Configure an External RAID-1 USB Drive for Backups - Ubuntu 12.04

I have a WD USB 3.0 500gb hard drive for my work laptop who's sole purpose in life is to record back-ups of my Ubuntu 12.04 system.  I've never formatted the device and it's always worked very well for me despite my cavalier treatment: tossing it around in my backpack during my commute mostly. I use a program called back-in-time for my backups and I like it for it's ease-of-use coupled with a healthy sense of fire-and-forget.

Recently, my back-ups started failing with generic write-fail errors so I committed to re-formatting the device.  Since I was going to nuke it, I decided to implement RAID-1 during it's resurrection just to have an "extra" copy of my back-ups.  The partition I back-up is only 100gb -- which is currently about 55% full -- so splitting the device into two logical drives and configuring under RAID shouldn't be a problem, right?

I mean, I have absolutely NO experience with RAID so how hard can this be?

After a few false starts, I honed down the process and this is what I'm sharing with you today.  Oh, and as side note -- in the first paragraph, I linked to this particular drive on Amazon as a courtesy -- I'm not getting and referrals or anything like that...just thought you might want to see what we're dealing with.

Phase 1 - Installation of Tools

Ok - so, to get started, I needed to install mdadm which was pretty easy:

[cc lang='bash' line_numbers='false']

# sudo apt-get install mdadm


What was somewhat confusing was the need for mdadm to install a mailing agent - for the monitoring tool, but during the installation I elected to not configure the mail program and everything still went smoothly.

Next, I installed gparted and all of the available options through the Ubuntu Software Center app.

I already had back-in-time installed and I provided the link above.  Use the Ubuntu Software Center to install this app if you don't already have it.

With all the software in-place, it's time to configure the device.

Part 2 - Wiping and Configuring the Device

Plug your device into your system and wait for it to automount -- which it will do if it still has the factory-equipped partition installed.  Once it's installed, go to your command line and unmount the device:

[cc lang='bash' line_numbers='false']

# umount /dev/{your_mount_point_here}


Either sudo this command, or sudo over to root.  I chose the latter as I get tired of forgetting to sudo.

Once the device is unmounted, start-up gparted,, authenticate, and wait for gparted to stop scanning for devices.  Change your device over to your external drive which was, for me: /dev/sdb.

(Side note: The WD drive comes with it's own software which just swell if you're a Windows or a Mac user.  Otherwise, the software is just taking up space.  Copy these programs if you want but you can always download replacements from the WD website.)

For me, the drive was formatted into one single, large, partition.  Delete this partition and don't look back.

Next, split your partition into two smaller partitions, equally dividing the available space between the two.  RAID-1 is mirroring - you're creating two logical partitions in this device but you will mount it (the system will see it) as a single device.  Data is written to the first partition and RAID copies the data over to the second partition, mirroring the data to the second logical partition.

I chose the ext3 filesystem for my partition simply because:

  • it's better than ext2
  • it's robust
  • I don't believe I'll get any speed benefits from ext4 (USB, eh?)
Make a note of the names of the logical partitions (Mine was /dev/sdb1 and /dev/sdb2.) as you're going to need to know this later.

Note too that you're not actually doing anything at this point - you're just building a task-list for gparted to execute when you're finished creating tasks.

Next, flag each partition as a RAID partition by right-clicking -> Manage Flags -> Raid.

Once this is done, you're ready to execute your task list which will partition and flag the devices.  When the task list completes successfully, you can quit gparted.

Step 3 - MDADM

Next, you need to create the software RAID volume using the mdadm tool.

This is very easy and is done with a single command:

[cc lang='bash' line_numbers='false']

# mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdb2


What this command does is:

  • invokes mdadm with the --create option (useful for creating RAID arrays)
  • specifies the "verbose" flag so you can get meaningful diagnostics should something head south
  • specifies the mount-point device (/dev/md0) for your RAID
  • specifies your RAID level (--level-1) (remember, 1 = mirroring)
  • specifies the number of devices in your RAID (--raid-devices=2)
  • lists the device links for the logical drives to be used in the RAID (/dev/sdb1, /dev/sdb2)
In the screen-shot below, I highlighted the raid mountpoint device, /dev/md0, because (a) I kept forgetting it was a required parameter to the command and (b) I am going to need this device name for the next command...
[cc lang='bash' line_numbers='false']

# mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdb2 mdadm: /dev/sdb1 appears to contain an ext2fs file system size=244174848K mtime=Wed Dec 31 16:00:00 1969 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: /dev/sdb2 appears to contain an ext2fs file system size=244177920K mtime=Wed Dec 31 16:00:00 1969 mdadm: size set to 244173688K Continue creating array? yes mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started.


Press enter to execute the command and you should get the prompt back after a few seconds.  Note that the program requires no input from you.

Step 4 - Making and Mounting

This is the step that most of the online and available pages on mdadm leave out.

You still need to format your filesystem!

So, let's do this with mkfs...

[cc lang='bash' line_numbers='false']

# mkfs -t ext3 /dev/md0 mke2fs 1.42 (29-Nov-2011) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 15261696 inodes, 61043422 blocks 3052171 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=4294967296 1863 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done


Note that I use the RAID mount-point address of /dev/md0 to create the filesystem.

This will take a few minutes to run so go take a stretch break and play with the dog or annoy your hot receptionist or something.

Once this step completes, you're pretty much done -- all that's left is mounting the device and using it.

[cc lang='bash' line_numbers='false']

# mount /dev/md0 /media/raid

<do stuff like start a back-up>

# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/PAMCAKES-root 84G 9.4G 70G 12% / udev 3.9G 4.0K 3.9G 1% /dev tmpfs 1.6G 920K 1.6G 1% /run none 5.0M 0 5.0M 0% /run/lock none 3.9G 49M 3.9G 2% /run/shm /dev/sda2 242M 159M 71M 70% /boot /dev/mapper/PAMCAKES-user 93G 48G 42G 54% /home /dev/md0 230G 22G 196G 11% /media/raid


Note that your available filesystem space is approximate one-half the capacity of the drive.

That's right - you paid a whopping $89 for a 500gb device out of which you can use about 230gb.

But you have two of them.  So if your first partition fails, you've got a back-up of your back-up thanks to the wondrous magic of RAID.  And, you learned how to set-up a RAID device, so, win.

Unless one of your kids (or grandkids) decides that your sleek, new, external device looks better in the fish tank than it does on your desk, you have an additional and available option for recovering lost data.

Keep in mind that this is a software solution -- hardware RAID is still the preferred way to go when dealing with issues of redundant data storage.  But this works, too.

Step 5 - Maintenance

To stop your RAID device use this command:

# mdadm --stop /dev/md0

To see the state of your RAID, cat /proc/mdstat to your terminal:

[cc lang='bash' line_numbers='false']

# cat /proc/mdstat Personalities : [raid0] [raid1] md0 : active raid1 sdb2[1] sdb1[0] 244173688 blocks super 1.2 [2/2] [UU] [>....................] resync = 1.8% (4474560/244173688) finish=4948.7min speed=806K/sec

unused devices: <none>


To get details about your RAID, use mdadm:

[cc lang='bash' line_numbers='false']

# mdadm --detail /dev/md0 /dev/md0: Version : 1.2 Creation Time : Tue Jul 24 14:41:55 2012 Raid Level : raid1 Array Size : 244173688 (232.86 GiB 250.03 GB) Used Dev Size : 244173688 (232.86 GiB 250.03 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent

Update Time : Tue Jul 24 14:42:51 2012 State : active, resyncing Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0

Resync Status : 2% complete

Name : pamcakes:0 (local to host pamcakes) UUID : b542c27c:1d620c3e:6f07b9e8:53aee08d Events : 1

Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 18 1 active sync /dev/sdb2


To uncouple the RAID device:

# mdadm --remove /dev/md0


That's it for today...hope this helps...there's a lot of good information already available on the web for using mdadm and software RAID - I just wanted to consolidate everything into a contiguous process.

Later, should sufficient motivation present itself, I'll follow-up with alerts and what not...


Waaaaake-up! Hello? Lion? You awake? WAKE-UP!

Oh, Apple.  What did you do now?

It's one thing to introduce broken (or bent) functionality in an upgrade release.  It's quite another to break (or bend) existing functionality in the same upgrade.

I really like Lion so far.  What I thought I would miss, I don't, and I've already become dependent on several of the base features that the upgrade offers.

And, hey, Microsoft (you big wad o' suck) take note:  a major update for $30 that I can install on all of my machines!  And I don't have to pay attention to see if it's ultimate home premium 64, too!

(aside:  I'm more pissed that usual at Microsuck.  Earlier, using Bootcamp, I was playing Rift and I noticed that performance was lagging badly.  To the point where I just decided to log-out and get some work done.  After logging, I see that my tx/rx light on the dsl modem is solid.  During shut-down, I see the usual dire-imprecations and deadly warning spew that pops when you update a Microsuck in-progress system update download.

WTF?  I explicitly turned off the "feature" of independent updates in favor of only-update-when-I-tell-you option.  You know, the way real operating systems do it.  Apparently this setting means jack-shit as the crapware decided, again and on it's own, to go out and download god knows what from the 'net.  Pure and unadulterated hubris.

Now I don't mind the constant virus updates -- I deleted three security exceptions from the Windows box today alone.  But this constant updating without my permission really is pushing it.  You confirm everything I want to do, concerning downloaded content, several times.  But true to the "do as I say not as I do" philosophy of this bloatware, Windows continues to ignore user selections and configurations and just farts and whistles it's way through a continuous stream of critical updates.  Pure crapware.

Wanna end the wars in Iraq and Afghanistan?  Send them free copies of Windows to install on all their military infrastructure.  War will be over in a week, guaranteed.)

That was a long "aside".  Or rant.  Or some factual observations.  Whatever.

Anyway, back to a real operating system that not only let's you get real-work done, but also listens, remembers, and then doesn't ignore your configuration settings...

I've been having problems with my Lion installation not waking from deep sleep.  I define two levels of sleep.  One is light-sleep: where the computer's screen saver kicks-in, and a simple mouse-twitch brings it back.  The other is deep sleep: this would be when you explicitly put the computer to sleep, or your power management settings kick in.

What I've been experiencing has been happening either on weekend-mornings, or in the evenings when I get home from work.  I sit down at the computer and poke the shift key, twitch the mouse, tap the space bar and ... nothing.  Repeat shit-key poke, mouse twitching, space bar tapping. ... Still nothing.

I poke the caps-lock key.  ... No light.  This is not good.

Both my keyboard and my mouse are wired USB peripherals.  So I dis(re)connect the devices from the hub and, again, twitch the mouse, poke the cap-lock key and ... black screen.  There is no power indicator on the new 27" iMacs.  So I have no idea what state the computer thinks it's in.  Time for some drastics.

I tap the power button.  This is usually enough, on my MacBook Pro, to jog it awake but, on my iMac...nothing.

Eventually, frustration wins out and I do a hard-reset by holding down the power key until it powers off and then I reboot.


I have a three support contract with Apple on this desktop but I'll be damned if I'm going to call them to confess that I've no idea on how to wake-up my desktop from sleep.  So, I google it.

I found this article, which explains how to reset the PRAM and NVRAM on your iMac because, you know, batteries get old and flash memory gets stupid over time.  So I follow the steps and, when the computer restarts, it's definitely brighter.  (I'm not that good of a touch typist and I tend to inadvertently do things to both the brightness and volume controls...)

But, the next day when I get home from work, the computer is back in Rainman mode and I have to power-down to bring it back.

So I google it again, and this time I see a post on a mac-forum that blames the problem on disk permissions.  Sure.  Why not?  So I run verify disk and, lo'!  I have a bunch of crap that gets re-perm'd.

Still not going to call Apple.

I'm writing this article and I guess I'll see what happens the next time I try to roust the machine from deep-sleep.  I'm pretty confident that it's going to fail and, if it does, then I'll log a call to tech support.

In the meantime, if any of you have suggestions, I'm open...