Backup, Backup, Backup

by Henry Farrell on October 20, 2008

“Michael Froomkin”:http://www.discourse.net/archives/2008/10/surveying_the_wreckage.html has a tale of terror.

Recently, the system has been a bi[t] weird, with very slow file access times (windows explorer would take forever to open, ditto with file dialogs in programs), and I also was worried that my copy of Firefox was compromised … First, I decided to take the plunge and migrate to a larger disk, and ordered up a “green” WD7500AACS. (Three quarters of a terrabyte! Whoohoo!) About three or four weeks ago, I copied my files on to it using using XXClone, a nice piece of freeware that basically makes an entire copy of Drive A (including operating system) onto drive B. … But things were still slow sometimes. I decided it was time to kill the trojan, or whatever, that seemed to be infesting my system. I also decided that I should go back to hardware RAID, since I don’t back up my files enough. … When I got back, the files were there, and I ran the first one. It duly called for a reboot and I did it — only to get error messages and a lockup. …at which point the disk wouldn’t boot any more. But no problem, I had my backup, the 160GB version. … But now that the two disks are in the system, with the 750Gb disk on the second pair of SATA ports, which are RAID capable (but were properly set for ordinary non-RAID use in the bios), the Windows system on the first 160GB disk decided they needed to be reactivated. … But the 750GB version worked. So that’s good. But now I’m nervous, things seemed jinxed. So I order up a second WD7500AACS, and plan to RAID mirror them. … Now, time extra backups. I’m a little nervous about hardware raid, in part because I’m a little dyslexic. … So I decided to make a software clone onto the new disk with XXClone, so that whichever way I copied the data would be OK. … I installed the disk, started up the format, and went of to do some stuff. When I got back, I found a blue screen of death, a 0024 failure (that I gather means a loose wire, something version one the sata hardware standard made all too easy). When I tried to reboot, I got a smart drive error – the disk is bad. I flip some disks around. One of the 160GB disks won’t boot either — “Disk error”. When the dust settles I have some very high-tech paperweights. … I’ve lost 3 weeks or more of personal data, only most of which can be reconstructed. . My work files, on the other hand, either on a unix server or on a USB stick, which I religiously back up at home and work, so that’s OK. My personal financial info, which isn’t backed up for the last 3+ weeks, I can recreate

Some life lessons here – the most obvious being the frequently repeated one of backup, backup, backup and _keep non-local copies of your data_ in case of massive system breakdown/fire/nuclear war etc. If it can happen to Michael Froomkin, who is much more technically adept and conscientious about backing up than you are (for most local values of ‘you’) it can happen to anyone. Happily, Michael appears to have lost nothing more than some easily recreatable data (and a lot of time, assuming he can get refunds for non-functioning hardware). If he were someone who didn’t religiously back up his material, he’d be in far worse shape. Non-local file backup is pretty easy to do these days, and relatively cheap. I use “Sugarsync”:http://www.sugarsync.com/ which synchronizes my three Windows machines very nicely 1 and as a side-benefit provides me with backups against local hardware errors. Kieran discusses a couple of alternatives “here”:http://www.kieranhealy.org/files/misc/workflow-apps.pdf (PDF), but whatever system you use, I really recommend that you institute _some system for doing this_ and that you _do it today_ rather than putting it on the long finger (which will most likely mean, given most people’s heuristics for this kind of stuff, that you won’t do it until you REALLY NEED TO, at which point it will sadly be too late).

1 I understand that it doesn’t work as well for Macs, which to my deep and everlasting regret isn’t a problem for me. The week before last, my university unexpectedly delivered me a lovely new MacBook Pro, which I had some eight hours to fall in love with before I discovered that it had been sent to me thanks to an administrative error, and that it in fact belonged to one of my colleagues. I’m still bitter, as you can tell (but in the unlikely event that an Apple executive is reading this post, and wants to reach out to the crucial academic-blogger constituency by handing out one of their new machines, they can find an enthusiastic evangelist for their product at this address …) More generally (and to get back from the griping), be aware that Sugarsync is not designed as a back-up product as such, and will do _nothing_ to save you from user generated errors (indeed it may make them more devastating). If you delete the one and only copy of your dissertation datafile from a synchronized folder, you will find of course that it is deleted from the copies of this folder on your other machines too. So caveat emptor.

{ 31 comments }

1

sanbikinoraion 10.20.08 at 3:07 pm

I never backed up until I got a Maxtor OneTouch USB HDD. The software that comes with it for Windows is great – just set what folders you want backed up and set a schedule, and let it get on with it. Obviously again doesn’t defend against accidental file deletion (though since you know when your next backup is scheduled for then you should be able to grab it off the OneTouch before it gets deleted).

2

Sam C 10.20.08 at 3:08 pm

OK, I’m sold – but does anyone have recommendations for good backup software for my (not especially new or fast, OSX 10.3.9-running) iBook? Cheers…

3

Chris Bertram 10.20.08 at 3:28 pm

I’ve been using Unison for a couple of years now, following Kieran’s recommendation. On the downside, it took some time to work out at first (and I spent ages puzzling over on-line guides), but it has worked pretty well since. It has also, lately, been coughing and complaining about files generated by Microsoft Office (well who wouldn’t) and refusing to copy them because of “conflicting permissions”. I’m sure there’s a fix for this but I haven’t put in the time to working it out yet.

4

Bill Gardner 10.20.08 at 3:29 pm

I use the Time Machine and an external HD on my MacBook with moderate consistency. Perhaps Mac experts can kindly tell us whether this is sufficient?

5

MarkUp 10.20.08 at 4:00 pm

For the Mac, it SuperDuper overall, there are/can be many ‘it depends’ involved. One thing to always be concerned with of course is data corruption and proprietary software/compression restrictions.

Henry, I bet you could sue for pain and suffering for having the MBPro taken away.

6

JSE 10.20.08 at 4:11 pm

I’m using Mozy (a daily online backup system) for my Mac; not sure how well it works because so far I haven’t had the need to recover anything. But it _says_it works….

7

Slocum 10.20.08 at 4:32 pm

I’m using Mozy (a daily online backup system) for my Mac; not sure how well it works because so far I haven’t had the need to recover anything. But it saysit works….

Yep — same here (only for Windows). Mozy is much better, though for small important files (documents) than big ones (photos & video) due to the slow upstream bandwidth most of us have. For large files (including my outlook mailbox files), external hard drives and recordable DVDs are better. I’d also suggest that Michael Froomkin had problems partly because of his sophistication — simple is better (for me, anyway).

8

bizzah 10.20.08 at 4:50 pm

Mac users should also be aware that there is a (very good) option already installed on their machines: rsync.

9

Karl Steel 10.20.08 at 5:01 pm

Big fan of Dropbox

10

Ben Alpers 10.20.08 at 5:39 pm

I’m a Mac user.

I used SuperDuper (which is a terrific, inexpensive program) until I upgraded to OS X 10.5, and have been using Time Machine ever since.

I back up to a Drobo, which I also highly recommend, which I’ve partitioned. I have about 500 GB devoted to Time Machine backups (of my MacBook Pro’s 150 GB internal drive as well as a similarly sized external Firewire drive), plus about 500 GB of primary storage.

This works very well.

11

Kieran 10.20.08 at 6:00 pm

These days I use a combination of syncing and backing up: laptop synced with desktop in office, which is backed up locally via Time Machine. I was using Mozy but it was having problems running on my desktop, so I’m looking for an alternative.

Since I’ve started using version control for stuff, I need to use unison less often — a dvcs makes it easy to have multiple, current clones of a project in different locations, each one of which is also a complete, annotated backup of every step of the development of the project.

12

Kieran 10.20.08 at 6:01 pm

Super Duper is also great.

13

Watson Aname 10.20.08 at 6:03 pm

What a lot of people miss, in my experience, is that you really don’t want only one type of backup. For typical use, you want to periodically mirror you drive entirely, but you also want incremental backups of your more important data. Depending how badly a rebuild from scratch will set you back, you may want at least two copies of incrementals, one stored remotely. Drives are cheap enough now many people opt to just replace these occasionally, leaving the old ones as stored backups. For your most important/annoying to replace data, your incremental backups and/or a mirror of just that should be stored a few places.

I have a script that mirrors my home machines personal data to a work machine and vice versa, every night. I also have incremental backups of personal data (mirrors alone aren’t any good if you don’t catch a problem before the next nights update!). On top of that, I have a bootable duplicate of my desktops hard drive, mirrored once every few days.

Since all of this is scripts, I don’t have to do anything or remember anything. There are a lot of programs for any program that will do one or both aspects of this, but I suspect that automating it is one of the most important aspects. The backup you forgot to do is useless.

14

Laura 10.20.08 at 6:03 pm

I do appreciate the reminder about backing up, but I just burned my most important documents to a cd. I know that’s not a complete solution, but if I have a massive computer failure, I’ll expect to be working from library computers for a while, and I don’t want to run into any of the downloading permission problems that I might get into with a public computer and an online storage solution like Mozy.

15

novakant 10.20.08 at 7:41 pm

Michael Froomkin had problems partly because of his sophistication—simple is better

This is correct. I really don’t want to add insult to injury, so please consider the following as a constructive attempt to prevent further disasters.

It can happen to anyone at any time (a certain percentage of hard drives just fail), so backups are a good idea, but the most important thing is to prevent these things from happening in the first place and take the right steps, if they occur. The vast majority of problems is caused by users who overestimate their knowledge and fiddle around with stuff they don’t really understand sufficiently. I used to be one of those users once, so I know what I’m talking about and this not meant to sound condescending.

The last time I had to rely on a backup was about ten years ago, maybe because I was lucky, but more likely because I generally know what I’m doing. And tellingly the people whom I sometimes help with their computers (relatives and friends, you wouldn’t believe how popular you become, lol) and who know next to nothing about computers never had to rely on a backup either, probably because they just use Word, Outlook and the like and leave everything else alone.

So the upshot of this is: don’t fiddle around with stuff, you don’t really know about and if you start noticing something unusual (“things running a bit slow” in Froomkin’s case) back up your system and then try to analyse what the cause of that behaviour might be. As soon as you’re out of your depth, look for solutions by asking around on the net, but only apply this advice if the suggested tools or workflows seem familiar and safe. If you’re feeling unsure, it’s much better to invite some knowledgeable acquaintance of yours for lunch on the weekend and have him/her take a look at it. And then there are actually professionals who make a living out of this and can probably save you a lot of time, headaches and money.

16

John Quiggin 10.20.08 at 8:06 pm

I sync home and work Macs via my iPod and backup each machine locally with Time Machine. It’s a bit klutzy but not so hard once you get used to it. And in 25+ years I’ve never lost more than about an hour’s work (most common cause – crashes while working in a program without autosave). Still, I want to move to SugarSync or similar if I can get organised.

17

Steven 10.20.08 at 9:08 pm

I want to move to SugarSync or similar if I can get organised.

Some time over the months I was trialling SugarSync (across two Macs and a PC), it silently deleted a bunch of files deep in my folder structure. Luckily I noticed, and had a sufficiently old physical backup of those files. But it was a useful reminder that sync is not backup.

Since then I’ve been using DropBox, which looks less snazzy at first but hasn’t actually destroyed any of my data. (Yet.)

18

MattF 10.20.08 at 9:21 pm

SuperDuper is great– by default it creates a bootable duplicate of your hard disk– so, if you back up to a firewire HD, you’re all set… um, unless you get one of the new Macbooks, which don’t have a firewire port.

19

David 10.20.08 at 9:43 pm

SuperDuper deserves all the praise it gets. Worth every penny.

20

MarkUp 10.20.08 at 9:53 pm

”which don’t have a firewire port.”

USB is bootable for a while now. A word of reminder to any that rely solely on online backup solutions. Remember the dot.com bubble bust, oh and the Alamo.

As W might put it, ” see… I uh, what you need is a road map, with a road map you can find your way…”

Plan, then implement.

21

Watson Aname 10.20.08 at 10:17 pm

MattF: You can boot from USB, too. It’s worth reiterating that having a bootable mirror of your drive is great, but it’s a pretty lousy backup strategy all by itself.

You want something incremental also, at least for data that is volatile. Kieran makes a good point that using version control on your working files (highly recommended for other reasons as well) often reduces the need for incremental backups.

Super Duper is good. It’s completely replaceable with free tools (rsync, time machine) but that doesn’t do you much good if you won’t use rsync .. and SuperDuper is very easy to use.

22

MarkUp 10.20.08 at 10:34 pm

Multiple mega sudo ditto‘s have a tendency to throw most folks in to a world they’d rather not be in even if they can just copy/paste it all.

23

MattF 10.20.08 at 10:42 pm

Didn’t know that you could boot from a USB drive– so that’s one less technicality to worry about.

As it happens, all the computers I use in my day job have fully versioned, automatic networked backup– so my professional backup needs are taken care of automagically. Backing up my personal stuff, I admit, is less systematic– but there’s at least a copy of everything ‘collectable’ on separate media as well as the SuperDuper clone.

24

Nick Caldwell 10.20.08 at 11:48 pm

SuperDuper is able to copy more of the hidden magic in an HFS+ file system than any other method (possibly including Time Machine, which is a slightly different use case anyway). So the other free alternatives aren’t quite as Mac-friendly as it.

25

Watson Aname 10.21.08 at 1:09 am

Nick, as I understand it recent versions of rsync (as installed by default on OS X) copy the resource forks etc. just fine. It certainly isn’t as user friendly as Super Duper, though.

26

rea 10.21.08 at 3:04 am

I didn’t understand more than about 20% of the post. Oddly, I somehow make my living using a computer.

27

Watson Aname 10.21.08 at 4:04 am

Addendum to 25: I looked it up, it is *not* the version of rsync shipping with OS X, but rather more recent source versions that seem to have (finally) sorted out the issues with extensions. This requires new flags, too. If none of that made obvious sense to you, best avoid relying on rsync, I expect. The ditto command works as advertized, and preserves resource forks.

28

nick s 10.21.08 at 7:24 am

I like the JWZ way.

I don’t follow it — I basically have a Heath Robinson backup system — but I like it.

29

Jacob Christensen 10.21.08 at 12:20 pm

For local back-up on OSX, TimeMachine is lovely (I reconstructed a crashed harddisk last week with the help of TM) – but it fails the second criterion: keep non-local copies of your data.

Backing-up the full contents of my harddisk to dotmac imMobileMe would probably send most ISPs into a rage. Here’s to hoping that nuclear war will not happen.

30

trane 10.21.08 at 9:23 pm

Thank you very much for the advice… which I should have listened to two months ago when my Mac crashed completely and irreperably (the IT man at work sent it to a firm that specialises in data reconstruction, but there was nothing they could do).

So my PhD dissertation…

..
.
And I learned my lesson.
Anyway, I have now bought a socalled Time Capsule from Apple. Seems to work fine.

31

c.l. ball 10.22.08 at 11:35 pm

I think I’ve wasted too much time looking up all the neat stuff Kieran listed in that article.

For Mac users there is the free (donation requested) Carbon Copy Cloner, which is supported by its makers. It is very good for a clone of your drive.

Beware of WD’s My Books as HDs if you are a Mac user; some models will not produce clones via CCC.

Comments on this entry are closed.