A brilliant thinker ahead of his time, Vannevar Bush envisioned a device he called the memex which would supplant one’s memory, allowing you to store and retrieve texts, ideas, photographs, and other items of interest. These bits of data could be linked together in meaningful ways, providing a way to explore the relationships between various “memories.” Sound familiar (I recommend reading read this)?

Now that we use something far better than Mr. Bush’s Memex, have you taken a moment to reflect on the amount of data that you create? From documents to pictures to music to videos, we create and use a vast amount of data on a daily basis. Most of us don’t even think about this until we run out of space or have a catastrophic hard drive crash resulting in a gut-wrenching realization (“I just lost all my music…”). I started contemplating these things anew after reading my college’s recent post, as well as David’s reflections on the topic. Looking at these two articles reminded me of an old post (May ’06) by Mark that also touched on the topic of data preservation.

What exactly is one to do? I’ve done some brief research on the topic, but have yet to find a solution that satisfies me. I use a desktop computer that houses the majority of my data (some 600 GB of stuff), in addition to a laptop (used only for doing stuff away from home). I also use a 2 GB thumb drive that travels with me, as well as a portable 160 GB hard drive (for lugging around larger data). All of these devices need to be backed up – and the older I get, the more important this becomes! Not only do I produce more data, but the importance and sensitivity of the data increases (e.g. I’ve started using Quicken). On top of all this, there’s this website – as I learned once before, it’s not impermeable, and the data can be lost – that would be almost two years of writing, up in smoke. Sure, maybe it’s not all the most valuable content in the world, but I fancy that I’d reminisce sometime in the future.

The backup and storage problem has a number of solutions, all sub-optimal and far to expensive for me. From co-located servers to storeage sites like StrongSpace, a myriad of options exist. A Network Attached Storage (NAS) device sounds like a decent solution to me, but at this current moment, I can’t afford one. How am I supposed to back up my precious files? The only solution that I currently use is having copies of the really important things on different devices, though none of them are dedicated backup mechanisms, nor are they particularly secure. I may investigate the possibility of creating a Linux file server out of leftover parts. The possibility of using Subversion to create re-visioned backups with incremental storage and the ability to “roll back” to an older version if something happens to a current version. By storing the subversion repository on a redundant RAID (level 5), you could reduce the risk to your data by a tremendous amount.

Anyone have a few 500 GB 1 TB hard drives they’d like to donate?

The second problem is digital obsolescence and the danger of proprietary file formats. As you’ve most likely noticed, the digital world moves extremely fast with a constant evolution of new formats, devices, and technology. As David alluded to in his post, when the software you use becomes obsolete or no longer functions, how do you get to that valuable data? What happens if you want to change operating systems? This is one of the biggest reasons why proprietary formats and DRM are so bad for your files – they tie you to a vendor and limit your options. When that vendor abandons a software line or goes out of business, what becomes of your stuff?

I would like to convert more of my documents to an open format, and I am actively trying to use as many open-source tools as possible. By employing open standards over proprietary junk, I am keeping my options open – I can change software, move to a new OS, and find the tools (or create them) to open an old document at some future date, despite the whims of some software vendor

Hopefully with thoughtful planning and management, I can archive my digital creations for many years to come. I’m sure that the simple thoughts presented here are not the best way (and maybe not even on the right track), but hopefully they will spark an interest in protecting your documents and media.

If you have any ideas or suggestions, be sure to leave a comment!