An interesting recurring topic keeps coming up around the office today.
Large capacity means more danger to your files.
What am I talking about?
- Customer #1. Has a 1 TB backup drive and a Macbook Pro with a 1 TB hard drive. He’s been backing up to the external drive using Time Machine for a long while, and his backup drive is now full. It’s the same capacity as his computer’s built-in hard drive, so he knew it would come to this.
No problem: he loaded up the external drive and deleted the backup set. No need to hold on to all these incremental file changes; he just wants to make sure the backup is current as of today. Clearing the drive means he now has a full 1 TB drive to work with again. Fantastic!
Now, run Time Machine and let ‘er go.
Half way through, the laptop hard drive crashed. Almost all of his family photos and videos, gone.
Data recovery is the only option (a pricey option at that). - Customer #2. Has a central server in their place of business and is meant to save all their documents and files to that server through the local area network. The server gets backed up on a nightly basis and I.T. takes the backup off-site on a regular rotation.
Their old computer is getting really slow and frustrating, so the boss authorized them to go to the local big box store and take advantage of the “back to school” sales by buying a cheap computer. That’ll bring them up to a more modern system.
Thing is, they don’t know the difference between a consumer-grade system and a commercial-grade system. And the commission-hungry sales guy doesn’t mention it.
So, in comes the new, super speedy consumer-grade system to replace the old one. This is fantastic! It’s fast, was only $600, and the hard drive is a whopping 2 TB. Now, they don’t have to store everything on the server.
Six months goes by and by now, almost all current documents, and even some personal files are being stored exclusively on the desktop computer. The files on the server are about six months out of date.
A single lightning strike and off goes the power. A few systems got damaged, including the server. But that’s not a problem for this company; the server is still backed up nightly.
The computer with the 2 TB hard drive however, won’t boot up when the power comes on. “Invalid System Disk”. What’s that?
The data recovery lab said the PCB on the hard drive was fried and quoted $2,300. - Customer #3. Saw a great deal in the local computer store flyer, likely a door crasher special. A 32 GB USB Flash Drive for only $14.99.
“Wow”, he thinks. “I remember when a 1 GB drive was $100”.
He picks one up just because it seems like a great deal. He doesn’t know what “Writes per block” means.
The drive works great. It’s amazing that he can store so much on it!
After some time, he starts offloading his family photos to the flash drive. Not a backup; actually moving the images onto the flash drive. That way, it’s easy access if he wants to get some prints while browsing the local superstore, or if he wants to share them with family or friends.
32 GB is a lot of space, and eventually, the drive holds the only copy of about 2,000 of his family photos. Basically all the photos he’d originally moved off his computer, and every photo taken since he bought the drive a few months back.
This drive could crash. Flash drives are very unreliable. However, in this customer’s case, he was lucky. The drive never failed. All the files remained safely on the 32 GB flash drive.
But he lost it.
A cold winter day and he was getting into his car in his work parking lot. The flash drive was in his leather jacket pocket, along with the glove for his right hand. Unbeknownst to him, as he removed the glove and climbed into his car, the flash drive fell to the parking lot at his feet. He drove home before he’d realized it.
He drove back to work with hopes of finding the drive, but the snow plow had been and gone, and try as he might, he never found the drive. He lost it all, with no chance of ever recovering his priceless photos.
A flash drive is good for:
- Copying files so you can bring them up on another computer.
- Taking a copy of images to the local photofinishing store for printing.
- A small bit of redundancy such as making a copy of very important files and keeping it in your purse for easy removal in emergency.
A flash drive is not to be trusted for:
- The sole copy of any file.
- Your sole backup.
I’m extremely particular about backups as you know. T0B33 in the chat room said to me today “My SD card which i used to store my data died today. Sooooooooooooo glad i had a back up!“, “Robbie, I always say i will back up later–then say the exact same thing later on–but you’re always talking about backups on the show and you tell stories which remind me of the time I lost all my school photos. 🙁 So it gets me off my back side every time. Thanks for that and please don’t stop!”
An SD card to store his data. But he had a backup, so he’s safe. How are your backups going?
And when you’re in the store looking at a 32 GB USB flash drive, or a computer with a 2 TB hard drive, think about how you’re going to make it redundant. Are you buying a 2 TB computer but only have a 500 MB backup drive? You’ve got it backwards!
Don’t put yourself in that situation. Yes, keep good backups, but also think about the capacity you’re buying. Why do you need 32 GB on a USB flash drive? Could you fall into the temptation to save files to it? If not, what is the drive for? Do you have the backup capacity for your new 2 TB computer?
I hope you get what I’m saying. Please post your comments below.
– Robbie
Technically, SD cards never fails and can resist every bad condition you can think of. However, I have many cards and flash drives being corrupted for no reason, even after very few uses This includes SD cards I bought for backups, as I don`t trust hard drives.. I have many problems with them too.
A technician told me to buy cards/ flash drivers not bigger than 16GB, as bigger ones can easily have many problems, he says…
Even these ones fail as well.. may be online backup is the best and the only safe option.. but you need a suitable connection speed.
NAND wear is real, and leads to SD card failure. This can be perceived as data corruption, or complete inaccessibility of the device. The size of the card is irrelevant other than the fact that on a larger card, you’ll probably read/write more data which will wear out the NAND and lose more data (hey, it is larger after all). Your technician’s advice is incorrect. The proper advice is to always have a backup, and a backup of your backup, because all storage fails, it’s just a question of when.
Great Media Monarch of the Maple Leaf:
Being a mere mortal I thought it would be a splendiferous idea to store my media treasures I.e. Beach Boys-Beatles-Groovy 60’s records, Western movies etc on an external USB hard drive to circumvent meltdown periods of despair. Then I discovered how junky the darn external hard drive I blew my fun money on was ,and that it was was obviously created out of recycled nitro. My question is…..what sort of affordable safe solution would you suggest for someone to bulk store readily accessable mp3’s and mp4’s say for instance 2 TB’s worth? I tried using DVD-R’s but then your stuck with this physical storage problem, because houses only come with a fixed amount of closet’s………Help!
Roku Rocks!
Haha, love the writing. “Great Media Monarch of the Maple Leaf”… I want that on a “hello my name is” sticker. 🙂
DVD-R’s might have a shelf life of about 5 years, so that’s not a good option. They’ll start having read issues after a few years.
With the amount of data you’re holding, I’d say your cheapest solution would be to buy two (yes, two) external hard drives of at least 1.5x the capacity you need. Say 3 or 4 TB each. Then, replicate everything on both drives, and store one off site. That’d be way too much data to lose in a fire. You don’t want both copies at the same site.
There’s really no cheaper option I could recommend. Realistically, I would lean more toward a RAID 1 or unRAID server solution, with network access, backed up off-site. For me, I use a Pogoplug with an external hard drive in a cupboard somewhere at a family member’s house, backed up from my unRAID server (see Episode 103) via SSH & rsync.
Good luck!
Roku Rocks, indeed!
what about a size, i can’t find any external hard drives that are anywhere close to the size of a flash drive. does anyone know of a company with really small portably back up drive that does not need an external power source.
You can get USB flash drives as large as 256 GB (possibly even bigger… I don’t really keep up with it). Check this out: http://www.amazon.com/dp/B00FHL3F0E/ref=nosim?tag=cat5tv-20 – but remember they are flash media… you need to always have a backup of the data that is on the flash drive because these kinds of devices are not reliable long-term.
As far as an actual spinning hard drive goes, they are physical hard drives in an external chassis. So the “small” ones are going to be a laptop hard drive… 2.5″. At least they’re more reliable, as long as you don’t drop them or move them while they’re powered on.
You do not need an external power source with USB external 2.5″ drives–only 3.5″ drives. The 2.5″ models use 2x USB ports: one for data and one for power.
So in that case, which type of external HDD should be connected to my HDD? 2.5 or 3.5. Is there reliability issues between these two types of drives?
I like the smaller USB drive concept. Better with 3 smaller drives each with some data than all of you data crashing of one bigger one (as well as good backups!)
I would never trust S3 for archiving (not even sure about Glacier), just backup. Nice to hear about your S3 loss, thanks.
My backups are uploaded as objects named after their hash. This gives me deduplication and versions. On each run, if an object is missing it gets reuploaded (although I may lose an older version). Thus my risk of loss is the odds of loss divided by the time between my last backup and the catastrophe at home.
And yes, the important stuff I should dup across Google Drive and maybe Box. This exponentially decreases the chance of loss.
Speaking of permanence, I wonder if I should dust off my old magneto-optical drive for storing Bitcoins…. 🙂 I don’t currently have any coins but people that store stuff like that on flash drives are nuts.
What do you know about M-disc? (mdisc.com)
It seems like a rather bold claim.
Wow–I’d never heard of M-Disc. Thanks for the link. Basically they’re claiming to sell a Blu-Ray disc that gets around the crap lifespan of traditional optical media like DVD and Blu-Ray writable discs. I suppose it is possible if the inner layers are completely different, so I’d say it could be a legit claim, but I have no experience with them. Want me to see if I can get them on the show?
I’d also encourage you to check out some of the PogoPlug hardware since it lets you stick a (for example) 4 TB external USB drive on a family member’s Internet connection and gives you cloud-like access for your backups. This could supplement your S3 backups quite well (a mirror) and would be much easier to recover in event of catastrophe (go pick up the drive and plug it in).
Thanks for commenting.
Are your DVDs in the same house as your computer?
Are they in a fire-proof safe?
How often do you try to read back the DVDs to see if they’re still readable?
You are one house-fire away from losing it all.
And DVDs are not permanent / long term storage. They will lose your data.
(personally I encrypt then backup to Amazon S3, and buy hard drives in pairs)
Indeed, DVDs are just bad news waiting to happen. A DVD-R disk, burnt in a clean environment, placed directly into a jewel case and stored in a comfortable temperature in a dark storage area will still likely have trouble reading the data 5 years later. They are not at all reliable for long-term storage in my experience. When we hear of DVDs “lasting forever”, they’re talking about pressed DVDs (think: a purchased movie), not burned consumer DVDs (think: a consumer writeable disc burnt on your computer). They may be read by the same device, but they are altogether different in their physical makeup and longevity.
Fire and theft are two big threats to our data and if you don’t have the data stored in multiple places at once, expect to lose it.
Regarding S3, it’s a good idea (encrypted first), but do keep in mind the policies which make it clear that they basically expect to lose your files every now and again. We’ve had to re-upload one or two images to S3 (not RRS) that have been lost by their system and began giving errors. Easy to do since I have local copies, but what if that lost file was your backup.
The key to data security is: Redundancy. Redundancy in your copies. Redundancy in your locations.
And good solid encryption will reduce the chances of someone getting a hold of your data.
You get it! Thanks for posting.
Yes, i remember someone telling me a long time ago, i your data does not exist in three different places, it does not exist.
i have accrued hundreds of DVDs backup of old photos and videos, written and stock off in decade and more, though never need to use even if upgrading PC and hard disk, due to redundant and active backups. but i can sleep tight all night knowing my treasured files are safe since learning the hard ways during my 486DX4 days when my hard disk died, i cried for months, and have misgivings for years.