A storage crisis

The best new phones come with the ability to shoot 108 megapixel photos, record 4K video with stereo sound, and pack the results into a terabyte of onboard storage. But what do you do when that storage fills up?

If you want to keep those files, you’ll need to offload them somewhere. Since your computer probably doesn’t have more than 2Tb of storage, you’ll need an external drive. Or two. Or three. Or more. Over time, a lot more.

Welcome to my world.

Gathered here for a portrait in a corner of my desk in Manhattan are 22 hard drives, plus three SD cards that each exceed the capacities of the drives they’re laying on. And then there’s the 2Tb one in the laptop I’m using now. That one has 357.33Gb available. Most of the others you see are also full, dead, or both. Five have FireWire connections, which my current laptop doesn’t comprehend at all. I also have a similar collection of drives in Santa Barbara, and several more in Bloomington, Indiana. (Yes, I live in all three places. This is much less fancy than it seems.)

Photos occupy most of the data I’ve stored on all those drives. Currently, my photo archives are spread between two portable drives and my laptop and total about 7Tb. I also have a 5Tb portable drive for videos, which is back in Santa Barbara awaiting dubs off tapes. The portable photo drives are among those in the picture above. Earlier today, my laptop gave me this news about the main one, called Black 4Tb WD Photo Drive:

That’s why I’m transferring its contents over to the 10Tb drive called Elements, on the far left. A progress report:

About 5Tb of Elements is occupied by Apple Time Machine backups. After the transfer is done, there won’t be room for more backups. So my project now is figuring what to do next.

I could get some Network Attached Storage (NAS), I suppose, but I’d need three of those. My experience with the option so far is with a used 2012-vintage 18Tb QNAP one that was cast off by a friendly university. It sits at the end of an Ethernet cable in Santa Barbara, and I’ve never been able to make work, except (no kidding) at dial-up speeds. (I welcome help.)

The obvious next option, of course, is to put it all in a cloud. Two problems there. One is cost, at least if you’re talking Apple, Azure or AWS. The other is that upstream speeds have been highly sphinctered by ISPs for decades. Here in New York, where our ISP is Spectrum, our speeds have long run 100-400 Mbps down, but only 10 Mbps up. However, I just checked again with Speedtest.net, and got this:

And that’s over wi-fi. So color me encouraged. But before I commit to a supplier, I’d like to hear what others recommend. Currently I’m considering Backblaze, which is top rated here. The costs $6/month, or less for unlimited sums of data. But I’m open to whatever.

[Later…] Hmm. At that last link it says this:

What We Don’t Like:

Something I should mention is that some users have had bad experiences with Backblaze because of a not-so-apparent feature that maybe should be a lot more obvious: Backblaze doesn’t function as a permanent archive of all of your data, but instead as a mirror.

In other words, if you delete files on your computer, or the drive fails and you’re connected to Backblaze’s website, Backblaze will see that those files are gone and will remove them from your online account, too.

Granted, signing up for the forever version history option would eliminate any issues with this, but it still poses a problem for anyone using one of the limited version history options.

Alas, the forever thing is complicated.

To be clear, I want more than a mirroring of what I have on my laptop and external drives. I want to replace those external drives with cloud storage. Is that possible? Not clear.

Alas, for all of us, this problem remains.

Oh, and Spectrum now only measures under 10Mbps upstream. So forget the cloud.



5 responses to “A storage crisis”

  1. Hi Doc,

    One combination I have found really handy is Backblaze B2 (the storage API, not the backup app), and ODrive (a unified storage app, which provides file syncing to a variety of cloud services, and can also provide temporary or permanent shared links to your files).

    B2 is reasonably cheap ($5/mo for a TB), and keeps file revision history. Meanwhile, ODrive allows you to mount your storage as a shared drive, without requiring hard drive space to cache the files locally (like Dropbox does).

    I use it B2 both as primary storage for large files, and as a backup mirror for my QNAP fileserver (with its Hybrid Sync feature, I can get offsite backups of my backups really easily).

  2. Best Laptop for Agriculture Students

    Doc Searls Weblog · A storage crisis

  3. Thanks, ksfiles. Great advice.

    I plan to do something like that when I get back to a university setting next month, where I expect to take advantage of fat fiber upstream capacity. The cable connection where I’ll be staying is, alas (I am told) just 10 MBp/s upstream.

    And if I can get my QNAP rig going for real at home in Santa Barbara in September, I’ll follow your footsteps there as well.

    Thanks again!

  4. I notice Backblaze only works as a backup, so the HD’s have to be connected to the computer at least every 30 days. So strictly speaking its not storage 🙁

  5. Hmm it seems like your blog ate my first comment (it was super long) so I guess I’ll just sum it
    up what I submitted and say, I’m thoroughly enjoying your blog.
    I as well am an aspiring blog blogger but I’m still new to
    the whole thing. Do you have any suggestions for novice blog writers?
    I’d certainly appreciate it.

Leave a Reply

Your email address will not be published. Required fields are marked *