Storage and backup of 5+ TB of photos

[helged]

Advertisement (gone after registration)

Hi folks. 

It's time for me (and I may not be the only one) to rethink storage and backup of images. In my case, some TB, say 5+ TB, of data. 

I guess external disks are commonly used, and I have several disks, all having backups located off-house. This procedure works, but disks/parts of disks will eventually fail. And using multiple disks is not the most elegant way to store and get access to images.

Which flexible and safe storage/backup system are in use? Anyone experience with eg Synology NAS servers?

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[charlesphoto99]

59 minutes ago, helged said:

Hi folks. 

It's time for me (and I may not be the only one) to rethink storage and backup of images. In my case, some TB, say 5+ TB, of data. 

I guess external disks are commonly used, and I have several disks, all having backups located off-house. This procedure works, but disks/parts of disks will eventually fail. And using multiple disks is not the most elegant way to store and get access to images.

Which flexible and safe storage/backup system are in use? Anyone experience with eg Synology NAS servers?

I use a pair of 20TB Thunderbolt 3 Glyph Drives in RAID1 as my working storage, and then back those up to a Synology 4 bay NAS, and a spare 16TB drive in an OWC dock, and an old Drobo 5D I have kicking around as well. I should have an offsite drive but behind on that. The nice thing about Synology (and Drobo, but don't go there with a used one as the company is no more) is that one can use a mix of drive sizes and types that one might have laying around already (not possible with traditional raid solutions) and then upgrade individual drives as your storage solutions changes (mine is currently hovering around 14TB total of files). The Synology is not the fastest solution - if one needs speed then you have to look at SSD solutions which becomes expensive quick depending on the amount of space you need. I prefer for expediency sakes to have everything on one drive, even if I'm not using most of the files. LR does not really need a super fast image storage drive, only for its catalog. 

[PhotoCruiser]

I have a lot of external disks as i usually buy one per year.

The main backup medium is a 24TB G-Raid (now SanDisk G-Drive) in mirrored disks configuration and it works well, no problem so far.
The old housing as below looks nicer on my Apple Setup, however those things can be loud.

I was thinking over a NAS, but i don't want to hassle around to set it up and maybe have a data breach so i just use external disks and backup every now and then. For the rest TimeMachine takes care of the other data but backup on another G-Drive single disk. However a better backup software than TimeMachine would be a better solution.

Chris

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

Edited January 26 by PhotoCruiser

[charlesphoto99]

58 minutes ago, PhotoCruiser said:

I have a lot of external disks as i usually buy one per year.

The main backup medium is a 24TB G-Raid (now SanDisk G-Drive) in mirrored disks configuration and it works well, no problem so far.
The old housing as below looks nicer on my Apple Setup, however those things can be loud.

I was thinking over a NAS, but i don't want to hassle around to set it up and maybe have a data breach so i just use external disks and backup every now and then. For the rest TimeMachine takes care of the other data but backup on another G-Drive single disk. However a better backup software than TimeMachine would be a better solution.

Chris

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

Do NOT have a RAID 1 as your only backup. If the controller goes, so goes both disks. It's great for realtime backup, but always backup new files asap to another disk(s) in another box(es). You can also use a bare disk drive dock (they're pretty cheap) and then store the bare disks. I use Carbon Copy Cloner software for doing backups. 

EDIT: and not saying you are (I don't know), just for others reference. 

Edited January 26 by charlesphoto99

[PhotoCruiser]

I need to check how the two disks are configured,
but i remember that they are mirrored RAID1, so if one goes bad the other has the same data on it.
I have only 12TB available from two 12Tb disks and not 24TB and i had to reformat them as the drive came as 24TB drive.

However, i backup important things on one of the many external disks i have, hard disks or SSD disks, so data loss should be limited.
But you are right, for mission critical or very important data it would be best to have a backup software running and copy all over night on another set of disks or in a cloud or similar.

To do it perfectly and avoiding to copy virus or other damaging stuff on the backup there should be also many backups of the backups (i.g. 7 disks for 1 backup every day) what makes the use rather complicated. This is one of the reasons that i have so many disks and rotate them every now and then.

Chris

 

 

 

Edited January 26 by PhotoCruiser

[PhotoCruiser]

vor 3 Stunden schrieb helged:

Which flexible and safe storage/backup system are in use? Anyone experience with eg Synology NAS servers?

Synology is one of the better brands, however you have to stay behind the software updates as also they have some software bugs whatccan lead to compromised safety and i personally would never attach them in any way to internet to avoid external persons being able to access my data. All my memory devices are also encrypted and that saved my butt ass two external disks, two MacBooks and a iPad was stolen.

Chris

[charlesphoto99]

Advertisement (gone after registration)

What I'm saying about RAID 1 is not a disk going bad (the reason to use RAID 1 is exactly that - one doesn't lose downtime while replacing the second disk) but the controller chip for the external box itself. That can potentially corrupt both disks. Many years ago I had a single disk that was fine itself, but the box controller went wonky. I was behind on backups, but using DOS was able to slowly extract the files. Once that was done, reformatting the disk itself was fine. 

 

[LocalHero1953]

Terramaster 5 bay RAID here, with 5x4Tb drives configured as RAID 5. I have a similar volume of photo storage as you, although I have recently reduced many of them from DNG/TIFF to JPG. These all come into the category of 'photos for others': events, theatre/music/dance photos. I take them in large numbers; I'm unlikely to want to revisit them for editing, though I retain them in case they're needed again - and for the record. This reduction made a big difference to the occupied storage space.

I used to have a QNAP device, which was highly regarded, but I found the software impenetrable. Terramaster software is much more friendly.

I also have a basic subscription to Backblaze, which has no storage limit, but doesn't retain files longer than, I think, 30 days.

Edited January 26 by LocalHero1953

[hansvons]

On 1/26/2025 at 3:33 PM, helged said:

I guess external disks are commonly used, and I have several disks, all having backups located off-house. This procedure works, but disks/parts of disks will eventually fail. And using multiple disks is not the most elegant way to store and get access to images.

Don't do raids, NAS servers etc. Proper posthouses backup their work on LTO tape drives. They use their raids only for online work, which is too unreliable for long-term storage (not the discs but the controllers will be the issue). LTO works similarly to a simple HD; only it's supposed to last longer and is less prone to failure. The other simple way of backing up data is to use two independent HDs to mirror your data on both discs. A few years ago, it was recommended to use spinning HDs and not solid-state HDs because, in the event of failure, data on spinning discs can often be fully retrieved. A dead solid state HD remains dead, regardless of what you try—no idea what science says today. I have used two 8TB discs for 3 years now. They will be filled up by Summer. The next pair will be 12TB each (e.g., SanDisk Professional G-Drive (SDPHF1A).

Another option is fast 4x 3TB SDs for copy-1 and a large 12TB for copy-2. This way, you'll have quick access to your files and the security of an old-fashioned HD as the backup disc.

[charlesphoto99]

LTO external drives are $$$$$ (the tape itself not so much). For the cost, you could do a LOT of redundant backups on HDD's and have money left over for a nice lens, used body, Eizo monitor, printer, trip, etc. etc. 

If you are worried about controllers, get one of these and some bare internal drives to the size of your choice and some drive storage boxes. Slower than Thunderbolt, but you're just making backups.

https://eshop.macsales.com/item/OWC/TCDRVDCK/

[BernardC]

The only thing I would add to @hansvons's excellent advice is that RAID addresses one risk: that of a disk going bad. It's not a long-term solution, but it's much better than relying on a single disk.

I copy (technically "sync") files from my Synology array to offline disks weekly. LTO would be nice, but I can't justify the expense. The trick, touch wood, is to make multiple copies, and to store them in different places. You also need to test/refresh your backups regularly, as I'm sure a lot of people who backed-up onto CD-R or DVD-R have found-out.

You really need to ask yourself how much risk you can live with. In your case (5TB), a paid online option might be warranted, on top of other backups. Keep in mind that it will take quite a while to do the initial upload (it could be several days), and that the company you chose could close shop, or cancel your account, at any time. It will also take days to download your data if you ever need it, so don't expect to be back on your feet right away.

[LocalHero1953]

Uploading all my Tb to Backblaze took many days initially, but now runs well in the background. I have never had to retrieve bulk data (you can pay to be sent a hard copy by mail), but I have downloaded old versions of files that I had deleted or destructively edited some time back (and so had been deleted/edited in my local backup as well). That worked well.

[Chuck Albertson]

I just bought a 16TB WD external HD for backup. Way more capacity than I need, but that's what I said when I bought a 4 TB HD a few years ago; it's now chockablock.

[SrMi]

It is good to have an offsite backup as well. I am using BackBlaze and iDrive. I also use iDrive as a backup when I am traveling (laptop images and catalog backups).

[PetPhoto]

but remember, most of those place can  happily delete any data you have stored on their fancy offsite server system when you forget to make your monthly payment to them. 

 

And the hard disk spinning away is a wonderful thing.   Disks made in 1970s are still spinning. Voyager 1 and 2 still work just fine after all this time.

[algrove]

On 1/26/2025 at 9:33 AM, helged said:

Hi folks. 

It's time for me (and I may not be the only one) to rethink storage and backup of images. In my case, some TB, say 5+ TB, of data. 

I guess external disks are commonly used, and I have several disks, all having backups located off-house. This procedure works, but disks/parts of disks will eventually fail. And using multiple disks is not the most elegant way to store and get access to images.

Which flexible and safe storage/backup system are in use? Anyone experience with eg Synology NAS servers?

I often use Sabrent no tools closing aluminum case that takes 8TB NVe cards. It is not large, weighs little for what it does and is lightening fast. 

[Luke_Miller]

I use a hard drive dock and bare drives for backup.  When not in use the drives reside in hard cases with labels.  Previously I used a WD external drive.  When its controller died, I removed its hard drive for re-use.  I then discovered that WD uses a proprietary disk format and the contents of the drive were not readable. 

[Archiver]

My general strategy is to have two sets of drives, a working set and a backup set. Every two or three years, when drive prices have become lower, I buy new working and backup drives and move everything across. The old drives become multiply redundant backups.

For 5TB+ of files, just buy two 8TB drives if budget allows. Move everything to both drives, and back up new files in the first drive to the second drive regularly. This gives you access to everything, and a backup in case either drive goes down.

A word about Synology: it appears NAS units purchased from 2025 onwards will lock out some functions if the NAS has drives that are not of the Synology brand. I've long considered a Synology NAS, but am now loathe to buy into that system. It is a doddle to buy recertified enterprise Seagate drives quite inexpensively, but if Synology will only provide full functionality and support to their own branded drives, then I'm looking elsewhere.

https://nascompares.com/2025/04/16/synology-2025-nas-hard-drive-and-ssd-lock-in-confirmed-bye-bye-seagate-and-wd/

[Advertisement]

Hallo,

Look here: Interesting Leica gear

[BWColor]

I keep my working files on a five drive Synology NAS with four 16TB drives and one 24TB drive with dual redundancy.  I regularly process TIFFs created with .fff X2D DNGs, I use Phocus, Lightroom Classic and Photoshop.  I use smart previews and generate sidecar files within LRC.  I have two external SSD drives on Thunderbolt 4, but use the NAS for working image storage.  Until I upgrade all drives to 24TB drves, 8TB of large drive not used.  The Synology backs up to a second Synology NAS that can handle one bad drive.  These two NAS units are as far apart in the house as possible.  Only jpg are offsite.

[LeicaGuyUK]

HI,

My setup may seem a little complex but for me its spot on. i use:

Internal storage comprising of 4x 2tb NVME SSD`s in my PC. One pair (raid1 mirror also giving 12gb/s read/writes) for WIP 2 years and then the other pair the same for two years before these

1. Synology NAS (24TB SHR Raid 3+1 spare), a sync client on my pc replicates ALL contents of the PC drives to this
2. Another Synology NAS (same build as the first, this is a block level replciation from the first
3. Another Synology NAS (same build as the first two), this is an offsite replication at a relatives house.
4. With this it would take 7 drives to fail before i lose anything. i Also vault off a copy of all RAW images and my lightroom catalogues to external USB drives from the NAS`s just in case. As i have super quick local storage everything is responsive, i generally only work on anything in the WIP so rarely go to the previous years for anything. 

So in a nutshell everything is quick, fully replicated both onsite and offsite and I have probably 15+ years worth of storage available.

 

Edited April 30 by LeicaGuyUK