Jump to content

how best to store these large files


Recommended Posts

Advertisement (gone after registration)

**note to moderators** if this is in the wrong forum please feel free to move it---

 

The files the Q makes are amazing.. they are also very large....

I have a mac mini with a 500GB spinning drive and these files are big and not getting any smaller.  i am already off loading copies of the SD cards to an attached 3 TB HD which gets backed up via Backblaze and time machine.. but at this rate my attached HD will fill up --- then what???

 

Does anyone have any workflows they would like to share???

Link to post
Share on other sites

At the risk of sounding flippant: how about deleting photos, or not taking so many in the first place?  Are they all so good that you really want to keep full res DNGs?  Or could some be classified as 'snaps' which could just be down-sampled or stored only as JPEGs?  I struggle with this too, and I'm trying to be more disciplined on keeping only 'keepers' ... photos that I'd actually be proud to show other people.  I try to use small capacity sim cards to keep myself from just manic spray-and-pray photography: if there's a risk the card will actually fill up, you really pause and consider whether a given shot is worth it.  Perhaps try using analogue film from time to time to train / recall the discipline of taking just a few key shots per location / event?

 

Or you could completely ignore what I just said, and just get a bunch of external hard drives ... they're cheap (compared to a Leica Q)  :)

Link to post
Share on other sites

Don't store your images on external drives, hard drives are meant to spin, not sit on a shelf. There is currently no economical true archival method for consumers. I use LTO tapes to archive my images. If you must use hard drives, make sure you have at least two sets and you regularly use them. Cloud storage is a great option.

Link to post
Share on other sites

There was a study a while back that concluded it was actually cheaper not to delete any images, and instead just archive them all on hard drives. This was based upon the comparison of a photographer's hourly rate and the time it takes to sort and delete images, versus the cost of a hard drive.

 

Personally, I shoot in raw, and keep the current year order so of raw images on my local drive. After I have processed I export a 1400 pixel jpg image to Dropbox. When my local drive is getting full I move the raw files off to an external hard drive, and once that is full I get another external hard drive. I typically don't need to access the raw files once I have exported then to jpg, unless I want a high quality export to print. I also back everything up to my NAS so that I always have two copies.

  • Like 1
Link to post
Share on other sites

I use a Synology NAS unit with two mirrored drives, it is set to backup automatically to Google Drive. Works well for me and gives me access to all of my photos on the move if I need them. The Synology unit has a heap of extra plugins which I don't use but they might be handy for others. I also purge my photos fairly aggressively every 6 months to remove any I no longer need/like.

  • Like 2
Link to post
Share on other sites

At the risk of sounding flippant: how about deleting photos, or not taking so many in the first place? Are they all so good that you really want to keep full res DNGs? Or could some be classified as 'snaps' which could just be down-sampled or stored only as JPEGs? I struggle with this too, and I'm trying to be more disciplined on keeping only 'keepers' ... photos that I'd actually be proud to show other people. I try to use small capacity sim cards to keep myself from just manic spray-and-pray photography: if there's a risk the card will actually fill up, you really pause and consider whether a given shot is worth it. Perhaps try using analogue film from time to time to train / recall the discipline of taking just a few key shots per location / event?

 

Or you could completely ignore what I just said, and just get a bunch of external hard drives ... they're cheap (compared to a Leica Q) :)

35 photos was just under 900 MB which got me thinking. 1 roll of 'film' 1 days shoot is a lot of data

  • Like 1
Link to post
Share on other sites

35 photos was just under 900 MB which got me thinking. 1 roll of 'film' 1 days shoot is a lot of data

To simplify the calculation just assume 1 roll to be 1GB. One roll a day will fill 365GB in a year or about 700GB in two years. I think that ought to be manageable. 

  • Like 2
Link to post
Share on other sites

To simplify the calculation just assume 1 roll to be 1GB. One roll a day will fill 365GB in a year or about 700GB in two years. I think that ought to be manageable.

 

The Mac mini does not have easily upgradable HDD (at least for me). With my music and movie collections I only have 75-100MB free. Which makes an external drive a necessity.

Link to post
Share on other sites

As Ricky says, get a synology NAS, with a UPS connected to it.

Plus backup outside of where your NAS is located.

My synology NAS has 12 3TB disks in it with RAID6 config, meaning two disks can die and be swapped out without loosing any data.

+1. If you use an NAS as the primary location for your files, be sure to have a backup of the NAS as well. I had an NAS die on me after five years (power supply), and even though the disks were fine, the files can't easily be retrieved in that situation. Eventually I found a power supply on ebay that I could swap - but now I'm using two NAS, plus a an external hard drive as additional backup.

Link to post
Share on other sites

There was a study a while back that concluded it was actually cheaper not to delete any images, and instead just archive them all on hard drives. This was based upon the comparison of a photographer's hourly rate and the time it takes to sort and delete images, versus the cost of a hard drive.

  

Right, but (1) I, and probably at least some of the other people on this forum, are not professionals and (2) if you can't even be bothered (or don't have time) to view every image you shoot, what's the point of shooting them?  Surely if you go through the process of choosing a subset of your shots for post-processing, you can just select then delete all the rest in one go?  It takes seconds to delete hundreds of photos.  Are you going to go back and find photos from years ago on your storage drives?  You start getting needle-in-haystack search issues, unless you flag photos while reviewing them, which takes the same time as deleting...

 

Look, I also shoot lots and keep practically everything.  But I have a nagging feeling it is not the right way to go about making a portfolio to be proud of.  How many motifs / scenes / events really benefit from more than 5-6 shots?  Photography should be more like poetry than prose: narrative in a single shot where possible.

 

But for the OP, as you point out, I think the real issue now is that each individual shot is so big.  Personally I'm not very enthusiastic about ever increasing pixel count on the same size of sensor.  Nearly a gig per roll is a lot (too much?), isn't it?

Link to post
Share on other sites

Right, but (1) I, and probably at least some of the other people on this forum, are not professionals and (2) if you can't even be bothered (or don't have time) to view every image you shoot, what's the point of shooting them? Surely if you go through the process of choosing a subset of your shots for post-processing, you can just select then delete all the rest in one go? It takes seconds to delete hundreds of photos. Are you going to go back and find photos from years ago on your storage drives? You start getting needle-in-haystack search issues, unless you flag photos while reviewing them, which takes the same time as deleting...

 

Look, I also shoot lots and keep practically everything. But I have a nagging feeling it is not the right way to go about making a portfolio to be proud of. How many motifs / scenes / events really benefit from more than 5-6 shots? Photography should be more like poetry than prose: narrative in a single shot where possible.

 

But for the OP, as you point out, I think the real issue now is that each individual shot is so big. Personally I'm not very enthusiastic about ever increasing pixel count on the same size of sensor. Nearly a gig per roll is a lot (too much?), isn't it?

Not really sure what your point is since you disagree with what I've written, and then say you do the same. But anyway, that's where good rating and tagging comes into play.

Link to post
Share on other sites

I don't know ... you're certainly a much better photographer than I am so your method obviously works for you  :)

 

All I know is that the less I shoot and the more I throw away the happier I am with my images.  The OP asked for workflow ... this is one alternative to ever bigger and ever more drives and back ups.

Link to post
Share on other sites

I keep all my files in 6 month/half-yearly batches organised in LR6. The current 6 month period and previous one are kept on my laptop with the past periods on external hard-drive and backed up to a 2nd hard-drive for extra safety. My laptop with current data is backed up with 'time-machine' onto one portable hard-drive and one fixed one and I also use Carbon copy cloner. So when I travel I have a portable drive which is in 2 partitions one with the files and the other for time-machine. This way I have double backups of all files.

  • Like 1
Link to post
Share on other sites

... So when I travel I have a portable drive which is in 2 partitions one with the files and the other for time-machine. This way I have double backups of all files.

You realise that practically every mishap that would damage your files would affect both partitions, don't you?

  • Like 1
Link to post
Share on other sites

You realise that practically every mishap that would damage your files would affect both partitions, don't you?

Of course but as I said I have another backup of all the files and laptop at home on a RAID drive. I only use the partitioned  drive when travelling

Edited by viramati
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...