Shooting in DNG and how people deal with storage

[irenedp]

Advertisement (gone after registration)

2200 images at 47 MP is a lot to process. Maybe she will become more selective in the future, otherwise she will spend a lot in storage 🙂

 

I usually reuse the cards (that means that when they are full I format them and start again), but some photographers want to keep them as a safeguard system. 

Lightroom as a catalog is good and solid. Most books about it have chapters specific to organization.

Storing in external hard drives is a good solution.  Regardless of how much I have stored in one, I replace each drive every three years.  That should suffice. 

I also keep a NAS with a backup copy of everything and an Ice storage online for the fraction that I consider critical to keep. But those are just building redundancy. 

Edited June 2, 2023 by irenedp

[mradey]

On 5/31/2023 at 9:35 PM, Jeff S said:

Best way to preserve photos IMO is by making prints.  I can’t imagine my heirs going through hard drives, even if still functional. I use drives for my benefit while still around, not for others.  
 

Jeff

I came to the same conclusion - with the best will in the world, my family will not be browsing through my extensive Lightroom catalogue. 
 

instead of (currently) 100K digital images to browse, I print out my best and are stored in a dedicated plan chest. A couple of hundred wonderful prints (A2/A3 mainly) to hold and view. Image details (date, location, camera and film details written in pencil on the back)

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[roydonian]

10 hours ago, dpitt said:

My point is that you should try not to rely on any hardware or software as much as possible to archive your stuff.

 

To my mind, the way to ensure long-term retrieval of one's work is to use formats and storage media that the world regards as a standard, but being prepared to move to a later standard when one becomes established. (But in my case, the problem will soon become academic. I'm 80 years old, and have no children. Since neither of my two surviving relatives is a photographer, my hard disks and stored data and images seem destined to become landfill...)

Back in the days when I wrote books to earn my living, I used an Apple II, but my publisher grumbled about having to pay an outside company to convert my copy to a PC-compatible format. So in 1990 I switched to a PC running MS-DOS and Microsoft Word. I started the task on converting my Apple files into my new format, but soon abandoned it on the grounds that time spent generating more copy generated income, but hours of converting past work did not.

The only minor compatibility problem I've had came when I was the editor of magazine. The editorial and administrative staff of the publishing house used Windows, but the layout people used Macs. I wanted to run the Windows version of the well known page-design package used by the layout department so that I could make final corrections to the individual pages, but the Mac and Windows versions of the software were not sufficiently compatible to make this possible.

I have no hands-on experience of the Mac world, but was aware that Mac VMs could be run on a Windows machine, so would have assumed that these VMs could run all Mac OS and/or software.

I take your point about ZIP and Jazz disks. I tried both for a while, but abandoned them as soon as it became obvious that these were not becoming widely-accepted standards.

[zeitz]

37 minutes ago, roydonian said:

I have no hands-on experience of the Mac world, but was aware that Mac VMs could be run on a Windows machine, so would have assumed that these VMs could run all Mac OS and/or software.

Apparently Mac VMs can run on a Windows machine.  I have no experience with that.

https://www.partitionwizard.com/resizepartition/mac-emulator-for-windows.html

I have used Windows XP, 7 and 10 for some time on Macs, using both vmWare Fusion and Parallels.  Macs could either run other OS VMs (not just Windows) when Apple used Intel processors.  Or you could boot to Windows with Boot Camp.  With the conversion to Apple silicon using ARM technology, the situation has changed.  I believe it is now only possible to use ARM versions of an OS (Windows or Linux) with VM software such as current Parallels or vmWare Fusion software.  I will be upgrading from a high-end Intel MacMini to a MacStudio in two months.  I will then have to learn the ARM OS an VM situation.  I will now longer be able to run Windows XP, my favorite OS ever.

https://communities.vmware.com/t5/VMware-Fusion-Documents/Running-Fusion-on-an-Intel-Mac-and-upgrading-to-an-M1-M2-Mac/ta-p/2888565

[Luke_Miller]

I bought my first digital camera in 2002 and now my Lightroom catalog contains more than 130K images.  Initially, the digital cameras produced relatively small files, but now those from my current cameras are quite large.  All total they currently take up 3 TB of storage.  I use a large internal drive (currently an SSD) for working storage that contains all image files.  For backup I have a five-drive Synology Network Attached Storage (NAS) as well as offline storage using individual hard drives.  The offline storage employs a hard drive dock which is connected via USB 3 to my PC.  The dock permits the insertion of individual hard drives which I use to maintain copies of my working image drive.  The individual hard drives are then kept in hard plastic anti-static cases.  A utility program is used periodically to keep the contents of the working storage, NAS, and individual hard drives synchronized.

For less demanding needs I think the hard drive dock and individual hard drives makes for a cost effective and relatively future-proof approach.

Edited June 2, 2023 by Luke_Miller

[roydonian]

1 hour ago, Luke_Miller said:

The individual hard drives are then kept in hard plastic anti-static cases. 

Sounds like a good idea! Is there a company that makes anti-static cases for hard drives?

 

 

[Luke_Miller]

Advertisement (gone after registration)

Here is what I use:

CRU-DataPort Drivebox Anti-Static Case for 3.5" Hard (bhphotovideo.com)

[marchyman]

16 hours ago, dpitt said:

AFAIK there are no easy ways to run a VM for Mac OS classic.

https://infinitemac.org

😁

[LocalHero1953]

On 5/31/2023 at 9:35 PM, Jeff S said:

Best way to preserve photos IMO is by making prints.  I can’t imagine my heirs going through hard drives, even if still functional. I use drives for my benefit while still around, not for others.  
 

Jeff

Roughly once a year, usually at Christmas when they are in residence here, my family (three daughters) work their way through my Lightroom catalogue, harvesting photos I have taken of them and their kids. I doubt that they (and their kids) will have any problem recovering my photos using whatever version of Lightroom I have left them - if they want the photos of course......

[Jeff S]

23 minutes ago, LocalHero1953 said:

Roughly once a year, usually at Christmas when they are in residence here, my family (three daughters) work their way through my Lightroom catalogue, harvesting photos I have taken of them and their kids. I doubt that they (and their kids) will have any problem recovering my photos using whatever version of Lightroom I have left them - if they want the photos of course......

If I told my family that my photos were stored in my “Lightroom”, they’d probably be searching through the sunniest room in my house looking through cabinets.  Good thing I gave up my darkroom long ago..  those prints and negs are stored along with digital prints.

Jeff

[LocalHero1953]

7 minutes ago, Jeff S said:

If I told my family that my photos were stored in my “Lightroom”, they’d probably be searching through the sunniest room in my house looking through cabinets.  Good thing I gave up my darkroom long ago..  those prints and negs are stored along with digital prints.

Jeff

I actually agree with you that the best form of archive is one that can be interrogated by the Mk I Eyeball. If I had the discipline I would have produced an annual photobook for all family, social and travel photos. As it is, I have produced two photobooks for the family than run from 1865 to 1953 and 1953 to 1976. The task of editing the plethora of analogue photos since then, let alone the digital explosion since around 2002, defeats me.

[dpitt]

On 6/2/2023 at 2:03 PM, roydonian said:

Back in the days when I wrote books to earn my living, I used an Apple II, but my publisher grumbled about having to pay an outside company to convert my copy to a PC-compatible format. So in 1990 I switched to a PC running MS-DOS and Microsoft Word. I started the task on converting my Apple files into my new format, but soon abandoned it on the grounds that time spent generating more copy generated income, but hours of converting past work did not.

...

I take your point about ZIP and Jazz disks. I tried both for a while, but abandoned them as soon as it became obvious that these were not becoming widely-accepted standards.

This is an illustration of what I tried to say. Even a 20-30 year gap can be hard to cross in this digital era.

To convert your Apple II files would now be even harder and more time consuming than it was in 1990, even if you have taken the precaution to copy them over to a current drive.

If your work was on a ZIP or Jazz disk today, it would add and additional hurdle. No VM machine can help with that.

Who knows what file format  and hardware will be main stream in 2050? Maybe it does not even exist yet. Maybe nobody will run anything remotely similar to the current Windows or Mac OS anymore. Will the software of 2050 still be able to open a JPEG or an M11 DNG file? And 20 years is a relatively short period in history. What if Ansel Adams would have lived now and we were living 80 years later, say in 2100? Would we be able to admire his pictures? And would we be able to bid absurd prices for his original files (RAW files) today like we do with his negatives now? I am afraid it is more likely we would never have heard of Ansel Adams and that we would not be able to open his RAW files.

On 6/2/2023 at 7:53 PM, marchyman said:

https://infinitemac.org

😁

Thank you for the site. It reminds me of how old I am...

But it also reminds me of how much fun I had with these old Mac systems. I can recommend the Games/Ambrosia Software folder in the Mac OS9 VM for a few of the best games ever made IMO.

[roydonian]

13 hours ago, dpitt said:

This is an illustration of what I tried to say. Even a 20-30 year gap can be hard to cross in this digital era.

 

1990 was more than 30 years ago, but I can still use all my PC-generated files from that era.

The GIF format is 36 years old but I can still read it.

The MS-DOS version Microsoft Word is close to 40 years old, but I can still read its files.

All I had to do was to move stuff from 5.25 inch disks to 3.5 inch disks and then to hard disks. It was just a matter of thinking ahead, sticking with widely-used types of media, and doing the necessary transfers.

When a new hardware architecture and operating system finally emerges, its large-scale acceptance will depend on giving the user the ability to run the older OS and software when this is required. I often wonder how the world of the personal computer would have evolved if Apple had given the Macintosh the ability to run Apple II software. 

I read some years ago that the US Department of Defense was running emulation programs in modern computers in order to simulate older computers that could be used run emulations of even older computers so that the DoD could run old software that was still needed. Running these off-the-shelf emulations within other off-the-self emulations was an inexpensive alternative to the cost of having to develop, debug, and maintain modern replacements for old software.

[dpitt]

2 hours ago, roydonian said:

 

1990 was more than 30 years ago, but I can still use all my PC-generated files from that era.

The GIF format is 36 years old but I can still read it.

The MS-DOS version Microsoft Word is close to 40 years old, but I can still read its files.

All I had to do was to move stuff from 5.25 inch disks to 3.5 inch disks and then to hard disks. It was just a matter of thinking ahead, sticking with widely-used types of media, and doing the necessary transfers.

When a new hardware architecture and operating system finally emerges, its large-scale acceptance will depend on giving the user the ability to run the older OS and software when this is required. I often wonder how the world of the personal computer would have evolved if Apple had given the Macintosh the ability to run Apple II software. 

I read some years ago that the US Department of Defense was running emulation programs in modern computers in order to simulate older computers that could be used run emulations of even older computers so that the DoD could run old software that was still needed. Running these off-the-shelf emulations within other off-the-self emulations was an inexpensive alternative to the cost of having to develop, debug, and maintain modern replacements for old software.

My main concern is not the main/plain file formats. GIF, JPEG and TIFF will probably last long enough. But it is the more proprietary stuff that is concerning.

Running VM inside VM does not sound easy. Nothing lasts forever in software although some users (like the US DoD seem to think so). I was involved in a complete rewrite of a some crucial piece of software used by a multinational. They were using it in multiple sites, and it was crucial for their quality control, so could not ship anything without it. They 'forgot' that everything was relying on hardware and software that did only run on Windows 95. In 2005 they realized they made a big mistake. No one could supply PCs or even parts capable of running Win 95 anymore!

It took several millions of EUR and about 2 years to completely rewrite the original package for XP (and for their instrumentation software that was by then 5 generations further) + develop new hardware for the new PCs to work with the package. 

You are lucky Microsoft Word still exists today. Ask MS DOS Word Perfect users if they can still read their files... And Word Perfect was in the late 80's as dominant as Word is now. Lots of other word processors have not survived and will probably not even run on the VM (like anything that relied on dongles to run). If it is that hard to read something as simple as a text with some make up, imaging how hard it will be to read an Aperture or even old Lightroom catalog 20 years from now. Image processing engines will have evolved beyond recognition, so you will probably be able to read the original RAW file, but any edits will have to be done again manually and the original edits will be lost. At least, that is what I think will happen as maintaining compatibility with old engines gets harder and harder. And if the company that makes your software goes out of business you are even worse off.

 

 

[marchyman]

My biggest issue with 25 year old digital images is that at that time I thought 512x384 was a decent resolution.  It took up most of a 640x480 screen. What more did I need?  🤣

[roydonian]

3 hours ago, dpitt said:

You are lucky Microsoft Word still exists today. Ask MS DOS Word Perfect users if they can still read their files...

In the late 1980s There was a program called Word Exchange that could convert from:

• DEC WPS PLUS (DX) Versions through 1.0
• DisplayWriterM 2, 3, & 4 Releases through 1.00
• MultiMate Versions through Advantage™ II
• Samna Word Versions through Samna Word IV
• Wang PC (IWP) Versions through 2.6
• WordPerfect Versions through 5.0
• WordStar Releases through 5.0
• WordStar 2000 Releases through 3.00
• Volkswriter 3 Versions through 1.0

I sent my copy to landfill a few months ago given that I haven't had to use it for for more than 30 years, and will never again be faced with any of these formats. But it would remain an invaluable tool for anyone responsible for archived software.

To my mind, anyone who trusted work files to software than needed a dongle without getting a good computer technician to patch the program deserves any problems they get! 

Edited June 7, 2023 by roydonian

[Alberti]

For the OP:

I once on a holiday tried storing on my ipad - that was a disaster. How to get it off? I have LR on the pad, even then, I had a problem getting it out. And copying to the ipad takes ages, even in file transfer mode. My suggestion is a USB 'hub' that allows a portable NVME flash drive to be connected and has a card reader in it; there are even small hubs that have both an internal NVME drive and a card reader. This should allow you to copy directly from the SD to that external disk to unload the photo card - bypassing the internal memory of the ipad. And it allows fast access at home again. [But I have this in mind, not in use yet]

JPG is hard to edit. DNG is great. I can understand. I admire the Q series and love to give my wife the newest.

For storage at home, I switched to two dual slot USB NVME readers, with 2TB cards. In total I have 3 TB available this way. Then I have even a faster card for the catalogue of LR. And its associated normal size temporary image files. This way I can import with high speed (the importing bottleneck is not just the card reader, but also writing elsewhere.)

2.200 images in a holiday in a week is normal & even with DNG this is doable (not many mortals do that 52 weeks a year, you'ld have 100K a year and you will need 4.000 hours just to look at them, select, etc etc plus editing time, reading this forum  🤕 - a full time job that some pro's let a assistant do).

Example at the amazon:

USB-C Hub with M.2 NVMe SSD Enclosure, SD/TF Card Reader for Windows, Mac OS, iPad OS

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

Edited June 13, 2023 by Alberti

[jaapv]

The real risk of using huge cards is that cards can and do fail. 

[Alberti]

Sure. There are standard back-up strategies

- for during import (LR manages that, write to external disk)

- and during use. And CCC manages this for me, Time Machine does not (nobody has a 4TB internal SSD).

[Advertisement]

Hallo,

Look here: Interesting Leica gear

[marchyman]

10 hours ago, Alberti said:

How to get it off? I have LR on the pad, even then, I had a problem getting it out.

Whatever I import into my iPad automagically shows up in my "LrSync" folder on my desktop in LrC.  I simply drag it from that folder to another folder inside of Lightroom.   That said, I've never imported more than a handful of images at a time and I do not leave them in the Adobe cloud.