Jump to content

backing up M8 DNG files to external hard drive


mikpeter

Recommended Posts

Advertisement (gone after registration)

If you are finding that Time Machine is gobbling up disk space, try excluding the cache directory. Also if you use something like VMware Fusion to provide Windows emulation you might want to exclude that as it uses a huge file to hold the Window's environment, and that file is going to change every time you use Fusion, and even _as_ you're using Fusion - therefore Time Machine will back up a different version of a multi gigabyte file every time it runs while you are using Fusion.

Link to post
Share on other sites

Robert,

 

I agree with your sentiment about too much FUD - but I think that Backup/restore is one area where I look for certainty that when a disaster strikes my system/house/office whatever, that I can find a Backup that I can (with certainty) use to recreate whatever has been lost. 100% is clearly unobtainable (airbuses crash, nuclear power plants explode etc.). But the probability of error must be very low. So Keep It Simple Soldier must be good advice.

 

My main point is that anyone using TM, to back up a LR catalog, needs to consider all the information available, and make their own judgment.

 

LR uses SQLite (which is a single user/machine product) - which suggests that making a copy while the database file is in use could be problematic. (see Lightroom Forums - Index). Doing a few tests will not prove anything (unless it fails), I made an OSX file copy in the middle of a 250 image import and the saved catalog showed 72 imports and seemed to work just fine.

 

Apple still states that one should not run TM on an active ( i.e.open) Aperture Library.

 

TM is great, just don't run it concurrently with LR or Aperture. There is also the problem of 'space', if a file has changed it will result in a full copy to the backup (my ~/Lightroom folder is currently 600 Mbytes for a library of under 1500 images (DNG and TIFF) held in separate folders. Previews account for 560Mbytes of that.

 

Regards

 

David

Link to post
Share on other sites

Here's how I organize and back up my photos. First of all I have an SD card reader in my computer and use a program called Picture Information Extractor or "PIE" from a company called Picmeta Systems that renames the camera files based on the file creation date. For example, if I shot some fireworks photos on the 4th of July then a file is renamed 090704-2045-10. That photo was taken at 8:45pm & 10 seconds on July 4th.

 

I also built a cheapo little computer that has no applications running, just the O/S and a huge hard drive. Every few days I copy all of my latest images over to that machine's HD. This cheapo machine is connected to my home network via Ethernet. This entire drive is my back up to the local drive on my main computer - the one I do all my work and image manipulation on.

 

The result is I have two hard drives containing all my photos. Should one computer's drive fail, I rebuild it from the other "good" drive. I could just as easily use an external hard drive for this.

 

All photos are sorted into folders by month and year and are relatively easy to find by looking at thumbnail images that the O/S can create.

 

I shoot with the camera set to basic jpg & DNG. I have two images with the same name, just different file extensions: jpg and DNG.

 

Oh, and occasionally I burn some DVD's of the folders to further improve the survivability of my images.

Link to post
Share on other sites

If you are finding that Time Machine is gobbling up disk space, try excluding the cache directory. Also if you use something like VMware Fusion to provide Windows emulation you might want to exclude that as it uses a huge file to hold the Window's environment, and that file is going to change every time you use Fusion, and even _as_ you're using Fusion - therefore Time Machine will back up a different version of a multi gigabyte file every time it runs while you are using Fusion.

 

Steve,

 

Apart from the eating up of HD space, I just don't trust the Time Machine program. Now it may well have improved since I gave it up last November when I had had a number of lock ups, corrupted sparse images and failed back ups to a Time Capsule. In fact Apple gave me a new TC and as the old one was a US model said I could keep it. I then got a lock up on the new TC, so switched back to Chronosync. Since then, using both my UK (European Mk2) and French (US model Mk1) TC's with Chronosync rather than TM, I have not had a single problem with either the old "faulty" TC or the replacement. I agree that the concept of Time Machine is great but I think there are still bugs to be worked out in it. I am guessing that the Snow Leopard version may be bug free. Sadly of course, it will not run on my 2xG5 Powermac or my wife's G5 iMac.

 

Wilson

Link to post
Share on other sites

The thing I dislike about many backup programs (Like Time Machine) is that they create a proprietary file that can only be read or restored from that application... I much prefer a application that creates a mirroring of ones data... An exact copy that can easily be accessed and understood... An external drive that is a mirrored can easily be plugged into a new computer and be instantly used with no additional software needed

Link to post
Share on other sites

The thing I dislike about many backup programs (Like Time Machine) is that they create a proprietary file that can only be read or restored from that application... I much prefer a application that creates a mirroring of ones data... An exact copy that can easily be accessed and understood... An external drive that is a mirrored can easily be plugged into a new computer and be instantly used with no additional software needed

 

That is why I like Chronosync on Macs. Apart from the back-up record file, which is a proprietary format, you can poke about, read, use and or modify the folders and files in the back up, just as if it was your regular hard disc. When the sparse image file on a Time Machine back becomes corrupt or locked up, it is a bit more effort to get inside it.

 

Wilson

Link to post
Share on other sites

Advertisement (gone after registration)

The thing I dislike about many backup programs (Like Time Machine) is that they create a proprietary file that can only be read or restored from that application... I much prefer a application that creates a mirroring of ones data... An exact copy that can easily be accessed and understood... An external drive that is a mirrored can easily be plugged into a new computer and be instantly used with no additional software needed

 

Have a look at:

 

Mac OS X 10.5 Leopard: the Ars Technica review - Ars Technica

 

Three quotes:

 

"The smallest unit of data that Time Machine will backup is an individual file. That means that if you change one byte in a 10GB file, that entire 10GB file needs to be copied to the backup volume."

 

Note LR on a Mac saves Develop actions in the DNG files rather than to a separate XMP file.

 

"On the bright side, the backup format isn't really a "format" at all. It's a bunch of plain files on disk in date-stamped folders. Don't like the loony space-motif file restoration interface in the Finder? Don't use it. You can wander around the Backups.backupdb folder using the Finder, the command line, or any other file manager."

 

"Time Machine's backup format also means that it's not possible to boot from a Time Machine backup volume. Instead, you must boot from a Leopard install DVD and restore the Time Machine backup onto a new volume."

 

Hope that helps

 

David

Link to post
Share on other sites

Have a look at:

 

Mac OS X 10.5 Leopard: the Ars Technica review - Ars Technica

 

Three quotes:

 

 

"On the bright side, the backup format isn't really a "format" at all. It's a bunch of plain files on disk in date-stamped folders. Don't like the loony space-motif file restoration interface in the Finder? Don't use it. You can wander around the Backups.backupdb folder using the Finder, the command line, or any other file manager."

 

 

Hope that helps

 

David

 

David,

 

Sadly if the sparse image file becomes corrupted or locked during a back up, you can often no longer access the TM back up with Finder and have to use a program such as Data Rescue. The files will often have lost their correct name and finding what you want is a nightmare. I hope that TM will improve on Snow Leopard but for the moment, I find it easier to avoid it.

 

Wilson

Link to post
Share on other sites

I don't see what is hard about having a backup of your image files.

Hopefully they are all in one place (folder and subfolders by year or job or whatever way you want to store them) on your hard drive.

Simple get a external HDD and copy those folder & subfolder over to your external.

 

Now the question is what program to use to copy newer image files over to that same external HDD. You can do it manually or you can look for some type of synchronization program that will check the internal HDD folder structure of your image file folders & subfolders and compare it to the external folder structure and copy over and or delete image files that are new/er and get rid of the dead weight of the files you deleted.

Using any type of backup program that writes some type of proprietary file that can only be read by that program is IMHO risky and you need to do multiple copies everytime you use it. Or at least that is my experience with backup/imaging programs.

 

That's not to say I don't use one of these type programs to do images of my main OS and program hard drives, I do. But for my image files, all of them ever shot with any camera, even scanned film images (the ones that IMHO are keepers), I want a real copy of them in there entirety.

Link to post
Share on other sites

David,

 

Sadly if the sparse image file becomes corrupted or locked during a back up, you can often no longer access the TM back up with Finder and have to use a program such as Data Rescue. The files will often have lost their correct name and finding what you want is a nightmare. I hope that TM will improve on Snow Leopard but for the moment, I find it easier to avoid it.

 

Wilson

 

Wilson,

 

May I ask - what is this 'sparse image' file?

 

Davis

Link to post
Share on other sites

Wilson,

 

May I ask - what is this 'sparse image' file?

 

Davis

 

It is what TM uses while the back up is being written. It is a proprietary data compaction format used by OSX.

 

Wilson

Link to post
Share on other sites

Wilson,

 

I have done some research that suggests that TM to a local (e.g. Firewire connected) disc produces a volume which has an 'apparently conventional' directory structure (which supports hard links at the directory level). But, if TM uses a Networked (remote) drive then it creates a sparse image file and backs up into that instead.

 

See,

Time Machine to network share [Archive] - Mac Forums

 

and here

 

macosxhints.com - 10.5: Time machine, corrupt sparse images, and panics

 

Conclusion? TM is not a panacea and needs to be used with care and understanding.

 

Thank-you for the insight

 

David

 

David

Link to post
Share on other sites

Wilson,

 

I have done some research that suggests that TM to a local (e.g. Firewire connected) disc produces a volume which has an 'apparently conventional' directory structure (which supports hard links at the directory level). But, if TM uses a Networked (remote) drive then it creates a sparse image file and backs up into that instead.

 

See,

Time Machine to network share [Archive] - Mac Forums

 

and here

 

macosxhints.com - 10.5: Time machine, corrupt sparse images, and panics

 

Conclusion? TM is not a panacea and needs to be used with care and understanding.

 

Thank-you for the insight

 

David

 

David

 

David,

 

I am in total agreement with you. I have one firewire connected drive (a 1TB Freecom Datatank) and one Ethernet connected wired with all Cat 6E cables via a hub to a Time Capsule. I had been doing my regular back ups from various Mac laptops as well as my scheduled back up from my iMac and store copy #1 of all my archived DNG's on the Datatank. I did not therefore, want to risk/pollute that with TM back up. Therefore I have only ever tried TM back ups to either of my remote TC's in the UK and France. As I posted above, not a happy experience. If you look at Apple discussion forums (fora?) I am not alone.

 

 

Wilson

Link to post
Share on other sites

I've just restored my entire system from a TM backup onto my MacBook, as my main Mac has had to go in for repair. Everything restored perfectly (Including some 40,000 photographs.) so it's not all doom and gloom. :D

Link to post
Share on other sites

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...