RAID systems experience?

[Westerwald-Leica]

Advertisement (gone after registration)

With more and more gigabytes of images piling up on my hard disk I am thinking about a RAID system to safely store them. Has anyone been able to collect experience with a RAID 5 or RAID 6 system? Brand, duration of use, flaws?

 

Thanks

Bernd

[Marty]

IMHO RAID 5 or 6 only makes sense in a corporate environment where data has to be safe from the moment it is written, and with spare disks to hand for replacement. For image storage I would simply suggest RAID 0 for speed, and a good backup policy. Just my view.

[wparsonsgisnet]

If you are the only user of your system, then a raid solution may not be necessary.

 

If you are scrupulous about your backups, and you can use an auto scheduler, mirroring external disks will be adequate -- and the disks may last longer. A raid array exercises the disks constantly.

[marban]

i'm using 4 x 2tb disks as 2x striped software raids in my mac pro - one for the actual data and the other for timemachine + a drobo pro with 8tb via iscsi that gets irregular snapshots for off-site storage.

it's an expensive but best of all worlds solution.

[k-hawinkler]

I have been using for a number of years SATA enclosures from FirmTek SeriTek/5PM: Five-Bay, Hot-Swap, External, eSATA Port Multiplier Enclosure on my Macs. The SeriTek/5PM 5-bay port multiplier enclosure only needs one cable to connect to a computer. When hard drive sizes were smaller and of lower speed I ran the enclosures as RAID 0. Nowadays, with fast 1 or 2TB hard drives available, I just use them as 5 normal drives. They are plenty fast for making multiple backups, of course on different drives or different enclosures.

 

With the SeriTek/2SM2-E two of these enclosures can be attached to a laptop. They also have cards to attach even more enclosures to PCI-X or PCI-Express adapters. The enclosures support up to 3.0Gbps transfer rates, needed as multiple drives can be active on the same cable simultaneously.

 

Occasionally, I run bit-for-bit comparisons to ensure data integrity between multiple backups and use software for incremental backups. I still use SilverKeeper for that, even on Snow Leopard, although Silverkeeper was never certified by the developer LaCie for Snow Leopard. So far, I have not encountered any problems. Of course, use at your own risk.

 

The beauty of this approach is that these enclosures and drives only need to be powered up briefly when archiving new data from the working hard drives or making backups.

 

These enclosures are pretty quiet and well suited for your digital music collection as well.

 

Other than being a customer, I have no connection to FirmTek. Previously, I used RAID enclosures from another vendor that never really worked reliably. Go figure.

 

K-H.

 

PS: There is currently another interesting thread dealing with DNGs with data validation hash http://www.l-camera-forum.com/leica-forum/leica-m8-forum/121626-m8-dngs-data-validation-hash.html.

[Bill W]

I use the back up "time machine" built into the Mac. I have 4 drives in my Mac and will occassionally just do a drop of my folder that has all of my photography into one of the extra drives. Probably not the best setup but I always have a backup of the images I need to keep the most. Drives are really cheap these days, you could do a back up and take it out and replace the drive with another. Place the drive in a safe enviornment..

[meino]

Advertisement (gone after registration)

With more and more gigabytes of images piling up on my hard disk I am thinking about a RAID system to safely store them. Has anyone been able to collect experience with a RAID 5 or RAID 6 system? Brand, duration of use, flaws?

 

Thanks

Bernd

I.m using a Synology DS210J NAS with two 1.5 tb samsung drives in a raid 1 configuration. After some startup problems, which were solved by a firmware upgrade, it has worked without problems.

[viramati]

I use this

 

G-Technology - G-RAID - Professional High-Performance Dual-Drive Storage System

 

with macbook pro 17" and keep everything over 6 months old on it. Also use it with 'time machine'

[sanyasi]

I was advised against RAID. You apparently run a greater risk of losing both hard drives because of interaction between them. I went with two hard drives.

 

Also, remember one important point. You should be worried about more than just hard drive failures. Floods and fire are a risk, too. Ideally, you should keep a complete set of your files off site--hobbyists might consider the trunk of your car or your office.

 

I would be interested if anyone has any experience with photos and a cloud system. I use a cloud for my small business. I would like to use it for photos, but that gets expensive given the size. Any good sites?

 

Finally, learn the delete shortcut. I kept every shot for five years. It got ridiculous. I had stuff I would never print. This past winter I went through the files and did massive deletes. I still have 3 times more than I will ever print. As with most spring cleaning, once the stuff is gone, you don't miss it. I am now far more brutal when I come back from a photo shoot.

[vanhulsenbeek]

Somewhat off-topic and perhaps a subject for the Digital Post-Processing Forum ?

[marban]

I was advised against RAID. You apparently run a greater risk of losing both hard drives because of interaction between them. I went with two hard drives.

 

that's why you use additional physical disks off-site, no matter what config you go for.

 

I would be interested if anyone has any experience with photos and a cloud system. I use a cloud for my small business. I would like to use it for photos, but that gets expensive given the size. Any good sites?

 

Online Backup for Mac | Arq | Haystack Software

 

backs up encrypted to amazon S3, i.e. no third party involved.

[Westerwald-Leica]

Thank you all. I may rethink RAID after all. May also save me some money ;-)

 

Bernd

[k-hawinkler]

You also may want to check your electricity usage with a Kill A Watt monitor.

 

K-H.

[Mauribix]

For image storage I would simply suggest RAID 0 for speed

 

That's the best way to risk and loose all your pictures.

Even with a backup policy, the risk is around the corner, and speed improvements with such a design is just a minor advantage (after all we're talking about few MB at picture) on the other hand for videos, speed advantage would be then quite considerable you're right.

RAID5 is obviously good for security and speed, but RAID6 is absolutely outstanding in every aspect.

 

You could create a RAID6 with as low as 3 disks, got two of them crashed and still get your data safe.

 

You could create a RAID5 with as low as 3 disks as well, but while you could be safe whit one crashed disk, a 2-disks failure would be simply unrecoverable.

 

You could create a RAID0 with as low as 2 disks then, but a single disk failure is a total disaster. So you get 50% of chances to loose your data, or most likely, you doubled your possibilities to get that same loss of data.

 

Statistics is not an opinion, the best way to keep data safe is a RAID1 or... an external drive with differential/incremental backups.

 

But if you need for speed, then the RAID6 is the way to go IMHO.

 

 

Oh well, keep in mind that a RAID system *need to be used* to perform well and be justified as a cost; otherway a couple of external drives for continuous backup are cheaper and safer.

[k-hawinkler]

Maurizio,

 

Great points.

 

To protect against theft, fire, etc you also need off-site backup.

Various online services come to mind.

 

K-H.

[Rick]

Drobo is a great backup system. And, the best backup system is one that you use.

[beewee]

I've written a short article on my blog regarding RAID here.

 

The skinny: Personally I run RAID 6 (which is slower than RAID5) but affords you up to 2 simultaneous drive failures and still lets you chug along without any downtime. The RAID management is done with a dedicated 3Ware raid card which is many times faster than software raid and doesn't tax your CPU.

 

The server I run which runs as both a web server as well as a storage and ftp server is on 24/7 and allows me to access my data anywhere with an internet connection. The RAID card has an independent battery backup incase the computer turns off due to a power supply blowing up, when power is restored, the data will finish writing to the drives from the RAID card memory. The server itself is also on battery backup incase of power failures and fluctuation.

 

I backup onto an offline drive periodically and store the backup drive offsite incase my server dies in a fire, flood or is stolen. When I'm shooting in the field, I download my images onto two seperate drives simultaneously and always have 2 copies, one working copy and one backup.

 

I do a lot of work with GPS data processing and my standard procedure is to always have 3 copies before I do anything. I have 2 backups in seperate locations and a third working copy used for processing to avoid data corruption on the original files.

 

Call me paranoid but Murphy was an optimist.

[S. Wong]

Based on my experience with large storage arrays, unless you're using truly massive amounts of data, just full doing fully copies (Raid 1, Mirroring) is the best option. This is what I use personally on my Win 2k3 server at home.

 

1TB hdds are only $100, and will provide the performance you need via usb/sata. Once full, label and store somewhere, and you should be able to power it on later and access the backups.

 

If you really want to run a RAID 5, its best to use about 4+1 (5 disks) 1tb drives for maximum use and protection. If you want more protection, run 6+2 (8 drive) RAID 6. I don't know of separate/standalone systems which I would recommend for home use, but for business/enterprise, I have some experience.

 

If you want to put this in a home PC/MAC, the Mac Pro towers only have slots for 5 drives, and I wouldn't recommend internal except for Raid 1 due to the space constraints. If you have the space, internal RAID cards from Adaptec or LSI is the way to go. Onboard RAID won't give you the same performance as these.

 

Again, none of these choices are cheap, so something to think about.

 

Thanks,

-Steven

[haroldp]

At home for my photo's and files I use an external jbod tower with Raid 1 implemented via Apple Leopard disk management. While software RAID such as this is marginally slower than an embedded raid chipset, It offers the advantages of continuous support from APPLE, simple topology, and the ability to access each drive separately as a single disk with integrity, which no striped topology can do.

 

I have had the experience of having hardware raid in a multidrive device fail at the chipset, and the vendor was onto a new release and chipset which could not read the striping, and they would not support the prior level. It is a very well known vendor. I was able to recover from from an asynchronous backup ( I was a professional Data Base Administrator for large financial institutions for 25 years before going the enteprenuerial software route) and totally paranoid about data.

 

I also use Time Machine for near time async backup to a different drive system, and NTI shadow to back up daily to a NAS device in a fire resistant room (4 poured concrete walls) on my network backbone, in the opposite corner of the house.

 

I am considering cloud backup ( MOZY ) as well, but am nervous about how long the initial image might take since I am talking over a terabyte.

 

I do use a Raid 0 array for cache / temp for photoshop and Capture NX2, and another Raid 0 array for the Operating System boot drive, to make system paging faster. This made a significant difference in photo editing performance.

 

I hope this is helpful.

 

Regards .... H

[Advertisement]

Hallo,

Look here: Interesting Leica gear

[SJP]

I am happy enough with this configured as RAID 1

 

29/10/2008: Secure data backup with Verbatim | Verbatim Europe - Data Storage, Computer & Imaging Consumables

 

using USB for the time being until I can find time to get the eSATA working

 

If the failure chance of a single drive is p then RAID 1 failure chance is p^2 - so much more secure than a single disk.