[Tirade]

My music archive went down a few days ago. I had everything backed up (and even the backups were backed up) but going RAID1 with an external backup of that was a tremendous amount of wasted space. I had close to 2TB of FLAC files but was taking up 6TB of space and everything was housed on separate drives and a small pain to manage. I sold off my old drives and was going to invest in some 1TB drives but the price point just isnt there yet so I went with 750's. I was stuck with my existing RAID before because I could not rebuild/expand it without losing everything (software RAID), this time around I went hardware RAID.

So on to my questions.

As I mentioned I went hardware RAID this time and invested in a decent RAID controller with online capacity expansion and online RAID level migration. (heres the card http://www.highpoint-tech.com/USA/rr2340.htm). Whats the general consensus on RAID5 reliability? The Highpoint card doesnt support RAID6 so Ive been thinking of moving to RAID50 for a little extra reliability. Whats the feeling on it? Worth losing the extra storage of a 2nd drive?

I was using Helium for my music archive database before and Im just curious if there is anything else out there? Im not unhappy with it, but something a little quicker would be nice (although the speed may have been the old PC and not Helium)

How do you guys have your music sorted? I had it split on to drives (A-F, I-N, etc etc), but its not ideal. Having them all in one folder seems a little easier but scrolling through a few thousand CD's at a time would get old fast I imagine. Maybe I dont have a choice

Everything for the new server is ordered and should be at the house by Tues so I hope to come back from travel on Wed and put it all together. Here are the parts.

http://img292.imageshack.us/my.php?image=mediaservereq2.jpg



Edit: All the parts arrived yesterday and I got it built last night!

http://s275.photobucket.com/albums/jj305/Tirade75/Media%20Server/


I ended up going with an Areca 1280 RAID6 controller that I got a lucky price on. My only mistake was the lack of a gigabit NIC on the motherboard so I had to buy a separate card this morning.

[BradJudy]

Whats the general consensus on RAID5 reliability? The Highpoint card doesnt support RAID6 so Ive been thinking of moving to RAID50 for a little extra reliability. Whats the feeling on it? Worth losing the extra storage of a 2nd drive?

It's not the reliability of RAID5 that's the question, it's the reliability of drives.  With RAID5, when a drive files and you replace it, it has to to a lot of math and I/O processing to fill the replaced drive with the proper data.  In the world of very large SATA drives, this can take a LONG time.  The risk is that sometime during the rebuild, another drive will fail and you'll lose everything.  Thus, RAID 6 has been gaining popularity, allowing you to lose two drives without losing all of your data. 

With RAID 50, you hedge your bets with another gamble.  You're splitting six drives into two RAID 5 arrays.  If a second drive fails during a rebuild of a new drive, you have a 40% chance of losing your data and a 60% chance it will be a drive in the second array and you'll have to do two simultaneous rebuilds, but won't lose any data. 

To me, the question becomes: Do you have any other backups?  If you're only going to store the data in this array, then go for more redundancy.  If you'll have another backup, then I wouldn't worry about dual drive failures too much. 

[JEaton]

My music archive went down a few days ago. I had everything backed up (and even the backups were backed up) but going RAID1 with an external backup of that was a tremendous amount of wasted space. I had close to 2TB of FLAC files but was taking up 6TB of space and everything was housed on separate drives and a small pain to manage. I sold off my old drives and was going to invest in some 1TB drives but the price point just isnt there yet so I went with 750's. I was stuck with my existing RAID before because I could not rebuild/expand it without losing everything (software RAID), this time around I went hardware RAID.

If you have backups then RAID is almost a complete wast of money and drive space.  What RAID enables you to do is to keep on running in the event of a hard disk failure.  Read that sentence again - it does NOT protect your data in any way.  It protects your UP TIME.

It's great for corporate data, where a company cannot afford to have their data systems go down for a few days or even a few hours.  The systems must run 24x7, disk failures and all.  Ask yourself, how important that is for your use?  If your music library were offline for a day or two while you installed a replacement drive and restored from your backup, is that such an inconvenience that it's worth spending hundreds of extra dollars to avoid? Thankfully, for myself, I can say I'd live through it without much trama.

As I mentioned I went hardware RAID this time and invested in a decent RAID controller with online capacity expansion and online RAID level migration. (heres the card http://www.highpoint-tech.com/USA/rr2340.htm). Whats the general consensus on RAID5 reliability? The Highpoint card doesnt support RAID6 so Ive been thinking of moving to RAID50 for a little extra reliability. Whats the feeling on it? Worth losing the extra storage of a 2nd drive?

If you still feel the need for RAID, then with your storage needs, RAID1 really isn't feasible.  RAID5 (or RAID6 if you want to spend even more money) is the only choice.  RAID50 will give you greater _PERFORMANCE_, but at double the cost in drives and nearly double the cost in controllers.  Probably the last thing you need to be buying is more performance for the lowly needs of a music server.

How do you guys have your music sorted? I had it split on to drives (A-F, I-N, etc etc), but its not ideal. Having them all in one folder seems a little easier but scrolling through a few thousand CD's at a time would get old fast I imagine. Maybe I dont have a choice

I organize my library by artist, A-M on one drive, N-Z on another.  It might be a little nicer to have it all on one drive, but not much.   That's probably the one nice thing about RAID5 - you can create gigantic drive spaces that can be be many times larger than that available on a single drive.

For RAID5 you need at least three drives, so you'll need at least a four drive SATA RAID controller.  Expect to spend $300-400 for a good controller of that capacity.  The 3ware 9650 series is nice, in that they use fast, low power Intel controller chips that don't need a fan, so it's one less thing to fail.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816116042

[Tirade]

It's not the reliability of RAID5 that's the question, it's the reliability of drives.  With RAID5, when a drive files and you replace it, it has to to a lot of math and I/O processing to fill the replaced drive with the proper data.  In the world of very large SATA drives, this can take a LONG time.  The risk is that sometime during the rebuild, another drive will fail and you'll lose everything.  Thus, RAID 6 has been gaining popularity, allowing you to lose two drives without losing all of your data.

Good info and after some searching/posting it seems that the "URE" rating of the drive should be 15 if using them in a parity based RAID. Luckily the Samsung drives I bought were indeed URE15.

[Tirade]

If you have backups then RAID is almost a complete wast of money and drive space.  What RAID enables you to do is to keep on running in the event of a hard disk failure.  Read that sentence again - it does NOT protect your data in any way.  It protects your UP TIME.

Id like to get away from having backups and just have the ability to grow my available space but be able to retain data in the case of a drive failure. Retaining a backup is costing me too much because Im having to pay for twice the storage.

It's great for corporate data, where a company cannot afford to have their data systems go down for a few days or even a few hours.  The systems must run 24x7, disk failures and all.  Ask yourself, how important that is for your use?  If your music library were offline for a day or two while you installed a replacement drive and restored from your backup, is that such an inconvenience that it's worth spending hundreds of extra dollars to avoid? Thankfully, for myself, I can say I'd live through it without much trama.

Downtime would not bother me, its the loss of data that would be an issue.

If you still feel the need for RAID, then with your storage needs, RAID1 really isn't feasible.  RAID5 (or RAID6 if you want to spend even more money) is the only choice.  RAID50 will give you greater _PERFORMANCE_, but at double the cost in drives and nearly double the cost in controllers.  Probably the last thing you need to be buying is more performance for the lowly needs of a music server.

RAID50 would only increase my drive needs by 1. I purchased 8 X 750GB drives so RAID5 would give me 7 available for storage and RAID6 & RAID50 would give me 6 available. Its only 1 extra parity drive, not double like RAID1.

I organize my library by artist, A-M on one drive, N-Z on another.  It might be a little nicer to have it all on one drive, but not much.   That's probably the one nice thing about RAID5 - you can create gigantic drive spaces that can be be many times larger than that available on a single drive.

Thats what Im planning to do with the new setup. I may keep them separated alphabetically but at least they will be on one drive.

For RAID5 you need at least three drives, so you'll need at least a four drive SATA RAID controller.  Expect to spend $300-400 for a good controller of that capacity.  The 3ware 9650 series is nice, in that they use fast, low power Intel controller chips that don't need a fan, so it's one less thing to fail.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816116042

You were right on the $ with this. The card I purchased was around $350. I linked it in my first post. The ability to grow the array without rebuilding it and have hot-spare/hot-swap capabilties is one of the reasons I choose that card, also the ability to handle 16 drives without using PM's

[JEaton]

Id like to get away from having backups and just have the ability to grow my available space but be able to retain data in the case of a drive failure.

IMO, a very bad move.  It won't protect against a fire, or theft of the server, or a computer virus run amok, or the accidental deletion of all or part of the library.

Retaining a backup is costing me too much because Im having to pay for twice the storage.

Yep.  No getting around it, I'm afraid.  The more data you have, the more it costs to back it up.

Google 'raid is not a backup' for about 7,880 additional viewpoints.

[Tirade]

I ended up buying a card that supports RAID6. The odds werent bad, I just didnt feel comfortable with them... doing 8 drives in a RAID5 set would give a 4.2% chance of catastrophic 2nd disk failure during rebuild using 750GB drives. Increasing my drives to 16 would increase it to a 9% chance of catastrophic failure and 24 drives put me over the 12% mark. RAID6 on the other hand would be sitting at around 0.76% chance of catastrophic failure which are odds I can line with.

I picked up the card as an open box item from Newegg so it saved me a few (hundred) dollars. It supports 24 drives and has the newer IOP341 processor. It should be here tomorrow so if all goes well I can get the archive back online Wed.

[bpape]

If you have the controller, the money, and the physical space, RAID 6 is great.  RAID 5 is just as good 99% of the time except for the things mentioned like theft, fire, etc.  If you have a fire, your backup is likely toast anyway.  If there's a theft, depends on how it's backed up and where as to whether they'd steal that or not.

In reality, for most people, a 4 drive RAID 5 setup is probably the best one can do.  You only lose 1 drive space/cost, controllers for 4 drives are built into many motherboards so not much on the cost there, no external enclosure cost if you have a decent case that holds 4 hard disks, etc.  Once you get beyond 4, then you get into needing the separate controllers, case, etc.

Personally, I've worked with a lot of servers for a lot of years using RAID5.  I can only remember one time when we lost a drive during the rebuild process - and that was when a drive controller crapped out. 

Bryan

[Tirade]

Intel makes it seem worse than it really is

http://www.intel.com/technology/magazine/computing/RAID-6-0505.htm

The truth though is, in a 4 disk RAID5 of 500GB drives you have around a 2% chance of failure during rebuild. 2% is an acceptable gamble for 99% of most home users.

Stepping up to 16 drives takes that risk to 9% roughly. Then again most home users wont have 16 drives. As was mentioned before, go with RAID5 but backup critical data to another source.

[EchiDna]

I did run a RAID 5 array in the past and probably will again in the future, but I found my collection came up to just under 1TB, so in my last round of upgrades, I just went with 4 500gb drives with two kept offsite at work as backups - simple, foolproof (sort of!) and 100% redundant

been contemplating an infrant readyNAS, but haven't bitten that bullet for about a year... once the Squeezebox 4 comes out, I think the readyNAS NV+ (or equivalent) will be the way for me to go...
If the capacity is enough for your needs, RAID 5 is simple and it works, the risk of failure is something each user can only balance for themselves. I have also kept all my CD's and DVD's, so in the worst case scenario, I can re-rip them all anyway...

[brj]

Another option: http://www.lime-technology.com/

[Tirade]

New archive is up and running. Im in the processes of restoring backups as we speak (only 20+ hours to go).

http://s275.photobucket.com/albums/jj305/Tirade75/Media%20Server/