Raid Array Question


Raid Array Question

Author
Message
Matt Wade
Matt Wade
Forum Sage
Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)

Group: Forum Members
Posts: 297, Visits: 876
Hi All,

We are installing a new server at HR Chelmsford and my technical guys has asked the following question:-

We have 2 ways of arranging the 3 drives. My initial idea was to have 1 drive as a mirror of the other (RAID 1, what we have now for the Audio/data folders) with the third as a 'Hot' spare - if one of the 2 drives failed then the spare would kick-in and replace it. Meaning we would regain a fault tolerant RAID automatically within minutes. It is not something we have had before, whether it is worth it is up for debate.

Another option is to arrange the 3 discs as RAID 5. This may give better performance and would give more capacity. It would also mean that if a disc failed we would lose fault tolerance until we replaced the failed drive (rather like we have now) However because there are 3 discs rather than 2 there is a 50% greater likelihood of a disc failure at some point.

Does anyone have an opinion on this or not. Should we do Raid 1 or Raid 5? Or something completely different?

ta

Matt Wade
Need Training or Consultancy for Broadcast Radio Software Products?
Get in touch
w: https://mattwadeonline.co.uk
e: training@mattwadeonline.co.uk
glenmitch
glenmitch
Forum Sage
Forum Sage (33K reputation)Forum Sage (33K reputation)Forum Sage (33K reputation)Forum Sage (33K reputation)Forum Sage (33K reputation)Forum Sage (33K reputation)Forum Sage (33K reputation)Forum Sage (33K reputation)Forum Sage (33K reputation)

Group: Forum Members
Posts: 107, Visits: 709
To be honest, we did think about having RAID, however we decided to go for 2 hard disks in 2 sep servers that are mirrored automatically every few hours using robocopy. If one server dies for what ever reason we can run myriad on a backup profile using the other server.



Glen Mitchell
Systems Manager
Hot Radio
Poole & Bournemouth
glen.mitchell@hot1028.com

Roger Woods
Roger Woods
Forum Sage
Forum Sage (41K reputation)Forum Sage (41K reputation)Forum Sage (41K reputation)Forum Sage (41K reputation)Forum Sage (41K reputation)Forum Sage (41K reputation)Forum Sage (41K reputation)Forum Sage (41K reputation)Forum Sage (41K reputation)

Group: Forum Members
Posts: 136, Visits: 623
We're RAID 1 on both our 'Admin' (500GB) and 'Myriad' (1TB) servers with a daily hard disk caddy back up of both and a weekly of both kept off-site.

Roger.

Roger Woods
Producer/Presenter/Engineer
TerryH
TerryH
Forum Sage
Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)Forum Sage (3.7K reputation)

Group: Forum Members
Posts: 15, Visits: 11
Hi Matt,
We at Whipps Cross have had a dual drive RAID system in place for years, but have never been 100 percent comfortable about it. For example we found that you could not just take a drive out and grab the data off it in the event of a failure.
When we were building our new server (also with RAID) last year and before it went live we tested a number of different scenarios by unplugging a drive to simulate a failure and see what happed.
We found a number of shortcomings such as that while the machine would keep going it would not re-start should the power and the UPS fail.

We are now in the final stages of building a new server so that we can do exactly what Glen does by having a completely separate server machine to fall back to. Also since we started using Myriad we have had a system in place to create weekly offsite backups.

Terry

Adam James
Adam James
Forum Sage
Forum Sage (29K reputation)Forum Sage (29K reputation)Forum Sage (29K reputation)Forum Sage (29K reputation)Forum Sage (29K reputation)Forum Sage (29K reputation)Forum Sage (29K reputation)Forum Sage (29K reputation)Forum Sage (29K reputation)

Group: Forum Members
Posts: 85, Visits: 432
Matt,

At Seaside HR we have a RAID 1 setup on two disks which although not perfect (if both discs fail at the same time, we're stuffed!) it was all we could really stretch to, particuarly with space and budget. We have a NAS which, if I get my ar$e into gear would be set to recieve a backup of the RAID consistently with Robocopy and I back up regularly to an external drive which is kept offsite (just in case!). We had a problem with Robocopy in that it was copying all of it all of the time and not just the changed files, something I've done wrong with the initial setup of the NAS I shouldn't wonder.

We will also eventually get round to installing our Dead Air Detector, which would take away the concern about the server falling over and nothing being broadcast; just unfortunately would put us out of Myriad action until we replaced the failed parts but for us that's an inconvenience we are comfortable with for the time being.

Anyway, I think it's best to have a balance of fault tolerance and convenience, I quite like the idea of a Hot Spare; at least then you have something to fall back on and bides you some time to source a replacement disc (or 2 as invariably when one goes they both go!). Either that or a backup server which you can either keep a consistent backup to or just a dormant spare to get to kick in if the main server fails.

RAID 5 makes me nervous. You only need a two disk failure and you have greater problems to deal with. And as you said, more chance of failure and of course more cost as you'd have more drives to deal with. Although a RAID 6 would give you greater fault tolerance and more time to get the data across, but that's more hard disks than the cost may justify.

Computers eh, who needs them. Wink

Adam
Seaside HR
Peter Jarrett
Peter Jarrett
Forum Sage
Forum Sage (518K reputation)Forum Sage (518K reputation)Forum Sage (518K reputation)Forum Sage (518K reputation)Forum Sage (518K reputation)Forum Sage (518K reputation)Forum Sage (518K reputation)Forum Sage (518K reputation)Forum Sage (518K reputation)

Group: Broadcast Radio
Posts: 1.8K, Visits: 3.7K
We've found its a big problem juggling cost vs desired resilience - after all if money is no object, just build 2 big RAID 10 servers connected to a massive distributed Storage Area Network across fiber linked sites and run Myriad using teamed network cards with dual switches etc BigGrin Fine if you are financed by a nice rich Sheikh, not so much for the rest of us...

In practice we're found something much more modest is reasonable.

Internally here we used to use RAID5 with a spare drive all in hot swap caddys (as the caddys are pretty cheap when used with SATA), but to be honest we're now dropped to straight mirrors (some servers have hot spares) as with 2TB drives it's more than enough storage space for our needs.

It's one of those questions that you really need to ask alongside the question of "Great, but what if the box itself dies (i.e. motherboard etc.)" and then just to confuse matters : "What if the building burns down/gets smoke damaged/floods/server is stolen"

When you start to look at that set of situations it can change you're thoughts - afterall, for a fixed budget you may well get better Disaster Resilience (DR) by spending a little less on that one server, and investing in a cheap backup server or large USB drive, and a pair of USB Drives that you rotate off-site.

I appreciate that our budgets here are a little different from others (more in some cases, a LOT less in the cases of some of our customers!) but if it helps anyone, here's how we do it on our 'live test' servers. (I know I'm not really answering the question about drive setup, but I do think that thinking about the wider recovery situations may change your mind on what you want to achieve!)

1) Primary Server has a pair of mirrors in it, one small pair for the operating System (100Gb of usable space or so), the second pair gives us a larger 2TB usable mirror. All 4 drives are in caddys so can be swapped in the event of a failure. We then use Intel Hardare Raid (on the motherboards) to run the mirrors. This gives us a very fast setup with good resilience.

2) Secondary server has a similar drive setup, but isn't such a beefy processor as it's a slightly older server. It does have caddys, but could easily not have them to save a few pennies as it's only being used (in theory!) in an emergency.  We use Robocopy to copy data from the primary every 30 mins or so.

3) Backup Server - ok this is where we do start to get a BIT belt and braces I'll admit. We have a very large server in the building running Microsoft DPM Server 2010 that backs up ALL the servers in the building and gives us the last 10 days "snapshots" of every file so we can go a recover any file from any time in the last 10 days. It also would be used to "barebones" recover to a new server if we ever suffer a catastrophic failure of both primary and secondary servers. Most places obviously don't have this, so I'm just including it for completeness.

4) USB Drive : (Very important!) The Secondary Server has a USB backup drive plugged into the back of it - a cheap off the shelf Toshiba unit I think - that stores 2 TB of data on. These drives are Bitlocker (from Microsoft) encrypted, and Robocopy runs every night to copy the contents of the server onto the drive so there is always a copy there. This drive is then taken off site to a secure location once and week and swapped with it's twin which runs for the next week. The key bit is that we NEVER have both of these drives on site together, and where possible we don't transport them at the same time or even have them in the same postcode Smile   These are the drives that give us the "Site Disaster Recovery" - if the building was inaccessible for some reason (fire/flood/pestilence etc.) then we can plug these drives into a suitable PC, obtain the decryption key from our key holder and then we are up and running, at worst we've only lost up to the last 6 days of data. Some people actually use 3 drives for this to keep down the likelihood of ending up with both drives in the same postcode at the same time!

5) What about up to the last 7 days of data? For this we use an backup service which we backup to every day (more often for some critical data) over the internet. This is a slow process as obviously it's limited by the speed of your internet connection, but it means that we are never TOO far from the "current data" It only needs to be the frequently changing data as the USB drives covers the rest, but we are actually aiming to backup the lot in time.

So thats how we do it - and yes it's massively overcomplicated in some ways Smile

If I was doing a backup setup for a radio station and had a bit more limited funding I would build a big serer with a mirror and a hot spare. Then buy 2 or 3 USB drives, backup onto them each night and rotate them on and off site as often as practicable - once a week say.

If a drive dies, the hot spare takes over and I replace the failed drive (make sure you setup the email notifications, otherwise how will you know if it fails!).

If the server dies, I plug the USB drive into my studio computer and change Myriad's file locations.

If the building is out of action, I plug the backup USB drive that I took off site last week into my Laptop, call Helen or Sarah here and get a new license code for Myriad and I'm backup and running before the smoke has cleared.

Anyway, hopefully that will help someone!

(Note: as a side note, some of the data is actually replicated using Microsoft DFS but this won't copy open files so is only used for areas which don't have persistent open files)

------------
Peter Jarrett, Technical Director
Broadcast Radio Ltd.

Bill Bailey: No win, no fee, no basis in reality. Just a room above a minicab office in Acton and a steady stream of greedy simpletons whose delusion is only matched by their clumsiness


Matt Wade
Matt Wade
Forum Sage
Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)Forum Sage (96K reputation)

Group: Forum Members
Posts: 297, Visits: 876
Thankyou all for your comments. My technical guy went for RAID5 in the end and we have 2 backup drives. One working via Firewire 800 and one on USB. The Firewire drive is taken off site once backed up and the USB drive is connected all the time and is backed up every night and we use this USB drive with the copy of myriad (Audiowall only) that we have on our OB laptop - this ensure it is always up to date when we are out and about.

cheers again all

M

Matt Wade
Need Training or Consultancy for Broadcast Radio Software Products?
Get in touch
w: https://mattwadeonline.co.uk
e: training@mattwadeonline.co.uk
GO


Reading This Topic


Login
Existing Account
Email Address:


Password:


Select a Forum....





























Broadcast Radio Forums


Search