Announcement

**PewterTA** · 10 February 2009, 14:17 Tuesday

Wow, would you want to put your os which is intensive onto a flash drive which has a large tendency to fail? I mean I know anything can fail...but using a USB flash drive... eh, I've had 2x the amount fail as physical HDs.

Just make sure you back up your system a lot incase the flash drive fails!

And get a good tape backup system to back the whole thing up! ha ha. :T

But that looks like it's going to be a really good server!!!

**Kevin P** · 10 February 2009, 14:24 Tuesday

Nice.

A couple of thoughts:

If you're going to put the OS on a USB flash drive, you should use a high-speed one like this one: http://www.newegg.com/Product/Produc...82E16820220252 You'll notice much faster boot/loading times.

For the prior poster who is concerned about flash drive failure: the OS is relatively static, back it up once (even copy it to another flash drive) and if it dies, just pop another one in and you're back in business. Keep the important data on the RAID array and you're golden. That said, I wouldn't put any Windows on a flash drive, but there's ways to put Linux on a flash and optimize it to minimize writes to the flash drive (which they have a finite number-of-writes lifespan).

2 GB RAM should be enough for Linux without a lot of other things running. If you go 4GB or higher and use a 64-bit distro, you could forego the swap partition for even more speed. You could even load the flash drive's contents into a RAM disk and then the OS will scream! Also, more RAM means more space for disk cache, improving performance.

The quad-core CPU is probably overkill for even a large RAID setup. Maybe look at one of the 45-watt Athlon 64 dual cores, they'll run a lot cooler and draw less power. I'm going to be building a media server myself sometime and will probably go with that CPU, unless I decide to make it a screaming VM box in which case I might spring for the quad core.

How do you plan to back up 20 TB of data?

**impala454** · 10 February 2009, 14:26 Tuesday

Interesting project man. A USB OS drive sounds interesting too. With RAM as cheap as it is going to 4 wouldn't add too much cost. Which PSU did you get? The link is the same as the same as the case.

**Hdale85** · 10 February 2009, 14:35 Tuesday

I fixed the link for the PSU. It's a PC Power and Cooling 500 watt that is 80%+ efficient.

As for the USB flash drive yeah I noticed it wasn't one of the highspeed units after it had shipped. I may try it and if I find its to slow I'll just order another one and use this one for whatever. Maybe backup the OS onto it or something?

Backing up 20TB's of drives haha. Well the guy with the 48TB server built another server to backup to. Me personally I don't want to lose my data but if I do it's all media stuff and not the end of the world. Running a raid 6 array and being able to lose 2 drives at the same time I feel is enough security for my needs. Some may not feel the same way and I understand everyone's needs and wants are different.

I ordered extra ram for my desktop and I've been considering just sticking it in the server instead. I have a stick of 1GB laying around but then it wouldn't be running in dual channel so that's why it hasn't gotten used.

Ah almost forgot. I have not experienced any issues with failing USB flash drives. I actually have a 2GB PNY Attache flash drive that I bought something like 4-5 years ago for 200 bucks when they first came out. I have not experienced any loss or corruption of data on it and it still works great.

**Kevin P** · 10 February 2009, 14:46 Tuesday

You can never have too many flash drives. Use it as a backup for the OS, or just put your pr0n collection on it and keep it on your keychain.

RAID isn't a replacement for backups, of course. You're protected from a hard drive failure (or 2, for RAID 6), but not from accidentally deleting files, or files getting corrupted for whatever reason, a virus overwriting stuff, etc. etc. I'd organize the data in such a way that the stuff that changes often/needs to be backed up can be kept in one place (folder, partition, etc.), and you'd back just that up regularly. You'll probably find that that data is minimal in size, and could just be backed up to an external drive. Static data like DVD/BD rips can probably be left unbacked up since you can just re-rip from your discs. DVRed TV shows are at your discretion... don't bother backing up the watch-once-and-delete shows, but if you're keeping any long-term, back those up.

In fact, you might find that you have 5 categories of data:

1. Important data that changes regularly, such as personal finances, email, etc. That's the stuff you'd back up frequently.
2. Important data that tends to be static, but new files are added regularly - such as photos or downloaded music, or DVRd shows you want to save - just back up new/changed files, how often depending on how much the data changes.
3. Archival data such as old photo collections, that don't change but you don't want to lose - make a couple backups, keep one offsite. This sort of thing could be burned to DVD or BDs.
4. Data that can be easily recreated, such as rips from your physical media - no need to back up at all, unless you want to save time re-ripping. An offsite backup of this data could be handy in case of a disaster (e.g. fire or theft) that destroys your server and your media collection.
5. Transient data that comes and goes, such as watch-and-delete DVR shows. You can get away with not backing this up.

I have some plans for a media server too, so I'll be posting a "competing" thread on this, as I have some different ideas.

**Hdale85** · 10 February 2009, 14:53 Tuesday

Love to see your idea's

And who knows I may adapt some of yours. I have a solid foundation I think that I can build off of though. I figure even a slower flash drive is faster then an IDE drive. Which is what I have laying around that would end up being used. But certainly interested in how your setup will be

**Kevin P** · 10 February 2009, 14:59 Tuesday

USB is slower than IDE though, and especially SATA. On the other hand, flash memory is generally faster than physical hard drives.

Here's another idea to consider when you've added a few more drives to your box. Split them up into more than one array. A RAID 5 with a lot of drives has an increased chance of issues due to drives failing (the more drives you have, the better the chances of one or more failing), especially if you lose 2, you're screwed unless you use RAID 6. Plus, segregating the data can help with backups like I mentioned previously. And, it's easier to create another array than to expand an existing one. By the time you're ready to add more capacity, you might decide that 2 TB or larger drives are cheap enough and you could build an array with those, separate from the array of 1 TB drives you have now.

Another thing you can do is designate 1 or 2 drives as hot spares. If you have more than one array, they can share the hot spares. Then, if a drive fails, the affected array will "grab" the hot spare and rebuild itself onto it. Then, at your convenience, replace the failed drive, and have the replacement be the new hot spare.

**Hdale85** · 10 February 2009, 15:56 Tuesday

The problem I see with more then one array is partially that it wouldn't all be shown as one big space right? If you run multiple arrays then doesn't the amount of space your losing for redundancy go up? Or does it work out to be the same?

**Hdale85** · 10 February 2009, 17:14 Tuesday

Kevin, I have a 3800+ X2 in my desktop, do you think it would be safe to stick that in the server? I've been considering putting the Phenom in the desktop lol. Kind of silly to have an HTPC and server that have a better CPU then my desktop :B

**impala454** · 10 February 2009, 17:54 Tuesday

Just to use my setup as an example, I have a 2TB server and a 1TB Western Digital "MyBook" for backup. I wrote a command line app that will check specific folders every night and copy over any files that have been modified. My documents, source code, music, and pictures (raw pictures only) all get copied over. Movie/TV rips and such do not get backed up. Works great, in a fire or something I could just grab the mybook and go and not lose anything important.

**Hdale85** · 10 February 2009, 18:00 Tuesday

Yeah, even my music is getting rather large though lol. Something like 350gb's at the moment I believe. But even so a 1TB drive would probably be enough to keep up for at least a little while.

**Kevin P** · 10 February 2009, 19:05 Tuesday

Originally posted by Dougie085

The problem I see with more then one array is partially that it wouldn't all be shown as one big space right? If you run multiple arrays then doesn't the amount of space your losing for redundancy go up? Or does it work out to be the same?

Assuming RAID 5 arrays, each array would lose 1 drive's worth of capacity for redundancy, or 2 for RAID 6. But, with a large number of drives, you'll want extra redundancy anyway. Where multiple arrays is better is if you have multiple RAID 5 arrays plus a shared hot spare vs. multiple RAID 6 arrays.

For example, taking 10 1 TB drives:
1 big RAID 5 = 9 TB + 1 for parity. Only 1 drive can fail without taking the array out.
1 big RAID 6 = 8 TB + 2 for parity. Up to 2 drives can fail without taking the array out.
Two equal sized RAID 5s: 4 TB + 4 TB + 1 parity per array. One drive can fail per array without data loss. Two drives failing in one array would cause loss of that array, but the other would remain.
Two equal sized RAID 6es: 3 TB + 3 TB + 2 parity per array. Plenty of redundancy, but only 6 TB out of 10 available.
Now, let's add one more drive, and make it a hot spare, and go with 2 RAID 5 arrays.
4 TB + 4 TB + 1 parity per array + 1 hot spare. Now, if a drive fails in either array,it will be rebuilt onto the spare. Once the rebuild completes, either array can then tolerate one more drive failure. So, two RAID 5 + 1 hot spare gives you (almost) the same redundancy as two RAID 6 but with 1 less drive.

But, going with multiple arrays depends on how you plan to organize your data. One big array might be better if you're ripping a lot of BluRay or something. But you should consider at least one hot spare, when you add some more drives.

I may organize my server into more than one array (or partitions within an array). Put the often-changed files in one array/partition, and archival stuff on another. Then I can backup each partition/array on different schedules.

Originally posted by Dougie085

Kevin, I have a 3800+ X2 in my desktop, do you think it would be safe to stick that in the server? I've been considering putting the Phenom in the desktop lol. Kind of silly to have an HTPC and server that have a better CPU then my desktop :B

That would work, as long as both chips/mobos are the same socket type. (Socket AM2/AM2+ are the current trend for AMD).

Originally posted by impala454

Works great, in a fire or something I could just grab the mybook and go and not lose anything important.

But what if you aren't home when the fire breaks out? (Speaking from experience here...)

Oh yeah, forgot to mention. If you do go with more than one array, you can use LVM (Logical Volume Management) to make the multiple arrays look like one big logical drive. This makes expansion easier. For example, say you start with an array of 1TB drives. You RAID them and then put a LVM partition on it, and then put your file system on that. Down the road, that volume is getting full and you want to add more space, but by then 2 TB drives are cheap and readily available, so you buy a bunch of those. Make those into an array and then expand the LVM and filesystem onto it. Now you have 2 arrays, acting as 1 big volume, and that also eliminates the issue of the different drive sizes in 1 array. A couple years later, you buy up a bunch of 4 TB drives, you can make another array with those, and so forth and so on. Well, by then, your old 1 TB drives are starting to die, so you copy everything over to the new array, and retire the old drives. Repeat ad infinitum...

**impala454** · 10 February 2009, 20:21 Tuesday

Originally posted by Kevin P

But what if you aren't home when the fire breaks out? (Speaking from experience here...)

The stuff I care about most (photos, source code, documents) still takes up less than a DVD worth of data, so every couple months or so I burn them and stick the disc in my fireproof safe. I also occasionally copy my source code folder & a few documents up to my web server.

**PewterTA** · 10 February 2009, 21:12 Tuesday

I guess I'm anti USB flash drives for important data since I send back so many every day. We've got a fail ratio of 1:8 and almost every one we've sent out has, at some point, come back to be replaced. Some last longer than others, but most fail within 1 to 2 years of heavy use...which all of our people use. Biggest thing that kills them is dust in the USB port...which you wouldn't have here keeping it plugged in all the time.

I do however, love all the solid state drives we use. They definitely show a nice performance increase over mechanical drives. But I'm still leery on them eventually failing. That's why I only use it as my OS drive and regular drives in my raid 6 config for my 4TB total on my server, but that's almost full so I'll have to be expanding.

That's why I use Veritas with a tape backup to backup the server and all PCs on my network.

**Hdale85** · 11 February 2009, 01:08 Wednesday

Yes both CPU's are AM2. All 3 computers in my house are AM2/AM2+ except the notebook haha. Anyways I think that's what I'll do. The 3800+ should be good for the server.

**PewterTA** · 11 February 2009, 09:46 Wednesday

It should work perfectly fine...you don't need a huge CPU just to send data to the network.

LOL my 2008 server is only a P4 3.2GHz cpu. Works perfectly fine for what it is.

**impala454** · 11 February 2009, 10:18 Wednesday

CPU isn't important if you just have a "dumb" file server, but make sure you keep in mind anything you might want to do to the files on the server. I.e. say you decide to re-encode a large amount of files or something. Just something to keep in mind.

**Hdale85** · 11 February 2009, 21:26 Wednesday

Well got it all put together and what not. Instead of putting 6gb of ram in the desktop I put 4gb in the server and 4 in the desktop. I'm installing Ubuntu Server 64bit as we speak. See how this goes

**Hdale85** · 12 February 2009, 13:31 Thursday

Anyone got AIM or something and could help me out with samba? I've got Webmin installed which is a nice browser interface admin tool. I'm trying to setup Samba. I don't want it to require a password but it's not letting me browse the files at all. I can see the Media-Server on the network from my other computers but when I try and go into it it just says I don't have permission or what not.

**Hdale85** · 12 February 2009, 13:34 Thursday

Ah nevermind! I got it now

Ok now I got it to where I can browse the files but I can't write to it. I set Samba to allow all users to write. The array has write protections on the server it self so maybe that's why?

**Kevin P** · 12 February 2009, 14:03 Thursday

If the server can't write to the array (due to mounting read-only or file permissions) Samba won't be able to either.

If you want to talk to me on AIM, send me a PM with your screen name so I can add you to my buddy list. EDIT: never mind, I see you have it in your profile. I'll add you to my list now. If you want to IM me, my screen name is in my profile.

**Hdale85** · 12 February 2009, 14:06 Thursday

Looks like I got it working. I just changed the owner to nobody which is what I have Samba set to forced user. It seems to be working pretty quickly. Takes like 2 minutes to copy a 4.4gb file? I guess that's pretty quick lol.

I setup VNC on it as well so I can just control it from my desktop.

**Kevin P** · 12 February 2009, 14:09 Thursday

That's pretty good speed, especially for Samba (or SMB, the underlying protocol, which, true to Mickeysoft form, is very inefficient).

If you have other Linux boxes and you want to share between them, NFS is a lot faster. But for Winders you'll need to use Samba.

**Hdale85** · 12 February 2009, 14:18 Thursday

Yeah my other boxes are all windows for now. I just have the HTPC, Desktop, and a notebook. Since I'm using a switch I need to figure out how to get my wireless router clients to show up on the same network. My friend is coming over today to help me tinker with it so we'll see how that goes.

**Hdale85** · 12 February 2009, 14:21 Thursday

One thing I'm going to say is I was actually surprised how easy it was to setup Samba and MDADM. It's all rather straight forward I'm not even sure I could have done it more quickly in a windows server setup. Even to add extra drives and what not it will be super easy in linux.

**Kevin P** · 12 February 2009, 14:37 Thursday

How is your network set up? Assuming the wireless router provides your Internet connection, just connect a port on the switch to one of the LAN ports on the router. If either device has an "uplink" port, use that.

Or, if you have enough LAN ports on the router, those act as a switch, and then you wouldn't need the other switch at all, unless it's in a different location or you need more ports, or want gigabit.

**Hdale85** · 12 February 2009, 14:50 Thursday

I have a Dell 16 port gigabit switch. I have 6 network connections. The wireless router has only 4. I have the wireless router connected to the switch via the routers WAN port I think they call it. So the computers on the wireless network (which currently is just the notebook) do not show up on the same network as my desktop and HTPC. I've been wondering if I just connect it to a lan port on the back of the wireless router if it would work then.

**Kevin P** · 12 February 2009, 18:50 Thursday

Yes, connect the switch to one of the LAN ports on the router. Just make sure to disable the DHCP server on the router to avoid conflicts.

Alternatively, you can connect your DSL modem to the router's WAN port, then connect a LAN port on the router to the switch. Then connect your computers to either the router or the switch (connect gigabit-capable machines to the switch). But, if your DSL modem has its own router/firewall, you could use the first method above, if you disable DHCP on the wireless router.

**Hdale85** · 12 February 2009, 20:31 Thursday

The DSL modem doesn't have router capabilities it's just a straight through DSL modem. So it should work fine connected to the wireless router and then to the switch through one of the lan ports.

On another note Kevin, disabling the swap seemed to only fix it for a little bit. It started acting up again but not as often so I'm going to pull the flash drive and install a regular hard drive or use the USB laptop drive I have.

**Hdale85** · 18 February 2009, 22:18 Wednesday

Ok well I finally got the server up and going. I had quite a few instances where I ended up just formating and starting from scratch. Some was stupid mistakes on my part others was bad advice from someone on a different forum. Either way it seems to be running much better then previously now! It runs great really extremely fast and very easily serves up even the most demanding Blu-ray movies over the network. I snapped a couple pics. I didn't get as many of the server not sure why? I guess it's nothing to terribly exciting.

I will say that the cables are all bundled up and nice now. I also taped up the holes under the hard drive fans so that it pulls more air through the drive bays. This is something a lot of people have been doing.

You can also see my modest little network setup. The Dlink box is the wireless router. It does 108 super g or whatever they call it. Works very well and pretty quick. Certainly not quick enough for 1080p streaming as I've been there tried that. Then the DSL modem is on top of it. Next to that is my Dell 2716 Managed gigabit switch that I snagged on ebay for about 110 bucks shipped. Excellent switch extremely fast and very well built. The antenna for my wireless is out in the hall way. Basically goes up through the ceiling panels and it has magnets on it so it just sits out in the hallway. For some reason my wifes laptop is very sensative to the signal? It's not very far away so I need to look into that soon.

**bbecker1983** · 26 February 2009, 22:08 Thursday

Sweet work man. Did you redo the network yet or??

I hope to have space to put use to my Cisco Fiber switch... ;x(

MMM 100Base-TX dual Mode fiber, should have zero issue streaming HD content over a network!

**bbecker1983** · 27 February 2009, 17:07 Friday

Dude if you want to tame th level of noise in that bad boy here is what ya need!

:T

Lose the 60mm fans and replace them with this, it will pull a lot hotair out and it will be very quiet

http://www.frozencpu.com/products/8254/cpa-260/Lian-Li_BS-08_Dual_120mm_Fan_Internal_Dual_Slot_PCI_Cooler_-_Black_BS-08B.html?tl=g36c15s62

Even with 20HDD's 3 of these can move enough air to cool them

http://www.frozencpu.com/products/4258/fan-237/Enermax_80mm_Marathon_Case_Fan_UC-8EB.html?tl=g36c15s58

Announcement

Media server capable of 20+ TB's

Media server capable of 20+ TB's

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment