In case you didn't read my blog about my learning project, I have decided I want to get some hands-on experience with the Cisco UCS (Unified Computing System) line of hardware, and you can get the older hardware (circa 2009) pretty cheap. Last year, I spec'd out a new data center build for a client and relied on spec sheets and a colleague that runs them in a data center for his day job to make sure I had the design right, but there really is no replacement for actual hands on experience, so hence the project.
There are two varieties of the Cisco UCS platform, the C-series, which is a single rack-mounted server (and is what we spec'd for the client), and the B-Series, which is a blade-style system that has a chassis that can accommodate 4 full-width server blades, or 8 half-width server blades. The B-Series are slightly more complicated in their design, given the shared fabric extenders in the rear of the chassis and the virtual mezzanine ethernet interfaces, so why not go for the more complex solution so we can learn a little more!
So I went shopping on Ebay . The first thing to know, if you are ordering one for your home lab like me, is the shipping aspect of getting one. The empty chassis alone, without servers, power supplies, fans, etc. is about 100 lbs. and very bulky. Fully loaded, they're almost 300 lbs. so this thing is not getting delivered by FedEx ground. I found two interesting deals, one was for a 5108 chassis with 48 Cores and 592 GB of RAM going for $600 with shipping included. The second deal was for a 5108 chassis with 32 cores and 288 GB of RAM going for $300 with a $300 shipping charge.
Both of them are a good fit for what I want to experiment with. Ideally, I can install KVM on one blade, VMware on a second, and MS HyperV on a third, so the build-out is juuuuust right... the first deal is the better one, though, given it's almost twice the RAM and 50% more CPU for the same price.
So I placed an order for the first deal, but upon finding that it was shipping to a non-commercial address (or really, any address without a loading dock), the seller quoted me an additional $300 shipping charge to ship with a different shipper that had a lift on the back of their truck so that it could be dropped off at my home. The seller was nice enough to void out the order, so I went back to the other seller and made an offer of $250 (which was accepted), and with the $300 shipping charge, and I am in business for a total of $550 out the door.
Final spec's on my purchase:
2 x UCS B250 M2 2x 2.67GHZ X5650 HEX CORES,96GB RAM 24X 4GB,2 X 146 GB 15K HARD DRIVES
1 x UCS B250 M1 2x 2.67GHZ X5550 QUAD CORES,96GB RAM 24X 4GB,2 X 73 GB 15K HARD DRIVES
CENTER HAS 8 FANS, 4 POWER SUPPLIES
2X UCS 2208XP MODULE
To build the equivalent server capacity out of new parts would cost me thousands of dollars just for the RAM alone, and even if I bought the individual parts used on Ebay I'd end up spending 2-3x what I paid for this whole box, so I think I did pretty good on this purchase...time will tell, but I can always part the thing out and make a small profit.
First things first, where am I going to put this beast? I need a place that has room, available power, and where the noise (while it's running, which probably won't be 24x7) isn't going to pose a problem to the household or my office work. In my case, I think that means the garage. I can't really tell from the breaker box if there's enough power there (there's no dedicated circuit for the garage), but knowing my luck it's on the same circuit as the microwave. This would be both good and bad if it was the case...it would be good in that it's a 20 amp circuit, but bad in that there's a real risk that if my servers are spinning the garage and somebody microwave's a piece of bread, my servers could be toast and the bread will be cold!
If only I could estimate the power requirements! Actually, I can! I used Cisco's handy power calculator with my previous data center project, so that's where I headed off to.
5108 Chassis and 1 B250 M2 blade
Max: 687W
Idle: 302W
50% load: 500W
Max Input Draw: 5A
5108 Chassis and all 3 blades:
Max: 1373 W
Idle: 551W
50% load: 900W
Max Input Draw: 12A
Okay, that's not looking too bad. It could be a little tight on a 15A circuit at full load, but I'm not planning on mining crypto currency or anything so I doubt the load will ever approach 12A except maybe on initial power-on. Next up is cabling.
These things never ship with any cables, and we're definitely going to need some. First, let's take a look at the power supplies.
The spec's in the listing don't say much about the power supplies, other than them being AC, so I'm going to have pull the supplies once they show up to verify the model. The hope is that they will be the "gold" power supplies, which accept 110-240V input vs. the "platinum" which is only 200-240V. Regardless of what the supplies end up being, I'm still going to need 110V power cables because the connectors are not standard IEC (thanks a lot, Cisco). I first ordered 4 AC 110V power cables off Amazon to feed the the supplies.
Second, I was going to need a way to connect to the console. I'll need what's called a KVM console cable that gives you a VGA, 2xUSB and 9-pin D-style serial port connector (what?? I haven't used one of those in two decades). I ordered mine off Ebay for about $10, but here's a link to one on Amazon.
And since nobody has serial ports in their computers anymore (for a dinosaur age, it seems) you'll need to order a USB to Serial adapter for your computer. Most of the ones on Amazon that come up on a search are USB to MALE connectors, which isn't going to work for you, so make sure you order something like this USB to Female RS232 connector.
Third, the 2204XP modules have no GBIC's installed in them, so get yourself some of these. I only ordered one for now, as that should be fine for bringing up my first server (I hope), and my switch only has 8 ports anyway. I'll add more later if I need to. Theoretically, I shouldn't need to as I can run 802.1q on the interface for as many interfaces as I need, but I don't know what I don't know right now, maybe I'll find out differently.
The un-answered questions I have at this point are:
1) Will I get power supplies that I can run on 110V, and
2) Can I use the 2204XP fabric extender to connect directly to my LAN switch, or do I absolutely have to have a 6100 or 6200 fabric interconnect as mention in this spec sheet? It looks like a 6100 can be had for as little as $24 on E-bay, so it's not a cost thing, but I really don't need one if it's only purpose is to help manage multiple UCS servers / blade centers.
The internets are absolutely no help here, at least as far as determining if I can run without it, although the Cisco diagram below is helpful in illustrating how they are meant to be used.
A large truck just showed up at my house. Looks like the next installment of this story will be coming soon.