You can't have much of a lab without some switch ports.
My little Netgear 5-port GigE switch was fine when I had a consumer network and only a few devices to plug in in under my consumer WiFi router, but not only was I going to need more ports, I was going to need features like:
1) 802.1q trunking, so that I could have physical servers on multiple subnets/VLANs
2) Power over Ethernet (PoE) to power my enterprise-grade WiFi Access Points (WAPs)'
3) Remote management for visibility. sflow support would be a nice to have
4) while 10GE ports would be a nice to have, plenty of GigE ports were mandatory, and
5) LACP (Link Aggregation Control Protocol) would also be a nice to have for future use, although it's not really in my current design.
My initial search on Ebay for a 1G switch turned up an Aruba 2400 for $20, and it advertised having an integrated mobility controller. Perfect! This would save me from having to install virtual mobility controllers initially and get my enterprise WiFi network up and running more quickly. I ordered it, along with an AP-125 ($17), and an AP-105 ($13) to get me off and running. When the switch turns up and I plug it in, it sounds like a jet engine. My wife was complaining about the noise, which could be heard out in the living room, almost right away. Additionally, it turns out that there are only two GigE ports (both the old, wide GBICs) and the rest were fast ethernet. On top of that, what I actually got was a 2400-E, not the 2400, which is an extender for a 2400 platform. What this means is that it doesn't have an integrated mobility controller, it's just a basic switch with an integrated "muxport" functionality to tunnel the ports directly up to a 2400. This thing is probably going in the trash.
This also led me to get some eval licenses for a virtual mobility master and virtual mobility controller setup, which I'll cover the setup of in a future post, but the problem this created is that the latest ArubaOS (8.6) doesn't support the AP-125 that I bought, and even the AP-105 will be discontinued in ArubaOS 8.7. So off I go to Ebay, and found an AP-225 and an AP-170 (outdoor) for pretty cheap.
So I went out looking for a model of switch that was a bit more recent. I found an Aruba 2530-8 PoE switch for $60. It's a fan-less desktop unit, so it would have no perceivable noise, and according to the specs on Aruba's web site, it was an 8+2 GigE switch, where the 8 ports were GigE with POE, and the 2 additional ports were designed as trunk ports without PoE. This should have barely met my needs, but in combination with my old 5-port Netgear everything would just fit with a couple of ports to spare.
But when it shows up, I discover that it only has two GigE ports and 8xFE! WTF! After some careful examination of part numbers, I realized that the spec sheet I had been looking at on Aruba's web site was for a 2530-8G-POE and they don't even list the 2530-POE in their portfolio anymore. I currently have this switch up and operational, powering the AP-225, so it does the job, but since I'm limited to 100meg on the PoE ports, this is not a good permanent solution.
I've also come to find that the Cisco UCS platform NEEDS a 10G SFP+ running at 10G in order to function. So, we're back to Ebay looking for a solution.
Once again, best switch deals seem to be on Aruba gear. Although I found a Juniper EX 4200 for $100, it only has two 10G ports. I found an Aruba S2500-48P for $100, which has 48 ports of wired GigE's and 4x10G SFP+ interfaces. Also, from a noise perspective, this thing seems to be roughly as loud as my current server, so I shouldn't have the 2400 jet engine problem this time around. This is perfect, and would actually allow me to put LAG ports in groups of two to each of my servers (the Intel system I built, and the UCS B-series chassis that I am still firing up). So yeah, that's on the way now.
And then there's the other thing I keep forgetting. SFP's!
I had originally purchased one for the UCS chassis, not realizing that a fabric interconnect was required to use the chassis. That was going to require more SFP's. I found some sold as a pair on Amazon for $20, and plugged them in. My fabric Interconnect can't seem to connect to the SFP's, though, perhaps because they were for Ubiquiti and not Cisco? Also, I thought I was getting something that could do 10G, but an SFP is only 1G and an SFP+ is 10G. Pretty big distinction, I would say. The Cisco documentation says that up to 8 of the ports can be configured for SFP GigE, but no luck here, maybe because of the compatibility stated on the part. I really wanted these ports between the fabric interconnect and the fabric extenders to be 10G anyway, so ... before I do anything else, I'm going to the whiteboard, because I am stick and tired of not having the parts I need!
So I've currently got 1x10gig SFP+ and to complete the diagram above I'm going to need a total of 12 SFP+'s. The only problem is, the best deal I could find an Amazon for a generic 10GbaseT SFP+ is $39, and it's not much cheaper to go to Ebay to get genuine Cisco SFP+'s, so I think I'm going to order 5 for now just for cost reasons. That will let me get my office server up to 10G, and have a single , complete 10G path into the UCS chassis.
I gotta say, I feel like a bit of a dummy for all of these mistakes I've made, but it hasn't really cost me much, other than shipping, as I can return/resell all these unused parts.
This is also what's making me remember that there's no replacement for hands-on experience. If you only read the spec-sheet alone, you will miss some details every now and then, and I always like to be as thorough as possible in my designs.