Developing a network upgrade plan
It's been a busy couple of weeks working on the network, and I feel like I owe you all an update on issues, resolutions, and current plans.
As of the last update on the lab, the production lab network looked something like this:

My primary objective over the next month is to have a fully functioning Network Access Control (NAC) deployment, but achieving that will require a lot of work and a lot of steps. It doesn't make sense to try and get there all in one step, but rather map out a roadmap to get there.
Some of the things I know I want/need to get as precursors are:
1) More IPv6 subnets, which will require either a tunnel to tunnelbroker.net, or a router that can request more a prefix larger than a /64 from Comcast, or maybe even both
2) A WiFi network that can support 802.1x authentication (Google WiFi cannot)
3) getting the Cisco UCS blade chassis online
I learned that you cannot establish a 6to4 tunnel to tunnelbroker.net from behind a Google WiFi router. The first problem is that Google has disabled ICMP on the external interface, so it won't respond to pings. This is a hard stop in the automated process at Hurricane Electric, but even if you could enable ICMP, you probably wouldn't get past the part where your router needs to be able to forward protocol 41, which is neither tcp, nor udp, and you would need to port-map protocol 41 to a device inside the LAN. I'm just not certain this is possible, but it's a moot point since we can't get past step one.
So the current list of dependencies for my first few pieces of my roadmap are:
a) Get a switch installed that can support PoE (see switch saga)
b) Get some Aruba WiFi access points off Ebay
c) acquire some cat 6 cable and related bits and bobs and run cable through the attic between the office and the garage, outdoor AP location, and basement
d) Get the UCS chassis up and running and record a walk-through video for Youtube
e) Get a eval license for Aruba Virtual Mobility Master and Virtual Mobility Controller and install them with walk-through videos
f) get a network appliance to replace the Google WiFi router, and select a routing platform for it
g) move the Google WiFi router below the network appliance
I ordered a Lanner Electronics FW-7535F off Ebay for $149, which came with pfSense pre-installed. It arrived with a corrupted 320GB hard drive, but luckily I had a 1TB drive laying around that would work just fine, so I downloaded a fresh version of pfSense and installed it on the new drive.
Meanwhile, the Aruba 2530-8 showed up and enabled me to get an AP-105 up and running on a trial install of the Virtual Mobility controller. Now I had two radios sitting right next to each other competing with each other for spectrum, but more on this later.
I unsuccessfully tried twice to get the Google WiFi router to sit under the pfSense router. I don't know what was causing the problem...maybe a lack of proper port-mapping the proper ports on the pfSense? The internets suggested that I would have to unplug two of my three mesh routers and put the Google WiFi into bridge mode, and now I'm getting really annoyed...until I realize after playing with the aruba controller system that I can simultaneously run multiple SSID's with different configurations on the Aruba system. If I get a fully functional Aruba network installed, I can run my current home network SSID with the existing WPA password and get all of my consumer devices running just as they were, and set up a separate SSID that uses 802.1x.
When it comes to project management, you're going to have critical path dependencies that have uncontrollable timelines. In this case, with COVID-19 messing up deliveries of gear from a lot of vendors, I don't have everything I need to jump right to the end-state. A friend of mine, Beth Hannan, used to always say "work the workable work." It's a good mantra for project management, because even if there's stuff you need to finish a project, there always always something you can do to move the project in a forward direction.
In this case, I can run cable, I can get familiar with pfSense and get as much of it ready to go as possible, I can get a good WiFi network installed in the house with 3 AP's and get familiar with the Aruba system, and I can get a new switch installed...all of this while waiting for some parts I need to get the UCS chassis online.
Over the past week, I have made a ton of changes to the network, and here's what it looks like now.

To review, we have installed and configured:
1) Aruba S2500-48P (48 port GigE, 4x10G) switch
2) pfSense router, pre-configured with one WAN interface and a 4xGE lag to the S2500
3) an Aruba AP-225 in the office, and an AP-175P outdoor AP with antennae's
4) Aruba Virtual Mobility Master and Virtual Mobility Controller
5) removed the Aruba 2530-8 and Netgear 5xGE switch from the network
We are now as ready as we can be for the final move to this design:

There is actually nothing stopping me from implementing the replacement of Google WiFi with the Aruba system and pfSense router except for one thing. Remember those "enterprise users" on the network, my family that depends on functioning Internet all day? When I re-install the mobility masters and controllers, I want to do it in a high-availability configuration, and for that, I need the UCS chassis up and running...and I will have to take down the Aruba network while I perform (and record) the process, which will probably be half a day of downtime. They always say that being a network administrator would be a lot easier if there were no users on the network! But hey, the users always come first!
There is not much more to do at this point but wait for my SFP+'s to arrive so I can plug the UCS box into the switch and connect the fabric interconnect to the 2204XP's fabric extenders in the chassis. I already have 2xCat6a cables run to the garage waiting for them to come in, so the fun should start soon.