How to build an NSX-T Home Lab
Updated: Feb 24
If you're learning NSX-T, it helps a lot to deploy it and configure it a few times. There's lots of new concepts, and reading about it or watching videos really isn't enough to solidify those concepts. In this blog, I'll highlight some considerations you should be thinking of before you build your lab. If you want a little extra help, I've created a FREE lab guide which provides all of the detail below and a lot more. It also includes Cisco switch configurations that you can use as a baseline, along with some diagrams. You can get your lab guide here.
Things to think about when building your NSX lab
There's a few things you need to think of when it comes to physical switch. I covered all of those things in this blog post, but to summarize it here, your physical switch should:
Provide physical connectivity to your servers (most likely via 1G copper)
Support Jumbo MTU of 1700+ (only required if testing network virtualization)
Support L3 interfaces (SVI's or VLAN interfaces on Cisco devices, RVI on Juniper)
Optional, but nice: Support DHCP server functionality
If you decide to go with a Cisco switch, be sure to check out my free lab guide which has sample Cisco configs in it that you can use as a starting point.
Distributed Firewall only, or Routing?
Are you interested in testing per-VM firewalling (Distributed Firewall) with NSX-T only? Or are you wanting to do network virtualization - meaning you define networks in NSX-T, along with routing and switching. If you are only wanting to test the distributed firewall, but don't care about network virtualization, you can get by without a physical or virtual router to peer NSX with. In addition to that, your footprint will be smaller as you won't need edge node VMs. If, however, you plan on creating networks in NSX-T, you'll need to think about what your NSX-T T0 Router is going to exchange routes with. The currently supported protocols are OSPF and BGP.
Check out my video below which details how NSX-T routing works:
Nested ESXi Sizing
When building your lab, you'll likely build (or buy) a system with lots of RAM, and install ESXi/vSphere on top of it. That's fine, but I highly recommend testing NSX-T with Nested ESXi instances. These are simply VMs on top of your physical server, that are running ESXi. If you're not familiar with Nested ESXi, check out my video below which covers it:
While you can absolutely have shared storage via a NAS (something like a Synology or QNAP, or OpenFiler), local storage is just fine if that's all you have. If you do this, I recommend getting a minimum of a 1TB drive. SSD is ideal, but spinning disk is alright if money is an issue. When you create your nested ESXi hosts, I prefer to give them two disks:
1x 40Gb disk for ESXi (thin provisioned)
1x 100Gb disk for datastore (thin provisioned also)
It's worth mentioning that you don't need to add the 100Gb disk to your nested ESXi if you have shared storage. Also, 40Gb disk is overkill for ESXi, but it's the default and if you thin provision, it won't use all 40Gb anyway.
My recommendation for your first NSX-T design is something like the following:
2x Nested ESXi hosts for workloads (VMs) - 6-8Gb RAM each
2x Nested ESXi hosts for edge nodes/cluster - 8-12Gb RAM each
1x NSX-T Manager (Medium form factor if you can)
Prep the workload ESXi hosts above for NSX-T only - NOT the edge cluster ESXi hosts
Make sure the edge cluster ESXi hosts have Distributed Switches with an MTU of 1800+.
Setup two VLANs on your physical network. One for Management, and one for TEP. The TEP VLAN should have Jumbo MTU enabled, and DHCP if possible.
One hosts are prepped, deploy an edge cluster with two edge VMs.
Configure a T0, T1, and some overlay segments.
Setup routing from the T0 to either a physical router in your lab, or a Cisco CSR1000V.
Get your FREE NSX-T Home Lab Guide!
I hope you found this article helpful! If you find yourself needing additional help on how to setup your VMware NSX-T Lab, you can register here for my mailing list and receive your FREE lab guide via email in just a few minutes.