Recently, I have had a number of customers who wanted recommendations on how to setup their networking when using vSphere on the Cisco UCS Platform.
One customer called Cisco and got this recommendation from the vendor:
- Setup multiple vNICs on UCS - one for each FI (A and B) for vMotion, one for Management and then one or more for other VLANs for maximum flexibility. If we then match the vNICs that are attached to FI-A together for vMotion into the same Distributed Switch, we can avoid Northbound (external switch) traffic. We can also perform QOS/Traffic shaping at the Cisco UCS level.
It makes sense that the hardware vendor would suggest a hardware solution that highlights and uses the capabilities of their platform.
However, I don't think this is the optimum solution.
I would just leave the vNICs at 2 - one per FI. And don't have them set for failover - because if a path fails, I want to see that failure at the vSphere level and not have it hidden from me. I will have the vNICs setup to handle failures already in vSphere, so I don't need it twice.
Then, just have more portgroups - one for Management, two for vMotion (one per FI/vNIC - so all vNICs that are tied to FI-A are together in one portgroup and all vNICs that are tied to FI-B are together in another portgroup), and then additional portgroups as needed for VMs.
Adding additional vNICs does not change the amount of bandwidth available or the number of physical connections coming out of a blade - it is limited to 1 connection to each FI....so the rest is semantics - adding portgroups is effectively the same as adding vNICs.
Plus, using the VMware method allows you to use a single pane of glass to control and manage all those features.
In reality, it really makes little difference whether you use vNICs or Portgroups - all that changes is the point at which you are separating the traffic. In the vNIC case, you are separating it before vSphere sees it....in the Portgroup case you are allow vSphere to handle the separation.
The decision comes down to whether your business prefers to control this from the hardware or software side. Plus, are the features you require available in UCS or vSphere? If you really want to use NIOC or other Distributed vSwitch features, then choose the vSphere method.
I would definitely not do both - don't enable QOS or Traffic Shaping at the vSphere level and then also do it at the UCS level. You are adding complexity without adding benefit. Plus diagnosing issues becomes much more difficult.
