Home Lab Upgrade–vSphere 5, Tyan Motherboard and iSCSI Multipathing

 

I have been running my home lab on two old HP ML350’s and they have been work horses – very reliable and stable, but not particularly fast and not 64 bit capable.

I have been wanting to try out a number of the new features of vSphere 5 and also run a number of Windows 2008 R2 64 bit OS instances, so I knew an upgrade was in order.

My current home lab consists of 2 HP ML350’s with 6 GB of RAM each, a Linksys 48 port Gigabit switch and my Norco multi-terabyte storage server that is running the free Starwind Software as an iSCSI target.

The setup has worked remarkably well and has allowed me to explore many different setups, but the time has come for an upgrade.

My initial thought was to find and buy some used gear that is on the VMware HCL, like maybe a couple of HP DL360 G5’s.  They would run me about $700 each.  After reviewing Jase's home lab build though, I decided to go ahead with something similar.

In the article, Jase goes with some more powerful CPU’s, but I decided that a) I always find that I run out of RAM before CPU in a VMware environment and b) I didn’t need the extra features that the other CPU’s provide – like VT-d (VMware DirectPass Through) etc. More important to me was cost.

So….here is what I purchased:

Tyan S5510GM3NR motherboard
Intel G620 2.6Ghz CPU with 3MB cache
Kingston KVR1333D3E9S/4G RAM (2 of these for 8 GB of RAM, could put in 4 for 16 GB of RAM)
Rosewill Case – FBM-01 RTL
Rosewill PSU – RG530-S12 530W RT

I ordered everything from Newegg.ca and my total cost (in CAD) was around $970 for 2 of these systems.

One motherboard from Newegg was DOA (dead on arrival), so I requested a Advanced Replacement from them (doesn’t cost extra and you get your stuff much sooner, plus you get a return shipment label).

So, I built one system.

What a GREAT MOTHERBOARD! 3 completely compatible NIC’s, 4 DDR3 slots so I can upgrade RAM easily, KVM over IP or also called IPMI and I could easily have put in a CPU that had 4 or more cores without too much more expense.

I just couldn’t get the motherboard to boot via the IPMI web interface, so I connected up a USB DVD drive directly to the system.

Also….the motherboard has a internal USB stick connection just ready and waiting to take a USB stick to boot and run from – so that is what I did – I bought a couple 4 GB Patriot USB drives and installed ESXi 5 directly onto that. Afterwards, you need to change your scratch drive to another location in order for your logs to be saved between reboots.

The other nice thing about this motherboard is that it has 3 NIC ports – one doubles as the IPMI port, but it can still be used by ESXi too.

The only bad thing I found about the gear I bought was that the motherboard expects a Front Fan to be hooked up to it and the fans that come with the case are only two wire fans that just take a power connection from the PSU.  Without the fan hooked to the motherboard, the motherboard squeals constantly to say that there is an alarm – a fan has stopped spinning – at least that it what it thinks. So, I ended up purchasing a couple $9 fans from a local store that had the 4 wire connections and replacing the front 120MM fans in the cases. Then all was good.

So, then I setup Round-Robin iSCSI multipathing – this is different from how it was in vSphere 4. VMware has added a GUI to allow this to be done directly from the VI Client.

Here is how that was done:

First I created the combined network like this – all three adapters as part of one vSwitch.

However, the iSCSI multipathing requires that each VMKernel port has only 1 uplink connection – so you have to make multiple VMKernel ports, assign them IP's and then override the default switch failover order for that VMkernel port.

Then you add the iSCSI storage adapter, and add the two VMkernel interfaces in the Network Configuration tab of the iSCSI Initiator properties.

Once that is done, you setup your iSCSI targets in the Dynamic Discovery tab as normal.

When finished, and you right-click a iSCSI target LUN/Disk, you will see multiple paths to the iSCSI target. Simply change the Path Selection to Round Robin and you are done!

Pretty darn happy with this new setup!