Microsoft Virtual Server inside of VMware ESXi virtual machine

Virtual machines inside of virtual machines like to sleep around.  But maybe I should give you some context!

I just moved an SBS 2008 installation from a physical server to a virtual machine. It’s a temporary thing until we rebuild the whole server in a couple of weeks, but it’s one of the steps. The physical SBS box had Microsoft Virtual Server installed on it running Blackberry software for Exchange (just for one user, but he’s the owner so it’s important that it keep working!). Once the virtual machine was up and running (hosted on VMware ESXi 5.0 free), everything seemed to be going fine until I took a look at the Blackberry server. I had to change the networking in Virtual Server to use the new virtual network card (instead of the old physical one). Then I booted the virtual-inside-virtual machine….and lost networking to the SBS VM!

I’ll save you the extended details of troubleshooting, which involved some reboots and reconfigurations and resets. The solution was go into the VMware vSphere Client, click on the host at the left, then on the Configuration tab, and then on Networking on the Hardware submenu. I clicked on Properties of vSwitch0, and then edited the vSwitch configuration. On the Security tab, I changed Promiscuous Mode from Reject to Accept, and OK’d my way out of all the settings screens. This is a great security feature but prevents the use of sub-virtualized machines that require promiscuous mode (now you see why I mentioned sleeping around?).

One thing I also did was add a second virtual network card to the SBS machine and unbind it from IPv4 and IPv6, but assign it as the physical card used by Microsoft Virtual Server for the Blackberry virtual machine. This gives me a little logical separation between the NIC used by the SBS system and the one used by Blackberry, though it doesn’t gain me much given the overall setup. It was more one of my troubleshooting changes I decided was worth keeping. You do still have to enable Promiscuous Mode for the Virtual Server to get network connectivity (a hint was that with the defaults, broadcast traffic worked because the NIC would get an IP via DHCP, but unicast traffic would fail; broadcast traffic was functioning even with Promiscuous Mode disabled!).

I’ve dealt with Promiscuous Mode before to get firewalls working in VMware, such as pfSense, which is why I though to check on the setting eventually. I’m happy that my past experience was able to help with my current one, and now I’ve shared with you and, hopefully, future Googlers looking for a solution to a similar problem!