10Gbit Ethernet: Killing Another Bottleneck?
- Posted in
- IT Computing
Delving Deeper
Let us take a closer look at the Neterion and Intel 10G chips configuration on VMware’s vSphere/ESX platform. First, we checked what the S2IO driver of Neterion did when ESX was booting.

If you look closely, you can see that eight Rx queues are recognized, but only one Tx queue. Compare this to the Intel ixgbe driver:

Eight Tx and Rx queues are recognized, one for each VM. This is also confirmed when we start up the VMs. Each VM gets its own Rx and Tx queue. The Xframe-E has eight transmit and eight receive paths, but it seems that for some reason the driver is not able to use the full potential of the card on ESX 4.0.
Conclusion
The goal of this short test was to discover the possibilities of 10 Gigabit Ethernet in a virtualized server. If you have suggestion for more real world testing, let us know.
CX4 is still the only affordable option that comes with reasonable power consumption. Our one-year-old dual-port CX4 card consumes only 6.5W; a similar 10GBase-T solution would probably need twice as much. The latest 10GBase-T (4W instead of >10W per port) advancements are very promising, as we might see power efficient 10G cards with CAT-6 UTP cables this year.
The Neterion Xframe-E could not fulfill the promise of near 10Gbit speeds at low CPU utilization, but our test can only give a limited indication. It is rather weird, as the card we tested was announced as one of the first to support NetQueue in ESX 3.5. We can only guess that driver support for ESX 4.0 is not optimal (yet). The Xframe X3100 is Neterion’s most advanced product and the spec sheet emphasizes its VMware NetQueue support. Neterion ships mostly to OEMs, so it is hard to get an idea of the pricing. When you spec your HP, Dell or IBM server for ESX 4.0 virtualization purposes, it is probably a good idea to check if the 10G Ethernet card is not an older Neterion card.
At a price of about $450-$550, the Supermicro AOC-STG-I2 dual-port with the Intel 82598EB chip is a very attractive solution. Typically, a quad-port gigabit Ethernet solution will cost you half as much, but it delivers only half the bandwidth at twice the CPU load in a virtualized environment.
Originally published at https://www.anandtech.com/show/2956/10gbit-ethernet-killing-another-bottleneck- (Wayback Machine).