Archive 1997–2024

10G Ethernet: More Than a Big Pipe

Consolidation and Choking

If we consolidate storage, network, and management traffic in a single or dual 10GbE cable, a simpler, cheaper and easier to manage datacenter is in reach. So how well do the new NICs cope with these I/O demands? We decided to mix a VM with an iSCSI initiator that is sending off lots of storage traffic with three VMs that are demanding lots of “normal” network traffic. In our first scenario we simulate the “I don’t have time to properly configure” scenario. We just hooked up one VM with an iSCSI initiator, and three other VMs were running IxChariot endpoints, creating as much traffic as they like.

iSCSI & Network traffic, no QoS

It is easy to notice that the four virtual function NICs allow the Neterion X3120 to spread the load more evenly. The iSCSI VM runs 50% faster on the Neterion chip, which is significant. Reading the disk blocks 50% faster can make a real difference to the end-user, as the user experience is in many applications highly depended on the slow disks.

Let us see how much CPU load was necessary to keep this kind of brutal network traffic under control. The two CPUs on the virtual node were Xeon X5670 2.93GHz.

CPU load, 3 network VMs + 1 iSCSI QoS

The Neterion chip does a better job at a slightly lower CPU load.

Originally published at https://www.anandtech.com/show/4014/10g-more-than-a-big-pipe (Wayback Machine).