What’s an e1000?
The e1000 is the Intel 82545EM Gigabit Ethernet Controller. VMware offers an emulated version of this controller. Most operating systems are shipped with a 82545EM driver. The 82545EM driver sucks! That’s why Intel replaced it with e1000e aka 82574L
What’s an e1000e?
The e1000e is the Intel 82574L Gigabit Ethernet Controller. In vSphere 5 (HW8), VMware offers an emulated version of the e1000e. Windows 7 and Windows 2008 are shipped with drivers for the 82574L. The 82574L is cool, but is it faster than an VMXNET3?
What’s VMXNET3?
The VMXNET3 adapter is the next generation of Para virtualized NIC designed for performance. The VMXNET3 network adapter is a 10Gb virtual NIC. Drivers are shipped with the VMware tools and most OS are supported. VMXNET3 is much faster than e1000 or e1000e. VMXNET3 has less CPU overhead compared to e1000 or e1000e. VMXNET3 is more stable than e1000 or e1000e
Wednesday, December 14. 2011
Myth - E1000 is faster than VMXNET3
Trackbacks
Trackback specific URI for this entry
No Trackbacks
Comments
Display comments as
(Linear | Threaded)
The default adapter when creating/adding a vNIC is an E1000. Is there a way to change this so it defaults to VMXNET3?
#1
Lieven
(Homepage)
on
2011-12-13 23:43
(Reply)
(the blog seemed to massively mangle my formatting for some reason)
Just a minor note, the e1000 covers far more than just the 82545EM, but at least these as well
82540
82541
82542
82543
82544
82545
82546
82547
(from looking at linux-2.6.39.4/drivers/net/e1000/e1000_hw.c)
It could be the vSphere emulated version sucks but e1000 by itself was quite good on normal hardware in Linux I have used it going back to 2003, and have easily been able to sustain 900Mbit/second on it without any tuning (using iperf as a test bed I remember back in 2005 was the last round of tests I did on it). I think around 2007 is when newer Intel NICs started coming out that did not work with e1000 anymore. I also seem to recall a particularly nasty bug with ESX 3.x at the time with regards to VLANs on e1000e (at the host/hypervisor level), was very annoying!
e1000 doesn't support newer NICs, so e1000e came out. e1000 was especially good because it was "stable" for so long, more recent Intel NICs have required more frequent driver updates. The e1000e that comes with Ubuntu 10.04 for example(1.5 yrs old) doesn't work with my Dell desktop(4 months old) I have to download a newer version and re-compile it whenever the kernel is updated. With the stock driver, the driver loads but doesn't detect any interfaces. I've had similar problems with e1000e for years now.
I've used e1000 in vsphere for a while too no issues though I have not run particularly bandwidth heavy apps on it.
#2
nate
(Homepage)
on
2011-12-14 00:04
(Reply)
I think you could make a case for the e1000 being more stable. VMware recommends the e1000 for thier connection servers. Botched VMware tool jobs can cause issues with VMXNET3. Also there is a know bugs right now with VMXNET3.
Do you know why the e1000e was created? David Davis had a post saying it was for performance. I can't find any info on the thing.
Overall a great post
#3
Dwayne Lessner
(Homepage)
on
2011-12-14 07:05
(Reply)
guys, I have clocked two virtual machines running on the same physical host and found totally different results (one of them is alligned with your, though)
- physical hosts : HP Proliant DL120 G6, one CPU, four cores but I don't remember exact model of CPU, 8GB physical RAM, local SATA storage 1x 160GB HDD [RAM and storage being not important in this test as long as we generate packets, right ?]
- esx v5.0 version 5.0.0 U1 fully patched up to ESXi500-201207001
- re-tested again with v5.1 last week
- simple vNetwork, Gigabit/auto or Gigabit/Full duplex manual settings do not change the results at all. Switch Dlink exact model unknown, 48 port enterprise class, 19" rackmountable, no other traffic there only my ESX hosts.
- no other virtual machines installed/running on those hosts. This was dedicated platform for my testing only.
- virtual machine v7 and v8
- Mikrotik linux distribution , versions v5.7, v5.15,
v5.17 and current latest v5.20 tested. Each VM allocated one core, later two cores in repeated test which honestly didn't change the results at all. 512MB vRAM as it's plenty enough for this system and test (used around 40MB RAM only, hey this is linux right), vmxnet3, e1000, e1000e virtual adapters. Seems like the version of Mikrotik had absolutely no influence on result of test, read more below.
There is built-in test in those Mikrotik distributions enabling user to test with random or compressible data. It is possible to have receive/send/both streams. Compressible data heavily load CPU as expected and is not representative measurement due to variations, but random generated uncompressible data leads to repeatitive statistically meaningful results as there is extremely low delta in results : usually it is within 3% which is excellent.
As far as I understand, the built-in throughput test in Mikrotik is based on iperf. Mikrotik has their own CLI so it is not standard linux command line thus no verification of this is possible. But I think this is not important, as far as we believe the vendor and generate packets (for sure they are not read/written from disk !! easily verified) and those packets are not compressible (verifiable but sorry I won't capture the traffic and analyze it afterwards), the test is performed in absolutely repeated and correct way.
Now the results : vmxnet3 gives little around 400Mbit/s throughput (380 to 410Mbit/s) while E1000 and E1000E both yield 990Mbit/s throughput with minimum of 975Mbit/s. Let me stress out stable and repeated EVERY TIME so this is extremely consistent, with every setting, every version of ESX, Mikrotik in virtual machines.
This leads me to think : is the VMXNET3 driver in Mikrotik flawed and not working optimally ? The answer is probably no. Explanation - the result of 400Mbit/s I achieved with VMXNET3 is extremely aligned with the result presented here in your blog. The only huge difference is with the E1000/E1000E results where I have three times as much performance.
I'm presenting detailed description so anyone of you can re-test. If you see any problems in my "methodology", let me know.
So no eggs are created equal and it is heavily possible that
E1000/E1000E are FASTER than VMXNET3. I dare to re-test this with
Windows 2008, 2003 and Windows7, stay tuned :)
#3.1
Lubomir Zvolensky
on
2012-10-13 20:00
(Reply)

