For a bit of fun, I decided to do a little bit of micro-benchmarking of Vert.x vs node.js HTTP performance.
Firstly, a disclaimer: This isn’t rigorous benchmarking, and I haven’t attempting to benchmark a wide range of use cases (I just test some HTTP stuff here). All the benchmarking is done on a single machine (my desktop). This is not ideal – a good benchmark would have clients and servers on different physical machines and a real network between them. So don’t read too much into these results. (You can read a little bit, just not too much 
) In the future, when I can get hold of some real hardware I intend to do some real benchmarking. Until then this will have to do.
Apparatus: All tests were run on my desktop: An AMD Phenom II X6 (that’s a six core, not as good as the latest Intels but pretty good), 8GB RAM (although only a fraction was used in the tests), Ubuntu 11.04.
Versions: vert.x-1.0.final, node.js 0.6.6
The tests:
Test 1 tests the performance of a trivially simple HTTP server which returns a 200-OK for every request.
Test 2 tests a slightly less trivial HTTP server which serves a real static file from the file system. The file is 72 bytes in size and just contains this:
<html>
<head>
<title>Some page</title>
</head>
<body>
<h1>Foo!</h1>
</body>
</html>
The client used was written in Java using Vert.x, and fires requests using 10 connections, and a maximum of 2000 pipelined requests at any one time per connection. I used 6 instances of the client so that means 60 connections in total.
Every 2000 responses received it sends a message on the Vert.x event bus which is picked up by another component which computes the overall and average rates and prints them to the console every three seconds.
Each test was run for 1 minute, and the average (mean) rate taken.
The code is here if you want to see it or try it out yourself.
For comparison I ran each tests against three server setups:
- A single node.js server
- A single Vert.x server process with 6 event loops.
- A node.js server with six child processes forked using the `cluster` module to distribute work out to them
One of the advantages of Vert.x over node.js is a single server instance can utilise all the cores on your server without having to manually fork child processes and without you having to write load-balancing code to farm out requests to the children.
For the Vert.x servers I tested a version of the server for each of the languages we support: currently JavaScript, Ruby, Groovy and Java.
Results:
In this test, the server simply returns 200-OK to each request. I tested Vert.x Java, Ruby, Groovy and JavaScript single server processes running with 6 event loops, against a node.js single server process. Since a node.js server only has 1 event loop, I also show the results for 6 node.js server processes using the cluster module.
As you can see a single Vert.x process hugely outperforms a single node.js process, and also greatly outperforms a cluster of node.js instances.
In this test, it serves a small static file. I tested Vert.x Java, Ruby, Groovy and JavaScript single server processes running with 6 event loops, against a node.js single server process. Since a node.js server only has one event loop, I also show the results for 6 node.js server processes using the cluster module.
For the node results I show the results both for using the `readFile` function to load the file, and also for using streams. For streams – I show three variations – one using a blocking call to stat the file to get the size, another using a non blocking call to stat the file, and a third using chunked transfer encoding (so it needs no stat).
Again, a single Vert.x process hugely outperforms a single node.js process, and also greatly outperforms a cluster of node.js instances.
Summary:
I don’t think I need to say much here. The numbers speak for themselves.
[Edit. I initially wasn't going to provide any summary, but several observers have said I should summarise the results. So here's my summary:]
1. Vert.x is much faster than Node.js when reading the same file from the filesystem more than once. Neither Vert.x nor the JVM is doing any explicit caching, so this is most likely due to the OS caching it (memory mapped files?). In a webserver-type app the same file is likely to be served many times so this is pretty significant
2. Node.js doesn’t appear to scale particularly well using the `cluster` module. The advice from the node.js folks is not to use it (for now). This leaves node with no out-of-the-box way to scale across multiple cores 
3. Many of the out of the box load testing tools don’t appear to pipeline very much. The Vert.x results are dramatically better when there’s a high degree of pipelining going on.
4. Java on the JVM is extremely fast. No surprises there But what is very encouraging is how the other JVM langs held up – in particular Rhino held up very well against V8 
[Second Edit: As I said in my replies, I should never posted the results with Vert.x "crippled". No-one is going to do this in real life. They will just be running Vert.x or Node.js on the same hardware in whatever way they can get it to exploit the available cores. This really makes the "crippled" Vert.x results redundant. So I've removed them. ]
