The Next Generation Open Compute Hardware: Tried and Tested
Benchmark Results
Though the systems are equipped with similar components, some notable observations can be made. Each benchmark was executed three times to verify the accuracy of our results. All ratios are calculated using the Dell R730 as baseline (=100%).
| OLAP | 95pct Response Time (Lower is better) |
Power (Lower is better) |
Throughput (Higher is better) |
|||
| Concurrency | Facebook Leopard |
Intel Decathlete v2 |
Facebook Leopard |
Intel Decathlete v2 |
Facebook Leopard |
Intel Decathlete v2 |
| 5 | 114% | 111% | 99% | 99% | 94% | 117% |
| 10 | 108% | 105% | 99% | 98% | 96% | 109% |
| 25 | 96% | 94% | 100% | 101% | 100% | 116% |
| 50 | 98% | 56% | 103% | 110% | 101% | 112% |
| 100 | 96% | 49% | 110% | 149% | 103% | 119% |
| 250 | 98% | 67% | 105% | 144% | 103% | 120% |
| 250 | 95% | 67% | 106% | 144% | 103% | 120% |
| 250 | 95% | 67% | 105% | 143% | 102% | 120% |
| 250 | 90% | 62% | 106% | 143% | 103% | 121% |
An interesting result: the Dell outperforms both Open Compute servers when it comes to energy efficiency. It is good to remember that the PSU used for the Facebook server is not optimal (it should get its power from a power shelf). Leopard is very close, and gets a slightly better performance mark, while Decathlete is a respectable 20% faster, but uses 40% more power in the process.
Next, we tested with our real-world Elasticsearch benchmark. Due to ES's internal queuing algorithm, the throughput and response time can vary wildly when it drops a 'heavy' request to allow it to pass many smaller ones, and we've seen the inverse happen as well. These tests were executed five times to confirm ensure the consistency of most results.
| ElasticSearch | 95pct Response Time (Lower is better) |
Power (Lower is better) |
Throughput (Higher is better) |
|||
| Concurrency | Leopard | Decathlete v2 | Leopard | Decathlete v2 | Leopard | Decathlete v2 |
| 5 | 115% | 73% | 107% | 106% | 96% | 104% |
| 10 | 88% | 88% | 75% | 94% | 101% | 101% |
| 25 | 74% | 26% | 90% | 98% | 118% | 165% |
| 50 | 96% | 41% | 95% | 104% | 100% | 153% |
| 100 | 97% | 74% | 98% | 108% | 107% | 124% |
| 200 | 103% | 85% | 97% | 112% | 108% | 129% |
| 200 | 96% | 75% | 100% | 110% | 108% | 130% |
| 200 | 89% | 66% | 95% | 112% | 110% | 137% |
| 200 | 96% | 80% | 95% | 108% | 113% | 128% |
| 250 | 92% | 70% | 97% | 113% | 101% | 121% |
Whereas the Dell and Leopard performed comparably in the OLAP test, ElasticSearch tests results are in favor of the Leopard. The throughput and response times are slightly better and Leopard uses slightly less power resulting in a tangibly better performance/watt with a sub optimal power supply. The Decathlete system gets the best scores on the board again, but this time the power usage increase is relatively modest: 8 to 13 percent.
Originally published at https://www.anandtech.com/show/9138/open-compute-hardware-tried-and-tested (Wayback Machine).