The OpenPOWER Saga Continues: Can You Get POWER Inside 1U?
Apache Spark Benchmarking
Spark is wonderful framework, but you need some decent input data and some good coding skills to really test it. Speeding up Big Data applications is the top priority project at the lab I work for (Sizing Servers Lab of the University College of West-Flanders), so I was able to turn to the coding skills of Wannes De Smet to produce a benchmark that uses many of the Spark features and is based upon real world usage.
The test is described in the graph above. We first start with 300 GB of compressed data gathered from the CommonCrawl. These compressed files are a large number of web archives. We decompress the data on the fly to avoid a long wait that is mostly storage related. We then extract the meaningful text data out of the archives by using the Java library "BoilerPipe". Using the Stanford CoreNLP Natural Language Processing Toolkit, we extract entities ("words that mean something") out of the text, and then count which URLs have the highest occurrence of these entities. The Alternating Least Square algorithm is then used to recommend which URLs are the most interesting for a certain subject.
We tested with Apache Spark 1.5 in standalone mode (non-clustered) as it took us a long time to make sure that the results were repetitive. For now, we're sticking with version 1.5 to be able to compare with earlier results.

The Xeon E5-2640 v4 and 8-core POWER8 finish neck-and-neck. And that is not good news for the POWER8. It needs to beat the lower-and cheaper Xeon by a large margin to make people switch.
Originally published at https://www.anandtech.com/show/10923/openpower-saga-tyans-1u-power8-gt75 (Wayback Machine).
