November 11, 2015 08:01 ET

EEMBC Reveals First Performance Benchmarks for Scale-Out Servers and Associated SoCs

ScaleMark-Caching Measures Memory Caching Latency and Throughput While Demonstrating EEMBC's Industry-Standard Methodology

EL DORADO HILLS, CA--(Marketwired - Nov 11, 2015) - The Embedded Microprocessor Benchmark Consortium (EEMBC, pronounced "embassy") today unveiled ScaleMark-Caching test, which measures a server's responsiveness to client requests for web data.. This benchmark, the first produced by EEMBC's Cloud and Big-Data Server working group, is based on the popular Memcached application, with the inclusion of a fixed set of operating parameters and utilization of a large-scale web-server workload. As Memcached is used in data centers to optimize performance and energy usage, EEMBC's ScaleMark-Caching is not a proxy benchmark, it benchmarks a real-world application utilizing real-world data.

The Cloud and Big-Data Server working group is chartered by EEMBC to build a standardized, industry-endorsed suite of speed and efficiency benchmarks that characterize SoC and system-level performance for modern cloud and big-data-related workloads, commonly called hyperscale and "scale-out" computing. Although ScaleMark-Caching is only one segment of the benchmark suite, it successfully embodies the comprehensive methodology that EEMBC has established. Like previous benchmarks, perhaps the most important attribute of the ScaleMark-Caching benchmark is its clear definition of parameters. This approach ensures repeatable and verifiable benchmark results. The EEMBC standard mode implements the "Facebook ETC" profile explained in "Workload Analysis of a Large-Scale Key-Value Store1." This profile defines the key size, value size, and inter-arrival distribution.

"ScaleMark-Caching is an extraordinarily complex application under the hood that accurately represents both the server under test as well as clients making requests to the server," said Markus Levy, EEMBC's president. "Consistent with our industry-proven methodology, our Cloud and Big-Data Server working group utilized various industry-standard tools to allow users to easily configure parameters to define and test their system's configuration."

"EEMBC's Cloud and Big Data Server Benchmark working group is filling a significant gap for modern web-scale infrastructure vendors and customers -- it is creating an accurate set of processor and system-architecture-neutral benchmarks for high-value workloads," said Paul Teich, Principal Analyst, Tirias Research. "This working group is enabling much more accurate total cost of ownership metrics for modern data centers, workload by workload. I'm looking forward to seeing EEMBC ScaleMark-Caching results."

Following the release of its ScaleMark-Caching benchmark, the group is developing a media-streaming benchmark and is defining requirements for a subsequent generation of benchmarks. Like all EEMBC benchmarks, all Cloud and Big-Data Server working group benchmarks will be based on real-world applications.

The Cloud and Big-Data Server working group, led by Shay Gal-On, principal engineer at Cavium, Inc., is comprised of EEMBC members with strong interest in the server market. These include AMD, ARM, Cavium, Intel, and others. EEMBC encourages vendors and manufacturers to join the consortium's working groups to contribute to the definition and development of its next-generation benchmark suites. To join the Cloud and Big Data Server, or other working group and/or try out the EEMBC ScaleMark-Caching benchmark, contact Markus Levy for details.

EEMBC was formed in 1997 to develop performance benchmarks for the hardware and software used in embedded systems. EEMBC benchmarks help predict the performance and energy consumption of embedded processors and systems in a range of applications (i.e. automotive/industrial, digital imaging and entertainment, networking, office automation, telecommunications, and connected devices) and disciplines (processor core functionality, floating-point, Java, multicore, and energy consumption).

EEMBC members include Ambiq Micro,, AMD, Analog Devices, Andes Technology, ARM, Atmel, C-Sky Microsystems, Cavium, Cypress Semiconductor, Dell, Freescale Semiconductor, Green Hills Software, IAR Systems, Imagination Technologies-MIPS, Infineon Technologies, Intel, Lockheed Martin, Marvell Semiconductor, MediaTek, Microchip Technology, Nokia Networks, Nordic Semiconductor, NVIDIA, NXP Semiconductors, Qualcomm, Realtek Semiconductor, Red Hat, Renesas Electronics, Samsung Electronics, Silicon Labs, Somnium Technologies, Sony Computer Entertainment, STMicroelectronics, Synopsys, Texas Instruments, TOPS Systems, and Wind River Systems.

1 Workload Analysis of a Large-Scale Key-Value Store, by B. Atikoglu, et al, June 2012

SIGMETRICS'12, June 11-15, 2012, London, England, UK.
Copyright 2012 ACM 978-1-4503-1097-0/12/06 ...$10.00

Contact Information