SPEC CPU 2017
SPEC CPU 2017 旨在提供性能測量,可用於比較不同電腦系統的計算密集型工作負載,包含 43 個基準測試,分為四個套件:SPECspeed 2017 Integer, SPECspeed 2017 Floating Point, SPECrate 2017 Integer, 和 SPECrate 2017 Floating Point。SPEC CPU 2017 還包括用於測量能耗的可選指標。
為您提供最佳使用者經驗
全新自動化:一個名為sysinfo的程式,為您自動補捉有關測試中的許多系統資訊,並包括由基準(benchmark)產生這些信息的報告之中。該程式為各種晶片和操作系統提供範例,並可隨著用戶擴展到新的系統自動化文件。
廣泛相容性:操作系統已可延伸應用至AIX、HP-UX、IRIX、Mac OS X、Solaris、Windows 和 2011 Linux 版本。SPEC還更新與最新的CPU、x86、Itanium、 POWER和SPARC處理器相容的工作群組。
回應您的需求:為回應您的建議和錯誤報告,已改進其代碼和文件。只要您使用SPEC工具的最新版本,就可增加與開源組件的相容性。
更好的設備:主要的設備升級是為了消除常見的錯誤、提供更多的資訊和實現更多的選擇性和管理效能。
更容易的提交維護:旗幟檔案包含編譯器選項,讓您更容易維護SPEC 網站的提交結果。
SPEC CPU®2017 System Requirements
System Requirements Overview | |||
CPU | ARM, Power ISA, SPARC, or x86. | Limitations apply. See: Supported Toolsets. | |
Physical Memory | SPECspeed® | 16 GB | May vary: memory detail. |
SPECrate® | 1 to 2 GB per copy | ||
Disk Space | Recommended | 250 GB. For SPECrate, add 1.2 GB per copy | May vary: see disk detail. |
Other options are available for smaller installations: see disk detail. | |||
Operating System | AIX, Linux, Mac OS X, Solaris, or Windows. | Limitations apply. See: Supported Toolsets. | |
Compilers | C99, C++2003, Fortran-2003. or Pre-compiled benchmarks. | All 3 are required for reportable. See: Compilers and Precompiled binaries. |
Introduction: Suites and Metrics
This document describes the system requirements for SPEC CPU®2017, a product of the SPEC® non-profit corporation (about SPEC). SPEC CPU 2017 includes source code and data sets for 43 benchmarks, organized into 4 suites:
Short Tag | Suite | Contents | Metrics | How many copies? What do Higher Scores Mean? |
intspeed | SPECspeed®2017 Integer | 10 integer benchmarks | SPECspeed2017_int_base SPECspeed2017_int_peak |
SPECspeed suites always run one copy of each benchmark. Higher scores indicate that less time is needed. |
fpspeed | SPECspeed®2017 Floating Point | 10 floating point benchmarks | SPECspeed2017_fp_base SPECspeed2017_fp_peak |
|
intrate | SPECrate®2017 Integer | 10 integer benchmarks | SPECrate2017_int_base SPECrate2017_int_peak |
SPECrate suites run multiple concurrent copies of each benchmark. The tester selects how many. Higher scores indicate more throughput (work per unit of time). |
fprate | SPECrate®2017 Floating Point | 13 floating point benchmarks | SPECrate2017_fp_base SPECrate2017_fp_peak |
|
he "Short Tag" is the canonical abbreviation for use with runcpu, where context is defined by the tools. In a published document, context may not be clear. To avoid ambiguity in published documents, the Suite Name or the Metrics should be spelled as shown above. |
You may choose which suite(s) you would like to run (there is no requirement to run all of them), and your choice affects hardware requirements: both memory and disk space differ between SPECspeed and SPECrate.
Having chosen a suite, if you will use your results in public, then you must run all the benchmarks in the suite (exceptions) and produce at least the base metric. The peak metric is optional. If producing both base and peak, you will need more disk space.
II. Hardware requirements
II.A. CPU
II.A.1 CPUs supported
SPEC supplies toolsets for ARM, Power ISA, SPARC, or x86.
Limitations apply:
Although SPEC CPU suites are intended to be useful with a wide range of chip architectures, in some cases it is possible that you may find that your chip is not compatible with the available toolsets (for example, if your chip is too old to run one of the supported OS+chip combinations). See the section on Supported Tools.
II.A.2 CPUs: Physical, Virtual, SMP, OpenMP
You can use SPEC CPU 2017 to measure the compute performance of a system with physical or virtual CPUs/processors. You can choose to measure all of the processors on a system, or a subset.
Typical: full system, all its CPUs
Usually SPEC CPU has been used to measure entire systems, with all of the physical CPU chips.
Alternatives: VM, partition, zone,...
There is no prohibition against using SPEC CPU 2017 to measure a subset of the CPU resources on a system, for example with a Virtual Machine (VM), partition, zone, domain, container, processor set, core, hardware thread, or other subset.
If you use a subset (such as a VM), it must have enough memory and disk. For public results, follow the usual rules. (Examples: use only methods that are documented, supported, and generally available to customers; fully disclose what you do, with sufficent detail so that the result can be reproduced; and if you enhance performance by doing something outside the subset -- such as temporarily silencing all the "noisy neighbor" VMs -- fully disclose that, too.)
How many CPUs? [link].
You can use SPEC CPU 2017 to measure performance with arbitrarily large numbers of processors. Usually, if your operating system can handle it, the SPEC CPU toolset can also handle it; if not, contact SPEC. (Of course, you must meet the other requirements of this document, such as sufficient disk and memory for your intended test.)
SPECrate suites run multiple identical jobs at the same time.
Typically, your OS spreads them across CPUs using capabilities known as SMP (Symmetric MultiProcessing).
SPECspeed suites run a single job at a time.
Some of the jobs can use multiple CPUs via OpenMP or compiler auto-parallelization.
The above points are summarized in the table below
SPEC CPU 2017 use of multiple CPUs | ||
SPECspeed | SPECrate | |
Copies | Only one copy of each benchmark is run at a time. | Multiple copies of each benchmark run simultaneously. |
Compiler Parallelism | May be compiled using OpenMP and/or compiler automatic parallelism. | OpenMP and compiler autopar are forbidden. |
How To | At build time, enable OpenMP. | Use runcpu --copies |
Use runcpu --threads=N | or the corresponding config file feature. | |
or the corresponding config file feature. |
II.B. Memory
(In this section, 1 GB = 2^30 bytes.)
Physical Memory
SPECrate | 1 GB | per copy, if compiled for a 32-bit address space. |
2 GB | per copy, if compiled for a 64-bit address space. | |
SPECspeed | 16 GB | minimum system main memory. |
Your needs may vary; see warnings below. |
Virtual Memory
More detail
The nominal memory requirements are based on pre-release testing by SPEC using Linux with GCC and
-m64 for SPECspeed
-m32 for SPECrate
ps -o vsz to record memory usage
which measured the SPECrate benchmarks at 0.89 GB and SPECspeed at 14.99 GB, thus leaving a small margin for overhead vs. the nominal 1 GB and 16 GB.
If you compile the SPECrate benchmarks in 64-bit mode, more memory will be needed. On one system that was tested prior to the release of SPEC CPU 2017, the largest benchmark used 1450 MB when compiled with -m64. Your system might need more.
The SPECspeed benchmark 627.cam4_s needs large stacks, both for the main process (typically controlled via ulimit -s) and for OpenMP threads (typically controlled through your config file setting of environment variables). See the FAQ item on cam4.
Warnings:
- Your memory needs may differ.
- Some systems have more overhead than others.
- Memory usage may vary depending on type of hardware, operating system, compiler, tuning, and other factors.
- Compiling SPECrate for 64-bit environments requires more memory than 32-bit.
- When an operating system runs out of memory, errors may occur that are difficult to diagnose.
- Your memory needs may differ.
II.C. Disk space
(In this section, 1 GB = 10^9 bytes.)
The disk space recommendations below are only estimates. Your environment may differ.
III. Software requirements
III.A. Operating System
The SPEC CPU 2017 toolset relies on several open source components, including GNU Make, Perl, and others. SPEC supplies pre-built versions of these for particular combinations of hardware and operating system, as shown in the table below.
What about other systems? For systems that are not listed in the table of supported toolsets:
- It is possible that the tools might work, but SPEC has not tested them.
- If the tools do not work, it might or might not be possible for you to build them yourself.
- If you would like to try to build the tools, please see the document Building the SPEC CPU 2017 Toolset.
- SPEC may be able to provide advice for your build; however, it will not always be practical for SPEC to do so.
- You might not succeed.
- SPEC CPU 2017 V1.0 includes two unsupported toolsets:
- tru64-alpha 64-bit Tru64 UNIX V5.1 or later
- hpux11iv3-ipf For IPF systems running HP-UX 11iv3 or later.
- These toolsets have received less testing than other tool sets. More importantly, if you try them, you are very likely to find unresolved benchmark portability problems. Nevertheless, the toolsets are provided as a courtesy for possible interest, perhaps when benchmarking historical systems. It is doubtful that you would complete a reportable run with them; you might be able to estimate performance from partial runs. If you develop performance-neutral source changes to allow more benchmarks to complete, you might want to write to SPEC to suggest inclusion of your patches in future versions.
III.B. Compilers or binaries
SPEC supplies the benchmarks in source code form. Therefore, you will need either:
1.A set of compilers. All three are needed:
C99
C++2003
Fortran 2003
--or--
2.A pre-compiled set of benchmark binaries (executables), given to you by another user of the same revision of SPEC CPU 2017.
Note that SPEC does not supply binaries. As mentioned above, SPEC supplies source code.
Binaries generated by a different revision of SPEC CPU 2017 will not work.
You might also need a set of run-time libraries for those executables.
Note that even if someone else compiles the benchmarks, the tester is responsible for compliance with the rules: do not accept binaries from untrusted sources.
New with CPU 2017, the integer suites include a Fortran benchmark.
Reportable runs for all suites require compilation of all three languages (C, C++, Fortran).
Config file
To compile the benchmarks, you will need a config file: a file that defines how to build, run, and report on the SPEC CPU benchmarks in a particular environment.
You might be able to use or modify one of the Example config files that are included with your installed copy of SPEC CPU 2017, or a config file from a result published at wwww.spec.org/cpu2017. The installation guides provide guidance (unix, windows).
Some compilers might not have working examples, and you may need to write your own config file using the documentation.
Occasionally, it might not be possible to create a working config file because a compiler does not support features that are used in the benchmarks. If that occurs, you will need to discuss such problems with the compiler vendor, not with SPEC.
IV. Install Media
You should be familiar either with basic shell commands for Unix (ls, cp, mkdir ...) or basic commands for the Windows Command Prompt (dir, copy, mkdir...).
You will need access to the SPEC CPU 2017 installation media, typically as an ISO image. The Installation Guides (Unix Windows) explain how to use it.
On some systems, the mount command may require privileges or may require additional software. In such cases, you might need to burn a physical DVD using some other system; or, you might need to use the procedure described in the appendix to the Unix installation guide to extract a tarball and use that instead.
Please note that the SPEC CPU 2017 license agreement does not allow you to post the SPEC CPU 2017 software on any public server. If your institution has a SPEC CPU 2017 license, then it's fine to post it on an internal server that is accessible only to members of your institution.
V. Don't be root. Don't run as Administrator. Turn privileges off. (Usually.)
Usually, you do not need privileges. The one known exception is that during installation, on some systems it is possible that you might need a privileged user to mount the installation media or to allocate resources, for example disk space.
After installation is complete, you should not need privileges to run SPEC CPU benchmarks. In general, it is recommended that you use an ordinary user account for SPEC CPU; that way, if your config file accidentally tries to delete the wrong directory, you are much less likely to damage your system.
Warning: SPEC CPU config files can execute arbitrary shell commands.
Read a config file before using it.
SPEC recommends:
- Do not become "root" on Unix systems.
- Do not "run as administrator" on Microsoft Windows systems.
- On all systems, turn privileges off.
- If you discover that an ordinary user account cannot run a benchmark, figure out what specific permission is needed (example: write permission for %SPEC%) or what specific resource needs adjusting (example: stack hard ulimit). Acquire privileges only long enough to fix the specific permission or specific resource, then turn privileges off again.
Cloud
SPEC Cloud IaaS 2018
SPEC Cloud IaaS 2018 builds on the original 2016 release, updates metrics, and workloads and adds easier setup. The benchmark stresses the provisioning, compute, storage, and network resources of infrastructure-as-a-service (IaaS) public and private cloud platforms with multiple multi-instance workloads. SPEC selected the social media NoSQL database transaction and K-Means clustering using Cassandra and Hadoop as two significant and representative workload types within cloud computing. For use by cloud providers, cloud consumers, hardware vendors, virtualization software vendors, application software vendors, and academic researchers.
Graphics / Workstations
SPECviewperf 2020
The SPECviewperf® 2020 v2.0 benchmark, released on June 23, 2021, is the worldwide standard for measuring graphics performance based on professional applications. The benchmark measures the 3D graphics performance of systems running under the OpenGL and DirectX application programming interfaces. The benchmark workloads are called viewsets, and represent graphics content and behavior from actual workstation-class applications, without the need to install the applications themselves.
SPECworkstation 3.1
The SPECworkstation® 3.1 benchmark, released on March 17, 2021, measures all key aspects of workstation performance based on diverse professional applications
The benchmark includes more than 30 workloads containing nearly 140 tests that exercise CPU, graphics, I/O and memory bandwidth. The updates in version 3.1 offer significant improvements in how workloads partition and distribute threads across multi-core CPUs, expand support for next generation GPUs, and add features such as vectorization that leverage advanced CPU hardware to accelerate performance.
SPECapc for 3ds Max™ 2020
The SPECapc®for 3ds Max®2020 benchmark is performance evaluation software for vendors and users of computing systems running Autodesk 3ds Max 2020. This version replaces the SPECapc for 3ds Max 2015 benchmark.
The SPECapc for 3ds Max 2020 benchmark builds upon the SPECapc for 3ds Max 2015 benchmark by including 43 tests for comprehensive measurement of CPU and GPU performance with modeling, interactive graphics and visual effects. Features of this benchmark, including shaders and vector maps, viewport enhancements, and dynamic and visual effects, have been migrated for use in Autodesk 3ds Max 2020. Additional enhancements include a new GUI, a new Results Manager for better results reporting and submission, and improved run-to-run consistency.
SPECapc for Maya 2017
The SPECapc® for Maya 2017 benchmark, released on September 27, 2017, is all-new performance evaluation software for systems running Autodesk Maya 2017 3D animation software.
On December 13, 2017, the SPECapc committee released an updated version of the SPECapc for Maya 2017 benchmark that improves the way multi-sample anti-aliasing (MSAA) is measured. Results from V1.0.6 are not comparable to those from previous versions of the benchmark.
The benchmark includes nearly 80 individual tests that reflect the processes used to model, animate and render scenes within the application. Composite scores are provided for CPU, GPU interactive, GPU animation and GPU-compute performance. There is an option to run the benchmark at 4K resolution.
The following workloads are included within the SPECapc for Maya 2017 benchmark:
- Sven animation – 10 copies of a character model, rigged for animation.
- Tiger – A 1.3-gigabyte realistically rendered tiger model.
- Bifrost bridge fall – A Bifrost simulation of a bridge falling into water.
- Sven space crash – A Bifrost simulation of a spaceship crashing into water.
- Sven textured model – A textured character model that measures retesselation performance.
- Sven space animation – An action-scene animation featuring the Sven character.
- Jungle escape – Another action-scene animation featuring the Sven character.
- Toy store – A walkthrough of a city scene.
SPECapc for PTC Creo 9
The SPECapc® for Creo 9 benchmark is all-new performance evaluation software for systems running PTC Creo 9, a 3D CAD solution that allows engineers to conceptualize, design, analyze, and validate products.
The SPECapc for PTC Creo 9 benchmark provides eight workflows (three CPU and five GPU) that exercise all aspects of system performance when running Creo 9. The benchmark generates composite scores for graphics, CPU and I/O performance. GPU scores are also generated for specific graphics display modes used in Creo 9.
SPECapc for Solidworks 2021
The SPECapc® for Solidworks® 2021 benchmark, released on February 17, 2021, is performance evaluation software for vendors and users of computing systems running Solidworks 2021 CAD/CAM software on Microsoft Windows 10 64-bit platforms.
The SPECapc for Solidworks 2021 benchmark includes 10 models and 50 tests exercising a full range of graphics and CPU functionality. Model sizes range from 392 MB to 2.3 GB in memory. The following models are included in the benchmark:
- Audi R8 (car) — 715 MB in memory, exercising graphics and Visualize CPU ray tracing
- Black owl (desktop computer) — 1.15 GB in memory, exercising graphics and Visualize CPU ray tracing
- Digger (backhoe) — 464 MB in memory, exercising graphics and Visualize CPU ray tracing
- Ferrari (car) — 533 MB in memory, exercising graphics and Visualize CPU ray tracing
- Jet engine — 665 MB in memory, exercising graphics and Visualize CPU ray tracing
- Menjac (motor assembly) — 514 MB in memory, exercising graphics
- Motor — 392 MB in memory, exercising graphics
- SpaceShipCrawler (NASA Crawler Transporter Model*) — 2.3 GB in memory, exercising graphics
- Super car (gullwing door) — 822 MB in memory, exercising graphics
- Tesla tower — 495 MB in memory, exercising graphics
SPECapc for Solidworks 2020
The SPECapc® for Solidworks® 2020 benchmark is performance evaluation software for vendors and users of computing systems running Solidworks 2020 CAD/CAM software on Microsoft Windows 10 64-bit platforms.
The SPECapc for Solidworks 2020 benchmark includes 10 models and 50 tests exercising a full range of graphics and CPU functionality. Model sizes range from 392 MB to 2.3 GB in memory.
MPI / OMP
SPEC ACCEL
SPEC ACCEL tests performance with a suite of computationally intensive parallel applications running under the OpenCL 1.1, OpenACC 1.0, and OpenMP 4.5 APIs. The suite exercises the performance of the accelerator, host CPU, memory transfer between host and accelerator, support libraries and drivers, and compilers.
SPEC MPI2007
SPEC MPI2007是SPEC的基準(benchmark)測試套件可評估MPI-parallel、浮點、跨廣域叢集的計算密集型性能及SMP 硬體。MPI2007延續SPEC 秉持的提供用戶最客觀及具代表性的高性能計算機系統測量,和比較的基準(benchmark)測試套件的傳統。
SPEC MPI2007著重於計算密集型應用程序所使用的消息傳遞介面(Message-Passing Interface ,MPI),這意味著這些基準(benchmark)強調性能表現如下:
● 電腦處理器的類型 (CPU)
● 電腦處理器的數量
● MPI 文庫
● 相互溝通
● 記憶體結構
● 編譯器
● 共享文件系統
SPEC OMP2012
延續OMP2001的測量性能設計,是使用基於共享記憶體並行處理的OpenMP3.1標準的應用程式所設計。此外,OMP2012還包括一個可選的公制測量能源消耗。
SPEChpc 2021
SPEChpc contains four suites, Tiny, Small, Medium, and Large, including groups of full applications or mini-apps covering a wide range of scientific domains and Fortran/C/C++ programming languages. Each suite uses increasingly larger workloads to allow for appropriate evaluation of HPC systems at different sizes, ranging from a single node to hundreds of nodes. All benchmarks are ported to use either pure-MPI or hybrid MPI+OpenACC, MPI+OpenMP (task/thread based), or MPI+OpenMP using "Target", thus allowing measurement on heterogenous system.
Java Client / Server
SPECjbb2015基準(benchmark)測試由最新的Java應用程序的功能來衡量性能來開發。這是包括JVM廠商、硬體開發者、Java應用程序開發人員、研究人員和學術界的成員們,眾所期待的Java服務器性能。
SPECjbb2015 的最新功能如下:
● 一個基於全球性跨國公司的IT基礎設施,可處理銷售時點情報系統(point-of-sale, POS)的各種需求、線上購物和資料探勘操作。
● 透過單純吞吐量度量方法和度量措施,可於服務水準協議(SLAs)指定的反應時間範圍為10ms至500ms之下,測量重要的吞吐量。
● 支持多項運行配置,使用戶能於多層級的系統堆疊進行分析和克服瓶頸,其包括硬體、OS、JVM和應用程序層級。
● Java的新功能和其他重要的表現元素的運作,包括最新的數據格式(XML),使用壓縮通訊和信息安全傳遞。
● 支援虛擬化和雲端環境。
SPECjEnterprise 2018
該SPECjEnterprise ® 2018的Web Profile基準措施的Java EE 7 Web Profile的應用程序服務器提供全系統性能和如JVM,數據庫,CPU,磁盤和服務器及其配套基礎設施。
工作負載包含一個基於端到端基於Web的汽車保險經紀應用程序,用於處理Web請求。
Technologies used in the benchmark:
- Expression Language (EL) 3.0
- Standard Tag Library for JavaServer Pages (JSTL) 1.2
- JavaServer Faces (JSF) 2.2
- Java API for RESTful Web Services (JAX-RS) 2.0
- Java API for WebSocket (WebSocket) 1.0
- Java API for JSON Processing (JSON-P) 1.0
- Common Annotations for the Java Platform 1.2
- Enterprise JavaBeans (EJB) 3.2 Lite
- Java Transaction API (JTA) 1.2
- Java Persistence API (JPA) 2.1
- Bean Validation 1.1
- Contexts and Dependency Injection for the Java 1.1
- Dependency Injection for Java 1.0
The SPECjEnterprise 2018 Web Profile benchmark is a newly designed benchmark. It has been designed from and developed to cover the Java EE 7 Web Profile specification's significantly expanded and simplified programming model, highlighting the major features used by developers in the industry today. The design of the benchmark enables scaling both vertically and horizontally. It is designed to support flexible deployment scenarios. The benchmark is designed to work as a single instance deployment or on many instances and can be deployed on bare-metal, virtualized and public or private cloud environments.
SPECjEnterprise2010
SPECjEnterprise2010 benchmark是一個完整的系統基準(benchmark),可允許Java EE5.0伺服器和配套的基礎設施,如JVM,數據庫,CPU,磁碟和伺服器的性能測量和表徵。
工作量由一個終端到終端的基於Web的訂單處理領域,RMI和Web服務驅動的製造領域,並利用基於文件的Web服務的供應鏈模式。應用程序是一個Java類別的集合,Java classes、Java Servlet、Java Server Pages、Enterprise Java Beans、Java Persistence Entities (pojo's) 和 Message Driven Beans。
SPECjEnterprise2010是SPEC的J2EE終端到終端的行業標準基準(benchmark)測試應用程序的第三代。新的SPECjEnterprise2010 benchmark已重新設計並發展到覆蓋Java EE5.0規範的顯著擴大和簡化的編程模式,凸顯出今日在同產業中被開發者使用的主要功能。這提供一個現實世界的驅動應用程序伺服器的Java EE規範的實施,特別是其最大潛力,並強調底層的硬體和軟體系統能允許的最大工作量。
SPEC已特別注意盡可能讓此基準(benchmark)容易安裝和運行。這同時成功地利用簡單化的Java EE 5.0平台功能,如備註、明智的錯誤和在開發和運行基準(benchmark)驅動器所使用的開放原始碼的Faban 設備。
SPECjEnterprise2010的最新設計跨越的Java EE5,包括新的EJB3.0和WSEE組件架構,信息驅動beans和功能水平交易。
SPECjvm 2008
SPECjvm 2008 is a benchmark suite for measuring the performance of a Java Runtime Environment (JRE), containing several real life applications and benchmarks focusing on core java functionality. The SPECjvm 2008 workload mimics a variety of common general purpose application computations.
Storage
SPECstorage Solution 2020
The most recent version of SPEC's benchmark suite designed to evaluate performance using file server throughput and response time
Power
SPECpower_ssj2008
SPECpower_ssj2008是第一個產業標準的SPEC 基準(benchmark),可評估磁碟機伺服器和多節點類別電腦的功能和性能特點。使用SPECpower_ssj2008,SPEC可以同樣的方式定義伺服器功率測量標準,就像我們已做過的性能測試。
來自對IT產業的認識、電腦製造商和政府,越來越關注在硬碟所造成的功耗和性能基準(benchmark)伺服器的能源使用。
目前,許多在廠商報告中的一些能源效率的數據,由於工作量,配置,測試環境等的差異,讓這些數據往往是沒有直接的可比較性。這個基準(benchmark)的發展提供了一種方法,可配合性能指標來衡量電源(AC輸入)。這應該可以幫助IT管理者考慮到功耗的特點以及與其他的選擇標準,以提高數據中心的效率。
最初的基準(benchmark)位址只有伺服器工作負載的一個子集:伺服器端的Java性能。它能行使的CPU、快取記憶體、記憶體階層架構和共享內存處理器(SMPS),以及JVM實行(Java Virtual Machine)、JIT (Just-In-Time)的編譯器、資源回收、線程和某些方面的可擴展性的操作系統。基準(benchmark)上運行多種操作系統和硬體架構,不需廣泛的客戶主機或儲存應用平台。另外正規劃額外的工作量。
由於這是第一次建立的電力性能基準(benchmark),一般用於功率測量方法已被證明可協助對其他基準(benchmark)感興趣的開發人員,在測量功率和建立一套通用的做法,並希望這成為一個新領域的基準(benchmark)。
Virtualization
SPEC VIRT_SC 2013
SPEC's updated benchmark addressing performance evaluation of datacenter servers used in virtualized server consolidation. SPEC VIRT_SC 2013 measures the end-to-end performance of all system components including the hardware, virtualization platform, and the virtualized guest operating system and application software. The benchmark supports hardware virtualization, operating system virtualization, and hardware partitioning schemes.
SPECvirt Datacenter 2021
The SPECvirt Datacenter 2021 benchmark is the third generation SPEC benchmark for evaluating the performance of virtualized environments and the first for measuring the performance of multiple hosts in virtualized datacenters.
SPEC Tools
SPEC SERT Suite 2.0
The SERT suite was created by Standard Performance Evaluation Corporation (SPEC) at the request of the US Environmental Protection Agency. The SERT suite 2.0 adds a single-value metric, reduces runtime, improves automation and testing, and broadens device and platform support. Designed to be simple to configure and use via a comprehensive graphical user interface, the SERT suite uses a set of synthetic worklets to test discrete system components such as processors, memory and storage, providing detailed power consumption data at different load levels.
The SERT suite metric, created with the support of the RG Power Working Group, rates the server efficiency of single- and multi-node servers across a broad span of configurations.
SPEC SERT Suite 1.1.1
The SERT suite was created by Standard Performance Evaluation Corporation (SPEC) at the request of the US Environmental Protection Agency. The SERT suite 1.1.1 is the most current SERT version supported by the U.S. EPA Energy Star v2.0 program. Designed to be simple to configure and use via a comprehensive graphical user interface, the SERT suite uses a set of synthetic worklets to test discrete system components such as processors, memory and storage, providing detailed power consumption data at different load levels.
SPEC Chauffeur WDK Tool
The Chauffeur™ WDK (Worklet Development Kit) Tool was designed to simplify the development of workloads for measuring both performance and energy efficiency. Because the Chauffeur WDK tool contains functions that are common to most workloads, developers of new workloads can focus on the actual business logic of the application, and take advantage of the Chauffeur WDK tool's capabilities for configuration, run-time, data collection, validation, and reporting.
The Chauffeur WDK tool was initially designed to meet the requirements of the SERT. However, SPEC recognized that the framework would also be useful for research and development purposes. The Chauffeur framework is now being made available as the Chauffeur WDK (Worklet Development Kit). This kit can be used to develop new workloads (or "worklets" in Chauffeur terminology). Researchers can also use the Chauffeur WDK to configure worklets to run in different ways, in order to mimic the behavior of different types of applications. These features can be used in the development and assessment of new technologies such as power management capabilities.
Version 2.0 is based on the SERT suite 2.0 infrastructure and includes significant enhancements to the hardware detection, customization options of generating HTML reports, and developer documentation. It has now reduced memory requirements for the Director when signing results files and reduced the size of the result output for large systems or clusters.
The Chauffeur WDK tool 2.0 added worklet-specific normalization of results and an updated list of supported operating systems including Ubuntu (14.04 LTS and 16.04 LTS) as well as current versions of Windows Server, RHEL, SLES, AIX, and Solaris. The Chauffeur WDK tool also includes the latest PTDaemon integration for power analyzers and temperature sensors, along with data collection, validation and reporting.