2.16.2.10. Yardstick Test Case Description TC012

Memory Bandwidth

test case id

OPNFV_YARDSTICK_TC012_MEMORY BANDWIDTH

metric

Memory read/write bandwidth (MBps)

test purpose

The purpose of TC012 is to evaluate the IaaS compute performance with regards to memory throughput. It measures the rate at which data can be read from and written to the memory (this includes all levels of memory).

The purpose is also to be able to spot the trends. Test results, graphs and similar shall be stored for comparison reasons and product evolution understanding between different OPNFV versions and/or configurations.

test tool

LMbench

LMbench is a suite of operating system microbenchmarks. This test uses bw_mem tool from that suite including:

  • Cached file read

  • Memory copy (bcopy)

  • Memory read

  • Memory write

  • Pipe

  • TCP

(LMbench is not always part of a Linux distribution, hence it needs to be installed. As an example see the /yardstick/tools/ directory for how to generate a Linux image with LMbench included.)

test description

LMbench bw_mem benchmark allocates twice the specified amount of memory, zeros it, and then times the copying of the first half to the second half. The benchmark is invoked in a host VM on a compute blade. Results are reported in megabytes moved per second.

configuration

File: opnfv_yardstick_tc012.yaml

  • SLA (optional): 15000 (MBps) min_bw: The minimum amount of memory bandwidth that is accepted.

  • Size: 10 240 kB - test allocates twice that size (20 480kB) zeros it and then measures the time it takes to copy from one side to another.

  • Benchmark: rdwr - measures the time to read data into memory and then write data to the same location.

  • Warmup: 0 - the number of iterations to perform before taking actual measurements.

  • Iterations: 10 - test is run 10 times iteratively.

  • Interval: 1 - there is 1 second delay between each iteration.

SLA is optional. The SLA in this test case serves as an example. Considerably higher bandwidth is expected. However, to cover most configurations, both baremetal and fully virtualized ones, this value should be possible to achieve and acceptable for black box testing. Many heavy IO applications start to suffer badly if the read/write bandwidths are lower than this.

applicability

Test can be configured with different:

  • memory sizes;

  • memory operations (such as rd, wr, rdwr, cp, frd, fwr, fcp, bzero, bcopy);

  • number of warmup iterations;

  • iterations and intervals.

Default values exist.

SLA (optional) : min_bandwidth: The minimun memory bandwidth that is accepted.

usability

This test case is one of Yardstick’s generic test. Thus it is runnable on most of the scenarios.

references

LMbench bw_mem

ETSI-NFV-TST001

pre-test conditions

The test case image needs to be installed into Glance with Lmbench included in the image.

No POD specific requirements have been identified.

test sequence

description and expected result

step 1

A host VM with LMbench installed is booted.

step 2

Yardstick is connected with the host VM by using ssh. “lmbench_bandwidth_benchmark” bash script is copied from Jump Host to the host VM via ssh tunnel.

step 3

‘lmbench_bandwidth_benchmark’ script is invoked. LMbench’s bw_mem benchmark starts to measures memory read/write bandwidth. Memory read/write bandwidth results are recorded and checked against the SLA. Logs are produced and stored.

Result: Logs are stored.

step 4

The host VM is deleted.

test verdict

Test fails if the measured memory bandwidth is below the SLA value or if there is a test case execution problem.