- The Yardstick framework, the Yardstick test cases are open-source software,
- licensed under the terms of the Apache License, Version 2.0.
1.1. Yardstick Release Notes¶
1.1.1. Abstract¶
This document compiles the release notes for the Hunter release of OPNFV Yardstick.
1.1.2. Version History¶
Date | Version | Comment |
May 10, 2019 | 8.0.0 | Yardstick for Hunter release |
1.1.3. Important Notes¶
The software delivered in the OPNFV Yardstick Project, comprising the Yardstick framework, and the Yardstick test cases is a realization of the methodology in ETSI-ISG NFV-TST001.
The Yardstick framework is installer, infrastructure and application independent.
1.1.4. OPNFV Hunter Release¶
This Hunter release provides Yardstick as a framework for NFVI testing and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
- Documentation generated with Sphinx
- User Guide
- Developer Guide
- Release notes (this document)
- Results
- Automated Yardstick test suite (daily, weekly)
- Jenkins Jobs for OPNFV community labs
- Automated Yardstick test results visualization
- Dashboard using Grafana (user:opnfv/password: opnfv), influxDB is used as backend
- Yardstick framework source code
- Yardstick test cases yaml files
- Yardstick plug-in configuration yaml files, plug-in install/remove scripts
For Hunter release, the Yardstick framework is used for the following testing:
- OPNFV platform testing - generic test cases to measure the categories:
- Compute
- Network
- Storage
- OPNFV platform network service benchmarking (NSB)
- NSB
- Test cases for the following OPNFV Projects:
- Container4NFV
- High Availability
- IPv6
- KVM
- Parser
- StorPerf
- VSperf
The Yardstick framework is developed in the OPNFV community, by the Yardstick team.
Note
The test case description template used for the Yardstick test cases is based on the document ETSI-ISG NFV-TST001; the results report template used for the Yardstick results is based on the IEEE Std 829-2008.
1.1.5. Release Data¶
Project | Yardstick |
Repo/tag | yardstick/opnfv-8.0.0 |
Yardstick Docker image tag | opnfv-8.0.0 |
Release designation | Hunter 8.0 |
Release date | May 10, 2019 |
Purpose of the delivery | OPNFV Hunter 8.0.0 |
1.1.6. Deliverables¶
1.1.6.1. Documents¶
- User Guide: <yardstick:userguide>
- Developer Guide: <yardstick:devguide>
1.1.6.2. Software Deliverables¶
- The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: opnfv-8.0.0)
1.1.6.2.1. List of Contexts¶
Context | Description |
Heat | Models orchestration using OpenStack Heat |
Node | Models Baremetal, Controller, Compute |
Standalone | Models VM running on Non-Managed NFVi |
Kubernetes | Models VM running on Non-Managed NFVi |
1.1.6.2.2. List of Runners¶
Runner | Description |
Arithmetic | Steps every run arithmetically according to specified input value |
Duration | Runs for a specified period of time |
Iteration | Runs for a specified number of iterations |
IterationIPC | Runs a configurable number of times before it returns. Each iteration has a configurable timeout. |
Sequence | Selects input value to a scenario from an input file and runs all entries sequentially |
Dynamictp | A runner that searches for the max throughput with binary search |
Search | A runner that runs a specific time before it returns |
1.1.6.2.3. List of Scenarios¶
Category | Delivered |
Availability | Attacker:
HA tools:
Monitor:
|
Compute |
|
Networking |
|
Parser | Tosca2Heat |
Storage |
|
StorPerf | storperf |
NSB | vFW thoughput test case |
1.1.6.3. New Test cases¶
Note
Yardstick Hunter 8.0.0 adds no new test cases.
1.1.7. Version Change¶
1.1.7.1. Module Version Changes¶
This is the seventh tracked release of Yardstick. It is based on following upstream versions:
- OpenStack Rocky
1.1.7.2. Document Version Changes¶
This is the seventh tracked version of the Yardstick framework in OPNFV. It includes the following documentation updates:
- Yardstick User Guide:
- Yardstick Developer Guide
- Yardstick Release Notes for Yardstick: this document
1.1.7.3. Feature additions¶
- Add TRex Baremetal/SA scale up test cases.
- Add possibility to distribute DPDK/vhost-user ports/queues between CPU cores.
- Adding scale up test case for l3fwd OvS-DPDK.
- Add Testcase Prox Standalone SRIOV.
- Add vBNG PPPoE test cases functionality.
- Add IXIA Baremetal scale up testcases.
- Add vIPSEC testcases.
- Add TRex Baremetal/SA scale up test cases.
- Add OpenStack test cases with different framesize and IMIX.
- Add new scenario NSPerf-RFC3511.
- Add VPP traffic profiles for TRex traffic generator.
- Add Trex traffic generator to initate Traffic for VPP IPSEC.
- Add IxNextgen API for setting IP priority, flow tracking options, creating BGP protocol layer.
- Add Pktgen traffic generator for vCMTS.
- Add Trex traffic generator to initate Traffic for VPP IPSEC.
- Add sipp trafficgen based SampleVNFTrafficGen.
- Add ansible scripts to deploy Kubernetes.
- Update NSB report to use Barometer metrics.
- Support using existing private key when using external heat template file.
1.1.8. Scenario Matrix¶
1.1.9. Test results¶
Test results are available in:
- jenkins logs on CI: https://build.opnfv.org/ci/view/yardstick/
1.1.9.1. Known Issues/Faults¶
1.1.9.2. Corrected Faults¶
1.1.10. Hunter 8.0.0 known restrictions/issues¶
1.1.11. Useful links¶
- wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick
- wiki Yardstick Hunter release planning page: https://wiki.opnfv.org/display/yardstick/Release+Hunter
- Yardstick repo: https://git.opnfv.org/yardstick
- Yardstick CI dashboard: https://build.opnfv.org/ci/view/yardstick
- Yardstick grafana dashboard: http://testresults.opnfv.org/grafana/
- Yardstick IRC channel: #opnfv-yardstick