C&I WIki Portal/OCP Checkbox
OCP Checkbox
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).
Origins
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time.
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.
About the Tools
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:
- plainbox-provider-opencompute-certification
- Provides the whitelists and test scripts, job definitions and other data for testing
- checkbox-ng
- The underlying test harness based on Plainbox
This will change slightly as development progresses.
Developer level documentation can be found here:
Keep in mind that the documentation is developer focused, not user focused, and is also actively developed and thus could change in the future.
How to Participate
There are a number of ways you can participate in making this tool better for OCP.
Join the dev team
There is a team set up on Launchpad for people who wish to contribute:
We hang out on Freenode IRC in the #opencompute-ci channel
Code!
You can always submit code to the project.
- The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox
- It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/
Coding can be anything from fixing bugs (if any) to assisting with the plainbox port to writing new test scripts and jobs. The latter is most important as we want to ensure that the OCP Ready and OCP Certified specs are thoroughly tested and test cases are represented in the test tools.
Docs
We can always use new documentation, be it user guides, development guides, wiki pages, etc. Currently, there is limited documentation available, and this is more akin to generic documentation and development documentation. We could really use user guides from an OCP point of view that explains how to perform testing, how to install the tools, how to USE the tools as well as documentation on how to write test cases and mapping existing test cases to OCP Ready and OCP Certified test cases in the specs.
Additionally, documentation on building alternate packaging (RPM) and installing/running the tools on OSs other than Ubuntu Server would be useful.
Testing
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases. It is packaged in .deb only so any additional formats will need to be created (.rpm, for instance).
Bug Reporting
Please feel free to file bugs on problems encountered or requests for enhancements:
Questions?
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.