Editing C&I Wiki Portal/ToolsDevelopmentGuideline (section)

== Project Hierarchy ==
=== opencompute-certification ===
==== What this provides ====
This is the trunk for the Open Compute Project's certification tooling.  It can be found on Launchpad:
https://code.launchpad.net/opencompute/opencompute-testing

This code is used to build the launcher packages:
* opencompute-ready
* opencompute-certification

and the provider package:
* plainbox-provider-opencompute-certification

==== What's a Provider? ====
A Provider is a package that provides the bits necessary for Plainbox/Checkbox-NG to perform testing.  These files include job descriptions, whitelists, test scripts and wrappers, helpers, custom libraries and anything else that you may need.

==== What's a Launcher? ====
A Launcher is a package that does two things.  
# It provides the actual program launcher (the command you issue to start the testing) such as <code>opencompute-certification</code>
# It ensures the lower-level packages and dependencies are installed
=== checkbox ===
==== What this provides ====
This is the main project for Canonical's certification tool.  This provides the back-end and underlying support structure as well as some of the general test scripts and jobs.  The following packages make up our use of Checkbox, in order from top to bottom.

===== plainbox-provider-checkbox =====
This package provides the actual checkbox test tools used by Canonical Certification. Many tests in OCP Certification and OCP Ready are based on the jobs, scripts and data in this provider.

===== plainbox-provider-resource-generic =====
This package provides system resource jobs and libraries.  These are used at run-time to gather system information and provide that data to CheckboxNG for job generation, to the results files for system hardware inventory, and to provide the validation that allows certain jobs to run.

To explain that last statement, one job of this is to determine if certain hardware exists, and if that hardware exists, then the jobs that test that hardware are configured and executed.  For example, there is a USB resource that determines if a given motherboard has a USB controller.  If the USB Controller is detected by the USB Resource, then the USB test jobs will be run.  If the USB Resource does NOT detect a USB controller, then the USB jobs will be skipped.

===== python3-checkbox-support =====
This package is an infrastructure package that provides several python3 support libraries that Checkbox needs to run.

===== checkbox-ng =====
This package is the back-end.  It provides Checkbox-NG, the Plainbox based test harness that replaces the old Checkbox tool.  It provides the internals that organize and run all the various test cases, performs all the parsing and heavy work of Certification Testing.

==== Documentation? ====
For the most part, Documentation for the OCP Tools can be found here on the OCP Certification Wiki.  For Checkbox, documentation can be found:
* http://checkbox.readthedocs.org/en/latest/
* http://plainbox.readthedocs.org/en/latest/

As CheckboxNG and Plainbox are under active development, this documentation is constantly evolving and improving.

==== How can I contribute to this? ====
There are several ways to contribute.  For the most part, you'll likely want to contribute to test scripts, jobs and whitelist changes, so you will mostly be interested in the '''opencompute-testing''' project.

To get started, follow the bits above to branch the project (lp:opencompute/opencompute-testing) and start hacking!

If you'd like to contribute to the Checkbox project, please see the [[#Questions?|Questions?]] section below.

=== Putting it all together ===
So now that you know about the various packages that make up the OCP Certification and OCP Ready test tools, let's install them so you can try them out to see what they do.
==== Installation of Packages ====
First, we need to install the PPAs.  Because this is based on Canonical's actual certification tooling instead of a fork, we need to add two PPAs:

<pre>
sudo apt-add-repository ppa:opencompute-developers/ocp-certification-tools-ppa
sudo apt-add-repository ppa:hardware-certification/public
</pre>

Next, we need to update the apt cache:

<pre>sudo apt update</pre>

And finally, install the packages:

<pre>sudo apt-get install opencompute-certification</pre>

==== Interaction ====
After you have done the above, you will see a long list of packages that need to be installed.  These all provide testing tools, libraries and other things necessary for Certification testing.  The number of packages will vary depending on how you installed the OS onto the SUT, what version is being used, what tool is used (Ready vs Certification) and so forth.  

Hit Y and the installation will begin.

Once installation is complete, to check it out qickly, simply run  the command <code>opencompute-certification</code> or <code>opencompute-ready</code>

To do proper testing, there is some configuration and other things that need to be performed first, but this will get you an idea of how the tool works and what it looks like.