[Rdo-list] Creating rpms for Tempest

[David Kranz]

There has been a lot of interest in running tempest against real RDO 
clusters. Having an rpm would make that a lot easier. There are several 
issues peculiar to tempest that need to be resolved.

1. The way tempest configures itself and does test discovery depends 
heavily on the tests being run from a directory containing the tests.
2. Unlike OpenStack python client libraries, tempest changes from 
release to release in incompatible ways, so you need "havana tempest" to 
test a havana cluster and an "icehouse tempest" to test icehouse.

The tempest group has little interest in changing either of these 
behaviors. Additionally, it would be desirable if a tempest user could 
install tempest rpms to test different RDO versions on the same machine. 
Here is a proposal for how this could work and what the user experience 
would be.

Dealing with these tempest issues suggests that the the tempest code 
should be copied to /var/lib/tempest/{4.0,5.0, etc.} and the user should 
configure a separate directory for each OpenStack cluster to be tested. 
Each directory needs to contain:

.testr.conf
an etc directory containing the tempest.conf and logging.conf files
a symlink to the tempest test modules for the appropriate version
a copy of the test run scripts that are in the tools directory of tempest

To help the user create such directories, there should be a global 
executable "configure-tempest-directory" that takes an optional version. 
If multiple versions are present in /var/lib/tempest and no version is 
specified then the user will be asked which version to configure.

User experience:

1. Install tempest rpm: yum install tempest-4.0
2. Run configure-tempest-directory
3. Make changes to tempest.conf to match the cluster being tested (and 
possibly logging.conf and .testr.conf as well)
4. Run tempest with desired test selection using 
tools/pretty_tox_serial.sh or tools/ pretty_tox_serial

Does any one have any comments/suggestions about this?

  -David