[Rdo-list] Creating rpms for Tempest

[Lon Hohberger]

On 02/03/2014 03:55 PM, David Kranz wrote:
> On 02/03/2014 02:27 PM, Lon Hohberger wrote:
>> On 02/03/2014 10:50 AM, David Kranz wrote:
>>> There has been a lot of interest in running tempest against real RDO
>>> clusters. Having an rpm would make that a lot easier. There are several
>>> issues peculiar to tempest that need to be resolved.
>>>
>>> 1. The way tempest configures itself and does test discovery depends
>>> heavily on the tests being run from a directory containing the tests.
>>> 2. Unlike OpenStack python client libraries, tempest changes from
>>> release to release in incompatible ways, so you need "havana tempest" to
>>> test a havana cluster and an "icehouse tempest" to test icehouse.
>>>
>>> The tempest group has little interest in changing either of these
>>> behaviors. Additionally, it would be desirable if a tempest user could
>>> install tempest rpms to test different RDO versions on the same machine.
>>> Here is a proposal for how this could work and what the user experience
>>> would be.
>>>
>>> Dealing with these tempest issues suggests that the the tempest code
>>> should be copied to /var/lib/tempest/{4.0,5.0, etc.} and the user should
>>> configure a separate directory for each OpenStack cluster to be tested.
>>> Each directory needs to contain:
>>>
>>> .testr.conf
>>> an etc directory containing the tempest.conf and logging.conf files
>>> a symlink to the tempest test modules for the appropriate version
>>> a copy of the test run scripts that are in the tools directory of
>>> tempest
>>>
>>> To help the user create such directories, there should be a global
>>> executable "configure-tempest-directory" that takes an optional version.
>>> If multiple versions are present in /var/lib/tempest and no version is
>>> specified then the user will be asked which version to configure.
>> Would it make sense to make the package name itself to have it?
>>
>> This way, you could say install tempest for grizzly and update tempest
>> for havana - e.g.:
>>
>>     yum install tempest-grizzly
>>     yum update -y tempest-havana
>>
>> ... and the RPMs would not obsolete each other.  If we use RPM
>> versioning, "yum update" will blow away testing environments of "older"
>> tempest versions.
>>
>> We could perhaps do it using sub-rpms - which would allow us to add and
>> remove tempest suites as we move along:
>>
>>    * ship no files except perhaps license in 'tempest' itself
>>      and stand-up environment bits
>>    * use subpackages for
>> tempest-grizzly/tempest-havana/tempest-icehouse/...
>>    * get a special exception to *not* make tempest-* child
>>      RPMs require a fully-versioned tempest base package all
>>      the time just in case you wanted to have older tempest
>>      for a given release
>>        e.g. tempest-2.0.0
>>             tempest-grizzly-1.0.0
>>             tempest-havana-2.0.0
>>
>>      ... could all be installed, and if done right, you could
>>      'yum update -y tempest-grizzly' to 3.0.0 without breaking
>>      the tempest-havana-2.0.0 package.
>>
>> Just a thought.
>>
>>> User experience:
>>>
>>> 1. Install tempest rpm: yum install tempest-4.0
>> The above would mean:
>>
>> yum install -y tempest-havana
>> yum install -y tempest-grizzly
>>
>> etc.
>>
>>
>>> 2. Run configure-tempest-directory
>>> 3. Make changes to tempest.conf to match the cluster being tested (and
>>> possibly logging.conf and .testr.conf as well)
>>> 4. Run tempest with desired test selection using
>>> tools/pretty_tox_serial.sh or tools/ pretty_tox_serial
>>>
>>> Does any one have any comments/suggestions about this?
>>>
>> -- Lon
>>
> Lon, I was not clear enough but I was proposing something very similar
> to your suggestion, just without subpackages which I don't understand
> well. Now that I understand the rules a little better and that the base
> package names should not have things that look like version numbers, I
> propose:
> 
> package-names: tempest-havana.{version stuff}, tempest-icehouse.{version
> stuff}
> 
> And a tempest-base package on which they all depend that creates
> "/var/lib/tempest" and the configure-tempest-directory script.
> 
>  -David

Sorry for not responding on-list (instead of just in-person) - that
sounds great.

-- Lon