[rdo-list] Tempest Packaging in RDO

[Daniel Mellado]

Hi everyone,

As a follow-up from the conversations that we had on the irc, I wanted
to summarize the current issues and possible actions. As we ran out of
time during the meeting I'd really appreciate some feedback on this.

1) Tempest Plugins

With the current situation (project package and project-test package) we
do install the entry point with the main package, even if the
sub-package is not installed. In this case, Tempest will discover the
test entry point without the code and fail.

On this, I'd propose to integrate-back the -tests packages for the
in-tree plugins, even if it would mean adding some more dependencies to
them and remove tempest requirement as a dependency for out-tree ones
(i.e. Designate[2] or tempest-horizon[3]), which will get rid of any
circular dependency.

This seems to breaks RPM logic. This is only to allow mixing packages
and git/pip installations which is a source of troubles. At best, it
could be a temporary measure. If it's about circular dependencies, RPM
knows how to handle them, and the best way to break them would be
plugins requiring tempest and not the reverse.

We could have a a subpackage tempest-all that requires all the plugins
for an AIO install. (alternatively: tempest requires all plugins, and
plugins would require a tempest-core packages. tempest package being
empty and tempest-core containing the framework)

Also, the plugin state is not really optimal, as some of those plugins
would pin to an specific commit of tempest. All of this should be sorted
out in the end by [1] but for now we'll need to find some solution on this.


2) Tempest Requirements

   As tempest is installed in the undercloud, it will use whatever
plugins are installed there, so if checking every test from the
overcloud is a must it'd need to have a lot of packages installed as
dependencies (that's the current status).

Some ideas for this have been.

- Create a parent-package/metapackage called 'tempest-all-test' or
similar, which will allow to install all the sub-componants packages.
This would allow not to install all the tempest dependencies but rather
only one component at a time. It would Requires: all test packages for
anyone who needs to install them all.

- Puppet-Tempest: it will install tempest itself (albeit only from
source right now, this can be addressed:
https://bugs.launchpad.net/puppet-tempest/+bug/1549366) but it will also
install tempest plugins based on the parameters that define the
availability of services. For example, if "ceilometer_available" is set
to true, it will install python-ceilometer-tests and set the config in
tempest.conf "[service_available] ceilometer=True"


3) Tempest Config Tool
    Right now is the only diff between upstream vanilla and our
downstream fork, on here, our proposal could be to move it to its own
repo and decouple it from tempest so we could use vanilla and not depend
on the downstream fork. This will later on also integrated and used on
RefStack/DefCore (WIP).

There has been quite some discussion around this, and there are
duplicated projects doing the same work with some difference (basically
to use or not api_discovery)

During the past summit and afterwards, we agreed on creating something
that would depend on the installer (fetching the configuration from
TripleO), as the tool as it is won't be accepted on upstream tempest.

This had been dropped dead since, so I'd like to resume the discussion.

Thanks for any ideas!

Daniel

---
[1]
http://lists.openstack.org/pipermail/openstack-dev/2016-August/101552.html
[2] https://review.rdoproject.org/r/#/c/1820/
[3]
https://github.com/openstack/tempest-horizon/blob/master/requirements.txt#L11