Hi everyone, As a follow-up from the conversations that we had on the irc, I wanted to summarize the current issues and possible actions. As we ran out of time during the meeting I'd really appreciate some feedback on this. 1) Tempest Plugins With the current situation (project package and project-test package) we do install the entry point with the main package, even if the sub-package is not installed. In this case, Tempest will discover the test entry point without the code and fail. On this, I'd propose to integrate-back the -tests packages for the in-tree plugins, even if it would mean adding some more dependencies to them and remove tempest requirement as a dependency for out-tree ones (i.e. Designate[2] or tempest-horizon[3]), which will get rid of any circular dependency. This seems to breaks RPM logic. This is only to allow mixing packages and git/pip installations which is a source of troubles. At best, it could be a temporary measure. If it's about circular dependencies, RPM knows how to handle them, and the best way to break them would be plugins requiring tempest and not the reverse. We could have a a subpackage tempest-all that requires all the plugins for an AIO install. (alternatively: tempest requires all plugins, and plugins would require a tempest-core packages. tempest package being empty and tempest-core containing the framework) Also, the plugin state is not really optimal, as some of those plugins would pin to an specific commit of tempest. All of this should be sorted out in the end by [1] but for now we'll need to find some solution on this. 2) Tempest Requirements As tempest is installed in the undercloud, it will use whatever plugins are installed there, so if checking every test from the overcloud is a must it'd need to have a lot of packages installed as dependencies (that's the current status). Some ideas for this have been. - Create a parent-package/metapackage called 'tempest-all-test' or similar, which will allow to install all the sub-componants packages. This would allow not to install all the tempest dependencies but rather only one component at a time. It would Requires: all test packages for anyone who needs to install them all. - Puppet-Tempest: it will install tempest itself (albeit only from source right now, this can be addressed: https://bugs.launchpad.net/puppet-tempest/+bug/1549366) but it will also install tempest plugins based on the parameters that define the availability of services. For example, if "ceilometer_available" is set to true, it will install python-ceilometer-tests and set the config in tempest.conf "[service_available] ceilometer=True" 3) Tempest Config Tool Right now is the only diff between upstream vanilla and our downstream fork, on here, our proposal could be to move it to its own repo and decouple it from tempest so we could use vanilla and not depend on the downstream fork. This will later on also integrated and used on RefStack/DefCore (WIP). There has been quite some discussion around this, and there are duplicated projects doing the same work with some difference (basically to use or not api_discovery) During the past summit and afterwards, we agreed on creating something that would depend on the installer (fetching the configuration from TripleO), as the tool as it is won't be accepted on upstream tempest. This had been dropped dead since, so I'd like to resume the discussion. Thanks for any ideas! Daniel --- [1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101552.html [2] https://review.rdoproject.org/r/#/c/1820/ [3] https://github.com/openstack/tempest-horizon/blob/master/requirements.txt...