[Rdo-list] Improving RDO continuous integration/testing

Hi, Continuous integration jobs for RDO trunk (liberty) are not in good shape right now [1] and they also provide poor coverage. The only test that is run is tempest.scenario.test_server_basic_ops [2] which adds a ssh-keypair. As part of our efforts to speed up the release process of RDO and improve the quality and stability of what we ship, I am working to improve the CI. The good news is that I've managed to get promising results with khaleesi+rdo+packstack+aio with selinux permissive locally (selinux enforced being blocked right now [3]) so we are on the right track. As I've been trying to improve test coverage, a good first step would be to enforce tempest smoke tests. However, I've noticed that khaleesi uses a fork of tempest [4] and this generated a failure [5] on a test that has since been fixed upstream [6]. I'm very concerned about testing trunk RDO against a fork of tempest. We should be testing trunk RDO against trunk tempest. Running against a fork means we might lack some important changes to test coverage or can unnecessarily encounter failures which have already been resolved upstream. My understanding is that Red Hat maintains a fork of tempest to run test suites against products which have a longer release and support cycles and that is fine. Should we switch RDO CI testing to the upstream branches ? Thanks, [1]: https://prod-rdojenkins.rhcloud.com/view/RDO-Liberty-Delorean-Trunk/ [2]: https://github.com/openstack/tempest/blob/master/tempest/scenario/test_se... [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1249685 [4]: https://github.com/redhat-openstack/khaleesi-settings/blob/master/setting... [5]: http://paste.openstack.org/show/463360/ [6]: https://github.com/openstack/tempest/commit/986b9e6fda6c76806a329bd53f5f7... David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter]