[Rdo-list] Improving RDO continuous integration/testing

[Haïkel]

2015-09-15 20:37 GMT+02:00 David Moreau Simard <dms at redhat.com>:
> Hi,
>
> Continuous integration jobs for RDO trunk (liberty) are not in good
> shape right now [1] and they also provide poor coverage.
> The only test that is run is tempest.scenario.test_server_basic_ops
> [2] which adds a ssh-keypair.
>
> As part of our efforts to speed up the release process of RDO and
> improve the quality and stability of what we ship, I am working to
> improve the CI.
> The good news is that I've managed to get promising results with
> khaleesi+rdo+packstack+aio with selinux permissive locally (selinux
> enforced being blocked right now [3]) so we are on the right track.
>

(CC'ing Emilien with whom we worked on integrating RDO into Puppet
Modules upstream CI)

Thanks for dealing with our CI, as we have a more robust continuous
packaging machinery with delorean, this will be the next hot topic.

We need more complete coverage for RDO as it will help detecting
integration issues on Fedora/EL and fix them earlier.
This is an important step to make Fedora/EL first-class citizens
upstream, and encourage upstream maintainers to develop on these
platforms.

> As I've been trying to improve test coverage, a good first step would
> be to enforce tempest smoke tests.
> However, I've noticed that khaleesi uses a fork of tempest [4] and
> this generated a failure [5] on a test that has since been fixed
> upstream [6].
>
> I'm very concerned about testing trunk RDO against a fork of tempest.
> We should be testing trunk RDO against trunk tempest.
> Running against a fork means we might lack some important changes to
> test coverage or can unnecessarily encounter failures which have
> already been resolved upstream.
>
> My understanding is that Red Hat maintains a fork of tempest to run
> test suites against products which have a longer release and support
> cycles and that is fine.
> Should we switch RDO CI testing to the upstream branches ?
>
> Thanks,
>

Yes, we should definitively set this as a goal, at the very least, we
should have running it in parallel and work on fixing issues.\

Please update trello accordingly so we could track this effort.

Regards,
H.


> [1]: https://prod-rdojenkins.rhcloud.com/view/RDO-Liberty-Delorean-Trunk/
> [2]: https://github.com/openstack/tempest/blob/master/tempest/scenario/test_server_basic_ops.py
> [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1249685
> [4]: https://github.com/redhat-openstack/khaleesi-settings/blob/master/settings/tester/tempest/setup/git.yml#L4
> [5]: http://paste.openstack.org/show/463360/
> [6]: https://github.com/openstack/tempest/commit/986b9e6fda6c76806a329bd53f5f73f557da903e#diff-06c9ec2d3ed36f96bfce6f75e82a4f45
>
> David Moreau Simard
> Senior Software Engineer | Openstack RDO
>
> dmsimard = [irc, github, twitter]
>
> _______________________________________________
> Rdo-list mailing list
> Rdo-list at redhat.com
> https://www.redhat.com/mailman/listinfo/rdo-list
>
> To unsubscribe: rdo-list-unsubscribe at redhat.com