Before going down to my specific context, I would like to raise that we're having more and more dependencies outside of OpenStack (Ceph, MariaDB, Ansible, OVS, etc); and we know that all of them can easily "break" projects like TripleO and therefore cause problems in the production chain. I would like to understand if we have a common path for these projects, to include them in delorean and in our machinery so we can easily test them in RDO CI (and stop manual testing like we are doing now).

We all know that we can't and don't want to consume EPEL but for some projects it's going to be problematic, I'll discuss about one of them a bit later in this email. My proposal is the following: - Identify which projects we have had troubles in the past months and that we are not automatically testing when new versions bump up (OVS? Ceph? etc)

automatically bump them at each new tag, with proper CI job in place or 2) have a repo that is nightly build from imports on latest tags available from their upstream repos), and have periodic jobs that test these bits.

What I like with 1) is that we can easily test them independently (e.g. a new version of OVS) versus all together in 2) (a new version of OVS + a new version of MariaDB, etc).

Which leads to my second question... how are going to ship Ansible. I talked with Sam Doran (in cc) today and it seems like in the near future Ansible would be shipped via releases.ansible.com or EPE (and not from CentOS Extras anymore). So here are the questions: 1) How are we going to test new versions of Ansible in RDO CI context? 2) Where should we ship it? Back in my proposal, would it make sense to import Ansible in RDO and gate each bump (proposal #1) or take upstream, put in a deps repo (as we have current I believe) and test it with a periodic jobs among other new deps.

Discussion is open, thanks for participating, -- Emilien Macchi _______________________________________________ dev mailing list dev(a)lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/dev To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org

> - Identify which projects we have had troubles in the past months and that > we are not automatically testing when new versions bump up (OVS? Ceph? etc) > - For these projects, how could we either 1) import them in delorean and Pretty please s/delorean/RDO Trunk/ DLRN is a tool, RDO Trunk are the repos built by it. (see Alfredo's blog series starting with [1] for the introduction to RDO repos) > automatically bump them at each new tag, with proper CI job in place or 2) > have a repo that is nightly build from imports on latest tags available from > their upstream repos), and have periodic jobs that test these bits. Upstream trunk chasing requires involvement from the people working closely in that upstream project, and reality is that e.g. even for OpenStack projects FTBFs in RDO Trunk are fixed by CI and infra teams most of the time. Until we have clear buy-in from subject-matter-experts, we cannot add a project directly to the trunk chase. There could a compromise, read on. > What I like with 1) is that we can easily test them independently (e.g. a > new version of OVS) versus all together in 2) (a new version of OVS + a new > version of MariaDB, etc). For OVS specifically, Javier and I looked it and opened a story for DLRN support [2] tl;dr project would be in rdoinfo but pinned and every pin update would be gated by rdoinfo CI Open questions are which CI jobs give good enough coverage while not too expense (hint: running full TripleO CI is NOT answer). This would avoid limit amount of racing against upstream master as pin updates would be proposed by the interested person who would then own it until it works and passes gate. Note this is all for the OpenStack/RDO release in _development_ i.e. master CI, for stable releases we would use stable release of dependencies, ideally built and released in the CentOS SIG repos i.e. Ceph from Storage SIG (this is the case now), OVS from NFV SIG (not the case now), Ansible from ConfigManagement SIG (not the case now) ... > Which leads to my second question... how are going to ship Ansible. > I talked with Sam Doran (in cc) today and it seems like in the near future > Ansible would be shipped via releases.ansible.com or EPE (and not from CentOS Extras anymore). Ansible RPMs are already there http://releases.ansible.com/ansible/rpm/release/epel-7-x86_64/ but they depend on EPEL for additional deps. To make it work in CentOS SIG ecosystem, we need them rebuilt and released in the appropriate SIG. > So here are the questions: > > 1) How are we going to test new versions of Ansible in RDO CI context? > 2) Where should we ship it? Back in my proposal, would it make sense to > import Ansible in RDO and gate each bump (proposal #1) or take upstream, put > in a deps repo (as we have current I believe) and test it with a periodic jobs among other new deps. Ansible is not in deps repo, we are getting it from Extras. NB there are multiple Ansibles in the CI context: Zuul/softwarefactory is pinned to an older version and pypi installed and the same is tripleo-quickstart (ideally those would use RPM Ansible but that's a separate discussion). Ansible RPM is only used by tripleo itself and it is currently coming from Extras where latest version is 2.4.2 [3] and I saw requests to get newer. For master we could try the same plan as OVS above, for stable releases I'd ask Ansible team to get involved with ConfigManagement SIG.

BTW ideal approach would be to insert OpenStack use-cases into Ansible upstream CI and make it voting, this could become reality with cross-project CI efforts lead by openstack-infra. With that, Ansible master would never break us!