[rdo-dev] Strategy on packaging external dependencies in RDO + include Ansible in RDO

[Alan Pevec]

> - Identify which projects we have had troubles in the past months and that
> we are not automatically testing when new versions bump up (OVS? Ceph? etc)
> - For these projects, how could we either 1) import them in delorean and

Pretty please s/delorean/RDO Trunk/
DLRN is a tool, RDO Trunk are the repos built by it.
(see Alfredo's blog series starting with [1] for the introduction to RDO repos)

> automatically bump them at each new tag, with proper CI job in place or 2)
> have a repo that is nightly build from imports on latest tags available from
> their upstream repos), and have periodic jobs that test these bits.

Upstream trunk chasing requires involvement from the people working
closely in that upstream project,
and reality is that e.g. even for OpenStack projects FTBFs in RDO
Trunk are fixed by CI and infra teams most of the time.
Until we have clear buy-in from subject-matter-experts, we cannot add
a project directly to the trunk chase.
There could a compromise, read on.

> What I like with 1) is that we can easily test them independently (e.g. a
> new version of OVS) versus all together in 2) (a new version of OVS + a new
> version of MariaDB, etc).

For OVS specifically, Javier and I looked it and opened a story for
DLRN support [2]
tl;dr project would be in rdoinfo but pinned and every pin update
would be gated by rdoinfo CI
Open questions are which CI jobs give good enough coverage while not
too expense (hint: running full TripleO CI is NOT answer).
This would avoid limit amount of racing against upstream master as pin
updates would be proposed by the interested person who would then own
it until it works and passes gate.

Note this is all for the OpenStack/RDO release in _development_ i.e.
master CI, for stable releases we would use stable release of
dependencies, ideally built and released in the CentOS SIG repos
i.e. Ceph from Storage SIG (this is the case now), OVS from NFV SIG
(not the case now), Ansible from ConfigManagement SIG (not the case
now) ...

> Which leads to my second question... how are going to ship Ansible.
> I talked with Sam Doran (in cc) today and it seems like in the near future
> Ansible would be shipped via releases.ansible.com or EPE (and not from CentOS Extras anymore).

Ansible RPMs are already there
http://releases.ansible.com/ansible/rpm/release/epel-7-x86_64/
but they depend on EPEL for additional deps. To make it work in CentOS
SIG ecosystem, we need them rebuilt and released in the appropriate
SIG.

> So here are the questions:
>
> 1) How are we going to test new versions of Ansible in RDO CI context?
> 2) Where should we ship it? Back in my proposal, would it make sense to
> import Ansible in RDO and gate each bump (proposal #1) or take upstream, put
> in a deps repo (as we have current I believe) and test it with a periodic jobs among other new deps.

Ansible is not in deps repo, we are getting it from Extras.
NB there are multiple Ansibles in the CI context: Zuul/softwarefactory
is pinned to an older version and pypi installed and the same is
tripleo-quickstart (ideally those would use RPM Ansible but that's a
separate discussion).
Ansible RPM is only used by tripleo itself and it is currently coming
from Extras where latest version is 2.4.2 [3] and I saw requests to
get newer.
For master we could try the same plan as OVS above, for stable
releases I'd ask Ansible team to get involved with ConfigManagement
SIG.

BTW ideal approach would be to insert OpenStack use-cases into Ansible
upstream CI and make it voting, this could become reality with
cross-project CI efforts lead by openstack-infra. With that, Ansible
master would never break us!

> Discussion is open, thanks for participating,

Thanks for starting the discussion!

Alan

[1] https://blogs.rdoproject.org/2016/04/new-in-rdo-repos-one-size-doesn-t-fit-all/
[2] https://tree.taiga.io/project/morucci-software-factory/us/1044
[3] https://git.centos.org/log/rpms!ansible.git/c7-extras