12:28 p.m.
----- Original Message -----
On Fri, Mar 22, 2019 at 8:35 PM Javier Pena < jpena(a)redhat.com
> wrote:
> ----- Original Message -----
> > I've been working with mjturek and baha on this a bit. I've responded
> > inline
> > below, but also want to clarify on the desired workflow.
> >
> > TL;DR: The desired workflow is to have ppc64le and x86_64 seamlessly
> > integrated and uploaded. This can be done with docker manifest list
> > images.
> >
> > The following link explains in greater detail:
> > https://docs.docker.com/registry/spec/manifest-v2-2/
> >
> > The process boils down to the following steps:
> >
> > 1) Upload an image of the first architecture (ex: image1:x86_64_01012019)
> > 2) Upload an image of the second architecture (ex:
> > image1:ppc64le_01012019)
> > 3) Upload manifest list image of the image (ex: image1:01012019)
> >
> This is one of the details where I had my doubts. Currently, the
images
> uploaded to the registry use the following naming convention:
>
tripleomaster/centos-binary-neutron-l3-agent:42a882962919b867c91a182b83acca6d8004096e_ee467b40
> Where:
> - tripleomaster is associated to the release (we have
tripleomaster,
> tripleostein, tripleorocky...)
> - centos is associated to the OS (we have centos and fedora)
> - 42a882962919b867c91a182b83acca6d8004096e_ee467b40 refers to the
> repository
> in trunk.rdoproject.org used to build the image (commit hash and short
> distro hash)
> If we want to go multi-arch, we need to change that tag to
include the
> architecture, is this correct? Otherwise, we could have conflicts between
> the x86_64 and ppc64le pipelines trying to upload the same image.
Yup. The idea is that the enpoint URL
(tripleomaster/centos-binary-neutron-l3-agent:42a882962919b867c91a182b83acca6d8004096e_ee467b40)
is a container manifest. Where we include the arch would be with an
additional tag:
tripleomaster/centos-binary-neutron-l3-agent:42a882962919b867c91a182b83acca6d8004096e_ee467b40_$arch
but nothing else should change and *explicitly* do not want different orgs
per architecture. So the publish pipeline would look like:
* Each architecture builds and publishes all the containers per
branch and OS
[1] all the containers and publishes a container image/layer to:
'tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s_%(arch)s'
* Then checks to see if the manifest exists. manifest =
'tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s' if
exists(manifest):
add_to_manifest(arch_layer=tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s_%(arch)s')
else:
create_manifest(arch_layer=tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s_%(arch)s')
I have been running some tests to check how this could work, and I've found an issue.
It looks like the OpenShift Registry (what we use for our RDO Registry) does not properly
support manifest lists, see [2]. It actually failed for me when I tried it, while a plain
Docker registry worked (using manifest-tool).
Would the manifest upload be required in the RDO Registry (which is used as an
intermediate step), or just in DockerHub (which is used for actual content delivery)? If
it's the second case, we're still fine.
Regards,
Javier
[2] - https://trello.com/c/4EcAIJrd/1303-5-quayregistry-add-support-for-manifes...
This shouldn't break existing consumers as docker and podman both
do the
correct thing when encountering a manifest. and does mean that multi-arch
consumers can use the same URL scheme. This is how downstream currently
works
It's possible and possibly even desirable, due to resource
constraints, for
the ppc64le build to be triggered only when updating current-passed-ci.
That's exactly what we discussed in Dublin.
Tony.
[1] for ppc64le we're starting with centos and master but over
time this
would need to grow out from master to include stein, u etc etc We haven't
looked at Fedora due to using centos CI but if Fedora is going to stick
around we can work on that too.