[rdo-dev] [infra][tripleoci] ppc64le container images in registry.rdoproject.org

Tue Apr 2 20:50:24 UTC 2019

rebooting this conversation...  see inline at the bottom

On Mon, Mar 25, 2019 at 11:29 AM Javier Pena <jpena at redhat.com> wrote:

>
>
> ------------------------------
>
> On Fri, Mar 22, 2019 at 8:35 PM Javier Pena <jpena at redhat.com> wrote:
>
>>
>>
>> ----- Original Message -----
>> > I've been working with mjturek and baha on this a bit.  I've responded
>> inline
>> > below, but also want to clarify on the desired workflow.
>> >
>> > TL;DR: The desired workflow is to have ppc64le and x86_64 seamlessly
>> > integrated and uploaded.  This can be done with docker manifest list
>> images.
>> >
>> > The following link explains in greater detail:
>> > https://docs.docker.com/registry/spec/manifest-v2-2/
>> >
>> > The process boils down to the following steps:
>> >
>> > 1) Upload an image of the first architecture  (ex:
>> image1:x86_64_01012019)
>> > 2) Upload an image of the second architecture  (ex:
>> image1:ppc64le_01012019)
>> > 3) Upload manifest list image of the image  (ex: image1:01012019)
>> >
>>
>> This is one of the details where I had my doubts. Currently, the images
>> uploaded to the registry use the following naming convention:
>>
>>
>> tripleomaster/centos-binary-neutron-l3-agent:42a882962919b867c91a182b83acca6d8004096e_ee467b40
>>
>>
>> Where:
>>
>> - tripleomaster is associated to the release (we have tripleomaster,
>> tripleostein, tripleorocky...)
>> - centos is associated to the OS (we have centos and fedora)
>> - 42a882962919b867c91a182b83acca6d8004096e_ee467b40 refers to the
>> repository in trunk.rdoproject.org used to build the image (commit hash
>> and short distro hash)
>>
>> If we want to go multi-arch, we need to change that tag to include the
>> architecture, is this correct? Otherwise, we could have conflicts between
>> the x86_64 and ppc64le pipelines trying to upload the same image.
>>
>
> Yup.  The idea is that the enpoint URL
> (tripleomaster/centos-binary-neutron-l3-agent:42a882962919b867c91a182b83acca6d8004096e_ee467b40)
> is a container manifest.  Where we include the arch would be with an
> additional tag:
>
> tripleomaster/centos-binary-neutron-l3-agent:42a882962919b867c91a182b83acca6d8004096e_ee467b40_$arch
> but nothing else should change and *explicitly* do not want different orgs
> per architecture.  So the publish pipeline would look like:
>
>
>    - Each architecture builds and publishes all the containers per branch
>    and OS [1] all the containers and publishes a container image/layer to:
>
>    'tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s_%(arch)s'
>    - Then checks to see if the manifest exists.
>    manifest =
>    'tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s'
>    if exists(manifest):
>
>    add_to_manifest(arch_layer=tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s_%(arch)s')
>    else:
>
>    create_manifest(arch_layer=tripleo%(branch)s/%(os)s-%(build_type)s-%(container)s:%(repo)s_%(arch)s')
>
> I have been running some tests to check how this could work, and I've
> found an issue. It looks like the OpenShift Registry (what we use for our
> RDO Registry) does not properly support manifest lists, see [2]. It
> actually failed for me when I tried it, while a plain Docker registry
> worked (using manifest-tool).
>
> Would the manifest upload be required in the RDO Registry (which is used
> as an intermediate step), or just in DockerHub (which is used for actual
> content delivery)? If it's the second case, we're still fine.
>
> Regards,
> Javier
>
> [2] -
> https://trello.com/c/4EcAIJrd/1303-5-quayregistry-add-support-for-manifest-list-rd
>
> This shouldn't break existing consumers as docker and podman both do the
> correct thing when encountering a manifest.  and does mean that multi-arch
> consumers can use the same URL scheme.  This is how downstream currently
> works
>
> It's possible and possibly even desirable, due to resource constraints,
> for the ppc64le build to be triggered only when updating
> current-passed-ci.  That's exactly what we discussed in Dublin.
>
> Tony.
>
> [1] for ppc64le we're starting with centos and master but over time this
> would need to grow out from master to include stein, u etc etc  We haven't
> looked at Fedora due to using centos CI but if Fedora is going to stick
> around we can work on that too.
>
>
> _______________________________________________
> dev mailing list
> dev at lists.rdoproject.org
> http://lists.rdoproject.org/mailman/listinfo/dev
>
> To unsubscribe: dev-unsubscribe at lists.rdoproject.org

OK... so here we are.
Trevor, myself and Steve hashed out a PPC64LE workflow that works alongside
the x86_64 workflow.
Our notes are here [1] .

Although x86_64 and PPC64LE should try their best to use the same container
build methods and process the workflows are completely independent and can
remain that way.   The convergence occurs on the tripleo-ci promotion
server after reading test results from the dlrn_api ( notes in the etherpad
)

Some additional points.
* Alan Pevec and I both agree getting the PPC hooked into a zuul workflow
would benefit everyone

OK... comment away and feel free to comment / update the etherpad.

[1] https://etherpad.openstack.org/p/x86_ppc_tripleo_container_promotion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20190402/450e6f0c/attachment.html>