[rdo-dev] [infra][tripleoci] ppc64le container images in registry.rdoproject.org

Fri Mar 22 09:35:55 UTC 2019

----- Original Message -----
> I've been working with mjturek and baha on this a bit.  I've responded inline
> below, but also want to clarify on the desired workflow.
> 
> TL;DR: The desired workflow is to have ppc64le and x86_64 seamlessly
> integrated and uploaded.  This can be done with docker manifest list images.
> 
> The following link explains in greater detail:
> https://docs.docker.com/registry/spec/manifest-v2-2/
> 
> The process boils down to the following steps:
> 
> 1) Upload an image of the first architecture  (ex: image1:x86_64_01012019)
> 2) Upload an image of the second architecture  (ex: image1:ppc64le_01012019)
> 3) Upload manifest list image of the image  (ex: image1:01012019)
> 

This is one of the details where I had my doubts. Currently, the images uploaded to the registry use the following naming convention:

tripleomaster/centos-binary-neutron-l3-agent:42a882962919b867c91a182b83acca6d8004096e_ee467b40

Where:

- tripleomaster is associated to the release (we have tripleomaster, tripleostein, tripleorocky...)
- centos is associated to the OS (we have centos and fedora)
- 42a882962919b867c91a182b83acca6d8004096e_ee467b40 refers to the repository in trunk.rdoproject.org used to build the image (commit hash and short distro hash)

If we want to go multi-arch, we need to change that tag to include the architecture, is this correct? Otherwise, we could have conflicts between the x86_64 and ppc64le pipelines trying to upload the same image.

Regards,
Javier

> Step 3 is essentially just pushing a JSON body that has descriptors and
> references to the other two images, such that when someone does a pull
> request of the manifest list image, it will gather the appropriate
> architecture for that image based on the host's architecture.
> 
> -Trevor
> 
> PS. If I've missed something important with the overall concerns here I
> apologize, but thought it necessary to spell out the goal as I understand
> it.
> 
> > On Mar 21, 2019, at 12:28 PM, Javier Pena <jpena at redhat.com> wrote:
> > 
> > 
> > ----- Original Message -----
> >> Hi all,
> >> 
> >> Over the last few weeks, mjturek and baha have been busy working on a set
> >> of
> >> periodic jobs to build TripleO images for the ppc64le arch [1].
> >> 
> >> The current missing step is publishing those images, and they are
> >> proposing
> >> to push those to the RDO Registry instance at registry.rdoproject.org,
> >> just
> >> like we do with our TripleO images. I have tried to understand the
> >> requirements, and would like to get input on the following topics:
> >> 
> >> - Which namespace would these images use? Based on some logs [2] it looks
> >> like they use tripleomaster-ppc64le, will they also push the images to
> >> that
> >> namespace?
> 
> I have no experience in namespaces inside of a registry or how that
> differentiates images from one another, but the images should be pushed (in
> my opinion) to the same location in which the x86 images reside.
> 
> >> - Could this create any conflicts with our current promotion pipeline?
> 
> This should not cause conflicts in current promotion pipeline, as the process
> should be an extension to existing functionality.
> 
> >> - Is registry.rdo the right place for those images? I'm not familiar with
> >> the
> >> next steps for ppc64le images after that (will it then go through a
> >> promotion pipeline?), so that might affect.
> 
> If the x86 images exist in registry.rdo, then the ppc64le (and any other
> architecture image) should exist there as well.  I can't think of a reason
> to differentiate between architectures when the desired result is parity and
> support of more architectures.
> 
> >> 
> >> If we decide to upload the images to images.rdo, we'll need to do the
> > 
> > Correction: this should read "registry.rdo"
> > 
> >> following:
> >> 
> >> - Create the tripleomaster-ppc64le namespace in registry.rdo, following a
> >> similar pattern to [3].
> >> - Schedule a short registry downtime to increase its disk space, since it
> >> is
> >> currently near its limit.
> 
> This is definitely necessary, given the capacity requirement will double,
> give or take, to support the additional architecture.
> 
> >> - Update the job at ci.centos to include the REGISTRY_PASSWORD environment
> >> variable with the right token (see [4]). This is missing today, and
> >> causing
> >> the job failure.
> >> 
> >> Once we get input from all interested parties, we will decide on the next
> >> steps.
> >> 
> >> Thanks,
> >> Javier
> >> 
> >> 
> >> [1] -
> >> https://ci.centos.org/job/tripleo-upstream-containers-build-master-ppc64le/
> >> [2] -
> >> https://centos.logs.rdoproject.org/tripleo-upstream-containers-build-master-ppc64le/422/logs/logs/000_FAILED_tripleoclient.log
> >> [3] - https://review.rdoproject.org/r/19063
> >> [4] -
> >> https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-ci-periodic-base/containers-build.yaml#L12-L20
> >> _______________________________________________
> >> dev mailing list
> >> dev at lists.rdoproject.org
> >> http://lists.rdoproject.org/mailman/listinfo/dev
> >> 
> >> To unsubscribe: dev-unsubscribe at lists.rdoproject.org
> >> 
> > _______________________________________________
> > dev mailing list
> > dev at lists.rdoproject.org
> > http://lists.rdoproject.org/mailman/listinfo/dev
> > 
> > To unsubscribe: dev-unsubscribe at lists.rdoproject.org
> 
> _______________________________________________
> dev mailing list
> dev at lists.rdoproject.org
> http://lists.rdoproject.org/mailman/listinfo/dev
> 
> To unsubscribe: dev-unsubscribe at lists.rdoproject.org
> 