[rdo-list] [tripleo] Moving mirror server images to tarballs.o.o (Was: RDO-CI Scrum Weekly Scrum Recap)

[Paul Belanger]

On Wed, Feb 08, 2017 at 08:50:10AM -0500, John Trowbridge wrote:
> Moving this thread from rdo-list, but leaving in copy as it is relevant
> there too. However it is more of a TripleO topic.
> 
> On 02/07/2017 03:56 PM, Paul Belanger wrote:
> > On Tue, Feb 07, 2017 at 03:34:09PM -0500, Ronelle Landy wrote:
> >> Greetings All,
> >>
> >> Links to the RDO-CI team scrum[1] and recording[2] from this week's
> >> meeting are below.
> >>
> >> Highlights:
> >>
> >>  - TripleO CI images outage 
> >>   [3] details the images missing from the mirror, causing CI failures
> >>   We need a less 'reactive' approach to checking that the gates are working
> >>   Action Items:
> >>     Attila added [4] to run check jobs and measure passing
> >>     IRC Bot needed to ping #oooq with failures
> > It would be great if we can finally replace this private infrastructure in
> > tripleo-ci with our community openstack-infra servers. This avoids the need for
> > tripleo-ci project to maintain servers.  Also, we likely have more HDD space
> > available.
> > 
> > I recently helped kolla move there images to tarballs.o.o, we can do the same
> > for tripleo-ci.
> > 
> 
> That would be awesome. We want to cover stable release images as well,
> and the current tripleo-ci mirror server seems inadequate for that. Are
> there any docs on how to publish to tarballs.o.o? Right now, the image
> upload happens in two stages. First, all periodic jobs of a particular
> type (HA I think?) upload the images they built to the mirror server.
> Second, when all periodic jobs pass on a particular dlrn hash the
> symlink for the "known good" image is updated.
> 
We basically create a publisher in JJB to handle this, which looks in the
$WORKSPACE/images folder for things to scp to tarballs.o.o.  Take a look at the
kolla job for example.

What we can do, is create an experimental job first test the uploads to
tarballs.o.o, once working, we can then update the jobs you have listed.

> It seems like to start we could maybe update this symlink swapping logic
> to instead publish the image on tarballs.o.o, and use that location as
> the known good. That would still result in using the current mirror
> server as a store of images under test, but it would give the gates
> which consume the known good images (and users/devs who do that) much
> more stability.
> 
> Maybe it would be possible to use tarballs.o.o for the testing images as
> well, but then we would need some sort of cleanup and some way to mark
> which images there are actually good (which seems a bit more complicated).
> 
> >>  -  Upgrade CI status
> >>   There is a periodic job running on internal jenkins [5] testing upgrades
> >>   [6] adds the sanity checks so that upgrade jobs will be able to check services without a full overcloud deployment
> >>  - Ocata Pipeline status
> >>   Phase 1 passed on Friday - but still some reviews outstanding (image build, release configs)
> >>   Phase 2 - jobs have been set up internally
> >>     All baremetal jobs are failing due to JJB mismatches (master vs rdo_trunk for ocata)
> >>  - DCI team is interested in using quickstart-extras
> >>   RDO-CI team to support this usage 
> >>   
> >>
> >> [1] - https://review.rdoproject.org/etherpad/p/rdo-infra-scrum
> >> [2] - https://bluejeans.com/s/ugrW6/
> >> [3] - https://bugs.launchpad.net/tripleo/+bug/1661656
> >> [4] - https://review.openstack.org/430277
> >> [5] - <internal jenkins>/RDO/view/openstack_virtual_baremetal/job/tripleo-quickstart-newton-to-master-upgrade-ovb-qeos7-rhos-dev-ci-multi-nic/
> >> [6] - https://review.openstack.org/#/c/429716/
> >>
> >> /R
> >>
> >> Ronelle Landy
> >>
> >> _______________________________________________
> >> rdo-list mailing list
> >> rdo-list at redhat.com
> >> https://www.redhat.com/mailman/listinfo/rdo-list
> >>
> >> To unsubscribe: rdo-list-unsubscribe at redhat.com
> > 
> > _______________________________________________
> > rdo-list mailing list
> > rdo-list at redhat.com
> > https://www.redhat.com/mailman/listinfo/rdo-list
> > 
> > To unsubscribe: rdo-list-unsubscribe at redhat.com
> >