[rdo-users] RDO registry outage

[David Moreau Simard]

Hi,

We incurred a loss of data on the RDO registry today around 4PM UTC during
the SSL certificate renewal for the different domains and services used for
the registry.
Around 10PM UTC, everything should be back to normal.

Instead of re-building the lost images in-place, we expedited the upgrade
of our registry to OpenShift 3.7.
Our implementation of this brand new release contains, amongst other
things, a fix to prevent this issue from occurring again.

We built and pushed the following images to recover from what we had:
- master/tripleo-ci-testing: https://trunk.rdoproject.org/
centos7-master/df/24/df24d6e51a178263940fa929d52276f3697f75bd_3b7cc045
- master/current-tripleo-rdo: https://trunk.rdoproject.org/
centos7-master/3b/85/3b85715c6488a692fcdf4e4c6a589d77c60eb5a6_a3792c8d
- pike/tripleo-ci-testing: https://trunk.rdoproject.org/centos7-pike/16/ee/
16ee91b0fdf092037bd8cd788ffcd6476367ff08_940a986d
- pike/current-tripleo-rdo: https://trunk.rdoproject.org/centos7-pike/ca/08/
ca0834685d9bf226816e49b195251bd553fe5d9f_c74c332d

​The last images builds and tags were pushed around 10PM UTC.

Note that we disabled Jenkins in order to prevent the periodic promotion
jobs from interfering or failing outright.
For this reason, if you see errors in your patches with 'NOT_REGISTERED',
feel free to do a recheck.

​Thanks,​

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/users/attachments/20171112/89f60d03/attachment.html>