<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Hi,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">We incurred a loss of data on the RDO registry today around 4PM UTC during the SSL certificate renewal for the different domains and services used for the registry.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Around 10PM UTC, everything should be back to normal.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Instead of re-building the lost images in-place, we expedited the upgrade of our registry to OpenShift 3.7.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Our implementation of this brand new release contains, amongst other things, a fix to prevent this issue from occurring again.</div><div><div class="gmail_signature"><br></div><div class="gmail_signature"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">We built and pushed the following images to recover from what we had:</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">- master/tripleo-ci-testing: <a href="https://trunk.rdoproject.org/centos7-master/df/24/df24d6e51a178263940fa929d52276f3697f75bd_3b7cc045">https://trunk.rdoproject.org/centos7-master/df/24/df24d6e51a178263940fa929d52276f3697f75bd_3b7cc045</a></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">- master/current-tripleo-rdo: <a href="https://trunk.rdoproject.org/centos7-master/3b/85/3b85715c6488a692fcdf4e4c6a589d77c60eb5a6_a3792c8d">https://trunk.rdoproject.org/centos7-master/3b/85/3b85715c6488a692fcdf4e4c6a589d77c60eb5a6_a3792c8d</a></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">- pike/tripleo-ci-testing: <a href="https://trunk.rdoproject.org/centos7-pike/16/ee/16ee91b0fdf092037bd8cd788ffcd6476367ff08_940a986d">https://trunk.rdoproject.org/centos7-pike/16/ee/16ee91b0fdf092037bd8cd788ffcd6476367ff08_940a986d</a></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">- pike/current-tripleo-rdo: <a href="https://trunk.rdoproject.org/centos7-pike/ca/08/ca0834685d9bf226816e49b195251bd553fe5d9f_c74c332d">https://trunk.rdoproject.org/centos7-pike/ca/08/ca0834685d9bf226816e49b195251bd553fe5d9f_c74c332d</a></div></div><div class="gmail_signature"><br></div><div class="gmail_signature"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">​The last images builds and tags were pushed around 10PM UTC.</div></div><div class="gmail_signature"><br></div><div class="gmail_signature"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Note that we disabled Jenkins in order to prevent the periodic promotion jobs from interfering or failing outright.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">For this reason, if you see errors in your patches with 'NOT_REGISTERED', feel free to do a recheck.</div></div><div class="gmail_signature"><br></div><div class="gmail_signature"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">​Thanks,​</div></div><div class="gmail_signature"><br>David Moreau Simard<br>Senior Software Engineer | OpenStack RDO<br><br>dmsimard = [irc, github, twitter]</div></div>
</div>