[rdo-dev] [infra][outage] trunk.rdoproject.org down

Javier Pena jpena at redhat.com
Tue Sep 3 08:25:17 UTC 2019


> > Hi all,
> > 
> > Due to an issue in the cloud provider, trunk.rdoproject.org has been down
> > during the weekend.
> > 
> > While we are still trying to recover it, we have switched to our secondary
> > server inside RDO Cloud. Depending on the DNS propagation time, it may
> > still
> > take a few minutes to be visible, but normal operations will be resumed
> > soon.
> > 
> > We will send another email as soon as everything is back to normal. Please
> > do
> > not hesitate to contact me if you need additional information.
> > 
> 
> Hi,
> 
> Apparently, the volumes where the RDO Trunk data was stored in
> trunk.rdoproject.org are not recoverable. Thus, we have set up a new
> trunk.rdo machine, and will follow this procedure to recover data:
> 
> - We have stopped new builds on RDO Trunk to sync the data to the new
> machine.
> - Tomorrow morning, once all data has been synced, we will switch the
> trunk.rdo DNS entry to the new machine and resume normal operations.
> 
> All repositories will remain accessible during the process, with only a brief
> outage expected tomorrow while the DNS is propagating.
> 

Hi,

The DNS entries have been updated. I am currently monitoring DNS propagation, please be aware that there may be some flakiness until it is completed. For example, some Google DNS servers are currently showing the new IP, while some others are not.

Once DNS propagation is stabilized, we will resume normal operations.

Regards,
Javier

> Regards,
> Javier
> 
> 
> > Regards,
> > Javier
> > _______________________________________________
> > dev mailing list
> > dev at lists.rdoproject.org
> > http://lists.rdoproject.org/mailman/listinfo/dev
> > 
> > To unsubscribe: dev-unsubscribe at lists.rdoproject.org
> > 
> 


More information about the dev mailing list