Re: [rdo-dev] Long queue in RDO SF
Hi,
I see no issues in nodepool. Looking at the current Zuul queue, we have a single job stuck
for ~90 hours, queued on
"gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens".
When that happens, it's usually a configuration issue, and this is the case here: we
have no definition for the queens gate job for featureset035 in
https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/job....
An easy way to troubleshoot this is:
- If we find one or more jobs queued, first check at
https://review.rdoproject.org/jenkins/ and see if there are nodes available to jenkins.
- If there are, just check the list of jobs available to Jenkins. If it's not there,
we need to double-check the jjb configuration and find what is missing.
My only doubt is why this does not show up as "NOT_REGISTERED" in Zuul as it did
before.
I have proposed https://review.rdoproject.org/r/12038 as a fix for this.
Regards,
Javier
----- Original Message -----
FWIW no alerts during the weekend and I have been able to spawn 10+
instances without issue.
Cheers
David Manchado
Senior Software Engineer - SysOps Team
Red Hat
dmanchad(a)redhat.com
On 11 February 2018 at 16:39, Paul Belanger <pabelanger(a)redhat.com> wrote:
> On Sun, Feb 11, 2018 at 12:44:52PM +0100, Haïkel Guémar wrote:
>> On 02/11/2018 12:17 AM, Sagi Shnaidman wrote:
>> > Hi,
>> >
>> > I see openstack-check has 53 hours queue when 1 job only is queued:
>> > https://review.rdoproject.org/zuul/
>> >
>> > Seems like problem with nodepool?
>> >
>> > Thanks
>> >
>> > --
>> > Best regards
>> > Sagi Shnaidman
>> >
>> >
>> > _______________________________________________
>> > dev mailing list
>> > dev(a)lists.rdoproject.org
>> > http://lists.rdoproject.org/mailman/listinfo/dev
>> >
>> > To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org
>> >
>>
>> Ok, it looks bad enough that a simple nodepool list fails with that error:
>> os_client_config.exceptions.OpenStackConfigException: Cloud rdo-cloud was
>> not found.
>>
>> Despite RDO Cloud looks up, there might be an outage or incident hence
>> copying David Manchado.
>>
>> Regards,
>> H.
>>
> Okay, I have to run, but this looks like a configuration issue. It is hard
> to
> tell without debug logs for nodepool or zuul, but please double check your
> node
> is setup properly.
>
> I have to run now.
>