Re: [Rdo-list] [RDO-Manager] Solving "nova is locked" errors from Ironic
On 06/23/2015 01:26 PM, Dmitry Tantsur wrote:
Hi all,
So, Ironic task manager is attacking us again:
https://bugzilla.redhat.com/show_bug.cgi?id=1233452
Previously we already had
https://bugzilla.redhat.com/show_bug.cgi?id=1212134
I've implemented retries upstream in ironicclient and backported to our
packages. Later I had to bump the retry timeout in instack-undercloud to
1 minute:
https://github.com/rdo-management/instack-undercloud/blob/master/scripts/...
Now we have the same problem in another place and I wonder how to fix
it. I have 2 obvious idea:
1. Patch ironicclient to have longer default timeout (2 mins?)
2. Update stackrc to carry longer IRONIC_MAX_RETRIES and
IRONIC_RETRY_INTERVAL
I'd prefer the latter, as it does not touch ironicclient package, only
undercloud installation tool. It can also be changed more easily in run
time. WDYT?
Hi Dmitry,
These env variables work with the unified CLI too, not just with the
instack script, right? I assume the answer is yes (as it seems the
script is using the CLI), so #2 seems to be a good option. However, I
was wondering whether others will also hit this issue outside RDO, which
would make #1 a better option. Considering that, we can try to fix it in
ironicclient and if there's any push-back, we can still go with updating
stackrc.
Also, as an improvement to #1, can we make the timeout configurable in
ironicclient eg. by passing a --timeout flag?
Imre
I wonder what the root cause is as well. I suspect some very slow BMC's
take too much time to do power actions or power syncs. It's possible
we'll make a wrong guess, and the problem will persist despite becoming
rare.
Thanks,
Dmitry
_______________________________________________
Rdo-list mailing list
Rdo-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/rdo-list
To unsubscribe: rdo-list-unsubscribe(a)redhat.com