Re: [Rdo-list] [RDO-Manager] Solving "nova is locked" errors from Ironic

On 06/23/2015 01:26 PM, Dmitry Tantsur wrote: > Hi all, > > So, Ironic task manager is attacking us again: > https://bugzilla.redhat.com/show_bug.cgi?id=1233452 > > Previously we already had > https://bugzilla.redhat.com/show_bug.cgi?id=1212134 > I've implemented retries upstream in ironicclient and backported to our > packages. Later I had to bump the retry timeout in instack-undercloud to > 1 minute: > https://github.com/rdo-management/instack-undercloud/blob/master/scripts/... > > > > Now we have the same problem in another place and I wonder how to fix > it. I have 2 obvious idea: > 1. Patch ironicclient to have longer default timeout (2 mins?) > 2. Update stackrc to carry longer IRONIC_MAX_RETRIES and > IRONIC_RETRY_INTERVAL > > I'd prefer the latter, as it does not touch ironicclient package, only > undercloud installation tool. It can also be changed more easily in run > time. WDYT? Hi Dmitry, These env variables work with the unified CLI too, not just with the instack script, right? I assume the answer is yes (as it seems the script is using the CLI), so #2 seems to be a good option. However, I was wondering whether others will also hit this issue outside RDO, which would make #1 a better option. Considering that, we can try to fix it in ironicclient and if there's any push-back, we can still go with updating stackrc. Also, as an improvement to #1, can we make the timeout configurable in ironicclient eg. by passing a --timeout flag?

Imre > > I wonder what the root cause is as well. I suspect some very slow BMC's > take too much time to do power actions or power syncs. It's possible > we'll make a wrong guess, and the problem will persist despite becoming > rare. > > Thanks, > Dmitry > > _______________________________________________ > Rdo-list mailing list > Rdo-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > > To unsubscribe: rdo-list-unsubscribe(a)redhat.com