10:10 a.m.
Hello,
I am having a problem with my icehouse testbed, which has one
controller, one network node (OVS/VLAN) and ~50 compute nodes.
It runs fine for some time, but then recently, only a small portion of
new instances can start, most of the new instances are stuck in BUILD
forever, without an IP assigned. The scheduler log shows the scheduler
has chosen the compute node to start the instance, but the neutron
server log shows no sign of allocating MAC/IP addresses for it, I
suspect the neutron server didn't get the request for some reason.
Any suggestions on how to debug this?
Thanks,
Xin
10:26 a.m.
On Fri, Jan 23, 2015 at 11:10:35AM -0500, Zhao, Xin wrote:
without an IP assigned. The scheduler log shows the scheduler has
chosen
the compute node to start the instance, but the neutron server log shows no
sign of allocating MAC/IP addresses for it, I suspect the neutron server
didn't get the request for some reason.
Any suggestions on how to debug this?
Are there any relevant errors in the nova-compute log on the target
compute hosts?
--
Lars Kellogg-Stedman <lars(a)redhat.com> | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack | http://blog.oddbit.com/
10:33 a.m.
On 1/23/2015 11:26 AM, Lars Kellogg-Stedman wrote:
On Fri, Jan 23, 2015 at 11:10:35AM -0500, Zhao, Xin wrote:
> without an IP assigned. The scheduler log shows the scheduler has chosen
> the compute node to start the instance, but the neutron server log shows no
> sign of allocating MAC/IP addresses for it, I suspect the neutron server
> didn't get the request for some reason.
>
> Any suggestions on how to debug this?
Are there any relevant errors in the nova-compute log on the target
compute hosts?
Nothing in the compute.log on the compute host. But, I don't have debug
turned on there, I will turn it on and see if there are anything...
Thanks,
Xin
1:49 p.m.
On 01/23/2015 09:33 AM, Zhao, Xin wrote:
On 1/23/2015 11:26 AM, Lars Kellogg-Stedman wrote:
> On Fri, Jan 23, 2015 at 11:10:35AM -0500, Zhao, Xin wrote:
>> without an IP assigned. The scheduler log shows the scheduler has
>> chosen
>> the compute node to start the instance, but the neutron server log
>> shows no
>> sign of allocating MAC/IP addresses for it, I suspect the neutron
>> server
>> didn't get the request for some reason.
>>
>> Any suggestions on how to debug this?
> Are there any relevant errors in the nova-compute log on the target
> compute hosts?
Nothing in the compute.log on the compute host. But, I don't have
debug turned on there, I will turn it on and see if there are anything...
Thanks,
Xin
Are you using qpid or rabbitmq? Does restarting either allow the rest
of the compute services to proceed? (OpenStack will reconnect when you
restart the amqp service).
Regards
-steve
_______________________________________________
Rdo-list mailing list
Rdo-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/rdo-list
To unsubscribe: rdo-list-unsubscribe(a)redhat.com
2:19 p.m.
On 1/23/2015 2:49 PM, Steven Dake wrote:
On 01/23/2015 09:33 AM, Zhao, Xin wrote:
>
> On 1/23/2015 11:26 AM, Lars Kellogg-Stedman wrote:
>> On Fri, Jan 23, 2015 at 11:10:35AM -0500, Zhao, Xin wrote:
>>> without an IP assigned. The scheduler log shows the scheduler has
>>> chosen
>>> the compute node to start the instance, but the neutron server log
>>> shows no
>>> sign of allocating MAC/IP addresses for it, I suspect the neutron
>>> server
>>> didn't get the request for some reason.
>>>
>>> Any suggestions on how to debug this?
>> Are there any relevant errors in the nova-compute log on the target
>> compute hosts?
>
> Nothing in the compute.log on the compute host. But, I don't have
> debug turned on there, I will turn it on and see if there are
> anything...
>
> Thanks,
> Xin
>
Are you using qpid or rabbitmq? Does restarting either allow the rest
of the compute services to proceed? (OpenStack will reconnect when
you restart the amqp service).
Yes I tried restarting rabbitmq, but that doesn't help.
Thanks,
Xin
Regards
-steve
> _______________________________________________
> Rdo-list mailing list
> Rdo-list(a)redhat.com
> https://www.redhat.com/mailman/listinfo/rdo-list
>
> To unsubscribe: rdo-list-unsubscribe(a)redhat.com
3:59 p.m.
Well, as I turn on debug mode and restart services (both nova-compute
and neutron-ovs-agent) on the compute nodes, things get better, so far
all new instances start fine. We will see how long this lasts ...
Thanks,
Xin
On 1/23/2015 3:19 PM, Zhao, Xin wrote:
On 1/23/2015 2:49 PM, Steven Dake wrote:
> On 01/23/2015 09:33 AM, Zhao, Xin wrote:
>>
>> On 1/23/2015 11:26 AM, Lars Kellogg-Stedman wrote:
>>> On Fri, Jan 23, 2015 at 11:10:35AM -0500, Zhao, Xin wrote:
>>>> without an IP assigned. The scheduler log shows the scheduler has
>>>> chosen
>>>> the compute node to start the instance, but the neutron server log
>>>> shows no
>>>> sign of allocating MAC/IP addresses for it, I suspect the neutron
>>>> server
>>>> didn't get the request for some reason.
>>>>
>>>> Any suggestions on how to debug this?
>>> Are there any relevant errors in the nova-compute log on the target
>>> compute hosts?
>>
>> Nothing in the compute.log on the compute host. But, I don't have
>> debug turned on there, I will turn it on and see if there are
>> anything...
>>
>> Thanks,
>> Xin
>>
> Are you using qpid or rabbitmq? Does restarting either allow the
> rest of the compute services to proceed? (OpenStack will reconnect
> when you restart the amqp service).
>
Yes I tried restarting rabbitmq, but that doesn't help.
Thanks,
Xin
> Regards
> -steve
>
>> _______________________________________________
>> Rdo-list mailing list
>> Rdo-list(a)redhat.com
>> https://www.redhat.com/mailman/listinfo/rdo-list
>>
>> To unsubscribe: rdo-list-unsubscribe(a)redhat.com
>
_______________________________________________
Rdo-list mailing list
Rdo-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/rdo-list
To unsubscribe: rdo-list-unsubscribe(a)redhat.com
3387
days inactive
3387
days old
5 comments
3 participants
participants (3)
-
Lars Kellogg-Stedman -
Steven Dake -
Zhao, Xin