[Rdo-list] [RDO][Instack] heat is not able to create stack with instack

Peeyush Gupta peeygupt at in.ibm.com
Wed Jul 23 10:09:38 UTC 2014


Hi James,

I reinstalled the undercloud and retraced all the steps with debug=True
in /etc/nova/nova/conf.
Here are the logs:

Nova-compute logs:

2014-07-23 05:58:03.357 2915 WARNING nova.virt.baremetal.driver
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] Destroy called on non-existing instance
dd613a79-a790-434e-a40e-ea787c6d5035
2014-07-23 05:58:03.806 2915 ERROR nova.compute.manager
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Error: PXE deploy failed for instance
dd613a79-a790-434e-a40e-ea787c6d5035
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Traceback (most recent call last):
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     set_access_ip=set_access_ip)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     return function(self, context,
*args, **kwargs)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1717, in
_spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     LOG.exception(_('Instance failed
to spawn'), instance=instance)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714, in
_spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     block_device_info)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]
block_device_info=block_device_info)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     _update_state(context, node,
None, baremetal_states.DELETED)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 281,
in _spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     self.driver.activate_node
(context, node, instance)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py", line 500, in
activate_node
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]     locals['error'] % instance
['uuid'])
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] InstanceDeployFailure: PXE deploy
failed for instance dd613a79-a790-434e-a40e-ea787c6d5035
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]
2014-07-23 05:59:27.981 2915 ERROR nova.virt.baremetal.driver
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] Error deploying instance
ba65d12e-b664-4ad9-93fa-992a2886ab01 on baremetal node
68bf3440-7eb6-4b95-b439-74dd8edbe1bf.
2014-07-23 05:59:31.794 2915 ERROR nova.compute.manager
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Instance failed to spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Traceback (most recent call last):
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714, in
_spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     block_device_info)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
block_device_info=block_device_info)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     _update_state(context, node,
None, baremetal_states.DELETED)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 281,
in _spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     self.driver.activate_node
(context, node, instance)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py", line 500, in
activate_node
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     locals['error'] % instance
['uuid'])
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] InstanceDeployFailure: PXE deploy
failed for instance ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
2014-07-23 05:59:31.870 2915 WARNING nova.virt.baremetal.driver
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] Destroy called on non-existing instance
ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:32.239 2915 ERROR nova.compute.manager
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Error: PXE deploy failed for instance
ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Traceback (most recent call last):
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     set_access_ip=set_access_ip)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     return function(self, context,
*args, **kwargs)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1717, in
_spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     LOG.exception(_('Instance failed
to spawn'), instance=instance)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714, in
_spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     block_device_info)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
block_device_info=block_device_info)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     _update_state(context, node,
None, baremetal_states.DELETED)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 281,
in _spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     self.driver.activate_node
(context, node, instance)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]   File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py", line 500, in
activate_node
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]     locals['error'] % instance
['uuid'])
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] InstanceDeployFailure: PXE deploy
failed for instance ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
2014-07-23 06:05:43.572 2915 WARNING nova.compute.manager [-] Found 4 in
the database and 0 on the hypervisor.

Nova-api logs:


2014-07-23 05:32:42.670 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:32:43.404 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:32:44.431 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:32:46.417 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:45:12.903 2947 ERROR nova.wsgi [-] Could not bind to
0.0.0.0:8773
2014-07-23 05:45:12.903 2947 CRITICAL nova [-] error: [Errno 98] Address
already in use

Nova-scheduler logs:
2014-07-23 05:32:45.277 32697 INFO oslo.messaging._drivers.impl_qpid [-]
Connected to AMQP server on 127.0.0.1:5672
2014-07-23 05:45:17.997 2962 INFO oslo.messaging._drivers.impl_qpid [-]
Connected to AMQP server on 127.0.0.1:5672
2014-07-23 05:51:44.675 2962 INFO oslo.messaging._drivers.impl_qpid [-]
Connected to AMQP server on 127.0.0.1:5672
2014-07-23 05:52:06.321 2962 ERROR nova.scheduler.filter_scheduler
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Error from last host: undercloud
(node dde604c0-d7f9-4212-bb26-c9e166b3df7e): [u'Traceback (most recent call
last):\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1272, in
_build_instance\n    with rt.instance_claim(context, instance, limits):\n',
u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line
249, in inner\n    return f(*args, **kwargs)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line
122, in instance_claim\n    overhead=overhead, limits=limits)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/claims.py", line 95, in
__init__\n    self._claim_test(resources, limits)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/claims.py", line 148, in
_claim_test\n    "; ".join(reasons))\n', u'ComputeResourcesUnavailable:
Insufficient compute resources: Free memory 0.00 MB < requested 2048
MB.\n']
2014-07-23 05:55:53.648 2962 ERROR nova.scheduler.filter_scheduler
[req-af4756b6-9630-4790-b782-629f3033a055 bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
923d2903-d202-487f-82a9-cd7288e0cfe0] Error from last host: undercloud
(node 72b92fba-fd38-40f6-8b80-7216ed8dc72c): [u'Traceback (most recent call
last):\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n    set_access_ip=set_access_ip)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n    return function(self, context, *args, **kwargs)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n    LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n    block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n    block_device_info=block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n    _update_state(context, node, None,
baremetal_states.DELETED)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n    self.driver.activate_node(context, node, instance)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n    locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
923d2903-d202-487f-82a9-cd7288e0cfe0\n']
2014-07-23 05:55:53.673 2962 WARNING nova.scheduler.driver
[req-af4756b6-9630-4790-b782-629f3033a055 bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
923d2903-d202-487f-82a9-cd7288e0cfe0] Setting instance to ERROR state.
2014-07-23 05:57:01.605 2962 ERROR nova.scheduler.filter_scheduler
[req-3122d59f-d1d2-4b8d-a1d6-59db30d59acd bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
00626822-43b1-46c6-b896-f1b69a738ac7] Error from last host: undercloud
(node 4cb58619-b383-4f97-be81-7dd0bfa51f8e): [u'Traceback (most recent call
last):\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n    set_access_ip=set_access_ip)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n    return function(self, context, *args, **kwargs)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n    LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n    block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n    block_device_info=block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n    _update_state(context, node, None,
baremetal_states.DELETED)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n    self.driver.activate_node(context, node, instance)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n    locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
00626822-43b1-46c6-b896-f1b69a738ac7\n']
2014-07-23 05:57:01.638 2962 WARNING nova.scheduler.driver
[req-3122d59f-d1d2-4b8d-a1d6-59db30d59acd bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
00626822-43b1-46c6-b896-f1b69a738ac7] Setting instance to ERROR state.
2014-07-23 05:58:03.819 2962 ERROR nova.scheduler.filter_scheduler
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Error from last host: undercloud
(node dde604c0-d7f9-4212-bb26-c9e166b3df7e): [u'Traceback (most recent call
last):\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n    set_access_ip=set_access_ip)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n    return function(self, context, *args, **kwargs)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n    LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n    block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n    block_device_info=block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n    _update_state(context, node, None,
baremetal_states.DELETED)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n    self.driver.activate_node(context, node, instance)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n    locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
dd613a79-a790-434e-a40e-ea787c6d5035\n']
2014-07-23 05:58:03.829 2962 WARNING nova.scheduler.driver
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Setting instance to ERROR state.
2014-07-23 05:59:32.248 2962 ERROR nova.scheduler.filter_scheduler
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Error from last host: undercloud
(node 68bf3440-7eb6-4b95-b439-74dd8edbe1bf): [u'Traceback (most recent call
last):\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n    set_access_ip=set_access_ip)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n    return function(self, context, *args, **kwargs)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n    LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n    block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n    block_device_info=block_device_info)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n    _update_state(context, node, None,
baremetal_states.DELETED)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n    self.driver.activate_node(context, node, instance)\n',
u'  File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n    locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
ba65d12e-b664-4ad9-93fa-992a2886ab01\n']
2014-07-23 05:59:32.267 2962 WARNING nova.scheduler.driver
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Setting instance to ERROR state.

I am not getting the "No valid host" error anymore. It's now PXE deploy
failure.

Regards,
Peeyush Gupta




From:	James Slagle <jslagle at redhat.com>
To:	Peeyush Gupta/India/IBM at IBMIN
Cc:	anantyog at linux.vnet.ibm.com, deepthi at linux.vnet.ibm.com,
            Pradeep K Surisetty/India/IBM at IBMIN, Pradeep Kumar Surisetty
            <psuriset at linux.vnet.ibm.com>, rdo-list at redhat.com
Date:	07/23/2014 02:08 AM
Subject:	Re: [Rdo-list] [RDO][Instack] heat is not able to create stack
            with instack



On Tue, Jul 22, 2014 at 11:56:28AM +0530, Peeyush Gupta wrote:
> | fault                                | {"message": "No valid host was
> found. ", "code": 500, "details": "  File
> \"/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py\",
> line 108, in schedule_run_instance |
> |                                      |     raise exception.NoValidHost
> (reason=\"\")
> |
> 2014-07-21 04:25:41.473 31078 WARNING nova.scheduler.driver
> [req-a00295d2-7308-4bb9-a40d-0740fe852bf5
921abf3732ce40d0b1502e9aa13c6c2a
> ae8b85d781ad443792f2a3516f38ed88] [instance:
> b3b3f6f9-25fc-45db-b14e-f7daae1f3216] Setting instance to ERROR state.

Unfortunately these error messages really aren't enough to debug the issue.
Can
you set debug=True in /etc/nova/nova.conf aynd restart the following
services:

openstack-nova-api
openstack-nova-compute
openstack-nova-scheduler
openstack-nova-conductor

Delete the overcloud with instack-delete-overcloud, and then try a
deployment
again. If you get another failure, hopefully there will be more useful data
in
the nova logs.

>
> Though I would like to point out here that I am seeing 5 VMs in my setup
> instead of 4.

You should have 5 vm's.

>
> [stack at devstack ~]$ virsh list --all
>  Id    Name                           State
> ----------------------------------------------------
>  2     instack                        running
>  9     baremetal_1                    running
>  10    baremetal_2                    running
>  11    baremetal_3                    running
>  -     baremetal_0                    shut off
>
> Regards,
> Peeyush Gupta
>
>
>
> From:		 James Slagle <jslagle at redhat.com>
> To:		 Pradeep Kumar Surisetty <psuriset at linux.vnet.ibm.com>
> Cc:		 rdo-list at redhat.com, deepthi at linux.vnet.ibm.com, Peeyush
>             Gupta/India/IBM at IBMIN, Pradeep K Surisetty/India/IBM at IBMIN,
>             anantyog at linux.vnet.ibm.com
> Date:		 07/21/2014 09:50 PM
> Subject:		 Re: [Rdo-list] [RDO][Instack] heat is not able to create
stack
>             with instack
>
>
>
> On Mon, Jul 21, 2014 at 05:59:25PM +0530, Pradeep Kumar Surisetty wrote:
> >    Hi All
> >
> >    I have been trying to set instack with RDO. I have successfully
> installed
> >    undercloud and moving on to overcloud. Now, when I run
> >    "instack-deploy-overcloud", I get the following error:
> >
> >  + OVERCLOUD_YAML_PATH=overcloud.yaml
> >  + heat stack-create -f overcloud.yaml -P
> AdminToken=b003d63242f5db3e1ad4864ae66911e02ba19bcb -P
> AdminPassword=7bfe4d4a18280752ad07f259a69a3ed00db2ab44 -P
> CinderPassword=df0893b4355f3511a6d67538dd592d02d1bc11d3 -P
> GlancePassword=066f65f878157b438a916ccbd44e0b7037ee!
> >   118f -P HeatPassword=58fda0e4d6708e0164167b11fe6fca6ab6b35ec6 -P
> NeutronPassword=80853ad029feb77bb7c60d035542f21aa5c24177 -P
> NovaPassword=331474580be53b78e40c91dfdfc2323578a035e7 -P
> NeutronPublicInterface=eth0 -P
> SwiftPassword=b0eca57b45ebf3dd5cae071dc3880888fb1d4840 -P
> SwiftHashSuffix=a8d87f3952d6f91da589fbef801bb92141fd1461 -P
> NovaComputeLibvirtType=qemu -P 'GlanceLogFile='\'''\''' -P
> NeutronDnsmasqOptions=dhcp-option-force=26,1400 overcloud
> >
>
+--------------------------------------+------------+--------------------+----------------------+

>
> >  | id                                   | stack_name | stack_status
> | creation_time        |
> >
>
+--------------------------------------+------------+--------------------+----------------------+

>
> >  | 0ca028e7-682b-41ef-8af0-b2eb67bee272 | overcloud  |
CREATE_IN_PROGRESS
> | 2014-07-18T10:50:48Z |
> >
>
+--------------------------------------+------------+--------------------+----------------------+

>
> >  + tripleo wait_for_stack_ready 220 10 overcloud
> >  Command output matched 'CREATE_FAILED'. Exiting...
> >
> >  Now, i understand that the stack isn't being created. So, I tried to
> check out the state of the stack:
> >
> >  [stack at localhost ~]$ heat stack-list
> >
>
+--------------------------------------+------------+---------------+----------------------+

>
> >  | id                                   | stack_name | stack_status  |
> creation_time        |
> >
>
+--------------------------------------+------------+---------------+----------------------+

>
> >  | 0ca028e7-682b-41ef-8af0-b2eb67bee272 | overcloud  | CREATE_FAILED |
> 2014-07-18T10:50:48Z |
> >
>
+--------------------------------------+------------+---------------+----------------------+

>
> >
> >
> >    i even tried to create stack manually, but ended up getting the same
> >    error.
> >
> >    Update: Here is the heat log:
> >
> >  2014-07-18 06:51:11.884 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> >  2014-07-18 06:51:12.921 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> >  2014-07-18 06:51:16.058 30750 ERROR heat.engine.resource [-] CREATE :
> Server "SwiftStorage0" [07e42c3d-0f1b-4bb9-b980-ffbb74ac770d] Stack
> "overcloud" [0ca028e7-682b-41ef-8af0-b2eb67bee272]
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource Traceback
(most
> recent call last):
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource   File
> "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 420, in
> _do_action
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource     while not
> check(handle_data):
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource   File
> "/usr/lib/python2.7/site-packages/heat/engine/resources/server.py", line
> 545, in check_create_complete
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource     return
> self._check_active(server)
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource   File
> "/usr/lib/python2.7/site-packages/heat/engine/resources/server.py", line
> 561, in _check_active
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource     raise exc
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource Error:
Creation
> of server overcloud-SwiftStorage0-qdjqbif6peva failed.
> >  2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource
> >  2014-07-18 06:51:16.255 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> >  2014-07-18 06:51:16.939 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> >  2014-07-18 06:51:17.368 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> >  2014-07-18 06:51:17.638 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> >  2014-07-18 06:51:18.158 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> >  2014-07-18 06:51:18.613 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back ...
>
> Hi Pradeep,
>
> Can you run a "nova show <instance-id>" on the failed instance? And also
> provide any tracebacks or errors from the nova compute log
> under /var/log/nova?
>
> --
> -- James Slagle
> --
>
>
>
--
-- James Slagle
--






More information about the dev mailing list