[Rdo-list] [RDO][Instack] heat is not able to create stack with instack
Peeyush Gupta
peeygupt at in.ibm.com
Wed Jul 23 10:09:38 UTC 2014
Hi James,
I reinstalled the undercloud and retraced all the steps with debug=True
in /etc/nova/nova/conf.
Here are the logs:
Nova-compute logs:
2014-07-23 05:58:03.357 2915 WARNING nova.virt.baremetal.driver
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] Destroy called on non-existing instance
dd613a79-a790-434e-a40e-ea787c6d5035
2014-07-23 05:58:03.806 2915 ERROR nova.compute.manager
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Error: PXE deploy failed for instance
dd613a79-a790-434e-a40e-ea787c6d5035
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Traceback (most recent call last):
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] set_access_ip=set_access_ip)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] return function(self, context,
*args, **kwargs)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1717, in
_spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] LOG.exception(_('Instance failed
to spawn'), instance=instance)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714, in
_spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] block_device_info)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]
block_device_info=block_device_info)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] _update_state(context, node,
None, baremetal_states.DELETED)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 281,
in _spawn
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] self.driver.activate_node
(context, node, instance)
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py", line 500, in
activate_node
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] locals['error'] % instance
['uuid'])
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] InstanceDeployFailure: PXE deploy
failed for instance dd613a79-a790-434e-a40e-ea787c6d5035
2014-07-23 05:58:03.806 2915 TRACE nova.compute.manager [instance:
dd613a79-a790-434e-a40e-ea787c6d5035]
2014-07-23 05:59:27.981 2915 ERROR nova.virt.baremetal.driver
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] Error deploying instance
ba65d12e-b664-4ad9-93fa-992a2886ab01 on baremetal node
68bf3440-7eb6-4b95-b439-74dd8edbe1bf.
2014-07-23 05:59:31.794 2915 ERROR nova.compute.manager
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Instance failed to spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Traceback (most recent call last):
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714, in
_spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] block_device_info)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
block_device_info=block_device_info)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] _update_state(context, node,
None, baremetal_states.DELETED)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 281,
in _spawn
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] self.driver.activate_node
(context, node, instance)
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py", line 500, in
activate_node
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] locals['error'] % instance
['uuid'])
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] InstanceDeployFailure: PXE deploy
failed for instance ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:31.794 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
2014-07-23 05:59:31.870 2915 WARNING nova.virt.baremetal.driver
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] Destroy called on non-existing instance
ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:32.239 2915 ERROR nova.compute.manager
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Error: PXE deploy failed for instance
ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Traceback (most recent call last):
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] set_access_ip=set_access_ip)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] return function(self, context,
*args, **kwargs)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1717, in
_spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] LOG.exception(_('Instance failed
to spawn'), instance=instance)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714, in
_spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] block_device_info)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
block_device_info=block_device_info)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] _update_state(context, node,
None, baremetal_states.DELETED)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] six.reraise(self.type_,
self.value, self.tb)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 281,
in _spawn
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] self.driver.activate_node
(context, node, instance)
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py", line 500, in
activate_node
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] locals['error'] % instance
['uuid'])
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] InstanceDeployFailure: PXE deploy
failed for instance ba65d12e-b664-4ad9-93fa-992a2886ab01
2014-07-23 05:59:32.239 2915 TRACE nova.compute.manager [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01]
2014-07-23 06:05:43.572 2915 WARNING nova.compute.manager [-] Found 4 in
the database and 0 on the hypervisor.
Nova-api logs:
2014-07-23 05:32:42.670 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:32:43.404 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:32:44.431 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:32:46.417 32685 WARNING keystoneclient.middleware.auth_token
[-] Configuring admin URI using auth fragments. This is deprecated, use
'identity_uri' instead.
2014-07-23 05:45:12.903 2947 ERROR nova.wsgi [-] Could not bind to
0.0.0.0:8773
2014-07-23 05:45:12.903 2947 CRITICAL nova [-] error: [Errno 98] Address
already in use
Nova-scheduler logs:
2014-07-23 05:32:45.277 32697 INFO oslo.messaging._drivers.impl_qpid [-]
Connected to AMQP server on 127.0.0.1:5672
2014-07-23 05:45:17.997 2962 INFO oslo.messaging._drivers.impl_qpid [-]
Connected to AMQP server on 127.0.0.1:5672
2014-07-23 05:51:44.675 2962 INFO oslo.messaging._drivers.impl_qpid [-]
Connected to AMQP server on 127.0.0.1:5672
2014-07-23 05:52:06.321 2962 ERROR nova.scheduler.filter_scheduler
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Error from last host: undercloud
(node dde604c0-d7f9-4212-bb26-c9e166b3df7e): [u'Traceback (most recent call
last):\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1272, in
_build_instance\n with rt.instance_claim(context, instance, limits):\n',
u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line
249, in inner\n return f(*args, **kwargs)\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line
122, in instance_claim\n overhead=overhead, limits=limits)\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/claims.py", line 95, in
__init__\n self._claim_test(resources, limits)\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/claims.py", line 148, in
_claim_test\n "; ".join(reasons))\n', u'ComputeResourcesUnavailable:
Insufficient compute resources: Free memory 0.00 MB < requested 2048
MB.\n']
2014-07-23 05:55:53.648 2962 ERROR nova.scheduler.filter_scheduler
[req-af4756b6-9630-4790-b782-629f3033a055 bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
923d2903-d202-487f-82a9-cd7288e0cfe0] Error from last host: undercloud
(node 72b92fba-fd38-40f6-8b80-7216ed8dc72c): [u'Traceback (most recent call
last):\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n set_access_ip=set_access_ip)\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n return function(self, context, *args, **kwargs)\n',
u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n block_device_info=block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n _update_state(context, node, None,
baremetal_states.DELETED)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n self.driver.activate_node(context, node, instance)\n',
u' File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
923d2903-d202-487f-82a9-cd7288e0cfe0\n']
2014-07-23 05:55:53.673 2962 WARNING nova.scheduler.driver
[req-af4756b6-9630-4790-b782-629f3033a055 bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
923d2903-d202-487f-82a9-cd7288e0cfe0] Setting instance to ERROR state.
2014-07-23 05:57:01.605 2962 ERROR nova.scheduler.filter_scheduler
[req-3122d59f-d1d2-4b8d-a1d6-59db30d59acd bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
00626822-43b1-46c6-b896-f1b69a738ac7] Error from last host: undercloud
(node 4cb58619-b383-4f97-be81-7dd0bfa51f8e): [u'Traceback (most recent call
last):\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n set_access_ip=set_access_ip)\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n return function(self, context, *args, **kwargs)\n',
u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n block_device_info=block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n _update_state(context, node, None,
baremetal_states.DELETED)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n self.driver.activate_node(context, node, instance)\n',
u' File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
00626822-43b1-46c6-b896-f1b69a738ac7\n']
2014-07-23 05:57:01.638 2962 WARNING nova.scheduler.driver
[req-3122d59f-d1d2-4b8d-a1d6-59db30d59acd bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
00626822-43b1-46c6-b896-f1b69a738ac7] Setting instance to ERROR state.
2014-07-23 05:58:03.819 2962 ERROR nova.scheduler.filter_scheduler
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Error from last host: undercloud
(node dde604c0-d7f9-4212-bb26-c9e166b3df7e): [u'Traceback (most recent call
last):\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n set_access_ip=set_access_ip)\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n return function(self, context, *args, **kwargs)\n',
u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n block_device_info=block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n _update_state(context, node, None,
baremetal_states.DELETED)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n self.driver.activate_node(context, node, instance)\n',
u' File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
dd613a79-a790-434e-a40e-ea787c6d5035\n']
2014-07-23 05:58:03.829 2962 WARNING nova.scheduler.driver
[req-648fe5c2-1152-44e6-83f2-87686e7ce87a bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
dd613a79-a790-434e-a40e-ea787c6d5035] Setting instance to ERROR state.
2014-07-23 05:59:32.248 2962 ERROR nova.scheduler.filter_scheduler
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Error from last host: undercloud
(node 68bf3440-7eb6-4b95-b439-74dd8edbe1bf): [u'Traceback (most recent call
last):\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1305, in
_build_instance\n set_access_ip=set_access_ip)\n', u' File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 393, in
decorated_function\n return function(self, context, *args, **kwargs)\n',
u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
1717, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1714,
in _spawn\n block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 245,
in spawn\n block_device_info=block_device_info)\n', u' File
"/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 303,
in _spawn\n _update_state(context, node, None,
baremetal_states.DELETED)\n', u' File
"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line
68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u'
File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line
281, in _spawn\n self.driver.activate_node(context, node, instance)\n',
u' File "/usr/lib/python2.7/site-packages/nova/virt/baremetal/pxe.py",
line 500, in activate_node\n locals[\'error\'] % instance[\'uuid\'])\n',
u'InstanceDeployFailure: PXE deploy failed for instance
ba65d12e-b664-4ad9-93fa-992a2886ab01\n']
2014-07-23 05:59:32.267 2962 WARNING nova.scheduler.driver
[req-bc984c41-18c0-4d4c-8dd8-2dc6e3b7044b bd2463eed41a47dbaad173888e8e4e81
5db26e088245493b9d07dcd52be97734] [instance:
ba65d12e-b664-4ad9-93fa-992a2886ab01] Setting instance to ERROR state.
I am not getting the "No valid host" error anymore. It's now PXE deploy
failure.
Regards,
Peeyush Gupta
From: James Slagle <jslagle at redhat.com>
To: Peeyush Gupta/India/IBM at IBMIN
Cc: anantyog at linux.vnet.ibm.com, deepthi at linux.vnet.ibm.com,
Pradeep K Surisetty/India/IBM at IBMIN, Pradeep Kumar Surisetty
<psuriset at linux.vnet.ibm.com>, rdo-list at redhat.com
Date: 07/23/2014 02:08 AM
Subject: Re: [Rdo-list] [RDO][Instack] heat is not able to create stack
with instack
On Tue, Jul 22, 2014 at 11:56:28AM +0530, Peeyush Gupta wrote:
> | fault | {"message": "No valid host was
> found. ", "code": 500, "details": " File
> \"/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py\",
> line 108, in schedule_run_instance |
> | | raise exception.NoValidHost
> (reason=\"\")
> |
> 2014-07-21 04:25:41.473 31078 WARNING nova.scheduler.driver
> [req-a00295d2-7308-4bb9-a40d-0740fe852bf5
921abf3732ce40d0b1502e9aa13c6c2a
> ae8b85d781ad443792f2a3516f38ed88] [instance:
> b3b3f6f9-25fc-45db-b14e-f7daae1f3216] Setting instance to ERROR state.
Unfortunately these error messages really aren't enough to debug the issue.
Can
you set debug=True in /etc/nova/nova.conf aynd restart the following
services:
openstack-nova-api
openstack-nova-compute
openstack-nova-scheduler
openstack-nova-conductor
Delete the overcloud with instack-delete-overcloud, and then try a
deployment
again. If you get another failure, hopefully there will be more useful data
in
the nova logs.
>
> Though I would like to point out here that I am seeing 5 VMs in my setup
> instead of 4.
You should have 5 vm's.
>
> [stack at devstack ~]$ virsh list --all
> Id Name State
> ----------------------------------------------------
> 2 instack running
> 9 baremetal_1 running
> 10 baremetal_2 running
> 11 baremetal_3 running
> - baremetal_0 shut off
>
> Regards,
> Peeyush Gupta
>
>
>
> From: James Slagle <jslagle at redhat.com>
> To: Pradeep Kumar Surisetty <psuriset at linux.vnet.ibm.com>
> Cc: rdo-list at redhat.com, deepthi at linux.vnet.ibm.com, Peeyush
> Gupta/India/IBM at IBMIN, Pradeep K Surisetty/India/IBM at IBMIN,
> anantyog at linux.vnet.ibm.com
> Date: 07/21/2014 09:50 PM
> Subject: Re: [Rdo-list] [RDO][Instack] heat is not able to create
stack
> with instack
>
>
>
> On Mon, Jul 21, 2014 at 05:59:25PM +0530, Pradeep Kumar Surisetty wrote:
> > Hi All
> >
> > I have been trying to set instack with RDO. I have successfully
> installed
> > undercloud and moving on to overcloud. Now, when I run
> > "instack-deploy-overcloud", I get the following error:
> >
> > + OVERCLOUD_YAML_PATH=overcloud.yaml
> > + heat stack-create -f overcloud.yaml -P
> AdminToken=b003d63242f5db3e1ad4864ae66911e02ba19bcb -P
> AdminPassword=7bfe4d4a18280752ad07f259a69a3ed00db2ab44 -P
> CinderPassword=df0893b4355f3511a6d67538dd592d02d1bc11d3 -P
> GlancePassword=066f65f878157b438a916ccbd44e0b7037ee!
> > 118f -P HeatPassword=58fda0e4d6708e0164167b11fe6fca6ab6b35ec6 -P
> NeutronPassword=80853ad029feb77bb7c60d035542f21aa5c24177 -P
> NovaPassword=331474580be53b78e40c91dfdfc2323578a035e7 -P
> NeutronPublicInterface=eth0 -P
> SwiftPassword=b0eca57b45ebf3dd5cae071dc3880888fb1d4840 -P
> SwiftHashSuffix=a8d87f3952d6f91da589fbef801bb92141fd1461 -P
> NovaComputeLibvirtType=qemu -P 'GlanceLogFile='\'''\''' -P
> NeutronDnsmasqOptions=dhcp-option-force=26,1400 overcloud
> >
>
+--------------------------------------+------------+--------------------+----------------------+
>
> > | id | stack_name | stack_status
> | creation_time |
> >
>
+--------------------------------------+------------+--------------------+----------------------+
>
> > | 0ca028e7-682b-41ef-8af0-b2eb67bee272 | overcloud |
CREATE_IN_PROGRESS
> | 2014-07-18T10:50:48Z |
> >
>
+--------------------------------------+------------+--------------------+----------------------+
>
> > + tripleo wait_for_stack_ready 220 10 overcloud
> > Command output matched 'CREATE_FAILED'. Exiting...
> >
> > Now, i understand that the stack isn't being created. So, I tried to
> check out the state of the stack:
> >
> > [stack at localhost ~]$ heat stack-list
> >
>
+--------------------------------------+------------+---------------+----------------------+
>
> > | id | stack_name | stack_status |
> creation_time |
> >
>
+--------------------------------------+------------+---------------+----------------------+
>
> > | 0ca028e7-682b-41ef-8af0-b2eb67bee272 | overcloud | CREATE_FAILED |
> 2014-07-18T10:50:48Z |
> >
>
+--------------------------------------+------------+---------------+----------------------+
>
> >
> >
> > i even tried to create stack manually, but ended up getting the same
> > error.
> >
> > Update: Here is the heat log:
> >
> > 2014-07-18 06:51:11.884 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> > 2014-07-18 06:51:12.921 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> > 2014-07-18 06:51:16.058 30750 ERROR heat.engine.resource [-] CREATE :
> Server "SwiftStorage0" [07e42c3d-0f1b-4bb9-b980-ffbb74ac770d] Stack
> "overcloud" [0ca028e7-682b-41ef-8af0-b2eb67bee272]
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource Traceback
(most
> recent call last):
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource File
> "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 420, in
> _do_action
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource while not
> check(handle_data):
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource File
> "/usr/lib/python2.7/site-packages/heat/engine/resources/server.py", line
> 545, in check_create_complete
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource return
> self._check_active(server)
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource File
> "/usr/lib/python2.7/site-packages/heat/engine/resources/server.py", line
> 561, in _check_active
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource raise exc
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource Error:
Creation
> of server overcloud-SwiftStorage0-qdjqbif6peva failed.
> > 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource
> > 2014-07-18 06:51:16.255 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> > 2014-07-18 06:51:16.939 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> > 2014-07-18 06:51:17.368 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> > 2014-07-18 06:51:17.638 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> > 2014-07-18 06:51:18.158 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back to using default
> > 2014-07-18 06:51:18.613 30750 WARNING heat.common.keystoneclient [-]
> stack_user_domain ID not set in heat.conf falling back ...
>
> Hi Pradeep,
>
> Can you run a "nova show <instance-id>" on the failed instance? And also
> provide any tracebacks or errors from the nova compute log
> under /var/log/nova?
>
> --
> -- James Slagle
> --
>
>
>
--
-- James Slagle
--
More information about the dev
mailing list