On 05/18/2013 07:24 PM, Gary Kotton wrote:
Hi,
I Have encountered a problem and maybe someone can help (RHEL-6.4 with
upstream devstack). I deploy a VM and it works. If I restart my host,
install devstack and then try and deploy a VM then it fails. If I try
again it works. After the next reboot in order to get a VM up and
running I need to deploy 3 VM's.
The trace on nova compute is:
2013-05-16 08:18:11.742 ERROR nova.openstack.common.rpc.amqp
[req-cd75ed38-1d04-4611-9f34-cce01b388413 demo demo] Exception during
message handling
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp Traceback
(most recent call last):
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 433, in
_process_data
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp **args)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 148,
in dispatch
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
return getattr(proxyobj, method)(ctxt, **kwargs)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/exception.py", line 98, in wrapped
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
temp_level, payload)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
self.gen.next()
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/exception.py", line 75, in wrapped
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
return f(self, context, *args, **kw)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 213, in
decorated_function
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp pass
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
self.gen.next()
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 199, in
decorated_function
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
return function(self, context, *args, **kwargs)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 264, in
decorated_function
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
function(self, context, *args, **kwargs)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 241, in
decorated_function
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp e,
sys.exc_info())
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
self.gen.next()
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 228, in
decorated_function
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
return function(self, context, *args, **kwargs)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 1320, in run_instance
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
do_run_instance()
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/openstack/common/lockutils.py", line 246, in inner
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
retval = f(*args, **kwargs)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 1319, in do_run_instance
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
admin_password, is_first_time, node, instance)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 877, in _run_instance
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
notify("error", msg=unicode(e)) # notify that build failed
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
self.gen.next()
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 853, in _run_instance
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
image_meta = self._prebuild_instance(context, instance)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 880, in
_prebuild_instance
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
self._check_instance_exists(context, instance)
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp File
"/opt/stack/nova/nova/compute/manager.py", line 1083, in
_check_instance_exists
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp raise
exception.InstanceExists(name=instance['name'])
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
InstanceExists: Instance instance-00000001 already exists.
2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
Problem seems to be that instances hang around:
[openstack@dhcp-4-126 devstack]$ virsh list --all
Id Name State
----------------------------------------------------
[openstack@dhcp-4-126 devstack]$ sudo virsh list --all
Id Name State
----------------------------------------------------
- instance-00000001 shut off
- instance-00000002 shut off
- instance-00000003 shut off
- instance-00000004 shut off
Issue appears to be that when one reboots a host running devstack (with
running VMS) that the domains are saved:
./etc/libvirt/qemu/instance-00000001.xml
./var/lib/libvirt/qemu/save/instance-00000001.save
./var/log/libvirt/qemu/instance-00000001.log
By deleting the files it enables one to run VM's after reboot.
Is there a libvirt configuration variable that will ensure that this is
not persistent? Or do we need to purge these each time devstack is
restarted?
Problem is that I am unable to delete any of these.
Any ideas?
Thanks
Gary