[Rdo-list] Fwd: Re: Nova boot problems with devstack

Gary Kotton gkotton at redhat.com
Sun May 19 15:00:48 UTC 2013


On 05/18/2013 07:24 PM, Gary Kotton wrote:
>  Hi,
>  I Have encountered a problem and maybe someone can help (RHEL-6.4 with
>  upstream devstack). I deploy a VM and it works. If I restart my host,
>  install devstack and then try and deploy a VM then it fails. If I try
>  again it works. After the next reboot in order to get a VM up and
>  running I need to deploy 3 VM's.
>
>  The trace on nova compute is:
>
>  2013-05-16 08:18:11.742 ERROR nova.openstack.common.rpc.amqp
>  [req-cd75ed38-1d04-4611-9f34-cce01b388413 demo demo] Exception during
>  message handling
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp Traceback
>  (most recent call last):
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 433, in
>  _process_data
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp     **args)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 148,
>  in dispatch
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  return getattr(proxyobj, method)(ctxt, **kwargs)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/exception.py", line 98, in wrapped
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  temp_level, payload)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  self.gen.next()
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/exception.py", line 75, in wrapped
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  return f(self, context, *args, **kw)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 213, in
>  decorated_function
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp     pass
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  self.gen.next()
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 199, in
>  decorated_function
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  return function(self, context, *args, **kwargs)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 264, in
>  decorated_function
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  function(self, context, *args, **kwargs)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 241, in
>  decorated_function
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp     e,
>  sys.exc_info())
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  self.gen.next()
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 228, in
>  decorated_function
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  return function(self, context, *args, **kwargs)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 1320, in run_instance
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  do_run_instance()
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/openstack/common/lockutils.py", line 246, in inner
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  retval = f(*args, **kwargs)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 1319, in do_run_instance
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  admin_password, is_first_time, node, instance)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 877, in _run_instance
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  notify("error", msg=unicode(e))  # notify that build failed
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  self.gen.next()
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 853, in _run_instance
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  image_meta = self._prebuild_instance(context, instance)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 880, in
>  _prebuild_instance
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  self._check_instance_exists(context, instance)
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp   File
>  "/opt/stack/nova/nova/compute/manager.py", line 1083, in
>  _check_instance_exists
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp     raise
>  exception.InstanceExists(name=instance['name'])
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>  InstanceExists: Instance instance-00000001 already exists.
>  2013-05-16 08:18:11.742 TRACE nova.openstack.common.rpc.amqp
>
>  Problem seems to be that instances hang around:
>
>  [openstack at dhcp-4-126 devstack]$ virsh list --all
>   Id    Name                           State
>  ----------------------------------------------------
>
>  [openstack at dhcp-4-126 devstack]$ sudo virsh list --all
>   Id    Name                           State
>  ----------------------------------------------------
>   -     instance-00000001              shut off
>   -     instance-00000002              shut off
>   -     instance-00000003              shut off
>   -     instance-00000004              shut off


Issue appears to be that when one reboots a host running devstack (with
running VMS) that the domains are saved:

./etc/libvirt/qemu/instance-00000001.xml
./var/lib/libvirt/qemu/save/instance-00000001.save
./var/log/libvirt/qemu/instance-00000001.log

By deleting the files it enables one to run VM's after reboot.

Is there a libvirt configuration variable that will ensure that this is
not persistent? Or do we need to purge these each time devstack is
restarted?

>
>  Problem is that I am unable to delete any of these.
>
>  Any ideas?
>  Thanks
>  Gary





More information about the dev mailing list