Hello again,

Ok on controller node I modify the neutron server configuration with 

nova_admin_tenant_id = f23ed5be5f534fdba31d23f60621347d

where id is "services"  in keystone and now it's working with "vif_plugging_is_fatal = True". Good thing. 

Well by the way the migrate doesnt working ... 

2014-07-15 14:20 GMT+02:00 Benoit ML <ben42ml@gmail.com>:

Hello,

Thank you for taking time  !

Well on the compute node, when i activate "vif_plugging_is_fatal = True", the vm creation stuck in spawning state, and in neutron server log i have : 

=======================================

2014-07-15 14:12:52.351 18448 DEBUG neutron.notifiers.nova [-] Sending events: [{'status': 'completed', 'tag': u'ba56921c-4628-4fcc-9dc5-2b324cd91faf', 'name': 'network-vif-plugged', 'server_uuid': u'f8554441-565c-49ec-bc88-3cbc628b0579'}] send_events /usr/lib/python2.7/site-packages/neutron/notifiers/nova.py:218

2014-07-15 14:12:52.354 18448 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): localhost

2014-07-15 14:12:52.360 18448 DEBUG urllib3.connectionpool [-] "POST /v2/5c9c186a909e499e9da0dd5cf2c403e0/os-server-external-events HTTP/1.1" 401 23 _make_request /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:295

2014-07-15 14:12:52.362 18448 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): localhost

2014-07-15 14:12:52.452 18448 DEBUG urllib3.connectionpool [-] "POST /v2.0/tokens HTTP/1.1" 401 114 _make_request /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:295

2014-07-15 14:12:52.453 18448 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'status': 'completed', 'tag': u'ba56921c-4628-4fcc-9dc5-2b324cd91faf', 'name': 'network-vif-plugged', 'server_uuid': u'f8554441-565c-49ec-bc88-3cbc628b0579'}]

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova Traceback (most recent call last):

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova   File "/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py", line 221, in send_events

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova     batched_events)

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova   File "/usr/lib/python2.7/site-packages/novaclient/v1_1/contrib/server_external_events.py", line 39, in create

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova     return_raw=True)

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova   File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 152, in _create

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova     _resp, body = self.api.client.post(url, body=body)

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova   File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 312, in post

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova     return self._cs_request(url, 'POST', **kwargs)

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova   File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 301, in _cs_request

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova     raise e

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova Unauthorized: Unauthorized (HTTP 401)

2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova 

2014-07-15 14:12:58.321 18448 DEBUG neutron.openstack.common.rpc.amqp [-] received {u'_context_roles': [u'admin'], u'_context_request_id': u'req-9bf35c42-3477-4ed3-8092-af729c21198c', u'_context_read_deleted': u'no', u'_context_user_name': None, u'_context_project_name': None, u'namespace': None, u'_context_tenant_id': None, u'args': {u'agent_state': {u'agent_state': {u'topic': u'N/A', u'binary': u'neutron-openvswitch-agent', u'host': u'pvidgsh006.pvi', u'agent_type': u'Open vSwitch agent', u'configurations': {u'tunnel_types': [u'vxlan'], u'tunneling_ip': u'192.168.40.5', u'bridge_mappings': {}, u'l2_population': False, u'devices': 1}}}, u'time': u'2014-07-15T12:12:58.313995'}, u'_context_tenant': None, u'_unique_id': u'7c9a4dfcd256494caf6e1327c8051e29', u'_context_is_admin': True, u'version': u'1.0', u'_context_timestamp': u'2014-07-15 12:01:28.190772', u'_context_tenant_name': None, u'_context_user': None, u'_context_user_id': None, u'method': u'report_state', u'_context_project_id': None} _safe_log /usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/common.py:280

=======================================

Well I'm supposed it's related ... Perhaps with those options in neutron.conf : 

======================================

notify_nova_on_port_status_changes = True

notify_nova_on_port_data_changes = True

nova_url = http://localhost:8774/v2

nova_admin_tenant_name = services

nova_admin_username = nova

nova_admin_password = nova

nova_admin_auth_url = http://localhost:35357/v2.0

======================================

But well didnt see anything wrong ...

Thank you in advance !

Regards,

2014-07-11 16:08 GMT+02:00 Vimal Kumar <vimal7370@gmail.com>:

-----

File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 239, in authenticate\\n    content_type="application/json")\\n\', u\'  File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 163, in _cs_request\\n    raise exceptions.Unauthorized(message=body)\\n\', u\'Unauthorized: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}\\n\'].\n']

-----

Looks like HTTP connection to neutron server is resulting in 401 error.

Try enabling debug mode for neutron server and then tail /var/log/neutron/server.log , hopefully you should get more info.

On Fri, Jul 11, 2014 at 7:13 PM, Benoit ML <ben42ml@gmail.com> wrote:

Hello,

Ok I see. Nova telles neutron/openvswitch to create the bridge qbr prior to the migration itself.

I ve already activate debug and verbose ... But well i'm really stuck, dont know how and where to search/look ...

Regards,

2014-07-11 15:09 GMT+02:00 Miguel Angel <miguelangel@ajo.es>:

Hi Benoit,

  A manual virsh migration should fail, because the

network ports are not migrated to the destination host.

  You must investigate on the authentication problem itself,

and let nova handle all the underlying API calls which should happen...

  May be it's worth setting nova.conf to debug=True

---

irc: ajo / mangelajo

Miguel Angel Ajo Pelayo
+34 636 52 25 69
skype: ajoajoajo

2014-07-11 14:41 GMT+02:00 Benoit ML <ben42ml@gmail.com>:

Hello,

cat /etc/redhat-release
CentOS Linux release 7 (Rebuilt from: RHEL 7.0)

Regards,

2014-07-11 13:40 GMT+02:00 Boris Derzhavets <bderzhavets@hotmail.com>:

Could you please post /etc/redhat-release.

Boris.

---

Date: Fri, 11 Jul 2014 11:57:12 +0200
From: ben42ml@gmail.com
To: rdo-list@redhat.com
Subject: [Rdo-list] Icehouse multi-node - Centos7 - live migration failed because of "network qbr no such device"

Hello,

I'm working on a multi-node setup of openstack Icehouse using centos7.

Well i have : 

  - one controllor node with all server services thing stuff

  - one network node with openvswitch agent, l3-agent, dhcp-agent

  - two compute node with nova-compute and neutron-openvswitch

  - one storage nfs node

NetworkManager is deleted on compute nodes and network node.

My network use is configured to use vxlan.  I can create VM, tenant-network, external-network, routeur, assign floating-ip to VM, push ssh-key into VM, create volume from glance image, etc... Evrything is conected and reacheable. Pretty cool :) 

But when i try to migrate VM things go wrong ...  I have configured  nova, libvirtd and qemu to use migration through libvirt-tcp.

I have create and exchanged ssh-key for nova user on all  node. I have verified userid and groupid of nova.

Well nova-compute log, on the target compute node, : 

2014-07-11 11:45:02.749 6984 TRACE nova.compute.manager [instance: a5326fe1-4faa-4347-ba29-159fce26a85c] RemoteError: Remote error: Unauthorized {"error": {"m

essage": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}

So well after searching a lots in all logs, i have fount that i cant simply migration VM between compute node with a simple virsh :

virsh migrate instance-00000084 qemu+tcp://<dest>/system

The error is : 

erreur :Cannot get interface MTU on 'qbr3ca65809-05': No such device

Well when i look on the source hyperviseur the bridge "qbr3ca65809" exists  and have a network tap device.  And moreover i manually create qbr3ca65809 on the target hypervisor, virsh migrate succed !

Can you help me plz ?

What can i do wrong ? Perhpas neutron must create the bridge before migration but didnt for a mis configuration ? 

Plz ask anything you need !

Thank you in advance.

The full nova-compute log attached.

Regards,

--
--
Benoit

_______________________________________________ Rdo-list mailing list Rdo-list@redhat.com https://www.redhat.com/mailman/listinfo/rdo-list

--
--
Benoit

_______________________________________________
Rdo-list mailing list
Rdo-list@redhat.com
https://www.redhat.com/mailman/listinfo/rdo-list

--
--
Benoit

_______________________________________________
Rdo-list mailing list
Rdo-list@redhat.com
https://www.redhat.com/mailman/listinfo/rdo-list

--
--
Benoit

--
--
Benoit