<div dir="ltr">Hello again,<div><br></div><div>Ok on controller node I modify the neutron server configuration with </div><div><div>nova_admin_tenant_id = f23ed5be5f534fdba31d23f60621347d</div></div><div><br></div><div>where id is "services" in keystone and now it's working with "<span style="font-family:arial,sans-serif;font-size:16px">vif_plugging_is_fatal = True". Good thing. </span></div>
<div><span style="font-family:arial,sans-serif;font-size:16px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:16px">Well by the way the migrate doesnt working ... </span></div><div><br></div><div>
<br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-07-15 14:20 GMT+02:00 Benoit ML <span dir="ltr"><<a href="mailto:ben42ml@gmail.com" target="_blank">ben42ml@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hello,<div><br></div><div>Thank you for taking time !</div><div><br></div><div>Well on the compute node, when i activate "vif_plugging_is_fatal = True", the vm creation stuck in spawning state, and in neutron server log i have : </div>
<div><br></div><div>=======================================</div><div><div>2014-07-15 14:12:52.351 18448 DEBUG neutron.notifiers.nova [-] Sending events: [{'status': 'completed', 'tag': u'ba56921c-4628-4fcc-9dc5-2b324cd91faf', 'name': 'network-vif-plugged', 'server_uuid': u'f8554441-565c-49ec-bc88-3cbc628b0579'}] send_events /usr/lib/python2.7/site-packages/neutron/notifiers/nova.py:218</div>
<div>2014-07-15 14:12:52.354 18448 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): localhost</div><div>2014-07-15 14:12:52.360 18448 DEBUG urllib3.connectionpool [-] "POST /v2/5c9c186a909e499e9da0dd5cf2c403e0/os-server-external-events HTTP/1.1" 401 23 _make_request /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:295</div>
<div>2014-07-15 14:12:52.362 18448 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): localhost</div><div>2014-07-15 14:12:52.452 18448 DEBUG urllib3.connectionpool [-] "POST /v2.0/tokens HTTP/1.1" 401 114 _make_request /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:295</div>
<div>2014-07-15 14:12:52.453 18448 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'status': 'completed', 'tag': u'ba56921c-4628-4fcc-9dc5-2b324cd91faf', 'name': 'network-vif-plugged', 'server_uuid': u'f8554441-565c-49ec-bc88-3cbc628b0579'}]</div>
<div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova Traceback (most recent call last):</div><div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py", line 221, in send_events</div>
<div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova batched_events)</div><div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/site-packages/novaclient/v1_1/contrib/server_external_events.py", line 39, in create</div>
<div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova return_raw=True)</div><div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 152, in _create</div>
<div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova _resp, body = self.api.client.post(url, body=body)</div><div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 312, in post</div>
<div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova return self._cs_request(url, 'POST', **kwargs)</div><div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 301, in _cs_request</div>
<div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova raise e</div><div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova Unauthorized: Unauthorized (HTTP 401)</div><div>2014-07-15 14:12:52.453 18448 TRACE neutron.notifiers.nova </div>
<div>2014-07-15 14:12:58.321 18448 DEBUG neutron.openstack.common.rpc.amqp [-] received {u'_context_roles': [u'admin'], u'_context_request_id': u'req-9bf35c42-3477-4ed3-8092-af729c21198c', u'_context_read_deleted': u'no', u'_context_user_name': None, u'_context_project_name': None, u'namespace': None, u'_context_tenant_id': None, u'args': {u'agent_state': {u'agent_state': {u'topic': u'N/A', u'binary': u'neutron-openvswitch-agent', u'host': u'pvidgsh006.pvi', u'agent_type': u'Open vSwitch agent', u'configurations': {u'tunnel_types': [u'vxlan'], u'tunneling_ip': u'192.168.40.5', u'bridge_mappings': {}, u'l2_population': False, u'devices': 1}}}, u'time': u'2014-07-15T12:12:58.313995'}, u'_context_tenant': None, u'_unique_id': u'7c9a4dfcd256494caf6e1327c8051e29', u'_context_is_admin': True, u'version': u'1.0', u'_context_timestamp': u'2014-07-15 12:01:28.190772', u'_context_tenant_name': None, u'_context_user': None, u'_context_user_id': None, u'method': u'report_state', u'_context_project_id': None} _safe_log /usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/common.py:280</div>
</div><div><div>=======================================</div><div></div></div><div><br></div><div>Well I'm supposed it's related ... Perhaps with those options in neutron.conf : </div><div><div>======================================</div>
<div>notify_nova_on_port_status_changes = True</div><div>notify_nova_on_port_data_changes = True</div></div><div><div>nova_url = <a href="http://localhost:8774/v2" target="_blank">http://localhost:8774/v2</a></div></div>
<div><div>nova_admin_tenant_name = services</div>
</div><div><div>nova_admin_username = nova</div></div><div><div>nova_admin_password = nova</div></div><div><div>nova_admin_auth_url = <a href="http://localhost:35357/v2.0" target="_blank">http://localhost:35357/v2.0</a></div>
</div><div><div>
======================================</div></div><div><br></div><div>But well didnt see anything wrong ...</div><div><br></div><div>Thank you in advance !</div><div><br></div><div>Regards,</div><div><br></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">2014-07-11 16:08 GMT+02:00 Vimal Kumar <span dir="ltr"><<a href="mailto:vimal7370@gmail.com" target="_blank">vimal7370@gmail.com</a>></span>:<div><div class="h5"><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">-----<div><span style="color:rgb(0,0,0);font-family:'Courier New',Courier,monospace;font-size:14px">File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 239, in authenticate\\n content_type="application/json")\\n\', u\' File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 163, in _cs_request\\n raise exceptions.Unauthorized(message=body)\\n\', u\'Unauthorized: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}\\n\'].\n']</span><br>
<div>-----</div></div><div class="gmail_extra"><br></div><div class="gmail_extra">Looks like HTTP connection to neutron server is resulting in 401 error.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Try enabling debug mode for neutron server and then tail /var/log/neutron/server.log , hopefully you should get more info.</div>
<div><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jul 11, 2014 at 7:13 PM, Benoit ML <span dir="ltr"><<a href="mailto:ben42ml@gmail.com" target="_blank">ben42ml@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div>Hello,<br><br></div><div>Ok I see. Nova telles neutron/openvswitch to create the bridge qbr prior to the migration itself.<br></div>I ve already activate debug and verbose ... But well i'm really stuck, dont know how and where to search/look ...<br>
<br><br><br></div>Regards,<br><div><br><br><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-07-11 15:09 GMT+02:00 Miguel Angel <span dir="ltr"><<a href="mailto:miguelangel@ajo.es" target="_blank">miguelangel@ajo.es</a>></span>:<div>
<div><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Benoit,<div><br></div><div> A manual virsh migration should fail, because the</div><div>network ports are not migrated to the destination host.</div>
<div><br></div><div> You must investigate on the authentication problem itself,</div>
<div>and let nova handle all the underlying API calls which should happen...</div><div><br></div><div> May be it's worth setting nova.conf to debug=True</div><div><br></div><div><br></div></div><div class="gmail_extra">
<br clear="all"><div><div dir="ltr"><div>---</div><div>irc: ajo / mangelajo</div>Miguel Angel Ajo Pelayo<br><a href="tel:%2B34%20636%2052%2025%2069" value="+34636522569" target="_blank">+34 636 52 25 69</a><br>skype: ajoajoajo</div>
</div>
<br><br><div class="gmail_quote">2014-07-11 14:41 GMT+02:00 Benoit ML <span dir="ltr"><<a href="mailto:ben42ml@gmail.com" target="_blank">ben42ml@gmail.com</a>></span>:<div><div><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div>Hello,<br><br>cat /etc/redhat-release <br>CentOS Linux release 7 (Rebuilt from: RHEL 7.0)<br><br><br></div>Regards,<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-07-11 13:40 GMT+02:00 Boris Derzhavets <span dir="ltr"><<a href="mailto:bderzhavets@hotmail.com" target="_blank">bderzhavets@hotmail.com</a>></span>:<div>
<div><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr">Could you please post /etc/redhat-release.<br><br>Boris.<br><br><div><hr>Date: Fri, 11 Jul 2014 11:57:12 +0200<br>From: <a href="mailto:ben42ml@gmail.com" target="_blank">ben42ml@gmail.com</a><br>To: <a href="mailto:rdo-list@redhat.com" target="_blank">rdo-list@redhat.com</a><br>
Subject: [Rdo-list] Icehouse multi-node - Centos7 - live migration failed because of "network qbr no such device"<div><div><br><br><div dir="ltr">Hello,<div><br></div><div>I'm working on a multi-node setup of openstack Icehouse using centos7.</div>
<div>Well i have : </div><div> - one controllor node with all server services thing stuff</div><div>
- one network node with openvswitch agent, l3-agent, dhcp-agent</div><div> - two compute node with nova-compute and neutron-openvswitch</div><div> - one storage nfs node</div><div><br></div><div>NetworkManager is deleted on compute nodes and network node.</div>
<div><br></div><div>My network use is configured to use vxlan. I can create VM, tenant-network, external-network, routeur, assign floating-ip to VM, push ssh-key into VM, create volume from glance image, etc... Evrything is conected and reacheable. Pretty cool :) </div>
<div><br></div><div>But when i try to migrate VM things go wrong ... I have configured nova, libvirtd and qemu to use migration through libvirt-tcp.</div><div>I have create and exchanged ssh-key for nova user on all node. I have verified userid and groupid of nova.</div>
<div><br></div><div>Well nova-compute log, on the target compute node, : </div><div><div>2014-07-11 11:45:02.749 6984 TRACE nova.compute.manager [instance: a5326fe1-4faa-4347-ba29-159fce26a85c] RemoteError: Remote error: Unauthorized {"error": {"m</div>
<div>essage": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}</div></div><div><br></div><div><br></div><div>So well after searching a lots in all logs, i have fount that i cant simply migration VM between compute node with a simple virsh :</div>
<div><div>virsh migrate instance-00000084 qemu+tcp://<dest>/system<br></div></div><div><br></div><div>The error is : </div><div><div>erreur :Cannot get interface MTU on 'qbr3ca65809-05': No such device</div>
</div><div><br></div><div>Well when i look on the source hyperviseur the bridge "qbr3ca65809" exists and have a network tap device. And moreover i manually create qbr3ca65809 on the target hypervisor, virsh migrate succed !</div>
<div><br></div><div>Can you help me plz ?</div><div>What can i do wrong ? Perhpas neutron must create the bridge before migration but didnt for a mis configuration ? </div><div><br></div><div>Plz ask anything you need !</div>
<div><br></div><div>Thank you in advance.</div><div><br></div><div><br></div><div>The full nova-compute log attached.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>Regards,<br>
<div><div><br></div>-- <br>--<br>Benoit<br>
</div></div></div>
<br></div></div>_______________________________________________
Rdo-list mailing list
<a href="mailto:Rdo-list@redhat.com" target="_blank">Rdo-list@redhat.com</a>
<a href="https://www.redhat.com/mailman/listinfo/rdo-list" target="_blank">https://www.redhat.com/mailman/listinfo/rdo-list</a></div> </div></div>
</blockquote></div></div></div><span><font color="#888888"><br><br clear="all"><br>-- <br>--<br>Benoit<br>
</font></span></div>
<br>_______________________________________________<br>
Rdo-list mailing list<br>
<a href="mailto:Rdo-list@redhat.com" target="_blank">Rdo-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/rdo-list" target="_blank">https://www.redhat.com/mailman/listinfo/rdo-list</a><br>
<br></blockquote></div></div></div><br></div>
</blockquote></div></div></div><span><font color="#888888"><br><br clear="all"><br>-- <br>--<br>Benoit<br>
</font></span></div>
<br>_______________________________________________<br>
Rdo-list mailing list<br>
<a href="mailto:Rdo-list@redhat.com" target="_blank">Rdo-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/rdo-list" target="_blank">https://www.redhat.com/mailman/listinfo/rdo-list</a><br>
<br></blockquote></div><br></div></div></div></div>
</blockquote></div></div></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br>--<br>Benoit<br>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>--<br>Benoit<br>
</div>