Hi, I followed the instructions at: http://openstack.redhat.com/Neutron-Quickstart http://openstack.redhat.com/Running_an_instance_with_Neutron I ran this on a RHEL 6.4 VM with latest updates from 6.4.z. I made sure to install the netns enabled kernel from RDO repos and reboot with that kernel before running packstack so that I didn't need to reboot the VM after the packstack install (and have br-ex disappear) The packstack install went without incident. And I was able to follow the launch an instance instructions. I noticed that the cirros VM took a long time to get to a login prompt on the VNC console. From looking at the console output it appears that the instance was waiting for a dhcp address.

Once the VNC session got me to a login prompt, I logged in (as the cirros user) and confirmed that eth0 did not have an ip address.

So, something networking related prevented the instance from getting an IP which of course makes ssh'ing into the instance via the floating ip later in the instructions not work properly. I tried ifup'ing eth0 and dhcp discovers were sent out but not responded to. One thing is that on the host running OpenStack services (the VM I ran packstack on), I don't see dnsmasq running except for the default libvirt network: > [admin@rdo-mgmt ~(keystone_demo)]$ ps -ef | grep dnsmas > nobody 1968 1 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq --strict-order --local=// --domain-needed --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --bind-interfaces --listen-address 192.168.122.1 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override --dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile --addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts So... that seems to be a problem :)

From my setup, interfaces info inside the DHCP n/w namespace:

Just to confirm, I am running the right kernel: > [root@rdo-mgmt log(keystone_demo)]# uname -a > Linux rdo-mgmt 2.6.32-358.114.1.openstack.el6.x86_64 #1 SMP Wed Jul 3 02:11:25 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux > [root@rdo-mgmt log(keystone_demo)]# rpm -q iproute kernel > iproute-2.6.32-23.el6_4.netns.1.x86_64 > kernel-2.6.32-358.114.1.openstack.el6.x86_64 >From quantum server.log: > 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: Error opening certificate file /var/lib/quantum/keystone-signing/signing_cert.pem > 140222780139336:error:02001002:system library:fopen:No such file or directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/signing_cert.pem','r') > 140222780139336:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:129: > > 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: Error loading file /var/lib/quantum/keystone-signing/cacert.pem > 140279285741384:error:02001002:system library:fopen:No such file or directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/cacert.pem','r') > 140279285741384:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:129: > 140279285741384:error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib:by_file.c:279:

>From quantum dhcp-agent.log: > 2013-08-04 09:08:05 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ > data = self._dataqueue.get(timeout=self._timeout) > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get > return waiter.wait() > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait > return get_hub().switch() > File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch > return self.greenlet.switch() > Empty > 2013-08-04 09:08:05 ERROR [quantum.agent.dhcp_agent] Failed reporting state! > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 702, in _report_state > self.agent_state) > File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line 66, in report_state > topic=self.topic) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call > return rpc.call(context, self._get_topic(topic), msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call > return _get_impl().call(CONF, context, topic, msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call > rpc_amqp.get_connection_pool(conf, Connection)) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call > rv = list(rv) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ > raise rpc_common.Timeout() > Timeout: Timeout while waiting on RPC response. > 2013-08-04 09:08:05 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.853869 sec > 2013-08-04 09:08:06 INFO [quantum.agent.dhcp_agent] Synchronizing state > 2013-08-04 09:32:34 ERROR [quantum.agent.dhcp_agent] Unable to enable dhcp. > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 131, in call_driver > getattr(driver, action)() > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", line 124, in enable > reuse_existing=True) > File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 554, in setup > namespace=namespace) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug > ns_dev.link.set_address(mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address > self._as_root('set', self.name, 'address', mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root > kwargs.get('use_root_namespace', False)) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root > namespace) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute > root_helper=root_helper) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute > raise RuntimeError(m) > RuntimeError: > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', 'link', 'set', 'tap07d8cc77-fc', 'address', 'fa:16:3e:da:66:28'] > Exit code: 2 > Stdout: '' > Stderr: 'RTNETLINK answers: Device or resource busy\n' > 2013-08-04 09:32:36 INFO [quantum.agent.dhcp_agent] Synchronizing state > 2013-08-04 09:32:41 ERROR [quantum.agent.dhcp_agent] Unable to enable dhcp. > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 131, in call_driver > getattr(driver, action)() > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", line 124, in enable > reuse_existing=True) > File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 554, in setup > namespace=namespace) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug > ns_dev.link.set_address(mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address > self._as_root('set', self.name, 'address', mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root > kwargs.get('use_root_namespace', False)) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root > namespace) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute > root_helper=root_helper) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute > raise RuntimeError(m) The RTNETLINK errors just repeat indefinitely >From openvswitch-agent.log: > 2013-08-04 09:08:29 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ > data = self._dataqueue.get(timeout=self._timeout) > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get > return waiter.wait() > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait > return get_hub().switch() > File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch > return self.greenlet.switch() > Empty > 2013-08-04 09:08:29 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 201, in _report_state > self.agent_state) > File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line 66, in report_state > topic=self.topic) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call > return rpc.call(context, self._get_topic(topic), msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call > return _get_impl().call(CONF, context, topic, msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call > rpc_amqp.get_connection_pool(conf, Connection)) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call > rv = list(rv) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ > raise rpc_common.Timeout() > Timeout: Timeout while waiting on RPC response. Do we have a race condition wrt various Quantum agents connecting to the qpid bus that is just generating initial qpid connection error messages that can be safely ignored?

If so, is there any way we can clean this up? >From l3-agent.log: > 2013-08-04 09:08:06 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ > data = self._dataqueue.get(timeout=self._timeout) > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get > return waiter.wait() > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait > return get_hub().switch() > File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch > return self.greenlet.switch() > Empty > 2013-08-04 09:08:06 ERROR [quantum.agent.l3_agent] Failed reporting state! > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 723, in _report_state > self.agent_state) > File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line 66, in report_state > topic=self.topic) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call > return rpc.call(context, self._get_topic(topic), msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call > return _get_impl().call(CONF, context, topic, msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call > rpc_amqp.get_connection_pool(conf, Connection)) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call > rv = list(rv) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ > raise rpc_common.Timeout() > Timeout: Timeout while waiting on RPC response. > 2013-08-04 09:08:06 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.554131 sec > 2013-08-04 09:08:10 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ > data = self._dataqueue.get(timeout=self._timeout) > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get > return waiter.wait() > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait > return get_hub().switch() > File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch > return self.greenlet.switch() > Empty > 2013-08-04 09:08:10 ERROR [quantum.agent.l3_agent] Failed synchronizing routers > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 637, in _sync_routers_task > context, router_id) > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 77, in get_routers > topic=self.topic) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call > return rpc.call(context, self._get_topic(topic), msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call > return _get_impl().call(CONF, context, topic, msg, timeout) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call > rpc_amqp.get_connection_pool(conf, Connection)) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call > rv = list(rv) > File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ > raise rpc_common.Timeout() > Timeout: Timeout while waiting on RPC response. > 2013-08-04 09:08:10 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 20.022704 sec > 2013-08-04 09:11:33 ERROR [quantum.agent.l3_agent] Failed synchronizing routers > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 638, in _sync_routers_task > self._process_routers(routers, all_routers=True) > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 621, in _process_routers > self.process_router(ri) > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 319, in process_router > self.external_gateway_added(ri, ex_gw_port, internal_cidrs) > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 410, in external_gateway_added > prefix=EXTERNAL_DEV_PREFIX) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug > ns_dev.link.set_address(mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address > self._as_root('set', self.name, 'address', mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root > kwargs.get('use_root_namespace', False)) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root > namespace) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute > root_helper=root_helper) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute > raise RuntimeError(m) > RuntimeError: > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', 'link', 'set', 'qg-46ed452c-5e', 'address', 'fa:16:3e:e7:d8:30'] > Exit code: 2 > Stdout: '' > Stderr: 'RTNETLINK answers: Device or resource busy\n' > 2013-08-04 09:12:11 ERROR [quantum.agent.l3_agent] Failed synchronizing routers > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 638, in _sync_routers_task > self._process_routers(routers, all_routers=True) > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 621, in _process_routers > self.process_router(ri) > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 319, in process_router > self.external_gateway_added(ri, ex_gw_port, internal_cidrs) > File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 410, in external_gateway_added > prefix=EXTERNAL_DEV_PREFIX) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug > ns_dev.link.set_address(mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address > self._as_root('set', self.name, 'address', mac_address) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root > kwargs.get('use_root_namespace', False)) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root > namespace) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute > root_helper=root_helper) > File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute > raise RuntimeError(m) Same qpid connection issue, which I'm assuming can just be ignored at this point. But also similar device busy errors with creating the namespace for the l2 agent

It appears that the issue with both the l2 agent and the dhcp agent that the namespace can't be created, which causes both of them to fail.

Anyone have any thoughts on what to look at next here? Perry

On 08/04/2013 05:47 PM, Kashyap Chamarthy wrote: >>> 2013-08-04 09:08:05 WARNING [quantum.openstack.common.loopingcall] >>> task run outlasted interval by 56.853869 sec >>> 2013-08-04 09:08:06 INFO [quantum.agent.dhcp_agent] >>> Synchronizing state >>> 2013-08-04 09:32:34 ERROR [quantum.agent.dhcp_agent] Unable to >>> enable dhcp. >>> Traceback (most recent call last): >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line >>> 131, in call_driver >>> getattr(driver, action)() >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", line >>> 124, in enable >>> reuse_existing=True) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line >>> 554, in setup >>> namespace=namespace) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", >>> line 181, in plug >>> ns_dev.link.set_address(mac_address) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>> line 180, in set_address >>> self._as_root('set', self.name, 'address', mac_address) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>> line 167, in _as_root >>> kwargs.get('use_root_namespace', False)) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>> line 47, in _as_root >>> namespace) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>> line 58, in _execute >>> root_helper=root_helper) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", >>> line 61, in execute >>> raise RuntimeError(m) >>> RuntimeError: >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ip', 'link', 'set', 'tap07d8cc77-fc', 'address', 'fa:16:3e:da:66:28'] >>> Exit code: 2 >>> Stdout: '' >>> Stderr: 'RTNETLINK answers: Device or resource busy\n' Quantum attempts to change the MAC address while the link is up. The live MAC address change feature is not supported in the openstack kernel at this point. We can attempt a backport of the feature to the openstack kernel and enable it for tap and veth devices or we modify quantum to bring down the interface before changing the mac address and bring it up again afterwards.

On 08/05/2013 04:27 AM, Thomas Graf wrote: > On 08/04/2013 05:47 PM, Kashyap Chamarthy wrote: >>>> 2013-08-04 09:08:05 WARNING [quantum.openstack.common.loopingcall] >>>> task run outlasted interval by 56.853869 sec >>>> 2013-08-04 09:08:06 INFO [quantum.agent.dhcp_agent] >>>> Synchronizing state >>>> 2013-08-04 09:32:34 ERROR [quantum.agent.dhcp_agent] Unable to >>>> enable dhcp. >>>> Traceback (most recent call last): >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line >>>> 131, in call_driver >>>> getattr(driver, action)() >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", line >>>> 124, in enable >>>> reuse_existing=True) >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line >>>> 554, in setup >>>> namespace=namespace) >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", >>>> line 181, in plug >>>> ns_dev.link.set_address(mac_address) >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>>> line 180, in set_address >>>> self._as_root('set', self.name, 'address', mac_address) >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>>> line 167, in _as_root >>>> kwargs.get('use_root_namespace', False)) >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>>> line 47, in _as_root >>>> namespace) >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", >>>> line 58, in _execute >>>> root_helper=root_helper) >>>> File >>>> "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", >>>> line 61, in execute >>>> raise RuntimeError(m) >>>> RuntimeError: >>>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>>> 'ip', 'link', 'set', 'tap07d8cc77-fc', 'address', 'fa:16:3e:da:66:28'] >>>> Exit code: 2 >>>> Stdout: '' >>>> Stderr: 'RTNETLINK answers: Device or resource busy\n' > > Quantum attempts to change the MAC address while the link is up. The > live MAC address change feature is not supported in the openstack > kernel at this point. > > We can attempt a backport of the feature to the openstack kernel and > enable it for tap and veth devices or we modify quantum to bring down > the interface before changing the mac address and bring it up again > afterwards. Thanks Thomas. Or perhaps we need a fix to Quantum itself to create the link with the proper MAC address to begin with rather than changing it in a second step? With the above error, I wonder if the Quantum Quickstart ever fully worked at all on either RHOS or RDO Grizzly? Terry, how did you work around the above issue when testing on RHOS?

> > >From quantum server.log: >> 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: Error opening certificate file /var/lib/quantum/keystone-signing/signing_cert.pem >> 140222780139336:error:02001002:system library:fopen:No such file or directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/signing_cert.pem','r') >> 140222780139336:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:129: >> >> 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: Error loading file /var/lib/quantum/keystone-signing/cacert.pem >> 140279285741384:error:02001002:system library:fopen:No such file or directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/cacert.pem','r') >> 140279285741384:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:129: >> 140279285741384:error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib:by_file.c:279: Do you have the PEM cert file that system call 'fopen' is trying to locate? $ file /var/lib/quantum/keystone-signing/cacert.pem

On 08/04/2013 11:27 AM, Perry Myers wrote: > Hi, > > I followed the instructions at: > http://openstack.redhat.com/Neutron-Quickstart > http://openstack.redhat.com/Running_an_instance_with_Neutron > > I ran this on a RHEL 6.4 VM with latest updates from 6.4.z. I made sure > to install the netns enabled kernel from RDO repos and reboot with that > kernel before running packstack so that I didn't need to reboot the VM > after the packstack install (and have br-ex disappear) > > The packstack install went without incident. And I was able to follow > the launch an instance instructions. > > I noticed that the cirros VM took a long time to get to a login prompt > on the VNC console. From looking at the console output it appears that > the instance was waiting for a dhcp address. > > Once the VNC session got me to a login prompt, I logged in (as the > cirros user) and confirmed that eth0 did not have an ip address. > > So, something networking related prevented the instance from getting an > IP which of course makes ssh'ing into the instance via the floating ip > later in the instructions not work properly. > > I tried ifup'ing eth0 and dhcp discovers were sent out but not responded to. > > One thing is that on the host running OpenStack services (the VM I ran > packstack on), I don't see dnsmasq running except for the default > libvirt network: > >> [admin@rdo-mgmt ~(keystone_demo)]$ ps -ef | grep dnsmas >> nobody 1968 1 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq --strict-order --local=// --domain-needed --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --bind-interfaces --listen-address 192.168.122.1 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override --dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile --addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts > > So... that seems to be a problem :) > > Just to confirm, I am running the right kernel: >> [root@rdo-mgmt log(keystone_demo)]# uname -a >> Linux rdo-mgmt 2.6.32-358.114.1.openstack.el6.x86_64 #1 SMP Wed Jul 3 02:11:25 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux > >> [root@rdo-mgmt log(keystone_demo)]# rpm -q iproute kernel >> iproute-2.6.32-23.el6_4.netns.1.x86_64 >> kernel-2.6.32-358.114.1.openstack.el6.x86_64 > > From quantum server.log: >> 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: Error opening certificate file /var/lib/quantum/keystone-signing/signing_cert.pem >> 140222780139336:error:02001002:system library:fopen:No such file or directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/signing_cert.pem','r') >> 140222780139336:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:129: >> >> 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: Error loading file /var/lib/quantum/keystone-signing/cacert.pem >> 140279285741384:error:02001002:system library:fopen:No such file or directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/cacert.pem','r') >> 140279285741384:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:129: >> 140279285741384:error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib:by_file.c:279: > > From quantum dhcp-agent.log: > >> 2013-08-04 09:08:05 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ >> data = self._dataqueue.get(timeout=self._timeout) >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get >> return waiter.wait() >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait >> return get_hub().switch() >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch >> return self.greenlet.switch() >> Empty >> 2013-08-04 09:08:05 ERROR [quantum.agent.dhcp_agent] Failed reporting state! >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 702, in _report_state >> self.agent_state) >> File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line 66, in report_state >> topic=self.topic) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call >> return rpc.call(context, self._get_topic(topic), msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call >> return _get_impl().call(CONF, context, topic, msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call >> rpc_amqp.get_connection_pool(conf, Connection)) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call >> rv = list(rv) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ >> raise rpc_common.Timeout() >> Timeout: Timeout while waiting on RPC response. >> 2013-08-04 09:08:05 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.853869 sec >> 2013-08-04 09:08:06 INFO [quantum.agent.dhcp_agent] Synchronizing state >> 2013-08-04 09:32:34 ERROR [quantum.agent.dhcp_agent] Unable to enable dhcp. >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 131, in call_driver >> getattr(driver, action)() >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", line 124, in enable >> reuse_existing=True) >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 554, in setup >> namespace=namespace) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug >> ns_dev.link.set_address(mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address >> self._as_root('set', self.name, 'address', mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root >> kwargs.get('use_root_namespace', False)) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root >> namespace) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute >> root_helper=root_helper) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute >> raise RuntimeError(m) >> RuntimeError: >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', 'link', 'set', 'tap07d8cc77-fc', 'address', 'fa:16:3e:da:66:28'] >> Exit code: 2 >> Stdout: '' >> Stderr: 'RTNETLINK answers: Device or resource busy\n' >> 2013-08-04 09:32:36 INFO [quantum.agent.dhcp_agent] Synchronizing state >> 2013-08-04 09:32:41 ERROR [quantum.agent.dhcp_agent] Unable to enable dhcp. >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 131, in call_driver >> getattr(driver, action)() >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", line 124, in enable >> reuse_existing=True) >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", line 554, in setup >> namespace=namespace) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug >> ns_dev.link.set_address(mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address >> self._as_root('set', self.name, 'address', mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root >> kwargs.get('use_root_namespace', False)) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root >> namespace) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute >> root_helper=root_helper) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute >> raise RuntimeError(m) > > The RTNETLINK errors just repeat indefinitely > > From openvswitch-agent.log: > >> 2013-08-04 09:08:29 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ >> data = self._dataqueue.get(timeout=self._timeout) >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get >> return waiter.wait() >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait >> return get_hub().switch() >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch >> return self.greenlet.switch() >> Empty >> 2013-08-04 09:08:29 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 201, in _report_state >> self.agent_state) >> File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line 66, in report_state >> topic=self.topic) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call >> return rpc.call(context, self._get_topic(topic), msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call >> return _get_impl().call(CONF, context, topic, msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call >> rpc_amqp.get_connection_pool(conf, Connection)) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call >> rv = list(rv) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ >> raise rpc_common.Timeout() >> Timeout: Timeout while waiting on RPC response. > > Do we have a race condition wrt various Quantum agents connecting to the > qpid bus that is just generating initial qpid connection error messages > that can be safely ignored? > > If so, is there any way we can clean this up? > > From l3-agent.log: > >> 2013-08-04 09:08:06 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ >> data = self._dataqueue.get(timeout=self._timeout) >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get >> return waiter.wait() >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait >> return get_hub().switch() >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch >> return self.greenlet.switch() >> Empty >> 2013-08-04 09:08:06 ERROR [quantum.agent.l3_agent] Failed reporting state! >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 723, in _report_state >> self.agent_state) >> File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line 66, in report_state >> topic=self.topic) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call >> return rpc.call(context, self._get_topic(topic), msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call >> return _get_impl().call(CONF, context, topic, msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call >> rpc_amqp.get_connection_pool(conf, Connection)) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call >> rv = list(rv) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ >> raise rpc_common.Timeout() >> Timeout: Timeout while waiting on RPC response. >> 2013-08-04 09:08:06 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.554131 sec >> 2013-08-04 09:08:10 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ >> data = self._dataqueue.get(timeout=self._timeout) >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get >> return waiter.wait() >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait >> return get_hub().switch() >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch >> return self.greenlet.switch() >> Empty >> 2013-08-04 09:08:10 ERROR [quantum.agent.l3_agent] Failed synchronizing routers >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 637, in _sync_routers_task >> context, router_id) >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 77, in get_routers >> topic=self.topic) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call >> return rpc.call(context, self._get_topic(topic), msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call >> return _get_impl().call(CONF, context, topic, msg, timeout) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 611, in call >> rpc_amqp.get_connection_pool(conf, Connection)) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call >> rv = list(rv) >> File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ >> raise rpc_common.Timeout() >> Timeout: Timeout while waiting on RPC response. >> 2013-08-04 09:08:10 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 20.022704 sec >> 2013-08-04 09:11:33 ERROR [quantum.agent.l3_agent] Failed synchronizing routers >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 638, in _sync_routers_task >> self._process_routers(routers, all_routers=True) >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 621, in _process_routers >> self.process_router(ri) >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 319, in process_router >> self.external_gateway_added(ri, ex_gw_port, internal_cidrs) >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 410, in external_gateway_added >> prefix=EXTERNAL_DEV_PREFIX) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug >> ns_dev.link.set_address(mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address >> self._as_root('set', self.name, 'address', mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root >> kwargs.get('use_root_namespace', False)) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root >> namespace) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute >> root_helper=root_helper) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute >> raise RuntimeError(m) >> RuntimeError: >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', 'link', 'set', 'qg-46ed452c-5e', 'address', 'fa:16:3e:e7:d8:30'] >> Exit code: 2 >> Stdout: '' >> Stderr: 'RTNETLINK answers: Device or resource busy\n' >> 2013-08-04 09:12:11 ERROR [quantum.agent.l3_agent] Failed synchronizing routers >> Traceback (most recent call last): >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 638, in _sync_routers_task >> self._process_routers(routers, all_routers=True) >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 621, in _process_routers >> self.process_router(ri) >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 319, in process_router >> self.external_gateway_added(ri, ex_gw_port, internal_cidrs) >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", line 410, in external_gateway_added >> prefix=EXTERNAL_DEV_PREFIX) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", line 181, in plug >> ns_dev.link.set_address(mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 180, in set_address >> self._as_root('set', self.name, 'address', mac_address) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 167, in _as_root >> kwargs.get('use_root_namespace', False)) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 47, in _as_root >> namespace) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", line 58, in _execute >> root_helper=root_helper) >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", line 61, in execute >> raise RuntimeError(m) > > Same qpid connection issue, which I'm assuming can just be ignored at > this point. But also similar device busy errors with creating the > namespace for the l2 agent > > It appears that the issue with both the l2 agent and the dhcp agent that > the namespace can't be created, which causes both of them to fail. > > Anyone have any thoughts on what to look at next here? > > Perry I ran into these issues as well. I noticed that ovs_use_veth was commented out in dhcp_agent.ini and l3_agent.ini. I uncommented them and set them to True and restarted. The vm now has an IP address. I noticed something else peculiar though... the public network.. the one set as the gateway for the router has dhcp enabled. I'm not sure why we would do that.

Cheers, Brent

----- Original Message ----- > On 08/04/2013 11:27 AM, Perry Myers wrote: > > Hi, > > > > I followed the instructions at: > > http://openstack.redhat.com/Neutron-Quickstart > > http://openstack.redhat.com/Running_an_instance_with_Neutron > > > > I ran this on a RHEL 6.4 VM with latest updates from 6.4.z. I made sure > > to install the netns enabled kernel from RDO repos and reboot with that > > kernel before running packstack so that I didn't need to reboot the VM > > after the packstack install (and have br-ex disappear) > > > > The packstack install went without incident. And I was able to follow > > the launch an instance instructions. > > > > I noticed that the cirros VM took a long time to get to a login prompt > > on the VNC console. From looking at the console output it appears that > > the instance was waiting for a dhcp address. > > > > Once the VNC session got me to a login prompt, I logged in (as the > > cirros user) and confirmed that eth0 did not have an ip address. > > > > So, something networking related prevented the instance from getting an > > IP which of course makes ssh'ing into the instance via the floating ip > > later in the instructions not work properly. > > > > I tried ifup'ing eth0 and dhcp discovers were sent out but not responded > > to. > > > > One thing is that on the host running OpenStack services (the VM I ran > > packstack on), I don't see dnsmasq running except for the default > > libvirt network: > > > >> [admin@rdo-mgmt ~(keystone_demo)]$ ps -ef | grep dnsmas > >> nobody 1968 1 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq > >> --strict-order --local=// --domain-needed > >> --pid-file=/var/run/libvirt/network/default.pid --conf-file= > >> --except-interface lo --bind-interfaces --listen-address 192.168.122.1 > >> --dhcp-range 192.168.122.2,192.168.122.254 > >> --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases > >> --dhcp-lease-max=253 --dhcp-no-override > >> --dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile > >> --addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts > > > > So... that seems to be a problem :) > > > > Just to confirm, I am running the right kernel: > >> [root@rdo-mgmt log(keystone_demo)]# uname -a > >> Linux rdo-mgmt 2.6.32-358.114.1.openstack.el6.x86_64 #1 SMP Wed Jul 3 > >> 02:11:25 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux > > > >> [root@rdo-mgmt log(keystone_demo)]# rpm -q iproute kernel > >> iproute-2.6.32-23.el6_4.netns.1.x86_64 > >> kernel-2.6.32-358.114.1.openstack.el6.x86_64 > > > > From quantum server.log: > >> 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: > >> Error opening certificate file > >> /var/lib/quantum/keystone-signing/signing_cert.pem > >> 140222780139336:error:02001002:system library:fopen:No such file or > >> directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/signing_cert.pem','r') > >> 140222780139336:error:2006D080:BIO routines:BIO_new_file:no such > >> file:bss_file.c:129: > >> > >> 2013-08-04 09:10:48 ERROR [keystoneclient.common.cms] Verify error: > >> Error loading file /var/lib/quantum/keystone-signing/cacert.pem > >> 140279285741384:error:02001002:system library:fopen:No such file or > >> directory:bss_file.c:126:fopen('/var/lib/quantum/keystone-signing/cacert.pem','r') > >> 140279285741384:error:2006D080:BIO routines:BIO_new_file:no such > >> file:bss_file.c:129: > >> 140279285741384:error:0B084002:x509 certificate > >> routines:X509_load_cert_crl_file:system lib:by_file.c:279: > > > > From quantum dhcp-agent.log: > > > >> 2013-08-04 09:08:05 ERROR [quantum.openstack.common.rpc.amqp] Timed > >> out > >> waiting for RPC response. > >> Traceback (most recent call last): > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 495, in __iter__ > >> data = self._dataqueue.get(timeout=self._timeout) > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, > >> in > >> get > >> return waiter.wait() > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, > >> in > >> wait > >> return get_hub().switch() > >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line > >> 177, > >> in switch > >> return self.greenlet.switch() > >> Empty > >> 2013-08-04 09:08:05 ERROR [quantum.agent.dhcp_agent] Failed reporting > >> state! > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", > >> line 702, in _report_state > >> self.agent_state) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line > >> 66, > >> in report_state > >> topic=self.topic) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", > >> line 80, in call > >> return rpc.call(context, self._get_topic(topic), msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", > >> line 140, in call > >> return _get_impl().call(CONF, context, topic, msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", > >> line 611, in call > >> rpc_amqp.get_connection_pool(conf, Connection)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 614, in call > >> rv = list(rv) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 500, in __iter__ > >> raise rpc_common.Timeout() > >> Timeout: Timeout while waiting on RPC response. > >> 2013-08-04 09:08:05 WARNING [quantum.openstack.common.loopingcall] task > >> run outlasted interval by 56.853869 sec > >> 2013-08-04 09:08:06 INFO [quantum.agent.dhcp_agent] Synchronizing > >> state > >> 2013-08-04 09:32:34 ERROR [quantum.agent.dhcp_agent] Unable to enable > >> dhcp. > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", > >> line 131, in call_driver > >> getattr(driver, action)() > >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", > >> line 124, in enable > >> reuse_existing=True) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", > >> line 554, in setup > >> namespace=namespace) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", > >> line 181, in plug > >> ns_dev.link.set_address(mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 180, in set_address > >> self._as_root('set', self.name, 'address', mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 167, in _as_root > >> kwargs.get('use_root_namespace', False)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 47, in _as_root > >> namespace) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 58, in _execute > >> root_helper=root_helper) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", > >> line 61, in execute > >> raise RuntimeError(m) > >> RuntimeError: > >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > >> 'ip', > >> 'link', 'set', 'tap07d8cc77-fc', 'address', 'fa:16:3e:da:66:28'] > >> Exit code: 2 > >> Stdout: '' > >> Stderr: 'RTNETLINK answers: Device or resource busy\n' > >> 2013-08-04 09:32:36 INFO [quantum.agent.dhcp_agent] Synchronizing > >> state > >> 2013-08-04 09:32:41 ERROR [quantum.agent.dhcp_agent] Unable to enable > >> dhcp. > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", > >> line 131, in call_driver > >> getattr(driver, action)() > >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/dhcp.py", > >> line 124, in enable > >> reuse_existing=True) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/dhcp_agent.py", > >> line 554, in setup > >> namespace=namespace) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", > >> line 181, in plug > >> ns_dev.link.set_address(mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 180, in set_address > >> self._as_root('set', self.name, 'address', mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 167, in _as_root > >> kwargs.get('use_root_namespace', False)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 47, in _as_root > >> namespace) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 58, in _execute > >> root_helper=root_helper) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", > >> line 61, in execute > >> raise RuntimeError(m) > > > > The RTNETLINK errors just repeat indefinitely > > > > From openvswitch-agent.log: > > > >> 2013-08-04 09:08:29 ERROR [quantum.openstack.common.rpc.amqp] Timed > >> out > >> waiting for RPC response. > >> Traceback (most recent call last): > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 495, in __iter__ > >> data = self._dataqueue.get(timeout=self._timeout) > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, > >> in > >> get > >> return waiter.wait() > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, > >> in > >> wait > >> return get_hub().switch() > >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line > >> 177, > >> in switch > >> return self.greenlet.switch() > >> Empty > >> 2013-08-04 09:08:29 ERROR > >> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting > >> state! > >> Traceback (most recent call last): > >> File > >> "/usr/lib/python2.6/site-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", > >> line 201, in _report_state > >> self.agent_state) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line > >> 66, > >> in report_state > >> topic=self.topic) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", > >> line 80, in call > >> return rpc.call(context, self._get_topic(topic), msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", > >> line 140, in call > >> return _get_impl().call(CONF, context, topic, msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", > >> line 611, in call > >> rpc_amqp.get_connection_pool(conf, Connection)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 614, in call > >> rv = list(rv) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 500, in __iter__ > >> raise rpc_common.Timeout() > >> Timeout: Timeout while waiting on RPC response. > > > > Do we have a race condition wrt various Quantum agents connecting to the > > qpid bus that is just generating initial qpid connection error messages > > that can be safely ignored? > > > > If so, is there any way we can clean this up? > > > > From l3-agent.log: > > > >> 2013-08-04 09:08:06 ERROR [quantum.openstack.common.rpc.amqp] Timed > >> out > >> waiting for RPC response. > >> Traceback (most recent call last): > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 495, in __iter__ > >> data = self._dataqueue.get(timeout=self._timeout) > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, > >> in > >> get > >> return waiter.wait() > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, > >> in > >> wait > >> return get_hub().switch() > >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line > >> 177, > >> in switch > >> return self.greenlet.switch() > >> Empty > >> 2013-08-04 09:08:06 ERROR [quantum.agent.l3_agent] Failed reporting > >> state! > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 723, in _report_state > >> self.agent_state) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/rpc.py", line > >> 66, > >> in report_state > >> topic=self.topic) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", > >> line 80, in call > >> return rpc.call(context, self._get_topic(topic), msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", > >> line 140, in call > >> return _get_impl().call(CONF, context, topic, msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", > >> line 611, in call > >> rpc_amqp.get_connection_pool(conf, Connection)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 614, in call > >> rv = list(rv) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 500, in __iter__ > >> raise rpc_common.Timeout() > >> Timeout: Timeout while waiting on RPC response. > >> 2013-08-04 09:08:06 WARNING [quantum.openstack.common.loopingcall] task > >> run outlasted interval by 56.554131 sec > >> 2013-08-04 09:08:10 ERROR [quantum.openstack.common.rpc.amqp] Timed > >> out > >> waiting for RPC response. > >> Traceback (most recent call last): > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 495, in __iter__ > >> data = self._dataqueue.get(timeout=self._timeout) > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, > >> in > >> get > >> return waiter.wait() > >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, > >> in > >> wait > >> return get_hub().switch() > >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line > >> 177, > >> in switch > >> return self.greenlet.switch() > >> Empty > >> 2013-08-04 09:08:10 ERROR [quantum.agent.l3_agent] Failed > >> synchronizing > >> routers > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 637, in _sync_routers_task > >> context, router_id) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 77, in get_routers > >> topic=self.topic) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", > >> line 80, in call > >> return rpc.call(context, self._get_topic(topic), msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", > >> line 140, in call > >> return _get_impl().call(CONF, context, topic, msg, timeout) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", > >> line 611, in call > >> rpc_amqp.get_connection_pool(conf, Connection)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 614, in call > >> rv = list(rv) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > >> line 500, in __iter__ > >> raise rpc_common.Timeout() > >> Timeout: Timeout while waiting on RPC response. > >> 2013-08-04 09:08:10 WARNING [quantum.openstack.common.loopingcall] task > >> run outlasted interval by 20.022704 sec > >> 2013-08-04 09:11:33 ERROR [quantum.agent.l3_agent] Failed > >> synchronizing > >> routers > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 638, in _sync_routers_task > >> self._process_routers(routers, all_routers=True) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 621, in _process_routers > >> self.process_router(ri) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 319, in process_router > >> self.external_gateway_added(ri, ex_gw_port, internal_cidrs) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 410, in external_gateway_added > >> prefix=EXTERNAL_DEV_PREFIX) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", > >> line 181, in plug > >> ns_dev.link.set_address(mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 180, in set_address > >> self._as_root('set', self.name, 'address', mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 167, in _as_root > >> kwargs.get('use_root_namespace', False)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 47, in _as_root > >> namespace) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 58, in _execute > >> root_helper=root_helper) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", > >> line 61, in execute > >> raise RuntimeError(m) > >> RuntimeError: > >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > >> 'ip', > >> 'link', 'set', 'qg-46ed452c-5e', 'address', 'fa:16:3e:e7:d8:30'] > >> Exit code: 2 > >> Stdout: '' > >> Stderr: 'RTNETLINK answers: Device or resource busy\n' > >> 2013-08-04 09:12:11 ERROR [quantum.agent.l3_agent] Failed > >> synchronizing > >> routers > >> Traceback (most recent call last): > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 638, in _sync_routers_task > >> self._process_routers(routers, all_routers=True) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 621, in _process_routers > >> self.process_router(ri) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 319, in process_router > >> self.external_gateway_added(ri, ex_gw_port, internal_cidrs) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/l3_agent.py", > >> line > >> 410, in external_gateway_added > >> prefix=EXTERNAL_DEV_PREFIX) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/interface.py", > >> line 181, in plug > >> ns_dev.link.set_address(mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 180, in set_address > >> self._as_root('set', self.name, 'address', mac_address) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 167, in _as_root > >> kwargs.get('use_root_namespace', False)) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 47, in _as_root > >> namespace) > >> File > >> "/usr/lib/python2.6/site-packages/quantum/agent/linux/ip_lib.py", > >> line 58, in _execute > >> root_helper=root_helper) > >> File "/usr/lib/python2.6/site-packages/quantum/agent/linux/utils.py", > >> line 61, in execute > >> raise RuntimeError(m) > > > > Same qpid connection issue, which I'm assuming can just be ignored at > > this point. But also similar device busy errors with creating the > > namespace for the l2 agent > > > > It appears that the issue with both the l2 agent and the dhcp agent that > > the namespace can't be created, which causes both of them to fail. > > > > Anyone have any thoughts on what to look at next here? > > > > Perry > > I ran into these issues as well. I noticed that ovs_use_veth was > commented out in dhcp_agent.ini and l3_agent.ini. I uncommented them and > set them to True and restarted. The vm now has an IP address. > > I noticed something else peculiar though... the public network.. the one > set as the gateway for the router has dhcp enabled. I'm not sure why we > would do that. > > Cheers, > > Brent I re-ran an installation with a yum updated/rebooted RDO VM. packstack --allinone, then logged in as 'demo' to horizon and created a VM. Everything worked. Apparently I have the magic test VM. I don't hit the "dhcp enabled on public" issue because horizon passes the --nic net-id stuff to make sure that the VM doesn't attach an interface to the public network, thereby spawning the dhcp stuff that is enabled on it. None of my configs have ovs_use_veth and it *still* works perfectly on my machine--which it seems like shouldn't be possible. But it is. Yum repos installed: [root@rhel-6 yum.repos.d(keystone_admin)]# cat `grep -l enabled=1 /etc/yum.repos.d/*` [epel] name=Extra Packages for Enterprise Linux 6 - $basearch baseurl=http://192.168.122.1/epel #baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch failovermethod=priority enabled=1 gpgcheck=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6 [openstack-grizzly] name=OpenStack Grizzly Repository baseurl=http://192.168.122.1/openstack-grizzly #baseurl=http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/ enabled=1 skip_if_unavailable=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-RDO-Grizzly priority=98 [rhel-server] name=Red Hat Enterprise Linux $releasever - $basearch - Server baseurl=http://192.168.122.1/rhel-server enabled=1 gpgcheck=0 [rhel-ha] name=Red Hat Enterprise Linux $releasever - $basearch - HA baseurl=http://192.168.122.1/rhel-ha #baseurl=http://download.devel.redhat.com/composes/latest-RHEL6/6/Server/$basearch/os/HighAvailability enabled=1 gpgcheck=0 [rhel-rs] name=Red Hat Enterprise Linux $releasever - $basearch - RS baseurl=http://192.168.122.1/rhel-rs #baseurl=http://download.devel.redhat.com/composes/latest-RHEL6/6/Server/$basearch/os/ResilientStorage enabled=1 gpgcheck=0 [rhel-z] name=Red Hat Enterprise Linux $releasever - $basearch - Z-Stream baseurl=http://192.168.122.1/rhel-z #baseurl=http://download.lab.bos.redhat.com/rel-eng/repos/RHEL-6.4-Z/$basearch/ enabled=1 gpgcheck=0 _______________________________________________ Rdo-list mailing list Rdo-list(a)redhat.com https://www.redhat.com/mailman/listinfo/rdo-list

I ran into these issues as well. I noticed that ovs_use_veth was commented out in dhcp_agent.ini and l3_agent.ini. I uncommented them and set them to True and restarted. The vm now has an IP address. I noticed something else peculiar though... the public network.. the one set as the gateway for the router has dhcp enabled. I'm not sure why we would do that.

On 08/04/2013 09:57 AM, Perry Myers wrote: > Hi, > > I followed the instructions at: > http://openstack.redhat.com/Neutron-Quickstart > http://openstack.redhat.com/Running_an_instance_with_Neutron > > I ran this on a RHEL 6.4 VM with latest updates from 6.4.z. I made sure > to install the netns enabled kernel from RDO repos and reboot with that > kernel before running packstack so that I didn't need to reboot the VM > after the packstack install (and have br-ex disappear) > > The packstack install went without incident. And I was able to follow > the launch an instance instructions. Ok, retried this but took advice from folks on this thread. Since l3 agent and dhcp agent in RDO are not right (they comment out ovs_use_veth=True and veths are required for the netns support in RHEL kernels) marun summarized this nicely: "if ovs_use_veth is set to false, a regular interface and an internal ovs port will be used, and the regular interface will be moved to a namespace during setup. if ovs_use_veth is set to true, a veth pair will be used with one endpoint created in the namespace. it is a limitation of rhel's netns implementation that requires the second approach, as virtual interfaces can only be created in namespaces, not moved post-creation." With manually enabling ovs_use_veth=True for l3 and dhcp agents, I was able to get cirros VM to get an ip address on launching. What doesn't work now is pinging/sshing to the floating ip address from the host (which is itself a VM) Yes, I did open those ports in the default security group, and I also made sure the instance was launched with the default security group. But that being said, I wanted to check the logs to see if some of the previous errors went away. dhcp-agent and l3 agent logs look clean now (aside from the amqp initial connection errors) My next test will be to run this exact same scenario but with NetworkManager disabled.

[admin@rdo-mgmt ~(keystone_demo)]$ nova secgroup-list-rules default +-------------+-----------+---------+-----------+--------------+ | IP Protocol | From Port | To Port | IP Range | Source Group | +-------------+-----------+---------+-----------+--------------+ | | | | | default | | | | | | default | | icmp | -1 | -1 | 0.0.0.0/0 | | | tcp | 22 | 22 | 0.0.0.0/0 | | +-------------+-----------+---------+-----------+--------------+

[admin@rdo-mgmt ~(keystone_demo)]$ sudo ovs-vsctl show 25588688-af82-4bc9-b053-8009d2718738 Bridge br-int Port "tap1582253a-01" Interface "tap1582253a-01" Port "qr-1582253a-01" tag: 1 Interface "qr-1582253a-01" type: internal Port "tapdce36595-4c" tag: 1 Interface "tapdce36595-4c" Port br-int Interface br-int type: internal Port "qvocd93d0e2-69" tag: 1 Interface "qvocd93d0e2-69" Port "tap99bc4804-f3" tag: 2 Interface "tap99bc4804-f3" Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-1448e7df-47" Interface "qg-1448e7df-47" type: internal Port "tap1448e7df-47" Interface "tap1448e7df-47" ovs_version: "1.10.0"

2013-08-06 01:39:32 WARNING [quantum.db.agentschedulers_db] Fail scheduling network {'status': u'ACTIVE', 'subnets': [u'585ec59b-005d-4460-a094-5394be2bb3a1'], 'name': u'private', 'provider:physical_network': None, 'admin_state_up': True, 'tenant_id': u'd297494482aa44ebb30243f624f9d5fc', 'provider:network_type': u'local', 'router:external': False, 'shared': False, 'id': u'7878056e-b4eb-4d26-a711-95ced35e7f98', 'provider:segmentation_id': None}

On 08/04/2013 09:57 AM, Perry Myers wrote: > Hi, > > I followed the instructions at: > http://openstack.redhat.com/Neutron-Quickstart > http://openstack.redhat.com/Running_an_instance_with_Neutron Ok, I wanted to close the loop on this thread with some things I found. It appears that the issues I was seeing had to do specifically with the 114.openstack kernel in the RDO Grizzly repos. This kernel had issues with being able to change MAC addresses with an interface that is in the up state. A new kernel was uploaded today 114.openstack.gre.2 and rkukura sent an announcement out about it, as it enables gre tunnel support. A side effect of this new kernel is that it fixed the bugs with changing MAC addresses on interfaces that are up. So, with this new kernel, I'm now able to successfully get RDO Grizzly on RHEL 6.4 with the 114.openstack.gre.2 kernel running. I can boot a Cirros VM, get an ip address, get external connectivity from the VM and also ssh/ping it via the floating IP. So... all in all, success! Now, the next step is doing a multi host setup. This is where the new support for gre tunnels will be handy, since moving from 1 to many nodes requires either VLAN support or tunnels, and since I don't have an intelligent switch, gre tunnels it is.