[Rdo-list] neutron server failing to start

Hello everyone: I'm attempting to build up an openstack install using the rdo packages after playing around with packstack. Up until now everything has been going smoothly. After installing the dashboard I attempted to login. My password was accepted but I had the "somethings wrong" error page pop up. httpd logs showed: [error] ConnectionFailed: Connection to neutron failed: Maximum attempts reached I tried running neutron net-list and received the same message. service neutron-server shows: neutron dead but pid file exists I started the server and it showed okay. However I was still receiving the same errors. I looked at the logs for neutron server and found the following: 2014-08-06 17:40:47.866 22869 INFO neutron.common.config [-] Logging enabled! 2014-08-06 17:40:47.872 22869 INFO neutron.common.config [-] Config paste file: /usr/share/neutron/api-paste.ini 2014-08-06 17:40:47.931 22869 INFO neutron.manager [-] Loading core plugin: neutron.plugins.openvswitch.ovs_neutron_plugin.OVSNeutronPluginV2 2014-08-06 17:40:48.044 22869 WARNING neutron.openstack.common.db.sqlalchemy.session [-] This application has not enabled MySQL traditional mode, which means silent data corruption may occur. Please encourage the application developers to enable this mode. 2014-08-06 17:40:48.067 22869 INFO neutron.plugins.openvswitch.ovs_neutron_plugin [-] Network VLAN ranges: {} 2014-08-06 17:40:48.157 22869 INFO neutron.plugins.openvswitch.ovs_neutron_plugin [-] Tunnel ID ranges: [(1, 1000)] 2014-08-06 17:40:48.190 22869 ERROR neutron.common.config [-] Unable to load neutron from configuration file /usr/share/neutron/api-paste.ini. 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config Traceback (most recent call last): 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/common/config.py", line 170, in load_paste_app 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config app = deploy.loadapp("config:%s" % config_path, name=app_name) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return loadobj(APP, uri, name=name, **kw) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return context.create() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 710, in create 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return self.object_type.invoke(self) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config **context.local_conf) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/util.py", line 56, in fix_call 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config val = callable(*args, **kw) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/urlmap.py", line 25, in urlmap_factory 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config app = loader.get_app(app_name, global_conf=global_conf) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config name=name, global_conf=global_conf).create() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 710, in create 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return self.object_type.invoke(self) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config **context.local_conf) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/util.py", line 56, in fix_call 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config val = callable(*args, **kw) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/auth.py", line 69, in pipeline_factory 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config app = loader.get_app(pipeline[-1]) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config name=name, global_conf=global_conf).create() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 710, in create 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return self.object_type.invoke(self) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return fix_call(context.object, context.global_conf, **context.local_conf) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/util.py", line 56, in fix_call 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config val = callable(*args, **kw) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/api/v2/router.py", line 71, in factory 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return cls(**local_config) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/api/v2/router.py", line 75, in __init__ 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config plugin = manager.NeutronManager.get_plugin() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/manager.py", line 211, in get_plugin 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return cls.get_instance().plugin 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/manager.py", line 206, in get_instance 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config cls._create_instance() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py", line 249, in inner 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return f(*args, **kwargs) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__ 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config self.gen.throw(type, value, traceback) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py", line 212, in lock 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config yield sem 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py", line 249, in inner 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return f(*args, **kwargs) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/manager.py", line 200, in _create_instance 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config cls._instance = cls() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/manager.py", line 112, in __init__ 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config plugin_provider) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/manager.py", line 140, in _get_plugin_instance 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return plugin_class() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/ovs_neutron_plugin.py", line 325, in __init__ 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config self.setup_rpc() 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/ovs_neutron_plugin.py", line 337, in setup_rpc 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config self.conn = rpc.create_connection(new=True) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/__init__.py", line 89, in create_connection 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config return _get_impl().create_connection(CONF, new=new) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/__init__.py", line 274, in _get_impl 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config _RPCIMPL = importutils.import_module(impl) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/openstack/common/importutils.py", line 57, in import_module 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config __import__(import_str) 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config ImportError: No module named rabbit 2014-08-06 17:40:48.190 22869 TRACE neutron.common.config 2014-08-06 17:40:48.193 22869 ERROR neutron.service [-] Error occurred: trying old api-paste.ini. 2014-08-06 17:40:48.193 22869 TRACE neutron.service Traceback (most recent call last): 2014-08-06 17:40:48.193 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/service.py", line 106, in serve_wsgi 2014-08-06 17:40:48.193 22869 TRACE neutron.service service.start() 2014-08-06 17:40:48.193 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/service.py", line 75, in start 2014-08-06 17:40:48.193 22869 TRACE neutron.service self.wsgi_app = _run_wsgi(self.app_name) 2014-08-06 17:40:48.193 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/service.py", line 175, in _run_wsgi 2014-08-06 17:40:48.193 22869 TRACE neutron.service app = config.load_paste_app(app_name) 2014-08-06 17:40:48.193 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/common/config.py", line 177, in load_paste_app 2014-08-06 17:40:48.193 22869 TRACE neutron.service raise RuntimeError(msg) 2014-08-06 17:40:48.193 22869 TRACE neutron.service RuntimeError: Unable to load neutron from configuration file /usr/share/neutron/api-paste.ini. 2014-08-06 17:40:48.193 22869 TRACE neutron.service 2014-08-06 17:40:48.194 22869 INFO neutron.common.config [-] Logging enabled! 2014-08-06 17:40:48.202 22869 INFO neutron.common.config [-] Config paste file: /usr/share/neutron/api-paste.ini 2014-08-06 17:40:48.202 22869 ERROR neutron.common.config [-] Unable to load quantum from configuration file /usr/share/neutron/api-paste.ini. 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config Traceback (most recent call last): 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/neutron/common/config.py", line 170, in load_paste_app 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config app = deploy.loadapp("config:%s" % config_path, name=app_name) 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config return loadobj(APP, uri, name=name, **kw) 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config global_conf=global_conf) 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config global_conf=global_conf) 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config return loader.get_context(object_type, name, global_conf) 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 408, in get_context 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config object_type, name=name) 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config File "/usr/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 587, in find_config_section 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config self.filename)) 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config LookupError: No section 'quantum' (prefixed by 'app' or 'application' or 'composite' or 'composit' or 'pipeline' or 'filter-app') found in config /usr/share/neutron/api-paste.ini 2014-08-06 17:40:48.202 22869 TRACE neutron.common.config 2014-08-06 17:40:48.203 22869 ERROR neutron.service [-] Unrecoverable error: please check log for details. 2014-08-06 17:40:48.203 22869 TRACE neutron.service Traceback (most recent call last): 2014-08-06 17:40:48.203 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/service.py", line 110, in serve_wsgi 2014-08-06 17:40:48.203 22869 TRACE neutron.service service.start() 2014-08-06 17:40:48.203 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/service.py", line 75, in start 2014-08-06 17:40:48.203 22869 TRACE neutron.service self.wsgi_app = _run_wsgi(self.app_name) 2014-08-06 17:40:48.203 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/service.py", line 175, in _run_wsgi 2014-08-06 17:40:48.203 22869 TRACE neutron.service app = config.load_paste_app(app_name) 2014-08-06 17:40:48.203 22869 TRACE neutron.service File "/usr/lib/python2.6/site-packages/neutron/common/config.py", line 177, in load_paste_app 2014-08-06 17:40:48.203 22869 TRACE neutron.service raise RuntimeError(msg) 2014-08-06 17:40:48.203 22869 TRACE neutron.service RuntimeError: Unable to load quantum from configuration file /usr/share/neutron/api-paste.ini. 2014-08-06 17:40:48.203 22869 TRACE neutron.service I googled for these errors and found http://lists.openstack.org/pipermail/openstack/2013-November/003464.html as the only similar (to my eyes) result. However the answer there was that python-keystoneclient wasn't installed. I have checked my controller and network nodes and they both have it installed. Does anyone have any idea what's going on? I'm not doing anything crazy config wise, just following the icehouse install guide. Thanks!