[Rdo-list] [Neutron] Fwd: Icehouse with Neutron and ML2 on RHEL
Steve Gordon
sgordon at redhat.com
Tue Mar 4 17:20:45 UTC 2014
Hi all,
I got some feedback about some issues with Icehouse RDO packages, specifically the Neutron ones and the way they handle (or don't handle) the database. Are these issues we are aware of/tracking? Matt is using the EL6 builds at the moment. We are hanging out in #openstack-docs on Freenode if you need any more info from Matt (Sam-I-Am on IRC).
Thanks,
Steve
----- Forwarded Message -----
> FYI...
>
> ---------- Forwarded message ----------
> From: Phil Hopkins <phil.hopkins at rackspace.com>
> Date: Sat, Mar 1, 2014 at 8:13 PM
> Subject: Re: Icehouse with Neutron and ML2 on RHEL
> To: Matt Kassawara <mkassawara at gmail.com>
> Cc: Carl Perry <edolnx at gmail.com>, Edgar Magana <emagana at plumgrid.com>,
> Nick Chase <nchase at mirantis.com>
>
>
> Very interesting, something is broke bad there. There should be a new
> build of the packages this week. Lets hope that fixes this.problem.
>
> Phil
>
> Sent from my iPhone
>
> On Mar 1, 2014, at 8:29 PM, "Matt Kassawara" <mkassawara at gmail.com> wrote:
>
> The following tables appear after starting neutron-server with a fresh
> database:
>
> mysql> show tables;
> +---------------------+
> | Tables_in_neutron |
> +---------------------+
> | agents |
> | ml2_gre_allocations |
> | ml2_gre_endpoints |
> | networks |
> | quotas |
> | securitygroups |
> +---------------------+
> 6 rows in set (0.00 sec)
>
> The log shows plenty of errors regarding missing database information. I'm
> particularly curious about this one:
>
> 2014-03-01 18:58:55.133 2877 INFO neutron.db.api [-] Database registration
> exception: (OperationalError) (1005, "Can't create table
> 'neutron.networkdhcpagentbindings' (errno: 150)") '\nCREATE TABLE
> networkdhcpagentbindings (\n\tnetwork_id VARCHAR(36) NOT NULL,
> \n\tdhcp_agent_id VARCHAR(36) NOT NULL, \n\tPRIMARY KEY (network_id,
> dhcp_agent_id), \n\tFOREIGN KEY(network_id) REFERENCES networks (id) ON
> DELETE CASCADE, \n\tFOREIGN KEY(dhcp_agent_id) REFERENCES agents (id) ON
> DELETE CASCADE\n)ENGINE=InnoDB\n\n' ()
>
> Interestingly, trying this SQL command manually also fails:
>
> mysql> create table networkdhcpagentbindings (network_id varchar(36) not
> null, dhcp_agent_id varchar(36) not null, primary key (network_id,
> dhcp_agent_id), foreign key(network_id) references networks (id) on delete
> cascade, foreign key(dhcp_agent_id) references agents (id) on delete
> cascade ) engine=InnoDB;
> ERROR 1005 (HY000): Can't create table 'neutron.networkdhcpagentbindings'
> (errno: 150)
>
> However, it works without "engine=InnoDB" appended at the end which leads
> me to some sort of referential integrity problem... or I'm mistyping
> something. :)
>
> mysql> show engine innodb status;
> LATEST FOREIGN KEY ERROR
> ------------------------
> 140301 19:24:40 Error in foreign key constraint of table
> neutron/networkdhcpagentbindings:
> foreign key(dhcp_agent_id) references agents (id) on delete cascade)
> engine=InnoDB:
> Cannot resolve table name close to:
> (id) on delete cascade) engine=InnoDB
>
> In comparison, here's the neutron database tables from my functional
> Ubuntu installation:
>
> mysql> show tables;
> +---------------------------+
> | Tables_in_neutron |
> +---------------------------+
> | agents |
> | allowedaddresspairs |
> | dnsnameservers |
> | externalnetworks |
> | extradhcpopts |
> | floatingips |
> | ipallocationpools |
> | ipallocations |
> | ipavailabilityranges |
> | ml2_gre_allocations |
> | ml2_gre_endpoints |
> | ml2_network_segments |
> | ml2_port_bindings |
> | networkdhcpagentbindings |
> | networks |
> | ports |
> | quotas |
> | routerl3agentbindings |
> | routerroutes |
> | routers |
> | securitygroupportbindings |
> | securitygrouprules |
> | securitygroups |
> | subnetroutes |
> | subnets |
> +---------------------------+
> 25 rows in set (0.00 sec)
>
>
>
> On Sat, Mar 1, 2014 at 6:46 PM, Phil Hopkins
> <phil.hopkins at rackspace.com>wrote:
>
> > I was thinking nova not neutron. Try dropping the neutron database,
> > recreate it and restart the neutron-server service.
> >
> > That should rebuild the tables. Check and see if they are created. If so
> > then see if the error reoccurs.
> >
> > Also see if any errors show up when the neutron-server service is
> > restarted. Maybe it is having trouble creating the database tables.
> >
> > Phil
> >
> > Sent from my iPhone
> >
> > On Mar 1, 2014, at 7:00 PM, "Matt Kassawara" <mkassawara at gmail.com> wrote:
> >
> > I did that a few times. Unlike the other similar utilities,
> > neutron-db-manage doesn't offer a "sync" command.
> >
> >
> > On Sat, Mar 1, 2014 at 5:52 PM, Phil Hopkins
> > <phil.hopkins at rackspace.com>wrote:
> >
> >> You might try rerunning the db sync to ensure the database tables are
> >> built correctly.
> >>
> >> Phil
> >>
> >> Sent from my iPhone
> >>
> >> On Mar 1, 2014, at 6:21 PM, "Matt Kassawara" <mkassawara at gmail.com>
> >> wrote:
> >>
> >> Fresh.
> >>
> >>
> >> On Sat, Mar 1, 2014 at 5:19 PM, Phil Hopkins
> >> <phil.hopkins at rackspace.com>wrote:
> >>
> >>> Was this a fresh install or an upgrade?
> >>>
> >>> Phil
> >>>
> >>> Sent from my iPhone
> >>>
> >>> On Mar 1, 2014, at 5:04 PM, "Matt Kassawara" <mkassawara at gmail.com>
> >>> wrote:
> >>>
> >>> If you thought getting Icehouse with Neutron/ML2 working on Ubuntu
> >>> took some effort, the RHEL situation appears much worse. All of the
> >>> Neutron
> >>> daemons spew database errors about missing tables and/or inability to
> >>> create them.
> >>>
> >>> For example:
> >>>
> >>> [root at hst-osctl8 neutron]# neutron net-list
> >>> Request Failed: internal server error while processing your request.
> >>>
> >>> From the server log...
> >>>
> >>> 2014-03-01 15:37:07.970 2584 INFO urllib3.connectionpool [-] Starting
> >>> new HTTP connection (1): hst-osctl8
> >>> 2014-03-01 15:37:08.133 2584 ERROR neutron.api.v2.resource
> >>> [req-9ec4eec0-bf1b-4720-af03-fbdba6a03c14 None] index failed
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource Traceback
> >>> (most recent call last):
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib/python2.6/site-packages/neutron/api/v2/resource.py", line 84,
> >>> in
> >>> resource
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource result =
> >>> method(request=request, **args)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib/python2.6/site-packages/neutron/api/v2/base.py", line 279, in
> >>> index
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource return
> >>> self._items(request, True, parent_id)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib/python2.6/site-packages/neutron/api/v2/base.py", line 233, in
> >>> _items
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource obj_list
> >>> = obj_getter(request.context, **kwargs)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib/python2.6/site-packages/neutron/plugins/ml2/plugin.py", line
> >>> 367,
> >>> in get_networks
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource limit,
> >>> marker, page_reverse)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib/python2.6/site-packages/neutron/db/db_base_plugin_v2.py", line
> >>> 1029, in get_networks
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource
> >>> page_reverse=page_reverse)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib/python2.6/site-packages/neutron/db/db_base_plugin_v2.py", line
> >>> 196, in _get_collection
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource items =
> >>> [dict_func(c, fields) for c in query]
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/orm/query.py",
> >>> line 2227, in __iter__
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource return
> >>> self._execute_and_instances(context)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/orm/query.py",
> >>> line 2242, in _execute_and_instances
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource result =
> >>> conn.execute(querycontext.statement, self._params)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
> >>> line 1449, in execute
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource params)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
> >>> line 1584, in _execute_clauseelement
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource
> >>> compiled_sql, distilled_params
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
> >>> line 1698, in _execute_context
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource context)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
> >>> line 1691, in _execute_context
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource context)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/default.py",
> >>> line 331, in do_execute
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource
> >>> cursor.execute(statement, parameters)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 173, in
> >>> execute
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource
> >>> self.errorhandler(self, exc, value)
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource File
> >>> "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 36, in
> >>> defaulterrorhandler
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource raise
> >>> errorclass, errorvalue
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource
> >>> ProgrammingError: (ProgrammingError) (1146, "Table
> >>> 'neutron.externalnetworks' doesn't exist") 'SELECT networks.tenant_id AS
> >>> networks_tenant_id, networks.id AS networks_id, networks.name AS
> >>> networks_name, networks.status AS networks_status,
> >>> networks.admin_state_up
> >>> AS networks_admin_state_up, networks.shared AS networks_shared,
> >>> externalnetworks_1.network_id AS externalnetworks_1_network_id \nFROM
> >>> networks LEFT OUTER JOIN externalnetworks ON networks.id =
> >>> externalnetworks.network_id LEFT OUTER JOIN externalnetworks AS
> >>> externalnetworks_1 ON networks.id = externalnetworks_1.network_id' ()
> >>> 2014-03-01 15:37:08.133 2584 TRACE neutron.api.v2.resource
> >>>
> >>> Although I haven't needed to use it before, I poked around at various
> >>> permutations of neutron-db-manage, all of which failed with various SQL
> >>> errors. I also dissected the functional neutron-server package on Ubuntu
> >>> and found no post-installation script calls to neutron-db-manage. On a
> >>> related note, Steve Gordon told me that RH should release icehouse-3
> >>> packages this week which might address this issue.
> >>>
> >>>
> >>
> >
>
--
Steve Gordon, RHCE
Product Manager, Red Hat Enterprise Linux OpenStack Platform
Red Hat Canada (Toronto, Ontario)
More information about the dev
mailing list