[rdo-users] packstack allinone POC - unable to get openstack running ERROR oslo_service.service

Tue Nov 14 12:18:47 UTC 2017

Hi Sonali, 

The source for the issues is here: 

Error: /Stage[main]/Nova:db::Sync/Exec[nova-db-sync]: Failed to call refresh: Command exceeded timeout 

That means that the "nova-manage db sync" command took too long to execute, and Puppet considered that as a failure. This usually happens when your disk is very slow or the machine you are using to run Packstack is low on RAM and constantly swapping. 

As a quick fix, I'd suggest to: 

- Run "nova-manage db sync" as root. 

- Re-run packstack using the same answer file. 

Regards, 

Javier 

----- Original Message -----

> Hi,

> I am trying to implement all-in-one concept cloud with Packstack on Centos 7.
> But I am having problem.

> My command “#packstack –allinone” end with following error,

> Preparing Puppet manifests [ DONE ]

> Copying Puppet modules and manifests [ DONE ]

> Applying 172.16.5.184_controller.pp

> 172.16.5.184_controller.pp: [ ERROR ]

> Applying Puppet manifests [ ERROR ]

> ERROR : Error appeared during Puppet run: 192.6.5.184_controller.pp

> Error: /Stage[main]/Nova::Db::Sync/Exec[nova-db-sync]: Failed to call
> refresh: Command exceeded timeout

> You will find full trace in log
> /var/tmp/packstack/20171113-141949-w6oJW4/manifests/172.16.5.184_controller.pp.log

> Please check log file
> /var/tmp/packstack/20171113-141949-w6oJW4/openstack-setup.log for more
> information

> Additional information:

> * A new answerfile was created in:
> /root/packstack-answers-20171113-141950.txt

> * Time synchronization installation was skipped. Please note that
> unsynchronized time on server instances might be problem for some OpenStack
> components.

> * File /root/keystonerc_admin has been created on OpenStack client host
> 192.6.5.184. To use the command line tools you need to source the file.

> * To access the OpenStack Dashboard browse to http://192.6.5.184/dashboard .

> Please, find your login credentials stored in the keystonerc_admin in your
> home directory.

> I am able to bowse dashboard but once I enter username and pwd , it’s not
> loading further.

> I checked nova logs,

> I am getting these errors,

> 2017-11-14 12:51:22.061 30815 INFO nova.service [-] Starting consoleauth node
> (version 16.0.2-1.el7)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> [req-6df91cfa-5d3e-4738-9d5e-06e0109a8175 - - - - -] DBAPIError exception
> wrapped from (pymysql.err.InternalError) (1054, u"Unknown column
> 'services.uuid' in 'field list'") [SQL: u'SELECT services.created_at AS
> services_created_at, services.updated_at AS services_updated_at,
> services.deleted_at AS services_deleted_at, services.deleted AS
> services_deleted, services.id AS services_id, services.uuid AS
> services_uuid, services.host AS services_host, services.`binary` AS
> services_binary, services.topic AS services_topic, services.report_count AS
> services_report_count, services.disabled AS services_disabled,
> services.disabled_reason AS services_disabled_reason, services.last_seen_up
> AS services_last_seen_up, services.forced_down AS services_forced_down,
> services.version AS services_version \nFROM services \nWHERE
> services.deleted = %(deleted_1)s AND services.host = %(host_1)s AND
> services.`binary` = %(binary_1)s \n LIMIT %(param_1)s'] [parameters:
> {u'host_1': 'ostest', u'param_1': 1, u'deleted_1': 0, u'binary_1':
> 'nova-consoleauth'}]: InternalError: (1054, u"Unknown column 'services.uuid'
> in 'field list'")

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters Traceback
> (most recent call last):

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182,
> in _execute_context

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters context)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 470,
> in do_execute

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> cursor.execute(statement, parameters)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters result =
> self._query(query)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> conn.query(q)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 856, in
> query

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> self._affected_rows = self._read_query_result(unbuffered=unbuffered)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1057, in
> _read_query_result

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> result.read()

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1340, in
> read

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> first_packet = self.connection._read_packet()

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in
> _read_packet

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> packet.check_error()

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in
> check_error

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> err.raise_mysql_exception(self._data)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in
> raise_mysql_exception

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters raise
> errorclass(errno, errval)

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters
> InternalError: (1054, u"Unknown column 'services.uuid' in 'field list'")

> 2017-11-14 12:51:22.465 30815 ERROR oslo_db.sqlalchemy.exc_filters

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service
> [req-6df91cfa-5d3e-4738-9d5e-06e0109a8175 - - - - -] Error starting thread.:
> DBError: (pymysql.err.InternalError) (1054, u"Unknown column 'services.uuid'
> in 'field list'") [SQL: u'SELECT services.created_at AS services_created_at,
> services.updated_at AS services_updated_at, services.deleted_at AS
> services_deleted_at, services.deleted AS services_deleted, services.id AS
> services_id, services.uuid AS services_uuid, services.host AS services_host,
> services.`binary` AS services_binary, services.topic AS services_topic,
> services.report_count AS services_report_count, services.disabled AS
> services_disabled, services.disabled_reason AS services_disabled_reason,
> services.last_seen_up AS services_last_seen_up, services.forced_down AS
> services_forced_down, services.version AS services_version \nFROM services
> \nWHERE services.deleted = %(deleted_1)s AND services.host = %(host_1)s AND
> services.`binary` = %(binary_1)s \n LIMIT %(param_1)s'] [parameters:
> {u'host_1': 'ostest', u'param_1': 1, u'deleted_1': 0, u'binary_1':
> 'nova-consoleauth'}]

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service Traceback (most
> recent call last):

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 721, in
> run_service

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service service.start()

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/nova/service.py", line 147, in start

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service ctxt, self.host,
> self.binary)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184,
> in wrapper

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service result = fn(cls,
> context, *args, **kwargs)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/nova/objects/service.py", line 296, in
> get_by_host_and_binary

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service host, binary)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/nova/db/api.py", line 133, in
> service_get_by_host_and_binary

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service return
> IMPL.service_get_by_host_and_binary(context, host, binary)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 265, in
> wrapped

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service return f(context,
> *args, **kwargs)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 554, in
> service_get_by_host_and_binary

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service
> filter_by(binary=binary).\

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2755, in
> first

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service ret =
> list(self[0:1])

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2547, in
> __getitem__

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service return list(res)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in
> __iter__

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service return
> self._execute_and_instances(context)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2878, in
> _execute_and_instances

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service result =
> conn.execute(querycontext.statement, self._params)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in
> execute

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service return meth(self,
> multiparams, params)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263,
> in _execute_on_connection

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service return
> connection._execute_clauseelement(self, multiparams, params)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053,
> in _execute_clauseelement

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service compiled_sql,
> distilled_params

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189,
> in _execute_context

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service context)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1398,
> in _handle_dbapi_exception

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service
> util.raise_from_cause(newraise, exc_info)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in
> raise_from_cause

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service
> reraise(type(exception), exception, tb=exc_tb, cause=cause)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182,
> in _execute_context

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service context)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 470,
> in do_execute

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service
> cursor.execute(statement, parameters)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service result =
> self._query(query)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service conn.query(q)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 856, in
> query

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service self._affected_rows
> = self._read_query_result(unbuffered=unbuffered)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1057, in
> _read_query_result

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service result.read()

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1340, in
> read

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service first_packet =
> self.connection._read_packet()

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in
> _read_packet

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service packet.check_error()

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in
> check_error

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service
> err.raise_mysql_exception(self._data)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service File
> "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in
> raise_mysql_exception

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service raise
> errorclass(errno, errval)

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service DBError:
> (pymysql.err.InternalError) (1054, u"Unknown column 'services.uuid' in
> 'field list'") [SQL: u'SELECT services.created_at AS services_created_at,
> services.updated_at AS services_updated_at, services.deleted_at AS
> services_deleted_at, services.deleted AS services_deleted, services.id AS
> services_id, services.uuid AS services_uuid, services.host AS services_host,
> services.`binary` AS services_binary, services.topic AS services_topic,
> services.report_count AS services_report_count, services.disabled AS
> services_disabled, services.disabled_reason AS services_disabled_reason,
> services.last_seen_up AS services_last_seen_up, services.forced_down AS
> services_forced_down, services.version AS services_version \nFROM services
> \nWHERE services.deleted = %(deleted_1)s AND services.host = %(host_1)s AND
> services.`binary` = %(binary_1)s \n LIMIT %(param_1)s'] [parameters:
> {u'host_1': 'ostest', u'param_1': 1, u'deleted_1': 0, u'binary_1':
> 'nova-consoleauth'}]

> 2017-11-14 12:51:22.469 30815 ERROR oslo_service.service

> ^C

> 2017-11-13 15:17:13.194 2949 ERROR oslo_db.sqlalchemy.exc_filters File
> "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in
> raise_mysql_exception

> 2017-11-13 15:17:13.194 2949 ERROR oslo_db.sqlalchemy.exc_filters raise
> errorclass(errno, errval)

> 2017-11-13 15:17:13.194 2949 ERROR oslo_db.sqlalchemy.exc_filters
> InternalError: (1054, u"Unknown column 'aggregates.uuid' in 'field list'")

> 2017-11-13 15:17:13.194 2949 ERROR oslo_db.sqlalchemy.exc_filters

> 2017-11-13 15:17:13.197 2949 CRITICAL nova
> [req-2d6ddb32-c998-4123-a7ba-e1a0da9648e1 - - - - -] Unhandled error:
> DBError: (pymysql.err.InternalError) (1054, u"Unknown column
> 'aggregates.uuid' in 'field list'") [SQL: u'SELECT aggregates.created_at AS
> aggregates_created_at, aggregates.updated_at AS aggregates_updated_at,
> aggregates.deleted_at AS aggregates_deleted_at, aggregates.deleted AS
> aggregates_deleted, aggregates.id AS aggregates_id, aggregates.uuid AS
> aggregates_uuid, aggregates.name AS aggregates_name,
> aggregate_hosts_1.created_at AS aggregate_hosts_1_created_at,
> aggregate_hosts_1.updated_at AS aggregate_hosts_1_updated_at,
> aggregate_hosts_1.deleted_at AS aggregate_hosts_1_deleted_at,
> aggregate_hosts_1.deleted AS aggregate_hosts_1_deleted, aggregate_hosts_1.id
> AS aggregate_hosts_1_id, aggregate_hosts_1.host AS aggregate_hosts_1_host,
> aggregate_hosts_1.aggregate_id AS aggregate_hosts_1_aggregate_id,
> aggregate_metadata_1.created_at AS aggregate_metadata_1_created_at,
> aggregate_metadata_1.updated_at AS aggregate_metadata_1_updated_at,
> aggregate_metadata_1.deleted_at AS aggregate_metadata_1_deleted_at,
> aggregate_metadata_1.deleted AS aggregate_metadata_1_deleted,
> aggregate_metadata_1.id AS aggregate_metadata_1_id,
> aggregate_metadata_1.`key` AS aggregate_metadata_1_key,
> aggregate_metadata_1.value AS aggregate_metadata_1_value,
> aggregate_metadata_1.aggregate_id AS aggregate_metadata_1_aggregate_id
> \nFROM aggregates LEFT OUTER JOIN aggregate_hosts AS aggregate_hosts_1 ON
> aggregates.id = aggregate_hosts_1.aggregate_id AND aggregate_hosts_1.deleted
> = %(deleted_1)s AND aggregates.deleted = %(deleted_2)s LEFT OUTER JOIN
> aggregate_metadata AS aggregate_metadata_1 ON aggregates.id =
> aggregate_metadata_1.aggregate_id AND aggregate_metadata_1.deleted =
> %(deleted_3)s AND aggregates.deleted = %(deleted_4)s \nWHERE
> aggregates.deleted = %(deleted_5)s'] [parameters: {u'deleted_5': 0,
> u'deleted_4': 0, u'deleted_3': 0, u'deleted_2': 0, u'deleted_1': 0}]

> 2017-11-13 15:17:13.197 2949 ERROR nova Traceback (most recent call last):

> 2017-11-13 15:17:13.197 2949 ERROR nova File "/usr/bin/nova-scheduler", line
> 10, in <module>

> 2017-11-13 15:17:13.197 2949 ERROR nova sys.exit(main())

> I am unable to figure out the issue here. Can anyone help me figure it out?

> Thanks,

> /Sonali

> _______________________________________________
> users mailing list
> users at lists.rdoproject.org
> http://lists.rdoproject.org/mailman/listinfo/users

> To unsubscribe: users-unsubscribe at lists.rdoproject.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/users/attachments/20171114/5b9a255f/attachment-0001.html>