6:37 a.m.
Hi,
I am trying to implement AIO POC with Trove enabled using packstack on centos 7. I have
created image for mongodb and data store is also configured.
I am having issue when I try to create trove instance, instance status is error. Though
trove was deployed by packstack, I cross checked all trove config files as per this guide
https://docs.openstack.org/trove/latest/install/install-rdo.html they seems ok.
trove-api.log has following error,
2017-12-08 13:14:48.021 15950 INFO trove.instance.service [-] Creating a database instance
for tenant 'b20cf7ccbd1a4636bb8a8bdf8cd3973d'
2017-12-08 13:14:48.048 15950 WARNING trove.common.cfg [-] Manager name
('datastore_manager') not defined, using 'mysql' options instead.
2017-12-08 13:14:48.049 15950 WARNING trove.common.cfg [-] Manager name
('datastore_manager') not defined, using 'mysql' options instead.
2017-12-08 13:14:48.062 15950 WARNING novaclient.v2.client [-] Property `management_url`
is deprecated for SessionClient. It should be set via `endpoint_override` variable while
class initialization.
2017-12-08 13:14:49.426 15950 INFO eventlet.wsgi [-] 172.16.5.184 - - [08/Dec/2017
13:14:49] "POST /v1.0/b20cf7ccbd1a4636bb8a8bdf8cd3973d/instances HTTP/1.1" 200
1085 1.411768
2017-12-08 13:14:49.851 15963 INFO trove.instance.service [-] Listing database instances
for tenant 'b20cf7ccbd1a4636bb8a8bdf8cd3973d'
2017-12-08 13:14:49.855 15963 WARNING novaclient.v2.client [-] Property `management_url`
is deprecated for SessionClient. It should be set via `endpoint_override` variable while
class initialization.
2017-12-08 13:14:51.215 15963 INFO eventlet.wsgi [-] 172.16.5.184 - - [08/Dec/2017
13:14:51] "GET /v1.0/b20cf7ccbd1a4636bb8a8bdf8cd3973d/instances?limit=20
HTTP/1.1" 200 949 1.738005
2017-12-08 13:14:51.230 15963 WARNING novaclient.v2.client [-] Property `management_url`
is deprecated for SessionClient. It should be set via `endpoint_override` variable while
class initialization.
2017-12-08 13:14:52.299 15963 INFO eventlet.wsgi [-] 172.16.5.184 - - [08/Dec/2017
13:14:52] "GET /v1.0/b20cf7ccbd1a4636bb8a8bdf8cd3973d/flavors HTTP/1.1" 200 1936
1.080438
2017-12-08 13:14:52.306 15955 WARNING keystonemiddleware.auth_token [-] Using the
in-process token cache is deprecated as of the 4.2.0 release and may be removed in the
5.0.0 release or the 'O' development cycle. The in-process cache causes
inconsistent results and high memory usage. When the feature is removed the auth_token
middleware will not cache tokens by default which may result in performance issues. It is
recommended to use memcache for the auth_token token cache by setting the
memcached_servers option.
2017-12-08 13:14:53.313 15955 INFO trove.instance.service [-] Showing database instance
'62b40b11-9440-41ad-929f-7fd8b0bb6208' for tenant
'b20cf7ccbd1a4636bb8a8bdf8cd3973d'
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api [-] Error calling
get_filesystem_stats: MessagingTimeout: Timed out waiting for a reply to message ID
5827c0618531497f9e7da6cf01de28b1
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api Traceback (most recent call
last):
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api File
"/usr/lib/python2.7/site-packages/trove/guestagent/api.py", line 90, in _call
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api result =
cctxt.call(self.context, method_name, **kwargs)
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in
call
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api retry=self.retry)
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 123, in
_send
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api timeout=timeout,
retry=retry)
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
578, in send
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api retry=retry)
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
567, in _send
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api result =
self._waiter.wait(msg_id, timeout)
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
459, in wait
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api message =
self.waiters.get(msg_id, timeout=timeout)
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
347, in get
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api 'to message ID %s' %
msg_id)
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api MessagingTimeout: Timed out
waiting for a reply to message ID 5827c0618531497f9e7da6cf01de28b1
2017-12-08 13:14:58.422 15955 ERROR trove.guestagent.api
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models [-] An error occurred
communicating with the guest: Timed out waiting for a reply to message ID
5827c0618531497f9e7da6cf01de28b1.: GuestError: An error occurred communicating with the
guest: Timed out waiting for a reply to message ID 5827c0618531497f9e7da6cf01de28b1.
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models Traceback (most recent call
last):
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models File
"/usr/lib/python2.7/site-packages/trove/instance/models.py", line 561, in
load_guest_info
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models volume_info =
guest.get_volume_info()
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models File
"/usr/lib/python2.7/site-packages/trove/guestagent/api.py", line 423, in
get_volume_info
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models version=version,
fs_path=None)
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models File
"/usr/lib/python2.7/site-packages/trove/guestagent/api.py", line 99, in _call
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models raise
exception.GuestError(original_message=str(e))
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models GuestError: An error occurred
communicating with the guest: Timed out waiting for a reply to message ID
5827c0618531497f9e7da6cf01de28b1.
2017-12-08 13:14:58.424 15955 ERROR trove.instance.models
2017-12-08 13:14:58.483 15955 WARNING novaclient.v2.client [-] Property `management_url`
is deprecated for SessionClient. It should be set via `endpoint_override` variable while
class initialization.
2017-12-08 13:14:59.795 15955 INFO eventlet.wsgi [-] 172.16.5.184 - - [08/Dec/2017
13:14:59] "GET
/v1.0/b20cf7ccbd1a4636bb8a8bdf8cd3973d/instances/62b40b11-9440-41ad-929f-7fd8b0bb6208
HTTP/1.1" 200 2327 7.491198
Whereas trove-taskmanager.log has following error,
2017-12-08 13:07:25.719 15981 WARNING novaclient.v2.client [-] Property `management_url`
is deprecated for SessionClient. It should be set via `endpoint_override` variable while
class initialization.
2017-12-08 13:07:26.715 15981 ERROR trove.instance.models [-] Could not find nova
server_id(None).: NotFound: Instance None could not be found. (HTTP 404) (Request-ID:
req-dba69b39-19ed-49bc-8f32-d37d7d356a24)
2017-12-08 13:07:26.716 15981 ERROR trove.common.exception [-] Cannot find compute
instance None for instance 3f9f4c53-7ba6-4e78-9278-ffd9918de42b.: NotFound: Instance None
could not be found. (HTTP 404) (Request-ID: req-dba69b39-19ed-49bc-8f32-d37d7d356a24)
2017-12-08 13:07:26.716 15981 ERROR trove.instance.models [-] Could not load compute
instance None.: ComputeInstanceNotFound: Resource 3f9f4c53-7ba6-4e78-9278-ffd9918de42b can
not be retrieved.
2017-12-08 13:07:26.724 15981 WARNING novaclient.v2.client [-] Property `management_url`
is deprecated for SessionClient. It should be set via `endpoint_override` variable while
class initialization.: UnprocessableEntity: Instance 3f9f4c53-7ba6-4e78-9278-ffd9918de42b
is not ready.
2017-12-08 13:14:49.336 15981 INFO trove.taskmanager.models [-] Creating instance
62b40b11-9440-41ad-929f-7fd8b0bb6208.
2017-12-08 13:14:53.094 15981 ERROR trove.taskmanager.models [-] Failed to create volume
for instance 62b40b11-9440-41ad-929f-7fd8b0bb6208
The resource could not be found.<br /><br />
(HTTP 404)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/trove/taskmanager/models.py", line 811,
in _build_volume_info
volume_size, volume_type, datastore_manager)
File "/usr/lib/python2.7/site-packages/trove/taskmanager/models.py", line 848,
in _create_volume
volume_type=volume_type)
File "/usr/lib/python2.7/site-packages/cinderclient/v2/volumes.py", line 290,
in create
return self._create('/volumes', body, 'volume')
File "/usr/lib/python2.7/site-packages/cinderclient/base.py", line 324, in
_create
resp, body = self.api.client.post(url, body=body)
File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 434, in
post
return self._cs_request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 387, in
_cs_request
resp, body = self.request(url, method, **kwargs)
File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 369, in
request
raise exceptions.from_response(resp, body)
NotFound: The resource could not be found.<br /><br />
(HTTP 404)
: NotFound: The resource could not be found.<br /><br />
(HTTP 404)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server [-] Exception during message
handling: TroveError: Failed to create volume for instance
62b40b11-9440-41ad-929f-7fd8b0bb6208
The resource could not be found.<br /><br />
(HTTP 404)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server Traceback (most recent call
last):
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in
_process_incoming
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server res =
self.dispatcher.dispatch(message)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213,
in dispatch
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server return
self._do_dispatch(endpoint, method, ctxt, args)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183,
in _do_dispatch
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server result = func(ctxt,
**new_args)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server return f(*args,
**kwargs)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/trove/taskmanager/manager.py", line 384, in
create_instance
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server locality)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/trove/taskmanager/manager.py", line 364, in
_create_instance
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server scheduler_hints)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/trove/taskmanager/models.py", line 505, in
create_instance
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server scheduler_hints)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/trove/taskmanager/models.py", line 782, in
_create_server_volume_individually
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server
volume_type=volume_type)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/trove/taskmanager/models.py", line 815, in
_build_volume_info
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server self._log_and_raise(e,
msg, err)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/site-packages/trove/taskmanager/models.py", line 839, in
_log_and_raise
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server raise
TroveError(message=full_message)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server TroveError: Failed to create
volume for instance 62b40b11-9440-41ad-929f-7fd8b0bb6208
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server The resource could not be
found.<br /><br />
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server (HTTP 404)
2017-12-08 13:14:53.247 15981 ERROR oslo_messaging.rpc.server
Regards,
/Sonali