7:08 a.m.
Hi ,
I am getting the ConnectFailure error message upon triggering “nova
image-list” command. nova-api process should be listening on 8774. It
doesn't look like it is not running. Also I do not find any error logs in
nova-api.log nova-compute.log and nova-conductor.log. I am using openstack-
mitaka release on host (Cent OS 7.2). How can I debug and know what
prevents it from running ? please suggest.
Note: This was working while back and got this issue all of a sudden.
Here are some logs.
[root@localhost ~(keystone_admin)]# nova image-list
ERROR (ConnectFailure): Unable to establish connection to
http://172.18.121.48:8774/v2/4bc608763cee41d9a8df26d3ef919825
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# service openstack-nova-api restart
Redirecting to /bin/systemctl restart openstack-nova-api.service
Job for openstack-nova-api.service failed because the control process
exited with error code. See "systemctl status openstack-nova-api.service"
and "journalctl -xe" for details.
[root@localhost ~(keystone_admin)]# systemctl status
openstack-nova-api.service
â openstack-nova-api.service - OpenStack Nova API Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-api.service;
enabled; vendor preset: disabled)
Active: activating (start) since Wed 2016-06-08 07:59:20 EDT; 2s ago
Main PID: 179955 (nova-api)
CGroup: /system.slice/openstack-nova-api.service
ââ179955 /usr/bin/python2 /usr/bin/nova-api
Jun 08 07:59:20 localhost systemd[1]: Starting OpenStack Nova API Server...
Jun 08 07:59:22 localhost python2[179955]: detected unhandled Python
exception in '/usr/bin/nova-api'
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# keystone endpoint-list
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64:
DeprecationWarning: The keystone CLI is deprecated in favor of
python-openstackclient. For a Python library, continue using
python-keystoneclient.
'python-keystoneclient.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145:
DeprecationWarning: Constructing an instance of the
keystoneclient.v2_0.client.Client class without a session is deprecated as
of the 1.7.0 release and may be removed in the 2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147:
DeprecationWarning: Using the 'tenant_name' argument is deprecated in
version '1.7.0' and will be removed in version '2.0.0', please use the
'project_name' argument instead
super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/site-packages/debtcollector/renames.py:45:
DeprecationWarning: Using the 'tenant_id' argument is deprecated in version
'1.7.0' and will be removed in version '2.0.0', please use the
'project_id'
argument instead
return f(*args, **kwargs)
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:371:
DeprecationWarning: Constructing an HTTPClient instance without using a
session is deprecated as of the 1.7.0 release and may be removed in the
2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/session.py:140:
DeprecationWarning: keystoneclient.session.Session is deprecated as of the
2.1.0 release in favor of keystoneauth1.session.Session. It will be removed
in future releases.
DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py:56:
DeprecationWarning: keystoneclient auth plugins are deprecated as of the
2.1.0 release in favor of keystoneauth1 plugins. They will be removed in
future releases.
'in future releases.', DeprecationWarning)
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
| id | region |
publicurl |
internalurl |
adminurl | service_id |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
| 02fcec9a7b834128b3e30403c4ed0de7 | RegionOne |
http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s |
http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s |
http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s |
5533324a63d8402888040832640a19d0 |
| 295802909413422cb7c22dc1e268bce9 | RegionOne |
http://172.18.121.48:8774/v2/%(tenant_id)s |
http://172.18.121.48:8774/v2/%(tenant_id)s |
http://172.18.121.48:8774/v2/%(tenant_id)s |
f7fe68bf4cec47a4a3c942f3916dc377 |
| 2a125f10b0d04f8a9306dede85b65514 | RegionOne |
http://172.18.121.48:9696 |
http://172.18.121.48:9696
| http://172.18.121.48:9696 |
b2a60cdc144e40a49757f13c2264f030 |
| 2d1a91d39f3d421cb1b2fe73fba5fd3a | RegionOne |
http://172.18.121.48:8777 |
http://172.18.121.48:8777
| http://172.18.121.48:8777 |
e6d750ac5ef3433799d4fe39518a3fe6 |
| 47b634f3e18e4caf914521a1a4157008 | RegionOne |
http://172.18.121.48:8042 |
http://172.18.121.48:8042
| http://172.18.121.48:8042 |
07cd8adf66254b4ab9b07be03a24084b |
| 595913f7227b44dc8753db3b0cf6acdc | RegionOne |
http://172.18.121.48:8041 |
http://172.18.121.48:8041
| http://172.18.121.48:8041 |
f43240abe5f3476ea64a8bd381fe4da7 |
| 64381b509bc84639b6a4710e6d99a23b | RegionOne |
http://172.18.121.48:8776/v1/%(tenant_id)s |
http://172.18.121.48:8776/v1/%(tenant_id)s |
http://172.18.121.48:8776/v1/%(tenant_id)s |
7edc7bedf93d4f388185699b9793ec7f |
| 727d25775be54c9f8453f697ae5cb625 | RegionOne |
http://172.18.121.48:5000/v2.0 |
http://172.18.121.48:5000/v2.0 |
http://172.18.121.48:35357/v2.0 | 25e99a2a98f244d9a73bf965acdd39da |
| 9049338c57574b2d8ff8308b1a4265a5 | RegionOne |
http://172.18.121.48:8776/v2/%(tenant_id)s |
http://172.18.121.48:8776/v2/%(tenant_id)s |
http://172.18.121.48:8776/v2/%(tenant_id)s |
6e070f0629094b72b66025250fdbda64 |
| c051c0f9649143f6b29eaf0895940abe | RegionOne |
http://172.18.121.48:9292 |
http://172.18.121.48:9292
| http://172.18.121.48:9292 |
40874e10139a47eb88dfec2114047a34 |
| ee4a00c1e8334cb8921fa3f2a7c82f1b | RegionOne |
http://172.18.121.48:8774/v3 |
http://172.18.121.48:8774/v3 |
http://172.18.121.48:8774/v3 | 24c92ce4cd354e3db6c5ad59b8beeae8 |
| fa60b5ba0ab7436ab1ffebb2982d3ccc | RegionOne |
http://127.0.0.1:8776/v3/%(tenant_id)s |
http://127.0.0.1:8776/v3/%(tenant_id)s |
http://127.0.0.1:8776/v3/%(tenant_id)s |
b0b6b97cf9d649c9800300cc64b0e866 |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# netstat -ntlp | grep 8774
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# ps -ef | grep nova-api
nova 156427 1 86 07:51 ? 00:00:01 /usr/bin/python2
/usr/bin/nova-api
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# lsof -i :8774
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# keystone user-list
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64:
DeprecationWarning: The keystone CLI is deprecated in favor of
python-openstackclient. For a Python library, continue using
python-keystoneclient.
'python-keystoneclient.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145:
DeprecationWarning: Constructing an instance of the
keystoneclient.v2_0.client.Client class without a session is deprecated as
of the 1.7.0 release and may be removed in the 2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147:
DeprecationWarning: Using the 'tenant_name' argument is deprecated in
version '1.7.0' and will be removed in version '2.0.0', please use the
'project_name' argument instead
super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/site-packages/debtcollector/renames.py:45:
DeprecationWarning: Using the 'tenant_id' argument is deprecated in version
'1.7.0' and will be removed in version '2.0.0', please use the
'project_id'
argument instead
return f(*args, **kwargs)
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:371:
DeprecationWarning: Constructing an HTTPClient instance without using a
session is deprecated as of the 1.7.0 release and may be removed in the
2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/session.py:140:
DeprecationWarning: keystoneclient.session.Session is deprecated as of the
2.1.0 release in favor of keystoneauth1.session.Session. It will be removed
in future releases.
DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py:56:
DeprecationWarning: keystoneclient auth plugins are deprecated as of the
2.1.0 release in favor of keystoneauth1 plugins. They will be removed in
future releases.
'in future releases.', DeprecationWarning)
+----------------------------------+------------+---------+----------------------+
| id | name | enabled |
email |
+----------------------------------+------------+---------+----------------------+
| 266f5859848e4f39b9725203dda5c3f2 | admin | True |
root@localhost |
| 79a6ff3cc7cc4d018247c750adbc18e7 | aodh | True |
aodh@localhost |
| 90f28a2a80054132a901d39da307213f | ceilometer | True |
ceilometer@localhost |
| 16fa5ffa60e147d89ad84646b6519278 | cinder | True |
cinder@localhost |
| c6312ec6c2c444288a412f32173fcd99 | glance | True |
glance@localhost |
| ac8fb9c33d404a1697d576d428db90b3 | gnocchi | True |
gnocchi@localhost |
| 1a5b4da4ed974ac8a6c78b752ac8fab6 | neutron | True |
neutron@localhost |
| f21e8a15da5c40b7957416de4fa91b62 | nova | True |
nova@localhost |
| b843358d7ae44944b11af38ce4b61f4d | swift | True |
swift@localhost |
+----------------------------------+------------+---------+----------------------+
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# nova-manage service list
Option "verbose" from group "DEFAULT" is deprecated for removal. Its
value
may be silently ignored in the future.
Option "notification_driver" from group "DEFAULT" is deprecated. Use
option
"driver" from group "oslo_messaging_notifications".
Option "notification_topics" from group "DEFAULT" is deprecated. Use
option
"topics" from group "oslo_messaging_notifications".
DEPRECATED: Use the nova service-* commands from python-novaclient instead
or the os-services REST resource. The service subcommand will be removed in
the 14.0 release.
Binary Host Zone
Status State Updated_At
nova-osapi_compute 0.0.0.0 internal
enabled XXX None
nova-metadata 0.0.0.0 internal
enabled XXX None
nova-cert localhost internal
enabled XXX 2016-06-08 06:12:38
nova-consoleauth localhost internal
enabled XXX 2016-06-08 06:12:37
nova-scheduler localhost internal
enabled XXX 2016-06-08 06:12:38
nova-conductor localhost internal
enabled XXX 2016-06-08 06:12:37
nova-compute localhost nova
enabled XXX 2016-06-08 06:12:43
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# ls -l /var/log/nova/
total 4
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-api.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-cert.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-compute.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-conductor.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-consoleauth.log
-rw-r--r--. 1 root root 0 Jun 8 05:22 nova-manage.log
-rw-r--r--. 1 nova nova 995 Jun 8 05:32 nova-novncproxy.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:23 nova-scheduler.log
[root@localhost ~(keystone_admin)]#
Regards,
Chinmaya