7:58 a.m.
________________________________
From: Chinmaya Dwibedy <ckdwibedy(a)gmail.com>
Sent: Wednesday, June 8, 2016 8:39 AM
To: Boris Derzhavets
Cc: rdo-list(a)redhat.com
Subject: Re: [rdo-list] Fwd: ConnectFailure error upon triggering “nova image-list”
command using openstack-mitaka release
Hi Boris,
It appears that, the nova-api process is not running .
Check /var/log/nova/api.log (or nova-api.log , I just forgot exact name of log file )
[root@localhost ~(keystone_admin)]# netstat -antp | grep 8774
[root@localhost ~(keystone_admin)]# iptables-save | grep 8774
-A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 nova
api incoming nova_api" -j ACCEPT
[root@localhost ~(keystone_admin)]#
Regards,
Chinmaya
On Wed, Jun 8, 2016 at 5:49 PM, Boris Derzhavets
<bderzhavets@hotmail.com<mailto:bderzhavets@hotmail.com>> wrote:
________________________________
From: rdo-list-bounces@redhat.com<mailto:rdo-list-bounces@redhat.com>
<rdo-list-bounces@redhat.com<mailto:rdo-list-bounces@redhat.com>> on behalf of
Chinmaya Dwibedy <ckdwibedy@gmail.com<mailto:ckdwibedy@gmail.com>>
Sent: Wednesday, June 8, 2016 8:08 AM
To: rdo-list@redhat.com<mailto:rdo-list@redhat.com>
Subject: [rdo-list] Fwd: ConnectFailure error upon triggering “nova image-list” command
using openstack-mitaka release
Hi ,
I am getting the ConnectFailure error message upon triggering “nova image-list” command.
nova-api process should be listening on 8774. It doesn't look like it is not running.
Also I do not find any error logs in nova-api.log
[BD]
# netstat -antp | grep 8774
# iptables-save | grep 8774
If first command return xxxxxx/python
then `ps -ef | grep xxxxxx`
nova-compute.log and nova-conductor.log. I am using openstack-mitaka release on host (Cent
OS 7.2). How can I debug and know what prevents it from running ? please suggest.
Note: This was working while back and got this issue all of a sudden.
Here are some logs.
[root@localhost ~(keystone_admin)]# nova image-list
ERROR (ConnectFailure): Unable to establish connection to
http://172.18.121.48:8774/v2/4bc608763cee41d9a8df26d3ef919825
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# service openstack-nova-api restart
Redirecting to /bin/systemctl restart openstack-nova-api.service
Job for openstack-nova-api.service failed because the control process exited with error
code. See "systemctl status openstack-nova-api.service" and "journalctl
-xe" for details.
[root@localhost ~(keystone_admin)]# systemctl status openstack-nova-api.service
â openstack-nova-api.service - OpenStack Nova API Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-api.service; enabled; vendor
preset: disabled)
Active: activating (start) since Wed 2016-06-08 07:59:20 EDT; 2s ago
Main PID: 179955 (nova-api)
CGroup: /system.slice/openstack-nova-api.service
ââ179955 /usr/bin/python2 /usr/bin/nova-api
Jun 08 07:59:20 localhost systemd[1]: Starting OpenStack Nova API Server...
Jun 08 07:59:22 localhost python2[179955]: detected unhandled Python exception in
'/usr/bin/nova-api'
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# keystone endpoint-list
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64: DeprecationWarning: The
keystone CLI is deprecated in favor of python-openstackclient. For a Python library,
continue using python-keystoneclient.
'python-keystoneclient.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning:
Constructing an instance of the keystoneclient.v2_0.client.Client class without a session
is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning:
Using the 'tenant_name' argument is deprecated in version '1.7.0' and will
be removed in version '2.0.0', please use the 'project_name' argument
instead
super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/site-packages/debtcollector/renames.py:45: DeprecationWarning: Using
the 'tenant_id' argument is deprecated in version '1.7.0' and will be
removed in version '2.0.0', please use the 'project_id' argument instead
return f(*args, **kwargs)
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:371: DeprecationWarning:
Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0
release and may be removed in the 2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/session.py:140: DeprecationWarning:
keystoneclient.session.Session is deprecated as of the 2.1.0 release in favor of
keystoneauth1.session.Session. It will be removed in future releases.
DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py:56:
DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in
favor of keystoneauth1 plugins. They will be removed in future releases.
'in future releases.', DeprecationWarning)
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
| id | region | publicurl
| internalurl | adminurl
| service_id |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
| 02fcec9a7b834128b3e30403c4ed0de7 | RegionOne |
http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s |
http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s |
http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s | 5533324a63d8402888040832640a19d0 |
| 295802909413422cb7c22dc1e268bce9 | RegionOne |
http://172.18.121.48:8774/v2/%(tenant_id)s |
http://172.18.121.48:8774/v2/%(tenant_id)s |
http://172.18.121.48:8774/v2/%(tenant_id)s | f7fe68bf4cec47a4a3c942f3916dc377 |
| 2a125f10b0d04f8a9306dede85b65514 | RegionOne | http://172.18.121.48:9696
| http://172.18.121.48:9696 |
http://172.18.121.48:9696 | b2a60cdc144e40a49757f13c2264f030 |
| 2d1a91d39f3d421cb1b2fe73fba5fd3a | RegionOne | http://172.18.121.48:8777
| http://172.18.121.48:8777 |
http://172.18.121.48:8777 | e6d750ac5ef3433799d4fe39518a3fe6 |
| 47b634f3e18e4caf914521a1a4157008 | RegionOne | http://172.18.121.48:8042
| http://172.18.121.48:8042 |
http://172.18.121.48:8042 | 07cd8adf66254b4ab9b07be03a24084b |
| 595913f7227b44dc8753db3b0cf6acdc | RegionOne | http://172.18.121.48:8041
| http://172.18.121.48:8041 |
http://172.18.121.48:8041 | f43240abe5f3476ea64a8bd381fe4da7 |
| 64381b509bc84639b6a4710e6d99a23b | RegionOne |
http://172.18.121.48:8776/v1/%(tenant_id)s |
http://172.18.121.48:8776/v1/%(tenant_id)s |
http://172.18.121.48:8776/v1/%(tenant_id)s | 7edc7bedf93d4f388185699b9793ec7f |
| 727d25775be54c9f8453f697ae5cb625 | RegionOne | http://172.18.121.48:5000/v2.0
| http://172.18.121.48:5000/v2.0 |
http://172.18.121.48:35357/v2.0 | 25e99a2a98f244d9a73bf965acdd39da |
| 9049338c57574b2d8ff8308b1a4265a5 | RegionOne |
http://172.18.121.48:8776/v2/%(tenant_id)s |
http://172.18.121.48:8776/v2/%(tenant_id)s |
http://172.18.121.48:8776/v2/%(tenant_id)s | 6e070f0629094b72b66025250fdbda64 |
| c051c0f9649143f6b29eaf0895940abe | RegionOne | http://172.18.121.48:9292
| http://172.18.121.48:9292 |
http://172.18.121.48:9292 | 40874e10139a47eb88dfec2114047a34 |
| ee4a00c1e8334cb8921fa3f2a7c82f1b | RegionOne | http://172.18.121.48:8774/v3
| http://172.18.121.48:8774/v3 |
http://172.18.121.48:8774/v3 | 24c92ce4cd354e3db6c5ad59b8beeae8 |
| fa60b5ba0ab7436ab1ffebb2982d3ccc | RegionOne |
http://127.0.0.1:8776/v3/%(tenant_id)s | http://127.0.0.1:8776/v3/%(tenant_id)s
| http://127.0.0.1:8776/v3/%(tenant_id)s | b0b6b97cf9d649c9800300cc64b0e866 |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# netstat -ntlp | grep 8774
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# ps -ef | grep nova-api
nova 156427 1 86 07:51 ? 00:00:01 /usr/bin/python2 /usr/bin/nova-api
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# lsof -i :8774
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# keystone user-list
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64: DeprecationWarning: The
keystone CLI is deprecated in favor of python-openstackclient. For a Python library,
continue using python-keystoneclient.
'python-keystoneclient.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning:
Constructing an instance of the keystoneclient.v2_0.client.Client class without a session
is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning:
Using the 'tenant_name' argument is deprecated in version '1.7.0' and will
be removed in version '2.0.0', please use the 'project_name' argument
instead
super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/site-packages/debtcollector/renames.py:45: DeprecationWarning: Using
the 'tenant_id' argument is deprecated in version '1.7.0' and will be
removed in version '2.0.0', please use the 'project_id' argument instead
return f(*args, **kwargs)
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:371: DeprecationWarning:
Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0
release and may be removed in the 2.0.0 release.
'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/session.py:140: DeprecationWarning:
keystoneclient.session.Session is deprecated as of the 2.1.0 release in favor of
keystoneauth1.session.Session. It will be removed in future releases.
DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py:56:
DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in
favor of keystoneauth1 plugins. They will be removed in future releases.
'in future releases.', DeprecationWarning)
+----------------------------------+------------+---------+----------------------+
| id | name | enabled | email |
+----------------------------------+------------+---------+----------------------+
| 266f5859848e4f39b9725203dda5c3f2 | admin | True | root@localhost |
| 79a6ff3cc7cc4d018247c750adbc18e7 | aodh | True | aodh@localhost |
| 90f28a2a80054132a901d39da307213f | ceilometer | True | ceilometer@localhost |
| 16fa5ffa60e147d89ad84646b6519278 | cinder | True | cinder@localhost |
| c6312ec6c2c444288a412f32173fcd99 | glance | True | glance@localhost |
| ac8fb9c33d404a1697d576d428db90b3 | gnocchi | True | gnocchi@localhost |
| 1a5b4da4ed974ac8a6c78b752ac8fab6 | neutron | True | neutron@localhost |
| f21e8a15da5c40b7957416de4fa91b62 | nova | True | nova@localhost |
| b843358d7ae44944b11af38ce4b61f4d | swift | True | swift@localhost |
+----------------------------------+------------+---------+----------------------+
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# nova-manage service list
Option "verbose" from group "DEFAULT" is deprecated for removal. Its
value may be silently ignored in the future.
Option "notification_driver" from group "DEFAULT" is deprecated. Use
option "driver" from group "oslo_messaging_notifications".
Option "notification_topics" from group "DEFAULT" is deprecated. Use
option "topics" from group "oslo_messaging_notifications".
DEPRECATED: Use the nova service-* commands from python-novaclient instead or the
os-services REST resource. The service subcommand will be removed in the 14.0 release.
Binary Host Zone Status State
Updated_At
nova-osapi_compute 0.0.0.0 internal enabled XXX
None
nova-metadata 0.0.0.0 internal enabled XXX
None
nova-cert localhost internal enabled XXX
2016-06-08 06:12:38
nova-consoleauth localhost internal enabled XXX
2016-06-08 06:12:37
nova-scheduler localhost internal enabled XXX
2016-06-08 06:12:38
nova-conductor localhost internal enabled XXX
2016-06-08 06:12:37
nova-compute localhost nova enabled XXX
2016-06-08 06:12:43
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# ls -l /var/log/nova/
total 4
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-api.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-cert.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-compute.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-conductor.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:22 nova-consoleauth.log
-rw-r--r--. 1 root root 0 Jun 8 05:22 nova-manage.log
-rw-r--r--. 1 nova nova 995 Jun 8 05:32 nova-novncproxy.log
-rw-r--r--. 1 nova nova 0 Jun 8 05:23 nova-scheduler.log
[root@localhost ~(keystone_admin)]#
Regards,
Chinmaya