[rdo-list] Fwd: ConnectFailure error upon triggering “nova image-list” command using openstack-mitaka release

Boris Derzhavets bderzhavets at hotmail.com
Wed Jun 8 12:58:38 UTC 2016




________________________________
From: Chinmaya Dwibedy <ckdwibedy at gmail.com>
Sent: Wednesday, June 8, 2016 8:39 AM
To: Boris Derzhavets
Cc: rdo-list at redhat.com
Subject: Re: [rdo-list] Fwd: ConnectFailure error upon triggering “nova image-list” command using openstack-mitaka release


Hi Boris,

It appears that,  the nova-api process is not running .

  Check /var/log/nova/api.log (or nova-api.log , I just forgot exact name of log  file )


[root at localhost ~(keystone_admin)]#  netstat -antp | grep 8774
[root at localhost ~(keystone_admin)]# iptables-save | grep 8774
-A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 nova api incoming nova_api" -j ACCEPT
[root at localhost ~(keystone_admin)]#


Regards,
Chinmaya



On Wed, Jun 8, 2016 at 5:49 PM, Boris Derzhavets <bderzhavets at hotmail.com<mailto:bderzhavets at hotmail.com>> wrote:



________________________________
From: rdo-list-bounces at redhat.com<mailto:rdo-list-bounces at redhat.com> <rdo-list-bounces at redhat.com<mailto:rdo-list-bounces at redhat.com>> on behalf of Chinmaya Dwibedy <ckdwibedy at gmail.com<mailto:ckdwibedy at gmail.com>>
Sent: Wednesday, June 8, 2016 8:08 AM
To: rdo-list at redhat.com<mailto:rdo-list at redhat.com>
Subject: [rdo-list] Fwd: ConnectFailure error upon triggering “nova image-list” command using openstack-mitaka release


Hi ,

I am getting the ConnectFailure error message upon triggering “nova image-list” command. nova-api process should be listening on 8774. It doesn't look like it is not running. Also I do not find any error logs in nova-api.log

  [BD]

  # netstat -antp | grep 8774
  # iptables-save | grep 8774
  If first command return   xxxxxx/python
  then `ps -ef | grep xxxxxx`

nova-compute.log and nova-conductor.log. I am using openstack-mitaka release on host (Cent OS 7.2). How can I debug and know what prevents it from running ? please suggest.

Note: This was working while back and got this issue all of a sudden.

Here are some logs.

[root at localhost ~(keystone_admin)]# nova image-list
ERROR (ConnectFailure): Unable to establish connection to http://172.18.121.48:8774/v2/4bc608763cee41d9a8df26d3ef919825
[root at localhost ~(keystone_admin)]#

[root at localhost ~(keystone_admin)]# service openstack-nova-api restart
Redirecting to /bin/systemctl restart  openstack-nova-api.service
Job for openstack-nova-api.service failed because the control process exited with error code. See "systemctl status openstack-nova-api.service" and "journalctl -xe" for details.
[root at localhost ~(keystone_admin)]# systemctl status openstack-nova-api.service
â openstack-nova-api.service - OpenStack Nova API Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-nova-api.service; enabled; vendor preset: disabled)
   Active: activating (start) since Wed 2016-06-08 07:59:20 EDT; 2s ago
 Main PID: 179955 (nova-api)
   CGroup: /system.slice/openstack-nova-api.service
           ââ179955 /usr/bin/python2 /usr/bin/nova-api

Jun 08 07:59:20 localhost systemd[1]: Starting OpenStack Nova API Server...
Jun 08 07:59:22 localhost python2[179955]: detected unhandled Python exception in '/usr/bin/nova-api'
[root at localhost ~(keystone_admin)]#

[root at localhost ~(keystone_admin)]#  keystone endpoint-list
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient.
  'python-keystoneclient.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning: Constructing an instance of the keystoneclient.v2_0.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead
  super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/site-packages/debtcollector/renames.py:45: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead
  return f(*args, **kwargs)
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:371: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/session.py:140: DeprecationWarning: keystoneclient.session.Session is deprecated as of the 2.1.0 release in favor of keystoneauth1.session.Session. It will be removed in future releases.
  DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py:56: DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in favor of keystoneauth1 plugins. They will be removed in future releases.
  'in future releases.', DeprecationWarning)
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
|                id                |   region  |                    publicurl                    |                   internalurl                   |                     adminurl                    |            service_id            |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
| 02fcec9a7b834128b3e30403c4ed0de7 | RegionOne | http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s | http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s | http://172.18.121.48:8080/v1/AUTH_%(tenant_id)s | 5533324a63d8402888040832640a19d0 |
| 295802909413422cb7c22dc1e268bce9 | RegionOne |    http://172.18.121.48:8774/v2/%(tenant_id)s   |    http://172.18.121.48:8774/v2/%(tenant_id)s   |    http://172.18.121.48:8774/v2/%(tenant_id)s   | f7fe68bf4cec47a4a3c942f3916dc377 |
| 2a125f10b0d04f8a9306dede85b65514 | RegionOne |            http://172.18.121.48:9696            |            http://172.18.121.48:9696            |            http://172.18.121.48:9696            | b2a60cdc144e40a49757f13c2264f030 |
| 2d1a91d39f3d421cb1b2fe73fba5fd3a | RegionOne |            http://172.18.121.48:8777            |            http://172.18.121.48:8777            |            http://172.18.121.48:8777            | e6d750ac5ef3433799d4fe39518a3fe6 |
| 47b634f3e18e4caf914521a1a4157008 | RegionOne |            http://172.18.121.48:8042            |            http://172.18.121.48:8042            |            http://172.18.121.48:8042            | 07cd8adf66254b4ab9b07be03a24084b |
| 595913f7227b44dc8753db3b0cf6acdc | RegionOne |            http://172.18.121.48:8041            |            http://172.18.121.48:8041            |            http://172.18.121.48:8041            | f43240abe5f3476ea64a8bd381fe4da7 |
| 64381b509bc84639b6a4710e6d99a23b | RegionOne |    http://172.18.121.48:8776/v1/%(tenant_id)s   |    http://172.18.121.48:8776/v1/%(tenant_id)s   |    http://172.18.121.48:8776/v1/%(tenant_id)s   | 7edc7bedf93d4f388185699b9793ec7f |
| 727d25775be54c9f8453f697ae5cb625 | RegionOne |          http://172.18.121.48:5000/v2.0         |          http://172.18.121.48:5000/v2.0         |         http://172.18.121.48:35357/v2.0         | 25e99a2a98f244d9a73bf965acdd39da |
| 9049338c57574b2d8ff8308b1a4265a5 | RegionOne |    http://172.18.121.48:8776/v2/%(tenant_id)s   |    http://172.18.121.48:8776/v2/%(tenant_id)s   |    http://172.18.121.48:8776/v2/%(tenant_id)s   | 6e070f0629094b72b66025250fdbda64 |
| c051c0f9649143f6b29eaf0895940abe | RegionOne |            http://172.18.121.48:9292            |            http://172.18.121.48:9292            |            http://172.18.121.48:9292            | 40874e10139a47eb88dfec2114047a34 |
| ee4a00c1e8334cb8921fa3f2a7c82f1b | RegionOne |           http://172.18.121.48:8774/v3          |           http://172.18.121.48:8774/v3          |           http://172.18.121.48:8774/v3          | 24c92ce4cd354e3db6c5ad59b8beeae8 |
| fa60b5ba0ab7436ab1ffebb2982d3ccc | RegionOne |      http://127.0.0.1:8776/v3/%(tenant_id)s     |      http://127.0.0.1:8776/v3/%(tenant_id)s     |      http://127.0.0.1:8776/v3/%(tenant_id)s     | b0b6b97cf9d649c9800300cc64b0e866 |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+-------------------------------------------------+----------------------------------+
[root at localhost ~(keystone_admin)]#


[root at localhost ~(keystone_admin)]# netstat -ntlp | grep 8774
[root at localhost ~(keystone_admin)]#

[root at localhost ~(keystone_admin)]# ps -ef | grep nova-api
nova     156427      1 86 07:51 ?        00:00:01 /usr/bin/python2 /usr/bin/nova-api
 [root at localhost ~(keystone_admin)]#

[root at localhost ~(keystone_admin)]#  lsof -i :8774
[root at localhost ~(keystone_admin)]#


[root at localhost ~(keystone_admin)]# keystone  user-list
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient.
  'python-keystoneclient.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning: Constructing an instance of the keystoneclient.v2_0.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead
  super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/site-packages/debtcollector/renames.py:45: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead
  return f(*args, **kwargs)
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:371: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/session.py:140: DeprecationWarning: keystoneclient.session.Session is deprecated as of the 2.1.0 release in favor of keystoneauth1.session.Session. It will be removed in future releases.
  DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py:56: DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in favor of keystoneauth1 plugins. They will be removed in future releases.
  'in future releases.', DeprecationWarning)
+----------------------------------+------------+---------+----------------------+
|                id                |    name    | enabled |        email         |
+----------------------------------+------------+---------+----------------------+
| 266f5859848e4f39b9725203dda5c3f2 |   admin    |   True  |    root at localhost    |
| 79a6ff3cc7cc4d018247c750adbc18e7 |    aodh    |   True  |    aodh at localhost    |
| 90f28a2a80054132a901d39da307213f | ceilometer |   True  | ceilometer at localhost |
| 16fa5ffa60e147d89ad84646b6519278 |   cinder   |   True  |   cinder at localhost   |
| c6312ec6c2c444288a412f32173fcd99 |   glance   |   True  |   glance at localhost   |
| ac8fb9c33d404a1697d576d428db90b3 |  gnocchi   |   True  |  gnocchi at localhost   |
| 1a5b4da4ed974ac8a6c78b752ac8fab6 |  neutron   |   True  |  neutron at localhost   |
| f21e8a15da5c40b7957416de4fa91b62 |    nova    |   True  |    nova at localhost    |
| b843358d7ae44944b11af38ce4b61f4d |   swift    |   True  |   swift at localhost    |
+----------------------------------+------------+---------+----------------------+
[root at localhost ~(keystone_admin)]#

[root at localhost ~(keystone_admin)]# nova-manage service list
Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
Option "notification_driver" from group "DEFAULT" is deprecated. Use option "driver" from group "oslo_messaging_notifications".
Option "notification_topics" from group "DEFAULT" is deprecated. Use option "topics" from group "oslo_messaging_notifications".
DEPRECATED: Use the nova service-* commands from python-novaclient instead or the os-services REST resource. The service subcommand will be removed in the 14.0 release.
Binary           Host                                 Zone             Status     State Updated_At
nova-osapi_compute 0.0.0.0                              internal         enabled    XXX   None
nova-metadata    0.0.0.0                              internal         enabled    XXX   None
nova-cert        localhost                               internal         enabled    XXX   2016-06-08 06:12:38
nova-consoleauth localhost                               internal         enabled    XXX   2016-06-08 06:12:37
nova-scheduler   localhost                               internal         enabled    XXX   2016-06-08 06:12:38
nova-conductor   localhost                               internal         enabled    XXX   2016-06-08 06:12:37
nova-compute     localhost                               nova             enabled    XXX   2016-06-08 06:12:43
[root at localhost ~(keystone_admin)]#

[root at localhost ~(keystone_admin)]# ls -l /var/log/nova/
total 4
-rw-r--r--. 1 nova nova   0 Jun  8 05:22 nova-api.log
-rw-r--r--. 1 nova nova   0 Jun  8 05:22 nova-cert.log
-rw-r--r--. 1 nova nova   0 Jun  8 05:22 nova-compute.log
-rw-r--r--. 1 nova nova   0 Jun  8 05:22 nova-conductor.log
-rw-r--r--. 1 nova nova   0 Jun  8 05:22 nova-consoleauth.log
-rw-r--r--. 1 root root   0 Jun  8 05:22 nova-manage.log
-rw-r--r--. 1 nova nova 995 Jun  8 05:32 nova-novncproxy.log
-rw-r--r--. 1 nova nova   0 Jun  8 05:23 nova-scheduler.log
[root at localhost ~(keystone_admin)]#


Regards,
Chinmaya


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20160608/1b76309e/attachment.html>


More information about the dev mailing list