8:32 a.m.
I've altered Keystone on my Icehouse cloud to use Apache/mod_ssl. The
Keystone and Nova clients are working (more or less) but I'm having
trouble with Glance.
Here's an example of the sort of error I'm seeing from the Glance api.log:
2014-07-15 14:24:00.551 24063 DEBUG
glance.api.middleware.version_negotiation [-] Determining version of
request: GET /v1/shared-images/e35356df747b4c5aa663fae2897facba
Accept: process_request
/usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:44
2014-07-15 14:24:00.552 24063 DEBUG
glance.api.middleware.version_negotiation [-] Using url versioning
process_request
/usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:57
2014-07-15 14:24:00.552 24063 DEBUG
glance.api.middleware.version_negotiation [-] Matched version: v1
process_request
/usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:69
2014-07-15 14:24:00.552 24063 DEBUG
glance.api.middleware.version_negotiation [-] new path
/v1/shared-images/e35356df747b4c5aa663fae2897facba process_request
/usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:70
2014-07-15 14:24:00.553 24063 DEBUG
keystoneclient.middleware.auth_token [-] Authenticating user token
__call__ /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:666
2014-07-15 14:24:00.553 24063 DEBUG
keystoneclient.middleware.auth_token [-] Removing headers from request
environment:
X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role
_remove_auth_headers
/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:725
2014-07-15 14:24:00.591 24063 INFO urllib3.connectionpool [-] Starting
new HTTPS connection (1): <hostname>
2014-07-15 14:24:01.921 24063 DEBUG urllib3.connectionpool [-] "POST
/v2.0/tokens HTTP/1.1" 200 7003 _make_request
/usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295
2014-07-15 14:24:01.931 24063 INFO urllib3.connectionpool [-] Starting
new HTTPS connection (1): <hostname>
2014-07-15 14:24:03.243 24063 DEBUG urllib3.connectionpool [-] "GET
/v2.0/tokens/revoked HTTP/1.1" 200 682 _make_request
/usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295
2014-07-15 14:24:03.252 24063 INFO urllib3.connectionpool [-] Starting
new HTTPS connection (1): <hostname>
2014-07-15 14:24:04.529 24063 DEBUG urllib3.connectionpool [-] "GET /
HTTP/1.1" 300 384 _make_request
/usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295
2014-07-15 14:24:04.530 24063 DEBUG
keystoneclient.middleware.auth_token [-] Server reports support for
api versions: v3.0 _get_supported_versions
/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:656
2014-07-15 14:24:04.531 24063 INFO
keystoneclient.middleware.auth_token [-] Auth Token confirmed use of
v3.0 apis
2014-07-15 14:24:04.531 24063 INFO urllib3.connectionpool [-] Starting
new HTTPS connection (1): <hostname>
2014-07-15 14:24:04.667 24063 DEBUG urllib3.connectionpool [-] "GET
/v3/OS-SIMPLE-CERT/certificates HTTP/1.1" 404 93 _make_request
/usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295
2014-07-15 14:24:04.669 24063 DEBUG
keystoneclient.middleware.auth_token [-] Token validation failure.
_validate_user_token
/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:943
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token Traceback (most recent call
last):
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 930, in _validate_user_token
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token verified =
self.verify_signed_token(user_token, token_ids)
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1347, in verify_signed_token
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token if
self.is_signed_token_revoked(token_ids):
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1299, in is_signed_token_revoked
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token if
self._is_token_id_in_revoked_list(token_id):
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1306, in _is_token_id_in_revoked_list
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token revocation_list =
self.token_revocation_list
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1413, in token_revocation_list
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token self.token_revocation_list =
self.fetch_revocation_list()
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1459, in fetch_revocation_list
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token return
self.cms_verify(data['signed'])
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1333, in cms_verify
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token self.fetch_signing_cert()
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1477, in fetch_signing_cert
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token
self._fetch_cert_file(self.signing_cert_file_name, 'signing')
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 1473, in _fetch_cert_file
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token raise
exceptions.CertificateConfigError(response.text)
2014-07-15 14:24:04.669 24063 TRACE
keystoneclient.middleware.auth_token CertificateConfigError: Unable to
load certificate. Ensure your system is configured properly.
2014-07-15 14:24:04.669 24063 TRACE keystoneclient.middleware.auth_token
2014-07-15 14:24:04.671 24063 DEBUG
keystoneclient.middleware.auth_token [-] Marking token as unauthorized
in cache _cache_store_invalid
/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:1239
2014-07-15 14:24:04.672 24063 WARNING
keystoneclient.middleware.auth_token [-] Authorization failed for
token
2014-07-15 14:24:04.672 24063 INFO
keystoneclient.middleware.auth_token [-] Invalid user token -
deferring reject downstream
2014-07-15 14:24:04.674 24063 INFO glance.wsgi.server [-] <IP address>
- - [15/Jul/2014 14:24:04] "GET
/v1/shared-images/e35356df747b4c5aa663fae2897facba HTTP/1.1" 401 381
4.124231
There is a bug report about a race condition involving Cinder, but
that was supposed to have been fixed.
Any suggestions appreciated.
Best Wishes,
Adam