Re: [Rdo-list] Glance/Keystone problem

Hi Flavio, Thanks for looking. In the end, the cause here was an omission in the api-paste file for Keystone, now fixed. Best Wishes, Adam On Wed, Jul 16, 2014 at 9:11 AM, Flavio Percoco <flavio(a)redhat.com&gt; wrote: > On 07/15/2014 03:32 PM, Adam Huffman wrote: >> I've altered Keystone on my Icehouse cloud to use Apache/mod_ssl. The >> Keystone and Nova clients are working (more or less) but I'm having >> trouble with Glance. > > Hi Adam, > > We'd need your config files to have a better idea of what the issue > could be. Based on the logs you just sent, keystone's middleware can't > find/load the certification file: > > "Unable to load certificate. Ensure your system is configured properly" > > Some things you could check: > > 1. Is the file path in your config file correct? > 2. Is the config option name correct? > 3. Is the file readable? > > Hope the above helps, > Flavio > > >> >> Here's an example of the sort of error I'm seeing from the Glance api.log: >> >> >> 2014-07-15 14:24:00.551 24063 DEBUG >> glance.api.middleware.version_negotiation [-] Determining version of >> request: GET /v1/shared-images/e35356df747b4c5aa663fae2897facba >> Accept: process_request >> /usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:44 >> 2014-07-15 14:24:00.552 24063 DEBUG >> glance.api.middleware.version_negotiation [-] Using url versioning >> process_request >> /usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:57 >> 2014-07-15 14:24:00.552 24063 DEBUG >> glance.api.middleware.version_negotiation [-] Matched version: v1 >> process_request >> /usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:69 >> 2014-07-15 14:24:00.552 24063 DEBUG >> glance.api.middleware.version_negotiation [-] new path >> /v1/shared-images/e35356df747b4c5aa663fae2897facba process_request >> /usr/lib/python2.6/site-packages/glance/api/middleware/version_negotiation.py:70 >> 2014-07-15 14:24:00.553 24063 DEBUG >> keystoneclient.middleware.auth_token [-] Authenticating user token >> __call__ /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:666 >> 2014-07-15 14:24:00.553 24063 DEBUG >> keystoneclient.middleware.auth_token [-] Removing headers from request >> environment: X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role >> _remove_auth_headers >> /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:725 >> 2014-07-15 14:24:00.591 24063 INFO urllib3.connectionpool [-] Starting >> new HTTPS connection (1): <hostname> >> 2014-07-15 14:24:01.921 24063 DEBUG urllib3.connectionpool [-] "POST >> /v2.0/tokens HTTP/1.1" 200 7003 _make_request >> /usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295 >> 2014-07-15 14:24:01.931 24063 INFO urllib3.connectionpool [-] Starting >> new HTTPS connection (1): <hostname> >> 2014-07-15 14:24:03.243 24063 DEBUG urllib3.connectionpool [-] "GET >> /v2.0/tokens/revoked HTTP/1.1" 200 682 _make_request >> /usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295 >> 2014-07-15 14:24:03.252 24063 INFO urllib3.connectionpool [-] Starting >> new HTTPS connection (1): <hostname> >> 2014-07-15 14:24:04.529 24063 DEBUG urllib3.connectionpool [-] "GET / >> HTTP/1.1" 300 384 _make_request >> /usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295 >> 2014-07-15 14:24:04.530 24063 DEBUG >> keystoneclient.middleware.auth_token [-] Server reports support for >> api versions: v3.0 _get_supported_versions >> /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:656 >> 2014-07-15 14:24:04.531 24063 INFO >> keystoneclient.middleware.auth_token [-] Auth Token confirmed use of >> v3.0 apis >> 2014-07-15 14:24:04.531 24063 INFO urllib3.connectionpool [-] Starting >> new HTTPS connection (1): <hostname> >> 2014-07-15 14:24:04.667 24063 DEBUG urllib3.connectionpool [-] "GET >> /v3/OS-SIMPLE-CERT/certificates HTTP/1.1" 404 93 _make_request >> /usr/lib/python2.6/site-packages/urllib3/connectionpool.py:295 >> 2014-07-15 14:24:04.669 24063 DEBUG >> keystoneclient.middleware.auth_token [-] Token validation failure. >> _validate_user_token >> /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:943 >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token Traceback (most recent call >> last): >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 930, in _validate_user_token >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token verified = >> self.verify_signed_token(user_token, token_ids) >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1347, in verify_signed_token >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token if >> self.is_signed_token_revoked(token_ids): >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1299, in is_signed_token_revoked >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token if >> self._is_token_id_in_revoked_list(token_id): >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1306, in _is_token_id_in_revoked_list >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token revocation_list = >> self.token_revocation_list >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1413, in token_revocation_list >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token self.token_revocation_list = >> self.fetch_revocation_list() >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1459, in fetch_revocation_list >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token return >> self.cms_verify(data['signed']) >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1333, in cms_verify >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token self.fetch_signing_cert() >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1477, in fetch_signing_cert >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token >> self._fetch_cert_file(self.signing_cert_file_name, 'signing') >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token File >> "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", >> line 1473, in _fetch_cert_file >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token raise >> exceptions.CertificateConfigError(response.text) >> 2014-07-15 14:24:04.669 24063 TRACE >> keystoneclient.middleware.auth_token CertificateConfigError: Unable to >> load certificate. Ensure your system is configured properly. >> 2014-07-15 14:24:04.669 24063 TRACE keystoneclient.middleware.auth_token >> 2014-07-15 14:24:04.671 24063 DEBUG >> keystoneclient.middleware.auth_token [-] Marking token as unauthorized >> in cache _cache_store_invalid >> /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:1239 >> 2014-07-15 14:24:04.672 24063 WARNING >> keystoneclient.middleware.auth_token [-] Authorization failed for >> token >> 2014-07-15 14:24:04.672 24063 INFO >> keystoneclient.middleware.auth_token [-] Invalid user token - >> deferring reject downstream >> 2014-07-15 14:24:04.674 24063 INFO glance.wsgi.server [-] <IP address> >> - - [15/Jul/2014 14:24:04] "GET >> /v1/shared-images/e35356df747b4c5aa663fae2897facba HTTP/1.1" 401 381 >> 4.124231 >> >> There is a bug report about a race condition involving Cinder, but >> that was supposed to have been fixed. >> >> Any suggestions appreciated. >> >> Best Wishes, >> Adam >> >> _______________________________________________ >> Rdo-list mailing list >> Rdo-list(a)redhat.com >> https://www.redhat.com/mailman/listinfo/rdo-list >> > > > -- > @flaper87 > Flavio Percoco > > _______________________________________________ > Rdo-list mailing list > Rdo-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list