[rdo-users] Error importing node definition on undercloud Wallaby

Tue Jun 15 15:48:00 UTC 2021

You're hitting https://bugs.launchpad.net/tripleo/+bug/1931995 because
wallaby incorrectly fetches master containers. iscsi has been removed from
master
https://review.opendev.org/c/openstack/tripleo-heat-templates/+/749475.
Currently the tripleowallaby containers are out of sync on quay.io and we
need to land https://review.opendev.org/c/openstack/tripleo-common/+/796516.

On Tue, Jun 15, 2021 at 9:42 AM wodel youchi <wodel.youchi at gmail.com> wrote:

> Hello,
>
> I am trying to redeploy wallaby undercloud using "container-tools:3.0"
> since I got a problem using the defaut stream.
>
> The installation seems to go without issues, but now I am trying to import
> the definition of my nodes, I am getting this error message :
>
> (undercloud) [stack at ooounder ~]$ openstack overcloud node import
>  overnodes.json --debug
> ...
> ...
> auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert':
> '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None,
> 'key': None, 'baremetal_status_code_retries': '5',
> 'baremetal_introspection_status_code_retries': '5',
> 'image_status_code_retries':
> '5', 'disable_vendor_agent': {}, 'interface': 'public',
> 'floating_ip_source': 'neutron', 'image_api_use_tasks': False,
> 'image_format'
> : 'qcow2', 'message': '', 'network_api_version': '2',
> 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status':
> 'activ
> e', 'auth': {'user_domain_name': 'Default', 'project_domain_name':
> 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('o
> sc-lib', '2.3.1')], 'verbose_level': 3, 'deferred_help': False, 'debug':
> True, 'default_domain': 'default', 'timing': False, 'inspect
> or_api_version': '1', 'auth_url': 'https://10.168.24.2:13000',
> 'username': 'admin', 'password': '***', 'beta_command': False, 'identi
> ty_api_version': '3', 'queues_api_version': '2',
> 'orchestration_api_version': '1', 'workflow_api_version': '2',
> 'database_api_version
> ': '1', 'share_api_version': '2.63', 'container_infra_api_version': '1',
> 'dns_api_version': '2', 'key_manager_api_version': '1', 'ala
> rming_api_version': '2', 'baremetal_api_version': '1.69',
> 'loadbalancer_api_version': '2.0', 'data_processing_api_version': '1.1', 't
> ripleoclient_api_version': '2', 'metrics_api_version': '1',
> 'event_api_version': '2', 'placement_api_version': '1.0', 'region_name':
> '', 'auth_type': 'password', 'networks': []}
> Using auth plugin: password
> Using parameters {'auth_url': 'https://10.168.24.2:13000',
> 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'ad
> min', 'user_domain_name': 'Default', 'password': '***'}
> Get auth_ref
> REQ: curl -g -i --cacert
> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET
> https://10.168.24.2:13000 -H "Accept: application/
> json" -H "User-Agent: openstacksdk/0.55.0 keystoneauth1/4.3.1
> python-requests/2.25.1 CPython/3.6.8"
> Starting new HTTPS connection (1): 10.168.24.2:13000
> https://10.168.24.2:13000 "GET / HTTP/1.1" 300 268
> RESP: [300] Content-Length: 268 Content-Type: application/json Date: Tue,
> 15 Jun 2021 15:37:03 GMT Location: https://10.168.24.2:1300
> 0/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id:
> req-d482da26-c336-41ed-93fe-610b2d98fbac
> RESP BODY: {"versions": {"values": [{"id": "v3.14", "status": "stable",
> "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self",
> "href": "https://10.168.24.2:13000/v3/"}], "media-types": [{"base":
> "application/json", "type": "application/vnd.openstack.identity-v
> 3+json"}]}]}}
> GET call to https://10.168.24.2:13000/ used request id
> req-d482da26-c336-41ed-93fe-610b2d98fbac
> Making authentication request to https://10.168.24.2:13000/v3/auth/tokens
> https://10.168.24.2:13000 "POST /v3/auth/tokens HTTP/1.1" 201 3615
> {"token": {"methods": ["password"], "user": {"domain": {"id": "default",
> "name": "Default"}, "id": "06ff4d4c15664efe8af20d7224df59d3"
> , "name": "admin", "password_expires_at": null}, "audit_ids":
> ["2KZLymr6ScOyqhAsJUXbfQ"], "expires_at": "2021-06-15T19:37:03.000000Z"
> , "issued_at": "2021-06-15T15:37:03.000000Z", "project": {"domain": {"id":
> "default", "name": "Default"}, "id": "8308c2e160204978b86e
> 5fcb5d8b524c", "name": "admin"}, "is_domain": false, "roles": [{"id":
> "115bc5caafd245e99859f2edc8cebcdf", "name": "member"}, {"id": "
> 4c8c8681ff924e3f82c6ec9f6385e77a", "name": "_member_"}, {"id":
> "db493429c9be44c499027b80f06fa3a8", "name": "admin"}, {"id": "5f20877f
> d53c47258a8c1d50361a8db4", "name": "reader"}], "catalog": [{"endpoints":
> [{"id": "24adacb7cded408dac4663143018f192", "interface": "pu
> blic", "region_id": "regionOne", "url": "https://10.168.24.2:13000",
> "region": "regionOne"}, {"id": "7c4fe5c4d13c486f8a0de078e4a50381
> ", "interface": "internal", "region_id": "regionOne", "url": "
> http://10.168.24.3:5000", "region": "regionOne"}, {"id": "ca18e36f33284
> 239907ec4d88040a6e4", "interface": "admin", "region_id": "regionOne",
> "url": "http://10.168.24.3:35357", "region": "regionOne"}], "id
> ": "12ab546ac7004930ba0a54d99fe36919", "type": "identity", "name":
> "keystone"}, {"endpoints": [{"id": "1bb5ba46095f4d679bf056f7e460c7
> 14", "interface": "admin", "region_id": "regionOne", "url": "
> http://10.168.24.3:6385", "region": "regionOne"}, {"id": "7fdd0d20af114f
> f2a83d53b340533cb3", "interface": "internal", "region_id": "regionOne",
> "url": "http://10.168.24.3:6385", "region": "regionOne"}, {"i
> d": "8884ad607c2e479984cd3e0a894352db", "interface": "public",
> "region_id": "regionOne", "url": "https://10.168.24.2:13385", "region"
> : "regionOne"}], "id": "4e48a20808a94dcb9b14adcf81eb9a68", "type":
> "baremetal", "name": "ironic"}, {"endpoints": [{"id": "14e31cce879
> 64d4ab26ecfacbcf365e3", "interface": "internal", "region_id": "regionOne",
> "url": "http://10.168.24.3:5050", "region": "regionOne"},
> {"id": "5add117abc43462a8901ec486bde8c55", "interface": "public",
> "region_id": "regionOne", "url": "https://10.168.24.2:13050", "regi
> on": "regionOne"}, {"id": "d3009b5b54d74f2a842f6b3e4b0019c7", "interface":
> "admin", "region_id": "regionOne", "url": "http://10.168.2
> 4.3:5050", "region": "regionOne"}], "id":
> "639c382276be488b8719f3ddf321ec00", "type": "baremetal-introspection",
> "name": "ironic-insp
> ector"}, {"endpoints": [{"id": "8107f8f6c79a4aaa84f7538fb17ce394",
> "interface": "admin", "region_id": "regionOne", "url": "http://10.
> 168.24.3:8004/v1/8308c2e160204978b86e5fcb5d8b524c", "region":
> "regionOne"}, {"id": "ef09fc92358d476a88f8d33b02a6bf23", "interface": "
> public", "region_id": "regionOne", "url": "
> https://10.168.24.2:13004/v1/8308c2e160204978b86e5fcb5d8b524c", "region":
> "regionOne"}, {"
> id": "f0dfb22acd124c33933687b9398f4351", "interface": "internal",
> "region_id": "regionOne", "url": "http://10.168.24.3:8004/v1/8308c2
> e160204978b86e5fcb5d8b524c", "region": "regionOne"}], "id":
> "88ced00488484104a9dcf0a2e1834084", "type": "orchestration", "name": "hea
> t"}, {"endpoints": [{"id": "1de6dfdb1a2e489cb83fdbeaa3a232d7",
> "interface": "internal", "region_id": "regionOne", "url": "http://10.1
> 68.24.3:9696", "region": "regionOne"}, {"id":
> "8894b604813547159b982c7df131b232", "interface": "public", "region_id":
> "regionOne", "u
> rl": "https://10.168.24.2:13696", "region": "regionOne"}, {"id":
> "cfa5ee424c7e46268075c79a7e29314d", "interface": "admin", "region_id
> ": "regionOne", "url": "http://10.168.24.3:9696", "region":
> "regionOne"}], "id": "b1bde18dbab2440ebe8474b53ad0cba2", "type": "network
> ", "name": "neutron"}]}}
> Directory "/home/stack/.tripleo" was not created because it already
> exists.
> run(Namespace(concurrency=20, env_file=<_io.TextIOWrapper
> name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/lib/ironi
> c/httpboot', instance_boot_option=None, introspect=False,
> no_deploy_image=False, provide=False, run_validations=False, validate_only=
> False))
> take_action(Namespace(concurrency=20, env_file=<_io.TextIOWrapper
> name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/l
> ib/ironic/httpboot', instance_boot_option=None, introspect=False,
> no_deploy_image=False, provide=False, run_validations=False, valida
> te_only=False))
> Instantiating baremetal client: <class 'ironicclient.v1.client.Client'>
> Baremetal API version: latest
> Registering all nodes.
> Populating list of registered nodes.
> REQ: curl -g -i --cacert
> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET
> https://10.168.24.2:13385/v1/nodes/detail -H "Acce
> pt: application/json" -H "Content-Type: application/json" -H "User-Agent:
> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495
> 887eff3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
> "X-OpenStack-Ironic-API-Version: 1.69"
> Starting new HTTPS connection (1): 10.168.24.2:13385
> https://10.168.24.2:13385 "GET /v1/nodes/detail HTTP/1.1" 200 None
> RESP: [200] Content-Encoding: gzip Content-Type: application/json Date:
> Tue, 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-fc046
> 05f-b359-4f83-bf2e-c92b0daccaf9 Server: Apache Transfer-Encoding: chunked
> Vary: Accept-Encoding X-OpenStack-Ironic-API-Maximum-Versio
> n: 1.74 X-OpenStack-Ironic-API-Minimum-Version: 1.1
> X-OpenStack-Ironic-API-Version: 1.69
> RESP BODY: {"nodes": []}
> ...
> REQ: curl -g -i --cacert
> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST
> https://10.168.24.2:13385/v1/nodes -H "Accept: ap
> plication/json" -H "Content-Type: application/json" -H "User-Agent:
> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
> 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
> "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties":
> {"cpus
> ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"},
> "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
> boot/agent.kernel", "rescue_kernel":
> "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk":
> "file:///var/lib/ironic/httpbo
> ot/agent.ramdisk", "rescue_ramdisk":
> "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username":
> "poweruser", "ipmi_password":
> "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"},
> "resource_class": "baremetal", "name": "controller-a"}'
> https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139
> RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue,
> 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-9de57250
> -ea09-4490-ab9c-e5bfed236412 Server: Apache
> X-OpenStack-Ironic-API-Maximum-Version: 1.74
> X-OpenStack-Ironic-API-Minimum-Version: 1.1
> X-OpenStack-Ironic-API-Version: 1.69
> RESP BODY: {"error_message": "{\"faultcode\": \"Server\", \"faultstring\":
> \"Resource temporarily unavailable, please retry.\", \"deb
> uginfo\": null}"}
> Error contacting Ironic server: Resource temporarily unavailable, please
> retry. (HTTP 503). Attempt 1 of 6
> REQ: curl -g -i --cacert
> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST
> https://10.168.24.2:13385/v1/nodes -H "Accept: ap
> plication/json" -H "Content-Type: application/json" -H "User-Agent:
> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
> 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
> "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties":
> {"cpus
> ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"},
> "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
> boot/agent.kernel", "rescue_kernel":
> "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk":
> "file:///var/lib/ironic/httpbo
> ot/agent.ramdisk", "rescue_ramdisk":
> "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username":
> "poweruser", "ipmi_password":
> "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"},
> "resource_class": "baremetal", "name": "controller-a"}'
> https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139
> RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue,
> 15 Jun 2021 15:37:05 GMT Openstack-Request-Id: req-3a448355
> -c367-46ad-b968-081fdd077487 Server: Apache
> X-OpenStack-Ironic-API-Maximum-Version: 1.74
> X-OpenStack-Ironic-API-Minimum-Version: 1.1
> X-OpenStack-Ironic-API-Version: 1.69
> RESP BODY: {"error_message": "{\"faultcode\": \"Server\", \"faultstring\":
> \"Resource temporarily unavailable, please retry.\", \"deb
> uginfo\": null}"}
> Error contacting Ironic server: Resource temporarily unavailable, please
> retry. (HTTP 503). Attempt 2 of 6
> REQ: curl -g -i --cacert
> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST
> https://10.168.24.2:13385/v1/nodes -H "Accept: ap
> plication/json" -H "Content-Type: application/json" -H "User-Agent:
> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
> 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
> "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties":
> {"cpus
> ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"},
> "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
> boot/agent.kernel", "rescue_kernel":
> "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk":
> "file:///var/lib/ironic/httpbo
> ot/agent.ramdisk", "rescue_ramdisk":
> "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username":
> "poweruser", "ipmi_password":
> "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"},
> "resource_class": "baremetal", "name": "controller-a"}'
>
>
>
>
>
> *https://10.168.24.2:13385 <https://10.168.24.2:13385> "POST /v1/nodes
> HTTP/1.1" 503 139 RESP: [503] Content-Length: 139 Content-Type:
> application/json Date: Tue, 15 Jun 2021 15:37:07 GMT Openstack-Request-Id:
> req-159b74d8-183e-4f84-9a86-569e7f29d313 Server: Apache
> X-OpenStack-Ironic-API-Maximum-Version: 1.74
> X-OpenStack-Ironic-API-Minimum-Version: 1.1 X-OpenStack-Ironic-API-Version:
> 1.69 RESP BODY: {"error_message": "{\"faultcode\": \"Server\",
> \"faultstring\": \"Resource temporarily unavailable, please retry.\",
> \"debuginfo\": null}"}*
>
> ...
> ..
>
>
> From /var/log/containers/ironic/ironic-conductor.log
> I get this
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
> [req-94d7c168-4c7f-4601-a302-3fd52b9132af - - - - -] Error starting
> thread.: iro
> nic.common.exception.DriverNotFoundInEntrypoint: Could not find the
> following items in the 'ironic.hardware.interfaces.deploy' entryp
> oint: iscsi.
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service Traceback (most
> recent call last):
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 807, in
> run_service
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     service.start()
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/common/rpc_service.py", line 63,
> in start
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>     self.manager.init_host(admin_context)
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line
> 171, in init_host
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>     self._collect_periodic_tasks(admin_context)
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line
> 300, in _collect_periodic_tasks
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     for ifaces in
> driver_factory.all_interfaces().values():
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
> 286, in all_interfaces
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return {iface:
> interfaces(iface) for iface in _INTERFACE_LOADERS}
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
> 286, in <dictcomp>
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return {iface:
> interfaces(iface) for iface in _INTERFACE_LOADERS}
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
> 277, in interfaces
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return
> _get_all_drivers(_INTERFACE_LOADERS[interface_type]())
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
> 338, in __init__
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>     self.__class__._init_extension_manager()
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 36
> 0, in inner
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return f(*args,
> **kwargs)
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
> 408, in _init_extension_manager
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>     on_missing_entrypoints_callback=missing_callback))
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/stevedore/named.py", line 85, in __init
> __
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>     on_missing_entrypoints_callback(self._missing_names)
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
> 399, in missing_callback
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     names=names,
> entrypoint=cls._entrypoint_name)
>
> *2021-06-15 16:39:04.163 7 ERROR oslo_service.service
> ironic.common.exception.DriverNotFoundInEntrypoint: Could not find the
> following items in the 'ironic.hardware.interfaces.deploy' entrypoint:
> iscsi. *
> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
> 2021-06-15 16:39:05.172 7 WARNING amqp [-] Received method (60, 30) during
> closing channel 1. This method will be ignored: SystemExit
> : 1
>
> *2021-06-15 16:39:05.174 7 INFO ironic.common.rpc_service [-] Stopped RPC
> server for service ironic.conductor_manager on host localhost.localdomain. *
> 2021-06-15 16:39:05.174 7 DEBUG oslo_concurrency.lockutils [-] Acquired
> lock "singleton_lock" lock /usr/lib/python3.6/site-packages/o
> slo_concurrency/lockutils.py:266
> 2021-06-15 16:39:05.175 7 DEBUG oslo_concurrency.lockutils [-] Releasing
> lock "singleton_lock" lock /usr/lib/python3.6/site-packages/
> oslo_concurrency/lockutils.py:282
>
>
> Any ideas?
>
> Regards.
> _______________________________________________
> users mailing list
> users at lists.rdoproject.org
> http://lists.rdoproject.org/mailman/listinfo/users
>
> To unsubscribe: users-unsubscribe at lists.rdoproject.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/users/attachments/20210615/cd400a05/attachment-0001.html>