Hello,
I am trying to redeploy wallaby undercloud using "container-tools:3.0" since I got a problem using the defaut stream.
The installation seems to go without issues, but now I am trying to import the definition of my nodes, I am getting this error message :
(undercloud) [stack@ooounder ~]$ openstack overcloud node import overnodes.json --debug
...
...
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None,
'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries':
'5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format'
: 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'activ
e', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('o
sc-lib', '2.3.1')], 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'default_domain': 'default', 'timing': False, 'inspect
or_api_version': '1', 'auth_url': 'https://10.168.24.2:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'identi
ty_api_version': '3', 'queues_api_version': '2', 'orchestration_api_version': '1', 'workflow_api_version': '2', 'database_api_version
': '1', 'share_api_version': '2.63', 'container_infra_api_version': '1', 'dns_api_version': '2', 'key_manager_api_version': '1', 'ala
rming_api_version': '2', 'baremetal_api_version': '1.69', 'loadbalancer_api_version': '2.0', 'data_processing_api_version': '1.1', 't
ripleoclient_api_version': '2', 'metrics_api_version': '1', 'event_api_version': '2', 'placement_api_version': '1.0', 'region_name':
'', 'auth_type': 'password', 'networks': []}
Using auth plugin: password
Using parameters {'auth_url': 'https://10.168.24.2:13000', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'ad
min', 'user_domain_name': 'Default', 'password': '***'}
Get auth_ref
REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET https://10.168.24.2:13000 -H "Accept: application/
json" -H "User-Agent: openstacksdk/0.55.0 keystoneauth1/4.3.1 python-requests/2.25.1 CPython/3.6.8"
Starting new HTTPS connection (1): 10.168.24.2:13000
https://10.168.24.2:13000 "GET / HTTP/1.1" 300 268
RESP: [300] Content-Length: 268 Content-Type: application/json Date: Tue, 15 Jun 2021 15:37:03 GMT Location: https://10.168.24.2:1300
0/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id: req-d482da26-c336-41ed-93fe-610b2d98fbac
RESP BODY: {"versions": {"values": [{"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self",
"href": "https://10.168.24.2:13000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v
3+json"}]}]}}
GET call to https://10.168.24.2:13000/ used request id req-d482da26-c336-41ed-93fe-610b2d98fbac
Making authentication request to https://10.168.24.2:13000/v3/auth/tokens
https://10.168.24.2:13000 "POST /v3/auth/tokens HTTP/1.1" 201 3615
{"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "06ff4d4c15664efe8af20d7224df59d3"
, "name": "admin", "password_expires_at": null}, "audit_ids": ["2KZLymr6ScOyqhAsJUXbfQ"], "expires_at": "2021-06-15T19:37:03.000000Z"
, "issued_at": "2021-06-15T15:37:03.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "8308c2e160204978b86e
5fcb5d8b524c", "name": "admin"}, "is_domain": false, "roles": [{"id": "115bc5caafd245e99859f2edc8cebcdf", "name": "member"}, {"id": "
4c8c8681ff924e3f82c6ec9f6385e77a", "name": "_member_"}, {"id": "db493429c9be44c499027b80f06fa3a8", "name": "admin"}, {"id": "5f20877f
d53c47258a8c1d50361a8db4", "name": "reader"}], "catalog": [{"endpoints": [{"id": "24adacb7cded408dac4663143018f192", "interface": "pu
blic", "region_id": "regionOne", "url": "https://10.168.24.2:13000", "region": "regionOne"}, {"id": "7c4fe5c4d13c486f8a0de078e4a50381
", "interface": "internal", "region_id": "regionOne", "url": "http://10.168.24.3:5000", "region": "regionOne"}, {"id": "ca18e36f33284
239907ec4d88040a6e4", "interface": "admin", "region_id": "regionOne", "url": "http://10.168.24.3:35357", "region": "regionOne"}], "id
": "12ab546ac7004930ba0a54d99fe36919", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "1bb5ba46095f4d679bf056f7e460c7
14", "interface": "admin", "region_id": "regionOne", "url": "http://10.168.24.3:6385", "region": "regionOne"}, {"id": "7fdd0d20af114f
f2a83d53b340533cb3", "interface": "internal", "region_id": "regionOne", "url": "http://10.168.24.3:6385", "region": "regionOne"}, {"i
d": "8884ad607c2e479984cd3e0a894352db", "interface": "public", "region_id": "regionOne", "url": "https://10.168.24.2:13385", "region"
: "regionOne"}], "id": "4e48a20808a94dcb9b14adcf81eb9a68", "type": "baremetal", "name": "ironic"}, {"endpoints": [{"id": "14e31cce879
64d4ab26ecfacbcf365e3", "interface": "internal", "region_id": "regionOne", "url": "http://10.168.24.3:5050", "region": "regionOne"},
{"id": "5add117abc43462a8901ec486bde8c55", "interface": "public", "region_id": "regionOne", "url": "https://10.168.24.2:13050", "regi
on": "regionOne"}, {"id": "d3009b5b54d74f2a842f6b3e4b0019c7", "interface": "admin", "region_id": "regionOne", "url": "http://10.168.2
4.3:5050", "region": "regionOne"}], "id": "639c382276be488b8719f3ddf321ec00", "type": "baremetal-introspection", "name": "ironic-insp
ector"}, {"endpoints": [{"id": "8107f8f6c79a4aaa84f7538fb17ce394", "interface": "admin", "region_id": "regionOne", "url": "http://10.
168.24.3:8004/v1/8308c2e160204978b86e5fcb5d8b524c", "region": "regionOne"}, {"id": "ef09fc92358d476a88f8d33b02a6bf23", "interface": "
public", "region_id": "regionOne", "url": "https://10.168.24.2:13004/v1/8308c2e160204978b86e5fcb5d8b524c", "region": "regionOne"}, {"
id": "f0dfb22acd124c33933687b9398f4351", "interface": "internal", "region_id": "regionOne", "url": "http://10.168.24.3:8004/v1/8308c2
e160204978b86e5fcb5d8b524c", "region": "regionOne"}], "id": "88ced00488484104a9dcf0a2e1834084", "type": "orchestration", "name": "hea
t"}, {"endpoints": [{"id": "1de6dfdb1a2e489cb83fdbeaa3a232d7", "interface": "internal", "region_id": "regionOne", "url": "http://10.1
68.24.3:9696", "region": "regionOne"}, {"id": "8894b604813547159b982c7df131b232", "interface": "public", "region_id": "regionOne", "u
rl": "https://10.168.24.2:13696", "region": "regionOne"}, {"id": "cfa5ee424c7e46268075c79a7e29314d", "interface": "admin", "region_id
": "regionOne", "url": "http://10.168.24.3:9696", "region": "regionOne"}], "id": "b1bde18dbab2440ebe8474b53ad0cba2", "type": "network
", "name": "neutron"}]}}
Directory "/home/stack/.tripleo" was not created because it already exists.
run(Namespace(concurrency=20, env_file=<_io.TextIOWrapper name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/lib/ironi
c/httpboot', instance_boot_option=None, introspect=False, no_deploy_image=False, provide=False, run_validations=False, validate_only=
False))
take_action(Namespace(concurrency=20, env_file=<_io.TextIOWrapper name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/l
ib/ironic/httpboot', instance_boot_option=None, introspect=False, no_deploy_image=False, provide=False, run_validations=False, valida
te_only=False))
Instantiating baremetal client: <class 'ironicclient.v1.client.Client'>
Baremetal API version: latest
Registering all nodes.
Populating list of registered nodes.
REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET https://10.168.24.2:13385/v1/nodes/detail -H "Acce
pt: application/json" -H "Content-Type: application/json" -H "User-Agent: python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495
887eff3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H "X-OpenStack-Ironic-API-Version: 1.69"
Starting new HTTPS connection (1): 10.168.24.2:13385
https://10.168.24.2:13385 "GET /v1/nodes/detail HTTP/1.1" 200 None
RESP: [200] Content-Encoding: gzip Content-Type: application/json Date: Tue, 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-fc046
05f-b359-4f83-bf2e-c92b0daccaf9 Server: Apache Transfer-Encoding: chunked Vary: Accept-Encoding X-OpenStack-Ironic-API-Maximum-Versio
n: 1.74 X-OpenStack-Ironic-API-Minimum-Version: 1.1 X-OpenStack-Ironic-API-Version: 1.69
RESP BODY: {"nodes": []}
... REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST https://10.168.24.2:13385/v1/nodes -H "Accept: ap
plication/json" -H "Content-Type: application/json" -H "User-Agent: python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties": {"cpus
": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"}, "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
boot/agent.kernel", "rescue_kernel": "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk": "file:///var/lib/ironic/httpbo
ot/agent.ramdisk", "rescue_ramdisk": "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username": "poweruser", "ipmi_password":
"ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"}, "resource_class": "baremetal", "name": "controller-a"}'
https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139
RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue, 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-9de57250
-ea09-4490-ab9c-e5bfed236412 Server: Apache X-OpenStack-Ironic-API-Maximum-Version: 1.74 X-OpenStack-Ironic-API-Minimum-Version: 1.1
X-OpenStack-Ironic-API-Version: 1.69
RESP BODY: {"error_message": "{\"faultcode\": \"Server\", \"faultstring\": \"Resource temporarily unavailable, please retry.\", \"deb
uginfo\": null}"}
Error contacting Ironic server: Resource temporarily unavailable, please retry. (HTTP 503). Attempt 1 of 6
REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST https://10.168.24.2:13385/v1/nodes -H "Accept: ap
plication/json" -H "Content-Type: application/json" -H "User-Agent: python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties": {"cpus
": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"}, "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
boot/agent.kernel", "rescue_kernel": "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk": "file:///var/lib/ironic/httpbo
ot/agent.ramdisk", "rescue_ramdisk": "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username": "poweruser", "ipmi_password":
"ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"}, "resource_class": "baremetal", "name": "controller-a"}'
https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139
RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue, 15 Jun 2021 15:37:05 GMT Openstack-Request-Id: req-3a448355
-c367-46ad-b968-081fdd077487 Server: Apache X-OpenStack-Ironic-API-Maximum-Version: 1.74 X-OpenStack-Ironic-API-Minimum-Version: 1.1
X-OpenStack-Ironic-API-Version: 1.69
RESP BODY: {"error_message": "{\"faultcode\": \"Server\", \"faultstring\": \"Resource temporarily unavailable, please retry.\", \"deb
uginfo\": null}"}
Error contacting Ironic server: Resource temporarily unavailable, please retry. (HTTP 503). Attempt 2 of 6
REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST https://10.168.24.2:13385/v1/nodes -H "Accept: ap
plication/json" -H "Content-Type: application/json" -H "User-Agent: python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties": {"cpus
": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"}, "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
boot/agent.kernel", "rescue_kernel": "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk": "file:///var/lib/ironic/httpbo
ot/agent.ramdisk", "rescue_ramdisk": "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username": "poweruser", "ipmi_password":
"ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"}, "resource_class": "baremetal", "name": "controller-a"}'
https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139
RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue, 15 Jun 2021 15:37:07 GMT Openstack-Request-Id: req-159b74d8
-183e-4f84-9a86-569e7f29d313 Server: Apache X-OpenStack-Ironic-API-Maximum-Version: 1.74 X-OpenStack-Ironic-API-Minimum-Version: 1.1
X-OpenStack-Ironic-API-Version: 1.69
RESP BODY: {"error_message": "{\"faultcode\": \"Server\", \"faultstring\": \"Resource temporarily unavailable, please retry.\", \"deb
uginfo\": null}"}
... ..
From /var/log/containers/ironic/ironic-conductor.log
I get this
2021-06-15 16:39:04.163 7 ERROR oslo_service.service [req-94d7c168-4c7f-4601-a302-3fd52b9132af - - - - -] Error starting thread.: iro
nic.common.exception.DriverNotFoundInEntrypoint: Could not find the following items in the 'ironic.hardware.interfaces.deploy' entryp
oint: iscsi.
2021-06-15 16:39:04.163 7 ERROR oslo_service.service Traceback (most recent call last):
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 807, in
run_service
2021-06-15 16:39:04.163 7 ERROR oslo_service.service service.start()
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/common/rpc_service.py", line 63,
in start
2021-06-15 16:39:04.163 7 ERROR oslo_service.service self.manager.init_host(admin_context)
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line
171, in init_host
2021-06-15 16:39:04.163 7 ERROR oslo_service.service self._collect_periodic_tasks(admin_context)
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line
300, in _collect_periodic_tasks
2021-06-15 16:39:04.163 7 ERROR oslo_service.service for ifaces in driver_factory.all_interfaces().values():
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
286, in all_interfaces
2021-06-15 16:39:04.163 7 ERROR oslo_service.service return {iface: interfaces(iface) for iface in _INTERFACE_LOADERS}
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
286, in <dictcomp>
2021-06-15 16:39:04.163 7 ERROR oslo_service.service return {iface: interfaces(iface) for iface in _INTERFACE_LOADERS}
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
277, in interfaces
2021-06-15 16:39:04.163 7 ERROR oslo_service.service return _get_all_drivers(_INTERFACE_LOADERS[interface_type]())
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
338, in __init__
2021-06-15 16:39:04.163 7 ERROR oslo_service.service self.__class__._init_extension_manager()
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 36
0, in inner
2021-06-15 16:39:04.163 7 ERROR oslo_service.service return f(*args, **kwargs)
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
408, in _init_extension_manager
2021-06-15 16:39:04.163 7 ERROR oslo_service.service on_missing_entrypoints_callback=missing_callback))
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/stevedore/named.py", line 85, in __init
__
2021-06-15 16:39:04.163 7 ERROR oslo_service.service on_missing_entrypoints_callback(self._missing_names)
2021-06-15 16:39:04.163 7 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
399, in missing_callback
2021-06-15 16:39:04.163 7 ERROR oslo_service.service names=names, entrypoint=cls._entrypoint_name)
2021-06-15 16:39:04.163 7 ERROR oslo_service.service ironic.common.exception.DriverNotFoundInEntrypoint: Could not find the following
items in the 'ironic.hardware.interfaces.deploy' entrypoint: iscsi.
2021-06-15 16:39:04.163 7 ERROR oslo_service.service
2021-06-15 16:39:05.172 7 WARNING amqp [-] Received method (60, 30) during closing channel 1. This method will be ignored: SystemExit
: 1
2021-06-15 16:39:05.174 7 INFO ironic.common.rpc_service [-] Stopped RPC server for service ironic.conductor_manager on host localhos
t.localdomain.
2021-06-15 16:39:05.174 7 DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" lock /usr/lib/python3.6/site-packages/o
slo_concurrency/lockutils.py:266
2021-06-15 16:39:05.175 7 DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" lock /usr/lib/python3.6/site-packages/
oslo_concurrency/lockutils.py:282
Any ideas?
Regards.