Re: [rdo-users] Error importing node definition on undercloud Wallaby

You're hitting https://bugs.launchpad.net/tripleo/+bug/1931995 because wallaby incorrectly fetches master containers. iscsi has been removed from master https://review.opendev.org/c/openstack/tripleo-heat-templates/+/749475. Currently the tripleowallaby containers are out of sync on quay.io and we need to land https://review.opendev.org/c/openstack/tripleo-common/+/796516. On Tue, Jun 15, 2021 at 9:42 AM wodel youchi <wodel.youchi(a)gmail.com&gt; wrote: > Hello, > > I am trying to redeploy wallaby undercloud using "container-tools:3.0" > since I got a problem using the defaut stream. > > The installation seems to go without issues, but now I am trying to > import the definition of my nodes, I am getting this error message : > > (undercloud) [stack@ooounder ~]$ openstack overcloud node import > overnodes.json --debug > ... > ... > auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': > '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None, > 'key': None, 'baremetal_status_code_retries': '5', > 'baremetal_introspection_status_code_retries': '5', > 'image_status_code_retries': > '5', 'disable_vendor_agent': {}, 'interface': 'public', > 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, > 'image_format' > : 'qcow2', 'message': '', 'network_api_version': '2', > 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': > 'activ > e', 'auth': {'user_domain_name': 'Default', 'project_domain_name': > 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('o > sc-lib', '2.3.1')], 'verbose_level': 3, 'deferred_help': False, 'debug': > True, 'default_domain': 'default', 'timing': False, 'inspect > or_api_version': '1', 'auth_url': 'https://10.168.24.2:13000', > 'username': 'admin', 'password': '***', 'beta_command': False, 'identi > ty_api_version': '3', 'queues_api_version': '2', > 'orchestration_api_version': '1', 'workflow_api_version': '2', > 'database_api_version > ': '1', 'share_api_version': '2.63', 'container_infra_api_version': '1', > 'dns_api_version': '2', 'key_manager_api_version': '1', 'ala > rming_api_version': '2', 'baremetal_api_version': '1.69', > 'loadbalancer_api_version': '2.0', 'data_processing_api_version': '1.1', 't > ripleoclient_api_version': '2', 'metrics_api_version': '1', > 'event_api_version': '2', 'placement_api_version': '1.0', 'region_name': > '', 'auth_type': 'password', 'networks': []} > Using auth plugin: password > Using parameters {'auth_url': 'https://10.168.24.2:13000', > 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'ad > min', 'user_domain_name': 'Default', 'password': '***'} > Get auth_ref > REQ: curl -g -i --cacert > "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET > https://10.168.24.2:13000 -H "Accept: application/ > json" -H "User-Agent: openstacksdk/0.55.0 keystoneauth1/4.3.1 > python-requests/2.25.1 CPython/3.6.8" > Starting new HTTPS connection (1): 10.168.24.2:13000 > https://10.168.24.2:13000 "GET / HTTP/1.1" 300 268 > RESP: [300] Content-Length: 268 Content-Type: application/json Date: Tue, > 15 Jun 2021 15:37:03 GMT Location: https://10.168.24.2:1300 > 0/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id: > req-d482da26-c336-41ed-93fe-610b2d98fbac > RESP BODY: {"versions": {"values": [{"id": "v3.14", "status": "stable", > "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", > "href": "https://10.168.24.2:13000/v3/"}], "media-types": [{"base": > "application/json", "type": "application/vnd.openstack.identity-v > 3+json"}]}]}} > GET call to https://10.168.24.2:13000/ used request id > req-d482da26-c336-41ed-93fe-610b2d98fbac > Making authentication request to https://10.168.24.2:13000/v3/auth/tokens > https://10.168.24.2:13000 "POST /v3/auth/tokens HTTP/1.1" 201 3615 > {"token": {"methods": ["password"], "user": {"domain": {"id": "default", > "name": "Default"}, "id": "06ff4d4c15664efe8af20d7224df59d3" > , "name": "admin", "password_expires_at": null}, "audit_ids": > ["2KZLymr6ScOyqhAsJUXbfQ"], "expires_at": "2021-06-15T19:37:03.000000Z" > , "issued_at": "2021-06-15T15:37:03.000000Z", "project": {"domain": > {"id": "default", "name": "Default"}, "id": "8308c2e160204978b86e > 5fcb5d8b524c", "name": "admin"}, "is_domain": false, "roles": [{"id": > "115bc5caafd245e99859f2edc8cebcdf", "name": "member"}, {"id": " > 4c8c8681ff924e3f82c6ec9f6385e77a", "name": "_member_"}, {"id": > "db493429c9be44c499027b80f06fa3a8", "name": "admin"}, {"id": "5f20877f > d53c47258a8c1d50361a8db4", "name": "reader"}], "catalog": [{"endpoints": > [{"id": "24adacb7cded408dac4663143018f192", "interface": "pu > blic", "region_id": "regionOne", "url": "https://10.168.24.2:13000", > "region": "regionOne"}, {"id": "7c4fe5c4d13c486f8a0de078e4a50381 > ", "interface": "internal", "region_id": "regionOne", "url": " > http://10.168.24.3:5000";, "region": "regionOne"}, {"id": "ca18e36f33284 > 239907ec4d88040a6e4", "interface": "admin", "region_id": "regionOne", > "url": "http://10.168.24.3:35357", "region": "regionOne"}], "id > ": "12ab546ac7004930ba0a54d99fe36919", "type": "identity", "name": > "keystone"}, {"endpoints": [{"id": "1bb5ba46095f4d679bf056f7e460c7 > 14", "interface": "admin", "region_id": "regionOne", "url": " > http://10.168.24.3:6385";, "region": "regionOne"}, {"id": "7fdd0d20af114f > f2a83d53b340533cb3", "interface": "internal", "region_id": "regionOne", > "url": "http://10.168.24.3:6385", "region": "regionOne"}, {"i > d": "8884ad607c2e479984cd3e0a894352db", "interface": "public", > "region_id": "regionOne", "url": "https://10.168.24.2:13385", "region" > : "regionOne"}], "id": "4e48a20808a94dcb9b14adcf81eb9a68", "type": > "baremetal", "name": "ironic"}, {"endpoints": [{"id": "14e31cce879 > 64d4ab26ecfacbcf365e3", "interface": "internal", "region_id": > "regionOne", "url": "http://10.168.24.3:5050", "region": "regionOne"}, > {"id": "5add117abc43462a8901ec486bde8c55", "interface": "public", > "region_id": "regionOne", "url": "https://10.168.24.2:13050", "regi > on": "regionOne"}, {"id": "d3009b5b54d74f2a842f6b3e4b0019c7", > "interface": "admin", "region_id": "regionOne", "url": "http://10.168.2 > 4.3:5050", "region": "regionOne"}], "id": > "639c382276be488b8719f3ddf321ec00", "type": "baremetal-introspection", > "name": "ironic-insp > ector"}, {"endpoints": [{"id": "8107f8f6c79a4aaa84f7538fb17ce394", > "interface": "admin", "region_id": "regionOne", "url": "http://10. > 168.24.3:8004/v1/8308c2e160204978b86e5fcb5d8b524c", "region": > "regionOne"}, {"id": "ef09fc92358d476a88f8d33b02a6bf23", "interface": " > public", "region_id": "regionOne", "url": " > https://10.168.24.2:13004/v1/8308c2e160204978b86e5fcb5d8b524c";, > "region": "regionOne"}, {" > id": "f0dfb22acd124c33933687b9398f4351", "interface": "internal", > "region_id": "regionOne", "url": "http://10.168.24.3:8004/v1/8308c2 > e160204978b86e5fcb5d8b524c", "region": "regionOne"}], "id": > "88ced00488484104a9dcf0a2e1834084", "type": "orchestration", "name": "hea > t"}, {"endpoints": [{"id": "1de6dfdb1a2e489cb83fdbeaa3a232d7", > "interface": "internal", "region_id": "regionOne", "url": "http://10.1 > 68.24.3:9696", "region": "regionOne"}, {"id": > "8894b604813547159b982c7df131b232", "interface": "public", "region_id": > "regionOne", "u > rl": "https://10.168.24.2:13696", "region": "regionOne"}, {"id": > "cfa5ee424c7e46268075c79a7e29314d", "interface": "admin", "region_id > ": "regionOne", "url": "http://10.168.24.3:9696", "region": > "regionOne"}], "id": "b1bde18dbab2440ebe8474b53ad0cba2", "type": "network > ", "name": "neutron"}]}} > Directory "/home/stack/.tripleo" was not created because it already > exists. > run(Namespace(concurrency=20, env_file=<_io.TextIOWrapper > name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/lib/ironi > c/httpboot', instance_boot_option=None, introspect=False, > no_deploy_image=False, provide=False, run_validations=False, validate_only= > False)) > take_action(Namespace(concurrency=20, env_file=<_io.TextIOWrapper > name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/l > ib/ironic/httpboot', instance_boot_option=None, introspect=False, > no_deploy_image=False, provide=False, run_validations=False, valida > te_only=False)) > Instantiating baremetal client: <class 'ironicclient.v1.client.Client'> > Baremetal API version: latest > Registering all nodes. > Populating list of registered nodes. > REQ: curl -g -i --cacert > "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET > https://10.168.24.2:13385/v1/nodes/detail -H "Acce > pt: application/json" -H "Content-Type: application/json" -H "User-Agent: > python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495 > 887eff3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H > "X-OpenStack-Ironic-API-Version: 1.69" > Starting new HTTPS connection (1): 10.168.24.2:13385 > https://10.168.24.2:13385 "GET /v1/nodes/detail HTTP/1.1" 200 None > RESP: [200] Content-Encoding: gzip Content-Type: application/json Date: > Tue, 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-fc046 > 05f-b359-4f83-bf2e-c92b0daccaf9 Server: Apache Transfer-Encoding: chunked > Vary: Accept-Encoding X-OpenStack-Ironic-API-Maximum-Versio > n: 1.74 X-OpenStack-Ironic-API-Minimum-Version: 1.1 > X-OpenStack-Ironic-API-Version: 1.69 > RESP BODY: {"nodes": []} > ... > REQ: curl -g -i --cacert > "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST > https://10.168.24.2:13385/v1/nodes -H "Accept: ap > plication/json" -H "Content-Type: application/json" -H "User-Agent: > python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff > 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H > "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties": > {"cpus > ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"}, > "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http > boot/agent.kernel", "rescue_kernel": > "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk": > "file:///var/lib/ironic/httpbo > ot/agent.ramdisk", "rescue_ramdisk": > "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username": > "poweruser", "ipmi_password": > "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"}, > "resource_class": "baremetal", "name": "controller-a"}' > https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139 > RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue, > 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-9de57250 > -ea09-4490-ab9c-e5bfed236412 Server: Apache > X-OpenStack-Ironic-API-Maximum-Version: 1.74 > X-OpenStack-Ironic-API-Minimum-Version: 1.1 > X-OpenStack-Ironic-API-Version: 1.69 > RESP BODY: {"error_message": "{\"faultcode\": \"Server\", > \"faultstring\": \"Resource temporarily unavailable, please retry.\", \"deb > uginfo\": null}"} > Error contacting Ironic server: Resource temporarily unavailable, please > retry. (HTTP 503). Attempt 1 of 6 > REQ: curl -g -i --cacert > "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST > https://10.168.24.2:13385/v1/nodes -H "Accept: ap > plication/json" -H "Content-Type: application/json" -H "User-Agent: > python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff > 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H > "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties": > {"cpus > ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"}, > "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http > boot/agent.kernel", "rescue_kernel": > "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk": > "file:///var/lib/ironic/httpbo > ot/agent.ramdisk", "rescue_ramdisk": > "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username": > "poweruser", "ipmi_password": > "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"}, > "resource_class": "baremetal", "name": "controller-a"}' > https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139 > RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue, > 15 Jun 2021 15:37:05 GMT Openstack-Request-Id: req-3a448355 > -c367-46ad-b968-081fdd077487 Server: Apache > X-OpenStack-Ironic-API-Maximum-Version: 1.74 > X-OpenStack-Ironic-API-Minimum-Version: 1.1 > X-OpenStack-Ironic-API-Version: 1.69 > RESP BODY: {"error_message": "{\"faultcode\": \"Server\", > \"faultstring\": \"Resource temporarily unavailable, please retry.\", \"deb > uginfo\": null}"} > Error contacting Ironic server: Resource temporarily unavailable, please > retry. (HTTP 503). Attempt 2 of 6 > REQ: curl -g -i --cacert > "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST > https://10.168.24.2:13385/v1/nodes -H "Accept: ap > plication/json" -H "Content-Type: application/json" -H "User-Agent: > python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff > 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H > "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties": > {"cpus > ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"}, > "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http > boot/agent.kernel", "rescue_kernel": > "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk": > "file:///var/lib/ironic/httpbo > ot/agent.ramdisk", "rescue_ramdisk": > "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username": > "poweruser", "ipmi_password": > "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"}, > "resource_class": "baremetal", "name": "controller-a"}' > > > > > > *https://10.168.24.2:13385 <https://10.168.24.2:13385> "POST /v1/nodes > HTTP/1.1" 503 139 RESP: [503] Content-Length: 139 Content-Type: > application/json Date: Tue, 15 Jun 2021 15:37:07 GMT Openstack-Request-Id: > req-159b74d8-183e-4f84-9a86-569e7f29d313 Server: Apache > X-OpenStack-Ironic-API-Maximum-Version: 1.74 > X-OpenStack-Ironic-API-Minimum-Version: 1.1 X-OpenStack-Ironic-API-Version: > 1.69 RESP BODY: {"error_message": "{\"faultcode\": \"Server\", > \"faultstring\": \"Resource temporarily unavailable, please retry.\", > \"debuginfo\": null}"}* > > ... > .. > > > From /var/log/containers/ironic/ironic-conductor.log > I get this > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service > [req-94d7c168-4c7f-4601-a302-3fd52b9132af - - - - -] Error starting > thread.: iro > nic.common.exception.DriverNotFoundInEntrypoint: Could not find the > following items in the 'ironic.hardware.interfaces.deploy' entryp > oint: iscsi. > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service Traceback (most > recent call last): > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 807, in > run_service > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service service.start() > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/common/rpc_service.py", line 63, > in start > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service > self.manager.init_host(admin_context) > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line > 171, in init_host > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service > self._collect_periodic_tasks(admin_context) > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line > 300, in _collect_periodic_tasks > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service for ifaces in > driver_factory.all_interfaces().values(): > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line > 286, in all_interfaces > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service return {iface: > interfaces(iface) for iface in _INTERFACE_LOADERS} > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line > 286, in <dictcomp> > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service return {iface: > interfaces(iface) for iface in _INTERFACE_LOADERS} > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line > 277, in interfaces > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service return > _get_all_drivers(_INTERFACE_LOADERS[interface_type]()) > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line > 338, in __init__ > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service > self.__class__._init_extension_manager() > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 36 > 0, in inner > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service return f(*args, > **kwargs) > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line > 408, in _init_extension_manager > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service > on_missing_entrypoints_callback=missing_callback)) > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/stevedore/named.py", line 85, in __init > __ > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service > on_missing_entrypoints_callback(self._missing_names) > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service File > "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line > 399, in missing_callback > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service names=names, > entrypoint=cls._entrypoint_name) > > *2021-06-15 16:39:04.163 7 ERROR oslo_service.service > ironic.common.exception.DriverNotFoundInEntrypoint: Could not find the > following items in the 'ironic.hardware.interfaces.deploy' entrypoint: > iscsi. * > 2021-06-15 16:39:04.163 7 ERROR oslo_service.service > 2021-06-15 16:39:05.172 7 WARNING amqp [-] Received method (60, 30) > during closing channel 1. This method will be ignored: SystemExit > : 1 > > *2021-06-15 16:39:05.174 7 INFO ironic.common.rpc_service [-] Stopped RPC > server for service ironic.conductor_manager on host localhost.localdomain. * > 2021-06-15 16:39:05.174 7 DEBUG oslo_concurrency.lockutils [-] Acquired > lock "singleton_lock" lock /usr/lib/python3.6/site-packages/o > slo_concurrency/lockutils.py:266 > 2021-06-15 16:39:05.175 7 DEBUG oslo_concurrency.lockutils [-] Releasing > lock "singleton_lock" lock /usr/lib/python3.6/site-packages/ > oslo_concurrency/lockutils.py:282 > > > Any ideas? > > Regards. > _______________________________________________ > users mailing list > users(a)lists.rdoproject.org > http://lists.rdoproject.org/mailman/listinfo/users > > To unsubscribe: users-unsubscribe(a)lists.rdoproject.org >