[rdo-users] Error importing node definition on undercloud Wallaby

wodel youchi wodel.youchi at gmail.com
Tue Jun 15 21:22:01 UTC 2021


Hi,

Do I have to redeploy the undercloud from scratch after the repo sync?

Regards.

Le mar. 15 juin 2021 à 16:48, Alex Schultz <aschultz at redhat.com> a écrit :

> You're hitting https://bugs.launchpad.net/tripleo/+bug/1931995 because
> wallaby incorrectly fetches master containers. iscsi has been removed from
> master
> https://review.opendev.org/c/openstack/tripleo-heat-templates/+/749475.
> Currently the tripleowallaby containers are out of sync on quay.io and we
> need to land
> https://review.opendev.org/c/openstack/tripleo-common/+/796516.
>
> On Tue, Jun 15, 2021 at 9:42 AM wodel youchi <wodel.youchi at gmail.com>
> wrote:
>
>> Hello,
>>
>> I am trying to redeploy wallaby undercloud using "container-tools:3.0"
>> since I got a problem using the defaut stream.
>>
>> The installation seems to go without issues, but now I am trying to
>> import the definition of my nodes, I am getting this error message :
>>
>> (undercloud) [stack at ooounder ~]$ openstack overcloud node import
>>  overnodes.json --debug
>> ...
>> ...
>> auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert':
>> '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem', 'cert': None,
>> 'key': None, 'baremetal_status_code_retries': '5',
>> 'baremetal_introspection_status_code_retries': '5',
>> 'image_status_code_retries':
>> '5', 'disable_vendor_agent': {}, 'interface': 'public',
>> 'floating_ip_source': 'neutron', 'image_api_use_tasks': False,
>> 'image_format'
>> : 'qcow2', 'message': '', 'network_api_version': '2',
>> 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status':
>> 'activ
>> e', 'auth': {'user_domain_name': 'Default', 'project_domain_name':
>> 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('o
>> sc-lib', '2.3.1')], 'verbose_level': 3, 'deferred_help': False, 'debug':
>> True, 'default_domain': 'default', 'timing': False, 'inspect
>> or_api_version': '1', 'auth_url': 'https://10.168.24.2:13000',
>> 'username': 'admin', 'password': '***', 'beta_command': False, 'identi
>> ty_api_version': '3', 'queues_api_version': '2',
>> 'orchestration_api_version': '1', 'workflow_api_version': '2',
>> 'database_api_version
>> ': '1', 'share_api_version': '2.63', 'container_infra_api_version': '1',
>> 'dns_api_version': '2', 'key_manager_api_version': '1', 'ala
>> rming_api_version': '2', 'baremetal_api_version': '1.69',
>> 'loadbalancer_api_version': '2.0', 'data_processing_api_version': '1.1', 't
>> ripleoclient_api_version': '2', 'metrics_api_version': '1',
>> 'event_api_version': '2', 'placement_api_version': '1.0', 'region_name':
>> '', 'auth_type': 'password', 'networks': []}
>> Using auth plugin: password
>> Using parameters {'auth_url': 'https://10.168.24.2:13000',
>> 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'ad
>> min', 'user_domain_name': 'Default', 'password': '***'}
>> Get auth_ref
>> REQ: curl -g -i --cacert
>> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET
>> https://10.168.24.2:13000 -H "Accept: application/
>> json" -H "User-Agent: openstacksdk/0.55.0 keystoneauth1/4.3.1
>> python-requests/2.25.1 CPython/3.6.8"
>> Starting new HTTPS connection (1): 10.168.24.2:13000
>> https://10.168.24.2:13000 "GET / HTTP/1.1" 300 268
>> RESP: [300] Content-Length: 268 Content-Type: application/json Date: Tue,
>> 15 Jun 2021 15:37:03 GMT Location: https://10.168.24.2:1300
>> 0/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id:
>> req-d482da26-c336-41ed-93fe-610b2d98fbac
>> RESP BODY: {"versions": {"values": [{"id": "v3.14", "status": "stable",
>> "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self",
>> "href": "https://10.168.24.2:13000/v3/"}], "media-types": [{"base":
>> "application/json", "type": "application/vnd.openstack.identity-v
>> 3+json"}]}]}}
>> GET call to https://10.168.24.2:13000/ used request id
>> req-d482da26-c336-41ed-93fe-610b2d98fbac
>> Making authentication request to https://10.168.24.2:13000/v3/auth/tokens
>> https://10.168.24.2:13000 "POST /v3/auth/tokens HTTP/1.1" 201 3615
>> {"token": {"methods": ["password"], "user": {"domain": {"id": "default",
>> "name": "Default"}, "id": "06ff4d4c15664efe8af20d7224df59d3"
>> , "name": "admin", "password_expires_at": null}, "audit_ids":
>> ["2KZLymr6ScOyqhAsJUXbfQ"], "expires_at": "2021-06-15T19:37:03.000000Z"
>> , "issued_at": "2021-06-15T15:37:03.000000Z", "project": {"domain":
>> {"id": "default", "name": "Default"}, "id": "8308c2e160204978b86e
>> 5fcb5d8b524c", "name": "admin"}, "is_domain": false, "roles": [{"id":
>> "115bc5caafd245e99859f2edc8cebcdf", "name": "member"}, {"id": "
>> 4c8c8681ff924e3f82c6ec9f6385e77a", "name": "_member_"}, {"id":
>> "db493429c9be44c499027b80f06fa3a8", "name": "admin"}, {"id": "5f20877f
>> d53c47258a8c1d50361a8db4", "name": "reader"}], "catalog": [{"endpoints":
>> [{"id": "24adacb7cded408dac4663143018f192", "interface": "pu
>> blic", "region_id": "regionOne", "url": "https://10.168.24.2:13000",
>> "region": "regionOne"}, {"id": "7c4fe5c4d13c486f8a0de078e4a50381
>> ", "interface": "internal", "region_id": "regionOne", "url": "
>> http://10.168.24.3:5000", "region": "regionOne"}, {"id": "ca18e36f33284
>> 239907ec4d88040a6e4", "interface": "admin", "region_id": "regionOne",
>> "url": "http://10.168.24.3:35357", "region": "regionOne"}], "id
>> ": "12ab546ac7004930ba0a54d99fe36919", "type": "identity", "name":
>> "keystone"}, {"endpoints": [{"id": "1bb5ba46095f4d679bf056f7e460c7
>> 14", "interface": "admin", "region_id": "regionOne", "url": "
>> http://10.168.24.3:6385", "region": "regionOne"}, {"id": "7fdd0d20af114f
>> f2a83d53b340533cb3", "interface": "internal", "region_id": "regionOne",
>> "url": "http://10.168.24.3:6385", "region": "regionOne"}, {"i
>> d": "8884ad607c2e479984cd3e0a894352db", "interface": "public",
>> "region_id": "regionOne", "url": "https://10.168.24.2:13385", "region"
>> : "regionOne"}], "id": "4e48a20808a94dcb9b14adcf81eb9a68", "type":
>> "baremetal", "name": "ironic"}, {"endpoints": [{"id": "14e31cce879
>> 64d4ab26ecfacbcf365e3", "interface": "internal", "region_id":
>> "regionOne", "url": "http://10.168.24.3:5050", "region": "regionOne"},
>> {"id": "5add117abc43462a8901ec486bde8c55", "interface": "public",
>> "region_id": "regionOne", "url": "https://10.168.24.2:13050", "regi
>> on": "regionOne"}, {"id": "d3009b5b54d74f2a842f6b3e4b0019c7",
>> "interface": "admin", "region_id": "regionOne", "url": "http://10.168.2
>> 4.3:5050", "region": "regionOne"}], "id":
>> "639c382276be488b8719f3ddf321ec00", "type": "baremetal-introspection",
>> "name": "ironic-insp
>> ector"}, {"endpoints": [{"id": "8107f8f6c79a4aaa84f7538fb17ce394",
>> "interface": "admin", "region_id": "regionOne", "url": "http://10.
>> 168.24.3:8004/v1/8308c2e160204978b86e5fcb5d8b524c", "region":
>> "regionOne"}, {"id": "ef09fc92358d476a88f8d33b02a6bf23", "interface": "
>> public", "region_id": "regionOne", "url": "
>> https://10.168.24.2:13004/v1/8308c2e160204978b86e5fcb5d8b524c",
>> "region": "regionOne"}, {"
>> id": "f0dfb22acd124c33933687b9398f4351", "interface": "internal",
>> "region_id": "regionOne", "url": "http://10.168.24.3:8004/v1/8308c2
>> e160204978b86e5fcb5d8b524c", "region": "regionOne"}], "id":
>> "88ced00488484104a9dcf0a2e1834084", "type": "orchestration", "name": "hea
>> t"}, {"endpoints": [{"id": "1de6dfdb1a2e489cb83fdbeaa3a232d7",
>> "interface": "internal", "region_id": "regionOne", "url": "http://10.1
>> 68.24.3:9696", "region": "regionOne"}, {"id":
>> "8894b604813547159b982c7df131b232", "interface": "public", "region_id":
>> "regionOne", "u
>> rl": "https://10.168.24.2:13696", "region": "regionOne"}, {"id":
>> "cfa5ee424c7e46268075c79a7e29314d", "interface": "admin", "region_id
>> ": "regionOne", "url": "http://10.168.24.3:9696", "region":
>> "regionOne"}], "id": "b1bde18dbab2440ebe8474b53ad0cba2", "type": "network
>> ", "name": "neutron"}]}}
>> Directory "/home/stack/.tripleo" was not created because it already
>> exists.
>> run(Namespace(concurrency=20, env_file=<_io.TextIOWrapper
>> name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/lib/ironi
>> c/httpboot', instance_boot_option=None, introspect=False,
>> no_deploy_image=False, provide=False, run_validations=False, validate_only=
>> False))
>> take_action(Namespace(concurrency=20, env_file=<_io.TextIOWrapper
>> name='overnodes.json' mode='r' encoding='UTF-8'>, http_boot='/var/l
>> ib/ironic/httpboot', instance_boot_option=None, introspect=False,
>> no_deploy_image=False, provide=False, run_validations=False, valida
>> te_only=False))
>> Instantiating baremetal client: <class 'ironicclient.v1.client.Client'>
>> Baremetal API version: latest
>> Registering all nodes.
>> Populating list of registered nodes.
>> REQ: curl -g -i --cacert
>> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X GET
>> https://10.168.24.2:13385/v1/nodes/detail -H "Acce
>> pt: application/json" -H "Content-Type: application/json" -H "User-Agent:
>> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495
>> 887eff3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
>> "X-OpenStack-Ironic-API-Version: 1.69"
>> Starting new HTTPS connection (1): 10.168.24.2:13385
>> https://10.168.24.2:13385 "GET /v1/nodes/detail HTTP/1.1" 200 None
>> RESP: [200] Content-Encoding: gzip Content-Type: application/json Date:
>> Tue, 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-fc046
>> 05f-b359-4f83-bf2e-c92b0daccaf9 Server: Apache Transfer-Encoding: chunked
>> Vary: Accept-Encoding X-OpenStack-Ironic-API-Maximum-Versio
>> n: 1.74 X-OpenStack-Ironic-API-Minimum-Version: 1.1
>> X-OpenStack-Ironic-API-Version: 1.69
>> RESP BODY: {"nodes": []}
>> ...
>> REQ: curl -g -i --cacert
>> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST
>> https://10.168.24.2:13385/v1/nodes -H "Accept: ap
>> plication/json" -H "Content-Type: application/json" -H "User-Agent:
>> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
>> 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
>> "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties":
>> {"cpus
>> ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"},
>> "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
>> boot/agent.kernel", "rescue_kernel":
>> "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk":
>> "file:///var/lib/ironic/httpbo
>> ot/agent.ramdisk", "rescue_ramdisk":
>> "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username":
>> "poweruser", "ipmi_password":
>> "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"},
>> "resource_class": "baremetal", "name": "controller-a"}'
>> https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139
>> RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue,
>> 15 Jun 2021 15:37:03 GMT Openstack-Request-Id: req-9de57250
>> -ea09-4490-ab9c-e5bfed236412 Server: Apache
>> X-OpenStack-Ironic-API-Maximum-Version: 1.74
>> X-OpenStack-Ironic-API-Minimum-Version: 1.1
>> X-OpenStack-Ironic-API-Version: 1.69
>> RESP BODY: {"error_message": "{\"faultcode\": \"Server\",
>> \"faultstring\": \"Resource temporarily unavailable, please retry.\", \"deb
>> uginfo\": null}"}
>> Error contacting Ironic server: Resource temporarily unavailable, please
>> retry. (HTTP 503). Attempt 1 of 6
>> REQ: curl -g -i --cacert
>> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST
>> https://10.168.24.2:13385/v1/nodes -H "Accept: ap
>> plication/json" -H "Content-Type: application/json" -H "User-Agent:
>> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
>> 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
>> "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties":
>> {"cpus
>> ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"},
>> "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
>> boot/agent.kernel", "rescue_kernel":
>> "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk":
>> "file:///var/lib/ironic/httpbo
>> ot/agent.ramdisk", "rescue_ramdisk":
>> "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username":
>> "poweruser", "ipmi_password":
>> "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"},
>> "resource_class": "baremetal", "name": "controller-a"}'
>> https://10.168.24.2:13385 "POST /v1/nodes HTTP/1.1" 503 139
>> RESP: [503] Content-Length: 139 Content-Type: application/json Date: Tue,
>> 15 Jun 2021 15:37:05 GMT Openstack-Request-Id: req-3a448355
>> -c367-46ad-b968-081fdd077487 Server: Apache
>> X-OpenStack-Ironic-API-Maximum-Version: 1.74
>> X-OpenStack-Ironic-API-Minimum-Version: 1.1
>> X-OpenStack-Ironic-API-Version: 1.69
>> RESP BODY: {"error_message": "{\"faultcode\": \"Server\",
>> \"faultstring\": \"Resource temporarily unavailable, please retry.\", \"deb
>> uginfo\": null}"}
>> Error contacting Ironic server: Resource temporarily unavailable, please
>> retry. (HTTP 503). Attempt 2 of 6
>> REQ: curl -g -i --cacert
>> "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST
>> https://10.168.24.2:13385/v1/nodes -H "Accept: ap
>> plication/json" -H "Content-Type: application/json" -H "User-Agent:
>> python-ironicclient" -H "X-Auth-Token: {SHA256}ab641f711495887eff
>> 3fd05f3cfdcd931e5abca41b5329c22e9a7513c5eb60bb" -H
>> "X-OpenStack-Ironic-API-Version: 1.69" -d '{"driver": "ipmi", "properties":
>> {"cpus
>> ": "8", "memory_mb": "16384", "local_gb": "100", "cpu_arch": "x86_64"},
>> "driver_info": {"deploy_kernel": "file:///var/lib/ironic/http
>> boot/agent.kernel", "rescue_kernel":
>> "file:///var/lib/ironic/httpboot/agent.kernel", "deploy_ramdisk":
>> "file:///var/lib/ironic/httpbo
>> ot/agent.ramdisk", "rescue_ramdisk":
>> "file:///var/lib/ironic/httpboot/agent.ramdisk", "ipmi_username":
>> "poweruser", "ipmi_password":
>> "ertyuiop", "ipmi_address": "127.0.0.1", "ipmi_port": "6200"},
>> "resource_class": "baremetal", "name": "controller-a"}'
>>
>>
>>
>>
>>
>> *https://10.168.24.2:13385 <https://10.168.24.2:13385> "POST /v1/nodes
>> HTTP/1.1" 503 139 RESP: [503] Content-Length: 139 Content-Type:
>> application/json Date: Tue, 15 Jun 2021 15:37:07 GMT Openstack-Request-Id:
>> req-159b74d8-183e-4f84-9a86-569e7f29d313 Server: Apache
>> X-OpenStack-Ironic-API-Maximum-Version: 1.74
>> X-OpenStack-Ironic-API-Minimum-Version: 1.1 X-OpenStack-Ironic-API-Version:
>> 1.69 RESP BODY: {"error_message": "{\"faultcode\": \"Server\",
>> \"faultstring\": \"Resource temporarily unavailable, please retry.\",
>> \"debuginfo\": null}"}*
>>
>> ...
>> ..
>>
>>
>> From /var/log/containers/ironic/ironic-conductor.log
>> I get this
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>> [req-94d7c168-4c7f-4601-a302-3fd52b9132af - - - - -] Error starting
>> thread.: iro
>> nic.common.exception.DriverNotFoundInEntrypoint: Could not find the
>> following items in the 'ironic.hardware.interfaces.deploy' entryp
>> oint: iscsi.
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service Traceback (most
>> recent call last):
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 807, in
>> run_service
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     service.start()
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/common/rpc_service.py", line 63,
>> in start
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>>     self.manager.init_host(admin_context)
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line
>> 171, in init_host
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>>     self._collect_periodic_tasks(admin_context)
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/conductor/base_manager.py", line
>> 300, in _collect_periodic_tasks
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     for ifaces in
>> driver_factory.all_interfaces().values():
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
>> 286, in all_interfaces
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return {iface:
>> interfaces(iface) for iface in _INTERFACE_LOADERS}
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
>> 286, in <dictcomp>
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return {iface:
>> interfaces(iface) for iface in _INTERFACE_LOADERS}
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
>> 277, in interfaces
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return
>> _get_all_drivers(_INTERFACE_LOADERS[interface_type]())
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
>> 338, in __init__
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>>     self.__class__._init_extension_manager()
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 36
>> 0, in inner
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     return f(*args,
>> **kwargs)
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
>> 408, in _init_extension_manager
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>>     on_missing_entrypoints_callback=missing_callback))
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/stevedore/named.py", line 85, in __init
>> __
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>>     on_missing_entrypoints_callback(self._missing_names)
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service   File
>> "/usr/lib/python3.6/site-packages/ironic/common/driver_factory.py", line
>> 399, in missing_callback
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service     names=names,
>> entrypoint=cls._entrypoint_name)
>>
>> *2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>> ironic.common.exception.DriverNotFoundInEntrypoint: Could not find the
>> following items in the 'ironic.hardware.interfaces.deploy' entrypoint:
>> iscsi. *
>> 2021-06-15 16:39:04.163 7 ERROR oslo_service.service
>> 2021-06-15 16:39:05.172 7 WARNING amqp [-] Received method (60, 30)
>> during closing channel 1. This method will be ignored: SystemExit
>> : 1
>>
>> *2021-06-15 16:39:05.174 7 INFO ironic.common.rpc_service [-] Stopped RPC
>> server for service ironic.conductor_manager on host localhost.localdomain. *
>> 2021-06-15 16:39:05.174 7 DEBUG oslo_concurrency.lockutils [-] Acquired
>> lock "singleton_lock" lock /usr/lib/python3.6/site-packages/o
>> slo_concurrency/lockutils.py:266
>> 2021-06-15 16:39:05.175 7 DEBUG oslo_concurrency.lockutils [-] Releasing
>> lock "singleton_lock" lock /usr/lib/python3.6/site-packages/
>> oslo_concurrency/lockutils.py:282
>>
>>
>> Any ideas?
>>
>> Regards.
>> _______________________________________________
>> users mailing list
>> users at lists.rdoproject.org
>> http://lists.rdoproject.org/mailman/listinfo/users
>>
>> To unsubscribe: users-unsubscribe at lists.rdoproject.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/users/attachments/20210615/c7718738/attachment-0001.html>


More information about the users mailing list