[rdo-users] Possible regression for podman starting containers after a hard shutdown

Wed Jul 7 12:53:03 UTC 2021

Hello *,

i have encountered an error after a hard power loss on
nearly each compute node. The neutron-haproxy-ovnmeta container didn't 
came up after the hard shutdown. Down below under "error message" i've
posted the exact error message.

After a quick search i found the following bug reported
in the red hat bug tracker. It looks like the exact same issue.

https://bugzilla.redhat.com/show_bug.cgi?id=1797892

But it is supposed to fixed in the "Train" release.
https://bugzilla.redhat.com/show_bug.cgi?id=1797892

Any ideas on this? Is it the same issue or something similar?
Would an update to Victoria solve this issue?

I think it's non of a big deal so since we try to avoid
unexpected shutdowns but it may happen due to power loss.

Now some details on the Openstack setup.
We are using Ussuri release and did a pretty basic
tirpleO Setup. We are using 3 Controller Nodes
and 8 Computes with Ceph storage attached.

I think there is nothing special in our configuration
that may favor this type of issue but if you think
so i will post the necessary configuration details.

Error message:
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event [-] Unexpected 
exception in notify_loop: neutron_lib.exceptions.ProcessExecutionError: 
Exit code: 125; Stdin: ; Stdout: Starting a new child container 
neutron-haproxy-ovnmeta-4c8e69e6-3e1a-4d7e-bde0-33241c0d383e
; Stderr: Error: error creating container storage: the container name 
"neutron-haproxy-ovnmeta-4c8e69e6-3e1a-4d7e-bde0-33241c0d383e" is 
already in use by 
"37b687a1d31adf73275ffdcadc3c5fd5ed1a72fe0c2547f38ec571556480c852". You 
have to remove that container to be able to reuse that name.: that name 
is already in use
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event Traceback (most recent 
call last):
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/ovsdbapp/event.py", line 143, in 
notify_loop
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event     match.run(event, 
row, updates)
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/neutron/agent/ovn/metadata/agent.py", 
line 83, in run
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event 
self.agent.update_datapath(str(row.datapath.uuid))
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/neutron/agent/ovn/metadata/agent.py", 
line 342, in update_datapath
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event 
self.provision_datapath(datapath)
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/neutron/agent/ovn/metadata/agent.py", 
line 457, in provision_datapath
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event     network_id=datapath)
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/neutron/agent/ovn/metadata/driver.py", 
line 200, in spawn_monitored_metadata_proxy
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event     pm.enable()
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/neutron/agent/linux/external_process.py", 
line 90, in enable
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event 
run_as_root=self.run_as_root)
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 
724, in execute
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event 
run_as_root=run_as_root)
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event   File 
"/usr/lib/python3.6/site-packages/neutron/agent/linux/utils.py", line 
147, in execute
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event     returncode=returncode)
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event 
neutron_lib.exceptions.ProcessExecutionError: Exit code: 125; Stdin: ; 
Stdout: Starting a new child container 
neutron-haproxy-ovnmeta-4c8e69e6-3e1a-4d7e-bde0-33241c0d383e
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event ; Stderr: Error: error 
creating container storage: the container name 
"neutron-haproxy-ovnmeta-4c8e69e6-3e1a-4d7e-bde0-33241c0d383e" is 
already in use by 
"37b687a1d31adf73275ffdcadc3c5fd5ed1a72fe0c2547f38ec571556480c852". You 
have to remove that container to be able to reuse that name.: that name 
is already in use
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event
2021-06-23 08:11:28.928 4823 ERROR ovsdbapp.event
2021-06-23 08:16:34.975 4823 INFO neutron.agent.ovn.metadata.agent [-] 
Port 983831dd-8aae-4b01-9953-8cbce86f89a8 in datapath 
d98d35e0-765e-4128-bf7f-ab71b2701986 bound to our chassis
2021-06-23 08:16:35.965 4823 ERROR neutron.agent.linux.utils [-] Exit 
code: 125; Stdin: ; Stdout: Starting a new child container 
neutron-haproxy-ovnmeta-d98d35e0-765e-4128-bf7f-ab71b2701986
; Stderr: Error: error creating container storage: the container name 
"neutron-haproxy-ovnmeta-d98d35e0-765e-4128-bf7f-ab71b2701986" is 
already in use by 
"bd037c9c5688efd56ccc0521bdebc0bcdc261ffd584e151da6a8e690aefdfb22". You 
have to remove that container to be able to reuse that name.: that name 
is already in use

-- 
Benjamin Zapiec <benjamin.zapiec at gonicus.de> (System Engineer)
* GONICUS GmbH * Moehnestrasse 55 (Kaiserhaus) * D-59755 Arnsberg
* Tel.: +49 2932 916-0 * Fax: +49 2932 916-245
* http://www.GONICUS.de

* Sitz der Gesellschaft: Moehnestrasse 55 * D-59755 Arnsberg
* Geschaeftsfuehrer: Rainer Luelsdorf, Alfred Schroeder
* Vorsitzender des Beirats: Juergen Michels
* Amtsgericht Arnsberg * HRB 1968

Wir erfüllen unsere Informationspflichten zum Datenschutz gem. der 
Artikel 13
und 14 DS-GVO durch Veröffentlichung auf unserer Internetseite unter:
https://www.gonicus.de/datenschutz oder durch Zusendung auf Ihre 
formlose Anfrage.