On Fri, Feb 23, 2018 at 07:52:49AM +0000, Boris Derzhavets wrote:
> In meantime script overcloud_prep_containers.sh crashes with log log.
>
> This behavior stays the same for about several days already.
>
> -----------------------------------------------
>
> overcloud_prep_containers.log
>
> -----------------------------------------------
>
> 2018-02-23 07:21:27 | Trying paths: ['/home/stack/.docker/config.json', '/home/stack/.dockercfg']
> 2018-02-23 07:21:27 | No config file found
> 2018-02-23 07:21:27 |
http://localhost:None "GET /version HTTP/1.1" 200 299
> 2018-02-23 07:21:27 | Running skopeo inspect docker://docker.io/tripleomaster/centos-binary-gnocchi-api:current-tripleo-rdo
> 2018-02-23 07:21:28 | Extracting
> 2018-02-23 07:21:28 | Extracting
> 2018-02-23 07:21:28 | None: None
> 2018-02-23 07:21:28 | Traceback (most recent call last):
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
> 2018-02-23 07:21:28 | result = cmd.run(parsed_args)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 25, in run
> 2018-02-23 07:21:28 | super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
> 2018-02-23 07:21:28 | return super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
> 2018-02-23 07:21:28 | return_code = self.take_action(parsed_args) or 0
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleoclient/v1/container_image.py", line 66, in take_action
> 2018-02-23 07:21:28 | uploader.upload()
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 94, in upload
> 2018-02-23 07:21:28 | uploader.run_tasks()
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 354, in run_tasks
> 2018-02-23 07:21:28 | for result in p.map(docker_upload, self.upload_tasks):
> 2018-02-23 07:21:28 | File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
> 2018-02-23 07:21:28 | return self.map_async(func, iterable, chunksize).get()
> 2018-02-23 07:21:28 | File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
> 2018-02-23 07:21:28 | raise self._value
> 2018-02-23 07:21:28 | ReadTimeoutError: None: None
> 2018-02-23 07:21:28 | clean_up UploadImage: None: None
> 2018-02-23 07:21:28 | Traceback (most recent call last):
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 134, in run
> 2018-02-23 07:21:28 | ret_val = super(OpenStackShell, self).run(argv)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/cliff/app.py", line 279, in run
> 2018-02-23 07:21:28 | result = self.run_subcommand(remainder)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 169, in run_subcommand
> 2018-02-23 07:21:28 | ret_value = super(OpenStackShell, self).run_subcommand(argv)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
> 2018-02-23 07:21:28 | result = cmd.run(parsed_args)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 25, in run
> 2018-02-23 07:21:28 | super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
> 2018-02-23 07:21:28 | return super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
> 2018-02-23 07:21:28 | return_code = self.take_action(parsed_args) or 0
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleoclient/v1/container_image.py", line 66, in take_action
> 2018-02-23 07:21:28 | uploader.upload()
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 94, in upload
> 2018-02-23 07:21:28 | uploader.run_tasks()
> 2018-02-23 07:21:28 | File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 354, in run_tasks
> 2018-02-23 07:21:28 | for result in p.map(docker_upload, self.upload_tasks):
> 2018-02-23 07:21:28 | File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
> 2018-02-23 07:21:28 | return self.map_async(func, iterable, chunksize).get()
> 2018-02-23 07:21:28 | File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
> 2018-02-23 07:21:28 | raise self._value
> 2018-02-23 07:21:28 | ReadTimeoutError: None: None
> 2018-02-23 07:21:28 |
> 2018-02-23 07:21:28 | END return value: 1
FWIW I have observed this one too (although not 100% of time, it is
frequent enough):
https://bugs.launchpad.net/tripleo/+bug/1749663
bugs.launchpad.net
Via https://bugs.launchpad.net/tripleo/+bug/1746305 and https://review.openstack.org/#/c/539383/ we made the image upload a bit more resilient. I seem to be still hitting it rather often: Extracting Extracting Extracting Pull complete imagename: docker.io/tripleomaster/centos-binary-neutron-dhcp-agent:current-tripleo-rdo
Trying paths: ['/home/stack/.docker/config.json', '/home/stack/.dockercfg'] No config file found None: None Traceback (most recent call last): File "/usr/lib/python2.7/site...
|
Rerunning the command a few times gets past it.
--
Michele Baldessari <michele@acksyn.org>
C2A5 9DA3 9961 4FFB E01B D0BC DDD4 DCCB 7515 5C6D