[rdo-dev] Attempt to reproduce Carlos Camacho simplest deployment (controller+compute) . It fails when running overcloud_prep_containers.sh

[Michele Baldessari]

On Fri, Feb 23, 2018 at 07:52:49AM +0000, Boris Derzhavets wrote:
> In meantime script  overcloud_prep_containers.sh crashes with log log.
> 
> This behavior stays the same for about several  days already.
> 
> -----------------------------------------------
> 
> overcloud_prep_containers.log
> 
> -----------------------------------------------
> 
> 2018-02-23 07:21:27 | Trying paths: ['/home/stack/.docker/config.json', '/home/stack/.dockercfg']
> 2018-02-23 07:21:27 | No config file found
> 2018-02-23 07:21:27 | http://localhost:None "GET /version HTTP/1.1" 200 299
> 2018-02-23 07:21:27 | Running skopeo inspect docker://docker.io/tripleomaster/centos-binary-gnocchi-api:current-tripleo-rdo
> 2018-02-23 07:21:28 | Extracting
> 2018-02-23 07:21:28 | Extracting
> 2018-02-23 07:21:28 | None: None
> 2018-02-23 07:21:28 | Traceback (most recent call last):
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
> 2018-02-23 07:21:28 |     result = cmd.run(parsed_args)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 25, in run
> 2018-02-23 07:21:28 |     super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
> 2018-02-23 07:21:28 |     return super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
> 2018-02-23 07:21:28 |     return_code = self.take_action(parsed_args) or 0
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleoclient/v1/container_image.py", line 66, in take_action
> 2018-02-23 07:21:28 |     uploader.upload()
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 94, in upload
> 2018-02-23 07:21:28 |     uploader.run_tasks()
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 354, in run_tasks
> 2018-02-23 07:21:28 |     for result in p.map(docker_upload, self.upload_tasks):
> 2018-02-23 07:21:28 |   File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
> 2018-02-23 07:21:28 |     return self.map_async(func, iterable, chunksize).get()
> 2018-02-23 07:21:28 |   File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
> 2018-02-23 07:21:28 |     raise self._value
> 2018-02-23 07:21:28 | ReadTimeoutError: None: None
> 2018-02-23 07:21:28 | clean_up UploadImage: None: None
> 2018-02-23 07:21:28 | Traceback (most recent call last):
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 134, in run
> 2018-02-23 07:21:28 |     ret_val = super(OpenStackShell, self).run(argv)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/cliff/app.py", line 279, in run
> 2018-02-23 07:21:28 |     result = self.run_subcommand(remainder)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 169, in run_subcommand
> 2018-02-23 07:21:28 |     ret_value = super(OpenStackShell, self).run_subcommand(argv)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
> 2018-02-23 07:21:28 |     result = cmd.run(parsed_args)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 25, in run
> 2018-02-23 07:21:28 |     super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
> 2018-02-23 07:21:28 |     return super(Command, self).run(parsed_args)
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
> 2018-02-23 07:21:28 |     return_code = self.take_action(parsed_args) or 0
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleoclient/v1/container_image.py", line 66, in take_action
> 2018-02-23 07:21:28 |     uploader.upload()
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 94, in upload
> 2018-02-23 07:21:28 |     uploader.run_tasks()
> 2018-02-23 07:21:28 |   File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 354, in run_tasks
> 2018-02-23 07:21:28 |     for result in p.map(docker_upload, self.upload_tasks):
> 2018-02-23 07:21:28 |   File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
> 2018-02-23 07:21:28 |     return self.map_async(func, iterable, chunksize).get()
> 2018-02-23 07:21:28 |   File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
> 2018-02-23 07:21:28 |     raise self._value
> 2018-02-23 07:21:28 | ReadTimeoutError: None: None
> 2018-02-23 07:21:28 |
> 2018-02-23 07:21:28 | END return value: 1

FWIW I have observed this one too (although not 100% of time, it is
frequent enough): https://bugs.launchpad.net/tripleo/+bug/1749663
Rerunning the command a few times gets past it.
-- 
Michele Baldessari            <michele at acksyn.org>
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D