[rdo-dev] [Infra] images server cleanup

Thu Nov 23 14:16:43 UTC 2017

Since there hasn't been any progress on this, I'll implement a basic
conservative cron on the image server for the time being in order to
automatically delete older images.

It won't be any different than the logic I've been using to clean things up
manually so far.
It won't have any logic around "keep 'N' last promotions", it will be based
on an amount of days -- excluding the symlinked images.

Please let me know once you have something else ready to use and I'll
disable the cron.

Thanks.

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

On Tue, Oct 31, 2017 at 8:42 AM, David Moreau Simard <dms at redhat.com> wrote:

> Has there been any progress on this ?
>
> I'm still fairly manually cleaning up older images because the server
> keeps filling up.
> I had planned additional storage for the server but it might not have been
> necessary because I heard about only keeping a few images worth of backlog.
>
> Pike is always the worst offender:
> ==
> # du -sh /var/www/html/images/* |sort -h
> 8.3G ./aarch64
> 9.9G ./ooo-snap
> 66G ./master
> 74G ./ocata
> 93G ./newton
> 193G ./pike
> ==
> # du -sh /var/www/html/images/pike/rdo_trunk/* |sort -h
> 0 /var/www/html/images/pike/rdo_trunk/current-tripleo
> 0 /var/www/html/images/pike/rdo_trunk/current-tripleo-rdo
> 0 /var/www/html/images/pike/rdo_trunk/tripleo-ci-testing
> 1.6G /var/www/html/images/pike/rdo_trunk/tripleo-upstream
> 4.3G /var/www/html/images/pike/rdo_trunk/old-tripleo
> 8.5G /var/www/html/images/pike/rdo_trunk/0712ed3b8c6193ca4978becf70da62
> c6c31edabc_90cbfd4f
> 8.5G /var/www/html/images/pike/rdo_trunk/0de7665e14f222802fbed40fa7df93
> b4a4082b2d_90cbfd4f
> 8.5G /var/www/html/images/pike/rdo_trunk/480e79b7a3d2f0f6e6e22b92c62894
> 26352d492c_c2957bbf
> 8.5G /var/www/html/images/pike/rdo_trunk/60d6e87cac10ff1f95a028c6176e76
> 8214ec8b77_9e72cb29
> 8.5G /var/www/html/images/pike/rdo_trunk/6beba54a71510525d5bbc4956d20d2
> 7bffa982e5_75873c3c
> 8.5G /var/www/html/images/pike/rdo_trunk/6d54c627703522921f41b5a8354838
> 0f1961034b_5b18c6af
> 8.5G /var/www/html/images/pike/rdo_trunk/75baddd6522aa86bd9028258937709
> c828fa1404_9e324686
> 8.5G /var/www/html/images/pike/rdo_trunk/8ef8f2bc46ac58385c6f92fbc9812a
> b6804a7ed2_4b120b84
> 8.5G /var/www/html/images/pike/rdo_trunk/a2369c6e219fe50fb0100806479009
> 055ada73dc_566fe0ed
> 8.5G /var/www/html/images/pike/rdo_trunk/cf3665fe1c60d43aa39f1880e42787
> 5c9c571058_5b18c6af
> 8.5G /var/www/html/images/pike/rdo_trunk/d335965eb848cfde5cc06136b5b2fc
> c6b436a419_7941156c
> 8.5G /var/www/html/images/pike/rdo_trunk/ec8fc5d5154cab4b5167b917b11056
> d4bff4ef06_37239c88
> 8.5G /var/www/html/images/pike/rdo_trunk/f9a2508318c8e6b2c6083f1fd8f719
> 9aba6fe1a4_0a2693a1
> 8.5G /var/www/html/images/pike/rdo_trunk/fd979d95d613e40be228695c0471c7
> 3cf9a5e3f4_9e324686
> 8.6G /var/www/html/images/pike/rdo_trunk/3c59aa392805d862096ed8ed6d9dbe
> 4ee72f0630_e400a1b4
> 8.6G /var/www/html/images/pike/rdo_trunk/6bef899ed13e0dcc5ba6a99bc1859f
> b77682bb4c_566fe0ed
> 8.6G /var/www/html/images/pike/rdo_trunk/7153e0cbc5b0e6433313a3bc6051b2
> c0775d3804_7df0efc2
> 8.6G /var/www/html/images/pike/rdo_trunk/fe2afcb87218af6a3523be5a885d26
> 0ec54d24a5_31d058df
> 9.4G /var/www/html/images/pike/rdo_trunk/9adfb9df52a0c22da87d528da14a32
> d2c2b516b7_0a2693a1
> ==
>
> On another note, the undercloud.qcow2 image went from 2.8GB in Newton to
> 6.9GB in Pike... is that legitimate ? There's no bloat in there ?
>
>
>
> David Moreau Simard
> Senior Software Engineer | OpenStack RDO
>
> dmsimard = [irc, github, twitter]
>
> On Tue, Oct 24, 2017 at 12:25 PM, David Moreau Simard <dms at redhat.com>
> wrote:
>
>> If the directory structure is the same across all directories and
>> releases, is there a reason why we couldn't simply run a cron on the
>> machine that woule regularly delete older images ?
>>
>> The migration from the previous (fedora) machine on OS1 to the new CentOS
>> server on RDO Cloud was largely manual and there wasn't any playbooks
>> involved.
>> We'd like to run full automation like upstream -infrastructure does. This
>> would allow anyone to submit a change to our playbooks, they'd be reviewed
>> and applied automatically.
>>
>> Setting up this cron could be one part of the tasks involved in setting
>> up the image server.
>>
>>
>>
>> David Moreau Simard
>> Senior Software Engineer | OpenStack RDO
>>
>> dmsimard = [irc, github, twitter]
>>
>> On Tue, Oct 24, 2017 at 10:55 AM, Gabriele Cerami <gcerami at redhat.com>
>> wrote:
>>
>>> Hi,
>>>
>>> we'd like to participate actively in cleaning up after ourselves the
>>> images we upload at each tripleo promotion. We are planning to do so
>>> also for the container images in dockerhub, so part of this process has
>>> to be done anyway. (maybe we should also do it in rdoregistry)
>>> Since our access to the server is limited to sftp, we are thinking about
>>> using paramiko library in our promoter script, to get the list of hashes
>>> uploaded and their mtimes, so we can delete the oldest ones.
>>>
>>> Is there any better solution ?
>>>
>>> Thanks
>>> _______________________________________________
>>> dev mailing list
>>> dev at lists.rdoproject.org
>>> http://lists.rdoproject.org/mailman/listinfo/dev
>>>
>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20171123/e6481ebc/attachment.html>