[rdo-dev] [Infra] images server cleanup

[David Moreau Simard]

Has there been any progress on this ?

I'm still fairly manually cleaning up older images because the server keeps
filling up.
I had planned additional storage for the server but it might not have been
necessary because I heard about only keeping a few images worth of backlog.

Pike is always the worst offender:
==
# du -sh /var/www/html/images/* |sort -h
8.3G ./aarch64
9.9G ./ooo-snap
66G ./master
74G ./ocata
93G ./newton
193G ./pike
==
# du -sh /var/www/html/images/pike/rdo_trunk/* |sort -h
0 /var/www/html/images/pike/rdo_trunk/current-tripleo
0 /var/www/html/images/pike/rdo_trunk/current-tripleo-rdo
0 /var/www/html/images/pike/rdo_trunk/tripleo-ci-testing
1.6G /var/www/html/images/pike/rdo_trunk/tripleo-upstream
4.3G /var/www/html/images/pike/rdo_trunk/old-tripleo
8.5G
/var/www/html/images/pike/rdo_trunk/0712ed3b8c6193ca4978becf70da62c6c31edabc_90cbfd4f
8.5G
/var/www/html/images/pike/rdo_trunk/0de7665e14f222802fbed40fa7df93b4a4082b2d_90cbfd4f
8.5G
/var/www/html/images/pike/rdo_trunk/480e79b7a3d2f0f6e6e22b92c6289426352d492c_c2957bbf
8.5G
/var/www/html/images/pike/rdo_trunk/60d6e87cac10ff1f95a028c6176e768214ec8b77_9e72cb29
8.5G
/var/www/html/images/pike/rdo_trunk/6beba54a71510525d5bbc4956d20d27bffa982e5_75873c3c
8.5G
/var/www/html/images/pike/rdo_trunk/6d54c627703522921f41b5a83548380f1961034b_5b18c6af
8.5G
/var/www/html/images/pike/rdo_trunk/75baddd6522aa86bd9028258937709c828fa1404_9e324686
8.5G
/var/www/html/images/pike/rdo_trunk/8ef8f2bc46ac58385c6f92fbc9812ab6804a7ed2_4b120b84
8.5G
/var/www/html/images/pike/rdo_trunk/a2369c6e219fe50fb0100806479009055ada73dc_566fe0ed
8.5G
/var/www/html/images/pike/rdo_trunk/cf3665fe1c60d43aa39f1880e427875c9c571058_5b18c6af
8.5G
/var/www/html/images/pike/rdo_trunk/d335965eb848cfde5cc06136b5b2fcc6b436a419_7941156c
8.5G
/var/www/html/images/pike/rdo_trunk/ec8fc5d5154cab4b5167b917b11056d4bff4ef06_37239c88
8.5G
/var/www/html/images/pike/rdo_trunk/f9a2508318c8e6b2c6083f1fd8f7199aba6fe1a4_0a2693a1
8.5G
/var/www/html/images/pike/rdo_trunk/fd979d95d613e40be228695c0471c73cf9a5e3f4_9e324686
8.6G
/var/www/html/images/pike/rdo_trunk/3c59aa392805d862096ed8ed6d9dbe4ee72f0630_e400a1b4
8.6G
/var/www/html/images/pike/rdo_trunk/6bef899ed13e0dcc5ba6a99bc1859fb77682bb4c_566fe0ed
8.6G
/var/www/html/images/pike/rdo_trunk/7153e0cbc5b0e6433313a3bc6051b2c0775d3804_7df0efc2
8.6G
/var/www/html/images/pike/rdo_trunk/fe2afcb87218af6a3523be5a885d260ec54d24a5_31d058df
9.4G
/var/www/html/images/pike/rdo_trunk/9adfb9df52a0c22da87d528da14a32d2c2b516b7_0a2693a1
==

On another note, the undercloud.qcow2 image went from 2.8GB in Newton to
6.9GB in Pike... is that legitimate ? There's no bloat in there ?



David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

On Tue, Oct 24, 2017 at 12:25 PM, David Moreau Simard <dms at redhat.com>
wrote:

> If the directory structure is the same across all directories and
> releases, is there a reason why we couldn't simply run a cron on the
> machine that woule regularly delete older images ?
>
> The migration from the previous (fedora) machine on OS1 to the new CentOS
> server on RDO Cloud was largely manual and there wasn't any playbooks
> involved.
> We'd like to run full automation like upstream -infrastructure does. This
> would allow anyone to submit a change to our playbooks, they'd be reviewed
> and applied automatically.
>
> Setting up this cron could be one part of the tasks involved in setting up
> the image server.
>
>
>
> David Moreau Simard
> Senior Software Engineer | OpenStack RDO
>
> dmsimard = [irc, github, twitter]
>
> On Tue, Oct 24, 2017 at 10:55 AM, Gabriele Cerami <gcerami at redhat.com>
> wrote:
>
>> Hi,
>>
>> we'd like to participate actively in cleaning up after ourselves the
>> images we upload at each tripleo promotion. We are planning to do so
>> also for the container images in dockerhub, so part of this process has
>> to be done anyway. (maybe we should also do it in rdoregistry)
>> Since our access to the server is limited to sftp, we are thinking about
>> using paramiko library in our promoter script, to get the list of hashes
>> uploaded and their mtimes, so we can delete the oldest ones.
>>
>> Is there any better solution ?
>>
>> Thanks
>> _______________________________________________
>> dev mailing list
>> dev at lists.rdoproject.org
>> http://lists.rdoproject.org/mailman/listinfo/dev
>>
>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20171031/cea3d6ae/attachment.html>