<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Since there hasn't been any progress on this, I'll implement a basic conservative cron on the image server for the time being in order to automatically delete older images.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">It won't be any different than the logic I've been using to clean things up manually so far.<br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">It won't have any logic around "keep 'N' last promotions", it will be based on an amount of days -- excluding the symlinked images.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Please let me know once you have something else ready to use and I'll disable the cron.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Thanks.<br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><br>David Moreau Simard<br>Senior Software Engineer | OpenStack RDO<br><br>dmsimard = [irc, github, twitter]</div></div>
<br><div class="gmail_quote">On Tue, Oct 31, 2017 at 8:42 AM, David Moreau Simard <span dir="ltr"><<a href="mailto:dms@redhat.com" target="_blank">dms@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Has there been any progress on this ?</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">I'm still fairly manually cleaning up older images because the server keeps filling up.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">I had planned additional storage for the server but it might not have been necessary because I heard about only keeping a few images worth of backlog.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Pike is always the worst offender:</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">==</div><div class="gmail_default"><div class="gmail_default"><font face="arial, helvetica, sans-serif"># du -sh /var/www/html/images/* |sort -h</font></div><div class="gmail_default"><span style="font-family:arial,helvetica,sans-serif">8.3G</span><span style="font-family:arial,helvetica,sans-serif;white-space:pre-wrap"> </span><span style="font-family:arial,helvetica,sans-serif">./aarch64</span><br></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">9.9G<span style="white-space:pre-wrap"> </span>./ooo-snap</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">66G<span style="white-space:pre-wrap"> </span>./master</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">74G<span style="white-space:pre-wrap"> </span>./ocata</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">93G<span style="white-space:pre-wrap"> </span>./newton</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">193G<span style="white-space:pre-wrap"> </span>./pike</font></div><div style="font-family:arial,helvetica,sans-serif">==<br></div><div><div><font face="arial, helvetica, sans-serif"># du -sh /var/www/html/images/pike/rdo_<wbr>trunk/* |sort -h</font></div><div><font face="arial, helvetica, sans-serif">0<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/current-tripleo</font></div><div><font face="arial, helvetica, sans-serif">0<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/current-tripleo-rdo</font></div><div><font face="arial, helvetica, sans-serif">0<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/tripleo-ci-testing</font></div><div><font face="arial, helvetica, sans-serif">1.6G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/tripleo-upstream</font></div><div><font face="arial, helvetica, sans-serif">4.3G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/old-tripleo</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>0712ed3b8c6193ca4978becf70da62<wbr>c6c31edabc_90cbfd4f</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>0de7665e14f222802fbed40fa7df93<wbr>b4a4082b2d_90cbfd4f</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>480e79b7a3d2f0f6e6e22b92c62894<wbr>26352d492c_c2957bbf</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>60d6e87cac10ff1f95a028c6176e76<wbr>8214ec8b77_9e72cb29</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>6beba54a71510525d5bbc4956d20d2<wbr>7bffa982e5_75873c3c</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>6d54c627703522921f41b5a8354838<wbr>0f1961034b_5b18c6af</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>75baddd6522aa86bd9028258937709<wbr>c828fa1404_9e324686</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>8ef8f2bc46ac58385c6f92fbc9812a<wbr>b6804a7ed2_4b120b84</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>a2369c6e219fe50fb0100806479009<wbr>055ada73dc_566fe0ed</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>cf3665fe1c60d43aa39f1880e42787<wbr>5c9c571058_5b18c6af</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>d335965eb848cfde5cc06136b5b2fc<wbr>c6b436a419_7941156c</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>ec8fc5d5154cab4b5167b917b11056<wbr>d4bff4ef06_37239c88</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>f9a2508318c8e6b2c6083f1fd8f719<wbr>9aba6fe1a4_0a2693a1</font></div><div><font face="arial, helvetica, sans-serif">8.5G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>fd979d95d613e40be228695c0471c7<wbr>3cf9a5e3f4_9e324686</font></div><div><font face="arial, helvetica, sans-serif">8.6G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>3c59aa392805d862096ed8ed6d9dbe<wbr>4ee72f0630_e400a1b4</font></div><div><font face="arial, helvetica, sans-serif">8.6G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>6bef899ed13e0dcc5ba6a99bc1859f<wbr>b77682bb4c_566fe0ed</font></div><div><font face="arial, helvetica, sans-serif">8.6G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>7153e0cbc5b0e6433313a3bc6051b2<wbr>c0775d3804_7df0efc2</font></div><div><font face="arial, helvetica, sans-serif">8.6G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>fe2afcb87218af6a3523be5a885d26<wbr>0ec54d24a5_31d058df</font></div><div><font face="arial, helvetica, sans-serif">9.4G<span style="white-space:pre-wrap"> </span>/var/www/html/images/pike/rdo_<wbr>trunk/<wbr>9adfb9df52a0c22da87d528da14a32<wbr>d2c2b516b7_0a2693a1</font></div></div></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">==</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">On another note, the undercloud.qcow2 image went from 2.8GB in Newton to 6.9GB in Pike... is that legitimate ? There's no bloat in there ?</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><div class="gmail_extra"><span class=""><br clear="all"><div><div class="m_-6377017802326806738gmail_signature" data-smartmail="gmail_signature"><br>David Moreau Simard<br>Senior Software Engineer | OpenStack RDO<br><br>dmsimard = [irc, github, twitter]</div></div>
<br></span><div><div class="h5"><div class="gmail_quote">On Tue, Oct 24, 2017 at 12:25 PM, David Moreau Simard <span dir="ltr"><<a href="mailto:dms@redhat.com" target="_blank">dms@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">If the directory structure is the same across all directories and releases, is there a reason why we couldn't simply run a cron on the machine that woule regularly delete older images ?</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">The migration from the previous (fedora) machine on OS1 to the new CentOS server on RDO Cloud was largely manual and there wasn't any playbooks involved.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">We'd like to run full automation like upstream -infrastructure does. This would allow anyone to submit a change to our playbooks, they'd be reviewed and applied automatically.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Setting up this cron could be one part of the tasks involved in setting up the image server.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><div class="gmail_extra"><br clear="all"><div><div class="m_-6377017802326806738m_316247670048448999gmail_signature" data-smartmail="gmail_signature"><br>David Moreau Simard<br>Senior Software Engineer | OpenStack RDO<br><br>dmsimard = [irc, github, twitter]</div></div><div><div class="m_-6377017802326806738h5">
<br><div class="gmail_quote">On Tue, Oct 24, 2017 at 10:55 AM, Gabriele Cerami <span dir="ltr"><<a href="mailto:gcerami@redhat.com" target="_blank">gcerami@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
we'd like to participate actively in cleaning up after ourselves the<br>
images we upload at each tripleo promotion. We are planning to do so<br>
also for the container images in dockerhub, so part of this process has<br>
to be done anyway. (maybe we should also do it in rdoregistry)<br>
Since our access to the server is limited to sftp, we are thinking about<br>
using paramiko library in our promoter script, to get the list of hashes<br>
uploaded and their mtimes, so we can delete the oldest ones.<br>
<br>
Is there any better solution ?<br>
<br>
Thanks<br>
______________________________<wbr>_________________<br>
dev mailing list<br>
<a href="mailto:dev@lists.rdoproject.org" target="_blank">dev@lists.rdoproject.org</a><br>
<a href="http://lists.rdoproject.org/mailman/listinfo/dev" rel="noreferrer" target="_blank">http://lists.rdoproject.org/ma<wbr>ilman/listinfo/dev</a><br>
<br>
To unsubscribe: <a href="mailto:rdo-list-unsubscribe@redhat.com" target="_blank">rdo-list-unsubscribe@redhat.co<wbr>m</a><br>
</blockquote></div><br></div></div></div>
</blockquote></div><br></div></div></div>
</blockquote></div><br></div>