Hi Bernard, I've added this as a topic for the https://etherpad.openstack.org/p/RDO-Meeting today, with some initial questions to explore. On Wed, Jan 10, 2018 at 1:50 PM, Bernard Cafarelli <bcafarel(a)redhat.com&gt; wrote: > * easier install/maintenance for the user, tripleo can consume the > image directly (from a package) How do you plan to distribute the image, wrapped inside RPM? > * ensuring up-to-date amphora images (to match the controller version, > for security updates, …) That's the most critical part of the process to figure out, how to automate image updates, just run it daily, trigger when included packages change...

> * allow to test and confirm the amphora works properly with latest > changes, including enforcing SELinux (CentOS upstream gate runs in > permissive) And that's the second part, which CI job would test it properly? > * use this image in tripleo CI (instead of having to build it there) Related to the up-to-date issue: how to ensure it's latest for CI? > * (future) extend this system for other system VM images Which ones? Cheers, Alan _______________________________________________ dev mailing list dev(a)lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/dev To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org

On 10 January 2018 at 14:41, Assaf Muller <assaf(a)redhat.com&gt; wrote: > On Wed, Jan 10, 2018 at 8:10 AM, Alan Pevec <apevec(a)redhat.com&gt; wrote: >> Hi Bernard, >> >> I've added this as a topic for the >> https://etherpad.openstack.org/p/RDO-Meeting today, >> with some initial questions to explore. Thanks, I will be there in #rdo >> >> On Wed, Jan 10, 2018 at 1:50 PM, Bernard Cafarelli <bcafarel(a)redhat.com&gt; wrote: >>> * easier install/maintenance for the user, tripleo can consume the >>> image directly (from a package) >> >> How do you plan to distribute the image, wrapped inside RPM? If possible, that is my initial idea yes (easier to fetch, and allows tracking available updates with yum) >> >>> * ensuring up-to-date amphora images (to match the controller version, >>> for security updates, …) >> >> That's the most critical part of the process to figure out, how to >> automate image updates, >> just run it daily, trigger when included packages change... > > On Octavia patches (because the agent in the image might change), but > it's worth pointing out that other elements in the image may need > updating - Say, on a kernel CVE fix, haproxy update, etc. Maybe on > Octavia changes + on a nightly basis? A side-effect here would be that the user will see a new image available, while there are maybe no real changes. But determining if an image has real changes may not be easy (thinking out loud, comparing the installed packages versions?) > >> >>> * allow to test and confirm the amphora works properly with latest >>> changes, including enforcing SELinux (CentOS upstream gate runs in >>> permissive) >> >> And that's the second part, which CI job would test it properly? A bit outside of the RDO pipeline, tripleo CI would use it to test Octavia deployments (I will check here with relevant folks) In the pipeline yes it would be nice to test to confirm that Octavia parts work fine with this repository version >> >>> * use this image in tripleo CI (instead of having to build it there) >> >> Related to the up-to-date issue: how to ensure it's latest for CI? >> >>> * (future) extend this system for other system VM images >> >> Which ones? Currently off the top of my head, Sahara and Trove? Though I did not study it further if it is possible/worth it for these projects Projects like tacker also have service VMs, but these are probably too specific/customizable

>> >> Cheers, >> Alan >> _______________________________________________ >> dev mailing list >> dev(a)lists.rdoproject.org >> http://lists.rdoproject.org/mailman/listinfo/dev >> >> To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org _______________________________________________ dev mailing list dev(a)lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/dev To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org

If we want to deliver via RPM and build on each Octavia change, we could try to add it to the octavia spec and build it using DLRN. Does the script require many external resources besides diskimage-builder? I'm not sure if that would work on CBS though, if we need to have network connectivity during the build process.

----- Original Message ----- > On Wed, Jan 10, 2018 at 7:50 PM, Javier Pena <jpena(a)redhat.com&gt; wrote: > > If we want to deliver via RPM and build on each Octavia change, we could > > try to add it to the octavia spec and build it using DLRN. Does the script > > require many external resources besides diskimage-builder? > > I'm not sure if that would work on CBS though, if we need to have network > > connectivity during the build process.

> > I would be concerned with the storage required, also we need to > trigger not only on Octavia distgit or upstream changes, all included > RPMs need to be checked checked for updates. > This could be simulated with dummy commits in distgit to force e.g. > nightly refresh but due to storage requirements, I'd keep image builds > outside trunk repos. > I have been doing some tests, and it looks like running diskimage-builder from a chroot is not the best idea (it tries to mount some tmpfs and fails), so even if we solved the storage issue it wouldn't work. I think our best chance is to create a periodic job to rebuild the images (daily) then upload them to images.rdoproject.org. This would be a similar approach to what we are currently doing with containers.

The only drawback of this alternative is that we would be distributing the qcow2 images instead of an RPM package, but we could still apply retention policies, and add some CI jobs to test them if needed.

Bumping the thead, upstream patches are merged now [0] With current upstream code, I can generate an image from master packages with: $ wget https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-180... $ virt-customize -a CentOS-7-x86_64-GenericCloud-1801-01.qcow2 --selinux-relabel --run-command 'yum-config-manager --add-repo http://trunk.rdoproject.org/centos7/delorean-deps.repo' $ virt-customize -a CentOS-7-x86_64-GenericCloud-1801-01.qcow2 --selinux-relabel --run-command 'yum-config-manager --add-repo https://trunk.rdoproject.org/centos7/current-passed-ci/delorean.repo' $ DIB_LOCAL_IMAGE=/home/stack/CentOS-7-x86_64-GenericCloud-1801-01.qcow2 /opt/stack/octavia/diskimage-create/diskimage-create.sh -p -i centos -o amphora-x64-haproxy-centos.qcow2 This is with devstack, but will be mostly the same when RDO packages are updated (just the script location that then comes from openstack-octavia-diskimage-create package)

So what are the next steps here? missing information, place to track this, item for next meeting, action items, … ?

[0] https://review.openstack.org/#/c/522626/ On 12 January 2018 at 13:05, Bernard Cafarelli <bcafarel(a)redhat.com&gt; wrote: > On 11 January 2018 at 11:53, Javier Pena <jpena(a)redhat.com&gt; wrote: >> ----- Original Message ----- >>> On Wed, Jan 10, 2018 at 7:50 PM, Javier Pena <jpena(a)redhat.com&gt; wrote: >>> > If we want to deliver via RPM and build on each Octavia change, we >>> > could >>> > try to add it to the octavia spec and build it using DLRN. Does the >>> > script >>> > require many external resources besides diskimage-builder? >>> > I'm not sure if that would work on CBS though, if we need to have >>> > network >>> > connectivity during the build process. > I looked a bit initially into building the image directly in spec, one > problem was how to pass the needed RDO packages properly to > diskimage-builder (as a repo so that yum pulls them in). > Apart from some configuration tweaks, most of the steps sum up to yum > calls (system update - install haproxy, keepalived, … - install > openstack-octavia-amphora-agent), these need network access, or at > least local mirrors. >>> >>> I would be concerned with the storage required, also we need to >>> trigger not only on Octavia distgit or upstream changes, all included >>> RPMs need to be checked checked for updates. >>> This could be simulated with dummy commits in distgit to force e.g. >>> nightly refresh but due to storage requirements, I'd keep image builds >>> outside trunk repos. >>> >> >> I have been doing some tests, and it looks like running diskimage-builder >> from a chroot is not the best idea (it tries to mount some tmpfs and >> fails), so even if we solved the storage issue it wouldn't work. >> I think our best chance is to create a periodic job to rebuild the images >> (daily) then upload them to images.rdoproject.org. This would be a >> similar approach to what we are currently doing with containers. > That would work for "keeping other packages up to date" too > >> The only drawback of this alternative is that we would be distributing the >> qcow2 images instead of an RPM package, but we could still apply >> retention policies, and add some CI jobs to test them if needed. > On disk usage and retention polices, the images I build locally (with > CentOS) are 500-500 MB qcow2 files > > -- > Bernard -- Bernard Cafarelli _______________________________________________ dev mailing list dev(a)lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/dev To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org

----- Original Message ----- > Bumping the thead, upstream patches are merged now [0] > > With current upstream code, I can generate an image from master packages > with: > $ wget > https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-180... > $ virt-customize -a CentOS-7-x86_64-GenericCloud-1801-01.qcow2 > --selinux-relabel --run-command 'yum-config-manager --add-repo > http://trunk.rdoproject.org/centos7/delorean-deps.repo' > $ virt-customize -a CentOS-7-x86_64-GenericCloud-1801-01.qcow2 > --selinux-relabel --run-command 'yum-config-manager --add-repo > https://trunk.rdoproject.org/centos7/current-passed-ci/delorean.repo' > $ DIB_LOCAL_IMAGE=/home/stack/CentOS-7-x86_64-GenericCloud-1801-01.qcow2 > /opt/stack/octavia/diskimage-create/diskimage-create.sh -p -i centos > -o amphora-x64-haproxy-centos.qcow2 > > This is with devstack, but will be mostly the same when RDO packages > are updated (just the script location that then comes from > openstack-octavia-diskimage-create package) > I have run a quick test with the latest openstack-octavia-diskimage-create package from RDO Trunk, and it works like a charm. > So what are the next steps here? missing information, place to track > this, item for next meeting, action items, … ? > Let's add it as an item for the next meeting, so we define a plan. My proposal would be a daily build using a periodic job, storing images in a new path under images.rdoproject.org.

Regards, Javier > > [0] https://review.openstack.org/#/c/522626/ > > On 12 January 2018 at 13:05, Bernard Cafarelli <bcafarel(a)redhat.com&gt; wrote: > > On 11 January 2018 at 11:53, Javier Pena <jpena(a)redhat.com&gt; wrote: > >> ----- Original Message ----- > >>> On Wed, Jan 10, 2018 at 7:50 PM, Javier Pena <jpena(a)redhat.com&gt; wrote: > >>> > If we want to deliver via RPM and build on each Octavia change, we > >>> > could > >>> > try to add it to the octavia spec and build it using DLRN. Does the > >>> > script > >>> > require many external resources besides diskimage-builder? > >>> > I'm not sure if that would work on CBS though, if we need to have > >>> > network > >>> > connectivity during the build process. > > I looked a bit initially into building the image directly in spec, one > > problem was how to pass the needed RDO packages properly to > > diskimage-builder (as a repo so that yum pulls them in). > > Apart from some configuration tweaks, most of the steps sum up to yum > > calls (system update - install haproxy, keepalived, … - install > > openstack-octavia-amphora-agent), these need network access, or at > > least local mirrors. > >>> > >>> I would be concerned with the storage required, also we need to > >>> trigger not only on Octavia distgit or upstream changes, all included > >>> RPMs need to be checked checked for updates. > >>> This could be simulated with dummy commits in distgit to force e.g. > >>> nightly refresh but due to storage requirements, I'd keep image builds > >>> outside trunk repos. > >>> > >> > >> I have been doing some tests, and it looks like running > >> diskimage-builder > >> from a chroot is not the best idea (it tries to mount some tmpfs and > >> fails), so even if we solved the storage issue it wouldn't work. > >> I think our best chance is to create a periodic job to rebuild the > >> images > >> (daily) then upload them to images.rdoproject.org. This would be a > >> similar approach to what we are currently doing with containers. > > That would work for "keeping other packages up to date" too > > > >> The only drawback of this alternative is that we would be distributing > >> the > >> qcow2 images instead of an RPM package, but we could still apply > >> retention policies, and add some CI jobs to test them if needed. > > On disk usage and retention polices, the images I build locally (with > > CentOS) are 500-500 MB qcow2 files > > > > -- > > Bernard > > > > -- > Bernard Cafarelli > _______________________________________________ > dev mailing list > dev(a)lists.rdoproject.org > http://lists.rdoproject.org/mailman/listinfo/dev > > To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org > _______________________________________________ dev mailing list dev(a)lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/dev To unsubscribe: dev-unsubscribe(a)lists.rdoproject.org