[rdo-list] static delorean-deps repos

[Haïkel]

2017-09-28 19:44 GMT+02:00 Wesley Hayutin <whayutin at redhat.com>:
>
>
> On Thu, Sep 28, 2017 at 1:30 PM, Haïkel <hguemar at fedoraproject.org> wrote:
>>
>> 2017-09-28 18:07 GMT+02:00 Alfredo Moralejo Alonso <amoralej at redhat.com>:
>> > On Thu, Sep 28, 2017 at 4:39 PM, Wesley Hayutin <whayutin at redhat.com>
>> > wrote:
>> >> Greetings,
>> >>
>> >> Had a brief discussion on the release delivery meeting this morning
>> >> with Jon
>> >> Schlueter where as the rel-del had a requirement for the CI to
>> >> constantly
>> >> execute to retest any potential changes to the delorean-deps yum repo.
>> >> This is required because it informs the import process.
>> >>
>> >> Ideally we don't have to constantly tax both our hardware and people
>> >> resources because we have have an unpinned repo that can change at any
>> >> moment.
>> >>
>> >> A static delorean-deps repo married to a delorean repo would be more
>> >> efficient for everyone.  How that is achieved is under discussion.
>> >>
>> >
>> > Having static delorean-deps repo for a RDO Trunk hash may be
>> > problematic also because centos, virtualization and ceph repos are
>> > changing repos and we may need to introduce new
>> > dependencies for the same hash.
>> >
>>
>> Actually, it can be done for third-party repo (See below). We just
>> need a step to retrieve,
>> save yum metadata. Possibly, we may have to mangle base url but it's easy
>> to do
>> (yum metadata are just sqlite and xml databases).
>>
>> > What i'd propose is to have versioned dependencies repos so that jobs
>> > can easily report the version of deps used and replay the job with a
>> > specific dependencies snapshot if needed.
>> >
>>
>> Just to clarify to the readers, we don't need to version the whole
>> repositories only the yum
>> metadata. By convention, published packages are not removed from
>> public repos, and
>> yum/dnf won't see newer packages that are not in metadata.
>>
>> So it's quite a storage-efficient way to handle the issue mentioned by
>> Wes.
>
>
> Haikel,
>
> Would we be able to version the CentOS base repos as well via metadata?

I am just pointing that this is possible without affecting RDO jobs
(aka phase 1)
nor CentOS repo.

> To Alfredo's point, if somethign changes in CentOS that requires a change to
> the delorean deps then we may have an issue.
>

No changes is required, we would just save the metadata somewhere and make it
consumable for you.
E.g : At the beginning of a promotion job, we would save the metadata
of all the repositories
used, and we could publish it somewhere with the right *.repo file.
That does not mean or implies the promotion job would be using it.

It can be used for other use-cases than yours but I'd like to keep
that as a separate
discussion to avoid confusion.



In short, we're just adding a new *artefact* to the CI jobs, nothing
more, nothing less.



> I'm not sure how often a change in CentOS requires a change to
> delorean-deps.
> I suspect if that happens often we will hit a lot failures.
>
> Thanks for the comments and feedback thus far!
>
>>
>>
>> >> This email is only to serve as an entry point for a public discussion
>> >> around
>> >> that issue.
>> >>
>> >> Thanks all!
>> >>
>> >> _______________________________________________
>> >> rdo-list mailing list
>> >> rdo-list at redhat.com
>> >> https://www.redhat.com/mailman/listinfo/rdo-list
>> >>
>> >> To unsubscribe: rdo-list-unsubscribe at redhat.com
>> >
>> > _______________________________________________
>> > rdo-list mailing list
>> > rdo-list at redhat.com
>> > https://www.redhat.com/mailman/listinfo/rdo-list
>> >
>> > To unsubscribe: rdo-list-unsubscribe at redhat.com
>
>