3:12 p.m.
Greetings,
With the zuulv3 migration wrapping up, I wanted to start a thread about projects
that use the package-distgit-check-jobs template. These are projects like:
openstack/cloudkittyclient-distgit
I wanted to raise the idea of maybe pushing these projects directly upstream into
git.openstack.org. The main reason would be to leverage the upstream testing
infrastructure upstream, and maybe increase adoption of rpm with other
openstack teams. It is also one less this we as RDO have to manage on our own.
From a governance POV these could be under an rdoproject team, tripleo
or some
other.
To me the main question is around publishing of RPM and DLRN. Given secrets are
now part of zuulv3, is there any external services running that we'd need to
worry about? Could the publishing process be run from openstack-infra service?
- Paul
4:42 p.m.
With the zuulv3 migration wrapping up, I wanted to start a thread
about projects
that use the package-distgit-check-jobs template. These are projects like:
openstack/cloudkittyclient-distgit
I wanted to raise the idea of maybe pushing these projects directly upstream into
git.openstack.org. The main reason would be to leverage the upstream testing
infrastructure upstream, and maybe increase adoption of rpm with other
openstack teams. It is also one less this we as RDO have to manage on our own.
From a governance POV these could be under an rdoproject team,
tripleo or some
other.
Ideally we would not have one core team but keep them synced with
maintainers listed in rdoinfo,
how could that work with distgit in review.o.o ?
In review.rdo we're using SF resource manager to create gerrit ACL, is
that manual operation in openstack gerrit?
To me the main question is around publishing of RPM and DLRN. Given
secrets are
now part of zuulv3, is there any external services running that we'd need to
worry about? Could the publishing process be run from openstack-infra service?
I would not be comfortable putting CBS Koji certs in openstack-infra.
Could we instead trigger CBS Koji builds via 3rd party CI in review.rdo ?
Cheers,
Alan
9:26 a.m.
On Tue, Jul 03, 2018 at 11:42:09PM +0200, Alan Pevec wrote:
> With the zuulv3 migration wrapping up, I wanted to start a
thread about projects
> that use the package-distgit-check-jobs template. These are projects like:
>
> openstack/cloudkittyclient-distgit
>
> I wanted to raise the idea of maybe pushing these projects directly upstream into
> git.openstack.org. The main reason would be to leverage the upstream testing
> infrastructure upstream, and maybe increase adoption of rpm with other
> openstack teams. It is also one less this we as RDO have to manage on our own.
> From a governance POV these could be under an rdoproject team, tripleo or some
> other.
Ideally we would not have one core team but keep them synced with
maintainers listed in rdoinfo,
how could that work with distgit in review.o.o ?
In review.rdo we're using SF resource manager to create gerrit ACL, is
that manual operation in openstack gerrit?
Yes, we manage ACLs via code review too, so we can create one per distgit or
have a core group. We bootstrap each ACL with the PTL (or person who created
the projects) then they manage users vi gerrit.
> To me the main question is around publishing of RPM and DLRN.
Given secrets are
> now part of zuulv3, is there any external services running that we'd need to
> worry about? Could the publishing process be run from openstack-infra service?
I would not be comfortable putting CBS Koji certs in openstack-infra.
Could we instead trigger CBS Koji builds via 3rd party CI in review.rdo ?
Why not comfortable? It the same process as RDO zuul for upstream, no root would
be able to access them. However, yes, we can setup rdo zuul as 3pci and trigger
CBS builds from a post pipeline if needed.
Cheers,
Alan
9:33 a.m.
Hello,
On July 4, 2018 2:26 pm, Paul Belanger wrote:
On Tue, Jul 03, 2018 at 11:42:09PM +0200, Alan Pevec wrote:
> > With the zuulv3 migration wrapping up, I wanted to start a thread about
projects
> > that use the package-distgit-check-jobs template. These are projects like:
> >
> > openstack/cloudkittyclient-distgit
> >
> > I wanted to raise the idea of maybe pushing these projects directly upstream
into
> > git.openstack.org. The main reason would be to leverage the upstream testing
> > infrastructure upstream, and maybe increase adoption of rpm with other
> > openstack teams. It is also one less this we as RDO have to manage on our own.
>
>
> > From a governance POV these could be under an rdoproject team, tripleo or some
> > other.
>
> Ideally we would not have one core team but keep them synced with
> maintainers listed in rdoinfo,
> how could that work with distgit in review.o.o ?
> In review.rdo we're using SF resource manager to create gerrit ACL, is
> that manual operation in openstack gerrit?
>
Yes, we manage ACLs via code review too, so we can create one per distgit or
have a core group. We bootstrap each ACL with the PTL (or person who created
the projects) then they manage users vi gerrit.
The SF resource manager to update gerrit ACL is different from using the
gerrit interface as it is fully managed through code review in Software Factory.
See this example of an ACL update: https://review.rdoproject.org/r/10152
We could contribute the resource manager to jeepby, but that's a critical
workflow modification. Do you think the openstack-infra would accept this?
Note that branch creation and suppression are also managed by the
resource manager.
For tracability reasons the resource manager is a very important feature, but
perhaps we could adopt the openstack-infra way of doing this instead.
> > To me the main question is around publishing of RPM and
DLRN. Given secrets are
> > now part of zuulv3, is there any external services running that we'd need
to
> > worry about? Could the publishing process be run from openstack-infra service?
>
> I would not be comfortable putting CBS Koji certs in openstack-infra.
> Could we instead trigger CBS Koji builds via 3rd party CI in review.rdo ?
>
Why not comfortable? It the same process as RDO zuul for upstream, no root would
be able to access them. However, yes, we can setup rdo zuul as 3pci and trigger
CBS builds from a post pipeline if needed.
Beside the above concerns, I'd like to note a few other things we need to
be careful about undertaking such a migration.
Using review.openstack.org requires an UbuntuOne account...
review.rdoproject.org is currently configured to use GitHub identity,
but the next version of Software Factory supports openid
(e.g. fedora account system) and SAML2 (e.g. keycloack).
Tripleo-ci users are currently using a config repository to manage
check and gate jobs that need secrets as well as to update the
openstack-periodic pipeline timer to re-schedule the promotion jobs.
Could those users have +3 permissions on a config repository upstream?
We have also been working on new Zuul features[0] to support enqueue and
abort from the REST api[1], Unfortunately the change needed in Zuul
doesn't get much attention. Do you think we could enable them in
openstack-infra zuul? Or are we going to be relying on infra-core
to stop and re-enqueue RDO promotion jobs when needed?
There are lots of pros to doing such migration, but the blockers are
very important. Is there an etherpad we could list and comment on the
pros/cons of this migration?
Best regards,
-Tristan
[0] https://review.openstack.org/#/c/562321/
[0] https://tree.taiga.io/project/morucci-software-factory/us/960
9:50 a.m.
On Mon, Jul 09, 2018 at 02:33:23PM +0000, Tristan Cacqueray wrote:
Hello,
On July 4, 2018 2:26 pm, Paul Belanger wrote:
> On Tue, Jul 03, 2018 at 11:42:09PM +0200, Alan Pevec wrote:
> > > With the zuulv3 migration wrapping up, I wanted to start a thread about
projects
> > > that use the package-distgit-check-jobs template. These are projects
like:
> > >
> > > openstack/cloudkittyclient-distgit
> > >
> > > I wanted to raise the idea of maybe pushing these projects directly
upstream into
> > > git.openstack.org. The main reason would be to leverage the upstream
testing
> > > infrastructure upstream, and maybe increase adoption of rpm with other
> > > openstack teams. It is also one less this we as RDO have to manage on our
own.
> >
> >
> > > From a governance POV these could be under an rdoproject team, tripleo or
some
> > > other.
> >
> > Ideally we would not have one core team but keep them synced with
> > maintainers listed in rdoinfo,
> > how could that work with distgit in review.o.o ?
> > In review.rdo we're using SF resource manager to create gerrit ACL, is
> > that manual operation in openstack gerrit?
> >
> Yes, we manage ACLs via code review too, so we can create one per distgit or
> have a core group. We bootstrap each ACL with the PTL (or person who created
> the projects) then they manage users vi gerrit.
>
The SF resource manager to update gerrit ACL is different from using the
gerrit interface as it is fully managed through code review in Software Factory.
See this example of an ACL update: https://review.rdoproject.org/r/10152
We could contribute the resource manager to jeepby, but that's a critical
workflow modification. Do you think the openstack-infra would accept this?
Note that branch creation and suppression are also managed by the
resource manager.
For tracability reasons the resource manager is a very important feature, but
perhaps we could adopt the openstack-infra way of doing this instead.
I would reach out to infra team on Tuesday and maybe start a spec to contribute
this feature upstream. Give our love of all things yaml and code review, this
might be a welcome feature.
> > > To me the main question is around publishing of RPM
and DLRN. Given secrets are
> > > now part of zuulv3, is there any external services running that we'd
need to
> > > worry about? Could the publishing process be run from openstack-infra
service?
> >
> > I would not be comfortable putting CBS Koji certs in openstack-infra.
> > Could we instead trigger CBS Koji builds via 3rd party CI in review.rdo ?
> >
> Why not comfortable? It the same process as RDO zuul for upstream, no root would
> be able to access them. However, yes, we can setup rdo zuul as 3pci and trigger
> CBS builds from a post pipeline if needed.
>
Beside the above concerns, I'd like to note a few other things we need to
be careful about undertaking such a migration.
Using review.openstack.org requires an UbuntuOne account...
review.rdoproject.org is currently configured to use GitHub identity,
but the next version of Software Factory supports openid
(e.g. fedora account system) and SAML2 (e.g. keycloack).
Been on the table for a long while to migrate off UbuntuOne for SSO. I believe
this will get active again with winterscale effort happening.
Tripleo-ci users are currently using a config repository to manage
check and gate jobs that need secrets as well as to update the
openstack-periodic pipeline timer to re-schedule the promotion jobs.
Could those users have +3 permissions on a config repository upstream?
We don't allow untrusted jobs to have secrets in check / gate pipelines
today,
I don't believe we'd give +3 just to use secrets. We'd likely want to work
with
upstream here and better understand why those jobs need it. Projects are using
secrets in tree, but more for post / artifact publishers.
We have also been working on new Zuul features[0] to support enqueue
and
abort from the REST api[1], Unfortunately the change needed in Zuul
doesn't get much attention. Do you think we could enable them in
openstack-infra zuul? Or are we going to be relying on infra-core
to stop and re-enqueue RDO promotion jobs when needed?
My first reaction is no, we don't want random users doing enqueue / abort on
zuul.o.o. I can see places where this would be exploited. Give how little we
actually enqueue changes, I still see this being an infra-root only task.
However, I think there is some confusion here, we wouldn't move promotion jobs
upstream, they are specific to tripleo and require OVB (single cloud). We moved
them downstream to give tripleo folks more freedom to manage them, so pushing
them upstream now, isn't the first goal. distgit packages seem to be generic
enough to run on any cloud and do not have these special requirements.
There are lots of pros to doing such migration, but the blockers are
very important. Is there an etherpad we could list and comment on the
pros/cons of this migration?
https://etherpad.openstack.org/p/rdo-distgit-upstream
I'll start collecting thoughts there.
Best regards,
-Tristan
[0] https://review.openstack.org/#/c/562321/
[0] https://tree.taiga.io/project/morucci-software-factory/us/960
2120
days inactive
2126
days old
4 comments
3 participants
participants (3)
-
Alan Pevec -
Paul Belanger -
Tristan Cacqueray