On 05/25/2015 02:55 PM, Hugh O. Brock wrote: And probably for clients. I've heard from upstream folks that stable/kilo on clients is not designed for distributions, only for CI. So we should probably consider moving forward (e.g. with ironicclient).

On 05/25/2015 11:46 PM, Alan Pevec wrote: Nothing recorded, just got it from private conversations e.g. with Devananda.

My 2 cents as an operator: I upgraded a production cloud to Juno. Juno had an unexpected regression... https://bugs.launchpad.net/neutron/+bug/1422476 Bug filed on: 2015-02-16 I proposed a working fix tested and working on our production cloud: Feb 18, 2015 2:01 PM The patch hit upstream Juno: 2015-05-07 12:48:47 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Upstream cares about the long term care and feeding of the code base more then the stability of it. If you follow the review, no one really commented on the fix, only nitpicked the unit tests. I'm all for the long term unit testing code being rock solid, but the fix should have gone in right away, and then care been spent on making nice unit tests. The project got what it needed, but the ops guys suffered in the mean time. If you don't provide a way to apply fixes before they get upstreamed, your going to get some very unhappy operators I think since upstream doesn't care. :/ Thanks, Kevin ________________________________________ From: rdo-list-bounces(a)redhat.com [rdo-list-bounces(a)redhat.com] on behalf of James Slagle [jslagle(a)redhat.com] Sent: Tuesday, May 26, 2015 12:06 PM To: Hugh O. Brock Cc: rdo-list(a)redhat.com Subject: Re: [Rdo-list] reconsidering midstream repos On Mon, May 25, 2015 at 02:55:57PM +0200, Hugh O. Brock wrote: No flying bricks from me anyway :) I agree with what you're saying, but would like to agree on what we're actually meaning. I think it's great that pretty much exactly what we said would happen has happened: as Kilo nears completion, the patches that we're having to carry for RDO-Manager is approaching 0. But before we say, let's kill the midstream repos...let's consider why we even have them to begin with. We didn't come up with the idea to fork a bunch of projects because it would make our *development* lives for RDO-Manager easier. In fact, it's exactly the opposite, they cause pain. So why do we have them? Quite simply it's because of the demand that we show, commit to, and support features in RDO-Manager prior to them necessarily merging in upstream OpenStack projects. If we can lift that requirement, we could likely do away with the midstream repos entirely. I know this is rdo-list, but in the spirit of being open, if we don't have midstream repos for RDO-Manager, then we shouldn't have them for OSP either, and we should only be committing to features as they land in upstream OpenStack. It doesn't make sense to me to drop the midstream repos for RDO-Manager, but then turn around and set them up privately for OSP. At least with them setup for RDO, we're doing the development of the management product in the open. In fact if we were to split like this, I think the pain would only intensify. It's pragmatic to say that anyone looking to productize OpenStack is going to have their own bug fixes/etc that they need, and maybe even some features. But setting up private forks as a general development principal to drive OpenStack based product development should be avoided at all costs. So yes, I'm +1 on dropping the midstream repos if they're no longer needed. But I'm -1 on then turning around and setting them up privately so that we can commit code to private forks because that code hasn't landed in upstream OpenStack yet. If we can agree that this isn't needed, then I think we can do without the midstream. Otherwise, if we're going to insist on forks to "land" features prior to them merging upstream, let's at least keep them public. There's a couple other facets to the discussion as well: The midstream repos is a place where we can preview where the management product is headed. For example, the Heat breakpoint work was available from the midstream repos much earlier than it ended up merging into Heat/heatclient. Some devs/users might find this sort of thing really useful, and could provide some early feedback for RDO-Manager. The midstream repos are also a place where we can quickly land reverts to unblock work. The Neutron regression that broke provisioning via Ironic this past cycle immediately comes to mind...it took a couple of weeks before the upstream revert even landed in Neutron. While that was broken, we carried the proposed revert in the midstream repos so that people could install a Neutron that actually worked for RDO-Manager. -- -- James Slagle -- _______________________________________________ Rdo-list mailing list Rdo-list(a)redhat.com https://www.redhat.com/mailman/listinfo/rdo-list To unsubscribe: rdo-list-unsubscribe(a)redhat.com

On Thu, May 28, 2015 at 10:34:29AM +0200, Dmitry Tantsur wrote: I very much agree on *not* making things worse by doing a bunch of secret-squirrel stuff in private repos. And you are right that the big-picture purpose of having the midstream repos is in fact to demonstrate (and build on) features we believe will land upstream, before they have actually landed. The "slower" the upstream project is, the more we need to consider taking the pain of having a midstream repo for that project. What is giving me pause is that it seems like we have midstream repos for projects that are not "slow" at all. The tripleo heat templates are a great example of this; there is little or no delay for our commits on this project, and in fact we benefit by pushing there first because of the tripleo-CI coverage. So why have a midstream repo for it, why not work entirely upstream? The same thing goes for the os-* tools, and a number of other repos I believe. I think we probably do benefit from maintaining midstreams of Heat and Ironic (I *think*), but I question the rest of them. Let me know please if I'm way off-base here. Agree, this makes sense. So is it correct to say that the more unwieldy an upstream project is, the more likely it is we need a mid-stream repo for it? Thanks, --Hugh -- == Hugh Brock, hbrock(a)redhat.com == == Senior Engineering Manager, Cloud Engineering == == RDO Manager: Install, configure, and scale OpenStack == == http://rdoproject.org == "I know that you believe you understand what you think I said, but I’m not sure you realize that what you heard is not what I meant." --Robert McCloskey

On 05/25/2015 02:55 PM, Hugh O. Brock wrote: I have to bring one more thing here: the way we handle patches is error-prone and opaque. `git push -f` is rarely a good way to solve problems. It's easy to forget a patch on rebase, it requires explicit coordination between people doing rebases, it does not (by default) provide history of changes to patch series, it's hard to revert to previous rebase if you forget to tag it. Delorean can also be confused with multi-patch rebase. Can we consider some specialized solution for handling patch stacks? I've heard Rackspace is using Ply [1], looks nice at first glance. It's idea is to store patch series in a separate repo - which looks like what we want. So we can use our fork only for mgt-* branch and have all patches tracked outside. Then we (maybe) could even use gerrit for patches (which I don't like but some people dream of)! [1] https://github.com/rconradharris/ply