On Tue, Jun 13, 2017 at 4:39 PM, Numan Siddique <nusiddiq@redhat.com> wrote:

On Tue, Jun 13, 2017 at 4:18 PM, Alan Pevec <apevec@redhat.com> wrote:

Hi Numan,

thanks for testing! Few questions inline:

>>>> > I tested by deploying upstream tripleo on my local setup by making use of these rpms and it works fine for me.
>>>> Did you also try upgrade tests? That was the pain-point with the previous OVS updates.
>>> No. I didn't try that.
> I did some testing locally and these are my findings
>  - After deploying a overcloud (with 3 controllers and 1 compute ) using
> tripleo-quickstart, I added the OVS 2.7 repo from here [1] manually on all

Was [1] http://cbs.centos.org/repos/cloud7-openstack-pike-candidate/x86_64/os/ ?

No. It was http://cbs.centos.org/repos/cloud7-openstack-common-candidate/x86_64/os/

 

Can you confirm which openvswitch NVR you got?

I am not sure which one. I will get back to you on this.

> the nodes and triggered "openstack overcloud update stack".
> - For some reason update was always failing. Not sure what is going wrong
> - So I manually ran "yum update" on the nodes
>  - OVS 2.7 was updated successfully, but the service was  not restarted.

Numan, please report update stack failure as tripleo upstream LP bug.
Adding Sofer from the upgrades team: is there upstream tripleo
update/upgrade CI job which is testing with OVS 2.7 ?

I want to test once more before I submit a bug to be really sure if I haven't done any mistake.

Hi Alan, I ran the stack update again, but now with 1 controller and 1 compute node and the stack update was successful.

I have captured the terminal logs which you can find here - https://paste.fedoraproject.org/paste/FELBFKHNWiIwgNsd3o22~A

Before starting the update, I created a VM with ip 10.0.0.9. I started a ping command in the controller node from the neutron dhcp namespace. The ping was running successfully without any issues.

As expected, the OVS packages were updated to 2.7 and openvswitch service was not restarted. So essentially OVS 2.6 was running even after the update.

Then I manually restarted openvswitch service first on the compute node and then on controller node. The ping was only interrupted during the service restarts and was back again.

Below is the repo file I added to get the OVS 2.7

******************************************************

 cat /etc/yum.repos.d/cbs.centos.org_repos_cloud7-openstack-pike-candidate_x86_64_os.repo 

[cbs.centos.org_repos_cloud7-openstack-pike-candidate_x86_64_os]

name=added from: http://cbs.centos.org/repos/cloud7-openstack-pike-candidate/x86_64/os

baseurl=http://cbs.centos.org/repos/cloud7-openstack-pike-candidate/x86_64/os

enabled=1

gpgcheck=0

includepkgs=openvswitch,openvswitch-ovn*

*********************************************************

 

Thanks

Numan

I don't think there is any CI job yet. I tried to be bold and submitted a patch to test it here (please note it's only for testing) - https://review.openstack.org/#/c/473191/

In the puppet/services/openvswitch_upgrade.yaml, I tried to install the OVS 2.7 and restart it.

From the logs I could see that ovs was never updated (not sure why) and the ansible task to restart the openvswitch (still with version 2.6) makes the node loose the network completely. I will explore more on this job and see how it goes.

But from what I understand upgrading OVS (presently) has a down time overhead. i.e We can update the ovs package. But restarting OVS with the updated version would cause some down time or reboot. I may be wrong here. But that's what I understand. If this is the case, not sure if we can test this scenario in CI other than just updating the package without restart.

>  - After the update everything was working fine.

Working fine without ovs service restart?

That's right.

 

>  - Then I restarted all the nodes to have OVS 2.7 started up.
>  - After restart I was able to have my tenant networks working fine.

So that sounds promising but I'd like to get details on update stack
failure before we proceed promotion OVS 2.7 packages to the RDO Pike
repo.
Also note that 2.7.0-1 RPM from v2.7.0 tag is missing > 100 backports
on branch-2.7, upstream should really consider releasing 2.7.1.

I have requested for a new tag in the OVS mailing list. Hope it would be done soon.

Thanks

Numan

 

Cheers,
Alan

> Thanks
> Numan

>>>> > Is it possible to cross-tag for RDO pike ?
>>>> I've tagged openvswitch-2.7.0-1.el7 for pike-candidate for now so we
>>>> can ran full RDO CI on it.
>>>> Alfredo, David, Emilien, can you please run pipeline jobs with
>>>> http://cbs.centos.org/repos/cloud7-openstack-pike-candidate/x86_64/os/
>>>> repo added ?