<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'><br><br><div>DVR setup on RDO Kilo (Controller/Network)+Compute+compute (ML2&OVS&VXLAN<br>Per <a href="http://www.linux.com/community/blogs/133-general-linux/850749-rdo-juno-dvr-deployment-controllernetworkcomputecompute-ml2aovsavxlan-on-centos-71" target="_blank">http://www.linux.com/community/blogs/133-general-linux/850749-rdo-juno-dvr-deployment-controllernetworkcomputecompute-ml2aovsavxlan-on-centos-71</a><br><br>However, to have VXLAN tunnels alive I was forced to set on each node <br>l2population,enable_distributed_routing,arp_responder to False.<br><br>[root@ip-192-169-142-147 openvswitch]# cat ovs_neutron_plugin.ini | grep -v ^#| grep -v ^$<br>[ovs]<br>enable_tunneling = True<br>integration_bridge = br-int<br>tunnel_bridge = br-tun<br>local_ip =10.0.0.147<br>bridge_mappings =physnet1:br-ex<br>[agent]<br>polling_interval = 2<br>tunnel_types =vxlan<br>vxlan_udp_port =4789<br>l2population = False<br>enable_distributed_routing = False<br>arp_responder = False<br>[securitygroup]<br>firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver<br><br>So, I was expecting all traffic to go via br-ex on Controller 192.169.142.127.<br>However, when I created as admin distributed router belongs to demo.<br>As demo tenant attached public and private networks to router.<br>Created VM (F22) and assigned floating IP no VM.<br><br>On Compute Node 192.169.142.147 appeared two namespaces :-<br><br>[root@ip-192-169-142-147 ~]# ip netns<br>fip-a33f05a2-f6ca-4dd9-ba84-1b9e2f3f256d<br>qrouter-1ca178ff-2ee1-4f40-a65a-fbc590dc18a4<br><br>Then I logged into Fedora VM via floating IP and started download of<br>CentOS-7-x86_64-Everything-1503-01.iso (7.5 GB) image via wget.<br><br>[root@vf22devs01 ~]# wget http://centos-mirror.rbc.ru/pub/centos/7/isos/x86_64/CentOS-7-x86_64-Everything-1503-01.iso<br>--2015-09-29 11:22:28-- http://centos-mirror.rbc.ru/pub/centos/7/isos/x86_64/CentOS-7-x86_64-Everything-1503-01.iso<br>Resolving centos-mirror.rbc.ru (centos-mirror.rbc.ru)... 80.68.250.218<br>Connecting to centos-mirror.rbc.ru (centos-mirror.rbc.ru)|80.68.250.218|:80... connected.<br>HTTP request sent, awaiting response... 200 OK<br>Length: 7591690240 (7.1G) [application/octet-stream]<br>Saving to: ‘CentOS-7-x86_64-Everything-1503-01.iso’<br><br>CentOS-7-x86_64-Everything-1 100%[==============================================>] 7.07G 1.71MB/s in 71m 49ss<br><br>2015-09-29 12:34:16 (1.68 MB/s) - ‘CentOS-7-x86_64-Everything-1503-01.iso’ saved [7591690240/7591690240]<br><br> <br>Protocol bellow clearly shows that traffic was coming via br-ex on Compute Node<br>Looks like if Compute nodes are running required services, ml2_conf.ini and metadata_agent.ini <br>are replicated as required and admin creates:-<br><br># source keystonerc_admin<br># neutron router-create RouterDemo --distributed True --tenant-id demo-tenants-id<br><br>DVR will work with l2population, arp_responder, enable_distributed_routing set to False<br>across all landscape.<br><br><br>***********************************<br>Unchanging picture on Controller<br>***********************************<br>[root@ip-192-169-142-127 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.127 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::44e1:5aff:fe40:8244 prefixlen 64 scopeid 0x20<link><br> ether 46:e1:5a:40:82:44 txqueuelen 0 (Ethernet)<br> RX packets 2599455 bytes 1150042934 (1.0 GiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 2564510 bytes 1643558459 (1.5 GiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:feeb:df21 prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:eb:df:21 txqueuelen 1000 (Ethernet)<br> RX packets 2639669 bytes 1154683451 (1.0 GiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 2598079 bytes 1647031397 (1.5 GiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>******************************************************************<br>Download running via br-ex on Compute Node (192.169.142.147):-<br>******************************************************************<br><br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 384115 bytes 257195666 (245.2 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 765238 bytes 338726146 (323.0 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 3264206 bytes 4290640925 (3.9 GiB) <====<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 1819329 bytes 410834141 (391.8 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 385282 bytes 257337071 (245.4 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 766473 bytes 338883813 (323.1 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 3435000 bytes 4529492380 (4.2 GiB) <====<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 1876798 bytes 414801100 (395.5 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 387008 bytes 257539547 (245.6 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 768296 bytes 339107545 (323.3 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 3691373 bytes 4888357764 (4.5 GiB) <======<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 1961572 bytes 420645258 (401.1 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 387525 bytes 257602938 (245.6 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 768848 bytes 339178423 (323.4 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 3767710 bytes 4995080257 (4.6 GiB) <=======<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 1987149 bytes 422411726 (402.8 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 389681 bytes 257850620 (245.9 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 771110 bytes 339453335 (323.7 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 4089304 bytes 5445503015 (5.0 GiB) <========<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 2093163 bytes 429718230 (409.8 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 393357 bytes 258277943 (246.3 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 775035 bytes 340005436 (324.2 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 4588907 bytes 6145182030 (5.7 GiB) <=====<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 2256743 bytes 441092293 (420.6 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 395350 bytes 258511452 (246.5 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 777148 bytes 340266919 (324.5 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 4888982 bytes 6565013893 (6.1 GiB) <=======<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 2353216 bytes 447751576 (427.0 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 397729 bytes 258782156 (246.7 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 779703 bytes 340646185 (324.8 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 5199268 bytes 6998765919 (6.5 GiB) <=======<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 2454158 bytes 454799344 (433.7 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br>[root@ip-192-169-142-147 ~]# ifconfig | head -16<br>br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255<br> inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link><br> ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)<br> RX packets 402423 bytes 259332767 (247.3 MiB)<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 784670 bytes 341253943 (325.4 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br> inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link><br> ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)<br> RX packets 5900362 bytes 7979888974 (7.4 GiB) <========<br> RX errors 0 dropped 0 overruns 0 frame 0<br> TX packets 2684043 bytes 470658266 (448.8 MiB)<br> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0<br><br><br>Thanks<br>Boris<br><br></div> </div></body>
</html>