DVR setup on RDO Kilo (Controller/Network)+Compute+compute (ML2&OVS&VXLAN
Per
http://www.linux.com/community/blogs/133-general-linux/850749-rdo-juno-dvr-deployment-controllernetworkcomputecompute-ml2aovsavxlan-on-centos-71However, to have VXLAN tunnels alive I was forced to set on each node
l2population,enable_distributed_routing,arp_responder to False.
[root@ip-192-169-142-147 openvswitch]# cat ovs_neutron_plugin.ini | grep -v ^#| grep -v ^$
[ovs]
enable_tunneling = True
integration_bridge = br-int
tunnel_bridge = br-tun
local_ip =10.0.0.147
bridge_mappings =physnet1:br-ex
[agent]
polling_interval = 2
tunnel_types =vxlan
vxlan_udp_port =4789
l2population = False
enable_distributed_routing = False
arp_responder = False
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
So, I was expecting all traffic to go via br-ex on Controller 192.169.142.127.
However, when I created as admin distributed router belongs to demo.
As demo tenant attached public and private networks to router.
Created VM (F22) and assigned floating IP no VM.
On Compute Node 192.169.142.147 appeared two namespaces :-
[root@ip-192-169-142-147 ~]# ip netns
fip-a33f05a2-f6ca-4dd9-ba84-1b9e2f3f256d
qrouter-1ca178ff-2ee1-4f40-a65a-fbc590dc18a4
Then I logged into Fedora VM via floating IP and started download of
CentOS-7-x86_64-Everything-1503-01.iso (7.5 GB) image via wget.
[root@vf22devs01 ~]# wget http://centos-mirror.rbc.ru/pub/centos/7/isos/x86_64/CentOS-7-x86_64-Everything-1503-01.iso
--2015-09-29 11:22:28-- http://centos-mirror.rbc.ru/pub/centos/7/isos/x86_64/CentOS-7-x86_64-Everything-1503-01.iso
Resolving centos-mirror.rbc.ru (centos-mirror.rbc.ru)... 80.68.250.218
Connecting to centos-mirror.rbc.ru (centos-mirror.rbc.ru)|80.68.250.218|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7591690240 (7.1G) [application/octet-stream]
Saving to: ‘CentOS-7-x86_64-Everything-1503-01.iso’
CentOS-7-x86_64-Everything-1 100%[==============================================>] 7.07G 1.71MB/s in 71m 49ss
2015-09-29 12:34:16 (1.68 MB/s) - ‘CentOS-7-x86_64-Everything-1503-01.iso’ saved [7591690240/7591690240]
Protocol bellow clearly shows that traffic was coming via br-ex on Compute Node
Looks like if Compute nodes are running required services, ml2_conf.ini and metadata_agent.ini
are replicated as required and admin creates:-
# source keystonerc_admin
# neutron router-create RouterDemo --distributed True --tenant-id demo-tenants-id
DVR will work with l2population, arp_responder, enable_distributed_routing set to False
across all landscape.
***********************************
Unchanging picture on Controller
***********************************
[root@ip-192-169-142-127 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.127 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::44e1:5aff:fe40:8244 prefixlen 64 scopeid 0x20<link>
ether 46:e1:5a:40:82:44 txqueuelen 0 (Ethernet)
RX packets 2599455 bytes 1150042934 (1.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2564510 bytes 1643558459 (1.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:feeb:df21 prefixlen 64 scopeid 0x20<link>
ether 52:54:00:eb:df:21 txqueuelen 1000 (Ethernet)
RX packets 2639669 bytes 1154683451 (1.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2598079 bytes 1647031397 (1.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
******************************************************************
Download running via br-ex on Compute Node (192.169.142.147):-
******************************************************************
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 384115 bytes 257195666 (245.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 765238 bytes 338726146 (323.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 3264206 bytes 4290640925 (3.9 GiB) <====
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1819329 bytes 410834141 (391.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 385282 bytes 257337071 (245.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 766473 bytes 338883813 (323.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 3435000 bytes 4529492380 (4.2 GiB) <====
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1876798 bytes 414801100 (395.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 387008 bytes 257539547 (245.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 768296 bytes 339107545 (323.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 3691373 bytes 4888357764 (4.5 GiB) <======
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1961572 bytes 420645258 (401.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 387525 bytes 257602938 (245.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 768848 bytes 339178423 (323.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 3767710 bytes 4995080257 (4.6 GiB) <=======
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1987149 bytes 422411726 (402.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 389681 bytes 257850620 (245.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 771110 bytes 339453335 (323.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 4089304 bytes 5445503015 (5.0 GiB) <========
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2093163 bytes 429718230 (409.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 393357 bytes 258277943 (246.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 775035 bytes 340005436 (324.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 4588907 bytes 6145182030 (5.7 GiB) <=====
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2256743 bytes 441092293 (420.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 395350 bytes 258511452 (246.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 777148 bytes 340266919 (324.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 4888982 bytes 6565013893 (6.1 GiB) <=======
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2353216 bytes 447751576 (427.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 397729 bytes 258782156 (246.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 779703 bytes 340646185 (324.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 5199268 bytes 6998765919 (6.5 GiB) <=======
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2454158 bytes 454799344 (433.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@ip-192-169-142-147 ~]# ifconfig | head -16
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.142.147 netmask 255.255.255.0 broadcast 192.169.142.255
inet6 fe80::38a0:86ff:fec6:8a4d prefixlen 64 scopeid 0x20<link>
ether 3a:a0:86:c6:8a:4d txqueuelen 0 (Ethernet)
RX packets 402423 bytes 259332767 (247.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 784670 bytes 341253943 (325.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::5054:ff:fe3f:a45d prefixlen 64 scopeid 0x20<link>
ether 52:54:00:3f:a4:5d txqueuelen 1000 (Ethernet)
RX packets 5900362 bytes 7979888974 (7.4 GiB) <========
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2684043 bytes 470658266 (448.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Thanks
Boris