[Rdo-list] R: R: Jumbo MTU to instances in Kilo?
Dan Sneddon
dsneddon at redhat.com
Fri Oct 9 17:54:40 UTC 2015
I forgot to link to the MTU Selection and Advertisement spec:
http://specs.openstack.org/openstack/neutron-specs/specs/kilo/mtu-selection-and-advertisement.html
On 10/09/2015 10:39 AM, Dan Sneddon wrote:
> Amedeo,
>
> Thanks for pointing this out. Although the KB article now includes this
> setting, some of our other documentation doesn't include this setting.
> I'll make sure it gets added.
>
> I'm curious whether anyone has tested out the new MTU-related options
> that were added to Kilo:
>
> advertise_mtu
> path_mtu
> segment_mtu
> physnet_mtus
>
> I haven't gotten a chance to test and document these new options
> myself. They serve to simplify configuration a bit, but also the new
> physnet_mtus option allows you to set a different MTU per interface:
>
> Example:
> physnet_mtus = physnet1:1550, physnet2:1500
> Or, to set MTU for physnet2 and leave physnet1 as default:
> physnet_mtus = physnet2:1550
>
> Lastly, has anyone ever run into problems when running (MTU - 50 bytes)
> as the veth_mtu with VXLAN? I see documentation all over recommending
> (MTU - 100 bytes), but I don't see why VXLAN should take that many
> extra bytes. I've done extensive testing at VM MTU 8950 over a 9000 MTU
> link, and never run into an issue. Is this just cargo-culting, or is
> there a reason to give VXLAN additional headroom in some scenarios?
>
> --
> Dan Sneddon | Principal OpenStack Engineer
> dsneddon at redhat.com | redhat.com/openstack
> 650.254.4025 | dsneddon:irc @dxs:twitter
>
> On 10/09/2015 05:24 AM, Salvati Amedeo wrote:
>> Erich you are welcome in the club :D
>>
>> One side note: as we have rhosp and not rdo, we asked to rh to document this and they wrote a solution on their kb:
>>
>> https://access.redhat.com/solutions/1417133
>>
>> Regards,
>> Amedeo
>>
>> -----Messaggio originale-----
>> Da: Erich Weiler [mailto:weiler at soe.ucsc.edu]
>> Inviato: giovedì 8 ottobre 2015 18:23
>> A: Salvati Amedeo; Pedro Navarro Perez
>> Cc: rdo-list at redhat.com
>> Oggetto: Re: R: [Rdo-list] Jumbo MTU to instances in Kilo?
>>
>> Thanks Amedeo,
>>
>> The bit about the config item in the l3_agent.ini file is new to me - I couldn't find that in the documentation, or even as a comment in the file as a config option. If it is a config item as you point out, maybe it should have a commented section in l3_agent.ini?
>>
>> Thanks for the insight!
>>
>> cheers,
>> erich
>>
>> On 10/08/2015 03:02 AM, Salvati Amedeo wrote:
>>> Eric,
>>>
>>> also, to set jumbo frames on your env, you have to set mtu from VM to controller:
>>>
>>> # echo "dhcp-option-force=26,8900" > /etc/neutron/dnsmasq-neutron.conf
>>> # openstack-config --set /etc/neutron/dhcp_agent.ini DEFAULT dnsmasq_config_file /etc/neutron/dnsmasq-neutron.conf
>>> # openstack-config --set /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini agent veth_mtu 8900
>>> # openstack-config --set /etc/neutron/l3_agent.ini DEFAULT network_device_mtu 9000
>>> # openstack-config --set /etc/nova/nova.conf DEFAULT network_device_mtu 9000 <--- this on every nova-compute
>>>
>>> take a look at l3_agent.ini file, without network_device_mtu every new
>>> router will use default mtu at 1500
>>>
>>> # ip netns exec qrouter-26f64a08-52ab-4643-b903-9aea6eae047a /bin/bash
>>> # ip a | grep mtu
>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
>>> 69: ha-89546945-ab: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc
>>> noqueue state UNKNOWN
>>> 74: qr-f207f652-da: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc
>>> noqueue state UNKNOWN
>>> 81: qg-ab978cd0-ad: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc
>>> noqueue state UNKNOWN
>>>
>>> HTH
>>> Amedeo
>>>
>>> -----Messaggio originale-----
>>> Da: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com]
>>> Per conto di Erich Weiler
>>> Inviato: giovedì 8 ottobre 2015 01:53
>>> A: Pedro Navarro Perez
>>> Cc: rdo-list at redhat.com
>>> Oggetto: Re: [Rdo-list] Jumbo MTU to instances in Kilo?
>>>
>>> Actually I was wrong, it WAS on the network node. The virtual router interfaces were not set to MTU=9000. On network node:
>>>
>>> [root at os-net-01 ~]# ip netns
>>> qdhcp-c395cff9-af7b-4456-91e3-3c55e6c2c5f5
>>> qrouter-0b52e3a6-135c-4481-b286-7c96229f6555
>>>
>>> i[root at os-net-01 ~]# ip netns exec
>>> qrouter-0b52e3a6-135c-4481-b286-7c96229f6555 ifconfig
>>> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
>>> inet 127.0.0.1 netmask 255.0.0.0
>>> inet6 ::1 prefixlen 128 scopeid 0x10<host>
>>> loop txqueuelen 0 (Local Loopback)
>>> RX packets 0 bytes 0 (0.0 B)
>>> RX errors 0 dropped 0 overruns 0 frame 0
>>> TX packets 0 bytes 0 (0.0 B)
>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>
>>> qg-fa1e2a28-25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>>> inet 10.50.100.1 netmask 255.255.0.0 broadcast 10.50.255.255
>>> inet6 fe80::f816:3eff:fe6a:608b prefixlen 64 scopeid 0x20<link>
>>> ether fa:16:3e:6a:60:8b txqueuelen 0 (Ethernet)
>>> RX packets 34071065 bytes 5046408745 (4.6 GiB)
>>> RX errors 0 dropped 0 overruns 0 frame 0
>>> TX packets 442 bytes 51915 (50.6 KiB)
>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>
>>> qr-51904c89-b8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>>> inet 10.100.0.1 netmask 255.255.0.0 broadcast 10.100.255.255
>>> inet6 fe80::f816:3eff:fe37:eca6 prefixlen 64 scopeid 0x20<link>
>>> ether fa:16:3e:37:ec:a6 txqueuelen 0 (Ethernet)
>>> RX packets 702 bytes 75369 (73.6 KiB)
>>> RX errors 0 dropped 0 overruns 0 frame 0
>>> TX packets 814 bytes 92259 (90.0 KiB)
>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>
>>> I can fix it manually:
>>>
>>> [root at os-net-01 neutron]# ip netns exec
>>> qrouter-0b52e3a6-135c-4481-b286-7c96229f6555 ifconfig qg-fa1e2a28-25
>>> mtu
>>> 9000
>>> [root at os-net-01 neutron]# ip netns exec
>>> qrouter-0b52e3a6-135c-4481-b286-7c96229f6555 ifconfig qr-51904c89-b8
>>> mtu
>>> 9000
>>> [root at os-net-01 neutron]# ip netns exec
>>> qrouter-0b52e3a6-135c-4481-b286-7c96229f6555 ifconfig
>>> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
>>> inet 127.0.0.1 netmask 255.0.0.0
>>> inet6 ::1 prefixlen 128 scopeid 0x10<host>
>>> loop txqueuelen 0 (Local Loopback)
>>> RX packets 0 bytes 0 (0.0 B)
>>> RX errors 0 dropped 0 overruns 0 frame 0
>>> TX packets 0 bytes 0 (0.0 B)
>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>
>>> qg-fa1e2a28-25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>> inet 10.50.100.1 netmask 255.255.0.0 broadcast 10.50.255.255
>>> inet6 fe80::f816:3eff:fe6a:608b prefixlen 64 scopeid 0x20<link>
>>> ether fa:16:3e:6a:60:8b txqueuelen 0 (Ethernet)
>>> RX packets 34086053 bytes 5048637833 (4.7 GiB)
>>> RX errors 0 dropped 0 overruns 0 frame 0
>>> TX packets 442 bytes 51915 (50.6 KiB)
>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>
>>> qr-51904c89-b8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>> inet 10.100.0.1 netmask 255.255.0.0 broadcast 10.100.255.255
>>> inet6 fe80::f816:3eff:fe37:eca6 prefixlen 64 scopeid 0x20<link>
>>> ether fa:16:3e:37:ec:a6 txqueuelen 0 (Ethernet)
>>> RX packets 702 bytes 75369 (73.6 KiB)
>>> RX errors 0 dropped 0 overruns 0 frame 0
>>> TX packets 814 bytes 92259 (90.0 KiB)
>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>
>>> And then I have a jumbo clean path everywhere! All is good then.
>>> But... How to set this in a config file or something so I don't have to do it manually?
>>>
>>> I found this bug report:
>>>
>>> https://bugs.launchpad.net/neutron/+bug/1311097
>>>
>>> Anyone know if that bug is still out there? Or how can I set virtual router interfaces MTU by default when I create the router?
>>>
>>> cheers,
>>> erich
>>>
>>> On 10/07/2015 04:35 PM, Erich Weiler wrote:
>>>> Actually I think I'm closer - on the compute nodes, I set this in
>>>> nova.conf:
>>>>
>>>> network_device_mtu=9000
>>>>
>>>> even though there was a big note above it that said not to use it
>>>> because this option was deprecated. But after setting that option,
>>>> and restarting nova and openvswitch, br-int, my tap device and my qvb
>>>> device all got set to MTU=9000. So I'm closer! But still one item
>>>> is blocking me. I show this tracepath from my controller node direct
>>>> to the VM (which is on a compute node on the local network):
>>>>
>>>> # tracepath 10.50.100.4
>>>> 1?: [LOCALHOST] pmtu 9000
>>>> 1: 10.50.100.4 0.682ms
>>>> 1: 10.50.100.4 0.241ms
>>>> 2: 10.50.100.4 0.297ms pmtu
>>>> 1500
>>>> 2: 10.50.100.4 1.664ms reached
>>>>
>>>> 10.50.100.4 is the VM. It looks like the path is jumbo clean up
>>>> until that third hop. But the thing is, I don't know what the third hop is.
>>>> ;)
>>>>
>>>> On my compute node I still see some stuff with MTU=1500, but I'm not
>>>> sure if one of those is blocking me:
>>>>
>>>> # ifconfig
>>>> br-enp3s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet6 fe80::ec4:7aff:fe58:423e prefixlen 64 scopeid 0x20<link>
>>>> ether 0c:c4:7a:58:42:3e txqueuelen 0 (Ethernet)
>>>> RX packets 2401498 bytes 359284253 (342.6 MiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 30 bytes 1572 (1.5 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> br-int: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet6 fe80::64dc:94ff:fe35:db4c prefixlen 64 scopeid 0x20<link>
>>>> ether 66:dc:94:35:db:4c txqueuelen 0 (Ethernet)
>>>> RX packets 133 bytes 12934 (12.6 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 8 bytes 648 (648.0 B)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> enp3s0f0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 9000
>>>> inet6 fe80::ec4:7aff:fe58:423e prefixlen 64 scopeid 0x20<link>
>>>> ether 0c:c4:7a:58:42:3e txqueuelen 1000 (Ethernet)
>>>> RX packets 165957142 bytes 20333410092 (18.9 GiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 23299881 bytes 5950708819 (5.5 GiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> enp3s0f0.50: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet 10.50.1.236 netmask 255.255.0.0 broadcast 10.50.255.255
>>>> inet6 fe80::ec4:7aff:fe58:423e prefixlen 64 scopeid 0x20<link>
>>>> ether 0c:c4:7a:58:42:3e txqueuelen 0 (Ethernet)
>>>> RX packets 6014767 bytes 813880745 (776.1 MiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 79301 bytes 19052451 (18.1 MiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
>>>> inet 127.0.0.1 netmask 255.0.0.0
>>>> inet6 ::1 prefixlen 128 scopeid 0x10<host>
>>>> loop txqueuelen 0 (Local Loopback)
>>>> RX packets 22462729 bytes 1202484822 (1.1 GiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 22462729 bytes 1202484822 (1.1 GiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> qbr922bd9f5-bb: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet6 fe80::4c1a:55ff:feba:14c3 prefixlen 64 scopeid 0x20<link>
>>>> ether 56:a6:a6:db:83:c4 txqueuelen 0 (Ethernet)
>>>> RX packets 16 bytes 1520 (1.4 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 8 bytes 648 (648.0 B)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> qbrf42ea01f-fe: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>>>> inet6 fe80::f484:f1ff:fe53:fb2e prefixlen 64 scopeid 0x20<link>
>>>> ether c2:a6:d8:25:63:ea txqueuelen 0 (Ethernet)
>>>> RX packets 15 bytes 1456 (1.4 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 8 bytes 648 (648.0 B)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> qvb922bd9f5-bb: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>> mtu
>>>> 9000
>>>> inet6 fe80::54a6:a6ff:fedb:83c4 prefixlen 64 scopeid 0x20<link>
>>>> ether 56:a6:a6:db:83:c4 txqueuelen 1000 (Ethernet)
>>>> RX packets 86 bytes 9610 (9.3 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 133 bytes 12767 (12.4 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> qvbf42ea01f-fe: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>> mtu
>>>> 1500
>>>> inet6 fe80::c0a6:d8ff:fe25:63ea prefixlen 64 scopeid 0x20<link>
>>>> ether c2:a6:d8:25:63:ea txqueuelen 1000 (Ethernet)
>>>> RX packets 377 bytes 57664 (56.3 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 333 bytes 38765 (37.8 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> qvo922bd9f5-bb: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>> mtu
>>>> 9000
>>>> inet6 fe80::b44a:bff:fe72:aaea prefixlen 64 scopeid 0x20<link>
>>>> ether b6:4a:0b:72:aa:ea txqueuelen 1000 (Ethernet)
>>>> RX packets 133 bytes 12767 (12.4 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 86 bytes 9610 (9.3 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> qvof42ea01f-fe: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>> mtu
>>>> 1500
>>>> inet6 fe80::f03e:35ff:fefe:e52 prefixlen 64 scopeid 0x20<link>
>>>> ether f2:3e:35:fe:0e:52 txqueuelen 1000 (Ethernet)
>>>> RX packets 333 bytes 38765 (37.8 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 377 bytes 57664 (56.3 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> tap922bd9f5-bb: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet6 fe80::fc16:3eff:fefa:9945 prefixlen 64 scopeid 0x20<link>
>>>> ether fe:16:3e:fa:99:45 txqueuelen 500 (Ethernet)
>>>> RX packets 118 bytes 11561 (11.2 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 95 bytes 10316 (10.0 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
>>>> inet 192.168.122.1 netmask 255.255.255.0 broadcast
>>>> 192.168.122.255
>>>> ether 52:54:00:c4:75:9f txqueuelen 0 (Ethernet)
>>>> RX packets 0 bytes 0 (0.0 B)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 0 bytes 0 (0.0 B)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> My network node has all interfaces set to MTU=9000. I thought maybe the
>>>> bottleneck might be there but I don't think it is. Here's ifconfig
>>>> from my network node:
>>>>
>>>> # ifconfig
>>>> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
>>>> inet 127.0.0.1 netmask 255.0.0.0
>>>> inet6 ::1 prefixlen 128 scopeid 0x10<host>
>>>> loop txqueuelen 0 (Local Loopback)
>>>> RX packets 2042 bytes 238727 (233.1 KiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 2042 bytes 238727 (233.1 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> p1p2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet6 fe80::207:43ff:fe10:deb8 prefixlen 64 scopeid 0x20<link>
>>>> ether 00:07:43:10:de:b8 txqueuelen 1000 (Ethernet)
>>>> RX packets 2156053308 bytes 325330839639 (302.9 GiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 223004 bytes 24769304 (23.6 MiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>> device interrupt 72
>>>>
>>>> p2p1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet 10.50.1.51 netmask 255.255.0.0 broadcast 10.50.255.255
>>>> inet6 fe80::260:ddff:fe44:2aea prefixlen 64 scopeid 0x20<link>
>>>> ether 00:60:dd:44:2a:ea txqueuelen 1000 (Ethernet)
>>>> RX packets 49352916 bytes 3501547231 (3.2 GiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 18876911 bytes 3768900461 (3.5 GiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> p2p2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>> inet6 fe80::260:ddff:fe44:2aeb prefixlen 64 scopeid 0x20<link>
>>>> ether 00:60:dd:44:2a:eb txqueuelen 1000 (Ethernet)
>>>> RX packets 2491224974 bytes 348058319500 (324.1 GiB)
>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>> TX packets 1597 bytes 204525 (199.7 KiB)
>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>>>>
>>>> Any way I can figure out what the third hop is from my tracepath?
>>>>
>>>> Thanks as always for the sage advice!
>>>>
>>>> -erich
>>>>
>>>> On 10/07/2015 09:57 AM, Erich Weiler wrote:
>>>>> Yeah, I made the changes and then recreated all the networks. For
>>>>> some reason br-int and the individual virtual instance interfaces on
>>>>> the compute node still show 1500 byte frames.
>>>>>
>>>>> Has anyone else configured jumbo frames in a Kilo environment? Or
>>>>> maybe I'm just an outlier... ;)
>>>>>
>>>>> -erich
>>>>>
>>>>> On 10/07/2015 01:46 AM, Pedro Navarro Perez wrote:
>>>>>> Hi Erich,
>>>>>>
>>>>>> did you recreate the neutron networks after the configuration changes?
>>>>>>
>>>>>> Pedro Navarro Pérez
>>>>>> OpenStack product specialist
>>>>>> Red Hat Iberia
>>>>>> Passeig de Gràcia 120,
>>>>>> 08008 Barcelona
>>>>>> Spain
>>>>>> M +34 639 642 379
>>>>>> E pnavarro at redhat.com
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: "Erich Weiler" <weiler at soe.ucsc.edu>
>>>>>> To: rdo-list at redhat.com
>>>>>> Sent: Wednesday, 7 October, 2015 2:34:28 AM
>>>>>> Subject: [Rdo-list] Jumbo MTU to instances in Kilo?
>>>>>>
>>>>>> Hi Y'all,
>>>>>>
>>>>>> I know someone must have figured this one out, but I can't seem to
>>>>>> get
>>>>>> 9000 byte MTUs working. I have it set in plugin.ini, etc, my nodes
>>>>>> have
>>>>>> MTU=9000 on their interfaces, so does the network node. dnsmasq
>>>>>> also is configured to set MTU=9000 on instances, which works. But
>>>>>> I still can't ping with large packets to my instance:
>>>>>>
>>>>>> [weiler at stacker ~]$ ping 10.50.100.2 PING 10.50.100.2 (10.50.100.2)
>>>>>> 56(84) bytes of data.
>>>>>> 64 bytes from 10.50.100.2: icmp_seq=1 ttl=63 time=2.95 ms
>>>>>> 64 bytes from 10.50.100.2: icmp_seq=2 ttl=63 time=1.14 ms
>>>>>> 64 bytes from 10.50.100.2: icmp_seq=3 ttl=63 time=0.661 ms
>>>>>>
>>>>>> That works fine. This however doesn't work:
>>>>>>
>>>>>> [root at stacker ~]# ping -M do -s 8000 10.50.100.2 PING 10.50.100.2
>>>>>> (10.50.100.2) 8000(8028) bytes of data.
>>>>>> From 10.50.100.2 icmp_seq=1 Frag needed and DF set (mtu = 1500)
>>>>>> ping: local error: Message too long, mtu=1500
>>>>>> ping: local error: Message too long, mtu=1500
>>>>>> ping: local error: Message too long, mtu=1500
>>>>>> ping: local error: Message too long, mtu=1500
>>>>>>
>>>>>> It looks like somehow the br-int interface for OVS isn't set at
>>>>>> 9000, but I can't figure out how to do that...
>>>>>>
>>>>>> Here's ifconfig on my compute node:
>>>>>>
>>>>>> br-enp3s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>>>> inet6 fe80::ec4:7aff:fe58:423e prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether 0c:c4:7a:58:42:3e txqueuelen 0 (Ethernet)
>>>>>> RX packets 2401432 bytes 359276713 (342.6 MiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 30 bytes 1572 (1.5 KiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> br-int: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>>>>>> inet6 fe80::64dc:94ff:fe35:db4c prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether 66:dc:94:35:db:4c txqueuelen 0 (Ethernet)
>>>>>> RX packets 69 bytes 6866 (6.7 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 8 bytes 648 (648.0 B)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> enp3s0f0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 9000
>>>>>> inet6 fe80::ec4:7aff:fe58:423e prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether 0c:c4:7a:58:42:3e txqueuelen 1000 (Ethernet)
>>>>>> RX packets 130174458 bytes 15334807929 (14.2 GiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 22919305 bytes 5859090420 (5.4 GiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> enp3s0f0.50: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
>>>>>> inet 10.50.1.236 netmask 255.255.0.0 broadcast 10.50.255.255
>>>>>> inet6 fe80::ec4:7aff:fe58:423e prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether 0c:c4:7a:58:42:3e txqueuelen 0 (Ethernet)
>>>>>> RX packets 38429352 bytes 5152853436 (4.7 GiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 419842 bytes 101161981 (96.4 MiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
>>>>>> inet 127.0.0.1 netmask 255.0.0.0
>>>>>> inet6 ::1 prefixlen 128 scopeid 0x10<host>
>>>>>> loop txqueuelen 0 (Local Loopback)
>>>>>> RX packets 22141566 bytes 1185622090 (1.1 GiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 22141566 bytes 1185622090 (1.1 GiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> qbr247da3ed-a4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>>>>>> inet6 fe80::5c8f:c0ff:fe79:bc11 prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether b6:1f:54:3f:3d:48 txqueuelen 0 (Ethernet)
>>>>>> RX packets 16 bytes 1472 (1.4 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 8 bytes 648 (648.0 B)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> qbrf42ea01f-fe: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>>>>>> inet6 fe80::f484:f1ff:fe53:fb2e prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether c2:a6:d8:25:63:ea txqueuelen 0 (Ethernet)
>>>>>> RX packets 15 bytes 1456 (1.4 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 8 bytes 648 (648.0 B)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> qvb247da3ed-a4: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>>>> mtu 1500
>>>>>> inet6 fe80::b41f:54ff:fe3f:3d48 prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether b6:1f:54:3f:3d:48 txqueuelen 1000 (Ethernet)
>>>>>> RX packets 247 bytes 28323 (27.6 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 233 bytes 25355 (24.7 KiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> qvbf42ea01f-fe: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>>>> mtu 1500
>>>>>> inet6 fe80::c0a6:d8ff:fe25:63ea prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether c2:a6:d8:25:63:ea txqueuelen 1000 (Ethernet)
>>>>>> RX packets 377 bytes 57664 (56.3 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 333 bytes 38765 (37.8 KiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> qvo247da3ed-a4: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>>>> mtu 1500
>>>>>> inet6 fe80::dcfa:f1ff:fe03:ee88 prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether de:fa:f1:03:ee:88 txqueuelen 1000 (Ethernet)
>>>>>> RX packets 233 bytes 25355 (24.7 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 247 bytes 28323 (27.6 KiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> qvof42ea01f-fe: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>
>>>>>> mtu 1500
>>>>>> inet6 fe80::f03e:35ff:fefe:e52 prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether f2:3e:35:fe:0e:52 txqueuelen 1000 (Ethernet)
>>>>>> RX packets 333 bytes 38765 (37.8 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 377 bytes 57664 (56.3 KiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> tap247da3ed-a4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>>>>>> inet6 fe80::fc16:3eff:fede:5eea prefixlen 64 scopeid
>>>>>> 0x20<link>
>>>>>> ether fe:16:3e:de:5e:ea txqueuelen 500 (Ethernet)
>>>>>> RX packets 219 bytes 24239 (23.6 KiB)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 224 bytes 26661 (26.0 KiB)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
>>>>>> inet 192.168.122.1 netmask 255.255.255.0 broadcast
>>>>>> 192.168.122.255
>>>>>> ether 52:54:00:c4:75:9f txqueuelen 0 (Ethernet)
>>>>>> RX packets 0 bytes 0 (0.0 B)
>>>>>> RX errors 0 dropped 0 overruns 0 frame 0
>>>>>> TX packets 0 bytes 0 (0.0 B)
>>>>>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions
>>>>>> 0
>>>>>>
>>>>>> This is on RHEL 7.1. Any obvious way I can get all the
>>>>>> intermediate bridges to MTU=9000? I've RTFM'd and googled to no avail...
>>>>>>
>>>>>> Here's the ovs-vsctl outout:
>>>>>>
>>>>>> [root at node-136 ~]# ovs-vsctl show
>>>>>> 6f5a5f00-59e2-4420-aeaf-7ad464ead232
>>>>>> Bridge br-int
>>>>>> fail_mode: secure
>>>>>> Port br-int
>>>>>> Interface br-int
>>>>>> type: internal
>>>>>> Port "qvo247da3ed-a4"
>>>>>> tag: 1
>>>>>> Interface "qvo247da3ed-a4"
>>>>>> Port "int-br-eth1"
>>>>>> Interface "int-br-eth1"
>>>>>> Port "int-br-enp3s0f0"
>>>>>> Interface "int-br-enp3s0f0"
>>>>>> type: patch
>>>>>> options: {peer="phy-br-enp3s0f0"}
>>>>>> Bridge "br-enp3s0f0"
>>>>>> Port "enp3s0f0"
>>>>>> Interface "enp3s0f0"
>>>>>> Port "br-enp3s0f0"
>>>>>> Interface "br-enp3s0f0"
>>>>>> type: internal
>>>>>> Port "phy-br-enp3s0f0"
>>>>>> Interface "phy-br-enp3s0f0"
>>>>>> type: patch
>>>>>> options: {peer="int-br-enp3s0f0"}
>>>>>> ovs_version: "2.3.1"
>>>>>>
>>>>>> Many thanks if anyone has any information on this topic! Or can
>>>>>> point me to some documentation I missed...
>>>>>>
>>>>>> Thanks,
>>>>>> erich
>>>>>>
>>>>>> _______________________________________________
>>>>>> Rdo-list mailing list
>>>>>> Rdo-list at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>>>>
>>>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>>>>>
>>>
>>> _______________________________________________
>>> Rdo-list mailing list
>>> Rdo-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>
>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>>
>>
>> _______________________________________________
>> Rdo-list mailing list
>> Rdo-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/rdo-list
>>
>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>
>
--
Dan Sneddon | Principal OpenStack Engineer
dsneddon at redhat.com | redhat.com/openstack
650.254.4025 | dsneddon:irc @dxs:twitter
More information about the dev
mailing list