[Rdo-list] Jumbo MTU to instances in Kilo?

Erich Weiler weiler at soe.ucsc.edu
Wed Oct 7 00:34:28 UTC 2015 

Hi Y'all,

I know someone must have figured this one out, but I can't seem to get 
9000 byte MTUs working.  I have it set in plugin.ini, etc, my nodes have 
MTU=9000 on their interfaces, so does the network node.  dnsmasq also is 
configured to set MTU=9000 on instances, which works.  But I still can't 
ping with large packets to my instance:

[weiler at stacker ~]$ ping 10.50.100.2
PING 10.50.100.2 (10.50.100.2) 56(84) bytes of data.
64 bytes from 10.50.100.2: icmp_seq=1 ttl=63 time=2.95 ms
64 bytes from 10.50.100.2: icmp_seq=2 ttl=63 time=1.14 ms
64 bytes from 10.50.100.2: icmp_seq=3 ttl=63 time=0.661 ms

That works fine.  This however doesn't work:

[root at stacker ~]# ping -M do -s 8000 10.50.100.2
PING 10.50.100.2 (10.50.100.2) 8000(8028) bytes of data.
 From 10.50.100.2 icmp_seq=1 Frag needed and DF set (mtu = 1500)
ping: local error: Message too long, mtu=1500
ping: local error: Message too long, mtu=1500
ping: local error: Message too long, mtu=1500
ping: local error: Message too long, mtu=1500

It looks like somehow the br-int interface for OVS isn't set at 9000, 
but I can't figure out how to do that...

Here's ifconfig on my compute node:

br-enp3s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
         inet6 fe80::ec4:7aff:fe58:423e  prefixlen 64  scopeid 0x20<link>
         ether 0c:c4:7a:58:42:3e  txqueuelen 0  (Ethernet)
         RX packets 2401432  bytes 359276713 (342.6 MiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 30  bytes 1572 (1.5 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-int: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet6 fe80::64dc:94ff:fe35:db4c  prefixlen 64  scopeid 0x20<link>
         ether 66:dc:94:35:db:4c  txqueuelen 0  (Ethernet)
         RX packets 69  bytes 6866 (6.7 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 8  bytes 648 (648.0 B)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp3s0f0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 9000
         inet6 fe80::ec4:7aff:fe58:423e  prefixlen 64  scopeid 0x20<link>
         ether 0c:c4:7a:58:42:3e  txqueuelen 1000  (Ethernet)
         RX packets 130174458  bytes 15334807929 (14.2 GiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 22919305  bytes 5859090420 (5.4 GiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp3s0f0.50: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
         inet 10.50.1.236  netmask 255.255.0.0  broadcast 10.50.255.255
         inet6 fe80::ec4:7aff:fe58:423e  prefixlen 64  scopeid 0x20<link>
         ether 0c:c4:7a:58:42:3e  txqueuelen 0  (Ethernet)
         RX packets 38429352  bytes 5152853436 (4.7 GiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 419842  bytes 101161981 (96.4 MiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
         inet 127.0.0.1  netmask 255.0.0.0
         inet6 ::1  prefixlen 128  scopeid 0x10<host>
         loop  txqueuelen 0  (Local Loopback)
         RX packets 22141566  bytes 1185622090 (1.1 GiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 22141566  bytes 1185622090 (1.1 GiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qbr247da3ed-a4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet6 fe80::5c8f:c0ff:fe79:bc11  prefixlen 64  scopeid 0x20<link>
         ether b6:1f:54:3f:3d:48  txqueuelen 0  (Ethernet)
         RX packets 16  bytes 1472 (1.4 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 8  bytes 648 (648.0 B)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qbrf42ea01f-fe: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet6 fe80::f484:f1ff:fe53:fb2e  prefixlen 64  scopeid 0x20<link>
         ether c2:a6:d8:25:63:ea  txqueuelen 0  (Ethernet)
         RX packets 15  bytes 1456 (1.4 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 8  bytes 648 (648.0 B)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qvb247da3ed-a4: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
         inet6 fe80::b41f:54ff:fe3f:3d48  prefixlen 64  scopeid 0x20<link>
         ether b6:1f:54:3f:3d:48  txqueuelen 1000  (Ethernet)
         RX packets 247  bytes 28323 (27.6 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 233  bytes 25355 (24.7 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qvbf42ea01f-fe: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
         inet6 fe80::c0a6:d8ff:fe25:63ea  prefixlen 64  scopeid 0x20<link>
         ether c2:a6:d8:25:63:ea  txqueuelen 1000  (Ethernet)
         RX packets 377  bytes 57664 (56.3 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 333  bytes 38765 (37.8 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qvo247da3ed-a4: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
         inet6 fe80::dcfa:f1ff:fe03:ee88  prefixlen 64  scopeid 0x20<link>
         ether de:fa:f1:03:ee:88  txqueuelen 1000  (Ethernet)
         RX packets 233  bytes 25355 (24.7 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 247  bytes 28323 (27.6 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qvof42ea01f-fe: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
         inet6 fe80::f03e:35ff:fefe:e52  prefixlen 64  scopeid 0x20<link>
         ether f2:3e:35:fe:0e:52  txqueuelen 1000  (Ethernet)
         RX packets 333  bytes 38765 (37.8 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 377  bytes 57664 (56.3 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap247da3ed-a4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet6 fe80::fc16:3eff:fede:5eea  prefixlen 64  scopeid 0x20<link>
         ether fe:16:3e:de:5e:ea  txqueuelen 500  (Ethernet)
         RX packets 219  bytes 24239 (23.6 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 224  bytes 26661 (26.0 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
         inet 192.168.122.1  netmask 255.255.255.0  broadcast 
192.168.122.255
         ether 52:54:00:c4:75:9f  txqueuelen 0  (Ethernet)
         RX packets 0  bytes 0 (0.0 B)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 0  bytes 0 (0.0 B)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

This is on RHEL 7.1.  Any obvious way I can get all the intermediate 
bridges to MTU=9000?  I've RTFM'd and googled to no avail...

Here's the ovs-vsctl outout:

[root at node-136 ~]# ovs-vsctl show
6f5a5f00-59e2-4420-aeaf-7ad464ead232
     Bridge br-int
         fail_mode: secure
         Port br-int
             Interface br-int
                 type: internal
         Port "qvo247da3ed-a4"
             tag: 1
             Interface "qvo247da3ed-a4"
         Port "int-br-eth1"
             Interface "int-br-eth1"
         Port "int-br-enp3s0f0"
             Interface "int-br-enp3s0f0"
                 type: patch
                 options: {peer="phy-br-enp3s0f0"}
     Bridge "br-enp3s0f0"
         Port "enp3s0f0"
             Interface "enp3s0f0"
         Port "br-enp3s0f0"
             Interface "br-enp3s0f0"
                 type: internal
         Port "phy-br-enp3s0f0"
             Interface "phy-br-enp3s0f0"
                 type: patch
                 options: {peer="int-br-enp3s0f0"}
     ovs_version: "2.3.1"

Many thanks if anyone has any information on this topic!  Or can point 
me to some documentation I missed...

Thanks,
erich

More information about the dev mailing list