[Rdo-list] R: R: Jumbo MTU to instances in Kilo?

[Dan Sneddon]

On 10/09/2015 12:46 PM, Erich Weiler wrote:
>> Lastly, has anyone ever run into problems when running (MTU - 50 bytes)
>> as the veth_mtu with VXLAN? I see documentation all over recommending
>> (MTU - 100 bytes), but I don't see why VXLAN should take that many
>> extra bytes. I've done extensive testing at VM MTU 8950 over a 9000 MTU
>> link, and never run into an issue. Is this just cargo-culting, or is
>> there a reason to give VXLAN additional headroom in some scenarios?
> 
> Along those same lines...  I'm running tenant segregation via regular
> VLAN segregation (not VXLAN or GRE), using all interfaces set to
> MTU=9000 (the instance all the way up to the main network gateway, all
> router interfaces, etc).  My physical switches have MTU=9260 on all
> ports however.
> 
> It seems to work and perform OK, but is there a recommendation on
> giving regular VLANs a little VLAN-tag headroom as well?  Like, should
> I be setting my instance MTU to 8950 or something?

Relax, Neutron VLAN mode uses the line MTU, no headroom is needed.

"What about the VLAN tag header?" you might ask. Well, every switch
vendor reserves 4 bytes on top of whatever you set the MTU to on an
interface to make room for VLAN tags. So if you set the MTU to 9000 on
an interface, the actual MTU the switch hardware uses is 9004. This
used to be a problem in the very early days of VLANs, where some Cisco
switches wouldn't reserve the 4-bytes beyond a certain upper limit, so
people got used to setting their host MTU 4-bytes less *just in case*
they went through an affected switch. This hasn't been necessary for
any switch that I'm aware of produced since about ca. 2000.

Internally, OVS will strip the VLAN tag, and replace it with an
internal VLAN tag that is used by OVS to separate each tenant's
traffic. Because the outer tag gets replaced, no additional headroom is
required. Under normal circumstances, the VM has no idea it is on a
VLAN, so it also uses the full MTU of the physical interface with no
additional overhead.

Things are a little different if you are using Q-in-Q to pass VLAN tags
down to the host, but you would know if you were doing that.

-- 
Dan Sneddon         |  Principal OpenStack Engineer
dsneddon at redhat.com |  redhat.com/openstack
650.254.4025        |  dsneddon:irc   @dxs:twitter