[Rdo-list] port qg-xxxxxx of external gateway goes to br-int instead of br-eth2
Dan Sneddon
dsneddon at redhat.com
Thu Aug 6 18:30:02 UTC 2015
On 08/06/2015 11:21 AM, Dan Sneddon wrote:
> On 08/06/2015 03:34 AM, ICHIBA Sara wrote:
>> Hello guys,
>>
>> I'm trying to add a second external network to my openstack. I followed
>> the documentation, created a new router and set the new gateway. I
>> can't ping my external gateway from my router namespace and I can ping
>> this gateway from my server.
>>
>> I kept comparing the configuration of my new external gateway (which
>> doesn't work in neutron) and my oldest one (which keeps working) and
>> noticed that when I set a gateway to my router this creates a port with
>> a name that bigins with qg. Normally this port has to be assigned to
>> the external bridge (in my case it should be the new external bridge
>> br-eth2). BUT what I saw is that this port qg-xxx is mapped instead to
>> the br-int.
>>
>> For more explanation please see my confs and some command outputs:
>>
>>
>> [root at OScontroller templates(keystone_admin)]# neutron net-external-list
>> +--------------------------------------+-------------------+------------------------------------------------------+
>> | id | name |
>> subnets |
>> +--------------------------------------+-------------------+------------------------------------------------------+
>> | 59c6f54f-26e3-4360-8a05-1a63285c846c | public |
>> 6744a5b3-1a33-42ae-8ae7-1467ebdc1b13 192.168.5.0/24
>> <http://192.168.5.0/24> |
>> | 7ee9f199-2c22-42a9-a6af-1b06c2d62a35 | ruckus_management |
>> cba357ee-f9a7-49f3-89e4-9f5d864747cd 192.168.31.0/24
>> <http://192.168.31.0/24> |
>> +--------------------------------------+-------------------+------------------------------------------------------+
>>
>>
>>
>>
>>
>>
>>
>> ****this router with its gateway are working***
>>
>> [root at OScontroller templates(keystone_admin)]# ip netns exec
>> qrouter-e8243f85-4c56-47bd-a1ee-40724a861dc6 ip a
>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
>> valid_lft forever preferred_lft forever
>> inet6 ::1/128 scope host
>> valid_lft forever preferred_lft forever
>> 25: qr-2b96e8b6-38: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>> noqueue state UNKNOWN
>> link/ether fa:16:3e:b5:54:4d brd ff:ff:ff:ff:ff:ff
>> inet 10.0.0.1/24 <http://10.0.0.1/24> brd 10.0.0.255 scope global
>> qr-2b96e8b6-38
>> valid_lft forever preferred_lft forever
>> inet6 fe80::f816:3eff:feb5:544d/64 scope link
>> valid_lft forever preferred_lft forever
>> 26: qr-9e50bd2e-fa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>> noqueue state UNKNOWN
>> link/ether fa:16:3e:fc:06:9b brd ff:ff:ff:ff:ff:ff
>> inet 20.0.0.1/24 <http://20.0.0.1/24> brd 20.0.0.255 scope global
>> qr-9e50bd2e-fa
>> valid_lft forever preferred_lft forever
>> inet6 fe80::f816:3eff:fefc:69b/64 scope link
>> valid_lft forever preferred_lft forever
>> 27: qg-cd45a565-0b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>> noqueue state UNKNOWN
>> link/ether fa:16:3e:20:81:60 brd ff:ff:ff:ff:ff:ff
>> inet 192.168.5.100/24 <http://192.168.5.100/24> brd 192.168.5.255
>> scope global qg-cd45a565-0b
>> valid_lft forever preferred_lft forever
>> inet 192.168.5.101/32 <http://192.168.5.101/32> brd 192.168.5.101
>> scope global qg-cd45a565-0b
>> valid_lft forever preferred_lft forever
>> inet 192.168.5.103/32 <http://192.168.5.103/32> brd 192.168.5.103
>> scope global qg-cd45a565-0b
>> valid_lft forever preferred_lft forever
>> inet6 fe80::f816:3eff:fe20:8160/64 scope link
>> valid_lft forever preferred_lft forever
>>
>>
>>
>>
>>
>>
>> ************* this is the new router and gateway that I configured and
>> they are not working
>> [root at OScontroller templates(keystone_admin)]# ip netns exec
>> qrouter-b3c51abb-4a14-4af2-ae72-b41e7cba4e84 ip a
>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
>> valid_lft forever preferred_lft forever
>> inet6 ::1/128 scope host
>> valid_lft forever preferred_lft forever
>> 38: qg-6406f3e3-72: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>> noqueue state UNKNOWN
>> link/ether fa:16:3e:63:48:64 brd ff:ff:ff:ff:ff:ff
>> inet 192.168.31.70/24 <http://192.168.31.70/24> brd 192.168.31.255
>> scope global qg-6406f3e3-72
>> valid_lft forever preferred_lft forever
>> inet6 fe80::f816:3eff:fe63:4864/64 scope link
>> valid_lft forever preferred_lft forever
>>
>>
>>
>>
>> [root at OScontroller templates(keystone_admin)]# ovs-vsctl show
>> 9b3e4cc7-6761-483e-a538-1a132734d1a5
>> Bridge br-tun
>> Port br-tun
>> Interface br-tun
>> type: internal
>> Port "vxlan-c0a80523"
>> Interface "vxlan-c0a80523"
>> type: vxlan
>> options: {df_default="true", in_key=flow,
>> local_ip="192.168.5.34", out_key=flow, remote_ip="192.168.5.35"}
>> Port patch-int
>> Interface patch-int
>> type: patch
>> options: {peer=patch-tun}
>> Bridge "br-eth2"
>> Port "phy-br-eth2"
>> Interface "phy-br-eth2"
>> type: patch
>> options: {peer="int-br-eth2"}
>> Port "eth2"
>> Interface "eth2"
>> Port "br-eth2"
>> Interface "br-eth2"
>> type: internal
>> Bridge br-int
>> fail_mode: secure
>> Port "qr-9e50bd2e-fa"
>> tag: 1
>> Interface "qr-9e50bd2e-fa"
>> type: internal
>> Port "tap61f76964-85"
>> tag: 1
>> Interface "tap61f76964-85"
>> type: internal
>> Port "qvo0952a802-99"
>> tag: 2
>> Interface "qvo0952a802-99"
>> Port "qvo5cc221dc-e8"
>> tag: 1
>> Interface "qvo5cc221dc-e8"
>> Port "tap599b2f77-21"
>> tag: 4095
>> Interface "tap599b2f77-21"
>> type: internal
>> Port "tapa383b7ae-99"
>> tag: 2
>> Interface "tapa383b7ae-99"
>> type: internal
>> Port "tapbea1a763-0b"
>> tag: 2
>> Interface "tapbea1a763-0b"
>> type: internal
>> Port "qr-2b96e8b6-38"
>> tag: 2
>> Interface "qr-2b96e8b6-38"
>> type: internal
>> Port int-br-ex
>> Interface int-br-ex
>> type: patch
>> options: {peer=phy-br-ex}
>> Port "qg-6406f3e3-72"
>> tag: 9
>> Interface "qg-6406f3e3-72"
>> type: internal
>> Port patch-tun
>> Interface patch-tun
>> type: patch
>> options: {peer=patch-int}
>> Port "int-br-eth2"
>> Interface "int-br-eth2"
>> type: patch
>> options: {peer="phy-br-eth2"}
>> Port br-int
>> Interface br-int
>> type: internal
>> Bridge br-ex
>> Port "qg-cd45a565-0b"
>> Interface "qg-cd45a565-0b"
>> type: internal
>> Port br-ex
>> Interface br-ex
>> type: internal
>> Port "eth0"
>> Interface "eth0"
>> Port phy-br-ex
>> Interface phy-br-ex
>> type: patch
>> options: {peer=int-br-ex}
>> ovs_version: "2.3.1"
>> [root at OScontroller templates(keystone_admin)]#
>>
>>
>>
>> ************************fichiers de conf*********************
>>
>> [root at OScontroller neutron]# cat l3_agent.ini | grep -v ^# | grep -v ^$
>> [DEFAULT]
>> debug = False
>> interface_driver =neutron.agent.linux.
>> interface.OVSInterfaceDriver
>> use_namespaces = True
>> gateway_external_network_id =
>> handle_internal_only_routers = True
>> external_network_bridge =
>> metadata_port = 9697
>> send_arp_for_ha = 3
>> periodic_interval = 40
>> periodic_fuzzy_delay = 5
>> enable_metadata_proxy = True
>> router_delete_namespaces = False
>> agent_mode = legacy
>> allow_automatic_l3agent_failover=False
>>
>>
>>
>> root at OScontroller neutron]# cat plugin.ini | grep -v ^# | grep -v ^$
>> [ml2]
>> type_drivers = vxlan
>> tenant_network_types = vxlan
>> mechanism_drivers =openvswitch
>> [ml2_type_flat]
>> [ml2_type_vlan]
>> [ml2_type_gre]
>> [ml2_type_vxlan]
>> vni_ranges =10:100
>> vxlan_group =224.0.0.1
>> [securitygroup]
>> enable_security_group = True
>>
>>
>> [root at OScontroller openvswitch]# cat ovs_neutron_plugin.ini | grep -v
>> ^# | grep -v ^$
>> [ovs]
>> enable_tunneling = True
>> integration_bridge = br-int
>> tunnel_bridge = br-tun
>> local_ip =192.168.5.34
>> network_vlan_ranges = physnet1,physnet2
>> bridge_mappings =physnet1:br-ex,physnet2:br-eth2
>> [agent]
>> polling_interval = 2
>> tunnel_types =vxlan
>> vxlan_udp_port =4789
>> l2_population = False
>> arp_responder = False
>> enable_distributed_routing = False
>> [securitygroup]
>> firewall_driver =
>> neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
>>
>>
>> [root at OScontroller network-scripts]# cat ifcfg-eth0
>> NAME=eth0
>> UUID=1b3a9854-df55-43cf-a9b8-21c1a9cc1e5b
>> DEVICE=eth0
>> HWADDR=44:1e:a1:75:ea:d6
>> DEVICETYPE=ovs
>> OVS_BRIDGE=br-ex
>> ONBOOT=yes
>> TYPE=OVSPort
>> NM_CONTROLLED=yes
>>
>>
>> [root at OScontroller network-scripts]# cat ifcfg-eth2
>> NAME=eth2
>> DEVICE=eth2
>> HWADDR=44:1e:a1:75:ea:d2
>> DEVICETYPE=ovs
>> OVS_BRIDGE=br-eth2
>> ONBOOT=yes
>> TYPE=OVSPort
>> NM_CONTROLLED=yes
>>
>>
>> [root at OScontroller network-scripts]# cat ifcfg-br-eth2
>> DEVICE=br-eth2
>> DEVICETYPE=ovs
>> TYPE=OVSBridge
>> BOOTPROTO=static
>> IPADDR=192.168.31.34
>> NETMASK=255.255.255.0
>> ONBOOT=yes
>> GATEWAY=192.168.31.1
>> DNS1=8.8.8.8
>>
>>
>> [root at OScontroller network-scripts]# cat ifcfg-br-ex
>> DEVICE=br-ex
>> DEVICETYPE=ovs
>> TYPE=OVSBridge
>> BOOTPROTO=static
>> IPADDR=192.168.5.34
>> NETMASK=255.255.255.0
>> ONBOOT=yes
>> GATEWAY=192.168.5.1
>> DNS1=8.8.8.8
>> DNS2=192.168.5.1
>>
>>
>> Thank you for your help,
>> Sara
>>
>>
>>
>>
>>
>> _______________________________________________
>> Rdo-list mailing list
>> Rdo-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/rdo-list
>>
>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>
>
> In /etc/neutron/l3_agent.ini, there is a setting which influences the
> use of the external bridge:
>
> external_network_bridge =
>
> If this is set to 'br-ex', then the floating IP network is tied to
> br-ex, and you can have only one floating IP network.
>
> If on the other hand, this setting is blank, then the floating IP
> networks get patched into br-int, and you can have as many as you want.
>
> The bridges that do get used for the floating IP networks should be
> included in the bridge mappings.
>
> What I would do in your case is to rename the br-ex bridge to something
> else (br-eth1?). Then your bridge mappings becomes:
>
> /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini:
> bridge_mappings =physnet1:br-eth1,physnet2:br-eth2
>
> You will have to delete br-ex, add br-eth1 (or whatever is
> appropriate), and then add the interface to the new bridge. Change the
> Neutron plugin files, and restart the Neutron services.
>
Actually, now that I think about it, you might be able to get away with
leaving the bridge named br-ex, as long as the external_network_bridge
is set to ''. This isn't tested, though, and I wouldn't be terribly
surprised if Neutron didn't like that.
We have a combination of command-line parameters and Heat parameters
that will set up the overcloud for multiple floating IP nets at
deployment time. First of all, the external_network_bridge can be set
in the network-environment.yaml file:
parameters:
Controller-1::NeutronExternalNetworkBridge: "''"
parameter_defaults:
NeutronExternalNetworkBridge: "''"
Then, in the controller.yaml NIC template, you will want the external
network on a bridge that is named appropriately (not br-ex).
Finally, when you deploy, use the --neutron-bridge-mappings parameter:
openstack overcloud deploy \
--neutron-bridge-mappings physnet1:br-eth1,physnet2:br-eth2 \
-e
/usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml
\
-e /home/stack/network-isolation.yaml \
[Other params...]
You will still need to create the floating IP networks after
deployment, we no longer do that at deployment time.
--
Dan Sneddon | Principal OpenStack Engineer
dsneddon at redhat.com | redhat.com/openstack
650.254.4025 | dsneddon:irc @dxs:twitter
More information about the dev
mailing list