Re: [Rdo-list] RDO + floating IPs

On 05/27/2015 12:28 PM, Rhys Oxenham wrote: > > > On 27 May 2015, at 11:13, Tomas Sedovic <tsedovic(a)redhat.com (mailto:tsedovic@redhat.com)> wrote: > > > > On 05/26/2015 05:16 PM, Kashyap Chamarthy wrote: > > > On Tue, May 26, 2015 at 03:54:36PM +0200, Tomas Sedovic wrote: > > > > Hey everyone, > > > > > > > > I tried to get RDO set up with floating IP addresses, but I'm running into > > > > problems I'm not sure how to debug (not that familiar with networking and > > > > Neutron). > > > > > > > > I followed these guides on a clean Fedora 21 x86_64 server: > > > > > > > > https://www.rdoproject.org/Quickstart > > > > https://www.rdoproject.org/Floating_IP_range > > > > > > > > > > [. . .] > > > > > > > once all 20 requests failed, it got to a login screen, but I could not ping > > > > or SSH into it: > > > > > > > > # ping 10.40.128.81 > > > > PING 10.40.128.81 (10.40.128.81) 56(84) bytes of data. > > > > From 10.40.128.44 icmp_seq=1 Destination Host Unreachable > > > > From 10.40.128.44 icmp_seq=2 Destination Host Unreachable > > > > From 10.40.128.44 icmp_seq=3 Destination Host Unreachable > > > > From 10.40.128.44 icmp_seq=4 Destination Host Unreachable > > > > > > > > # ssh cirros(a)10.40.128.81 (mailto:cirros@10.40.128.81) > > > > ssh: connect to host 10.40.128.81 port 22: No route to host > > > > > > > > > > > > > It could be any no. of reasons, as I don't know what's going on in your > > > network. But, your steps sound reasonably correct. Just for comparision, > > > that's what I normally do: > > > > > > # Create new private network: > > > $ neutron net-create $privnetname > > > > > > # Create a subnet > > > neutron subnet-create $privnetname \ > > > $subnetspace/24 \ > > > --name $privsubnetname > > > > > > # Create a router > > > neutron router-create $routername > > > > > > # Associate the router to the external network by setting its gateway > > > # NOTE: This assumes the external network name is 'ext' > > > > > > export EXT_NET=$(neutron net-list | grep ext | awk '{print $2;}') > > > export PRIV_NET=$(neutron subnet-list | grep $privsubnetname | awk '{print $2;}') > > > export ROUTER_ID=$(neutron router-list | grep $routername | awk '{print $2;}' > > > > > > neutron router-gateway-set \ > > > $ROUTER_ID $EXT_NET_ID > > > > > > neutron router-interface-add \ > > > $ROUTER_ID $PRIV_NET_ID > > > > > > > > > # Add Neutron security groups for this test tenant > > > neutron security-group-rule-create \ > > > --protocol icmp \ > > > --direction ingress \ > > > --remote-ip-prefix 0.0.0.0/0 \ > > > default > > > > > > neutron security-group-rule-create \ > > > --protocol tcp \ > > > --port-range-min 22 \ > > > --port-range-max 22 \ > > > --direction ingress \ > > > --remote-ip-prefix 0.0.0.0/0 \ > > > default > > > > > > > > > On a related note, all the above, inlcuding creating the Keystone > > > tenant, user, etc is put together in this trivial script[1], which > > > allows me to create tenant networks this way: > > > > > > $ ./create-new-tenant-network.sh (http://create-new-tenant-network.sh) \ > > > demoten1 tuser1 \ > > > 14.0.0.0 trouter1 \ > > > priv-net1 priv-subnet1 > > > > > > It assumes your external network is named as "ext", but you can modify > > > the script trivially to change that. > > > > > > > > > [1] https://github.com/kashyapc/ostack-misc/blob/master/create-new-tenant-net... > > > > Thanks Kashyab, much appreciated. I've tried all this out, but the result seems to be the same (timeouts in cloud-init, the VM is unreachable). > > So it seems there’s two problems here, correct me if I’m wrong - > > 1) VM’s getting access to the metadata service > > 2) VM’s accessible via their floating IP’s from the outside > > I would say that (2) is the more important one we need to fix right now. If it’s pingable from the namespace then your overlays (or inter-node communication) and DHCP is working OK. That means that it’s likely the link between the external network bridge and the outside. From the output below, it suggests that you’ve defined your external network as a non-provider network. Therefore, you have to tell the L3 agent the specific bridge it needs to use to route traffic. Thanks. For the record, it's only pingable when I route my private network to "public". When the router's gateway is set to "ext", the floating IP isn't pingable at all (including from the namespace): ip netns exec qrouter-f2bfd294-c90c-4c98-9b6d-b33e28b7c9ef ping 10.40.128.84 connect: Network is unreachable (f2bfd...9ef is the ID of the router between the private and external network, 10.40.128.84 is the floting IP). > > In your L3 agent configuration file you’ll have ‘external_network_bridge’ option. This will need to be set to ‘br-ex’ (or the name of your external bridge) for the flows to be set up correctly. If this is blank, you’ll need to recreate your external network as a provider network, and ensure that you have the correct bridge mappings enabled on your Neutron network node. > > So I guess my question is this… what is ‘external_network_bridge’ set to in /etc/neutron/l3-agent.ini? > > [root@stack-node1 ~]# grep external_network_bridge /etc/neutron/l3_agent.ini > # When external_network_bridge is set, each L3 agent can be associated > # networks, both the external_network_bridge and gateway_external_network_id > # external_network_bridge = br-ex > external_network_bridge = br-ex > It is indeed set to br-ex. > > Cheers > Rhys > > > > > > > When I switched the router's gateway from "ext" to "public" (a network created by packstack) and booted the VM in my private network, it got to the login screen immediately and the floating IP was pingable through `ip netns exec`. Changing the gateway back to "ext", I got the timeouts again. That seems to indicate that the issue is related to "ext" rather then the way I set up a private network or boot the VM. > > > > There doesn't seem to be a significant difference between "ext" and "public" networks and their subnets: > > > > # neutron net-show public > > +---------------------------+--------------------------------------+ > > | Field | Value | > > +---------------------------+--------------------------------------+ > > | admin_state_up | True | > > | id | 5d2a0846-4244-4d3b-ad68-033a18224459 | > > | mtu | 0 | > > | name | public | > > | provider:network_type | vxlan | > > | provider:physical_network | | > > | provider:segmentation_id | 10 | > > | router:external | True | > > | shared | True | > > | status | ACTIVE | > > | subnets | 5285ff33-1bed-449b-b629-8ecc5ec0f642 | > > | tenant_id | 3c7799abd0af430696428247d377ceaf | > > +---------------------------+--------------------------------------+ > > # neutron net-show ext > > +---------------------------+--------------------------------------+ > > | Field | Value | > > +---------------------------+--------------------------------------+ > > | admin_state_up | True | > > | id | 376e6c88-4752-476b-8feb-ae3346a98006 | > > | mtu | 0 | > > | name | ext | > > | provider:network_type | vxlan | > > | provider:physical_network | | > > | provider:segmentation_id | 12 | > > | router:external | True | > > | shared | False | > > | status | ACTIVE | > > | subnets | db336afd-8d41-4938-97ac-39ec912597df | > > | tenant_id | 3c7799abd0af430696428247d377ceaf | > > +---------------------------+--------------------------------------+ > > # neutron subnet-show public_subnet > > +-------------------+--------------------------------------------------+ > > | Field | Value | > > +-------------------+--------------------------------------------------+ > > | allocation_pools | {"start": "172.24.4.226", "end": "172.24.4.238"} | > > | cidr | 172.24.4.224/28 | > > | dns_nameservers | | > > | enable_dhcp | False | > > | gateway_ip | 172.24.4.225 | > > | host_routes | | > > | id | 5285ff33-1bed-449b-b629-8ecc5ec0f642 | > > | ip_version | 4 | > > | ipv6_address_mode | | > > | ipv6_ra_mode | | > > | name | public_subnet | > > | network_id | 5d2a0846-4244-4d3b-ad68-033a18224459 | > > | subnetpool_id | | > > | tenant_id | 3c7799abd0af430696428247d377ceaf | > > +-------------------+--------------------------------------------------+ > > # neutron subnet-show ext_subnet > > +-------------------+--------------------------------------------------+ > > | Field | Value | > > +-------------------+--------------------------------------------------+ > > | allocation_pools | {"start": "10.40.128.80", "end": "10.40.128.84"} | > > | cidr | 10.40.128.0/20 | > > | dns_nameservers | | > > | enable_dhcp | False | > > | gateway_ip | 10.40.143.254 | > > | host_routes | | > > | id | db336afd-8d41-4938-97ac-39ec912597df | > > | ip_version | 4 | > > | ipv6_address_mode | | > > | ipv6_ra_mode | | > > | name | ext_subnet | > > | network_id | 376e6c88-4752-476b-8feb-ae3346a98006 | > > | subnetpool_id | | > > | tenant_id | 3c7799abd0af430696428247d377ceaf | > > +-------------------+--------------------------------------------------+ > > > > > > I've also seen this: https://www.rdoproject.org/Neutron_with_existing_external_network > > > > Tried to follow it some time ago, but whenever I got to the `service network restart`, I got disconnected from my box and it was unreachable even after reboot. > > > > > > Is there anything else that jumps at you? Or do you have any ideas how to investigate this further? > > > > I was also thinking I could change "public"'s subnet to the floating IP range I have available, but I worry that may screw everything up. Is it worth a try? > > > > Thanks, > > Tomas > > > > > > > > _______________________________________________ > > Rdo-list mailing list > > Rdo-list(a)redhat.com (mailto:Rdo-list@redhat.com) > > https://www.redhat.com/mailman/listinfo/rdo-list > > > > To unsubscribe: rdo-list-unsubscribe(a)redhat.com (mailto:rdo-list-unsubscribe@redhat.com) _______________________________________________ Rdo-list mailing list Rdo-list(a)redhat.com (mailto:Rdo-list@redhat.com) https://www.redhat.com/mailman/listinfo/rdo-list To unsubscribe: rdo-list-unsubscribe(a)redhat.com (mailto:rdo-list-unsubscribe@redhat.com)