[Rdo-list] Outbound packet traffic with nova-network
msolberg at redhat.com
Fri Jun 28 13:00:05 UTC 2013
On 06/27/2013 01:13 PM, Russell Bryant wrote:
> On 06/26/2013 11:58 PM, Michael Solberg wrote:
>> Hi list.
>> I'm having some trouble with my network setup and I thought maybe the
>> group could help. I have three compute nodes with two network
>> interfaces. There's a private 192.168.32.0/24 network (fixed range) on
>> eth1 and a 10.17.12.128/25 network (floating range) on eth0 on these nodes.
>> Networking works as expected. Connectivity is good on the 192 network
>> and traffic is forwarded from the floating range inbound to the fixed
>> range correctly. That is, I can ssh into an instance from the outside
>> world using the floating address. However, when traffic is leaving the
>> instance, it doesn't seem to get translated. Here's a concrete example:
>> Instance has a fixed address of 192.168.32.4.
>> Instance is assigned a floating address of 10.17.12.139.
>> Instance is running on a hypervisor with the address 10.17.12.12.
>> Pings from the instance to 10.17.12.12 return fine, but pings from the
>> instance out to the internet don't work. When I run a tcpdump, I see
>> that the pings reach the destination with the source address set to
>> [root at 10.17.12.11 ~]# tcpdump -n -i eth0 icmp
>> tcpdump: WARNING: eth0: no IPv4 address assigned
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
>> 23:55:38.238739 IP 192.168.32.4 > 10.17.12.11: ICMP echo request, id
>> 45321, seq 1, length 64
>> 23:55:38.238774 IP 10.17.12.11 > 192.168.32.4: ICMP echo reply, id
>> 45321, seq 1, length 64
>> Is this expected? If so, is there any way to get nova-network to
>> translate the outbound traffic so that it looks like it's coming from
>> 10.17.12.139 instead of 192.168.32.4?
> Can you provide more information on your network topology? What
> nova-network mode are you using?
I'm using single-host FlatDHCPManager. Here's a crude diagram of the
(eth0 in VM) 192.168.32.4 -> (on hypervisor) vnet0 -> br100 -> (eth0
10.17.12.12) -> (across switch to other host) 10.17.12.11 (eth0).
The host at 10.17.12.11 didn't have a route back to 192.168.32.4, so the
reply packets were just dropped.
> Your example is for "pings from the instance out to the internet", but
> you're pinging an address (10.17.12.11) that's on the same private
> network as the compute node (10.17.12.12). My guess is that is
> confusing things.
I was able to get the behavior I wanted by adding a masquerade rule to
the POSTROUTING table:
iptables -t nat -I POSTROUTING 1 -s 192.168.32.0/22 ! -d 192.168.32.0/22
Now all traffic out from the virtual machine gets NAT'd when it leaves
the nova network host:
[root at virt-two ~]# tcpdump -n -i eth0 icmp
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
08:39:30.685952 IP 10.17.12.12 > 10.17.12.11: ICMP echo request, id
45596, seq 1, length 64
08:39:30.686007 IP 10.17.12.11 > 10.17.12.12: ICMP echo reply, id 45596,
seq 1, length 64
10.17.12.12 routes the packets back to 192.168.32.4.
More information about the dev