10:58 p.m.
Hi list.
I'm having some trouble with my network setup and I thought maybe the
group could help. I have three compute nodes with two network
interfaces. There's a private 192.168.32.0/24 network (fixed range) on
eth1 and a 10.17.12.128/25 network (floating range) on eth0 on these nodes.
Networking works as expected. Connectivity is good on the 192 network
and traffic is forwarded from the floating range inbound to the fixed
range correctly. That is, I can ssh into an instance from the outside
world using the floating address. However, when traffic is leaving the
instance, it doesn't seem to get translated. Here's a concrete example:
Instance has a fixed address of 192.168.32.4.
Instance is assigned a floating address of 10.17.12.139.
Instance is running on a hypervisor with the address 10.17.12.12.
Pings from the instance to 10.17.12.12 return fine, but pings from the
instance out to the internet don't work. When I run a tcpdump, I see
that the pings reach the destination with the source address set to
192.168.32.4:
[root(a)10.17.12.11 ~]# tcpdump -n -i eth0 icmp
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:55:38.238739 IP 192.168.32.4 > 10.17.12.11: ICMP echo request, id
45321, seq 1, length 64
23:55:38.238774 IP 10.17.12.11 > 192.168.32.4: ICMP echo reply, id
45321, seq 1, length 64
Is this expected? If so, is there any way to get nova-network to
translate the outbound traffic so that it looks like it's coming from
10.17.12.139 instead of 192.168.32.4?
Thanks.
Michael.
--
Michael Solberg
Principal Architect, Red Hat, Inc.
12:13 p.m.
On 06/26/2013 11:58 PM, Michael Solberg wrote:
Hi list.
I'm having some trouble with my network setup and I thought maybe the
group could help. I have three compute nodes with two network
interfaces. There's a private 192.168.32.0/24 network (fixed range) on
eth1 and a 10.17.12.128/25 network (floating range) on eth0 on these nodes.
Networking works as expected. Connectivity is good on the 192 network
and traffic is forwarded from the floating range inbound to the fixed
range correctly. That is, I can ssh into an instance from the outside
world using the floating address. However, when traffic is leaving the
instance, it doesn't seem to get translated. Here's a concrete example:
Instance has a fixed address of 192.168.32.4.
Instance is assigned a floating address of 10.17.12.139.
Instance is running on a hypervisor with the address 10.17.12.12.
Pings from the instance to 10.17.12.12 return fine, but pings from the
instance out to the internet don't work. When I run a tcpdump, I see
that the pings reach the destination with the source address set to
192.168.32.4:
[root(a)10.17.12.11 ~]# tcpdump -n -i eth0 icmp
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:55:38.238739 IP 192.168.32.4 > 10.17.12.11: ICMP echo request, id
45321, seq 1, length 64
23:55:38.238774 IP 10.17.12.11 > 192.168.32.4: ICMP echo reply, id
45321, seq 1, length 64
Is this expected? If so, is there any way to get nova-network to
translate the outbound traffic so that it looks like it's coming from
10.17.12.139 instead of 192.168.32.4?
Can you provide more information on your network topology? What
nova-network mode are you using?
Your example is for "pings from the instance out to the internet", but
you're pinging an address (10.17.12.11) that's on the same private
network as the compute node (10.17.12.12). My guess is that is
confusing things.
--
Russell Bryant
8 a.m.
On 06/27/2013 01:13 PM, Russell Bryant wrote:
On 06/26/2013 11:58 PM, Michael Solberg wrote:
> Hi list.
>
> I'm having some trouble with my network setup and I thought maybe the
> group could help. I have three compute nodes with two network
> interfaces. There's a private 192.168.32.0/24 network (fixed range) on
> eth1 and a 10.17.12.128/25 network (floating range) on eth0 on these nodes.
>
> Networking works as expected. Connectivity is good on the 192 network
> and traffic is forwarded from the floating range inbound to the fixed
> range correctly. That is, I can ssh into an instance from the outside
> world using the floating address. However, when traffic is leaving the
> instance, it doesn't seem to get translated. Here's a concrete example:
>
> Instance has a fixed address of 192.168.32.4.
> Instance is assigned a floating address of 10.17.12.139.
> Instance is running on a hypervisor with the address 10.17.12.12.
>
> Pings from the instance to 10.17.12.12 return fine, but pings from the
> instance out to the internet don't work. When I run a tcpdump, I see
> that the pings reach the destination with the source address set to
> 192.168.32.4:
>
> [root(a)10.17.12.11 ~]# tcpdump -n -i eth0 icmp
> tcpdump: WARNING: eth0: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> 23:55:38.238739 IP 192.168.32.4 > 10.17.12.11: ICMP echo request, id
> 45321, seq 1, length 64
> 23:55:38.238774 IP 10.17.12.11 > 192.168.32.4: ICMP echo reply, id
> 45321, seq 1, length 64
>
> Is this expected? If so, is there any way to get nova-network to
> translate the outbound traffic so that it looks like it's coming from
> 10.17.12.139 instead of 192.168.32.4?
Can you provide more information on your network topology? What
nova-network mode are you using?
I'm using single-host FlatDHCPManager. Here's a crude diagram of the
traffic flow:
(eth0 in VM) 192.168.32.4 -> (on hypervisor) vnet0 -> br100 -> (eth0
10.17.12.12) -> (across switch to other host) 10.17.12.11 (eth0).
The host at 10.17.12.11 didn't have a route back to 192.168.32.4, so the
reply packets were just dropped.
Your example is for "pings from the instance out to the
internet", but
you're pinging an address (10.17.12.11) that's on the same private
network as the compute node (10.17.12.12). My guess is that is
confusing things.
I was able to get the behavior I wanted by adding a masquerade rule to
the POSTROUTING table:
iptables -t nat -I POSTROUTING 1 -s 192.168.32.0/22 ! -d 192.168.32.0/22
-j MASQUERADE
Now all traffic out from the virtual machine gets NAT'd when it leaves
the nova network host:
[root@virt-two ~]# tcpdump -n -i eth0 icmp
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
08:39:30.685952 IP 10.17.12.12 > 10.17.12.11: ICMP echo request, id
45596, seq 1, length 64
08:39:30.686007 IP 10.17.12.11 > 10.17.12.12: ICMP echo reply, id 45596,
seq 1, length 64
10.17.12.12 routes the packets back to 192.168.32.4.
Michael.
4198
days inactive
4199
days old
2 comments
2 participants
participants (2)
-
Michael Solberg -
Russell Bryant