[Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan

Sun Apr 28 08:47:14 UTC 2013

Thanks.
One more question - on the network node, did you run quantum-dhcp-setup?
I am nearly ready with my setup. Hopefully I'll have a reproduction or
some additional questions.
Thanks
Gary

On 04/28/2013 11:41 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote:
>
> Sure, my answers below. :)
>
> Regards,
>
> Kimi Zhang
>
> MP: +86 186 0800 8182
>
> Call me(NCS): sip:+86018608008182
>
> *From:*ext Gary Kotton [mailto:gkotton at redhat.com]
> *Sent:* Sunday, April 28, 2013 4:31 PM
> *To:* Zhang, Kimi (NSN - CN/Cheng Du)
> *Cc:* rdo-list at redhat.com
> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum +
> Openvswitch + Vlan
>
> Hi,
> I have a few questions (please be patient with me):
> 1. On the compute node, which services are running?
>
> <A> nova-compute, nova-novncproxy, quantum-openvswitch-agent, openvswitch
>
>
> 2. Can you please print the iptables on the compute node?
>
> <A> I disabled it already, here's output before I do it.
>
> [root at computer-2 ~]# iptables-save
>
> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013
>
> *filter
>
> :INPUT ACCEPT [22634:3487580]
>
> :FORWARD ACCEPT [22:704]
>
> :OUTPUT ACCEPT [22619:5860198]
>
> :nova-compute-FORWARD - [0:0]
>
> :nova-compute-INPUT - [0:0]
>
> :nova-compute-OUTPUT - [0:0]
>
> :nova-compute-inst-26 - [0:0]
>
> :nova-compute-local - [0:0]
>
> :nova-compute-provider - [0:0]
>
> :nova-compute-sg-fallback - [0:0]
>
> :nova-filter-top - [0:0]
>
> -A INPUT -j nova-compute-INPUT
>
> -A FORWARD -j nova-filter-top
>
> -A FORWARD -j nova-compute-FORWARD
>
> -A OUTPUT -j nova-filter-top
>
> -A OUTPUT -j nova-compute-OUTPUT
>
> -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m
> udp --sport 68 --dport 67 -j ACCEPT
>
> -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m
> udp --sport 68 --dport 67 -j ACCEPT
>
> -A nova-compute-inst-26 -m state --state INVALID -j DROP
>
> -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT
>
> -A nova-compute-inst-26 -j nova-compute-provider
>
> -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67
> --dport 68 -j ACCEPT
>
> -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT
>
> -A nova-compute-inst-26 -p icmp -j ACCEPT
>
> -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT
>
> -A nova-compute-inst-26 -j nova-compute-sg-fallback
>
> -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26
>
> -A nova-compute-sg-fallback -j DROP
>
> -A nova-filter-top -j nova-compute-local
>
> COMMIT
>
> # Completed on Sun Apr 28 16:37:18 2013
>
> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013
>
> *mangle
>
> :PREROUTING ACCEPT [22733:3519752]
>
> :INPUT ACCEPT [22733:3519752]
>
> :FORWARD ACCEPT [175:50468]
>
> :OUTPUT ACCEPT [22705:5868566]
>
> :POSTROUTING ACCEPT [22880:5919034]
>
> :nova-compute-POSTROUTING - [0:0]
>
> -A POSTROUTING -j nova-compute-POSTROUTING
>
> COMMIT
>
> # Completed on Sun Apr 28 16:37:18 2013
>
> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013
>
> *nat
>
> :PREROUTING ACCEPT [16:14570]
>
> :POSTROUTING ACCEPT [338:22855]
>
> :OUTPUT ACCEPT [331:20579]
>
> :nova-compute-OUTPUT - [0:0]
>
> :nova-compute-POSTROUTING - [0:0]
>
> :nova-compute-PREROUTING - [0:0]
>
> :nova-compute-float-snat - [0:0]
>
> :nova-compute-snat - [0:0]
>
> :nova-postrouting-bottom - [0:0]
>
> -A PREROUTING -j nova-compute-PREROUTING
>
> -A POSTROUTING -j nova-compute-POSTROUTING
>
> -A POSTROUTING -j nova-postrouting-bottom
>
> -A OUTPUT -j nova-compute-OUTPUT
>
> -A nova-compute-snat -j nova-compute-float-snat
>
> -A nova-postrouting-bottom -j nova-compute-snat
>
> COMMIT
>
> # Completed on Sun Apr 28 16:37:18 2013
>
>
> 3. Can you please print the flow table rules (ovs-dpctl dump-flows
> br-int)?
>
> <A> I suppose you mean ovs-ofctl dump-flows br-int ?
>
> [root at computer-2 ~]# ovs-ofctl dump-flows br-int
>
> NXST_FLOW reply (xid=0x4):
>
> cookie=0x0, duration=4125.444s, table=0, n_packets=1707,
> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL
>
> cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688,
> idle_age=20, priority=2,in_port=1 actions=drop
>
> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0,
> idle_age=3349, priority=3,in_port=1,dl_vlan=1001
> actions=mod_vlan_vid:1,NORMAL
>
> Here’s also ovs-dpctl show:
>
> [root at computer-2 ~]# ovs-dpctl show
>
> system at br-p3p1:
>
> lookups: hit:3967 missed:314 lost:0
>
> flows: 1
>
> port 0: br-p3p1 (internal)
>
> port 1: p3p1
>
> port 2: phy-br-p3p1
>
> system at br-int:
>
> lookups: hit:1575 missed:302 lost:0
>
> flows: 0
>
> port 0: br-int (internal)
>
> port 1: int-br-p3p1
>
> port 4: qvo39242f22-ec
>
>
> Thanks
> Gary
>
> On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote:
>
> Hi，Gary
>
> I tried capture packet while keeping VM to restart it’s network.
>
> I can see dhcp request broadcast packet on tap, qbr, qvb and qvo
> interfaces.
>
> Failed to see packet on int-br-p3p1 on bridge br-int.
>
> Not sure if it has something to do with openflow setting? I attach
> some ovs-ofctl outputs
>
> I have not seen “veth” port anywhere…
>
> ---Record---
>
> [root at computer-2 ~]# brctl show
>
> bridge name bridge id STP enabled interfaces
>
> qbr39242f22-ec 8000.c6f95e6a859a no qvb39242f22-ec
>
> tap39242f22-ec
>
> virbr0 8000.525400c47f62 yes virbr0-nic
>
> [root at computer-2 ~]# ovs-vsctl show
>
> 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b
>
> Bridge br-int
>
> Port br-int
>
> Interface br-int
>
> type: internal
>
> Port "int-br-p3p1"
>
> Interface "int-br-p3p1"
>
> Port "qvo39242f22-ec"
>
> tag: 1
>
> Interface "qvo39242f22-ec"
>
> Bridge "br-p3p1"
>
> Port "phy-br-p3p1"
>
> Interface "phy-br-p3p1"
>
> Port "p3p1"
>
> Interface "p3p1"
>
> Port "br-p3p1"
>
> Interface "br-p3p1"
>
> type: internal
>
> ovs_version: "1.9.0"
>
> [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67
>
> tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned
>
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>
> listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture size
> 65535 bytes
>
> 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300
>
> 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300
>
> ^C
>
> 2 packets captured
>
> 2 packets received by filter
>
> 0 packets dropped by kernel
>
> [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67
>
> tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned
>
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>
> listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture size
> 65535 bytes
>
> 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300
>
> ^C
>
> 1 packets captured
>
> 1 packets received by filter
>
> 0 packets dropped by kernel
>
> [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67
>
> tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned
>
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>
> listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture size
> 65535 bytes
>
> 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300
>
> ^C
>
> 1 packets captured
>
> 1 packets received by filter
>
> 0 packets dropped by kernel
>
> [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67
>
> tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned
>
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>
> listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture size
> 65535 bytes
>
> 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300
>
> 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300
>
> ^C
>
> 2 packets captured
>
> 2 packets received by filter
>
> 0 packets dropped by kernel
>
> [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67
>
> tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned
>
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>
> listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size
> 65535 bytes
>
> ^C
>
> 0 packets captured
>
> 0 packets received by filter
>
> 0 packets dropped by kernel
>
> ---output of ovs-ofctl---
>
> [root at computer-2 ~]# ovs-ofctl show br-int
>
> OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142
>
> n_tables:255, n_buffers:256
>
> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
>
> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC
> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
>
> 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d
>
> config: 0
>
> state: 0
>
> current: 10GB-FD COPPER
>
> speed: 10000 Mbps now, 100 Mbps max
>
> 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78
>
> config: 0
>
> state: 0
>
> current: 10GB-FD COPPER
>
> speed: 10000 Mbps now, 100 Mbps max
>
> LOCAL(br-int): addr:86:40:18:20:f1:42
>
> config: PORT_DOWN
>
> state: LINK_DOWN
>
> speed: 100 Mbps now, 100 Mbps max
>
> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0
>
> [root at computer-2 ~]#
>
> [root at computer-2 ~]# ovs-ofctl show br-p3p1
>
> OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424
>
> n_tables:255, n_buffers:256
>
> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
>
> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC
> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
>
> 1(p3p1): addr:a0:36:9f:15:d4:24
>
> config: 0
>
> state: 0
>
> current: 10GB-FD
>
> advertised: 10GB-FD FIBER
>
> supported: 10GB-FD FIBER
>
> speed: 10000 Mbps now, 10000 Mbps max
>
> 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0
>
> config: 0
>
> state: 0
>
> current: 10GB-FD COPPER
>
> speed: 10000 Mbps now, 100 Mbps max
>
> LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24
>
> config: PORT_DOWN
>
> state: LINK_DOWN
>
> speed: 100 Mbps now, 100 Mbps max
>
> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0
>
> [root at computer-2 ~]# ovs-ofctl dump-flows br-int
>
> NXST_FLOW reply (xid=0x4):
>
> cookie=0x0, duration=4125.444s, table=0, n_packets=1707,
> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL
>
> cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688,
> idle_age=20, priority=2,in_port=1 actions=drop
>
> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0,
> idle_age=3349, priority=3,in_port=1,dl_vlan=1001
> actions=mod_vlan_vid:1,NORMAL
>
> [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1
>
> NXST_FLOW reply (xid=0x4):
>
> cookie=0x0, duration=4129.629s, table=0, n_packets=2175,
> n_bytes=138652, idle_age=0, priority=1 actions=NORMAL
>
> cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224,
> idle_age=1045, priority=2,in_port=2 actions=drop
>
> cookie=0x0, duration=3354.578s, table=0, n_packets=1697,
> n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1
> actions=mod_vlan_vid:1001,NORMAL
>
> Regards,
>
> Kimi Zhang
>
> MP: +86 186 0800 8182
>
> Call me(NCS): sip:+86018608008182
>
> *From:*rdo-list-bounces at redhat.com
> <mailto:rdo-list-bounces at redhat.com>
> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Zhang, Kimi
> (NSN - CN/Cheng Du)
> *Sent:* Sunday, April 28, 2013 3:40 PM
> *To:* gkotton at redhat.com <mailto:gkotton at redhat.com>;
> rdo-list at redhat.com <mailto:rdo-list at redhat.com>
> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum +
> Openvswitch + Vlan
>
> Very nice pic, I am going to try to capture packet on each port.
>
> I did not configure to use quantum to manage firewall , just leave it
> to nova-compute, will try your configs later.
>
> Regards,
>
> Kimi Zhang
>
> MP: +86 186 0800 8182
>
> Call me(NCS): sip:+86018608008182
>
> *From:*rdo-list-bounces at redhat.com
> <mailto:rdo-list-bounces at redhat.com>
> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton
> *Sent:* Sunday, April 28, 2013 3:33 PM
> *To:* rdo-list at redhat.com <mailto:rdo-list at redhat.com>
> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum +
> Openvswitch + Vlan
>
> Hi,
> Can you also please check that firewall_driver =
> quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
> is configured in plugin.ini file.And security_group_api = quantum is
> set in nova.conf
> Thanks
> Gary
>
> On 04/28/2013 10:21 AM, Gary Kotton wrote:
>
> On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote:
>
> Hi, Gary
>
> I tried to disable iptables on both network and compute nodes, still
> does not work out L
>
>
> Can you please look at
> https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing
>
> When using the OVS there are a number of devices. Would it be possible
> that you try and capture on each device so that we can try and see
> where the packet is discarded.
>
> I will have a setup ready in about an hour.
>
>
> From quantum openvswitch agent logs, following messages keeps coming
> out repeatly every 2-3 seconds, not sure if they matter or not?
>
>
> The messages below are OK - this is how the OVS agent works. It polls
> the OVS every interval to check if new ports are created.
>
>
> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making
> synchronous call on q-plugin ...
>
> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID
> is 92f4e83cf92c46f1b9304c879f9b7a41
>
> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp]
> UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b.
>
> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command:
> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
>
> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils]
>
> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
>
> Exit code: 0
>
> Stdout: 'int-br-p3p1\n'
>
> Stderr: ''
>
> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command:
> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1',
> 'external_ids']
>
> 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils]
>
> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1',
> 'external_ids']
>
> Exit code: 0
>
> Stdout: '{}\n'
>
> Stderr: ''
>
> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command:
> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
>
> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command:
> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
>
> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils]
>
> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
>
> Exit code: 0
>
> Stdout: 'int-br-p3p1\n'
>
> Stderr: ''
>
> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command:
> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1',
> 'external_ids']
>
> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils]
>
> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
>
> Exit code: 0
>
> Stdout: 'int-br-p3p1\n'
>
> Stderr: ''
>
> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command:
> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1',
> 'external_ids']
>
> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils]
>
> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1',
> 'external_ids']
>
> Exit code: 0
>
> Stdout: '{}\n'
>
> Stderr: ''
>
> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils]
>
> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1',
> 'external_ids']
>
> Exit code: 0
>
> Stdout: '{}\n'
>
> Stderr: ''
>
> Regards,
>
> Kimi Zhang
>
> MP: +86 186 0800 8182
>
> Call me(NCS): sip:+86018608008182
>
> *From:*ext Gary Kotton [mailto:gkotton at redhat.com]
> *Sent:* Sunday, April 28, 2013 3:08 PM
> *To:* Zhang, Kimi (NSN - CN/Cheng Du)
> *Cc:* rdo-list at redhat.com <mailto:rdo-list at redhat.com>
> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum +
> Openvswitch + Vlan
>
> On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote:
>
> I tried that too, no lucky.
>
> From tcpdump ,it seems br-int does not forward any packet to
> interfaces connect to br-p3p1, which connects to physical network…
>
>
> There could be a number of issues here:
> 1. The iptables are dropping the traffic (I am in the process of
> getting a setup up and running)
> 2. The network connectivity
>
> In order to ensure that it is not the first one can you try and see
> which iptables rules are matched or disable the iptables?
>
>
>
> Regards,
>
> Kimi Zhang
>
> MP: +86 186 0800 8182
>
> Call me(NCS): sip:+86018608008182
>
> *From:*ext Gary Kotton [mailto:gkotton at redhat.com]
> *Sent:* Sunday, April 28, 2013 3:01 PM
> *To:* Zhang, Kimi (NSN - CN/Cheng Du)
> *Cc:* rdo-list at redhat.com <mailto:rdo-list at redhat.com>
> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum +
> Openvswitch + Vlan
>
> On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote:
>
> Hi, Gary
>
> Yes, I’m aware of that packstack does not support quantum yet. The
> whole setup was installed manually.
>
> I did run quantum-server-setup and quantum-host-setup, I tried
> linuxbridge plugin too, it has no issue for VM to get IP address, but
> openvswitch has issues on this…
>
>
> ok.
>
> if you configure and IP address manually on the VM are you able to
> ping the port of the DHCP agent?
>
> you can get the IP from quantum port-list
>
>
>
>
>
> Regards,
>
> Kimi
>
> *From:*rdo-list-bounces at redhat.com
> <mailto:rdo-list-bounces at redhat.com>
> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton
> *Sent:* Sunday, April 28, 2013 2:50 PM
> *To:* rdo-list at redhat.com <mailto:rdo-list at redhat.com>
> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum +
> Openvswitch + Vlan
>
> Hi Kimi,
> Thanks for the mail. Please see the inline comments below. Please note
> that at the moment we do not have packstack support for Quantum so
> there is a little manual plumbing that needs to be done (not sure if
> you have done this already).
> On the host where the quantum service is running you need to run
> quantum-server-setup and on the compute nodes you need to run
> quantum-host-setup (please note that the relevant keystone credentials
> need to be set too).
> Thanks
> Gary
>
> On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote:
>
> converted from rtf
>
> When I start VM instance, the VM can’t get IP address. Could someone
> help me on this ?
>
>
> I will try
>
>
>
>
> 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository.
>
> ・Controller node:
>
> Services: Keystone+Glance+Cinder+Quantum server + Nova services
>
> Network: bond0(10.68.125.11 for O&M)
>
> ・Network node:
>
> Services: quantum-openvswitch-agent, quantum-l3-agent,
> quantum-dhcp-agent, quantum-metadata-agent
>
> Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network,
> p3p2 for external network
>
>
> Please note that RHEL currently does not support namespaces so there
> are a number of limitations. We are addressing this at the moment. If
> namespaces are not used then it is suggested that one does not run the
> DHCP agent and the L3 agent on the same host. The reason for this is
> that there is no network isolation.
>
>
>
>
>
> ・Compute node:
>
> Services: nove-compute and quantum-openvswitch-agent
>
> Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network
>
> ・Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of
> network and compute nodes.
>
> 1.Quantum.conf:
>
> [DEFAULT]
>
> debug = True
>
> verbose = True
>
> lock_path = $state_path/lock
>
> bind_host = 0.0.0.0
>
> bind_port = 9696
>
> core_plugin =
> quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2
>
> api_paste_config = api-paste.ini
>
> rpc_backend = quantum.openstack.common.rpc.impl_kombu
>
>
> Are you using rabbit or qpid?
>
>
>
>
>
> control_exchange = quantum
>
> rabbit_host = 10.68.125.11
>
> notification_driver = quantum.openstack.common.notifier.rpc_notifier
>
> default_notification_level = INFO
>
> notification_topics = notifications
>
> [QUOTAS]
>
> [DEFAULT_SERVICETYPE]
>
> [AGENT]
>
> polling_interval = 2
>
> root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf
>
> [keystone_authtoken]
>
> auth_host = 10.68.125.11
>
> auth_port = 35357
>
> auth_protocol = http
>
> signing_dir = /var/lib/quantum/keystone-signing
>
> admin_tenant_name = service
>
> admin_user = quantum
>
> admin_password = password
>
> 2.ovs_quantum_plugin.ini
>
> [DATABASE]
>
> sql_connection = mysql://quantum:quantum@10.68.125.11:3306/ovs_quantum
> <mailto:mysql://quantum:quantum@10.68.125.11:3306/ovs_quantum>
>
> reconnect_interval = 2
>
> [OVS]
>
> tenant_network_type = vlan
>
> network_vlan_ranges = physnet1:1000:2999
>
> bridge_mappings = physnet1:br-p3p1
>
> [AGENT]
>
> polling_interval = 2
>
> [SECURITYGROUP]
>
> 3.nova.conf
>
> [DEFAULT]
>
> verbose=true
>
> logdir = /var/log/nova
>
> state_path = /var/lib/nova
>
> lock_path = /var/lib/nova/tmp
>
> volumes_dir = /etc/nova/volumes
>
> dhcpbridge = /usr/bin/nova-dhcpbridge
>
> dhcpbridge_flagfile = /etc/nova/nova.conf
>
> force_dhcp_release = True
>
> injected_network_template = /usr/share/nova/interfaces.template
>
> libvirt_nonblocking = True
>
> libvirt_inject_partition = -1
>
> network_manager = nova.network.manager.FlatDHCPManager
>
> iscsi_helper = tgtadm
>
> compute_driver = libvirt.LibvirtDriver
>
> libvirt_type=kvm
>
> libvirt_ovs_bridge=br-int
>
> firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver
>
> manager=nova.conductor.manager.ConductorManager
>
> rpc_backend = nova.openstack.common.rpc.impl_kombu
>
> rabbit_host = 10.68.125.11
>
> rootwrap_config = /etc/nova/rootwrap.conf
>
> use_deprecated_auth=false
>
> auth_strategy=keystone
>
> glance_api_servers=10.68.125.11:9292
>
> image_service=nova.image.glance.GlanceImageService
>
> novnc_enabled=true
>
> novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html
>
> novncproxy_port=6080
>
> vncserver_proxyclient_address=10.68.125.16
>
> vncserver_listen=0.0.0.0
>
> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
>
> libvirt_use_virtio_for_bridges=True
>
> network_api_class=nova.network.quantumv2.api.API
>
> quantum_url=http://10.68.125.11:9696
>
> quantum_auth_strategy=keystone
>
> quantum_admin_tenant_name=service
>
> quantum_admin_username=quantum
>
> quantum_admin_password=password
>
> quantum_admin_auth_url=http://10.68.125.11:35357/v2.0
>
> linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
>
> libvirt_vif_type=ethernet
>
> service_quantum_metadata_proxy = True
>
> quantum_metadata_proxy_shared_secret = helloOpenStack
>
> metadata_host = 10.68.125.11
>
> metadata_listen = 0.0.0.0
>
> metadata_listen_port = 8775
>
> [keystone_authtoken]
>
> admin_tenant_name = service
>
> admin_user = nova
>
> admin_password = password
>
> auth_host = 10.68.125.11
>
> auth_port = 35357
>
> auth_protocol = http
>
> signing_dir = /tmp/keystone-signing-nova
>
> 4.ovs-vsctl show on network node:
>
> aeeb6cf7-271b-405a-aa17-1b95bcd9e301
>
> Bridge "br-p3p1"
>
> Port "p3p1"
>
> Interface "p3p1"
>
> Port "phy-br-p3p1"
>
> Interface "phy-br-p3p1"
>
> Port "br-p3p1"
>
> Interface "br-p3p1"
>
> type: internal
>
> Bridge br-ex
>
> Port br-ex
>
> Interface br-ex
>
> type: internal
>
> Port "qg-a83c0abd-f4"
>
> Interface "qg-a83c0abd-f4"
>
> type: internal
>
> Port "p3p2"
>
> Interface "p3p2"
>
> Bridge br-int
>
> Port br-int
>
> Interface br-int
>
> type: internal
>
> Port "int-br-p3p1"
>
> Interface "int-br-p3p1"
>
> Port "tap1f386a2a-12"
>
> tag: 1
>
> Interface "tap1f386a2a-12"
>
> type: internal
>
> ovs_version: "1.9.0"
>
> 5.ovs-vsctl show on compute node:
>
> 8d6c2637-ff69-4a2d-a7db-e4f181273bc0
>
> Bridge "br-p3p1"
>
> Port "br-p3p1"
>
> Interface "br-p3p1"
>
> type: internal
>
> Port "phy-br-p3p1"
>
> Interface "phy-br-p3p1"
>
> Port "p3p1"
>
> Interface "p3p1"
>
> Bridge br-int
>
> Port "qvo56a4572c-dc"
>
> tag: 2
>
> Interface "qvo56a4572c-dc"
>
> Port "int-br-p3p1"
>
> Interface "int-br-p3p1"
>
> Port br-int
>
> Interface br-int
>
> type: internal
>
> ovs_version: "1.9.0"
>
> On compute node, I can see dhcp request packet from tcpdump on
> qvo56a4572c-dc, but it seems the packet is not forwarded out since I
> can’t see packet from int-br-p3p1 on br-int or any port from br-p3p1.
>
>
> Any chance to get the DHCP and the L3 agent configuration files?
> Please check that use_namespaces = False in both of these files.
>
> Are there any log errors?
>
>
>
>
>
> Thank you!
>
> Regards,
>
> Kimi
>
>
>
>
>
>
>
> _______________________________________________
> Rdo-list mailing list
> Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> https://www.redhat.com/mailman/listinfo/rdo-list
>
>
>
>
>
> _______________________________________________
> Rdo-list mailing list
> Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> https://www.redhat.com/mailman/listinfo/rdo-list
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20130428/bb2631bc/attachment.html>