Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan

Hi, I have found a few problems and hopefully one or more may be related to the case that you have experienced: 1. When using OVS it is important you run the service ovs-quantum-cleanup when the host boots. This is due to the fact that OVS will store all tap device. This causes havoc when restarting hosts (in particular ones that have dhcp and l3 agents). So please make sure you have run "chkconfig quantum-ovs-cleanup on" on all hosts that are running the OVS. You can verify if this is the case by checking of the DHCP agent has created an IP address on the host. [Please note that we have a problem here - in the file /etc/init.d/quantum-ovs-cleanup "--config-file /usr/share/$proj/$proj-dist.conf" needs to be removed]. 2. Which dnsmasq version are you using? If this is 2.48 then there is a problem with the DHCP agent running. We are in the process of resolving this. If you make use of a version with tag support then this will work. 3. The quantum client needs to be updated to support the security groups. Hopefully we will have solutions for all of the above ASAP. Thanks Gary On 04/28/2013 12:26 PM, Gary Kotton wrote: > Hi, > I have been able to reproduce the problem. I'll get back to you as > soon as I have any information. > Thanks > Gary > > On 04/28/2013 11:56 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Yes, I did run quantum-dhcp-setup on network node. >> >> Thanks, good luck there. >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton@redhat.com] >> *Sent:* Sunday, April 28, 2013 4:47 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list(a)redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Thanks. >> One more question - on the network node, did you run quantum-dhcp-setup? >> I am nearly ready with my setup. Hopefully I'll have a reproduction >> or some additional questions. >> Thanks >> Gary >> >> On 04/28/2013 11:41 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Sure, my answers below. :) >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton@redhat.com] >> *Sent:* Sunday, April 28, 2013 4:31 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list(a)redhat.com <mailto:rdo-list@redhat.com> >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Hi, >> I have a few questions (please be patient with me): >> 1. On the compute node, which services are running? >> >> <A> nova-compute, nova-novncproxy, quantum-openvswitch-agent, >> openvswitch >> >> >> 2. Can you please print the iptables on the compute node? >> >> <A> I disabled it already, here's output before I do it. >> >> [root@computer-2 ~]# iptables-save >> >> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >> >> *filter >> >> :INPUT ACCEPT [22634:3487580] >> >> :FORWARD ACCEPT [22:704] >> >> :OUTPUT ACCEPT [22619:5860198] >> >> :nova-compute-FORWARD - [0:0] >> >> :nova-compute-INPUT - [0:0] >> >> :nova-compute-OUTPUT - [0:0] >> >> :nova-compute-inst-26 - [0:0] >> >> :nova-compute-local - [0:0] >> >> :nova-compute-provider - [0:0] >> >> :nova-compute-sg-fallback - [0:0] >> >> :nova-filter-top - [0:0] >> >> -A INPUT -j nova-compute-INPUT >> >> -A FORWARD -j nova-filter-top >> >> -A FORWARD -j nova-compute-FORWARD >> >> -A OUTPUT -j nova-filter-top >> >> -A OUTPUT -j nova-compute-OUTPUT >> >> -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp >> -m udp --sport 68 --dport 67 -j ACCEPT >> >> -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m >> udp --sport 68 --dport 67 -j ACCEPT >> >> -A nova-compute-inst-26 -m state --state INVALID -j DROP >> >> -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT >> >> -A nova-compute-inst-26 -j nova-compute-provider >> >> -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67 >> --dport 68 -j ACCEPT >> >> -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT >> >> -A nova-compute-inst-26 -p icmp -j ACCEPT >> >> -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT >> >> -A nova-compute-inst-26 -j nova-compute-sg-fallback >> >> -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26 >> >> -A nova-compute-sg-fallback -j DROP >> >> -A nova-filter-top -j nova-compute-local >> >> COMMIT >> >> # Completed on Sun Apr 28 16:37:18 2013 >> >> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >> >> *mangle >> >> :PREROUTING ACCEPT [22733:3519752] >> >> :INPUT ACCEPT [22733:3519752] >> >> :FORWARD ACCEPT [175:50468] >> >> :OUTPUT ACCEPT [22705:5868566] >> >> :POSTROUTING ACCEPT [22880:5919034] >> >> :nova-compute-POSTROUTING - [0:0] >> >> -A POSTROUTING -j nova-compute-POSTROUTING >> >> COMMIT >> >> # Completed on Sun Apr 28 16:37:18 2013 >> >> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >> >> *nat >> >> :PREROUTING ACCEPT [16:14570] >> >> :POSTROUTING ACCEPT [338:22855] >> >> :OUTPUT ACCEPT [331:20579] >> >> :nova-compute-OUTPUT - [0:0] >> >> :nova-compute-POSTROUTING - [0:0] >> >> :nova-compute-PREROUTING - [0:0] >> >> :nova-compute-float-snat - [0:0] >> >> :nova-compute-snat - [0:0] >> >> :nova-postrouting-bottom - [0:0] >> >> -A PREROUTING -j nova-compute-PREROUTING >> >> -A POSTROUTING -j nova-compute-POSTROUTING >> >> -A POSTROUTING -j nova-postrouting-bottom >> >> -A OUTPUT -j nova-compute-OUTPUT >> >> -A nova-compute-snat -j nova-compute-float-snat >> >> -A nova-postrouting-bottom -j nova-compute-snat >> >> COMMIT >> >> # Completed on Sun Apr 28 16:37:18 2013 >> >> >> 3. Can you please print the flow table rules (ovs-dpctl dump-flows >> br-int)? >> >> <A> I suppose you mean ovs-ofctl dump-flows br-int ? >> >> [root@computer-2 ~]# ovs-ofctl dump-flows br-int >> >> NXST_FLOW reply (xid=0x4): >> >> cookie=0x0, duration=4125.444s, table=0, n_packets=1707, >> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL >> >> cookie=0x0, duration=4123.006s, table=0, n_packets=143, >> n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop >> >> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, >> idle_age=3349, priority=3,in_port=1,dl_vlan=1001 >> actions=mod_vlan_vid:1,NORMAL >> >> Here's also ovs-dpctl show: >> >> [root@computer-2 ~]# ovs-dpctl show >> >> system@br-p3p1: >> >> lookups: hit:3967 missed:314 lost:0 >> >> flows: 1 >> >> port 0: br-p3p1 (internal) >> >> port 1: p3p1 >> >> port 2: phy-br-p3p1 >> >> system@br-int: >> >> lookups: hit:1575 missed:302 lost:0 >> >> flows: 0 >> >> port 0: br-int (internal) >> >> port 1: int-br-p3p1 >> >> port 4: qvo39242f22-ec >> >> >> Thanks >> Gary >> >> On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi,Gary >> >> I tried capture packet while keeping VM to restart it's network. >> >> I can see dhcp request broadcast packet on tap, qbr, qvb and qvo >> interfaces. >> >> Failed to see packet on int-br-p3p1 on bridge br-int. >> >> Not sure if it has something to do with openflow setting? I attach >> some ovs-ofctl outputs >> >> I have not seen "veth" port anywhere... >> >> ---Record--- >> >> [root@computer-2 ~]# brctl show >> >> bridge name bridge id STP enabled interfaces >> >> qbr39242f22-ec 8000.c6f95e6a859a no >> qvb39242f22-ec >> >> tap39242f22-ec >> >> virbr0 8000.525400c47f62 yes virbr0-nic >> >> [root@computer-2 ~]# ovs-vsctl show >> >> 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b >> >> Bridge br-int >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port "qvo39242f22-ec" >> >> tag: 1 >> >> Interface "qvo39242f22-ec" >> >> Bridge "br-p3p1" >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> ovs_version: "1.9.0" >> >> [root@computer-2 ~]# tcpdump -i tap39242f22-ec port 67 >> >> tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 2 packets captured >> >> 2 packets received by filter >> >> 0 packets dropped by kernel >> >> [root@computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 >> >> tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 1 packets captured >> >> 1 packets received by filter >> >> 0 packets dropped by kernel >> >> [root@computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 >> >> tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 1 packets captured >> >> 1 packets received by filter >> >> 0 packets dropped by kernel >> >> [root@computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 >> >> tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 2 packets captured >> >> 2 packets received by filter >> >> 0 packets dropped by kernel >> >> [root@computer-2 ~]# tcpdump -i int-br-p3p1 port 67 >> >> tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size >> 65535 bytes >> >> ^C >> >> 0 packets captured >> >> 0 packets received by filter >> >> 0 packets dropped by kernel >> >> ---output of ovs-ofctl--- >> >> [root@computer-2 ~]# ovs-ofctl show br-int >> >> OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 >> >> n_tables:255, n_buffers:256 >> >> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP >> >> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC >> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST >> ENQUEUE >> >> 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD COPPER >> >> speed: 10000 Mbps now, 100 Mbps max >> >> 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD COPPER >> >> speed: 10000 Mbps now, 100 Mbps max >> >> LOCAL(br-int): addr:86:40:18:20:f1:42 >> >> config: PORT_DOWN >> >> state: LINK_DOWN >> >> speed: 100 Mbps now, 100 Mbps max >> >> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 >> >> [root@computer-2 ~]# >> >> [root@computer-2 ~]# ovs-ofctl show br-p3p1 >> >> OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 >> >> n_tables:255, n_buffers:256 >> >> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP >> >> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC >> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST >> ENQUEUE >> >> 1(p3p1): addr:a0:36:9f:15:d4:24 >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD >> >> advertised: 10GB-FD FIBER >> >> supported: 10GB-FD FIBER >> >> speed: 10000 Mbps now, 10000 Mbps max >> >> 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD COPPER >> >> speed: 10000 Mbps now, 100 Mbps max >> >> LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 >> >> config: PORT_DOWN >> >> state: LINK_DOWN >> >> speed: 100 Mbps now, 100 Mbps max >> >> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 >> >> [root@computer-2 ~]# ovs-ofctl dump-flows br-int >> >> NXST_FLOW reply (xid=0x4): >> >> cookie=0x0, duration=4125.444s, table=0, n_packets=1707, >> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL >> >> cookie=0x0, duration=4123.006s, table=0, n_packets=143, >> n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop >> >> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, >> idle_age=3349, priority=3,in_port=1,dl_vlan=1001 >> actions=mod_vlan_vid:1,NORMAL >> >> [root@computer-2 ~]# ovs-ofctl dump-flows br-p3p1 >> >> NXST_FLOW reply (xid=0x4): >> >> cookie=0x0, duration=4129.629s, table=0, n_packets=2175, >> n_bytes=138652, idle_age=0, priority=1 actions=NORMAL >> >> cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, >> idle_age=1045, priority=2,in_port=2 actions=drop >> >> cookie=0x0, duration=3354.578s, table=0, n_packets=1697, >> n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 >> actions=mod_vlan_vid:1001,NORMAL >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*rdo-list-bounces@redhat.com >> <mailto:rdo-list-bounces@redhat.com> >> [mailto:rdo-list-bounces@redhat.com] *On Behalf Of *ext Zhang, Kimi >> (NSN - CN/Cheng Du) >> *Sent:* Sunday, April 28, 2013 3:40 PM >> *To:* gkotton(a)redhat.com <mailto:gkotton@redhat.com>; >> rdo-list(a)redhat.com <mailto:rdo-list@redhat.com> >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Very nice pic, I am going to try to capture packet on each port. >> >> I did not configure to use quantum to manage firewall , just leave >> it to nova-compute, will try your configs later. >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*rdo-list-bounces@redhat.com >> <mailto:rdo-list-bounces@redhat.com> >> [mailto:rdo-list-bounces@redhat.com] *On Behalf Of *ext Gary Kotton >> *Sent:* Sunday, April 28, 2013 3:33 PM >> *To:* rdo-list(a)redhat.com <mailto:rdo-list@redhat.com> >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Hi, >> Can you also please check that firewall_driver = >> quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is >> configured in plugin.ini file.And security_group_api = quantum is >> set in nova.conf >> Thanks >> Gary >> >> On 04/28/2013 10:21 AM, Gary Kotton wrote: >> >> On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi, Gary >> >> I tried to disable iptables on both network and compute nodes, still >> does not work out L >> >> >> Can you please look at >> https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B5... >> >> When using the OVS there are a number of devices. Would it be >> possible that you try and capture on each device so that we can try >> and see where the packet is discarded. >> >> I will have a setup ready in about an hour. >> >> >> >> From quantum openvswitch agent logs, following messages keeps coming >> out repeatly every 2-3 seconds, not sure if they matter or not? >> >> >> The messages below are OK - this is how the OVS agent works. It >> polls the OVS every interval to check if new ports are created. >> >> >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> Making synchronous call on q-plugin ... >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton@redhat.com] >> *Sent:* Sunday, April 28, 2013 3:08 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list(a)redhat.com <mailto:rdo-list@redhat.com> >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> I tried that too, no lucky. >> >> From tcpdump ,it seems br-int does not forward any packet to >> interfaces connect to br-p3p1, which connects to physical network... >> >> >> There could be a number of issues here: >> 1. The iptables are dropping the traffic (I am in the process of >> getting a setup up and running) >> 2. The network connectivity >> >> In order to ensure that it is not the first one can you try and see >> which iptables rules are matched or disable the iptables? >> >> >> >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton@redhat.com] >> *Sent:* Sunday, April 28, 2013 3:01 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list(a)redhat.com <mailto:rdo-list@redhat.com> >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi, Gary >> >> Yes, I'm aware of that packstack does not support quantum yet. The >> whole setup was installed manually. >> >> I did run quantum-server-setup and quantum-host-setup, I tried >> linuxbridge plugin too, it has no issue for VM to get IP address, >> but openvswitch has issues on this... >> >> >> ok. >> >> if you configure and IP address manually on the VM are you able to >> ping the port of the DHCP agent? >> >> you can get the IP from quantum port-list >> >> >> >> >> >> >> Regards, >> >> Kimi >> >> *From:*rdo-list-bounces@redhat.com >> <mailto:rdo-list-bounces@redhat.com> >> [mailto:rdo-list-bounces@redhat.com] *On Behalf Of *ext Gary Kotton >> *Sent:* Sunday, April 28, 2013 2:50 PM >> *To:* rdo-list(a)redhat.com <mailto:rdo-list@redhat.com> >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Hi Kimi, >> Thanks for the mail. Please see the inline comments below. Please >> note that at the moment we do not have packstack support for Quantum >> so there is a little manual plumbing that needs to be done (not sure >> if you have done this already). >> On the host where the quantum service is running you need to run >> quantum-server-setup and on the compute nodes you need to run >> quantum-host-setup (please note that the relevant keystone >> credentials need to be set too). >> Thanks >> Gary >> >> On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> converted from rtf >> >> When I start VM instance, the VM can't get IP address. Could someone >> help me on this ? >> >> >> I will try >> >> >> >> >> >> 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. >> >> ·Controller node: >> >> Services: Keystone+Glance+Cinder+Quantum server + Nova services >> >> Network: bond0(10.68.125.11 for O&M) >> >> ·Network node: >> >> Services: quantum-openvswitch-agent, quantum-l3-agent, >> quantum-dhcp-agent, quantum-metadata-agent >> >> Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, >> p3p2 for external network >> >> >> Please note that RHEL currently does not support namespaces so there >> are a number of limitations. We are addressing this at the moment. >> If namespaces are not used then it is suggested that one does not >> run the DHCP agent and the L3 agent on the same host. The reason for >> this is that there is no network isolation. >> >> >> >> >> >> >> ·Compute node: >> >> Services: nove-compute and quantum-openvswitch-agent >> >> Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network >> >> ·Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) >> of network and compute nodes. >> >> 1.Quantum.conf: >> >> [DEFAULT] >> >> debug = True >> >> verbose = True >> >> lock_path = $state_path/lock >> >> bind_host = 0.0.0.0 >> >> bind_port = 9696 >> >> core_plugin = >> quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 >> >> api_paste_config = api-paste.ini >> >> rpc_backend = quantum.openstack.common.rpc.impl_kombu >> >> >> Are you using rabbit or qpid? >> >> >> >> >> >> >> control_exchange = quantum >> >> rabbit_host = 10.68.125.11 >> >> notification_driver = quantum.openstack.common.notifier.rpc_notifier >> >> default_notification_level = INFO >> >> notification_topics = notifications >> >> [QUOTAS] >> >> [DEFAULT_SERVICETYPE] >> >> [AGENT] >> >> polling_interval = 2 >> >> root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf >> >> [keystone_authtoken] >> >> auth_host = 10.68.125.11 >> >> auth_port = 35357 >> >> auth_protocol = http >> >> signing_dir = /var/lib/quantum/keystone-signing >> >> admin_tenant_name = service >> >> admin_user = quantum >> >> admin_password = password >> >> 2.ovs_quantum_plugin.ini >> >> [DATABASE] >> >> sql_connection = >> mysql://quantum:quantum@10.68.125.11:3306/ovs_quantum >> <mailto:mysql://quantum:quantum@10.68.125.11:3306/ovs_quantum> >> >> reconnect_interval = 2 >> >> [OVS] >> >> tenant_network_type = vlan >> >> network_vlan_ranges = physnet1:1000:2999 >> >> bridge_mappings = physnet1:br-p3p1 >> >> [AGENT] >> >> polling_interval = 2 >> >> [SECURITYGROUP] >> >> 3.nova.conf >> >> [DEFAULT] >> >> verbose=true >> >> logdir = /var/log/nova >> >> state_path = /var/lib/nova >> >> lock_path = /var/lib/nova/tmp >> >> volumes_dir = /etc/nova/volumes >> >> dhcpbridge = /usr/bin/nova-dhcpbridge >> >> dhcpbridge_flagfile = /etc/nova/nova.conf >> >> force_dhcp_release = True >> >> injected_network_template = /usr/share/nova/interfaces.template >> >> libvirt_nonblocking = True >> >> libvirt_inject_partition = -1 >> >> network_manager = nova.network.manager.FlatDHCPManager >> >> iscsi_helper = tgtadm >> >> compute_driver = libvirt.LibvirtDriver >> >> libvirt_type=kvm >> >> libvirt_ovs_bridge=br-int >> >> firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver >> >> manager=nova.conductor.manager.ConductorManager >> >> rpc_backend = nova.openstack.common.rpc.impl_kombu >> >> rabbit_host = 10.68.125.11 >> >> rootwrap_config = /etc/nova/rootwrap.conf >> >> use_deprecated_auth=false >> >> auth_strategy=keystone >> >> glance_api_servers=10.68.125.11:9292 >> >> image_service=nova.image.glance.GlanceImageService >> >> novnc_enabled=true >> >> novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html >> >> novncproxy_port=6080 >> >> vncserver_proxyclient_address=10.68.125.16 >> >> vncserver_listen=0.0.0.0 >> >> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver >> >> libvirt_use_virtio_for_bridges=True >> >> network_api_class=nova.network.quantumv2.api.API >> >> quantum_url=http://10.68.125.11:9696 >> >> quantum_auth_strategy=keystone >> >> quantum_admin_tenant_name=service >> >> quantum_admin_username=quantum >> >> quantum_admin_password=password >> >> quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 >> >> linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver >> >> libvirt_vif_type=ethernet >> >> service_quantum_metadata_proxy = True >> >> quantum_metadata_proxy_shared_secret = helloOpenStack >> >> metadata_host = 10.68.125.11 >> >> metadata_listen = 0.0.0.0 >> >> metadata_listen_port = 8775 >> >> [keystone_authtoken] >> >> admin_tenant_name = service >> >> admin_user = nova >> >> admin_password = password >> >> auth_host = 10.68.125.11 >> >> auth_port = 35357 >> >> auth_protocol = http >> >> signing_dir = /tmp/keystone-signing-nova >> >> 4.ovs-vsctl show on network node: >> >> aeeb6cf7-271b-405a-aa17-1b95bcd9e301 >> >> Bridge "br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> Bridge br-ex >> >> Port br-ex >> >> Interface br-ex >> >> type: internal >> >> Port "qg-a83c0abd-f4" >> >> Interface "qg-a83c0abd-f4" >> >> type: internal >> >> Port "p3p2" >> >> Interface "p3p2" >> >> Bridge br-int >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port "tap1f386a2a-12" >> >> tag: 1 >> >> Interface "tap1f386a2a-12" >> >> type: internal >> >> ovs_version: "1.9.0" >> >> 5.ovs-vsctl show on compute node: >> >> 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 >> >> Bridge "br-p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Bridge br-int >> >> Port "qvo56a4572c-dc" >> >> tag: 2 >> >> Interface "qvo56a4572c-dc" >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> ovs_version: "1.9.0" >> >> On compute node, I can see dhcp request packet from tcpdump on >> qvo56a4572c-dc, but it seems the packet is not forwarded out since I >> can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. >> >> >> Any chance to get the DHCP and the L3 agent configuration files? >> Please check that use_namespaces = False in both of these files. >> >> Are there any log errors? >> >> >> >> >> >> >> Thank you! >> >> Regards, >> >> Kimi >> >> >> >> >> >> >> >> >> _______________________________________________ >> Rdo-list mailing list >> Rdo-list(a)redhat.com <mailto:Rdo-list@redhat.com> >> https://www.redhat.com/mailman/listinfo/rdo-list >> >> >> >> >> >> >> _______________________________________________ >> Rdo-list mailing list >> Rdo-list(a)redhat.com <mailto:Rdo-list@redhat.com> >> https://www.redhat.com/mailman/listinfo/rdo-list >> > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list _______________________________________________ Rdo-list mailing list Rdo-list(a)redhat.com https://www.redhat.com/mailman/listinfo/rdo-list