[Rdo-list] networking issues on JUNO redhead openstack.
Ns, Rushi
rushi.ns at sap.com
Sat Jan 10 00:16:45 UTC 2015
I can’t assign a port to the floating ip . I don’t’ see any ports available ?
[cid:CD828843-4C9C-442E-9F18-65F4084FCB55]
Best Regards,
Rushi.
Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race
From: Marius Cornea <marius at remote-lab.net<mailto:marius at remote-lab.net>>
Date: Friday, January 9, 2015 at 4:06 PM
To: SAP SAP <rushi.ns at sap.com<mailto:rushi.ns at sap.com>>
Cc: rdo-list <rdo-list at redhat.com<mailto:rdo-list at redhat.com>>
Subject: Re: [Rdo-list] networking issues on JUNO redhead openstack.
You've already defined router1 which is already connected to the external network. You can use that. ext-net is the 'public' network in your case.
On Sat, Jan 10, 2015 at 1:03 AM, Ns, Rushi <rushi.ns at sap.com<mailto:rushi.ns at sap.com>> wrote:
I can’t run this command
$ neutron router-gateway-set demo-router ext-net
Its getting error (ext-net not found) ? What is the ext-net in this case ?
Best Regards,
Rushi.
Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race
From: <Ns>, SAP SAP <rushi.ns at sap.com<mailto:rushi.ns at sap.com>>
Date: Friday, January 9, 2015 at 3:58 PM
To: Marius Cornea <marius at remote-lab.net<mailto:marius at remote-lab.net>>
Cc: rdo-list <rdo-list at redhat.com<mailto:rdo-list at redhat.com>>
Subject: Re: [Rdo-list] networking issues on JUNO redhead openstack.
Ok I did the folowing steps for tenant.
1) neutron net-create demo-net
2) neutron subnet-create demo-net --name demo-subnet --gateway 10.48.146.1 10.48.146.0/24<http://10.48.146.0/24>
3) neutron router-create demo-router
4) neutron router-interface-add demo-router demo-subnet
Anything left .?
Best Regards,
Rushi.
Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race
From: <Ns>, SAP SAP <rushi.ns at sap.com<mailto:rushi.ns at sap.com>>
Date: Friday, January 9, 2015 at 3:55 PM
To: Marius Cornea <marius at remote-lab.net<mailto:marius at remote-lab.net>>
Cc: rdo-list <rdo-list at redhat.com<mailto:rdo-list at redhat.com>>
Subject: Re: [Rdo-list] networking issues on JUNO redhead openstack.
Hi Marius,
Yes, I can ping 10.48.146.113 (gateway) from the host , can you provide the comamnds or link to create tenant network, subjet and attach to router ?
Best Regards,
Rushi.
Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race
From: Marius Cornea <marius at remote-lab.net<mailto:marius at remote-lab.net>>
Date: Friday, January 9, 2015 at 3:48 PM
To: SAP SAP <rushi.ns at sap.com<mailto:rushi.ns at sap.com>>
Cc: rdo-list <rdo-list at redhat.com<mailto:rdo-list at redhat.com>>
Subject: Re: [Rdo-list] networking issues on JUNO redhead openstack.
Can you ping 10.48.146.113 from your network? You also need to create a tenant network, subnet and attach it to the router. Boot the instance with the tenant network attached and then assign it the floating IP.
On Sat, Jan 10, 2015 at 12:20 AM, Ns, Rushi <rushi.ns at sap.com<mailto:rushi.ns at sap.com>> wrote:
Hi Marius,
I did this first and now can see the BR-ex is the same IP as my eno1 interface , you are right I lost first while doing the first command(ip addr del 10.48.146.77/24<http://10.48.146.77/24> dev eno1) and then conneting to console I did the second comand (ovs-vsctl add-port br-ex eno1 and also changed the IP address of br-ex to 10.48.146.77)
ip addr del 10.48.146.77/24<http://10.48.146.77/24> dev eno1; ovs-vsctl add-port br-ex eno1
Then after I removed all the routers, networks and interfaces from openstack and ran the following 4 commands.
1) source /root/keystonerc_admin
2) neutron net-create public --router:external=True
3) neutron subnet-create public 10.48.146.0/24<http://10.48.146.0/24> --name vlan --enable_dhcp=False --allocation_pool start=10.48.146.113,end=10.48.146.130 --gateway 10.48.146.1
4) neutron router-create router1
5) neutron router-gateway-set 7e07fef0-e92b-4956-9b23-82157d81ebf3 1605ac0b-4f59-473d-b20f-911c3401dc2c
6) neutron floatingip-create public
The above command generated a floating IP 10.48.146.115
Then I spin up a new instance and it got IP assigned automatically 10.48.146.115, howevr I cannot ping or ssh to that ip : 10.48.146.115 ?
Here are the screenshots
[cid:964628AC-2686-40F5-B655-00EAF252B65D]
[cid:2B55990D-8C1A-445B-8C55-BF8428FCA0F2]
[root at lspal117 ~(keystone_admin)]# neutron net-list
+--------------------------------------+--------+-----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+--------+-----------------------------------------------------+
| 1605ac0b-4f59-473d-b20f-911c3401dc2c | public | 24f385ad-bfeb-4c62-91ff-ab8756d3983b 10.48.146.0/24<http://10.48.146.0/24> |
+--------------------------------------+--------+——————————————————————————+
[root at lspal117 ~(keystone_admin)]# neutron subnet-list
+--------------------------------------+------+----------------+----------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+------+----------------+----------------------------------------------------+
| 24f385ad-bfeb-4c62-91ff-ab8756d3983b | vlan | 10.48.146.0/24<http://10.48.146.0/24> | {"start": "10.48.146.113", "end": "10.48.146.130"} |
+--------------------------------------+------+----------------+----------------------------------------------------+
[root at lspal117 ~(keystone_admin)]# neutron router-list
+--------------------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| id | name | external_gateway_info | distributed | ha |
+--------------------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| 7e07fef0-e92b-4956-9b23-82157d81ebf3 | router1 | {"network_id": "1605ac0b-4f59-473d-b20f-911c3401dc2c", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "24f385ad-bfeb-4c62-91ff-ab8756d3983b", "ip_address": "10.48.146.113"}]} | False | False |
+--------------------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
Any ideas what is going wrong, I still can’t ping or ssh to that IP address and its not listening anywhere ?
Best Regards,
Rushi.
Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race
From: Marius Cornea <marius at remote-lab.net<mailto:marius at remote-lab.net>>
Date: Friday, January 9, 2015 at 2:30 PM
To: SAP SAP <rushi.ns at sap.com<mailto:rushi.ns at sap.com>>, rdo-list <rdo-list at redhat.com<mailto:rdo-list at redhat.com>>
Subject: Re: [Rdo-list] networking issues on JUNO redhead openstack.
As Kashyap kindly suggested we're not a support line and information here should be available for all the community members. First, I don't see how you can ping the br-ex IP address from an external network since it doesn't contain any physical interface. Make sure that 10.48.146.112 isn't used by other host on you network segment. You can do the following to remove the ip address from the physical interface and add it to the br-ex bridge. Please make sure that you've got console access to the server before doing this as you may lose connectivity.
# ip addr del 10.48.146.77/24<http://10.48.146.77/24> dev eno1; ovs-vsctl add-port br-ex eno1
You should then follow the documentation[1] to recreate the public/tenant networks and router. In your case FLOATING_IP_START, FLOATING_IP_END should be part of the 10.48.146.0/24<http://10.48.146.0/24> network. Let's say you wish to allocate IP addresses of the 10.48.146.200 - 10.48.146.210 range to your Openstack routers/instances, then FLOATING_IP_START=10.48.146.200 and FLOATING_IP_END=10.48.146.210. EXTERNAL_NETWORK_GATEWAY should be set to what 'ip r | grep default | awk {'print $3'}' outputs and EXTERNAL_NETWORK_CIDR is 10.48.146.0/24<http://10.48.146.0/24>.
Note that the instance will get an IP address of the tenant network. By assigning a floating IP you will be able to access it through an IP address from your physical network segment.
[1] http://docs.openstack.org/juno/install-guide/install/yum/content/neutron_initial-external-network.html
On Fri, Jan 9, 2015 at 10:52 PM, Ns, Rushi <rushi.ns at sap.com<mailto:rushi.ns at sap.com>> wrote:
Hi Marius,
Here are the outputs. Is it possible to reach you on phone so that I can explain more than email. You can call me at 650 849 3915<tel:650%20849%203915> or let me know your number.
#ip a
[root at lspal117 network-scripts(keystone_admin)]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8<http://127.0.0.1/8> scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether d8:9d:67:29:c6:94 brd ff:ff:ff:ff:ff:ff
inet 10.48.146.77/24<http://10.48.146.77/24> brd 10.48.146.255 scope global eno1
valid_lft forever preferred_lft forever
inet6 fe80::da9d:67ff:fe29:c694/64 scope link
valid_lft forever preferred_lft forever
3: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether d8:9d:67:29:c6:95 brd ff:ff:ff:ff:ff:ff
4: eno3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether d8:9d:67:29:c6:96 brd ff:ff:ff:ff:ff:ff
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether d8:9d:67:29:c6:97 brd ff:ff:ff:ff:ff:ff
6: ens1f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 38:ea:a7:8f:bf:b4 brd ff:ff:ff:ff:ff:ff
7: ens1f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 38:ea:a7:8f:bf:b5 brd ff:ff:ff:ff:ff:ff
8: ens2f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 38:ea:a7:8f:bf:40 brd ff:ff:ff:ff:ff:ff
9: ens2f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 38:ea:a7:8f:bf:41 brd ff:ff:ff:ff:ff:ff
10: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether 16:02:ac:3f:05:f4 brd ff:ff:ff:ff:ff:ff
11: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether be:8b:24:03:a9:47 brd ff:ff:ff:ff:ff:ff
inet 10.48.146.112/24<http://10.48.146.112/24> brd 10.48.146.255 scope global br-ex
valid_lft forever preferred_lft forever
inet6 fe80::bc8b:24ff:fe03:a947/64 scope link
valid_lft forever preferred_lft forever
12: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether 02:35:a4:7c:e5:4c brd ff:ff:ff:ff:ff:ff
16: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether fe:61:30:fb:a3:47 brd ff:ff:ff:ff:ff:ff
17: qbr63bd6704-47: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 8e:fe:6e:19:61:5c brd ff:ff:ff:ff:ff:ff
inet6 fe80::3822:bcff:fe6a:b74f/64 scope link
valid_lft forever preferred_lft forever
18: qvo63bd6704-47: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP qlen 1000
link/ether a2:fe:04:5f:04:b4 brd ff:ff:ff:ff:ff:ff
19: qvb63bd6704-47: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbr63bd6704-47 state UP qlen 1000
link/ether 8e:fe:6e:19:61:5c brd ff:ff:ff:ff:ff:ff
22: tap63bd6704-47: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbr63bd6704-47 state UNKNOWN qlen 500
link/ether fe:16:3e:d9:5e:ba brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fed9:5eba/64 scope link
valid_lft forever preferred_lft forever
# ovs-vsctl show
[root at lspal117 network-scripts(keystone_admin)]# ovs-vsctl show
c5bcc161-86cd-4004-9729-c1ba05d28569
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Port "qg-c3a1b534-36"
Interface "qg-c3a1b534-36"
type: internal
Bridge br-int
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tape386c0cb-d8"
tag: 1
Interface "tape386c0cb-d8"
type: internal
Port br-int
Interface br-int
type: internal
Port "qvo63bd6704-47"
tag: 1
Interface "qvo63bd6704-47"
Port "tap44dd465f-d4"
tag: 4095
Interface "tap44dd465f-d4"
type: internal
Port "qr-e0ac3c25-2b"
tag: 2
Interface "qr-e0ac3c25-2b"
type: internal
Bridge br-tun
Port br-tun
Interface br-tun
type: internal
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
ovs_version: “2.1.3"
# neutron net-list
[root at lspal117 network-scripts(keystone_admin)]# neutron net-list
+--------------------------------------+---------+------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+---------+------------------------------------------------------+
| 5abe660b-52da-4d36-8275-4c1488df88b4 | public | 79ee8c72-4c60-4b39-9b53-e6a3e9591e6a 172.24.4.224/28<http://172.24.4.224/28> |
| d825e820-66e5-44b1-a91d-0c660aa822a5 | private | f0fc905d-ad3b-4ba6-8094-bf4dbbc55cd9 10.0.0.0/24<http://10.0.0.0/24> |
| 6eb6d123-8fec-4984-986a-5eeabf895883 | sap | ea2b21e0-bf9f-4195-b210-b75b6f9f7943 10.48.146.0/24<http://10.48.146.0/24> |
+--------------------------------------+---------+------------------------------------------------------+
# neutron subnet-list
[root at lspal117 network-scripts(keystone_admin)]# neutron subnet-list
+--------------------------------------+----------------+-----------------+----------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+----------------+-----------------+----------------------------------------------------+
| 79ee8c72-4c60-4b39-9b53-e6a3e9591e6a | public_subnet | 172.24.4.224/28<http://172.24.4.224/28> | {"start": "172.24.4.226", "end": "172.24.4.238"} |
| f0fc905d-ad3b-4ba6-8094-bf4dbbc55cd9 | private_subnet | 10.0.0.0/24<http://10.0.0.0/24> | {"start": "10.0.0.2", "end": "10.0.0.254"} |
| ea2b21e0-bf9f-4195-b210-b75b6f9f7943 | sap_local | 10.48.146.0/24<http://10.48.146.0/24> | {"start": "10.48.146.112", "end": "10.48.146.120"} |
+--------------------------------------+----------------+-----------------+——————————————————————————+
#neutron router-list
[root at lspal117 network-scripts(keystone_admin)]# neutron router-list
+--------------------------------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| id | name | external_gateway_info | distributed | ha |
+--------------------------------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| d6325c88-e5b0-448e-8ab1-af5908a189d4 | router1 | {"network_id": "5abe660b-52da-4d36-8275-4c1488df88b4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "79ee8c72-4c60-4b39-9b53-e6a3e9591e6a", "ip_address": "172.24.4.226"}]} | False | False |
+--------------------------------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+———+
In my system I have IP address for BR-ex and ENO1 (interface connecting ), should I disable eno1 as I can ping both (br-ex and eno1) from my network to reach to this host ? Probably its a dump question.
Here are the outputs.
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.48.146.112 netmask 255.255.255.0 broadcast 10.48.146.255
inet6 fe80::bc8b:24ff:fe03:a947 prefixlen 64 scopeid 0x20<link>
ether be:8b:24:03:a9:47 txqueuelen 0 (Ethernet)
RX packets 10 bytes 864 (864.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 25 bytes 3767 (3.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br-int: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether 02:35:a4:7c:e5:4c txqueuelen 0 (Ethernet)
RX packets 135 bytes 7574 (7.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br-tun: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether fe:61:30:fb:a3:47 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.48.146.77 netmask 255.255.255.0 broadcast 10.48.146.255
inet6 fe80::da9d:67ff:fe29:c694 prefixlen 64 scopeid 0x20<link>
ether d8:9d:67:29:c6:94 txqueuelen 1000 (Ethernet)
RX packets 467319 bytes 48391354 (46.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 14794 bytes 5238733 (4.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 32
Best Regards,
Rushi.
Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race
From: Marius Cornea <marius at remote-lab.net<mailto:marius at remote-lab.net>>
Date: Friday, January 9, 2015 at 1:46 PM
To: SAP SAP <rushi.ns at sap.com<mailto:rushi.ns at sap.com>>
Subject: Re: [Rdo-list] networking issues on JUNO redhead openstack.
Hi Rushi,
Can you provide the output of the following commands on your node ?
# ip a
# ovs-vsctl show
# neutron net-list
# neutron subnet-list
# neutron router-list
Please note that only the br-ex bridge requires an IP address, the interface that's part of it shouldn't have one assigned. Also the floating IP addresses will be used for DNAT on the tenant router. You can check the documentation here[1] that describes the use of floating IPs.
You can also check the upstream documentation[2] on how to create the networks.
[1] https://openstack.redhat.com/Difference_between_Floating_IP_and_private_IP
[2] http://docs.openstack.org/juno/install-guide/install/yum/content/neutron_initial-external-network.html
Tnx,
Marius
On Fri, Jan 9, 2015 at 10:22 PM, Ns, Rushi <rushi.ns at sap.com<mailto:rushi.ns at sap.com>> wrote:
Hi Support,
I have a problem that a launched instance from openstack cannot connect to SSH or PING
First of all I have dump question whether my system should be NOVA network or Neutron network. Well to keep everyone on the same page. Here is my history.
1. I did install RDO (allinone host) using the bellow steps https://openstack.redhat.com/Quickstart
2. Then I have used the following steps for floating IP https://openstack.redhat.com/Floating_IP_range.
My situtation is as follows.
* Floating IP address assignment:
I have assigned a floating IP , I did follow this steps.
https://openstack.redhat.com/Floating_IP_range
* Bridge configuration:My Ethernet device eth0(actually in Redhat 7 it is eno1 name)is on the same bridge as the compute host
Yes, ethernet device on 10.48.146.78 address and br-ex interface on 10.48.146.112
* DHCP configuration: Yes, DHCP configuration is set and Neutron hearing and responding to DHCP
requests and the instance is getting IP from 10.48.146.113 (next hop) however I cannot ping/ssh to that IP address (ICMP/SSH is added allowed
rule on default security group)
Here is the little diagram of my system.
[cid:EB813931-0B05-459C-A0B6-DE7C98D3D2FA]
Can someone help to find what is missing.
Best Regards,
Rushi.
Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race
_______________________________________________
Rdo-list mailing list
Rdo-list at redhat.com<mailto:Rdo-list at redhat.com>
https://www.redhat.com/mailman/listinfo/rdo-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20150110/7ec3c4e6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CD828843-4C9C-442E-9F18-65F4084FCB55.png
Type: image/png
Size: 44700 bytes
Desc: CD828843-4C9C-442E-9F18-65F4084FCB55.png
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20150110/7ec3c4e6/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EB813931-0B05-459C-A0B6-DE7C98D3D2FA.jpg
Type: image/jpeg
Size: 31140 bytes
Desc: EB813931-0B05-459C-A0B6-DE7C98D3D2FA.jpg
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20150110/7ec3c4e6/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2B55990D-8C1A-445B-8C55-BF8428FCA0F2.png
Type: image/png
Size: 35381 bytes
Desc: 2B55990D-8C1A-445B-8C55-BF8428FCA0F2.png
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20150110/7ec3c4e6/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 964628AC-2686-40F5-B655-00EAF252B65D.png
Type: image/png
Size: 73280 bytes
Desc: 964628AC-2686-40F5-B655-00EAF252B65D.png
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20150110/7ec3c4e6/attachment-0002.png>
More information about the dev
mailing list