[Rdo-list] I can't get access to VM instances
Dan Sneddon
dsneddon at redhat.com
Sun Feb 22 08:03:39 UTC 2015
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 02/21/2015 01:27 PM, Pasquale Salza wrote:
> I have a question. If I want to add any public network, do I need
> to statically assign every compute node to the same network on one
> of the interfaces? I mean, in order to access to VMs which have the
> floating IP on that network.
>
> For example, having the VMs on 172.16.58.0/24
> <http://172.16.58.0/24> external network and compute nodes with
> interfaces assigned with different networks.
>
> Il 21/feb/2015 21:34 "Dan Sneddon" <dsneddon at redhat.com
> <mailto:dsneddon at redhat.com>> ha scritto:
>
> On 02/21/2015 12:14 AM, Pasquale Salza wrote:
>> Thank you! Yes you were right, I meant to chose 6 VMs and give
>> them 6 IPs. I forgot the router IP.
>
>> Is there any problem in not giving direct internet access to
>> machines, but using IP forwarding on controller?
>
>> Il 21/feb/2015 01:35 "Dan Sneddon" <dsneddon at redhat.com
> <mailto:dsneddon at redhat.com>
>> <mailto:dsneddon at redhat.com <mailto:dsneddon at redhat.com>>> ha
>> scritto:
>
>> On 02/20/2015 03:29 PM, Pasquale Salza wrote:
>>> Whops! I figured out just few seconds after I sent the mail!
>>> Ok, tomorrow I'll try with it. :) I'd like to share how I want
>>> to organise my network in order to get some advices.
>
>>> Let's say I have 7 machines and 7 spare IPs on the network
>>> 172.16.58.0/24 <http://172.16.58.0/24> <http://172.16.58.0/24>
> <http://172.16.58.0/24>
>> which are also associated to
>>> 7 public (internet) IPs.
>
>>> I'd like to reserve 6 IPs for 6 VMs I could instanciate on
>>> OpenStack.
>
>>> So I planned to do this: the controller node has a static IP
>>> on eth0 of the 7 in 172.16.58.50/24 <http://172.16.58.50/24>
> <http://172.16.58.50/24>
>> <http://172.16.58.50/24> network
>>> so as I can access it from outside. I add an alias eth0:0 with
>>> which I connect the controller to the Management network of
>>> OpenStack, the 10.0.1.0/24 <http://10.0.1.0/24>
>>> <http://10.0.1.0/24>
>> <http://10.0.1.0/24> network. Also on
>>> the controller, I set statically the IP for eth1 with one of
>>> float IPs network 192.168.0.0/16 <http://192.168.0.0/16>
> <http://192.168.0.0/16>
>> <http://192.168.0.0/16> network. With
>>> iptables, I add the rule of forwarding everithing on eth0 and
>>> eth1, so the other nodes can get Internet access on network
>>> 10.0.1.0/24 <http://10.0.1.0/24> <http://10.0.1.0/24>
> <http://10.0.1.0/24>.
>
>>> On the compute nodes I set eth0 as one of IPs on 10.0.1.0/24
> <http://10.0.1.0/24>
>> <http://10.0.1.0/24>
>>> <http://10.0.1.0/24> management network and eth1 as one on
>>> 192.168.0.0/16 <http://192.168.0.0/16> <http://192.168.0.0/16>
> <http://192.168.0.0/16>.
>
>>> Om each node I put the bridge on eth1.
>
>>> With RDO I put virtualisation and tunneling only on eth1.
>
>>> When the installatation has finished, I create a private
>>> neutron network 10.100.0.0/16 <http://10.100.0.0/16>
>>> <http://10.100.0.0/16>
>> <http://10.100.0.0/16> and two public
>>> networks of floating IPs. The first is 192.168.0.0/24
> <http://192.168.0.0/24>
>> <http://192.168.0.0/24>
>>> <http://192.168.0.0/24> for any kind of VM. The other is the
>>> 172.16.58.0/24 <http://172.16.58.0/24> <http://172.16.58.0/24>
> <http://172.16.58.0/24>
>> network, limited to the 6
>>> available IPs with which I can put virtual machines on
>>> Internet.
>
>>> Does it make sense or I'm doing some mistakes? Do you have any
>>> other idea?
>
>>> Thank you very much indeed!
>
>>> Pasquale
>
>>> On 02/20/2015 02:07 PM, Pasquale Salza wrote:
>>>> Hi Rhys, I suppose so, because these are my iptables rules:
>
>>>> iptables -F iptables -t nat -F iptables -P INPUT ACCEPT
>>>> iptables -P OUTPUT ACCEPT iptables -P FORWARD ACCEPT
>>>> iptables -A INPUT -d 172.16.58.0/24 <http://172.16.58.0/24>
> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>>> -m
>>> state --state
>>>> ESTABLISHED,RELATED -j ACCEPT iptables -A INPUT -d
>>>> 172.16.58.0/24 <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24> -p tcp --dport ssh -j ACCEPT
>>>> iptables -A INPUT -d 172.16.58.0/24 <http://172.16.58.0/24>
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24> -p tcp --dport www
>>>> -j ACCEPT iptables -A INPUT -d 172.16.58.0/24
> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24> -p tcp --dport pptp -j ACCEPT
>>>> iptables -A INPUT -d 172.16.58.0/24 <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24> -p tcp --sport
>>>> domain -j ACCEPT iptables -A INPUT -d 172.16.58.0/24
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24> -p tcp --dport domain -j ACCEPT
>>>> iptables -A INPUT -d 172.16.58.0/24 <http://172.16.58.0/24>
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24> -p udp --sport
>>>> domain -j ACCEPT iptables -A INPUT -d 172.16.58.0/24
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24> -p udp --dport domain -j ACCEPT
>>>> iptables -A INPUT -d 172.16.58.0/24 <http://172.16.58.0/24>
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24> -p gre -j ACCEPT
>>>> iptables -A INPUT -d 172.16.58.0/24 <http://172.16.58.0/24>
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24> -p icmp
>>>> -j ACCEPT iptables -A INPUT -d 172.16.58.0/24
> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24> -j DROP iptables -t nat -A
>>>> POSTROUTING -o eth0 -j MASQUERADE service iptables save
>
>>>> Firstly, do you think I planned the network organisation
>>>> well? Do you have other suggestion (best practices) with 2
>>>> interfaces?
>
>
>>>> 2015-02-20 18:30 GMT+01:00 Rhys Oxenham <roxenham at redhat.com
> <mailto:roxenham at redhat.com>
>> <mailto:roxenham at redhat.com <mailto:roxenham at redhat.com>>
>>> <mailto:roxenham at redhat.com <mailto:roxenham at redhat.com>
> <mailto:roxenham at redhat.com <mailto:roxenham at redhat.com>>>
>>>> <mailto:roxenham at redhat.com <mailto:roxenham at redhat.com>
> <mailto:roxenham at redhat.com <mailto:roxenham at redhat.com>>
>> <mailto:roxenham at redhat.com <mailto:roxenham at redhat.com>
> <mailto:roxenham at redhat.com <mailto:roxenham at redhat.com>>>>>:
>
>>>> Hi Pasquale,
>
>>>> Did you modify your security group rules to allow ICMP
>>>> and/or 22:tcp access?
>
>>>> Many thanks Rhys
>
>>>>> On 20 Feb 2015, at 17:11, Pasquale Salza
>>>>> <pasquale.salza at gmail.com
>>>>> <mailto:pasquale.salza at gmail.com>
> <mailto:pasquale.salza at gmail.com
> <mailto:pasquale.salza at gmail.com>>
>> <mailto:pasquale.salza at gmail.com
>> <mailto:pasquale.salza at gmail.com>
>> <mailto:pasquale.salza at gmail.com
>> <mailto:pasquale.salza at gmail.com>>>
>>>> <mailto:pasquale.salza at gmail.com
>>>> <mailto:pasquale.salza at gmail.com>
>>>> <mailto:pasquale.salza at gmail.com
>>>> <mailto:pasquale.salza at gmail.com>>
>>>> <mailto:pasquale.salza at gmail.com
>>>> <mailto:pasquale.salza at gmail.com>
>>>> <mailto:pasquale.salza at gmail.com
> <mailto:pasquale.salza at gmail.com>>>>>
>>> wrote:
>>>>>
>>>>> Hi there, I have a lot of problems with RDO/OpenStack
>>>> configuration. Firstly, I need to describe my network
>>>> situation.
>>>>>
>>>>> I have 7 machine, each of them with 2 NIC. I would like to
>>>>> use one
>>>> machine as a controller/network node and the others as
>>>> compute nodes.
>>>>>
>>>>> I would like to use the eth0 to connect nodes to internet
>>>>> (and get
>>>> access by remote sessions) with the network "172.16.58.0/24
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24>
>>> <http://172.16.58.0/24>
>>>> <http://172.16.58.0/24>", in which I have just 7 available
>>>> IPs, and eth1 as configuration network on the network
>>>> 10.42.100.0/42 <http://10.42.100.0/42>
>> <http://10.42.100.0/42>
>>> <http://10.42.100.0/42>
>>>> <http://10.42.100.0/42>.
>>>>>
>>>>> This is my current configuration, for each node (varying
>>>>> the IPs
>>>> on each machine):
>>>>>
>>>>> eth0: DEVICE=eth0 TYPE=Ethernet ONBOOT=yes
>>>>> BOOTPROTO=static IPADDR=172.16.58.50 NETMASK=255.255.255.0
>>>>> GATEWAY=172.16.58.254 DNS1=172.16.58.50 DOMAIN=###
>>>>> DEFROUTE="yes"
>>>>>
>>>>> eth1: DEVICE=eth1 TYPE=OVSPort DEVICETYPE=ovs
>>>>> OVS_BRIDGE=br-ex ONBOOT=yes
>>>>>
>>>>> br-ex: DEVICE=br-ex DEVICETYPE=ovs TYPE=OVSBridge
>>>>> BOOTPROTO=static IPADDR=10.42.100.1 NETMASK=255.255.255.0
>>>>> ONBOOT=yes
>>>>>
>>>>> I'd like to have instances on 10.42.200.0/24
> <http://10.42.200.0/24>
>>>>> <http://10.42.200.0/24> <http://10.42.200.0/24>
>>>> <http://10.42.200.0/24> virtual private network and the
>>>> remaining IPs of 10.42.100.0/24 <http://10.42.100.0/24>
> <http://10.42.100.0/24>
>> <http://10.42.100.0/24>
>>>> <http://10.42.100.0/24>
>>> network as floating
>>>> IPs.
>>>>>
>>>>> These are the relevant parts of my answers.txt file:
>>>>>
>>>>> CONFIG_CONTROLLER_HOST=10.42.100.1
>>>>>
>
>
>
> CONFIG_COMPUTE_HOSTS=10.42.100.10,10.42.100.11,10.42.100.12,10.42.100.13,10.42.100.14,10.42.100.15
>
>
>
>
>>>> CONFIG_NETWORK_HOSTS=10.42.100.1
>>>>> CONFIG_AMQP_HOST=10.42.100.1
>>>>> CONFIG_MARIADB_HOST=10.42.100.1
>>>>> CONFIG_NOVA_COMPUTE_PRIVIF=eth1
>>>>> CONFIG_NOVA_NETWORK_PUBIF=eth1
>>>>> CONFIG_NOVA_NETWORK_PRIVIF=eth1
>>>>> CONFIG_NOVA_NETWORK_FIXEDRANGE=10.42.200.0/24
> <http://10.42.200.0/24>
>> <http://10.42.200.0/24>
>>>>> <http://10.42.200.0/24>
>>>> <http://10.42.200.0/24>
>>>>> CONFIG_NOVA_NETWORK_FLOATRANGE=10.42.100.0/24
> <http://10.42.100.0/24>
>> <http://10.42.100.0/24>
>>>>> <http://10.42.100.0/24>
>>>> <http://10.42.100.0/24>
>>>>> CONFIG_NEUTRON_L3_EXT_BRIDGE=br-ex
>>>>> CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vxlan
>>>>> CONFIG_NEUTRON_ML2_TENANT_NETWORK_TYPES=vxlan
>>>>> CONFIG_NEUTRON_ML2_VNI_RANGES=10:100
>>>>> CONFIG_NEUTRON_LB_INTERFACE_MAPPINGS=
>>>>> CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=
>>>>> CONFIG_NEUTRON_OVS_BRIDGE_IFACES=
>>>>> CONFIG_NEUTRON_OVS_TUNNEL_IF=eth1
>>>>>
>>>>> After the installation, I configure the network like this:
>>>>>
>>>>> neutron router-create router neutron net-create private
>>>>> neutron subnet-create private 10.42.200.0/24
> <http://10.42.200.0/24>
>>>>> <http://10.42.200.0/24> <http://10.42.200.0/24>
>>>> <http://10.42.200.0/24> --name private-subnet
>>>>> neutron router-interface-add router private-subnet neutron
>>>>> net-create public --router:external=True neutron
>>>>> subnet-create public 10.42.100.0/24
>>>>> <http://10.42.100.0/24>
> <http://10.42.100.0/24>
>> <http://10.42.100.0/24>
>>>> <http://10.42.100.0/24> --name public-subnet
>>>> --enable_dhcp=False --allocation-pool
>>>> start=10.42.100.100,end=10.42.100.200 --no-gateway
>>>>> neutron router-gateway-set router public
>>>>>
>>>>> I'm able to launch instances but I can't get access
>>>>> (ping/ssh) to
>>>> them.
>>>>>
>>>>> I don't know if I'm doing something wrong starting from
>>>>> planning.
>>>>>
>>>>> Please, help me!
>>>>>
>>>>> _______________________________________________ Rdo-list
>>>>> mailing list Rdo-list at redhat.com
>>>>> <mailto:Rdo-list at redhat.com> <mailto:Rdo-list at redhat.com
>>>>> <mailto:Rdo-list at redhat.com>>
>> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>>>
>>> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>>
>> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>>>>
>>>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>>>
>>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>
>>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>>
>>>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>
>>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>>>
>
>
>
>
>>>> -- Pasquale Salza
>
>>>> e-mail: pasquale.salza at gmail.com
>>>> <mailto:pasquale.salza at gmail.com>
>>>> <mailto:pasquale.salza at gmail.com
>>>> <mailto:pasquale.salza at gmail.com>>
>>>> <mailto:pasquale.salza at gmail.com
>>>> <mailto:pasquale.salza at gmail.com>
>>>> <mailto:pasquale.salza at gmail.com
>>>> <mailto:pasquale.salza at gmail.com>>>
>>> <mailto:pasquale.salza at gmail.com
>>> <mailto:pasquale.salza at gmail.com>
>>> <mailto:pasquale.salza at gmail.com
>>> <mailto:pasquale.salza at gmail.com>>
>>> <mailto:pasquale.salza at gmail.com
>>> <mailto:pasquale.salza at gmail.com>
>>> <mailto:pasquale.salza at gmail.com
>>> <mailto:pasquale.salza at gmail.com>>>>
>>>> phone: +39 393 4415978 <tel:%2B39%20393%204415978>
> <tel:%2B39%20393%204415978>
>> <tel:%2B39%20393%204415978> fax: +39 089
>>> 8422939 <tel:%2B39%20089%208422939> skype: pasquale.salza
>>>> linkedin: http://it.linkedin.com/in/psalza/
>
>
>>>> _______________________________________________ Rdo-list
>>>> mailing list Rdo-list at redhat.com
>>>> <mailto:Rdo-list at redhat.com>
> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>>
>> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>>>
>>>> https://www.redhat.com/mailman/listinfo/rdo-list
>
>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>
>>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>>
>
>
>>> Those look like the iptables rule on the hypervisor. Rhys is
>>> talking about the Neutron security group rules. By default,
>>> ssh into VMs is not allowed. You need to permit ICMP and SSH in
>>> the security rules on the neutron network.
>
>>> I don't see anything wrong with your network architecture at
>>> first glance, but floating IPs can be tricky at first. Start
>>> with basic VM-to-VM connectivity and add on from there.
>
>>> Good luck!
>
>
>>> _______________________________________________ Rdo-list
>>> mailing list Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>>
>> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>
> <mailto:Rdo-list at redhat.com <mailto:Rdo-list at redhat.com>>>
>>> https://www.redhat.com/mailman/listinfo/rdo-list
>
>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>
>>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>> <mailto:rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>>>
>
>> That sounds like it should work, but one of those 6 IP addresses
>> will need to be used for the Neutron router (that IP will be
>> used for SNAT for VMs that have no floating IP).
>
>> I'm not sure what you mean when you say "I'd like to reserve 6
>> IPs for 6 VMs I could instanciate on OpenStack." You can
>> instantiate more than one VM on each compute node, and if you
>> have 6 compute nodes then depending on size you could have dozens
>> of VMs. Maybe you just mean you could instantiate 6 VMs with
>> public IPs? Actually, due to the router IP, you would be limited
>> to 5.
>
>> Make sure you add the floating IP network as an external net.
>> Since your router will not be taking the .1 address, you will
>> need to create the port by hand with the chosen IP and add it to
>> the router.
>
>> $ neutron net-create externalnet -- --router:external=True $
>> neutron subnet-create externalnet 172.16.58.0/24
> <http://172.16.58.0/24>
>> <http://172.16.58.0/24> --name external \ --enable_dhcp=False
>> --allocation_pool start=172.16.58.x,\ end=172.16.58.x --gateway
>> 172.16.58.x (use your network gateway here - change the IP
>> addresses in the allocation range to match what is available on
>> your network) $ neutron router-create extrouter (name of your
>> router) $ neutron port-create externalnet --fixed-ip 172.16.58.x
>> (use desired router IP) $ neutron router-interface-add extrouter
>> port=$portid (port id from previous command) $ neutron
>> router-interface-add extrouter subnet=public (replace public
>> with the name of the 192.168.0.0/24 <http://192.168.0.0/24>
> <http://192.168.0.0/24> network)
>
>> Once that is done, you should be able to assign a floating IP to
>> any VM that has an interface on the 192.168.0.0/24
> <http://192.168.0.0/24>
>> <http://192.168.0.0/24> network.
>
>> P.S. - Several times in your email you mentioned 192.168.0.0/16
> <http://192.168.0.0/16>
>> <http://192.168.0.0/16>, but that's not a valid network. I
>> assume you mean 192.168.0.0/24 <http://192.168.0.0/24>
> <http://192.168.0.0/24>.
>
>
>
> That depends what you are trying to do. There are plenty of
> reasons why it might not work at first. You may need to
> troubleshoot.
>
> One issue that might come up is that you will be doing multiple
> levels of NAT. Some protocols won't work with multiple layers of
> translation.
>
> If your goal is to eventually make these VMs reachable from the
> Internet, there are a lot of factors in play above the OpenStack
> cloud.
>
>
No, the external network is only attached to the Neutron controller.
The public IP actually lives on the l3agent, which runs the router you
created and attached to that network. When traffic goes back and forth
from outside, the l3agent does source NAT and swaps the public IP with
the VM IP. The controller isn't actually attached to the external network.
In general, the only IPs in use on the External network are the IP you
assign to the router attached to the External network, the upstream
gateway router, and the floating IPs handled by Neutron.
If a VM doesn't have a floating IP, the Neutron router will use its
own IP address for the NAT. That Internet access is outbound-only.
- --
Dan Sneddon | Principal OpenStack Engineer
dsneddon at redhat.com | redhat.com/openstack
650.254.4025 | @dxs on twitter
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBAgAGBQJU6Y1bAAoJEFkV3ypsGNbjTIMH/iajE5q30wfCKcghkWaTu0AW
VckXJyPSdtucrewUb+oUriGFx3OPMZU1hnGxCYqDTjsj/iTx3JsSFzCozmKzdXAY
hWEO/nNmD4lWljWghjTac13t+6rhM5lJVA3posQoZEPWwyrdh6bmcHwCM93HYZ3H
QYaXv7RKasSool6Kq9MxOyRq2+O0DvmVWk8BOKHzy2ZnP1OrRjhotSRIRIh1O3Ti
3PEYZJ+QZOzxAMfWDWcRjNONuGscaIVvPxrU5/i6jH5FK1ymJarIRJmVPO1a58BW
cYEcsuz/L6wYhaYthRCY14EkLQ7bsSTT4JMse68s0/u3WgQPyjZOR2NBk6QAAu8=
=0N5i
-----END PGP SIGNATURE-----
More information about the dev
mailing list