July 2014 - dev - Rdo List Archives

by John Eckersberg

(In the spirit of "Concerning Hobbits") Ryan O'Hara and I have been investigating RabbitMQ as it pertains to RDO recently. There has been a lot of discussion on several disparate threads, so I wanted to try and capture it on the list for the benefit of everyone. Ryan has been working on getting RabbitMQ running in a multi-node HA configuration. I won't steal his thunder, and he can speak to it better than I can, so I'll defer to him on the details. As for me, I've been working on el7 support and bug squashing along the way. The first bug[1] causes the daemon to load incredibly slow, or outright fail by timing out. This is due to the SELinux policy disallowing name_bind on ports lower than 32768. RabbitMQ tries to name_bind to a port starting at 10000, and increments if it fails. So if you have SELinux in enforcing mode, you'll get 22768 AVC denials in the log before it finally starts. The second bug[2] causes the daemon to intermittently fail to start due to a race condition in the creation of the erlang cookie file. This happens only the first time the service starts. Really this is an Erlang bug, but there's a workaround for the RabbitMQ case. I've submitted patches for both issues. Until those get merged in, I've rebuilt[3] RabbitMQ for F20 which includes the fixes. Beyond bugs, I've also built out RabbitMQ and all the build/runtime dependencies for el7. I have a yum repo[4] on my fedorapeople page containing all the bits. This is all the stuff that is presently missing from EPEL7. In time, I would hope the maintainers build all this stuff, but for now it'll work for testing. You will also need the EPEL 7 Beta repository[5] enabled. As a side note, I built everything using mock with a local override repo on my workstation. I've not used copr before but it seems relevant to this sort of thing, so if it's any benefit I'll look to rebuilt the el7 stack there for easier consumption. Hopefully this helps get the discussion into one place, and provide a baseline for further investigation by everyone interested in RabbitMQ. John. --- [1] Is really two bugzillas, but the same bug: [1a] https://bugzilla.redhat.com/show_bug.cgi?id=998682 [1b] https://bugzilla.redhat.com/show_bug.cgi?id=1032595 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1059913 [3] http://jeckersb.fedorapeople.org/rabbitmq-server-3.1.5-3.fc20.noarch.rpm [4] http://jeckersb.fedorapeople.org/rabbitmq-el7/ [5] http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/

9 years, 6 months

6
11
0 / 0

[Rdo-list] mysqld failure on --allinone, centos7

by Rich Bowen

I'm running `packstack --allinone` on a fresh install of the new CentOS7, and I'm getting a failure at: 192.168.0.176_mysql.pp: [ ERROR ] Applying Puppet manifests [ ERROR ] ERROR : Error appeared during Puppet run: 192.168.0.176_mysql.pp Error: Could not enable mysqld: You will find full trace in log /var/tmp/packstack/20140708-092703-ZMkytw/manifests/192.168.0.176_mysql.pp.log Please check log file /var/tmp/packstack/20140708-092703-ZMkytw/openstack-setup.log for more information The log message is: Notice: /Stage[main]/Neutron::Db::Mysql/Mysql::Db[neutron]/Database_grant[neutron@127.0.0.1/neutron]: Dependency Service[mysqld] has failures: true Warning: /Stage[main]/Neutron::Db::Mysql/Mysql::Db[neutron]/Database_grant[neutron@127.0.0.1/neutron]: Skipping because of failed dependencies mysqld was successfully installed, and is running. Before I start digging deeper, I wondered if this is something that's already been encountered. Thanks. -- Rich Bowen - rbowen(a)redhat.com OpenStack Community Liaison http://openstack.redhat.com/

9 years, 7 months

6
13
0 / 0

[Rdo-list] foreman-installer puppet problems: " Error 400 on SERVER: Must pass admin_password to Class[Quickstack::Nova]"

by Jonas Hagberg

Hej I have installed foreman-installer from yum repo icehouce el6 on Scientific linux 6.5 I got foreman up and running and got hostgroups and smart parameters. But when assigning a node to a hostgroup (Neutron controller or neutron compute) and running puppet I get the following error. err: Could not retrieve catalog from remote server: Error 400 on SERVER: Must pass admin_password to Class[Quickstack::Nova] at /usr/share/openstack-foreman-installer/puppet/modules/quickstack/manifests/nova.pp:65 on "fqdn" admin_password is set in hostgroup. Any tips on where the problem could be? Cheers -- Jonas Hagberg BILS - Bioinformatics Infrastructure for Life Sciences - http://bils.se e-mail: jonas.hagberg(a)bils.se, jonas.hagberg(a)scilifelab.se phone: +46-(0)70 6683869 address: SciLifeLab, Box 1031, 171 21 Solna, Sweden

9 years, 9 months

2
3
0 / 0

[Rdo-list] Metadata block until restart iptables

by Lakmal Warusawithana

Hi, Configured multi node RDO setup on centos 6.5 with gre tunneling. Instances are spin up correctly but metadata not received to instances. When restart the iptables on compute node, metadata received only instances that spin up in that compute node. Seems like iptables issue, wonder how to figure it out. Is anyone notice this? highly appreciate your help. thanks -- Lakmal Warusawithana

9 years, 9 months

2
1
0 / 0

[Rdo-list] [RDO][Instack] heat is not able to create stack with instack

by Pradeep Kumar Surisetty

Hi All I have been trying to set instack with RDO. I have successfully installed undercloud and moving on to overcloud. Now, when I run "instack-deploy-overcloud", I get the following error: |+ OVERCLOUD_YAML_PATH=overcloud.yaml + heat stack-create-f overcloud.yaml-PAdminToken=b003d63242f5db3e1ad4864ae66911e02ba19bcb-PAdminPassword=7bfe4d4a18280752ad07f259a69a3ed00db2ab44 -PCinderPassword=df0893b4355f3511a6d67538dd592d02d1bc11d3-PGlancePassword=066f65f878157b438a916ccbd44e0b7037ee118f -PHeatPassword=58fda0e4d6708e0164167b11fe6fca6ab6b35ec6 -PNeutronPassword=80853ad029feb77bb7c60d035542f21aa5c24177 -PNovaPassword=331474580be53b78e40c91dfdfc2323578a035e7 -PNeutronPublicInterface=eth0-PSwiftPassword=b0eca57b45ebf3dd5cae071dc3880888fb1d4840-PSwiftHashSuffix=a8d87f3952d6f91da589fbef801bb92141fd1461-PNovaComputeLibvirtType=qemu-P'GlanceLogFile='\'''\''' -PNeutronDnsmasqOptions=dhcp-option-force=26,1400 overcloud +--------------------------------------+------------+--------------------+----------------------+ | id| stack_name| stack_status| creation_time| +--------------------------------------+------------+--------------------+----------------------+ | 0ca028e7-682b-41ef-8af0-b2eb67bee272| overcloud| CREATE_IN_PROGRESS| 2014-07-18T10:50:48Z | +--------------------------------------+------------+--------------------+----------------------+ + tripleo wait_for_stack_ready220 10 overcloud Command output matched'CREATE_FAILED'. Exiting... |Now, i understand that the stack isn't being created. So, I tried to check out the state of the stack: |[stack@localhost~]$ heat stack-list +--------------------------------------+------------+---------------+----------------------+ | id| stack_name| stack_status| creation_time| +--------------------------------------+------------+---------------+----------------------+ | 0ca028e7-682b-41ef-8af0-b2eb67bee272| overcloud| CREATE_FAILED| 2014-07-18T10:50:48Z | +--------------------------------------+------------+---------------+----------------------+ | i even tried to create stack manually, but ended up getting the same error. Update: Here is the heat log: |2014-07-18 06:51:11.884 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back tousing default 2014-07-18 06:51:12.921 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back tousing default 2014-07-18 06:51:16.058 30750 ERROR heat.engine.resource[-] CREATE: Server "SwiftStorage0" [07e42c3d-0f1b-4bb9-b980-ffbb74ac770d] Stack "overcloud" [0ca028e7-682b-41ef-8af0-b2eb67bee272] 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourceTraceback (most recent calllast): 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourceFile "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line420, in _do_action 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourcewhile not check(handle_data): 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourceFile "/usr/lib/python2.7/site-packages/heat/engine/resources/server.py", line545, in check_create_complete 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourcereturn self._check_active(server) 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourceFile "/usr/lib/python2.7/site-packages/heat/engine/resources/server.py", line561, in _check_active 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourceraise exc 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resourceError: Creation of server overcloud-SwiftStorage0-qdjqbif6peva failed. 2014-07-18 06:51:16.058 30750 TRACE heat.engine.resource 2014-07-18 06:51:16.255 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back tousing default 2014-07-18 06:51:16.939 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back tousing default 2014-07-18 06:51:17.368 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back tousing default 2014-07-18 06:51:17.638 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back tousing default 2014-07-18 06:51:18.158 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back tousing default 2014-07-18 06:51:18.613 30750 WARNING heat.common.keystoneclient[-] stack_user_domain IDnot set in heat.conf falling back... Earlier posted in wrong/different forum. Ref: https://ask.openstack.org/en/question/43017/heat-is-not-able-to-create-st... | --Pradeep

9 years, 9 months

3
5
0 / 0

[Rdo-list] Issues with rdo-release-4 and external networking / br-tun

by uberj

Hello, I'm attempting to get rdo working on Centos6.5 with external networking. I am following the steps outlined on http://openstack.redhat.com/Neutron_with_existing_external_network To install openstack, I'm running the following commands: sudo yum -y update sudo yum install -y http://rdo.fedorapeople.org/rdo-release.rpm sudo yum install -y openstack-packstack packstack --allinone --provision-all-in-one-ovs-bridge=n --os-client-install=y --os-heat-install=y After it completes I do 'tail -f /var/log/neutron/*.log' and then 'service neutron-openvswitch-agent restart'. In the neutron log I see the following errors: 2014-07-18 16:06:03.255 28794 ERROR neutron.agent.linux.ovs_lib [req-2fc4abd3-483a-4476-b14d-78c09b04368c None] Unable to execute ['ovs-ofctl', 'del-flows', 'br-tun']. Exception: Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'del-flows', 'br-tun'] Exit code: 1 Stdout: '' Stderr: 'ovs-ofctl: br-tun is not a bridge or a socket\n' 2014-07-18 16:06:03.325 28794 ERROR neutron.agent.linux.ovs_lib [req-2fc4abd3-483a-4476-b14d-78c09b04368c None] Unable to execute ['ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=1,in_port=1,actions=resubmit(,1)']. Exception: Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=1,in_port=1,actions=resubmit(,1)'] Exit code: 1 Stdout: '' Stderr: 'ovs-ofctl: br-tun is not a bridge or a socket\n' 2014-07-18 16:06:03.380 28794 ERROR neutron.agent.linux.ovs_lib [req-2fc4abd3-483a-4476-b14d-78c09b04368c None] Unable to execute ['ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=0,actions=drop']. Exception: Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-tun', 'hard_timeout=0,idle_timeout=0,priority=0,actions=drop'] Exit code: 1 Stdout: '' Stderr: 'ovs-ofctl: br-tun is not a bridge or a socket\n' When I go to look for br-tun I do 'ovs-vsctl show' and I'll *sometimes* see: ... Bridge br-tun Port br-tun Interface br-tun type: internal ... Now, this is kind of weird, but if I do "watch -n 0.5 ovs-vsctl show" I don't always see br-tun! In fact the br-tun seems to jump around a lot (sometimes its there, sometimes its listed above br-ex, sometimes below.) More info: [root@localhost ~]# ovsdb-server --version ovsdb-server (Open vSwitch) 1.11.0 Compiled Jul 30 2013 18:14:53 [root@localhost ~]# ovs-vswitchd --version ovs-vswitchd (Open vSwitch) 1.11.0 Compiled Jul 30 2013 18:14:54 OpenFlow versions 0x1:0x1 Any help would be appreciated. -- (uberj) Jacques Uber Mozilla IT

9 years, 9 months

2
2
0 / 0

[Rdo-list] RDO Icehouse with CentOS 6.5 Openvswitch Issue

by Lakmal Warusawithana

Hi, I have tried out RDO with CentOS 6.5 following [1]. I am getting an openvswitch issue, it keep adding and deleting br-tun. my /var/log/message hitting continues message of following. Is any one com across this issue? And note, I had working setup with multi node (10 days ago), but suddenly with an yum update and re-run packstack lead to this issue. Now even allinone setup also getting this issue. Jul 24 10:22:20 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --if-exists del-br br-tun Jul 24 10:22:20 openstack01 kernel: device br-tun left promiscuous mode Jul 24 10:22:20 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --may-exist add-br br-tun Jul 24 10:22:20 openstack01 kernel: device br-tun entered promiscuous mode Jul 24 10:22:20 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --if-exists del-br br-tun Jul 24 10:22:20 openstack01 kernel: device br-tun left promiscuous mode Jul 24 10:22:20 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --may-exist add-br br-tun Jul 24 10:22:20 openstack01 kernel: device br-tun entered promiscuous mode Jul 24 10:22:21 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --if-exists del-br br-tun Jul 24 10:22:21 openstack01 kernel: device br-tun left promiscuous mode Jul 24 10:22:21 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --may-exist add-br br-tun Jul 24 10:22:21 openstack01 kernel: device br-tun entered promiscuous mode Jul 24 10:22:21 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --if-exists del-br br-tun Jul 24 10:22:21 openstack01 kernel: device br-tun left promiscuous mode Jul 24 10:22:21 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --may-exist add-br br-tun Jul 24 10:22:21 openstack01 kernel: device br-tun entered promiscuous mode Jul 24 10:22:22 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --if-exists del-br br-tun Jul 24 10:22:22 openstack01 kernel: device br-tun left promiscuous mode Jul 24 10:22:22 openstack01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -t 10 -- --may-exist add-br br-tun Jul 24 10:22:22 openstack01 kernel: device br-tun entered promiscuous mode [1]http://openstack.redhat.com/Neutron_with_existing_external_network -- Lakmal Warusawithana

9 years, 9 months

2
4
0 / 0

[Rdo-list] Outgoing traffic from instances! RDO icehouse with GRE fedora20

by Antonio C. Velez

Hi everyone, I'm using icehouse with GRE on fedora20 in two nodes 1 controller 1 compute, everything is working great, I can loging to ssh to the instances and ping to the internet ect. but when I try to install something with yum it says conection timeout, conecting to public servers using ramdon ports 443, 80, 22 doesn't work. Someone can give me any clue of what is wrong? ------------------ Antonio C. Velez Baez Linux Consultant Vidalinux.com RHCE, RHCI, RHCX, RHCOE Red Hat Certified Training Center Email: acvelez(a)vidalinux.com Tel: 1-787-439-2983 Skype: vidalinuxpr Twitter: @vidalinux.com Website: www.vidalinux.com -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

9 years, 9 months

6
11
0 / 0

Re: [Rdo-list] Icehouse MariaDB-Galera -Server problem.

by john decot

Hi, the output of uname -a is Linux virtualbox.localdomain 2.6.32-431.20.3.el6.i686 #1 SMP Thu Jun 19 19:51:30 UTC 2014 i686 i686 i386 GNU/Linux John. On Tue, Jul 22, 2014 at 6:19 AM, john decot <johndecot(a)gmail.com> wrote: > Hi, > the output of uname -a is > > Linux virtualbox.localdomain 2.6.32-431.20.3.el6.i686 #1 SMP Thu Jun 19 > 19:51:30 UTC 2014 i686 i686 i386 GNU/Linux > > John. > > > > On Mon, Jul 21, 2014 at 8:47 PM, uberj <juber(a)mozilla.com> wrote: > >> Hi John, >> >> I have seen similar symptoms (specifically the absense of >> mariadb-galera-server) when I trying to install packstack on a non x86 >> version of centos. What does 'uname -a' say? Some packages needed by >> packstack are pinned to the x86 architecture so that when you install rdo >> on something other than x86 you only will be installing the noarch packages >> and miss any architecture specific packages. >> >> >> On 07/21/2014 02:06 AM, john decot wrote: >> >> Hi, >> >> I am new to openstack. I am on the way to RDO for the installation >> of openstack. >> >> >> packstack --allinone command generates output error : cannot find >> mariadb-galera-server in repo. >> >> any help will be appreciated. >> >> >> Thank You, >> >> John. >> >> >> >> >> >> _______________________________________________ >> Rdo-list mailing listRdo-list@redhat.comhttps://www.redhat.com/mailman/listinfo/rdo-list >> >> >> -- >> (uberj) Jacques Uber >> Mozilla IT >> >> >

9 years, 9 months

2
2
0 / 0

[Rdo-list] packstack error messages

by Zhao, Xin

Hello, I am installing icehouse from RDO, on a RHEL7 VM, using packstack. Get the following error: ...... Applying 130.199.185.76_glance.pp Applying 130.199.185.76_cinder.pp 130.199.185.76_keystone.pp: [ ERROR ] Applying Puppet manifests [ ERROR ] ERROR : Error appeared during Puppet run: 130.199.185.76_keystone.pp Error: /Stage[main]/Keystone::Roles::Admin/Keystone_role[_member_]: Could not evaluate: Execution of '/usr/bin/keystone --os-endpoint http://127.0.0.1:35357/v2.0/ role-list' returned 1: 'NoneType' object has no attribute '__getitem__' You will find full trace in log /var/tmp/packstack/20140721-102751-Mviqzb/manifests/130.199.185.76_keystone.pp.log Please check log file /var/tmp/packstack/20140721-102751-Mviqzb/openstack-setup.log for more information Any idea what caused this ? I am using the rdo repo from rdo-release-icehouse-4 rpm. Thanks in advance, Xin

9 years, 9 months

2
2
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

dev July 2014