I think I figured it out.
I started looking into saslauthd configuration and the configuration looked right, so I
checked the service status, it was off. I checked the chkconfig status of saslauthd, and
it was off for all init levels. I ran "/etc/init.d/saslauthd start", it turned
on, so I changed the /etc/qpidd.conf "auth=no" to "auth=yes", and
restarted qpidd service while tailing the /var/log/nova/compute.log of my compute node. It
had 2 failure notices immediately, but then right after said communication was successful,
and all my hypervisors are showing in the dashboard, communication in logs still looks
good.
I guess for some reason the Foreman controller host group doesn't turn on saslauthd
service and doesn't turn on chkconfig for saslauthd?
-----Original Message-----
From: Lodgen, Brad
Sent: Thursday, July 03, 2014 11:52 AM
To: 'Rhys Oxenham'
Cc: 'rdo-list(a)redhat.com'
Subject: RE: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup: First Compute Node
Doesn't Show Up in Hypervisor List
Well, it had the same result with the second compute node I brought up which was a fresh
system with RHEL6.5/RHOS package updates.
I checked the nova.conf on controller and both compute nodes. All the same configuration,
username, passwords, everything.
rpc_backend=nova.openstack.common.rpc.impl_qpid
qpid_hostname={controller node private IP}
qpid_port=5672
#qpid_hosts=$qpid_hostname:$qpid_port
qpid_username={same username}
qpid_password={same password}
#qpid_sasl_mechanisms=
qpid_heartbeat=60
qpid_protocol=tcp
qpid_tcp_nodelay=True
#qpid_topology_version=1
Should the qpid client be installed on the compute nodes? Because this page notes that
doing it manually, it should be
(
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_Op...)
but, as you can see via Foreman host group deployment it IS installed on the controller
and IS NOT on the compute nodes.
[root@ctlr ~]# yum list installed | grep qpid This system is receiving updates from Red
Hat Subscription Management.
python-qpid.noarch 0.14-11.el6_3 @rhel-6-server-rpms
qpid-cpp-client.x86_64 0.14-22.el6_3 @rhel-6-server-rpms
qpid-cpp-server.x86_64 0.14-22.el6_3 @rhel-6-server-rpms
[root@comp1 ~]# yum list installed | grep qpid This system is receiving updates from Red
Hat Subscription Management.
python-qpid.noarch 0.14-11.el6_3 @rhel-6-server-rpms
[root@comp2 ~]# yum list installed | grep qpid This system is receiving updates from Red
Hat Subscription Management.
python-qpid.noarch 0.14-11.el6_3 @rhel-6-server-rpms
-----Original Message-----
From: Rhys Oxenham [mailto:roxenham@redhat.com]
Sent: Thursday, July 03, 2014 11:22 AM
To: Lodgen, Brad
Cc: rdo-list(a)redhat.com
Subject: Re: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup: First Compute Node
Doesn't Show Up in Hypervisor List
OK looks like a bug somewhere... if qpid auth is enabled it requires the authentication
mechanism to be completed properly.
See:
http://qpid.apache.org/releases/qpid-0.14/books/AMQP-Messaging-Broker-CPP...
From looking at puppet-qpid it should have done this for you.
Have you been able to reproduce this issue on a clean system?
Cheers
Rhys
On 3 Jul 2014, at 17:16, Lodgen, Brad <Brad.Lodgen(a)centurylink.com> wrote:
No worries. I understand you're busy and thank you for the
assistance. To answer your question: yes, following the change and qpidd restart, the logs
showed successful communication and the initial compute node showed up as a hypervisor in
the dashboard. I also successfully added a second compute node. Success here relies upon
disabling the puppet agent so it doesn't change auth back to yes; otherwise,
communication fails.
-----Original Message-----
From: Rhys Oxenham [mailto:roxenham@redhat.com]
Sent: Thursday, July 03, 2014 11:13 AM
To: Lodgen, Brad
Cc: rdo-list(a)redhat.com
Subject: Re: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup:
First Compute Node Doesn't Show Up in Hypervisor List
Sorry I didn't respond to this... I have auth set to no in my environment, but
that's just for testing. Do things work when auth is set to no and the service is
restarted?
On 3 Jul 2014, at 17:11, Lodgen, Brad <Brad.Lodgen(a)centurylink.com> wrote:
> Follow-up from yesterday... is this the same default in RDO, to have qpidd.conf
default to auth=yes?
>
> Does that mean I have something on the compute side misconfigured? It looks to me
like the username/password is the same on the controller/compute.
>
> For now, I've had to disable puppet agent on the controller, as it keeps
resetting "auth=no" back to "auth=yes", and I don't see a host
group parameter that would change that.
>
>
>
>
> -----Original Message-----
> From: Lodgen, Brad
> Sent: Wednesday, July 02, 2014 2:01 PM
> To: 'Rhys Oxenham'
> Cc: 'rdo-list(a)redhat.com'
> Subject: RE: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup:
> First Compute Node Doesn't Show Up in Hypervisor List
>
> So even though the default controller puppet module configures qpidd.conf to say
"auth=yes", I changed it to "auth=no" and restarted qpidd service. I
now see my compute host in the dashboard.
>
> Is that a misconfiguration in RHOSv4 that I should submit for a change somewhere?
>
>
>
> -----Original Message-----
> From: Lodgen, Brad
> Sent: Wednesday, July 02, 2014 12:53 PM
> To: 'Rhys Oxenham'
> Cc: 'rdo-list(a)redhat.com'
> Subject: RE: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup:
> First Compute Node Doesn't Show Up in Hypervisor List
>
> The settings in the controller/compute host group parameters regarding qpidd are all
default, except for the host, which is the private IP of the controller.
>
> I haven't made any changes outside of the Foreman host group parameters and I
don't see any compute host group parameters that would allow me to specify whether a
service uses Qpid authentication or not.
>
> I did change the compute host group parameter "auth_host" and
"nova_host" (originally by default set to 127.0.0.1) to the private IP of the
controller. Would that have any effect?
>
>
> -----Original Message-----
> From: Rhys Oxenham [mailto:roxenham@redhat.com]
> Sent: Wednesday, July 02, 2014 12:35 PM
> To: Lodgen, Brad
> Cc: rdo-list(a)redhat.com
> Subject: Re: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup:
> First Compute Node Doesn't Show Up in Hypervisor List
>
> Have you specified Qpid authentication in any of the rest of the services? I suspect
that Qpid is set up to use authentication but none of the other services are.
>
> On 2 Jul 2014, at 18:30, Lodgen, Brad <Brad.Lodgen(a)centurylink.com> wrote:
>
>> # GENERATED BY PUPPET
>> #
>> # Configuration file for qpidd. Entries are of the form:
>> # name=value
>> #
>> # (Note: no spaces on either side of '='). Using default settings:
>> # "qpidd --help" or "man qpidd" for more details.
>> port=5672
>> max-connections=65535
>> worker-threads=17
>> connection-backlog=10
>> auth=yes
>> realm=QPID
>>
>>
>> -----Original Message-----
>> From: Rhys Oxenham [mailto:roxenham@redhat.com]
>> Sent: Wednesday, July 02, 2014 12:27 PM
>> To: Lodgen, Brad
>> Cc: rdo-list(a)redhat.com
>> Subject: Re: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup:
>> First Compute Node Doesn't Show Up in Hypervisor List
>>
>> No worries!
>>
>> Can you paste out your /etc/qpidd.conf file from the controller?
>> (Make sure you sanitise the output)
>>
>> Cheers
>> Rhys
>>
>>
>> On 2 Jul 2014, at 18:23, Lodgen, Brad <Brad.Lodgen(a)centurylink.com> wrote:
>>
>>> Thanks for the quick response! Based on the below log findings and what I
just found searching, is this caused by the controller host group parameter
"freeipa" being set to the default "false"? Change it to
"true"?
>>>
>>>
>>>
>>> On the compute node, I'm seeing this over and over in the compute log:
>>>
>>> Unable to connect to AMQP server: Error in sasl_client_start (-1)
>>> SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
>>> Minor code may provide more information (Cannot determine realm for
>>> numeric host address). Sleeping 5 seconds
>>>
>>> On the controller conductor log:
>>>
>>> Unable to connect to AMQP server: Error in sasl_client_start (-1)
>>> SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.
>>> Minor code may provide more information (Cannot determine realm for
>>> numeric host address). Sleeping 5 seconds
>>>
>>> In the controller messages file:
>>>
>>> python: GSSAPI Error: Unspecified GSS failure. Minor code may
>>> provide more information (Cannot determine realm for numeric host
>>> address)
>>>
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Rhys Oxenham [mailto:roxenham@redhat.com]
>>> Sent: Wednesday, July 02, 2014 12:14 PM
>>> To: Lodgen, Brad
>>> Cc: rdo-list(a)redhat.com
>>> Subject: Re: [Rdo-list] RH OpenStack v4 Evaluation: Initial Setup:
>>> First Compute Node Doesn't Show Up in Hypervisor List
>>>
>>> Hi Brad,
>>>
>>> Have you checked the nova-compute logs in /var/log/nova/compute.log
>>> (on your new compute node?)
>>>
>>> This should point towards why it's unable to connect/start etc. I suspect
that it's unable to join the message queue, and hence show up as an available
hypervisor.
>>>
>>> Many thanks
>>> Rhys
>>>
>>> On 2 Jul 2014, at 18:05, Lodgen, Brad <Brad.Lodgen(a)centurylink.com>
wrote:
>>>
>>>> Hi folks,
>>>>
>>>> I have an issue where I've just done the initial setup and added a
controller node, it finished, then I added a compute node, it finished, but I don't
see the compute node in the hypervisor list on the dashboard. I'm using the RH
OpenStack evaluation version 4. I have five hosts present in Foreman.
>>>>
>>>> -Foreman host (purely for Foreman) -Controller host (applied
>>>> Controller(Nova) host group) -Compute Host (applied Compute(Nova)
>>>> host group)
>>>> -2 other hosts (not host group applied, but one will be compute
>>>> and one will be storage)
>>>>
>>>> Did I miss something on the controller/compute host group parameters that
would cause it to not show up in the dashboard?
>>>> _______________________________________________
>>>> Rdo-list mailing list
>>>> Rdo-list(a)redhat.com
>>>>
https://www.redhat.com/mailman/listinfo/rdo-list
>>>
>>
>