[rdo-list] Newton HA galera-ready dependency error

Tue Oct 11 18:39:16 UTC 2016

I think the issue is caused by the addresses in wsrep_cluster_address
not matching the pacemaker node names:
wsrep_cluster_address=gcomm://overcloud-controller-0.internalapi.localdomain,overcloud-controller-1.internalapi.localdomain,overcloud-controller-2.internalapi.localdomain

Could you please confirm what version of puppet-tripleo you've got
installed on the overcloud nodes and if it contains the following
patch: https://review.openstack.org/#/c/382883/1/manifests/profile/pacemaker/database/mysql.pp

Thanks,
Marius

On Tue, Oct 11, 2016 at 7:42 PM, Charles Short <cems at ebi.ac.uk> wrote:
> Ok install finished with same error
> The latest pcs status etc
>
> http://pastebin.com/ZK683gZe
>
>
> On 11/10/2016 17:35, Charles Short wrote:
>>
>> Deployment almost finished...so
>>
>>
>> http://pastebin.com/zE9B19XB
>>
>> This shows the pcs status as the deployment nears the end, and pcs
>> resource show galera
>>
>> Charles
>>
>> On 11/10/2016 16:59, Marius Cornea wrote:
>>>
>>> Great, thanks for checking this.
>>>
>>> On Tue, Oct 11, 2016 at 5:58 PM, Charles Short <cems at ebi.ac.uk> wrote:
>>>>
>>>> Currently having more generic deployment issues (no valid host found
>>>> etc).
>>>> I can work around/solve these.
>>>> I don't yet have another stack to analyse, but will do soon.
>>>>
>>>> Charles
>>>>
>>>>
>>>> On 11/10/2016 16:35, Marius Cornea wrote:
>>>>>
>>>>> Did it succeed in bringing the Galera nodes to Master? You can ssh to
>>>>> the nodes and run 'pcs resource show galera' even though the
>>>>> deployment hasn't finished. I'm interested to see how the
>>>>> wsrep_cluster_address is set to see if it's affected by the resource
>>>>> agent issue described in
>>>>> https://bugs.launchpad.net/tripleo/+bug/1628521
>>>>>
>>>>> On Tue, Oct 11, 2016 at 5:18 PM, Charles Short <cems at ebi.ac.uk> wrote:
>>>>>>
>>>>>> Looks similar to this bug (still waiting on deployment to finish)
>>>>>>
>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1368214
>>>>>>
>>>>>>
>>>>>> On 11/10/2016 15:25, Charles Short wrote:
>>>>>>>
>>>>>>> Sorry for the delay.
>>>>>>>
>>>>>>>
>>>>>>> Just redeploying to make sure I can repeat the same error. Should not
>>>>>>> be
>>>>>>> long.
>>>>>>>
>>>>>>> Charles
>>>>>>>
>>>>>>> On 11/10/2016 14:24, Marius Cornea wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Could you also please paste the output for 'pcs resource show
>>>>>>>> galera',
>>>>>>>> it looks that all the galera nodes show up as slaves?
>>>>>>>>
>>>>>>>>     Master/Slave Set: galera-master [galera]
>>>>>>>>         Slaves: [ overcloud-controller-0 overcloud-controller-1
>>>>>>>> overcloud-controller-2 ]
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Tue, Oct 11, 2016 at 2:16 PM, Charles Short <cems at ebi.ac.uk>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Here you are -
>>>>>>>>>
>>>>>>>>>     - Heat stack error  - http://pastebin.com/E8KZa2vE
>>>>>>>>>     - PCS status  - http://pastebin.com/z34gSLq6
>>>>>>>>>     - mariadb.log - http://pastebin.com/APFXPBLc
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> Charles
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 11/10/2016 12:07, Marius Cornea wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Charles,
>>>>>>>>>>
>>>>>>>>>> Could you please paste the output of 'pcs status' ? The log in
>>>>>>>>>> /var/log/mariadb/mariadb.log might also be a good indicator.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Oct 11, 2016 at 11:16 AM, Charles Short <cems at ebi.ac.uk>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> To add I built my own image from
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
>>>>>>>>>>>
>>>>>>>>>>> as the images in
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/newton/delorean/
>>>>>>>>>>> caused sporadic ramdisk loading errors  (hung at x% loaded on
>>>>>>>>>>> boot)
>>>>>>>>>>>
>>>>>>>>>>> Does my image now need to be customised in any way for HA to
>>>>>>>>>>> work?
>>>>>>>>>>>
>>>>>>>>>>> Charles
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 11/10/2016 09:55, Charles Short wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I am installing Newton with TripleO on baremetal HP blades.
>>>>>>>>>>>> I can deploy a single controller stack overcloud no problem,
>>>>>>>>>>>> however
>>>>>>>>>>>> when
>>>>>>>>>>>> I choose three controllers the deployment fails (including
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml)
>>>>>>>>>>>> .
>>>>>>>>>>>>
>>>>>>>>>>>> The heat stack error first complains "Dependency
>>>>>>>>>>>> Exec[galera-ready]
>>>>>>>>>>>> has
>>>>>>>>>>>> failures" which in turn causes lots of other errors.
>>>>>>>>>>>>
>>>>>>>>>>>> I have deployed Liberty and Mitaka successfully in the past on
>>>>>>>>>>>> baremetal
>>>>>>>>>>>> with three controllers, and this is the first time I have seen
>>>>>>>>>>>> this
>>>>>>>>>>>> error.
>>>>>>>>>>>>
>>>>>>>>>>>> Charles
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Charles Short
>>>>>>>>>>> Cloud Engineer
>>>>>>>>>>> Virtualization and Cloud Team
>>>>>>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>>>>>>> Tel: +44 (0)1223 494205
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> rdo-list mailing list
>>>>>>>>>>> rdo-list at redhat.com
>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>>>>>>>>>
>>>>>>>>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Charles Short
>>>>>>>>> Cloud Engineer
>>>>>>>>> Virtualization and Cloud Team
>>>>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>>>>> Tel: +44 (0)1223 494205
>>>>>>>>>
>>>>>> --
>>>>>> Charles Short
>>>>>> Cloud Engineer
>>>>>> Virtualization and Cloud Team
>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>> Tel: +44 (0)1223 494205
>>>>>>
>>>> --
>>>> Charles Short
>>>> Cloud Engineer
>>>> Virtualization and Cloud Team
>>>> European Bioinformatics Institute (EMBL-EBI)
>>>> Tel: +44 (0)1223 494205
>>>>
>>
>
> --
> Charles Short
> Cloud Engineer
> Virtualization and Cloud Team
> European Bioinformatics Institute (EMBL-EBI)
> Tel: +44 (0)1223 494205
>