Re: [rdo-list] Newton HA galera-ready dependency error

Hi, puppet-tripleo-5.2.0-0.20161007035759.f32e484.el7.centos.noarch The patch seems to be already present - grep short /usr/share/openstack-puppet/modules/tripleo/manifests/profile/pacemaker/database/mysql.pp # short name which is already registered in pacemaker until we get around $galera_node_names_lookup = hiera('mysql_short_node_names', hiera('mysql_node_names', $::hostname)) Charles On 11/10/2016 19:39, Marius Cornea wrote: > > I think the issue is caused by the addresses in wsrep_cluster_address > not matching the pacemaker node names: > > wsrep_cluster_address=gcomm://overcloud-controller-0.internalapi.localdomain,overcloud-controller-1.internalapi.localdomain,overcloud-controller-2.internalapi.localdomain > > Could you please confirm what version of puppet-tripleo you've got > installed on the overcloud nodes and if it contains the following > patch: > https://review.openstack.org/#/c/382883/1/manifests/profile/pacemaker/dat... > > Thanks, > Marius > > On Tue, Oct 11, 2016 at 7:42 PM, Charles Short <cems(a)ebi.ac.uk&gt; wrote: >> >> Ok install finished with same error >> The latest pcs status etc >> >> http://pastebin.com/ZK683gZe >> >> >> On 11/10/2016 17:35, Charles Short wrote: >>> >>> Deployment almost finished...so >>> >>> >>> http://pastebin.com/zE9B19XB >>> >>> This shows the pcs status as the deployment nears the end, and pcs >>> resource show galera >>> >>> Charles >>> >>> On 11/10/2016 16:59, Marius Cornea wrote: >>>> >>>> Great, thanks for checking this. >>>> >>>> On Tue, Oct 11, 2016 at 5:58 PM, Charles Short <cems(a)ebi.ac.uk&gt; wrote: >>>>> >>>>> Currently having more generic deployment issues (no valid host found >>>>> etc). >>>>> I can work around/solve these. >>>>> I don't yet have another stack to analyse, but will do soon. >>>>> >>>>> Charles >>>>> >>>>> >>>>> On 11/10/2016 16:35, Marius Cornea wrote: >>>>>> >>>>>> Did it succeed in bringing the Galera nodes to Master? You can ssh to >>>>>> the nodes and run 'pcs resource show galera' even though the >>>>>> deployment hasn't finished. I'm interested to see how the >>>>>> wsrep_cluster_address is set to see if it's affected by the resource >>>>>> agent issue described in >>>>>> https://bugs.launchpad.net/tripleo/+bug/1628521 >>>>>> >>>>>> On Tue, Oct 11, 2016 at 5:18 PM, Charles Short <cems(a)ebi.ac.uk&gt; >>>>>> wrote: >>>>>>> >>>>>>> Looks similar to this bug (still waiting on deployment to finish) >>>>>>> >>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1368214 >>>>>>> >>>>>>> >>>>>>> On 11/10/2016 15:25, Charles Short wrote: >>>>>>>> >>>>>>>> Sorry for the delay. >>>>>>>> >>>>>>>> >>>>>>>> Just redeploying to make sure I can repeat the same error. Should >>>>>>>> not >>>>>>>> be >>>>>>>> long. >>>>>>>> >>>>>>>> Charles >>>>>>>> >>>>>>>> On 11/10/2016 14:24, Marius Cornea wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Could you also please paste the output for 'pcs resource show >>>>>>>>> galera', >>>>>>>>> it looks that all the galera nodes show up as slaves? >>>>>>>>> >>>>>>>>> Master/Slave Set: galera-master [galera] >>>>>>>>> Slaves: [ overcloud-controller-0 overcloud-controller-1 >>>>>>>>> overcloud-controller-2 ] >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> On Tue, Oct 11, 2016 at 2:16 PM, Charles Short <cems(a)ebi.ac.uk&gt; >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Here you are - >>>>>>>>>> >>>>>>>>>> - Heat stack error - http://pastebin.com/E8KZa2vE >>>>>>>>>> - PCS status - http://pastebin.com/z34gSLq6 >>>>>>>>>> - mariadb.log - http://pastebin.com/APFXPBLc >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> Charles >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/10/2016 12:07, Marius Cornea wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Charles, >>>>>>>>>>> >>>>>>>>>>> Could you please paste the output of 'pcs status' ? The log in >>>>>>>>>>> /var/log/mariadb/mariadb.log might also be a good indicator. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Oct 11, 2016 at 11:16 AM, Charles Short <cems(a)ebi.ac.uk&gt; >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> To add I built my own image from >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 >>>>>>>>>>>> >>>>>>>>>>>> as the images in >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/newton/d... >>>>>>>>>>>> caused sporadic ramdisk loading errors (hung at x% loaded on >>>>>>>>>>>> boot) >>>>>>>>>>>> >>>>>>>>>>>> Does my image now need to be customised in any way for HA to >>>>>>>>>>>> work? >>>>>>>>>>>> >>>>>>>>>>>> Charles >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/10/2016 09:55, Charles Short wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I am installing Newton with TripleO on baremetal HP blades. >>>>>>>>>>>>> I can deploy a single controller stack overcloud no problem, >>>>>>>>>>>>> however >>>>>>>>>>>>> when >>>>>>>>>>>>> I choose three controllers the deployment fails (including >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml) >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>>> The heat stack error first complains "Dependency >>>>>>>>>>>>> Exec[galera-ready] >>>>>>>>>>>>> has >>>>>>>>>>>>> failures" which in turn causes lots of other errors. >>>>>>>>>>>>> >>>>>>>>>>>>> I have deployed Liberty and Mitaka successfully in the past on >>>>>>>>>>>>> baremetal >>>>>>>>>>>>> with three controllers, and this is the first time I have seen >>>>>>>>>>>>> this >>>>>>>>>>>>> error. >>>>>>>>>>>>> >>>>>>>>>>>>> Charles >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Charles Short >>>>>>>>>>>> Cloud Engineer >>>>>>>>>>>> Virtualization and Cloud Team >>>>>>>>>>>> European Bioinformatics Institute (EMBL-EBI) >>>>>>>>>>>> Tel: +44 (0)1223 494205 >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> rdo-list mailing list >>>>>>>>>>>> rdo-list(a)redhat.com >>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/rdo-list >>>>>>>>>>>> >>>>>>>>>>>> To unsubscribe: rdo-list-unsubscribe(a)redhat.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Charles Short >>>>>>>>>> Cloud Engineer >>>>>>>>>> Virtualization and Cloud Team >>>>>>>>>> European Bioinformatics Institute (EMBL-EBI) >>>>>>>>>> Tel: +44 (0)1223 494205 >>>>>>>>>> >>>>>>> -- >>>>>>> Charles Short >>>>>>> Cloud Engineer >>>>>>> Virtualization and Cloud Team >>>>>>> European Bioinformatics Institute (EMBL-EBI) >>>>>>> Tel: +44 (0)1223 494205 >>>>>>> >>>>> -- >>>>> Charles Short >>>>> Cloud Engineer >>>>> Virtualization and Cloud Team >>>>> European Bioinformatics Institute (EMBL-EBI) >>>>> Tel: +44 (0)1223 494205 >>>>> >> -- >> Charles Short >> Cloud Engineer >> Virtualization and Cloud Team >> European Bioinformatics Institute (EMBL-EBI) >> Tel: +44 (0)1223 494205 >> -- Charles Short Cloud Engineer Virtualization and Cloud Team European Bioinformatics Institute (EMBL-EBI) Tel: +44 (0)1223 494205