[rdo-list] Newton HA galera-ready dependency error
Charles Short
cems at ebi.ac.uk
Wed Oct 12 09:45:29 UTC 2016
Just checked what I did to build the image previously.
I used
export
DELOREAN_TRUNK_REPO="http://trunk.rdoproject.org/centos7-newton/current/"
Which is the same repo I used to install the Undercloud.
Maybe I need to do a yum update in the image with libguestfs-tools
prior to building the image?
I will rebuild again anyway in case I made an error
C
On 12/10/2016 10:28, Charles Short wrote:
> Hi,
>
> Ok, I will rebuild using the undercloud repo and report back.
>
> Thanks for your help
>
> Charles
>
> On 12/10/2016 09:44, Marius Cornea wrote:
>> Oh, that explains it. It looks that the overcloud image doesn't
>> contain the patch.
>>
>> I'm not familiar with the image build process but according to the
>> docs[1] I think the packages get installed from the repo specified by
>> export DELOREAN_TRUNK_REPO so maybe you should try to rebuild the
>> image and use the same repo as the one set on the undercloud.
>>
>> [1]
>> http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html
>>
>> On Wed, Oct 12, 2016 at 10:30 AM, Charles Short <cems at ebi.ac.uk> wrote:
>>> Hi,
>>>
>>> We noticed something that may be the reason for this patch not
>>> working which
>>> may be related to the way the Undercloud built the image? -
>>>
>>> This is difference between the puppet files in the undercloud and
>>> puppet in the image:
>>>
>>> [stack at hh-extcl05-undercloud ~]$ grep mysql_short_node_names
>>> /usr/share/openstack-puppet/modules/tripleo/manifests/profile/pacemaker/database/mysql.pp
>>>
>>>
>>> $galera_node_names_lookup = hiera('mysql_short_node_names',
>>> hiera('mysql_node_names', $::hostname))
>>>
>>> [root at overcloud-controller-1 puppet]# grep mysql_short_node_names
>>> /etc/puppet/modules/tripleo/manifests/profile/pacemaker/database/mysql.pp
>>>
>>>
>>> (nothing found)
>>>
>>>
>>> On 12/10/2016 09:09, Marius Cornea wrote:
>>>> That's odd. I encountered the same issue and it was caused by missing
>>>> this patch. What do you get if you do sudo hiera
>>>> mysql_short_node_names on the controller node?
>>>>
>>>> On Wed, Oct 12, 2016 at 12:48 AM, Charles Short <cems at ebi.ac.uk>
>>>> wrote:
>>>>> Hi,
>>>>>
>>>>> puppet-tripleo-5.2.0-0.20161007035759.f32e484.el7.centos.noarch
>>>>>
>>>>> The patch seems to be already present -
>>>>>
>>>>> grep short
>>>>>
>>>>> /usr/share/openstack-puppet/modules/tripleo/manifests/profile/pacemaker/database/mysql.pp
>>>>>
>>>>>
>>>>> # short name which is already registered in pacemaker until we
>>>>> get
>>>>> around
>>>>> $galera_node_names_lookup = hiera('mysql_short_node_names',
>>>>> hiera('mysql_node_names', $::hostname))
>>>>>
>>>>> Charles
>>>>>
>>>>> On 11/10/2016 19:39, Marius Cornea wrote:
>>>>>> I think the issue is caused by the addresses in
>>>>>> wsrep_cluster_address
>>>>>> not matching the pacemaker node names:
>>>>>>
>>>>>>
>>>>>> wsrep_cluster_address=gcomm://overcloud-controller-0.internalapi.localdomain,overcloud-controller-1.internalapi.localdomain,overcloud-controller-2.internalapi.localdomain
>>>>>>
>>>>>>
>>>>>> Could you please confirm what version of puppet-tripleo you've got
>>>>>> installed on the overcloud nodes and if it contains the following
>>>>>> patch:
>>>>>>
>>>>>> https://review.openstack.org/#/c/382883/1/manifests/profile/pacemaker/database/mysql.pp
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Marius
>>>>>>
>>>>>> On Tue, Oct 11, 2016 at 7:42 PM, Charles Short <cems at ebi.ac.uk>
>>>>>> wrote:
>>>>>>> Ok install finished with same error
>>>>>>> The latest pcs status etc
>>>>>>>
>>>>>>> http://pastebin.com/ZK683gZe
>>>>>>>
>>>>>>>
>>>>>>> On 11/10/2016 17:35, Charles Short wrote:
>>>>>>>> Deployment almost finished...so
>>>>>>>>
>>>>>>>>
>>>>>>>> http://pastebin.com/zE9B19XB
>>>>>>>>
>>>>>>>> This shows the pcs status as the deployment nears the end, and pcs
>>>>>>>> resource show galera
>>>>>>>>
>>>>>>>> Charles
>>>>>>>>
>>>>>>>> On 11/10/2016 16:59, Marius Cornea wrote:
>>>>>>>>> Great, thanks for checking this.
>>>>>>>>>
>>>>>>>>> On Tue, Oct 11, 2016 at 5:58 PM, Charles Short <cems at ebi.ac.uk>
>>>>>>>>> wrote:
>>>>>>>>>> Currently having more generic deployment issues (no valid
>>>>>>>>>> host found
>>>>>>>>>> etc).
>>>>>>>>>> I can work around/solve these.
>>>>>>>>>> I don't yet have another stack to analyse, but will do soon.
>>>>>>>>>>
>>>>>>>>>> Charles
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 11/10/2016 16:35, Marius Cornea wrote:
>>>>>>>>>>> Did it succeed in bringing the Galera nodes to Master? You
>>>>>>>>>>> can ssh
>>>>>>>>>>> to
>>>>>>>>>>> the nodes and run 'pcs resource show galera' even though the
>>>>>>>>>>> deployment hasn't finished. I'm interested to see how the
>>>>>>>>>>> wsrep_cluster_address is set to see if it's affected by the
>>>>>>>>>>> resource
>>>>>>>>>>> agent issue described in
>>>>>>>>>>> https://bugs.launchpad.net/tripleo/+bug/1628521
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Oct 11, 2016 at 5:18 PM, Charles Short <cems at ebi.ac.uk>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Looks similar to this bug (still waiting on deployment to
>>>>>>>>>>>> finish)
>>>>>>>>>>>>
>>>>>>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1368214
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 11/10/2016 15:25, Charles Short wrote:
>>>>>>>>>>>>> Sorry for the delay.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Just redeploying to make sure I can repeat the same error.
>>>>>>>>>>>>> Should
>>>>>>>>>>>>> not
>>>>>>>>>>>>> be
>>>>>>>>>>>>> long.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Charles
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 11/10/2016 14:24, Marius Cornea wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Could you also please paste the output for 'pcs resource
>>>>>>>>>>>>>> show
>>>>>>>>>>>>>> galera',
>>>>>>>>>>>>>> it looks that all the galera nodes show up as slaves?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Master/Slave Set: galera-master [galera]
>>>>>>>>>>>>>> Slaves: [ overcloud-controller-0
>>>>>>>>>>>>>> overcloud-controller-1
>>>>>>>>>>>>>> overcloud-controller-2 ]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 2:16 PM, Charles Short
>>>>>>>>>>>>>> <cems at ebi.ac.uk>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Here you are -
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Heat stack error - http://pastebin.com/E8KZa2vE
>>>>>>>>>>>>>>> - PCS status - http://pastebin.com/z34gSLq6
>>>>>>>>>>>>>>> - mariadb.log - http://pastebin.com/APFXPBLc
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Charles
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 11/10/2016 12:07, Marius Cornea wrote:
>>>>>>>>>>>>>>>> Hi Charles,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Could you please paste the output of 'pcs status' ? The
>>>>>>>>>>>>>>>> log in
>>>>>>>>>>>>>>>> /var/log/mariadb/mariadb.log might also be a good
>>>>>>>>>>>>>>>> indicator.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 11:16 AM, Charles Short
>>>>>>>>>>>>>>>> <cems at ebi.ac.uk>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> To add I built my own image from
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> as the images in
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/newton/delorean/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> caused sporadic ramdisk loading errors (hung at x%
>>>>>>>>>>>>>>>>> loaded on
>>>>>>>>>>>>>>>>> boot)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Does my image now need to be customised in any way for
>>>>>>>>>>>>>>>>> HA to
>>>>>>>>>>>>>>>>> work?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Charles
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 11/10/2016 09:55, Charles Short wrote:
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I am installing Newton with TripleO on baremetal HP
>>>>>>>>>>>>>>>>>> blades.
>>>>>>>>>>>>>>>>>> I can deploy a single controller stack overcloud no
>>>>>>>>>>>>>>>>>> problem,
>>>>>>>>>>>>>>>>>> however
>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>> I choose three controllers the deployment fails
>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The heat stack error first complains "Dependency
>>>>>>>>>>>>>>>>>> Exec[galera-ready]
>>>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>>> failures" which in turn causes lots of other errors.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I have deployed Liberty and Mitaka successfully in
>>>>>>>>>>>>>>>>>> the past
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> baremetal
>>>>>>>>>>>>>>>>>> with three controllers, and this is the first time I
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> seen
>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> error.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Charles
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Charles Short
>>>>>>>>>>>>>>>>> Cloud Engineer
>>>>>>>>>>>>>>>>> Virtualization and Cloud Team
>>>>>>>>>>>>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>>>>>>>>>>>>> Tel: +44 (0)1223 494205
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> rdo-list mailing list
>>>>>>>>>>>>>>>>> rdo-list at redhat.com
>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Charles Short
>>>>>>>>>>>>>>> Cloud Engineer
>>>>>>>>>>>>>>> Virtualization and Cloud Team
>>>>>>>>>>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>>>>>>>>>>> Tel: +44 (0)1223 494205
>>>>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Charles Short
>>>>>>>>>>>> Cloud Engineer
>>>>>>>>>>>> Virtualization and Cloud Team
>>>>>>>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>>>>>>>> Tel: +44 (0)1223 494205
>>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Charles Short
>>>>>>>>>> Cloud Engineer
>>>>>>>>>> Virtualization and Cloud Team
>>>>>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>>>>>> Tel: +44 (0)1223 494205
>>>>>>>>>>
>>>>>>> --
>>>>>>> Charles Short
>>>>>>> Cloud Engineer
>>>>>>> Virtualization and Cloud Team
>>>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>>>> Tel: +44 (0)1223 494205
>>>>>>>
>>>>> --
>>>>> Charles Short
>>>>> Cloud Engineer
>>>>> Virtualization and Cloud Team
>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>> Tel: +44 (0)1223 494205
>>>>>
>>> --
>>> Charles Short
>>> Cloud Engineer
>>> Virtualization and Cloud Team
>>> European Bioinformatics Institute (EMBL-EBI)
>>> Tel: +44 (0)1223 494205
>>>
>
--
Charles Short
Cloud Engineer
Virtualization and Cloud Team
European Bioinformatics Institute (EMBL-EBI)
Tel: +44 (0)1223 494205
More information about the dev
mailing list