<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<br>
<div style="color: rgb(49, 55, 57);">
<div>
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> John Trowbridge <trown@redhat.com><br>
<b>Sent:</b> Friday, June 3, 2016 5:43 PM<br>
<b>To:</b> Boris Derzhavets; Lars Kellogg-Stedman<br>
<b>Cc:</b> rdo-list<br>
<b>Subject:</b> Re: [rdo-list] Tripleo QuickStart HA deployment attempts constantly crash</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText"><br>
<br>
On 06/03/2016 04:53 PM, John Trowbridge wrote:<br>
> I just did an HA deploy locally on master, and I see the same thing wrt<br>
> telemetry services being down due to failed redis import. That could be<br>
> a packaging bug (something should depend on python-redis, maybe<br>
> python-tooz?). That said, it does not appear fatal in my case. Is there<br>
> some issue other than telemetry services being down that you are seeing?<br>
> That is certainly something we should fix, but I wouldn't characterize<br>
> it as the deployment is constantly crashing.<br>
<br>
That was told by me in regards of comment #3 in <a title="Ctrl+Click or tap to follow the link" id="LPlnk385006" href="https://bugzilla.redhat.com/show_bug.cgi?id=1340865" class="OWAAutoLink">https://bugzilla.redhat.com/show_bug.cgi?id=1340865</a><br>
Of course , issue with telemetry services is not "<font size="2"><span style="font-size:10pt;">constantly crashing"</span></font><br>
> <br>
<br>
Confirmed that installing python-redis fixes the telemetry issue by<br>
doing the following from the undercloud:<br>
<br>
sudo LIBGUESTFS_BACKEND=direct virt-customize -a overcloud-full.qcow2<br>
--install python-redis<br>
openstack overcloud image upload --update-existing<br>
<br>
> Then deleting the failed overcloud stack, and re-running<br>
> overcloud-deploy.sh.<br>
<br>
Doesn't work for me. Re-running fails to recreate overcloud stack.<br>
<br>
> On 06/03/2016 11:30 AM, Boris Derzhavets wrote:<br>
>> 1. Attempting to address your concern ( if I understood you correct )<br>
>><br>
>> First log :-<br>
>><br>
>> [root@overcloud-controller-0 ceilometer]# cat central.log | grep ERROR<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service [req-4db5f172-0bf0-4200-9cf4-174859cdc00b admin - - - -] Error starting thread.<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service Traceback (most recent call last):<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 680, in run_service<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service service.start()<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ceilometer/agent/manager.py", line 384, in start<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service self.partition_coordinator.start()<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ceilometer/coordination.py", line 84, in start<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service backend_url, self._my_id)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/tooz/coordination.py", line 539, in get_coordinator<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service invoke_args=(member_id, parsed_url, options)).driver<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/driver.py", line 46, in __init__<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements=verify_requirements,<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 55, in __init__<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 171, in _load_plugins<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service self._on_load_failure_callback(self, ep, err)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 163, in _load_plugins<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements,<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 123, in _load_one_plugin<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements,<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 186, in _load_one_plugin<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service plugin = ep.load(require=verify_requirements)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2260, in load<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service entry = __import__(self.module_name, globals(),globals(), ['__name__'])<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/tooz/drivers/redis.py", line 27, in <module><br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service import redis<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service ImportError: No module named redis<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service <br>
>> [root@overcloud-controller-0 ceilometer]# clear<br>
>> [3;J<br>
>> [root@overcloud-controller-0 ceilometer]# cat central.log | grep ERROR<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service [req-4db5f172-0bf0-4200-9cf4-174859cdc00b admin - - - -] Error starting thread.<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service Traceback (most recent call last):<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 680, in run_service<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service service.start()<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ceilometer/agent/manager.py", line 384, in start<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service self.partition_coordinator.start()<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ceilometer/coordination.py", line 84, in start<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service backend_url, self._my_id)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/tooz/coordination.py", line 539, in get_coordinator<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service invoke_args=(member_id, parsed_url, options)).driver<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/driver.py", line 46, in __init__<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements=verify_requirements,<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 55, in __init__<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 171, in _load_plugins<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service self._on_load_failure_callback(self, ep, err)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 163, in _load_plugins<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements,<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 123, in _load_one_plugin<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service verify_requirements,<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 186, in _load_one_plugin<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service plugin = ep.load(require=verify_requirements)<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2260, in load<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service entry = __import__(self.module_name, globals(),globals(), ['__name__'])<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/tooz/drivers/redis.py", line 27, in <module><br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service import redis<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service ImportError: No module named redis<br>
>> 2016-06-03 08:50:04.405 17503 ERROR oslo_service.service <br>
>><br>
>> Second log :-<br>
>><br>
>> [root@overcloud-controller-0 ceilometer]# cd -<br>
>> /var/log/aodh<br>
>> [root@overcloud-controller-0 aodh]# cat evaluator.log | grep ERROR<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service [-] Error starting thread.<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service Traceback (most recent call last):<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 680, in run_service<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service service.start()<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/aodh/evaluator/__init__.py", line 229, in start<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service self.partition_coordinator.start()<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/aodh/coordination.py", line 133, in start<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service self.backend_url, self._my_id)<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/tooz/coordination.py", line 539, in get_coordinator<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service invoke_args=(member_id, parsed_url, options)).driver<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/driver.py", line 46, in __init__<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service verify_requirements=verify_requirements,<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 55, in __init__<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service verify_requirements)<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 171, in _load_plugins<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service self._on_load_failure_callback(self, ep, err)<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 163, in _load_plugins<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service verify_requirements,<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 123, in _load_one_plugin<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service verify_requirements,<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 186, in _load_one_plugin<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service plugin = ep.load(require=verify_requirements)<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2260, in load<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service entry = __import__(self.module_name, globals(),globals(), ['__name__'])<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/tooz/drivers/redis.py", line 27, in <module><br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service import redis<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service ImportError: No module named redis<br>
>> 2016-06-03 08:46:20.552 32101 ERROR oslo_service.service <br>
>><br>
>> 2 . Memory DIMMs DDR3 ( Kingston HyperX 1600 MHZ ) is not a problem<br>
>> My board ASUS Z97-P cannot support more 32 GB. So .... <br>
>><br>
>> 3. i7 4790 surprised me on doing deployment on TripleO Quickstart , in particular, Controller+2xComputes ( --compute-scale 2 )<br>
>><br>
>> Thank you<br>
>> Boris.<br>
>> ________________________________________<br>
>> From: John Trowbridge <trown@redhat.com><br>
>> Sent: Friday, June 3, 2016 8:43 AM<br>
>> To: Boris Derzhavets; John Trowbridge; Lars Kellogg-Stedman<br>
>> Cc: rdo-list<br>
>> Subject: Re: [rdo-list] Tripleo QuickStart HA deployment attempts constantly crash<br>
>><br>
>> So this last one looks like telemetry services went down. You could<br>
>> check the logs on the controllers to see if it was OOM killed. My bet<br>
>> would be this is what is happening.<br>
>><br>
>> The reason that HA is not the default for tripleo-quickstart is exactly<br>
>> this type of issue. It is pretty difficult to fit a full HA deployment<br>
>> of TripleO on a 32G virthost. I think there is near 100% chance that the<br>
>> default HA config will crash when trying to do anything on the<br>
>> deployed overcloud, due to running out of memory.<br>
>><br>
>> I have had some success in my local test setup using KSM [1] on the<br>
>> virthost, and then changing the HA config to give the controllers more<br>
>> memory. This results in overcommiting, but KSM can handle overcommiting<br>
>> without going into swap. It might even be possible to try to setup KSM<br>
>> in the environment setup part of quickstart. I would certainly accept an<br>
>> RFE/patch for this [2,3].<br>
>><br>
>> If you have a larger virthost than 32G, you could similarly bump the<br>
>> memory for the controllers, which should lead to a much higher success rate.<br>
>><br>
>> There is also a feature coming in TripleO [4] that will allow choosing<br>
>> what services get deployed in each role, which will allow us to tweak<br>
>> the tripleo-quickstart HA config to deploy a minimal service layout in<br>
>> order to reduce memory requirements.<br>
>><br>
>> Thanks a ton for giving tripleo-quickstart a go!<br>
>><br>
>> [1] <a id="LPlnk962918" href="https://en.wikipedia.org/wiki/Kernel_same-page_merging">
https://en.wikipedia.org/wiki/Kernel_same-page_merging</a>
<div style="margin-bottom: 20px; overflow: auto; width: 100%; text-indent: 0px;" id="LPBorder_GT_14650260554490.3936381882308414">
<table style="width: 90%; background-color: rgb(255, 255, 255); position: relative; overflow: auto; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top: 1px dotted rgb(200, 200, 200); border-bottom: 1px dotted rgb(200, 200, 200);" id="LPContainer_14650260554440.701349580149489" cellspacing="0">
<tbody>
<tr style="border-spacing: 0px;" valign="top">
<td colspan="2" style="vertical-align: top; position: relative; padding: 0px; display: table-cell;" id="TextCell_14650260554460.18838864386823084">
<div id="LPRemovePreviewContainer_14650260554460.19810186651802908"></div>
<div style="top: 0px; color: rgb(0, 120, 215); font-weight: 400; font-size: 21px; font-family: "wf_segoe-ui_light","Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; line-height: 21px;" id="LPTitle_14650260554460.48416885728657455">
<a target="_blank" href="https://en.wikipedia.org/wiki/Kernel_same-page_merging" style="text-decoration: none;" id="LPUrlAnchor_14650260554470.3672877066423522">Kernel same-page merging - Wikipedia, the free encyclopedia</a></div>
<div style="margin: 10px 0px 16px; color: rgb(102, 102, 102); font-weight: 400; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; line-height: 14px;" id="LPMetadata_14650260554470.629506483522918">
en.wikipedia.org</div>
<div style="display: block; color: rgb(102, 102, 102); font-weight: 400; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; line-height: 20px; max-height: 100px; overflow: hidden;" id="LPDescription_14650260554480.9088606682559274">
In computing, kernel same-page merging (abbreviated as KSM, and also known as kernel shared memory and memory merging) is a kernel feature that makes it possible for ...</div>
</td>
</tr>
</tbody>
</table>
</div>
<br>
>> [2] <a href="https://bugs.launchpad.net/tripleo-quickstart">https://bugs.launchpad.net/tripleo-quickstart</a><br>
>> [3] <a href="https://review.openstack.org/#/q/project:openstack/tripleo-quickstart">
https://review.openstack.org/#/q/project:openstack/tripleo-quickstart</a><br>
>> [4]<br>
>> <a href="https://blueprints.launchpad.net/tripleo/+spec/composable-services-within-roles">
https://blueprints.launchpad.net/tripleo/+spec/composable-services-within-roles</a><br>
>><br>
>> On 06/03/2016 06:20 AM, Boris Derzhavets wrote:<br>
>>> =====================================<br>
>>><br>
>>> Fresh HA deployment attempt<br>
>>><br>
>>> =====================================<br>
>>><br>
>>> [stack@undercloud ~]$ date<br>
>>> Fri Jun 3 10:05:35 UTC 2016<br>
>>> [stack@undercloud ~]$ heat stack-list<br>
>>> +--------------------------------------+------------+-----------------+---------------------+--------------+<br>
>>> | id | stack_name | stack_status | creation_time | updated_time |<br>
>>> +--------------------------------------+------------+-----------------+---------------------+--------------+<br>
>>> | 0c6b8205-be86-4a24-be36-fd4ece956c6d | overcloud | CREATE_COMPLETE | 2016-06-03T08:14:19 | None |<br>
>>> +--------------------------------------+------------+-----------------+---------------------+--------------+<br>
>>> [stack@undercloud ~]$ nova list<br>
>>> +--------------------------------------+-------------------------+--------+------------+-------------+---------------------+<br>
>>> | ID | Name | Status | Task State | Power State | Networks |<br>
>>> +--------------------------------------+-------------------------+--------+------------+-------------+---------------------+<br>
>>> | 6a38b7be-3743-4339-970b-6121e687741d | overcloud-controller-0 | ACTIVE | - | Running | ctlplane=192.0.2.10 |<br>
>>> | 9222dc1b-5974-495b-8b98-b8176ac742f4 | overcloud-controller-1 | ACTIVE | - | Running | ctlplane=192.0.2.9 |<br>
>>> | 76adbb27-220f-42ef-9691-94729ee28749 | overcloud-controller-2 | ACTIVE | - | Running | ctlplane=192.0.2.11 |<br>
>>> | 8f57f7b6-a2d8-4b7b-b435-1c675e63ea84 | overcloud-novacompute-0 | ACTIVE | - | Running | ctlplane=192.0.2.8 |<br>
>>> +--------------------------------------+-------------------------+--------+------------+-------------+---------------------+<br>
>>> [stack@undercloud ~]$ ssh heat-admin@192.0.2.10<br>
>>> Last login: Fri Jun 3 10:01:44 2016 from gateway<br>
>>> [heat-admin@overcloud-controller-0 ~]$ sudo su -<br>
>>> Last login: Fri Jun 3 10:01:49 UTC 2016 on pts/0<br>
>>> [root@overcloud-controller-0 ~]# . keystonerc_admin<br>
>>><br>
>>> [root@overcloud-controller-0 ~]# pcs status<br>
>>> Cluster name: tripleo_cluster<br>
>>> Last updated: Fri Jun 3 10:07:22 2016 Last change: Fri Jun 3 08:50:59 2016 by root via cibadmin on overcloud-controller-0<br>
>>> Stack: corosync<br>
>>> Current DC: overcloud-controller-0 (version 1.1.13-10.el7_2.2-44eb2dd) - partition with quorum<br>
>>> 3 nodes and 123 resources configured<br>
>>><br>
>>> Online: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>><br>
>>> Full list of resources:<br>
>>><br>
>>> ip-192.0.2.6 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0<br>
>>> Clone Set: haproxy-clone [haproxy]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> ip-192.0.2.7 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1<br>
>>> Master/Slave Set: galera-master [galera]<br>
>>> Masters: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: memcached-clone [memcached]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: rabbitmq-clone [rabbitmq]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-core-clone [openstack-core]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Master/Slave Set: redis-master [redis]<br>
>>> Masters: [ overcloud-controller-1 ]<br>
>>> Slaves: [ overcloud-controller-0 overcloud-controller-2 ]<br>
>>> Clone Set: mongod-clone [mongod]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-aodh-evaluator-clone [openstack-aodh-evaluator]<br>
>>> Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-nova-scheduler-clone [openstack-nova-scheduler]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: neutron-l3-agent-clone [neutron-l3-agent]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: neutron-netns-cleanup-clone [neutron-netns-cleanup]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: neutron-ovs-cleanup-clone [neutron-ovs-cleanup]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> openstack-cinder-volume (systemd:openstack-cinder-volume): Started overcloud-controller-2<br>
>>> Clone Set: openstack-heat-engine-clone [openstack-heat-engine]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-ceilometer-api-clone [openstack-ceilometer-api]<br>
>>> Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-aodh-listener-clone [openstack-aodh-listener]<br>
>>> Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: neutron-metadata-agent-clone [neutron-metadata-agent]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-gnocchi-metricd-clone [openstack-gnocchi-metricd]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-aodh-notifier-clone [openstack-aodh-notifier]<br>
>>> Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-heat-api-clone [openstack-heat-api]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-ceilometer-collector-clone [openstack-ceilometer-collector]<br>
>>> Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-glance-api-clone [openstack-glance-api]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-cinder-scheduler-clone [openstack-cinder-scheduler]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-nova-api-clone [openstack-nova-api]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-nova-consoleauth-clone [openstack-nova-consoleauth]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-sahara-api-clone [openstack-sahara-api]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-heat-api-cloudwatch-clone [openstack-heat-api-cloudwatch]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-sahara-engine-clone [openstack-sahara-engine]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-glance-registry-clone [openstack-glance-registry]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-gnocchi-statsd-clone [openstack-gnocchi-statsd]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-ceilometer-notification-clone [openstack-ceilometer-notification]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-cinder-api-clone [openstack-cinder-api]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: neutron-dhcp-agent-clone [neutron-dhcp-agent]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: neutron-openvswitch-agent-clone [neutron-openvswitch-agent]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-nova-novncproxy-clone [openstack-nova-novncproxy]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: delay-clone [delay]<br>
>>> Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: neutron-server-clone [neutron-server]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-ceilometer-central-clone [openstack-ceilometer-central]<br>
>>> Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: httpd-clone [httpd]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-heat-api-cfn-clone [openstack-heat-api-cfn]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>> Clone Set: openstack-nova-conductor-clone [openstack-nova-conductor]<br>
>>> Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]<br>
>>><br>
>>> Failed Actions:<br>
>>> * openstack-aodh-evaluator_monitor_60000 on overcloud-controller-1 'not running' (7): call=76, status=complete, exitreason='none',<br>
>>> last-rc-change='Fri Jun 3 08:47:22 2016', queued=0ms, exec=0ms<br>
>>> * openstack-ceilometer-central_start_0 on overcloud-controller-1 'not running' (7): call=290, status=complete, exitreason='none',<br>
>>> last-rc-change='Fri Jun 3 08:51:18 2016', queued=0ms, exec=2132ms<br>
>>> * openstack-aodh-evaluator_monitor_60000 on overcloud-controller-2 'not running' (7): call=76, status=complete, exitreason='none',<br>
>>> last-rc-change='Fri Jun 3 08:47:16 2016', queued=0ms, exec=0ms<br>
>>> * openstack-ceilometer-central_start_0 on overcloud-controller-2 'not running' (7): call=292, status=complete, exitreason='none',<br>
>>> last-rc-change='Fri Jun 3 08:51:31 2016', queued=0ms, exec=2102ms<br>
>>> * openstack-aodh-evaluator_monitor_60000 on overcloud-controller-0 'not running' (7): call=77, status=complete, exitreason='none',<br>
>>> last-rc-change='Fri Jun 3 08:47:19 2016', queued=0ms, exec=0ms<br>
>>> * openstack-ceilometer-central_start_0 on overcloud-controller-0 'not running' (7): call=270, status=complete, exitreason='none',<br>
>>> last-rc-change='Fri Jun 3 08:50:02 2016', queued=0ms, exec=2199ms<br>
>>><br>
>>><br>
>>> PCSD Status:<br>
>>> overcloud-controller-0: Online<br>
>>> overcloud-controller-1: Online<br>
>>> overcloud-controller-2: Online<br>
>>><br>
>>> Daemon Status:<br>
>>> corosync: active/enabled<br>
>>> pacemaker: active/enabled<br>
>>> pcsd: active/enabled<br>
>>><br>
>>><br>
>>> ________________________________<br>
>>> From: rdo-list-bounces@redhat.com <rdo-list-bounces@redhat.com> on behalf of Boris Derzhavets <bderzhavets@hotmail.com><br>
>>> Sent: Monday, May 30, 2016 4:56 AM<br>
>>> To: John Trowbridge; Lars Kellogg-Stedman<br>
>>> Cc: rdo-list<br>
>>> Subject: Re: [rdo-list] Tripleo QuickStart HA deployment attempts constantly crash<br>
>>><br>
>>><br>
>>> Done one more time :-<br>
>>><br>
>>><br>
>>> [stack@undercloud ~]$ heat deployment-show 9cc8087a-6d82-4261-8a13-ee8c46e3a02d<br>
>>><br>
>>> Uploaded here :-<br>
>>><br>
>>> <a href="http://textuploader.com/5bm5v">http://textuploader.com/5bm5v</a><br>
>>> ________________________________<br>
>>> From: rdo-list-bounces@redhat.com <rdo-list-bounces@redhat.com> on behalf of Boris Derzhavets <bderzhavets@hotmail.com><br>
>>> Sent: Sunday, May 29, 2016 3:39 AM<br>
>>> To: John Trowbridge; Lars Kellogg-Stedman<br>
>>> Cc: rdo-list<br>
>>> Subject: [rdo-list] Tripleo QuickStart HA deploymemt attempts constantly crash<br>
>>><br>
>>><br>
>>> Error every time is the same :-<br>
>>><br>
>>><br>
>>> 2016-05-29 07:20:17 [0]: CREATE_FAILED Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6<br>
>>> 2016-05-29 07:20:18 [0]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:20:18 [overcloud-ControllerNodesPostDeployment-dzawjmjyaidt-ControllerServicesBaseDeployment_Step2-ufz2ccs5egd7]: CREATE_FAILED Resource CREATE failed: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited
with non-zero status code: 6<br>
>>> 2016-05-29 07:20:18 [0]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:20:19 [ControllerServicesBaseDeployment_Step2]: CREATE_FAILED Error: resources.ControllerServicesBaseDeployment_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6<br>
>>> 2016-05-29 07:20:19 [0]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:20:19 [0]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:20:20 [ControllerDeployment]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:20:20 [overcloud-ControllerNodesPostDeployment-dzawjmjyaidt]: CREATE_FAILED Resource CREATE failed: Error: resources.ControllerServicesBaseDeployment_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with
non-zero status code: 6<br>
>>> 2016-05-29 07:20:21 [ControllerNodesPostDeployment]: CREATE_FAILED Error: resources.ControllerNodesPostDeployment.resources.ControllerServicesBaseDeployment_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero
status code: 6<br>
>>> 2016-05-29 07:20:21 [0]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:20:22 [NetworkDeployment]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:20:22 [0]: SIGNAL_COMPLETE Unknown<br>
>>> 2016-05-29 07:24:22 [ComputeNodesPostDeployment]: CREATE_FAILED CREATE aborted<br>
>>> 2016-05-29 07:24:22 [overcloud]: CREATE_FAILED Resource CREATE failed: Error: resources.ControllerNodesPostDeployment.resources.ControllerServicesBaseDeployment_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with
non-zero status code: 6<br>
>>> Stack overcloud CREATE_FAILED<br>
>>> Deployment failed: Heat Stack create failed.<br>
>>> + heat stack-list<br>
>>> + grep -q CREATE_FAILED<br>
>>> + deploy_status=1<br>
>>> ++ heat resource-list --nested-depth 5 overcloud<br>
>>> ++ grep FAILED<br>
>>> ++ grep 'StructuredDeployment '<br>
>>> ++ cut -d '|' -f3<br>
>>> + for failed in '$(heat resource-list --nested-depth 5 overcloud | grep FAILED |<br>
>>> grep '\''StructuredDeployment '\'' | cut -d '\''|'\'' -f3)'<br>
>>> + heat deployment-show 66bd3fbe-296b-4f88-87a7-5ceafd05c1d3<br>
>>> + exit 1<br>
>>><br>
>>><br>
>>> Minimal configuration deployments run with no errors and build completely functional environment.<br>
>>><br>
>>><br>
>>> However, template :-<br>
>>><br>
>>><br>
>>> #################################<br>
>>> # Test Controller + 2*Compute nodes<br>
>>> #################################<br>
>>> control_memory: 6144<br>
>>> compute_memory: 6144<br>
>>><br>
>>> undercloud_memory: 8192<br>
>>><br>
>>> # Giving the undercloud additional CPUs can greatly improve heat's<br>
>>> # performance (and result in a shorter deploy time).<br>
>>> undercloud_vcpu: 4<br>
>>><br>
>>> # We set introspection to true and use only the minimal amount of nodes<br>
>>> # for this job, but test all defaults otherwise.<br>
>>> step_introspect: true<br>
>>><br>
>>> # Define a single controller node and a single compute node.<br>
>>> overcloud_nodes:<br>
>>> - name: control_0<br>
>>> flavor: control<br>
>>><br>
>>> - name: compute_0<br>
>>> flavor: compute<br>
>>><br>
>>> - name: compute_1<br>
>>> flavor: compute<br>
>>><br>
>>> # Tell tripleo how we want things done.<br>
>>> extra_args: >-<br>
>>> --neutron-network-type vxlan<br>
>>> --neutron-tunnel-types vxlan<br>
>>> --ntp-server pool.ntp.org<br>
>>><br>
>>> network_isolation: true<br>
>>><br>
>>><br>
>>> Picks up new memory setting but doesn't create second Compute Node.<br>
>>><br>
>>> Every time just Controller && (1)* Compute.<br>
>>><br>
>>><br>
>>> HW - i74790 , 32 GB RAM<br>
>>><br>
>>><br>
>>> Thanks.<br>
>>><br>
>>> Boris<br>
>>><br>
>>> ________________________________<br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> rdo-list mailing list<br>
>>> rdo-list@redhat.com<br>
>>> <a href="https://www.redhat.com/mailman/listinfo/rdo-list">https://www.redhat.com/mailman/listinfo/rdo-list</a><br>
>>><br>
>>> To unsubscribe: rdo-list-unsubscribe@redhat.com<br>
>>><br>
> <br>
> _______________________________________________<br>
> rdo-list mailing list<br>
> rdo-list@redhat.com<br>
> <a href="https://www.redhat.com/mailman/listinfo/rdo-list">https://www.redhat.com/mailman/listinfo/rdo-list</a><br>
> <br>
> To unsubscribe: rdo-list-unsubscribe@redhat.com<br>
> <br>
</div>
</span></font></div>
</div>
</body>
</html>