<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<!--[if lte mso 15]-->
<div id="OwaReferenceAttachments" contenteditable="false">
<table style="padding-bottom: 13px; border-width: 0px; border-style: none;">
<tbody>
<tr valign="top">
<td>
<table style="border-width: 0px 0px 1px 0px; border-color:#C7C7C7; border-style: none none dotted none;">
<tbody>
<tr valign="top">
<td style="padding-bottom:7px;">
<table align="left" style="padding-right: 28px; border-width: 0px; background-color: rgb(255, 255, 255); border-spacing: 0px">
<tbody>
<tr valign="top">
<td style="padding: 0px;">
<div id="OwaReferenceAttachmentDescription" style="padding-left: 3px; font-size: 14px; font-family: 'Segoe UI', 'Segoe WP', 'Segoe UI WPC', Tahoma, Arial, sans-serif; color: rgb(102, 102, 102);">
Boris Derzhavets has shared a OneDrive file with you. To view it, click the link below.
</div>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr valign="top">
<td><a href="https://1drv.ms/u/s!AqjiDzRpwaKogSHAekH8ZluOaclk" target="_blank">
<table align="left" style="padding-right: 28px; padding-bottom:10px; border-width: 0px; height:20px; background-color: rgb(255, 255, 255); border-spacing: 0px">
<tbody>
<tr valign="top">
<td style="padding: 0px;">
<div style="background-color: rgb(255, 255, 255); height: 20px; width: 20px; max-height: 20px;">
<a href="https://1drv.ms/u/s!AqjiDzRpwaKogSHAekH8ZluOaclk" target="_blank"><img width="20" style="border:0px;" src="https://r1.res.office365.com/owa/prem/images/dc-generic_20.png"></a></div>
</td>
<td>
<div id="OwaReferenceAttachmentFileName2" style="padding: 0px 0px 0px 5px; font-size: 14px; font-family: 'Segoe UI', 'Segoe WP', 'Segoe UI WPC', Tahoma, Arial, sans-serif; color: rgb(0, 114, 198);">
<a href="https://1drv.ms/u/s!AqjiDzRpwaKogSHAekH8ZluOaclk" target="_blank" style="text-decoration: none; margin: 0px; font-size: 14px; font-family: 'Segoe UI', 'Segoe WP', 'Segoe UI WPC', Tahoma, Arial, sans-serif; color: rgb(0, 114, 198);">HeatCrash2.txt 1.gz</a></div>
</td>
<td style="display:none;visibility:hidden;" width="0" height="0"><img width="0" height="0" style="visibility:hidden;border:0px;display:none" title="HeatCrash2.txt 1.gz" src="cid:f9f34e43-6742-4ef5-85d5-5b6c91153147"></td>
</tr>
</tbody>
</table>
</a></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
<div id="OwaReferenceAttachmentsEnd" style="display:none;visibility:hidden;"></div>
<!--[endif]-->
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Reattach gzip archive via One Drive<br>
</p>
<br>
<br>
<div style="color: rgb(49, 55, 57);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> rdo-list-bounces@redhat.com <rdo-list-bounces@redhat.com> on behalf of Boris Derzhavets <bderzhavets@hotmail.com><br>
<b>Sent:</b> Wednesday, June 29, 2016 9:36 AM<br>
<b>To:</b> John Trowbridge; shardy@redhat.com<br>
<b>Cc:</b> rdo-list@redhat.com<br>
<b>Subject:</b> [rdo-list] HA overcloud-deploy.sh crashes again ( ControllerOvercloudServicesDeployment_Step4 )</font>
<div> </div>
</div>
<div>
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Attempt to follow steps suggested in <a id="LPlnk168835" href="http://hardysteven.blogspot.ru/2016/06/tripleo-partial-stack-updates.html" class="OWAAutoLink">http://hardysteven.blogspot.ru/2016/06/tripleo-partial-stack-updates.html</a></p>
<p><br>
</p>
<p>./deploy-overstack crashes<br>
</p>
<p><br>
</p>
<p></p>
<div>2016-06-29 12:42:41 [overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk-ControllerOvercloudServicesDeployment_Step4-nzdoizlgrmx2]: CREATE_FAILED Resource CREATE failed: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment
exited with non-zero status code: 6<br>
2016-06-29 12:42:42 [ControllerOvercloudServicesDeployment_Step4]: CREATE_FAILED Error: resources.ControllerOvercloudServicesDeployment_Step4.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6<br>
2016-06-29 12:42:43 [overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk]: CREATE_FAILED Resource CREATE failed: Error: resources.ControllerOvercloudServicesDeployment_Step4.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with
non-zero status code: 6<br>
2016-06-29 12:42:44 [ControllerNodesPostDeployment]: CREATE_FAILED Error: resources.ControllerNodesPostDeployment.resources.ControllerOvercloudServicesDeployment_Step4.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero
status code: 6<br>
2016-06-29 12:42:44 [2]: SIGNAL_COMPLETE Unknown<br>
2016-06-29 12:42:45 [2]: SIGNAL_COMPLETE Unknown<br>
2016-06-29 12:42:45 [2]: SIGNAL_COMPLETE Unknown<br>
2016-06-29 12:42:46 [overcloud]: CREATE_FAILED Resource CREATE failed: Error: resources.ControllerNodesPostDeployment.resources.ControllerOvercloudServicesDeployment_Step4.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with
non-zero status code: 6<br>
2016-06-29 12:42:46 [2]: SIGNAL_COMPLETE Unknown<br>
2016-06-29 12:42:47 [2]: SIGNAL_COMPLETE Unknown<br>
2016-06-29 12:42:47 [ControllerDeployment]: SIGNAL_COMPLETE Unknown<br>
2016-06-29 12:42:48 [NetworkDeployment]: SIGNAL_COMPLETE Unknown<br>
2016-06-29 12:42:48 [2]: SIGNAL_COMPLETE Unknown<br>
Stack overcloud CREATE_FAILED<br>
Deployment failed: Heat Stack create failed.<br>
+ heat stack-list<br>
+ grep -q CREATE_FAILED<br>
+ deploy_status=1<br>
++ heat resource-list --nested-depth 5 overcloud<br>
++ grep FAILED<br>
++ grep 'StructuredDeployment '<br>
++ cut -d '|' -f3<br>
+ for failed in '$(heat resource-list --nested-depth 5 overcloud | grep FAILED |<br>
grep '\''StructuredDeployment '\'' | cut -d '\''|'\'' -f3)'<br>
+ heat deployment-show 655c77fc-6a78-4cca-b4b7-a153a3f4ad52<br>
+ for failed in '$(heat resource-list --nested-depth 5 overcloud | grep FAILED |<br>
grep '\''StructuredDeployment '\'' | cut -d '\''|'\'' -f3)'<br>
+ heat deployment-show 1fe5153c-e017-4ee5-823a-3d1524430c1d<br>
+ for failed in '$(heat resource-list --nested-depth 5 overcloud | grep FAILED |<br>
grep '\''StructuredDeployment '\'' | cut -d '\''|'\'' -f3)'<br>
+ heat deployment-show bf6f25f4-d812-41e9-a7a8-122de619a624<br>
+ exit 1<br>
<br>
*****************************<br>
<span>Troubleshooting steps :-</span><br>
*****************************<br>
<br>
[stack@undercloud ~]$ . stackrc<br>
[stack@undercloud ~]$ heat resource-list overcloud | grep ControllerNodesPost<br>
| ControllerNodesPostDeployment | f1d6a474-c946-46bf-ab0c-2fdaeb55d0b3 | OS::TripleO::ControllerPostDeployment | CREATE_FAILED | 2016-06-29T12:11:21 |<br>
<br>
<br>
[stack@undercloud ~]$ heat stack-list -n | grep "^| f1d6a474-c946-46bf-ab0c-2fdaeb55d0b3"<br>
| f1d6a474-c946-46bf-ab0c-2fdaeb55d0b3 | overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk | CREATE_FAILED | 2016-06-29T12:31:11 | None | 17f82f6e-e0ca-44c6-9058-de82c00d4f79 |<br>
<br>
<br>
<br>
[stack@undercloud ~]$ heat event-list -m f1d6a474-c946-46bf-ab0c-2fdaeb55d0b3 overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk<br>
<br>
+------------------------------------------------------+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------+<br>
| resource_name | id | resource_status_reason
| resource_status | event_time |<br>
+------------------------------------------------------+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------+<br>
| overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk | 10ec0cf9-b3c9-4191-9966-3f4d47f27e2a | Stack CREATE started <br>
. . . . . . . . . . . . . . . . . <br>
Step1,2,3 succeeded<br>
. . . . . . . . . . . . . . . . .<br>
| CREATE_IN_PROGRESS | 2016-06-29T12:31:14 |<br>
| ControllerPuppetConfig | a2a1df33-5106-425c-b16d-8d2df709b19f | state changed
| CREATE_COMPLETE | 2016-06-29T12:35:02 |<br>
| ControllerOvercloudServicesDeployment_Step4 | 1e151333-4de5-4e7b-907c-ea0f42d31a47 | state changed
| CREATE_IN_PROGRESS | 2016-06-29T12:35:03 |<br>
| ControllerOvercloudServicesDeployment_Step4 | 7bf36334-3d92-4554-b6c0-41294a072ab6 | Error: resources.ControllerOvercloudServicesDeployment_Step4.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status
code: 6 | CREATE_FAILED | 2016-06-29T12:42:42 |<br>
| overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk<br>
| e72fb6f4-c2aa-4fe8-9bd1-5f5ad152685c | Resource CREATE failed: Error: resources.ControllerOvercloudServicesDeployment_Step4.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6 | CREATE_FAILED
| 2016-06-29T12:42:43 |<br>
+------------------------------------------------------+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------+<br>
<br>
[stack@undercloud ~]$ heat stack-show overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk | grep NodeConfigIdentifiers<br>
| | "NodeConfigIdentifiers": "{u'deployment_identifier': 1467202276, u'controller_config': {u'1': u'os-apply-config deployment 796df02a-7550-414b-a084-8b591a13e6db completed,Root CA cert injection not enabled.,TLS not enabled.,None,',
u'0': u'os-apply-config deployment 613ec889-d852-470a-8e4c-6e243e1d2033 completed,Root CA cert injection not enabled.,TLS not enabled.,None,', u'2': u'os-apply-config deployment c8b099d0-3af4-4ba0-a056-a0ce60f40e2d completed,Root CA cert injection not enabled.,TLS
not enabled.,None,'}, u'allnodes_extra': u'none'}" |<br>
<br>
However, when stack creating crashed update wouldn't help.<br>
<br>
[stack@undercloud ~]$ heat stack-update -x overcloud-ControllerNodesPostDeployment-2r4tlv5icaxk -e update_env.yaml<br>
ERROR: PATCH update to non-COMPLETE stack is not supported.<br>
<br>
DUE TO :-<br>
<br>
[stack@undercloud ~]$ heat stack-list<br>
+--------------------------------------+------------+---------------+---------------------+--------------+<br>
| id | stack_name | stack_status | creation_time | updated_time |<br>
+--------------------------------------+------------+---------------+---------------------+--------------+<br>
| 17f82f6e-e0ca-44c6-9058-de82c00d4f79 | overcloud | CREATE_FAILED | 2016-06-29T12:11:20 | None |<br>
+--------------------------------------+------------+---------------+---------------------+------<br>
</div>
<p></p>
<p><br>
</p>
<p>Complete error file <span>`heat deployment-show 655c77fc-6a78-4cca-b4b7-a153a3f4ad52` is
</span>attached a gzip archive.</p>
<p><br>
</p>
<p>Thanks.</p>
<p>Boris.<br>
</p>
</div>
</div>
</div>
</div>
</body>
</html>