[rdo-list] RDO TripleO Mitaka Non-HA Overcloud Failing
Marius Cornea
marius at remote-lab.net
Wed Aug 3 20:00:26 UTC 2016
Hi,
Could you please ssh to the nodes, gather the os-collect-config
journals (journalctl -l -u os-collect-config) and attach them here?
Thank you,
Marius
On Wed, Aug 3, 2016 at 8:40 PM, Gunjan, Milind [CTO]
<Milind.Gunjan at sprint.com> wrote:
> Hi All,
>
>
>
> I am currently working on Tripleo Mitaka Openstack deployment on baremetal
> servers:
>
> Undercloud – 1 baremetal server with 2 NIC (1 for provisioning and 2nd for
> external network connectivity)
>
> Controller – 1 baremetal server ( 6 NICs with each openstack VLANs on
> separate NIC)
>
> Compute – 1 baremetal server
>
>
>
> I followed Graeme's instructions here :
> https://www.redhat.com/archives/rdo-list/2016-June/msg00049.html to set up
> Undercloud . Undercloud deployment was successful and all the images
> required for overcloud deployment was properly built as per the instruction.
> I would like to mention that I used libvirt tools to modify the root
> password on overcloud-full.qcow2 and we also modified the grub file to
> include “net.ifnames=0 biosdevname=0” to restore old interface naming.
>
>
>
> I was able to successfully introspect 2 serves to be used for controller and
> compute nodes. Also , we added the serial device discovered during
> introspection as root device:
>
> ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add
> properties/root_device='{"serial": "618e728372833010c79bead9066f0f9e"}'
>
> ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add
> properties/root_device='{"serial": "618e7283728347101f2107b511603adc"}'
>
>
>
> Next, we added compute and control tag to respective introspected node with
> local boot option:
>
>
>
> ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add
> properties/capabilities='profile:control,boot_option:local'
>
> ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add
> properties/capabilities='profile:compute,boot_option:local'
>
>
>
> We used multiple NIC templates for control and compute node which has been
> attached along with network-environment.yaml file. Default network isolation
> template file has been used.
>
>
>
>
>
> Deployment script looks like this :
>
> #!/bin/bash
>
> DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
>
> template_base_dir="$DIR"
>
> ntpserver=<sprint.ntp.server.ip> #Sprint LAB
>
> openstack overcloud deploy --templates \
>
> -e
> /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml
> \
>
> -e ${template_base_dir}/environments/network-environment.yaml \
>
> --control-flavor control --compute-flavor compute \
>
> --control-scale 1 --compute-scale 1 \
>
> --ntp-server $ntpserver \
>
> --neutron-network-type vxlan --neutron-tunnel-types vxlan --debug
>
>
>
> Heat stack deployment goes on more really long time (more than 4 hours) and
> gets stuck at postdeployment configurations. Please find below the capture
> during install :
>
>
>
>
>
> Every 2.0s: ironic node-list && nova list && heat stack-list && heat
> resource-list -n5 overcloud | grep -vi complete
> Wed Aug 3 17:33:37 2016
>
>
>
> +--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
>
> | UUID | Name | Instance UUID
> | Power State | Provisioning State | Maintenance |
>
> +--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
>
> | 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 | None |
> 9e7aae15-cabc-4489-a1b2-778915a78df2 | power on | active |
> False |
>
> | afcfbee3-3108-48da-a6da-aba8f422642c | None |
> c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | power on | active |
> False |
>
> +--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
>
> +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
>
> | ID | Name | Status |
> Task State | Power State | Networks |
>
> +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
>
> | 9e7aae15-cabc-4489-a1b2-778915a78df2 | overcloud-controller-0 | ACTIVE |
> - | Running | ctlplane=192.168.149.9 |
>
> | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | overcloud-novacompute-0 | ACTIVE |
> - | Running | ctlplane=192.168.149.8 |
>
> +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
>
> +--------------------------------------+------------+---------------+---------------------+--------------+
>
> | id | stack_name | stack_status |
> creation_time | updated_time |
>
> +--------------------------------------+------------+---------------+---------------------+--------------+
>
> | 26ee0150-4cfa-4268-9107-8bfbf6712913 | overcloud | CREATE_FAILED |
> 2016-08-03T08:11:34 | None |
>
> +--------------------------------------+------------+---------------+---------------------+--------------+
>
> +---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------
>
> ---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+
>
> | resource_name | physical_resource_id
> | resource_type
>
> | resource_status | updated_time | stack_name
> |
>
> +---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------
>
> ---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+
>
> | ComputeNodesPostDeployment |
> 3797aec6-e543-4dda-9cd1-c7261e827a64 |
> OS::TripleO::ComputePostDeployment
>
> | CREATE_FAILED | 2016-08-03T08:11:35 | overcloud
> |
>
> | ControllerNodesPostDeployment |
> 6ad9f88c-5c55-4125-97f1-eb0e33329d16 |
> OS::TripleO::ControllerPostDeployment
>
> | CREATE_FAILED | 2016-08-03T08:11:35 | overcloud
> |
>
> | ComputePuppetDeployment |
> 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f |
> OS::Heat::StructuredDeployments
>
> | CREATE_FAILED | 2016-08-03T08:29:19 |
> overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy
> |
>
> | ControllerOvercloudServicesDeployment_Step4 |
> 15509f59-ff28-43af-95dd-6247a6a32c2d |
> OS::Heat::StructuredDeployments
>
> | CREATE_FAILED | 2016-08-03T08:29:20 |
> overcloud-ControllerNodesPostDeployment-35y7uafngfwj
> |
>
> | 0 |
> 7cd0aa3d-742f-4e78-99ca-b2a575913f8e |
> OS::Heat::StructuredDeployment
>
> | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 |
> overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy-ComputePuppetDeployment-cpahcct3tfw3
> |
>
> | 0 |
> 5e9308f7-c3a9-4a94-a017-e1acb694c036 |
> OS::Heat::StructuredDeployment
>
>
>
>
>
> [stack at mitaka-uc ~]$ openstack software deployment show
> 5e9308f7-c3a9-4a94-a017-e1acb694c036
>
> +---------------+--------------------------------------+
>
> | Field | Value |
>
> +---------------+--------------------------------------+
>
> | id | 5e9308f7-c3a9-4a94-a017-e1acb694c036 |
>
> | server_id | 9e7aae15-cabc-4489-a1b2-778915a78df2 |
>
> | config_id | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 |
>
> | creation_time | 2016-08-03T08:32:10 |
>
> | updated_time | |
>
> | status | IN_PROGRESS |
>
> | status_reason | Deploy data available |
>
> | input_values | {} |
>
> | action | CREATE |
>
> +---------------+--------------------------------------+
>
>
>
> [stack at mitaka-uc ~]$ openstack software deployment show --long
> 5e9308f7-c3a9-4a94-a017-e1acb694c036
>
> +---------------+--------------------------------------+
>
> | Field | Value |
>
> +---------------+--------------------------------------+
>
> | id | 5e9308f7-c3a9-4a94-a017-e1acb694c036 |
>
> | server_id | 9e7aae15-cabc-4489-a1b2-778915a78df2 |
>
> | config_id | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 |
>
> | creation_time | 2016-08-03T08:32:10 |
>
> | updated_time | |
>
> | status | IN_PROGRESS |
>
> | status_reason | Deploy data available |
>
> | input_values | {} |
>
> | action | CREATE |
>
> | output_values | None |
>
> +---------------+--------------------------------------+
>
>
>
> [stack at mitaka-uc ~]$ openstack stack resource list
> 3797aec6-e543-4dda-9cd1-c7261e827a64
>
> +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+
>
> | resource_name | physical_resource_id |
> resource_type | resource_status |
> updated_time |
>
> +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+
>
> | ComputeArtifactsConfig | a33cd04d-61ab-4429-8565-182409c2b97f |
> file:///usr/share/openstack-tripleo-heat- | CREATE_COMPLETE |
> 2016-08-03T08:29:19 |
>
> | | |
> templates/puppet/deploy-artifacts.yaml | |
> |
>
> | ComputePuppetConfig | 5bb712b0-5358-46c7-a444-f9adedfedd50 |
> OS::Heat::SoftwareConfig | CREATE_COMPLETE |
> 2016-08-03T08:29:19 |
>
> | ComputePuppetDeployment | 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f |
> OS::Heat::StructuredDeployments | CREATE_FAILED |
> 2016-08-03T08:29:19 |
>
> | ComputeArtifactsDeploy | 1d13bf34-fc66-4bf1-a3b7-1dd815f58f5a |
> OS::Heat::StructuredDeployments | CREATE_COMPLETE |
> 2016-08-03T08:29:19 |
>
> | ExtraConfig | |
> OS::TripleO::NodeExtraConfigPost | INIT_COMPLETE |
> 2016-08-03T08:29:19 |
>
> +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+
>
>
>
> [stack at mitaka-uc ~]$ openstack stack resource list
> 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f
>
> +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+
>
> | resource_name | physical_resource_id | resource_type
> | resource_status | updated_time |
>
> +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+
>
> | 0 | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e |
> OS::Heat::StructuredDeployment | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 |
>
> +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+
>
> [stack at mitaka-uc ~]$ openstack software deployment show
> 7cd0aa3d-742f-4e78-99ca-b2a575913f8e
>
> +---------------+--------------------------------------+
>
> | Field | Value |
>
> +---------------+--------------------------------------+
>
> | id | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e |
>
> | server_id | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a |
>
> | config_id | 24e5c0db-f84f-4a94-8f8e-8e38e73ccc86 |
>
> | creation_time | 2016-08-03T08:30:05 |
>
> | updated_time | |
>
> | status | IN_PROGRESS |
>
> | status_reason | Deploy data available |
>
> | input_values | {} |
>
> | action | CREATE |
>
> +---------------+--------------------------------------+
>
>
>
> Keystonerc file was not generated. Please find below openstack status
> command result on controller and compute.
>
>
>
> [heat-admin at overcloud-controller-0 ~]$ openstack-status
>
> == Nova services ==
>
> openstack-nova-api: active
>
> openstack-nova-compute: inactive (disabled on boot)
>
> openstack-nova-network: inactive (disabled on boot)
>
> openstack-nova-scheduler: activating(disabled on boot)
>
> openstack-nova-cert: active
>
> openstack-nova-conductor: active
>
> openstack-nova-console: inactive (disabled on boot)
>
> openstack-nova-consoleauth: active
>
> openstack-nova-xvpvncproxy: inactive (disabled on boot)
>
> == Glance services ==
>
> openstack-glance-api: active
>
> openstack-glance-registry: active
>
> == Keystone service ==
>
> openstack-keystone: inactive (disabled on boot)
>
> == Horizon service ==
>
> openstack-dashboard: uncontactable
>
> == neutron services ==
>
> neutron-server: failed (disabled on boot)
>
> neutron-dhcp-agent: inactive (disabled on boot)
>
> neutron-l3-agent: inactive (disabled on boot)
>
> neutron-metadata-agent: inactive (disabled on boot)
>
> neutron-lbaas-agent: inactive (disabled on boot)
>
> neutron-openvswitch-agent: inactive (disabled on boot)
>
> neutron-metering-agent: inactive (disabled on boot)
>
> == Swift services ==
>
> openstack-swift-proxy: active
>
> openstack-swift-account: active
>
> openstack-swift-container: active
>
> openstack-swift-object: active
>
> == Cinder services ==
>
> openstack-cinder-api: active
>
> openstack-cinder-scheduler: active
>
> openstack-cinder-volume: active
>
> openstack-cinder-backup: inactive (disabled on boot)
>
> == Ceilometer services ==
>
> openstack-ceilometer-api: active
>
> openstack-ceilometer-central: active
>
> openstack-ceilometer-compute: inactive (disabled on boot)
>
> openstack-ceilometer-collector: active
>
> openstack-ceilometer-notification: active
>
> == Heat services ==
>
> openstack-heat-api: inactive (disabled on boot)
>
> openstack-heat-api-cfn: active
>
> openstack-heat-api-cloudwatch: inactive (disabled on boot)
>
> openstack-heat-engine: inactive (disabled on boot)
>
> == Sahara services ==
>
> openstack-sahara-api: active
>
> openstack-sahara-engine: active
>
> == Support services ==
>
> libvirtd: active
>
> openvswitch: active
>
> dbus: active
>
> target: active
>
> rabbitmq-server: active
>
> memcached: active
>
>
>
>
>
> [heat-admin at overcloud-novacompute-0 ~]$ openstack-status
>
> == Nova services ==
>
> openstack-nova-api: inactive (disabled on boot)
>
> openstack-nova-compute: activating(disabled on boot)
>
> openstack-nova-network: inactive (disabled on boot)
>
> openstack-nova-scheduler: inactive (disabled on boot)
>
> openstack-nova-cert: inactive (disabled on boot)
>
> openstack-nova-conductor: inactive (disabled on boot)
>
> openstack-nova-console: inactive (disabled on boot)
>
> openstack-nova-consoleauth: inactive (disabled on boot)
>
> openstack-nova-xvpvncproxy: inactive (disabled on boot)
>
> == Glance services ==
>
> openstack-glance-api: inactive (disabled on boot)
>
> openstack-glance-registry: inactive (disabled on boot)
>
> == Keystone service ==
>
> openstack-keystone: inactive (disabled on boot)
>
> == Horizon service ==
>
> openstack-dashboard: uncontactable
>
> == neutron services ==
>
> neutron-server: inactive (disabled on boot)
>
> neutron-dhcp-agent: inactive (disabled on boot)
>
> neutron-l3-agent: inactive (disabled on boot)
>
> neutron-metadata-agent: inactive (disabled on boot)
>
> neutron-lbaas-agent: inactive (disabled on boot)
>
> neutron-openvswitch-agent: active
>
> neutron-metering-agent: inactive (disabled on boot)
>
> == Swift services ==
>
> openstack-swift-proxy: inactive (disabled on boot)
>
> openstack-swift-account: inactive (disabled on boot)
>
> openstack-swift-container: inactive (disabled on boot)
>
> openstack-swift-object: inactive (disabled on boot)
>
> == Cinder services ==
>
> openstack-cinder-api: inactive (disabled on boot)
>
> openstack-cinder-scheduler: inactive (disabled on boot)
>
> openstack-cinder-volume: inactive (disabled on boot)
>
> openstack-cinder-backup: inactive (disabled on boot)
>
> == Ceilometer services ==
>
> openstack-ceilometer-api: inactive (disabled on boot)
>
> openstack-ceilometer-central: inactive (disabled on boot)
>
> openstack-ceilometer-compute: inactive (disabled on boot)
>
> openstack-ceilometer-collector: inactive (disabled on boot)
>
> openstack-ceilometer-notification: inactive (disabled on boot)
>
> == Heat services ==
>
> openstack-heat-api: inactive (disabled on boot)
>
> openstack-heat-api-cfn: inactive (disabled on boot)
>
> openstack-heat-api-cloudwatch: inactive (disabled on boot)
>
> openstack-heat-engine: inactive (disabled on boot)
>
> == Sahara services ==
>
> openstack-sahara-all: inactive (disabled on boot)
>
> == Support services ==
>
> libvirtd: active
>
> openvswitch: active
>
> dbus: active
>
> rabbitmq-server: inactive (disabled on boot)
>
> memcached: inactive (disabled on boot)
>
>
>
>
>
>
>
> Please let me know if there is any other logs which I can provide that can
> help in troubleshooting.
>
>
>
>
>
> Thanks a lot in Advance for your help and support.
>
>
>
> Best Regards,
>
> Milind Gunjan
>
>
>
>
> ________________________________
>
> This e-mail may contain Sprint proprietary information intended for the sole
> use of the recipient(s). Any use by others is prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies of the
> message.
>
> _______________________________________________
> rdo-list mailing list
> rdo-list at redhat.com
> https://www.redhat.com/mailman/listinfo/rdo-list
>
> To unsubscribe: rdo-list-unsubscribe at redhat.com
More information about the dev
mailing list