Re: [rdo-list] [tripleo] Troubles deploying Libery with HA setup

Hello, I'm deploying Liberty on a set of server using tripleo. I imported and correctly introspected 6 nodes. First i did a setup with 1 controller node and 3 computing nodes. No issues during this deploy, everything went fine. On this deployment i used custom hostname formats (defined via yaml environment file) and custom dns_domain (changed in /etc/nova/nova.conf and /etc/neutron/dhcp_agent.ini and all /usr/share/openstack-tripleo-heat-templates/*.yaml files that contains localdomain) Now I'm working to have a 3 controller setup with HA. I'm running this command to deploy: openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/network- isolation.yaml -e ~/templates/network-environment.yaml -e ~/templates/puppet-ceph-external.yaml -e ~/hostname-nostri.yaml --neutron-bridge-mappings datacentre:br-ex,storage-pub:br-stg-pub -e /usr/share/openstack-tripleo-heat-templates/environments/puppet- pacemaker.yaml --ntp-server timesrv1 --control-scale 3 --compute-scale 3 --ceph-storage-scale 0 --control-flavor controlhp --compute-flavor computehp --neutron-network-type vxlan --neutron-tunnel-types vxlan --verbose --debug --log-file overcloud_$(date +%T).log I see that stack creation starts and correctly deploy os on the nodes. Nodes are named according to ControllerHostnameFormat and ComputeHostnameFormat that i specified into ~/hostname-nostri.yaml. Everything goes well until HA configuration starts. I see this stack creation failing: overcloud-ControllerNodesPostDeployment-bbp3c47jgau2- ControllerLoadBalancerDeployment_Step1-sla6ce7n2arq The error message of the deployment command is: Stack failed with status: Resource CREATE failed: resources.ControllerLoadBalancerDeployment_Step1: resources.ControllerNodesPostDeployment.Error: resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6 Heat Stack create failed. If i go in depth with heat deployment-show i see that all resources report this deploy_stderr: Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\ Error: Command exceeded timeout Error: /Stage[main]/Pacemaker::Corosync/Exec[auth-successful-across- all-nodes]/returns: change from notrun to 0 failed: Command exceeded timeout Warning: /Stage[main]/Pacemaker::Corosync/Exec[Create Cluster tripleo_cluster]: Skipping because of failed dependencies Warning: /Stage[main]/Pacemaker::Corosync/Exec[Start Cluster tripleo_cluster]: Skipping because of failed dependencies Warning: /Stage[main]/Pacemaker::Corosync/Exec[wait-for-settle]: Skipping because of failed dependencies Warning: /Stage[main]/Pacemaker::Corosync/Notify[pacemaker settled]: Skipping because of failed dependencies Warning: /Stage[main]/Pacemaker::Stonith/Exec[Disable STONITH]: Skipping because of failed dependencies", I see nodes are stuck on puppet step "auth-successful-across-all-nodes" (defined in /etc/puppet/modules/pacemaker/manifests/corosync.pp) /usr/bin/python2 /usr/sbin/pcs cluster auth opsctrl0 opsctrl1 opsctrl2 -u hacluster -p PASSWORD --force I suppose that the problem is due to corosync service not yet started. But as far as i can see corosync will never start because /etc/corosync/corosync.conf file is missing. in /etc/hosts opsctrl{0-2} are correctly defined and each host can talk with the others. I'm stuck and i don't know what to do. Anyone had similar issues? I'm missing something that i have to enter in the configuration? I'm using a in-house mirror of rdo-release made on April 20, and custom images build from rhel image and this repository. A week ago i tried with latest repository but had the same error (and also others, that's why i returned to this older mirror that was working). This is the list of packages installed from rdo-release repository: crudini-0.7-1.el7.noarch dib-utils-0.0.9-1.el7.noarch dibbler-client-1.0.1-0.RC1.2.el7.x86_64 diskimage-builder-1.4.0-1.el7.noarch erlang-asn1-R16B-03.16.el7.x86_64 erlang-compiler-R16B-03.16.el7.x86_64 erlang-crypto-R16B-03.16.el7.x86_64 erlang-erts-R16B-03.16.el7.x86_64 erlang-hipe-R16B-03.16.el7.x86_64 erlang-inets-R16B-03.16.el7.x86_64 erlang-kernel-R16B-03.16.el7.x86_64 erlang-mnesia-R16B-03.16.el7.x86_64 erlang-os_mon-R16B-03.16.el7.x86_64 erlang-otp_mibs-R16B-03.16.el7.x86_64 erlang-public_key-R16B-03.16.el7.x86_64 erlang-runtime_tools-R16B-03.16.el7.x86_64 erlang-sasl-R16B-03.16.el7.x86_64 erlang-sd_notify-0.1-1.el7.x86_64 erlang-snmp-R16B-03.16.el7.x86_64 erlang-ssl-R16B-03.16.el7.x86_64 erlang-stdlib-R16B-03.16.el7.x86_64 erlang-syntax_tools-R16B-03.16.el7.x86_64 erlang-tools-R16B-03.16.el7.x86_64 erlang-xmerl-R16B-03.16.el7.x86_64 hiera-1.3.4-1.el7.noarch instack-0.0.8-1.el7.noarch instack-undercloud-2.1.3-1.el7.noarch jq-1.3-2.el7.x86_64 liberasurecode-1.1.1-1.el7.x86_64 libnetfilter_queue-1.0.2-2.el7.x86_64 memcached-1.4.25-1.el7.x86_64 1:openstack-ceilometer-alarm-5.0.2-1.el7.noarch 1:openstack-ceilometer-api-5.0.2-1.el7.noarch 1:openstack-ceilometer-central-5.0.2-1.el7.noarch 1:openstack-ceilometer-collector-5.0.2-1.el7.noarch 1:openstack-ceilometer-common-5.0.2-1.el7.noarch 1:openstack-ceilometer-notification-5.0.2-1.el7.noarch 1:openstack-ceilometer-polling-5.0.2-1.el7.noarch 1:openstack-glance-11.0.1-2.el7.noarch 1:openstack-heat-api-5.0.0-1.el7.noarch 1:openstack-heat-api-cfn-5.0.0-1.el7.noarch 1:openstack-heat-api-cloudwatch-5.0.0-1.el7.noarch 1:openstack-heat-common-5.0.0-1.el7.noarch 1:openstack-heat-engine-5.0.0-1.el7.noarch openstack-heat-templates-0-0.1.20151019.el7.noarch 1:openstack-ironic-api-4.2.2-1.el7.noarch 1:openstack-ironic-common-4.2.2-1.el7.noarch 1:openstack-ironic-conductor-4.2.2-1.el7.noarch openstack-ironic-inspector-2.2.2-1.el7.noarch 1:openstack-keystone-8.0.1-1.el7.noarch 1:openstack-neutron-7.0.3-1.el7.noarch 1:openstack-neutron-common-7.0.3-1.el7.noarch 1:openstack-neutron-ml2-7.0.3-1.el7.noarch 1:openstack-neutron-openvswitch-7.0.3-1.el7.noarch 1:openstack-nova-api-12.0.1-1.el7.noarch 1:openstack-nova-cert-12.0.1-1.el7.noarch 1:openstack-nova-common-12.0.1-1.el7.noarch 1:openstack-nova-compute-12.0.1-1.el7.noarch 1:openstack-nova-conductor-12.0.1-1.el7.noarch 1:openstack-nova-scheduler-12.0.1-1.el7.noarch 1:openstack-puppet-modules-7.0.1-1.el7.noarch openstack-selinux-0.6.41-1.el7.noarch openstack-swift-2.5.0-1.el7.noarch openstack-swift-account-2.5.0-1.el7.noarch openstack-swift-container-2.5.0-1.el7.noarch openstack-swift-object-2.5.0-1.el7.noarch openstack-swift-plugin-swift3-1.7-4.el7.noarch openstack-swift-proxy-2.5.0-1.el7.noarch openstack-tripleo-0.0.6-1.el7.noarch openstack-tripleo-heat-templates-0.8.7-1.el7.noarch openstack-tripleo-image-elements-0.9.7-1.el7.noarch openstack-tripleo-puppet-elements-0.0.2-1.el7.noarch openstack-utils-2015.2-1.el7.noarch openvswitch-2.4.0-1.el7.x86_64 os-apply-config-0.1.32-3.el7.noarch os-cloud-config-0.2.10-2.el7.noarch os-collect-config-0.1.36-4.el7.noarch os-net-config-0.1.5-3.el7.noarch os-refresh-config-0.1.11-2.el7.noarch puppet-3.6.2-3.el7.noarch pyOpenSSL-0.15.1-1.el7.noarch pyparsing-2.0.3-1.el7.noarch pysendfile-2.0.0-5.el7.x86_64 pysnmp-4.2.5-2.el7.noarch pystache-0.5.3-2.el7.noarch python-alembic-0.8.3-3.el7.noarch python-amqp-1.4.6-1.el7.noarch python-anyjson-0.3.3-3.el7.noarch python-automaton-0.7.0-1.el7.noarch python-babel-1.3-6.el7.noarch python-bson-3.0.3-1.el7.x86_64 python-cachetools-1.0.3-2.el7.noarch 1:python-ceilometer-5.0.2-1.el7.noarch python-ceilometerclient-1.5.0-1.el7.noarch python-cinderclient-1.4.0-1.el7.noarch python-cliff-1.15.0-1.el7.noarch python-cliff-tablib-1.1-3.el7.noarch python-cmd2-0.6.8-3.el7.noarch python-contextlib2-0.4.0-1.el7.noarch python-croniter-0.3.4-2.el7.noarch python-dogpile-cache-0.5.7-3.el7.noarch python-dogpile-core-0.4.1-2.el7.noarch python-ecdsa-0.11-3.el7.noarch python-editor-0.4-4.el7.noarch python-elasticsearch-1.4.0-2.el7.noarch python-extras-0.0.3-2.el7.noarch python-fixtures-1.4.0-2.el7.noarch python-futures-3.0.3-1.el7.noarch 1:python-glance-11.0.1-2.el7.noarch python-glance-store-0.9.1-1.el7.noarch 1:python-glanceclient-1.1.0-1.el7.noarch python-heatclient-0.8.0-1.el7.noarch python-httplib2-0.9.2-1.el7.noarch python-idna-2.0-1.el7.noarch python-ipaddress-1.0.7-4.el7.noarch python-ironicclient-0.8.1-1.el7.noarch python-jsonpatch-1.2-2.el7.noarch python-jsonpath-rw-1.2.3-2.el7.noarch python-jsonschema-2.3.0-1.el7.noarch python-keyring-5.0-4.el7.noarch 1:python-keystone-8.0.1-1.el7.noarch 1:python-keystoneclient-1.7.2-1.el7.noarch python-keystonemiddleware-2.3.1-1.el7.noarch 1:python-kombu-3.0.32-1.el7.noarch python-ldappool-1.0-4.el7.noarch python-linecache2-1.0.0-1.el7.noarch python-logutils-0.3.3-3.el7.noarch python-memcached-1.54-3.el7.noarch python-migrate-0.10.0-1.el7.noarch python-mimeparse-0.1.4-1.el7.noarch python-monotonic-0.3-1.el7.noarch python-ncclient-0.4.2-2.el7.noarch python-netaddr-0.7.18-1.el7.noarch python-netifaces-0.10.4-1.el7.x86_64 python-networkx-core-1.10-1.el7.noarch 1:python-neutron-7.0.3-1.el7.noarch python-neutronclient-3.1.0-1.el7.noarch python-nose-1.3.7-7.el7.noarch 1:python-nova-12.0.1-1.el7.noarch 1:python-novaclient-2.30.1-1.el7.noarch python-oauthlib-0.7.2-5.20150520git514cad7.el7.noarch python-openstackclient-1.7.2-1.el7.noarch python-openvswitch-2.4.0-1.el7.noarch python-oslo-cache-0.7.0-1.el7.noarch python-oslo-concurrency-2.6.0-1.el7.noarch python-oslo-db-2.6.0-3.el7.noarch python-oslo-log-1.10.0-1.el7.noarch python-oslo-messaging-2.5.0-1.el7.noarch python-oslo-middleware-2.8.0-1.el7.noarch python-oslo-policy-0.11.0-1.el7.noarch python-oslo-rootwrap-2.3.0-1.el7.noarch python-oslo-service-0.9.0-1.el7.noarch python-oslo-versionedobjects-0.10.0-1.el7.noarch python-oslo-vmware-1.21.0-1.el7.noarch python-osprofiler-0.3.0-1.el7.noarch python-paramiko-1.15.1-1.el7.noarch python-paste-deploy-1.5.2-6.el7.noarch python-pbr-1.8.1-2.el7.noarch python-posix_ipc-0.9.8-1.el7.x86_64 python-prettytable-0.7.2-1.el7.noarch python-proliantutils-2.1.7-1.el7.noarch python-psutil-1.2.1-1.el7.x86_64 python-pycadf-1.1.0-1.el7.noarch python-pyeclib-1.2.0-1.el7.x86_64 python-pyghmi-0.8.0-2.el7.noarch python-pygments-2.0.2-4.el7.noarch python-pymongo-3.0.3-1.el7.x86_64 python-pysaml2-3.0.2-1.el7.noarch python-qpid-0.30-1.el7.noarch python-qpid-common-0.30-1.el7.noarch python-repoze-lru-0.4-3.el7.noarch python-repoze-who-2.1-1.el7.noarch python-requests-2.9.1-2.el7.noarch python-retrying-1.2.3-4.el7.noarch python-rfc3986-0.2.0-1.el7.noarch python-routes-1.13-2.el7.noarch python-saharaclient-0.11.1-1.el7.noarch python-semantic_version-2.4.2-1.el7.noarch python-simplegeneric-0.8-7.el7.noarch python-simplejson-3.5.3-5.el7.x86_64 python-sqlalchemy-1.0.11-1.el7.x86_64 python-sqlparse-0.1.18-5.el7.noarch python-stevedore-1.8.0-1.el7.noarch python-swiftclient-2.6.0-1.el7.noarch python-tablib-0.10.0-1.el7.noarch python-taskflow-1.21.0-1.el7.noarch python-tempita-0.5.1-8.el7.noarch python-testtools-1.8.0-2.el7.noarch python-tooz-1.24.0-1.el7.noarch python-traceback2-1.4.0-2.el7.noarch python-tripleoclient-0.0.11-3.el7.noarch python-troveclient-1.3.0-1.el7.noarch python-unicodecsv-0.14.1-1.el7.noarch python-unittest2-1.0.1-1.el7.noarch python-urllib3-1.13.1-3.el7.noarch python-warlock-1.0.1-1.el7.noarch python-webob-1.4.1-2.el7.noarch python-websockify-0.6.0-2.el7.noarch python-wrapt-1.10.5-3.el7.x86_64 python2-PyMySQL-0.6.7-2.el7.noarch python2-appdirs-1.4.0-4.el7.noarch python2-castellan-0.3.1-1.el7.noarch python2-cffi-1.5.2-1.el7.x86_64 python2-cryptography-1.2.1-3.el7.x86_64 python2-debtcollector-0.8.0-1.el7.noarch python2-eventlet-0.17.4-4.el7.noarch python2-fasteners-0.14.1-4.el7.noarch python2-funcsigs-0.4-2.el7.noarch python2-futurist-0.5.0-1.el7.noarch python2-greenlet-0.4.9-1.el7.x86_64 python2-ironic-inspector-client-1.2.0-2.el7.noarch python2-iso8601-0.1.11-1.el7.noarch python2-jsonpath-rw-ext-0.1.7-1.1.el7.noarch python2-mock-1.3.0-1.el7.noarch python2-os-brick-0.5.0-1.el7.noarch python2-os-client-config-1.7.4-1.el7.noarch 2:python2-oslo-config-2.4.0-1.el7.noarch python2-oslo-context-0.6.0-1.el7.noarch python2-oslo-i18n-2.6.0-1.el7.noarch python2-oslo-reports-0.5.0-1.el7.noarch python2-oslo-serialization-1.9.0-1.el7.noarch python2-oslo-utils-2.5.0-1.el7.noarch python2-passlib-1.6.5-1.el7.noarch python2-pecan-1.0.2-2.el7.noarch python2-pyasn1-0.1.9-6.el7.1.noarch python2-rsa-3.3-2.el7.noarch python2-singledispatch-3.4.0.3-4.el7.noarch python2-suds-0.7-0.1.94664ddd46a6.el7.noarch python2-wsme-0.7.0-2.el7.noarch rabbitmq-server-3.3.5-17.el7.noarch ruby-augeas-0.5.0-1.el7.x86_64 ruby-shadow-1.4.1-23.el7.x86_64 rubygem-rgen-0.6.6-2.el7.noarch tripleo-common-0.0.1-2.el7.noarch Thank you, Luca -- "E' assurdo impiegare gli uomini di intelligenza eccellente per fare calcoli che potrebbero essere affidati a chiunque se si usassero delle macchine" Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716) "Internet è la più grande biblioteca del mondo. Ma il problema è che i libri sono tutti sparsi sul pavimento" John Allen Paulos, Matematico (1945-vivente) Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , <lorenzetto.luca(a)gmail.com&gt; _______________________________________________ rdo-list mailing list rdo-list(a)redhat.com https://www.redhat.com/mailman/listinfo/rdo-list To unsubscribe: rdo-list-unsubscribe(a)redhat.com