<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Hi Dan, Sasha, <br>
<br>
Thanks for your answers and hints. <br>
I looked up the heat/etc log files and stack/node status. <br>
Only thing I found by far is "timed out". I don't know what's
the reason. IPMI looks good. <br>
<br>
Tried with HEAT_INCLUDE_PASSWORD=1 but same error message
(Please try again
with option --include-password or export HEAT_INCLUDE_PASSWORD=1
Authentication required)<br>
<br>
BTW. I only followed the exact instruction as shown in the
guide: (openstack overcloud deploy --templates) No more options.
I thought this is good for a demo deployment. If not sufficient,
which one I should follow? See some of your discussions, but not
very clear. Should I follow the example from
<meta charset="utf-8">
<span style="color: rgb(0, 0, 0); font-family: 'DejaVu Sans',
'Liberation Sans', sans-serif; font-size: small; font-style:
normal; font-variant: normal; font-weight: bold; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: 1; word-spacing: 0px; -webkit-text-stroke-width: 0px;
display: inline !important; float: none; background-color:
rgb(255, 255, 255);"><a class="moz-txt-link-abbreviated" href="mailto:jliberma@redhat.com">jliberma@redhat.com</a>?</span><br>
<br>
Below are my investigation:<br>
By runnig: $ heat resource-list overcloud<br>
Found that just controller and compute are failed:
CREATE_FAILED<br>
<br>
Checked the reason it says: resource_status_reason | CREATE
aborted<br>
<br>
I then logged into the running overcloud nodes (e.g. the
controller): <br>
<br>
<small><small><small><br>
[heat-admin@overcloud-controller-0 ~]$ ifconfig<br>
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu
1500<br>
inet6 fe80::21:5eff:fecd:9df3 prefixlen 64 scopeid
0x20<link><br>
ether 02:21:5e:cd:9d:f3 txqueuelen 0 (Ethernet)<br>
RX packets 29926 bytes 2364154 (2.2 MiB)<br>
RX errors 0 dropped 0 overruns 0 frame 0<br>
TX packets 81 bytes 25614 (25.0 KiB)<br>
TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0<br>
<br>
enp0s29f0u2:
flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500<br>
inet6 fe80::21:5eff:fecd:9df3 prefixlen 64 scopeid
0x20<link><br>
ether 02:21:5e:cd:9d:f3 txqueuelen 1000 (Ethernet)<br>
RX packets 29956 bytes 1947140 (1.8 MiB)<br>
RX errors 0 dropped 0 overruns 0 frame 0<br>
TX packets 102 bytes 28620 (27.9 KiB)<br>
TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0<br>
<br>
enp11s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>
mtu 1500<br>
inet 10.0.6.64 netmask 255.255.0.0 broadcast
10.0.255.255<br>
inet6 fe80::221:5eff:fec9:abd8 prefixlen 64
scopeid 0x20<link><br>
ether 00:21:5e:c9:ab:d8 txqueuelen 1000 (Ethernet)<br>
RX packets 66256 bytes 21109918 (20.1 MiB)<br>
RX errors 0 dropped 0 overruns 0 frame 0<br>
TX packets 35938 bytes 4641202 (4.4 MiB)<br>
TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0<br>
<br>
enp11s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>
mtu 1500<br>
inet6 fe80::221:5eff:fec9:abda prefixlen 64
scopeid 0x20<link><br>
ether 00:21:5e:c9:ab:da txqueuelen 1000 (Ethernet)<br>
RX packets 25429 bytes 2004574 (1.9 MiB)<br>
RX errors 0 dropped 0 overruns 0 frame 0<br>
TX packets 6 bytes 532 (532.0 B)<br>
TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0<br>
<br>
ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu
2044<br>
inet6 fe80::202:c902:23:baf9 prefixlen 64 scopeid
0x20<link><br>
Infiniband hardware address can be incorrect! Please read
BUGS section in ifconfig(8).<br>
infiniband
80:00:04:04:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
txqueuelen 256 (InfiniBand)<br>
RX packets 183678 bytes 10292768 (9.8 MiB)<br>
RX errors 0 dropped 0 overruns 0 frame 0<br>
TX packets 17 bytes 5380 (5.2 KiB)<br>
TX errors 0 dropped 7 overruns 0 carrier 0
collisions 0<br>
<br>
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536<br>
inet 127.0.0.1 netmask 255.0.0.0<br>
inet6 ::1 prefixlen 128 scopeid 0x10<host><br>
loop txqueuelen 0 (Local Loopback)<br>
RX packets 138 bytes 11792 (11.5 KiB)<br>
RX errors 0 dropped 0 overruns 0 frame 0<br>
TX packets 138 bytes 11792 (11.5 KiB)<br>
TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0<br>
<br>
[heat-admin@overcloud-controller-0 ~]$ ovs-vsctl show<br>
ovs-vsctl: unix:/var/run/openvswitch/db.sock: database
connection failed (Permission denied)<br>
[heat-admin@overcloud-controller-0 ~]$ sudo ovs-vsctl show<br>
76e6f8a7-88cf-4920-b133-b4d15a4b9092<br>
Bridge br-ex<br>
Port br-ex<br>
Interface br-ex<br>
type: internal<br>
Port "enp0s29f0u2"<br>
Interface "enp0s29f0u2"<br>
ovs_version: "2.3.1"<br>
[heat-admin@overcloud-controller-0 ~]$ <br>
[heat-admin@overcloud-controller-0 ~]$ ping 10.0.6.65<br>
PING 10.0.6.65 (10.0.6.65) 56(84) bytes of data.<br>
64 bytes from 10.0.6.65: icmp_seq=1 ttl=64 time=0.176 ms<br>
64 bytes from 10.0.6.65: icmp_seq=2 ttl=64 time=0.195 ms<br>
^C<br>
--- 10.0.6.65 ping statistics ---<br>
2 packets transmitted, 2 received, 0% packet loss, time
999ms<br>
rtt min/avg/max/mdev = 0.176/0.185/0.195/0.016 ms<br>
[heat-admin@overcloud-controller-0 ~]$ <br>
[heat-admin@overcloud-controller-0 ~]$ ping 10.0.6.64<br>
PING 10.0.6.64 (10.0.6.64) 56(84) bytes of data.<br>
64 bytes from 10.0.6.64: icmp_seq=1 ttl=64 time=0.015 ms<br>
^C<br>
--- 10.0.6.64 ping statistics ---<br>
1 packets transmitted, 1 received, 0% packet loss, time 0ms<br>
rtt min/avg/max/mdev = 0.015/0.015/0.015/0.000 ms<br>
<br>
[heat-admin@overcloud-controller-0 ~]$ cat
/etc/os-net-config/config.json<br>
{"network_config": [{"use_dhcp": true, "type": "ovs_bridge",
"name": "br-ex", "members": [{"type": "interface", "name":
"nic1", "primary": true}]}]}<br>
[heat-admin@overcloud-controller-0 ~]$ <br>
[heat-admin@overcloud-controller-0 ~]$ <br>
[heat-admin@overcloud-controller-0 ~]$ sudo os-net-config
--debug -c /etc/os-net-config/config.json<br>
[2015/10/15 07:52:08 PM] [INFO] Using config file at:
/etc/os-net-config/config.json<br>
[2015/10/15 07:52:08 PM] [INFO] Using mapping file at:
/etc/os-net-config/mapping.yaml<br>
[2015/10/15 07:52:08 PM] [INFO] Ifcfg net config provider
created.<br>
[2015/10/15 07:52:08 PM] [DEBUG] network_config JSON:
[{'use_dhcp': True, 'type': 'ovs_bridge', 'name': 'br-ex',
'members': [{'type': 'interface', 'name': 'nic1', 'primary':
True}]}]<br>
[2015/10/15 07:52:08 PM] [INFO] nic1 mapped to: enp0s29f0u2<br>
[2015/10/15 07:52:08 PM] [INFO] nic2 mapped to: enp11s0f0<br>
[2015/10/15 07:52:08 PM] [INFO] nic3 mapped to: enp11s0f1<br>
[2015/10/15 07:52:08 PM] [INFO] nic4 mapped to: ib0<br>
[2015/10/15 07:52:08 PM] [INFO] adding bridge: br-ex<br>
[2015/10/15 07:52:08 PM] [DEBUG] bridge data: DEVICE=br-ex<br>
ONBOOT=yes<br>
HOTPLUG=no<br>
DEVICETYPE=ovs<br>
TYPE=OVSBridge<br>
OVSBOOTPROTO=dhcp<br>
OVSDHCPINTERFACES="enp0s29f0u2"<br>
OVS_EXTRA="set bridge br-ex
other-config:hwaddr=02:21:5e:cd:9d:f3"<br>
<br>
[2015/10/15 07:52:08 PM] [INFO] adding interface:
enp0s29f0u2<br>
[2015/10/15 07:52:08 PM] [DEBUG] interface data:
DEVICE=enp0s29f0u2<br>
ONBOOT=yes<br>
HOTPLUG=no<br>
DEVICETYPE=ovs<br>
TYPE=OVSPort<br>
OVS_BRIDGE=br-ex<br>
BOOTPROTO=none<br>
<br>
[2015/10/15 07:52:08 PM] [INFO] applying network configs...<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff file data:<br>
DEVICE=enp0s29f0u2<br>
ONBOOT=yes<br>
HOTPLUG=no<br>
DEVICETYPE=ovs<br>
TYPE=OVSPort<br>
OVS_BRIDGE=br-ex<br>
BOOTPROTO=none<br>
<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff data:<br>
DEVICE=enp0s29f0u2<br>
ONBOOT=yes<br>
HOTPLUG=no<br>
DEVICETYPE=ovs<br>
TYPE=OVSPort<br>
OVS_BRIDGE=br-ex<br>
BOOTPROTO=none<br>
<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff file data:<br>
<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff data:<br>
<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff file data:<br>
DEVICE=br-ex<br>
ONBOOT=yes<br>
HOTPLUG=no<br>
DEVICETYPE=ovs<br>
TYPE=OVSBridge<br>
OVSBOOTPROTO=dhcp<br>
OVSDHCPINTERFACES="enp0s29f0u2"<br>
OVS_EXTRA="set bridge br-ex
other-config:hwaddr=02:21:5e:cd:9d:f3"<br>
<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff data:<br>
DEVICE=br-ex<br>
ONBOOT=yes<br>
HOTPLUG=no<br>
DEVICETYPE=ovs<br>
TYPE=OVSBridge<br>
OVSBOOTPROTO=dhcp<br>
OVSDHCPINTERFACES="enp0s29f0u2"<br>
OVS_EXTRA="set bridge br-ex
other-config:hwaddr=02:21:5e:cd:9d:f3"<br>
<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff file data:<br>
<br>
[2015/10/15 07:52:08 PM] [DEBUG] Diff data:</small></small></small><br>
<br>
<small><br>
<br>
[heat-admin@overcloud-controller-0 ~]$ openstack-status<br>
== Nova services ==<br>
openstack-nova-api: inactive (disabled on
boot)<br>
openstack-nova-cert: inactive (disabled on
boot)<br>
openstack-nova-compute: inactive (disabled on
boot)<br>
openstack-nova-network: inactive (disabled on
boot)<br>
openstack-nova-scheduler: inactive (disabled on
boot)<br>
openstack-nova-conductor: inactive (disabled on
boot)<br>
== Glance services ==<br>
openstack-glance-api: inactive (disabled on
boot)<br>
openstack-glance-registry: inactive (disabled on
boot)<br>
== Keystone service ==<br>
openstack-keystone: inactive (disabled on
boot)<br>
== Horizon service ==<br>
openstack-dashboard: uncontactable<br>
== neutron services ==<br>
neutron-server: inactive (disabled on
boot)<br>
neutron-dhcp-agent: inactive (disabled on
boot)<br>
neutron-l3-agent: inactive (disabled on
boot)<br>
neutron-metadata-agent: inactive (disabled on
boot)<br>
neutron-lbaas-agent: inactive (disabled on
boot)<br>
neutron-openvswitch-agent: inactive (disabled on
boot)<br>
neutron-metering-agent: inactive (disabled on
boot)<br>
== Swift services ==<br>
openstack-swift-proxy: inactive (disabled on
boot)<br>
openstack-swift-account: inactive (disabled on
boot)<br>
openstack-swift-container: inactive (disabled on
boot)<br>
openstack-swift-object: inactive (disabled on
boot)<br>
== Cinder services ==<br>
openstack-cinder-api: inactive (disabled on
boot)<br>
openstack-cinder-scheduler: inactive (disabled on
boot)<br>
openstack-cinder-volume: inactive (disabled on
boot)<br>
openstack-cinder-backup: inactive (disabled on
boot)<br>
== Ceilometer services ==<br>
openstack-ceilometer-api: inactive (disabled on
boot)<br>
openstack-ceilometer-central: inactive (disabled on
boot)<br>
openstack-ceilometer-compute: inactive (disabled on
boot)<br>
openstack-ceilometer-collector: inactive (disabled on
boot)<br>
openstack-ceilometer-alarm-notifier: inactive (disabled on
boot)<br>
openstack-ceilometer-alarm-evaluator: inactive (disabled on
boot)<br>
openstack-ceilometer-notification: inactive (disabled on
boot)<br>
== Heat services ==<br>
openstack-heat-api: inactive (disabled on
boot)<br>
openstack-heat-api-cfn: inactive (disabled on
boot)<br>
openstack-heat-api-cloudwatch: inactive (disabled on
boot)<br>
openstack-heat-engine: inactive (disabled on
boot)<br>
== Support services ==<br>
libvirtd: active<br>
openvswitch: active<br>
dbus: active<br>
rabbitmq-server: inactive (disabled on
boot)<br>
memcached: inactive (disabled on
boot)<br>
== Keystone users ==<br>
Warning keystonerc not sourced</small><br>
<br>
<br>
<br>
<br>
Thanks,<br>
<br>
Erming<br>
<br>
<br>
On 10/14/15, 5:23 PM, Dan Sneddon wrote:<br>
</div>
<blockquote cite="mid:561EE408.6030302@redhat.com" type="cite">
<pre wrap="">On 10/14/2015 03:03 PM, Erming Pei wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
I am deploying the overcloud in baremetal way and after a couple of
hours, it showed:
$ openstack overcloud deploy --templates
Deploying templates in the directory
/usr/share/openstack-tripleo-heat-templates
^[[A^[[BERROR: openstack ERROR: Authentication failed. Please try again
with option --include-password or export HEAT_INCLUDE_PASSWORD=1
Authentication required
But I checked the nodes are now running:
[stack@gcloudcon-3 ~]$ nova list
+--------------------------------------+-------------------------+--------+------------+-------------+--------------------+
| ID | Name |
Status | Task State | Power State | Networks |
+--------------------------------------+-------------------------+--------+------------+-------------+--------------------+
| 1ba04ac0-fe2b-4318-aa31-2e5f4d8422a6 | overcloud-controller-0 |
ACTIVE | - | Running | ctlplane=10.0.6.60 |
| c152ba59-3aed-4fb0-81fa-e3fed7e35cf6 | overcloud-novacompute-0 |
ACTIVE | - | Running | ctlplane=10.0.6.61 |
+--------------------------------------+-------------------------+--------+------------+-------------+--------------------+
1. Should I re-deploy the nodes or there is a way to do update/makeup
for the authentication issue?
2.
I don't know how to access to the nodes.
There is not an overcloudrc file produced.
$ ls overcloud*
overcloud-env.json overcloud-full.initrd overcloud-full.qcow2
overcloud-full.vmlinuz
overcloud-full.d:
dib-manifests
Is it via ssh key or password? Should I set the authentication method
somewhere?
Thanks,
Erming
_______________________________________________
Rdo-list mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Rdo-list@redhat.com">Rdo-list@redhat.com</a>
<a class="moz-txt-link-freetext" href="https://www.redhat.com/mailman/listinfo/rdo-list">https://www.redhat.com/mailman/listinfo/rdo-list</a>
To unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:rdo-list-unsubscribe@redhat.com">rdo-list-unsubscribe@redhat.com</a>
</pre>
</blockquote>
<pre wrap="">
This error generally means that something in the deployment got stuck,
and the deployment hung until the token expired after 4 hours. When
that happens, there is no overcloudrc generated (because there is not a
working overcloud). You won't be able to recover with a stack update,
you'll need to perform a stack-delete and redeploy once you know what
went wrong.
Generally a deployment shouldn't take anywhere near that long, a bare
metal deployment with 6 hosts takes me less than an hour, and less than
2 including a Ceph deployment. In fact, I usually set a timeout using
the --timeout option, because if it hasn't finished after, say 90
minutes (depending on how complicated the deployment is), then I want
it to bomb out so I can diagnose what went wrong and redeploy.
Often when a deployment times out it is because there were connectivity
issues between the nodes. Since you can log in to the hosts, you might
want to do some basic network troubleshooting, such as:
$ ip address # check to see that all the interfaces are there, and
that the IP addresses have been assigned
$ sudo ovs-vsctl show # make sure that the bridges have the proper
interfaces, vlans, and that all the expected bridges show up
$ ping <other overcloud nodes> # you can try this on all VLANs to make
sure that any VLAN trunks are working properly
$ sudo ovs-appctl bond/show # if running bonding, check to see the
bond status
$ sudo os-net-config --debug -c /etc/os-net-config/config.json # run
the network configuration script again to make sure that it is able to
configure the interfaces without error. WARNING, MAY BE DISRUPTIVE as
this will reset the network interfaces, run on console if possible.
However, I want to first double-check that you had a valid command
line. You only show "openstack deploy overcloud --templates" in your
original email. You did have a full command-line, right? Refer to the
official installation guide for the right parameters.
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
---------------------------------------------
Erming Pei, Ph.D
Senior System Analyst; Grid/Cloud Specialist
Research Computing Group
Information Services & Technology
University of Alberta, Canada
Tel: +1 7804929914 Fax: +1 7804921729
---------------------------------------------</pre>
</body>
</html>