[Rdo-list] [rdo-manager] Authentication required during overcloud deployment
Dan Sneddon
dsneddon at redhat.com
Thu Oct 15 23:00:45 UTC 2015
If you are doing the most basic deployment (no network isolation) on
bare metal, you will want to specify which interface is your external
network interface. This will be the interface that gets attached to
br-ex (this defaults to your first interface, which may not be correct
in your case). In the basic deployment, this network will require a
DHCP server to give the controller an address, and if you want to use
floating IPs you will need a range of IPs that are free (won't be
assigned to other hosts by DHCP server).
So, if your external interface is 'enp11s0f0' (just guessing, since
that one actually has a DHCP address), then your command-line will be
at least:
openstack overcloud deploy --templates \
--neutron-public-interface enp11s0f0
But you should probably include a reference to an NTP server:
openstack overcloud deploy --templates \
--neutron-public-interface enp11s0f0 \
--ntp-server pool.ntp.org
Some other options to consider:
--timeout <mins>
--debug
Selecting tunnel type:
--neutron-network-type <vxlan|gre>
--neutron-tunnel-types <vxlan,gre>
--
Dan Sneddon | Principal OpenStack Engineer
dsneddon at redhat.com | redhat.com/openstack
650.254.4025 | dsneddon:irc @dxs:twitter
On 10/15/2015 01:03 PM, Erming Pei wrote:
> Hi Dan, Sasha,
>
> Thanks for your answers and hints.
> I looked up the heat/etc log files and stack/node status.
> Only thing I found by far is "timed out". I don't know what's the
> reason. IPMI looks good.
>
> Tried with HEAT_INCLUDE_PASSWORD=1 but same error message (Please
> try again with option --include-password or export
> HEAT_INCLUDE_PASSWORD=1 Authentication required)
>
> BTW. I only followed the exact instruction as shown in the guide:
> (openstack overcloud deploy --templates) No more options. I thought
> this is good for a demo deployment. If not sufficient, which one I
> should follow? See some of your discussions, but not very clear. Should
> I follow the example from jliberma at redhat.com?
>
> Below are my investigation:
> By runnig: $ heat resource-list overcloud
> Found that just controller and compute are failed: CREATE_FAILED
>
> Checked the reason it says: resource_status_reason | CREATE aborted
>
> I then logged into the running overcloud nodes (e.g. the controller):
>
>
> [heat-admin at overcloud-controller-0 ~]$ ifconfig
> br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::21:5eff:fecd:9df3 prefixlen 64 scopeid 0x20<link>
> ether 02:21:5e:cd:9d:f3 txqueuelen 0 (Ethernet)
> RX packets 29926 bytes 2364154 (2.2 MiB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 81 bytes 25614 (25.0 KiB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> enp0s29f0u2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::21:5eff:fecd:9df3 prefixlen 64 scopeid 0x20<link>
> ether 02:21:5e:cd:9d:f3 txqueuelen 1000 (Ethernet)
> RX packets 29956 bytes 1947140 (1.8 MiB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 102 bytes 28620 (27.9 KiB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> enp11s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet 10.0.6.64 netmask 255.255.0.0 broadcast 10.0.255.255
> inet6 fe80::221:5eff:fec9:abd8 prefixlen 64 scopeid 0x20<link>
> ether 00:21:5e:c9:ab:d8 txqueuelen 1000 (Ethernet)
> RX packets 66256 bytes 21109918 (20.1 MiB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 35938 bytes 4641202 (4.4 MiB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> enp11s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::221:5eff:fec9:abda prefixlen 64 scopeid 0x20<link>
> ether 00:21:5e:c9:ab:da txqueuelen 1000 (Ethernet)
> RX packets 25429 bytes 2004574 (1.9 MiB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 6 bytes 532 (532.0 B)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 2044
> inet6 fe80::202:c902:23:baf9 prefixlen 64 scopeid 0x20<link>
> Infiniband hardware address can be incorrect! Please read BUGS section
> in ifconfig(8).
> infiniband
> 80:00:04:04:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 txqueuelen
> 256 (InfiniBand)
> RX packets 183678 bytes 10292768 (9.8 MiB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 17 bytes 5380 (5.2 KiB)
> TX errors 0 dropped 7 overruns 0 carrier 0 collisions 0
>
> lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
> inet 127.0.0.1 netmask 255.0.0.0
> inet6 ::1 prefixlen 128 scopeid 0x10<host>
> loop txqueuelen 0 (Local Loopback)
> RX packets 138 bytes 11792 (11.5 KiB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 138 bytes 11792 (11.5 KiB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> [heat-admin at overcloud-controller-0 ~]$ ovs-vsctl show
> ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection
> failed (Permission denied)
> [heat-admin at overcloud-controller-0 ~]$ sudo ovs-vsctl show
> 76e6f8a7-88cf-4920-b133-b4d15a4b9092
> Bridge br-ex
> Port br-ex
> Interface br-ex
> type: internal
> Port "enp0s29f0u2"
> Interface "enp0s29f0u2"
> ovs_version: "2.3.1"
> [heat-admin at overcloud-controller-0 ~]$
> [heat-admin at overcloud-controller-0 ~]$ ping 10.0.6.65
> PING 10.0.6.65 (10.0.6.65) 56(84) bytes of data.
> 64 bytes from 10.0.6.65: icmp_seq=1 ttl=64 time=0.176 ms
> 64 bytes from 10.0.6.65: icmp_seq=2 ttl=64 time=0.195 ms
> ^C
> --- 10.0.6.65 ping statistics ---
> 2 packets transmitted, 2 received, 0% packet loss, time 999ms
> rtt min/avg/max/mdev = 0.176/0.185/0.195/0.016 ms
> [heat-admin at overcloud-controller-0 ~]$
> [heat-admin at overcloud-controller-0 ~]$ ping 10.0.6.64
> PING 10.0.6.64 (10.0.6.64) 56(84) bytes of data.
> 64 bytes from 10.0.6.64: icmp_seq=1 ttl=64 time=0.015 ms
> ^C
> --- 10.0.6.64 ping statistics ---
> 1 packets transmitted, 1 received, 0% packet loss, time 0ms
> rtt min/avg/max/mdev = 0.015/0.015/0.015/0.000 ms
>
> [heat-admin at overcloud-controller-0 ~]$ cat /etc/os-net-config/config.json
> {"network_config": [{"use_dhcp": true, "type": "ovs_bridge", "name":
> "br-ex", "members": [{"type": "interface", "name": "nic1", "primary":
> true}]}]}
> [heat-admin at overcloud-controller-0 ~]$
> [heat-admin at overcloud-controller-0 ~]$
> [heat-admin at overcloud-controller-0 ~]$ sudo os-net-config --debug -c
> /etc/os-net-config/config.json
> [2015/10/15 07:52:08 PM] [INFO] Using config file at:
> /etc/os-net-config/config.json
> [2015/10/15 07:52:08 PM] [INFO] Using mapping file at:
> /etc/os-net-config/mapping.yaml
> [2015/10/15 07:52:08 PM] [INFO] Ifcfg net config provider created.
> [2015/10/15 07:52:08 PM] [DEBUG] network_config JSON: [{'use_dhcp':
> True, 'type': 'ovs_bridge', 'name': 'br-ex', 'members': [{'type':
> 'interface', 'name': 'nic1', 'primary': True}]}]
> [2015/10/15 07:52:08 PM] [INFO] nic1 mapped to: enp0s29f0u2
> [2015/10/15 07:52:08 PM] [INFO] nic2 mapped to: enp11s0f0
> [2015/10/15 07:52:08 PM] [INFO] nic3 mapped to: enp11s0f1
> [2015/10/15 07:52:08 PM] [INFO] nic4 mapped to: ib0
> [2015/10/15 07:52:08 PM] [INFO] adding bridge: br-ex
> [2015/10/15 07:52:08 PM] [DEBUG] bridge data: DEVICE=br-ex
> ONBOOT=yes
> HOTPLUG=no
> DEVICETYPE=ovs
> TYPE=OVSBridge
> OVSBOOTPROTO=dhcp
> OVSDHCPINTERFACES="enp0s29f0u2"
> OVS_EXTRA="set bridge br-ex other-config:hwaddr=02:21:5e:cd:9d:f3"
>
> [2015/10/15 07:52:08 PM] [INFO] adding interface: enp0s29f0u2
> [2015/10/15 07:52:08 PM] [DEBUG] interface data: DEVICE=enp0s29f0u2
> ONBOOT=yes
> HOTPLUG=no
> DEVICETYPE=ovs
> TYPE=OVSPort
> OVS_BRIDGE=br-ex
> BOOTPROTO=none
>
> [2015/10/15 07:52:08 PM] [INFO] applying network configs...
> [2015/10/15 07:52:08 PM] [DEBUG] Diff file data:
> DEVICE=enp0s29f0u2
> ONBOOT=yes
> HOTPLUG=no
> DEVICETYPE=ovs
> TYPE=OVSPort
> OVS_BRIDGE=br-ex
> BOOTPROTO=none
>
> [2015/10/15 07:52:08 PM] [DEBUG] Diff data:
> DEVICE=enp0s29f0u2
> ONBOOT=yes
> HOTPLUG=no
> DEVICETYPE=ovs
> TYPE=OVSPort
> OVS_BRIDGE=br-ex
> BOOTPROTO=none
>
> [2015/10/15 07:52:08 PM] [DEBUG] Diff file data:
>
> [2015/10/15 07:52:08 PM] [DEBUG] Diff data:
>
> [2015/10/15 07:52:08 PM] [DEBUG] Diff file data:
> DEVICE=br-ex
> ONBOOT=yes
> HOTPLUG=no
> DEVICETYPE=ovs
> TYPE=OVSBridge
> OVSBOOTPROTO=dhcp
> OVSDHCPINTERFACES="enp0s29f0u2"
> OVS_EXTRA="set bridge br-ex other-config:hwaddr=02:21:5e:cd:9d:f3"
>
> [2015/10/15 07:52:08 PM] [DEBUG] Diff data:
> DEVICE=br-ex
> ONBOOT=yes
> HOTPLUG=no
> DEVICETYPE=ovs
> TYPE=OVSBridge
> OVSBOOTPROTO=dhcp
> OVSDHCPINTERFACES="enp0s29f0u2"
> OVS_EXTRA="set bridge br-ex other-config:hwaddr=02:21:5e:cd:9d:f3"
>
> [2015/10/15 07:52:08 PM] [DEBUG] Diff file data:
>
> [2015/10/15 07:52:08 PM] [DEBUG] Diff data:
>
>
>
> [heat-admin at overcloud-controller-0 ~]$ openstack-status
> == Nova services ==
> openstack-nova-api: inactive (disabled on boot)
> openstack-nova-cert: inactive (disabled on boot)
> openstack-nova-compute: inactive (disabled on boot)
> openstack-nova-network: inactive (disabled on boot)
> openstack-nova-scheduler: inactive (disabled on boot)
> openstack-nova-conductor: inactive (disabled on boot)
> == Glance services ==
> openstack-glance-api: inactive (disabled on boot)
> openstack-glance-registry: inactive (disabled on boot)
> == Keystone service ==
> openstack-keystone: inactive (disabled on boot)
> == Horizon service ==
> openstack-dashboard: uncontactable
> == neutron services ==
> neutron-server: inactive (disabled on boot)
> neutron-dhcp-agent: inactive (disabled on boot)
> neutron-l3-agent: inactive (disabled on boot)
> neutron-metadata-agent: inactive (disabled on boot)
> neutron-lbaas-agent: inactive (disabled on boot)
> neutron-openvswitch-agent: inactive (disabled on boot)
> neutron-metering-agent: inactive (disabled on boot)
> == Swift services ==
> openstack-swift-proxy: inactive (disabled on boot)
> openstack-swift-account: inactive (disabled on boot)
> openstack-swift-container: inactive (disabled on boot)
> openstack-swift-object: inactive (disabled on boot)
> == Cinder services ==
> openstack-cinder-api: inactive (disabled on boot)
> openstack-cinder-scheduler: inactive (disabled on boot)
> openstack-cinder-volume: inactive (disabled on boot)
> openstack-cinder-backup: inactive (disabled on boot)
> == Ceilometer services ==
> openstack-ceilometer-api: inactive (disabled on boot)
> openstack-ceilometer-central: inactive (disabled on boot)
> openstack-ceilometer-compute: inactive (disabled on boot)
> openstack-ceilometer-collector: inactive (disabled on boot)
> openstack-ceilometer-alarm-notifier: inactive (disabled on boot)
> openstack-ceilometer-alarm-evaluator: inactive (disabled on boot)
> openstack-ceilometer-notification: inactive (disabled on boot)
> == Heat services ==
> openstack-heat-api: inactive (disabled on boot)
> openstack-heat-api-cfn: inactive (disabled on boot)
> openstack-heat-api-cloudwatch: inactive (disabled on boot)
> openstack-heat-engine: inactive (disabled on boot)
> == Support services ==
> libvirtd: active
> openvswitch: active
> dbus: active
> rabbitmq-server: inactive (disabled on boot)
> memcached: inactive (disabled on boot)
> == Keystone users ==
> Warning keystonerc not sourced
>
>
>
>
> Thanks,
>
> Erming
>
>
> On 10/14/15, 5:23 PM, Dan Sneddon wrote:
>> On 10/14/2015 03:03 PM, Erming Pei wrote:
>>> Hi,
>>>
>>> I am deploying the overcloud in baremetal way and after a couple of
>>> hours, it showed:
>>>
>>> $ openstack overcloud deploy --templates
>>> Deploying templates in the directory
>>> /usr/share/openstack-tripleo-heat-templates
>>> ^[[A^[[BERROR: openstack ERROR: Authentication failed. Please try again
>>> with option --include-password or export HEAT_INCLUDE_PASSWORD=1
>>> Authentication required
>>>
>>>
>>> But I checked the nodes are now running:
>>>
>>> [stack at gcloudcon-3 ~]$ nova list
>>> +--------------------------------------+-------------------------+--------+------------+-------------+--------------------+
>>>
>>> | ID | Name |
>>> Status | Task State | Power State | Networks |
>>> +--------------------------------------+-------------------------+--------+------------+-------------+--------------------+
>>>
>>> | 1ba04ac0-fe2b-4318-aa31-2e5f4d8422a6 | overcloud-controller-0 |
>>> ACTIVE | - | Running | ctlplane=10.0.6.60 |
>>> | c152ba59-3aed-4fb0-81fa-e3fed7e35cf6 | overcloud-novacompute-0 |
>>> ACTIVE | - | Running | ctlplane=10.0.6.61 |
>>> +--------------------------------------+-------------------------+--------+------------+-------------+--------------------+
>>>
>>>
>>> 1. Should I re-deploy the nodes or there is a way to do update/makeup
>>> for the authentication issue?
>>>
>>> 2.
>>> I don't know how to access to the nodes.
>>> There is not an overcloudrc file produced.
>>>
>>> $ ls overcloud*
>>> overcloud-env.json overcloud-full.initrd overcloud-full.qcow2
>>> overcloud-full.vmlinuz
>>>
>>> overcloud-full.d:
>>> dib-manifests
>>>
>>> Is it via ssh key or password? Should I set the authentication method
>>> somewhere?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Erming
>>>
>>>
>>> _______________________________________________
>>> Rdo-list mailing list
>>> Rdo-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>
>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>> This error generally means that something in the deployment got stuck,
>> and the deployment hung until the token expired after 4 hours. When
>> that happens, there is no overcloudrc generated (because there is not a
>> working overcloud). You won't be able to recover with a stack update,
>> you'll need to perform a stack-delete and redeploy once you know what
>> went wrong.
>>
>> Generally a deployment shouldn't take anywhere near that long, a bare
>> metal deployment with 6 hosts takes me less than an hour, and less than
>> 2 including a Ceph deployment. In fact, I usually set a timeout using
>> the --timeout option, because if it hasn't finished after, say 90
>> minutes (depending on how complicated the deployment is), then I want
>> it to bomb out so I can diagnose what went wrong and redeploy.
>>
>> Often when a deployment times out it is because there were connectivity
>> issues between the nodes. Since you can log in to the hosts, you might
>> want to do some basic network troubleshooting, such as:
>>
>> $ ip address # check to see that all the interfaces are there, and
>> that the IP addresses have been assigned
>>
>> $ sudo ovs-vsctl show # make sure that the bridges have the proper
>> interfaces, vlans, and that all the expected bridges show up
>>
>> $ ping <other overcloud nodes> # you can try this on all VLANs to make
>> sure that any VLAN trunks are working properly
>>
>> $ sudo ovs-appctl bond/show # if running bonding, check to see the
>> bond status
>>
>> $ sudo os-net-config --debug -c /etc/os-net-config/config.json # run
>> the network configuration script again to make sure that it is able to
>> configure the interfaces without error. WARNING, MAY BE DISRUPTIVE as
>> this will reset the network interfaces, run on console if possible.
>>
>> However, I want to first double-check that you had a valid command
>> line. You only show "openstack deploy overcloud --templates" in your
>> original email. You did have a full command-line, right? Refer to the
>> official installation guide for the right parameters.
>>
>
>
> --
> ---------------------------------------------
> Erming Pei, Ph.D
> Senior System Analyst; Grid/Cloud Specialist
>
> Research Computing Group
> Information Services & Technology
> University of Alberta, Canada
>
> Tel: +1 7804929914 Fax: +1 7804921729
> ---------------------------------------------
>
More information about the dev
mailing list