[Rdo-list] networking issues with the nova-docker driver
by ICHIBA Sara
Hello people,
The nova-docker finally worked for me. Now I can launch containers with
nova, in the dashboard it is said that the instances are running. But when
I try to go into the containers I fail. It seems that the problem is due to
networking issues. Nova fail to create netns for the containers in question
and thus all what comes after fail as well.
When I check the compute.log I find these messages. Would you please take a
look and see if they are familiar to you and if so suggest me something ?
Any suggestion or hint would be very appreciated.
Thank you all ^^,
Sara
====compute.log
2014-09-18 11:22:37.899 902 AUDIT nova.compute.claims
[req-faa4b1ac-869a-4613-9a3b-9f97794b3cfd 6b8a39ff0bb9417eb1a3ce8bdf09cf00
3ca2fa62ac434e8f942e5823969f23db] [instance:
f0be5fc3-9a00-4e9b-a2d5-af8ddacd5d94] Claim successful
2014-09-18 11:22:44.029 902 ERROR novadocker.virt.docker.vifs
[req-faa4b1ac-869a-4613-9a3b-9f97794b3cfd 6b8a39ff0bb9417eb1a3ce8bdf09cf00
3ca2fa62ac434e8f942e5823969f23db] *Failed to attach vif*
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs Traceback
(most recent call last):
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs File
"/usr/lib/python2.6/site-packages/novadocker/virt/docker/vifs.py", line
206, in attach
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs
container_id, run_as_root=True)
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs File
"/usr/lib/python2.6/site-packages/nova/utils.py", line 165, in execute
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs return
processutils.execute(*cmd, **kwargs)
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs File
"/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py",
line 193, in execute
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs cmd='
'.join(cmd))
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs
ProcessExecutionError: Unexpected error while running command.
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs Command: sudo
nova-rootwrap /etc/nova/rootwrap.conf* ip link set ns59fc4e34-bc netns
afb0b5b7b02aef73f07c34b7f456ace080bb9944d21376f7a05e2d08206c4b67*
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs Exit code: 255
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs Stdout: ''
*2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs Stderr:
'Error: argument
"afb0b5b7b02aef73f07c34b7f456ace080bb9944d21376f7a05e2d08206c4b67" is
wrong: Invalid "netns" value\n\n'*
2014-09-18 11:22:44.029 902 TRACE novadocker.virt.docker.vifs
2014-09-18 11:23:22.087 902 AUDIT nova.compute.resource_tracker [-]
Auditing locally available compute resources
2014-09-18 11:23:22.459 902 AUDIT nova.compute.resource_tracker [-] Free
ram (MB): -683
2014-09-18 11:23:22.459 902 AUDIT nova.compute.resource_tracker [-] Free
disk (GB): 9
2014-09-18 11:23:22.459 902 AUDIT nova.compute.resource_tracker [-] Free
VCPUS: -3
2014-09-18 11:23:22.627 902 INFO nova.compute.resource_tracker [-]
Compute_service record updated for otvmi307s.priv.atos.fr:o
tvmi307s.priv.atos.fr
==== the output of the command *ovs-vsctl show*
95915800-961a-45de-ba73-09bc8c9c329b
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Port "qg-79be5a9c-97"
Interface "qg-79be5a9c-97"
type: internal
Bridge br-int
fail_mode: secure
Port "tap9683ca3b-87"
tag: 1
Interface "tap9683ca3b-87"
type: internal
Port br-int
Interface br-int
type: internal
Port "tapc8dd2048-aa"
tag: 3
Interface "tapc8dd2048-aa"
Port "qr-234b2621-29"
tag: 1
Interface "qr-234b2621-29"
type: internal
Port "tap59fc4e34-bc"
tag: 3
Interface "*tap59fc4e34-bc*"
Port "tap360101f0-ee"
tag: 3
Interface "tap360101f0-ee"
Port "tap16ebed2a-3e"
tag: 3
Interface "tap16ebed2a-3e"
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tap741f9199-b9"
tag: 4095
Interface "tap741f9199-b9"
Bridge br-tun
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
ovs_version: "1.11.0"
========the output of the command ip link show
55: *ns59fc4e34-bc*: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
qlen 1000
link/ether 7e:5f:86:0e:1f:52 brd ff:ff:ff:ff:ff:ff
56: *tap59fc4e34-bc*: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc
pfifo_fast state DOWN qlen 1000
link/ether a6:e5:4b:fa:9d:84 brd ff:ff:ff:ff:ff:ff
==========the output of the command ls /var/run/netns
09ccbe2f093c904972bd886735b718e9e9fec41f1ee5afd5d53b5d1b8c7f0ae2
a96f3ec37b7aa764a86c61e0317b1360c449dec272a7fd6ce3cab4b45ea6c98b
5eda9416e30d2aa9e8bf1f37bbadca1c4688985b5a3272338557cd24ecefaab6
* afb0b5b7b02aef73f07c34b7f456ace080bb9944d21376f7a05e2d08206c4b67*
87992907919ec457bf93161326f472c485fdff0b7ff6ba0b6762dac66fcd2626
qdhcp-fecbcfdd-92bb-41aa-86da-4050b65d360b
9abc5b0dba7faa0d2ca1b34cedb93acceb7ec4c46e3390ed9d33843310548325
qrouter-d8cc218e-b99f-440a-9096-b18b4c447caf
======the output of the command docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
*afb0b5b7b02a * busybox:buildroot-2014.02 /bin/sh 11
minutes ago Up 11 minutes
nova-f0be5fc3-9a00-4e9b-a2d5-af8ddacd5d94
10 years, 2 months
[Rdo-list] define service endpoints
by Zhao, Xin
Hello,
When one defines services endpoints in keystone, eg. for neutron, the
publicurl should be using the outfacing NIC IP (or external hostname of
the controller), while the internalurl and adminurl should be using the
internal management subnet NIC IP (or the internal hostname of the
controller). Do I understand this right? My controller has an out-facing
IP/hostname and an internal hostname/IP on the management subnet.
Thanks,
Xin
10 years, 2 months
[Rdo-list] python-kombu for Icehouse
by Belmiro Moreira
Hi,
we are upgrading to RDO Icehouse and we see that "python-kombu" version
installed is "1.1.3-2.el6" that we get from epel on our SLC6 nodes.
However, nova Icehouse requirement for python-kombu is >=2.4.8
We started building your RPM but it would be good to have a more recent
version
on el6 or even RDO for all infrastructures that are using RabbitMQ.
thanks,
Belmiro
10 years, 2 months
Re: [Rdo-list] selinux preventing Horizon access?
by Kaul, Yaniv
Ahoy!
Filed https://bugzilla.redhat.com/show_bug.cgi?id=1144539 , ARGH!
Cap'n Y.
(Ay, it's Talk like a pirate day)
From: Kaul, Yaniv
Sent: Thursday, September 18, 2014 9:01 PM
To: rdo-list(a)redhat.com
Subject: selinux preventing Horizon access?
IceHouse / CentOS 7- after reboot post install.
type=AVC msg=audit(1411063019.099:1848): avc: denied { name_connect } for pid=5684 comm="httpd" dest=8776 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket
getenforce Permissive solved it.
[root@lgdrm403 httpd(keystone_admin)]# rpm -qa |grep -E "openstack|selinux"
openstack-utils-2014.1-3.el7.noarch
selinux-policy-targeted-3.12.1-153.el7_0.10.noarch
openstack-nova-cert-2014.1.2-1.el7.centos.noarch
python-django-openstack-auth-1.1.5-1.el7.noarch
libselinux-2.2.2-6.el7.x86_64
openstack-glance-2014.1.2-4.el7.centos.noarch
openstack-packstack-puppet-2014.1.1-0.28.dev1238.el7.noarch
openstack-nova-novncproxy-2014.1.2-1.el7.centos.noarch
openstack-dashboard-2014.1.2-2.el7.centos.noarch
openstack-cinder-2014.1-2.el7.noarch
libselinux-utils-2.2.2-6.el7.x86_64
openstack-nova-console-2014.1.2-1.el7.centos.noarch
openstack-keystone-2014.1.2.1-1.el7.centos.noarch
libselinux-python-2.2.2-6.el7.x86_64
openstack-puppet-modules-2014.1-23.el7.noarch
libselinux-ruby-2.2.2-6.el7.x86_64
openstack-nova-api-2014.1.2-1.el7.centos.noarch
openstack-nova-compute-2014.1.2-1.el7.centos.noarch
openstack-nova-conductor-2014.1.2-1.el7.centos.noarch
openstack-nova-scheduler-2014.1.2-1.el7.centos.noarch
openstack-packstack-2014.1.1-0.28.dev1238.el7.noarch
selinux-policy-3.12.1-153.el7_0.10.noarch
openstack-selinux-0.5.15-1.el7ost.noarch
openstack-nova-common-2014.1.2-1.el7.centos.noarch
openstack-nova-network-2014.1.2-1.el7.centos.noarch
10 years, 2 months
[Rdo-list] OVS bridge not coming up at boot on centos7
by Arif Ali
Hi chaps,
I have been testing rdo-openstack for the last few months, and finally got
the basic stuff working yesterday with all the necessary core bits, :)
The one issue I have found with the system at the moment is that when I
reboot any of the machines in the cluster, whether it's the controller or
the nova node, the OVSBridge ports do not acquire an IP address.
I then go onto the machine via IPMI SOL, and restart the networking through
"systemctl restart network.service"
So an extracts of commands below
If someone can shed any light, hat would be great
# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
# uname -r
3.10.0-123.6.3.el7.x86_64
[root@stack03 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp2s1f0
DEVICE=enp2s1f0
ONBOOT=yes
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-xcat
[root@stack03 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br-xcat
BOOTPROTO=static
DEVICE=br-xcat
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE=ovs
IPADDR=10.0.0.3
NETMASK=255.255.254.0
# ovs-vsctl show
663f3055-d146-4e59-979d-741c8488edb8
Bridge br-int
fail_mode: secure
Port "qvo9bb27bce-15"
tag: 1
Interface "qvo9bb27bce-15"
Port "qvo6424e2c9-ec"
tag: 1
Interface "qvo6424e2c9-ec"
Port "qvo89552e51-eb"
tag: 1
Interface "qvo89552e51-eb"
Port "qvo8d8059a9-b7"
tag: 1
Interface "qvo8d8059a9-b7"
Port int-br-xcat
Interface int-br-xcat
Port br-int
Interface br-int
type: internal
Port "qvo0ca24fd5-3d"
tag: 1
Interface "qvo0ca24fd5-3d"
Bridge br-xcat
Port "enp2s1f0"
Interface "enp2s1f0"
Port br-xcat
Interface br-xcat
type: internal
ovs_version: "2.0.0"
Below are the list of the core RPMs installed on a nova compute node
# rpm -qa | grep "openvswitch\|openstack\|libvirt\|qemu"
libvirt-daemon-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-driver-nwfilter-1.1.1-29.el7_0.1.x86_64
libvirt-python-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-kvm-1.1.1-29.el7_0.1.x86_64
openstack-neutron-2014.1.2-1.el7.centos.noarch
qemu-kvm-1.5.3-60.el7_0.7.x86_64
qemu-img-1.5.3-60.el7_0.7.x86_64
ipxe-roms-qemu-20130517-5.gitc4bce43.el7.noarch
libvirt-client-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-driver-nodedev-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-driver-interface-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-driver-secret-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-config-network-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-driver-qemu-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-driver-lxc-1.1.1-29.el7_0.1.x86_64
openstack-nova-compute-2014.1.2-1.el7.centos.noarch
openstack-utils-2014.1-3.el7.noarch
openvswitch-2.0.0-6.el7.x86_64
openstack-neutron-ml2-2014.1.2-1.el7.centos.noarch
libvirt-daemon-driver-storage-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-config-nwfilter-1.1.1-29.el7_0.1.x86_64
libvirt-daemon-driver-network-1.1.1-29.el7_0.1.x86_64
libvirt-1.1.1-29.el7_0.1.x86_64
openstack-nova-common-2014.1.2-1.el7.centos.noarch
qemu-kvm-common-1.5.3-60.el7_0.7.x86_64
openstack-neutron-openvswitch-2014.1.2-1.el7.centos.noarch
--
Arif Ali
IRC: arif-ali at freenode
LinkedIn: http://uk.linkedin.com/in/arifali
10 years, 2 months
[Rdo-list] selinux preventing Horizon access?
by Kaul, Yaniv
IceHouse / CentOS 7- after reboot post install.
type=AVC msg=audit(1411063019.099:1848): avc: denied { name_connect } for pid=5684 comm="httpd" dest=8776 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket
getenforce Permissive solved it.
[root@lgdrm403 httpd(keystone_admin)]# rpm -qa |grep -E "openstack|selinux"
openstack-utils-2014.1-3.el7.noarch
selinux-policy-targeted-3.12.1-153.el7_0.10.noarch
openstack-nova-cert-2014.1.2-1.el7.centos.noarch
python-django-openstack-auth-1.1.5-1.el7.noarch
libselinux-2.2.2-6.el7.x86_64
openstack-glance-2014.1.2-4.el7.centos.noarch
openstack-packstack-puppet-2014.1.1-0.28.dev1238.el7.noarch
openstack-nova-novncproxy-2014.1.2-1.el7.centos.noarch
openstack-dashboard-2014.1.2-2.el7.centos.noarch
openstack-cinder-2014.1-2.el7.noarch
libselinux-utils-2.2.2-6.el7.x86_64
openstack-nova-console-2014.1.2-1.el7.centos.noarch
openstack-keystone-2014.1.2.1-1.el7.centos.noarch
libselinux-python-2.2.2-6.el7.x86_64
openstack-puppet-modules-2014.1-23.el7.noarch
libselinux-ruby-2.2.2-6.el7.x86_64
openstack-nova-api-2014.1.2-1.el7.centos.noarch
openstack-nova-compute-2014.1.2-1.el7.centos.noarch
openstack-nova-conductor-2014.1.2-1.el7.centos.noarch
openstack-nova-scheduler-2014.1.2-1.el7.centos.noarch
openstack-packstack-2014.1.1-0.28.dev1238.el7.noarch
selinux-policy-3.12.1-153.el7_0.10.noarch
openstack-selinux-0.5.15-1.el7ost.noarch
openstack-nova-common-2014.1.2-1.el7.centos.noarch
openstack-nova-network-2014.1.2-1.el7.centos.noarch
10 years, 2 months