[Rdo-list] issue with numa and cpu pinning using SRIOV ports

[Pedro Sousa]

Hi all,

I have a rdo kilo deployment, using sr-iov ports to my instances. I'm
trying to configure NUMA topology and CPU pinning for some telco based
workloads based on this doc:
http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-topology-awareness-in-openstack-compute/

I have 3 compute nodes, I'm trying to use one of them to use cpu pinning.

I've configured it like this:

*Compute Node (total 24 cpus)*
*/etc/nova/nova.conf*
vcpu_pin_set=2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,23

Changed grub to isolate my cpus:
#grubby --update-kernel=ALL
--args="isolcpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,23"

#grub2-install /dev/sda

*Controller Nodes:* */etc/nova/nova.conf*
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter
scheduler_available_filters = nova.scheduler.filters.all_filters
scheduler_available_filters =
nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter *Created
host aggregate performance * #nova aggregate-create performance #nova
aggregate-set-metadata 1 pinned=true

#nova aggregate-add-host 1 compute03

*Created host aggregate normal*
#nova aggregate-create normal
#nova aggregate-set-metadata 2 pinned=false

#nova aggregate-add-host 2 compute01

#nova aggregate-add-host 2 compute02

*Created the flavor with cpu pinning* #nova flavor-create m1.performance 6
2048 20 4 #nova flavor-key 6 set hw:cpu_policy=dedicated #nova flavor-key 6
set aggregate_instance_extra_specs:pinned=true *The issue is:* With SR-IOV
ports it only let's me create instances with 6 vcpus in total with the conf
described above. Without SR-IOV, using OVS, I don't have that limitation.
Is this a bug or something? I've seen this:
https://bugs.launchpad.net/nova/+bug/1441169, however I have the patch, and
as I said it works for the first 6 vcpus with my configuration.

*Some relevant logs:*

*/var/log/nova/nova-scheduler.log*

2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Starting with 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:70

2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter RetryFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter AvailabilityZoneFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter RamFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ComputeFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ComputeCapabilitiesFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ImagePropertiesFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ServerGroupAntiAffinityFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ServerGroupAffinityFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.957 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter PciPassthroughFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84*2015-11-06
11:18:17.959 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter NUMATopologyFilter returned 2 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84*

Any help would be appreciated.


Thanks,
Pedro Sousa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20151106/6f491c85/attachment.html>