5:24 a.m.
Hi all,
I have a rdo kilo deployment, using sr-iov ports to my instances. I'm
trying to configure NUMA topology and CPU pinning for some telco based
workloads based on this doc:
http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-topolog...
I have 3 compute nodes, I'm trying to use one of them to use cpu pinning.
I've configured it like this:
*Compute Node (total 24 cpus)*
*/etc/nova/nova.conf*
vcpu_pin_set=2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,23
Changed grub to isolate my cpus:
#grubby --update-kernel=ALL
--args="isolcpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,23"
#grub2-install /dev/sda
*Controller Nodes:* */etc/nova/nova.conf*
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter
scheduler_available_filters = nova.scheduler.filters.all_filters
scheduler_available_filters =
nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter *Created
host aggregate performance * #nova aggregate-create performance #nova
aggregate-set-metadata 1 pinned=true
#nova aggregate-add-host 1 compute03
*Created host aggregate normal*
#nova aggregate-create normal
#nova aggregate-set-metadata 2 pinned=false
#nova aggregate-add-host 2 compute01
#nova aggregate-add-host 2 compute02
*Created the flavor with cpu pinning* #nova flavor-create m1.performance 6
2048 20 4 #nova flavor-key 6 set hw:cpu_policy=dedicated #nova flavor-key 6
set aggregate_instance_extra_specs:pinned=true *The issue is:* With SR-IOV
ports it only let's me create instances with 6 vcpus in total with the conf
described above. Without SR-IOV, using OVS, I don't have that limitation.
Is this a bug or something? I've seen this:
https://bugs.launchpad.net/nova/+bug/1441169, however I have the patch, and
as I said it works for the first 6 vcpus with my configuration.
*Some relevant logs:*
*/var/log/nova/nova-scheduler.log*
2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Starting with 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:70
2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter RetryFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter AvailabilityZoneFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.955 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter RamFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ComputeFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ComputeCapabilitiesFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ImagePropertiesFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ServerGroupAntiAffinityFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter ServerGroupAffinityFilter returned 3 host(s)
get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.957 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter PciPassthroughFilter returned 3 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84*2015-11-06
11:18:17.959 59494 DEBUG nova.filters
[req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d
9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - -
-] Filter NUMATopologyFilter returned 2 host(s) get_filtered_objects
/usr/lib/python2.7/site-packages/nova/filters.py:84*
Any help would be appreciated.
Thanks,
Pedro Sousa