<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Which method you are using?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Neutron sriov plugin? Nova (via flavor definition)? Neutron without sriov plugin (which I think is the best)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> rdo-list-bounces@redhat.com [mailto:rdo-list-bounces@redhat.com]
<b>On Behalf Of </b>Chinmaya Dwibedy<br>
<b>Sent:</b> Monday, June 20, 2016 14:53<br>
<b>To:</b> rdo-list@redhat.com<br>
<b>Subject:</b> Re: [rdo-list] Issue with assigning multiple VFs to VM instance<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi ,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Can anyone please suggest how to assign multiple VF devices to VM instance using open-stack openstack-mitaka release? Thank you in advance for your time and support.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Chinmaya<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Thu, Jun 16, 2016 at 5:42 PM, Chinmaya Dwibedy <<a href="mailto:ckdwibedy@gmail.com" target="_blank">ckdwibedy@gmail.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi All,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I have installed open-stack openstack-mitaka release on CentO7 system . It has two Intel QAT devices. There are 32 VF devices available per QAT Device/DH895xCC device. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">[root@localhost nova(keystone_admin)]# lspci -nn | grep 0435<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">83:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [8086:0435]<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">88:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [8086:0435]<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">[root@localhost nova(keystone_admin)]# cat /sys/bus/pci/devices/0000\:88\:00.0/sriov_numvfs<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">32<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">[root@localhost nova(keystone_admin)]# cat /sys/bus/pci/devices/0000\:83\:00.0/sriov_numvfs<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">32<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">[root@localhost nova(keystone_admin)]#<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Changed the nova configuration (as stated below) for exposing VF ( via PCI-passthrough) to the instances.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">pci_alias = {"name": "QuickAssist", "product_id": "0443", "vendor_id": "8086", "device_type": "type-VF"}<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">pci_passthrough_whitelist = [{"vendor_id":"8086","product_id":"0443"}}]<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Restarted the nova compute, nova API and nova scheduler service
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">service openstack-nova-compute restart;service openstack-nova-api restart;systemctl restart openstack-nova-scheduler;<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">scheduler_available_filters=nova.scheduler.filters.all_filters<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">scheduler_default_filters=RamFilter,ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,PciPassthroughFilter<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thereafter it shows all the available VFs (64) in nova database upon select * from pci_devices. Set the flavor 4 to allow passing two VFs to instances.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">[root@localhost nova(keystone_admin)]# nova flavor-show 4<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">+----------------------------+------------------------------------------------------------+<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| Property | Value |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">+----------------------------+------------------------------------------------------------+<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| OS-FLV-DISABLED:disabled | False |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| OS-FLV-EXT-DATA:ephemeral | 0 |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| disk | 80 |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| extra_specs | {"pci_passthrough:alias": "QuickAssist:2"} |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| id | 4 |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| name | m1.large |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| os-flavor-access:is_public | True |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| ram | 8192 |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| rxtx_factor | 1.0 |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| swap | |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">| vcpus | 4 |<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">+----------------------------+------------------------------------------------------------+<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">[root@localhost nova(keystone_admin)]#<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> Also when I launch an instance using this new flavor, it goes into an error state</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3">nova boot --flavor 4 --key_name oskey1 --image bc859dc5-103b-428b-814f-d36e59009454 --nic net-id=e2ca118d-1f25-47de-8524-bb2a2635c4be
TEST</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3">Here goes the output of nova-conductor.log</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3">2016-06-16 07:55:34.640 5094 WARNING nova.scheduler.utils [req-6189d3c8-5587-4350-8cd3-704fd35cf2ad 266f5859848e4f39b9725203dda5c3f2
4bc608763cee41d9a8df26d3ef919825 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3">Traceback (most recent call last):</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 150, in inner</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> return func(*args, **kwargs)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 104, in select_destinations</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> dests = self.driver.select_destinations(ctxt, spec_obj)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 74, in select_destinations</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> raise exception.NoValidHost(reason=reason)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3">NoValidHost: No valid host was found. There are not enough hosts available.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3">Here goes the output of nova-compute.log</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:57:32.502 170789 INFO nova.compute.resource_tracker [req-4529a2a8-390f-4620-98b3-d3eb77e077a3 - - - - -] Total usable vcpus: 36, total allocated vcpus: 16<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:57:32.502 170789 INFO nova.compute.resource_tracker [req-4529a2a8-390f-4620-98b3-d3eb77e077a3 - - - - -] Final resource view: name=localhost phys_ram=128721MB used_ram=33280MB
phys_disk=49GB used_disk=320GB total_vcpus=36 used_vcpus=16 pci_stats=[PciDevicePool(count=0,numa_node=0,product_id='10fb',tags={dev_type='type-PF'},vendor_id='8086'), PciDevicePool(count=63,numa_node=1,product_id='0443',tags={dev_type='type-VF'},vendor_id='8086')]<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:57:33.803 170789 INFO nova.compute.resource_tracker [req-4529a2a8-390f-4620-98b3-d3eb77e077a3 - - - - -] Compute_service record updated for localhost:localhost<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#525359;background:#F7F6F3">Here goes the output of nova-scheduler.log</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:55:34.636 171018 WARNING nova.scheduler.host_manager [req-6189d3c8-5587-4350-8cd3-704fd35cf2ad 266f5859848e4f39b9725203dda5c3f2 4bc608763cee41d9a8df26d3ef919825 -
- -] Host localhost has more disk space than database expected (-141 GB > -271 GB)<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:55:34.637 171018 INFO nova.filters [req-6189d3c8-5587-4350-8cd3-704fd35cf2ad 266f5859848e4f39b9725203dda5c3f2 4bc608763cee41d9a8df26d3ef919825 - - -] Filter PciPassthroughFilter
returned 0 hosts<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:55:34.638 171018 INFO nova.filters [req-6189d3c8-5587-4350-8cd3-704fd35cf2ad 266f5859848e4f39b9725203dda5c3f2 4bc608763cee41d9a8df26d3ef919825 - - -] Filtering removed
all hosts for the request with instance ID '4f68c680-5a17-4a38-a6df-5cdb6d76d75b'. Filter results: ['RamFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'ComputeCapabilitiesFilter: (start: 1, end:
1)', 'ImagePropertiesFilter: (start: 1, end: 1)', 'PciPassthroughFilter: (start: 1, end: 0)']<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:56:14.743 171018 INFO nova.scheduler.host_manager [req-64a8dc31-f2ab-4d93-8579-6b9f8210ece7 - - - - -] Successfully synced instances from host 'localhost'.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">2016-06-16 07:58:17.748 171018 INFO nova.scheduler.host_manager [req-152ac777-1f77-433d-8493-6cd86ab3e0fc - - - - -] Successfully synced instances from host 'localhost'.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Note that, If I set the flavor as (#nova flavor-key 4 set "pci_passthrough:alias"="QuickAssist:1") , it assigns a single VF to VM instance. I think, multiple PFs can be assigned
per VM. Can anyone please suggest , where I am wrong and the way to solve this ? Thank you in advance for your support and help.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Regards,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Chinmaya<o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>