Red Hat OpenStack Platform v10 can be deployed via OSP-director by enabling SR- IOV on the compute overcloud nodes. This post is going to detail the steps involved in this deployment, along with other required details of SR-IOV deployment.
The following list of changes has to be added on top of the usual OSP10 deployment in order to enable SR-IOV in compute nodes. Here we are going disucss on enabling SR-IOV on the Compute role itself. The steps for deploying it as a new custom role along with existing Compute role will be added shortly and will be linked here.
- Environment file
- Parameters
Environment file
SR-IOV can be enabled by adding the neutron-sriov.yaml
environment file to
the deploy command, like below
-e /usr/share/openstack-tripleo-heat-templates/environments/neutron-sriov.yaml
This only enables the SR-IOV composable service in the Compute
role.
Additionally, we need to give the following parameters to assist the
succesfully deployment.
Parameters
Following are the list of parameters which needs to be configured addit
- NeutronPhysicalDevMappings
- NeutronSriovNumVFs
- NovaPCIPassthrough
- NovaSchedulerDefaultFilters
- NovaSchedulerAvailableFilters
- NeutronSupportedPCIVendorDevs
NeutronPhysicalDevMappings
This parameter contains the list of physical network to the physical device mapping.
Example:
parameter_defaults:
NeutronPhysicalDevMappings: "datacentre:ens20f2"
NeutronSriovNumVFs
This parameter contains the number of VFs to be created on a physical interface as a list. The number of VFs will be enabled on the corresponding interface.
echo "<N>" > /sys/class/net/<interface>/device/sriov_numvfs
As this configuration is transitent, there is a boot-up script written on the
SR-IOV deployed node via ifup-local
to invoke this command every time the
overcloud node reboots.
Example:
parameter_defaults:
NeutronSriovNumVFs: "ens20f2:4"
NovaPCIPassthrough
This parameter contains the white list of PCI devices available to VMs.
Examples:
This example is to illustrate the different format of this parameter that can be given in the templates. Choose the best format which suits your deployment.
parameter_defaults:
NovaPCIPassthrough:
- devname: "ens20f2"
physical_network: "datacentre"
- address: "*:0a:00.*"
- address: ":0a:00."
physical_network: "datacentre"
- vendor_id: 1137
product_id: 0071
- vendor_id: 1137
product_id: 0071
address: "0000:0a:00.1"
physical_network: "datacentre"
Note::
Following format is an invalid format, as they specifiy mutually exclusive options.
parameter_defaults:
NovaPCIPassthrough:
- devname: "ens20f2"
physical_network: "datacentre"
address: "*:0a:00.*"
All these details are specificed in the nova.conf documenation for
pci_passthrough_whitelist
. This option is being moved from the DEFAULT
secion to the pci
section and renamed as passthrough_whitelist
in the
Ocata release.
NovaSchedulerDefaultFilters
parameter_defaults:
NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter']
NovaSchedulerAvailableFilters
parameter_defaults:
NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"]
NeutronSupportedPCIVendorDevs
This parameter id deprecated in OSP10. This parameter contains the list of vendor id and product id of the Virtual Function (VF) which will be used for SR-IOV deployment. Usually deployers make a mistake of assuming this as a the vendor and product id of the Physical Function (PF) which is not the case. As of now there is no straighforward way to identify these ids. Either we should get this information from nic vendor or we have exeucute below command in one of the node which used for deployment:
echo "1" > /sys/class/net/<interface>/device/sriov_numvfs
lspci -nn | grep -i 'network\|ether'
Example:
parameter_defaults:
NeutronSupportedPCIVendorDevs: ['8086:154c','8086:10ca','8086:1520']
Note:
NeutronMechanismDrivers
parameter is modified in the SR-IOV TripleO
environment file to include sriovnicswitch
. On adding the environment file
neutron-sriov.yaml
to the deploy command, no need to override this
paramter in tempaltes.
Complete List of SR-IOV Template Parameters
Create a file /home/stack/sriov-environment.yaml
with below contents
modified to your environment:
parameter_defaults:
NeutronSupportedPCIVendorDevs: ['8086:154c','8086:10ca','8086:1520']
NeutronPhysicalDevMappings: "datacentre:ens20f2"
NeutronSriovNumVFs: "ens20f2:4"
NovaPCIPassthrough:
- devname: "ens20f2"
physical_network: "datacentre"
NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter']
NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"]
Deploy Command:
openstack overcloud deploy --templates \
-e /usr/share/openstack-tripleo-heat-templates/environments/neutron-sriov.yaml \
-e /home/stack/sriov-environment.yaml \
...