[Rdo-list] Network Isolation setup check

Hi everybody, I'm trying to deploy a tripleo environment with network isolation using a pool of 8 machines: 3 controller, 2 compute and 3 storage. Each of those machine has got 2 network interfaces, the first one (em1) connected to the lan, the second one (em2) used for the undercloud provisioning. The ultimate goal of the setup is to have the ExternalNet on the em1 (so to be able to put instances with floating Ips in the LAN) and all the other networks (InternalApi, Storage and StorageMgmt) on the em2. To produce what described I created this network-environment.yaml configuration: resource_registry: OS::TripleO::BlockStorage::Net::SoftwareConfig: /home/stack/nic-configs/cinder-storage.yaml OS::TripleO::Compute::Net::SoftwareConfig: /home/stack/nic-configs/compute.yaml OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/nic-configs/controller.yaml OS::TripleO::ObjectStorage::Net::SoftwareConfig: /home/stack/nic-configs/swift-storage.yaml OS::TripleO::CephStorage::Net::SoftwareConfig: /home/stack/nic-configs/ceph-storage.yaml parameter_defaults: # Customize the IP subnets to match the local environment InternalApiNetCidr: 172.17.0.0/24 StorageNetCidr: 172.18.0.0/24 StorageMgmtNetCidr: 172.19.0.0/24 TenantNetCidr: 172.16.0.0/24 ExternalNetCidr: 10.1.240.0/24 ControlPlaneSubnetCidr: '24' InternalApiAllocationPools: [{'start': '172.17.0.10', 'end': '172.17.0.200'}] StorageAllocationPools: [{'start': '172.18.0.10', 'end': '172.18.0.200'}] StorageMgmtAllocationPools: [{'start': '172.19.0.10', 'end': '172.19.0.200'}] TenantAllocationPools: [{'start': '172.16.0.10', 'end': '172.16.0.200'}] ExternalAllocationPools: [{'start': '10.1.240.10', 'end': '10.1.240.200'}] # Specify the gateway on the external network. ExternalInterfaceDefaultRoute: 10.1.240.254 # Gateway router for the provisioning network (or Undercloud IP) ControlPlaneDefaultRoute: 192.0.2.1 # Generally the IP of the Undercloud EC2MetadataIp: 192.0.2.1 DnsServers: ["10.1.241.2"] InternalApiNetworkVlanID: 2201 StorageNetworkVlanID: 2203 StorageMgmtNetworkVlanID: 2204 TenantNetworkVlanID: 2202 # This won't actually be used since external is on native VLAN, just here for reference #ExternalNetworkVlanID: 38 # Floating IP networks do not have to use br-ex, they can use any bridge as long as the NeutronExternalNetworkBridge is set to "''". NeutronExternalNetworkBridge: "''" And modified the controller.yaml file in this way (default parts are omitted, nic1 == em1 and nic2 == em2): ... ... resources: OsNetConfigImpl: type: OS::Heat::StructuredConfig properties: group: os-apply-config config: os_net_config: network_config: - type: ovs_bridge name: {get_input: bridge_name} use_dhcp: false dns_servers: {get_param: DnsServers} addresses: - ip_netmask: list_join: - '/' - - {get_param: ControlPlaneIp} - {get_param: ControlPlaneSubnetCidr} routes: - ip_netmask: 169.254.169.254/32 next_hop: {get_param: EC2MetadataIp} members: - type: interface name: nic1 addresses: - ip_netmask: {get_param: ExternalIpSubnet} routes: - ip_netmask: 0.0.0.0/0 next_hop: {get_param: ExternalInterfaceDefaultRoute} - type: interface name: nic2 # force the MAC address of the bridge to this interface primary: true - type: vlan vlan_id: {get_param: InternalApiNetworkVlanID} addresses: - ip_netmask: {get_param: InternalApiIpSubnet} - type: vlan vlan_id: {get_param: StorageNetworkVlanID} addresses: - ip_netmask: {get_param: StorageIpSubnet} - type: vlan vlan_id: {get_param: StorageMgmtNetworkVlanID} addresses: - ip_netmask: {get_param: StorageMgmtIpSubnet} - type: vlan vlan_id: {get_param: TenantNetworkVlanID} addresses: - ip_netmask: {get_param: TenantIpSubnet} outputs: OS::stack_id: description: The OsNetConfigImpl resource. value: {get_resource: OsNetConfigImpl} The deploy of the overcloud was invoked with this command: openstack overcloud deploy --templates --libvirt-type=kvm --ntp-server 10.5.26.10 --control-scale 3 --compute-scale 2 --ceph-storage-scale 3 --block-storage-scale 0 --swift-storage-scale 0 --control-flavor baremetal --compute-flavor baremetal --ceph-storage-flavor baremetal --block-storage-flavor baremetal --swift-storage-flavor baremetal --templates -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /home/stack/network-environment.yaml Now the point is that I need to know if my configurations are formally correct since I got some network problems once the post deployment status were done. I still don't know (we're investigating) if those problems are related to the switch configurations (so hardware side) but for some reason everything exploded. What I saw until the machines were reachable was what I was expecting: the external address assigned to em1 and the vlans correctly assigned to the em2. With all the external IPs address pingable one to each other. But I was not able to do further tests. Many thanks, -- Raoul Scarazzini rasca(a)redhat.com