The previous article introduced the architecture and packet flow of Open vSwitch with Provider Networks. Today, we will introduce Open vSwitch with self-service networks.

Contents

Open vSwitch: Self-service Networks

Architecture

The diagram above provides an overview of the entire Open vSwitch – Self-service networks architecture. You can see which components run on the controller node, which on the compute node, and which on the network node.

This diagram illustrates how the components are interconnected within a single untagged (flat) provider network and a single self-service network. In this example, the Instance and the DHCP agent are on the same machine, but in practice, the DHCP agent might reside on a different compute node.

Traffic Flow

This section introduces how packets actually flow through the architecture under various scenarios.

Architecture Configuration

Below is our envisioned architecture, featuring two self-service networks isolated via VXLAN, each with one instance.

Provider network 1 (VLAN)
- VLAN ID 101 (tagged)
Self-service network 1 (VXLAN)
- VXLAN ID (VNI) 101
Self-service network 2 (VXLAN)
- VXLAN ID (VNI) 102
Self-service router
- Gateway on the provider network
- Interface on self-service network 1
- Interface on self-service network 2
Instance 1
Instance 2

North-South Traffic 1: Instance with Fixed IP

For instances with fixed IPv4 addresses, the network node performs SNAT on North-South traffic traveling from self-service networks to external networks like the Internet. For instances with fixed IPv6 addresses, the network node sends packets to the external network via regular routing.

Instance on compute node 1 using self-service network 1
Instance sends packets to the external network

The instance's network interface (1) sends packets via a veth pair (2) to the instance port on the Linux bridge.
On the Linux bridge, iptables (3) handles firewalling and connection tracking.
The OVS interface (4) on the Linux bridge sends packets via a veth pair to the OVS integration bridge security group interface (5).
The OVS integration bridge tags the packet with an internal VLAN tag.
The OVS integration bridge int-br-provider patch interface (6) sends the packet to the OVS tunnel bridge r patch interface (7).
The OVS tunnel bridge (8) encapsulates the packet into VNI 101.
The physical network interface (9) sends the packet to the network node via the overlay network (10).

After the packet reaches the network node:

The physical network interface (11) corresponding to the overlay network sends the packet to the OVS tunnel bridge (12).
The OVS tunnel bridge decapsulates the VXLAN packet and adds an internal tunnel ID.
The OVS tunnel bridge replaces the internal tunnel ID with an internal VLAN tag.
The OVS tunnel bridge patch port (13) sends the packet to the OVS integration bridge patch port (14).
The OVS integration bridge port (15) of the self-service network strips the internal VLAN tag and sends the packet to the self-service network interface (16) in the router namespace.
- In the case of IPv4, the router uses SNAT to change the source IP to the router's IP and then sends the packet to the gateway via the gateway interface (17) on the provider network.
- In the case of IPv6, the router sends the packet to the next-hop IP address, typically the provider network gateway, via the gateway interface (17) on the provider network.
The router sends the packet to the OVS integration bridge interface (18) corresponding to the router's provider network.
The OVS integration bridge adds an internal VLAN tag to the packet.
The int-br-provider patch port (19) on the OVS integration bridge sends the packet to the OVS provider bridge phy-br-provider patch port (20).
The OVS provider bridge replaces the internal VLAN tag with the actual provider network VLAN tag 101.
The provider physical network interface (14) adds VLAN tag 101 to the packet and sends it to the physical network infrastructure (15).
The provider network interface (21) on the OVS provider bridge sends the packet to the physical network interface (22).
The physical network interface sends the packet to the switch (23) in the physical network infrastructure.

The subsequent part is the same as general networking knowledge: the packet is sent to the router within the physical network and then transmitted out. The return path for the packet follows the exact reverse process.

North-South Traffic 2: Instance with Floating IP

For instances with fixed IPv4 addresses, the network node performs SNAT on North-South traffic from self-service networks to external networks like the Internet and performs DNAT on traffic returning from the external network. Floating IPs do not apply to IPv6.

Since the packet flow from the instance to the external network is the same as in Example 1, I won't go into detail here. The only difference is that the SNAT IP is replaced by the floating IP instead of the router's IP. This section will introduce the traffic returning from the external network.

Instance on compute node 1 using self-service network 1
The external network sends packets to the instance.

After the external network packet reaches the network node:

Physical network infrastructure (1) sends packets to the provider physical network interface (2).
The provider physical network interface (2) removes VLAN tag 101 and sends the packet to the provider network port (3) on the OVS provider bridge.
The OVS provider bridge replaces VLAN tag 101 with an internal VLAN tag.
The phy-br-provider port (4) on the OVS provider bridge sends the packet to the int-br-provider port (5) on the OVS integration bridge.
The provider network port (6) on the OVS integration bridge removes the internal VLAN tag and sends the packet to the provider network interface (7) in the router namespace.
- In the case of IPv4, the router uses DNAT to change the destination IP to the instance's IP and then sends the packet to the self-service network via the self-service interface (8).
- In the case of IPv6, the router sends the packet to the next-hop IP address, which is typically the gateway of the self-service network, also via the self-service interface (8).
The router sends the packet to the interface (9) on the OVS integration bridge corresponding to the self-service network.
The OVS integration bridge adds an internal VLAN tag to the packet.
The OVS tunnel bridge replaces the internal tunnel ID with an internal VLAN tag.
The patch-tun patch port (10) on the OVS integration bridge sends the packet to the OVS tunnel bridge patch-int patch port (11).
The OVS tunnel bridge (12) encapsulates the packet into VNI 101.
The physical network interface (13) sends the VXLAN packet to the compute node via the overlay network (14).

After the packet reaches the compute node:

The physical network interface (15) corresponding to the overlay network sends the packet to the OVS tunnel bridge (16).
The OVS tunnel bridge decapsulates the VXLAN packet and adds an internal tunnel ID.
The OVS tunnel bridge replaces the internal tunnel ID with an internal VLAN tag.
The OVS tunnel bridge patch-int port (17) sends the packet to the OVS integration bridge patch-tun patch port (18).
The OVS integration bridge removes the internal VLAN tag.
The OVS integration bridge security group interface (19) sends the packet to the OVS interface (20) on the Linux bridge.
On the Linux bridge, iptables (21) handles the firewall and connection tracking.
The instance port (22) of the Linux bridge sends the packet to the instance's network interface (23) via a veth pair.

The above describes the return packet flow when using a Floating IP.

East-West Traffic 1: Instances on the same network

Instance 1 is located on compute node 1 and uses self-service network 1.
Instance 2 is located on compute node 2 and uses self-service network 1.
Instance 1 sends packets to Instance 2

On compute node 1:

The instance's network interface (1) sends packets via a veth pair (2) to the instance port on the Linux bridge.
On the Linux bridge, iptables (3) handles firewalling and connection tracking.
The OVS interface (4) on the Linux bridge sends packets via a veth pair to the OVS integration bridge security group interface (5).
The OVS integration bridge tags the packet with an internal VLAN tag.
The OVS integration bridge int-br-provider patch interface (6) sends the packet to the OVS tunnel bridge r patch interface (7).
The OVS tunnel bridge (8) encapsulates the packet into VNI 101.
The physical network interface (9) sends the packet to compute node 2 via the overlay network (10).

After arriving at compute node 2:

The physical network interface (11) corresponding to the overlay network sends the packet to the OVS tunnel bridge (12).
The OVS tunnel bridge decapsulates the VXLAN packet and adds an internal tunnel ID.
The OVS tunnel bridge replaces the internal tunnel ID with an internal VLAN tag.
The OVS tunnel bridge patch-int port (13) sends the packet to the OVS integration bridge patch-tun patch port (14).
The OVS integration bridge removes the internal VLAN tag.
The OVS integration bridge security group interface (15) sends the packet to the OVS interface (16) on the Linux bridge.
On the Linux bridge, iptables (17) handles firewall rules and connection tracking.
The instance port (18) of the Linux bridge sends the packet to the Instance's network interface (19) via a veth pair.

East-West Traffic 2: Instances on different networks

Instance 1 is on compute node 1 and uses self-service network 1
Instance 2 is on compute node 1 and uses self-service network 2
Instance 1 sends packets to Instance 2

On the compute node:

The instance's network interface (1) sends packets via a veth pair (2) to the instance port on the Linux bridge.
On the Linux bridge, iptables (3) handles firewalling and connection tracking.
The OVS interface (4) on the Linux bridge sends packets via a veth pair to the OVS integration bridge security group interface (5).
The OVS integration bridge tags the packet with an internal VLAN tag.
The OVS integration bridge int-br-provider patch interface (6) sends the packet to the OVS tunnel bridge r patch interface (7).
The OVS tunnel bridge (8) encapsulates the packet into VNI 101.
The physical network interface (9) sends the packet to the network node via the overlay network (10).

After the packet reaches the network node:

The physical network interface (11) corresponding to the overlay network sends the packet to the OVS tunnel bridge (12).
The OVS tunnel bridge decapsulates the VXLAN packet and adds an internal tunnel ID.
The OVS tunnel bridge replaces the internal tunnel ID with an internal VLAN tag.
The OVS tunnel bridge patch port (13) sends the packet to the OVS integration bridge patch-tun patch port (14).
The OVS integration bridge port (15) of self-service network 1 strips the internal VLAN tag and sends the packet to the self-service network 1 interface (16) in the router namespace.
The router sends the packet to the next-hop IP address, typically via the self-service network 2 interface (17) to the gateway of self-service network 2.
The router sends the packet to the OVS integration bridge interface (18) corresponding to the router's self-service network 2.
The OVS integration bridge adds an internal VLAN tag to the packet.
The OVS integration bridge replaces the internal VLAN tag with the internal tunnel ID.
The patch-tun patch port (19) on the OVS integration bridge sends the packet to the OVS tunnel bridge patch-int patch port (20).
The OVS tunnel bridge (21) encapsulates the packet into VNI 102.
The physical network interface (22) sends the VXLAN packet to the compute node via the overlay network (23).

After the packet returns to the compute node:

The physical network interface (24) corresponding to the overlay network sends the packet to the OVS tunnel bridge (25).
The OVS tunnel bridge decapsulates the VXLAN packet and adds an internal tunnel ID.
The OVS tunnel bridge replaces the internal tunnel ID with an internal VLAN tag.
The OVS tunnel bridge patch-int patch port (26) sends the packet to the OVS integration bridge patch-tun patch port (27).
The OVS integration bridge removes the internal VLAN tag.
The OVS integration bridge security group interface (28) sends the packet to the OVS interface (29) on the Linux bridge.
On the Linux bridge, iptables (30) handles firewall rules and connection tracking.
The instance port (31) of the Linux bridge sends packets to the instance's network interface (32) via a veth pair.

The above covers the packet flows for Open vSwitch self-service networks under various scenarios.

Summary

This article introduced the architecture and packet flow of Open vSwitch self-service networks. Similarly, understanding how network packets flow will give you a better conceptual grasp when debugging connectivity issues. The next article will still focus on Neutron, but it will introduce a newer plugin: the OVN plugin architecture.

From Bare Metal to the Cloud: Introduction to OpenStack Neutron — OVS Self-service Networks