The previous article introduced the architecture and packet flow of Linux Bridge with Provider Network. This article will cover Linux Bridge with self-service networks.

Table of Contents

Linux Bridge: Self-Service Networks

Self-service networks provide users with virtually unlimited network capacity. Neutron natively supports VLAN self-service networks, but this example uses VXLAN self-service networks.

Architecture

The diagram above shows the complete architecture of Linux Bridge – Self-Service Networks, clearly indicating which components run on the controller node, which on the compute node, and which on the network node.

This diagram illustrates how components are connected within self-service networks under a single provider network. In this example, the instance and DHCP agent are on the same host, but in real-world scenarios, the DHCP agent may be located on a different compute node.

Traffic Flow

This section will explain how packets actually flow through the architecture in various scenarios.

Architecture Setup

Below is the envisioned architecture, featuring two self-service networks separated by VXLAN, each with one instance.

Provider Network 1 (VLAN)
- VLAN ID 101 (Tagged)
Self-Service Network 1 (VXLAN)
- VXLAN ID (VNI) 101
Self-Service Network 2 (VXLAN)
- VXLAN ID (VNI) 102
Self-Service Router
- Gateway on the provider network
- Interface on Self-Service Network 1
- Interface on Self-Service Network 2
Instance 1
Instance 2

North-south traffic flow 1: Instance with fixed IP

For instances with fixed IPv4 addresses, the network node performs SNAT to route traffic from self-service networks to external networks like the Internet. For instances with fixed IPv6 addresses, the network node forwards packets via standard routing to external networks.

Instance on compute node 1 and using self-service network 1
Instance will forward packets to external network

Instance network interface (1) through veth pair (2) sends packets to the instance port of the self-service bridge
On the self-service bridge, iptables (3) handles firewall rules and connection tracking
The VXLAN interface (4) on the self-service bridge encapsulates packets into VNI 101
The physical network interface (5) corresponding to the VXLAN interface forwards packets over the overlay network to the network node (6)

Packet flow after reaching the network node:

The physical network interface (7) corresponding to the VXLAN interface sends packets to the VXLAN interface (8) and decapsulates the VXLAN packet
路由器的 self-service bridge 介面 (9) 將封包送往 self-service 路由器 namespace 內的 self-service network 介面 (10)
- 在 IPv4 的狀況下，路由器會用 SNAT 將 source IP 改成 router 的 IP 之後將封包投過 provider network 上的 gateway 介面 (11) 送往 gateway。
- 在 IPv6 的狀況下，路由器會將封包送往 next-hop IP 地址，通常是 provider network 的 gateway，也是會透過 provider network 上的 gateway 介面 (11)
路由器將封包送往路由器的 provider bridge 介面 (12)
VLAN sub-interface port (13) on the provider bridge will forward packets to the provider physical network interface (14)
The provider physical network interface (14) will add VLAN tag 101 to the packets and forward them to the provider network device (15)

The following portion works as described in standard networking—packets are forwarded through the actual network to the router and then sent out. The return path is simply reversed.

North-south traffic flow 2: Instance with Floating IP

For instances with fixed IPv4 addresses, the network node performs SNAT to route traffic from self-service networks to external networks like the Internet, and also performs DNAT to route incoming packets from external networks back to the instance. Floating IPs are not applicable to IPv6.

The packet flow from instance to external network is similar to Example 1. No further explanation is needed, except that the SNAT IP is replaced with a floating IP instead of the router’s IP. This will be discussed in detail for incoming traffic from external networks.

Instance on compute node 1 and using self-service network 1
The external network will forward packets to the instance

Packet flow from external network after reaching the network node:

The provider network device (1) will forward packets to the provider physical network interface (2)
The provider physical network interface (2) will remove VLAN tag 101 and forward packets to the VLAN sub-interface port (3) on the provider bridge
The provider bridge (4) will forward packets to the provider gateway interface (5) of the self-service router
- Under IPv4 conditions, the router will use DNAT to change the destination IP to the instance's IP and then forward packets through the self-service interface (6) to the self-service network
- Under IPv6 conditions, the router will forward packets to the next-hop IP address, typically the gateway of the self-service network, which also passes through the self-service interface (6)
The router will forward packets to the self-service bridge router interface (7)
The self-service bridge will forward packets to the VXLAN interface (8) and encapsulate packets with VNI 101
The provider network interface (9) will forward VXLAN packets through the overlay network (10) to the compute node

Packet flow after reaching the compute node:

The provider network interface (11) will forward packets to the VXLAN interface (12) and decapsulate VNI 101
On the self-service bridge, iptables (13) will handle firewall rules and connection tracking
The instance port (14) on the self-service bridge will forward packets via the veth pair to the instance network interface (15)

The above is the packet flow when using Floating IP.

East-west traffic flow 1: Instance on the same network

Instance 1 on compute node 1 also uses self-service network 1
Instance 2 on compute node 2 also uses self-service network 1
Instance 1 sends the packet to Instance 2

The situation described earlier is entirely identical.

Instance network interface (1) through veth pair (2) sends packets to the instance port of the self-service bridge
On the self-service bridge, iptables (3) handles firewall rules and connection tracking
The VXLAN interface (4) on the self-service bridge encapsulates packets into VNI 101
VXLAN interface corresponding to the physical network interface (5) forwards the packet through the overlay network to compute node 2 (6)

After the packet reaches compute node 2:

Physical network interface (7) forwards the packet to the VXLAN interface (8) and decapsulates VNI 101
On the self-service bridge, iptables (9) handles firewall rules and connection tracking
Instance port on the self-service bridge (10) uses the veth pair to forward the packet to the instance's network interface (11)

East-west traffic flow 2: Instance on different networks

Instance 1 on compute node 1 and uses self-service network 1
Instance 2 on compute node 1 and uses self-service network 2
Instance 1 sends the packet to Instance 2

The packet flow initiated and sent out is identical.

Instance network interface (1) through veth pair (2) sends packets to the instance port of the self-service bridge
On the self-service bridge, iptables (3) handles firewall rules and connection tracking
The VXLAN interface (4) on the self-service bridge encapsulates packets into VNI 101
VXLAN interface corresponding to the physical network interface (5) forwards the packet through the overlay network to compute node 2 (6)

Packet flow after reaching the network node:

The physical network interface (7) corresponding to the VXLAN interface sends packets to the VXLAN interface (8) and decapsulates the VXLAN packet
The self-service bridge interface (9) on the router forwards the packet to the self-service network 1 interface (10) inside the self-service router namespace
The router forwards the packet through the self-service network 2 interface (11) to the gateway of the self-service network
The router forwards the packet to the self-service network 2 bridge interface (12) on the router
Self-service network 2 bridge encapsulates the packet with VNI 102 and forwards it to the VXLAN interface
The physical network interface (14) corresponding to the VXLAN interface forwards the packet through the overlay network to compute node 1

Returning to compute node 1:

Physical network interface (16) forwards the packet to the VXLAN interface (17) and decapsulates VNI 102
On the self-service bridge, iptables (18) handles firewall rules and connection tracking
self-service bridge 的 instance port (19) 透過 veth pair 將封包送往 Instance 的網路介面 (20)

The above is the packet flow for Linux Bridge self-service networks under various scenarios.

Summary

This article explained the architecture and packet flow of Linux Bridge Self-Service Networks. After understanding how packets flow through the network, you will have a much clearer concept when debugging network connectivity issues. The next article will cover the architecture and packet flow of Open vSwitch: Provider networks.

From the front to the back: OpenStack Neutron Introduction – Linux Bridge – Self-Service Networks