What Is VXLAN and How to Use It on Linux with iproute2

VXLAN (Virtual eXtensible LAN) is a network overlay technology defined in RFC 7348 that encapsulates Ethernet frames inside UDP packets. It was originally designed to address the scaling limits of traditional VLANs in large datacenters, and today it underpins virtual networking stacks from Docker and Kubernetes to commercial SDN platforms.

If you have ever wondered how container orchestrators give every pod its own private Layer 2 segment across thousands of physical nodes — VXLAN is a big part of the answer.

This guide covers everything from the fundamental problem VXLAN solves to hands-on iproute2 commands you can run today on any modern Linux machine.

Related reading: VXLAN separates network namespaces at the overlay level. For the underlying Linux mechanism that creates isolated network stacks per container or process, see our companion article What Is a Linux Network Namespace.

---

1. What Is VXLAN?

The Problem: VLAN ID Exhaustion

Traditional VLANs use a 12-bit identifier field in the 802.1Q header. That gives you exactly 4,096 VLAN IDs (0–4095, with a few reserved). In a small enterprise network that is plenty. In a multi-tenant datacenter where each customer needs isolated Layer 2 segments — and where you might serve thousands of tenants, each with dozens of segments — 4,096 runs out quickly.

Beyond the ID limit, VLANs are also tied to physical switch topology. Spanning Tree Protocol (STP) blocks redundant paths to prevent loops, which wastes bandwidth and adds convergence delays. Migrating a workload from one rack to another while keeping it in the same VLAN requires careful switch reconfiguration. Live VM migration across racks becomes a coordination problem rather than a routine operation.

The Solution: Overlay Networks

VXLAN solves this by creating an overlay network — a virtual Layer 2 segment that runs on top of an existing Layer 3 (IP) infrastructure. The physical underlay can be anything: a simple routed datacenter fabric, a public cloud VPC, or an internet connection between two offices. VXLAN does not care about the underlay topology; it just needs UDP reachability between endpoints.

How VXLAN Works

The key components are:

• VTEP (VXLAN Tunnel Endpoint): A device — physical or virtual — that encapsulates outgoing Ethernet frames into VXLAN/UDP packets and decapsulates incoming ones. On Linux, a VXLAN interface acts as a software VTEP.
• VNI (VXLAN Network Identifier): A 24-bit field in the VXLAN header that identifies the virtual network. With 24 bits you get 16,777,216 unique network IDs — enough for even the largest multi-tenant environments.
• UDP Port 4789: The IANA-assigned destination port for VXLAN traffic. (Some older implementations used 8472 — notably the original Linux kernel default — but 4789 is the standard since RFC 7348.)

When Host A wants to send an Ethernet frame to Host B across a VXLAN tunnel:

1. The frame leaves a VM or container and arrives at the VTEP on Host A.
2. The VTEP wraps the original frame in a VXLAN header (containing the VNI), then wraps that in UDP/IP headers addressed to Host B's VTEP.
3. The packet travels through the underlay network as a normal UDP datagram.
4. Host B's VTEP receives the UDP packet, strips the outer headers, and delivers the original Ethernet frame to the correct VM or container.

The result: two machines that might be in different racks, different datacenters, or different cloud regions appear to share a flat Layer 2 segment. ARP, broadcast, and multicast all work as expected from the perspective of the workloads — the tunnel is transparent.

---

2. VXLAN vs. Other Overlay Technologies

Understanding where VXLAN fits helps you choose the right tool:

| Technology | Encapsulation | Header overhead | Key strength | |------------|---------------|-----------------|--------------| | VLAN (802.1Q) | Ethernet tag | 4 bytes | Simple, hardware support everywhere | | GRE | IP-in-IP | 24+ bytes | Point-to-point, no UDP, older compatibility | | VXLAN | L2-in-UDP | ~50 bytes | Multi-tenant scale, ECMP-friendly, Linux native | | GENEVE | L2-in-UDP + TLVs | Variable | Extensible metadata, used by OVN/Kubernetes |

GRE predates VXLAN and is simpler to configure for point-to-point tunnels, but it does not include a multi-tenant identifier and its IP-in-IP encapsulation is not ECMP-friendly (routers can't load-balance it across multiple paths without looking inside the packet). VXLAN's UDP encapsulation exposes source/destination ports to ECMP hash functions, enabling proper load balancing across bonded uplinks.

GENEVE is the spiritual successor to VXLAN, adding a flexible TLV extension header for carrying metadata. It is used by Open Virtual Network (OVN) and some Kubernetes CNI plugins. If you are building new infrastructure from scratch and need extensibility, GENEVE is worth evaluating — but VXLAN has wider tooling support today.

---

3. Linux Kernel Support

VXLAN has been built into the Linux kernel since version 3.7 (released December 2012). On any modern distribution — Ubuntu 20.04+, Debian 11+, CentOS 8+, Fedora 36+ — the kernel module is present and loads automatically when you create a VXLAN interface.

You do not need to install any additional software. Everything in this guide uses iproute2, which ships with all major Linux distributions. For bridge operations you will also use bridge (part of the iproute2 package) and optionally tcpdump for inspection.

Verify the module is available:

modinfo vxlan
# filename: /lib/modules/.../drivers/net/vxlan/vxlan.ko.zst
# description: Driver for VXLAN encapsulated traffic

---

4. Point-to-Point VXLAN with iproute2

The simplest and most portable VXLAN setup is a static unicast tunnel between two hosts. No multicast, no external controller — just two endpoints that know each other's IP addresses.

Suppose you have:

• Host A with underlay IP 192.168.1.10
• Host B with underlay IP 192.168.1.20

You want them to share a virtual Layer 2 segment using VNI 42, with overlay IPs 10.0.0.1 and 10.0.0.2.

On Host A:

# Create the VXLAN interface
ip link add vxlan0 type vxlan id 42 remote 192.168.1.20 dstport 4789 dev eth0

# Assign an overlay IP address
ip addr add 10.0.0.1/24 dev vxlan0

# Bring the interface up
ip link set vxlan0 up

On Host B:

# Create the VXLAN interface
ip link add vxlan0 type vxlan id 42 remote 192.168.1.10 dstport 4789 dev eth0

# Assign an overlay IP address
ip addr add 10.0.0.2/24 dev vxlan0

# Bring the interface up
ip link set vxlan0 up

Now test connectivity across the tunnel:

# From Host A
ping 10.0.0.2

# From Host B
ping 10.0.0.1

The remote parameter sets the unicast destination for all frames sent out of the interface. The dev eth0 parameter binds the tunnel to a specific underlay interface — useful when the host has multiple NICs. dstport 4789 explicitly sets the IANA standard port (recommended to avoid ambiguity with older kernels that defaulted to 8472).

To make these interfaces survive a reboot, add the commands to a systemd network unit, a NetworkManager dispatcher script, or an ip-up hook depending on your distribution.

---

5. Multicast VXLAN

In LAN environments where the underlay network supports IP multicast, you can use a multicast group instead of a static remote address. This enables dynamic MAC learning: VTEPs broadcast ARP requests to the multicast group and learn which MAC addresses live behind which VTEPs, just like a physical switch learning table.

# Host A — join multicast group 239.1.1.1 for VNI 100
ip link add vxlan100 type vxlan id 100 group 239.1.1.1 dstport 4789 dev eth0

ip addr add 10.1.0.1/24 dev vxlan100
ip link set vxlan100 up

# Host B — same group
ip link add vxlan100 type vxlan id 100 group 239.1.1.1 dstport 4789 dev eth0

ip addr add 10.1.0.2/24 dev vxlan100
ip link set vxlan100 up

Any VTEP that joins the group receives broadcast/multicast frames for VNI 100. The kernel's VXLAN driver populates the Forwarding Database (FDB) automatically as it sees traffic from remote VTEPs.

Limitation: Most WAN connections and public cloud environments do not support IP multicast. For cross-datacenter or cloud setups, use static FDB entries instead (see the next section).

---

6. Static FDB Entries for Production

In environments without multicast — which is the majority of production deployments — you manage the VXLAN Forwarding Database manually using bridge fdb. This is the approach used by Kubernetes CNI plugins like Flannel in host-gw mode and by Docker's overlay driver in swarm mode.

Create a VXLAN interface without a remote or group parameter (learning mode disabled):

# Host A
ip link add vxlan0 type vxlan id 42 dstport 4789 dev eth0 nolearning
ip addr add 10.0.0.1/24 dev vxlan0
ip link set vxlan0 up

Add a static FDB entry that maps the broadcast address to the remote VTEP's underlay IP. This tells the kernel: "for VNI 42, send BUM (Broadcast, Unknown-unicast, Multicast) traffic to 192.168.1.20":

# Direct all unknown/broadcast traffic for this VNI to Host B's VTEP
bridge fdb add 00:00:00:00:00:00 dev vxlan0 dst 192.168.1.20 via eth0 permanent

Once you know the MAC address of a specific remote VM, you can add a unicast FDB entry to bypass the broadcast fallback entirely:

# Learned MAC of a VM on Host B
bridge fdb add aa:bb:cc:dd:ee:ff dev vxlan0 dst 192.168.1.20 via eth0 permanent

On Host B, mirror the configuration pointing back at Host A:

ip link add vxlan0 type vxlan id 42 dstport 4789 dev eth0 nolearning
ip addr add 10.0.0.2/24 dev vxlan0
ip link set vxlan0 up
bridge fdb add 00:00:00:00:00:00 dev vxlan0 dst 192.168.1.10 via eth0 permanent

Static FDB entries are deterministic, scale to large clusters (you can script them from a central database), and work across any underlay that has IP connectivity.

---

7. VXLAN over IPv6

VXLAN works equally well over IPv6 underlay networks. Simply use IPv6 addresses for the VTEP endpoints:

# Host A (underlay IPv6: 2001:db8::1)
ip link add vxlan0 type vxlan id 42 remote 2001:db8::2 dstport 4789 dev eth0
ip addr add 10.0.0.1/24 dev vxlan0
ip link set vxlan0 up

# Host B (underlay IPv6: 2001:db8::2)
ip link add vxlan0 type vxlan id 42 remote 2001:db8::1 dstport 4789 dev eth0
ip addr add 10.0.0.2/24 dev vxlan0
ip link set vxlan0 up

The outer packet will be a UDP/IPv6 datagram; the inner payload is still a complete Ethernet frame that can carry IPv4, IPv6, or any other Ethertype. There is no restriction on mixing IPv4 overlay traffic over an IPv6 underlay.

---

8. Inspecting VXLAN

Detailed Interface Information

ip -d link show prints the full VXLAN configuration attached to an interface, including VNI, remote VTEP, port, and flags:

ip -d link show vxlan0

Sample output:

5: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN
    link/ether 6e:a4:c1:23:45:67 brd ff:ff:ff:ff:ff:ff
    vxlan id 42 remote 192.168.1.20 dev eth0 srcport 0 0 dstport 4789
    ageing 300 noudpcsum noudp6zerocsumtx noudp6zerocsumrx

The MTU of 1450 bytes — not 1500 — is expected. VXLAN adds roughly 50 bytes of overhead per packet (VXLAN header 8 bytes + UDP 8 bytes + IP 20 bytes + Ethernet 14 bytes), so the inner MTU must be smaller than the underlay MTU to avoid fragmentation.

Forwarding Database

Inspect which MAC-to-VTEP mappings are currently active:

bridge fdb show dev vxlan0

Sample output:

00:00:00:00:00:00 dev vxlan0 dst 192.168.1.20 via eth0 self permanent
aa:bb:cc:dd:ee:ff dev vxlan0 dst 192.168.1.20 via eth0 self permanent

The all-zeros entry is the BUM (broadcast/unknown/multicast) fallback. Specific MAC entries are learned dynamically or set statically with bridge fdb add.

Packet Capture

To verify that VXLAN encapsulation is actually happening on the wire, capture UDP traffic on port 4789 with tcpdump:

sudo tcpdump -i eth0 -n port 4789

Sample output:

12:34:56.789012 IP 192.168.1.10.54321 > 192.168.1.20.4789: VXLAN, flags [I], vni 42
        IP 10.0.0.1 > 10.0.0.2: ICMP echo request, id 1, seq 1, length 64

The outer IP header carries the underlay addresses (192.168.1.10 → 192.168.1.20). Inside the VXLAN frame you can see the original IP packet (10.0.0.1 → 10.0.0.2) and the VNI (42). This nested view is a reliable way to confirm that your tunnel is functioning and using the correct VNI.

---

9. Connecting VXLAN to a Linux Bridge

In VM and container networking, a VXLAN interface is rarely used in isolation. Instead, it is enslaved to a Linux bridge along with one or more tap/veth interfaces. This allows multiple local VMs or containers to share the same VXLAN segment, and the bridge handles local switching between them without going out to the underlay.

# Create the bridge
ip link add br0 type bridge

# Attach the VXLAN interface to the bridge
ip link set vxlan0 master br0

# Bring both up
ip link set vxlan0 up
ip link set br0 up

Now attach container or VM interfaces (veth pairs or tap devices) to br0 in the same way:

# Attach a veth peer that belongs to a container's network namespace
ip link set veth0 master br0
ip link set veth0 up

Traffic between two containers on the same host is switched locally by br0 without ever touching the VXLAN tunnel. Traffic destined for a container on a remote host travels through br0 → vxlan0 → underlay network → remote VTEP → remote br0 → remote container. The workloads see a single flat Ethernet segment regardless of where they are physically located.

This is precisely the pattern used by Docker's overlay network driver and Flannel's VXLAN backend for Kubernetes.

---

10. Performance Considerations

MTU Configuration

Getting the MTU right is the single most important performance tuning step for VXLAN. If the effective MTU is too large, packets get fragmented — or silently dropped if the DF (Don't Fragment) bit is set — causing visible performance degradation and mysterious timeouts.

The standard recommendation is to set the VXLAN interface MTU to 1450 bytes when the underlay MTU is 1500 bytes. If your underlay already runs jumbo frames (9000 bytes), you can set the VXLAN MTU to 8950 bytes and avoid fragmentation entirely.

Check and set the MTU explicitly:

# Check current MTU
ip link show vxlan0 | grep mtu

# Set MTU to 1450 (standard recommendation)
ip link set vxlan0 mtu 1450

Also configure the MTU on the bridge if you are using one:

ip link set br0 mtu 1450

Hardware Offloading

Modern NICs can offload VXLAN encapsulation and decapsulation to hardware, reducing CPU overhead significantly. Check whether your NIC supports VXLAN offload and enable it:

# Check current offload settings
ethtool -k eth0 | grep -i vxlan

# Enable tx-udp_tnl-segmentation (VXLAN TX offload)
ethtool -K eth0 tx-udp_tnl-segmentation on

# Enable rx-gro-hw (GRO offload for VXLAN RX)
ethtool -K eth0 rx-gro-hw on

Not all NICs support these flags. If ethtool -K returns an error, the NIC does not have hardware VXLAN support — the kernel will handle encapsulation in software, which is still fast but uses more CPU at high packet rates.

UDP Checksum

By default, Linux VXLAN interfaces disable UDP checksums on the outer header (noudpcsum flag visible in ip -d link show). This is intentional: the inner Ethernet frame already has its own checksums, and adding an outer UDP checksum at line rate is expensive. This is safe on trusted networks; on untrusted links you may want to re-enable it:

ip link add vxlan0 type vxlan id 42 remote 192.168.1.20 dstport 4789 dev eth0 udpcsum

---

11. Real-World Use: Docker, Kubernetes, and FastSox

VXLAN is not a niche tool — it is the backbone of some of the most widely deployed networking stacks in production:

Docker Overlay Networks: When you create a Docker swarm network (docker network create --driver overlay), Docker provisions a VXLAN interface on each participating node and manages FDB entries automatically via a distributed key-value store (etcd or Docker's built-in Raft). Every container on the overlay gets an IP on a shared virtual L2 segment regardless of which physical host it runs on.

Kubernetes CNI: Flannel's VXLAN backend, Calico (in VXLAN mode), and Cilium all create VXLAN tunnels between cluster nodes to provide pod-to-pod connectivity. The CNI plugin handles FDB population and ARP proxy so that pods see a flat network even across nodes in different availability zones.

FastSox Gateway Mesh: FastSox uses overlay networking principles — including VXLAN-style tunneling — to build its multi-gateway mesh. When traffic from a client needs to reach the optimal exit node, FastSox routes it through the gateway mesh transparently, giving end users consistent performance regardless of network topology. You can read more about how FastSox builds on these primitives at fastsox.com.

---

Summary

Here is a quick reference for the key commands used in this guide:

| Task | Command | |------|---------| | Create unicast VXLAN | ip link add vxlan0 type vxlan id <VNI> remote <IP> dstport 4789 dev eth0 | | Create multicast VXLAN | ip link add vxlan0 type vxlan id <VNI> group <MCAST> dstport 4789 dev eth0 | | Assign overlay IP | ip addr add 10.0.0.1/24 dev vxlan0 | | Bring interface up | ip link set vxlan0 up | | Add BUM FDB entry | bridge fdb add 00:00:00:00:00:00 dev vxlan0 dst <VTEP_IP> via eth0 permanent | | Add unicast FDB entry | bridge fdb add <MAC> dev vxlan0 dst <VTEP_IP> via eth0 permanent | | Show VXLAN details | ip -d link show vxlan0 | | Show FDB | bridge fdb show dev vxlan0 | | Capture VXLAN traffic | tcpdump -i eth0 -n port 4789 | | Attach to bridge | ip link set vxlan0 master br0 | | Set MTU | ip link set vxlan0 mtu 1450 |

VXLAN is a foundational technology for any engineer working on datacenter networking, container platforms, or multi-tenant infrastructure. Its 24-bit VNI space gives you practically unlimited virtual segments, its UDP encapsulation plays well with ECMP load balancing, and its first-class Linux kernel support means you can experiment with it on any modern machine without installing anything extra.

Understanding VXLAN at this level — VTEPs, VNIs, FDB entries, MTU implications — gives you the mental model to debug overlay networks confidently and to understand what tools like Docker, Kubernetes, and FastSox are doing under the hood.

FastSox is developed by OneDotNet Ltd.