Gateway

The gateway is responsible for forwarding the internet traffic. We use an Ubuntu server host as the basis for this setup.

Note

We will provide you with some sample configurations in this chapter. Keep in mind that you likely have to update these for sour specific setup depending on the number of VNIs you want to use and the IP space available to you.

Setting Up FRR

Install frr on the gateway.
```
sudo apt install frr
```

Create the file sample-configs/gateway/frr.conf with the following content. Remember to insert the IP of the gateway and the route reflector.

/etc/frr/daemons

log syslog informational
ip nht resolve-via-default
ip6 nht resolve-via-default
router bgp 65000
  bgp router-id <OWN-IP-ADDRESS>
  no bgp default ipv4-unicast
  neighbor fabric peer-group
  neighbor fabric remote-as 65000
  ! BGP sessions with route reflectors
  ! add one line per route reflector
  neighbor <IP-ADDRESS-OF-ROUTE-REFLECTOR> peer-group fabric
  !
  address-family l2vpn evpn
  neighbor fabric activate
  advertise-all-vni
  exit-address-family
  !
!

Create the file /etc/frr/daemons with the following content

/etc/frr/daemons

bgpd=yes
ospfd=no
ospf6d=no
ripd=no
ripngd=no
isisd=no
pimd=no
ldpd=no
nhrpd=no
eigrpd=no
babeld=no
sharpd=no
pathd=no
pbrd=no
bfdd=no
fabricd=no
vrrpd=no


vtysh_enable=yes
zebra_options="  -A 127.0.0.1 -s 90000000"
bgpd_options="   -A 127.0.0.1"
ospfd_options="  -A 127.0.0.1"
ospf6d_options=" -A ::1"
ripd_options="   -A 127.0.0.1"
ripngd_options=" -A ::1"
isisd_options="  -A 127.0.0.1"
pimd_options="   -A 127.0.0.1"
ldpd_options="   -A 127.0.0.1"
nhrpd_options="  -A 127.0.0.1"
eigrpd_options=" -A 127.0.0.1"
babeld_options=" -A 127.0.0.1"
sharpd_options=" -A 127.0.0.1"
pbrd_options="   -A 127.0.0.1"
staticd_options="-A 127.0.0.1"
bfdd_options="   -A 127.0.0.1"
fabricd_options="-A 127.0.0.1"
vrrpd_options="  -A 127.0.0.1"

Ensure these configuration files are owned by the frr user and group by running

sudo chown -R frr:frr /etc/frr/

Ensure that only the frr user has read and write permissions to the file and the frr group can read the file. All other users should not have any access. To achieve this, run
```
sudo chmod 640 /etc/frr/*
```
Restart frr by executing
```
sudo systemctl restart frr
```

Setting Up VXLAN Interfaces and Bridges

We will use the Ubuntu tool netplan to set up and persist the required network devices. Check below for a method without the need for netplan.

Overwrite the file /etc/netplan/00-installer-config.yaml with the following content. Remember to replace the interface name for your current internet connection and set the local IP address.

/etc/netplan/00-installer-config.yaml

network:
    ethernets:
        <UPLINK IF NAME>:
            dhcp4: true
    tunnels:
        vxlan1:
            mode: vxlan
            id: 1
            local: <GATEWAY IP>
            ttl: 10
            mac-learning: false
            port: 4789

        vxlan2:
            mode: vxlan
            id: 2
            local: <GATEWAY IP>
            ttl: 10
            mac-learning: false
            port: 4789

        vxlan3:
            mode: vxlan
            id: 3
            local: <GATEWAY IP>
            ttl: 10
            mac-learning: false
            port: 4789

    bridges:
        br1:
            interfaces:
                - vxlan1
            parameters:
                stp: false
            addresses:
                - 10.1.2.1/24
        br2:
            interfaces:
                - vxlan2
            parameters:
                stp: false
            addresses:
                - 10.1.3.1/24
        br3:
            interfaces:
                - vxlan3
            parameters:
                stp: false
            addresses:
                - 10.1.4.1/24

Warning

This setup is for a very basic gateway. Please make sure you understand what it does and change it for your needs. Especially, you may want to increase the number of VNIs depending on what you set in the APs.

Refer back to the parameters you chose for your setup.

Restart the machine

Tip

In theory, using netplan apply should work. In our experience however, this did not give us the desired result so rebooting is the best way to go here.

Setting up VXLANs without netplan

As an alternative, you can also set up the interfaces using the ip command directly. Note that these interfaces will not persist on reboot so you have to run the following script on every boot:

#!/bin/bash
# Copied from Vincent Bernat's blog post:
for vni in {1..20}; do
    # Create VXLAN interface
    ip link add vxlan"${vni}" type vxlan
        id "${vni}" \
        dstport 4789 \
        nolearning
    # Create companion bridge
    brctl addbr br"${vni}"
    brctl addif br"${vni}" vxlan"${vni}"
    brctl stp br"${vni}" off
    ip link set up dev br"${vni}"
    ip a add 10.0."${vni}".1/24 dev br"${vni}"
    ip link set up dev vxlan"${vni}"
done

Enable IP Forwarding

Open the file /etc/sysctl.conf in an editor of your choice with root permissions:
```
sudo nano /etc/sysctl.conf
```
Uncomment the line
```
net.ipv4.ip_forward=1
```
to enable IPv4 forwarding
Uncomment the line
```
net.ipv6.conf.all.forwarding=1
```
to enable IPv6 forwarding
Reload the configuration file by executing
```
sudo sysctl -p
```

Setting Up Nftables

Make sure you have nftables installed by running
```
nft -v
```
Tip

nftables should come preinstalled with Ubuntu Server 22.04 and later

Create/Update the file /etc/nftables.conf with root privileges and paste the following content. You may want to update the vxlans variable to match your setup and set the uplink interface correctly.

/etc/nftables.conf

#!/usr/sbin/nft -f

# Setup for 3 VNI's
define vxlans = { br1, br2, br3, }

define uplink = <Uplink interface>

flush ruleset

# NAT for all VNIs

table inet filter {
        chain input {
                type filter hook input priority filter;
        }
        chain forward {
                type filter hook forward priority filter; policy drop;
                ct state established, related accept
                iifname $vxlans oifname $uplink counter accept
        }
        chain output {
                type filter hook output priority filter;
        }
        chain postrouting {
                type nat hook postrouting priority 100; policy accept;
                iifname $vxlans oifname $uplink masquerade
        }
}

Restart the system

Setting Up Dnsmasq

Install dnsmasq as a DHCP server on the gateway.
```
sudo apt install dnsmasq
```
Update the file /etc/dnsmasq.conf with the following content. You likely need to add more entries for the correct IP ranges when you have more than 3 VNIs in use.

Tip

Refer back to the parameters you decided on for the IPv4 range for each VNI
/etc/dnsmasq.conf
```
interface=br1
interface=br2
interface=br3

bind-interfaces
no-resolv
dhcp-authoritative

dhcp-range=10.0.1.100,10.0.1.200,255.255.255.0,5m
dhcp-range=10.0.2.100,10.0.2.200,255.255.255.0,5m
dhcp-range=10.0.3.100,10.0.3.200,255.255.255.0,5m
```
Restart dnsmasq
```
sudo systemctl restart dnsmasq
```