1. Creating the data center side of the IPsec VPN

To facilitate zero touch provisioning of new spokes to establish VPNs on each data center FortiGate, this example uses dial-up VPNs with auto-discovery-sender enabled in the ADVPN configuration.

Also, add-route is disabled to support multiple dynamic tunnels to the same host advertising the same network. This dynamic discovery of the network is facilitated by the BGP configuration.

Wildcard security associations are used for phase 2 since BGP routes determine whether traffic is sent over the IPSec VPN tunnel. In this example, IPsec VPN is added to each FortiGate interface connected to the Internet.

The Phase 1 configuration includes:

• A dynamic VPN tunnel name that is 11 characters or less.
• Setting type to dynamic

• Setting interface to the Internet connected interface
• Setting peertype to any

• Setting add-route to disable

• Setting auto-discovery-sender to enable


config vpn ipsec phase1-interface
    edit "vpn-br1-1"
        set type dynamic
        set interface "vlan-3510"
        set peertype any
        set proposal aes256-sha256
        set add-route disable
        set dhgrp 5
        set auto-discovery-sender enable
        set psksecret <password>
    next
    edit "vpn-br1-2"
        set type dynamic
        set interface "vlan-3511"
        set peertype any
        set proposal aes256-sha256
        set add-route disable
        set dhgrp 5
        set auto-discovery-sender enable
        set psksecret <password>
    end

The Phase 2 configuration includes:

• Setting phase1name to the name of the phase 1 configuration
• Disabling pfs and replay

config vpn ipsec phase2-interface
    edit "vpn-br1-1_p2"
        set phase1name "vpn-isp-a"
        set proposal aes256-sha256
        set pfs disable
        set replay disable
     next
     edit "vpn-br1-2_p2"
        set phase1name "vpn-isp-b"
        set proposal aes256-sha256
        set pfs disable
        set replay disable
     end
 

2. Adding addresses to the tunnel interfaces

The BGP configuration requires IP addresses assigned to the IPsec VPN tunnel interfaces that BGP peers over. The ADVPN feature enabled by set auto-discovery-sender enable allows FortiOS to establish a point-to-multipoint connection to each FortiGate.

The IPsec VPN tunnel interface ip is set to the IP address that the tunnels will connect to, and remote-ip is set to the highest unused IP address that is part of your tunnel network. This adds two host-based routes to the FortiGate’s routing table that point directly back to the branch FortiGate.

The IPsec VPN interface configuration includes:

• Setting the ip to <vpn interface ip> 255.255.255.255
• Setting type to tunnel
• Setting remote-ip to the highest unused IP address in the VPN subnet
• Setting allowaccess to ping to allow for confirmation that a point-to-point tunnel has been established between the data center FortiGate and the branch FortiGate.

config system interface
    edit "vpn-br1-1"
        set vdom "root"
        set ip 10.254.0.1 255.255.255.255
        set allowaccess ping
        set type tunnel
        set remote-ip 10.254.0.254
        set interface "port1"
    next
    edit "vpn-br1-2"
        set vdom "root"
        set ip 10.254.1.1 255.255.255.255
        set allowaccess ping
        set type tunnel
        set remote-ip 10.254.1.254
        set interface "port2"
    end

3. Implementing route discovery with BGP

Network route discovery is facilitated by BGP and EBGP, which prevent the redistribution of routes learned that are contained in the same autonomous system number as the host. Also, EBGP influences route selection on the branches because of AS-Path prepending.

Enable ebgp-multipath to allow the FortiGate to dynamically discover multiple paths for networks advertised at branches.

Configure neighbor-range and neighbor-group to allow peering relationships to be established without defining each individual peer. The branch IPsec VPN tunnel interface addresses must be in the BGP peer range.

The BGP configuration includes:

• Enabling ebgp-multipath
• Enabling soft-reconfiguration and link-down-failover for each BGP peer in the neighbor group
• Adding the branch remote-as (which is 65501) to each peer configuration
• Setting the prefix for the neighbor range to the network matching the BGP peers
• Configuring a network with the prefix of the network advertised into BGP

To facilitate the fastest route failovers, the following timers are set to their lowest values:

• scan-time
• advertisement-interval
• keep-alive timer
• holdtime-timer

config router bgp
    set as 65500
    set router-id 10.10.0.1
    set ebgp-multipath enable
    set scan-time 5
    set graceful-restart enable
    config neighbor-group
        edit "branch-peers-1"
            set advertisement-interval 1
            set link-down-failover enable
            set soft-reconfiguration enable
            set remote-as 65501
            set keep-alive-timer 1
            set holdtime-timer 3
        next
        edit "branch-peers-2"
            set advertisement-interval 1
            set link-down-failover enable
            set soft-reconfiguration enable
            set remote-as 65501
            set keep-alive-timer 1
            set holdtime-timer 3
        next
    end
    config neighbor-range
        edit 1
            set prefix 10.254.0.0 255.255.255.0
            set neighbor-group "branch-peers-1"
        next
        edit 2
            set prefix 10.254.1.0 255.255.255.0
            set neighbor-group "branch-peers-2"
        next
    end
    config network
        edit 1
            set prefix 10.200.1.0 255.255.255.0
        next
        edit 2
            set prefix 10.200.0.0 255.255.255.0
        next
        edit 3
            set prefix 10.200.3.0 255.255.255.0
        next
    end
end

4. Controlling access to data center networks

Create firewall policies to allow users on the branch office networks to access the data center networks (behind the FortiGate). Security profiles can be added to these firewall policies to inspect of layer 7 traffic.

Include a policy on the data center FortiGate to allow a branch FortiGate to check the health of the data center FortiGate by allowing the branch FortiGate to ping the data center FortiGate IPsec VPN interface:

• Source interface: IPsec VPN interface
• Destination interface: Internal interface
• Source Address: Tunnel IP addresses of branch
• Destination Address: Data Center 1 FortiGate Internal interface
• Action: Accept
• Schedule: Always
• Service: ICMP

Policies to allow traffic from branch networks to reach data center networks should have the following firewall settings:

• Source interface: IPsec VPN interface
• Destination interface: Internal interface
• Source Address: Branch networks
• Destination Address: Date center networks
• Action: Accept
• Schedule: Always (or define a more restrictive schedule)
• Service: Allowed Service(s)

5. Pointing to branch offices with black hole routes

It is a best practice to create black hole routes with destinations set to each branch  network. If the FortiGate temporarily loses connectivity with a branch network, traffic destined to that network is sent to the black hole until connectivity has been restored.

Each Black hole route includes:

• Setting dst to the branch network IP address
• Setting the distance to 255

config router static
    edit 1
        set dst 10.0.0.0/14
        set distance 255
        set blackhole enable
    next
end

1. Creating the branch side of the IPsec VPN

Create two Phase 1 configurations, one for each data center. These configurations include:

• Setting peertype to any
• Setting remote-gw to the IP address of the data center.

config vpn ipsec phase1-interface
    edit "vpn_dc1-1"
        set interface "vlan-3000"
        set peertype any
        set proposal aes256-sha256
        set dhgrp 5
        set remote-gw 172.20.10.10
        set psksecret <password>
    next
    edit "vpn_dc1-2"
        set interface "vlan-3001"
        set peertype any
        set proposal aes256-sha256
        set dhgrp 5
        set remote-gw 172.20.11.10
        set psksecret <password>
    next
end

Create two Phase 2 configurations, one for each data center. These configurations include:

• Disabling pfs and replay
• Enabling auto-negotiate to ensure VPN establishment

config vpn ipsec phase2-interface
    edit "vpn_dc1-1_p2"
        set phase1name "vpn_dc1-1"
        set proposal aes256-sha256
        set pfs disable
        set replay disable
        set auto-negotiate enable
    next
    edit "vpn_dc1-2_p2"
        set phase1name "vpn_dc1-2"
        set proposal aes256-sha256
        set pfs disable
        set replay disable
        set auto-negotiate enable
    next
end

2. Adding IP addresses to the tunnel interfaces

To establish the point-to-multipoint IPsec VPN between the branch and the data center, the tunnel interfaces must include the following IP addresses.

The IPsec VPN Interface configuration includes:

• Setting ip to the local IP address of the VPN interface
• Setting remote-ip to the data center FortiGate’s IPsec VPN interface IP address

config system interface
    edit "vpn_dc1-1"
        set vdom "root"
        set ip 10.254.0.2 255.255.255.255
        set allowaccess ping
        set type tunnel
        set remote-ip 10.254.0.1
        set interface "wan1"
    next
    edit "vpn_dc1-2"
        set vdom "root"
        set ip 10.254.1.2 255.255.255.255
        set allowaccess ping
        set type tunnel
        set remote-ip 10.254.1.1
        set interface "wan2"
    next
end

3. Implementing route discovery with BGP

BGP allows the branch and data center FortiGates to dynamically discover routes from each other. To make this happen add the data center FortiGate IPsec VPN tunnel interface IP addresses to the branch BGP configuration as  BGP peers.

Routes that have the same network mask, administrative distance, and priority are automatically considered for SD-WAN when the interfaces where those routes are learned are added to the SD-WAN interface group.

The branch BGP configuration includes:

• Enabling ebgp-multipath
• Enabling soft-reconfiguration and link-down-failover for each BGP peer
• Adding the data center remote-as (which is 65500) to each peer configuration
• Setting the prefix for the neighbor range to the network matching the BGP peers

To facilitate the fastest route failovers, the following timers are set to their lowest values:

• scan-time
• advertisement-interval
• keep-alive timer
• holdtime-timer

config router bgp
    set as 65501
    set router-id 10.254.0.2
    set keepalive-timer 1
    set holdtime-timer 3
    set ebgp-multipath enable
    set scan-time 5
    set distance-external 1
    config neighbor
        edit "10.254.0.1"
            set advertisement-interval 1
            set link-down-failover enable
            set soft-reconfiguration enable
            set remote-as 65500
        next
        edit "10.254.1.1"
            set advertisement-interval 1
            set link-down-failover enable
            set soft-reconfiguration enable
            set remote-as 65500
        next
    end
end

4. Setting up the load balancing SD-WAN configuration

The SD-WAN configuration sets up load balancing based on link quality. Link quality is determined by health checking; which measures jitter, packet loss, and latency on each link. FortiOS dynamically creates policy routes that send traffic over the link with the highest quality.

config system virtual-wan-link
    set status enable
    config members
        edit 1
            set interface "vpn_dc1-1"
        next
        edit 2
            set interface "vpn_dc1-2"
        next
    end
end

Create SD-WAN Health-Checks for each data center network. Set server to the IP address of a server on the data center network.

config system virtual-wan-link
    config health-check
        edit "datacenter1-net"
            set server "10.200.1.1"
            set interval 1
            set failtime 1
            set recoverytime 3
        next
        edit "datacenter2-net"
             set server "10.200.2.1"
             set interval 1
             set failtime 1
             set recoverytime 3
         end
     end

Add SD-WAN Service Rules to define the criteria for the policy routes. Criteria include:

• Protocol
• Destination Address
• Source Address
• Identity Based Group
• Internet Service Definition
• Source Port
• Destination Port
• Destination Tag

To dynamically determine the networks the policy routes point to, the routes learned from a BGP neighbor are matched against a route map and matching routes are tagged. The service rules determine the routes to use based on these tags.

config system virtual-wan-link
    config service
        edit 1
            set mode priority
            set dst-tag 10
            set health-check "datacenter1-net"
            set priority-members 1 2
        next
        edit 2
             set mode priority
             set dst-tag 10
             set health-check "datacenter2-net"
             set priority-members 1 2
         next
    end
end

5. Controlling access from branch networks

Create firewall policies to allow users on the branch office networks to access the data center networks. Security profiles can be enabled on these firewall policies to inspect layer 7 traffic.

Policies to Allow traffic from the branch office to the data center networks:

• Source interface: Internal interface
• Destination interface: SD-WAN interface
• Source Address: Branch networks
• Destination Address: Data center networks
• Action: Accept
• Schedule: Always (or define a more restrictive schedule)
• Service: Allowed services

Policies to allow traffic from the data center to the branch networks:

• Source interface: SD-WAN interface
• Destination interface: Internal interface
• Source Address: Data center networks
• Destination Address: Branch networks
• Action: Accept
• Schedule: Always (or define a more restrictive schedule)
• Service: Allowed Services