Decrypting ESP payloads using Wireshark

This recipe describes how to decrypt Encapsulated Security Payload (ESP) traffic on a FortiGate using the Security Association (SA) information from diag vpn tunnel list. This is useful for tracking whether the FortiGate is properly encrypting/decrypting IPsec VPN packets, and whether there is any packet loss.

This recipe assumes that NPU offloading is disabled on phase1-interface and that NAT is disabled. The example simulates a lost packet in a site-to-site IPsec VPN tunnel.

1. Establishing the tunnel
If the tunnel is currently down, go to Monitor > IPsec Monitor, right-click the tunnel, and select Bring Up.
2. Capturing packets
Go to Network > Packet Capture and create a new entry. Set Interface to the external-facing interface (in this case, wan1). Select Enable Filters and enter Protocol 50 (the protocol number for ESP).
In the Packet Capture list, highlight the new entry and select Start/Resume Capturing to begin capturing packets for the next step.
Ping through the tunnel to populate the packet capture with traffic.
For example, in Windows Command Prompt, enter: `ping x.x.x.x -n 100`, where `x.x.x.x` is the remote tunnel endpoint (`-n 100` will ping 100 times).
In the Packet Capture list on the FortiGate, select the Download option to save the .pcap file to your computer once the packets have been captured.
3. Configuring Wireshark
In Wireshark, open the .pcap file saved previously.
Go to Edit > Preferences and navigate to Protocol > ESP. Check all BUT Attempt to detect/decode NULL encrypted ESP payloads. Select Edit… to open the ESP SAs configuration table.
On the FortiGate, open the CLI Console and enter the command `diag vpn tunnel list`. Make note of the information next to dec: and enc:. You will need the SPI information, as well as the ESP and AH keys for both the remote and local FortiGates.
In Wireshark’s ESP SAs configuration table, add a new entry for each direction of the tunnel.
Note the image in the example: Src IP and Dest IP refer to the gateway addresses. The SPI information in the diag output will help you determine which encryption and authentication keys to use for each direction. Note that `0x` must be prepended to the SPI entries as well as each of the Encyrption and Authentication Keys. Click OK when you are done.
4. Results
In this example, a missing packet is identified in the packet capture by the ICMP error “No response seen to ICMP request“.
Shown here is a packet capture without any errors.