ADVPN场景

网络拓扑与需求

image-20231226183400201

  1. 使用IPsec VPN建立动态的多隧道,Hub和Spoke之间是固定的IPSec隧道,而Spoke和Spoke之间则是流量触发的按需动态创建的VPN隧道。
  2. 使用动态路由协议BGP进行路由的分发和学习,HUB充当BGP路由反射器进行路由的传递,同时分支需要开启“additional-path”,形成VPN业务路由的负载分担BGP路由,这是SD-WAN可以调度的前提(路由表中有路由)。
  3. 使用SD-WAN规则对流量进行分配和调度,可以配置特定的SD-WAN算法进行流量的调度。

配置步骤

基础配置

Hub

  1. 在Hub设备上配置WAN1(port2)、WAN2(port3)以及LAN(port4)的接口IP(略)。

  2. 配置两个WAN接口上访问Internet的默认路由。

    image-20231225144236699

    config router static
        edit 1
            set gateway 101.103.254.1
            set device "port2"
        next
        edit 2
            set gateway 202.103.254.1
            set device "port3"
        next
    end
    
    Hub # get router info routing-table all
    Routing table for VRF=0
    S*      0.0.0.0/0 [10/0] via 101.103.254.1, port2, [1/0]
                      [10/0] via 202.103.254.1, port3, [1/0]
    C       10.10.254.0/24 is directly connected, port4
    C       101.103.254.0/24 is directly connected, port2
    C       202.103.254.0/24 is directly connected, port3
    

Spoke

  1. 配置每个Spoke FortiGate运营商线路的IP(如果是PPPoE线路,PPPoE请通过CLI配置,参考部署前注意事项→PPPoE接口配置章节)与内网口IP(略)。
  2. 在每个Spoke配置WAN接口上访问Internet的默认路由(略,参考Hub上的配置)。

IPSec相关配置

Hub

  1. 通过ADVPN向导来配置ADVPN,进入“VPN→IPSec向导”页面,模板类型选择“Hub-and-Spoke”类型,角色选择Hub,点击下一步。

    通过ADVPN向导来配置ADVPN可以快速地部署ADVPN+BGP环境,而不用在每个Spoke上进行相关的具体功能配置,大大增加了部署效率与难度,部署完成后,可以再根据实际需求对配置进行调整。

    image-20231225151309474

  2. 配置WAN1(port2)接口上的IPSec拨号连接,流入接口选择WAN1(port2),配置认证方法,这里以预共享密钥为例,填入预共享密钥,点击下一步。

    image-20231225145837667

  3. 配置Hub端WAN1(port2)上的IPSec Tunnel IP,用于BGP邻居建立,配置“远程IP地址/掩码”为一个24位网络地址(根据Spoke的数量判断掩码位数),点击下一步。

    image-20231225150206676

  4. 配置BGP路由的本地AS号(此例中Hub和所有的Spoke之间为IBGP邻居),Hub端内网接口与网段(用于自动创建安全策略和BGP路由发布),Spoke type选择“范围”,填写Spoke prefix范围为WAN1线路IPSec对应的Spoke Tunnel接口IP的范围子网,点击Spoke邻居组中的新建按钮。

    Spoke type推荐使用“范围”模式,并结合BGP邻居范围和BGP邻居组使用,否则如果Spoke较多,每个Spoke配置一个BGP邻居将是一个巨大的工作量。

    image-20231225163320190

  5. 在弹出的BGP邻居组窗口中,配置远程AS(此例中Hub和所有的Spoke之间为IBGP邻居),开启路由反射器和软重配。点击确认按钮下发BGP邻居组配置。

    image-20231225163859319

  6. Spoke邻居组选择上步创建的BGP邻居组,点击下一步。

    image-20231225164036970

  7. 查看即将创建的配置内容,点击完成按钮。

    image-20231225151519521

  8. 检查对象摘要中的配置是否全部下发成功。然后点击下方的“生成简易配置密钥”按钮,在右侧弹出的窗口中,添加WAN1(port2)线路IPSec隧道对应的三个Spoke的Tunnel接口IP。然后点击“生成简易配置密钥”按钮,将生成的3个Spoke简易配置密钥复制出来并保存。

    image-20231225164324327

  9. 配置WAN2(port3)接口上的IPSec拨号连接,流入接口选择WAN2(port3),配置认证方法,这里以预共享密钥为例,填入预共享密钥,点击下一步。

    image-20231225152045300

  10. 配置Hub端WAN2(port3)上的IPSec Tunnel IP,用于BGP邻居建立,配置“远程IP地址/掩码”为一个24位网络地址(根据Spoke的数量判断掩码位数),点击下一步。

    image-20231225152213735

  11. 配置BGP路由的本地AS号(此例中Hub和所有的Spoke之间为IBGP邻居),Hub端内网接口与网段(用于自动创建安全策略和BGP路由发布),Spoke type选择“范围”,填写Spoke prefix范围为WAN1线路IPSec对应的Spoke Tunnel接口IP的范围子网,点击Spoke邻居组中的新建按钮。

    Spoke type推荐使用“范围”模式,并结合BGP邻居范围和BGP邻居组使用,否则如果Spoke较多,每个Spoke配置一个BGP邻居将是一个巨大的工作量。

    image-20231225165222128

  12. 在弹出的BGP邻居组窗口中,配置远程AS(此例中Hub和所有的Spoke之间为IBGP邻居),开启路由反射器和软重配。点击确认按钮下发BGP邻居组配置。

    image-20231225165420216

  13. Spoke邻居组选择上步创建的BGP邻居组,点击下一步。

    image-20231225165502315

  14. 查看即将创建的配置内容,点击完成按钮。

    image-20231225165522127

  15. 检查对象摘要中的配置是否全部下发成功。然后点击下方的“生成简易配置密钥”按钮,在右侧弹出的窗口中,添加WAN1(port2)线路IPSec隧道对应的三个Spoke的Tunnel接口IP。然后点击“生成简易配置密钥”按钮,将生成的3个Spoke简易配置密钥复制出来并保存。

    image-20231225165635883

  16. 查看IPSec向导下发的IPSec配置,以及IPSec Tunnel接口配置。

    config vpn ipsec phase1-interface
        edit "Hub_WAN1"
            set type dynamic
            set interface "port2"
            set peertype any
            set net-device disable
            set proposal aes128-sha256 aes256-sha256 aes128-sha1 aes256-sha1
            set add-route disable
            set dpd on-idle
            set comments "VPN: Hub_WAN1 (Created by VPN wizard)"
            set wizard-type hub-fortigate-auto-discovery
            set auto-discovery-sender enable    //确认开启了ADVPN sender//
            set psksecret ENC LRPVkrjrtBp5qTlA1Bo5E3M0Cr/79r/kqV7vZmqrNqckeTk2kJf2/W5Xi2lni2CPCfSukYx0CY9cyWwu6C2IuwxiK4chuJXZbuhYejUguiHmZE=
        next
        edit "Hub_WAN2"
            set type dynamic
            set interface "port3"
            set peertype any
            set net-device disable
            set proposal aes128-sha256 aes256-sha256 aes128-sha1 aes256-sha1
            set add-route disable
            set dpd on-idle
            set comments "VPN: Hub_WAN2 (Created by VPN wizard)"
            set wizard-type hub-fortigate-auto-discovery
            set auto-discovery-sender enable    //确认开启了ADVPN sender//
            set psksecret ENC g3UIj62MLpQkr791Zf+pujPQ2QqbzDnvcwLDmF5xr5apZLk8FDRs+YtDJGtJ5vnMEG6K2lEhluoO/M5gjGu5qjJatbTRvblp0FPBUQGW2X2eFY=
        next
    end
    
    config vpn ipsec phase2-interface
        edit "Hub_WAN1"
            set phase1name "Hub_WAN1"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
            set comments "VPN: Hub_WAN1 (Created by VPN wizard)"
        next
        edit "Hub_WAN2"
            set phase1name "Hub_WAN2"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
            set comments "VPN: Hub_WAN2 (Created by VPN wizard)"
        next
    end
    
    config system interface
        edit "Hub_WAN1"
            set vdom "root"
            set ip 100.1.1.254 255.255.255.255
            set type tunnel
            set remote-ip 100.1.1.253 255.255.255.0
            set interface "port2"
        next
        edit "Hub_WAN2"
            set vdom "root"
            set ip 200.1.1.254 255.255.255.255
            set type tunnel
            set remote-ip 200.1.1.253 255.255.255.0
            set interface "port3"
        next
    end
    
  17. 查看IPSec向导下发的地址、地址组和安全策略配置,包含放通Hub端IPSec隧道的入向流量、放通IPSec隧道的中转流量,可根据实际需求进行调整或新增(如Hub向Spoke主动发起的方向策略)。

    后续如果需要在Hub上配置SD-WAN,安全策略配置需要删除,否则IPSec Tunnel接口无法加入SD-WAN区域。
    config firewall address
        edit "Hub_WAN1_local_subnet_1"
            set allow-routing enable
            set subnet 10.10.254.0 255.255.255.0
        next
        edit "Hub_WAN2_local_subnet_1"
            set allow-routing enable
            set subnet 10.10.254.0 255.255.255.0
        next
    end
    config firewall addrgrp
        edit "Hub_WAN1_local"
            set member "Hub_WAN1_local_subnet_1"
            set comment "VPN: Hub_WAN1 (Created by VPN wizard)"
            set allow-routing enable
        next
        edit "Hub_WAN2_local"
            set member "Hub_WAN2_local_subnet_1"
            set comment "VPN: Hub_WAN2 (Created by VPN wizard)"
            set allow-routing enable
        next
    end
    
    config firewall policy
        edit 1    //放通了WAN1上IPSec隧道的入向流量//
            set name "vpn_Hub_WAN1_spoke2hub_0"
            set srcintf "Hub_WAN1"
            set dstintf "port4"
            set action accept
            set srcaddr "all"
            set dstaddr "Hub_WAN1_local"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Hub_WAN1 (Created by VPN wizard)"
        next
        edit 2    //放通了WAN1上IPSec隧道的中转流量//
            set name "vpn_Hub_WAN1_spoke2spoke_0"
            set srcintf "Hub_WAN1"
            set dstintf "Hub_WAN1"
            set action accept
            set srcaddr "all"
            set dstaddr "all"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Hub_WAN1 (Created by VPN wizard)"
        next
        edit 3    //放通了WAN2上IPSec隧道的入向流量//
            set name "vpn_Hub_WAN2_spoke2hub_0"
            set srcintf "Hub_WAN2"
            set dstintf "port4"
            set action accept
            set srcaddr "all"
            set dstaddr "Hub_WAN2_local"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Hub_WAN2 (Created by VPN wizard)"
        next
        edit 4    //放通了WAN2上IPSec隧道的中转流量//
            set name "vpn_Hub_WAN2_spoke2spoke_0"
            set srcintf "Hub_WAN2"
            set dstintf "Hub_WAN2"
            set action accept
            set srcaddr "all"
            set dstaddr "all"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Hub_WAN2 (Created by VPN wizard)"
        next
    end
    
  18. 查看IPSec向导下发的BGP配置。

    config router bgp
        set as 10086
        config neighbor-group
            edit "BGP_NBR_GRP_WAN1"
                set soft-reconfiguration enable
                set remote-as 10086
                set route-reflector-client enable
            next
            edit "BGP_NBR_GRP_WAN2"
                set soft-reconfiguration enable
                set remote-as 10086
                set route-reflector-client enable
            next
        end
        config neighbor-range
            edit 1
                set prefix 100.1.1.0 255.255.255.0
                set neighbor-group "BGP_NBR_GRP_WAN1"
            next
            edit 2
                set prefix 200.1.1.0 255.255.255.0
                set neighbor-group "BGP_NBR_GRP_WAN2"
            next
        end
        config network
            edit 1
                set prefix 10.10.254.0 255.255.255.0
            next
        end
    end
    
  19. 在ADVPN环境下,以上BGP配置还需要手动增加如下参数。

    config router bgp
        set ibgp-multipath enable    //必须配置,否则去往Spoke的路由无法负载//
        set additional-path enable    //多线路ADVPN,Hub端必须配置,否则Spoke学到的路由可能不全//
        set additional-path-select 4
        config neighbor-group
            edit "BGP_NBR_GRP_WAN1"
                set link-down-failover enable
                set additional-path send    //多线路ADVPN,Hub端必须配置,否则Spoke学到的路由可能不全//
                set adv-additional-path 4    //配置从几条额外路径上去发送路由//
                set soft-reconfiguration enable
            next
            edit "BGP_NBR_GRP_WAN2"
                set link-down-failover enable
                set additional-path send    //多线路ADVPN,Hub端必须配置,否则Spoke学到的路由可能不全//
                set adv-additional-path 4    //配置从几条额外路径上去发送路由//
                set soft-reconfiguration enable
            next
        end
    end
    

Spoke

  1. Hub端通过向导配置ADVPN后,在Spoke端的配置就轻松很多。准备好在Hub端复制出的Spoke简易配置密钥。

  2. 在Spoke1 FortiGate上进入VPN→IPSec隧道页面,新建WAN1(port2)上的IPSec隧道,选择“Hub-and-Spoke”模式,角色选择Spoke,将Spoke1 WAN1的IPSec隧道对应的简易配置密钥粘贴进去,然后点击右侧的应用按钮,会弹出提示“配置已应用”,然后点击下一步。

    image-20231225172640885

  3. Spoke1自动将Hub端WAN1(port2)的IP地址填入,配置流出接口为WAN1(port2),配置预共享密钥,点击下一步。

    image-20231225172754912

  4. Spoke1自动配置了WAN1(port2)线路IPSec Tunnel口的地址(在Hub端向导中的配置),远程IP地址/掩码为Hub端WAN1线路IPSec Tunnel接口地址。点击下一步。

    image-20231225172947565

  5. 配置BGP本地AS(此例中Hub和所有的Spoke之间为IBGP邻居),配置本地接口和本地子网(用于自动创建安全策略和BGP路由发布)。点击下一步。

    image-20231225173212581

  6. 查看即将下发的相关配置,点击完成按钮。

    image-20231225173423144

  7. 检查相关配置均下发成功后,点击“添加另一个”按钮。

    image-20231225173455533

  8. 新建WAN2(port3)上的IPSec隧道,选择“Hub-and-Spoke”模式,角色选择Spoke,将Spoke1 WAN2的IPSec隧道对应的简易配置密钥粘贴进去,然后点击右侧的应用按钮,会弹出提示“配置已应用”,然后点击下一步。

    image-20231225173817728

  9. Spoke1自动将Hub端WAN2(port3)的IP地址填入,配置流出接口为WAN2(port3),配置预共享密钥,点击下一步。

    image-20231225174037762

  10. Spoke1自动配置了WAN2(port3)线路IPSec Tunnel口的地址(在Hub端向导中的配置),远程IP地址/掩码为Hub端WAN2线路IPSec Tunnel接口地址。点击下一步。

    image-20231225174115022

  11. 配置BGP本地AS(此例中Hub和所有的Spoke之间为IBGP邻居),配置本地接口和本地子网(用于自动创建安全策略和BGP路由发布)。点击下一步。

    image-20231225174205055

  12. 查看即将下发的相关配置,点击完成按钮。

    image-20231225174224188

  13. 检查相关配置均下发成功。

    image-20231225174255847

  14. 在Spoke1上查看向导下发的IPSec配置以及IPSec Tunnel接口配置。

    config vpn ipsec phase1-interface
        edit "Spoke1_WAN1"
            set interface "port2"
            set peertype any
            set net-device enable    //确保Spoke的net-device开启,否则使用SD-WAN时,Shortcut的健康检查无法正常工作//
            set proposal aes128-sha256 aes256-sha256 aes128-sha1 aes256-sha1
            set add-route disable
            set dpd on-idle
            set comments "VPN: Spoke1_WAN1 (Created by VPN wizard)"
            set wizard-type spoke-fortigate-auto-discovery
            set auto-discovery-receiver enable    //确认开启了ADVPN receiver//
            set remote-gw 101.103.254.2
            set psksecret ENC jZJjdqGe4N3alz21hgoMkyE1t5HDNMSanAQpDEtCwitgjX2hvApP24ZC3axQP4f9qdqT/oeVmj/5bWBXt1drV/rrwPOHteGqFhMmDuRGpHK7qA=
        next
        edit "Spoke1_WAN2"
            set interface "port3"
            set peertype any
            set net-device enable    //确保Spoke的net-device开启,否则使用SD-WAN时,Shortcut的健康检查无法正常工作//
            set proposal aes128-sha256 aes256-sha256 aes128-sha1 aes256-sha1
            set add-route disable
            set dpd on-idle
            set comments "VPN: Spoke1_WAN2 (Created by VPN wizard)"
            set wizard-type spoke-fortigate-auto-discovery
            set auto-discovery-receiver enable    //确认开启了ADVPN receiver//
            set remote-gw 202.103.254.2
            set psksecret ENC uH/eLzEpwUxUfKKcFdZYZ9q/bNsDSKCal2iF7LuF978SppZrpdHQi2BpWdTcSlQaDWZ6QaeyeXn+8crQLCavpKv7fa9XhyI+U/OLf7J0LSHc/n=
        next
    end
    
    config vpn ipsec phase2-interface
        edit "Spoke1_WAN1"
            set phase1name "Spoke1_WAN1"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
            set comments "VPN: Spoke1_WAN1 (Created by VPN wizard)"
        next
        edit "Spoke1_WAN2"
            set phase1name "Spoke1_WAN2"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
            set comments "VPN: Spoke1_WAN2 (Created by VPN wizard)"
        next
    end
    
    config system interface
        edit "Spoke1_WAN1"
            set vdom "root"
            set ip 100.1.1.1 255.255.255.255
            set type tunnel
            set remote-ip 100.1.1.254 255.255.255.0
            set interface "port2"
        next
        edit "Spoke1_WAN2"
            set vdom "root"
            set ip 200.1.1.1 255.255.255.255
            set type tunnel
            set remote-ip 200.1.1.254 255.255.255.0
            set interface "port3"
        next
    end
    
  15. 在SD-WAN(配置了健康检查) + ADVPN环境下,以上IPSec Tunnel接口还需要手动增加允许Ping的配置,用于SD-WAN在Spoke之间的Shortcut隧道自动创建的健康检查。

    SD-WAN在Shortcut间自动创建的健康检查为Ping模式,即使配置的健去往Hub的健康检查为其他模式。
    config system interface
        edit "Spoke1_WAN1"
            append allowaccess ping
        next
        edit "Spoke1_WAN2"
            append allowaccess ping
        next
    end
    
  16. 查看Spoke1上IPSec向导下发的地址、地址组和安全策略配置,包含放通Spoke IPSec隧道的双向流量的策略。

    后续要在Spoke配置SD-WAN,这些安全策略配置需要删除,否则IPSec接口无法加入SD-WAN区域。
    config firewall address
        edit "Spoke1_WAN1_local_subnet_1"
            set allow-routing enable
            set subnet 10.10.1.0 255.255.255.0
        next
        edit "Spoke1_WAN2_local_subnet_1"
            set allow-routing enable
            set subnet 10.10.1.0 255.255.255.0
        next
    end
    
    config firewall addrgrp
        edit "Spoke1_WAN1_local"
            set member "Spoke1_WAN1_local_subnet_1"
            set comment "VPN: Spoke1_WAN1 (Created by VPN wizard)"
            set allow-routing enable
        next
        edit "Spoke1_WAN2_local"
            set member "Spoke1_WAN2_local_subnet_1"
            set comment "VPN: Spoke1_WAN2 (Created by VPN wizard)"
            set allow-routing enable
        next
    end
    
    config firewall policy
        edit 1
            set name "vpn_Spoke1_WAN1_remote_0"
            set srcintf "Spoke1_WAN1"
            set dstintf "port4"
            set action accept
            set srcaddr "all"
            set dstaddr "Spoke1_WAN1_local"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Spoke1_WAN1 (Created by VPN wizard)"
        next
        edit 2
            set name "vpn_Spoke1_WAN1_local_0"
            set srcintf "port4"
            set dstintf "Spoke1_WAN1"
            set action accept
            set srcaddr "all"
            set dstaddr "all"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Spoke1_WAN1 (Created by VPN wizard)"
        next 
        edit 3
            set name "vpn_Spoke1_WAN2_remote_0"
            set srcintf "Spoke1_WAN2"
            set dstintf "port4"
            set action accept
            set srcaddr "all"
            set dstaddr "Spoke1_WAN2_local"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Spoke1_WAN2 (Created by VPN wizard)"
        next
        edit 4
            set name "vpn_Spoke1_WAN2_local_0"
            set srcintf "port4"
            set dstintf "Spoke1_WAN2"
            set action accept
            set srcaddr "all"
            set dstaddr "all"
            set schedule "always"
            set service "ALL"
            set comments "VPN: Spoke1_WAN2 (Created by VPN wizard)"
        next 
    end
    
  17. 查看Spoke1上IPSec向导下发的BGP配置。

    config router bgp
        set as 10086
        config neighbor
            edit "100.1.1.254"
                set remote-as 10086
            next
            edit "200.1.1.254"
                set remote-as 10086
            next
        end
        config network
            edit 1
                set prefix 10.10.1.0 255.255.255.0
            next
        end
    end
    
  18. 在ADVPN环境下,以上BGP配置还需要手动增加如下参数。

    config router bgp
        set ibgp-multipath enable    //必须配置,否则去往Hub或其他Spoke的路由无法负载//
        set additional-path enable     //多线路ADVPN,Hub端必须配置,否则Spoke学到的路由可能不全//
        set additional-path-select 4
        config neighbor-group
            edit "BGP_NBR_GRP_WAN1"
                set additional-path receive    //多线路ADVPN,Spoke端必须配置,否则Spoke学到的路由可能不全//
                set soft-reconfiguration enable
            next
            edit "BGP_NBR_GRP_WAN2"
                set additional-path receive    //多线路ADVPN,Spoke端必须配置,否则Spoke学到的路由可能不全//
                set soft-reconfiguration enable
            next
        end
    end
    
  19. Spoke2与Spoke3的配置参考Spoke1,这里就不再赘述(注意Spoke3只有一条Internet线路,与Hub的两条线路建立IPSec)。

SD-WAN基础配置

Spoke

  1. 在Spoke1上删除IPSec向导自动创建的安全策略(否则IPSec接口无法加入SD-WAN区域)。

  2. 进入分支FortiGate的网络→SD-WAN→SD-WAN区域页面,新建用于ADVPN访问的区域“Spoke1_ADVPN”。

    image-20231214174022211

    image-20231226102243844

    config system sdwan
        set status enable
        config zone
            edit "Spoke1_ADVPN"
            next
        end
    end
    
  3. 新建用于访问Hub内网和其他Spoke的SD-WAN成员(两个IPSec Tunnel接口)加入步骤2创建的SD-WAN区域“Spoke1_ADVPN”。注意IPSec Tunnel接口加入SD-WAN区域时不需要填写网关。

    如果IPSec口存在相关配置引用的情况下,是无法加入到SD-WAN接口成员的,需要将WAN接口的相关配置删除引用,包括策略、Link-monitor、所属区域等。

    image-20231226102357474

    image-20231226102909206

    image-20231226103007824

    config system sdwan
        config members
            edit 1
                set interface "Spoke1_WAN1"
                set zone "Spoke1_ADVPN"
            next
            edit 2
                set interface "Spoke1_WAN2"
                set zone "Spoke1_ADVPN"
            next
        end
    end
    
  4. 配置防火墙策略1,放通内网port4(LAN)到SD-WAN ADVPN区域的流量,通常情况下内网之间的VPN访问不需要开启SNAT。根据需求开启UTM功能。

    image-20231226103402676

    config firewall policy
        edit 1
            set name "to_Spoke1_ADVPN"
            set srcintf "port4"
            set dstintf "Spoke1_ADVPN"
            set action accept
            set srcaddr "Spoke1_WAN1_local_subnet_1"
            set dstaddr "all"
            set schedule "always"
            set service "ALL"
        next
    end
    
  5. 右键点击防火墙策略1,选择反向复制,生成策略2。放通SD-WAN ADVPN区域到内网port4(LAN)的流量,通常情况下内网之间的VPN访问不需要开启SNAT。根据需求开启UTM功能。反向复制的策略名称为空,且为禁用状态,需要手动设置策略名称,然后手动启用该策略。

    image-20231226103719118

    image-20231226103853625

    config firewall policy
        edit 2
            set name "from_Spoke1_ADVPN"
            set srcintf "Spoke1_ADVPN"
            set dstintf "port4"
            set action accept
            set srcaddr "all"
            set dstaddr "Spoke1_WAN1_local_subnet_1"
            set schedule "always"
            set service "ALL"
            set comments " (Copy of to_Spoke1_ADVPN) (Reverse of to_Spoke1_ADVPN)"
        next
    end
    

    image-20231226104114192

  6. Spoke2和Spoke3的配置参考Spoke1,这里就不再赘述。

Hub

Hub可以不配置SD-WAN。如果不配置SD-WAN选路,可以通过静态路由不同的优先级进行选路。这里我们以Hub端配置SD-WAN为例。
  1. 删除IPSec向导生成的所有安全策略(否则IPSec接口无法加入SD-WAN区域)。

  2. 在Hub上新建SD-WAN区域“Hub_ADVPN”与SD-WAN成员(包含WAN1和WAN2上的IPSec拨号连接),将两个IPSec接口加入SD-WAN区域。

    如果IPSec口存在相关配置引用的情况下,是无法加入到SD-WAN接口成员的,需要将WAN接口的相关配置删除引用,包括策略、Link-monitor、所属区域等。

    image-20231226105821861

    config system sdwan
        set status enable
        config zone
            edit "Hub_ADVPN"
            next
        end
        config members
            edit 1
                set interface "Hub_WAN1"
                set zone "Hub_ADVPN"
            next
            edit 2
                set interface "Hub_WAN2"
                set zone "Hub_ADVPN"
            next
        end
    end
    
  3. 配置防火墙策略1和防火墙策略2,放通内网port4(LAN)与所有Spoke内网之间互相访问的流量,通常情况下内网之间的VPN访问不需要开启SNAT。根据需求开启UTM功能。

    image-20231226110722532

    config firewall policy
        edit 1
            set name "to_Hub_ADVPN"
            set srcintf "port4"
            set dstintf "Hub_ADVPN"
            set action accept
            set srcaddr "Hub_WAN1_local_subnet_1"
            set dstaddr "all"
            set schedule "always"
            set service "ALL"
        next
        edit 2
            set name "from_Hub_ADVPN"
            set srcintf "Hub_ADVPN"
            set dstintf "port4"
            set action accept
            set srcaddr "all"
            set dstaddr "Hub_WAN1_local_subnet_1"
            set schedule "always"
            set service "ALL"
            set comments " (Copy of to_Hub_ADVPN) (Reverse of to_Hub_ADVPN)"
        next
    end
    
  4. 配置防火墙策略3,放通通过Hub中转的Spoke之间的流量。通常情况下VPN之间的VPN访问不需要开启SNAT。根据需求开启UTM功能。

    image-20231226170619819

    config firewall policy
        edit 3
            set name "Spoke_to_Spoke"
            set srcintf "Hub_ADVPN"
            set dstintf "Hub_ADVPN"
            set action accept
            set srcaddr "Spoke1_10.10.1.0/24" "Spoke2_10.10.2.0/24" "Spoke3_10.10.3.0/24"
            set dstaddr "Spoke1_10.10.1.0/24" "Spoke2_10.10.2.0/24" "Spoke3_10.10.3.0/24"
            set schedule "always"
            set service "ALL"
        next
    end
    

    image-20231226170713710

状态检查

  1. 查看Hub端IPSec隧道的建立状态,所有Spoke的线路与Hub的两条线路的IPSec建立成功(3个Spoke共6条隧道)。

    image-20231226112526254

    Hub # get vpn ipsec tunnel summary 
    'Hub_WAN1_0' 101.103.2.2:0  selectors(total,up): 1/1  rx(pkt,err): 84/0  tx(pkt,err): 83/0
    'Hub_WAN1_1' 101.103.3.2:0  selectors(total,up): 1/1  rx(pkt,err): 20/0  tx(pkt,err): 27/0
    'Hub_WAN1_2' 101.103.1.2:0  selectors(total,up): 1/1  rx(pkt,err): 84/0  tx(pkt,err): 83/0
    'Hub_WAN2_0' 202.103.2.2:0  selectors(total,up): 1/1  rx(pkt,err): 84/0  tx(pkt,err): 83/1
    'Hub_WAN2_1' 202.103.1.2:0  selectors(total,up): 1/1  rx(pkt,err): 84/0  tx(pkt,err): 83/1
    'Hub_WAN2_2' 101.103.3.2:0  selectors(total,up): 1/1  rx(pkt,err): 20/0  tx(pkt,err): 27/0
    
  2. 查看Hub端FortiGate路由表中的BGP路由,Hub去往每个Spoke的路由在两条线路的IPSec隧道上负载。

    Hub # get router info routing-table bgp
    Routing table for VRF=0
    B       10.10.1.0/24 [200/0] via 100.1.1.1 (recursive is directly connected, Hub_WAN1), 00:28:29, [1/0]
                         [200/0] via 200.1.1.1 (recursive is directly connected, Hub_WAN2), 00:28:29, [1/0]
    B       10.10.2.0/24 [200/0] via 100.1.1.2 (recursive is directly connected, Hub_WAN1), 00:28:32, [1/0]
                         [200/0] via 200.1.1.2 (recursive is directly connected, Hub_WAN2), 00:28:32, [1/0]
    B       10.10.3.0/24 [200/0] via 100.1.1.3 (recursive is directly connected, Hub_WAN1), 00:05:24, [1/0]
                         [200/0] via 200.1.1.3 (recursive is directly connected, Hub_WAN2), 00:05:24, [1/0]
    
  3. 查看Spoke1 FortiGate路由表中的BGP路由:

    • 去往Hub内网的路由在两个线路的IPSec隧道上负载;
    • 去往其他Spoke内网的路由通过Hub端的BGP路由反射器反射后学习,通过两个IPSec隧道负载,出接口为去往Hub的IPSec隧道接口(由于Hub和Spoke开启了BGP additional-path send/receive,且Hub端BGP在两个IPSec隧道上都开启了路由反射,Spoke1每条隧道学习另一个Spoke内网的路由会有2条相同的路由)。
    Spoke1 # get router info routing-table bgp
    Routing table for VRF=0
    B       10.10.2.0/24 [200/0] via 100.1.1.2 [2] (recursive via Spoke1_WAN1 tunnel 101.103.254.2), 00:29:51, [1/0]
                         [200/0] via 200.1.1.2 [2] (recursive via Spoke1_WAN2 tunnel 202.103.254.2), 00:29:51, [1/0]
    B       10.10.3.0/24 [200/0] via 100.1.1.3 [2] (recursive via Spoke1_WAN1 tunnel 101.103.254.2), 00:06:55, [1/0]
                         [200/0] via 200.1.1.3 [2] (recursive via Spoke1_WAN2 tunnel 202.103.254.2), 00:06:55, [1/0]
    B       10.10.254.0/24 [200/0] via 100.1.1.254 (recursive via Spoke1_WAN1 tunnel 101.103.254.2), 00:30:21, [1/0]
                           [200/0] via 200.1.1.254 (recursive via Spoke1_WAN2 tunnel 202.103.254.2), 00:30:21, [1/0]
    
  4. Spoke2、Spoke3的路由表与Spoke1类似,这里不再赘述。

SD-WAN策略配置

Spoke

需求1:Spoke去往Hub内网的流量优先从WAN1(port2)的IPSec隧道转发,WAN1的IPSec隧道中断或不满足SLA标准后,从WAN2(port3)的IPSec隧道转发。

  1. 在Spoke1的SD-WAN中配置探测Hub端内网服务器的健康检查:

    • 成员选择WAN1的IPSec Tunnel接口和WAN2的IPSec Tunnel接口。
    • 探测模式为Ping。
    • 开启SLA目标:延迟<250ms/抖动<50ms/丢包率<5%。

    image-20231226143805269

    config system sdwan
        config health-check
            edit "HC_to_Hub_Ping"
                set server "10.10.254.100"
                set interval 1000
                set recoverytime 10
                set members 0
                config sla
                    edit 1
                        set latency-threshold 250
                        set jitter-threshold 50
                        set packetloss-threshold 5
                    next
                end
            next
        end
    end
    
  2. 查看健康检查的状态,可以看到目前2条去往Hub内网的线路都满足SLA标准。

    image-20231226145035046

    Spoke1 # diagnose sys sdwan health-check status HC_to_Hub_Ping
    Health Check(HC_to_Hub_Ping): 
    Seq(1 Spoke1_WAN1): state(alive), packet-loss(0.000%) latency(48.060), jitter(20.974), mos(4.356), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x1
    Seq(2 Spoke1_WAN2): state(alive), packet-loss(0.000%) latency(106.973), jitter(30.848), mos(4.268), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x1
    
  3. 新建SD-WAN规则1,用于Spoke内网去往Hub内网的流量:

    image-20231215153625634

    • 源地址选择Spoke1内网网段;目标地址为Hub内网网段。
    • 接口选择策略:选择Lowest Cost (SLA),Spoke内部访问Hub内网的流量优先从满足SLA标准(延迟<250ms/抖动<50ms/丢包率<5%)的线路转发。如果2条线路都满足SLA标准,则优先从接口偏好中靠前的接口转发。
    • 接口偏好:选择WAN1线路IPSec接口与WAN2线路IPSec接口。WAN1线路IPSec接口排序在前,其次是WAN2线路IPSec接口。
    • 需要的SLA目标:选择上步创建的SLA目标(延迟<250ms/抖动<50ms/丢包率<5%)。

    image-20231226150153711

    config system sdwan
        config service
            edit 1
                set name "to_Hub"
                set mode sla
                set dst "Hub_10.10.254.0/24"
                set src "Spoke1_WAN1_local_subnet_1"
                config sla
                    edit "HC_to_Hub_Ping"
                        set id 1
                    next
                end
                set priority-members 1 2
            next
        end
    end
    
  4. 查看上步创建的SD-WAN规则目前的选路状态,由于目前2条去往Hub内网的线路均满足SLA目标,所以选择接口偏好配置中靠前的WAN1上的IPSec线路。

    image-20231226150539009

    Spoke1 #  diagnose sys sdwan service 1
    
    Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: cfg
      Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Members(2): 
        1: Seq_num(1 Spoke1_WAN1), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected    //优先从WAN1上的IPSec隧道转发//
        2: Seq_num(2 Spoke1_WAN2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
      Src address(1): 
            10.10.1.0-10.10.1.255
    
      Dst address(1): 
            10.10.254.0-10.10.254.255
    
  5. 查看SD-WAN规则对应的策略路由状态,出接口优先选择MPLS线路接口(port4)。

    Spoke1 # diagnose firewall proute list
    list route policy info(vf=root):
    
    id=2130837505(0x7f020001) vwl_service=1(to_Hub) vwl_mbr_seq=1 2 dscp_tag=0xfc 0xfc flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0-65535 iif=0(any) dport=1-65535 path(2) oif=17(Spoke1_WAN1) oif=18(Spoke1_WAN2)    //优先从WAN1上的IPSec隧道转发//
    source(1): 10.10.1.0-10.10.1.255 
    destination(1): 10.10.254.0-10.10.254.255 
    hit_count=0 last_used=2023-12-26 15:03:45
    
  6. Spoke2与Spoke3的配置请参考Spoke1,这里不再赘述。

需求2:Spoke内网访问其他Spoke内网的流量优先从Shortcut隧道转发。Shortcut隧道无法建立时,通过Hub中转。

  1. 新建SD-WAN规则2:

    • 源地址选择Spoke1内网网段;目标地址为all(或其他Spoke内网的网段集合)。
    • 接口选择策略:选择Lowest Cost (SLA),Spoke内部访问Internet的流量优先从满足SLA标准(延迟<250ms/抖动<50ms/丢包率<5%)的IPSec线路创建的Shortcut隧道转发(如果Shortcut无法建立,则通过去往Hub的IPSec隧道转发)。如果两条IPSec线路都满足SLA标准,则优先从接口偏好中靠前的IPSec隧道创建的Shortcut转发(如果Shortcut无法建立,则通过去往Hub的IPSec隧道转发)。
    • 接口偏好:选择WAN1线路IPSec接口与WAN2线路IPSec接口。WAN1线路IPSec接口排序在前,其次是WAN2线路IPSec接口。
    • 需要SLA目标:选择上步创建的SLA目标(延迟<250ms/抖动<50ms/丢包率<5%)。

    image-20231226152801103

    config system sdwan
        config service
            edit 2
                set name "to_Other_Spoke"
                set mode sla
                set dst "all"
                set src "Spoke1_WAN1_local_subnet_1"
                config sla
                    edit "HC_to_Hub_Ping"
                        set id 1
                    next
                end
                set priority-members 1 2
            next
        end
    end
    
  2. 查看上步创建的SD-WAN规则目前的选路状态,由于目前两条IPSec线路均满足SLA目标,所以选择接口偏好配置中靠前的Spoke1_WAN1隧道。

    image-20231226153645587

    Spoke1 # diagnose sys sdwan service 2
    
    Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: cfg
      Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Members(2): 
        1: Seq_num(1 Spoke1_WAN1), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected    //优先从WAN1上的IPSec隧道转发//
        2: Seq_num(2 Spoke1_WAN2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
      Src address(1): 
            10.10.1.0-10.10.1.255
    
      Dst address(1): 
            0.0.0.0-255.255.255.255
    
  3. Spoke2与Spoke3的配置请参考Spoke1,这里不再赘述。

Hub

需求3:Hub内网主动访问Spoke内网的流量强制从WAN1的IPSec线路转发,当WAN1的IPSec线路中断时,从WAN2的IPSec线路转发。

  1. 新建SD-WAN规则1(可根据需求使用其他模式):

    • 源地址选择Hub内网网段;目标地址为Spoke1内网网段。
    • 接口选择策略:选择Manual,Hub内网访问Spoke1的流量优先从WAN1的IPSec线路转发,当WAN1的IPSec线路中断时,从WAN2的IPSec线路转发。
    • 接口偏好:选择WAN1线路IPSec接口与WAN2线路IPSec接口。WAN1线路IPSec接口排序在前,其次是WAN2线路IPSec接口。

    image-20231226160906939

    config system sdwan
        config service
            edit 1
                set name "to_Spoke1"
                set dst "Spoke1_10.10.1.0/24"
                set src "Hub_WAN1_local_subnet_1"
                set priority-members 1 2
            next
        end
    end
    
  2. 查看上步创建的SD-WAN规则目前的选路状态,选择接口偏好配置中靠前的Spoke1_WAN1隧道。

    image-20231226164910718

    Hub # diagnose sys sdwan service 1
    
    Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: cfg
      Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Members(2): 
        1: Seq_num(1 Hub_WAN1), alive, selected    //Hub访问Spoke1的流量优先从WAN1的IPSec隧道转发//
        2: Seq_num(2 Hub_WAN2), alive, selected
      Src address(1): 
            10.10.254.0-10.10.254.255
    
      Dst address(1): 
            10.10.1.0-10.10.1.255
    
  3. 去往Spoke2和Spoke3的SD-WAN规则可参考步骤1,这里不再赘述。

需求4:当Spoke内网之间的Shortcut隧道无法建立,或Shortcut还未建立Spoke内网之间第一次进行访问时,流量要通过Hub进行中转。Hub中转这类流量时,需要保证流入IPSec隧道和流出IPSec隧道为同一个隧道。

  1. 通过CLI新建SD-WAN规则2:

    • 源地址(src)选择和目标地址(dst)都选择所有Spoke的内网网段地址。
    • 接口选择策略(mode):使用Manual。
    • 接口偏好(priority-members):选择WAN1的IPSec隧道和WAN2的IPSec隧道。
    • tie-break:选择input-device方式,根据入接口来决定SD-WAN策略转发时使用相同的接口作为出接口,接口偏好中配置的接口顺序不会生效(该选项需要7.2.1及以上版本支持,如低于7.2.1版本,请使用策略路由实现此需求)。
    config system sdwan
        config service
            edit 2
                set name "Spoke_to_Spoke"
                set mode manual
                set dst "Spoke1_10.10.1.0/24" "Spoke2_10.10.2.0/24" "Spoke3_10.10.3.0/24"
                set src "Spoke1_10.10.1.0/24" "Spoke2_10.10.2.0/24" "Spoke3_10.10.3.0/24"
                set priority-members 1 2
                set tie-break input-device    //重要配置//
            next
        end
    end
    
  2. 查看上步创建的SD-WAN规则目前的状态,Tie break模式为input-device,下方的Member顺序不会生效。

    Hub # diagnose sys sdwan service 2
    
    Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: input-device    //tie-break模式为input-device时,下方的Member顺序不会生效//
      Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Members(2): 
        1: Seq_num(1 Hub_WAN1), alive, selected
        2: Seq_num(2 Hub_WAN2), alive, selected
      Src address(3): 
            10.10.3.0-10.10.3.255
            10.10.2.0-10.10.2.255
            10.10.1.0-10.10.1.255
    
      Dst address(3): 
            10.10.3.0-10.10.3.255
            10.10.2.0-10.10.2.255
            10.10.1.0-10.10.1.255
    

结果验证

Spoke访问Hub

  1. 使用Spoke1内网PC客户端访问Hub端的内网业务,流量可以正确从WAN1上的IPSec隧道转发。

    image-20231226161604975

  2. 模拟Spoke1 WAN1上IPSec隧道的健康检查的延迟增加到300ms。

    image-20231226161759320

    Spoke1 # diagnose sys sdwan health-check status HC_to_Hub_Ping
    Health Check(HC_to_Hub_Ping): 
    Seq(1 Spoke1_WAN1): state(alive), packet-loss(0.000%) latency(303.576), jitter(19.449), mos(3.595), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x0
    Seq(2 Spoke1_WAN2): state(alive), packet-loss(0.000%) latency(103.569), jitter(34.299), mos(4.258), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x1
    
  3. 在Spoke1上查看SD-WAN规则1的选路优先选择联通线路IPSec接口去往Hub内网。

    image-20231226161917777

    Spoke1 # diagnose sys sdwan service 1
    
    Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: cfg
      Gen(2), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Members(2): 
        1: Seq_num(2 Spoke1_WAN2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected    //去往Hub内网的流量优先从WAN2的IPSec隧道转发//
        2: Seq_num(1 Spoke1_WAN1), alive, sla(0x0), gid(0), cfg_order(0), local cost(0), selected
      Src address(1): 
            10.10.1.0-10.10.1.255
    
      Dst address(1): 
            10.10.254.0-10.10.254.255
    
  4. 对应的SD-WAN线路切换系统日志。

    image-20231226162632224

    date=2023-12-26 time=16:17:30 eventtime=1703578649518771555 tz="+0800" logid="0113022923" type="event" subtype="sdwan" level="notice" vd="root" logdesc="SDWAN status" eventtype="Service" serviceid=1 service="to_Hub" seq="2,1" msg="Service prioritized by SLA will be redirected in sequence order."
    
  5. 再次使用Spoke1内网PC客户端访问Hub端内网的业务,流量开始从WAN2的IPSec接口转发。

    image-20231226162502690

  6. Spoke2、Spoke3内网访问Hub内网的测试参考Spoke1,这里不再赘述。

Spoke访问Spoke

  1. Spoke与Spoke之间互访流量会触发Shortcut隧道的建立(前提是需要Hub开启BGP RR,在Spoke之间反射路由,且Hub开启ADVPN sender,Spoke开启ADVPN receiver),在创建Shortcut隧道之前,第一次Spoke访问其他Spoke的流量会经过Hub进行中转,同时通过Hub进行相关Shortcut信息专递。Shortcut隧道建立后,Spoke与Spoke之间直接通过Shortcut隧道通信。

  2. 保留之前线路的延迟状态(Spoke访问Hub时应优先从WAN2的IPSec转发),查看Spoke1的隧道建立状态,只有去往Hub的IPSec隧道,没有去往其他Spoke的Shortcut隧道。

    Spoke1 # get vpn ipsec tunnel summary 
    'Spoke1_WAN2' 202.103.254.2:0  selectors(total,up): 1/1  rx(pkt,err): 8423/0  tx(pkt,err): 8408/11
    'Spoke1_WAN1' 101.103.254.2:0  selectors(total,up): 1/1  rx(pkt,err): 9284/0  tx(pkt,err): 9193/11
    
  3. 使用Spoke1内网PC访问Spoke2内网PC,触发Shortcut隧道建立,可以看到Shortcut隧道已经建立,名称为“Spoke1_WAN2_0”,通过WAN2上的IPSec隧道建立。

    Spoke1 # get vpn ipsec tunnel summary
    'Spoke1_WAN2' 202.103.254.2:0  selectors(total,up): 1/1  rx(pkt,err): 8825/0  tx(pkt,err): 8810/13
    'Spoke1_WAN2_0' 202.103.2.2:0  selectors(total,up): 1/0  rx(pkt,err): 0/0  tx(pkt,err): 0/2
    'Spoke1_WAN1' 101.103.254.2:0  selectors(total,up): 1/1  rx(pkt,err): 9686/0  tx(pkt,err): 9595/11
    

    image-20231226171003402

    date=2023-12-26 time=17:08:55 eventtime=1703581736206155498 tz="+0800" logid="0101037122" type="event" subtype="vpn" level="notice" vd="root" logdesc="Negotiate IPsec phase 2" msg="negotiate IPsec phase 2" action="negotiate" remip=202.103.2.2 locip=202.103.1.2 remport=500 locport=500 outintf="port3" cookies="07fb4062cc0f70df/8341519d8c69969b" user="202.103.2.2" group="N/A" useralt="N/A" xauthuser="N/A" xauthgroup="N/A" assignip=N/A vpntunnel="Spoke1_WAN2_0" status="success" role="initiator" esptransform="ESP_AES" espauth="HMAC_SHA1" advpnsc=1
    
  4. 在Hub上抓包查看Spoke1内网访问Spoke2内网第一次访问时的流量转发,可以看到在Spoke1和Spoke2之间的Shortcut建立前,流量是通过Hub中转的,Hub中转时匹配SD-WAN规则2,tie-break的方式input-device生效,流量从Hub_WAN2进入,也从Hub_WAN2发出(不会按照Manual模式的SD-WAN Member顺序从Hub_WAN1发出)。

    Hub # diagnose sniffer packet any 'host 10.10.1.100 and host 10.10.2.100' 4
    Using Original Sniffing Mode
    interfaces=[any]
    filters=[host 10.10.1.100 and host 10.10.2.100]
    23.920132 Hub_WAN2 in 10.10.1.100 -> 10.10.2.100: icmp: echo request
    23.920239 Hub_WAN2 out 10.10.1.100 -> 10.10.2.100: icmp: echo request
    23.922899 Hub_WAN2 in 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    23.922913 Hub_WAN2 out 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    
  5. Shortcut隧道建立后,SD-WAN会基于父隧道(Spoke1_WAN2)关联的SLA配置,自动创建健康检查检测Spoke2的Tunnel接口IP,检测方式固定为Ping(这也就是为什么要在所有Spoke的IPSec Tunnel接口上开启允许Ping访问)。

    image-20231226171702525

    Spoke1 # diagnose sys sdwan health-check
    Health Check(HC_to_Hub_Ping): 
    Seq(1 Spoke1_WAN1): state(alive), packet-loss(0.000%) latency(301.470), jitter(20.919), mos(3.591), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x0
    Seq(2 Spoke1_WAN2): state(alive), packet-loss(0.000%) latency(127.110), jitter(22.480), mos(4.259), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x1
    Seq(2 Spoke1_WAN2_0): state(alive), packet-loss(0.000%) latency(112.727), jitter(29.479), mos(4.260), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x1
    
  6. 查看Spoke1上的BGP路由表,去往Spoke2内网路由的出接口已变为在Shortcut隧道接口(Spoke1_WAN2_0)和WAN1上的IPSec隧道接口(Spoke1_WAN1)之间负载。

    Spoke1 # get router info routing-table bgp
    Routing table for VRF=0
    B       10.10.2.0/24 [200/0] via 100.1.1.2 [2] (recursive via Spoke1_WAN1 tunnel 101.103.254.2), 00:19:40, [1/0]
                         [200/0] via 200.1.1.2 [2] (recursive is directly connected, Spoke1_WAN2_0), 00:19:40, [1/0]
    B       10.10.3.0/24 [200/0] via 100.1.1.3 [2] (recursive via Spoke1_WAN1 tunnel 101.103.254.2), 02:33:25, [1/0]
                         [200/0] via 200.1.1.3 [2] (recursive via Spoke1_WAN2 tunnel 202.103.254.2), 02:33:25, [1/0]
    B       10.10.254.0/24 [200/0] via 100.1.1.254 (recursive via Spoke1_WAN1 tunnel 101.103.254.2), 02:35:09, [1/0]
                           [200/0] via 200.1.1.254 (recursive via Spoke1_WAN2 tunnel 202.103.254.2), 02:35:09, [1/0]
    
  7. 查看Spoke1上SD-WAN规则2的选路状态,现在优先从WAN2上IPSec隧道的Shortcut(Spoke1_WAN2_0)进行转发,而不是从父隧道(Spoke1_WAN2)转发,这说明在SLA类型的SD-WAN规则中,Shortcut隧道优先于其父隧道,除非Shortcut隧道的SLA不满足目标。

    Spoke1 # diagnose sys sdwan service 2
    
    Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: cfg
      Gen(2), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Member sub interface(3): 
        2: seq_num(2), interface(Spoke1_WAN2):
           1: Spoke1_WAN2_0(22)
      Members(3): 
        1: Seq_num(2 Spoke1_WAN2_0), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
        2: Seq_num(2 Spoke1_WAN2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
        3: Seq_num(1 Spoke1_WAN1), alive, sla(0x0), gid(0), cfg_order(0), local cost(0), selected
      Src address(1): 
            10.10.1.0-10.10.1.255
    
      Dst address(1): 
            0.0.0.0-255.255.255.255
    
  8. 再次使用Spoke1内网PC访问Spoke2内网PC,在Spoke1上抓包可以看到流量通过WAN2上的Shortcut隧道(Spoke1_WAN2_0)隧道转发。

    Spoke1 # diagnose sniffer packet any 'host 10.10.2.100' 4
    Using Original Sniffing Mode
    interfaces=[any]
    filters=[host 10.10.2.100]
    7.778827 port4 in 10.10.1.100 -> 10.10.2.100: icmp: echo request
    7.778945 Spoke1_WAN2_0 out 10.10.1.100 -> 10.10.2.100: icmp: echo request
    7.870455 Spoke1_WAN2_0 in 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    7.870486 port4 out 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    8.798494 port4 in 10.10.1.100 -> 10.10.2.100: icmp: echo request
    8.798518 Spoke1_WAN2_0 out 10.10.1.100 -> 10.10.2.100: icmp: echo request
    8.870069 Spoke1_WAN2_0 in 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    8.870081 port4 out 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    
  9. 调整Spoke1与Spoke2之间Shortcut隧道(Spoke1_WAN2_0)的延迟到400ms,但父隧道(Spoke1_WAN2)延迟不变。

    image-20231226173921208

    Spoke1 # diagnose sys sdwan health-check
    Health Check(HC_to_Hub_Ping): 
    Seq(1 Spoke1_WAN1): state(alive), packet-loss(1.000%) latency(314.047), jitter(16.431), mos(3.573), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x0
    Seq(2 Spoke1_WAN2): state(alive), packet-loss(0.000%) latency(112.318), jitter(24.059), mos(4.291), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x1
    Seq(2 Spoke1_WAN2_0): state(alive), packet-loss(2.410%) latency(393.256), jitter(26.858), mos(3.071), bandwidth-up(65535000), bandwidth-dw(65535000), bandwidth-bi(131070000) sla_map=0x0
    
  10. 查看Spoke1上SD-WAN规则2的选路状态,现在优先从WAN2上IPSec父隧道(Spoke1_WAN2)进行转发,而不是从Shortcut隧道(Spoke1_WAN2_0)转发,事实的确如此吗?

    Spoke1 # diagnose sys sdwan service 2
    
    Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: cfg
      Gen(15), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Member sub interface(3): 
        1: seq_num(2), interface(Spoke1_WAN2):
           1: Spoke1_WAN2_0(25)
      Members(3): 
        1: Seq_num(2 Spoke1_WAN2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
        2: Seq_num(1 Spoke1_WAN1), alive, sla(0x0), gid(0), cfg_order(0), local cost(0), selected
        3: Seq_num(2 Spoke1_WAN2_0), alive, sla(0x0), gid(0), cfg_order(1), local cost(0), selected
      Src address(1): 
            10.10.1.0-10.10.1.255
    
      Dst address(1): 
            0.0.0.0-255.255.255.255
    
  11. 再次使用Spoke1内网PC访问Spoke2内网PC,在Spoke1上抓包可以看到流量通过WAN1上IPSec隧道(Spoke1_WAN1)隧道转发,而不是Spoke1_WAN2隧道,并没有按照SD-WAN规则状态中的选路从Spoke1_WAN2隧道转发。

    Spoke1 # diagnose sniffer packet any 'host 10.10.2.100' 4
    Using Original Sniffing Mode
    interfaces=[any]
    filters=[host 10.10.2.100]
    4.313351 port4 in 10.10.1.100 -> 10.10.2.100: icmp: echo request
    4.313453 Spoke1_WAN1 out 10.10.1.100 -> 10.10.2.100: icmp: echo request
    4.639634 Spoke1_WAN1 in 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    4.639664 port4 out 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    
  12. 这是由于SD-WAN规则选路时也会根据流量的目的地址查询路由表,如果最佳路由下一跳接口是SD-WAN成员,则通过该SD-WAN规则的SLA找到最佳的接口转发,该接口必须要有路由。在Spoke1的路由表中,通过Spoke1_WAN2隧道去往Spoke2内网的路由出接口已变为Shortcut隧道Spoke1_WAN2_0,而不是Spoke1_WAN2,所以跳过成员Spoke1_WAN2,从下一个成员Spoke1_WAN1转发(参考网络管理→SD-WAN→SD-WAN介绍→SD-WAN数据转发逻辑章节中的SD-WAN工作流程注意事项)。

  13. 随后Spoke1触发了Spoke1_WAN1隧道上的去往Spoke2的Shortcut隧道连接Spoke1_WAN1_0,后续流量通过Shortcut隧道连接Spoke1_WAN1_0转发。

    Spoke1 # diagnose sniffer packet any 'host 10.10.2.100' 4
    Using Original Sniffing Mode
    interfaces=[any]
    filters=[host 10.10.2.100]
    36.862757 port4 in 10.10.1.100 -> 10.10.2.100: icmp: echo request
    36.862888 Spoke1_WAN1_0 out 10.10.1.100 -> 10.10.2.100: icmp: echo request
    37.167399 Spoke1_WAN1_0 in 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    37.167437 port4 out 10.10.2.100 -> 10.10.1.100: icmp: echo reply
    
  14. Spoke2、Spoke3之间的Shortcut隧道测试与Spoke1一致,这里不再赘述。

Hub访问Spoke

  1. 保留之前线路的延迟状态(Spoke1主动访问Hub时应优先从WAN2的IPSec转发)。

  2. 使用Hub内网PC访问Spoke1的内网PC,由于Hub的SD-WAN规则配置为Manual模式,流量强制从Hub_WAN1转发(除非Hub_WAN1隧道中断)。

    image-20231226180727178

  3. 在Spoke1上查看返回的流量走向,只要路由表中存在返回的路由,就会遵循源进源出原则,而不是匹配SD-WAN规则。

    Spoke1 # diagnose sys sdwan service 1
    
    Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Tie break: cfg
      Gen(12), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Members(4): 
        1: Seq_num(2 Spoke1_WAN2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
        2: Seq_num(1 Spoke1_WAN1), alive, sla(0x0), gid(0), cfg_order(0), local cost(0), selected
      Src address(1): 
            10.10.1.0-10.10.1.255
    
      Dst address(1): 
            10.10.254.0-10.10.254.255
    
    Spoke1 # diagnose sniffer packet any 'host 10.10.254.100 and host 10.10.1.100' 4 
    Using Original Sniffing Mode
    interfaces=[any]
    filters=[host 10.10.254.100 and host 10.10.1.100]
    1.105333 Spoke1_WAN1 in 10.10.254.100 -> 10.10.1.100: icmp: echo request
    1.105377 port4 out 10.10.254.100 -> 10.10.1.100: icmp: echo request
    1.105956 port4 in 10.10.1.100 -> 10.10.254.100: icmp: echo reply
    1.105979 Spoke1_WAN1 out 10.10.1.100 -> 10.10.254.100: icmp: echo reply
    

注意事项

默认配置下,当VPN或专线线路(无SNAT)SD-WAN选路出现变化后,已有的会话会立即切换到新链路上,如果想在SD-WAN线路切换后让已有会话保持在原来的线路上,只有新会话创建在新线路上,需要在对应VPN接口或专线接口上使用如下命令。

config system interface
    edit "Spoke1_WAN1"
        set preserve-session-route enable    //默认为disable//
    next
end

Copyright © 2024 Fortinet Inc. All rights reserved. Powered by Fortinet TAC Team.
📲扫描下方二维码分享此页面👇
该页面修订于: 2023-12-27 10:26:54

results matching ""

    No results matching ""