- Topology
- Networking environment
From the diagram, Switch A/B/C are TOR switches, between them is Layer 3 network,
the downlink HOST are tenement test, it requires to isolate with other tenements.
To make sure the reliabilities, part of the servers need active-active connection, so
it needs to configure MLAG and configure virtual ip as server’s gateway on SWITCH
A/B. At the same time, making sure the gateway address is not changing after virtual
machine migration, and then deploy distributed gateway on SWITCH A/B/C.
- Configuration Steps
Configure MLAG on SWITCH A and B separately, is used for server active-active
connection, SWITCH C uses single downlink interface
SWITCH_A# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SWITCH_A(config)# interface range eth-0-9 – 10
SWITCH_A(config-if-range)# no shutdown
SWITCH_A(config-if-range)# static-channel-group 55
SWITCH_A(config-if-range)# exit
SWITCH_A(config)# interface agg 55
SWITCH_A(config-if)# switchport mode trunk
SWITCH_A(config-if)# switchport trunk allowed vlan all
SWITCH_A(config-if)# spanning-tree port disable
SWITCH_A(config-if)# exit
SWITCH_A(config)# vlan database
SWITCH_A(config-vlan)# vlan 4094
SWITCH_A(config-vlan)# exit
SWITCH_A(config)# interface vlan 4094
SWITCH_A(config-if)# ip address 40.94.0.1/24
SWITCH_A(config-if)# exit
SWITCH_A(config)# mlag configuration
SWITCH_A(config-mlag)# peer-link agg 55
SWITCH_A(config-mlag)# peer-address 40.94.0.2
SWITCH_A(config)# vlan database
SWITCH_A(config-vlan)# vlan 10,20
SWITCH_A(config-vlan)# exit
SWITCH_A(config)# interface eth-0-1
SWITCH_A(config-if)# switchport mode trunk
SWITCH_A(config-if)# switchport trunk allowed vlan add 10
SWITCH_A(config-if)# no shutdown
SWITCH_A(config-if)# static-channel-group 1
SWITCH_A(config-if)# exit
SWITCH_A(config)# interface eth-0-2
SWITCH_A(config-if)# switchport mode trunk
SWITCH_A(config-if)# switchport trunk allowed vlan add 20
SWITCH_A(config-if)# no shutdown
SWITCH_A(config-if)# static-channel-group 2
SWITCH_A(config-if)# exit
SWITCH_A(config)# interface agg 1
SWITCH_A(config-if)# mlag 1
SWITCH_A(config-if)# exit
SWITCH_A(config)# interface agg 2
SWITCH_A(config-if)# mlag 2
SWITCH_A(config-if)# end
########################################
SWITCH_B# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SWITCH_B(config)# interface range eth-0-9 – 10
SWITCH_B(config-if-range)# no shutdown
SWITCH_B(config-if-range)# static-channel-group 55
SWITCH_B(config-if-range)# exit
SWITCH_B(config)# interface agg 55
SWITCH_B(config-if)# switchport mode trunk
SWITCH_B(config-if)# switchport trunk allowed vlan all
SWITCH_B(config-if)# spanning-tree port disable
SWITCH_B(config-if)# exit
SWITCH_B(config)# vlan database
SWITCH_B(config-vlan)# vlan 4094
SWITCH_B(config-vlan)# exit
SWITCH_B(config)# interface vlan 4094
SWITCH_B(config-if)# ip address 40.94.0.2/24
SWITCH_B(config-if)# exit
SWITCH_B(config)# mlag configuration
SWITCH_B(config-mlag)# peer-link agg 55
SWITCH_B(config-mlag)# peer-address 40.94.0.1
SWITCH_B(config-mlag)# exit
SWITCH_B(config)# vlan database
SWITCH_B(config-vlan)# vlan 10,20
SWITCH_B(config-vlan)# exit
SWITCH_B(config)# interface eth-0-1
SWITCH_B(config-if)# switchport mode trunk
SWITCH_B(config-if)# switchport trunk allowed vlan add 10
SWITCH_B(config-if)# static-channel-group 1
SWITCH_B(config-if)# no shutdown
SWITCH_B(config-if)# exit
SWITCH_B(config)# interface eth-0-2
SWITCH_B(config-if)# switchport mode trunk
SWITCH_B(config-if)# switchport trunk allowed vlan add 20
SWITCH_B(config-if)# static-channel-group 2
SWITCH_B(config-if)# no shutdown
SWITCH_B(config-if)# exit
SWITCH_B(config)# interface agg 1
SWITCH_B(config-if)# mlag 1
SWITCH_B(config-if)# exit
SWITCH_B(config)# interface agg 2
SWITCH_B(config-if)# mlag 2
SWITCH_B(config-if)# end
#####################################
SWITCH_C# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SWITCH_C(config)# vlan database
SWITCH_C(config-vlan)# vlan 10
SWITCH_C(config-vlan)# exit
SWITCH_C(config)# interface eth-0-1
SWITCH_C(config-if)# switchport mode trunk
SWITCH_C(config-if)# switchport trunk allowed vlan add 10
SWITCH_C(config-if)# no shutdown
SWITCH_C(config-if)# end
Configure the address of network gateway and enable distributed gateway, use
virtual ip SWITCH_A
SWITCH_A(config)# ip vrf test
SWITCH_A(config-vrf)# exit
SWITCH_A(config)# interface vlan 10
SWITCH_A(config-if)# ip vrf forwarding test
SWITCH_A(config-if)# ip address 192.168.10.253/24
SWITCH_A(config-if)# ip virtual-router address 192.168.10.1
SWITCH_A(config-if)# overlay distributed-gateway enable
SWITCH_A(config-if)# exit
SWITCH_A(config)# interface vlan 20
SWITCH_A(config-if)# ip vrf forwarding test
SWITCH_A(config-if)# ip address 192.168.20.253/24
SWITCH_A(config-if)# ip virtual-router address 192.168.20.1
SWITCH_A(config-if)# overlay distributed-gateway enable
SWITCH_A(config-if)# exit
SWITCH_A(config)# ip virtual-router mac 0.0.1
SWITCH_A(config)# end
######################################
SWITCH_B(config)# ip vrf test
SWITCH_B(config-vrf)# exit
SWITCH_B(config)# interface vlan 10
SWITCH_B(config-if)# ip vrf forwarding test
SWITCH_B(config-if)# ip address 192.168.10.254/24
SWITCH_B(config-if)# ip virtual-router address 192.168.10.1
SWITCH_B(config-if)# overlay distributed-gateway enable
SWITCH_B(config-if)# exit
SWITCH_B(config)# interface vlan 20
SWITCH_B(config-if)# ip vrf forwarding test
SWITCH_B(config-if)# ip address 192.168.20.254/24
SWITCH_B(config-if)# ip virtual-router address 192.168.20.1
SWITCH_B(config-if)# overlay distributed-gateway enable
SWITCH_B(config-if)# exit
SWITCH_B(config)# ip virtual-router mac 0.0.1
SWITCH_B(config)# end
#####################################
SWITCH_C# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SWITCH_C(config)# ip vrf test
SWITCH_C(config-vrf)# exit
SWITCH_C(config)# interface vlan 10
SWITCH_C(config-if)# ip vrf forwarding test
SWITCH_C(config-if)# ip address 192.168.10.1/24
SWITCH_C(config-if)# overlay distributed-gateway enable
SWITCH_C(config-if)# end
Configure Layer 3 network between switches
SWITCH_A(config)# interface loopback 0
SWITCH_A(config-if)# ip address 10.1.1.1/32
SWITCH_A(config-if)# exit
SWITCH_A(config)# interface eth-0-17
SWITCH_A(config-if)# no switchport
SWITCH_A(config-if)# no shutdown
SWITCH_A(config-if)# ip address 192.168.17.1/24
SWITCH_A(config-if)# exit
SWITCH_A(config)# ip route 10.3.3.3/32 192.168.17.2
SWITCH_A(config)# ip route 10.3.3.3/32 40.94.0.2 100
SWITCH_A(config)#end
######################################
SWITCH_B(config)# interface loopback 0
SWITCH_B(config-if)# ip address 10.1.1.1/32
SWITCH_B(config-if)# exit
SWITCH_B(config)# interface eth-0-13
SWITCH_B(config-if)# no switchport
SWITCH_B(config-if)# no shutdown
SWITCH_B(config-if)# ip address 192.168.13.1/24
SWITCH_B(config-if)# exit
SWITCH_B(config)# ip route 10.3.3.3/32 192.168.13.2
SWITCH_B(config)# ip route 10.3.3.3/32 40.94.0.1 100
SWITCH_B(config)# end
#######################################
SWITCH_C(config)# interface loopback 0
SWITCH_C(config-if)# ip address 10.3.3.3/32
SWITCH_C(config-if)# exit
SWITCH_C(config)# interface eth-0-9
SWITCH_C(config-if)# no shutdown
SWITCH_C(config-if)# no switchport
SWITCH_C(config-if)# ip address 192.168.9.1/24
SWITCH_C(config-if)# exit
SWITCH_C(config)# ip route 10.1.1.1/32 192.168.9.2
SWITCH_C(config)# end
######################################
SWITCH_D(config)# interface eth-0-17
SWITCH_D(config-if)# no shutdown
SWITCH_D(config-if)# no switchport
SWITCH_D(config-if)# ip address 192.168.17.2/24
SWITCH_D(config-if)# exit
SWITCH_D(config)# interface eth-0-13
SWITCH_D(config-if)# no shutdown
SWITCH_D(config-if)# no switchport
SWITCH_D(config-if)# ip address 192.168.13.2/24
SWITCH_D(config-if)# exit
SWITCH_D(config)# interface eth-0-9
SWITCH_D(config-if)# no shutdown
SWITCH_D(config-if)# no switchport
SWITCH_D(config-if)# ip address 192.168.9.2/24
SWITCH_D(config-if)# exit
SWITCH_D(config)# ip route 10.1.1.1/32 192.168.17.1
SWITCH_D(config)# ip route 10.1.1.1/32 192.168.13.1
SWITCH_D(config)# ip route 10.3.3.3/32 192.168.9.1
SWITCH_D(config)# end
Configure VXLAN tunnels
SWITCH_A(config)# vlan database
SWITCH_A(config-vlan)# vlan 10 overlay enable
SWITCH_A(config-vlan)# vlan 20 overlay enable
SWITCH_A(config-vlan)# exit
SWITCH_A(config)# overlay
SWITCH_A(config-overlay)# source 10.1.1.1
SWITCH_A(config-overlay)# remote-vtep 1 ip-address 10.3.3.3 type vxlan
SWITCH_A(config-overlay)# vlan 10 vni 10000
SWITCH_A(config-overlay)# vlan 10 remote-vtep 1
SWITCH_A(config-overlay)# vlan 20 vni 20000
SWITCH_A(config-overlay)# vlan 20 remote-vtep 1
SWITCH_A(config-overlay)# exit
SWITCH_A(config)# interface eth-0-17
SWITCH_A(config-if)# vxlan uplink enable
SWITCH_A(config-if)# end
###############################################
SWITCH_B(config)# vlan database
SWITCH_B(config-vlan)# vlan 10 overlay enable
SWITCH_B(config-vlan)# vlan 20 overlay enable
SWITCH_B(config-vlan)# exit
SWITCH_B(config)# overlay
SWITCH_B(config-overlay)# source 10.1.1.1
SWITCH_B(config-overlay)# remote-vtep 1 ip-address 10.3.3.3 type vxlan
SWITCH_B(config-overlay)# vlan 10 vni 10000
SWITCH_B(config-overlay)# vlan 10 remote-vtep 1
SWITCH_B(config-overlay)# vlan 20 vni 20000
SWITCH_B(config-overlay)# vlan 20 remote-vtep 1
SWITCH_B(config-overlay)# exit
SWITCH_B(config)# interface eth-0-13
SWITCH_B(config-if)# vxlan uplink enable
SWITCH_B(config-if)# end
########################################
SWITCH_C(config)# vlan database
SWITCH_C(config-vlan)# vlan 10 overlay enable
SWITCH_C(config-vlan)# vlan 20
SWITCH_C(config-vlan)# vlan 20 overlay enable
SWITCH_C(config-vlan)# exit
SWITCH_C(config)# overlay
sSWITCH_C(config-overlay)# source 10.3.3.3
SWITCH_C(config-overlay)# remote-vtep 1 ip-address 10.1.1.1 type vxlan
SWITCH_C(config-overlay)# vlan 10 vni 10000
SWITCH_C(config-overlay)# vlan 10 remote-vtep 1
SWITCH_C(config-overlay)# vlan 20 vni 20000
SWITCH_C(config-overlay)# vlan 20 remote-vtep 1
SWITCH_C(config-overlay)# exit
SWITCH_C(config)# interface eth-0-9
SWITCH_C(config-if)# vxlan uplink enable
SWITCH_C(config-if)# end
configure DVR routing, make it interflows with different switches under different
segments’ hosts.
SWITCH_A# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SWITCH_A(config)# ip route vrf test 192.168.10.20/32 remote-vtep 1 vni 10000 inner-macda
b.b.b
SWITCH_A(config)# end
SWITCH_B# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SWITCH_B(config)# ip route vrf test 192.168.10.20/32 remote-vtep 1 vni 10000 inner-macda
b.b.b
SWITCH_B(config)# end
SWITCH_C# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SWITCH_C(config)# ip route vrf test 192.168.20.10/32 remote-vtep 1 vni 20000 inner-macda
c.c.c
SWITCH_C(config)# end
- Verify Configuration
#check the status of downlink interface on MLAG
SWITCH_A# show mlag interface
mlagid local-if local-state remote-state
1 agg1 up up
2 agg2 up up
# check the status of VXLAN
SWITCH_A# show overlay
—————————————————————
ECMP Mode : Normal
Source VTEP : 10.1.1.1
Vlan Vni Type Remote-vtep IP-Address
—————————————————————
10 10000 VxLAN 1 10.3.3.3
20 20000 VxLAN 1 10.3.3.3
# Check whether DVR routing is effective
SWITCH_A# show ip route vrf test
Codes: K – kernel, C – connected, S – static, R – RIP, B – BGP
O – OSPF, IA – OSPF inter area
N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2
E1 – OSPF external type 1, E2 – OSPF external type 2
i – IS-IS, L1 – IS-IS level-1, L2 – IS-IS level-2, ia – IS-IS inter area
Dc – DHCP Client
[*] – [AD/Metric]
* – candidate default
C 192.168.10.0/24 is directly connected, vlan10
C 192.168.10.253/32 is in local loopback, vlan10
C 192.168.10.1/32 is directly connected, vlan10
S 192.168.10.20/32 is in overlay remote vxlan vtep:10.3.3.3, vni:10000
C 192.168.20.0/24 is directly connected, vlan20
C 192.168.20.253/32 is in local loopback, vlan20
C 192.168.20.1/32 is directly connected, vlan20
