I needed to set up a few fail over routes in case an intermediate route became unavailable. And I needed to do this for two different scenarios.
Create your IP SLAs:
ip sla 1
icmp-echo 10.10.10.111 source-ip 10.10.10.123
timeout 1000
threshold 2
frequency 3
ip sla schedule 1 life forever start-time now
This is ping testing 10.10.10.111 from 10.10.10.123 to verify that the primary hop is available.
ip sla 2
icmp-echo 10.10.11.222 source-ip 10.10.22.123
timeout 1000
threshold 2
frequency 3
ip sla schedule 2 life forever start-time now
Likewise, this is ping testing 10.10.11.222 from 10.10.22.123 to verify that the primary hop is available.
Create your routes (Note: The IP security version of IOS is not needed for this.)::
ip route 0.0.0.0 0.0.0.0 10.10.10.111 track 1
ip route 0.0.0.0 0.0.0.0 10.10.10.222 10
In this case, the default route on the 3750 is to 10.10.10.111, however if it is unavailable, then the default route is to 10.10.10.222.
However, traffic to 10.10.11.10 and 192.168.0.20 should route to 10.10.11.222 by default and 10.10.11.223 when 10.10.11.222 is unavailable.
ip route 10.10.11.10 255.255.255.255 10.10.11.222 track 2
ip route 192.168.0.20 255.255.255.255 10.10.11.222 track 2
ip route 10.10.11.10 255.255.255.0 10.10.11.223
ip route 192.168.0.20 255.255.255.0 10.10.11.223
Activate the tracks with the following statements:
track 1 ip sla 1 reachability
track 2 ip sla 2 reachability
Here are some useful commands to look at the configuration/status:
#show ip sla application
IP Service Level Agreement Technologies
IPSLAs Infrastructure version: Engine-II
Supported Operation Types:
dhcp, dns, echo, ftp, http, jitter, mpls jitter, pathEcho
pathJitter, tcpConnect, udpEcho
Supported Features:
IPSLAs Event Publisher
IP SLAs low memory water mark: 8441624
Estimated system max number of entries: 6182
Estimated number of configurable operations: 4524
Number of Entries configured : 2
Number of active Entries : 2
Number of pending Entries : 0
Number of inactive Entries : 0
Last time the operation configuration changed: 17:54:49.625 PST Tue Nov 13 2018
#show ip sla configuration
IP SLAs, Infrastructure Engine-II
Entry number: 1
Owner:
Tag:
Type of operation to perform: echo
Target address: 10.10.10.111
Source address: 10.10.10.123
Request size (ARR data portion): 28
Operation timeout (milliseconds): 1000
Type Of Service parameters: 0x0
Verify data: No
Vrf Name:
Schedule:
Operation frequency (seconds): 3
Next Scheduled Start Time: Start Time already passed
Group Scheduled : FALSE
Randomly Scheduled : FALSE
Life (seconds): Forever
Entry Ageout (seconds): never
Recurring (Starting Everyday): FALSE
Status of entry (SNMP RowStatus): Active
Threshold (milliseconds): 2
Distribution Statistics:
Number of statistic hours kept: 2
Number of statistic distribution buckets kept: 1
Statistic distribution interval (milliseconds): 20
History Statistics:
Number of history Lives kept: 0
Number of history Buckets kept: 15
History Filter Type: None
Enhanced History:
Entry number: 2
Owner:
Tag:
Type of operation to perform: echo
Target address: 10.10.11.222
Source address: 10.10.22.123
Request size (ARR data portion): 28
Operation timeout (milliseconds): 1000
Type Of Service parameters: 0x0
Verify data: No
Vrf Name:
Schedule:
Operation frequency (seconds): 3
Next Scheduled Start Time: Start Time already passed
Group Scheduled : FALSE
Randomly Scheduled : FALSE
Life (seconds): Forever
Entry Ageout (seconds): never
Recurring (Starting Everyday): FALSE
Status of entry (SNMP RowStatus): Active
Threshold (milliseconds): 2
Distribution Statistics:
Number of statistic hours kept: 2
Number of statistic distribution buckets kept: 1
Statistic distribution interval (milliseconds): 20
History Statistics:
Number of history Lives kept: 0
Number of history Buckets kept: 15
History Filter Type: None
Enhanced History:
#show ip sla statistics
Round Trip Time (RTT) for Index 1
Type of operation: icmp-echo
Latest RTT: 1 ms
Latest operation start time: 11:29:19.092 PST Wed Nov 14 2018
Latest operation return code: OK
Number of successes: 151
Number of failures: 9
Operation time to live: Forever
Round Trip Time (RTT) for Index 2
Type of operation: icmp-echo
Latest RTT: 1 ms
Latest operation start time: 11:29:16.634 PST Wed Nov 14 2018
Latest operation return code: OK
Number of successes: 663
Number of failures: 26
Operation time to live: Forever