Traceroute is the go-to utility for troubleshooting many network related issues.

Overview

  • Traceroute launches a probe packet towards DST IP with a TTL of 1
    • Every router hop decrements the IP TTL of the packet by 1
    • When TTL equals 0 the packet is dropped, and the router sends an ICMP TTL exceeded packet to the SRC with the original probe packet as payloads
  • The received message is displayed as a “hop” by traceroute
  • The TTL is incremented by 1 and another probe sent
    • This process is repeated until traceroute receives ICMP Dest Unreachable

Probe Packets

  • Each OS has a slightly different of implementing traceroute and its probe packets
  • UNIX uses UDP probes
    • UDP DST port 33434 incrementing once per probe
  • Windows (and some others) use ICMP (each request) probes
  • Look for some traceroute implementation that allows you to use TCP, UDP, or ICMP
  • In most cases, three probes are sent per hop
    • This is why you will see three latency results or *’s
    • These three probes are important as in the case of load balancing path and latency might change

traceroute

Finding problems in your traceroute results

  • Network Latency is normal
    • It takes ~400 ms for a packet to travel across the world (around the equator) just in light propagation
    • Many times ICMP is forwarded, and processed by routers general CPU, which can cause higher than normal delays
  • Note the path your traffic is taking (see below for tips)

Tip to understanding traceroute outputs on the internet

Example Traceroute output from macOS

traceroute to google.com (172.217.7.142), 64 hops max, 52 byte packets
 1  192.168.1.1 (192.168.1.1)  2.980 ms  2.112 ms  1.820 ms
 2  96.120.0.0 (96.120.0.0)  14.111 ms  11.262 ms  15.064 ms
 3  xe-2-1-0-sur01.boulevard.va.richmond.comcast.net (68.86.174.21)  13.862 ms  11.079 ms  10.103 ms
 4  ae-20-ar02.charlvilleco.va.richmond.comcast.net (69.139.165.117)  20.135 ms  24.705 ms  22.751 ms
 5  be-21508-cr02.ashburn.va.ibone.comcast.net (68.86.91.53)  22.032 ms  221.995 ms  28.721 ms
 6  hu-0-10-0-0-pe07.ashburn.va.ibone.comcast.net (68.86.83.66)  26.192 ms  23.345 ms  26.009 ms
 7  as4436-1-c.111eighthave.ny.ibone.comcast.net (173.167.57.162)  23.656 ms  27.712 ms  23.995 ms —— as 4436 suggest Autonomous System number 4436 - owned by GTT … and we go to Eighth Ave NYC interesting —— 
 8  * * *
 9  216.239.48.14 (216.239.48.14)  24.979 ms —— Example of Load-balancing 1 ——  
    72.14.233.20 (72.14.233.20)  24.393 ms —— Example of Load-balancing 2 ——  
    66.249.95.20 (66.249.95.20)  28.957 ms —— Example of Load-balancing 3 ——  
10  108.170.246.49 (108.170.246.49)  43.923 ms
    108.170.246.34 (108.170.246.34)  23.696 ms  22.547 ms
11  108.170.232.199 (108.170.232.199)  22.059 ms
    216.239.49.185 (216.239.49.185)  24.893 ms
    216.239.48.101 (216.239.48.101)  34.441 ms
12  216.239.50.92 (216.239.50.92)  24.613 ms  25.921 ms
    216.239.35.163 (216.239.35.163)  21.525 ms
13  108.170.246.1 (108.170.246.1)  24.262 ms  26.109 ms  82.985 ms —— ~83 ms seems to be an anomaly as other probes are ~25 ms —— 
14  216.239.54.205 (216.239.54.205)  26.884 ms
    216.239.54.127 (216.239.54.127)  22.345 ms  21.584 ms
15  iad30s08-in-f142.1e100.net (172.217.7.142)  28.379 ms  21.229 ms  27.930 ms —— “iad” suggests Ashburn Va. ——

NANOG 47 Presentation by Richard Steenbergen inspires this post.