Ping Works. The Application Doesn’t: How to Troubleshoot MTU Problems

The server answers ping. SSH connects. Then a file transfer stalls, or a web request hangs just as you expect the page to arrive. Someone announces that the network is fine because ping works. The network, apparently, has an excellent public relations department.

A small ping proves that a small ICMP exchange succeeds. It does not prove that the path carries the packet sizes your application needs. In this guide, we’ll narrow down an MTU problem, reproduce it in an isolated Linux lab, and choose a fix based on the evidence.

What MTU actually measures

An interface’s IP MTU is the largest IP packet it sends without fragmentation at that interface. A path MTU is the smallest link MTU along a particular path. TCP MSS is different again: it describes TCP payload size, excluding the IP and TCP headers. Ethernet frame limits include other bytes, and vendors do not always present those limits using the same accounting.

That distinction matters when someone says, “Everything is set to 1500.” Are they talking about a host interface, a tunnel’s inner packet, or the switch’s maximum frame size? Write down the layer before comparing the numbers.

Classic IPv4 Path MTU Discovery uses the Don’t Fragment flag and ICMP fragmentation-needed feedback. An intermediate router that cannot forward a packet at its current size reports the restriction. If that feedback disappears, the sender may keep trying packets the path cannot carry. That is the basic shape of a PMTU black hole. RFC 1191 describes the mechanism.

Start with the actual application path

Record the source and destination IP addresses, address family, outgoing interface, tunnel use, and direction of the failing transfer. Test from the affected host or network namespace. A successful probe from your laptop may take an entirely different route.

ip route get 198.51.100.20
ip -br link
ip -s link show dev eth0

Replace the documentation address and interface with your own. Look for an unexpected VPN route, a lower interface MTU, or errors increasing during the failure. These commands establish context; none of them proves the entire path is healthy.

Find the size boundary

For IPv4 without IP options, a 1472-byte ICMP payload plus the 8-byte ICMP header and 20-byte IP header makes a 1500-byte IP packet. The 28-byte adjustment is specific to this probe format. It is not a universal subtraction for all traffic.

ping -4 -c 3 -M do -s 1200 198.51.100.20
ping -4 -c 3 -M do -s 1472 198.51.100.20
ping -4 -c 3 -M do -s 1372 198.51.100.20

On Linux, -s sets the ICMP data size and -M do requests discovery behavior that forbids fragmentation. If 1372 succeeds and 1472 does not, try sizes between them to narrow the boundary. A 1372-byte success supports a 1400-byte IP packet for that exchange; it does not, by itself, locate the limiting interface. See the iputils ping manual.

Read the failure carefully. An ICMP response identifying a smaller MTU is evidence of a path restriction. A local “message too long” can reflect the sender’s interface limit or a previously learned PMTU. A timeout alone could also mean filtering, loss, or a return-path problem. Don’t turn an absent reply into a diagnosis.

tracepath -4 -n 198.51.100.20

Tracepath uses UDP probes and can report discovered MTU changes. Filtering and different ECMP selection can make its view differ from the application’s. Treat it as another clue, not a definitive map of every flow. Tracepath reference.

Build a small, deliberately broken path

Use a disposable Ubuntu VM with sudo access. This lab creates three network namespaces and two virtual links. It changes forwarding only inside the lab router, not on your host’s production interfaces. Use a fresh VM or confirm the namespace names do not already exist. The commands are a documentation-reviewed lab recipe; the expected behavior below is not a captured test run.

sudo apt update
sudo apt install iproute2 iputils-ping iputils-tracepath tcpdump
sudo ip netns add mtu-a
sudo ip netns add mtu-r
sudo ip netns add mtu-b
sudo ip link add a0 type veth peer name r0
sudo ip link add r1 type veth peer name b0
sudo ip link set a0 netns mtu-a
sudo ip link set r0 netns mtu-r
sudo ip link set r1 netns mtu-r
sudo ip link set b0 netns mtu-b
sudo ip -n mtu-a addr add 192.0.2.2/24 dev a0
sudo ip -n mtu-r addr add 192.0.2.1/24 dev r0
sudo ip -n mtu-r addr add 198.51.100.1/24 dev r1
sudo ip -n mtu-b addr add 198.51.100.2/24 dev b0
sudo ip -n mtu-a link set lo up
sudo ip -n mtu-r link set lo up
sudo ip -n mtu-b link set lo up
sudo ip -n mtu-a link set a0 mtu 1500 up
sudo ip -n mtu-r link set r0 mtu 1500 up
sudo ip -n mtu-r link set r1 mtu 1400 up
sudo ip -n mtu-b link set b0 mtu 1400 up
sudo ip -n mtu-a route add default via 192.0.2.1
sudo ip -n mtu-b route add default via 198.51.100.1
sudo ip netns exec mtu-r sysctl -w net.ipv4.ip_forward=1

mtu-a                     mtu-r                     mtu-b
192.0.2.2 -- 1500 MTU -- 192.0.2.1
                        198.51.100.1 -- 1400 MTU -- 198.51.100.2

The two subnets are connected by the router namespace. The smaller routed link is intentional. Network namespaces provide separate network stacks, while the kernel’s IP sysctl documentation explains the forwarding switch.

Watch the router explain the problem

In a second terminal, capture ICMP on the router’s source-facing interface. Stop the capture with Ctrl+C when you finish.

sudo ip netns exec mtu-r tcpdump -ni r0 -vv icmp

In the first terminal, send a small probe, then one that requires a 1500-byte path, then one that fits 1400.

sudo ip netns exec mtu-a ping -4 -c 3 198.51.100.2
sudo ip netns exec mtu-a ping -4 -c 3 -M do -s 1472 198.51.100.2
sudo ip netns exec mtu-a ping -4 -c 3 -M do -s 1372 198.51.100.2

Expected behavior: small probes pass; the oversized probe fails with fragmentation-needed feedback indicating 1400; the 1400-byte probe passes. Later oversized probes may fail locally after the sender learns the limit. That is why the capture and the timing of the first failure matter.

This demonstrates a working MTU notification, not a black hole. To investigate a real black hole, capture near both the restricting router and the sender. If the router generates feedback but the sender never sees it, follow the return path and firewall policy. Avoid assuming every MTU problem requires increasing the MTU.

Fix the constraint you actually find

• An unintended low MTU: reconcile the host, routed interfaces, and switching frame limits against the approved design. Verify every relevant hop before increasing the value.
• A legitimate tunnel limit: budget for the actual encapsulation and configure an appropriate inner MTU. Tunnel protocols, outer IP versions, and optional headers change the overhead.
• Blocked PMTU feedback: permit the required ICMP errors through the relevant policy path. For IPv4 this includes destination-unreachable type 3, code 4. Verify receipt at the sender.
• A TCP-specific workaround: MSS adjustment at the appropriate boundary can constrain TCP segment size, but it does not fix UDP or all application protocols. Account for direction and verify new connections.

In our lab, assume both virtual links are intended to carry 1500-byte IP packets. Restore the smaller link at both ends:

sudo ip -n mtu-r link set r1 mtu 1500
sudo ip -n mtu-b link set b0 mtu 1500

A previously learned smaller PMTU may persist. For a clean lab retest, stop the capture, delete the three lab namespaces using the cleanup commands below, and rerun setup with both 1400 values changed to 1500. This avoids treating cached state as a failed fix. In production, inspect learned state and its expiry before changing or clearing it.

IPv6 and tunnels deserve their own checks

IPv6 routers do not fragment forwarded packets. ICMPv6 Packet Too Big feedback is central to conventional IPv6 PMTU discovery. Test IPv6 independently when applications use both families. For a simple IPv6 echo probe with no extension headers, 1452 bytes of data plus 40 bytes of IPv6 header and 8 bytes of ICMPv6 header totals 1500. RFC 8201 explains the discovery process.

For encapsulated traffic, keep inner and outer sizes separate. A host may generate a perfectly valid inner packet that becomes too large after encapsulation. Capture placement matters too: segmentation and receive offloads can make host captures look different from packets on the wire. Correlate host evidence with the tunnel endpoint or a suitable network capture.

Prove the application works, then clean up

Retest the original operation in both relevant directions, using the same address family, endpoints, and tunnel path. Open fresh application connections, repeat the size probes, and look for recurring retransmissions or stalls. The goal is a working transfer with an explained packet-size limit, not merely a larger number in an interface configuration.

After stopping the lab capture and any other processes you start inside these namespaces, remove only the lab namespaces. Their virtual interfaces disappear with them.

sudo ip netns del mtu-a
sudo ip netns del mtu-r
sudo ip netns del mtu-b

The useful habit is simple: when small exchanges work and larger ones fail, test size deliberately. A few controlled probes and a well-placed capture can turn “the network is acting weird” into a specific interface, policy, or tunnel setting.

Related Guides

• Five Causes of GPU Fabric Performance Problems, and How to Fix Them
• Deploying GB200 and GB300 on Ethernet: The Network Beyond the GPU Rack
• How to Calculate Network Oversubscription
• iperf3 Throughput Testing: Measure and Troubleshoot