It's not unusual for a server to also be a router in a layer 3 link aggregation setup. It's extremely common for IPs to be load-shared amongst servers using ECMP. If each server is connected to 2 Top-of-rack (TOR) switches and advertises the route to the shared IP through both TORs, you can very easily have ICMP probes used for PMTU take the wrong route and be dropped. The result is a TCP session with a default MTU that may not work along all traversed paths and will suffer from fragmentation.
Comments
It's not unusual for a server to also be a router in a layer 3 link aggregation setup. It's extremely common for IPs to be load-shared amongst servers using ECMP. If each server is connected to 2 Top-of-rack (TOR) switches and advertises the route to the shared IP through both TORs, you can very easily have ICMP probes used for PMTU take the wrong route and be dropped. The result is a TCP session with a default MTU that may not work along all traversed paths and will suffer from fragmentation.