Had something similar when a k8s node broke but k8s thought the pods (envoy) on it were still running so it routed 1/nth of traffic into a black hole