Comment on Mesh LLM: distributed AI computing on irohparentComments−DerivativeBS2moCurious about: does it have fault tolerance if one of the machines goes down mid-inference? Can it dynamically reroute, or does it just retry?−i3862moIt can dynamically route. If a machine drops out of split, the topology is recalculated and the request is automatically retried.
Comments
Curious about: does it have fault tolerance if one of the machines goes down mid-inference? Can it dynamically reroute, or does it just retry?
It can dynamically route. If a machine drops out of split, the topology is recalculated and the request is automatically retried.