Moving agent execution off localhost makes sense for parallel workflows, but giving an LLM direct provisioning power is terrifying from a billing perspective. If the agent gets into a retry loop or hallucination spiral, it could easily spin up expensive GPU instances (like that B200 example) without tearing them down. Do you enforce hard budget caps or instance-count limits at the API key level to prevent runaway provisioning? Also, how do you handle SSH key lifecycle management—are keys rotated per session to ensure no lingering access remains if the teardown command fails?
Comments
Moving agent execution off localhost makes sense for parallel workflows, but giving an LLM direct provisioning power is terrifying from a billing perspective. If the agent gets into a retry loop or hallucination spiral, it could easily spin up expensive GPU instances (like that B200 example) without tearing them down. Do you enforce hard budget caps or instance-count limits at the API key level to prevent runaway provisioning? Also, how do you handle SSH key lifecycle management—are keys rotated per session to ensure no lingering access remains if the teardown command fails?