Comment on cerebras: 450 tokens/sec llama 3.1 70BComments−davidfialaOP2y- 1,800tps on llama 3.1 8B- 450tps on llama 3.1 70Bfree chat interface is at: https://inference.cerebras.ai (requires login)
Comments
- 1,800tps on llama 3.1 8B
- 450tps on llama 3.1 70B
free chat interface is at: https://inference.cerebras.ai (requires login)