Comment on GroqchatparentComments−andygeorge2yfast but wrong/gibberish−razorguymania2yIts using vanilla llama-2 from Meta with no fine tuning. The point here is the speed and responsiveness of the underlying HW and SW.−chihuahua2yBut if the quality of the response is poor, it's irrelevant that it was generated quickly. If it was using different data to generate higher quality responses, would that not slow it down?−tome2ynomel gave a good answer in a different threadThis is not about the model, it’s about the relative speed improvement from the hardware, with this model as a demo.To compare apples to apples look at the tokens per second of other systems running Llama 2 70B 4096. We're by far the fastest!https://news.ycombinator.com/item?id=38742466−andygeorge2yDo you work there? Just curious−razorguymania2yyes−andygeorge2yThanks!−andygeorge2yah Llama 2 70B, no wonder
Comments
fast but wrong/gibberish
Its using vanilla llama-2 from Meta with no fine tuning. The point here is the speed and responsiveness of the underlying HW and SW.
But if the quality of the response is poor, it's irrelevant that it was generated quickly. If it was using different data to generate higher quality responses, would that not slow it down?
nomel gave a good answer in a different thread
To compare apples to apples look at the tokens per second of other systems running Llama 2 70B 4096. We're by far the fastest!
https://news.ycombinator.com/item?id=38742466
Do you work there? Just curious
yes
Thanks!
ah Llama 2 70B, no wonder