Comment on Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.parentComments−ricketycricket2yexcept if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.May I ask to which service you are referring?−cess112yThis one: https://jonatanklosko-bumblebee-tools.hf.space/apps/tokenize...It's linked in the Bumblebee README. Seems broken at the moment, maybe the PR it made is more informative: https://huggingface.co/Neprox/STT-Swedish-Whisper/discussion...−cess112yThe tokenizer generator has risen again.
Comments
May I ask to which service you are referring?
This one: https://jonatanklosko-bumblebee-tools.hf.space/apps/tokenize...
It's linked in the Bumblebee README. Seems broken at the moment, maybe the PR it made is more informative: https://huggingface.co/Neprox/STT-Swedish-Whisper/discussion...
The tokenizer generator has risen again.