I've already put one together for personal usage using the ESP32-S3. The advantage of using something like this instead of a raw raspberry PI is that you've got a basic far field microphone, screen, wake word support, etc. and then it's just a matter of wiring up the voice to go to deep whisper for recognition, pass it on to a large language model (in my case I'm using mistral.), generate TTS using Mycroft, and sending it back.
Biggest annoyance is having to have a dedicated server, since the ESP is simply not powerful enough to do real time voice recognition and LLM inference.
It's relatively easy to do - I had a workable prototype within a weekend.
Comments
All the major smart speaker manufacturers have plans to eventually back their services using LLM's.
https://www.theverge.com/2023/9/20/23880764/amazon-ai-alexa-...
I've already put one together for personal usage using the ESP32-S3. The advantage of using something like this instead of a raw raspberry PI is that you've got a basic far field microphone, screen, wake word support, etc. and then it's just a matter of wiring up the voice to go to deep whisper for recognition, pass it on to a large language model (in my case I'm using mistral.), generate TTS using Mycroft, and sending it back.
Biggest annoyance is having to have a dedicated server, since the ESP is simply not powerful enough to do real time voice recognition and LLM inference.
It's relatively easy to do - I had a workable prototype within a weekend.