The demo use WASM/browser when available, but also supports running only on ESP32. Figured out I could offload and give the users a better experience when available.
Thanks for noticing the Tailscale demo. Created a Tailscale-Rust client that supports NAT-traversial, encryption etc. and manage to run on ESP32 (together with the local LLM). Just the Tailscale part can be found here: https://github.com/punnerud/tailscale-mpe-rust
Comments
wow, that tailscale-in-browser demo is wild..!
re: language model - here, the embedding lookup, layers, logits all run on-chip and loops its own output back.
the only off-chip piece is the id to string table, i.e. the chip (once I get it!) will speak token ids and my laptop will print them as letters
The demo use WASM/browser when available, but also supports running only on ESP32. Figured out I could offload and give the users a better experience when available.
Thanks for noticing the Tailscale demo. Created a Tailscale-Rust client that supports NAT-traversial, encryption etc. and manage to run on ESP32 (together with the local LLM). Just the Tailscale part can be found here: https://github.com/punnerud/tailscale-mpe-rust