afaik it's mostly for cool demos and educational purposes, or extremely latency intensive applications that funny require large models.
for most deep learning models, training on the client is completely unreasonable, as they require weeks of training even on multi thousand $ pro equipement.
For training, tiny metalearning models are the only reasonable thing to train on the clients in most useful scenarios, as they are pretrained to take as few examples as possible to train on a specific task (see MAML).
For inference, aside from educational applications, the only potential advantage of in browser over in server is the lower latency. the main disadvantages are that you need to send the model to the client (multiple MB), with the longer loading time and potential problems with intellectual property this entails. maybe for some extremely small models in very latency driven applications, it can be worth it.
So, overall, either educational purposes, training and use of latency hungry tiny metalearning models or inference with tiny pretrained latency hungry models, like computer vision on webcam sort of deal
Comments
afaik it's mostly for cool demos and educational purposes, or extremely latency intensive applications that funny require large models.
for most deep learning models, training on the client is completely unreasonable, as they require weeks of training even on multi thousand $ pro equipement.
For training, tiny metalearning models are the only reasonable thing to train on the clients in most useful scenarios, as they are pretrained to take as few examples as possible to train on a specific task (see MAML).
For inference, aside from educational applications, the only potential advantage of in browser over in server is the lower latency. the main disadvantages are that you need to send the model to the client (multiple MB), with the longer loading time and potential problems with intellectual property this entails. maybe for some extremely small models in very latency driven applications, it can be worth it.
So, overall, either educational purposes, training and use of latency hungry tiny metalearning models or inference with tiny pretrained latency hungry models, like computer vision on webcam sort of deal
Keep in mind that while the tutorial is in javascript, in the web browser, the neural network easily apply for node based solutions as well.
We're adding GPU support that use either client side OR server side GPU, so that any case you mention can be handled.
yes, but afaik node with webgl is strictly slower than cuda/cudnn with Python or whatever serving
unless you also support cuda/cudnn directly now ofc