Thanks Robert for putting this together. Coming from PyTorch ecosystem. What is the actual requirement for this course? Would a smattering knowledge of ES6 do? Been leching at browser based implementations since Karpathy demoed his CNNs a few years ago but just havent had enough motivation and courage to pickup JS. Perhaps my main reson for anxiety is the confusing JS ecosystem.Also what are some real world use cases for training and deploying NNs on the browser rather than training and deploying it on a tradional cloud backend environment?
afaik it's mostly for cool demos and educational purposes, or extremely latency intensive applications that funny require large models.
for most deep learning models, training on the client is completely unreasonable, as they require weeks of training even on multi thousand $ pro equipement.
For training, tiny metalearning models are the only reasonable thing to train on the clients in most useful scenarios, as they are pretrained to take as few examples as possible to train on a specific task (see MAML).
For inference, aside from educational applications, the only potential advantage of in browser over in server is the lower latency. the main disadvantages are that you need to send the model to the client (multiple MB), with the longer loading time and potential problems with intellectual property this entails. maybe for some extremely small models in very latency driven applications, it can be worth it.
So, overall, either educational purposes, training and use of latency hungry tiny metalearning models or inference with tiny pretrained latency hungry models, like computer vision on webcam sort of deal
Real world use cases might be; Classifying the users mood from mouse movement, classifying the microphone audio. I.e. process real time data that might be too large to upload.
I personally haven't seen any NNs being used in browser apps, but there are plenty of existing mobile apps that has NNs to classify audio/video/etc directly on the device.
All you need know are basic function calls, objects and arrays. I share the same fears you have about the JS ecosystem, especially when they overcomplicate something simple. By keeping the course thought out and simple we can adhere to a wide audience, and show off that the priciples are the same.
Comments
Thanks Robert for putting this together. Coming from PyTorch ecosystem. What is the actual requirement for this course? Would a smattering knowledge of ES6 do? Been leching at browser based implementations since Karpathy demoed his CNNs a few years ago but just havent had enough motivation and courage to pickup JS. Perhaps my main reson for anxiety is the confusing JS ecosystem.Also what are some real world use cases for training and deploying NNs on the browser rather than training and deploying it on a tradional cloud backend environment?
afaik it's mostly for cool demos and educational purposes, or extremely latency intensive applications that funny require large models.
for most deep learning models, training on the client is completely unreasonable, as they require weeks of training even on multi thousand $ pro equipement.
For training, tiny metalearning models are the only reasonable thing to train on the clients in most useful scenarios, as they are pretrained to take as few examples as possible to train on a specific task (see MAML).
For inference, aside from educational applications, the only potential advantage of in browser over in server is the lower latency. the main disadvantages are that you need to send the model to the client (multiple MB), with the longer loading time and potential problems with intellectual property this entails. maybe for some extremely small models in very latency driven applications, it can be worth it.
So, overall, either educational purposes, training and use of latency hungry tiny metalearning models or inference with tiny pretrained latency hungry models, like computer vision on webcam sort of deal
Keep in mind that while the tutorial is in javascript, in the web browser, the neural network easily apply for node based solutions as well.
We're adding GPU support that use either client side OR server side GPU, so that any case you mention can be handled.
yes, but afaik node with webgl is strictly slower than cuda/cudnn with Python or whatever serving
unless you also support cuda/cudnn directly now ofc
Real world use cases might be; Classifying the users mood from mouse movement, classifying the microphone audio. I.e. process real time data that might be too large to upload.
I personally haven't seen any NNs being used in browser apps, but there are plenty of existing mobile apps that has NNs to classify audio/video/etc directly on the device.
All you need know are basic function calls, objects and arrays. I share the same fears you have about the JS ecosystem, especially when they overcomplicate something simple. By keeping the course thought out and simple we can adhere to a wide audience, and show off that the priciples are the same.