Comment on AITemplate, a revolutionary new inference engine by Meta AIparentComments−upbeat_general3yDo you know of any good explanations of the techniques you used for those who only touch PyTorch Eager + occasionally torchscript?−ipiszy3yYou could check "AITemplate optimizations" section in the blog (https://ai.facebook.com/blog/gpu-inference-engine-nvidia-amd...), and https://github.com/facebookincubator/AITemplate#more-about-a.... The basic idea is to do aggressive kernel fusions.
Comments
Do you know of any good explanations of the techniques you used for those who only touch PyTorch Eager + occasionally torchscript?
You could check "AITemplate optimizations" section in the blog (https://ai.facebook.com/blog/gpu-inference-engine-nvidia-amd...), and https://github.com/facebookincubator/AITemplate#more-about-a.... The basic idea is to do aggressive kernel fusions.