There are too many poor design decisions in the fast.ai library.
One should invest too much time just for the sake of learning the library's weird API, and then using it.
Doing something custom is too difficult, in contrast to Jax, PyTorch, and even (poor library) TensorFlow.
The coding practices are whimsical. The codebase wouldn’t pass code review in any respectable company.
Variable namings are weird and super-problematic.
I fully stick to what I said. Learn techniques, best practices, and, most importantly, Howard's attitude. Then take them with you and move onto something like PyTorch.
Howard is great with one problem: he kinda hates math. It might also seem that he ends up promoting anti-intellectualism.
Howard is great with one problem: he kinda hates math
I'm sorry what?
I run a math study group 4x per week.
Right now the book I'm reading during my rest time is a calculus book.
I've co-authored a lengthy paper on matrix calculus foundations for deep learning.
I wrote a lot of the math materials in our numerical linear programming course.
It really seems like you have very very little understanding of me or the software library I've created, but yet are nonetheless comfortable publicly pronouncing your opinions about both.
Can you explain this last sentence (which I understand to be insulting and without basis):
> Howard is great with one problem: he kinda hates math. It might also seem that he ends up promoting anti-intellectualism.
He says repeatedly "You don't need math", and stuff like that.
This is not insulting. That man is my hero, and I deeply respect him.
But his 2019/20 course was riddled with such statements. He repeatedly said that one doesn't need math, and showed tools like drawing math symbols on a website to learn their names and ride on that. No further math needed.
It's like you can wing it in Deep Learning without learning Math. His behavior throughout the course reinforced this attitude. It is harmful for new learners.
But I am fortunate that I didn't learn from that, but learned from some successful alumni example that Howard gave. One woman who was also a musician ('19/'20), she made it big, but Howard mentioned that she did the Ng course, and also read the Goodfellow book.
So, I took the cue, and did DL the proper way. Anybody I know in DL made it because they know the Math.
There are some influencer types in fastai community who has 10ks of followers and shills stuff and do media stuff. Other than that 1-2 people, everyone who made it in DL, did it because they knew the math.
So, I think that people might get the wrong idea hearing from Howard that "you don't need math".
This is one fault I find. It's not like I dislike him. I like the rest of him. I love his attitude on almost all other things. I love Jeremy Howard, and he is my hero.
You don't need the math in the beginning to train a model and get first results.
Later, you will need the math and Jeremy clearly knows the math.
He gives a great example: In sports, you don't start with learning about physiology and train individual muscles etc. (I paraphrase), you start playing basketball or baseball or soccer, and understand the overall game. And if you like it, you can then become better and better and get deeper and deeper.
It's not helpful to start with linear algebra if - what motivated you - was the application of ML. We lose people who could have otherwise become experts later.
It is good enough to not need heavy math to begin.
Yeah, I know.
But you need a lot of math to do Deep Learning.
But I do not think Howard tries to communicate that.
You can't show me people who knows high school math
only and gets to work in FAANG, or PhD in DL/related, or CTO of an AI start-up, or anyhow "made it" in DL.
I think co-writing and co-teaching a math course at a deep learning company he co-founded, pinning that to the github repo and moving it close to the top of the home page makes it pretty clear he does see value in math in deep learning. I mean, if you need other evidence beyond than the fact that he teaches math needed to understand and build things from scratch in the deep learning course...
In the courses he has always been clear you don't need a ton of math to begin. He's also always been clear that as you progress you will encounter math that you need to learn to continue. He's always clear that that is ok if you don't know it before you start and it's ok to learn it when you need it.
influencer types in the fastai community who have 10ks of followers and shills
Everyone that I can see that fits that profile work at real companies doing real deep learning work, or are building infrastructure and tools that we all use. Nvidia, Huggingface, Etc. I don't see pure media stuff at all, most people are developing libraries or doing other applied work, and talk about their work publicly. Frankly, your comments come across like you are salty. Being a unpleasant person in online forums that enjoys insulting people seems correlated, which likely doesn't bode well for your professional aspirations, regardless of how much math or python you do/don't know.
You don't need math
He's saying you don't need a PhD in math, not that you should ignore math all together. I have graduate level math and CS background and I don't thing either of those helped much, other than overcoming gatekeeping. The thing thats far more important for applied ML is to practice DL on lots of different problems to be effective. PhD level math might be useful for research, but that isn't necessary in practice for most people.
Definitely do the fast.ai course. Totally worth it.
But also use ISLR, Goodfellow, Bishop, etc.
Start with Andrew Ng's ML, then do the first part of Aurelien Geron book, then do Ng's DL specialization, then do fast.ai. Then learn PyTorch. A great book would be Sebastian Raschka's book. Also d2l.ai. A fast-paced, but really good course would be the Neuromatch DL tutorials.
Then move forward based on your interests.
Yann LeCun has THE best MOOC on DL on YouTube.
For the Math, I majored in Physics, so stuff came naturally. I suggest Imperial London's MOOC on Mathematics for ML specialization, Robert Ghrist's Calculus course, VMLS book for Linear Algebra. For stats, haven’t found a good one yet.
What you read, how much- these all depend on what you want to do. Where do you want to see yourself, and so on.
If you just want to brag about DL and put it on your resume so that you can get a job writing SQL queries and make PowerBI presentation as a "Data Scientist", then the bars are low.
If you want to do some DL, then that is another league altogether.
You need to be able to quickly read papers, understand ideas, use those for your own projects or papers.
Comments
There are too many poor design decisions in the fast.ai library.
One should invest too much time just for the sake of learning the library's weird API, and then using it.
Doing something custom is too difficult, in contrast to Jax, PyTorch, and even (poor library) TensorFlow.
The coding practices are whimsical. The codebase wouldn’t pass code review in any respectable company.
Variable namings are weird and super-problematic.
I fully stick to what I said. Learn techniques, best practices, and, most importantly, Howard's attitude. Then take them with you and move onto something like PyTorch.
Howard is great with one problem: he kinda hates math. It might also seem that he ends up promoting anti-intellectualism.
You've recently posted repeated comments that cross into personal attack. We ban accounts that do that, so please don't do it again.
We detached this subthread from https://news.ycombinator.com/item?id=32189308.
I'm sorry what?
I run a math study group 4x per week.
Right now the book I'm reading during my rest time is a calculus book.
I've co-authored a lengthy paper on matrix calculus foundations for deep learning.
I wrote a lot of the math materials in our numerical linear programming course.
It really seems like you have very very little understanding of me or the software library I've created, but yet are nonetheless comfortable publicly pronouncing your opinions about both.
I deleted an earlier, angrier comment of mine.
Can you explain this last sentence (which I understand to be insulting and without basis): > Howard is great with one problem: he kinda hates math. It might also seem that he ends up promoting anti-intellectualism.
He says repeatedly "You don't need math", and stuff like that.
This is not insulting. That man is my hero, and I deeply respect him.
But his 2019/20 course was riddled with such statements. He repeatedly said that one doesn't need math, and showed tools like drawing math symbols on a website to learn their names and ride on that. No further math needed.
It's like you can wing it in Deep Learning without learning Math. His behavior throughout the course reinforced this attitude. It is harmful for new learners.
But I am fortunate that I didn't learn from that, but learned from some successful alumni example that Howard gave. One woman who was also a musician ('19/'20), she made it big, but Howard mentioned that she did the Ng course, and also read the Goodfellow book.
So, I took the cue, and did DL the proper way. Anybody I know in DL made it because they know the Math.
There are some influencer types in fastai community who has 10ks of followers and shills stuff and do media stuff. Other than that 1-2 people, everyone who made it in DL, did it because they knew the math.
So, I think that people might get the wrong idea hearing from Howard that "you don't need math".
This is one fault I find. It's not like I dislike him. I like the rest of him. I love his attitude on almost all other things. I love Jeremy Howard, and he is my hero.
I think you completely misunderstand his stance.
You don't need the math in the beginning to train a model and get first results. Later, you will need the math and Jeremy clearly knows the math.
He gives a great example: In sports, you don't start with learning about physiology and train individual muscles etc. (I paraphrase), you start playing basketball or baseball or soccer, and understand the overall game. And if you like it, you can then become better and better and get deeper and deeper.
It's not helpful to start with linear algebra if - what motivated you - was the application of ML. We lose people who could have otherwise become experts later.
It is far from anti-intellectualism. It is about didactics. And Jeremy is spot on about this.
It is good enough to not need heavy math to begin.
Yeah, I know.
But you need a lot of math to do Deep Learning.
But I do not think Howard tries to communicate that.
You can't show me people who knows high school math only and gets to work in FAANG, or PhD in DL/related, or CTO of an AI start-up, or anyhow "made it" in DL.
I think co-writing and co-teaching a math course at a deep learning company he co-founded, pinning that to the github repo and moving it close to the top of the home page makes it pretty clear he does see value in math in deep learning. I mean, if you need other evidence beyond than the fact that he teaches math needed to understand and build things from scratch in the deep learning course...
In the courses he has always been clear you don't need a ton of math to begin. He's also always been clear that as you progress you will encounter math that you need to learn to continue. He's always clear that that is ok if you don't know it before you start and it's ok to learn it when you need it.
Everyone that I can see that fits that profile work at real companies doing real deep learning work, or are building infrastructure and tools that we all use. Nvidia, Huggingface, Etc. I don't see pure media stuff at all, most people are developing libraries or doing other applied work, and talk about their work publicly. Frankly, your comments come across like you are salty. Being a unpleasant person in online forums that enjoys insulting people seems correlated, which likely doesn't bode well for your professional aspirations, regardless of how much math or python you do/don't know.
He's saying you don't need a PhD in math, not that you should ignore math all together. I have graduate level math and CS background and I don't thing either of those helped much, other than overcoming gatekeeping. The thing thats far more important for applied ML is to practice DL on lots of different problems to be effective. PhD level math might be useful for research, but that isn't necessary in practice for most people.
Can you please provide some resources you used to learn?
Definitely do the fast.ai course. Totally worth it.
But also use ISLR, Goodfellow, Bishop, etc.
Start with Andrew Ng's ML, then do the first part of Aurelien Geron book, then do Ng's DL specialization, then do fast.ai. Then learn PyTorch. A great book would be Sebastian Raschka's book. Also d2l.ai. A fast-paced, but really good course would be the Neuromatch DL tutorials.
Then move forward based on your interests.
Yann LeCun has THE best MOOC on DL on YouTube.
For the Math, I majored in Physics, so stuff came naturally. I suggest Imperial London's MOOC on Mathematics for ML specialization, Robert Ghrist's Calculus course, VMLS book for Linear Algebra. For stats, haven’t found a good one yet.
What you read, how much- these all depend on what you want to do. Where do you want to see yourself, and so on.
If you just want to brag about DL and put it on your resume so that you can get a job writing SQL queries and make PowerBI presentation as a "Data Scientist", then the bars are low.
If you want to do some DL, then that is another league altogether.
You need to be able to quickly read papers, understand ideas, use those for your own projects or papers.
Makes sense?