I've taken part 1 of 3 in Andrew Ng's machine learning specialization which covers the math for supervised learning, linear regression, etc. As I started part 2 (neural networks), it built off the math from part 1 such as the sigmoid activation function. This is what I think of when the OP refers to learning ML from first principles. I highly recommend Andrew Ng's course and I feel like I need to take it again to really understand those basic building blocks.
Comments
I've taken part 1 of 3 in Andrew Ng's machine learning specialization which covers the math for supervised learning, linear regression, etc. As I started part 2 (neural networks), it built off the math from part 1 such as the sigmoid activation function. This is what I think of when the OP refers to learning ML from first principles. I highly recommend Andrew Ng's course and I feel like I need to take it again to really understand those basic building blocks.