Thanks a lot for this. It's nearly impossible to find this kind of deep dives on pretraining.