The having constant SRE support is a big deal (your point 2). While we had broad ownership, autonomy, and fulfilled many of the 12 factors described in the blog (and shipped ml projects to production), only recently did we have SREs supporting us continuously and it is a night and day improvement in QoL. It is as you said, so much time is fighting your own infra and it just takes a lot of time and mind space.
It really breaks down to acknowledging how many overlaps ML has with traditional software development. Great to hear that adding SREs worked for you - how did the transition go for the rest of the team? Any big learnings from your team about the transition?
Comments
The having constant SRE support is a big deal (your point 2). While we had broad ownership, autonomy, and fulfilled many of the 12 factors described in the blog (and shipped ml projects to production), only recently did we have SREs supporting us continuously and it is a night and day improvement in QoL. It is as you said, so much time is fighting your own infra and it just takes a lot of time and mind space.
It really breaks down to acknowledging how many overlaps ML has with traditional software development. Great to hear that adding SREs worked for you - how did the transition go for the rest of the team? Any big learnings from your team about the transition?