Skip to content

Ask HN: How did you learn to debug production incidents?

1 pointbinora2 comments
On HN

Comments

While debugging a crash or incident, I used to get lost in the specific line of code and trying to fixing symptoms. TLA+ gave me new prespective towards debugging, and started treating system crashes as a state problems rather than just code errors. I stop asking 'Why is this line failing?' and start asking 'How did the system get into this state?'.

And whenever a fix or patch feel like a duct-tape to me, start looking at the architecture and keep asking myself how can this be refactor to make it more resilient.

I realized my brain naturally wants to 'fill in the gaps' with assumptions. Learning to suppress that urge and force myself to verify what is actually happening rather than what I think is happening has been the most important part of my growth.

Its a continues process, you never stop learning.

Through tears and mud, how else?

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.