Skip to content

Comment on Sysadmin left finger on power button for an hour to avert SAP outage

Comments

Unscrew a bolt from the server rack and tape it on the button, done

Straight from the article:

"Jeremy told Who, me? that his mate asked to be relieved, as he was in a bit of pain. Those requests were denied due to the risk of the power going off and also out of a desire to make the poor chap suffer for his error."

Last time Indiana Jones tried that, the whole cave collapsed and guys were waiting with blow guns. Have you learned nothing?! :)

ETA: And he almost lost his hat! Come on man!

Jeremy told Who, me? that his mate asked to be relieved, as he was in a bit of pain. Those requests were denied due to the risk of the power going off and also out of a desire to make the poor chap suffer for his error.

Looks like they wanted him to suffer :p

You call it suffer, others call it corrective training ;)

Corrective training for the wrong person. Every time I hear a case like this, I'm reminded that when something like this goes wrong, it's usually the fault of something that failed prior to getting to this stage. The problem isn't that this person pressed the wrong button. The problem is that the wrong button was allowed to be pushed in the first place.

Normally I'm down with that, but I think there are some ground-state base cases where you have for perfectly sound reasons penetrated all the security and safety affordances and simply must be careful. "Make sure you've got the right power button before pushing it" is probably one of them. There's not much you can do about that. Maybe not zero, but not much, in a context like this where many servers are being updated, because, again, whatever protections you may have had in place have already been bypassed.

Similarly, when push comes to shove you'll never be able to eliminate the needs for somebody to jump to root on some server, at which point, well, be careful is all you've really got. Hopefully you've built some good habits into your fingers.

Would you risk it after the button is already pressed?

You'd have to take the bolt into the button without releasing it...

I suppose the point is, relying on a human to keep a button pressed without fatigue for an hour is also risking it.

A lot of people will pick inaction over action in situations like this. (There is also a link to a TDWTF article where someone held a button for the better part of a day)

Well, you raised the possibility of risk! I merely pointed out that both options were risky :)

and what if the tape comes loose?

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.