PagerDuty is a fantastic service. The flexibility in scheduling on-call rotations (they added a custom schedule for my team at my last company), the support for integrating with lots of different monitoring platforms and the generally fantastic support make it well worth the monthly cost.
Is PagerDuty worth it for $18 per month? For a 5 person team its $1080 for an entire year to do something that a small script + Google Calendar could take care of. Maybe I'm missing something? I'd love to hear a story about how a feature of PagerDuty saved somebody's butt.
Yes. Every penny ($90/mo). We've been using them at $SMALL_ISP since September 2010. The scheduling, reporting, and flexible alerting has been well worth it to us.
Before PagerDuty we were using Nagios with a modem dialing out to pagers. Later we went to Nagios with email to SMS gateways. We had "problems" with email to SMS delivery and no way to tell if it was a bad employee or flaky cellco gateway. We also had problems with scheduling on call rotation & overrides for vacation.
The escalation/accountability & reporting let us weed out a bad employee that was ignoring pages. We suspected it but phone call delivery with confirmation in Pagerduty ensured there weren't any plausible excuses.
Most importantly it makes scheduling oncall rotations painless.
Yes, you can do all this in plain old Nagios with enough time or hackery. We reached a point where we couldn't waste time on that kind of hackery (changing alert schedules & escalation every time someone went on vacation, google calendar to nagios config scripts) and needed it to just work.
These APIs are just icing on top. I have two projects that will benefit immediately:
1. Our customer service manager has been asking for a easy way to tell who is on call and we didn't want to give her a pager duty login. Now I can make a simple "who is on call?" dashboard so the call center knows who to call if there's a problem monitoring missed.
2. With the Integration Triggers we can open Jira tickets for incidents. This will streamline workflow.
You should checkout the following blog post as a possible solution to item 1 on your list. You can setup a phone number that will page the on-call person.
If you ever find yourself asking this question, consider the follow-up: "Is my business sought by PagerDuty?" If the answer to your question isn't screamingly obvious the answer to my question is.
n.b. This is OK. In a world with X customers and Y firms there are X x Y possible commercial relationships and the overwhelming majority of them will never happen. That's the nature of the beast.
What's your implementation (and testing) cost of your home-brew alternative?
How are you going to document your homebrew alternative?
Odds are good that your DevOps hire will be familiar with Pagerduty (or something like it, though probably worse).
PD works, is straightforward, has plugins to other alerting tools (Nagios and New Relic off the top of my head), has multiple contact methods (SMS, MMS, eMail, phone).
Pagerduty is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead. Or your site comes back up.
Just a happy customer (despite their core competency being waking me up at 4am -- but given that that of itself is a momentous task, that they're good at it says something).
I'd pay $50/mo/user for monitoring and alerting which was:
1) quite reliable (although not as reliable as my production app)
2) totally uncorrelated from failures in my own infrastructure
3) tuneable to eliminate false positives, get good confirmation from users that they've taken issues. An iOS app is the obvious solution here, smarter than IVR + SMS.
Even better:
4) tied into my ticketing system
5) tied into my config management/ops system, so I can let people have simple app/web tools for "roll back! roll back!", etc. At 3am if you get paged, it is nice to have good tools for that kind of thing.
I haven't found anything which is #1-3 without roll-your-own, and #4 and #5 I've never even built internally (but may this time).
If PagerDuty really is #1-3, then it is a bargain at $18/mo.
Understood. We've actually built solutions that can be installed on-premise previously (http://www.ifountain.com). But we found that there are some compelling technical reasons for this service to be cloud based.
- When users receive notifications, they are often outside the office, and for many enterprises connecting to the corporate network from a mobile device is not practical (even if it's technically possible). A cloud based service ensures that the recipients can access the information easily wherever they may be.
- Cloud based service enables cross organization collaboration without complicated infrastructure changes. OpsGenie enables incident focused collaboration. Only the specified recipients have access to the alert and supporting docs. For example, we use it to receive alerts from customer implementations and can collaborate with operations folks without having to have access to their internal operations systems, etc.
The text of their notification e-mails is ridiculously bad ("You have 1 triggered Incident", not customizable). The mails don't even contain direct links to the incidents, much less links to ack/resolve them...
There are no notifications about status updates at all, not even an e-mail. People still call each other all the time to synchronize about "Are you on it or should I take it?" and "Is it resolved yet?".
etc.
I'm still baffled how they managed to ruin such a trivial product so thoroughly.
Needless to say we went back to a simple shell-script with TSV schedule (and that amusingly works better in most regards).
Moe, at OpsGenie (http://www.opsgenie.com), we've attempted to solve this problem through the use of mobile apps (iPhone, Android and html5 app) on smartphones (and tablets). Apps allow us to present much richer information, with custom fields, tags, as well as attachments, and enable users to communicate, not only acknowledge or comment but also integrate with other management systems thru execution of custom actions. I'd love to have you give it a try.
Comments
PagerDuty is a fantastic service. The flexibility in scheduling on-call rotations (they added a custom schedule for my team at my last company), the support for integrating with lots of different monitoring platforms and the generally fantastic support make it well worth the monthly cost.
At Loggly, we integrated our alerting app with PagerDuty, and open-sourced the whole thing if anyone is interested:
https://github.com/loggly/alertbirds-community-edition
We're lucky in that we have so many technically astute users, who've already built cool stuff in spite of us. I'm excited to see what they'll do now.
Chiming in - PagerDuty has saved many butts at Airbnb, too. We love to hate getting pages from you guys.
Datadog hearts PagerDuty. We use it every day. And sometimes nights too...
And here's its awesome API in action in Datadog -http://www.datadoghq.com/2012/07/pagerduty/
TL;DR New Pagerduty API docs at: http://developer.pagerduty.com/
Is PagerDuty worth it for $18 per month? For a 5 person team its $1080 for an entire year to do something that a small script + Google Calendar could take care of. Maybe I'm missing something? I'd love to hear a story about how a feature of PagerDuty saved somebody's butt.
Yes. Every penny ($90/mo). We've been using them at $SMALL_ISP since September 2010. The scheduling, reporting, and flexible alerting has been well worth it to us.
Before PagerDuty we were using Nagios with a modem dialing out to pagers. Later we went to Nagios with email to SMS gateways. We had "problems" with email to SMS delivery and no way to tell if it was a bad employee or flaky cellco gateway. We also had problems with scheduling on call rotation & overrides for vacation.
The escalation/accountability & reporting let us weed out a bad employee that was ignoring pages. We suspected it but phone call delivery with confirmation in Pagerduty ensured there weren't any plausible excuses.
Most importantly it makes scheduling oncall rotations painless.
Yes, you can do all this in plain old Nagios with enough time or hackery. We reached a point where we couldn't waste time on that kind of hackery (changing alert schedules & escalation every time someone went on vacation, google calendar to nagios config scripts) and needed it to just work.
These APIs are just icing on top. I have two projects that will benefit immediately: 1. Our customer service manager has been asking for a easy way to tell who is on call and we didn't want to give her a pager duty login. Now I can make a simple "who is on call?" dashboard so the call center knows who to call if there's a problem monitoring missed. 2. With the Integration Triggers we can open Jira tickets for incidents. This will streamline workflow.
You should checkout the following blog post as a possible solution to item 1 on your list. You can setup a phone number that will page the on-call person.
http://blog.pagerduty.com/2012/08/how-to-provide-24x7-phone-...
Is PagerDuty worth it for $18 per month?
If you ever find yourself asking this question, consider the follow-up: "Is my business sought by PagerDuty?" If the answer to your question isn't screamingly obvious the answer to my question is.
n.b. This is OK. In a world with X customers and Y firms there are X x Y possible commercial relationships and the overwhelming majority of them will never happen. That's the nature of the beast.
Note that Pagerduty also has a $9 per month plan (up to 3 users), aiming at startups...
Do the math.
What's your downtime cost?
What's your implementation (and testing) cost of your home-brew alternative?
How are you going to document your homebrew alternative?
Odds are good that your DevOps hire will be familiar with Pagerduty (or something like it, though probably worse).
PD works, is straightforward, has plugins to other alerting tools (Nagios and New Relic off the top of my head), has multiple contact methods (SMS, MMS, eMail, phone).
Pagerduty is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead. Or your site comes back up.
Just a happy customer (despite their core competency being waking me up at 4am -- but given that that of itself is a momentous task, that they're good at it says something).
I'd pay $50/mo/user for monitoring and alerting which was:
1) quite reliable (although not as reliable as my production app)
2) totally uncorrelated from failures in my own infrastructure
3) tuneable to eliminate false positives, get good confirmation from users that they've taken issues. An iOS app is the obvious solution here, smarter than IVR + SMS.
Even better:
4) tied into my ticketing system
5) tied into my config management/ops system, so I can let people have simple app/web tools for "roll back! roll back!", etc. At 3am if you get paged, it is nice to have good tools for that kind of thing.
I haven't found anything which is #1-3 without roll-your-own, and #4 and #5 I've never even built internally (but may this time).
If PagerDuty really is #1-3, then it is a bargain at $18/mo.
This pretty much sums up why we've built OpsGenie ( http://www.opsgenie.com )
- a reliable notification system with multiple notification channels (iPhone/Android push, email, SMS, voice, etc)
- integrated with IT management systems (monitoring, ticketing, etc.)
- can provide full context with the alert like performance charts, change history, etc., and not just "something went wrong"
- enables recipients to initiate investigative/corrective actions, communicate with others, etc.
We've build iPhone/Android apps to take advantage of push notifications as well as ability to present rich information and configurable user actions.
I'd love to get your feedback and collaborate to develop it further to meet type of requirements you mention above.
Disclaimer: I'm one of the founders of OpsGenie.
This looks pretty interesting, although I'd strongly prefer a self-hostable system to SaaS (at least as an option).
Understood. We've actually built solutions that can be installed on-premise previously (http://www.ifountain.com). But we found that there are some compelling technical reasons for this service to be cloud based.
- When users receive notifications, they are often outside the office, and for many enterprises connecting to the corporate network from a mobile device is not practical (even if it's technically possible). A cloud based service ensures that the recipients can access the information easily wherever they may be.
- Cloud based service enables cross organization collaboration without complicated infrastructure changes. OpsGenie enables incident focused collaboration. Only the specified recipients have access to the alert and supporting docs. For example, we use it to receive alerts from customer implementations and can collaborate with operations folks without having to have access to their internal operations systems, etc.
> Is PagerDuty worth it for $18 per month?
Yes. We use them at Netflix.
The real value that they add is they take care of the edge cases you haven't thought of yet. :)
Sadly they do an embarrassingly poor job at it.
The text of their notification e-mails is ridiculously bad ("You have 1 triggered Incident", not customizable). The mails don't even contain direct links to the incidents, much less links to ack/resolve them...
There are no notifications about status updates at all, not even an e-mail. People still call each other all the time to synchronize about "Are you on it or should I take it?" and "Is it resolved yet?".
etc.
I'm still baffled how they managed to ruin such a trivial product so thoroughly.
Needless to say we went back to a simple shell-script with TSV schedule (and that amusingly works better in most regards).
Moe, at OpsGenie (http://www.opsgenie.com), we've attempted to solve this problem through the use of mobile apps (iPhone, Android and html5 app) on smartphones (and tablets). Apps allow us to present much richer information, with custom fields, tags, as well as attachments, and enable users to communicate, not only acknowledge or comment but also integrate with other management systems thru execution of custom actions. I'd love to have you give it a try.
Disclamer: I'm one of the founders of OpsGenie
Could you elaborate on one of these edge cases? A specific anecdote would be great.