At Cronitor we've just finished building out lambda infrastructure for a fan-out where I would otherwise use a worker application and Supervisord.
Lambda has a lot of strengths, and it gives you solid primitives like immutable versions, version aliases/pointers and stateless functions.
The problem I had was that it didn't give me any more than that. Here are just a few of the things we had figure out and then build ourselves:
1. The development workflow. How can I get a REPL workflow going that doesn't make me go crazy.
2. Deploys and Rollbacks. How can we safely deploy and rollback, especially in cases where you have the same lambda function in multiple regions and each region is at a different published version (because the version number is monotonically incrementing)
3. Permissions and Policies. The broad strokes are clear. But you want your S3 bucket or SNS topic to trigger lambda? Get ready to spend an hour trying to figure out what you've done wrong and what's missing from the vague directions. Hope you get that REPL flow solid first.
In the end, we built several scripts (create_build, publish_version, promote_to_prod, etc) and we use these directly during development and from a Fabric-based deploy script. When I have time I plan to release this tooling open-source.
If I had to do it all over again, I wouldn't. And I wouldn't use Apex (at least not yet). I would just use a t2.small instances with a simple worker.
Comments
At Cronitor we've just finished building out lambda infrastructure for a fan-out where I would otherwise use a worker application and Supervisord.
Lambda has a lot of strengths, and it gives you solid primitives like immutable versions, version aliases/pointers and stateless functions.
The problem I had was that it didn't give me any more than that. Here are just a few of the things we had figure out and then build ourselves:
1. The development workflow. How can I get a REPL workflow going that doesn't make me go crazy.
2. Deploys and Rollbacks. How can we safely deploy and rollback, especially in cases where you have the same lambda function in multiple regions and each region is at a different published version (because the version number is monotonically incrementing)
3. Permissions and Policies. The broad strokes are clear. But you want your S3 bucket or SNS topic to trigger lambda? Get ready to spend an hour trying to figure out what you've done wrong and what's missing from the vague directions. Hope you get that REPL flow solid first.
In the end, we built several scripts (create_build, publish_version, promote_to_prod, etc) and we use these directly during development and from a Fabric-based deploy script. When I have time I plan to release this tooling open-source.
If I had to do it all over again, I wouldn't. And I wouldn't use Apex (at least not yet). I would just use a t2.small instances with a simple worker.