From what I can tell, it's the invocation model and deployment that is unique here?
You invoke faast from your local machine (or build server, or cron job, whatever), and in turn it deploys some functions to a serverless platform and runs them, then tears them all down when complete. Eg, from the site, this code runs locally:
import { faast } from "faastjs";
import * as funcs from "./functions";
(async () => {
const m = await faast("aws", funcs);
try {
// m.functions.hello: string => Promise<string>
const result = await m.functions.hello("world");
console.log(result);
} finally {
await m.cleanup();
}
})();
You wouldn't want to run this code on serverless, as you'd be paying for compute time of just waiting for all the other tasks to complete.
It would be useful to see a discussion about how and where to host this entry code, may even a topic on "Running in production".
It's definitely a neat idea because if you control the event that kicks everything off anyway (eg: "create monthly invoices" or "build daily reports") you can deploy the latest version of everything, run it and clean it up in essentially a single step.
(Please correct me if I've misunderstood any of the details here!)
You're basically correct, and thanks for the suggestion to add documentation about deployment in production.
One special case is if your functions return a lot of data; outbound data charges can get expensive fast, and you'll be limited in getting responses by your network link. So you can run the coordinator code on, say, EC2 in the same region and then the link to Lambda is super fast and you won't have any outbound data costs.
This is how I interpreted it's usage too. We've all started an instance on DO/AWS/GCP/ETC for some batch job were we wanted 32 cores or whatnot. This lets you use lambda's for the scaling instead of the cores directly. How efficient this is performance wise I have no clue.
To serve as a data point, I effectively built an in-house version of this a few years ago built on top of AWS Lambda, all in Python. The "entry point" code or orchestration code was hosted normally on an EC2 instance. More specifically we were using Airflow, so our Airflow server would kick off a Python program that would then orchestrate a couple thousand Lambdas.
Very cool. Worth checking out a similar project Durable Functions. However those orchestrations can run in serverless and can scale to zero during the “waiting for other tasks” step.
Comments
From what I can tell, it's the invocation model and deployment that is unique here?
You invoke faast from your local machine (or build server, or cron job, whatever), and in turn it deploys some functions to a serverless platform and runs them, then tears them all down when complete. Eg, from the site, this code runs locally:
You wouldn't want to run this code on serverless, as you'd be paying for compute time of just waiting for all the other tasks to complete.It would be useful to see a discussion about how and where to host this entry code, may even a topic on "Running in production".
It's definitely a neat idea because if you control the event that kicks everything off anyway (eg: "create monthly invoices" or "build daily reports") you can deploy the latest version of everything, run it and clean it up in essentially a single step.
(Please correct me if I've misunderstood any of the details here!)
You're basically correct, and thanks for the suggestion to add documentation about deployment in production.
One special case is if your functions return a lot of data; outbound data charges can get expensive fast, and you'll be limited in getting responses by your network link. So you can run the coordinator code on, say, EC2 in the same region and then the link to Lambda is super fast and you won't have any outbound data costs.
This is how I interpreted it's usage too. We've all started an instance on DO/AWS/GCP/ETC for some batch job were we wanted 32 cores or whatnot. This lets you use lambda's for the scaling instead of the cores directly. How efficient this is performance wise I have no clue.
To serve as a data point, I effectively built an in-house version of this a few years ago built on top of AWS Lambda, all in Python. The "entry point" code or orchestration code was hosted normally on an EC2 instance. More specifically we were using Airflow, so our Airflow server would kick off a Python program that would then orchestrate a couple thousand Lambdas.
Very cool. Worth checking out a similar project Durable Functions. However those orchestrations can run in serverless and can scale to zero during the “waiting for other tasks” step.
https://docs.microsoft.com/en-us/azure/azure-functions/durab...
Disclaimer - product manager for Azure durable functions
I'd love to add Azure support but I'm not super familiar with it. Would be great to chat about it sometime.
Absolutely! And awesome job with this - very cool to see it being shared with the community
Contact me via DM on twitter, or on linkedin