REST vs GraphQL doesn't really matter for this type of app. Since you're restricting the choice between Phoenix/Elixir and Wt, I'd definitely go the former, but you'll have something out the door even faster with Ruby or Python. I'm not just speculating here, I worked with a C/C++ dev before when we used Ruby for a project and he was blown away at how much more productive he was.
As for CouchDB vs Postgres: "When in doubt, Postgres." Has never failed me. I burned a lot of time on other DB systems[0], but in the end Postgres was always easier for the whole picture of what a DB should do from backups to persistent to extensions, everything really. Never tried CouchDB though.
Anyway, seems like a good project to try out some new tech since it should be pretty easy. Whatever you choose should be fine for your first 100k users.
[0] Riak, Mongo, hand rolled custom graph DB based on BerkleyDB, tried to make Neo4J work, even wrote a persistence layer for NetworkX at one point.
What IS GraphQL good for? The only time it seems like it's needed is when you want to ensure your front end devs can work without understanding API code. Am I missing something?
P.S. not arguing - genuinely would like to know, since it never clicked.
P.S.S. Definitely use Postgres - "When in doubt, Postgres" is a good axiom unless you need something very specific and know what that is.
P.S.S.S. Elixir isn't that widely used. If you want something simple use FastAPI with Python or Flask. For real projects, .NET Core API will work well, but it's a little slower to set up if you don't use a template. Your C/C++ WILL carry you through C# though. This choice will REALLY depend on what language you want to learn - most of them have good API frameworks.
Say you have a REST interface for Student and Class.
What do you do in rest if you want to get just a class? What do you do if you want to get a class and its students? What do you do if you want to get a Student and also its class?
For every iteration of the way you need to access the data you have to modify and extend your REST endpoint or do separate queries. Every time the frontend devs want new data related to existing data, they have to do another query or ask the backend to include it.
Or lets say Class has 1000 attributes. Does your REST interface return all 1000 every time? Or can the frontend specify which attributes to load? Things like computed attributes and functions that return data related to the object. GraphQL allows this.
In graphql doing these relationships is dead easy. You define a class and how a student is related to it. Then you could even ask a query like “give me the classes of a student. For each class give me all the students in those classes. For each of those students, give me their classes”. Without the backend having to add anything. All the backend has to do is define the relationship and permissions about how to access the data.
Most GraphQL libs provide helpers for avoiding n+1 queries also, so things get optimized in the above case by only making 4 queries to the DB. Obviously thats a contrived example and most nested things wont go so crazy. But it goes to show how a powerful the API can become from defining such a simple relationship.
As a backend dev you just have to work on transforming data, defining the relationships and checking permissions. You let the frontend determine what data they want to access.
Before working with REST almost every page seemed to need a specialized endpoint to be implemented for the frontend to work, but with GraphQL, the frontend can work much more independently without having to ask the backend to provide a specialized view for more data.
I can write a Graphql endpoint in python (Using Django-Graphene). I can write python functions that return data to the graphql. I can write ORM queries that write data to grapqhl. I can specify access, permissions and certain queries based on the user.
Graphql is more of a relational FFI than a SQL replacement.
Also the frontend never sends SQL to get data because it is unsafe. This is what a GraphQL framework allows you to do, clean up and filter queries so they are safe and then translate them into efficient lookups in a RDBMS.
SQL is a leaky abstraction, which makes your system more brittle. The front-end now has intimate knowledge about the physical data management. You don't want that. If later you decide you want to change your physical data layout to improve data storage and retrieval performance then your queries have to be modified. The front-end and data tier can't vary independently from one another. That's what makes your system brittle.
That's not even getting into the tremendous attack surface you've just opened leaving your application vulnerable to SQL injection attacks. SQL injection is a real problem and scrubbing all the inputs is a pain and you can never be sure you got everything. One mistake and congratulations! - you've made headline news!
GraphQL forms a strongly and deliberately typed interface between a client and server, without frontend and backend teams having to collaborate on the shape of the endpoints (should such and such endpoint gain new options? should it be split into a separate endpoint? should the old one be deprecated, and if so, do clients still rely on it?).
Instead, they need only agree on the shape of the data available, and the frontend queries it however it needs to be queried while the backend resolves the nested calls with no additional effort* from those writing the backend.
Also, some clients come with additional affordances for hard problems, like pagination.
* okay, fine, you have to wrap your data resolution in Dataloaders, but that's not particularly challenging
* in my experience you basically have to write every call anyway, in other words, getting GraphQL set up on the backend is at least as much work as writing API boilerplate. Again, correct me if I am wrong. It's been a while since I last used GraphQL and I keep hearing about hearing about magic libraries that make it "effortless".
I abstain from writing graphql apis now because they’re too much work for me when toy apps are simple and I can render json.
But if a team does want graphql without the fuss of openapi, json schemas, and swagger docs, I recommend tools like Hasura that autogenerates graphql apis over my db. Auto generation of graphql resolvers are the way to go for productivity. Supabase even created a Postgres extension to perform graphql directly within postgres https://supabase.com/blog/2021/12/03/pg-graphql
CouchDB is a json document based noSql system that is exposed through traditional HTTP calls (e.g. GET to get a document, POST to create a new document, PUT to update a document, and DELETE to delete a document).
In the "expose data via REST" the stack is often "service doing SQL against a RDBMS" and that service is a either Java or Python or Ruby... that can often (not always) be done as "here is couch, with this data stored in it." It's memory and CPU footprint (even doing a 3x replication in our k8s environment) is a fraction of a Java container, much less a Java container and a database.
how to sync access control rules between couchdb and the rest of the business logic? do you just not care about security/privacy, or do you duplicate all the logic?
This isn't the right type of solution for all data - but there's a lot of situations where people have a bug lump of data that they're trying to expose... list of all current flights, or classes, or ongoing games.
It can be quite useful for a lot of use cases. One size never fits all.
Comments
REST vs GraphQL doesn't really matter for this type of app. Since you're restricting the choice between Phoenix/Elixir and Wt, I'd definitely go the former, but you'll have something out the door even faster with Ruby or Python. I'm not just speculating here, I worked with a C/C++ dev before when we used Ruby for a project and he was blown away at how much more productive he was.
As for CouchDB vs Postgres: "When in doubt, Postgres." Has never failed me. I burned a lot of time on other DB systems[0], but in the end Postgres was always easier for the whole picture of what a DB should do from backups to persistent to extensions, everything really. Never tried CouchDB though.
Anyway, seems like a good project to try out some new tech since it should be pretty easy. Whatever you choose should be fine for your first 100k users.
[0] Riak, Mongo, hand rolled custom graph DB based on BerkleyDB, tried to make Neo4J work, even wrote a persistence layer for NetworkX at one point.
What IS GraphQL good for? The only time it seems like it's needed is when you want to ensure your front end devs can work without understanding API code. Am I missing something?
P.S. not arguing - genuinely would like to know, since it never clicked.
P.S.S. Definitely use Postgres - "When in doubt, Postgres" is a good axiom unless you need something very specific and know what that is.
P.S.S.S. Elixir isn't that widely used. If you want something simple use FastAPI with Python or Flask. For real projects, .NET Core API will work well, but it's a little slower to set up if you don't use a template. Your C/C++ WILL carry you through C# though. This choice will REALLY depend on what language you want to learn - most of them have good API frameworks.
OK to give you a clear example.
Say you have a REST interface for Student and Class.
What do you do in rest if you want to get just a class? What do you do if you want to get a class and its students? What do you do if you want to get a Student and also its class?
For every iteration of the way you need to access the data you have to modify and extend your REST endpoint or do separate queries. Every time the frontend devs want new data related to existing data, they have to do another query or ask the backend to include it.
Or lets say Class has 1000 attributes. Does your REST interface return all 1000 every time? Or can the frontend specify which attributes to load? Things like computed attributes and functions that return data related to the object. GraphQL allows this.
In graphql doing these relationships is dead easy. You define a class and how a student is related to it. Then you could even ask a query like “give me the classes of a student. For each class give me all the students in those classes. For each of those students, give me their classes”. Without the backend having to add anything. All the backend has to do is define the relationship and permissions about how to access the data.
Most GraphQL libs provide helpers for avoiding n+1 queries also, so things get optimized in the above case by only making 4 queries to the DB. Obviously thats a contrived example and most nested things wont go so crazy. But it goes to show how a powerful the API can become from defining such a simple relationship.
As a backend dev you just have to work on transforming data, defining the relationships and checking permissions. You let the frontend determine what data they want to access.
Before working with REST almost every page seemed to need a specialized endpoint to be implemented for the frontend to work, but with GraphQL, the frontend can work much more independently without having to ask the backend to provide a specialized view for more data.
Isn't this exactly what SQL is supposed to be for? Why not just submit SQL queries and get the response directly from the DB?
That is to say, what differentiates GraphQL from SQL?
GraphQL is a layer in front of SQL.
I can write a Graphql endpoint in python (Using Django-Graphene). I can write python functions that return data to the graphql. I can write ORM queries that write data to grapqhl. I can specify access, permissions and certain queries based on the user.
Graphql is more of a relational FFI than a SQL replacement.
Also the frontend never sends SQL to get data because it is unsafe. This is what a GraphQL framework allows you to do, clean up and filter queries so they are safe and then translate them into efficient lookups in a RDBMS.
SQL is a leaky abstraction, which makes your system more brittle. The front-end now has intimate knowledge about the physical data management. You don't want that. If later you decide you want to change your physical data layout to improve data storage and retrieval performance then your queries have to be modified. The front-end and data tier can't vary independently from one another. That's what makes your system brittle.
That's not even getting into the tremendous attack surface you've just opened leaving your application vulnerable to SQL injection attacks. SQL injection is a real problem and scrubbing all the inputs is a pain and you can never be sure you got everything. One mistake and congratulations! - you've made headline news!
These are the problems GraphQL solves.
GraphQL forms a strongly and deliberately typed interface between a client and server, without frontend and backend teams having to collaborate on the shape of the endpoints (should such and such endpoint gain new options? should it be split into a separate endpoint? should the old one be deprecated, and if so, do clients still rely on it?).
Instead, they need only agree on the shape of the data available, and the frontend queries it however it needs to be queried while the backend resolves the nested calls with no additional effort* from those writing the backend.
Also, some clients come with additional affordances for hard problems, like pagination.
* okay, fine, you have to wrap your data resolution in Dataloaders, but that's not particularly challenging
* in my experience you basically have to write every call anyway, in other words, getting GraphQL set up on the backend is at least as much work as writing API boilerplate. Again, correct me if I am wrong. It's been a while since I last used GraphQL and I keep hearing about hearing about magic libraries that make it "effortless".
I abstain from writing graphql apis now because they’re too much work for me when toy apps are simple and I can render json.
But if a team does want graphql without the fuss of openapi, json schemas, and swagger docs, I recommend tools like Hasura that autogenerates graphql apis over my db. Auto generation of graphql resolvers are the way to go for productivity. Supabase even created a Postgres extension to perform graphql directly within postgres https://supabase.com/blog/2021/12/03/pg-graphql
Thanks for the info!
Couch db is awesome but as my most recent couch app got big I ended up moving to pg. go pg.
CouchDB is a json document based noSql system that is exposed through traditional HTTP calls (e.g. GET to get a document, POST to create a new document, PUT to update a document, and DELETE to delete a document).
In the "expose data via REST" the stack is often "service doing SQL against a RDBMS" and that service is a either Java or Python or Ruby... that can often (not always) be done as "here is couch, with this data stored in it." It's memory and CPU footprint (even doing a 3x replication in our k8s environment) is a fraction of a Java container, much less a Java container and a database.
Grab a couchdb docker compose and spin it up and give it a try. It has a built in admin console. https://docs.couchdb.org/en/3.2.2/index.html
how to sync access control rules between couchdb and the rest of the business logic? do you just not care about security/privacy, or do you duplicate all the logic?
You can set up design documents that implement access control logic. These are written in JavaScript.
https://docs.couchdb.org/en/3.2.0/ddocs/ddocs.html (note the design document for the users database)
This isn't the right type of solution for all data - but there's a lot of situations where people have a bug lump of data that they're trying to expose... list of all current flights, or classes, or ongoing games.
It can be quite useful for a lot of use cases. One size never fits all.