Keeping in mind that I've never built anything that is Rapportive-sized, it seems that the problems Martin is talking about here can be mitigated thus.
1. Never access models directly from controllers. Build an API layer that exposes discrete methods which can store and retrieve data, and use these exclusively from the higher layers of your app. Using the example given in the article, (using a hypothetical Python + SQLAlchemy app):
post = api.get_post(id=post_id)
posts_by_date = api.get_posts(before=end_date, after=start_date)
posts_by_author = api.get_posts(author_id=user_id)
2. APIs should not use auto-querying collections whenever possible, and should accept configurable options to use an abstract storage class representing tables, buckets, interfaces to externals services or whatever.
Essentially, the idea is to channel access to the data store through only discrete paths. First, split the model layer into methods that support higher layer needs -- these are what Martin is calling the dependencies I think. Identify methods like `get_post` which contains logic and information such as how to get posts by id, and whether or not comments should be eager loaded.
Second, abstract the actual dispatch of queries to the data store into domain-specific stores. Instead of using a generic data model, write stores that know about Posts, Comments, Users etc and wrap the generic model classes. In this way, a RelationalPostStore knows what it means to get a post from the store, along with its comments, users, author information etc. A quickly changed user-configurable setting can switch that out with an HDFSPostStore, which knows how to get those objects from a Hadoop backend. A CouchPostStore can do the same, etc.
This is the pattern that has been emerging through my own repeated web dev experiences. I'd be interested to know if there are obvious/subtle improvements or flaws.
That's what models are for though. The problem is your models are acting the part of light wrappers over a database. That's not what models are, and treating them like that is bad.
post = api.get_post(id=post_id)
It makes no sense to use api. Following the same style:
post = Post.get(post_id)
Post.get will figure get the data from whatever underlying data source is provided, whether that data source is a 3rd party API, the database, or a cache. But the model isn't loading that data, it's just requesting it, and once it's given that data, it returns it properly, making sure that regardless of the source, it's always the same format.
+1, that's exactly the pattern I've been exploring too. Here's an example project which illustrates the division of access between the api, model, and view layers: https://github.com/shazow/pyramid-sampleapp
All remote calls and fancy caching gets added to the api module so that neither the model nor other consumers, like the view, need to be aware of it. Also this keeps a stable interface for consumers to build on while the model and view evolves (great for unit tests, other apps using your app, etc).
Agree, this is going in the same direction. How models and data stores are structured will always depend on the particular application. But your point of "channeling access to the data store through only discrete paths" is very like the "separating communication logic from business logic" that I argue for.
However, I would like to take it further. Even if you move the data store access into a separate API layer, that API is still written in an imperative language, which limits the transformations that a framework can perform on it. I dream of a fully declarative way of specifying the communication dependencies, which would open up completely new processing modes, such as running all the business logic in Hadoop.
Somehow I think adding yet another layer to software that ultimately consumes some inputs and spits out some text is a little extreme. Hopefully we can figure out a solution that doesn't require it.
1) Which software does something other than merely operating on given inputs and deterministically generating text, graphics, or more generally, colored squares? I think that description fully characterizes all non-hardware-driving code.
2) Web apps do more than generate text. They modify stored data in domain-specific ways, and ultimately that data is what's important -- it's everything from opinions to orders.
Comments
Keeping in mind that I've never built anything that is Rapportive-sized, it seems that the problems Martin is talking about here can be mitigated thus.
1. Never access models directly from controllers. Build an API layer that exposes discrete methods which can store and retrieve data, and use these exclusively from the higher layers of your app. Using the example given in the article, (using a hypothetical Python + SQLAlchemy app):
This:
Becomes: 2. APIs should not use auto-querying collections whenever possible, and should accept configurable options to use an abstract storage class representing tables, buckets, interfaces to externals services or whatever.So, this:
becomes: Essentially, the idea is to channel access to the data store through only discrete paths. First, split the model layer into methods that support higher layer needs -- these are what Martin is calling the dependencies I think. Identify methods like `get_post` which contains logic and information such as how to get posts by id, and whether or not comments should be eager loaded.Second, abstract the actual dispatch of queries to the data store into domain-specific stores. Instead of using a generic data model, write stores that know about Posts, Comments, Users etc and wrap the generic model classes. In this way, a RelationalPostStore knows what it means to get a post from the store, along with its comments, users, author information etc. A quickly changed user-configurable setting can switch that out with an HDFSPostStore, which knows how to get those objects from a Hadoop backend. A CouchPostStore can do the same, etc.
This is the pattern that has been emerging through my own repeated web dev experiences. I'd be interested to know if there are obvious/subtle improvements or flaws.
> Never access models directly from controllers.
That's what models are for though. The problem is your models are acting the part of light wrappers over a database. That's not what models are, and treating them like that is bad.
It makes no sense to use api. Following the same style: Post.get will figure get the data from whatever underlying data source is provided, whether that data source is a 3rd party API, the database, or a cache. But the model isn't loading that data, it's just requesting it, and once it's given that data, it returns it properly, making sure that regardless of the source, it's always the same format.+1, that's exactly the pattern I've been exploring too. Here's an example project which illustrates the division of access between the api, model, and view layers: https://github.com/shazow/pyramid-sampleapp
All remote calls and fancy caching gets added to the api module so that neither the model nor other consumers, like the view, need to be aware of it. Also this keeps a stable interface for consumers to build on while the model and view evolves (great for unit tests, other apps using your app, etc).
Further, this lets you reuse the same "business logic" in different frontend scenarios, such as exposing parts of the API over HTTP: https://github.com/shazow/pyramid-sampleapp/blob/master/foo/...
Agree, this is going in the same direction. How models and data stores are structured will always depend on the particular application. But your point of "channeling access to the data store through only discrete paths" is very like the "separating communication logic from business logic" that I argue for.
However, I would like to take it further. Even if you move the data store access into a separate API layer, that API is still written in an imperative language, which limits the transformations that a framework can perform on it. I dream of a fully declarative way of specifying the communication dependencies, which would open up completely new processing modes, such as running all the business logic in Hadoop.
Somehow I think adding yet another layer to software that ultimately consumes some inputs and spits out some text is a little extreme. Hopefully we can figure out a solution that doesn't require it.
Two thoughts:
1) Which software does something other than merely operating on given inputs and deterministically generating text, graphics, or more generally, colored squares? I think that description fully characterizes all non-hardware-driving code.
2) Web apps do more than generate text. They modify stored data in domain-specific ways, and ultimately that data is what's important -- it's everything from opinions to orders.