I'm mostly curious about the details behind the flat-file persistence. Is the storage on the same physical machine as the API and Business Logic? If not how is communication done? How do you handle horizontal scalability for either perf or HA reasons?
Hi! That’s actually a good question, an aspect that I basically avoided in the article. Truth be told, we aren’t happy with the way things are, but indeed the storage is on the same physical machine (in fact a shared docker volume) as everything else. We have a read-only replica that we use sometimes to execute slow queries.
However, the vision is that eventually we would use Raft or other kinds of consensus on top of the current layer of storage. Event sourcing supposedly should make this kind of replication easy because data is always append-only. Unfortunately the reality of a startup is that we often have to do things quickly and this aspect has been neglected for pretty long.
Comments
Hey!
I'm mostly curious about the details behind the flat-file persistence. Is the storage on the same physical machine as the API and Business Logic? If not how is communication done? How do you handle horizontal scalability for either perf or HA reasons?
Hi! That’s actually a good question, an aspect that I basically avoided in the article. Truth be told, we aren’t happy with the way things are, but indeed the storage is on the same physical machine (in fact a shared docker volume) as everything else. We have a read-only replica that we use sometimes to execute slow queries.
However, the vision is that eventually we would use Raft or other kinds of consensus on top of the current layer of storage. Event sourcing supposedly should make this kind of replication easy because data is always append-only. Unfortunately the reality of a startup is that we often have to do things quickly and this aspect has been neglected for pretty long.