Skip to content

Comment on Postgres as queue

Comments

The idea that "performance" is the big problem, is the same as saying "performance" is why you don't use an IMAP server as a queue. Hey, you've already got an email server somewhere, and that means it's simpler, so that means it's a good idea. Based on the logic in this article you should be using your email server as your application's general purpose queue.

I feel so bad for the poor engineers that will believe this crap and later regret it. If you're going to advocate doing something like this, you should be forced to be honest and explain all the reasons it's bad idea. The fact is that people are lying about the problems of using Postgres as a queue, because they just don't want to believe that the world is really more complex than that.

But you aren’t storing application data in the IMAP server. I hope.

The idea (for me anyway) is that if you have a postgres db with data, and need a queue for tasks dealing with that data, building it into the existing db can make more sense than adding additional infrastructure and complexity. Especially if you are familiar with postgres already.

If all you need is a queue, then I agree that dedicated queue software will be better.

That's the fallacy. That keeping it all in one place with one tool is somehow better, because you can, or because it's all tightly coupled, or because it's one less piece of software. But in practice it just creates more problems (lots of them) that a dedicated, separate queue does not have.

If you want a very simple way of looking at it, think about how code becomes modular. You don't put all your code in one function, or one class, or one file, or even one application. Not just because it's easier to read, but because specific components should have clearly defined delineations from others, and forceably be kept separate at the borders of their functionality. Doing that leads to benefits that outweigh them sitting next to each other, tightly tied into each other.

At the very least, you should not be using one giant database for all your data in all your applications. Everybody should know that by now. And just like you shouldn't conflate different applications' data containers/interfaces/internals, you shouldn't conflate a queueing system with a completely separate data container. They need to be separate to avoid a bunch of problems. Again, not because you can't do it - like keeping all your code in a single function, or class - but because it's going to lead to a bad time.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.