Using databases as a queue is not an anti-pattern (delayed_job is an example).
A full blown message queue system is often overkill; it adds another system that you have to learn. (And frankly, queues are often backed by databases anyway)
That said, it's best to avoid having application state in the queue, it should be generic. This makes it more scalable and flexible.
Also, if your using the DB as a queue index on the status element. Inserts are still low overhead and the DB can handle poling that table 10,000 time a second without issue.
Edit: Just don't pole the 'finished' status as that can have a lot of elements in it.
A lot of databases can also provide partial indices, which are only maintained and searched when a WHERE clause matches (example: in this case, WHERE status != 'filtered' would be a good choice).
Comments
Using databases as a queue is not an anti-pattern (delayed_job is an example).
A full blown message queue system is often overkill; it adds another system that you have to learn. (And frankly, queues are often backed by databases anyway)
That said, it's best to avoid having application state in the queue, it should be generic. This makes it more scalable and flexible.
Also, if your using the DB as a queue index on the status element. Inserts are still low overhead and the DB can handle poling that table 10,000 time a second without issue.
Edit: Just don't pole the 'finished' status as that can have a lot of elements in it.
A lot of databases can also provide partial indices, which are only maintained and searched when a WHERE clause matches (example: in this case, WHERE status != 'filtered' would be a good choice).