Making something synchronous when it involves a job queue sounds like a recipe for disaster, IMO. Better to let both the consumer and producer act in a fire-and-forget manner with the original consumer producing a reply on a second queue which the original producer will eventually handle.
Our use case necessitated a synchronous interaction, but the message back was done exactly as you suggest - just uses a private second queue. The challenge was that while the initial job is fire-and-forget (or wait for a reply), the 2nd private queue was just a placeholder for a reliable messaging backend that needed to be implemented. It doesn't matter what worker gets the job, but it matters to whom the consumer of the job replies. I believe Redis provides enough primitives to make a highly reliable messaging system. We just have not done it.
Comments
Making something synchronous when it involves a job queue sounds like a recipe for disaster, IMO. Better to let both the consumer and producer act in a fire-and-forget manner with the original consumer producing a reply on a second queue which the original producer will eventually handle.
Our use case necessitated a synchronous interaction, but the message back was done exactly as you suggest - just uses a private second queue. The challenge was that while the initial job is fire-and-forget (or wait for a reply), the 2nd private queue was just a placeholder for a reliable messaging backend that needed to be implemented. It doesn't matter what worker gets the job, but it matters to whom the consumer of the job replies. I believe Redis provides enough primitives to make a highly reliable messaging system. We just have not done it.