No, you are not storing the job itself, just a flag that indicates that the job was not done/or even stored
You must be storing enough representation of the job to re-send it though, no? You couldn't do this with a simple flag for all job types I imagine.
in most cases it's just the absence of the desired effect that can be a giveaway
Sure, but not all jobs create side effects you can easily look for (e.g. sending an email). And secondly, you'd have to create a "sweeper" process for every type of job you create then.
At the point where all the other "normal" mechanisms monitoring storing the job on the queue have failed I would suppose I have a far more greater problem on that system. Personally I would not use flags for jobs types, I would monitor them with flower. Also, we need to keep in mind that it's one thing retrying sending a job to the queue and a whole other thing retrying a job already on the queue that failed for some reason. In my experience so far there a mechanisms that deal with both of these issues effectively and none involved using the webapps database for this.
Thanks, I will take a look at the Pythonic way of doing these things. Celery does look a lot more industrial-grade than most of the Ruby solutions so my curiosity is piqued.
Comments
You must be storing enough representation of the job to re-send it though, no? You couldn't do this with a simple flag for all job types I imagine.
Sure, but not all jobs create side effects you can easily look for (e.g. sending an email). And secondly, you'd have to create a "sweeper" process for every type of job you create then.
At the point where all the other "normal" mechanisms monitoring storing the job on the queue have failed I would suppose I have a far more greater problem on that system. Personally I would not use flags for jobs types, I would monitor them with flower. Also, we need to keep in mind that it's one thing retrying sending a job to the queue and a whole other thing retrying a job already on the queue that failed for some reason. In my experience so far there a mechanisms that deal with both of these issues effectively and none involved using the webapps database for this.
Thanks, I will take a look at the Pythonic way of doing these things. Celery does look a lot more industrial-grade than most of the Ruby solutions so my curiosity is piqued.
Feel free to ping me if you have any questions. I will do my best and try and answer them if I can.