Hypothesis does what you’re talking about. It stores failing examples and chooses them during subsequent runs.