Glad there's prior art on that. I had a small project where I iterated all the fonts on the system and used them to generate glyph training images. The next step was to dirty them up, but I never continued the project.

More generally, I really like the idea of generating controlled synthetic images and then messing them up for regularization.