My suspicion is that the spam filter programmers didn't do a comprehensive evaluation of every code point on every plane of Unicode because... well that's a massive job. So your "sanitize and denormalize" tasks are actually massive mappings which were likely imperfectly created.
I'm basing my comment on the article. Since they are susceptible to ASCII smuggling, they can't have been doing unicode normalization.
The task isn't that massive. Python's unicodedata (for example) contains all the info already. Including the normalization (not denormalization as I wrote before) function.
Comments
Do you have any evidence they didn't?
My suspicion is that the spam filter programmers didn't do a comprehensive evaluation of every code point on every plane of Unicode because... well that's a massive job. So your "sanitize and denormalize" tasks are actually massive mappings which were likely imperfectly created.
I'm basing my comment on the article. Since they are susceptible to ASCII smuggling, they can't have been doing unicode normalization.
The task isn't that massive. Python's unicodedata (for example) contains all the info already. Including the normalization (not denormalization as I wrote before) function.
Is there something I'm missing?