It seems that Aphantasia does not globally bin into two groups, since I don't fit in either.

I believe you’re correct, since I commonly see it presented as a spectrum. https://aphantasia.com/study/vviq/