Ah right, thank you for clarifying.

But I think my point still stands, isn't the geometry information THE information I referred to in the first place? Obviously the vector size gives you the granularity but it's kind of unavoidable to positionally encode information in a latent space...that's literally what they're for?

But yes, it is very cool to know that regardless of exact implementation finding x,y,z representations of some dataset with various relationships (like language) creates similar geometry/clues across all the implementations.