Not everything has to be directly informative or solve a problem. Sometimes data visualization can look pretty for pretty's sake.
Dimensionality reduction/clustering like this may be less useful for identifying trends in token embeddings, but for other types of embeddings it's extremely useful.
Agreed. The fact that it has any structure at all is fascinating (and super pretty). Could signal at interesting internal structures. I would love to see a version for Qwen-3 and Mistral too!
I wonder if being trained on significant amounts of synthetic data gave it any unique characteristics.
Comments
Not everything has to be directly informative or solve a problem. Sometimes data visualization can look pretty for pretty's sake.
Dimensionality reduction/clustering like this may be less useful for identifying trends in token embeddings, but for other types of embeddings it's extremely useful.
Agreed. The fact that it has any structure at all is fascinating (and super pretty). Could signal at interesting internal structures. I would love to see a version for Qwen-3 and Mistral too!
I wonder if being trained on significant amounts of synthetic data gave it any unique characteristics.