The original MRL paper (https://arxiv.org/pdf/2205.13147) reported an SVD baseline, which showed comparable performance (Table 1, top-1 accuracy) at d>=256, but much degraded performance at lower dims (d \in {8,32,64}). (Though Table 2, nearest-neighbor accuracy, doesn't show degradation until d <= 16.)
In your conclusion, you report that PCA won on most dimensions. Did you investigate why you found that PCA outperforms MRL when the original paper found that their SVD baseline did not?
Comments
The original MRL paper (https://arxiv.org/pdf/2205.13147) reported an SVD baseline, which showed comparable performance (Table 1, top-1 accuracy) at d>=256, but much degraded performance at lower dims (d \in {8,32,64}). (Though Table 2, nearest-neighbor accuracy, doesn't show degradation until d <= 16.)
In your conclusion, you report that PCA won on most dimensions. Did you investigate why you found that PCA outperforms MRL when the original paper found that their SVD baseline did not?
I also found that interesting but our evaluation methodology is quite different, so I didn’t go too deep into it
Yeah...so I'm having difficulty conceptualizing your result. If you were publishing this I feel like it would be an important distinction to make.
Are the goals different, or should the original paper have done something more similar to your benchmark? Or something else?