I think it's probably more like an "incorrect" focal length for what you'd expect. And it's kind of hard to know what you'd expect because you don't typically have arbitrary spatial control over your point-of-view, so some POVs you're used to only seeing through cameras (or even certain types of cameras), while others you're mostly used to seeing through your own eyes.

I imagine all of this will get better though.