It's not just that it triggers anxieties in some situations, but even when it doesn't, it is exhausting

Yes. For me, it's that small talk is so boring. Even not-so-small talk gets dull to me. Like, I recently took a trip to Vietnam for two weeks. Obviously everybody wants to hear about it, which I know at some level is because they like me and are interested in my life, but I am now just sick of talking about it! I've been telling the same stories over and over, and it just wears me out now to talk about it.

So I think extraverts love the opportunity to tell stories, even well-trod ones, whereas we introverts just abhor it.