LTO could handle this if you're compiling an executable, but not a library.

That's not the correct distinction, that's why I said default vs hidden visibility.

Libraries typically export more symbols but executables can also export them eg for plugins.