If Salinger had decided he never wanted any of his books published and no one should ever read them, that's his choice, not ours.

Absolutely. Until Salinger hits the metaphorical "publish" button, his books belong to him and no one else. But the minute he decides to share them with the world he enters into the realm of copyright law, which is (at least originally) designed to create a good enough bargain to incentivise him to publish, but which ultimately results in all these works entering the public domain.

There is no such thing as intellectual property - merely intellectual licensing agreements.