niobe's excellent reply covered it already, but just to be blunt: You usually share the channel with some of your neighbors' networks, so the assessment that only you are using it is usually not correct.

This is also why it's often better if everyone uses lower transmit power (while still retaining coverage), as networks farther away will see less interfering networks.