No it isn't a zero-sum game... but this isn't "invading" their network.

If they connected to the AP and ran nmap against my server yes it would be like that... but simply capturing packets that are broadcast into a public space I don't think is reasonable or realistic to expect it is some grave violation of privacy.