Actually, that is the crux of the matter. If you're at a party and making an obnoxious ass of yourself, the other partygoers can't kick you out. What they can do is go to the host and ask that he kick you out, which he can do if he so decides. If the host happens to be you (or someone sympathetic to you), then their only recourse (beyond attempting to persuade you to cut it out) is to leave themselves. The avenues of legal recourse are entirely determined by ownership of the piece of ground they are standing on.