If you have a tactful manager, bring it up in your next one on one. Your manager is in a unique position to handle these sorts of issues because they can gently probe for information and give more unbiased feedback than your coworkers.

That said, clumsy help is worse than none. If your manager is not particularly political then you can disregard that advice entirely.