No, that's not rude, it's honesty.

For what it's worth, Godaddy Broker has contacted me in the past to buy this domain name for a client.

The past is the past and you still don't have that offer do you? So it doesn't mean anything.

You are telling me what to do or not do with my own free time!

No, he gave you a suggestion, and his opinion.