Is there some downside that I am not seeing with relation to using <button> instead of a <div>?

I have always used <button> tags (occasionally <a> tags as well). Along with a css reset and some custom changes I can make the button look like anything I want.