It is not a predisposition. You are using a term that was actually deprecated this year as synonym of "hyper focussed" individuals which is actually just an stereotype, most people on the spectrum aren't hyper-focussed.

As an example, homosexuality once was considered a mental disorder and it is full of stereotypes. And I am sure that homosexual people would be very offended if someone starts using the term as a synonym of hiper-sexuality.

Also I am very sure that someone would say that is a fact based on anecdotical experience, and hence is not derogatory.

For me a misuse of a term using it as a cliché is indeed derogatory.