While I agree with the second part, I think it's a fallacy to connect it with a fictitious group of people labelled "drug users".

True, there is a class of people who "use" drugs and a class of people "abuse" drugs, but in the common vocabulary, there's unfortunately no distinction between the two. And there really ought to be.

We (society) get all of the negative consequences of drug abuse and none of the positive consequences of responsible drug use.