ML breaks down such classifications. Finding a fruit without having already figured out that it's a banana is pretty hard.

Can you elaborate? If I train a fruit detector that just gets the fruits bounding box vs a fruit detector and classifier, are you saying the latter will work better?