The baseline standards for natural selection being cruel are de novo genetic diseases, not some people being short. Nature has no way to tell whether or not a gene sequence will kill a 20 year old except wrapping a person around it and shooting it out.