The heuristics are generally implemented by an algorithm though. I think that is where the confusion arises.
In other words, you've got an algorithmic step that produces some output, followed by a heuristic step that ascribes some meaning to the output.
For example, consider spam detection with Naive Bayes. The parts that tokenize each email, accumulate the counts of each token, and estimate their conditional probability given the class are all algorithmic, even under the formal definition given by @zero_iq, below). Each of these steps is deterministic, finite, etc. The same holds for the part where you calculate the probability of a new email under each class, assuming that each token is independent (it's just multiplication!).
The heuristic part is the assumption that these numbers are meaningful (training data is representative, pre-processing is reasonable, independence assumption isn't bonkers, etc.) and where you choose to set a decision boundary for actually classifying each item.
Comments
The heuristics are generally implemented by an algorithm though. I think that is where the confusion arises.
In other words, you've got an algorithmic step that produces some output, followed by a heuristic step that ascribes some meaning to the output.
For example, consider spam detection with Naive Bayes. The parts that tokenize each email, accumulate the counts of each token, and estimate their conditional probability given the class are all algorithmic, even under the formal definition given by @zero_iq, below). Each of these steps is deterministic, finite, etc. The same holds for the part where you calculate the probability of a new email under each class, assuming that each token is independent (it's just multiplication!).
The heuristic part is the assumption that these numbers are meaningful (training data is representative, pre-processing is reasonable, independence assumption isn't bonkers, etc.) and where you choose to set a decision boundary for actually classifying each item.