You can supply the model with a list of facts already, that’s not the problem. Within the context window the model is able to learn and generalize new information.
Fine tuning is very unintelligent in the sense that it doesn’t take the context of the training samples into account. It’s a dumb optimizer that’s trying to minimize next token loss. Gradient descent is not beholden to the behaviors you taught in the instruct fine tune step.
Comments
What if we just train it to respect facts in general, then couldn't we just supply it a list of facts?
Sort of how they made chatGPT way more likely to obey requests?
You can supply the model with a list of facts already, that’s not the problem. Within the context window the model is able to learn and generalize new information.
Fine tuning is very unintelligent in the sense that it doesn’t take the context of the training samples into account. It’s a dumb optimizer that’s trying to minimize next token loss. Gradient descent is not beholden to the behaviors you taught in the instruct fine tune step.