Comment on Large Concept Models: Language modeling in a sentence representation spaceparentComments−mdp20211ymulti token prediction of sufficient lengthIs multi token prediction the same as predicting the embedding of a complex token (the articulation of those input tokens in a sentence)?−blackeyeblitzar1yTo be honest I don’t know. Maybe the only way to know is to build and measure all these variations.
Comments
Is multi token prediction the same as predicting the embedding of a complex token (the articulation of those input tokens in a sentence)?
To be honest I don’t know. Maybe the only way to know is to build and measure all these variations.