Comment on Large Concept Models: Language modeling in a sentence representation spaceparentComments−IanCal1yIs that true? I thought there was a desire to move towards byte level work rather than tokens, and that the benefits of tokens was more that you are reducing the context size for the same input.−fngjdflmdflg1ythere was a desire to move towards byte level work rather than tokensYeah, latest work on this is from Meta a last month.[0] It showed good results.[0] https://ai.meta.com/research/publications/byte-latent-transf... (https://news.ycombinator.com/item?id=42415122)
Comments
Is that true? I thought there was a desire to move towards byte level work rather than tokens, and that the benefits of tokens was more that you are reducing the context size for the same input.
Yeah, latest work on this is from Meta a last month.[0] It showed good results.
[0] https://ai.meta.com/research/publications/byte-latent-transf... (https://news.ycombinator.com/item?id=42415122)