- The model is said to be rewarded/penalized based on answers, and the plain meaning would leave someone with the impression of "answer to the question asked/prompt", the visual appears correct (the chalkboard indicates the question is "fill in the next token" and the answer is "the next most likely token")
- You end up with a strong sense there's 3 tables in the model called key, query, and value that carry the weight of the world. Are there only 3?
- "what part of data are we processing and are they relevant?" - what is they? what part of data is it processing? what is data in this context?
- "how well does this data answer my question?" - so the model is picking out answers from the training data and checking if it answers the prompt? This creates a strong sense of copying verbatim from training data
- "how should we improve the contextual representation of the input data?" - "contextual representation of the input data" isn't clear here, in my dummy brain, it's "here's information we can use to decide the next token: in this context, they meant cat cli, not cat the animal"
- "training a model is like doing Q&A with the neutral networks while it attends to the right data": Is it Q&A? If we double down on that (which means also doubling down on the idea being finding the answer in the training data, I don't think it's a good idea), then there's a gap in how the sentence connects within itself that's worth addressing. I assume "like doing Q&A with the neutral networks, and based on whether the answer is correct or not, it'll adjust and learn to attend to the right data"
Comments
Specific feedback:
- The model is said to be rewarded/penalized based on answers, and the plain meaning would leave someone with the impression of "answer to the question asked/prompt", the visual appears correct (the chalkboard indicates the question is "fill in the next token" and the answer is "the next most likely token")
- You end up with a strong sense there's 3 tables in the model called key, query, and value that carry the weight of the world. Are there only 3?
- "what part of data are we processing and are they relevant?" - what is they? what part of data is it processing? what is data in this context?
- "how well does this data answer my question?" - so the model is picking out answers from the training data and checking if it answers the prompt? This creates a strong sense of copying verbatim from training data
- "how should we improve the contextual representation of the input data?" - "contextual representation of the input data" isn't clear here, in my dummy brain, it's "here's information we can use to decide the next token: in this context, they meant cat cli, not cat the animal"
- "training a model is like doing Q&A with the neutral networks while it attends to the right data": Is it Q&A? If we double down on that (which means also doubling down on the idea being finding the answer in the training data, I don't think it's a good idea), then there's a gap in how the sentence connects within itself that's worth addressing. I assume "like doing Q&A with the neutral networks, and based on whether the answer is correct or not, it'll adjust and learn to attend to the right data"
THANK YOU!!! this is amazing. Will update the cartoons over next week.
Also thanks so much for taking the time to write these down!! Can't appreciate it enough
No problem at all, all the best, thank you for your hard work :) This isn't easy and it's extremely valuable