I suspect it's not so much the "expert" instruction but the list of subjects of the expertise. These words will generate embeddings that have a better chance of activating useful pathways and relationships within the LLM's generation path.
My understanding is the goal is to prime the LLM with context that it will use when generating answers, rather than hoping it will infer connections in the feed forward layers when generating answers based on a sparse prompt.
Comments
I suspect it's not so much the "expert" instruction but the list of subjects of the expertise. These words will generate embeddings that have a better chance of activating useful pathways and relationships within the LLM's generation path.
My understanding is the goal is to prime the LLM with context that it will use when generating answers, rather than hoping it will infer connections in the feed forward layers when generating answers based on a sparse prompt.