Agentic behavior is instilled in RLHF'd models [0, 1]. In this paper, "agency" is defined in terms of the LLM thinking like it's a person (agent) with consistent thoughts (outputs). The problem is this also severely limits the LLM's ability to break out from the norm and explore unknown paths (thoughts).
Most (if not all) agent frameworks use GPT-4, Claude Opus, etc. models which are heavily RLHF'd.
Comments
Agentic behavior is instilled in RLHF'd models [0, 1]. In this paper, "agency" is defined in terms of the LLM thinking like it's a person (agent) with consistent thoughts (outputs). The problem is this also severely limits the LLM's ability to break out from the norm and explore unknown paths (thoughts).
Most (if not all) agent frameworks use GPT-4, Claude Opus, etc. models which are heavily RLHF'd.
[0]: https://arxiv.org/abs/2406.05587 [1]: https://news.ycombinator.com/item?id=40702617