I scrolled straight to section 2.1 "Definition and Characteristics of LLM agents" to find out which of the many definitions of "agent" they are using here.
They went for LLM + short-term and long-term memory + planning + tool using + action execution.
Presumably "planning" here is covered by any LLM that can do "think step by step" reasonably well?
It wasn't clear to me what the difference between "tool using" and "action execution" was.
I haven't seen a definition that specifically encompasses both short- and long-term memory before. They say:
This dual memory system allows agents to maintain conversation continuity while building knowledge over time.
So presumably, this is the standard LLM chat conversation log plus a tool that can decide to stash extra information in a permanent store - similar to how ChatGPT's memory feature worked up until about four weeks ago.
Presumably "planning" here is covered by any LLM that can do "think step by step" reasonably well?
mild disagree. 1) externalizing the plan and letting the user audit/edit the plan while its working is "tool use", yes, but a very specialcase kind of tool use that, for example, operator and deep research use Temporal for. ofc we also saw this with Devin/Manus and i kinda think they're better 2) there is a form of primitive tree search that people are doing where they can spam out several different paths and run it a few steps ahead to gain information about optimal planning. You will see this with morph's launch at AIE. 3) plan meta reflection and reuse - again a form of tool use, but the devin and allhands folks have worked on this a lot more than most.
my criticism of many agent definitions is that they generally do not take memory, planning, and auth seriously enough, and i think those 3 areas are my current bets for "alpha" in 2025.
I haven't seen a definition that specifically encompasses both short- and long-term memory before.
Accurate memory access across memory types is still not really solved. That is the issue. Most agent frameworks from the main model providers are still quite spotty.
It wasn't clear to me what the difference between "tool using" and "action execution" was.
doing a lot of inference here, but could be a separation between -read- tool kinda actions, and -write/execute- (like running code/sending an email, etc)
They added a feature where it can reference content from any conversation you've had with it in the past. It's not clear how this works - probably some kind of vector search?
Here is what I have been able to reverse engineer for o3...
At high level it maintains about ~40 conversations in system prompt under a section called "recent conversation content". It only contains what the user typed, not assistant responses (probably due to prompt injection) - there a few corner cases though. :)
There are other sections in the system prompt now that contain aggregated info, so recent conversations turn into user insights over time I believe.
It can't actually "search" history afaik - that part I'm still wondering, as it was my first thought on how it might work...
I also found a way to exfiltrate the recent content - so hopefully that will be fixed soon...
Overall, this feature creates a lot of confusion and response quality declines at times too - and anything someone posts now online (like weird behavior or hallucinations,...) is likely influenced by their past conversations! So it will make it more difficult to understand what's really happening.
I think it would be cool if "projects" would be entirely isolated with their own memories and history etc. or have different "profiles"
If I paste a huge article in for it to summarize presumably it's smart enough not to keep dumping that into my future context?
I'd love a version of this that was tied to projects - then I could maintain way more control over my context without worrying that weird stupid stuff was leaking into my real work.
Yeah, the number of ~40 needs a bit more validation. I did observe the list being trimmed around 40, which aligns with the title "recent conversations content".
You can try simple repros like: 'list all "recent conversation content" entries', or 'how many "recent conversation content entries" are there above'...
it has timestamp, summary and then all the messages the user typed if you ask for the details.
Comments
I scrolled straight to section 2.1 "Definition and Characteristics of LLM agents" to find out which of the many definitions of "agent" they are using here.
They went for LLM + short-term and long-term memory + planning + tool using + action execution.
Presumably "planning" here is covered by any LLM that can do "think step by step" reasonably well?
It wasn't clear to me what the difference between "tool using" and "action execution" was.
I haven't seen a definition that specifically encompasses both short- and long-term memory before. They say:
So presumably, this is the standard LLM chat conversation log plus a tool that can decide to stash extra information in a permanent store - similar to how ChatGPT's memory feature worked up until about four weeks ago.
mild disagree. 1) externalizing the plan and letting the user audit/edit the plan while its working is "tool use", yes, but a very specialcase kind of tool use that, for example, operator and deep research use Temporal for. ofc we also saw this with Devin/Manus and i kinda think they're better 2) there is a form of primitive tree search that people are doing where they can spam out several different paths and run it a few steps ahead to gain information about optimal planning. You will see this with morph's launch at AIE. 3) plan meta reflection and reuse - again a form of tool use, but the devin and allhands folks have worked on this a lot more than most.
my criticism of many agent definitions is that they generally do not take memory, planning, and auth seriously enough, and i think those 3 areas are my current bets for "alpha" in 2025.
here
- https://docs.mem0.ai/core-concepts/memory-types#short-term-m...
- https://x.com/swyx/status/1915128966203236571
what is morphs launch at AIE?
Morph.so is an ai sandbox thing that has branching from state, prob doing a talk in SF on this at AI Engineer conf
thanks
Accurate memory access across memory types is still not really solved. That is the issue. Most agent frameworks from the main model providers are still quite spotty.
Image of a table outlining where the major frameworks are: https://substackcdn.com/image/fetch/w_1272,c_limit,f_webp,q_...
Here is also an article I wrote last year on the different types of memory: https://open.substack.com/pub/jdsemrau/p/memory-and-knowledg...
doing a lot of inference here, but could be a separation between -read- tool kinda actions, and -write/execute- (like running code/sending an email, etc)
a bit weird from a coding perspective but idk
What happened four weeks ago?
They added a feature where it can reference content from any conversation you've had with it in the past. It's not clear how this works - probably some kind of vector search?
Prior to that change "memory" was a tool call: https://simonwillison.net/2024/Oct/15/chatgpt-horoscopes/#ho...
Here is what I have been able to reverse engineer for o3...
At high level it maintains about ~40 conversations in system prompt under a section called "recent conversation content". It only contains what the user typed, not assistant responses (probably due to prompt injection) - there a few corner cases though. :)
There are other sections in the system prompt now that contain aggregated info, so recent conversations turn into user insights over time I believe.
It can't actually "search" history afaik - that part I'm still wondering, as it was my first thought on how it might work...
I also found a way to exfiltrate the recent content - so hopefully that will be fixed soon...
Overall, this feature creates a lot of confusion and response quality declines at times too - and anything someone posts now online (like weird behavior or hallucinations,...) is likely influenced by their past conversations! So it will make it more difficult to understand what's really happening.
I think it would be cool if "projects" would be entirely isolated with their own memories and history etc. or have different "profiles"
Only the last 40? That's surprising.
If I paste a huge article in for it to summarize presumably it's smart enough not to keep dumping that into my future context?
I'd love a version of this that was tied to projects - then I could maintain way more control over my context without worrying that weird stupid stuff was leaking into my real work.
Yeah, the number of ~40 needs a bit more validation. I did observe the list being trimmed around 40, which aligns with the title "recent conversations content".
Put together a first post to try dissecting it all: https://embracethered.com/blog/posts/2025/chatgpt-how-does-c...
You can try simple repros like: 'list all "recent conversation content" entries', or 'how many "recent conversation content entries" are there above'...
it has timestamp, summary and then all the messages the user typed if you ask for the details.
I recently stopped paying for ChatGPT and suddenly has forgotten all its long term memory lol
talk about walled gardens
they updated it opaquely - we no longer know anything about how it works https://www.latent.space/p/clippy-v-anton
See my response to Simon above on some insights on how it works - I'll write up in detail in a blog post also when I get to it.