The goal is to democratize building your own simulation environment with AI agents. Would love to see the community build more complex interactions on top of this. Let us know what you think!
Cooperative agents have a lot of utility in more applied settings and the interesting part here is the agents are autonomous agents operating over an abstract semantic space using LLMs unique ability to “reason” abductively. The key to my mind in these exercises is the construction of constraints, management of context, and delegation to task based optimizers (i.e., the agent has a goal, the LLM abstractly constructs a solution given the context, and optimizers like path finders carry it out)
I’ve thought a lot that these sorts of techniques can be used to make things much more generalized. A multimodal model backing say manufacturing equipment can make the equipment much easier to reconfigure to new tasks in an environment that’s more adhoc and includes other machine and human agents.
The key to my mind is the fact it’s not just LLMs it’s a mixture of techniques - constraints, optimizers, information retrieval, etc.
More specifically the demo involves a lot of Andreessen Horowitz investments to demonstrate them in an accessible way that people find interesting because it’s a game. And people like games.
Imagine being in your shop and telling a robot to get you a hammer. It can understand the instructions, look at the environment using an inverse image model, synthesize the environment with the instruction, delegate to path finders etc a route to the tool box and retrieve you a hammer without ever being programmed anything. Now abstract you to being other machine agents in the environment of a workshop being instructed goals along some plan. Today factory robots are highly specialized machines that require a very finite and controlled problem space and reconfiguration is expensive and time consuming, if possible at all.
Thank you very much @fnordpiglet for addressing my questions directly and thoroughly! Makes a lot of sense. I still don't understand why a16z wants us to do that though. what's in it for them?
The mission statement you quoted is incredibly clear. Very often both commercial and open source projects announced here on hacker news have completely impenetrable descriptions or mission statements. That is not the case here. I literally could not improve on it without quoting it directly.
why did they make this? what is the purpose? why do they want "to see the community build more complex interactions on top of this" - what's in it for them?
I'm making a conscious effort to do more things that have no extrinsic objective, to do things just for their own sake. This is a familiar concept, it's a hobby (or art?). I feel I'm missing some of the joy and satisfaction that life has to give by always needing a stated tangible objective.
Comments
To be crass and direct,
why? what's the point?
Cooperative agents have a lot of utility in more applied settings and the interesting part here is the agents are autonomous agents operating over an abstract semantic space using LLMs unique ability to “reason” abductively. The key to my mind in these exercises is the construction of constraints, management of context, and delegation to task based optimizers (i.e., the agent has a goal, the LLM abstractly constructs a solution given the context, and optimizers like path finders carry it out)
I’ve thought a lot that these sorts of techniques can be used to make things much more generalized. A multimodal model backing say manufacturing equipment can make the equipment much easier to reconfigure to new tasks in an environment that’s more adhoc and includes other machine and human agents.
The key to my mind is the fact it’s not just LLMs it’s a mixture of techniques - constraints, optimizers, information retrieval, etc.
More specifically the demo involves a lot of Andreessen Horowitz investments to demonstrate them in an accessible way that people find interesting because it’s a game. And people like games.
Thanks! Can you please elaborate on "Cooperative agents have a lot of utility in more applied settings?"
Imagine being in your shop and telling a robot to get you a hammer. It can understand the instructions, look at the environment using an inverse image model, synthesize the environment with the instruction, delegate to path finders etc a route to the tool box and retrieve you a hammer without ever being programmed anything. Now abstract you to being other machine agents in the environment of a workshop being instructed goals along some plan. Today factory robots are highly specialized machines that require a very finite and controlled problem space and reconfiguration is expensive and time consuming, if possible at all.
Thank you very much @fnordpiglet for addressing my questions directly and thoroughly! Makes a lot of sense. I still don't understand why a16z wants us to do that though. what's in it for them?
They’re showcasing a bunch of portfolio companies in their integrations. That’s why the setup is 300 steps long.
The mission statement you quoted is incredibly clear. Very often both commercial and open source projects announced here on hacker news have completely impenetrable descriptions or mission statements. That is not the case here. I literally could not improve on it without quoting it directly.
What is unclear about the portion you quoted?
why did they make this? what is the purpose? why do they want "to see the community build more complex interactions on top of this" - what's in it for them?
ITS FUN!!!
And maybe can be used to do interesting things.
Let’s work on the cool tech together.
I promise you it will not hack into the mainframe and turn the world into paperclips. (Odds are…near zero).
I'm making a conscious effort to do more things that have no extrinsic objective, to do things just for their own sake. This is a familiar concept, it's a hobby (or art?). I feel I'm missing some of the joy and satisfaction that life has to give by always needing a stated tangible objective.
This is made by a16z, ostensibly for business reasons. I am asking what their angle is.
As far as I know, they don't have much history in making things with "no extrinsic objective" - I could be wrong though.
You misunderstand what a16z-infra is. This is built by Convex, the game engine being used.
Convex is in a16z’s portfolio. https://a16z.com/2022/04/27/investing-in-convex/
That GitHub org is a16z.
Their other repos are also showcasing various AI toolings, often the tools are also startups they invest in.
Creation?