But... why? Computer use is probably the least efficient way to do this. There are agentic mods and plugins that are a lot better. See also SkyrimNet, people have been building agents for gaming and roleplay long before tool use was even a thing, let alone the current agentic craze.
Totally agree. I’ve built a bunch of harnesses for Minecraft and a specialized one will almost certainly out preform a general purpose one.
I believe the way we advance the capabilities for general computer use is to to attempt harder and harder tasks with it, and build up the necessary infrastructure to accomplish it along the way.
Even when we don’t achieve the goal, the infrastructure we build is often useful for easier and more obtainable goals as well.
If your computer use environment can pilot Minecraft, it can also probably do your taxes
I've played around with Mindcraft for a while. Fair warning that it is quite outdated and spaghetti-coded. Although all of its components required to make it work are somewhat hard to replicate from scratch, so it might still be your best option.
Comments
But... why? Computer use is probably the least efficient way to do this. There are agentic mods and plugins that are a lot better. See also SkyrimNet, people have been building agents for gaming and roleplay long before tool use was even a thing, let alone the current agentic craze.
Totally agree. I’ve built a bunch of harnesses for Minecraft and a specialized one will almost certainly out preform a general purpose one.
I believe the way we advance the capabilities for general computer use is to to attempt harder and harder tasks with it, and build up the necessary infrastructure to accomplish it along the way.
Even when we don’t achieve the goal, the infrastructure we build is often useful for easier and more obtainable goals as well.
If your computer use environment can pilot Minecraft, it can also probably do your taxes
that is a nothingburger :^)
for minecraft in particular, there is: https://github.com/mindcraft-bots/mindcraft
I've played around with Mindcraft for a while. Fair warning that it is quite outdated and spaghetti-coded. Although all of its components required to make it work are somewhat hard to replicate from scratch, so it might still be your best option.