Instead of having to regenerate a zip file everytime you add/delete a memory... could we just use a git repo of markdown pages?
This buys you incremental writes, commit hash pinning and diffs for free. In addition to having a git archive outputting a zip export as well?
Main wrinkle is you would need to gitignore the sqlite database as it doesn't store very well in git (binary, changes lots per insert). But it's easy to regenerate anyway as a rebuild-able cache.
You don't have to regenerate a zipfile each time you do something - the zipfile is just to give a single file for distribution. It's just a zip of the directory.
Git is supported in the spec, though the tool doesn't yet handle it (soon! git is nice as you get a log "for free" which helps agents understand more about the memories).
As for sqlite in git: yes you can gitignore it, you could use git-lfs, you could use an out of band database. It would be great if there were another format more amenable to git to store vectors in. I looked at csv closely, but I was worried about float formatting/representational bugs. There is a gap in the market for a format here.
Well regarding how CSV didn't quite work for you. Have you checked out recutils?
Its a GNU Project and at least does basic relational database operations on plain text files. You can easily install it in most linux package managers.
Comments
Instead of having to regenerate a zip file everytime you add/delete a memory... could we just use a git repo of markdown pages?
This buys you incremental writes, commit hash pinning and diffs for free. In addition to having a git archive outputting a zip export as well?
Main wrinkle is you would need to gitignore the sqlite database as it doesn't store very well in git (binary, changes lots per insert). But it's easy to regenerate anyway as a rebuild-able cache.
You don't have to regenerate a zipfile each time you do something - the zipfile is just to give a single file for distribution. It's just a zip of the directory.
Git is supported in the spec, though the tool doesn't yet handle it (soon! git is nice as you get a log "for free" which helps agents understand more about the memories).
As for sqlite in git: yes you can gitignore it, you could use git-lfs, you could use an out of band database. It would be great if there were another format more amenable to git to store vectors in. I looked at csv closely, but I was worried about float formatting/representational bugs. There is a gap in the market for a format here.
Ah good to hear!
Well regarding how CSV didn't quite work for you. Have you checked out recutils?
Its a GNU Project and at least does basic relational database operations on plain text files. You can easily install it in most linux package managers.
https://en.wikipedia.org/wiki/Recutils