We ported 65 OSS projects with AIakka.io 4 pointsTylerJewell10 days ago2 commentsSaveHideCopy link On HNComments−TylerJewellOP10dWe created a self-improving, spec-driven delivery harness to test whether frontier models can port complete systems.−TylerJewellOP10dThere were a few interesting findings:1. We were able to execute complete, unattended rewrites of existing systems that pass the original's unit and integration tests.2. Low effort models were more efficient than high effort models.3. The delivery harness and its structure had a bigger impact on consistency than model capability.4. We saw a LOC or performance improvement on 57 of 65 ports.
Comments
We created a self-improving, spec-driven delivery harness to test whether frontier models can port complete systems.
There were a few interesting findings:
1. We were able to execute complete, unattended rewrites of existing systems that pass the original's unit and integration tests.
2. Low effort models were more efficient than high effort models.
3. The delivery harness and its structure had a bigger impact on consistency than model capability.
4. We saw a LOC or performance improvement on 57 of 65 ports.