Skip to content

Ask HN: How to evaluate PRs against a dynamic rubric?

1 pointsujayk_33discuss
On HN

I'm working on a use case where every PR created in a repository needs to be evaluated against the topic that will be tagged to it. Based on which the AI feedback is to be given, based on the feedback, changes will need to be made, until done, the merging should be blocked. when the updation of code happens, it should be able to check it against the previously given feedback. Now it can ask for further improvements or give it a green light and unblock the merging.

To implement this, I'm looking for ideas to implement as the LLM shouldn't chase perfection and be generalized to make a call "this is enough".

If you have worked on something like this or have any ideas on how to get started, would really appreciate the help.

Thanks

Comments

No comments yet.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.