1. Use mechanical scripts at the start to gather a context and at the end to make the comments. Don't have an agent/llm do this part (make API calls), I do have the comment creation agent write a jsonl file, so it is writing the comments.
2. It should not need any tools beyond read and write (to a specific directory). Tests and the like should have already run as part of CI. PR review is not really intended review failing tests, human or agentic.
3. A way to run different level of agentic review based on how complex the PR is.
General PR agents are not really the right answer in my opinion, in large part because devs and orgs are opinionated and those need to be encoded into their PR review process.
There are a lot of code review tools popping up lately.
People assume these tools are very complex, while some might be, building one you can use on day to day basis is pretty simple and straightforward.
In this post I will take you through the whole process
Building a code review agent is easy, yes, but building a code review tool for humans is not that easy, been working on my own for a while.
Yeah, not only complex but very expensive at scale, you would want to preindex or heavily delegate to smaller models.
I'm currently building one based on: https://blog.cloudflare.com/ai-code-review/
The keys here are
1. Use mechanical scripts at the start to gather a context and at the end to make the comments. Don't have an agent/llm do this part (make API calls), I do have the comment creation agent write a jsonl file, so it is writing the comments.
2. It should not need any tools beyond read and write (to a specific directory). Tests and the like should have already run as part of CI. PR review is not really intended review failing tests, human or agentic.
3. A way to run different level of agentic review based on how complex the PR is.
General PR agents are not really the right answer in my opinion, in large part because devs and orgs are opinionated and those need to be encoded into their PR review process.