Unpopular opinion: it shouldn’t matter when it comes to code quality.
If it’s a dev with AI and the code is good, then it should be merged.
If it’s a dev with AI and the code is bad, then the code should be rejected.
If it’s a dev without AI and the code is good, then it should be merged.
If it’s a dev without AI and the code is bad, then the code should be rejected.
So in that case AI has no effect on the outcome. The only explanation I can see for making it visible is for the sake of licensing. Which I don’t have an opinion on right now because I haven’t looked into it enough.
People using AI push a lot more code in a shorter amount of time. That deluge makes it much more likely for bad code, intentional or not, to slip by.
“Just leave it to a few trusted devs”
Some bad actors can play the long game. The XZ Utils backdoor is a very recent example of a contributor trying to abuse a backdoor he put there himself, after 2 years of posing as a helpful person.
“Let AI review it”
If humans are prone to mistakes, why would anyone leave a probabilistic program that is known to make bad decisions at random and that has, on more than one occasion, “cheated” to pass quality tests, deleted files it shouldn’t and is still suceptible to prompt attacks, review it?
I understand the sentiment of “if the AI generated code is good, allow it”, but the “productivity” gains in writing code means that, sooner or later, the person using the AI will care less and less about reading and preemptively catching possible errors because there’ll be so many alterations that he either has to spend a month reading, or just skim over and accept.
The XZ Utils backdoor is a very recent example of a contributor trying to abuse a backdoor he put there himself, after 2 years of posing as a helpful person.
A game that has been played for decades. A game that LLM review of merge requests should be making very much harder to play in the future.
Harder for the people who want backdoors or critical security bugs approved, I’d say.
That promise of open source is getting a little boost here: many eyes on the code, insecure patterns revealed for lower effort in less time - able to be caught before wide release because the (robot) reviewers have the capacity to run the code through many more test scenarios before delivering a verdict.
Can you ensure that the llm reviewing the code will not hallucinate and, more importantly, will not fall to some prompt injection that can be within the code?
There are no guarantees in life, especially when dealing with people you’ve never met on the other side of the planet through the internet.
However, when an LLM finds a real bug or vulnerability, somewhat akin to an NP-hard problem, you can much more easily verify that the problem is real once it has been identified - you can do the verification with or without LLM assistance.
will not fall to some prompt injection that can be within the code?
that is a concern, one which has been identified and is in a state of continual improvement to make less of a concern in LLM agents / harnesses. It’s also not without human based precedent, with misleading comments and even obfuscated code structure leading humans to believe that code does something different than what it looks like to a casual or fast/shallow reader. The ability of LLM agents to more quickly run many actual tests of the code than a human tester is a huge advantage in determining what code actually does, and identifying rarely triggered pathways that may have concerning functionality.
The thing is: the review process now also has AI, and those AI assisted reviews are much more stringent than the traditional human only reviews. Neither are perfect, ideally both would give the submission all the attention it deserves - in real life that rarely happens, though at the moment the LLMs are fast and cheap enough that you really should let them review anything you are considering approving for merge - not necessarily take everything they output at face value, but I find that they find real problems that real humans often miss.
Unpopular opinion: it shouldn’t matter when it comes to code quality.
If it’s a dev with AI and the code is good, then it should be merged.
If it’s a dev with AI and the code is bad, then the code should be rejected.
If it’s a dev without AI and the code is good, then it should be merged.
If it’s a dev without AI and the code is bad, then the code should be rejected.
So in that case AI has no effect on the outcome. The only explanation I can see for making it visible is for the sake of licensing. Which I don’t have an opinion on right now because I haven’t looked into it enough.
People using AI push a lot more code in a shorter amount of time. That deluge makes it much more likely for bad code, intentional or not, to slip by.
“Just leave it to a few trusted devs”
Some bad actors can play the long game. The XZ Utils backdoor is a very recent example of a contributor trying to abuse a backdoor he put there himself, after 2 years of posing as a helpful person.
“Let AI review it”
If humans are prone to mistakes, why would anyone leave a probabilistic program that is known to make bad decisions at random and that has, on more than one occasion, “cheated” to pass quality tests, deleted files it shouldn’t and is still suceptible to prompt attacks, review it?
I understand the sentiment of “if the AI generated code is good, allow it”, but the “productivity” gains in writing code means that, sooner or later, the person using the AI will care less and less about reading and preemptively catching possible errors because there’ll be so many alterations that he either has to spend a month reading, or just skim over and accept.
A game that has been played for decades. A game that LLM review of merge requests should be making very much harder to play in the future.
Harder for the people who don’t want backdoors or critical security bugs approved, you mean
Harder for the people who want backdoors or critical security bugs approved, I’d say.
That promise of open source is getting a little boost here: many eyes on the code, insecure patterns revealed for lower effort in less time - able to be caught before wide release because the (robot) reviewers have the capacity to run the code through many more test scenarios before delivering a verdict.
Can you ensure that the llm reviewing the code will not hallucinate and, more importantly, will not fall to some prompt injection that can be within the code?
There are no guarantees in life, especially when dealing with people you’ve never met on the other side of the planet through the internet.
However, when an LLM finds a real bug or vulnerability, somewhat akin to an NP-hard problem, you can much more easily verify that the problem is real once it has been identified - you can do the verification with or without LLM assistance.
that is a concern, one which has been identified and is in a state of continual improvement to make less of a concern in LLM agents / harnesses. It’s also not without human based precedent, with misleading comments and even obfuscated code structure leading humans to believe that code does something different than what it looks like to a casual or fast/shallow reader. The ability of LLM agents to more quickly run many actual tests of the code than a human tester is a huge advantage in determining what code actually does, and identifying rarely triggered pathways that may have concerning functionality.
The thing is: the review process now also has AI, and those AI assisted reviews are much more stringent than the traditional human only reviews. Neither are perfect, ideally both would give the submission all the attention it deserves - in real life that rarely happens, though at the moment the LLMs are fast and cheap enough that you really should let them review anything you are considering approving for merge - not necessarily take everything they output at face value, but I find that they find real problems that real humans often miss.