AI generated code review

[u/Shitmfman]

Hi there,
How do you deal with managing and reviewing AI-generated code across the organization?

Over the past year it's become incredibly difficult to review the growing amount of AI-generated code, especially from vibecoders/non-engineers.
Reviewing it manually is hard, but it's still important to understand what's going on there.
For example, I've been at this company for some years so I have a lot of context and know many of the nuances, which helps me notice or suggest important things to keep everything secure, efficient and so on…

But any other teammate can use any model for coding nowadays. AI reviews of AI-generated code could be done in the MR, but then the reviewer model will most likely be smarter than the model that wrote the code, which might lead to a lot of threads and an infinite loop of changes. On top of that the reviewer model won't have all the context and input from the developer.
Another option is to distribute a "review" skill or rules across the organization and somehow make everyone do a self-review after making changes…

It feels like in this AI code generation era we'll stop focusing on how the code was written and just focus on whether the MR/PR works and if those changes achieves the goal, no matter how it was done, right?
How do you deal with this? Just accept the reality?

Thanks for attention

[u/theov666]

I wouldn't reduce review to just "does it work?" A PR can pass its functional tests and still violate an architectural boundary, introduce an unwanted dependency, or handle sensitive data in a way the organization never intended. I think the interesting part of your question is the context held by experienced engineers. They know not just what the rules are, but why certain decisions were made and when exceptions are acceptable. A second reviewing model doesn't automatically have that knowledge. I'd separate the problem into three layers: functional tests to verify behavior, explicit architectural and security constraints that can be checked automatically, and human review for changes that require judgment or exceptions. AGENTS.md and review skills can communicate those decisions to agents, but I'd avoid relying on instructions alone for critical constraints. Wondering whether the bigger problem in your organization is the sheer volume of generated changes, or that the engineering decisions needed to review them are still largely in people's heads?

[u/riickdiickulous]

I’ve been saying that AI coding is making testing and observability more important than ever. There must ultimately be a person responsible for owning the code AI writes. Review it the best you can. It is the Wild West out there right now and mistakes will be made.

[u/rayray5884 → u/riickdiickulous]

For real. Day in and day out it’s ’my robot wrote this, my other robot reviewed it, please have your robot review it and deploy it’.

So if it goes boom are we just blaming this on my robot or yours? 😂 😭

[u/trash-packer1983]

Have you considered implementing steering documents? A steering document provides persistent instructions that AI coding assistants can use when generating or reviewing code. Consider it a guide for AI. It won't eliminate the problem, but can help control.

https://medium.com/@Toglefritz/fast-code-slow-integration-0e9e2b63668b

[u/Plenty-Emphasis-5669 → u/trash-packer1983]

What's the difference with skills and plugins?

[u/trash-packer1983 → u/Plenty-Emphasis-5669]

Skill defines the review process.

Example :

When reviewing a pull request:
1. Read the architecture documentation.
2. Identify modified resources.
3. Evaluate changes against security standards.
4. Look for unnecessary permissions.
5. Validate test coverage.
6. Report findings by severity.
7. Suggest remediation

Plug-in: Capability

Retrieve pull requests from GitHub.
Run security scanning tools.
Query vulnerability databases.
Publish review comments.
Trigger CI/CD checks.

Consider implement
1. Organization standards
2. Skills
3. Plug-ins

[u/theManNowDog6 → u/trash-packer1983]

I treat agents like an outsourced team of juniors. Guardrails and guides like this are the kind of thing that makes that work, and you should do it in layers. If your infra is set up around assumptions you would make for how stable a senior or seasoned team operated without agents, then you've got gaps or a lot of headache keeping up with output that chruns.

[u/DoStopBelieving]

I use an agent to review the code in increments while it's being written, then when it's pushed a CI pipeline runs linting and tests/scans, and when the PR is opened another AI agent with access to our knowledge base reviews the PR and provides an independent summary plus comments. The AI agent will also close its own comments when the issues are resolved.

I also do a single full self-review pass before having another human review the code.

By the time another human needs to review the code there's a very high chance of the code already being production-ready and having AI explanations of code behavior that are often better than the developer's own explanations (sorry y'all, humans can do better but many devs suck at technical writing).

As for skills - I searched and collected a library of agent skills covering topics like planning, writing, reviewing, specific tools, sub-agents, meta-skills, etc. I even made my own skill for expediting re-reviews so the sub-agents waste less time reviewing work.

All that said, you should not approve code that you don't understand no matter how overwhelming the volume is. Code is fundamentally for humans to understand the instructions we give to machines.

[u/nightwillalwayswin]

Your value on a review is context. You know what this change was supposed to do, what the nuances are, what breaks if someone touches the auth path. A vibecoder’s MR arrives with none of that attached. The code is there. The intent that produced it is gone.

That’s why AI-reviewing-AI hits a wall. The reviewer model sees the diff and has to guess what the author was trying to do. So it flags everything, you get the infinite thread loop, because it’s reconstructing intent instead of checking against a stated one. Same problem a human reviewer has with a context-free MR, just faster.

On your last question- no. ‘Works’ is unverifiable when you don’t know what it was supposed to do or what it chose not to handle. A change can pass tests and still be wrong in a way only someone who knew the intent would catch. You’d be deciding the thing you know matters, your context, doesn’t matter anymore. It does. It just isn’t being captured.

Disclosure: I work on Atomic (atomic.dev), so biased. Stop treating the diff as the unit of review. Capture the workstream, the intent, what the agent considered, where a human made a call, and review that. You’re checking a decision against its stated goal instead of reverse-engineering the goal from the code. The self-review skill you mentioned is a lightweight version of this instinct. It’s right, it just lives in a doc instead of traveling with the change.

[u/First_Inspection_478]

Have my agent review it

[u/giovangonzalez → u/First_Inspection_478]

tell your code sucks without telling your code sucks

[u/First_Inspection_478 → u/giovangonzalez]

It was mostly a joke, but agents are good now if you give them enough context. Update your priors

[u/unitegondwanaland]

We have agents doing code reviews and we leverage AGENTS.md files in the repo + skills that the agent can leverage to enforce all the rules, best practices, etc.

You have to fight fire with fire. That way idiot vibe coders can't merge shitty Terraform unless it passes the sniff tests you have in place.

Go read about AWS AgentCore Harness

[u/Plenty-Emphasis-5669]

In my company we're being encouraged (aka almost forced) to use agents for reviews and don't wait for another person to merge.

[u/L3ima]

We route changes by what they can break. Anything that touches auth, payments or data migrations needs a reviewer who signs off before merge. Other tickets take the normal path.

Before that we were reviewing by author, but the non-engineer PRs got all the attention and risky changes from regular contributors would not be checked enough.

[u/usually_guilty99]

This is a growing problem and will get worse before we get to a perfect solution. The part I'd focus on is the knowledge your experienced reviewers are supplying that isn't written anywhere.

Why do they reject certain changes? Usually there's some unstated constraint around a shared dependency, backward compatibility, data ownership or a failure the team has seen before.

I'd start capturing those reasons and turning the repeatable ones into checks.

For everything else, require a short explanation of what the change affects and how it will be verified.

That seems more scalable than having two models argue over every PR while neither understands the production system.

[u/effectivescarequotes]

A few things. First, scope the work to keep PRs a reasonable size. Second adopt a highly structured architecture so it's obvious when things are out of place. Third, developers should perform an AI code review of their own PRs and post the findings and resolutions to the PR as comments.

The human reviewer should read these comments before reviewing manually, which if you've done all the other things, should only take a few minutes.

[u/Entire_Struggle_2236]

What helped us most was dropping the idea that a human reads every line. Put the boring checks in CI as hard gates: SAST, secret scanning, dependency checks, and required tests for anything touching auth, payments or data access.

Then route review by risk with CODEOWNERS, so changes in sensitive paths always go to someone with context like you, while low risk UI or script changes get a lighter pass.

For non engineers shipping code, require the PR description to explain what the change does and which tool or prompt produced it, which makes review much faster.

Also cap PR size, because huge AI generated PRs are exactly where things slip through.

[u/rafaelpirolla → u/Entire_Struggle_2236]

But "changes in sensitive paths always go to someone with context like you" is exactly the overburden issue in the post...

Most of my friends are abandoning PR reviews and focusing on testing and quality gateways but I don't believe it's the answer, though.

[u/Entire_Struggle_2236 → u/rafaelpirolla]

True, routing alone just moves the pile. The point is to shrink it: gates and small PRs mean only a slice of changes need that person, and every repeated review comment should become a check so the context stops living in one head.

Agree that tests alone won’t cover it, they only catch what someone already thought of.

[u/rafaelpirolla → u/Entire_Struggle_2236]

Are you hiring? :)

[u/Entire_Struggle_2236 → u/rafaelpirolla]

No, but you can be our referral agent if you want haha

[u/Triple-B-564]

Lots of people/companies/projects are approaching this differently right now. Here's what I've seen and what worked well in my opinion:

• Keep PRs small. Stacked PRs on GitHub help a lot, and a size cap (something like 400 changed lines, a bot flags anything bigger) keeps it that way. AI makes huge diffs cheap, reviewing them isn't.
• A person owns every PR. If the author can't explain a change, it doesn't merge, even if an AI agent wrote it.
• Let CI catch the boring stuff: lint, type checks, tests, coverage on changed lines, secret scanning.
• An early risk assessment based on the code change. If it comes out high, a human has to review it.
• CODEOWNERS plus a branch ruleset, so infra, auth and payments paths need a review from whoever owns them.
• AI review agents (CodeRabbit, some open-source PR-Agent, etc...) are useful as a first pass, but treat them like a linter, not an approval.
• For UI changes, one approach I liked: the PR includes a preview link, so you can see and try the actual change instead of just reading the diff.

The catch is that with more tests and more agents, pipeline run time goes up. That's a whole other topic (optimizing for both time and money), but it's solvable.

So to your last question: I don't think it's "just accept it".

How the code was written matters less now, but someone still has to own what it does.

[u/Apple_Master]

I don't review it because I don't allow slop code in repos or code that I maintain or own.

[u/Additional_Vast_5216]

only thing I review are tests, tests tests at every level, unit, integration, system, e2e etc if the tests pass I dont care about the implementation

[u/HappyJuggernaut4647 → u/Additional_Vast_5216]

the tests don't tell you if the AI slipped in a credential leak or a silent data grab though, that's where it gets sketchy when nobody actually reads the code

[u/Psychological_Poem64]

See how it's done here Https://forge.amitinfotech.net