LinkedIn Measures AI Code Review Against Merged Code
The company’s reported acceptance rate is a useful outcome measure, but the more transferable design is a controllable review pipeline: local rules, pre-posting filters and infrastructure built for monitoring.
Listen to this story
The audio brief
Story brief
3 key pointsLinkedIn’s code-review system is being judged by whether suggestions survive into the merged codebase, not merely by whether reviewers posted comments. In a sample of 5,230 comments from 1,727 pull requests, 90.1% could be assessed with high confidence and 63.9% were accepted. Results varied substantially by issue type, while the multi-agent workflow adds policy controls, pre-publication filtering, and operational...
- 01
Logic-error suggestions had an 80% acceptance rate, versus 43.5% for refactoring and 40.6% for security suggestions.
- 02
The reported 100% concurrency-bug acceptance rate comes from the sample, not a general benchmark.
- 03
Independent reviewers converge on issues; one-off findings receive additional verification before posting.
LinkedIn has built a multi-agent AI code-review platform that evaluates suggestions against the code that ultimately gets merged. The system is designed to apply organization and repository context while giving engineering teams a more manageable way to run AI review across pull requests.
Measuring the outcome, not just the comment
The company built an automated evaluation pipeline that compares each AI review suggestion with the resulting merged codebase. Its evaluation sampled 5,230 review comments across 1,727 pull requests, and it determined that 90.1% of those comments could be evaluated with high confidence from the merged result.
LinkedIn reported that 63.9% of suggestions in that evaluation were accepted. The result varies sharply by category: logic-error suggestions reached 80% acceptance, while refactoring suggestions reached 43.5% and security-related suggestions 40.6%. The 100% result for concurrency bugs is also based on the same sampled evaluation, not a general benchmark for AI reviewers.
LinkedIn reported this overall acceptance rate from its 5,230-comment sample across 1,727 pull requests.
High-confidence evaluation was based on comparing suggestions with the code that was merged.
Several reviewers, then a policy layer
The platform’s core design is to use multiple independent AI reviewers with distinct models and reasoning approaches. When reviewers converge on an issue, the system treats that as stronger evidence; findings raised by only one reviewer are verified separately. That approach aims to reduce the blind spots of relying on a single reviewer without automatically discarding unique findings.
- Rules can combine organization-wide policies, repository-level conventions and context-specific guidance.
- Before a comment is posted, the system filters suggestions that are cosmetic, already fixed, irrelevant or inconsistent with the repository.
- Kubernetes, durable queues, event-driven processing and horizontally scaled workers provide the operating layer for the review pipeline.
A code-review system that can be operated
The architecture is built to support monitoring of latency, acceptance and completion rates, as well as provider failures. That is the practical enterprise distinction in LinkedIn’s design: the model output sits inside a configurable and observable workflow, rather than arriving as an ungoverned stream of comments.
The reported acceptance rate shows that this particular system produced suggestions developers often incorporated in its sampled pull requests. It does not establish that another company’s codebase, rules, models or review practices would produce the same rate. What LinkedIn’s implementation makes clearer is where organizations can exert control: the rules supplied to reviewers, the checks before comments reach developers, and the operational data used to tune the service.
Sources
- infoq.comAI Code Review at Scale: LinkedIn
