AI Writes More Code—So Why Isn't More Software Shipping? What Harvard's New Study Found
A current Harvard working paper tracks AI coding tools across software firms and explains why more coding activity may not produce a matching rise in finished software.
AI coding tools can help developers produce code and open pull requests faster. But code is an intermediate step: a change still has to be understood, reviewed, tested, integrated and released. A new Harvard working paper asks what happens across that wider process when software companies adopt AI coding assistants and agents.
The distinction matters to business owners and engineering leaders. Counting generated lines or accepted suggestions can make a tool look productive even when the team’s delivery pace, reliability or customer value has not improved by the same amount.
What the Harvard research examined
Fiona Chen and James Stratton’s *Artificial Intelligence in the Firm: Bottlenecks in Software Production* is a Harvard job-market working paper. Its October 28, 2026 version analyzes 445 million engineering work events associated with 720,542 workers at 655 firms. The data includes GitHub activity, Jira issues and other engineering workflow records, alongside measured adoption of coding assistants and agents.
The authors report substantial increases in intermediate measures of code activity after firms adopted AI tools. The measured changes in completed Jira issues and larger epics were much smaller; for coding assistants, the estimated changes were statistically insignificant. Agent adoption showed somewhat larger output estimates, but the paper says the gains still did not scale one-for-one with code activity. It also reports longer review queues and more review work after agent adoption.
These findings do not mean that AI never helps an individual developer or that every company will see the same result. The paper studies firm-level adoption in a particular dataset, uses Jira issue completion as an output measure, and is preliminary research rather than a peer-reviewed journal result. Tools, team experience, codebase, task selection and review practices can all affect outcomes.
Why code review can become the bottleneck
Software delivery has complementary stages. If code creation speeds up while reviewer capacity, testing or integration stays fixed, more changes can arrive than the team can safely process. Reviewers may need to understand unfamiliar code, check assumptions the prompt did not capture, find security-sensitive paths and request revisions. A faster first draft can therefore move work into a queue instead of eliminating it.
This is an operations problem as much as a model problem. If a team rewards code volume or pull-request counts alone, it may create incentives that do not reflect completed work. The more useful question is whether the whole delivery system can absorb, verify and maintain the additional changes.
A practical way to evaluate an AI coding pilot
### 1. Pick a bounded task
Choose a repeatable, low-risk task with a clear acceptance test, such as drafting a small internal script or generating test scaffolding. Keep authentication, payments, sensitive data handling and high-impact decisions out of an initial experiment unless qualified reviewers and safeguards are already in place.
### 2. Record a baseline
Before enabling the tool, record task completion time, review time, revision requests, defects found before release, rework after merge and reviewer effort. Define each measure in advance. A comparison is misleading if the team changes task complexity, staffing or release rules at the same time.
### 3. Measure through release and maintenance
Track the complete path: work accepted, change reviewed, tests passed, feature released and follow-up defects. Include the time spent prompting, correcting output, reviewing dependencies and updating documentation. Lines of code and commits describe activity; they do not by themselves establish business value.
### 4. Protect code and business information
Use only tools approved for the organization’s data and intellectual property. Check retention and training settings, permissions, repository access, vendor terms and applicable contracts before sharing source code or customer information. Ask a qualified privacy or security professional where the data or obligations warrant it.
### 5. Decide using a balanced scorecard
Continue the pilot only if it improves a useful outcome without an unacceptable rise in defects, review backlog, cost or maintenance effort. Compare results by task type and developer experience instead of assuming one average applies to every role. Keep a human accountable for decisions and production changes.
For organizations assessing an AI-enabled workflow, iPragma’s AI automation services focus on process fit, data readiness and human review. If the idea requires a broader product or integration, see software and digital product services. For a practical companion on protecting code before merge, read the AI-generated code security review checklist.
What the study can—and cannot—tell a business
The paper is useful because it looks beyond individual coding speed to firm-level output and review capacity. It is not a universal forecast, a vendor comparison or proof that AI coding agents have no value. Its central operational lesson is narrower: measure the stages where work is reviewed and shipped, because gains in one task may not pass through the rest of the workflow automatically.