How to Measure Developer Experience in the AI Era

Candace Shamieh, Teddy Gesbert and Daniel de Juan argue that AI makes the usual output metrics useless. PR counts, commits and lines of code all go up whether or not anything valuable shipped, so the thing to measure instead is developer experience: the feedback loops, cognitive load and flow state around the work. They cite GitClear’s analysis of over 200 million lines of code, which found code churn nearly doubled after AI adoption became widespread.

A developer can now produce significantly more lines per session, but higher volume doesn’t guarantee that the code is stable, maintainable, or successfully running in production.

The numbers from their own estate are the useful part. Datadog keeps more than 3,000 engineers productive in an AI-augmented development cycle, and about 80% of PRs are now AI-assisted. Those PRs have slightly lower cycle times per change but much higher concurrency, which they read as AI not really speeding up any individual change so much as letting one developer run more changes at once. The strain shows up downstream: some teams saw review time rise by over 500%, even though the global average held steady.

That reframes where the cognitive load sits. Code-level complexity still matters for legacy systems and core libraries, but the primary load now comes from orchestrating several agents at once. Their concrete fix was to attack the queueing rather than the code: persistent runners, which cut out the cold starts that pile up under higher AI-driven PR volume and improved CI speed by 50%. Datadog has written about the same problem from other angles in harness-first development and Hackerbot.