How We Used AI Agents to Migrate GitLab Rate Limiting

Sam Wiskow on how a three-person pod at GitLab, plus floating engineers, used agents to move 121 rate limit keys onto a new implementation across 30-plus merge requests and 14 numbered specs. The work ran in six cohorts, one more than the five originally planned.

The loop was rigid on purpose: read the epic, write a spec, run an adversarial review on the spec with a two-round cap before a human is pulled in, implement only once blockers are cleared, verify with explicit evidence, adversarially review the merge request, then escalate for human review and merge.

The honest details are the good part. Cohort 1 went 1% to 10% to 50% on 4 May and 100% the next day, but the first pass missed 17 of the 121 keys, and a Redis connection test was halted at 75,000 connections rather than the intended 100,000. This sits in front of a RackAttack layer handling roughly four billion requests a day. One engineer’s mid-project verdict was I could’ve done this faster myself, which was irritating.

The lesson Wiskow draws is that the agents were never the constraint. Review capacity, rollout judgement and operator attention were.