testing

Allen Holub: if you really want to treat AI-generated code as a black box, your tests have to be immaculate and extensive, and he lists what that actually means, starting with all functionality and running through a long tail most suites never reach. “‘It works’ is not sufficient.”

The fine print on Uncle Bob’s gauntlet: not reading the code is only as safe as the test suite is complete, and software factories fail on exactly the qualities tests don’t price in.

Uncle Bob Martin (coding since the late 60s) no longer reads the code his agents write. Instead he surrounds them with extreme constraints, from unit and gherkin tests to mutation testing, and trusts code that has “run the gauntlet of all of my constraints and tests”.

The polar opposite of cognitive coverage: commit at zero understanding and bet the harness makes up for it. Harness engineering and Jason Swett’s TDD skill, taken to the logical end.

Jason Swett argues that agents only write useful tests if they are given a concrete process to follow. Models may know about TDD in the abstract, but they default to vague, performative, or overcomplicated tests.

Basing the agent loop on “specify, encode, fulfil” gives the agent an executable target, keeps speculative implementation (code/feature bloat) in check, and makes refactoring easier to review.