Skip to content
All tags

#testing

6 posts

Learning Agent Design from Mature Coding Agents (24): Testing a Moving Agent — fake-CLI Contracts, Recorded Streams, TUI Pilot

An agent's two dependencies — the LLM and external CLIs — are both non-deterministic, but mature projects separate 'the moving parts' from 'the shape of the boundary': codex fakes the Responses API with wiremock plus a scripted SSE server and pins its TUI with insta snapshots; opencode built a VCR-style http-recorder package; pi splits model-backed evals from unit tests into two vitest configs; omp wraps its edit benchmark itself in unit tests. looplane stacks four layers against external CLIs: unit tests, fake-CLI contract tests, recorded-stream integration proofs, and Textual pilot TUI tests. The methodology in one line: record real non-deterministic output, then make deterministic assertions about it.

Learning Agent Design from Mature Coding Agents (5): The Verification Gate — Changed Files Isn't Success, Verified Is

None of the five reference projects enforces 'all declared verification commands pass' at the harness level: pi leaves verification to the model, OpenCode and Codex put it in the system prompt, Claude Code uses a separate adversarial verifier subagent but as a soft contract, and only OMP's cleanse actually runs checks from harness code. looplane takes the hardest path: if files changed, every declared verification command must pass before terminal_reason=verified; with no changes, checks don't rerun (no_changes). Whether to verify is decided by code, not by the model.

tech deep-dive

Selenium Deep Dive: Browser Automation from WebDriver Sessions to Grid

Selenium drives real browsers through standardized WebDriver sessions, making it useful for cross-browser workflows, existing test assets, and remote Grid capacity; it can render JavaScript applications, but it does not guarantee bypassing CAPTCHAs or other anti-automation controls.

Vitest: Share Vite Configuration Without Mistaking Component Tests for E2E

Vitest's advantage is not merely a familiar Jest-style API. Tests share Vite transforms, aliases, and plugins with the application; Browser Mode adds real-browser confidence but does not replace full E2E testing.

CS146S Week 5: Express Scores 28, CockroachDB Scores 74 — Agent Readiness Is Measurable

Factory breaks 'can an agent work in this repo' into eight pillars and five levels, and published real scores: CockroachDB L4 (74%), FastAPI L3 (53%), Express L2 (28%). The thesis is that agent readiness approximates the density of deterministic validation loops — linters, type checkers, tests are reward signals for agents.

ai deep-dive

Uncle Bob Doesn't Read His Agents' Code: What He Runs Instead of Code Review

Uncle Bob's 4.18M-view post of 2026/7/23 isn't a manifesto — it's a reply to an engineer who started in 1983 asking whether needing to understand code psychologically makes him old-fashioned. And he doesn't skip the code entirely: his 6/1 four-stage pipeline post says 'I spot check the code,' with thresholds of crap ≤ 6 (convention is 30) and mutation runs that kill all survivors. Plus a breakdown of his open-sourced Acceptance-Pipeline-Specification and the three metric blind spots Grady Booch names.