Already have a harness: PiHarness. Chat where you assemble each model call: AIChatAgent; let the framework run it: Think. Not chat but need state, scheduling or MCP: Agent.
CS2680 Assignment 2 (Mads-Loop) has you write your own coding agent harness: with a fixed small model, qwen3.6-35b-a3b, it fixes real SWE-bench Pro issues inside Docker containers and logs every step in a required format. The assignment page says plainly not to trust the design it hands you, and some of the 10 tasks can't be solved. This post covers the spec, the places on the page worth doubting, the five places a harness most often breaks and how to handle each (with a snippet that has actually been run), how to build your own four-task eval set, and how to tell whether a task really is unsolvable. No official solution.
CS2680 Assignment 3 (Mads-Opt) is a harness competition: on the same SWE-bench Pro tasks, who solves the most, spends the least per solved task, and waits the least per solved task. The three metrics aren't weighted into one score. Instead submissions are ranked in skyline (Pareto) layers: if nobody is at least as good as you on all three and better on at least one, you're rank 1. This post covers the container setup and hard limits, how ranking and scoring work (with a worked example), a first estimate of what one task costs using the model prices on the assignment page, and how Lecture 8's cost and speed levers map onto the assignment. No official solution and no specific strategies.
CS2680 Lecture 1 (2026-09-02) starts with how AI systems differ from traditional systems: computation is split across CPUs and GPUs, floating-point math doesn't give the same answer every time, and state is large enough that it has to move between GPUs. It then compares an agent to a new kind of computer (the LLM is the CPU, the context window is memory, the harness is the operating system) and shows that one agent request becomes a long chain of inference → tool → inference. That makes tokens per second the wrong target; what matters is task success rate, end-to-end latency, and cost per success. This post works through two agents' numbers to show how those metrics are used.
CS2680 Lecture 2 (2026-09-09) takes Claude Code apart. You type one request and it calls the model many times. Each time, the model only proposes a tool to run; the harness checks permissions, runs it, and puts the result back into context, and when the context fills up it gets compacted. The second half covers memory (the CLAUDE.md you write and the auto memory Claude keeps) and four extensions: skills cover how to do things, MCP reaches outside systems, hooks cover what must always happen, and plugins handle packaging. This post first shows what one API call looks like, then walks the whole loop through fixing one test.
CS2680 Lecture 4 (2026-09-16) switches from using agents to designing them. An agent is a model plus a loop, and you decide which tools it gets and what it sees at each step. The tools half covers four ways to find code, four ways to edit it, the power and risk of a shell, and a set of design principles: few orthogonal tools, purpose-built over generic, token-cheap output, errors that teach. The context half covers why context is a finite resource, four ways to manage it (from guarding the door to summarizing as a last resort), and five ways context fails. It closes with a paper showing that when context is tight, the same model with only a different harness can solve 70% to more than twice as many tasks.
CS2680 Lecture 5 (2026-09-23) is about designing multi-agent systems. There are four reasons to split work across agents: specialization, isolation, parallelism, and context management. Any architecture can be described by three questions: who coordinates, how the work is split, and how messages move. The lecture lays out eight architectures, then covers six ways to decompose a task, a granularity formula for deciding whether to split, six common decomposition failures, and why communication is really a distributed systems problem: messages get lost or resent, and two agents end up changing the same thing at once. It closes by contrasting A2A with MCP, with "Is multi-agent really necessary?" as the first design principle.
The first half of CS2680 Lecture 6 (2026-09-28) is about agent memory. Keeping one fact means deciding, in order, whether to write it, what form it takes, where it lives, when to retrieve it, whether to generalize from it, and whether it still holds, plus who owns the memory. Don't simply delete outdated facts; record when they were true and where they came from. The second half is about reliability: at 95% success per step, a 20-step run is fully correct only about 36% of the time. Failures can be traced to five layers (task spec, harness, model, tool, environment), each with its own fixes. The two halves meet at one harness-layer failure: treating a stale note as fact.
CS2680 Lecture 8 (2026-10-07) has two halves. The first takes apart an agent's bill: cost is a product of users × sessions per user × turns per session × steps per turn × tokens per step × price per token, and the last four are the ones you can cut. The students' own Claude Code logs show 97.6% of tokens are cached re-reads, so keeping the prefix stable matters more than keeping the prompt short. Speed is set by the critical path, not the sum of every step. The second half compares three harness architectures (monolithic, modular, microkernel) and the design choices of eight open-source and commercial harnesses. This post works one bill by hand: put a timestamp at the start of the system prompt and the same task goes from 23.6 units back to 65 or more.
**tinyhumansai/openhuman** (42,159★) is a Rust agent harness that beat six other tools on a public SWE-bench run — over 2x faster, 2.6x fewer tokens, 8x less memory and CPU than Hermes and peers — by compressing tool output, using a tiny model (Jev) to pick the right tool, and running everything on one in-process plugin bus. **tigerless-labs/agent-memory** (4,385★) stores an agent's long-term memory as plain Markdown files with a rebuildable local index — delete the index and nothing is lost — so Claude Code and Codex CLI can share one memory store. **jub0t/Concat** (9,417★) is a local-first, watermark-free open-source CapCut alternative with a built-in MCP server, letting Claude and other agents cut video directly. Release: Agno v3.1.6 lets eval Cases carry a session_id, images, and other run arguments through a full turn, and has OS Manager query its own docs instead of delegating to Platform Manager.
**trueforge** (6,083★) is an open-source agent harness runtime that packages the chores of running an agent well — model calls, MCP tools, sandboxing, approvals, session state — behind a chat UI, an HTTP API, and an embeddable UI SDK. **agent-device** (4,937★) lets coding agents open an app on an iOS/Android simulator or real device, tap through the UI, and read accessibility snapshots to verify their own changes actually work; Expensify and Shopify already use it. **pi-pocket** (274★, 4 days old) ports an entire session of Pi — the open-source, deliberately minimal coding agent — into a mobile web app you can steer, watch with others, and schedule from your phone. Claude Code v2.1.294 fixed a bug where plain-language hooks (written as instructions like "Block commands that...") sometimes failed to block what they should; browser-use 0.13.11 previews a Claude-native browser toolset that can't actually run yet — it's waiting on an Anthropic SDK release that hasn't shipped.
Pi 1.0 and the experimental Pi Durable package shipped together in early October 2026. Pi Durable turns every model request, tool call and compaction into a task that writes a checkpoint, so a killed process resumes from its last checkpoint; Cloudflare followed with PiHarness on Durable Objects. The key design decision is classifying tools as safe or unsafe to replay.
CMU 11-768's first assignment starts from an empty ReAct loop: a bash-only CodeAgent fixes a bug in a chess app, context compaction is added to solve a SWE-bench task, and the same loop becomes a ChessAgent that runs a two-ply search with simulate_move, run_python, and a skill. All 100 points are graded by replaying submitted patches and trajectories offline.
affaan-m/ECC rode agent-harness optimization to 260k+ stars in eight months, though a growth rate that steep deserves skepticism; cactus-compute/needle trades chat ability for tool-calling precision in an 8-29MB model; Graphify-Labs/graphify builds knowledge graphs with local AST parsing instead of a vector store; tinyhumansai/openhuman makes 'getting to know the user' the core of its agent memory; IvanMurzak/Godot-MCP lets agents drive the Godot editor directly; Claude Code v2.1.277 adds AGENTS.md support
DeepSeek Harness (dsh) uses an everything-is-a-plugin architecture and hit 214K stars in 3 weeks; ponytail proves with real benchmarks that one skill can cut Claude Code's code output by 54%; Magnitude auto-picks and tunes local models for your coding agent; wigolo gives agents API-key-free web search, crawling, and research
AgentHarness Core Class, System Prompt Dynamic Assembly Flow, Skills Loading & Formatting, Prompt Templates System, How Harness Decides Tool Availability, Result Handling, Telemetry Schema Registration, Default Harness Construction, Extension Harness Extension.
deepseek-ai/deepseek-harness (dsh) uses a Cordis plugin architecture to make models, tools, sandboxes, and memory all swappable components, hitting nearly 200k stars a week after its developer preview launch; PrimeIntellect-ai/prime-agent runs long-lived research coding tasks on a Recursive Language Model architecture, surviving terminal disconnects via a persistent IPython session; liqiwa/mcp-radar automates this very kind of digest by scanning GitHub daily for newly ranked MCP servers. On the framework side, Mastra 1.61.0 adds a crash-resilient background task queue, and ComposioHQ/composio 0.17.0 extends SSRF protection to tool-execution downloads and S3 uploads.
Slack Code moves AI coding agents from individual terminals into shared Slack channels where teams can see diffs, previews, and plans in real time. But it solves management's visibility anxiety, not engineers' productivity bottleneck — the real battle is over who becomes the agent control plane.
DeepSeek's open-source agent harness 'dsh' crossed 20K stars within an hour of its 8/13 launch and has since accumulated ~158K stars, with 2000+ plugin proposals flooding in within two days. Its core is a Cordis-powered 'everything is a plugin' architecture that can even call Claude Code and Codex as sub-agents. RightNow-AI reimagines agents at the OS level with Rust (openfang), NetEase Youdao ships a desktop Agent built on OpenClaw (LobsterAI), and PrimeIntellect's prime-agent features a self-improving reasoning loop. CrewAI 1.15.16 adds execution context tracking and flow error logging.
Harness-IF reveals Coding Agent instruction following is overestimated by 3.6-7.4 pp because things the model would do anyway are counted as compliance; SHE decomposes the harness into four safety components and auto-evolves from trajectory failures, cutting ASR by 3.1x while improving correctness; SBCO uses a decomposed verifier bank with text gradients for harness self-improvement, matching Gödel Machine at 4-5.5x lower compute on planning tasks
Evo-Bench benchmarks nine models on self-improving harnesses — GPT-5.6 Sol tops at +16.6 but Office tasks barely move; MEGA uses a three-layer Wisdom Graph to make agent optimization infrastructure self-evolving, merging knowledge accumulation with optimization; SHE decomposes harnesses into four evolvable components that learn safety boundaries from failure trajectories, cutting ASR by 3.1x with cross-model transferability
OneDayAgent's decompose-remember-verify harness hits 0.821 new SOTA on AgentIF-OneDay and works unchanged across five backends; The Horizon Gap surveys 1,547 papers to find that six categories of long-horizon failure share a single structural pattern — outcome-only signals degrade as step count grows, driving the field toward denser process signals; Evo-Bench is the first benchmark for harness self-evolution — GPT-5.6 Sol peaks at +16.6 absolute gain, but Office tasks still need hand-crafted workflows
OpenAI wrapped the Codex harness as a JSON-RPC over stdio App Server, enabling VS Code, JetBrains, Web, and desktop apps to share a single agent loop. Three core primitives: Item, Turn, and Thread.
Agent memory isn't a plugin — it's part of the harness itself. Pick the right memory type, estimate data volume, then decide on the technology. And finally, figure out whether you actually own that memory.
An open-source Agent Harness framework from HKUDS (HKU Data Science Lab) that implements tool calling, skill loading, memory, permissions, and multi-agent collaboration as complete infrastructure, supporting Anthropic / OpenAI / GitHub Copilot API formats.
The model is the CPU, the harness is the operating system, and the agent is the application. No matter how powerful a model is, without a good harness it's just a demo. Phil Schmid argues that harness is the most critical infrastructure in AI engineering for 2026.