CS2680 Lecture 4 (2026-09-16) switches from using agents to designing them. An agent is a model plus a loop, and you decide which tools it gets and what it sees at each step. The tools half covers four ways to find code, four ways to edit it, the power and risk of a shell, and a set of design principles: few orthogonal tools, purpose-built over generic, token-cheap output, errors that teach. The context half covers why context is a finite resource, four ways to manage it (from guarding the door to summarizing as a last resort), and five ways context fails. It closes with a paper showing that when context is tight, the same model with only a different harness can solve 70% to more than twice as many tasks.
Claude's docs say tool-selection accuracy degrades past 30–50 tools, and both Claude and OpenAI now load tools on demand via tool search. But finding a tool isn't finishing the task: on ToolRet's 43k-tool benchmark, no retriever put the complete required toolset in its top 10 more than 35% of the time. This post splits the problem into five failure points, maps seven approaches onto them, and lays out an evaluation.
Agent interviews keep returning to eight topics: the agent loop, structured output, function calling, large tool sets, memory, design trade-offs, MCP versus A2A, and prompt caching. This post strings them into one line (concept, mechanism, how to answer) and corrects when the 2026-07 MCP spec changes actually landed.
L09 defines an AI agent in one line: the AI finishes the work you would otherwise do yourself. Yen-Lung Tsai follows Andrew Ng's four design patterns (Reflection, Tool Use, Planning, Multiagent Collaboration) but builds only the two easiest. Demo07a hands a draft between a "writer" and a "reviewer" LLM call. Demo07c splits the Lucky Vicky post generator into "think of five reasons, then write the post", a two-stage CoT. Both use AISuite with Groq and a Gradio front end. LangChain, AutoGen and CrewAI appear only on a further-learning list. The week 9 homework asks you to pick one of the two patterns.
A dialogue system that helps a user get something done is traditionally split into four modules: understand the utterance, track the state so far, decide the next action, and write the action as a sentence. A language model can act out all four in one go, and without external tools its "let me check" checks nothing. The first half of this lecture therefore covers representative systems that let a model use tools: LaMDA drafts first and then verifies and revises, BlenderBot adds search and long-term memory, WebGPT learns to search from records of human actions, and Toolformer has the model generate and filter its own training data. The second half moves from a single reply to multiple turns: the model can ask the user, learn an expert's strategy, and guess what the other party is thinking. It ends with how conversations are evaluated.
Lecture 7 of CME295 (2025) patches three LLM gaps: RAG fixes knowledge frozen at training time with a two-stage retrieve-then-rerank pipeline; tool calling fixes the inability to act by having a backend execute the function call the model writes; agents chain those calls with ReAct's observe-plan-act loop. The 2026 edition renames it AI Agents and adds context compaction, harness optimization, coding agents, and skills, the biggest rewrite in the course.
Lecture 1 of 11-768 strips an agent to its minimum: tool definitions and tool calls are just tokens, the harness parses, executes, and feeds results back into context, and running a ReAct loop makes it an agent. Neubig then lists six capabilities a good agent needs, each of which can be built through training or through the harness, and argues that an agent is a system of harness, sandbox, inference, training, and monitoring — not just a model.
Neubig splits tool use into five layers: capabilities, mechanics, constraints, interfaces, and systems. A tool call is just tokens the model emits; the harness parses, validates, and matches results back by call ID. Constrained decoding guarantees form, not correctness. MCP's real value is credential brokering. And the same model served by different providers can swing from roughly 15% tool-call errors to under 0.1%.
An agent with a workspace_browse tool said 'file not found' instead of searching. Anthropic, OpenAI, and Google's official guides all point to the same fix: put trigger conditions and workflows in the tool description. A 2025 study found 97.1% of MCP tool descriptions have quality issues.
looplane now ships a bounded tool-program DSL: read-only programs support list/read/search/diff, repeat, and if_contains; modify/check transactions receive whole-transaction approval and roll back touched paths on failure. This is not arbitrary JavaScript/Python code mode, and transaction execution is not parallel.
Looplane's disposable clone and SafePathPolicy protect the source repo. `--sandbox-checks` can now wrap verification commands with macOS sandbox-exec, Linux bubblewrap, or Landlock, while Cloudflare provides a separate bounded Sandbox slice. Network policy, external-runtime coverage, and production hardening are not yet consistent across those backends.
Looplane parallelizes calls only when they are read-only, concurrency-safe, and classified as READ. Tool programs provide bounded read-only repeat and branching, while transactions snapshot and restore possible workspace-file changes; external side effects are not rolled back.
screenshot-to-code is not a one-shot screenshot-to-HTML tool. Its core is a 30-step Agent Loop with 7 tools — extracting real assets from screenshots, self-verifying with Playwright, and running 4 models in parallel so users pick the best output. 74,500+ GitHub stars, MIT License.
Lecture 10 moves from question answering and RAG into language agents, then decomposes them into reasoning and planning, memory, tools, data, and evaluation. An agent is an inspectable loop between a model and external state.
This site covers MCP thoroughly but has never written about the layer underneath it: when your agent acts for ten thousand end users reading their own Gmail, whose database holds those refresh tokens, who rotates them, who revokes them. Composio is currently the most complete answer — MIT-licensed SDKs, a commercial hosted execution and OAuth layer. It claims 1,000+ toolkits; the managed-auth page actually lists 121 with a Composio OAuth app and 96 that require your own credentials. New pricing effective 2026-08-15: 100K free tool calls, $29/mo Pro. This post takes the authorization model down to an operational level and draws the line between wiring up MCP servers yourself and buying an integration platform.
Week 1 of CS146S is 'build Claude Code in 200 lines' plus a dissection of production system prompts. The agent loop really is that small. The course slides close with four things Claude does underneath, one of them being `<system-reminder>` tags scattered everywhere to stop the model drifting — which appears in no official documentation.
MCP governs agent-to-tool, A2A governs agent-to-agent, Skills govern reusable knowledge. The test is whether the data changes: if it changes between calls you need MCP; if it's stable enough to write down, a skill file is simpler and has no runtime that can fail on its own.
Memory Reward Inflation finds that self-improving agents' memory rewards self-inflate — wrong experiences grow more confident over time; LUCID boosts accuracy from 54.0% to 56.9% on BIRD. RoMeRL compresses memory state space with fixed-dimension semantic coordinates, cutting Cold-Q ratio by 80% and LLM calls by 21.1%. ToolLIFT abstracts tool trajectories into function-level workflow graphs, consistently outperforming existing methods on three OOD benchmarks
ToolLIFT abstracts tool trajectories into function-level workflow graphs, lifting OOD accuracy by 4+ points on average; HyperAgent builds tool-schema hypergraphs with deficit-oriented expansion, beating ReAct by 14.3 points on AppWorld with lower token cost; a multilingual multi-agent planning diagnosis finds that planning grounding failures rise with decreasing language resources, and the TART fix improves scores by 5.6 points on average
The industry has converged on using OpenTelemetry GenAI semantic conventions to turn every LLM call and tool call into a span. Detecting the three major failure modes then splits into three tracks: faithfulness + semantic entropy for hallucinations, framework-level symbolic guardrails for tool misuse, and max steps + action hash deduplication for infinite loops — all wired into a Final / Trajectory / Single-step three-layer evaluation framework.
Automatic prompt optimization (APO) has evolved from APE/OPRO to GEPA: replacing sparse rewards with linguistic reflection, winning over GRPO by ~6pp with 4-35x fewer rollouts. Meanwhile, tool descriptions are the overlooked prompt -- small wording changes can shift tool selection rates by 10x, and Anthropic's experiments show Claude self-rewriting tool descriptions outperforms human experts. These two lines are converging: eval-driven automatic optimization is eating hand-tuned prompts.
As tools scale up, selection accuracy doesn't degrade gracefully — it collapses: 4 to 51 tools drops from 43% to 2%, 10 to 100+ drops from 78% to 13.62%. The root fix is to stop stuffing everything in at once — Anthropic's Tool Search Tool uses defer loading plus retrieval to cut 85% of tokens, pushing Opus 4.5 accuracy from 79.5% to 88.1%. Description quality has conditional payoff: negligible in simple scenarios, but correctness jumps from 44% to 50% in multi-tool chaining.
Making 'chunk and embed every uploaded file automatically' the default behavior means making a decision for the LLM that it could have made itself. From Self-RAG (2310.11511) and Adaptive-RAG (2403.14403) to AgenticOCR (2602.24134), the academic trajectory is pushing three layers of decision-making -- whether to retrieve, whether to parse, and how to chunk -- from the ingestion pipeline back to the agent at conversation time.
The hard part of LLM agents is not building function calling, skills, code interpreter, and document tools individually -- it is assembling them into a system that selects the right tool, writes code when needed, decomposes tasks, verifies results, and resists prompt injection. This post organizes the key papers into six engineering decisions: function calling reliability, tool/skill selection, code-as-action, multi-step planning, skill systems, and safety plus document generation.
MCP is not going away, but its effective scope is narrower than most people think. For local development, CLI and raw API almost always beat MCP. MCP's truly irreplaceable niche is the narrow gap of 'cross-agent shared local tool layer.'
An open-source Agent Harness framework from HKUDS (HKU Data Science Lab) that implements tool calling, skill loading, memory, permissions, and multi-agent collaboration as complete infrastructure, supporting Anthropic / OpenAI / GitHub Copilot API formats.
Every one of Claude Code's 45 tools uses a prompt() method that dynamically adjusts based on user type, feature flags, and system capabilities. Applying this pattern to a ReAct Agent, tool descriptions are dynamically generated along three dimensions: orchestrator model capability, locale, and available tools. Small models automatically get few-shot examples; large models save tokens.
OpenClaw's hard requirement for a model is tool use plus a large enough context — onboarding only auto-suggests a local model when it confirms tool support and at least a 16K context window. The easier thing to get wrong is that provider, model, and agent runtime are three separate layers: an `openai/*` ref does not mean Codex.
Every AI tool has its own calling format, making integration costly. MCP (Model Context Protocol) is an open standard proposed by Anthropic that unifies the communication protocol between AI Agents and external tools/data sources, enabling tools to be reused across Agents.