Skip to content
All tags

#agent-reliability

2 posts

Agent Memory and Reliability: What to Write, How Long to Keep It, and Which Layer Failed | Harvard CS2680 Lecture 6

The first half of CS2680 Lecture 6 (2026-09-28) is about agent memory. Keeping one fact means deciding, in order, whether to write it, what form it takes, where it lives, when to retrieve it, whether to generalize from it, and whether it still holds, plus who owns the memory. Don't simply delete outdated facts; record when they were true and where they came from. The second half is about reliability: at 95% success per step, a 20-step run is fully correct only about 36% of the time. Failures can be traced to five layers (task spec, harness, model, tool, environment), each with its own fixes. The two halves meet at one harness-layer failure: treating a stale note as fact.

aidebug

10% of Conversations Were Making Things Up: Debugging a Silent Retriever Failure

About 10% of conversations on our AI assistant platform randomly lost knowledge base tools — the agent hallucinated answers from training data instead. Root cause: a bare except Exception swallowed Elasticsearch connection failures during retriever initialization, silently skipping tool registration. Fix: retry + surface failures to system prompt + structured metadata tracking.