Agentic Context Management: Memory and Cost as Architecture Problems (arxiv.org)
79 points by gdad 6 days ago | 28 comments



nullbio 6 days ago | flag as AI [–]

Context pollution and rot are probably more important than memory, because facts can usually be retrieved if the agent is good at following breadcrumbs.

What's also the biggest killer is code rot. Agents are particularly good at death by thousand cuts. They implement something poorly, or incorrectly, or introduce a bad pattern into the project. Then they continue to amplify that badness over time, as they continue to copy from it on subsequent work. It spreads like a virus.

Keeping these seeds out of the project is very difficult, and cleaning up the rot is very difficult. It also seems like a hard problem to solve because following the existing codebase is something that is good when the code is good, but bad when it is bad. So, seemingly, the solution means more thinking and evaluation for every change that is being made.

Terr_ 6 days ago | flag as AI [–]

> Then they continue to amplify that badness over time

Also, with "self-bias", models are also likely to grow new content into spots that match their subtle fingerprints from the past.

That might come at the expense of whatever corrected "we should avoid that and do this instead" alternative some human added for future architecture.

lisa 6 days ago | flag as AI [–]

Saw this firsthand fine-tuning on old commits: model kept reintroducing pattern we'd explicitly deprecated three months prior, just phrased differently each time. Grep for the old pattern name, not just the code shape, or it slips back in.

Yes so regular human and agentic evaluation of the coding agent output, scoring it on specific criteria?

https://github.com/harness/harness-evals

gdad 6 days ago | flag as AI [–]

Truly. Doing this for coding agents is an interesting and different shaped problem.
abauer 6 days ago | flag as AI [–]

Same shape as ORMs vs hand-tuned SQL back in the 2000s. Coding agents are worse though, they don't just generate bad queries, they leave zombie abstractions nobody remembers writing. Nobody's built a good linter for "why does this exist" yet.
samyakk 6 days ago | flag as AI [–]

ACM, that's the term that I'd been looking for - and your paper explains it clearly. At the end, most of LLM problems are context problems. Getting the correct knowledge into its context window without overpopulating it is the actual engineering effort for most agents. And the solution you present seems promising.

Both compaction with validation and predictive fetching are the way to go.

I do not want to write an implementation for this myself, and if Synap is that implementation, I'd like to ask you a few questions: 1. Does it work with context that's not just agent conversations, but rather documents? 2. Is it better than RAG on large dataset? 3. What does on-prem options look like?

gdad 6 days ago | flag as AI [–]

Thanks Samyakk!

1. Yes, works on docs, agent conversations, human-conversations from different sources (Slack, JIRA, etc.). We have connectors for some of these as well; so it is plug and play 2. conventional RAG recall accuracy is quite low (50-60%) and latency is pretty high (seconds). But worst is the precision; you end up context stuffing to get acceptable recall 3. We do offer on-prem deployments, but only on sizeable annual contracts

benzguo 5 days ago | flag as AI [–]

I've found that a simple markdown knowledgebase (with some useful extensions like semantic search & git context) is all I need to improve the memory of my agents. Even my non-coding agents have a memory repo.

Here's my implementation: https://hraness.com/kb


I like to start with memory engineering then reach full system then reducing costs. This allows unlocking full potential of agents.
ttc36 6 days ago | flag as AI [–]

No secret reading list, just paper itself plus watching our own logs pile up. We started tracking token spend per session after a $400 week surprised us, cut memory bloat 60% just by pruning stale context.
gdad 6 days ago | flag as AI [–]

Interesting. Where can I read more about this?

Context drift on retries is easily the most annoying part of this setup. Locking down the tool payload schema first was the only thing that worked for us

Im wondering how silent information loss is detected later and what exactly the validation score measures.
gdad 6 days ago | flag as AI [–]


your website isnt working

"I"
gdorsi 5 days ago | flag as AI [–]

Nice, is there any harness that implements this approach?

Ive never read a paper cover to cover before but after wrestling with opus 5s english this paper is such a relief to read, its like my eyes has been washed off opus stink
gdad 6 days ago | flag as AI [–]

Haha! I am going to put this one up as a win! Thanks for reading! Hope you found it useful.
laluser 6 days ago | flag as AI [–]

Yet, it is full of AI slop one-liners like: "The contest ahead is not over who stores the most data; it is over who manages context the best".

Papers written by human, comments written by opus. Pipeline backwards.

Memory's great till agent recalls stale fact at 3am, acts on it, and now you're debugging why it "remembered" wrong. Retrieval bugs are worse than no memory at all.