Learning While You Sleep: Memory, Dreaming, and Agent Intelligence
Context engineering—how agents manage and learn from memory—is the key multiplier of model intelligence in production. Lamis Mukta walks through the evolution from simple markdown files to autonomous memory systems, production guardrails (versioning, concurrency, permissions), and a new paradigm called 'dreaming': an out-of-band batch process that reviews agent transcripts, identifies patterns, and proposes memory updates so agents improve over time without competing for in-session resources.
Why Context Engineering Matters
Model Intelligence Alone Doesn't Compound
Releasing smarter models doesn't automatically make deployed agents smarter. Agents need contextual knowledge specific to your organization, codebase, and user preferences—orthogonal to raw model capability—to perform well. This context is the multiplier that makes intelligence compound over time.
The Core Problems Context Solves
Agents struggle with three key issues: not knowing how to navigate your codebase or understand user preferences, lacking continual learning (not improving on repeated tasks), and not learning from mistakes across sessions. Context engineering directly addresses all three.
Evolution of Context Engineering (Past Year)
CLAUDE.md: The Unreasonably Effective Start
A simple markdown file injected at session start—containing instructions about your codebase, organization, and user preferences—proved remarkably effective at steering agent behavior. The simplicity and human-readability made it powerful, but it hit scaling limits as files grew large and context window pressure increased.
Memory Tools: Autonomous In-Band Management
Agents were given tools to autonomously decide when to read, write, and update memories within a session. This in-band approach (happening during the session) worked well because agents had agency over their own memory, but it created a resource-allocation problem: agents had to balance task completion against memory curation.
Skills: Progressive Disclosure for Large Contexts
Skills solve context bloat through progressive disclosure: agents see only a brief summary (front matter) before deciding to load a skill, but the full procedural detail remains available. This is like scanning a bookshelf title before pulling a book—you don't load all context upfront, only what's relevant.
File Systems as State-of-the-Art Memory
The current best practice treats memory as a file system: agents use standard tools (bash, grep) to search and index markdown files. This aggregates learnings from prior approaches—markdown readability, agent autonomy, and progressive disclosure via intelligent search—without being opinionated about custom memory tools.
Production Guardrails for Memory Systems
Versioning: Track Changes and Rollback
Every memory update must be versioned with metadata: what context triggered it (which session, transcript), who made it (agent or human), and a hash of the state before and after. This enables rollback if an update is bad, malicious, or incorrect, and provides an audit trail.
Concurrency: Hashing to Prevent Conflicts
When thousands of agents write to shared memory simultaneously, use a hash-based commit system: agent takes a hash before editing, drafts the change, takes another hash, and only commits if hashes match. If they don't match, another agent updated memory in the meantime, so the agent re-reads, redrafts, and retries.
Permissions: Hierarchical Access Control
Memory hierarchies span from organization-wide knowledge (read-only, carefully curated) to agent scratchpads (read-write, individualized), with intermediate levels for teams or domains. Permissions prevent one agent from corrupting shared context and ensure sensitive updates require human review.
Portability: Clean APIs for Multi-System Access
Memory systems should be accessible across multiple product surfaces and systems via clean APIs. Since memory curation is a long-term investment, portability ensures your carefully built context isn't locked into one tool.
Production Results from Effective Memory
Measurable Improvements Across Metrics
Deploying large-scale memory systems in production yields better accuracy (agents perform tasks better on repeat runs), lower latency and cost (fewer tokens needed, one-shot capability), and freed developer capacity (agents self-learn while developers focus on product wins).
The In-Band Memory Bottleneck
Competing Resources and Limited Visibility
In-band memory (agents reading and writing within a session) creates two problems: agents must split focus between completing the task and curating memory for future runs, and they only see patterns within their own session, missing cross-session failures or fleet-wide issues that other agents encounter.
Why Out-of-Band Memory Helps
An out-of-band process (running separately, asynchronously) has dedicated capacity and visibility across all agent sessions and transcripts. It can spot patterns no single agent sees and propose memory updates without competing for in-session resources, analogous to how teachers and head teachers review student work to improve curriculum.
Dreaming: The New Paradigm
What Dreaming Is
Dreaming is a batch, out-of-band process that takes a memory store and a collection of agent session transcripts, analyzes them for patterns and gaps, and proposes updates to make the memory more effective. It runs asynchronously with dedicated resources, ensuring memories stay fresh, complete, and aligned with agent performance.
Dreaming in Practice: Three Examples
Example 1: All geography students answer a question incorrectly; dreaming notices the topic is missing from curriculum and adds it. Example 2: All math answers are in radians instead of degrees; dreaming notices tool misconfiguration and proposes a fix. Example 3: Fleet-wide issue (e.g., too many dashes); dreaming proposes an organization-wide context change. Dreaming scrutinizes tool calls and metadata, not just agent responses.
Designing Dreaming in Production
Configure which transcripts to include (respecting permissions), deploy subject agents to analyze them, and have an orchestrator review findings. You can steer the dreaming process by telling it what is and isn't important for your organization. The system outputs proposed changes with evidence (example transcripts and prevalence stats), letting you accept or reject each one.
Why Dreaming Isn't Expensive
While dreaming consumes extra tokens, it pays for itself: agents with better memories one-shot tasks more effectively, need fewer tokens overall, and perform better. The upfront cost of dreaming is offset by downstream efficiency gains.
Memory and Dreaming in Parallel
Two Complementary Processes
In-band memory lets agents write updates that improve their next run immediately, but they have limited visibility and competing resources. Out-of-band dreaming has broad visibility and dedicated capacity to spot fleet-wide patterns and propose updates. Together, they create a feedback loop: agents learn session-to-session, and dreaming ensures the memory store itself evolves and stays accurate.
Key Takeaways and Scope
Start Simple, Scale Thoughtfully
Begin with simple context management (CLAUDE.md, skills, autonomous memory tools). As you scale to many agents, long-running sessions, or complex domains, add production guardrails: versioning, concurrency control, permissions, and portability. Finally, add dreaming to consolidate and improve memory at scale.
Memory Is Not Coding-Specific
While memory systems are powerful for coding tasks, they apply broadly: presentations, customer service, domain expertise, etc. Any task where agents benefit from accumulated knowledge and preferences is a candidate for memory and dreaming.
An Open Research Area
Context engineering, especially dreaming, is very new (emerged in the past year) and remains an active area of research and development. The field welcomes contributions and experimentation.
Notable quotes
Context engineering is the multiplier of model intelligence even as models get smarter. — Lamis Mukta
Do the simple thing that works. Markdown is great for reading memories. — Lamis Mukta
Keep thinking, keep learning, and keep dreaming. — Lamis Mukta
Action items
- Implement a simple CLAUDE.md file or markdown memory store for your agents with organization, codebase, and user preference context.
- Give agents autonomous memory tools (read, write, update) and let them decide when to use them within sessions.
- As you scale to multiple agents, add production guardrails: versioning (track what changed and why), concurrency control (hashing to prevent conflicts), and permissions (read-only for shared context, read-write for individual scratchpads).
- Design your memory system as a file system with clean APIs so it's portable across product surfaces and accessible by multiple systems.
- Set up an out-of-band dreaming process: periodically collect agent transcripts, analyze them for patterns and gaps, and propose memory updates with evidence and prevalence stats.
- Configure dreaming to respect the same permissions as your agents and to focus on what matters for your organization.