Handle multi-turn conversations effectively by maintaining relevant context without overwhelming memory.
Managing Conversation Context and Memory
1. Purpose
Effective context management determines whether an agent provides coherent, relevant responses or disjointed, repetitive ones. This module defines how to maintain, prioritize, and prune conversational context across turns and sessions.
2. Context Layers
Layer
Scope
Persistence
Example
Immediate
Current turn
Until response sent
User's current question
Conversational
Current session
Until session ends
Previous Q&A in this chat
Persistent
Cross-session
Until explicitly cleared
User preferences, name
Global
All users
Permanent
Knowledge base, documentation
3. Context Window Management
Agent context windows have finite capacity. Prioritize information:
Priority 1: Current user query (always include)
Priority 2: Most recent 3-5 conversation turns
Priority 3: User preferences and persistent context
Priority 4: Relevant knowledge base excerpts
Priority 5: Older conversation history (summarized)
Priority 6: General system instructions
4. Memory Operations
Operation
When
How
Store
User provides preference or fact
Write to persistent memory with key
Recall
Relevant to current query
Retrieve by semantic similarity or key
Update
User corrects stored information
Overwrite with new value, log change
Forget
User requests deletion
Remove from all storage layers
Summarize
Context window filling up
Compress older turns into summary
5. Context Summarization Protocol
When conversation exceeds context window:
Keep the last 3 complete turns verbatim
Summarize turns 4-10 into key points:
Decisions made
Information provided
Actions taken
Unresolved questions
Compress turns 11+ into a single paragraph overview
Always retain: user name, stated preferences, current task goal
6. What to Remember Across Sessions
Remember
Don't Remember
User's name (if provided)
Specific conversation details
Stated preferences
Temporary workarounds
Technical environment (OS, tools)
One-time questions
Accessibility needs
Sensitive information
Communication style preference
Off-topic discussions
7. Context Relevance Scoring
Before including context in a response, score its relevance:
Score
Meaning
Action
0.9–1.0
Directly relevant
Include verbatim
0.7–0.8
Partially relevant
Include summarized
0.5–0.6
Tangentially relevant
Include if space permits
< 0.5
Not relevant
Exclude
8. Handling Context Conflicts
When stored context contradicts current input:
Prefer current input — User may have changed their mind
Acknowledge the change — "Previously you mentioned X. I'll use Y going forward."
Update stored context — Replace old value with new
Don't challenge — Unless the conflict might cause errors
9. Privacy in Context Management
Never store passwords, tokens, or financial data in persistent memory
Allow users to view what's stored about them
Allow users to delete any stored information
Don't infer sensitive attributes (health, politics) from context
Comply with data minimization: store only what improves service
10. Error Cases
Scenario
Response
Context retrieval fails
Operate with current turn only, inform user
Contradictory persistent data
Ask user to clarify current preference
Context window overflow
Apply summarization protocol
User requests context export
Provide all stored data in structured format
Stale context (> 30 days)
Verify before using: "Last time you preferred X. Still accurate?"
Protocol: Managing 1 Million Token Context Windows — Structured rules for AI agents operating within extended context windows. Covers memory management, context prioritization, and coherence maintenance across long sessions.