Context Engineering Part 2: Advanced Techniques for Using AI in Production

Pillar 3: COMPRESS (Reduce Token Usage)

The main idea is to keep the most important information and get rid of or summarize the rest, because every token of AI context competes for the same fixed budget.

Technique 3.1: Hierarchical Summarization

The Zoom Lens Approach: Consider you are describing your summer vacation:

Zoom Level 1 – Ultra Wide (5 words):

• “Family trip to the beach was fun”

Zoom Level 2 – Wide (50 words):

• “Spent two weeks at the beach with my family.” We swam every day, built sandcastles, ate ice cream, surfed, and saw dolphins. The best vacation ever!”

Zoom Level 3 – Medium (500 words):

• This is the whole story, including what you learnt while surfing, the funny sandcastle competition, and when you saw dolphins.

Zoom Level 4 – Full Detail (5000 words):

• Everything! Every second, every talk, every little thing, every caption on a photo…

The Smart Part:

• Want a quick summary? Use Level 1.
• Are you sending grandma an email? Use Level 2.
• Keeping a diary? Level 3 is what you should use.
• Making a book of pictures? Level 4.
• AI does the same thing!

Full Technical Specification (5000 words):
"Our company was founded in 2010 with the mission to revolutionize
cloud testing. Over the years, we've grown from a team of 5 to 500+
employees across 12 countries..."

Medium Summary (500 words):
"Testing platform founded 2010. Team of 500+ across 12 countries.
Processes 10M+ tests daily for 10K+ customers..."

Short Summary (50 words):
"Cloud testing platform. 500+ employees, 10K+ customers, 10M+ daily tests."

Ultra Short (5 words):
"Cloud testing platform, global scale"

Load the amount of detail you need for each job!

Technique 3.2: Sliding Window With Summarization

For long conversations, keep track of the details of recent messages and summarize the older ones.

The Conversation Memory Trick: Consider you are having a 2-hour phone call with your friend:

What You Remember:

Minutes 110-120 (Just Now) – Crystal Clear:

• Friend: “So should I get the blue or red shoes?”
• You: “Get the blue ones, they match your jacket!”
• Friend: “Good point! I’ll order them tonight.”

Minutes 1-109 (Earlier) – Fuzzy Summary:

• “We talked about school, weekend plans, and shopping”
• “Friend needs new shoes for the party”
• “Budget is around $50”

You DON’T Remember:

• Every single word from the first 109 minutes.
• Exact phrasing of everything.
• The tangent about weather.

What Happens:

• Recent stuff (last 10 minutes): Remember everything!
• Older stuff (first 109 minutes): Just the important summary.

Claude Code auto-compact feature implement this brilliantly that triggers at 95% context window capacity.

Technique 3.3: Tool Output Compression

Some tools give back HUGE answers. Before adding to the context, compress:

The “Report Card Summary” Approach:

Think about how your teacher grades 10,000 students on a spreadsheet:

Without Compression (The Overwhelming Way):

Show me all 10,000 students:
Row 1: John Smith, Math: 92, English: 88, Science: 91..
Row 2: Sarah Jones, Math: 85, English: 93, Science: 87..
Row 3: Mike Brown, Math: 78, English: 82, Science: 85..
[... 9,997 more rows ...]

AI Context: EXPLODED! Can't fit!

With Compression (The Smart Summary):

Query returned 10,000 student records.

Key Statistics:

- Average Math score: 84.5
- Average English score: 86.2
- Top 5 students: Sarah (94.3 avg), Mike (93.1 avg)...
- Bottom 5 students: Need tutoring support
- Grade distribution: 15% A's, 35% B's, 40% C's, 10% D's

Sample records:
Row 1: John Smith (90.3 avg) - Excellent
Row 2: Sarah Jones (88.3 avg) - Very Good

Full data saved to: student_grades.xlsx

Result: AI gets the important insights (200 tokens) instead of a lot of raw data (20,000 tokens).

Compression by Tool Type:

Code Search Results:

• Raw: 50 files, 10,000 lines.
• Compressed: “Found in 5 key files: auth.py (lines 45-120),middleware.py (lines 23-67)…”

Database Query:

• Raw: 10,000 rows.
• Compressed: “10,000 records. Stats: 8,500 active users, 1,500 inactive. Sample: [Row 1, Row 2]”.

Log Files:

• Raw: 50,000 log entries.
• Compressed: “23 ERROR logs (15 database timeouts, 5 API limits, 3 memory issues). First: 10:23 AM, Last: 11:42 AM”.

Technique 3.4: Lossy vs Lossless Compression

Lossless Compression: Get rid of extra data without losing any information.

Original: "The user wants to book a flight. The user prefers direct flights.
The user's budget is $500. The user is traveling next week."

Lossless: "User wants direct flight, $500 budget, traveling next week."

Information preserved: 100%
Token reduction: 40%

Lossy Compression: Accept some loss of information to get a big reduction.

Original: 50-page technical specification with exact implementation details

Lossy: "System processes payments via Stripe. Supports credit cards,
PayPal, and Apple Pay. Handles refunds within 30 days."

Information preserved: ~60%
Token reduction: 98%

When to Use Each:

• Lossless: Important policies, legal documents, code, and exact requirements.
• Lossy: General knowledge, background information, examples, and historical context.

Note: Test your AI agents across real-world scenarios. Try Agent Testing Today!

Key Takeaway: Compression keeps an agent inside a fixed token budget by retaining the critical information and summarizing or discarding the rest.

Pillar 4: ISOLATE (Focused Context per Task)

The main idea is to keep context from getting in the way, break up concerns into focused units.

Technique 4.1: Multi-Agent Architecture

Anthropic’s multi-agent research system shows that specialized agents with separate contexts work much better than single-agent systems. Their internal tests showed that “a multi-agent system with Claude Opus 4 as the main agent and Claude Sonnet 4 as subagents did 90.2% better than a single-agent Claude Opus 4.”

When one of those isolated subagents needs to control a real browser, the lowest-friction integration is the Playwright LangChain pattern, where Playwright actions are wrapped as LangChain tools and the subagent runs a focused ReAct loop with its own context window.

The main point is that “subagents make compression easier by working in parallel with their own context windows and looking at different parts of the question at the same time.” You can assign a narrow sub-task to each subagent’s context without having to worry about unrelated information getting in the way.

Architecture Pattern: You can consider it like a group project at school:

The Teacher (Orchestrator Agent):

• Reads the assignment: “Create a science fair project about volcanoes”
• Makes a plan and assigns tasks to different students

The Students (Specialist Agents):

• Research Student: Goes to library, finds books about volcanoes.
  • Only carries: Library card, notebook for notes.
  • Doesn’t need: Art supplies, poster board (not their job!).
• Art Student: Creates the volcano model and poster.
  • Only carries: Paint, clay, poster board.
  • Doesn’t need: Library books (already researched!).
• Data Student: Analyzes volcano eruption statistics
  • Only carries: Calculator, graph paper, the research notes.
  • Doesn’t need: Art supplies, library books.
• Quality Check Student: Reviews everything for accuracy
  • Only carries: the checklist, the completed work.
  • Doesn’t need: Any of the original materials.

• Only carries: Library card, notebook for notes.
• Doesn’t need: Art supplies, poster board (not their job!).

• Only carries: Paint, clay, poster board.
• Doesn’t need: Library books (already researched!).

• Only carries: Calculator, graph paper, the research notes.
• Doesn’t need: Art supplies, library books.

• Only carries: the checklist, the completed work.
• Doesn’t need: Any of the original materials.

Every student has their small, focused backpack! The teacher gathers everyone’s work at the end and puts it all together to make the final project. Each student only had to remember what they were supposed to do, not the whole project!

Real-World Diagram:

Task: "Write a comprehensive market analysis report"

┌─────────────────────────────────────────────────┐
│ Orchestrator Agent                               │
│ Context: Task description, plan, coordination   │
└────┬─────────┬─────────┬─────────┬──────────────┘
     │         │         │         │
     ▼         ▼         ▼         ▼
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│Research │ │Financial│ │Competitor│ │Synthesis│
│Agent    │ │Agent    │ │Agent     │ │Agent    │
│         │ │         │ │          │ │         │
│Context: │ │Context: │ │Context:  │ │Context: │
│-Search  │ │-Finance │ │-Competitor│ │-All     │
│ tools   │ │ data    │ │ data     │ │ summaries│
│-Market  │ │-Metrics │ │ frameworks│ │-Report  │
│ sources │ │ formulas│ │          │ │ template│
└─────────┘ └─────────┘ └─────────┘ └─────────┘

Each agent has isolated, focused context – no interference, no confusion!

Trade-offs of Multi-Agent Systems:

According to Anthropic’s research, multi-agent systems have significant benefits and costs:

Benefits:

• Dramatic performance improvements (90.2% improvement in Anthropic’s research eval).
• Parallel execution of independent tasks.
• Separation of concerns and cleaner context per agent.
• Can handle tasks exceeding single context windows.
• Excel at “breadth-first queries that involve pursuing multiple independent directions simultaneously”.

Costs:

• “Agents usually use about four times as many tokens as chat interactions, and multi-agent systems use about fifteen times as many tokens as chats.”
• requires complicated logic for coordination.
• Harder to build and fix
• “Compound nature of errors”: “One step failing can make agents go down completely different paths.”
• Without proper prompt engineering, there is a risk of “spawning 50 subagents for simple queries.”

When to Use Multi-Agent Systems:

Anthropic found that multi-agent systems excel at:

• “Valuable tasks that involve heavy parallelization, information that exceeds single context windows, and interfacing with numerous complex tools”.
• Open-ended research and exploration tasks.
• Tasks where multiple independent directions need exploration simultaneously.

When Not to Use Multi-Agent Systems:

• “Domains that require all agents to share the same context”.
• Tasks “involving many dependencies between agents”.
• “Most coding tasks involve fewer truly parallelizable tasks than research”.
• Simple queries where single-agent is sufficient.

Key Finding: In Anthropic’s BrowseComp evaluation, they found that token usage by itself explains 80% of performance variance. Multi-agent systems work primarily because they “help spend enough tokens to solve the problem” through parallel context windows.

Technique 4.2: Sandboxed Code Execution

HuggingFace’s CodeAgent approach shows how to isolate data-heavy operations.

The Sandbox is Like a Workshop:

Consider you’re building a huge LEGO castle:

Without Sandbox (Everything in Your Bedroom):

• 10,000 LEGO pieces scattered on your bed.
• Instructions spread across your desk.
• Half-built towers blocking your closet.
• Photos of your progress everywhere.
• Can’t even find your homework!
• Your bedroom is a disaster!

With Sandbox (Using a Separate Workshop):

• Build the entire castle in the garage (workshop/sandbox).
• Keep all 10,000 LEGO pieces there.
• All the mess stays in the garage.
• When you’re done, bring ONE THING to your bedroom:
  • A photo of the finished castle.
  • A note: “Built awesome castle, used 10,000 pieces, stored in garage”.

• A photo of the finished castle.
• A note: “Built awesome castle, used 10,000 pieces, stored in garage”.

Your Bedroom (AI’s Context) Only Sees:

• ✅ Small photo (100 KB)
• ✅ Short note (50 words)

The Garage (Sandbox) Holds:

• Entire castle.
• All the pieces.
• All the instructions.
• Progress photos.

Benefits:

• Your bedroom stays clean (AI context stays manageable).
• You can build huge things (work with massive datasets).
• Everything is saved in the garage (data persists).
• You can show others just the photo (not the whole castle).

Technique 4.3: State-Based Context Isolation

The Three-Drawer System: Consider your desk has three drawers with different rules:

Top Drawer (ALWAYS Open):

• Current homework assignment.
• Today’s schedule.
• What you did in the last 5 minutes.

This drawer is always visible. The AI sees this every time.

Middle Drawer (Open ONLY When Needed):

• Full conversation history from last week.
• Research notes from previous projects.
• Detailed data and analysis.

This drawer opens only when specifically asked. Most of the time it stays closed to keep your desk uncluttered.

Bottom Drawer (NEVER Show to AI):

• System secrets and passwords.
• Technical performance stats.
• Internal tracking numbers.

This drawer is locked. The AI never sees what’s inside.

Why This Works:

• AI’s “desk” (context) only shows the top drawer (clean and focused!).
• Need more info? Open middle drawer temporarily.
• Never clutter the workspace with locked drawer stuff.
• Everything is organized and easy to find.

Key Takeaway: Isolation gives each task its own context window, so unrelated tool output and instructions never compete for the same token budget.

Advanced Context Engineering Patterns

Now that you know what the four pillars are, let’s look at some more advanced patterns that are used in production systems:

Pattern 1: Context Tiering

Following best practices, as outlined in Daffodil Software Engineering Insights, organize information according to levels of importance:

The Five-Level Information Tower: Think of information like floors in a building – higher floors are more important:

Tier 0 – The Foundation (NEVER expires):

• “Who am I?” (The AI’s identity).
• “What am I allowed to do?” (Safety rules).
• “What can I do?” (Core abilities).
• Must ALWAYS load – This is like wearing clothes; you never skip it!

Tier 1 – The Ground Floor (Lasts 30 days):

• Company policies.
• Product documentation.
• How things work.
• Must ALWAYS load – Like bringing your student ID to school.

Tier 2 – Second Floor (Lasts 7 days):

• This week’s special offers.
• Temporary promotions.
• Current A/B tests.
• Load if backpack has room – Nice to have, not critical.

Tier 3 – Third Floor (Lasts 24 hours):

• Today’s conversation with this user.
• What we’re working on right now.
• User’s preferences for this session.
• Load if backpack has room – useful but optional.

Tier 4 – The Rooftop (lasts 5 minutes):

• Quick calculations.
• Temporary results from just now.
• Things you’ll throw away soon.
• Load if backpack has room – Very temporary.

How It Works:

• Start at the foundation (Tier 0) – must pack this!
• Add Ground Floor (Tier 1) – must pack this too!
• Got room? Add Second Floor (Tier 2).
• Still got room? Add Third Floor (Tier 3).
• Any space left? Add Rooftop (Tier 4).

The AI packs its backpack from most important to least important, stopping when the backpack is full!

Pattern 2: Long-Horizon Conversation Management

Anthropic’s production experience provides critical insights for managing extended conversations:

The Relay Race Strategy for Super Long Conversations: Consider you’re running a marathon (26 miles), but you can only run 5 miles before getting tired:

• Try to run all 26 miles yourself.
• Get exhausted at mile 5.
• Collapse! Can’t finish.

Runner 1 (Miles 1-5):

• Runs fresh and energetic!
• At mile 5: Writes summary note.
  • “Passed 3 water stations”.
  • “Route goes through park, then downtown”.
  • “Current pace: 8 min/mile”.
• Saves note to locker.
• Passes baton to Runner 2.

• “Passed 3 water stations”.
• “Route goes through park, then downtown”.
• “Current pace: 8 min/mile”.

Runner 2 (Miles 6-10):

• Starts fresh!
• Carries: Just the summary note (light!).
• Doesn’t carry: Every detail from miles 1-5 (too heavy!).
• At mile 10: Adds to the note, saves to locker.
• Passes baton to Runner 3.

Runners 3, 4, 5… Continue the pattern.

What Happens:

• Each runner only remembers their 5-mile section (small backpack!).
• Important info saved in locker (external memory).
• If needed, any runner can check the locker.
• The marathon gets finished!

Anthropic’s Three-Part Strategy:

• Phase Summarization: “Finished Phase 1: Found 10 sources on topic X” (store summary, forget details).
• Fresh Context Spawning: When the backpack is full, a new AI is spawned with a clean backpack and a summary note.
• Memory Retrieval: Need more information from Phase 1? Look in the locker! Don’t always carry it around. This is how AI can talk to people with hundreds of messages without losing track of what they are saying!

Key Takeaway: Context tiering and long-horizon conversation management let an agent load foundation rules first and page in optional detail only when space allows.

Production Challenges for Multi-Agent Systems

Building multi-agent systems that work in production requires solving challenges beyond basic Context Engineering. Anthropic’s engineering team shares critical lessons from deploying their research system.

Challenge 1: Stateful Errors Compound

The Problem:

Unlike traditional software where you can restart on error, agents can’t restart from the beginning – it’s “expensive and frustrating for users”.

The Solution – The Video Game Save Point Strategy:

Consider you are playing a video game with 20 levels:

Without Checkpoints (The Nightmare):

• Play from Level 1 to Level 18.
• Game crashes at Level 18.
• Start over from Level 1.
• Takes 2 hours to get back to where you were!

With Checkpoints (The Smart Way):

• ✅ Level 5 completed → Auto-save!
• ✅ Level 10 completed → Auto-save!
• ✅ Level 15 completed → Auto-save!
• Game crashes at Level 18.
• Restart from Level 15 save point!
• Only replay 3 levels (10 minutes).

Here are scenarios when things go wrong:

Scenario 1 – Tool Breaks:

• AI tries to use a hammer.
• The hammer is broken!
• AI says: “Okay, I’ll use a screwdriver instead.”
• Adapts and continues!

Scenario 2 – System Crashes:

• Working on Step 18 of 20.
• System crashes.
• Load last save (Step 15).
• Resume from there, not from Step 1!

Key Insight From Anthropic: “Letting the agent know when a tool is failing and letting it adapt works surprisingly well.” The AI is smart enough to find another way – just tell it what’s broken!

Challenge 2: Non-Deterministic Debugging

The Problem:

Users say, “The AI didn’t find obvious information” but when you try, it works fine. What happened?

The Solution: The Detective’s Notebook (Without Reading Private Diaries)

The problem is like: Consider your robot toy sometimes goes left and sometimes goes right, even with the same button press. How do you fix it if you can’t predict what it’ll do?

The Solution – Track Patterns, Not Content:

Instead of reading every private conversation (creepy!), track the patterns:

Here are aspects we track:

Decisions Made:

• “Used Google 73% of the time, Wikipedia 20%, ignored other tools 7%”.
• “Created 3 helper robots on average for complex tasks”.
• “Chose Strategy A vs Strategy B split: 60/40”.

Interaction Patterns:

• “Main robot → Helper robot handoff took 2 seconds on average”.
• “Used Tool 1, then Tool 2, then back to Tool 1 (inefficient!)”.
• “Context grew from 1000 words → 5000 words → 20,000 words”.

Performance Stats:

• “Each search took 1.5 seconds”.
• “Tool X failed 5% of the time”.
• “Average task: 15 steps, 3 minutes”.

Privacy Protected:

• We see: “User asked about topic category: Travel”.
• We DON’T see: “User asked about honeymoon in Paris”.

Anthropic emphasizes: “We monitor agent decision patterns and interaction structures, all without monitoring the contents of individual conversations, to maintain user privacy.”

The Detective Work:

• The pattern shows: When the context is more than 100k words, AI starts repeating old actions.
• Fix: Add checkpoint to summarize when reaching 100k.
• Problem solved! No need to read private conversations.

Challenge 3: Deployment Coordination

The Problem:

You can’t update all agents simultaneously without breaking running tasks.

The Solution: The Two-Playground Strategy‘

Here is the problem: Consider a theme park where 100 people are on different rides:

• Person 1: Halfway through the rollercoaster.
• Person 2: Just started the carousel.
• Person 3: Almost done with the ferris wheel.

Now you want to upgrade all the rides with new features. But you can’t:

• Stop everyone mid-ride (they’d be angry!).
• Swap rides while people are on them (dangerous!).
• Make everyone start over (frustrating!).

Rainbow Deployment (The Smart Way):

Step 1: Build a second, upgraded theme park next door.

Step 2: Make a simple rule:

• Anyone CURRENTLY on a ride? → Finish on OLD theme park.
• Anyone NEW arriving? → Send to NEW theme park.

Step 3: Wait patiently.

• Old park: People gradually finish and leave.
• New park: New visitors are having fun with upgrades!

Step 4: When the old park is empty:

• Close it down.
• Everyone’s now in the new park!

Nobody’s ride was interrupted. This is exactly how Anthropic deploys updates: “Gradually shifting traffic from old to new versions while keeping both running simultaneously” so no one’s work gets interrupted.

Challenge 4: Synchronous Bottlenecks

The Current State: Anthropic notes that currently their “lead agents execute subagents synchronously, waiting for each set of subagents to complete before proceeding.”

The Problem:

• The lead agent can’t steer subagents mid-execution.
• Subagents can’t coordinate with each other.
• The entire system blocked by slowest subagent.
• Missed opportunities for dynamic parallelism.

The Future:

• Asynchronous execution enabling concurrent work.
• Agents creating new subagents on-demand.
• Dynamic coordination during execution.
• But adds complexity: “result coordination, state consistency, and error propagation”.

Lessons from Anthropic’s Multi-Agent System

Anthropic’s research demonstrates that using multiple specialized agents with separate contexts significantly improves performance. By isolating responsibilities, parallelizing tasks, and managing context individually, multi-agent systems handle complex, large-scale workflows more efficiently and reliably than single-agent setups.

1. Think Like Your Agents

Build simulations with exact prompts and tools, and watch agents work step-by-step. This “immediately revealed failure modes: agents continuing when they already had sufficient results, using overly verbose search queries, or selecting incorrect tools.”

2. Teach the Orchestrator How to Delegate

Vague instructions like “research the semiconductor shortage” led to duplicated work and gaps. Instead, each subagent needs:

• Clear objective.
• Output format specification.
• Tool and source guidance.
• Explicit task boundaries.

3. Scale Effort to Query Complexity

Embed scaling rules in prompts:

• Simple fact-finding: 1 agent, 3-10 tool calls.
• Direct comparisons: 2-4 subagents, 10-15 calls each.
• Complex research: 10+ subagents with divided responsibilities.

4. Tool Design is Critical

“Agent-tool interfaces are as critical as human-computer interfaces.” The right tool makes tasks efficient; often it’s strictly necessary.

5. The Last Mile is Most of the Journey

Key Takeaway: Multi-agent systems fail in production mostly because stateful errors compound across steps, so error recovery matters more than the reasoning quality of any single agent.

How Do Agent Frameworks Handle Context Limits Now?

Agent frameworks handle context limits with three documented techniques: compaction, structured note-taking, and sub-agent architectures. Anthropic publishes all three as the standard approach for long-horizon agent work in its effective context engineering guidance, linked in the references below.

• Compaction: The agent summarizes a conversation nearing the context window limit and restarts a new window from that summary. Claude Code keeps architectural decisions, unresolved bugs, and implementation details, then continues with the compressed context plus the five most recently accessed files.
• Structured note-taking: The agent writes notes to memory outside the context window and pulls them back when a later step needs them. Anthropic reports a Pokemon-playing agent keeping tallies and maps of explored regions this way across thousands of game steps.
• Sub-agent architectures: Specialized sub-agents run focused work in clean context windows and return only a condensed summary, often 1,000 to 2,000 tokens, to the coordinating agent.

Pick the technique by workload. Compaction suits extensive back-and-forth conversation. Structured note-taking suits iterative development with clear milestones. Sub-agent architectures suit complex research and analysis, which is also where multi agent AI systems carry the most coordination cost.

All three techniques apply COMPRESS and ISOLATE at the session level instead of the single prompt level. The four pillars described in context engineering in AI stay the same. What changed is that agent frameworks now ship these behaviours by default rather than leaving each team to build them.

Key Takeaway: Compaction, structured note-taking, and sub-agent architectures are the three documented ways an agent keeps working once a conversation reaches the context window limit.

Common Mistakes and How to Fix Them?

Even well-designed AI agents fail when context handling goes wrong. These are the most frequent mistakes teams make when managing context at scale, and how to fix them with practical, production-tested methods.

Mistake 1: Treating All Context Equally

• Wrong: Load everything with equal priority.
• Right: Prioritize critical info; load optional info only if space permits.

The Backpack Analogy:

• Don’t pack your winter coat and beach toys equally for a summer trip.
• Pack summer essentials first; add extras if there’s room.

Mistake 2: Static Context Management

• Wrong: Use the same context for every task.
• Right: Adapt context to each task’s needs.

The Analogy:

• Don’t bring your entire closet to school.
• Gym class? Bring gym clothes.
• Art class? Bring art supplies.
• Math class? Bring a calculator.

Mistake 3: No Context Lifecycle Management

• Wrong: Keep adding context forever, never removing.
• Right: Regularly clean up old, irrelevant context.

The Analogy:

• Don’t keep last week’s lunch leftovers in your backpack.
• Remove old items, add fresh ones.

Mistake 4: Ignoring Context Versioning

• Wrong: Overwrite information without tracking changes.
• Right: Keep version history so you can roll back.

The Analogy:

• Like having “Track Changes” in Word documents.
• Can see what changed and when.
• Can undo if something breaks.

Mistake 5: No Context Observability

• Wrong: Treat context as a black box.
• Right: Monitor what’s in context, measure effectiveness.

The Analogy:

• Like checking your backpack weight before hiking.
• Too heavy? Remove something.
• Missing essentials? Add them.

Key Takeaway: The most common context engineering mistake is treating all context equally instead of prioritizing what the current step actually needs.

Measuring Success: Is Your Context Engineering Working?

Track these metrics to know if you’re on the right track:

Efficiency Metrics

These metrics show how effectively your AI is using and managing its context window for optimal performance.

Context Utilization:

• How much of the available context window are you using?
• Target: 70-90% (not too empty, not overflowing).

Information Density:

• How many unique facts per 1000 tokens?
• Higher density = better packing.

Retrieval Precision:

• How many retrieved chunks were actually used?
• Target: >80% precision (don’t retrieve junk).

Context Freshness:

• Average age of context items.
• Fewer stale items = better.

Redundancy Rate:

• How much duplicate information?
• Lower redundancy = more efficient.

Quality Metrics

These metrics measure the accuracy, relevance, and consistency of the AI’s responses based on the loaded context.

Relevance Score:

• How much loaded context was actually referenced in the response?
• Target: >70% relevance.

Sufficiency Score:

• Did the AI have enough information to answer properly?
• Check for incomplete or uncertain answers.

Consistency Score:

• Any contradictions in the context?
• Detect conflicting information automatically.

Conclusion: The Art Meets Science

Context Engineering is where the art of AI system design meets the science of optimization.

The Art:

• Understanding user needs and workflows.
• Designing intuitive information architectures.
• Balancing competing priorities (speed vs accuracy).
• Creating elegant solutions to complex problems.

The Science:

• Measuring token usage and costs.
• Optimizing retrieval algorithms.
• Testing different strategies empirically.
• Analyzing performance data.

The evidence is clear: as Drew Breunig’s research compilation shows, even frontier models with million-token context windows suffer from context poisoning, distraction, confusion, and clash. Simply having a large context window doesn’t solve the problem – you need thoughtful Context Engineering.

Key Takeaways from Part 2

• COMPRESS saves tokens while preserving meaning.
• ISOLATE prevents interference between different concerns.
• Production is hard – prototype success doesn’t guarantee production reliability.
• Measure everything – you can’t optimize what you don’t track.
• Learn from failures – track patterns to identify and fix issues.

The Four Pillars Together

• WRITE: Organize and save information.
• SELECT: Retrieve only what’s relevant.
• COMPRESS: Make it smaller without losing meaning.
• ISOLATE: Separate concerns to prevent interference.

Remember: An AI’s context window is like a backpack. Pack smart, not heavy. At TestMu AI, we’re committed to applying these principles across our AI-native products, continuously pushing the boundaries of what’s possible when context is engineered thoughtfully.

Further Reading and References

Essential Resources

• How Long Contexts Fail (and How to Fix Them): Drew Breunig’s essential guide covering the four context failure modes with extensive research citations, including DeepMind, Berkeley, Microsoft/Salesforce, and Databricks studies.
• Context Engineering for AI Agents: A comprehensive guide covering the four pillars (WRITE, SELECT, COMPRESS, ISOLATE) with implementation patterns.
• Anthropic’s Multi-Agent Research System: Deep dive into building production multi-agent systems (90.2% performance improvement).
• Anthropic’s Effective Context Engineering: Additional strategies from the Claude team.
• Microsoft’s AI Context Engineering
• Daffodil Software: Context Engineering Best Practices: Industry best practices.

Research Papers and Studies

• DeepMind Gemini 2.5 Technical Report: Context poisoning in game-playing agents.
• Anthropic Multi-Agent Eval: 90.2% performance improvement over single-agent.
• Berkeley Function-Calling Leaderboard: Every model performs worse with more tools.
• Microsoft/Salesforce Sharded Prompts Study: 39% performance drop from context clash.
• Anthropic BrowseComp Evaluation: Token usage explains 80% of performance variance.
• Hugging Face CodeAgent Paper: Sandboxed execution for context isolation.

Co-Author: Sai Krishna

Sai Krishna is a Director of Engineering at TestMu AI. As an active contributor to Appium and a member of the Appium organization, he is deeply involved in the open-source community. He is passionate about innovative thinking and love to contribute to open-source technologies. Additionally, he is a blogger, community builder, mentor, international speaker, and conference organizer.