Token Optimization and Context Engineering

Token fundamentals and cost accounting

Context overflow can truncate useful evidence, fail requests, or crowd out output. Cost estimation needs provider-specific prices and definitions; keep unit prices configurable and dated. Also measure latency, retries, model repair rate, and human review—not tokens alone.

estimated_cost = Σroute[(input_tokens × input_rate) + (output_tokens × output_rate)
                        + cached_tokens × cached_rate + tool/compute charges]
​
orchestration_tokens ≈ root_context + Σworkers(worker_context + worker_output)
                     + integration_context + validation/retry_context

This is accounting notation, not a provider bill formula: rates, cache discounts, reasoning usage, and tool charges vary by product and can change.

def orchestration_tokens(root_input, worker_inputs, worker_outputs, integration, validation):
    return root_input + sum(worker_inputs) + sum(worker_outputs) + integration + validation
​
example = orchestration_tokens(8000, [5000, 4000], [700, 500], 1800, 1200)

The calculation deliberately adds measured or estimated usage from every phase; production accounting should also preserve per-model and cached-token dimensions.

Optimization techniques

For every optimization, evaluate both tokens and task quality. Use a held-out benchmark and compare successful completion, regressions, false confidence, latency, and cost per accepted result.

Context engineering and trust boundaries

Assemble context from explicit layers, in order of authority and relevance:

1. System/developer policy and tool permission envelope.
2. User goal and constraints.
3. Project instructions relevant to the touched paths.
4. Repository facts: commit, symbol map, targeted source/tests, relevant diff.
5. Retrieved documentation with source, version/date, and access labels.
6. Tool results and prior agent reports, clearly marked as untrusted observations.
7. Task memory: decisions, evidence, unresolved questions, and next step.

Never merge a README, webpage, tool output, or subagent response into the instruction layer. Repository content can contain accidental or malicious instructions; treat it as data to analyze. Keep task context isolated by project/tenant, redact secrets before model access, remove stale facts, and preserve provenance. Summaries should say “observed in file/version” rather than turn claims into authoritative instructions.

Context package example

{
  "goal": "Add loading cancellation to the profile screen",
  "commit": "<current-commit-id>",
  "owned_paths": ["App/Profile/ProfileViewModel.swift", "Tests/ProfileTests.swift"],
  "read_only_references": ["App/API/ProfileClient.swift", "App/Profile/ProfileView.swift"],
  "constraints": ["Preserve existing API contract", "Do not log profile payloads"],
  "acceptance": ["Cancel stale request", "Test completion and cancellation"],
  "untrusted_observations": ["Current implementation uses task cancellation in the view"],
  "excluded": ["DerivedData", "build output", "unrelated feature modules"]
}

This compact structure provides task boundaries, not a substitute for repository-level authorization or validating the commit/path state.

Repository-aware coding agents

Reading every file is wasteful when most files are unrelated and can introduce more stale or adversarial text. Begin with the current directory map, build/test instructions, git status/diff, and project instructions. Search symbols and references; then inspect relevant declarations, callers, tests, protocol implementations, generated-code boundaries, and configuration that changes behavior. Use AST or language-server indexing for accurate symbol references where available; text search remains useful but is not complete for dynamic dispatch, macros, generated code, or reflection.

For Swift, search the view/model/repository protocol and test double, then inspect target membership, actor isolation, availability, and call sites. For TypeScript, trace exported symbol → route/schema → service → repository/query and its tests; inspect package scripts and lockfile rather than every dependency. Before editing, record in-scope paths. After editing, inspect git diff, run the smallest relevant test first, then broader validation based on risk. Do not assume code search results are complete without checking module boundaries and generated source.

Illustrative repository context budget

Assumptions for comparison only: a hypothetical repository has 200 files averaging 1,500 tokens of source each (=300,000 source tokens before tool metadata); a targeted five-file package averages 1,500 tokens/file (=7,500); model/tool overhead, history, summaries, and output are excluded. Actual tokenizer and source sizes vary.

These are not universal savings claims and do not compare reasoning or answer quality. Scenario D is sensible only if the tasks are truly independent and file ownership is non-overlapping. If the workers all need the same source, measure whether shared context/cache or one sequential agent is cheaper.

Subagent token efficiency

Budget formula: total = root input/output + Σ(worker input/output + tool results) + integration + validation + retries. Record actual token usage by run, task, route, and cache status where available. Estimate total cost with the provider's current pricing, not a hard-coded guide number.

Token budget manager

Reserve before delegation, account actual usage after each call, and never allow workers to borrow an unlimited shared pool. Protect a validation/report reserve so a task does not exhaust its budget before checking output. Prioritize correctness/security gates over optional polish; if budget is exhausted, stop or request a model/budget decision rather than silently weakening quality.

type Budget = { input: number; output: number; calls: number };
type Usage = Budget;
​
class BudgetManager {
  private reserved = new Map<string, Budget>();
  private spent: Budget = { input: 0, output: 0, calls: 0 };
​
  constructor(private readonly limit: Budget, private readonly validationReserve: Budget) {}
​
  reserve(taskId: string, amount: Budget): void {
    const current = [...this.reserved.values()].reduce(add, this.spent);
    const ceiling = subtract(this.limit, this.validationReserve);
    if (!fits(add(current, amount), ceiling)) throw new Error("Budget reservation denied");
    this.reserved.set(taskId, amount);
  }
​
  settle(taskId: string, actual: Usage): void {
    const held = this.reserved.get(taskId);
    if (!held) throw new Error("No reservation for task");
    this.reserved.delete(taskId);
    this.spent = add(this.spent, actual); // production code validates non-negative values
    if (!fits(this.spent, this.limit)) throw new Error("Actual usage exceeded global budget");
  }
}
​
function add(a: Budget, b: Budget): Budget {
  return { input: a.input + b.input, output: a.output + b.output, calls: a.calls + b.calls };
}
function subtract(a: Budget, b: Budget): Budget {
  return { input: a.input - b.input, output: a.output - b.output, calls: a.calls - b.calls };
}
function fits(a: Budget, limit: Budget): boolean {
  return a.input <= limit.input && a.output <= limit.output && a.calls <= limit.calls;
}

This is illustrative single-process pseudocode. A production manager needs atomic durable reservations, model-specific input/output rates, provider usage reconciliation, concurrency safety, currency limits, partial usage on failure, and escalation rules. Token limits do not equal dollar limits.

{
  "task_budget": { "input_tokens": 24000, "output_tokens": 6000, "tool_calls": 18 },
  "validation_reserve": { "input_tokens": 5000, "output_tokens": 1500, "tool_calls": 4 },
  "per_agent": {
    "architecture_review": { "input_tokens": 5000, "output_tokens": 1200, "tool_calls": 2 },
    "implementation": { "input_tokens": 10000, "output_tokens": 2500, "tool_calls": 10 },
    "security_review": { "input_tokens": 4000, "output_tokens": 1000, "tool_calls": 2 }
  },
  "on_exhaustion": "stop_and_report_with_evidence",
  "human_approval_required_for": ["external_write", "deployment", "destructive_change"]
}

Numbers are a sample configuration, not recommended universal budgets.

Model routing by quality-per-successful-task

Route a short deterministic extraction to a small model only after evaluation. Route difficult reasoning, high-impact design, or tool-plan ambiguity to a stronger approved model when its latency and privacy characteristics fit. Use local models for data/control constraints only after measuring task quality and operational cost. If confidence is low, abstain, retrieve more, or ask a human; automatic escalation must respect the same budget and data policy.

Context/token interview hit points

🧠 Say: “I minimize irrelevant context by locating symbols and dependencies, but I keep the contract, tests, instructions, and enough surrounding code to verify behavior. I judge token savings by cost per accepted, correct change—not by a smaller prompt.”

References

• OpenAI: Prompt engineering
• OpenAI: Prompt caching
• OpenAI: Models and pricing
• Anthropic: Context windows
• OWASP Top 10 for LLM Applications