LangChain Middleware Patterns (Python)

LangChain Middleware Patterns (Python)

Overview

Tenant A sends a prompt: "Summarize this support ticket from [email protected] about her overdue invoice." The chain's caching middleware ran before the PII redaction middleware, so the raw prompt — email and all — became part of the cache key. Thirty seconds later Tenant B sends a semantically identical prompt (different tenant, different customer, same shape). Cache hits. Tenant B's user gets back a summary that names [email protected] and her overdue invoice. That is pain-catalog entry P24 in production, and it is a real class of incident — post-mortems read like "we added caching to cut cost, leaked a customer's PII to a different tenant within an hour."

The sibling failure modes:

• P25 — Retry middleware runs the model call twice on a 429; both attempts fire on_llm_end; the token-usage aggregator sums both; a single logical call bills as two, tenant's per-session budget trips at 50% of true usage.
• P10 — Agent loops exceed 15 iterations on vague prompts. There is no default cost cap. A per-session token-budget middleware solves this; without one, a single "help me with my account" prompt can burn thousands of tokens.
• P34 — Runnable.invoke does not sanitize prompt injection. A RAG document containing "Ignore previous instructions and..." is followed verbatim. Guardrails middleware is your injection defense; without it, indirect prompt injection is a one-line exploit.
• P61 — set_llm_cache(InMemoryCache()) hashes the prompt string only. Two chains with different tool bindings return the same cached response; tools are silently ignored by the cache key.

This skill defines the canonical middleware order for LangChain 1.0 chains and LangGraph 1.0 agents, with an ordering-invariants matrix (every adjacent pair has a named failure mode if you swap them), six reference implementations, a cache-key hash that includes prompt plus bound-tools plus tenant_id, retry telemetry that deduplicates by request_id, and an integration test pattern that asserts the ordering invariant on every build.

Pin: langchain-core 1.0.x, langchain 1.0.x, langgraph 1.0.x. Pain-catalog anchors: P10, P24, P25, P34, P61, with supporting references to P27, P29, P30, P33.