---
name: agentsope/agentsop-query-routing
source: https://app.decimal.ai/s/agentsope-agentsop-query-routing@1/SKILL.md
source_sha256: 342c356a4735
---

# Query-Type Routing · Enhancement Overlay

> Third-person operating model for a coder agent that owns a multi-handler
> answering surface. Audience is the LLM agent writing/reviewing the routing
> code — not the end user.

> **One sentence**: *A retriever is shaped by the query type it was built for;
> a summary index, a vector index, and a text-to-SQL engine are not
> interchangeable — so classify the query and route first, then retrieve.*

This is an **ENHANCE overlay**. It distills the cross-framework *routing
pattern* from three source skills. For the per-framework API, cross-link the
base skill: `[[llamaindex]]` (`RouterQueryEngine`), `[[agentsop-dify]]` (Question
Classifier node), `[[agentsop-langgraph]]` (conditional edges).

---

## 1. 何时激活 (Activation Rules)

Activate when **any** of the following holds:

1. The system has **multiple indices / tools / engines** behind one entry point,
   and a query must be dispatched to exactly one (or a few) of them.
2. Inbound queries **differ in kind** — at least two of: *lookup* ("what does
   the contract say about termination"), *summarize* ("give me the gist of doc
   Y"), *compute/aggregate* ("how many tickets closed last week"), *compare*
   ("diff the 2024 vs 2025 policy").
3. A `VectorStoreIndex` (or any single retriever) is being stretched to answer
   query types it was not built for, and quality is uneven across the mix.
4. The user names a routing primitive: `RouterQueryEngine`,
   `SelectorPromptTemplate`, `LLMSingleSelector`, Dify *Question Classifier*,
   LangGraph `add_conditional_edges`, "intent classifier", "text-to-SQL or RAG".
5. PR review touches a function that reads a query and returns *which handler*
   to call.

Do **not** activate when:

- One index/tool serves every query correctly — routing adds an LLM hop and a
  failure mode for no benefit (see §6 anti-pattern A1).
- The split is by **who is asking** (tenant / ACL), not **what is asked** — that
  is filtering, route to `[[agentsop-multi-tenant-rag]]`.
- The downstream choice is **fixed** (always retrieve then summarize) — that is
  a static edge / linear pipeline, not a router.

---

## 2. 核心心智模型 (Core Mental Model)

Three principles. Violating any of them produces a router that misroutes
silently or routes when it should not.

### Principle 1 — One retriever cannot serve all query types

The index taxonomy is not cosmetic. From `[[llamaindex]]`: a `SummaryIndex` is
a "small, fan-out synthesis" primitive — it reads *every* node to digest a doc;
a `VectorStoreIndex` is top-k semantic lookup — it reads the *few* most similar
chunks; a text-to-SQL engine answers *aggregate/compute* queries that no chunk
contains the answer to. Ask a vector index to "summarize the whole document" and
it returns 4 arbitrary chunks; ask a summary index "what is the late-fee clause"
and it fans out over the whole corpus wastefully. **The query type names the
correct primitive.** Routing is the act of recovering that name at runtime.

> Operational corollary: `index.as_query_engine()` over a single
> `VectorStoreIndex` answering a heterogeneous query mix is the symptom this
> skill exists to fix. The fix is per-type handlers + a router on top.

### Principle 2 — Route first, retrieve second

Routing is a **classification** step that runs *before* any retrieval. It reads
only the query (and optionally light context) and emits a *destination*, not an
answer. This ordering is what bounds latency and cost: you pay for the router
once, then exactly one downstream handler, instead of fanning out to all of
them and merging. LlamaIndex's `RouterQueryEngine`, Dify's Question Classifier
node feeding IF/ELSE branches, and LangGraph's conditional edge over `state`
are the *same shape* — a selector function `(query) -> handler_id` evaluated up
front. The three frameworks differ only in *how* the selector is implemented
(§7).

### Principle 3 — A router is only as good as its destinations' descriptions

Every router — LLM, embedding, or keyword — picks among destinations described
in words or examples. In LlamaIndex the signal is the
`QueryEngineTool.description`; in Dify it is the class label + instruction; in
LangGraph it is whatever the routing function reads off state plus the node
names. From `[[llamaindex]]` Dilemma 3: *"invest in `QueryEngineTool.description`
— it's the only signal the router/agent sees."* A misroute is, four times out
of five, a **bad description**, not a bad model. Fix the description before
swapping the router type.

> The fourth case is genuinely ambiguous queries — for those, Principle of
> Fallback (§3 Stage 4) applies: route to a safe default, never guess silently.

---

## 3. SOP 工作流 (Agentic Protocol)

Five stages. Each gates the next. Stop and reconsider at the first "no".

### Stage 0 — Confirm routing is warranted

Gate questions:
- Are there genuinely **≥2 handlers** with different query-type fit? If no →
  single index, no router (anti-pattern A1).
- Does real traffic actually **span query kinds**? Sample 50 real queries and
  label them. If >90% are one kind → build that handler well; skip the router.
- Is the choice **query-dependent** (not fixed)? If fixed → static edge.

If all three are yes, continue.

### Stage 1 — Enumerate the query types and their handlers

Build the table *before* writing the router. One row per query kind:

| Query kind | Example | Correct handler | Primitive |
|---|---|---|---|
| Summarize / digest | "summarize the Q3 report" | summary engine | `SummaryIndex` |
| Fact lookup | "what is the late-fee clause" | vector engine | `VectorStoreIndex` + filters |
| Compute / aggregate | "how many orders shipped in Q3" | text-to-SQL engine | NL2SQL over the DB |
| Compare / multi-hop | "diff 2024 vs 2025 policy" | decomposition engine | `SubQuestionQueryEngine` |
| Out of scope | "what's the weather" | default / refuse | fallback handler |

This table is the spec for both the handlers and the router. Mapping mirrors
`[[llamaindex]]` Stage 4 ("Compose for query heterogeneity").

### Stage 2 — Build one handler per type, in isolation

Implement and test each engine **independently** against its own query kind
before wiring the router. A misroute is undebuggable if the handlers
themselves are wrong. Author each handler's **description / label** here
(Principle 3) — the destination metadata is part of the handler, not the router.

### Stage 3 — Write the router (pick the cheapest sufficient family)

Three router families, cheapest to most capable:

| Family | How it decides | Pick when |
|---|---|---|
| **Keyword / rule** | regex / substring / heuristic over the query | destinations are lexically distinct ("SELECT", "summarize", file extensions); latency-critical; cost-critical |
| **Embedding / semantic** | embed query, nearest destination description | destinations semantically distinct but not lexically; no per-call LLM budget; deterministic-ish |
| **LLM / selector** | LLM reads query + descriptions, returns choice (single or multi) | destinations need reasoning to disambiguate; multi-select needed; quality > latency |

Default ladder: **start keyword if the types are lexically separable; else
LLM selector**; reach for embedding when you want a middle point (no LLM hop,
better than keyword). Always emit a **confidence / score**, never just a label.

### Stage 4 — Add the fallback default (non-negotiable)

Every router must define behavior for the **unrouteable** query:
- LLM selector returns low confidence / no match → route to a documented
  default handler (usually the broad vector index) **or** an explicit "I can't
  answer that" path. Never let an ambiguous query silently hit a random branch.
- Set a **confidence threshold**: below it → fallback, not best-guess.
- Log every routing decision `{query_hash, chosen_handler, score, fell_back}`
  so misroutes are observable, not anecdotal.

A router with no fallback is the single most common production failure here
(anti-pattern A2).

---

## 4. 操作模型 (Operation Models)

Format: **Trigger / Action / Output / Evidence**.

### OP-01 EnumerateQueryTypes
- **Trigger**: A single endpoint fronts heterogeneous queries; no type table exists.
- **Action**: Sample ≥50 real queries, label each by kind (summarize / lookup /
  compute / compare / out-of-scope), map each kind to its correct handler +
  primitive (Stage 1 table).
- **Output**: A query-type → handler spec table that drives both handler build
  and router design.
- **Evidence**: `[[llamaindex]]` Stage 0 ("What is the query distribution?") +
  Stage 4 heterogeneity table.

### OP-02 BuildPerTypeHandler
- **Trigger**: Type table exists; handlers not yet built.
- **Action**: Implement one engine per type (`SummaryIndex` for digest,
  `VectorStoreIndex` for lookup, NL2SQL for compute, `SubQuestionQueryEngine`
  for compare). Test each against its own kind in isolation. Author its
  description/label.
- **Output**: N independently-verified handlers, each with a destination
  description (Principle 3).
- **Evidence**: `[[llamaindex]]` OP-06 RouteByQueryType; OP-07 DecomposeMultiHop.

### OP-03 KeywordRouter
- **Trigger**: Destinations are lexically distinct; latency/cost-critical path.
- **Action**: regex/substring rules over the query → handler id; explicit
  default for no-match. No model call.
- **Output**: Sub-millisecond, deterministic, free routing for the separable
  cases.
- **Evidence**: `[[agentsop-dify]]` IF/ELSE node over query; `[[agentsop-langgraph]]` conditional
  edge as a pure Python predicate (`add_conditional_edges`).

### OP-04 EmbeddingRouter
- **Trigger**: Destinations semantically (not lexically) distinct; no LLM budget per call.
- **Action**: Embed the query, cosine-match against pre-embedded destination
  descriptions / few examples per route; pick argmax; threshold for fallback.
- **Output**: Cheap, low-latency semantic routing without an LLM hop.
- **Evidence**: `[[llamaindex]]` `EmbeddingSingleSelector` family; router docs.

### OP-05 LLMSelectorRouter
- **Trigger**: Disambiguation needs reasoning; or a query maps to multiple handlers.
- **Action**: Use an LLM selector (single or multi) reading the query + each
  handler's description; return chosen id(s) + reasoning. In LlamaIndex:
  `RouterQueryEngine(selector=LLMSingleSelector.from_defaults(), query_engine_tools=[...])`.
- **Output**: Highest-accuracy routing including multi-route fan-out; costs one
  LLM round-trip.
- **Evidence**: `[[llamaindex]]` OP-06 + Dilemma 3; `SelectorPromptTemplate`.

### OP-06 ConfidenceThresholdFallback
- **Trigger**: Any router that can be uncertain (LLM/embedding) on ambiguous queries.
- **Action**: Read the selector's score; below threshold → route to documented
  default (broad vector index) or explicit refuse path. Never best-guess silently.
- **Output**: Unrouteable queries land somewhere safe and auditable.
- **Evidence**: `[[agentsop-langgraph]]` conditional edge can return a `"__default__"`
  branch; `[[agentsop-dify]]` Question Classifier has a built-in "other/else" class.

### OP-07 RouteDecisionLogging
- **Trigger**: Any production router.
- **Action**: Log `{ts, query_hash, chosen_handler, selector_score, fell_back}`
  on every decision; surface a misroute dashboard.
- **Output**: Misroutes become observable metrics, not user-reported anecdotes.
- **Evidence**: `[[agentsop-dify]]` 7-class trace incl. routing; `[[agentsop-langgraph]]`
  LangSmith trace of the conditional edge.

### OP-08 DescriptionFirstDebug
- **Trigger**: Router misroutes a known query class.
- **Action**: Before swapping router type, rewrite the destination description /
  add 1-2 disambiguating examples; re-evaluate on a labeled routing eval set.
- **Output**: Most misroutes fixed at the description layer, no model change.
- **Evidence**: `[[llamaindex]]` Dilemma 3 ("the only signal the router sees").

---

## 5. 困境决策案例 (Dilemma Cases)

### Dilemma 1 — Router misroutes: improve the classifier or add a fallback?

**困境**: A 3-way router (summary / lookup / SQL) misroutes ~12% of queries —
some lookup queries land on the SQL engine and error out. The team is split:
fine-tune / swap to a bigger LLM selector, vs. add a catch-all fallback.

**约束**:
- Bigger LLM selector adds latency + cost to *every* query to fix 12%.
- A pure fallback masks the misroute but still sends those queries somewhere
  generic, lowering answer quality for the 12%.
- The misroutes cluster: most are lookup queries phrased like questions the SQL
  engine "thinks" it can answer.

**决策步骤**:
1. **Diagnose before fixing** — pull the routing log (OP-07); cluster the 12%.
   The clustering reveals it's a *description* problem, not a model problem
   (the SQL engine's description over-claims).
2. **Fix descriptions first** (OP-08) — tighten the SQL handler's description to
   "aggregate/count/sum over the orders table only"; add 2 negative examples.
   Re-run the routing eval set. Misroute drops to ~4%.
3. **Then add fallback** (OP-06) for the residual ambiguous tail — low-confidence
   → broad vector index, not SQL.
4. **Only if still bad** consider the bigger selector — but now on 4%, the
   cost/benefit usually says no.

**结果**: Description fix + fallback, no model change. The two moves are
complementary, not either/or: improve the classifier *signal* (descriptions)
*and* add a fallback for the irreducible ambiguity. Mirrors `[[llamaindex]]`
Dilemma 3's "fix the supervisor before switching paradigms" logic and
`[[agentsop-langgraph]]`'s "hitting the limit means the logic is wrong" stance.

**可提取的操作**: OP-08, OP-06, OP-07.

### Dilemma 2 — LLM router latency vs cheap keyword router

**困境**: An LLM selector routes correctly but adds ~600ms + a token cost to
every query. Traffic is high-QPS and most queries are lexically obvious
("summarize…", SQL-shaped, or a plain question). Is the LLM hop worth it?

**约束**:
- The LLM selector is accurate but is now the p95 latency bottleneck and a
  per-query cost line.
- A keyword router is free and instant but brittle on the genuinely ambiguous
  tail.
- `[[agentsop-dify]]`'s known per-node latency overhead and `[[agentsop-langgraph]]`'s
  "checkpoint serialisation adds overhead, latency budget <200ms" boundary both
  argue against an LLM hop on every request.

**决策步骤**:
1. **Measure the separable fraction** — what % of queries a cheap keyword/regex
   router classifies with high confidence? Sample says ~80%.
2. **Tier the router**: cheap keyword/embedding router first; only the residual
   ~20% (no confident keyword match) escalates to the LLM selector. This is the
   routing analog of `[[llamaindex]]`'s "exhaust cheap knobs first" and
   `[[agentsop-langgraph]]`'s "promote upward only as needed" ladder.
3. **Set the keyword-confidence bar high** — a wrong cheap route is worse than a
   slow correct one; when in doubt, escalate.
4. **Re-measure p95 and cost**; the LLM hop now runs on 20% of traffic.

**结果**: A two-tier (cascade) router — cheap router handles the obvious
majority instantly, LLM selector handles the ambiguous minority. Cost and p95
drop ~5×; accuracy is preserved because the cheap tier only acts when confident.
Reserve the LLM selector for where reasoning is actually required (Principle 2 +
the §3 Stage 3 ladder).

**可提取的操作**: OP-03, OP-04, OP-05, OP-06.

### Dilemma 3 — Route to one handler vs multi-select fan-out

**困境**: Some queries legitimately need two handlers ("summarize the contract
*and* tell me the late-fee clause"). A single-select router forces a wrong
binary choice.

**约束**:
- Multi-select doubles downstream cost and needs a synthesis step to merge.
- Most queries are single-intent; defaulting to multi-select wastes spend.

**决策步骤**:
1. Quantify the multi-intent fraction from the routing log. If <10% → keep
   single-select + fallback; accept occasional follow-up.
2. If material → use a **multi-selector** (LlamaIndex `LLMMultiSelector` /
   Dify branching to multiple nodes / LangGraph `Send` fan-out to multiple
   handlers) and add a synthesis node to merge results.
3. Never make multi-select the default — it converts a routing problem into a
   fan-out + merge problem with the cost of both.

**结果**: Single-select by default; multi-select only on the measured
multi-intent slice, paired with explicit synthesis. Mirrors `[[agentsop-langgraph]]`
"don't fan out with `Send` for fixed-cardinality work."

**可提取的操作**: OP-05, OP-07.

---

## 6. 反模式与边界 (Anti-patterns & Boundaries)

### Top anti-patterns (red flags in code review)

| # | Anti-pattern | Why it's wrong | Correct move |
|---|---|---|---|
| A1 | Adding a router when one index already serves every query | Pure overhead: an extra hop + a new failure mode for zero benefit | Single index; route only when ≥2 handlers differ in fit (Stage 0) |
| A2 | Router with no fallback / default branch | Ambiguous or out-of-scope queries hit a random or erroring handler | Confidence threshold → documented default / refuse (OP-06) |
| A3 | LLM selector on every query when keyword would do | p95 latency + per-query token cost for separable traffic | Tier: cheap router first, LLM only for the ambiguous tail (Dilemma 2) |
| A4 | Vague destination descriptions | Router can't disambiguate; misroutes blamed on the model | Author precise descriptions + examples; fix here first (OP-08) |
| A5 | No routing decision logging | Misroutes surface as user complaints, not metrics | Log `{query, chosen, score, fell_back}` per decision (OP-07) |
| A6 | Routing by tenant/permission instead of query kind | That's access control, not query routing | Use `[[agentsop-multi-tenant-rag]]` filter at the store; route by *kind* only |
| A7 | Multi-select as the default | Doubles cost + needs merge for mostly single-intent traffic | Single-select default; multi only on measured multi-intent slice (Dilemma 3) |
| A8 | Best-guess on low confidence | Silent wrong route degrades the answer with no signal | Below threshold → fallback, never guess (OP-06) |
| A9 | Tuning the router before the handlers work | A misroute is undebuggable atop broken handlers | Build + verify each handler in isolation first (Stage 2) |

### Boundaries — when query routing is **not** the move

- **B1** One index/tool answers everything correctly → no router (A1).
- **B2** The split is by *who asks* (tenant, ACL) → `[[agentsop-multi-tenant-rag]]`.
- **B3** The downstream is a *fixed* sequence (always retrieve→summarize) →
  static edge / linear pipeline, no selector.
- **B4** Hard real-time (<200ms) and an LLM selector is the only correct router
  → measure first; tier to a keyword/embedding front, escalate rarely (Dilemma 2).
- **B5** The choice is *agentic/iterative* (retrieve→reflect→re-query in a loop)
  → that's an agent loop, not a one-shot router; use `[[agentsop-langgraph]]` cycles.

### PR review smells

- A `RouterQueryEngine` / classifier with no `default` / `other` branch.
- An LLM selector in a high-QPS hot path with no cheap pre-filter.
- `QueryEngineTool(description="index")` — uselessly vague description.
- Routing logic that reads `tenant_id` — that's filtering, not routing.
- A router whose handlers were never tested independently.
- No eval set of labeled `(query, expected_handler)` pairs gating router changes.

---

## 7. 跨框架对照 (Cross-Framework Reference)

The same selector `(query) -> handler_id` surface across the three base skills.
All verified against the source SKILLs (May 2026). Cross-link the base skill for
the full API.

### 7.1 LlamaIndex — `RouterQueryEngine` + selectors → `[[llamaindex]]`

```python
from llama_index.core.query_engine import RouterQueryEngine
from llama_index.core.selectors import LLMSingleSelector  # or LLMMultiSelector,
                                                           #    EmbeddingSingleSelector
from llama_index.core.tools import QueryEngineTool

tools = [
    QueryEngineTool.from_defaults(
        query_engine=summary_engine,
        description="Useful for SUMMARIZING or digesting an entire document."),
    QueryEngineTool.from_defaults(
        query_engine=vector_engine,
        description="Useful for LOOKING UP a specific fact or clause."),
]
router = RouterQueryEngine(
    selector=LLMSingleSelector.from_defaults(),   # LLM selector family
    query_engine_tools=tools,                     # descriptions are the routing signal
)
```

The selector reads each `QueryEngineTool.description` (the only routing signal —
Principle 3). Selector families: `LLMSingleSelector`, `LLMMultiSelector`,
`EmbeddingSingleSelector`, `PydanticSingleSelector`. `SelectorPromptTemplate`
customizes the LLM prompt. `RouterQueryEngine` over per-task indices is "often
the correct top-level shape, not a single monolithic `VectorStoreIndex`"
(`[[llamaindex]]` Principle 3 + OP-06).

### 7.2 Dify — Question Classifier node → `[[agentsop-dify]]`

The **Question Classifier** node (an LLM node) takes the query and emits one of
N declared classes; each class wires to a downstream branch (Knowledge
Retrieval / LLM / Code / HTTP / SQL-via-Code). It is the visual analog of an
LLM selector. The built-in "other" class is the fallback (OP-06). Routing logic
that doesn't need an LLM uses the **IF/ELSE** node (keyword/rule router, OP-03).
Node taxonomy: `Question Classifier`, `Parameter Extractor`, `IF/ELSE`. The
classifier's class label + instruction is the routing signal (Principle 3).

### 7.3 LangGraph — conditional edges as routers → `[[agentsop-langgraph]]`

```python
def route(state) -> str:                          # the selector function
    q = state["query"]
    if looks_like_sql(q):     return "sql"        # keyword tier (OP-03)
    if score := classify(q):  return score.label  # LLM/embedding tier (OP-04/05)
    return "vector_default"                        # fallback (OP-06)

graph.add_conditional_edges("router", route,
                            {"sql": "sql_node",
                             "summary": "summary_node",
                             "vector_default": "vector_node"})
```

"Graph topology is just routing logic over state… a conditional edge reads
state and picks a next node" (`[[agentsop-langgraph]]` Principle 3). The mapping dict's
keys are the destinations; the `route` function is the selector — it can be
keyword, embedding, or LLM, or a tier of all three (Dilemma 2). For genuine
multi-route fan-out, return a list of `Send(...)` instead of one label.

### 7.4 Side-by-side

| | LlamaIndex | Dify | LangGraph |
|---|---|---|---|
| Router primitive | `RouterQueryEngine` | Question Classifier node | `add_conditional_edges` |
| Selector impl | LLM / Embedding / Pydantic selector | LLM classifier (or IF/ELSE for rules) | any Python fn (keyword/embed/LLM) |
| Routing signal | `QueryEngineTool.description` | class label + instruction | node names + what `route()` reads |
| Fallback | selector default / catch-all tool | "other" class | a `"__default__"` branch |
| Multi-route | `LLMMultiSelector` | branch to multiple nodes | list of `Send(...)` |
| Deep skill | `[[llamaindex]]` | `[[agentsop-dify]]` | `[[agentsop-langgraph]]` |

The three are the **same pattern** — classify the query up front, dispatch to
the structurally-correct handler, fall back when uncertain. Pick the framework
your stack already uses; the routing discipline (Principles 1-3, Stages 0-4) is
identical.

---

## References

- `references/R1-source-evidence.md` — every cited claim resolved to its source
  SKILL line.
- `intermediate/operation_candidates.json` — machine-readable operation list.

### Source skills (cited inline as `[[name]]`)

- `[[llamaindex]]` — `llamaindex-sop-skill/SKILL.md` (`RouterQueryEngine`,
  selectors, OP-06 RouteByQueryType, Dilemma 3, Index taxonomy).
- `[[agentsop-dify]]` — `dify-sop-skill/SKILL.md` (Question Classifier / IF-ELSE nodes,
  node taxonomy, per-node latency boundary).
- `[[agentsop-langgraph]]` — `langgraph-sop-skill/SKILL.md` (conditional edges as
  routers, Principle 3 topology-as-routing, `Send` fan-out, latency boundary).