Looker Runtime Ingest

Looker Runtime Ingest

Looker runtime ingest turns API-staged dashboards, Looks, and explores into durable ktx memory. Runtime entities are evidence. They are not themselves the final knowledge shape.

Required Workflow

1. Read every rawFiles entry for the WorkUnit.
2. Read relevant dependencyPaths before making a decision. For dashboard and Look WUs this usually includes the referenced explore JSON, signal files, folders/tree.json, and users/<id>.json.
3. Treat signals/*.json, owners, folders, schedules, and favorites as prioritization or provenance context only.
4. Extract generalizable metric formulas, segment definitions, field semantics, and domain conventions.
5. Use wiki_search, sl_discover, and sl_read_source before writing so new content merges with existing memory instead of duplicating it.
6. Use context_evidence_search or context_evidence_read to obtain evidence chunk IDs for any wiki-bound knowledge candidate.
7. Use context_candidate_write for durable wiki-bound knowledge. Do not call wiki_write from a Looker WorkUnit; Stage 4 reconciliation promotes candidates and writes wiki pages.
8. Use looker_query_to_sl for each Look query or dashboard tile query that has a query object.
9. Write SL from Looker runtime evidence only through the staged warehouse target contract. For explores and inherited dashboard/Look queries, branch on targetTable.ok; when it is true, write on targetWarehouseConnectionId and use targetTable.canonicalTable as source.table. When it is false or missing, write wiki knowledge candidates and record emit_unmapped_fallback with the staged reason.
10. Run sl_validate after every SL write. If validation fails, fix the source or roll it back before the WorkUnit ends.

For every Looker field reference, call entity_details on the underlying schema.table.column before promoting it to sl_refs or quoting it in wiki body.

Identifier Verification Protocol

Before writing a wiki page or SL source on any topic:

1. discover_data({query: "<topic>"}) - see what wikis, SL sources, and raw tables already exist. Prefer updating existing pages over creating new ones.

Before emitting any schema.table or schema.table.column into a wiki body, SL source, tables: frontmatter, sl_refs, or emit_unmapped_fallback:

2. entity_details({connectionId, targets: [{display: "<identifier>"}]}) - confirm the identifier resolves; inspect native types, FK/PK, and sampleValues.
3. For literal values from the source, such as status codes or plan tiers, check whether they appear in entity_details sampleValues for the relevant column. If sampleValues is short or the sample may have missed real values, run a sql_execution probe with the same warehouse connection id: sql_execution({connectionId, sql: "SELECT DISTINCT <col> FROM <ref> LIMIT 50"}).
4. If the candidate identifier still does not resolve, do one of:
   • Use sql_execution({connectionId, sql: "SELECT 1 FROM <ref> LIMIT 0"}).