Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Generates SQL validation notebooks for dbt PR changes with before/after comparison queries.
.claude/skills/monte-carlo-validation-notebook/SKILL.md| Model | Eval pass | Runs |
|---|---|---|
| gemini-3.6-flash | 91% | 36 |
| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-11 | ✗→✓ | ▲ Improved | 37% | 0% |
| case-07 | ✗→✓ | ▲ Improved | -33% | 0% |
| case-08 | ✗→✓ | ▲ Improved | -75% | 0% |
| case-09 | ✗→✓ | ▲ Improved | -20% | 0% |
| case-12 | ✗→✓ | ▲ Improved | 1% | 0% |
> Tip: This skill works well with Sonnet. Run /model sonnet before invoking for faster generation.
Generate a SQL Notebook with validation queries for dbt changes.
Arguments: $ARGUMENTS
Read the detailed guide before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.
Use this skill when the user wants to validate dbt model or snapshot changes with Monte Carlo SQL Notebook queries, either from a GitHub PR or a local dbt repository.
Parse the arguments:
--mc-base-url <URL> — defaults to https://getmontecarlo.com--models <model1,model2,...> — comma-separated list of model filenames (without .sql extension) to generate queries for. Only these models will be included. By default, all changed models are included up to a maximum of 10.Prerequisites:
gh (GitHub CLI) — required for PR mode. Must be authenticated (gh auth status).python3 — required for helper scripts.pyyaml — install with pip3 install pyyaml (or pip install pyyaml, uv pip install pyyaml, etc.)Note: Generated SQL uses ANSI-compatible syntax that works across Snowflake, BigQuery, Redshift, and Athena. Minor adjustments may be needed for specific warehouse quirks.
This skill includes two helper scripts in ${CLAUDE_PLUGIN_ROOT}/skills/monte-carlo-validation-notebook/scripts/:
resolve_dbt_schema.py - Resolves dbt model output schemas from dbt_project.yml routing rules and model config overrides.generate_notebook_url.py - Encodes notebook YAML into a base64 import URL and opens it in the browser.Auto-detect mode from the target argument:
:// or github.com) -> PR mode., /path/to/repo, relative path) -> Local modeThis command generates a SQL Notebook containing validation queries for dbt changes. The notebook can be opened in the MC Bridge SQL Notebook interface for interactive validation.
The output is an import URL that opens directly in the notebook interface:
<MC_BASE_URL>/notebooks/import#<base64-encoded-yaml>Key Features:
text parameters (prod_db and dev_db) for selecting databasesdbt_project.yml and model configs{{prod_db}}.<SCHEMA>.<TABLE>{{prod_db}} vs {{dev_db}}Key structure:
yamlversion: 1 metadata: id: string # kebab-case + random suffix name: string # display name created_at: string # ISO 8601 updated_at: string # ISO 8601 default_context: # optional database/schema context database: string schema: string cells: - id: string type: sql | markdown | parameter content: string # SQL, markdown, or parameter config (JSON) display_type: table | bar | timeseries
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 30,308 | 19,007 | -37% | 1 | 1 | 0% | 4,925 | 1,397 | -72% | 0 | 0 | — |
case-11 | fail→pass | 16,213 | 12,092 | -25% | 1 | 1 | 0% | 1,613 | 2,207 | +37% | 0 | 0 | — |
case-02 | fail→fail | 51,031 | 24,087 | -53% | 1 | 1 | 0% | 1,580 | 1,341 | -15% | 0 | 0 | — |
case-03 | fail→fail | 18,215 | 23,148 | +27% | 1 | 1 | 0% | 1,911 | 1,325 | -31% | 0 | 0 | — |
case-04 | pass→pass | 23,498 | 19,185 | -18% | 1 | 1 | 0% | 3,401 | 3,303 | -3% | 0 | 0 | — |
case-05 | pass→pass | 16,371 | 12,577 | -23% | 1 | 1 | 0% | 1,890 | 2,243 | +19% | 0 | 0 | — |
case-06 | pass→pass | 17,444 | 17,440 | -0% | 1 | 1 | 0% | 2,489 | 3,092 | +24% | 0 | 0 | — |
case-07 | fail→pass | 19,980 | 5,442 | -73% | 1 | 1 | 0% | 2,469 | 1,655 | -33% | 0 | 0 | — |
case-08 | fail→pass | 38,694 | 5,076 | -87% | 1 | 1 | 0% | 6,750 | 1,661 | -75% | 0 | 0 | — |
case-09 | fail→pass | 13,064 | 8,417 | -36% | 1 | 1 | 0% | 1,753 | 1,407 | -20% | 0 | 0 | — |
case-10 | pass→pass | 60,887 | 8,910 | -85% | 1 | 1 | 0% | 1,559 | 1,469 | -6% | 0 | 0 | — |
case-12 | fail→pass | 16,016 | 17,711 | +11% | 1 | 1 | 0% | 1,595 | 1,604 | +1% | 0 | 0 | — |
case-13 | fail→pass | 15,919 | 8,563 | -46% | 1 | 1 | 0% | 1,671 | 1,341 | -20% | 0 | 0 | — |
case-14 | pass→pass | 9,032 | 5,567 | -38% | 1 | 1 | 0% | 562 | 1,232 | +119% | 0 | 0 | — |
case-15 | pass→pass | 25,720 | 6,263 | -76% | 1 | 1 | 0% | 2,331 | 1,719 | -26% | 0 | 0 | — |
case-16 | fail→pass | 22,408 | 12,984 | -42% | 1 | 1 | 0% | 2,363 | 2,142 | -9% | 0 | 0 | — |
case-17 | pass→pass | 15,823 | 9,716 | -39% | 1 | 1 | 0% | 1,472 | 1,603 | +9% | 0 | 0 | — |
case-18 | fail→pass | 18,314 | 12,088 | -34% | 1 | 1 | 0% | 2,016 | 2,809 | +39% | 0 | 0 | — |
case-19 | pass→pass | 20,787 | 9,039 | -57% | 1 | 1 | 0% | 2,288 | 1,475 | -36% | 0 | 0 | — |
case-20 | fail→pass | 15,263 | 7,645 | -50% | 1 | 1 | 0% | 1,446 | 1,253 | -13% | 0 | 0 | — |
case-21 | fail→pass | 12,318 | 9,432 | -23% | 1 | 1 | 0% | 1,643 | 1,391 | -15% | 0 | 0 | — |
case-22 | pass→pass | 14,420 | 8,574 | -41% | 1 | 1 | 0% | 1,235 | 1,243 | +1% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 19 counted toward the lift figure. The other 3 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +45 percentage points is the difference between those two pass rates over the 19 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
| Model | Method | Date | Lift |
|---|---|---|---|
| gemini-3.6-flash | verified | 7/28/2026 | +23% |
Other measured skills in the registry, with their headline benchmark lift.