Model Registry Maintainer

This skill provides guidance for maintaining MassGen's model registry across two key files:

1. massgen/backend/capabilities.py - Models, capabilities, release dates
2. massgen/token_manager/token_manager.py - Pricing, context windows

When to Use This Skill

• New model released by a provider
• Model pricing changes
• Context window limits updated
• Model capabilities changed
• New provider/backend added

Two Files to Maintain

File 1: capabilities.py (Models & Features)

What it contains:

• List of available models per provider
• Model capabilities (web search, code execution, vision, etc.)
• Release dates
• Default models

Used by:

• Config builder (--quickstart, --generate-config)
• Documentation generation
• Backend validation

Always update this file for new models.

File 2: token_manager.py (Pricing & Limits)

What it contains:

• Hardcoded pricing/context windows for models NOT in LiteLLM database
• On-demand loading from LiteLLM database (500+ models)

Used by:

• Cost estimation
• Token counting
• Context management

Pricing resolution order:

1. LiteLLM database (fetched on-demand, cached 1 hour)
2. Hardcoded PROVIDER_PRICING (fallback only)
3. Pattern matching heuristics

Only update PROVIDER_PRICING if:

• Model is NOT in LiteLLM database
• LiteLLM pricing is incorrect/outdated
• Model is custom/internal to your organization

Information to Gather for New Models

1. Release Date

• Format: "YYYY-MM"
• Sources:
• OpenAI: https://openai.com/index
• Anthropic: https://www.anthropic.com/news
• Google DeepMind: https://blog.google/technology/google-deepmind/
• xAI: https://x.ai/news

2. Context Window

• Input context size (tokens)
• Max output tokens
• Look for: "context window", "max tokens", "input/output limits"

3. Pricing

• Input cost per 1K tokens (USD)
• Output cost per 1K tokens (USD)
• Cached input cost (if applicable)
• Sources:
• OpenAI: https://openai.com/api/pricing/
• Anthropic: https://www.anthropic.com/pricing
• Google: https://ai.google.dev/pricing
• xAI: https://x.ai/api/pricing

4. Capabilities

• Web search, code execution, vision, reasoning, etc.
• Check official API documentation

5. Model Name

• Exact API identifier (case-sensitive)
• Check provider's model documentation

Adding a New Model - Complete Workflow

Step 1: Add to capabilities.py

Add model to the models list and model_release_dates:

python
# massgen/backend/capabilities.py

"openai": BackendCapabilities(
    # ... existing fields ...
    models=[
        "new-model-name",  # Add here (newest first)
        "gpt-5.1",
        # ... existing models ...
    ],
    model_release_dates={
        "new-model-name": "2025-12",  # Add here
        "gpt-5.1": "2025-11",
        # ... existing dates ...
    },
)

Step 2: Check if pricing is in LiteLLM (Usually Skip)

First, check if the model is already in LiteLLM database:

python
import requests

url = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
pricing_db = requests.get(url).json()

if "new-model-name" in pricing_db:
    print("✅ Model found in LiteLLM - no need to update token_manager.py")
    print(f"Pricing: ${pricing_db['new-model-name']['input_cost_per_token']*1000}/1K input")
else:
    print("❌ Model NOT in LiteLLM - need to add to PROVIDER_PRICING")

Only if NOT in LiteLLM, add to PROVIDER_PRICING:

python
# massgen/token_manager/token_manager.py

PROVIDER_PRICING: Dict[str, Dict[str, ModelPricing]] = {
    "OpenAI": {
        # Format: ModelPricing(input_per_1k, output_per_1k, context_window, max_output)
        "new-model-name": ModelPricing(0.00125, 0.01, 300000, 150000),
        # ... existing models ...
    },
}

Provider name mapping:

• "OpenAI" (not "openai")
• "Anthropic" (not "claude")
• "Google" (not "gemini")
• "xAI" (not "grok")

Step 3: Update Capabilities (if new features)

If the model introduces new capabilities:

python
supported_capabilities={
    "web_search",
    "code_execution",
    "new_capability",  # Add here
}

Step 4: Update Default Model (if appropriate)

Only change if the new model should be the recommended default:

python
default_model="new-model-name"

Step 5: Validate and Test

bash
# Run capabilities tests
uv run pytest massgen/tests/test_backend_capabilities.py -v

# Test config generation with new model
massgen --generate-config ./test.yaml --config-backend openai --config-model new-model-name

# Verify the config was created successfully
cat ./test.yaml

Step 6: Regenerate Documentation

bash
uv run python docs/scripts/generate_backend_tables.py
cd docs && make html

Current Model Data

OpenAI Models (as of Nov 2025)

In capabilities.py:

python
models=[
    "gpt-5.1",        # 2025-11
    "gpt-5-codex",    # 2025-09
    "gpt-5",          # 2025-08
    "gpt-5-mini",     # 2025-08
    "gpt-5-nano",     # 2025-08
    "gpt-4.1",        # 2025-04
    "gpt-4.1-mini",   # 2025-04
    "gpt-4.1-nano",   # 2025-04
    "gpt-4o",         # 2024-05
    "gpt-4o-mini",    # 2024-07
    "o4-mini",        # 2025-04
]

In token_manager.py (add missing models):

python
"OpenAI": {
    "gpt-5": ModelPricing(0.00125, 0.01, 400000, 128000),
    "gpt-5-mini": ModelPricing(0.00025, 0.002, 400000, 128000),
    "gpt-5-nano": ModelPricing(0.00005, 0.0004, 400000, 128000),
    "gpt-4o": ModelPricing(0.0025, 0.01, 128000, 16384),
    "gpt-4o-mini": ModelPricing(0.00015, 0.0006, 128000, 16384),
    # Missing: gpt-5.1, gpt-5-codex, gpt-4.1 family, o4-mini
}

Claude Models (as of Nov 2025)

In capabilities.py:

python
models=[
    "claude-haiku-4-5-20251001",    # 2025-10
    "claude-sonnet-4-5-20250929",   # 2025-09
    "claude-opus-4-1-20250805",     # 2025-08
    "claude-sonnet-4-20250514",     # 2025-05
]

In token_manager.py:

python
"Anthropic": {
    "claude-haiku-4-5": ModelPricing(0.001, 0.005, 200000, 65536),
    "claude-sonnet-4-5": ModelPricing(0.003, 0.015, 200000, 65536),
    "claude-opus-4.1": ModelPricing(0.015, 0.075, 200000, 32768),
    "claude-sonnet-4": ModelPricing(0.003, 0.015, 200000, 8192),
}

Gemini Models (as of Nov 2025)

In capabilities.py:

python
models=[
    "gemini-3-pro-preview",  # 2025-11
    "gemini-2.5-flash",      # 2025-06
    "gemini-2.5-pro",        # 2025-06
]

In token_manager.py (missing gemini-2.5 and gemini-3):

python
"Google": {
    "gemini-1.5-pro": ModelPricing(0.00125, 0.005, 2097152, 8192),
    "gemini-1.5-flash": ModelPricing(0.000075, 0.0003, 1048576, 8192),
    # Missing: gemini-2.5-pro, gemini-2.5-flash, gemini-3-pro-preview
}

Grok Models (as of Nov 2025)

In capabilities.py:

python
models=[
    "grok-4-1-fast-reasoning",      # 2025-11
    "grok-4-1-fast-non-reasoning",  # 2025-11
    "grok-code-fast-1",             # 2025-08
    "grok-4",                       # 2025-07
    "grok-4-fast",                  # 2025-09
    "grok-3",                       # 2025-02
    "grok-3-mini",                  # 2025-05
]

In token_manager.py (missing grok-3, grok-4 families):

python
"xAI": {
    "grok-2-latest": ModelPricing(0.005, 0.015, 131072, 131072),
    "grok-2": ModelPricing(0.005, 0.015, 131072, 131072),
    "grok-2-mini": ModelPricing(0.001, 0.003, 131072, 65536),
    # Missing: grok-3, grok-4, grok-4-1 families
}

Model Name Matching

Important: The names in PROVIDER_PRICING use simplified patterns:

• "gpt-5" matches gpt-5, gpt-5-preview, gpt-5-*
• "claude-sonnet-4-5" matches claude-sonnet-4-5-* (any date suffix)
• "gemini-2.5-pro" is exact match

The token manager uses prefix matching for flexibility.

Common Tasks

Task: Add brand new GPT-5.2 model

1. Research: Release date, pricing, context window, capabilities
2. Add to capabilities.py models list and release_dates
3. Add to token_manager.py PROVIDER_PRICING"OpenAI"]
4. Run tests
5. Regenerate docs

Task: Update pricing for existing model

1. Verify new pricing from official source
2. Update only token_manager.py PROVIDER_PRICING
3. No need to touch capabilities.py
4. Document change in notes if significant

Task: Add new capability to model

1. Update supported_capabilities in capabilities.py
2. Add to notes explaining when/how capability works
3. Update backend implementation if needed
4. Run tests

Validation Commands

bash
# Test capabilities registry
uv run pytest massgen/tests/test_backend_capabilities.py -v

# Test token manager
uv run pytest massgen/tests/test_token_manager.py -v

# Generate config with new model
massgen --generate-config ./test.yaml --config-backend openai --config-model new-model

# Build docs to verify tables
cd docs && make html

Programmatic Model Updates

LiteLLM Pricing Database (RECOMMENDED)

The easiest way to get comprehensive model pricing and context window data:

URL: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json

Coverage: 500+ models across 30+ providers including:

• OpenAI, Anthropic, Google, xAI
• Together AI, Groq, Cerebras, Fireworks
• AWS Bedrock, Azure, Cohere, and more

Data Available:

json
{
  "gpt-4o": {
    "input_cost_per_token": 0.0000025,
    "output_cost_per_token": 0.00001,
    "max_input_tokens": 128000,
    "max_output_tokens": 16384,
    "supports_vision": true,
    "supports_function_calling": true,
    "supports_prompt_caching": true
  }
}

Usage:

python
import requests

# Fetch latest pricing
url = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
pricing_db = requests.get(url).json()

# Get info for a model
model_info = pricing_db.get("gpt-4o")
input_per_1k = model_info["input_cost_per_token"] * 1000
output_per_1k = model_info["output_cost_per_token"] * 1000

Update token_manager.py from LiteLLM:

• Convert per-token costs to per-1K costs
• Extract context window and max output tokens
• Keep models in reverse chronological order

OpenRouter API (Real-Time)

For the most up-to-date model list with live pricing:

Endpoint: https://openrouter.ai/api/v1/models

Data Available:

• Real-time pricing (prompt, completion, reasoning, caching)
• Context windows and max completion tokens
• Model capabilities and modalities
• 200+ models from multiple providers

Usage:

python
import requests
import os

headers = {"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"}
response = requests.get("https://openrouter.ai/api/v1/models", headers=headers)
models = response.json()["data"]

for model in models:
    print(f"{model['id']}: ${model['pricing']['prompt']} input, ${model['pricing']['completion']} output")

Provider-Specific APIs

| Provider | Models API | Pricing in API? | Recommendation | |----------|------------|-----------------|----------------| | OpenAI | https://api.openai.com/v1/models | ❌ No | Use LiteLLM | | Claude | No public API | ❌ No | Use LiteLLM | | Gemini | https://generativelanguage.googleapis.com/v1beta/models | ❌ No | API + LiteLLM | | Grok (xAI) | https://api.x.ai/v1/models | ❌ No | Use LiteLLM | | Together AI | https://api.together.xyz/v1/models | ✅ Yes | API directly | | Groq | https://api.groq.com/openai/v1/models | ❌ No | Use LiteLLM | | Cerebras | https://api.cerebras.ai/v1/models | ❌ No | Use LiteLLM | | Fireworks | https://api.fireworks.ai/v1/accounts/{id}/models | ❌ No | Use LiteLLM | | Azure OpenAI | Azure Management API | ❌ Complex | Manual | | Claude Code | No API | ❌ No | Manual |

Automation Script

Create scripts/update_model_pricing.py to automate updates:

python
#!/usr/bin/env python3
"""Update token_manager.py pricing from LiteLLM database."""

import requests

# Fetch LiteLLM database
url = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
pricing_db = requests.get(url).json()

# Filter by provider
openai_models = {k: v for k, v in pricing_db.items()
                 if v.get("litellm_provider") == "openai"}
anthropic_models = {k: v for k, v in pricing_db.items()
                    if v.get("litellm_provider") == "anthropic"}

# Generate ModelPricing entries
for model_name, info in openai_models.items():
    input_per_1k = info["input_cost_per_token"] * 1000
    output_per_1k = info["output_cost_per_token"] * 1000
    context = info.get("max_input_tokens", 0)
    max_output = info.get("max_output_tokens", 0)

    print(f'    "{model_name}": ModelPricing({input_per_1k}, {output_per_1k}, {context}, {max_output}),')

Run weekly to keep pricing current:

bash
uv run python scripts/update_model_pricing.py

Reference Files

• Capabilities registry: massgen/backend/capabilities.py
• Token/pricing manager: massgen/token_manager/token_manager.py
• Capabilities tests: massgen/tests/test_backend_capabilities.py
• Config builder: massgen/config_builder.py
• Doc generator: docs/scripts/generate_backend_tables.py
• LiteLLM database: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json
• OpenRouter API: https://openrouter.ai/docs/overview/models

Important Maintenance Notes

• Keep models in reverse chronological order - Newest first
• Use exact API names - Match provider documentation exactly
• Verify pricing units - Always per 1K tokens in token_manager.py
• Document uncertainties - If info is estimated/unofficial, note it
• Update both files - Don't forget token_manager.py when adding models
• Use LiteLLM for pricing - Comprehensive and frequently updated
• Test after updates - Run pytest to verify no breaking changes