LLM Provider

LLM Provider

Critical

1. All providers MUST implement the LLMProvider interface from src/llm/types.ts with three methods:

   • call(options: LLMCallOptions): Promise — single non-streaming call returning text
   • stream(options: LLMStreamOptions, callbacks: LLMStreamCallbacks): Promise — streaming call invoking callbacks
   • listModels?(): Promise<string[]> — optional; list available models from the API

2. Initialize client in constructor and store defaultModel from config. Example: this.client = new YourSDK({ apiKey: config.apiKey }). Never lazy-initialize on first call — providers are instantiated once and cached in src/llm/index.ts.

3. For EVERY response in call() and stream(), invoke trackUsage(model, usage) from src/llm/usage.js before returning/ending. This is mandatory — it captures token metrics for CLI telemetry and cost analysis. If the API doesn't return usage data, estimate via estimateTokens(text), which assumes ~4 chars per token.

4. Both call() and stream() must respect the model parameter using pattern: options.model || this.defaultModel. Never hardcode model names. Callers supply model overrides via LLMCallOptions.model.

5. Error handling: catch all errors, preserve error messages unchanged. The retry logic in src/llm/index.ts handles transient errors (ECONNRESET, socket hang up, 529 overload). For seat-based providers (Cursor, Claude CLI), wrap stderr via parseSeatBasedError() for user-friendly messages.

6. Always update ProviderType union (Step 2), DEFAULT_MODELS (Step 4), and createProvider() switch case (Step 5) in lock-step. Missing any one breaks the build or causes runtime Unknown provider error.

Instructions

Step 1: Create provider class file

Verify directory exists: ls -la src/llm/. Create src/llm/your-provider.ts. Match existing provider patterns (src/llm/anthropic.ts, src/llm/openai-compat.ts).

Minimal structure:

import type { LLMProvider, LLMCallOptions, LLMStreamOptions, LLMStreamCallbacks, LLMConfig, TokenUsage } from './types.js';
import { trackUsage } from './usage.js';
import { estimateTokens } from './utils.js';

export class YourProviderProvider implements LLMProvider {
  private client: YourSDKType;
  private defaultModel: string;

  constructor(config: LLMConfig) {
    if (!config.apiKey) throw new Error('API key required');
    this.client = new YourSDK({ apiKey: config.apiKey, ...(config.baseUrl && { baseURL: config.baseUrl }) });
    this.defaultModel = config.model;
  }