LLM Text Streaming

LLM Text Streaming

When to Use This Skill

• Adding token or chunk progress to an LLM-powered step
• Choosing between completed generation and direct stream access
• Using onChunk, or onEnd / onError on streamText() / Agent.stream()
• Making stream failures trigger Temporal activity retries
• Streaming Agent responses or persisting streamed conversations

Choose the API

In workflow steps, prefer generateTextWithStreaming() or Agent.generateWithStreaming() when onChunk progress is sufficient. They consume the stream internally, return complete results like generateText() or Agent.generate(), and reject on provider, transport, or abort errors. Rejection allows Temporal to record the failed activity attempt and apply the step retry policy.

streamText() and Agent.stream() remain supported for code that needs direct control over stream consumption.

generateTextWithStreaming()

import { generateTextWithStreaming } from '@outputai/llm';

const result = await generateTextWithStreaming( {
  prompt: 'draft@v1',
  variables: { topic },
  onChunk( { chunk } ) {
    if ( chunk.type === 'text-delta' ) {
      process.stdout.write( chunk.text );
    }
  }
} );

return result.result;