Applies pipes-and-filters for sequential data transformations.
SkillScore79Maintained
Automatic
◆Data Engineer pack
Last verified 8/20/2026
What it changes
Passed 18 of 23 test cases with the skill, and 10 of 23 without it.
Without skill42%
With skill79%
Checks it passed that the model alone missed
Mandate load testing at 2x expected peak throughput.
Require individual contract tests for each processing stage.
Specify a load testing requirement at 2x expected peak throughput.
and 11 more in the full report
9 fixed10 passed either way5 still failing0 broke
Pass-rate lift
+38 pp
Tokens
-13%
Agent turns
0%
Response time
-42%
Model gemini-3.6-flashMeasured 8/20/2026v119 of 24 cases passed with skill · 23 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 8 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 8
Prompt
We are building a real-time event processing system to handle continuous log ingestion and telemetry transformations. Please generate an architectural design and adoption roadmap using a sequential transformation pattern. Include step-by-step implementation guidance, required deliverables, potential operational risks with mitigations, and final production readiness criteria.
What was checked, without and with the skill
Checked
Without skill
With skill
Recommend an Architecture Decision Record documenting data replay mechanisms.
Missed
Passed
Require individual contract tests for each processing stage.
Missed
Passed
Mandate load testing at 2x expected peak throughput.
Missed
Passed
case-01 · gemini-3.6-flash · 8/20/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md57 linessingle file
The Pipeline (Pipes and Filters) Paradigm
When to Employ This Paradigm
When data must flow through a fixed sequence of discrete transformations, such as in ETL jobs, streaming analytics, or CI/CD pipelines.
When reusing individual processing stages is needed, either independently or to scale bottleneck stages separately from others.
When failure isolation between stages is a critical requirement.
When NOT To Use
Interactive request/response systems (use
archetypes:architecture-paradigm-client-server)
Stages that must share mutable state, which the pattern cannot express
Adoption Steps
Define Filters: Design each stage (filter) to perform a single, well-defined transformation. Each filter must have a clear input and output data schema.
Connect via Pipes: Connect the filters using "pipes," which can be implemented as streams, message queues, or in-memory channels. validate these pipes support back-pressure and buffering.
Maintain Stateless Filters: Where possible, design filters to be stateless. Any required state should be persisted externally or managed at the boundaries of the pipeline.
Instrument Each Stage: Implement monitoring for each filter to track key metrics such as latency, throughput, and error rates.
Orchestrate Deployments: Design the deployment strategy to allow each stage to be scaled horizontally and upgraded independently.
Key Deliverables
An Architecture Decision Record (ADR) documenting the filters, the chosen pipe technology, the error-handling strategy, and the tools for replaying data.
A suite of contract tests for each filter, plus integration tests that cover representative end-to-end pipeline executions.
Observability dashboards that visualize stage-level Key Performance Indicators (KPIs).
Risks & Mitigations
Single-Stage Bottlenecks:
Mitigation: Implement auto-scaling for individual filters. If a single filter remains a bottleneck, consider refactoring it into a more granular sub-pipeline.
Schema Drift Between Stages:
Mitigation: Centralize schema definitions in a shared repository and enforce compatibility tests as part of the CI/CD process to prevent breaking changes.
Back-Pressure Failures:
Mitigation: Conduct rigorous load testing to simulate high-volume scenarios. Validate that buffering, retry logic, and back-pressure mechanisms behave as expected under stress.
Concrete Components
These vocabulary items name the concrete tools and abstractions
that show up when the paradigm is implemented. They are not
required dependencies and they are not part of the skill's
tools: frontmatter (which is reserved for Claude Code tool
restrictions). Use this list to disambiguate during architecture