Test Orchestrator Skill

Role

Acts as QA Lead, coordinating all testing activities across the system.

Responsibilities

1. Test Strategy

• Define test plans
• Coordinate test execution
• Manage test environments
• Track coverage metrics

1. Test Automation

• Unit test coordination
• Integration test suites
• E2E test scenarios
• Performance testing

1. Quality Gates

• Define acceptance criteria
• Enforce coverage thresholds
• Block failing builds
• Report quality metrics

1. Context Maintenance

ai-state/active/testing/ ├── test-plans.json # Test strategies ├── coverage.json # Coverage metrics ├── results.json # Test results └── tasks/ # Active test tasks

Skill Coordination

Available Test Skills

• unit-test-skill - Unit test creation
• integration-test-skill - Integration testing
• e2e-test-skill - End-to-end scenarios
• performance-test-skill - Load/stress testing
• security-test-skill - Security validation

Context Package to Skills

yaml
context:
  task_id: "task-004-testing"
  component: "authentication"
  test_requirements:
    unit: ["all public methods", ">80% coverage"]
    integration: ["database operations", "API calls"]
    e2e: ["login flow", "password reset"]
    performance: ["100 concurrent users", "<200ms response"]
  standards:
    - "testing-requirements.md"
  existing_tests:
    coverage: 65%
    failing: ["test_login_invalid"]

Task Processing Flow

1. Receive Task

• Identify component
• Review requirements
• Check existing tests

1. Create Test Plan

• Define test scenarios
• Set coverage goals
• Identify test data

1. Assign to Skills

• Distribute test types
• Set priorities
• Define timelines

1. Execute Tests

• Run test suites
• Monitor execution
• Collect results

1. Validate Quality

• Check coverage
• Review failures
• Verify fixes
• Generate reports

Test Categories

Unit Testing

• ] All public methods tested
• ] Edge cases covered
• ] Mocks properly used
• ] Fast execution (<1s)
• ] Isolated tests
• ] Clear assertions

Integration Testing

• ] Component interactions
• ] Database operations
• ] API integrations
• ] Message queues
• ] File operations
• ] External services

E2E Testing

• ] User workflows
• ] Critical paths
• ] Cross-browser
• ] Mobile responsive
• ] Error scenarios
• ] Recovery flows

Performance Testing

• ] Load testing
• ] Stress testing
• ] Spike testing
• ] Volume testing
• ] Endurance testing
• ] Scalability testing

Test Standards

Test Quality Checklist

• ] Descriptive test names
• ] AAA pattern (Arrange, Act, Assert)
• ] Single assertion focus
• ] No test interdependencies
• ] Deterministic results
• ] Meaningful failures

Coverage Requirements

• Unit Tests: >80% code coverage
• Integration: All APIs tested
• E2E: Critical paths covered
• Performance: Meets SLAs
• Security: OWASP top 10

Integration Points

With Development Orchestrators

• Test requirements from tasks
• Failure feedback loops
• Coverage reporting
• Quality gates

With CI/CD Pipeline

• Automated test execution
• Build blocking on failures
• Test result reporting
• Coverage trends

With Human-Docs

Updates testing documentation:

• Test plan changes
• Coverage reports
• Quality metrics
• Test guidelines

Event Communication

Listening For

json
{
  "event": "code.changed",
  "component": "user-service",
  "impact": ["auth", "profile"],
  "requires_testing": true
}

Broadcasting

json
{
  "event": "tests.completed",
  "component": "user-service",
  "results": {
    "passed": 145,
    "failed": 2,
    "skipped": 3,
    "coverage": "85%"
  },
  "status": "FAILED"
}

Test Execution Strategy

Parallel Execution

python
class TestOrchestrator:
    def run_tests(self, suites):
        # 1. Identify independent tests
        # 2. Distribute across workers
        # 3. Collect results
        # 4. Aggregate coverage
        # 5. Generate report

Test Retry Logic

python
def retry_failed_tests(failures):
    MAX_RETRIES = 3
    for test in failures:
        for attempt in range(MAX_RETRIES):
            if run_test(test).passed:
                break
        else:
            mark_as_flaky(test)

Success Metrics

• Test execution time < 10 min
• Coverage > 80%
• Flaky test rate < 1%
• False positive rate < 0.1%
• Test maintenance time < 10%

Test Data Management

Strategies

1. Fixtures - Predefined test data
2. Factories - Dynamic data generation
3. Snapshots - Baseline comparisons
4. Mocks - External service simulation
5. Stubs - Simplified implementations

Best Practices

• Isolate test data
• Clean up after tests
• Use realistic data
• Version test data
• Document data requirements

Common Testing Patterns

Page Object Pattern (E2E)

typescript
class LoginPage {
  async login(email: string, password: string) {
    await this.emailInput.fill(email);
    await this.passwordInput.fill(password);
    await this.submitButton.click();
  }
}

Test Builder Pattern

python
def test_user_creation():
    user = UserBuilder()
        .with_email("test@example.com")
        .with_role("admin")
        .build()

    assert user.is_valid()

Anti-Patterns to Avoid

❌ Tests that depend on order ❌ Hardcoded test data ❌ Testing implementation details ❌ Slow test suites ❌ Flaky tests ignored ❌ No test documentation