Files
pig-farm-controller/bmad/bmm/workflows/testarch/framework
2025-11-01 19:22:39 +08:00
..
2025-11-01 19:22:39 +08:00
2025-11-01 19:22:39 +08:00
2025-11-01 19:22:39 +08:00
2025-11-01 19:22:39 +08:00

Test Framework Setup Workflow

Initializes a production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, configuration, and industry best practices. This workflow scaffolds the complete testing infrastructure for modern web applications, providing a robust foundation for test automation.

Usage

bmad tea *framework

The TEA agent runs this workflow when:

  • Starting a new project that needs test infrastructure
  • Migrating from an older testing approach
  • Setting up testing from scratch
  • Standardizing test architecture across teams

Inputs

Required Context Files:

  • package.json: Project dependencies and scripts to detect project type and bundler

Optional Context Files:

  • Architecture docs (architecture.md, tech-spec.md): Informs framework configuration decisions
  • Existing tests: Detects current framework to avoid conflicts

Workflow Variables:

  • test_framework: Auto-detected (playwright/cypress) or manually specified
  • project_type: Auto-detected from package.json (react/vue/angular/next/node)
  • bundler: Auto-detected from package.json (vite/webpack/rollup/esbuild)
  • test_dir: Root test directory (default: {project-root}/tests)
  • use_typescript: Prefer TypeScript configuration (default: true)
  • framework_preference: Auto-detection or force specific framework (default: "auto")

Outputs

Primary Deliverables:

  1. Configuration File

    • playwright.config.ts or cypress.config.ts with production-ready settings
    • Timeouts: action 15s, navigation 30s, test 60s
    • Reporters: HTML + JUnit XML
    • Failure-only artifacts (traces, screenshots, videos)
  2. Directory Structure

    tests/
    ├── e2e/                          # Test files (organize as needed)
    ├── support/                      # Framework infrastructure (key pattern)
    │   ├── fixtures/                 # Test fixtures with auto-cleanup
    │   │   ├── index.ts             # Fixture merging
    │   │   └── factories/           # Data factories (faker-based)
    │   ├── helpers/                 # Utility functions
    │   └── page-objects/            # Page object models (optional)
    └── README.md                    # Setup and usage guide
    

    Note: Test organization (e2e/, api/, integration/, etc.) is flexible. The support/ folder contains reusable fixtures, helpers, and factories - the core framework pattern.

  3. Environment Configuration

    • .env.example with TEST_ENV, BASE_URL, API_URL, auth credentials
    • .nvmrc with Node version (LTS)
  4. Test Infrastructure

    • Fixture architecture using mergeTests pattern
    • Data factories with auto-cleanup (faker-based)
    • Sample tests demonstrating best practices
    • Helper utilities for common operations
  5. Documentation

    • tests/README.md with comprehensive setup instructions
    • Inline comments explaining configuration choices
    • References to TEA knowledge base

Secondary Deliverables:

  • Updated package.json with minimal test script (test:e2e)
  • Sample test demonstrating fixture usage
  • Network-first testing patterns
  • Selector strategy guidance (data-testid)

Validation Safeguards:

  • No existing framework detected (prevents conflicts)
  • package.json exists and is valid
  • Framework auto-detection successful or explicit choice provided
  • Sample test runs successfully
  • All generated files are syntactically correct

Key Features

Smart Framework Selection

  • Auto-detection logic based on project characteristics:
    • Playwright recommended for: Large repos (100+ files), performance-critical apps, multi-browser support, complex debugging needs
    • Cypress recommended for: Small teams prioritizing DX, component testing focus, real-time test development
  • Falls back to Playwright as default if uncertain

Production-Ready Patterns

  • Fixture Architecture: Pure function → fixture → mergeTests composition pattern
  • Auto-Cleanup: Fixtures automatically clean up test data in teardown
  • Network-First: Route interception before navigation to prevent race conditions
  • Failure-Only Artifacts: Screenshots/videos/traces only captured on failure to reduce storage
  • Parallel Execution: Configured for optimal CI performance

Industry Best Practices

  • Selector Strategy: Prescriptive guidance on data-testid attributes
  • Data Factories: Faker-based factories for realistic test data
  • Contract Testing: Recommends Pact for microservices architectures
  • Error Handling: Comprehensive timeout and retry configuration
  • Reporting: Multiple reporter formats (HTML, JUnit, console)

Knowledge Base Integration

Automatically consults TEA knowledge base:

  • fixture-architecture.md - Pure function → fixture → mergeTests pattern
  • data-factories.md - Faker-based factories with auto-cleanup
  • network-first.md - Network interception before navigation
  • playwright-config.md - Playwright-specific best practices
  • test-config.md - General configuration guidelines

Integration with Other Workflows

Before framework:

  • prd (Phase 2): Determines project scope and testing needs
  • workflow-status: Verifies project readiness

After framework:

  • ci: Scaffold CI/CD pipeline using framework configuration
  • test-design: Plan test coverage strategy for the project
  • atdd: Generate failing acceptance tests using the framework

Coordinates with:

  • architecture (Phase 3): Aligns test structure with system architecture
  • tech-spec: Uses technical specifications to inform test configuration

Updates:

  • bmm-workflow-status.md: Adds framework initialization to Quality & Testing Progress section

Important Notes

Preflight Checks

Critical requirements verified before scaffolding:

  • package.json exists in project root
  • No modern E2E framework already configured
  • Architecture/stack context available

If any check fails, workflow HALTS and notifies user.

Framework-Specific Guidance

Playwright Advantages:

  • Worker parallelism (significantly faster for large suites)
  • Trace viewer (powerful debugging with screenshots, network, console logs)
  • Multi-language support (TypeScript, JavaScript, Python, C#, Java)
  • Built-in API testing capabilities
  • Better handling of multiple browser contexts

Cypress Advantages:

  • Superior developer experience (real-time reloading)
  • Excellent for component testing
  • Simpler setup for small teams
  • Better suited for watch mode during development

Avoid Cypress when:

  • API chains are heavy and complex
  • Multi-tab/window scenarios are common
  • Worker parallelism is critical for CI performance

Selector Strategy

Always recommend:

  • data-testid attributes for UI elements (framework-agnostic)
  • data-cy attributes if Cypress is chosen (Cypress-specific)
  • Avoid brittle CSS selectors or XPath

Standalone Operation

This workflow operates independently:

  • No story required: Can be run at project initialization
  • No epic context needed: Works for greenfield and brownfield projects
  • Autonomous: Auto-detects configuration and proceeds without user input

Output Summary Format

After completion, provides structured summary:

## Framework Scaffold Complete

**Framework Selected**: Playwright (or Cypress)

**Artifacts Created**:

- ✅ Configuration file: playwright.config.ts
- ✅ Directory structure: tests/e2e/, tests/support/
- ✅ Environment config: .env.example
- ✅ Node version: .nvmrc
- ✅ Fixture architecture: tests/support/fixtures/
- ✅ Data factories: tests/support/fixtures/factories/
- ✅ Sample tests: tests/e2e/example.spec.ts
- ✅ Documentation: tests/README.md

**Next Steps**:

1. Copy .env.example to .env and fill in environment variables
2. Run npm install to install test dependencies
3. Run npm run test:e2e to execute sample tests
4. Review tests/README.md for detailed setup instructions

**Knowledge Base References Applied**:

- Fixture architecture pattern (pure functions + mergeTests)
- Data factories with auto-cleanup (faker-based)
- Network-first testing safeguards
- Failure-only artifact capture

Validation Checklist

After workflow completion, verify:

  • Configuration file created and syntactically valid
  • Directory structure exists with all folders
  • Environment configuration generated (.env.example, .nvmrc)
  • Sample tests run successfully (npm run test:e2e)
  • Documentation complete and accurate (tests/README.md)
  • No errors or warnings during scaffold
  • package.json scripts updated correctly
  • Fixtures and factories follow patterns from knowledge base

Refer to checklist.md for comprehensive validation criteria.

Example Execution

Scenario 1: New React + Vite project

# User runs framework workflow
bmad tea *framework

# TEA detects:
# - React project (from package.json)
# - Vite bundler
# - No existing test framework
# - 150+ files (recommends Playwright)

# TEA scaffolds:
# - playwright.config.ts with Vite detection
# - Component testing configuration
# - React Testing Library helpers
# - Sample component + E2E tests

Scenario 2: Existing Node.js API project

# User runs framework workflow
bmad tea *framework

# TEA detects:
# - Node.js backend (no frontend framework)
# - Express framework
# - Small project (50 files)
# - API endpoints in routes/

# TEA scaffolds:
# - playwright.config.ts focused on API testing
# - tests/api/ directory structure
# - API helper utilities
# - Sample API tests with auth

Scenario 3: Cypress preferred (explicit)

# User sets framework preference
# (in workflow config: framework_preference: "cypress")

bmad tea *framework

# TEA scaffolds:
# - cypress.config.ts
# - tests/e2e/ with Cypress patterns
# - Cypress-specific commands
# - data-cy selector strategy

Troubleshooting

Issue: "Existing test framework detected"

  • Cause: playwright.config._ or cypress.config._ already exists
  • Solution: Use upgrade-framework workflow (TBD) or manually remove existing config

Issue: "Cannot detect project type"

  • Cause: package.json missing or malformed
  • Solution: Ensure package.json exists and has valid dependencies

Issue: "Sample test fails to run"

  • Cause: Missing dependencies or incorrect BASE_URL
  • Solution: Run npm install and configure .env with correct URLs

Issue: "TypeScript compilation errors"

  • Cause: Missing @types packages or tsconfig misconfiguration
  • Solution: Ensure TypeScript and type definitions are installed

Version History

  • v4.0 (BMad v6): Pure markdown instructions, enhanced workflow.yaml, comprehensive README
  • v3.x: XML format instructions
  • v2.x: Legacy task-based approach