pig/pig-farm-controller

Fork 0

Files

huang 426ae41f54 bmad初始化

2025-11-01 19:22:39 +08:00

14 KiB

Raw Blame History

CI/CD Pipeline Setup

Workflow ID: bmad/bmm/testarch/ci Version: 4.0 (BMad v6)

Overview

Scaffolds a production-ready CI/CD quality pipeline with test execution, burn-in loops for flaky test detection, parallel sharding, artifact collection, and notification configuration. This workflow creates platform-specific CI configuration optimized for fast feedback and reliable test execution.

Preflight Requirements

Critical: Verify these requirements before proceeding. If any fail, HALT and notify the user.

✅ Git repository is initialized (.git/ directory exists)
✅ Local test suite passes (npm run test:e2e succeeds)
✅ Test framework is configured (from framework workflow)
✅ Team agrees on target CI platform (GitHub Actions, GitLab CI, Circle CI, etc.)
✅ Access to CI platform settings/secrets available (if updating existing pipeline)

Step 1: Run Preflight Checks

Actions

Verify Git Repository
- Check for .git/ directory
- Confirm remote repository configured (git remote -v)
- If not initialized, HALT with message: "Git repository required for CI/CD setup"
Validate Test Framework
- Look for playwright.config.* or cypress.config.*
- Read framework configuration to extract:
  - Test directory location
  - Test command
  - Reporter configuration
  - Timeout settings
- If not found, HALT with message: "Run framework workflow first to set up test infrastructure"
Run Local Tests
- Execute npm run test:e2e (or equivalent from package.json)
- Ensure tests pass before CI setup
- If tests fail, HALT with message: "Fix failing tests before setting up CI/CD"
Detect CI Platform
- Check for existing CI configuration:
  - .github/workflows/*.yml (GitHub Actions)
  - .gitlab-ci.yml (GitLab CI)
  - .circleci/config.yml (Circle CI)
  - Jenkinsfile (Jenkins)
- If found, ask user: "Update existing CI configuration or create new?"
- If not found, detect platform from git remote:
  - github.com → GitHub Actions (default)
  - gitlab.com → GitLab CI
  - Ask user if unable to auto-detect
Read Environment Configuration
- Check for .nvmrc to determine Node version
- Default to Node 20 LTS if not found
- Read package.json to identify dependencies (affects caching strategy)

Halt Condition: If preflight checks fail, stop immediately and report which requirement failed.

Step 2: Scaffold CI Pipeline

Actions

Select CI Platform Template

Based on detection or user preference, use the appropriate template:

GitHub Actions (.github/workflows/test.yml):
- Most common platform
- Excellent caching and matrix support
- Free for public repos, generous free tier for private
GitLab CI (.gitlab-ci.yml):
- Integrated with GitLab
- Built-in registry and runners
- Powerful pipeline features
Circle CI (.circleci/config.yml):
- Fast execution with parallelism
- Docker-first approach
- Enterprise features
Jenkins (Jenkinsfile):
- Self-hosted option
- Maximum customization
- Requires infrastructure management

Generate Pipeline Configuration

Use templates from {installed_path}/ directory:

github-actions-template.yml
gitlab-ci-template.yml

Key pipeline stages:

stages:
  - lint # Code quality checks
  - test # Test execution (parallel shards)
  - burn-in # Flaky test detection
  - report # Aggregate results and publish

Configure Test Execution

Parallel Sharding:

strategy:
  fail-fast: false
  matrix:
    shard: [1, 2, 3, 4]

steps:
  - name: Run tests
    run: npm run test:e2e -- --shard=${{ matrix.shard }}/${{ strategy.job-total }}

Purpose: Splits tests into N parallel jobs for faster execution (target: <10 min per shard)

Add Burn-In Loop

Critical pattern from production systems:

burn-in:
  name: Flaky Test Detection
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4

    - name: Setup Node
      uses: actions/setup-node@v4
      with:
        node-version-file: '.nvmrc'

    - name: Install dependencies
      run: npm ci

    - name: Run burn-in loop (10 iterations)
      run: |
        for i in {1..10}; do
          echo "🔥 Burn-in iteration $i/10"
          npm run test:e2e || exit 1
        done

    - name: Upload failure artifacts
      if: failure()
      uses: actions/upload-artifact@v4
      with:
        name: burn-in-failures
        path: test-results/
        retention-days: 30

Purpose: Runs tests multiple times to catch non-deterministic failures before they reach main branch.

When to run:

On pull requests to main/develop
Weekly on cron schedule
After significant test infrastructure changes

Configure Caching

Node modules cache:

- name: Cache dependencies
  uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

Browser binaries cache (Playwright):

- name: Cache Playwright browsers
  uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright
    key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}

Purpose: Reduces CI execution time by 2-5 minutes per run.

Configure Artifact Collection

Failure artifacts only:
```
- name: Upload test results
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: test-results-${{ matrix.shard }}
    path: |
      test-results/
      playwright-report/
    retention-days: 30
```
Artifacts to collect:
- Traces (Playwright) - full debugging context
- Screenshots - visual evidence of failures
- Videos - interaction playback
- HTML reports - detailed test results
- Console logs - error messages and warnings

Add Retry Logic

- name: Run tests with retries
  uses: nick-invision/retry@v2
  with:
    timeout_minutes: 30
    max_attempts: 3
    retry_on: error
    command: npm run test:e2e

Purpose: Handles transient failures (network issues, race conditions)

Configure Notifications (Optional)

If notify_on_failure is enabled:

- name: Notify on failure
  if: failure()
  uses: 8398a7/action-slack@v3
  with:
    status: ${{ job.status }}
    text: 'Test failures detected in PR #${{ github.event.pull_request.number }}'
    webhook_url: ${{ secrets.SLACK_WEBHOOK }}

Generate Helper Scripts

Selective testing script (scripts/test-changed.sh):

#!/bin/bash
# Run only tests for changed files

CHANGED_FILES=$(git diff --name-only HEAD~1)

if echo "$CHANGED_FILES" | grep -q "src/.*\.ts$"; then
  echo "Running affected tests..."
  npm run test:e2e -- --grep="$(echo $CHANGED_FILES | sed 's/src\///g' | sed 's/\.ts//g')"
else
  echo "No test-affecting changes detected"
fi

Local mirror script (scripts/ci-local.sh):

#!/bin/bash
# Mirror CI execution locally for debugging

echo "🔍 Running CI pipeline locally..."

# Lint
npm run lint || exit 1

# Tests
npm run test:e2e || exit 1

# Burn-in (reduced iterations)
for i in {1..3}; do
  echo "🔥 Burn-in $i/3"
  npm run test:e2e || exit 1
done

echo "✅ Local CI pipeline passed"

Generate Documentation

CI README (docs/ci.md):
- Pipeline stages and purpose
- How to run locally
- Debugging failed CI runs
- Secrets and environment variables needed
- Notification setup
- Badge URLs for README
Secrets checklist (docs/ci-secrets-checklist.md):
- Required secrets list (SLACK_WEBHOOK, etc.)
- Where to configure in CI platform
- Security best practices

Step 3: Deliverables

Primary Artifacts Created

CI Configuration File
- .github/workflows/test.yml (GitHub Actions)
- .gitlab-ci.yml (GitLab CI)
- .circleci/config.yml (Circle CI)
Pipeline Stages
- Lint: Code quality checks (ESLint, Prettier)
- Test: Parallel test execution (4 shards)
- Burn-in: Flaky test detection (10 iterations)
- Report: Result aggregation and publishing
Helper Scripts
- scripts/test-changed.sh - Selective testing
- scripts/ci-local.sh - Local CI mirror
- scripts/burn-in.sh - Standalone burn-in execution
Documentation
- docs/ci.md - CI pipeline guide
- docs/ci-secrets-checklist.md - Required secrets
- Inline comments in CI configuration
Optimization Features
- Dependency caching (npm, browser binaries)
- Parallel sharding (4 jobs default)
- Retry logic (2 retries on failure)
- Failure-only artifact upload

Performance Targets

Lint stage: <2 minutes
Test stage (per shard): <10 minutes
Burn-in stage: <30 minutes (10 iterations)
Total pipeline: <45 minutes

Speedup: 20× faster than sequential execution through parallelism and caching.

Important Notes

Knowledge Base Integration

Critical: Consult {project-root}/bmad/bmm/testarch/tea-index.csv to identify and load relevant knowledge fragments:

ci-burn-in.md - Burn-in loop patterns: 10-iteration detection, GitHub Actions workflow, shard orchestration, selective execution (678 lines, 4 examples)
selective-testing.md - Changed test detection strategies: tag-based, spec filters, diff-based selection, promotion rules (727 lines, 4 examples)
visual-debugging.md - Artifact collection best practices: trace viewer, HAR recording, custom artifacts, accessibility integration (522 lines, 5 examples)
test-quality.md - CI-specific test quality criteria: deterministic tests, isolated with cleanup, explicit assertions, length/time optimization (658 lines, 5 examples)
playwright-config.md - CI-optimized configuration: parallelization, artifact output, project dependencies, sharding (722 lines, 5 examples)

CI Platform-Specific Guidance

GitHub Actions:

Use actions/cache for caching
Matrix strategy for parallelism
Secrets in repository settings
Free 2000 minutes/month for private repos

GitLab CI:

Use .gitlab-ci.yml in root
cache: directive for caching
Parallel execution with parallel: 4
Variables in project CI/CD settings

Circle CI:

Use .circleci/config.yml
Docker executors recommended
Parallelism with parallelism: 4
Context for shared secrets

Burn-In Loop Strategy

When to run:

✅ On PRs to main/develop branches
✅ Weekly on schedule (cron)
✅ After test infrastructure changes
❌ Not on every commit (too slow)

Iterations:

10 iterations for thorough detection
3 iterations for quick feedback
100 iterations for high-confidence stability

Failure threshold:

Even ONE failure in burn-in → tests are flaky
Must fix before merging

Artifact Retention

Failure artifacts only:

Saves storage costs
Maintains debugging capability
30-day retention default

Artifact types:

Traces (Playwright) - 5-10 MB per test
Screenshots - 100-500 KB per screenshot
Videos - 2-5 MB per test
HTML reports - 1-2 MB per run

Selective Testing

Detect changed files:

git diff --name-only HEAD~1

Run affected tests only:

Faster feedback for small changes
Full suite still runs on main branch
Reduces CI time by 50-80% for focused PRs

Trade-off:

May miss integration issues
Run full suite at least on merge

Local CI Mirror

Purpose: Debug CI failures locally

Usage:

./scripts/ci-local.sh

Mirrors CI environment:

Same Node version
Same test command
Same stages (lint → test → burn-in)
Reduced burn-in iterations (3 vs 10)

Output Summary

After completing this workflow, provide a summary:

## CI/CD Pipeline Complete

**Platform**: GitHub Actions (or GitLab CI, etc.)

**Artifacts Created**:

- ✅ Pipeline configuration: .github/workflows/test.yml
- ✅ Burn-in loop: 10 iterations for flaky detection
- ✅ Parallel sharding: 4 jobs for fast execution
- ✅ Caching: Dependencies + browser binaries
- ✅ Artifact collection: Failure-only traces/screenshots/videos
- ✅ Helper scripts: test-changed.sh, ci-local.sh, burn-in.sh
- ✅ Documentation: docs/ci.md, docs/ci-secrets-checklist.md

**Performance:**

- Lint: <2 min
- Test (per shard): <10 min
- Burn-in: <30 min
- Total: <45 min (20× speedup vs sequential)

**Next Steps**:

1. Commit CI configuration: `git add .github/workflows/test.yml && git commit -m "ci: add test pipeline"`
2. Push to remote: `git push`
3. Configure required secrets in CI platform settings (see docs/ci-secrets-checklist.md)
4. Open a PR to trigger first CI run
5. Monitor pipeline execution and adjust parallelism if needed

**Knowledge Base References Applied**:

- Burn-in loop pattern (ci-burn-in.md)
- Selective testing strategy (selective-testing.md)
- Artifact collection (visual-debugging.md)
- Test quality criteria (test-quality.md)

Validation

After completing all steps, verify:

CI configuration file created and syntactically valid
Burn-in loop configured (10 iterations)
Parallel sharding enabled (4 jobs)
Caching configured (dependencies + browsers)
Artifact collection on failure only
Helper scripts created and executable (chmod +x)
Documentation complete (ci.md, secrets checklist)
No errors or warnings during scaffold

Refer to checklist.md for comprehensive validation criteria.

14 KiB Raw Blame History Unescape Escape

CI/CD Pipeline Setup

Overview

Preflight Requirements

Step 1: Run Preflight Checks

Actions

Step 2: Scaffold CI Pipeline

Actions

Step 3: Deliverables

Primary Artifacts Created

Performance Targets

Important Notes

Knowledge Base Integration

CI Platform-Specific Guidance

Burn-In Loop Strategy

Artifact Retention

Selective Testing

Local CI Mirror

Output Summary

Validation

14 KiB

Raw Blame History