pig-farm-controller/bmad/bmm/testarch/knowledge/test-priorities-matrix.md

<!-- Powered by BMAD-CORE™ -->

# Test Priorities Matrix

Guide for prioritizing test scenarios based on risk, criticality, and business impact.

## Priority Levels

### P0 - Critical (Must Test)

**Criteria:**

- Revenue-impacting functionality
- Security-critical paths
- Data integrity operations
- Regulatory compliance requirements
- Previously broken functionality (regression prevention)

**Examples:**

- Payment processing
- Authentication/authorization
- User data creation/deletion
- Financial calculations
- GDPR/privacy compliance

**Testing Requirements:**

- Comprehensive coverage at all levels
- Both happy and unhappy paths
- Edge cases and error scenarios
- Performance under load

### P1 - High (Should Test)

**Criteria:**

- Core user journeys
- Frequently used features
- Features with complex logic
- Integration points between systems
- Features affecting user experience

**Examples:**

- User registration flow
- Search functionality
- Data import/export
- Notification systems
- Dashboard displays

**Testing Requirements:**

- Primary happy paths required
- Key error scenarios
- Critical edge cases
- Basic performance validation

### P2 - Medium (Nice to Test)

**Criteria:**

- Secondary features
- Admin functionality
- Reporting features
- Configuration options
- UI polish and aesthetics

**Examples:**

- Admin settings panels
- Report generation
- Theme customization
- Help documentation
- Analytics tracking

**Testing Requirements:**

- Happy path coverage
- Basic error handling
- Can defer edge cases

### P3 - Low (Test if Time Permits)

**Criteria:**

- Rarely used features
- Nice-to-have functionality
- Cosmetic issues
- Non-critical optimizations

**Examples:**

- Advanced preferences
- Legacy feature support
- Experimental features
- Debug utilities

**Testing Requirements:**

- Smoke tests only
- Can rely on manual testing
- Document known limitations

## Risk-Based Priority Adjustments

### Increase Priority When:

- High user impact (affects >50% of users)
- High financial impact (>$10K potential loss)
- Security vulnerability potential
- Compliance/legal requirements
- Customer-reported issues
- Complex implementation (>500 LOC)
- Multiple system dependencies

### Decrease Priority When:

- Feature flag protected
- Gradual rollout planned
- Strong monitoring in place
- Easy rollback capability
- Low usage metrics
- Simple implementation
- Well-isolated component

## Test Coverage by Priority

| Priority | Unit Coverage | Integration Coverage | E2E Coverage       |
| -------- | ------------- | -------------------- | ------------------ |
| P0       | >90%          | >80%                 | All critical paths |
| P1       | >80%          | >60%                 | Main happy paths   |
| P2       | >60%          | >40%                 | Smoke tests        |
| P3       | Best effort   | Best effort          | Manual only        |

## Priority Assignment Rules

1. **Start with business impact** - What happens if this fails?
2. **Consider probability** - How likely is failure?
3. **Factor in detectability** - Would we know if it failed?
4. **Account for recoverability** - Can we fix it quickly?

## Priority Decision Tree

```
Is it revenue-critical?
├─ YES → P0
└─ NO → Does it affect core user journey?
    ├─ YES → Is it high-risk?
    │   ├─ YES → P0
    │   └─ NO → P1
    └─ NO → Is it frequently used?
        ├─ YES → P1
        └─ NO → Is it customer-facing?
            ├─ YES → P2
            └─ NO → P3
```

## Test Execution Order

1. Execute P0 tests first (fail fast on critical issues)
2. Execute P1 tests second (core functionality)
3. Execute P2 tests if time permits
4. P3 tests only in full regression cycles

## Continuous Adjustment

Review and adjust priorities based on:

- Production incident patterns
- User feedback and complaints
- Usage analytics
- Test failure history
- Business priority changes

---

## Automated Priority Classification

### Example: Priority Calculator (Risk-Based Automation)

```typescript
// src/testing/priority-calculator.ts

export type Priority = 'P0' | 'P1' | 'P2' | 'P3';

export type PriorityFactors = {
  revenueImpact: 'critical' | 'high' | 'medium' | 'low' | 'none';
  userImpact: 'all' | 'majority' | 'some' | 'few' | 'minimal';
  securityRisk: boolean;
  complianceRequired: boolean;
  previousFailure: boolean;
  complexity: 'high' | 'medium' | 'low';
  usage: 'frequent' | 'regular' | 'occasional' | 'rare';
};

/**
 * Calculate test priority based on multiple factors
 * Mirrors the priority decision tree with objective criteria
 */
export function calculatePriority(factors: PriorityFactors): Priority {
  const { revenueImpact, userImpact, securityRisk, complianceRequired, previousFailure, complexity, usage } = factors;

  // P0: Revenue-critical, security, or compliance
  if (revenueImpact === 'critical' || securityRisk || complianceRequired || (previousFailure && revenueImpact === 'high')) {
    return 'P0';
  }

  // P0: High revenue + high complexity + frequent usage
  if (revenueImpact === 'high' && complexity === 'high' && usage === 'frequent') {
    return 'P0';
  }

  // P1: Core user journey (majority impacted + frequent usage)
  if (userImpact === 'all' || userImpact === 'majority') {
    if (usage === 'frequent' || complexity === 'high') {
      return 'P1';
    }
  }

  // P1: High revenue OR high complexity with regular usage
  if ((revenueImpact === 'high' && usage === 'regular') || (complexity === 'high' && usage === 'frequent')) {
    return 'P1';
  }

  // P2: Secondary features (some impact, occasional usage)
  if (userImpact === 'some' || usage === 'occasional') {
    return 'P2';
  }

  // P3: Rarely used, low impact
  return 'P3';
}

/**
 * Generate priority justification (for audit trail)
 */
export function justifyPriority(factors: PriorityFactors): string {
  const priority = calculatePriority(factors);
  const reasons: string[] = [];

  if (factors.revenueImpact === 'critical') reasons.push('critical revenue impact');
  if (factors.securityRisk) reasons.push('security-critical');
  if (factors.complianceRequired) reasons.push('compliance requirement');
  if (factors.previousFailure) reasons.push('regression prevention');
  if (factors.userImpact === 'all' || factors.userImpact === 'majority') {
    reasons.push(`impacts ${factors.userImpact} users`);
  }
  if (factors.complexity === 'high') reasons.push('high complexity');
  if (factors.usage === 'frequent') reasons.push('frequently used');

  return `${priority}: ${reasons.join(', ')}`;
}

/**
 * Example: Payment scenario priority calculation
 */
const paymentScenario: PriorityFactors = {
  revenueImpact: 'critical',
  userImpact: 'all',
  securityRisk: true,
  complianceRequired: true,
  previousFailure: false,
  complexity: 'high',
  usage: 'frequent',
};

console.log(calculatePriority(paymentScenario)); // 'P0'
console.log(justifyPriority(paymentScenario));
// 'P0: critical revenue impact, security-critical, compliance requirement, impacts all users, high complexity, frequently used'
```

### Example: Test Suite Tagging Strategy

```typescript
// tests/e2e/checkout.spec.ts
import { test, expect } from '@playwright/test';

// Tag tests with priority for selective execution
test.describe('Checkout Flow', () => {
  test('valid payment completes successfully @p0 @smoke @revenue', async ({ page }) => {
    // P0: Revenue-critical happy path
    await page.goto('/checkout');
    await page.getByTestId('payment-method').selectOption('credit-card');
    await page.getByTestId('card-number').fill('4242424242424242');
    await page.getByRole('button', { name: 'Place Order' }).click();

    await expect(page.getByText('Order confirmed')).toBeVisible();
  });

  test('expired card shows user-friendly error @p1 @error-handling', async ({ page }) => {
    // P1: Core error scenario (frequent user impact)
    await page.goto('/checkout');
    await page.getByTestId('payment-method').selectOption('credit-card');
    await page.getByTestId('card-number').fill('4000000000000069'); // Test card: expired
    await page.getByRole('button', { name: 'Place Order' }).click();

    await expect(page.getByText('Card expired. Please use a different card.')).toBeVisible();
  });

  test('coupon code applies discount correctly @p2', async ({ page }) => {
    // P2: Secondary feature (nice-to-have)
    await page.goto('/checkout');
    await page.getByTestId('coupon-code').fill('SAVE10');
    await page.getByRole('button', { name: 'Apply' }).click();

    await expect(page.getByText('10% discount applied')).toBeVisible();
  });

  test('gift message formatting preserved @p3', async ({ page }) => {
    // P3: Cosmetic feature (rarely used)
    await page.goto('/checkout');
    await page.getByTestId('gift-message').fill('Happy Birthday!\n\nWith love.');
    await page.getByRole('button', { name: 'Place Order' }).click();

    // Message formatting preserved (linebreaks intact)
    await expect(page.getByTestId('order-summary')).toContainText('Happy Birthday!');
  });
});
```

**Run tests by priority:**

```bash
# P0 only (smoke tests, 2-5 min)
npx playwright test --grep @p0

# P0 + P1 (core functionality, 10-15 min)
npx playwright test --grep "@p0|@p1"

# Full regression (all priorities, 30+ min)
npx playwright test
```

---

## Integration with Risk Scoring

Priority should align with risk score from `probability-impact.md`:

| Risk Score | Typical Priority | Rationale                                  |
| ---------- | ---------------- | ------------------------------------------ |
| 9          | P0               | Critical blocker (probability=3, impact=3) |
| 6-8        | P0 or P1         | High risk (requires mitigation)            |
| 4-5        | P1 or P2         | Medium risk (monitor closely)              |
| 1-3        | P2 or P3         | Low risk (document and defer)              |

**Example**: Risk score 9 (checkout API failure) → P0 priority → comprehensive coverage required.

---

## Priority Checklist

Before finalizing test priorities:

- [ ] **Revenue impact assessed**: Payment, subscription, billing features → P0
- [ ] **Security risks identified**: Auth, data exposure, injection attacks → P0
- [ ] **Compliance requirements documented**: GDPR, PCI-DSS, SOC2 → P0
- [ ] **User impact quantified**: >50% users → P0/P1, <10% → P2/P3
- [ ] **Previous failures reviewed**: Regression prevention → increase priority
- [ ] **Complexity evaluated**: >500 LOC or multiple dependencies → increase priority
- [ ] **Usage metrics consulted**: Frequent use → P0/P1, rare use → P2/P3
- [ ] **Monitoring coverage confirmed**: Strong monitoring → can decrease priority
- [ ] **Rollback capability verified**: Easy rollback → can decrease priority
- [ ] **Priorities tagged in tests**: @p0, @p1, @p2, @p3 for selective execution

## Integration Points

- **Used in workflows**: `*automate` (priority-based test generation), `*test-design` (scenario prioritization), `*trace` (coverage validation by priority)
- **Related fragments**: `risk-governance.md` (risk scoring), `probability-impact.md` (impact assessment), `selective-testing.md` (tag-based execution)
- **Tools**: Playwright/Cypress grep for tag filtering, CI scripts for priority-based execution

_Source: Risk-based testing practices, test prioritization strategies, production incident analysis_