# Non-Functional Requirements Assessment - Instructions v4.0

**Workflow:** `testarch-nfr`
**Purpose:** Assess non-functional requirements (performance, security, reliability, maintainability) before release with evidence-based validation
**Agent:** Test Architect (TEA)
**Format:** Pure Markdown v4.0 (no XML blocks)

---

## Overview

This workflow performs a comprehensive assessment of non-functional requirements (NFRs) to validate that the implementation meets performance, security, reliability, and maintainability standards before release. It uses evidence-based validation with deterministic PASS/CONCERNS/FAIL rules and provides actionable recommendations for remediation.

**Key Capabilities:**

- Assess multiple NFR categories (performance, security, reliability, maintainability, custom)
- Validate NFRs against defined thresholds from tech specs, PRD, or defaults
- Classify status deterministically (PASS/CONCERNS/FAIL) based on evidence
- Never guess thresholds - mark as CONCERNS if unknown
- Generate gate-ready YAML snippets for CI/CD integration
- Provide quick wins and recommended actions for remediation
- Create evidence checklists for gaps

---

## Prerequisites

**Required:**

- Implementation deployed locally or accessible for evaluation
- Evidence sources available (test results, metrics, logs, CI results)

**Recommended:**

- NFR requirements defined in tech-spec.md, PRD.md, or story
- Test results from performance, security, reliability tests
- Application metrics (response times, error rates, throughput)
- CI/CD pipeline results for burn-in validation

**Halt Conditions:**

- If NFR targets are undefined and cannot be obtained, halt and request definition
- If implementation is not accessible for evaluation, halt and request deployment

---

## Workflow Steps

### Step 1: Load Context and Knowledge Base

**Actions:**

1. Load relevant knowledge fragments from `{project-root}/bmad/bmm/testarch/tea-index.csv`:
   - `nfr-criteria.md` - Non-functional requirements criteria and thresholds (security, performance, reliability, maintainability with code examples, 658 lines, 4 examples)
   - `ci-burn-in.md` - CI/CD burn-in patterns for reliability validation (10-iteration detection, sharding, selective execution, 678 lines, 4 examples)
   - `test-quality.md` - Test quality expectations for maintainability (deterministic, isolated, explicit assertions, length/time limits, 658 lines, 5 examples)
   - `playwright-config.md` - Performance configuration patterns: parallelization, timeout standards, artifact output (722 lines, 5 examples)
   - `error-handling.md` - Reliability validation patterns: scoped exceptions, retry validation, telemetry logging, graceful degradation (736 lines, 4 examples)

2. Read story file (if provided):
   - Extract NFR requirements
   - Identify specific thresholds or SLAs
   - Note any custom NFR categories

3. Read related BMad artifacts (if available):
   - `tech-spec.md` - Technical NFR requirements and targets
   - `PRD.md` - Product-level NFR context (user expectations)
   - `test-design.md` - NFR test plan and priorities

**Output:** Complete understanding of NFR targets, evidence sources, and validation criteria

---

### Step 2: Identify NFR Categories and Thresholds

**Actions:**

1. Determine which NFR categories to assess (default: performance, security, reliability, maintainability):
   - **Performance**: Response time, throughput, resource usage
   - **Security**: Authentication, authorization, data protection, vulnerability scanning
   - **Reliability**: Error handling, recovery, availability, fault tolerance
   - **Maintainability**: Code quality, test coverage, documentation, technical debt

2. Add custom NFR categories if specified (e.g., accessibility, internationalization, compliance)

3. Gather thresholds for each NFR:
   - From tech-spec.md (primary source)
   - From PRD.md (product-level SLAs)
   - From story file (feature-specific requirements)
   - From workflow variables (default thresholds)
   - Mark thresholds as UNKNOWN if not defined

4. Never guess thresholds - if a threshold is unknown, mark the NFR as CONCERNS

**Output:** Complete list of NFRs to assess with defined (or UNKNOWN) thresholds

---

### Step 3: Gather Evidence

**Actions:**

1. For each NFR category, discover evidence sources:

   **Performance Evidence:**
   - Load test results (JMeter, k6, Lighthouse)
   - Application metrics (response times, throughput, resource usage)
   - Performance monitoring data (New Relic, Datadog, APM)
   - Playwright performance traces (if applicable)

   **Security Evidence:**
   - Security scan results (SAST, DAST, dependency scanning)
   - Authentication/authorization test results
   - Penetration test reports
   - Vulnerability assessment reports
   - Compliance audit results

   **Reliability Evidence:**
   - Error logs and error rates
   - Uptime monitoring data
   - Chaos engineering test results
   - Failover/recovery test results
   - CI burn-in results (stability over time)

   **Maintainability Evidence:**
   - Code coverage reports (Istanbul, NYC, c8)
   - Static analysis results (ESLint, SonarQube)
   - Technical debt metrics
   - Documentation completeness
   - Test quality assessment (from test-review workflow)

2. Read relevant files from evidence directories:
   - `{test_results_dir}` for test execution results
   - `{metrics_dir}` for application metrics
   - `{logs_dir}` for application logs
   - CI/CD pipeline results (if `include_ci_results` is true)

3. Mark NFRs without evidence as "NO EVIDENCE" - never infer or assume

**Output:** Comprehensive evidence inventory for each NFR

---

### Step 4: Assess NFRs with Deterministic Rules

**Actions:**

1. For each NFR, apply deterministic PASS/CONCERNS/FAIL rules:

   **PASS Criteria:**
   - Evidence exists AND meets defined threshold
   - No concerns flagged in evidence
   - Example: Response time is 350ms (threshold: 500ms) → PASS

   **CONCERNS Criteria:**
   - Threshold is UNKNOWN (not defined)
   - Evidence is MISSING or INCOMPLETE
   - Evidence is close to threshold (within 10%)
   - Evidence shows intermittent issues
   - Example: Response time is 480ms (threshold: 500ms, 96% of threshold) → CONCERNS

   **FAIL Criteria:**
   - Evidence exists BUT does not meet threshold
   - Critical evidence is MISSING
   - Evidence shows consistent failures
   - Example: Response time is 750ms (threshold: 500ms) → FAIL

2. Document findings for each NFR:
   - Status (PASS/CONCERNS/FAIL)
   - Evidence source (file path, test name, metric name)
   - Actual value vs threshold
   - Justification for status classification

3. Classify severity based on category:
   - **CRITICAL**: Security failures, reliability failures (affect users immediately)
   - **HIGH**: Performance failures, maintainability failures (affect users soon)
   - **MEDIUM**: Concerns without failures (may affect users eventually)
   - **LOW**: Missing evidence for non-critical NFRs

**Output:** Complete NFR assessment with deterministic status classifications

---

### Step 5: Identify Quick Wins and Recommended Actions

**Actions:**

1. For each NFR with CONCERNS or FAIL status, identify quick wins:
   - Low-effort, high-impact improvements
   - Configuration changes (no code changes needed)
   - Optimization opportunities (caching, indexing, compression)
   - Monitoring additions (detect issues before they become failures)

2. Provide recommended actions for each issue:
   - Specific steps to remediate (not generic advice)
   - Priority (CRITICAL, HIGH, MEDIUM, LOW)
   - Estimated effort (hours, days)
   - Owner suggestion (dev, ops, security)

3. Suggest monitoring hooks for gaps:
   - Add performance monitoring (APM, synthetic monitoring)
   - Add error tracking (Sentry, Rollbar, error logs)
   - Add security monitoring (intrusion detection, audit logs)
   - Add alerting thresholds (notify before thresholds are breached)

4. Suggest fail-fast mechanisms:
   - Add circuit breakers for reliability
   - Add rate limiting for performance
   - Add validation gates for security
   - Add smoke tests for maintainability

**Output:** Actionable remediation plan with prioritized recommendations

---

### Step 6: Generate Deliverables

**Actions:**

1. Create NFR assessment markdown file:
   - Use template from `nfr-report-template.md`
   - Include executive summary (overall status, critical issues)
   - Add NFR-by-NFR assessment (status, evidence, thresholds)
   - Add findings summary (PASS count, CONCERNS count, FAIL count)
   - Add quick wins section
   - Add recommended actions section
   - Add evidence gaps checklist
   - Save to `{output_folder}/nfr-assessment.md`

2. Generate gate YAML snippet (if enabled):

   ```yaml
   nfr_assessment:
     date: '2025-10-14'
     categories:
       performance: 'PASS'
       security: 'CONCERNS'
       reliability: 'PASS'
       maintainability: 'PASS'
     overall_status: 'CONCERNS'
     critical_issues: 0
     high_priority_issues: 1
     concerns: 2
     blockers: false
   ```

3. Generate evidence checklist (if enabled):
   - List all NFRs with MISSING or INCOMPLETE evidence
   - Assign owners for evidence collection
   - Suggest evidence sources (tests, metrics, logs)
   - Set deadlines for evidence collection

4. Update story file (if enabled and requested):
   - Add "NFR Assessment" section to story markdown
   - Link to NFR assessment report
   - Include overall status and critical issues
   - Add gate status

**Output:** Complete NFR assessment documentation ready for review and CI/CD integration

---

## Non-Prescriptive Approach

**Minimal Examples:** This workflow provides principles and patterns, not rigid templates. Teams should adapt NFR categories, thresholds, and assessment criteria to their needs.

**Key Patterns to Follow:**

- Use evidence-based validation (no guessing or inference)
- Apply deterministic rules (consistent PASS/CONCERNS/FAIL classification)
- Never guess thresholds (mark as CONCERNS if unknown)
- Provide actionable recommendations (specific steps, not generic advice)
- Generate gate-ready artifacts (YAML snippets for CI/CD)

**Extend as Needed:**

- Add custom NFR categories (accessibility, internationalization, compliance)
- Integrate with external tools (New Relic, Datadog, SonarQube, JIRA)
- Add custom thresholds and rules
- Link to external assessment systems

---

## NFR Categories and Criteria

### Performance

**Criteria:**

- Response time (p50, p95, p99 percentiles)
- Throughput (requests per second, transactions per second)
- Resource usage (CPU, memory, disk, network)
- Scalability (horizontal, vertical)

**Thresholds (Default):**

- Response time p95: 500ms
- Throughput: 100 RPS
- CPU usage: < 70% average
- Memory usage: < 80% max

**Evidence Sources:**

- Load test results (JMeter, k6, Gatling)
- APM data (New Relic, Datadog, Dynatrace)
- Lighthouse reports (for web apps)
- Playwright performance traces

---

### Security

**Criteria:**

- Authentication (login security, session management)
- Authorization (access control, permissions)
- Data protection (encryption, PII handling)
- Vulnerability management (SAST, DAST, dependency scanning)
- Compliance (GDPR, HIPAA, PCI-DSS)

**Thresholds (Default):**

- Security score: >= 85/100
- Critical vulnerabilities: 0
- High vulnerabilities: < 3
- Authentication strength: MFA enabled

**Evidence Sources:**

- SAST results (SonarQube, Checkmarx, Veracode)
- DAST results (OWASP ZAP, Burp Suite)
- Dependency scanning (Snyk, Dependabot, npm audit)
- Penetration test reports
- Security audit logs

---

### Reliability

**Criteria:**

- Availability (uptime percentage)
- Error handling (graceful degradation, error recovery)
- Fault tolerance (redundancy, failover)
- Disaster recovery (backup, restore, RTO/RPO)
- Stability (CI burn-in, chaos engineering)

**Thresholds (Default):**

- Uptime: >= 99.9% (three nines)
- Error rate: < 0.1% (1 in 1000 requests)
- MTTR (Mean Time To Recovery): < 15 minutes
- CI burn-in: 100 consecutive successful runs

**Evidence Sources:**

- Uptime monitoring (Pingdom, UptimeRobot, StatusCake)
- Error logs and error rates
- CI burn-in results (see `ci-burn-in.md`)
- Chaos engineering test results (Chaos Monkey, Gremlin)
- Incident reports and postmortems

---

### Maintainability

**Criteria:**

- Code quality (complexity, duplication, code smells)
- Test coverage (unit, integration, E2E)
- Documentation (code comments, README, architecture docs)
- Technical debt (debt ratio, code churn)
- Test quality (from test-review workflow)

**Thresholds (Default):**

- Test coverage: >= 80%
- Code quality score: >= 85/100
- Technical debt ratio: < 5%
- Documentation completeness: >= 90%

**Evidence Sources:**

- Coverage reports (Istanbul, NYC, c8, JaCoCo)
- Static analysis (ESLint, SonarQube, CodeClimate)
- Documentation audit (manual or automated)
- Test review report (from test-review workflow)
- Git metrics (code churn, commit frequency)

---

## Deterministic Assessment Rules

### PASS Rules

- Evidence exists
- Evidence meets or exceeds threshold
- No concerns flagged
- Quality is acceptable

**Example:**

```markdown
NFR: Response Time p95
Threshold: 500ms
Evidence: Load test result shows 350ms p95
Status: PASS ✅
```

---

### CONCERNS Rules

- Threshold is UNKNOWN
- Evidence is MISSING or INCOMPLETE
- Evidence is close to threshold (within 10%)
- Evidence shows intermittent issues
- Quality is marginal

**Example:**

```markdown
NFR: Response Time p95
Threshold: 500ms
Evidence: Load test result shows 480ms p95 (96% of threshold)
Status: CONCERNS ⚠️
Recommendation: Optimize before production - very close to threshold
```

---

### FAIL Rules

- Evidence exists BUT does not meet threshold
- Critical evidence is MISSING
- Evidence shows consistent failures
- Quality is unacceptable

**Example:**

```markdown
NFR: Response Time p95
Threshold: 500ms
Evidence: Load test result shows 750ms p95 (150% of threshold)
Status: FAIL ❌
Recommendation: BLOCKER - optimize performance before release
```

---

## Integration with BMad Artifacts

### With tech-spec.md

- Primary source for NFR requirements and thresholds
- Load performance targets, security requirements, reliability SLAs
- Use architectural decisions to understand NFR trade-offs

### With test-design.md

- Understand NFR test plan and priorities
- Reference test priorities (P0/P1/P2/P3) for severity classification
- Align assessment with planned NFR validation

### With PRD.md

- Understand product-level NFR expectations
- Verify NFRs align with user experience goals
- Check for unstated NFR requirements (implied by product goals)

---

## Quality Gates

### Release Blocker (FAIL)

- Critical NFR has FAIL status (security, reliability)
- Performance failure affects user experience severely
- Do not release until FAIL is resolved

### PR Blocker (HIGH CONCERNS)

- High-priority NFR has FAIL status
- Multiple CONCERNS exist
- Block PR merge until addressed

### Warning (CONCERNS)

- Any NFR has CONCERNS status
- Evidence is missing or incomplete
- Address before next release

### Pass (PASS)

- All NFRs have PASS status
- No blockers or concerns
- Ready for release

---

## Example NFR Assessment

````markdown
# NFR Assessment - Story 1.3

**Feature:** User Authentication
**Date:** 2025-10-14
**Overall Status:** CONCERNS ⚠️ (1 HIGH issue)

## Executive Summary

**Assessment:** 3 PASS, 1 CONCERNS, 0 FAIL
**Blockers:** None
**High Priority Issues:** 1 (Security - MFA not enforced)
**Recommendation:** Address security concern before release

## Performance Assessment

### Response Time (p95)

- **Status:** PASS ✅
- **Threshold:** 500ms
- **Actual:** 320ms (64% of threshold)
- **Evidence:** Load test results (test-results/load-2025-10-14.json)
- **Findings:** Response time well below threshold across all percentiles

### Throughput

- **Status:** PASS ✅
- **Threshold:** 100 RPS
- **Actual:** 250 RPS (250% of threshold)
- **Evidence:** Load test results (test-results/load-2025-10-14.json)
- **Findings:** System handles 2.5x target load without degradation

## Security Assessment

### Authentication Strength

- **Status:** CONCERNS ⚠️
- **Threshold:** MFA enabled for all users
- **Actual:** MFA optional (not enforced)
- **Evidence:** Security audit (security-audit-2025-10-14.md)
- **Findings:** MFA is implemented but not enforced by default
- **Recommendation:** HIGH - Enforce MFA for all new accounts, provide migration path for existing users

### Data Protection

- **Status:** PASS ✅
- **Threshold:** PII encrypted at rest and in transit
- **Actual:** AES-256 at rest, TLS 1.3 in transit
- **Evidence:** Security scan (security-scan-2025-10-14.json)
- **Findings:** All PII properly encrypted

## Reliability Assessment

### Uptime

- **Status:** PASS ✅
- **Threshold:** 99.9% (three nines)
- **Actual:** 99.95% over 30 days
- **Evidence:** Uptime monitoring (uptime-report-2025-10-14.csv)
- **Findings:** Exceeds target with margin

### Error Rate

- **Status:** PASS ✅
- **Threshold:** < 0.1% (1 in 1000)
- **Actual:** 0.05% (1 in 2000)
- **Evidence:** Error logs (logs/errors-2025-10.log)
- **Findings:** Error rate well below threshold

## Maintainability Assessment

### Test Coverage

- **Status:** PASS ✅
- **Threshold:** >= 80%
- **Actual:** 87%
- **Evidence:** Coverage report (coverage/lcov-report/index.html)
- **Findings:** Coverage exceeds threshold with good distribution

### Code Quality

- **Status:** PASS ✅
- **Threshold:** >= 85/100
- **Actual:** 92/100
- **Evidence:** SonarQube analysis (sonarqube-report-2025-10-14.pdf)
- **Findings:** High code quality score with low technical debt

## Quick Wins

1. **Enforce MFA (Security)** - HIGH - 4 hours
   - Add configuration flag to enforce MFA for new accounts
   - No code changes needed, only config adjustment

## Recommended Actions

### Immediate (Before Release)

1. **Enforce MFA for all new accounts** - HIGH - 4 hours - Security Team
   - Add `ENFORCE_MFA=true` to production config
   - Update user onboarding flow to require MFA setup
   - Test MFA enforcement in staging environment

### Short-term (Next Sprint)

1. **Migrate existing users to MFA** - MEDIUM - 3 days - Product + Engineering
   - Design migration UX (prompt, incentives, deadline)
   - Implement migration flow with grace period
   - Communicate migration to existing users

## Evidence Gaps

- [ ] Chaos engineering test results (reliability)
  - Owner: DevOps Team
  - Deadline: 2025-10-21
  - Suggested evidence: Run chaos monkey tests in staging

- [ ] Penetration test report (security)
  - Owner: Security Team
  - Deadline: 2025-10-28
  - Suggested evidence: Schedule third-party pentest

## Gate YAML Snippet

```yaml
nfr_assessment:
  date: '2025-10-14'
  story_id: '1.3'
  categories:
    performance: 'PASS'
    security: 'CONCERNS'
    reliability: 'PASS'
    maintainability: 'PASS'
  overall_status: 'CONCERNS'
  critical_issues: 0
  high_priority_issues: 1
  medium_priority_issues: 0
  concerns: 1
  blockers: false
  recommendations:
    - 'Enforce MFA for all new accounts (HIGH - 4 hours)'
  evidence_gaps: 2
```
````

## Recommendations Summary

- **Release Blocker:** None ✅
- **High Priority:** 1 (Enforce MFA before release)
- **Medium Priority:** 1 (Migrate existing users to MFA)
- **Next Steps:** Address HIGH priority item, then proceed to gate workflow

```

---

## Validation Checklist

Before completing this workflow, verify:

- ✅ All NFR categories assessed (performance, security, reliability, maintainability, custom)
- ✅ Thresholds defined or marked as UNKNOWN
- ✅ Evidence gathered for each NFR (or marked as MISSING)
- ✅ Status classified deterministically (PASS/CONCERNS/FAIL)
- ✅ No thresholds were guessed (marked as CONCERNS if unknown)
- ✅ Quick wins identified for CONCERNS/FAIL
- ✅ Recommended actions are specific and actionable
- ✅ Evidence gaps documented with owners and deadlines
- ✅ NFR assessment report generated and saved
- ✅ Gate YAML snippet generated (if enabled)
- ✅ Evidence checklist generated (if enabled)

---

## Notes

- **Never Guess Thresholds:** If a threshold is unknown, mark as CONCERNS and recommend defining it
- **Evidence-Based:** Every assessment must be backed by evidence (tests, metrics, logs, CI results)
- **Deterministic Rules:** Use consistent PASS/CONCERNS/FAIL classification based on evidence
- **Actionable Recommendations:** Provide specific steps, not generic advice
- **Gate Integration:** Generate YAML snippets that can be consumed by CI/CD pipelines

---

## Troubleshooting

### "NFR thresholds not defined"
- Check tech-spec.md for NFR requirements
- Check PRD.md for product-level SLAs
- Check story file for feature-specific requirements
- If thresholds truly unknown, mark as CONCERNS and recommend defining them

### "No evidence found"
- Check evidence directories (test-results, metrics, logs)
- Check CI/CD pipeline for test results
- If evidence truly missing, mark NFR as "NO EVIDENCE" and recommend generating it

### "CONCERNS status but no threshold exceeded"
- CONCERNS is correct when threshold is UNKNOWN or evidence is MISSING/INCOMPLETE
- CONCERNS is also correct when evidence is close to threshold (within 10%)
- Document why CONCERNS was assigned

### "FAIL status blocks release"
- This is intentional - FAIL means critical NFR not met
- Recommend remediation actions with specific steps
- Re-run assessment after remediation

---

## Related Workflows

- **testarch-test-design** - Define NFR requirements and test plan
- **testarch-framework** - Set up performance/security testing frameworks
- **testarch-ci** - Configure CI/CD for NFR validation
- **testarch-gate** - Use NFR assessment as input for quality gate decisions
- **testarch-test-review** - Review test quality (maintainability NFR)

---

<!-- Powered by BMAD-CORE™ -->
```