Automated Testing

Automated testing is a cornerstone of quality software development, especially in regulated environments where testing evidence must be preserved for audits. This section covers the testing pyramid, best practices, and compliance considerations for each testing level.

Why Automated Testing Matters

In regulated software development, testing serves dual purposes:

1. Quality assurance: Ensure the software works correctly and safely
2. Compliance evidence: Provide documented proof of verification activities

Code is incomplete without tests. Write unit tests that are small and fast. Test where it makes sense, not just to hit 100% coverage.

The Testing Pyramid

Core Testing Principles

1. Test First Mentality

Write tests before or alongside implementation, not as an afterthought:

• TDD (Test-Driven Development): Write failing tests first, then implement
• BDD (Behavior-Driven Development): Write tests as specifications
• Specification by Example: Use examples as test cases

2. Arrange-Act-Assert Pattern

Structure tests consistently:

// Arrange: Set up test data and conditions
const user = createTestUser({ role: 'admin' });
const document = createTestDocument({ status: 'draft' });

// Act: Execute the code under test
const result = await documentService.publish(document, user);

// Assert: Verify the expected outcome
expect(result.status).toBe('published');
expect(result.publishedBy).toBe(user.id);

3. F.I.R.S.T. Principles

Testing in Regulated Environments

Compliance Requirements

For FDA, HIPAA, and other regulatory frameworks:

Test Documentation

For regulated software, document:

• Test Plan: Strategy, scope, and approach
• Test Cases: Individual test specifications
• Test Results: Execution records with pass/fail status
• Traceability Matrix: Requirements linked to tests

Traceability Example

Requirement: REQ-AUTH-001 - User must authenticate before accessing PHI

Test Cases:
├── TC-AUTH-001: Valid login grants access
├── TC-AUTH-002: Invalid password denies access
├── TC-AUTH-003: Locked account denies access
└── TC-AUTH-004: Session timeout requires re-authentication

Code Implementation:
└── src/auth/AuthService.authenticate()

Test Coverage

What to Measure

Coverage Guidance

• Don't chase 100%: Focus on critical paths and risk areas
• Quality over quantity: Better to have meaningful tests than many trivial ones
• Risk-based testing: Higher coverage for safety-critical code
• Exclude boilerplate: Generated code, simple getters/setters may be excluded

Coverage Example Configuration

// jest.config.js
module.exports = {
  collectCoverageFrom: [
    'src/**/*.{js,ts}',
    '!src/**/*.d.ts',
    '!src/**/*.test.{js,ts}',
    '!src/generated/**',
  ],
  coverageThreshold: {
    global: {
      branches: 70,
      functions: 80,
      lines: 80,
      statements: 80,
    },
    './src/core/**': {
      branches: 90,
      functions: 90,
      lines: 90,
    },
  },
};

Test Organization

Directory Structure

project/
├── src/
│   ├── components/
│   │   ├── Button.tsx
│   │   └── Button.test.tsx      # Co-located unit tests
│   └── services/
│       ├── AuthService.ts
│       └── AuthService.test.ts
├── tests/
│   ├── integration/              # Integration tests
│   │   └── auth.integration.test.ts
│   ├── e2e/                      # End-to-end tests
│   │   └── login.e2e.test.ts
│   └── fixtures/                 # Test data
│       └── users.json
└── coverage/                     # Coverage reports

Naming Conventions

Continuous Integration

Test Execution in CI/CD

# GitHub Actions example
name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install dependencies
        run: npm ci

      - name: Run unit tests
        run: npm run test:unit -- --coverage

      - name: Run integration tests
        run: npm run test:integration

      - name: Upload coverage
        uses: codecov/codecov-action@v3

      - name: Check coverage thresholds
        run: npm run test:coverage-check

Test Execution Order

1. Pre-commit: Fast unit tests (lint-staged)
2. PR/Push: Full unit test suite
3. PR/Push: Integration tests
4. Nightly/Deploy: Full E2E suite
5. Weekly: Performance tests

Testing Best Practices

Do

• Write tests for all new code
• Include tests in code reviews
• Run tests locally before pushing
• Keep tests fast (unit tests < 100ms each)
• Use meaningful test descriptions
• Test edge cases and error conditions
• Mock external dependencies appropriately

Don't

• Skip tests to meet deadlines
• Write tests that depend on execution order
• Test implementation details (test behavior)
• Mock too much (test real integration when possible)
• Ignore flaky tests (fix or remove them)
• Write tests that sleep/wait arbitrarily

Testing Tools Ecosystem

JavaScript/TypeScript

Other Languages

Related Resources

• Unit Testing
• Integration Testing
• E2E Testing
• Performance Testing
• Test Automation
• Code Reviews
• Microsoft Playbook: Automated Testing

Compliance

This section fulfills ISO 13485 requirements for design verification (7.3.6), design validation (7.3.7), process validation (7.5.6), product monitoring (8.2.4), and control of records (4.2.4), and ISO 27001 requirements for security testing (A.8.29), secure development lifecycle (A.8.25), secure architecture (A.8.27), and compliance verification (A.5.36).

View full compliance matrix