Automated Testing Standards

Playwright · TypeScript · BDD/Gherkin

Version: 1.0
Audience: Development Team · QA / Tester Team
Stack: Playwright, TypeScript, Cucumber (BDD), GitHub Actions

Table of Contents

1. Overview
2. Roles & Responsibilities
3. Project Structure
4. Standards for Developers (Automation Lead)
5. Standards for QA / Testers
6. Shared Standards (Everyone)
7. Test Coverage Policy
8. CI/CD Integration Standards
9. Reporting Standards
10. Flaky Test Policy
11. Definition of Done (Testing)
12. Onboarding Checklist

1. Overview

This document defines the automated testing standards for all development and QA activities. It establishes clear ownership, responsibilities, and conventions so every team member — regardless of role — understands what they are expected to produce and maintain.

Guiding Principles

• Tests are first-class code. They live in version control, go through code review, and are maintained like production code.
• QA owns scenarios. Developers own infrastructure. QA testers focus on defining and writing business-readable test cases. Developers build and maintain the technical foundation.
• Fail fast, fail visibly. Every pull request must pass all tests before it can be merged. Failures are surfaced immediately to the responsible team member.
• Stability over quantity. A small suite of reliable tests is more valuable than a large suite of flaky ones.

2. Roles & Responsibilities

2.1 Automation Lead (Senior Developer)

The Automation Lead is responsible for the technical foundation of the test suite. QA testers should never be blocked by infrastructure concerns.

2.2 QA / Tester

QA testers focus exclusively on business scenarios and UI coverage. They do not modify Page Objects, config files, or step definition implementations.

2.3 All Team Members

3. Project Structure

/
├── src/
│   ├── pages/                  # Page Object Model classes (Automation Lead only)
│   │   ├── LoginPage.ts
│   │   ├── CheckoutPage.ts
│   │   └── ...
│   ├── step-definitions/       # Gherkin step implementations (Automation Lead only)
│   │   ├── auth.steps.ts
│   │   ├── checkout.steps.ts
│   │   └── ...
│   ├── fixtures/               # Auth state, shared setup (Automation Lead only)
│   │   └── auth.fixture.ts
│   └── helpers/                # Reusable utilities (Automation Lead only)
│       ├── testData.ts
│       └── dateUtils.ts
│
├── features/                   # Gherkin feature files (QA / Tester writes here)
│   ├── auth/
│   │   └── login.feature
│   ├── checkout/
│   │   └── checkout.feature
│   └── ...
│
├── test-data/                  # Test data JSON files (QA / Tester writes here)
│   ├── users.json
│   ├── products.json
│   └── ...
│
├── .github/
│   └── workflows/
│       └── playwright.yml      # CI pipeline (Automation Lead only)
│
├── playwright.config.ts        # Automation Lead only
├── cucumber.config.ts          # Automation Lead only
└── package.json

Rule: QA testers write files only inside /features/ and /test-data/. All other directories are owned by the Automation Lead.

4. Standards for Developers (Automation Lead)

4.1 Page Object Model (POM) Standards

Every page or major UI component must have a corresponding Page Object class. Page Objects abstract all selector logic from test scenarios.

Rules:

• One Page Object class per page or major component
• All locators must be defined as class properties — never inline inside methods
• Use Playwright’s semantic locators in this priority order:
  1. getByRole() — preferred
  2. getByLabel()
  3. getByTestId() — requires data-testid attribute on the element
  4. getByText() — for unique text only
  5. CSS selectors — last resort only, must be documented with a reason

Example: Page Object class

// src/pages/CheckoutPage.ts
import { Page, Locator, expect } from '@playwright/test';

export class CheckoutPage {
  // Locators defined as properties
  readonly shippingFirstName: Locator;
  readonly shippingLastName: Locator;
  readonly shippingAddress: Locator;
  readonly cardNumber: Locator;
  readonly placeOrderButton: Locator;
  readonly confirmationHeading: Locator;
  readonly orderNumberText: Locator;

  constructor(private page: Page) {
    this.shippingFirstName  = page.getByLabel('First name');
    this.shippingLastName   = page.getByLabel('Last name');
    this.shippingAddress    = page.getByLabel('Street address');
    this.cardNumber         = page.getByLabel('Card number');
    this.placeOrderButton   = page.getByRole('button', { name: 'Place order' });
    this.confirmationHeading = page.getByRole('heading', { name: 'Order confirmed' });
    this.orderNumberText    = page.getByTestId('order-number');
  }

  async fillShipping(data: { firstName: string; lastName: string; address: string }) {
    await this.shippingFirstName.fill(data.firstName);
    await this.shippingLastName.fill(data.lastName);
    await this.shippingAddress.fill(data.address);
  }

  async payWithCard(cardNumber: string) {
    await this.cardNumber.fill(cardNumber);
    await this.placeOrderButton.click();
  }

  async assertOrderConfirmed(expectedOrderPattern = /ORD-\d+/) {
    await expect(this.confirmationHeading).toBeVisible();
    await expect(this.orderNumberText).toHaveText(expectedOrderPattern);
  }
}

4.2 Step Definition Standards

Step definitions are the bridge between Gherkin scenarios (written by QA) and Page Object methods (built by dev). They must stay thin — no UI logic inside step definitions.

Rules:

• Step definitions call Page Object methods only — no raw Playwright API calls
• Keep steps reusable across multiple feature files
• Use a World context object to share page and page object instances across steps
• Group step files by domain (auth, checkout, profile, etc.)

Example: Step definition

// src/step-definitions/checkout.steps.ts
import { Given, When, Then } from '@cucumber/cucumber';
import { CheckoutPage } from '../pages/CheckoutPage';
import { testUsers, testCards } from '../helpers/testData';
import { World } from '../fixtures/world';

When('I proceed to checkout', async function (this: World) {
  this.checkoutPage = new CheckoutPage(this.page);
  await this.checkoutPage.goTo();
});

When('I enter valid shipping details', async function (this: World) {
  await this.checkoutPage.fillShipping(testUsers.standard.shipping);
});

When('I pay with a valid card', async function (this: World) {
  await this.checkoutPage.payWithCard(testCards.visa.number);
});

Then('I should see an order confirmation', async function (this: World) {
  await this.checkoutPage.assertOrderConfirmed();
});

4.3 Auth Fixture Standards

Never write login steps inside test scenarios. Auth must be handled via Playwright’s storageState so tests start already authenticated.

// src/fixtures/auth.fixture.ts
import { test as base, Page } from '@playwright/test';

export const test = base.extend({
  authenticatedPage: async ({ browser }, use) => {
    const context = await browser.newContext({
      storageState: 'test-data/auth-state/standard-user.json'
    });
    const page = await context.newPage();
    await use(page);
    await context.close();
  }
});

How to generate auth state: Run npx playwright codegen --save-storage=test-data/auth-state/standard-user.json and log in manually. Commit the resulting JSON file.

4.4 data-testid Convention

Developers must add data-testid attributes to all interactive and key UI elements during feature development. This prevents QA tests from depending on fragile CSS classes or text content.

Naming convention: data-testid="[component]-[element]-[modifier]"

<!-- Good -->
<button data-testid="checkout-place-order-btn">Place order</button>
<p data-testid="checkout-order-number">ORD-12345</p>
<input data-testid="profile-email-input" />

<!-- Bad -->
<button class="btn-primary">Place order</button>

Rule: A feature is not considered dev-complete until all key interactive elements have data-testid attributes.

5. Standards for QA / Testers

5.1 Writing Feature Files (Gherkin)

QA testers write all .feature files. These files are the source of truth for what behavior is covered by automated tests.

Rules:

• Write in plain English from the user’s perspective
• One .feature file per business feature or user flow
• Each scenario must be independently executable (no shared state between scenarios)
• Use Background for steps that are identical across all scenarios in a file
• Do not embed technical details (URLs, CSS selectors, API responses) in scenarios

File naming: kebab-case.feature — e.g., user-login.feature, product-checkout.feature

Example: Feature file

# features/checkout/product-checkout.feature

Feature: Product checkout
  As a registered customer
  I want to complete a purchase
  So that I receive my order

  Background:
    Given I am logged in as a standard user
    And I have "Blue Classic T-Shirt" in my cart

  Scenario: Successful checkout with valid card
    When I proceed to checkout
    And I enter valid shipping details
    And I pay with a valid card
    Then I should see an order confirmation
    And I should receive a confirmation email

  Scenario: Checkout fails with expired card
    When I proceed to checkout
    And I enter valid shipping details
    And I pay with an expired card
    Then I should see a payment error message
    And my order should not be placed

  Scenario: Checkout requires all shipping fields
    When I proceed to checkout
    And I submit checkout without filling shipping details
    Then I should see validation errors on required fields

5.2 Scenario Writing Rules

5.3 Test Data Standards

All test data used in scenarios must be stored in /test-data/ as JSON files, not hardcoded inside feature files or step definitions.

Structure:

// test-data/users.json
{
  "standard": {
    "email": "[email protected]",
    "password": "Test@12345",
    "shipping": {
      "firstName": "Test",
      "lastName": "User",
      "address": "123 Test Street",
      "city": "Jakarta",
      "postcode": "12345"
    }
  },
  "admin": {
    "email": "[email protected]",
    "password": "Admin@12345"
  }
}

Rules:

• Never use real customer data in test files
• Never commit passwords or tokens — use environment variables for sensitive data
• Keep a separate data set per environment (staging, UAT)

5.4 Scenario Coverage Checklist

For every feature, QA must ensure these scenario types are covered before sign-off:

• Happy path — the standard successful user flow
• Negative path — what happens when inputs are invalid
• Edge cases — empty states, boundary values, maximum lengths
• UI validation — required field errors, format errors
• Business rule enforcement — e.g., out-of-stock items, permission restrictions
• Key UI elements visible — page headings, CTAs, confirmation messages

5.5 Raising a Step Definition Request

If a scenario requires a Gherkin step that doesn’t exist yet, QA must raise a step definition request to the Automation Lead — not implement it themselves.

Process:

1. Write the scenario in the .feature file with the new step
2. Open a ticket tagged [step-request] with the step wording and expected behavior
3. Automation Lead implements the step definition within 1 sprint
4. QA verifies the step behaves as expected

6. Shared Standards (Everyone)

6.1 Naming Conventions

6.2 Branch & Commit Standards

• Test code lives in the same branch as feature code — not separate branches
• Commit message prefix for test changes: test: — e.g., test: add checkout failure scenarios
• Feature files must be committed alongside the feature they test

6.3 Code Review Checklist

All test code (feature files and step definitions) must pass review before merge.

Reviewer checks:

• Scenarios are written from the user’s perspective
• No raw selectors or technical details in .feature files
• Step definitions delegate to Page Object methods only
• New Page Object locators use semantic selectors
• Test data is externalized to /test-data/
• No hardcoded credentials or environment-specific URLs
• All new UI elements have data-testid attributes

7. Test Coverage Policy

7.1 Coverage Targets

7.2 What Must Be Automated

The following must always have automated test coverage:

• User authentication (login, logout, session expiry)
• All checkout and payment flows
• User registration and profile management
• Permission and role-based access
• All form submissions (valid and invalid inputs)
• Critical notifications (confirmation emails, error messages)

7.3 What Should NOT Be Automated

• Exploratory testing — always manual
• Usability and UX judgment — always manual
• One-off or throwaway test scenarios
• Tests that require physical hardware (e.g., OTP via SMS — mock these)

8. CI/CD Integration Standards

8.1 Pipeline Rules

• All tests run automatically on every pull request
• PRs cannot be merged if any test fails — this is enforced at the branch protection level
• Tests run across Chromium, Firefox, and WebKit on nightly builds
• PRs run Chromium only (faster feedback)
• Parallel sharding is used to keep PR test time under 5 minutes

8.2 GitHub Actions Workflow

# .github/workflows/playwright.yml
name: Playwright Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3]

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shard }}/3
        env:
          BASE_URL: ${{ secrets.STAGING_URL }}
          TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}

      - name: Upload HTML report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report-shard-${{ matrix.shard }}
          path: playwright-report/
          retention-days: 14

8.3 Environment Variables

Never hardcode environment-specific values. Use environment variables for all of the following:

9. Reporting Standards

9.1 Reports Generated Per Run

9.2 Report Retention Policy

• HTML reports are retained for 14 days per CI run
• Allure historical data is retained indefinitely
• Videos and traces are captured on failure only (to manage storage)

9.3 Reviewing Failures

When a test fails in CI, the assigned developer or QA tester must:

1. Open the HTML report artifact from the CI run
2. Review the trace and screenshot at the point of failure
3. Determine: is this a product bug or a broken test?
4. If a product bug: raise a defect ticket and link it to the failing test
5. If a broken test: fix within the same PR or raise a [test-fix] ticket for the next sprint

10. Flaky Test Policy

A flaky test is any test that passes and fails inconsistently without code changes.

10.1 Detection

A test is considered flaky if it fails on 2 or more consecutive runs without any code change to the feature under test.

10.2 Response Protocol

10.3 Quarantine Tag

// Quarantined test — do not remove without fixing
test.fixme('user can complete checkout with valid card', async ({ page }) => {
  // FLAKY: Intermittently fails on the payment step — see ticket QA-142
  // Quarantined by: @john.doe on 2025-01-15
});

10.4 Zero Tolerance Rule

Flaky tests must never remain in the main suite untagged. A flaky suite erodes the team’s trust in CI and undermines the value of automated testing.

11. Definition of Done (Testing)

A feature is only considered done when ALL of the following are true:

• Feature file written by QA covering happy path, negative path, and edge cases
• All new UI elements have data-testid attributes added by the developer
• Step definitions implemented and reviewed by Automation Lead
• All tests pass locally and in CI
• Visual regression snapshots updated if UI has changed
• Test data added to /test-data/ for any new data requirements
• No new flaky tests introduced
• PR approved by both a developer (code review) and QA (scenario review)

12. Onboarding Checklist

For New Developers (Automation Lead)

• Read this document in full
• Clone the repo and run npm ci && npx playwright install
• Run the full test suite locally with npx playwright test
• Review the existing Page Object classes in /src/pages/
• Review the CI pipeline in .github/workflows/playwright.yml
• Attend a walkthrough session with the QA Lead
• Complete your first task: implement 1 step definition from the backlog

For New QA / Testers

• Read this document in full, focusing on sections 5 and 6
• Clone the repo and explore the /features/ directory
• Read 3 existing .feature files to understand the writing style
• Run the test suite locally: npx playwright test (ask the Automation Lead for setup help)
• Attend a Gherkin writing workshop with the QA Lead
• Complete your first task: write a .feature file for an existing untested flow
• Have your first feature file reviewed by the Automation Lead before merging

Appendix A — Quick Reference Card

For QA / Testers

✅ I write:     /features/**/*.feature
✅ I write:     /test-data/**/*.json
❌ I don't touch: /src/pages/
❌ I don't touch: /src/step-definitions/
❌ I don't touch: playwright.config.ts
❌ I don't touch: .github/workflows/

For Automation Lead

✅ I own:  /src/pages/        — Page Object classes
✅ I own:  /src/step-definitions/ — Gherkin step implementations
✅ I own:  /src/fixtures/     — Auth and shared setup
✅ I own:  playwright.config.ts
✅ I own:  .github/workflows/
✅ I review: all /features/ PRs from QA

Locator Priority (Developer)

1. getByRole()      ← always try this first
2. getByLabel()     ← for form inputs
3. getByTestId()    ← requires data-testid on element
4. getByText()      ← for unique visible text only
5. CSS selector     ← last resort, must be documented

Scenario Structure (QA)

Scenario: [describe the behavior, not the steps]
  Given [the starting condition]
  When  [the user action]
  Then  [the expected outcome]
  And   [additional outcome if needed]

This document is maintained by the Automation Lead. Proposed changes must be submitted as a PR and approved by both the QA Lead and the Engineering Manager.