qa-patrol

使用本地浏览器自动化对 Web 应用程序进行自动化 QA 测试。完全在您的机器上运行 - 没有数据离开，没有云服务，没有外部服务器。 1 级（冒烟测试）仅需要一个 URL。 2 级（身份验证/付款测试）使用可选的环境变量作为测试凭据。 3级（静态分析， DB 检查）可以选择读取本地文件并连接到用户提供的数据库。 支持 Supabase/Firebase 身份验证、Stripe 支付、React Native Web、Next.js 和 SPA。

## 概述（中文）

使用本地浏览器自动化对 Web 应用程序进行自动化 QA 测试。完全在您的机器上运行 -
没有数据离开，没有云服务，没有外部服务器。 1 级（冒烟测试）仅需要一个 URL。
2 级（身份验证/付款测试）使用可选的环境变量作为测试凭据。 3级（静态分析，
DB 检查）可以选择读取本地文件并连接到用户提供的数据库。
支持 Supabase/Firebase 身份验证、Stripe 支付、React Native Web、Next.js 和 SPA。

## 原文

# QA Patrol

Automated QA testing skill for web applications. Catches bugs that unit tests miss: cross-platform issues, auth state problems, data integrity failures, and integration breakages.

## Security & Privacy

**All tests run locally on your machine. Nothing is sent to external servers. The browser automation uses OpenClaw's built-in browser control — no cloud services involved.**

### Permissions by Level

| Level | What it does | Permissions needed | Env vars needed |
|-------|-------------|-------------------|-----------------|
| **1 — Smoke** | Loads pages, checks for errors | `browser` only | `APP_URL` (or pass `--url`) |
| **2 — Auth/Payments** | Tests sign-in, checkout flows | `browser` only | Test account credentials (see below) |
| **3 — Static Analysis** | Scans local source code for bug patterns | `browser` + `read` | None (uses local `repo_path`) |
| **3 — DB Integrity** | Compares DB values to UI display | `browser` | `DATABASE_URL` |

**The `read` permission is ONLY needed for Level 3 static analysis.** Level 1 and Level 2 tests use browser automation exclusively. If you only run Level 1/2 tests, the skill never accesses local files.

### Environment Variables (all optional)

| Variable | Required | Used by | Purpose |
|----------|----------|---------|---------|
| `APP_URL` | No | Level 1+ | Target app URL (can also use `--url` flag) |
| `ADMIN_EMAIL` | No | Level 2 | Admin test account email |
| `ADMIN_PASSWORD` | No | Level 2 | Admin test account password |
| `FREE_EMAIL` | No | Level 2 | Free-tier test account email |
| `FREE_PASSWORD` | No | Level 2 | Free-tier test account password |
| `PRO_EMAIL` | No | Level 2 | Pro test account email |
| `PRO_PASSWORD` | No | Level 2 | Pro test account password |
| `DATABASE_URL` | No | Level 3 | DB connection for data integrity checks |

**⚠️ Use test credentials only — never supply production passwords or production DATABASE_URL.**

### Secrets Handling
- **NEVER hardcode secrets** in test plans — always use environment variable interpolation: `${env.ADMIN_PASSWORD}`
- Credentials are read from your local environment at runtime
- Test plans in this skill's examples use only `${env.VAR}` placeholders
- The skill does not persist, log, or transmit credentials

### Security Pattern Detection (Not Exploitation)
The `references/bug-patterns.md` file contains regex patterns for **detecting exposed secrets** in codebases (e.g., `sk_live_`, `api_key=`). These are **detection patterns** used to help developers find and fix security issues — they are NOT exploitation tools. This is standard practice in security linters like ESLint, Semgrep, and GitHub's secret scanning.

### No Install Scripts, No Code Files
This is an **instruction-only skill** — it contains no executable code, no install scripts, and no third-party dependencies. The entire security surface is the SKILL.md instructions and OpenClaw's built-in browser/read capabilities.

## Quick Start

### Level 1: Zero-Config Smoke Test
```bash
# Just provide a URL
qa-patrol https://example.com
```

### Level 2: With Auth/Payments
```bash
# Use a test plan template
qa-patrol --plan auth-supabase.yaml --url https://example.com
```

### Level 3: Full Config
```bash
# Custom test plan with data integrity checks
qa-patrol --plan my-app.yaml
```

## Workflow

### 1. Load or Generate Test Plan

If a YAML test plan is provided, load it. Otherwise, generate a basic plan:

```yaml
app:
  url: <provided URL>
  name: <extracted from page title>

tests:
  smoke:
    - name: Homepage loads
      navigate: /
      assert:
        - element_exists: main
        - no_console_errors: true
```

See `assets/templates/` for test plan templates:
- `basic.yaml` - Zero-config smoke test
- `auth-supabase.yaml` - Supabase auth flows
- `payments-stripe.yaml` - Stripe checkout testing
- `full-saas.yaml` - Complete SaaS test plan

### 2. Execute Tests

Run tests in order: smoke → auth → payments → data_integrity → static_analysis.

For each test:
1. Navigate to the target URL
2. Execute steps (click, type, wait)
3. Capture snapshot and console logs
4. Evaluate assertions
5. Record PASS/FAIL/SKIP with evidence

#### Browser Automation Patterns

```python
# Navigate and snapshot
browser(action="navigate", targetUrl="https://example.com/page")
browser(action="snapshot")

# Form interaction
browser(action="act", request={"kind": "click", "ref": "email_input"})
browser(action="act", request={"kind": "type", "ref": "email_input", "text": "user@test.com"})
browser(action="act", request={"kind": "click", "ref": "submit_button"})

# Check console for errors
browser(action="console", level="error")
```

See `references/test-patterns.md` for complete automation patterns.

### 3. Check for Known Bug Patterns

Scan codebase (if accessible) for anti-patterns:

| Pattern | What to grep | Severity |
|---------|-------------|----------|
| Alert.alert on web | `Alert.alert` without `Platform.OS` guard | High |
| Linking in Modal | `Linking.openURL` inside Modal component | High |
| Missing RLS | Supabase queries without proper auth context | High |
| Hardcoded secrets | API keys in client code | Critical |

See `references/bug-patterns.md` for the full catalog.

### 4. Data Integrity Checks (Level 3)

When `data_integrity` tests are defined:

1. Execute the DB query (requires DB access)
2. Navigate to the UI path
3. Extract the displayed value
4. Compare against query result
5. Flag mismatches with severity based on % difference

### 5. Generate Report

Output a structured report:

```markdown
# QA Report: [App Name]
**Date**: YYYY-MM-DD HH:MM
**URL**: https://example.com
**Confidence**: 87%

## Summary
| Category | Pass | Fail | Skip |
|----------|------|------|------|
| Smoke    | 5    | 0    | 0    |
| Auth     | 3    | 1    | 0    |
| Payments | 0    | 0    | 2    |

## Failures

### [FAIL] Auth: Session persistence after refresh
**Steps**: Sign in → Refresh page → Check auth state
**Expected**: User remains signed in
**Actual**: Redirected to login page
**Evidence**: [screenshot]
**Severity**: High

## Recommendations
1. Fix session persistence (likely cookie/localStorage issue)
2. Add Platform.OS guards to Alert.alert calls
```

See `references/report-format.md` for the complete template.

## Test Plan Reference

### App Configuration

```yaml
app:
  url: https://example.com      # Required: base URL
  name: My App                  # Optional: display name
  stack: expo-web               # expo-web | nextjs | spa | static
```

### Auth Configuration

```yaml
auth:
  provider: supabase            # supabase | firebase | auth0 | custom
  login_path: /auth             # Path to login page
  accounts:
    admin:
      email: admin@test.com
      password: ${ADMIN_PASSWORD}  # Use env vars for secrets
    free:
      email: free@test.com
      password: ${FREE_PASSWORD}
    guest: true                 # Test anonymous/guest mode
```

### Test Types

#### Smoke Tests
```yaml
tests:
  smoke:
    - name: Homepage loads
      navigate: /
      assert:
        - element_exists: main
        - no_console_errors: true
        - no_network_errors: true
    
    - name: Navigation works
      navigate: /
      steps:
        - click: { ref: nav_link }
        - assert: { url_contains: "/target" }
```

#### Auth Tests
```yaml
tests:
  auth:
    - name: Sign in flow
      steps:
        - navigate: /auth
        - type: { ref: email_input, text: "${auth.accounts.free.email}" }
        - type: { ref: password_input, text: "${auth.accounts.free.password}" }
        - click: { ref: sign_in_button }
        - wait: { url_contains: "/home", timeout: 5000 }
        - assert: { element_exists: "user_avatar" }
    
    - name: Sign out flow
      requires: signed_in