nova-act-usability

AI-orchestrated usability testing using Amazon Nova Act. The agent generates personas, runs tests to collect raw data, interprets responses to determine goal achievement, and generates HTML reports. Tests real user workflows (booking, checkout, posting) with safety guardrails. Use when asked to "test website usability", "run usability test", "generate usability report", "evaluate user experience", "test checkout flow", "test booking process", or "analyze website UX".

# Nova Act Usability Testing v1.0.2

**AI-orchestrated** usability testing with digital twin personas powered by Amazon Nova Act.

## ⚠️ Prerequisites & Credentials

**This skill requires an Amazon Nova Act API key.**

| Requirement | Details |
|-------------|---------|
| **API Key** | Nova Act API key from [AWS Console](https://console.aws.amazon.com/) |
| **Config Location** | `~/.openclaw/config/nova-act.json` |
| **Format** | `{"apiKey": "your-nova-act-api-key-here"}` |
| **Dependencies** | `pip3 install nova-act pydantic playwright` |
| **Browser** | `playwright install chromium` (~300MB download) |

## 🔒 Data & Privacy Notice

**What this skill accesses:**
- **Reads:** `~/.openclaw/config/nova-act.json` (your API key)
- **Writes:** `./nova_act_logs/` (trace files with screenshots), `./test_results_adaptive.json`, `./nova_act_usability_report.html`

**What trace files contain:**
- Screenshots of every page visited
- Full page content (HTML, text)
- Browser actions and AI decisions

**Recommendations:**
- Run tests only on **non-production** or **test environments**
- Be aware traces may capture **PII or sensitive data** visible on tested pages
- Review/delete trace files after use if they contain sensitive content
- Consider running in a **sandboxed environment** (container/VM) for untrusted sites

---

## Features

**Agent-Driven Interpretation**: The script no longer interprets responses. YOU (the agent) must:
1. Run the test script → collect raw data
2. Read JSON → interpret each `raw_response` 
3. Set `goal_achieved` and `overall_success`
4. Generate the report

No hardcoded regex. No extra API calls. The agent doing the work is already running.

## Quick Start (For AI Agents)

**When a user asks to test a website, YOU (the AI agent) must complete ALL 4 phases:**

| Phase | What Happens | Who Does It |
|-------|--------------|-------------|
| 1. Setup | Generate personas, run test script | Agent + Script |
| 2. Collect | Script captures raw Nova Act responses | Script |
| 3. Interpret | Read JSON, determine goal_achieved for each step | **Agent** |
| 4. Report | Generate HTML report with interpreted results | Agent |

**⚠️ The script does NOT interpret responses or generate the final report. You must do phases 3-4.**

### 🎯 Recommended: AI Agent Generates Personas

**You're already an AI (Claude) - use your intelligence to generate contextual personas!**

```python
import subprocess
import os
import sys
import json
import tempfile

# Step 1: Check dependencies
try:
    import nova_act
    print("✅ Dependencies ready")
except ImportError:
    print("📦 Dependencies not installed. Please run:")
    print("   pip3 install nova-act pydantic playwright")
    print("   playwright install chromium")
    sys.exit(1)

# Step 2: Verify Nova Act API key
config_file = os.path.expanduser("~/.openclaw/config/nova-act.json")
with open(config_file, 'r') as f:
    config = json.load(f)
    if config.get('apiKey') == 'your-nova-act-api-key-here':
        print(f"⚠️  Please add your Nova Act API key to {config_file}")
        sys.exit(1)

# Step 3: YOU (the AI agent) generate personas
# Example for https://www.pgatour.com/ (golf tournament site)
website_url = "https://www.pgatour.com/"

personas = [
    {
        "name": "Marcus Chen",
        "archetype": "tournament_follower",
        "age": 42,
        "tech_proficiency": "high",
        "description": "Avid golf fan who follows multiple tours and tracks player stats",
        "goals": [
            "Check current tournament leaderboard",
            "View recent tournament results",
            "Track favorite player performance"
        ]
    },
    {
        "name": "Dorothy Williams",
        "archetype": "casual_viewer",
        "age": 68,
        "tech_proficiency": "low",
        "description": "Occasional golf viewer who watches major tournaments",
        "goals": [
            "Find when the next tournament is",
            "See who won recently",
            "Understand how to watch online"
        ]
    }
]

# Step 4: Save personas and run test
with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
    json.dump(personas, f, indent=2)
    personas_file = f.name

skill_dir = os.path.expanduser("~/.openclaw/skills/nova-act-usability")
test_script = os.path.join(skill_dir, "scripts", "run_adaptive_test.py")

# Run with AI-generated personas
subprocess.run([sys.executable, test_script, website_url, personas_file])

# Clean up temp file
os.unlink(personas_file)
```

**Persona Template:**
```json
{
  "name": "FirstName LastName",
  "archetype": "descriptive_identifier",
  "age": 30,
  "tech_proficiency": "low|medium|high",
  "description": "One sentence about who they are",
  "goals": [
    "First goal relevant to this website",
    "Second goal relevant to this website",
    "Third goal relevant to this website"
  ]
}
```

### 📝 Alternative: Simple Custom Persona

If user specifies a persona description, pass it as a string:

```python
# User: "Test PGA Tour site as a golf enthusiast"
website_url = "https://www.pgatour.com/"
user_persona = "golf enthusiast who follows tournaments closely"

subprocess.run([sys.executable, test_script, website_url, user_persona])
# Script will parse this and create personas automatically
```

### ⚠️ Fallback: Auto-Generation (Not Recommended)

Let the script guess personas based on basic category keywords:

```python
# Generic, less contextual personas
subprocess.run([sys.executable, test_script, website_url])
```

### Why YOU Should Generate Personas

**✅ Advantages:**
- **Better context:** You have full conversation history and domain knowledge
- **Smarter inference:** You can analyze the URL, industry, and user intent
- **No duplicate API calls:** You're already Claude - don't call yourself again!
- **User preferences:** You can adapt based on stated preferences
- **Clarifying questions:** You can ask the user about target demographics

**❌ What to avoid:**
- Don't let Python script make its own Claude API call (wasteful)
- Don't rely on generic fallback personas (less accurate)
- Don't skip persona generation (hurts test quality)

### 💡 Tips for Persona Generation

**Analyze the website:**
- **URL domain:** `.gov` → citizens, `.edu` → students/faculty
- **Keywords:** "shop" → shoppers, "book" → travelers, "play" → gamers
- **Industry:** Golf → fans/players, Banking → customers/businesses

**Create diverse personas:**
- Mix experience levels (beginner, intermediate, expert)
- Mix tech proficiency (low, medium, high)  
- Mix age ranges (young, middle-aged, senior)
- Mix motivations (casual, professional, enthusiastic)

**Generate realistic goals:**
- Specific to the website's purpose
- Actionable and measurable
- Match the persona's characteristics

**Examples by industry:**
- **E-commerce:** bargain_hunter, comparison_shopper, impulse_buyer
- **News:** daily_reader, topic_follower, casual_browser
- **Sports:** die_hard_fan, casual_viewer, stats_tracker
- **Travel:** business_traveler, vacation_planner, deal_seeker
- **SaaS:** power_user, evaluator, beginner
## User Invocation

Users can trigger this skill by saying:
- "Test the usability of [website URL]"
- "Run a usability test on [website URL]"
- "Generate a usability report for [website URL]"
- "Evaluate the UX of [website URL]"
- "Analyze [website URL] for usability issues"
- **NEW:** "Test the booking flow on [website]"
- **NEW:** "Test the checkout process on [e-commerce site]"
- **NEW:** "Test posting workflow on [social media site]"

**The AI will automatically:**
1. Load the Nova Act cookbook for guidance
2. Analyze the page to understand it
3. Detect if it's a workflow-based site (booking, e-commerce, social, etc.)
4. **Generate contextual personas:**
   - If custom persona specified → Create persona matching that description
   - If no custom persona → Use Claude AI to infer the 3 most plausible real-world user types
   - Fallback to category-based personas if AI unavailable