ds160-autofill

使用用于元素定位的 CDP、用于用户信息的 CSV 数据源、用于复杂案例（验证码、缺失元素）的 LLM 协助以及用于恢复功能的会话持久性，自动填写美国非移民签证 DS-160 表格。当用户需要执行以下操作时使用：(1) 自动填写 DS-160 签证申请表，(2) 恢复填写中断的 DS-160 申请表，(3) 在法学硕士的协助下处理验证码和复杂的表单元素。支持中文输入并自动翻译成英文。

# DS-160 Auto-fill Skill

This skill automates the filling of US nonimmigrant visa DS-160 forms by combining:
- CDP (Chrome DevTools Protocol) for fast element location
- CSV data source for user information
- LLM assistance for complex cases (captcha, missing elements)
- Session persistence for resume capability
- Chinese to English translation for user input

## Quick Start

### Starting a New Application

1. **Provide CSV data**: User must provide a CSV file with their personal information (use `ds160-user-info.csv` as template)

2. **Open DS-160 website**: Use `browser` tool to open `https://ceac.state.gov/genniv/` with profile `openclaw`

3. **Initialize session**: Load and execute `scripts/ds160-filler.js` to parse mappings and CSV

4. **Start filling**: Begin with the home page, handle captcha with LLM, then proceed page by page

### Resuming an Application

1. **Load existing session**: Check `ds160/ds160-session.json` for saved Application ID

2. **Open application**: Use the saved Application ID to resume on DS-160 website

3. **Continue filling**: Load CSV from saved location and continue from last page

## Core Workflow

### Step 1: Prepare Data

**Always check if session exists first:**
```javascript
// Read ds160/ds160-session.json
// If exists → Resume mode
// If not → New application mode
```

**New application mode:**
- Ask user for CSV file path or content
- Save CSV to `ds160/ds160-user-info.csv` using `write` tool
- Initialize session data in `ds160/ds160-session.json`:
  ```json
  {
    "applicationId": null,
    "securityQuestion": null,
    "securityAnswer": null,
    "currentPageIndex": 0,
    "completedPages": [],
    "startDate": "2026-02-06T21:00:00Z"
  }
  ```

**Resume mode:**
- Load CSV from `ds160/ds160-user-info.csv`
- Load session from `ds160/ds160-session.json`
- Report current progress to user

### Step 2: Initialize Browser

```javascript
// Open browser with openclaw profile
browser_start: { profile: "openclaw", targetUrl: "https://ceac.state.gov/genniv/" }

// For resume mode, add Application ID to URL
// Example: https://ceac.state.gov/GenNIV/Common/ConfirmApplicationID.aspx
```

### Step 3: Fill Current Page

**For each page in sequence:**

1. **Snapshot page** to understand current state:
   ```javascript
   browser_snapshot: { refs: "role", profile: "openclaw" }
   ```

2. **Load mappings** from `references/ds160-elements.yaml`

3. **Execute fill logic** using `browser act` with evaluate:
   ```javascript
   // Load ds160-filler.js
   // Call fillPage(page, currentUrl, userData, yamlData)
   ```

4. **Handle results**:
   - **Success**: Continue to next page
   - **Needs LLM assistance**: Call LLM and retry
   - **Needs user input**: Save progress and pause

### Step 4: Handle Special Cases

**Captcha:**
- Take screenshot of captcha area
- Use `image` tool to analyze captcha
- Fill captcha code via browser evaluate

**Missing element:**
- Call LLM with page snapshot
- Ask LLM to locate element and provide alternative selector
- Retry fill operation

**Translation needed:**
- Call LLM with field context and Chinese value
- Ask LLM to translate to appropriate English value
- Update CSV and retry fill operation

**Missing user data:**
- Identify which field is missing
- Report to user with field name and description
- **Important**: Save current page using browser evaluate before pausing
- Wait for user to provide data

### Step 5: Save Progress & Continue

**After completing a page:**

1. **Update session data**:
   ```json
   {
     "applicationId": "AA00FBLCQP",
     "securityQuestion": "What is the given name of your mother's mother?",
     "securityAnswer": "LiMei",
     "currentPageIndex": 5,
     "completedPages": ["home", "security_question", "personal_1", "personal_2"],
     "startDate": "2026-02-06T21:00:00Z"
   }
   ```

2. **Save to file**: `write(ds160/ds160-session.json, JSON.stringify(sessionData))`

3. **Report progress to user**:
   - Current page completed
   - Next page name
   - Application ID
   - Security question/answer (first time only)
   - Overall progress (X/Y pages)

4. **Click "Continue" button** to proceed to next page

## Data Structures

### CSV Format

The CSV must have these columns:
1. `页面` - Page name
2. `字段名称` - Field identifier (matches YAML `name` field)
3. `英文说明` - English description
4. `中文说明` - Chinese description
5. `必填` - Required? (是/否)
6. `示例值` - Example value
7. `用户填写` - **User-provided value** (this is what gets filled)

**Important**: Users can provide values in Chinese. The script includes a built-in translation dictionary for common fields (gender, marital status, countries, etc.). For fields not in the dictionary, LLM will be called for translation automatically.

### YAML Structure

Each page in `ds160-elements.yaml` contains:
- `page_id`: Unique page identifier
- `page_name`: Human-readable page name
- `url`: Page URL pattern
- `elements`: Array of form elements with:
  - `id`: Element ID
  - `name`: Field identifier (matches CSV)
  - `type`: Element type (text, select, radio, checkbox, button)
  - `label`: English label
  - `label_cn`: Chinese label
  - `required`: Is required?
  - `options`: Available options (for select)
  - `group`: Radio group name

## Error Handling

### Element Not Found

**Symptoms**: `fillPage` returns `needsLLM: true`

**Action**:
1. Take page snapshot
2. Call LLM with:
   - Current URL
   - Element information (id, name, type)
   - Page HTML snippet
   - Ask LLM to analyze and provide alternative selector
3. Retry with LLM-suggested selector

### Translation Needed

**Symptoms**: `fillPage` returns `needsTranslation: true`

**Action**:
1. Call LLM with translation context:
   - Field name and description (EN/CN)
   - Field type and available options
   - Original Chinese value
   - Ask LLM to translate to appropriate English value for the field
2. Update CSV with translated value
3. Retry fill operation with translated value

**Example LLM prompt for translation**:
```
I need to translate this Chinese value for a DS-160 form field:

Field: ${elementName}
Type: ${elementType}
English label: ${label}
Chinese label: ${label_cn}
Chinese value: "${originalValue}"

Available options (if select): ${options}

Please provide the correct English value for this field. Consider:
- The field type and context
- Available options (if applicable)
- Standard DS-160 terminology

Return only the English value, no explanation.
```

### Data Missing

**Symptoms**: `fillPage` returns `needsUserInput: true`

**Action**:
1. **CRITICAL**: Save current page before pausing:
   ```javascript
   // Find and click "Save" button
   browser_act: {
     request: { kind: "click", ref: "save_button" }
   }
   ```
2. Report missing field to user:
   - Field name (English and Chinese)
   - Description
   - Whether it's required
3. Update CSV with new data using `write` tool
4. Resume filling

### Captcha

**Symptoms**: Captcha image detected on page

**Action**:
1. Take screenshot of captcha area
2. Use `image` tool:
   ```javascript
   image: {
     image: "/path/to/captcha.png",
     prompt: "What is the code shown in this captcha? Return only the code characters."
   }
   ```
3. Fill captcha field via browser evaluate
4. Submit form

## Page Sequence

The pages are filled in this order:

1. **home** - Start application, select location, handle captcha
2. **security_question** - Confirm Application ID, set security question
3. **personal_1** - Personal information (name, gender, DOB)
4. **personal_2** - Nationality, other nationalities, IDs
5. **travel** - Travel purpose, dates, who is paying
6. **travel_companions** - People traveling with you
7. **previous_us_travel** - Previous US visits
8. **address_phone** - Current address and phone
9. **passport** - Passport information
10. **us_contact** - US point of contact
11. **family_relatives** - Family information
12. **work_education** - Work and education history
13. **security_background** - Security questions

#