tappi

Lightweight CDP browser control for AI agents. Token-efficient alternative to the built-in browser tool — 3-10x fewer tokens per interaction. Use when browsing websites, clicking elements, filling forms, uploading files, or extracting page content. Requires a Chrome/Chromium browser running with --remote-debugging-port (OpenClaw browser works out of the box). Signed-in sessions carry over automatically.

# tappi

Lightweight CLI that talks to Chrome via CDP (Chrome DevTools Protocol). Returns minimal, indexed output that agents can act on immediately — no accessibility tree parsing, no ref hunting.

## Setup

```bash
# Install dependency (one-time, in the skill scripts/ dir)
cd scripts && npm install

# Ensure browser is running with CDP enabled.
# With OpenClaw:
#   browser start profile=openclaw
# Or manually:
#   google-chrome --remote-debugging-port=18800 --user-data-dir=~/.browser-data
```

The tool connects to `http://127.0.0.1:18800` by default. Override with `CDP_URL` env var.

## Alias setup (optional)

```bash
mkdir -p ~/.local/bin
cat > ~/.local/bin/bjs << 'WRAPPER'
#!/bin/bash
exec node /path/to/scripts/browser.js "$@"
WRAPPER
chmod +x ~/.local/bin/bjs
```

## Commands

```
bjs tabs                    List open tabs
bjs open <url>              Navigate to URL
bjs tab <index>             Switch to tab
bjs newtab [url]            Open new tab
bjs close [index]           Close tab
bjs elements [selector]     List interactive elements (indexed)

Smart actions (handle focus, verification, fallbacks internally):
bjs click <index>           [Smart] Click + report what changed (navigation, checkbox, dialog)
bjs type <index> <text>     [Smart] Type into element (auto-focus, auto-verify). Good for short fields.
bjs paste <index> <text>    [Smart] Paste with auto-verify + fallback. PREFERRED for long content.
bjs paste <index> --file <path>  [Smart] Paste from file (.md, .txt)

Low-level actions (building blocks, rarely needed directly):
bjs focus <index>           [Low-level] Reclaim focus without click events. Use when smart actions report focus loss.
bjs check <index>           [Low-level] Read element value + focus state. Use to inspect without modifying.
bjs eval <js>               [Low-level] Run JavaScript. Last resort when smart actions can't solve the problem.

Read & filter:
bjs text [selector]         Extract visible page text (max 8KB)
bjs text | grep -i "pattern"  Filter text — much cheaper than full output
bjs elements | grep -i "send" Filter elements by pattern
bjs html <selector>         Get element HTML

Other:
bjs upload <path> [selector] Upload file to input (bypasses OS dialog)
bjs screenshot [path]       Save screenshot
bjs scroll <up|down|top|bottom> [px]
bjs url                     Current URL
bjs back / forward / refresh
bjs wait <ms>

Raw keyboard input (canvas apps: Google Sheets, Docs, Figma):
bjs keys <text>             Type text via CDP keyboard events (bypasses DOM)
bjs keys <text> --tab       Type text then press Tab
bjs keys <text> --enter     Type text then press Enter
bjs keys --combo cmd+b      Key combination (cmd/ctrl/shift/alt + key)
bjs keys --combo cmd+a      Select all
bjs keys "hello" --tab "world" --enter   Chain text + keys in one command
bjs keys --enter --delay 50 Flags: --enter --tab --escape --backspace --delete
                            --up --down --left --right --home --end --space

Coordinate commands (cross-origin iframes, captchas, overlays):
bjs click-xy <x> <y>       Click at page coordinates via CDP Input
bjs click-xy <x> <y> --double   Double-click at coordinates
bjs click-xy <x> <y> --right    Right-click at coordinates
bjs hover-xy <x> <y>       Hover at page coordinates
bjs drag-xy <x1> <y1> <x2> <y2>   Drag between coordinates
bjs iframe-rect <selector> Get iframe bounding box (for click-xy targeting)
```

## How it works

`elements` scans the page for all interactive elements (links, buttons, inputs, selects, etc.) — **including those inside shadow DOM** (web components). This means sites like Reddit, GitHub, and other modern SPAs that use shadow DOM are fully supported. The scan recursively pierces all shadow roots.

Returns a compact numbered list:

```
[0] (link) Hacker News → https://news.ycombinator.com/news
[1] (link) new → https://news.ycombinator.com/newest
[2] (input:text) q
[3] (button) Submit
```

Then `click 3` or `type 2 search query` — immediately actionable, no interpretation needed.

**Auto-indexing:** `click` and `type` auto-index elements if not already indexed. You can skip calling `elements` first and go straight to `click`/`type` after `open`. Call `elements` explicitly when you need to see what's on the page.

**After navigation or AJAX changes:** Elements get re-indexed automatically on next `click`/`type` if stamps are stale. For manual re-index, call `elements` again.

**Real mouse events:** `click` uses CDP `Input.dispatchMouseEvent` (mousePressed + mouseReleased) instead of JS `.click()`. This triggers React/Vue/Angular synthetic event handlers that ignore plain `.click()` calls. Works reliably on SPAs like Instagram, GitHub, LinkedIn.

## File uploads

`upload` uses CDP's `DOM.setFileInputFiles` to inject files directly into hidden `<input type="file">` elements — no OS file picker dialog. Works with Instagram, Twitter, any site with file uploads.

```bash
bjs upload ~/photos/image.jpg                    # auto-finds input[type=file]
bjs upload ~/docs/resume.pdf "input.file-drop"   # specific selector
```

## Token efficiency

| Approach | Tokens per interaction | Notes |
|----------|----------------------|-------|
| **bjs** | ~50-200 | Indexed list, 1-line responses |
| browser tool (snapshot) | ~2,000-5,000 | Full accessibility tree |
| browser tool + thinking | ~3,000-8,000 | Plus reasoning to find refs |

Over a 10-step flow: **~1,500 tokens (bjs) vs ~30,000-80,000 (browser tool)**.

## Typical flow

```bash
bjs open https://example.com       # Navigate
bjs elements                        # See what's clickable
bjs click 5                         # Click element [5]
bjs type 12 "hello world"          # Type into element [12]
bjs text                            # Read page content
bjs screenshot /tmp/result.png      # Verify visually
```

## Shadow DOM support

bjs automatically pierces shadow DOM boundaries. Sites built with web components (Reddit, GitHub, etc.) work out of the box — `elements`, `click`, `type`, and `text` all recurse into shadow roots. No special flags needed.

## Canvas-based apps (Google Sheets, Docs, Slides, Figma)

Some web apps render content on a `<canvas>` element instead of the DOM. In these apps, `bjs type` won't work for the content area because there are no DOM input elements to target. Use `bjs keys` instead — it sends raw CDP keyboard events that the canvas app picks up directly.

**Navigation elements** (menus, toolbars, name boxes, search bars) are still regular DOM — use `bjs click`/`bjs type` for those.

**Example: Google Sheets workflow**
```bash
# Navigate to a cell: click the Name Box (DOM element), type ref, press Enter
bjs click <namebox-index>             # Click name box input
bjs type <namebox-index> "A1"         # Type cell ref
bjs keys --enter                      # Press Enter to navigate

# Type a row using Tab to move between columns
bjs keys "Revenue" --tab "Q1" --tab "Q2" --tab "Total" --tab

# IMPORTANT: --enter does NOT reliably move to the next row in Sheets.
# Navigate to each row start via the Name Box instead:
bjs click <namebox-index> && bjs type <namebox-index> "A2" && bjs keys --enter
bjs keys "100" --tab "200" --tab "=SUM(B2:C2)" --tab

# Formatting: select a range via Name Box, then apply
bjs click <namebox-index> && bjs type <namebox-index> "A1:D1" && bjs keys --enter
bjs keys --combo cmd+b                # Bold the selection
```

**Pattern:** Name Box for row navigation + `--tab` within rows. Don't rely on `--enter` to advance rows.

## Coordinate commands (iframes, captchas, overlays)

When you can't use `click` by index — e.g. the target is inside a **cross-origin iframe** (captcha checkbox, payment form, OAuth widget) — use coordinate-based commands that dispatch real CDP Input events at the OS level. These bypass all DOM boundaries.

**Workflow for clicking inside an iframe:**
```bash
bjs iframe-rect 'iframe[title*="hCaptcha"]'    # Get bounding box
# Output: x=95 y=440 w=302 h=76 cente