pixel-lobster

像素艺术桌面龙虾与 OpenClaw TTS 语音进行口型同步。在以下情况下使用：(1) 用户想要为其 AI 代理提供视觉化身，(2) 用户想要在代理说话时显示动画的桌面覆盖层，(3) 用户要求设置或配置像素龙虾。指导用户安装、配置和启动捆绑的 Electron 应用程序。

## 概述（中文）

像素艺术桌面龙虾与 OpenClaw TTS 语音进行口型同步。在以下情况下使用：(1) 用户想要为其 AI 代理提供视觉化身，(2) 用户想要在代理说话时显示动画的桌面覆盖层，(3) 用户要求设置或配置像素龙虾。指导用户安装、配置和启动捆绑的 Electron 应用程序。

## 原文

# Pixel Lobster

A transparent desktop overlay featuring a pixel art lobster that animates when your OpenClaw agent speaks. Powered by envelope data from your local TTS server — the lobster's mouth only moves during AI speech, not music or system audio.

The app is fully bundled inside this skill — no external repository clone required.

## Requirements

- Node.js 18+ with `npx` available
- A running TTS server exposing `GET /audio/envelope` (XTTS on port 8787, or any OpenAI-compatible TTS via the OpenClaw TTS proxy)
- Windows or Linux desktop (macOS not supported)

## Install

The app is included in this skill at `<skill_dir>/app/`. Install dependencies once:

```bash
cd <skill_dir>/app
npm install
```

## Configure

Edit `<skill_dir>/app/config.json` before launching. Key settings:

| Key | Default | Description |
|-----|---------|-------------|
| `audioMode` | `"tts"` | `"tts"` reacts only to TTS speech; `"system"` captures all audio output |
| `ttsUrl` | `"http://127.0.0.1:8787"` | Base URL of your TTS server |
| `monitor` | `"primary"` | `"primary"`, `"secondary"`, `"left"`, `"right"`, or display index |
| `lobsterScale` | `4` | Sprite scale (4 = 480px tall lobster) |
| `clickThrough` | `false` | Start with click-through mode on so the lobster doesn't block clicks |
| `swimEnabled` | `true` | Enable swimming animation |

## Launch

```bash
cd <skill_dir>/app
npx electron .
```

Or use the included helper script (handles first-run `npm install` automatically):

```bash
bash <skill_dir>/scripts/launch.sh
```

## Keyboard Shortcuts

| Key | Action |
|-----|--------|
| F8 | Move window to next monitor (cycles through all displays) |
| F9 | Toggle click-through mode |
| F12 | Toggle DevTools |

## OpenClaw Integration

With OpenClaw and a local XTTS server, set `audioMode` to `"tts"` and point `ttsUrl` at your XTTS instance. The lobster polls the envelope endpoint at 45ms intervals during active speech and 500ms when idle — no perceptible CPU cost.

If you use the OpenClaw TTS proxy (port 8788), point `ttsUrl` at port 8787 (the XTTS server directly), not the proxy — the envelope endpoint is on the TTS server, not the proxy layer.

## Lip Sync Notes

If the mouth movement is ahead of or behind the audio:

- Mouth moves too early: increase `ttsPlayStartOffsetMs` (default 1100ms)
- Mouth moves too late: decrease `ttsPlayStartOffsetMs`

The default is tuned for PowerShell MediaPlayer on Windows. Other playback methods may need adjustment.

## Mouth Shapes

Six visemes drive natural speech animation:

- **A** — wide open "ah"
- **B** — wide grin "ee"
- **C** — round "oh"
- **D** — small pucker "oo"
- **E** — medium "eh"
- **F** — teeth "ff"

Plus **X** (closed) for silence and pauses. Spring physics and variety enforcement prevent robotic repetition.