enhanced-memory

通过混合向量+关键字评分、时间路由、文件路径评分、自适应加权、伪相关反馈、显着性评分和知识图交叉引用来增强记忆搜索。用 4 信号融合检索系统替换默认的内存搜索。在搜索内存、索引内存文件、构建交叉引用或对内存显着性进行评分时使用。需要 Ollama 和 nomic-embed-text 模型。

# Enhanced Memory

Drop-in enhancement for OpenClaw's memory system. Replaces flat vector search with a 4-signal hybrid retrieval pipeline that achieved **0.782 MRR** (vs ~0.45 baseline vector-only).

## Setup

```bash
# Install Ollama and pull the embedding model
ollama pull nomic-embed-text

# Index your memory files (run from workspace root)
python3 skills/enhanced-memory/scripts/embed_memories.py

# Optional: build cross-reference graph
python3 skills/enhanced-memory/scripts/crossref_memories.py build
```

Re-run `embed_memories.py` whenever memory files change significantly.

## Scripts

### `scripts/search_memory.py` — Primary Search

Hybrid 4-signal retrieval with automatic adaptation:

```bash
python3 skills/enhanced-memory/scripts/search_memory.py "query" [top_n]
```

**Signals fused:**
1. **Vector similarity** (0.4) — cosine similarity via nomic-embed-text embeddings
2. **Keyword matching** (0.25) — query term overlap with chunk text
3. **Header matching** (0.1) — query terms in section headers
4. **Filepath scoring** (0.25) — query terms matching file/directory names

**Automatic behaviors:**
- **Temporal routing** — date references ("yesterday", "Feb 8", "last Monday") get 3x boost on matching files
- **Adaptive weighting** — when keyword overlap is low, shifts to 85% vector weight
- **Pseudo-relevance feedback (PRF)** — when top score < 0.45, expands query with terms from initial results and re-scores

### `scripts/enhanced_memory_search.py` — JSON-Compatible Search

Same pipeline with JSON output format compatible with OpenClaw's memory_search tool:

```bash
python3 skills/enhanced-memory/scripts/enhanced_memory_search.py --json "query"
```

Returns `{results: [{path, startLine, endLine, score, snippet, header}], ...}`.

### `scripts/embed_memories.py` — Indexing

Chunks all `.md` files in `memory/` plus core workspace files (MEMORY.md, AGENTS.md, etc.) by markdown headers and embeds them:

```bash
python3 skills/enhanced-memory/scripts/embed_memories.py
```

Outputs `memory/vectors.json`. Batches embeddings in groups of 20, truncates chunks to 2000 chars.

### `scripts/memory_salience.py` — Salience Scoring

Surfaces stale/important memory items for heartbeat self-prompting:

```bash
python3 skills/enhanced-memory/scripts/memory_salience.py          # Human-readable prompts
python3 skills/enhanced-memory/scripts/memory_salience.py --json   # Programmatic output
python3 skills/enhanced-memory/scripts/memory_salience.py --top 5  # More items
```

Scores `importance × staleness` considering: file type (topic > core > daily), size, access frequency, and query gap correlation.

### `scripts/crossref_memories.py` — Knowledge Graph

Builds cross-reference links between memory chunks using embedding similarity:

```bash
python3 skills/enhanced-memory/scripts/crossref_memories.py build          # Build index
python3 skills/enhanced-memory/scripts/crossref_memories.py show <file>    # Show refs for file
python3 skills/enhanced-memory/scripts/crossref_memories.py graph          # Graph statistics
```

Uses file-representative approach (top 5 chunks per file) to reduce O(n²) to manageable comparisons. Threshold: 0.75 cosine similarity.

## Configuration

All tunable constants are at the top of each script. Key parameters:

| Parameter | Default | Script | Purpose |
|-----------|---------|--------|---------|
| `VECTOR_WEIGHT` | 0.4 | search_memory.py | Weight for vector similarity |
| `KEYWORD_WEIGHT` | 0.25 | search_memory.py | Weight for keyword overlap |
| `FILEPATH_WEIGHT` | 0.25 | search_memory.py | Weight for filepath matching |
| `TEMPORAL_BOOST` | 3.0 | search_memory.py | Multiplier for date-matching files |
| `PRF_THRESHOLD` | 0.45 | search_memory.py | Score below which PRF activates |
| `SIMILARITY_THRESHOLD` | 0.75 | crossref_memories.py | Min similarity for cross-ref links |
| `MODEL` | nomic-embed-text | all | Ollama embedding model |

To use a different embedding model (e.g., `mxbai-embed-large`), change `MODEL` in each script and re-run `embed_memories.py`.

## Integration

To replace the default memory search, point your agent's search tool at these scripts. The scripts expect:
- `memory/` directory relative to workspace root containing `.md` files
- `memory/vectors.json` (created by `embed_memories.py`)
- Ollama running locally on port 11434

All scripts use only Python stdlib + Ollama HTTP API. No pip dependencies.

---

## 中文说明

# Enhanced Memory

为 OpenClaw 的记忆系统提供的即插即用增强方案。用一条 4 信号混合检索流水线替换扁平的向量搜索，实现了 **0.782 MRR**（相比仅向量基线约 0.45）。

## 安装设置

```bash
# Install Ollama and pull the embedding model
ollama pull nomic-embed-text

# Index your memory files (run from workspace root)
python3 skills/enhanced-memory/scripts/embed_memories.py

# Optional: build cross-reference graph
python3 skills/enhanced-memory/scripts/crossref_memories.py build
```

每当记忆文件发生重大变化时，重新运行 `embed_memories.py`。

## 脚本

### `scripts/search_memory.py` — 主搜索

带自动适应的混合 4 信号检索：

```bash
python3 skills/enhanced-memory/scripts/search_memory.py "query" [top_n]
```

**融合的信号：**
1. **向量相似度**（0.4）—— 通过 nomic-embed-text 嵌入计算的余弦相似度
2. **关键字匹配**（0.25）—— 查询词与分块文本的重叠
3. **标题匹配**（0.1）—— 查询词出现在章节标题中
4. **文件路径评分**（0.25）—— 查询词与文件/目录名称的匹配

**自动行为：**
- **时间路由** —— 日期引用（"yesterday"、"Feb 8"、"last Monday"）会对匹配文件给予 3 倍加成
- **自适应加权** —— 当关键字重叠较低时，转向 85% 的向量权重
- **伪相关反馈（PRF）** —— 当最高得分 < 0.45 时，用初始结果中的词扩展查询并重新评分

### `scripts/enhanced_memory_search.py` — 兼容 JSON 的搜索

相同的流水线，输出与 OpenClaw 的 memory_search 工具兼容的 JSON 格式：

```bash
python3 skills/enhanced-memory/scripts/enhanced_memory_search.py --json "query"
```

返回 `{results: [{path, startLine, endLine, score, snippet, header}], ...}`。

### `scripts/embed_memories.py` — 索引构建

按 markdown 标题对 `memory/` 中所有 `.md` 文件以及核心工作区文件（MEMORY.md、AGENTS.md 等）进行分块并嵌入：

```bash
python3 skills/enhanced-memory/scripts/embed_memories.py
```

输出 `memory/vectors.json`。以 20 个为一组批量嵌入，将分块截断到 2000 字符。

### `scripts/memory_salience.py` — 显著性评分

为心跳自我提示浮现陈旧/重要的记忆条目：

```bash
python3 skills/enhanced-memory/scripts/memory_salience.py          # Human-readable prompts
python3 skills/enhanced-memory/scripts/memory_salience.py --json   # Programmatic output
python3 skills/enhanced-memory/scripts/memory_salience.py --top 5  # More items
```

考虑以下因素对 `importance × staleness` 评分：文件类型（topic > core > daily）、大小、访问频率以及查询缺口相关性。

### `scripts/crossref_memories.py` — 知识图谱

使用嵌入相似度在记忆分块之间构建交叉引用链接：

```bash
python3 skills/enhanced-memory/scripts/crossref_memories.py build          # Build index
python3 skills/enhanced-memory/scripts/crossref_memories.py show <file>    # Show refs for file
python3 skills/enhanced-memory/scripts/crossref_memories.py graph          # Graph statistics
```

采用文件代表法（每个文件取前 5 个分块）将 O(n²) 降低到可管理的比较量。阈值：0.75 余弦相似度。

## 配置

所有可调常量都位于每个脚本的顶部。关键参数：

| 参数 | 默认值 | 脚本 | 用途 |
|-----------|---------|--------|---------|
| `VECTOR_WEIGHT` | 0.4 | search_memory.py | 向量相似度的权重 |
| `KEYWORD_WEIGHT` | 0.25 | search_memory.py | 关键字重叠的权重 |
| `FILEPATH_WEIGHT` | 0.25 | search_memory.py | 文件路径匹配的权重 |
| `TEMPORAL_BOOST` | 3.0 | search_memory.py | 日期匹配文件的倍率 |
| `PRF_THRESHOLD` | 0.45 | search_memory.py | 低于此得分时激活 PRF |
| `SIMILARITY_THRESHOLD` | 0.75 | crossref_memories.py | 交叉引用链接的最小相似度 |
| `MODEL` | nomic-embed-text | all | Ollama 嵌入模型 |

要使用不同的嵌入模型（例如 `mxbai-embed-large`），请在每个脚本中更改 `MODEL` 并重新运行 `embed_memories.py`。

## 集成

要替换默认的记忆搜索，将你的代理搜索工具指向这些脚本。脚本要求：
- 工作区根目录下相对的 `memory/` 目录，其中包含 `.md` 文件
- `memory/vectors.json`（由 `embed_memories.py` 创建）
- 在本地 11434 端口运行的 Ollama

所有脚本仅使用 Python 标准库 + Ollama HTTP API。无 pip 依赖。