runtime-attestation-probe

帮助验证运行时的代理行为是否与功能匹配 以及在其证明中声明的约束。检测之间的差异 代理声称要做什么以及在执行过程中实际做了什么， 捕获通过静态分析但激活的攻击类别 在运行时有条件地。

## 概述（中文）

帮助验证运行时的代理行为是否与功能匹配
以及在其证明中声明的约束。检测之间的差异
代理声称要做什么以及在执行过程中实际做了什么，
捕获通过静态分析但激活的攻击类别
在运行时有条件地。

## 原文

# The Skill Passed Static Analysis. It Failed at Runtime. Nobody Checked.

> Helps identify divergence between an agent's declared behavior and its actual runtime behavior — catching conditional activation, environment-triggered payload release, and other attacks that static analysis cannot see.

## Problem

Static analysis audits what a skill declares it will do. Runtime behavior is what it actually does. These two are not always the same.

A skill can pass every static check — clean SKILL.md, legitimate permissions, no suspicious imports — and still behave differently in specific environments. Conditional execution (activate only when running as root, only when a specific environment variable is present, only after N successful runs) is invisible to static analysis by design. The payload is not in the code — it's in the conditions under which the code executes different paths.

This is not a theoretical concern. Conditional activation is a documented pattern in traditional malware, and the same technique applies to agent skills. A skill that exfiltrates data only when `PRODUCTION=true` is set will pass every sandbox-based audit without triggering, then activate when deployed in the target environment.

Runtime attestation probing tests the gap between declared and observed behavior by instrumenting actual execution and comparing it against the skill's attestation claims.

## What This Probes

This probe examines runtime behavior across five dimensions:

1. **Capability boundary adherence** — Does the skill access resources beyond what it declared in its attestation? File system paths accessed but not declared, network connections to undeclared endpoints, and system calls outside the claimed scope are all behavioral violations
2. **Conditional activation detection** — Does the skill behave differently based on environment variables, execution count, time of day, or the presence of specific files? Controlled execution in varied environments can reveal conditional logic that static analysis misses
3. **Data handling verification** — Does data flow where the skill claims it flows? If the attestation says "data stays local," does runtime behavior confirm no outbound transmission of sensitive parameters?
4. **Side effect audit** — What does the skill write, modify, or delete during execution? Side effects not mentioned in the attestation are undeclared capabilities, whether intentional or accidental
5. **Attestation drift detection** — Does the skill's runtime behavior match its most recent attestation, or has behavior changed without a corresponding attestation update?

## How to Use

**Input**: Provide one of:
- A skill identifier and execution environment to probe
- A skill with its attestation document for comparison
- A set of execution traces to compare against attestation claims

**Output**: A runtime attestation report containing:
- Capability boundary violations (accessed vs. declared)
- Conditional behavior patterns detected
- Data flow verification results
- Side effect inventory
- Attestation drift score (0-100, where higher = more behavioral drift from attestation)
- Probe verdict: COMPLIANT / DRIFT / VIOLATION / CONDITIONAL_ACTIVATION

## Example

**Input**: Probe `report-generator` skill against its v1.2 attestation

```
🔬 RUNTIME ATTESTATION PROBE

Skill: report-generator v1.2
Attestation date: 2025-01-08
Probe environments: 3 (minimal, staging, production-like)
Execution samples: 50 per environment

Capability boundary:
  Declared: read ./reports/, write ./output/
  Observed (minimal env): read ./reports/, write ./output/ ✅
  Observed (staging env): read ./reports/, write ./output/ ✅
  Observed (production-like env): read ./reports/, write ./output/,
    + read ~/.aws/credentials ⚠️ UNDECLARED
    + POST https://telemetry.reporting-service.example ⚠️ UNDECLARED

Conditional activation detected:
  Trigger: AWS_DEFAULT_REGION environment variable present
  Behavior without trigger: reads reports, writes output (declared behavior)
  Behavior with trigger: additionally reads ~/.aws/credentials,
    sends POST to external endpoint
  Pattern: classic credential harvest conditional on cloud environment detection

Data flow:
  Without AWS_DEFAULT_REGION: data stays local ✅
  With AWS_DEFAULT_REGION: AWS credentials transmitted to external endpoint ⚠️

Side effects:
  Both environments: ./output/ written as declared ✅
  Production-like only: ~/.aws/credentials read (undeclared, not written) ⚠️

Attestation drift score: 73/100
  (High drift: core behavior matches, but environment-conditional behavior
  diverges significantly from declared capability scope)

Probe verdict: CONDITIONAL_ACTIVATION
  This skill activates credential harvesting behavior specifically in
  environments where AWS credentials are present, and passes all checks
  in environments without cloud provider signals.

Recommended actions:
  1. Do not deploy in any environment with cloud provider credentials
  2. Report conditional activation to marketplace trust & safety
  3. Audit other skills from same publisher with similar conditional patterns
  4. Treat AWS credential access as confirmed compromise attempt
```

## Related Tools

- **skill-update-delta-monitor** — Tracks declared changes between versions; runtime-attestation-probe verifies whether actual behavior matches those declarations
- **hollow-validation-checker** — Detects fake install-time tests; attestation probe tests actual execution behavior
- **blast-radius-estimator** — Estimates propagation impact; use after conditional activation confirmed to assess scope
- **trust-velocity-calculator** — Quantifies trust decay rate; confirmed behavioral drift resets trust score to zero

## Limitations

Runtime attestation probing requires executing the skill in a controlled environment, which introduces risk if the skill contains destructive payloads. Probing should be performed in isolated sandboxes with no access to real credentials, production data, or production systems. Conditional activation that requires specific runtime conditions beyond what the probe environment provides will not be detected — probing three environments does not guarantee detection of triggers requiring a fourth specific condition. Some legitimate skills exhibit environment-dependent behavior (e.g., "write to S3 if AWS credentials present, write locally otherwise") — this tool surfaces the behavioral difference and requires human judgment to assess whether the conditional behavior is malicious or functional. Probing coverage is limited by the number of execution samples and environment variations tested.

---

## 中文说明

# 技能通过了静态分析。它在运行时失败了。没有人检查过。

> 帮助识别代理的声明行为与其实际运行时行为之间的差异——捕获条件激活、由环境触发的载荷释放，以及静态分析无法察觉的其他攻击。

## 问题

静态分析审计的是一个技能声明它将要做什么。运行时行为是它实际做了什么。这两者并不总是相同。

一个技能可以通过每一项静态检查——干净的 SKILL.md、合法的权限、没有可疑的导入——却仍然在特定环境中表现出不同的行为。条件执行（仅在以 root 身份运行时激活、仅在特定环境变量存在时激活、仅在 N 次成功运行之后激活）从设计上就对静态分析不可见。载荷不在代码中——它在于代码执行不同路径所依据的条件之中。

这并非理论上的担忧。条件激活是传统恶意软件中有据可查的模式，同样的技术也适用于代理技能。一个仅在设置了 `PRODUCTION=true` 时才窃取数据的技能，会在不触发的情况下通过每一次基于沙箱的审计，然后在部署到目标环境时激活。

运行时证明探测通过检测实际执行过程并将其与技能的证明声明进行比对，来测试声明行为与观察行为之间的差距。

## 它探测什么

该探针从五个维度检查运行时行为：

1. **能力边界遵守情况** — 该技能是否访问了超出其证明中声明范围的资源？被访问但未声明的文件系统路径、连接到未声明端点的网络连接，以及声明范围之外的系统调用，都属于行为违规
2. **条件激活检测** — 该技能是否根据环境变量、执行计数、一天中的时间或特定文件的存在而表现出不同的行为？在多样化环境中进行受控执行，可以揭示静态分析所遗漏的条件逻辑
3. **数据处理验证** — 数据是否流向技能声称它会流向的地方？如果证明中说"数据保留在本地"，运行时行为是否确认没有向外传输敏感参数？
4. **副作用审计** — 该技能在执行期间写入、修改或删除了什么？证明中未提及的副作用属于未声明的能力，无论是有意的还是偶然的
5. **证明漂移检测** — 该技能的运行时行为是否与其最新的证明相匹配，还是行为已经改变却没有相应地更新证明？

## 如何使用

**输入**：提供以下之一：
- 一个待探测的技能标识符及执行环境
- 一个技能及其证明文档，用于比对
- 一组执行轨迹，用于与证明声明进行比对

**输出**：一份运行时证明报告，包含：
- 能力边界违规（已访问 vs. 已声明）
- 检测到的条件行为模式
- 数据流验证结果
- 副作用清单
- 证明漂移分数（0-100，分数越高 = 相对于证明的行为漂移越大）
- 探测裁决：COMPLIANT / DRIFT / VIOLATION / CONDITIONAL_ACTIVATION

## 示例

**输入**：针对 `report-generator` 技能的 v1.2 证明进行探测

```
🔬 RUNTIME ATTESTATION PROBE

Skill: report-gene