What the plugin costs¶
The plugin buys ambient memory by spending latency and context. Until 2026-08-11 nobody had measured either, the README's "~10 ms, never on the agent's hot path" was carried forward as though it described the whole plugin, when it describes the capture hook alone.
Reproduce with:
uv run python scripts/measure_hook_overhead.py # realistic, dedup on
uv run python scripts/measure_hook_overhead.py --fresh # worst case, dedup defeated
uv run python scripts/measure_hook_overhead.py --json
Measured¶
Against a local server, real fleet memory, real agent identity, 7 samples per hook, per-session dedup active:
| Hook | Fires | p50 | p95 | Tokens (first → then) |
|---|---|---|---|---|
SessionStart |
per session | 696 ms | 729 ms | 612 → 612 |
UserPromptSubmit · inbox |
per prompt | 65 ms | 69 ms | 0 → 0 |
UserPromptSubmit · recall |
per prompt | 195 ms | 221 ms | 112 → 88 |
PreToolUse · gotcha |
per tool call | 189 ms | 204 ms | 123 → 0 |
Stop · capture |
per turn | 16 ms | 17 ms | 0 → 0 |
For a session of 20 prompts and 60 tool calls: ≈ 17.6 s of added wall-clock and ≈ 4,200 tokens of context.
Two things worth reading off that table:
- The dominant cost is
PreToolUse, because it fires on every tool call. At 189 ms it is most of the added wall-clock, and it is not the hook anyone would have guessed. -
The drainer does more since these numbers were taken. It now also extracts a token rollup from the same transcript slice and posts it to
/usage. That work is in the detached drainer, not the hook, so the 16 ms hot-path figure above still stands, but the measurement predates the change and the drainer's own cost has not been re-measured. -
Stop· capture is genuinely cheap, 16 ms, injects nothing. The original claim holds for the hook it was made about; it just never covered recall, inbox or gotcha.
Numbers to treat with care: this is localhost, so a remote instance is strictly worse; tokens are counted as characters/4; the inbox figure depends entirely on whether that agent has unread messages, and was 547 tokens for an identity that did.
An inert plugin is not a free plugin¶
With bad credentials every hook returns nothing, logs nothing, and injects nothing. The same session then costs 6.6 s and 0 tokens, all of the latency, none of the benefit, and no signal anywhere that it is happening.
measure_hook_overhead.py exits non-zero when total injection is zero, for exactly this reason. It is also a hazard for any A/B measurement: a treatment arm in this state is a placebo, and would produce a clean, entirely meaningless null result. Any benchmark comparing Artel on/off must assert non-zero injection while it runs, not merely check that the plugin is installed.