Capture Claude/Codex harness events as Basic Memory producer envelopes

[phernandez]

Summary

Add a safe, opt-in "harness WAL" path for the Claude Code and Codex plugins that normalizes supported hook/transcript events into Basic Memory producer envelopes and writes useful agent-memory artifacts.

Design note: memory://basic-memory-7020de4e925843c68c9056c60d101d9e/basic-memory-llc/product/ideas/harness-wal-capture-river-ingestion

Why

Claude Code and Codex sessions generate a stream of useful work context: prompts, tool calls, file changes, tests, blockers, decisions, handoffs, and compaction points. Today we capture some high-signal checkpoints, but the broader session stream still mostly evaporates.

The idea is to treat the harness output like an append-only event log:

Claude/Codex hook event
  -> normalized producer envelope
  -> durable inbox/raw event record
  -> coalesced artifact writer
  -> SessionNote / ToolLedger / RecallQuery / DecisionNote candidates
  -> consolidation or memory_routine follow-up

This implements the producer side of SPEC-55 and feeds SPEC-61 memory routines without turning Basic Memory into an agent runtime.

Scope for v0

• Add a shared event/envelope helper for Claude Code and Codex plugins.
• Capture supported lifecycle hooks into a normalized envelope shape.
• Include source, session id, turn id when available, cwd, project hint, timestamp, hook name, and safe payload summary.
• Coalesce events into existing artifact types from SPEC-55, especially SessionNote and ToolLedger.
• Keep hooks fast; expensive summarization and consolidation should happen out of band.
• Add idempotency keys so repeated hooks do not duplicate notes.
• Add opt-in configuration and clear redaction controls.
• Emit provenance observations such as [source] codex/<session_id> or [source] claude-code/<session_id>.

Possible normalized event types

• session_started
• user_prompted
• assistant_message
• tool_called
• tool_result
• file_changed
• test_ran
• decision_observed
• blocked
• handoff_requested
• compaction_imminent
• session_ended

V0 should only capture events exposed through supported harness hooks and visible transcript artifacts. It should not try to capture private model reasoning.

Privacy and safety constraints

• Capture must be opt-in or clearly tied to plugin installation/configuration.
• Do not capture hidden chain-of-thought or private model reasoning.
• Prefer summaries, metadata, and selected visible turns over raw transcript dumps.
• Redact obvious secrets before writing artifacts.
• Provide deny rules for paths, tools, environment-like values, and large payloads.
• Preserve source offsets or hook event ids so artifacts can be audited without storing everything forever.
• Fail fast on missing Basic Memory project mapping rather than writing to the wrong project.

Related specs

• SPEC-55: Agent Memory Pipeline (Producer Framework)
• SPEC-61: Event-Driven Memory Routines
• SPEC-56: Memory Consolidation (Dream Mode)
• SPEC-74: Semantic Diff for Basic Memory Notes

Open questions

• Should raw envelopes be stored locally in an inbox first, or should plugin hooks write directly through existing Basic Memory MCP/API calls for v0?
• Which hook events are available and stable enough across Claude Code and Codex today?
• Should Codex and Claude plugins share one helper package/script, or duplicate thin adapters with a shared envelope schema?
• What is the minimum useful artifact: session checkpoint, tool ledger, recall query, or all three?
• How should users configure redaction and excluded paths?
• Should the first version only run on PreCompact and SessionEnd, or also capture PostToolUse events?
• Where should raw event retention live once the local daemon exists?

[claude]

Claude finished @phernandez's task in 1m 55s —— View job

---

Issue Triage

• Load tools
• Explore existing plugin/hook structure
• Check for duplicate issues
• Classify issue type, priority, component, complexity
• Apply labels
• Provide triage summary

---

Classification

Dimension	Assessment
Type	Feature request — new functionality
Priority	Medium — high strategic value but no existing functionality is broken
Component	Plugins (plugins/claude-code/, plugins/codex/), with cross-cutting impact on integrations/openclaw/
Complexity	Complex — architectural work across multiple plugin packages

Labels applied: enhancement

---

No Duplicates Found

Searched for "harness WAL", "producer envelope", "SPEC-55", and "hook events" — no prior issues. This is a new initiative.

---

Current State of the Harness

The existing hook landscape is minimal but real:

Claude Code plugin (plugins/claude-code/hooks/):

• SessionStart → session-start.sh — briefs Claude from the knowledge graph
• PreCompact → pre-compact.sh — checkpoints the session before compaction

Codex plugin (plugins/codex/hooks/):

• Same two hooks, with matchers (startup|resume|compact and manual|auto)

OpenClaw integration (integrations/openclaw/hooks/capture.ts):

• Already has a post-turn capture handler that indexes conversation turns as notes via client.indexConversation()
• Filters by minimum char count, extracts last user+assistant turn — this is a working example of the producer side

The issue is asking for a more structured, normalized version of this pattern shared across Claude Code and Codex (not just OpenClaw), with defined envelope types, idempotency, opt-in config, and redaction.

---

Complexity Assessment

This is a complex feature for the following reasons:

1. Shared helper across two plugin ecosystems: Claude Code uses shell scripts + bash; Codex uses Python hooks (pre-compact.py, session-start.py) and shell. A shared envelope schema that works across both (and possibly OpenClaw's TypeScript) requires careful design.

2. Open architecture questions: The issue itself lists ~7 open design questions (inbox vs. direct write, which hooks are stable, where raw retention lives, etc.) — these need resolution before implementation.

3. Privacy/redaction infrastructure: Deny rules for paths, tools, env-like values, large payloads is non-trivial and easy to get wrong.

4. New event types: 12 proposed normalized event types, most of which have no current hook support (e.g. PostToolUse, file_changed, test_ran) — availability depends on what Claude Code and Codex actually expose.

5. SPEC-55 / SPEC-61 dependencies: This is described as the "producer side" of SPEC-55 and feeds SPEC-61 routines. Implementation correctness depends on those specs being stable.

---

Recommended Scope for v0

Based on the existing code, a minimal v0 could be:

1. Share the envelope shape — a simple JSON schema (or markdown spec) defining the normalized envelope fields (source, session_id, turn_id, cwd, project, timestamp, hook_name, payload_summary)
2. Hook PreCompact only — both plugins already have this; extend the checkpoint scripts to emit a structured envelope alongside the existing session note
3. Reuse the OpenClaw capture.ts pattern — adapt it for the shell/Python hooks
4. Idempotency key = sha256(session_id + hook_name + timestamp) — write to a note only if that key isn't already present

This deferral of PostToolUse and the more exotic event types (file_changed, test_ran) is consistent with the "V0 should only capture events exposed through supported harness hooks" note in the issue.

---

Open Questions to Resolve Before Implementation

The issue lists several — the two most blocking are:

1. Inbox vs. direct write: Direct write through existing MCP calls is simpler for v0; an inbox adds durability at the cost of a new persistence layer. Recommend direct write for v0.
2. Hook availability: Claude Code exposes PostToolUse — confirm whether Codex does, and whether it's stable enough to depend on.

[sourrris]

I am working on this now.