Memory: support categorized sections in MEMORY.md and USER.md (current flat §-delimited format doesn't scale)

[Interstellar-code]

Summary

memory_tool.py stores entries in MEMORY.md and USER.md as a flat list delimited by \n§\n (ENTRY_DELIMITER, line 59). This works for a small number of entries but doesn't scale as a profile accumulates user knowledge — every entry loads into context regardless of relevance, and there's no way to group entries by category (communication style, workflow rules, identity facts, domain context) or filter which categories a specialist agent needs.

Current behavior

• _read_file() (line 494) splits the entire file on ENTRY_DELIMITER into a flat List[str]
• _detect_external_drift() (line 515) enforces that the file survives a read → split → join round-trip; any structural change to the file format (e.g., adding markdown headers for categorization) fails the round-trip check and triggers a .bak backup, refusing all future writes
• All entries from both files are injected into the system prompt on every turn, with no filtering by category or relevance

Problem

In a multi-profile setup (orchestrator + domain specialists), the flat format means:

1. No categorization. A USER.md mixes communication preferences, workflow rules, identity facts, and domain-specific context with no structural separation. New entries added via the memory tool (action: add) just append to the flat list.

2. No domain filtering. Every specialist loads the full USER.md even when most entries are irrelevant to their domain. A finance specialist loads dev workflow rules; a code specialist loads billing preferences. This wastes context tokens and risks cross-domain leakage.

3. No validation. Garbage entries accumulate with no schema enforcement (observed: a standalone test entry survived for weeks). There's no way to mark an entry as needing verification (verified_date) or flag drift-prone facts.

4. Specialists start cold. Specialist USER.md files are empty stubs that never accumulate data because there's no mechanism to seed a specialist with only the relevant slice of user knowledge. Every specialist starts with zero user context on every task.

5. Forced workarounds. Users who want categorization must resort to in-band tagging (e.g., prefixing entries with [COMM], [WORKFLOW]) — which is a semantic hack that the parser treats as part of the entry text, not a structural feature.

Proposed enhancement

Extend the memory file format to support categorized sections:

Format

## Communication Style
- Entry one
- Entry two
§
- Entry three (still §-delimited within section)

## Workflow Rules
- Entry four

Parser changes

1. Section-aware parsing: _read_file() first splits on markdown headers (^## ), then splits each section's body on ENTRY_DELIMITER. Returns Dict[str, List[str]] (section → entries) instead of List[str].

2. Backward compatibility: files without headers are parsed as before (flat list, assigned to a default __uncategorized__ section). The drift detector's round-trip check accounts for header round-tripping.

3. Category filtering at injection: the system prompt builder accepts a categories filter (per-profile config) so specialists only load relevant sections. E.g., a finance specialist config specifies user_memory_categories: [communication-style, workflow-rules-finance].

4. Schema validation: entries can carry optional frontmatter-like tags (verified_date, domain, expires) for drift-prone facts (age, address, vendor names that change).

5. Seeding mechanism: a new tool action (memory action=seed) copies specified sections from the orchestrator's USER.md to a specialist's USER.md, respecting the category filter.

Motivating use case

SwitchUI manages a 3-tier profile system: T1 orchestrator → T2 permanent specialists → T3 ad-hoc subagents. Each specialist needs a different slice of user knowledge:

• Neo (code): communication style + dev workflow rules
• Morpheus (design): communication style + cognitive patterns + working relationship
• Trinity (finance): communication style + finance workflow rules + approval conventions

Currently impossible with the filed format. Each specialist either loads everything or nothing.

Environment

• Hermes Agent: current (profile hermes-switch)
• memory_tool.py: ENTRY_DELIMITER = "\n§\n" (line 59), _read_file() (line 494), _detect_external_drift() (line 515)
• Observed in production with a 14-entry USER.md that needed categorization

cc @rohits