Add callstack monitor for PII-safe live event tailing

[Justinabox]

Motivation

#22 deliberately shipped a minimal callstack CLI for status and send, leaving monitor as a follow-up if event streaming would make the first PR too large. Operators still need a safe way to tail live modem activity while bringing up hardware: inbound SMS, delivery reports, call state, USSD responses, reconnects, and signal changes.

This issue scopes a local CLI monitor command, not a network API. It complements #31 (/ws) by helping developers debug a Pi/modem from the terminal without opening another HTTP surface.

User journey

1. A developer SSHes into the Raspberry Pi running a modem.
2. They run callstack monitor --at-port /dev/ttyUSB2 --json during setup.
3. As typed events occur, the CLI prints one sanitized JSON line per event.
4. They can Ctrl-C cleanly without a traceback, leaked message content in errors, or orphaned modem tasks.
5. For human troubleshooting, they can omit --json and get concise text output that masks private identifiers by default.

API / UX sketch

callstack monitor \
  --at-port /dev/ttyUSB2 \
  --audio-port /dev/ttyUSB4 \
  --sms-db-path /var/lib/callstack/sms.sqlite \
  --events sms.received,sms.delivery_report,call.state,modem.state,signal.quality,ussd.response \
  --json

JSON-lines output example:

{"type":"modem.state","timestamp":"2026-06-25T00:00:00Z","data":{"connected":true}}
{"type":"sms.delivery_report","timestamp":"2026-06-25T00:00:10Z","data":{"reference":42,"status":"delivered"}}

Human output example:

[00:00:00] modem connected
[00:00:10] sms delivery report ref=42 status=delivered

Default output should avoid printing SMS bodies and full phone numbers. If a future operator really needs full payloads, make that an explicit opt-in flag in a separate security-reviewed change.

Technical approach

1. Add a monitor subcommand in callstack/cli.py using the existing global config flags and asyncio.run() style.
2. Create a small event serialization helper, preferably shared or compatible with the future WebSocket serializer from Add authenticated WebSocket realtime event stream #31, that maps typed events to PII-safe envelopes:
   • IncomingSMSEvent -> sms.received with masked sender and body length or redacted body.
   • SMSDeliveryReportEvent -> sms.delivery_report with reference/status and masked recipient if present.
   • CallStateEvent -> call.state.
   • ModemDisconnectedEvent / ModemReconnectedEvent -> modem.state.
   • SignalQualityEvent -> signal.quality.
   • USSDResponseEvent -> ussd.response with text redacted or length-only by default.
3. Subscribe to selected event types on modem.bus, place envelopes into a bounded asyncio.Queue, and print until interrupted.
4. Emit an initial modem.state/status snapshot after successful connection if it can be done without extra hardware assumptions.
5. Support --events filtering so developers can narrow output without changing code.
6. Handle KeyboardInterrupt/cancellation cleanly and ensure all subscriptions are removed before closing the modem.
7. Preserve CLI safety from Add a minimal callstack CLI for SMS send and modem status #22: no tracebacks by default and no full private numbers, SMS bodies, API keys, SIM identifiers, or raw AT lines in stderr/logs.

Affected modules and tests

Likely files:

• Modify: callstack/cli.py — parser, monitor runner, output modes.
• Create (optional): callstack/events/serialize.py or callstack/realtime.py — typed-event-to-envelope helper reused later by Add authenticated WebSocket realtime event stream #31.
• Modify: tests/test_cli.py — monitor parser/config mapping and clean cancellation behavior.
• Create (optional): tests/test_event_serialization.py — PII-safe envelope mapping for each public event type.

Existing context:

• callstack/cli.py already has _add_config_args(), _config_from_args(), and safe error handling.
• Modem.bus is public on the Modem object and already supports subscribe() / unsubscribe().
• events/types.py already defines the public event dataclasses needed for monitor output.
• Add authenticated WebSocket realtime event stream #31 covers authenticated WebSocket streaming; keep this CLI local and smaller.

Hardware / modem caveats

• Unit tests must use fake modem/event-bus objects only; no serial devices, SIM cards, or carrier network access.
• The monitor must not expose raw AT command lines, IMEI/serial/SIM identifiers, API keys, full phone numbers, or SMS/USSD content by default.
• Slow terminals should not cause unbounded memory growth; use a bounded queue and deterministic overflow notice.
• This should not assume a specific SIMCOM/Quectel URC shape; typed events are the boundary.

Acceptance criteria

• callstack --help and callstack monitor --help document the monitor command and shared port/config flags.
• callstack monitor --json prints one valid JSON object per event for the selected event types.
• Human output is concise and masks/redacts private fields by default.
• --events filters supported event types and rejects unknown names with a useful non-zero CLI error.
• Ctrl-C/cancellation closes the modem, unsubscribes handlers, and exits without a traceback.
• A bounded queue or equivalent backpressure policy prevents unbounded growth during event bursts.
• Tests cover event serialization, JSON output, human output masking, event filtering, and cleanup.
• No real hardware is required for tests.

Exact verification gates

git diff --check
PYTHONPATH=. uv run --no-project --with pytest --with pytest-asyncio --with pytest-aiohttp --with pyserial-asyncio --with aiosqlite pytest tests/test_cli.py tests/test_events.py tests/test_event_serialization.py -q
PYTHONPATH=. uv run --no-project --with pytest --with pytest-asyncio --with pytest-aiohttp --with pyserial-asyncio --with aiosqlite pytest tests/ -q

If tests/test_event_serialization.py is not created because the serializer stays inside callstack/cli.py, replace that path with the exact CLI test file(s) covering serialization.

Non-goals

• WebSocket/API event streaming (Add authenticated WebSocket realtime event stream #31).
• Dashboard/TUI UI.
• Durable event replay.
• Raw AT trace mode.
• Printing full SMS bodies/phone numbers by default.

[Justinabox]

Planner refinement — event serializer is now on main, make monitor the next local-DX slice

Planning pass evidence: clean main at 6926b369cdbaf2ec6cb99c129b4615aaf949c76b after the PII-safe typed-event serialization helper landed. I did not create a duplicate CLI/realtime issue; #50 remains the right feature issue for the local operator monitor.

Product / roadmap evidence

• North-star priority: developer experience for hardware bring-up should make modem quirks understandable without leaking private data.
• README.md now documents callstack status, callstack send, and safe callstack doctor, and explicitly lists PII-safe callstack monitor as a planned CLI follow-up.
• ROADMAP.md keeps local callstack monitor ahead of broader realtime/dashboard expansion in the hardening order.
• Current code has callstack/events/serialize.py::serialize_event(...), which already emits PII-safe envelopes for SMS, delivery reports, call state, modem state, signal quality, USSD, ring/caller-ID, DTMF, and sent-SMS events.
• callstack/cli.py still exposes only status, send, and doctor; there is no monitor subcommand yet.
• Dedupe confirmed existing tracking only: Add callstack monitor for PII-safe live event tailing #50 for monitor, Add authenticated WebSocket realtime event stream #31 for /ws; this update keeps the scope local and does not duplicate Add authenticated WebSocket realtime event stream #31.

Recommended next one-PR scope

Implement callstack monitor as a local terminal event tail that reuses the shipped serializer instead of inventing a new event schema.

Suggested CLI shape:

callstack monitor \
  --at-port /dev/ttyUSB2 \
  --audio-port /dev/ttyUSB4 \
  --events sms.received,sms.delivery_report,call.state,modem.state,signal.quality,ussd.response \
  --json

Implementation outline:

1. Add a monitor subcommand in callstack/cli.py, using the existing _add_config_args() and _config_from_args() helpers.
2. Map --events names to the typed event classes supported by callstack.events.serialize.serialize_event.
3. Subscribe handlers on modem.bus, push serialized envelopes into a bounded asyncio.Queue, and print JSON Lines for --json.
4. Add concise human output mode that still masks/redacts private fields by default.
5. On cancellation/KeyboardInterrupt, unsubscribe all handlers, close the modem context, and exit without a traceback.
6. Emit a deterministic overflow notice if the queue fills; do not let slow terminals grow memory unbounded.

Acceptance criteria to emphasize now

• callstack --help lists monitor; callstack monitor --help documents shared config flags plus --events and --json.
• callstack monitor --json prints one valid JSON object per selected typed event using serialize_event(...).
• Human output is concise and does not show raw SMS bodies, USSD text, full phone numbers, SIM identifiers, API keys, raw AT lines, or modem serial/IMEI values.
• Unknown --events names produce a useful non-zero CLI error before opening serial ports.
• Multiple selected event types can be subscribed at once; unsupported event classes are not silently streamed raw.
• Slow output/backpressure is bounded and tested with a deterministic overflow policy.
• Ctrl-C/cancellation unsubscribes handlers and closes the modem without traceback noise.
• Unit tests use fake modem/event-bus objects only; no serial hardware, SIM card, or carrier access is required.

Exact gates for the PR

git diff --check
PYTHONPATH=. uv run --no-project --with pytest --with pytest-asyncio --with pytest-aiohttp --with pyserial-asyncio --with aiosqlite pytest tests/test_cli.py tests/test_event_serialization.py tests/test_events.py -q
PYTHONPATH=. uv run --no-project --with pytest --with pytest-asyncio --with pytest-aiohttp --with pyserial-asyncio --with aiosqlite pytest tests/ -q

Non-goals

• No WebSocket/API streaming; keep Add authenticated WebSocket realtime event stream #31 separate and downstream of the network-auth/webhook hardening sequence.
• No durable event replay.
• No raw AT trace mode.
• No opt-in full-payload mode in this first PR; full SMS/USSD/body display should be a separate security-reviewed change if ever needed.

Planner gate evidence

git diff --check
# exit 0
PYTHONPATH=. uv run --no-project --with pytest --with pytest-asyncio --with pytest-aiohttp --with pyserial-asyncio --with aiosqlite pytest tests/ -q
# 570 passed in 5.67s