Skip to content

Plugin LLM observability — OpenTelemetry tracing via hooks + execution logs + optimization skill #297

@blafourcade

Description

@blafourcade

Objective

Add LLM observability to the framework: trace what users (and the LLM) actually do, expose execution logs, and feed an optimization loop.

Scope

  • Hooks-based tracing — capture LLM/agent actions and user interactions via lifecycle hooks (tool calls, skill invocations, prompts/outputs).
  • OpenTelemetry export — emit traces/spans in the OTel format so teams plug into existing observability backends (no proprietary lock-in).
  • LLM execution logs — make per-run execution visible (which skills ran, what was sent, latency, tokens, outcome).
  • Optimization skill — a skill that reads the traces/logs and proposes improvements (prompt/skill tuning, redundant steps, costly patterns).

Why

Today there is no visibility into how AIDD is used or how the LLM executes. Observability enables: debugging agent behavior, measuring real usage, and a data-driven optimization loop. Privacy: opt-in, no PII by default.

Notes

  • Distinct from CLI opt-in telemetry (install/adoption metrics) and aidd-team (team KPIs) — this is execution-level LLM tracing for devs.
  • Likely grows into sub-issues (hooks capture · OTel exporter · log viewer · optimization skill). Created as an epic.
  • Created during roadmap review 2026-06-18.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions