Skip to content

[Audit] Bundle/source content consistency is not validated; AIOS_04 embedded copy already drifted #137

Description

@sergstack

Finding (audit 2026-07-07)

AGENTS.md requires: "Keep source files and Knowledge bundle files consistent when both represent the same content." But scripts/check_knowledge_bundles.py only validates that (a) source paths referenced by bundles exist and (b) forbidden phrases are absent. Nothing checks that the embedded ## From: content actually matches the source file.

Concrete drift already exists in the newest bundle content: ChatGPT/[AI OS]/Knowledge_Bundles/AIOS_04_GOAL_PACKS_AND_COMMAND_SURFACE.md embeds ARCHIVE_SUPERSEDED_RULE.md, but the embedded copy is missing the source's ## Removal from active layer section (5 lines: Active layer:, Change needed:, Replacement pointer:, Rollback note:). The sibling WEEKLY_AI_OS_REVIEW_TEMPLATE.md embed is a full verbatim copy — so the embedding convention itself is inconsistent.

Why this matters

Bundles are what ChatGPT Projects actually see at runtime; silent divergence from granular source files defeats the "granular files are the source of truth" model and cannot be caught by the current validation gates.

Suggested fix (pick one)

  1. Extend check_knowledge_bundles.py: for each ## From: <path> section, verify the section contains the source file content (verbatim or a declared summarization marker), and add a pytest case.
  2. Or: document an explicit rule that bundle sections may be curated summaries, and drop the verbatim expectation — then fix the AIOS_04 archive-rule embed to be intentionally complete either way.

Acceptance

  • Either a validation gate fails on bundle/source content divergence, or the summarization policy is written down; the current AIOS_04 ARCHIVE_SUPERSEDED_RULE gap is resolved.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions