Skip to content

feat: incremental extraction — cb feed for adding/updating docs #9

Description

@rajnavakoti

Description

Currently adding a new document requires re-running the full extraction pipeline. Add a cb feed <file-or-folder> command that incrementally processes new or updated documents without touching existing entities. Uses the existing source document provenance and crash-safe resume infrastructure.

Acceptance Criteria

  • cb feed <file> extracts entities from a single new document and adds them to the existing KB
  • cb feed <folder> processes only files not already in the KB (by filename + content hash)
  • When a previously-processed file is fed again with changed content, existing entities from that file are updated (not duplicated)
  • Entities from unchanged source docs are not re-processed
  • Accumulated knowledge is updated with new entities after feed completes
  • Output clearly shows: N new entities created, M entities updated, K entities unchanged

Out of Scope

  • Structural fingerprinting for section-level diffing (P2)
  • Entity deletion when content is removed from source doc (needs design)
  • Cross-document concept synthesis

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1-nextNext after OSS releasepipelineExtraction pipeline

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions