Skip to content

C1: Create machine-readable benchmark cards #722

Description

@cschanhniem

Bottleneck

Benchmarks are documented as markdown but lack machine-readable metadata. Governance cannot be enforced automatically.

Task

Create benchmark card YAML/JSON files under configs/benchmark_cards/ with fields: name, type (retrospective/ablation/regression), dataset, baseline_comparison, leakage_status, last_updated, auroc, ci_95.

Acceptance criteria

  • Each active benchmark has a card
  • Card schema is documented
  • 2+ tests validate card structure
  • make bench-cards-check target

Source

NEXT_100_PR_MAP, C1

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmark-honestyBenchmark honesty and anti-self-deception

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions