Skip to content

[Perf]: speed up cache staleness #263

Description

@hyperpolymath

What is slow / heavy

Issue 2: [Feature] Offline Pre-fetching & Stale-While-Revalidate Cartridge Caching
Description Relying on on-demand dynamic cartridge fetching introduces cold-start latency on initial tool invocations and creates a single point of failure if boj-server-cartridges becomes unreachable or rate-limited at runtime.

Measurement

Proposed Solution

Local Bundling: Bundle a pre-warmed cache of the top 20 most frequently used capability cartridges directly inside the release distribution/container.
Stale-While-Revalidate Policy: Implement an asynchronous background fetch mechanism. When a cartridge is invoked, immediately execute the locally cached version while asynchronously checking for cartridge updates in the background to eliminate cold starts entirely.

Baseline vs now

No response

Environment / hardware

No response

Before submitting

  • I included reproducible measurements, not just a subjective impression.
  • I searched existing issues for this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions