Skip to content

[Benchmark]: Measure mutation engine throughput #56

Description

@rian-be

Summary

Benchmark the core mutation engine throughput across single mutation and batch execution paths.

Goal

Measure how the core runtime behaves under the execution shapes that matter most for throughput so regressions and improvements are visible over time.

Problem

The core benchmark umbrella needs at least one concrete throughput focused benchmark to establish useful baseline. Without measuring execution throughput directly, it is difficult to tell whether changes in the engine, batching logic, or state handling have meaningful performance impact.

This issue targets the core mutation engine itself and keeps the benchmark scenarios close to the actual runtime paths used by the library.

Scope

  • Benchmark single mutation execution throughput
  • Benchmark batch mutation execution throughput
  • Measure throughput across multiple state sizes or state shapes where it changes runtime cost materially
  • Keep the benchmark scenarios focused on the core mutation engine rather than governance or Redis provider behavior
  • Reuse existing benchmark harnesses or helpers where they already exist
  • Make the benchmark names and parameters clear enough to compare runs over time

Design Expectations

  • Benchmarks should reflect real execution paths rather than artificial loops.
  • Batch and single mutation measurements should be reported separately.
  • State size variations should be explicit so performance trends are easy to interpret.
  • The benchmark setup should stay lightweight and not require unrelated test infrastructure.
  • Any helper code should remain local to the benchmark project.

Suggested Measurements

  • one mutation on small state
  • one mutation on larger state
  • batch execution over a fixed set of mutations
  • batch execution over varying state sizes

Acceptance Criteria

  • Core mutation engine throughput is measured for both single and batch execution
  • The benchmark output makes it easy to compare different state sizes or shapes
  • The benchmark remains centered on the core runtime and does not drift into provider concerns
  • The scenario is organized in way that can be extended with more throughput cases later

Non-Goals

  • This issue does not optimize the engine prematurely
  • This issue does not add provider specific benchmarks
  • This issue does not design full load testing suite
  • This issue does not change runtime behavior

Notes

This issue is child of the benchmark umbrella #55.

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkBenchmark coverage and performance measurement changes

    Type

    No fields configured for Task.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions