ingest backpressure charges idempotent replays against the byte-rate budget (ledger checked after backpressure)

[bgmcmullen]

Summary

On POST /v1/ingest/{signal}, the per-gateway byte-rate token bucket is charged before the
idempotency ledger is consulted. An already-acked X-Hyp-Batch-Id returns 202 and does no
spooling work — yet it still spends rate budget. This lets a retrying client livelock:
re-sending an already-delivered prefix burns the burst and 429s before reaching new data.

Where

src/http/routes-ingest.js:

// :150-155  backpressure FIRST (charges Content-Length tokens)
const reject = backpressure.check(gatewayId, declaredLength)
if (reject) return sendJson(res, reject.status, { error: reject.kind },
                            { 'retry-after': String(reject.retryAfter) })

// :164  idempotency ledger SECOND (only reached if budget remained)
if (batchId && ledger.has(gatewayId, batchId)) return sendJson(res, 202, {})

Bucket: src/ingest/backpressure.js:19-64 (per-gateway token bucket, defaults 8 MB/s rate / 64
MB burst, src/config.js:65-66).

Impact

• A client that restarts a partition from chunk 0 (current hypaware behavior) re-spends budget
  on already-stored chunks, so partitions larger than the burst can never fully land — even
  though the new bytes do no extra backend work.
• Makes the rate limit punish idempotent retries, the exact thing the batch-id ledger was meant
  to make cheap.

Proposed fix

• Short-circuit the ledger before charging the bucket: if X-Hyp-Batch-Id is a known ledger
  hit, return 202 without consuming rate budget (a replay does no spooling). Keep charging by
  Content-Length for genuinely new batches.
• This breaks the livelock defensively, independent of client behavior.

Optional follow-ups

• Make Retry-After adaptive (derive from token deficit ÷ refill rate) instead of the hardcoded
  30 (backpressure.js:64).
• Document HYPSERVER_BYTE_RATE_PER_GATEWAY / HYPSERVER_BYTE_BURST_PER_GATEWAY defaults and
  tuning guidance.