Problem
The per-request query plan is stored on the prepared request object as _alternator_query_plan, but botocore creates a fresh prepared request object on SDK retry attempts. As a result, retries recreate a new plan instead of advancing through the remaining nodes.
This is especially visible with key affinity: if the preferred node is down, every retry can select the same preferred node again instead of trying the rest of the live-node list.
Evidence
A local reproduction with three nodes and retries={"max_attempts": 2} captured three before-send attempts. With a deterministic preferred affinity node, all three attempts used the same URL.
Relevant code:
alternator/core/handlers.py: _alternator_query_plan is attached to the prepared request object.tests/unit/test_key_affinity.py: the current test calls update_endpoint() repeatedly on the same mock request, which does not match botocore retry behavior.
Expected behavior
Retries for a single logical SDK operation should walk the request's node plan and try alternate nodes before exhausting attempts.
Test coverage needed
Add tests using a real botocore client and connection failures to verify that SDK retries move across different live nodes, including an affinity-preferred first attempt followed by non-preferred fallback nodes.
Problem
The per-request query plan is stored on the prepared request object as
_alternator_query_plan, but botocore creates a fresh prepared request object on SDK retry attempts. As a result, retries recreate a new plan instead of advancing through the remaining nodes.This is especially visible with key affinity: if the preferred node is down, every retry can select the same preferred node again instead of trying the rest of the live-node list.
Evidence
A local reproduction with three nodes and
retries={"max_attempts": 2}captured threebefore-sendattempts. With a deterministic preferred affinity node, all three attempts used the same URL.Relevant code:
alternator/core/handlers.py:_alternator_query_planis attached to the prepared request object.tests/unit/test_key_affinity.py: the current test callsupdate_endpoint()repeatedly on the same mock request, which does not match botocore retry behavior.Expected behavior
Retries for a single logical SDK operation should walk the request's node plan and try alternate nodes before exhausting attempts.
Test coverage needed
Add tests using a real botocore client and connection failures to verify that SDK retries move across different live nodes, including an affinity-preferred first attempt followed by non-preferred fallback nodes.