Parent
#176
What to build
Wire string sentinel declarations from the profile into Phase 2 and verify the full pipeline with an integration test.
ImputationOrchestrator.fit() (imputation/orchestrator.py):
- Change the existing
_resolve_effective_nulls(train_df) call to _resolve_effective_nulls(train_df, string_sentinels=profile.string_sentinels) so replace-semantics normalization applies to the training split before any fill values are computed.
- Pass
string_sentinels=profile.string_sentinels to the FittedImputer constructor so transform-time normalization uses the same declarations.
Integration test (tests/integration/test_imputation_end_to_end.py):
- DataFrame with a String/Utf8 column (
status) containing a mix of real values, declared sentinels ("N/A", "missing"), and the hardcoded default "?".
ProfileConfig declares string_sentinels={"status": ["N/A", "missing"]}.
StructuralProfiler.profile() runs; the returned profile carries the declarations.
ImputationOrchestrator.fit() produces a FittedImputer.
FittedImputer.transform() on the test split produces a result where:
"N/A" and "missing" values are filled (treated as effective null).
"?" values are not filled (replace semantics: "?" is not in the declaration and the hardcoded set does not apply for this column).
- The fill value is derived exclusively from non-sentinel observations.
Acceptance criteria
Blocked by
Parent
#176
What to build
Wire string sentinel declarations from the profile into Phase 2 and verify the full pipeline with an integration test.
ImputationOrchestrator.fit()(imputation/orchestrator.py):_resolve_effective_nulls(train_df)call to_resolve_effective_nulls(train_df, string_sentinels=profile.string_sentinels)so replace-semantics normalization applies to the training split before any fill values are computed.string_sentinels=profile.string_sentinelsto theFittedImputerconstructor so transform-time normalization uses the same declarations.Integration test (
tests/integration/test_imputation_end_to_end.py):status) containing a mix of real values, declared sentinels ("N/A","missing"), and the hardcoded default"?".ProfileConfigdeclaresstring_sentinels={"status": ["N/A", "missing"]}.StructuralProfiler.profile()runs; the returned profile carries the declarations.ImputationOrchestrator.fit()produces aFittedImputer.FittedImputer.transform()on the test split produces a result where:"N/A"and"missing"values are filled (treated as effective null)."?"values are not filled (replace semantics:"?"is not in the declaration and the hardcoded set does not apply for this column).Acceptance criteria
ImputationOrchestrator.fit()passesstring_sentinels=profile.string_sentinelsto_resolve_effective_nullsImputationOrchestrator.fit()passesstring_sentinels=profile.string_sentinelsto theFittedImputerconstructorFittedImputerreturned byfit()hasstring_sentinelsequal toprofile.string_sentinels"N/A"and"missing"are filled in the transform output"?"is not filled (replace semantics confirmed)Blocked by