Parent
#176
What to build
Update the MAR correlation detection loop in profiling/orchestrator.py to use user-declared string sentinels with replace semantics when building per-column missingness masks.
Where the orchestrator currently references _SENTINEL_STRINGS unconditionally for String/Utf8 columns (around lines 357 and 375), it should instead check profile_config.string_sentinels.get(col_name):
- If a declaration exists for the column → use only the declared values (case-insensitive) for sentinel matching in the mask
- If no declaration exists → fall back to
_SENTINEL_STRINGS (unchanged current behaviour)
Empty/whitespace detection is always applied unconditionally.
Acceptance criteria
Blocked by
Parent
#176
What to build
Update the MAR correlation detection loop in
profiling/orchestrator.pyto use user-declared string sentinels with replace semantics when building per-column missingness masks.Where the orchestrator currently references
_SENTINEL_STRINGSunconditionally for String/Utf8 columns (around lines 357 and 375), it should instead checkprofile_config.string_sentinels.get(col_name):_SENTINEL_STRINGS(unchanged current behaviour)Empty/whitespace detection is always applied unconditionally.
Acceptance criteria
_SENTINEL_STRINGS_SENTINEL_STRINGS(no regression)tests/unit/profiling/test_pipeline_config.pyortest_missingness_profiler.py(whichever already covers MAR detection paths) verify the replace-semantics maskBlocked by