Skip to content

feat: splitting/_profile_signals.py string sentinel replace semantics #181

Description

@DEVunderdog

Parent

#176

What to build

Update splitting/_profile_signals.py to use user-declared string sentinels with replace semantics when computing effective null masks for split-balance signals.

Where the module currently references _SENTINEL_STRINGS unconditionally for String/Utf8 columns (line 58), it should instead resolve the per-column sentinel set from the profile result:

  • If profile_result.string_sentinels has an entry for the column → use only the declared values (case-insensitive)
  • If no entry exists → fall back to _SENTINEL_STRINGS (unchanged current behaviour)

Empty/whitespace detection is always applied unconditionally.

StructuralProfileResult.string_sentinels (added in #177) is the source of the declarations — the split module already receives the profile result.

Acceptance criteria

  • Effective null mask for a String/Utf8 column with a declared sentinel uses only the declared values when computing split signals
  • Effective null mask for a String/Utf8 column with no declaration continues to use _SENTINEL_STRINGS (no regression)
  • Empty/whitespace strings are always included in the mask regardless of declaration
  • SplitImbalanceWarning fires correctly when sentinel-based missingness lands disproportionately in one split, using declared sentinels
  • Columns of other dtypes are completely unaffected
  • Unit tests covering the replace-semantics path in the relevant split-signal test file
  • Docstrings updated per ADR-0034 for any modified in-scope functions

Blocked by

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions