Skip to content

feat: extend _resolve_effective_nulls to normalize user-declared numeric sentinels #172

@DEVunderdog

Description

@DEVunderdog

Parent

#94

What to build

Extend _resolve_effective_nulls in utils/_null_normalization.py to accept an optional numeric_sentinels: dict[str, list[float]] | None parameter (default None).

For each column present in both the DataFrame and the sentinels dict, if the column's dtype passes _numeric_sentinel_eligible, append Polars expressions that replace each matching sentinel value with pl.lit(None, dtype=dtype). The function remains pure and side-effect-free; sentinel expressions are appended to the existing expression list alongside the existing string-sentinel and Inf/NaN rules.

When numeric_sentinels is None or {}, the function is strictly identical to its current behaviour — no existing caller is broken.

Acceptance criteria

  • _resolve_effective_nulls signature is (df, numeric_sentinels=None)
  • Int64 column: declared sentinel value converted to null; non-sentinel values unchanged
  • Float64 column: declared sentinel value converted to null; existing NaN/Inf rules still apply independently in the same call
  • Multiple sentinels declared for one column: all converted; any other value unchanged
  • Sentinel declared for a column not present in the DataFrame: no error, rest of DataFrame unchanged
  • Sentinel value that matches no rows: DataFrame unchanged (no-op, no error)
  • Column with no declared sentinel: completely unchanged even if its values happen to equal a sentinel declared for another column
  • numeric_sentinels=None: output is byte-for-byte identical to calling with no argument
  • All cases above covered by unit tests in tests/unit/imputation/test_null_normalization.py
  • Docstring updated per ADR-0034

Blocked by

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions