Skip to content

Feature() crashes on countries that declare a table but ship no microdata (e.g. Nepal) #492

@ligon

Description

@ligon

Problem

ll.Feature('individual_education')() (unscoped — all declaring countries) raises instead of returning a partial assembly:

make: *** No rule to make target '.../Nepal/var/individual_education.parquet'.  Stop.
Data file .../Nepal/var/individual_education.parquet still missing after running fallbacks.

Nepal declares individual_education in _/data_scheme.yml but ships no source microdata (see the Countries Without Microdata table in CLAUDE.md: Nepal, Armenia, Timor-Leste 2001, Guatemala). The cross-country Feature assembler tries to build every declaring country and dies on the first with no data, taking the whole assembly down.

Impact

Any Feature(name)() call where ≥1 declaring country lacks buildable data fails wholesale — not specific to education. Consumers must manually enumerate the country subset that has data, defeating the no-argument convenience call.

Suggested fix

In the Feature assembly path (and/or diagnostics.load_feature), catch the missing-data / "still missing after fallbacks" error per country, emit a warning, and continue assembling the rest. Optionally record skipped countries in the result .attrs or a log line.

Evidence

Found while validating the #171 education harmonization via Feature('individual_education'). Scoping the call to the 14 countries that have data works perfectly (1.26M rows, fully canonical).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions