Implement Novo CAGED structured ingestion

# Codex Design Contract: Novo CAGED Structured Ingestion

You are adding the next source to Brazil-RV: Novo CAGED / PDET formal-employment data.

Follow the existing architecture:

```text
raw -> bronze -> source-specific silver -> later gold/research panels
```

Keep this PR narrow. Do not add raw-to-research panels, modeling, backtesting, portfolio logic, event-risk overlays, NLP, PDF parsing, broad scraping, browser automation, or non-Novo-CAGED sources. Do not modify B3/BCB/IBGE/Tesouro/FRED/CVM/ONS/ANP code except for tiny shared config compatibility required by tests.

Read first:

- `AGENTS.md`
- `docs/PROJECT_PLAN.md`
- `docs/TIMING_AND_AVAILABILITY_POLICY.md`
- existing ingestion modules for ANP/ONS/CVM/FRED/Tesouro
- `src/bralpha/infra/`
- `src/bralpha/metadata/`
- `src/bralpha/parsing/common.py`
- `src/bralpha/timing/availability.py`

---

## Source decision

Novo CAGED is useful for our daily Brazil rates/FX/equity RV model because it is the official high-frequency formal-labor-market source. It captures labor-market momentum and wage-pressure context, which can matter for DI/rates, BRL, equities, inflation expectations, and activity regimes.

The earlier priority list explicitly included:

```text
Novo CAGED / formal employment | Yes | Ministério do Trabalho PDET / downloads | Labor-market momentum, wage/inflation pressure
```

### Implement live in this PR

Implement these source-specific silver datasets:

```text
novo_caged_movements_monthly
novo_caged_release_calendar
```

Rationale:

- `novo_caged_movements_monthly` is the core official movement microdata: admissions/dismissals, geography, sector, occupation, worker demographics, wage and contractual-hours fields where present.
- `novo_caged_release_calendar` is small and useful for point-in-time availability metadata. The official MTE calendar page lists expected release dates by reference competence.

### Source-map only in this PR

Add config/source-map entries, but do not implement live normalization yet:

```text
novo_caged_late_declarations_monthly
novo_caged_exclusions_monthly
novo_caged_official_monthly_tables
novo_caged_reference_stock
rais_microdata
legacy_caged_microdata
pdet_powerbi_panel
pdet_online_queries
```

Reasons:

- Late declarations (`CAGEDFOR`) and exclusions (`CAGEDEXC`) are useful for revision analysis and exact official restatements, but they should be handled in a dedicated revision-aware PR. Do not silently fold them into main movement rows in this first PR.
- Official monthly tables are linked from the MTE release page, often through a public Google Drive `Tabelas.xlsx` link. They are useful cross-checks but not deterministic enough for the first ingestion PR.
- Reference stock (`Estoque de Referência`) is useful for level/stock context, but is annual/reference data; source-map first.
- RAIS and legacy CAGED are separate historical/structural datasets and should be separate PRs.
- PDET PowerBI/online queries are useful for human validation but not deterministic ingestion sources.

Do not create fake endpoints for source-map-only datasets.

---

## Official MTE/PDET facts to respect

Official PDET page:

```text
https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho
```

Official Novo CAGED page:

```text
https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/o-pdet/o-que-e-o-novo-caged
```

Official current Novo CAGED release page example:

```text
https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/novo-caged
```

Official release calendar page:

```text
https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/o-pdet/calendario-de-divulgacao-do-novo-caged
```

Official facts:

- The PDET page says Novo CAGED lets users follow employment behavior monthly.
- The same PDET page says microdata for RAIS and CAGED are available in text format.
- The MTE “O que é Novo CAGED?” page says that since January 2020, CAGED has been replaced by eSocial for part of companies, while CAGED remains mandatory for public bodies and international organizations hiring CLT workers.
- The same page says Novo CAGED statistics are generated from eSocial, CAGED, and Empregador Web information.
- The same page says imputation was used during the transition to address missing dismissal information and preserve the quality/integrity of formal-employment statistics.
- The release calendar page lists expected Novo CAGED release dates by reference competence; for 2026 it shows, for example, January 2026 competence released on 2026-03-03 and May 2026 competence scheduled for 2026-06-30.
- The current Novo CAGED monthly page links `Sumário executivo.pdf`, `Apresentação.pdf`, `Tabelas.xlsx`, the PowerBI panel, the technical note, and reference stock.

Operational microdata resource pattern to fixture-test:

```text
https://ftp.mtps.gov.br/pdet/microdados/NOVO%20CAGED/{year}/{yyyymm}/CAGEDMOV{yyyymm}.7z
```

Examples:

```text
https://ftp.mtps.gov.br/pdet/microdados/NOVO%20CAGED/2024/202401/CAGEDMOV202401.7z
https://ftp.mtps.gov.br/pdet/microdados/NOVO%20CAGED/2026/202604/CAGEDMOV202604.7z
```

If MTE changes the FTP path or file naming, fail the resource clearly and write a failed manifest record. Do not infer hidden endpoints silently.

---

## Dependency policy

Novo CAGED microdata is commonly distributed as `.7z` archives containing text files. Python’s standard library does not read 7z archives.

One new dependency is allowed in this PR **only if needed to parse official 7z resources**:

```text
py7zr>=0.21
```

Rules:

- Add no other dependency.
- Use `py7zr` only in `src/bralpha/parsing/novo_caged_tabular.py` or a tiny archive helper.
- Parser tests must cover `.7z` fixture bytes.
- If Codex can parse official 7z files without adding a dependency using an already-present project tool, prefer that; otherwise add `py7zr` and document why.

---

## Files to add

Create only files with real code:

```text
docs/NOVO_CAGED_SOURCE_MAP.md

configs/datasets/novo_caged.yaml
configs/sources/novo_caged.yaml

src/bralpha/ingestion/novo_caged/common.py
src/bralpha/ingestion/novo_caged/downloads.py
src/bralpha/ingestion/novo_caged/resources.py

src/bralpha/parsing/novo_caged_tabular.py
src/bralpha/normalization/novo_caged_labor.py
src/bralpha/pipelines/novo_caged_ingest.py

tests/test_novo_caged_source_map.py
tests/test_novo_caged_resources.py
tests/test_novo_caged_downloads.py
tests/test_novo_caged_tabular_parsing.py
tests/test_novo_caged_normalization.py
tests/test_novo_caged_ingest_pipeline.py
```

Do not create empty modules.

Update `src/bralpha/infra/config.py` only as needed to add:

```python
load_novo_caged_dataset_registry(repo_root)
```

If `py7zr` is added, update only:

```text
pyproject.toml
```

with exactly that dependency.

---

## Storage layout

Use existing conventions:

```text
data/raw/novo_caged/{dataset_id}/{download_date}/
data/bronze/novo_caged/{dataset_id}/
data/silver/{dataset_id}/
data/manifests/novo_caged/downloads.jsonl
```

Do not commit downloaded data.

---

## Config: datasets

Create:

```text
configs/datasets/novo_caged.yaml
```

Minimum top-level shape:

```yaml
source: novo_caged
free_access_default: true
requires_auth_default: false
point_in_time_required: true

raw_storage:
  path_template: data/raw/novo_caged/{dataset_id}/{download_date}/
  manifest_path: data/manifests/novo_caged/downloads.jsonl

bronze_storage:
  path_template: data/bronze/novo_caged/{dataset_id}/
  format: parquet

silver_storage:
  path_template: data/silver/{dataset_id}/
  format: parquet
```

Datasets:

```yaml
datasets:
  - dataset_id: novo_caged_movements_monthly
    source: novo_caged
    priority: P0
    frequency: monthly
    raw_format: 7z_txt
    canonical_table: novo_caged_movement_record
    partition_keys: [year]
    primary_keys: [movement_record_id]
    quality_checks:
      - required_columns_present
      - no_duplicate_primary_keys
      - ref_date_not_null
      - available_date_not_null
      - available_date_on_or_after_ref_date
    source_map_status: live_download
    availability_policy: novo_caged_conservative_next_month_end_plus_2bd
    resource_families:
      - name: movement_records
        record_kind: movement
        mode: monthly_7z
        applies_from: "2020-01-01"
        url_template: "https://ftp.mtps.gov.br/pdet/microdados/NOVO%20CAGED/{year}/{period}/CAGEDMOV{period}.7z"
        filename_template: "CAGEDMOV{period}.7z"
    notes: Monthly Novo CAGED movement microdata. Preserve official movement rows; do not aggregate in ingestion.

  - dataset_id: novo_caged_release_calendar
    source: novo_caged
    priority: P0
    frequency: annual_calendar
    raw_format: html
    canonical_table: novo_caged_release_calendar
    partition_keys: [release_year]
    primary_keys: [ref_date]
    quality_checks:
      - required_columns_present
      - no_duplicate_primary_keys
      - ref_date_not_null
      - available_date_not_null
      - available_date_on_or_after_ref_date
    source_map_status: live_download
    source_urls:
      - name: official_release_calendar
        url_template: "https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/o-pdet/calendario-de-divulgacao-do-novo-caged"
        filename_template: "novo_caged_release_calendar.html"
    notes: Official release-calendar page. Current-year calendar only unless historical calendars are explicitly discoverable.

  - dataset_id: novo_caged_late_declarations_monthly
    source: novo_caged
    priority: P1
    frequency: monthly
    raw_format: 7z_txt
    canonical_table: novo_caged_late_declaration_record
    partition_keys: [year]
    primary_keys: [late_declaration_record_id]
    quality_checks: [required_columns_present, no_duplicate_primary_keys]
    source_map_status: source_map_only_revision_files
    source_urls: []
    notes: Likely CAGEDFOR monthly archive family. Useful for revisions/late declarations; defer to revision-aware PR.

  - dataset_id: novo_caged_exclusions_monthly
    source: novo_caged
    priority: P1
    frequency: monthly
    raw_format: 7z_txt
    canonical_table: novo_caged_exclusion_record
    partition_keys: [year]
    primary_keys: [exclusion_record_id]
    quality_checks: [required_columns_present, no_duplicate_primary_keys]
    source_map_status: source_map_only_revision_files
    source_urls: []
    notes: Likely CAGEDEXC monthly archive family. Useful for revisions/exclusions; defer to revision-aware PR.

  - dataset_id: novo_caged_official_monthly_tables
    source: novo_caged
    priority: P1
    frequency: monthly
    raw_format: xlsx_google_drive_link
    canonical_table: novo_caged_official_monthly_table
    partition_keys: [year]
    primary_keys: [ref_date, table_name, row_key]
    quality_checks: [required_columns_present, no_duplicate_primary_keys]
    source_map_status: source_map_only_official_xlsx_crosscheck
    source_urls: []
    notes: Official monthly page links Tabelas.xlsx, often through Google Drive. Useful cross-check; defer until deterministic fetch is designed.

  - dataset_id: novo_caged_reference_stock
    source: novo_caged
    priority: P2
    frequency: annual_or_reference
    raw_format: html_or_xlsx_pending
    canonical_table: novo_caged_reference_stock
    partition_keys: [year]
    primary_keys: [reference_year, geography_key, sector_key]
    quality_checks: [required_columns_present, no_duplicate_primary_keys]
    source_map_status: source_map_only_reference
    source_urls: []
    notes: Official reference-stock communication/data. Useful for level context; defer.

  - dataset_id: rais_microdata
    source: pdet
    priority: P2
    frequency: annual
    raw_format: txt_archive_large
    canonical_table: rais_microdata_record
    partition_keys: [year]
    primary_keys: [rais_record_id]
    quality_checks: [required_columns_present, no_duplicate_primary_keys]
    source_map_status: source_map_only_separate_source
    source_urls: []
    notes: RAIS is structural annual labor-market data, not Novo CAGED ingestion.

  - dataset_id: legacy_caged_microdata
    source: pdet
    priority: P2
    frequency: monthly_historical
    raw_format: txt_archive_large
    canonical_table: legacy_caged_record
    partition_keys: [year]
    primary_keys: [legacy_caged_record_id]
    quality_checks: [required_columns_present, no_duplicate_primary_keys]
    source_map_status: source_map_only_legacy_source
    source_urls: []
    notes: Pre-2020 CAGED history. Useful later for longer labor history; defer.

  - dataset_id: pdet_powerbi_panel
    source: pdet
    priority: P3
    frequency: dashboard
    raw_format: powerbi
    canonical_table: pdet_powerbi_snapshot
    partition_keys: [snapshot_year]
    primary_keys: [snapshot_date, panel_id]
    quality_checks: [required_columns_present, no_duplicate_primary_keys]
    source_map_status: source_map_only_human_validation
    source_urls: []
    notes: Dashboard is useful for validation, not deterministic ingestion.
```

---

## Config: source metadata

Create:

```text
configs/sources/novo_caged.yaml
```

Minimum:

```yaml
source: novo_caged
portal:
  name: Ministério do Trabalho e Emprego / PDET
  url: https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho
  license_note: Brazilian federal open-data terms; cite MTE/PDET and dataset/source.

official_pages:
  - name: pdet_home
    url: https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho
  - name: what_is_novo_caged
    dataset_id: novo_caged_movements_monthly
    url: https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/o-pdet/o-que-e-o-novo-caged
  - name: release_calendar
    dataset_id: novo_caged_release_calendar
    url: https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/o-pdet/calendario-de-divulgacao-do-novo-caged
  - name: current_release_page
    dataset_id: novo_caged_official_monthly_tables
    url: https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/novo-caged
  - name: microdata_page
    dataset_id: novo_caged_movements_monthly
    url: https://www.gov.br/trabalho-e-emprego/pt-br/assuntos/estatisticas-trabalho/microdados-rais-e-caged

endpoint_policy:
  live_download_requires:
    - stable_direct_url_or_official_page_link_discovery
    - no_login
    - no_browser_automation
    - fixture_test
    - manifest_failure_on_404
```

---

## Source map doc

Create:

```text
docs/NOVO_CAGED_SOURCE_MAP.md
```

Must include table columns:

```text
dataset_id
priority
status
source_page_or_endpoint
raw_format
expected_frequency
silver_output
known_limitations
```

Document:

- `novo_caged_movements_monthly` is live and uses monthly CAGEDMOV archives.
- `novo_caged_release_calendar` is live and uses the official MTE release-calendar page.
- Late declarations/exclusions are source-map-only revision files.
- Official monthly tables are source-map-only cross-checks because the current page links an XLSX through Google Drive.
- RAIS and legacy CAGED are separate sources and not part of this PR.
- No aggregation, no net-admission features, no wage growth, no seasonality, no rolling statistics, no model/backtest logic in ingestion.

---

## Resource generation

Create:

```text
src/bralpha/ingestion/novo_caged/resources.py
```

Implement:

```python
@dataclass(frozen=True)
class NovoCagedResourceRequest:
    dataset_id: str
    resource_name: str
    url: str
    filename: str
    period: str | None = None  # YYYYMM
    year: int | None = None
    month: int | None = None
    record_kind: str | None = None


def novo_caged_monthly_resources(dataset_config, start: date, end: date) -> list[NovoCagedResourceRequest]:
    ...


def novo_caged_release_calendar_resource(dataset_config) -> list[NovoCagedResourceRequest]:
    ...
```

Rules:

- `novo_caged_movements_monthly` supports monthly resources only from 2020-01 onward.
- Generate one `CAGEDMOV{YYYYMM}.7z` resource per month intersecting `[start, end]`.
- `period = YYYYMM`.
- Sort resources chronologically.
- Raise on inverted windows.
- Do not request daily files.
- Do not crawl FTP directory listings.
- Release calendar generates one HTML resource from the official configured URL.

Tests:

```text
2019 window returns no resources or raises clear pre-2020 message
2020-01 range -> CAGEDMOV202001.7z
2024-01 to 2024-03 -> three resources
inverted window raises
release calendar resource returns configured HTML URL
```

---

## Downloader

Create:

```text
src/bralpha/ingestion/novo_caged/common.py
src/bralpha/ingestion/novo_caged/downloads.py
```

Reuse existing infrastructure:

```text
HttpClient
RawStore
ManifestWriter
ManifestRecord
sha256 helpers
client_context pattern
```

Implement:

```python
def download_novo_caged_dataset(repo_root, dataset_id, start=None, end=None, client=None, downloaded_at=None):
    ...
```

Rules:

- `novo_caged_movements_monthly` requires `start` and `end`.
- Generate monthly resources through `resources.py`.
- `novo_caged_release_calendar` downloads the configured calendar HTML once per run.
- Source-map-only datasets raise a clear error and write no data.
- Write one raw file per resource.
- Write manifest records for success/failure.
- HTTP 404/5xx must create failure manifest records and no raw file.
- Do not parse inside downloader.
- Reuse one HTTP client per run.
- No browser automation.

---

## Parser

Create:

```text
src/bralpha/parsing/novo_caged_tabular.py
```

Support:

```text
7z_txt
txt
html
```

### Movement bronze

Bronze columns for `novo_caged_movements_monthly`:

```text
row_index
inner_filename
resource_name
record_kind
period
year
month
all raw_<normalized_source_column> fields
source
source_dataset
download_timestamp_utc
raw_path
sha256
```

Rules:

- Read text data as strings first.
- Use UTF-8 with fallback to Latin-1/CP1252.
- Detect semicolon/comma/tab delimiters, but default to semicolon for PDET text files.
- Preserve all source columns as `raw_<normalized_source_column>`.
- Do not create per-row `raw_fields_json`.
- Keep archive `inner_filename` lineage.
- Parse `.7z` members one at a time if practical.
- Use Polars.
- Do not parse XLSX/PDF.

### Release calendar bronze

For `novo_caged_release_calendar`, preserve:

```text
row_index
raw_text
source
source_dataset
download_timestamp_utc
raw_path
sha256
```

It is acceptable to parse the small HTML calendar directly in the calendar normalizer instead of creating a wide raw table.

---

## Normalization

Create:

```text
src/bralpha/normalization/novo_caged_labor.py
```

### Shared timing policy

Use:

```text
availability_policy = novo_caged_conservative_next_month_end_plus_2bd
```

For movement records:

```text
ref_date = last calendar day of competence month
available_date = add_business_days(last_business_day(next_month), 2)
```

Rationale:

- The official calendar shows release dates around the end of the following month, but some months can be early in the second following month.
- This conservative fallback avoids leakage when a historical release calendar is not available.
- Do not use `download_timestamp_utc` as historical availability.

The release-calendar silver table should preserve official release dates. Later raw-to-research work can prefer official calendar availability where complete.

### `novo_caged_movements_monthly` silver

Output:

```text
data/silver/novo_caged_movements_monthly/
```

Required columns:

```text
movement_record_id
ref_date
available_date
availability_policy
competence
year
month
record_kind
region
state
municipality_code
cnae_section
cnae_subclass
occupation_code
movement_type_code
movement_sign
employment_category
education_degree
age
sex
race_color
disability_type
employer_type
establishment_type
establishment_size_jan
contract_hours
wage
wage_unit
is_apprentice
is_intermittent
is_part_time
source_system
raw_competenciamov
raw_saldomovimentacao
raw_tipomovimentacao
raw_salario
raw_valorsalariofixo
source
source_dataset
download_timestamp_utc
raw_path
sha256
source_version
```

Alias mapping must be robust to accent/no-accent PDET headers. Match at least these aliases where present:

```text
competenciamov / competência / competencia -> competence
regiao / região -> region
uf -> state
municipio / município -> municipality_code
secao / seção -> cnae_section
subclasse -> cnae_subclass
cbo2002ocupacao / cbo2002ocupação / cbo -> occupation_code
saldomovimentacao / saldomovimentação -> movement_sign
tipomovimentacao / tipomovimentação -> movement_type_code
categoria -> employment_category
graudeinstrucao / grau de instrução -> education_degree
idade -> age
sexo -> sex
racacor / raça cor / raça_cor -> race_color
tipodedeficiencia / tipo de deficiência -> disability_type
tipoempregador -> employer_type
tipoestabelecimento -> establishment_type
tamestabjan -> establishment_size_jan
horascontratuais -> contract_hours
salario / salário -> wage
unidadesalariocodigo / unidade salario codigo -> wage_unit
valorsalariofixo -> optional raw wage value fallback
indicadoraprendiz -> is_apprentice
indtrabintermitente -> is_intermittent
indtrabparcial -> is_part_time
origemdainformacao / origem da informação -> source_system
```

Rules:

- Preserve `movement_sign` exactly as official signed value where available, typically `+1` for admission and `-1` for dismissal.
- Preserve `movement_type_code`; do not infer admission/dismissal categories beyond the official sign/code.
- Preserve wage and hours as official numeric observations.
- Do not compute admissions, dismissals, net employment, wage growth, sector totals, unemployment proxies, rolling values, seasonal adjustments, or labels in silver.
- `movement_record_id` must be deterministic from stable lineage and official row content:
  - source_dataset
  - resource_name
  - inner_filename
  - period
  - record_kind
  - row_index
  - competence
  - municipality_code
  - cnae_subclass
  - occupation_code
  - movement_type_code
  - movement_sign
  - wage
- Do not use `raw_path`, `download_timestamp_utc`, or `sha256` in the primary-key hash.
- Primary key: `movement_record_id`.
- Partition by `year` from `ref_date`.

### `novo_caged_release_calendar` silver

Output:

```text
data/silver/novo_caged_release_calendar/
```

Required columns:

```text
ref_date
release_date
available_date
availability_policy
release_year
competence_label
source
source_dataset
download_timestamp_utc
raw_path
sha256
source_version
```

Rules:

- Parse official calendar rows such as:
  - `03/03/2026 - Competência: janeiro de 2026`
  - `30/06/2026 - Competência: maio de 2026`
- `ref_date = month-end of the competence month`.
- `release_date = listed release date`.
- `available_date = usable_date_from_date_only(release_date)`.
- `availability_policy = novo_caged_official_release_calendar`.
- If no date rows are found, raise a clear error in explicit calendar ingestion.
- Primary key: `ref_date`.
- Partition by `release_year` or `year` if the shared writer only supports `year`; prefer `release_year` if straightforward.

---

## Pipeline

Create:

```text
src/bralpha/pipelines/novo_caged_ingest.py
```

CLI:

```bash
python -m bralpha.pipelines.novo_caged_ingest \
  --repo-root . \
  --dataset novo_caged_movements_monthly \
  --start 2020-01-01 \
  --end 2026-01-01
```

Release calendar:

```bash
python -m bralpha.pipelines.novo_caged_ingest \
  --repo-root . \
  --dataset novo_caged_release_calendar
```

Behavior:

```text
download raw -> parse bronze -> normalize silver -> quality checks
```

Rules:

- Do not run source-map-only datasets.
- `novo_caged_movements_monthly` requires `start` and `end`.
- Process monthly archives incrementally:
  - for each successful raw file: parse -> write bronze -> normalize -> date-filter -> quality-check -> write silver.
- Do not concatenate all Novo CAGED history into memory.
- Date filters should be applied after normalization.
- Idempotent reruns must not duplicate silver primary keys.
- Source-map-only dataset failure must write no data.

---

## Efficiency requirements

Novo CAGED movement microdata can be large.

Required:

- Generate one resource per month, not one resource per geography/sector.
- Parse/process one raw resource at a time.
- Do not create per-row JSON blobs.
- Use Polars.
- Partition bronze/silver by `year`.
- Avoid row-wise Python loops over movement microdata.
- Keep row-level movement records in silver; aggregation belongs in raw-to-research.
- Do not pivot wide.

---

## Tests

Use fixtures/mocks only. No live MTE/PDET calls.

### Source map/config

- `configs/datasets/novo_caged.yaml` loads.
- `configs/sources/novo_caged.yaml` loads.
- `docs/NOVO_CAGED_SOURCE_MAP.md` lists every configured dataset.
- Live datasets have source URLs/templates.
- Source-map-only datasets have empty `source_urls` or clear source-map-only status.
- No fake endpoints for deferred datasets.

### Resource generation

- 2020-01 generates `CAGEDMOV202001.7z` with period `202001`.
- Multi-month 2024-01 to 2024-03 generates three monthly resources.
- 2019-only range produces no resources or a clear pre-2020 error.
- Inverted windows raise.
- Release calendar resource points to the official MTE calendar page.

### Downloader

- Mocked monthly movement resources write raw files and manifests.
- Mocked release-calendar HTML writes raw file and manifest.
- HTTP failure writes failure manifest and no raw file.
- Source-map-only dataset raises clear error and writes no data.
- Downloader does not parse.

### Parser

- Delimited TXT fixture reads as strings.
- UTF-8 and Latin-1 fixtures parse.
- `.7z` fixture with one TXT member parses and preserves `inner_filename`.
- Raw columns become `raw_<normalized_name>`.
- `resource_name`, `period`, `year`, `month`, `record_kind`, and lineage are preserved.
- No `raw_fields_json`.

### Normalizers

Movement fixture:

- Maps competence, UF, municipality, CNAE section/subclass, CBO occupation, movement sign/type, wage, hours, age, sex, race/color, education, and source system where present.
- Handles accented and unaccented header variants.
- Computes conservative `available_date` without using download timestamp.
- Creates deterministic `movement_record_id` independent of raw path / timestamp / sha256.
- Preserves movement sign and wage separately.
- Does not compute admissions, dismissals, net jobs, wage growth, sector totals, or rolling features.

Release calendar fixture:

- Parses official-style lines:
  - `03/03/2026 - Competência: janeiro de 2026`
  - `30/06/2026 - Competência: maio de 2026`
- Computes month-end `ref_date` and listed `release_date`.
- Uses `available_date = usable_date_from_date_only(release_date)`.
- Primary keys are unique.

### Pipeline

- Mocked two-month movement raw -> bronze -> silver works resource-incrementally.
- Silver is partitioned by year.
- Idempotent rerun does not duplicate `movement_record_id`.
- Release calendar raw -> bronze -> silver works.
- Source-map-only dataset fails clearly and writes no data.

Do not add static banned-feature denylist tests.

---

## Acceptance criteria

The PR is complete when:

1. `docs/NOVO_CAGED_SOURCE_MAP.md` exists.
2. `configs/datasets/novo_caged.yaml` exists.
3. `configs/sources/novo_caged.yaml` exists.
4. `novo_caged_movements_monthly` live ingestion works with mocked 7z/TXT resources.
5. `novo_caged_release_calendar` live ingestion works with mocked official calendar HTML.
6. Late declarations, exclusions, official tables, reference stock, RAIS, legacy CAGED, PowerBI, and online queries are source-mapped/deferred with no fake endpoints.
7. Raw, bronze, silver, manifests, and quality checks work end to end.
8. Strict timing policy is respected; no download timestamp is used as historical availability.
9. Movement microdata remains row-level in silver; no aggregation in ingestion.
10. No admissions/dismissals/net-job aggregates, wage-growth features, ratios, rolling features, seasonal adjustments, labels, or portfolio/backtest features are added.
11. Only the justified `py7zr` dependency is added if necessary for official 7z parsing; otherwise no new dependency is added.
12. No XLSX/PDF/browser/scraping workflow is added.
13. `python -m pytest tests/test_novo_caged_source_map.py tests/test_novo_caged_resources.py tests/test_novo_caged_downloads.py tests/test_novo_caged_tabular_parsing.py tests/test_novo_caged_normalization.py tests/test_novo_caged_ingest_pipeline.py` passes.
14. `python -m pytest` passes.
15. `python -m ruff check .` passes.
16. No generated data is committed.

Keep the implementation deterministic, monthly-resource-based, point-in-time safe, row-level, and efficient.

Implement Novo CAGED structured ingestion #42

Description

Codex Design Contract: Novo CAGED Structured Ingestion

Source decision

Implement live in this PR

Source-map only in this PR

Official MTE/PDET facts to respect

Dependency policy

Files to add

Storage layout

Config: datasets

Config: source metadata

Source map doc

Resource generation

Downloader

Parser

Movement bronze

Release calendar bronze

Normalization

Shared timing policy

novo_caged_movements_monthly silver

novo_caged_release_calendar silver

Pipeline

Efficiency requirements

Tests

Source map/config

Resource generation

Downloader

Parser

Normalizers

Pipeline

Acceptance criteria

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`novo_caged_movements_monthly` silver

`novo_caged_release_calendar` silver