Pipeline de Ingestão de Dados

Pipeline de Ingestão de Dados

Ingestion is a roadmap-critical but not yet implemented feature

The project vision depends on continuous and reliable ingestion of legal source content, but the current repository does not yet implement this pipeline.

🚧 Planned Feature — End-to-end ingestion from external legal sources is approved in planning but not present in code.

Planned pipeline stages

Expected stages:

1. fetch source payloads from official providers;
2. normalize and parse legal structure;
3. persist into canonical schema (regulations, document_nodes, etc.);
4. validate integrity and hierarchy consistency;
5. emit freshness and failure telemetry.

Reliability requirements

Ingestion quality should include:

• idempotent retries;
• explicit freshness timestamping;
• deterministic handling of partial failures.

Edge cases to handle

Known legal edge cases include:

• future-validity norms (vacatio legis);
• partial vs total revocation chains;
• overlapping amendments affecting the same article.

Success criteria for first production iteration

A credible first ingestion release should prove:

• reproducible runs;
• no duplicate legal nodes under canonical keys;
• measurable source freshness reporting.