InterferenceETL/aidocs/project_context.md

InterferenceETL project context

2026-07-31: Initial hourly interference pipeline

• This repository is developed as the InterferenceETL submodule under Metrix and is intended to run later in Metrix Script Management on an hourly schedule.
• main.py reads the configured Metrix SFTP storage through Metrix API Token authentication or a local directory for tests. It selects the newest hour containing all seven known interference source types and falls back from a newer incomplete hour.
• Each selected ZIP must contain exactly one XLSX. The script strictly validates the known Sheet0 header, writes one full UTF-8-BOM CSV per source type, and writes one merged CSV containing hour_start, hour_end, source_type, cgi, cell_name, interference_dbm, and source_path.
• LTE/SDR CGI currently uses 460-00-{node}-{cell}; NR uses masterOperatorId unchanged. This is an explicit assumption pending user confirmation.
• Runs are idempotent at the output-window directory level. Generation happens in a scoped temporary directory, then replaces only the same window below the configured output root. Source storage is never modified.
• Database persistence is intentionally not implemented because the target schema and credentials are pending. manifest.json records input paths, sizes, SHA-256 hashes, row counts, warnings, and generated files for later ingestion auditing.
• The Metrix script container needs openpyxl==3.1.5, bridge networking, python main.py, a reachable METRIX_API_BASE_URL, and METRIX_API_TOKEN injected through project environment settings. Secrets must not be committed.
• Read-only validation against the current Metrix storage selected window 2026073110001100, processed all seven source ZIP files, and produced 1,831 summary rows. Per-source row counts were 7 / 804 / 45 / 86 / 700 / 164 / 25 in EXPECTED_TYPES order; sampled CGI, cell name, and interference values matched the source workbooks.
• The Linux/amd64 runtime image is interference-etl-runtime:1.0. Its offline archive is generated locally at dist/interference-etl-runtime-1.0.tar (46,828,032 bytes, SHA-256 B65B94941BB304CA955224B885582E36BE05E390CBD76FC4C069494352E77746) and remains excluded from Git.
• Mock tests pass on Windows Python and inside the runtime image. They cover complete-hour fallback, strict schema rejection, summary extraction, and cross-midnight window parsing.