103 lines
4.5 KiB
Markdown
103 lines
4.5 KiB
Markdown
# InterferenceETL
|
|
|
|
InterferenceETL processes the newest complete hour of interference KPI files from Metrix storage. It reads seven ZIP/XLSX source types, converts each `Sheet0` to CSV, and creates one merged interference summary.
|
|
|
|
## Output
|
|
|
|
Each run writes one window directory:
|
|
|
|
```text
|
|
output/2026073110001100/
|
|
├── converted/
|
|
│ ├── 5G下FDD干扰监控_2026073110001100.csv
|
|
│ ├── 5G干扰监控_2026073110001100.csv
|
|
│ └── ... seven source CSV files
|
|
├── interference_summary_2026073110001100.csv
|
|
└── manifest.json
|
|
```
|
|
|
|
The summary columns are:
|
|
|
|
```text
|
|
hour_start,hour_end,source_type,cgi,cell_name,interference_dbm,source_path
|
|
```
|
|
|
|
The script never modifies or deletes source storage files. By default it writes the complete selected hour to MySQL and keeps only that hour in the target table.
|
|
|
|
The database table contains one time column, `metric_time DATETIME`, which is the source KPI start time. A same-hour rerun replaces that whole batch. After a successful insert, rows for every other hour are deleted in the same transaction. The script refuses to replace a newer database hour with an older source hour.
|
|
|
|
## Local development
|
|
|
|
```powershell
|
|
python -m pip install -r requirements.txt
|
|
python -m unittest discover -s tests -v
|
|
python main.py --source-dir C:\path\to\mock-or-exported-tree --output-dir output --no-database
|
|
```
|
|
|
|
Use `--window 2026073110001100` to require one exact source window. Without it, the script selects the newest window containing all seven source types and falls back if the newest observed window is incomplete.
|
|
|
|
## Metrix Script Management
|
|
|
|
The target server is offline. Build the dependency image locally, then generate a self-contained Script Management upload ZIP:
|
|
|
|
```powershell
|
|
docker build -t interference-etl-runtime:1.1 .
|
|
python scripts\build_offline_package.py
|
|
```
|
|
|
|
The generated `dist/InterferenceETL-offline-1.1.zip` contains `main.py` plus `openpyxl`, `et_xmlfile`, and `PyMySQL`. The builder verifies imports using the standard `python:3.13.11-slim` image, so the uploaded workspace does not need an online `pip install` or a custom runtime image. The ZIP is 325,932 bytes with SHA-256 `D57930E36B3C70DF7D209C4A83546FEA3B73F87022910D1FDB548D6BED71675D`.
|
|
|
|
Create the project, then upload and extract the ZIP in its Script Management workspace. Use these project settings:
|
|
|
|
```text
|
|
Name: InterferenceETL
|
|
Language: python
|
|
Base image: python:3.13.11-slim
|
|
Network: bridge
|
|
Run command: python main.py
|
|
Timeout: 1800 seconds
|
|
```
|
|
|
|
Configure project environment variables in Metrix instead of committing secrets:
|
|
|
|
```json
|
|
{
|
|
"METRIX_API_BASE_URL": "http://172.17.0.1:8000",
|
|
"METRIX_API_TOKEN": "<Metrix API token>",
|
|
"METRIX_STORAGE_ID": "stg_4d9a910d72",
|
|
"INTERFERENCE_SOURCE_ROOT": "/网优日常优化数据文档/(勿删)干扰定时小时指标",
|
|
"INTERFERENCE_OUTPUT_DIR": "/workspace/output",
|
|
"INTERFERENCE_LOOKBACK_DAYS": "3",
|
|
"INTERFERENCE_PLMN": "460-00",
|
|
"INTERFERENCE_DB_HOST": "<reachable MySQL host>",
|
|
"INTERFERENCE_DB_PORT": "3306",
|
|
"INTERFERENCE_DB_USER": "<MySQL user>",
|
|
"INTERFERENCE_DB_PASSWORD": "<MySQL password>",
|
|
"INTERFERENCE_DB_NAME": "<database name>",
|
|
"INTERFERENCE_DB_TABLE": "interference_hourly_summary"
|
|
}
|
|
```
|
|
|
|
`172.17.0.1` is the current Linux Docker default-bridge gateway used to reach the Metrix host port. Verify both API and MySQL addresses before creating the online schedule because Docker network configuration can differ by server. A container name such as `ShareMySQL` resolves only when the script container joins the same Docker network.
|
|
|
|
## CLI options
|
|
|
|
```text
|
|
--source-dir PATH Read a local directory tree instead of Metrix API
|
|
--output-dir PATH Output root, default output or INTERFERENCE_OUTPUT_DIR
|
|
--window WINDOW Require an exact 16-digit source window
|
|
--lookback-days N Number of newest date directories scanned, default 3
|
|
--api-base URL Metrix API base URL
|
|
--api-token TOKEN Metrix API token
|
|
--storage-id ID Metrix storage connection ID
|
|
--root PATH Source directory in Metrix storage
|
|
--plmn PLMN LTE/SDR CGI PLMN prefix, default 460-00
|
|
--no-database Generate CSV files without writing MySQL
|
|
--db-host HOST MySQL host or INTERFERENCE_DB_HOST
|
|
--db-port PORT MySQL port, default 3306
|
|
--db-user USER MySQL user
|
|
--db-password VALUE MySQL password; prefer the environment variable
|
|
--db-name NAME MySQL database name
|
|
--db-table NAME Table name, default interference_hourly_summary
|
|
```
|