98 lines
4.2 KiB
Markdown
98 lines
4.2 KiB
Markdown
# InterferenceETL
|
|
|
|
InterferenceETL processes the newest complete hour of interference KPI files from Metrix storage. It reads seven ZIP/XLSX source types, converts each `Sheet0` to CSV, and creates one merged interference summary.
|
|
|
|
## Output
|
|
|
|
Each run writes one window directory:
|
|
|
|
```text
|
|
output/2026073110001100/
|
|
├── converted/
|
|
│ ├── 5G下FDD干扰监控_2026073110001100.csv
|
|
│ ├── 5G干扰监控_2026073110001100.csv
|
|
│ └── ... seven source CSV files
|
|
├── interference_summary_2026073110001100.csv
|
|
└── manifest.json
|
|
```
|
|
|
|
The summary columns are:
|
|
|
|
```text
|
|
hour_start,hour_end,source_type,cgi,cell_name,interference_dbm,source_path
|
|
```
|
|
|
|
The script never modifies or deletes source storage files. By default it writes the complete selected hour to MySQL and keeps only that hour in the target table.
|
|
|
|
The database table contains one time column, `metric_time DATETIME`, which is the source KPI start time. A same-hour rerun replaces that whole batch. After a successful insert, rows for every other hour are deleted in the same transaction. The script refuses to replace a newer database hour with an older source hour.
|
|
|
|
CGI is generated with fixed rules:
|
|
|
|
- NR (`5G干扰监控`, `700M干扰监控`): `{gNBplmn}-{gNBId}-{cellId}`.
|
|
- 4G (the other five source types): `460-00-{eNodeBID}-{小区ID}`, using each source schema's actual equivalent column names.
|
|
|
|
MySQL connection values, database `metrix`, and table `interference_hourly_summary` are constants at the top of `main.py`; they are not project environment variables.
|
|
|
|
## Local development
|
|
|
|
```powershell
|
|
python -m pip install -r requirements.txt
|
|
python -m unittest discover -s tests -v
|
|
python main.py --source-dir C:\path\to\mock-or-exported-tree --output-dir output --no-database
|
|
```
|
|
|
|
Use `--window 2026073110001100` to require one exact source window. Without it, the script selects the newest window containing all seven source types and falls back if the newest observed window is incomplete.
|
|
|
|
## Metrix Script Management
|
|
|
|
The target server is offline. Build the dependency image locally, then generate a self-contained Script Management upload ZIP:
|
|
|
|
```powershell
|
|
docker build -t interference-etl-runtime:1.1 .
|
|
python scripts\build_offline_package.py
|
|
```
|
|
|
|
The generated `dist/InterferenceETL-offline-1.1.zip` contains `main.py`, a ZIP execution entry point, and `openpyxl`, `et_xmlfile`, and `PyMySQL`. The builder verifies both imports and ZIP execution using the standard `python:3.13.11-slim` image, so the uploaded workspace does not need an online `pip install` or a custom runtime image.
|
|
|
|
Create the project, then upload and extract the ZIP in its Script Management workspace. The normal file tree should contain `main.py` at `/workspace/main.py`. Use these project settings:
|
|
|
|
```text
|
|
Name: InterferenceETL
|
|
Language: python
|
|
Base image: python:3.13.11-slim
|
|
Network: bridge
|
|
Run command: python main.py
|
|
Timeout: 1800 seconds
|
|
```
|
|
|
|
If the server keeps the ZIP as one file instead of extracting it, use `python InterferenceETL-offline-1.1.zip` as the run command. The ZIP contains `__main__.py` for this mode.
|
|
|
|
Configure only the Metrix source settings in the project environment:
|
|
|
|
```json
|
|
{
|
|
"METRIX_API_BASE_URL": "http://172.17.0.1:8000",
|
|
"METRIX_API_TOKEN": "<Metrix API token>",
|
|
"METRIX_STORAGE_ID": "stg_4d9a910d72",
|
|
"INTERFERENCE_SOURCE_ROOT": "/网优日常优化数据文档/(勿删)干扰定时小时指标",
|
|
"INTERFERENCE_OUTPUT_DIR": "/workspace/output",
|
|
"INTERFERENCE_LOOKBACK_DAYS": "3"
|
|
}
|
|
```
|
|
|
|
`172.17.0.1` is the current Linux Docker default-bridge gateway used by the script to reach both the Metrix API and the host-published MySQL port. `ShareMySQL` itself does not resolve from the script project's bridge network.
|
|
|
|
## CLI options
|
|
|
|
```text
|
|
--source-dir PATH Read a local directory tree instead of Metrix API
|
|
--output-dir PATH Output root, default output or INTERFERENCE_OUTPUT_DIR
|
|
--window WINDOW Require an exact 16-digit source window
|
|
--lookback-days N Number of newest date directories scanned, default 3
|
|
--api-base URL Metrix API base URL
|
|
--api-token TOKEN Metrix API token
|
|
--storage-id ID Metrix storage connection ID
|
|
--root PATH Source directory in Metrix storage
|
|
--no-database Generate CSV files without writing MySQL
|
|
```
|