feat: 改用Metrix API写入干扰数据

.gitignore

@@ -3,5 +3,6 @@ __pycache__/
 .pytest_cache/
 .venv/
 .env
+runtime_config.py
 output/
 dist/

README.md

@@ -24,7 +24,7 @@ The summary columns are:
 hour_start,hour_end,cgi,cell_name,interference_dbm,longitude,latitude
 ```
 
-The script never modifies or deletes source storage files. By default it writes the complete selected hour to MySQL and keeps only that hour in the target table.
+The script never modifies or deletes source storage files. By default it writes the complete selected hour through the Metrix Database API and keeps only that hour in the target table.
 
 The database table contains one time column, `metric_time DATETIME`, which is the source KPI start time, plus nullable `longitude` and `latitude` columns. A same-hour rerun replaces that whole batch. After a successful insert, rows for every other hour are deleted in the same transaction. The script refuses to replace a newer database hour with an older source hour.
 
@@ -33,7 +33,7 @@ CGI is generated with fixed rules:
 - NR (`5G干扰监控`, `700M干扰监控`): `{gNBplmn}-{gNBId}-{cellId}`.
 - 4G (the other five source types): `460-00-{eNodeBID}-{小区ID}`, using each source schema's actual equivalent column names.
 
-MySQL connection values, database `interference_etl`, and table `interference_hourly_summary` are constants at the top of `main.py`; they are not project environment variables. The script creates its database and table automatically when they do not exist.
+The Metrix API address, API Token, and database connection ID are constants in `runtime_config.py`; they are not project environment variables. Copy `runtime_config.example.py` to `runtime_config.py` and fill in the actual Token and the `conn_id` of `ShareMySQL`. The fixed database is `interference_etl`, the fixed table is `interference_hourly_summary`, and the script creates both automatically when they do not exist.
 
 The five CellData directories are fixed in `CELL_DATA_DIRECTORIES`. In each directory, only XLSX files ending with a valid `YYYYMMDD.xlsx` date are considered, and the latest date is selected.
 
@@ -56,7 +56,7 @@ docker build -t interference-etl-runtime:1.1 .
 python scripts\build_offline_package.py
 ```
 
-The generated `dist/InterferenceETL-offline-1.1.zip` contains `main.py`, a ZIP execution entry point, and a `vendor/` directory containing `openpyxl`, `et_xmlfile`, and `PyMySQL`. The builder verifies both imports and ZIP execution using the standard `python:3.13.11-slim` image, so the uploaded workspace does not need an online `pip install` or a custom runtime image.
+The generated `dist/InterferenceETL-offline-1.1.zip` contains `main.py`, `runtime_config.py`, a ZIP execution entry point, and a `vendor/` directory containing `openpyxl` and `et_xmlfile`. The builder verifies imports and ZIP execution using the standard `python:3.13.11-slim` image, so the uploaded workspace does not need an online `pip install` or a custom runtime image. The build fails when `runtime_config.py` is missing.
 
 Create the project, then upload and extract the ZIP in its Script Management workspace. The normal file tree should contain `main.py` at `/workspace/main.py`. Use these project settings:
 
@@ -71,12 +71,10 @@ Timeout: 1800 seconds
 
 If the server keeps the ZIP as one file instead of extracting it, use `python InterferenceETL-offline-1.1.zip` as the run command. The ZIP contains `__main__.py` for this mode.
 
-Configure only the Metrix source settings in the project environment:
+Configure only run-specific source and output settings in the project environment:
 
 ```json
 {
-  "METRIX_API_BASE_URL": "http://172.17.0.1:8000",
-  "METRIX_API_TOKEN": "<Metrix API token>",
   "METRIX_STORAGE_ID": "stg_4d9a910d72",
   "INTERFERENCE_SOURCE_ROOT": "/网优日常优化数据文档/(勿删)干扰定时小时指标",
   "INTERFERENCE_OUTPUT_DIR": "/workspace/output",
@@ -84,7 +82,7 @@ Configure only the Metrix source settings in the project environment:
 }
 ```
 
-`172.17.0.1` is the current Linux Docker default-bridge gateway used by the script to reach both the Metrix API and the host-published MySQL port. `ShareMySQL` itself does not resolve from the script project's bridge network.
+Both storage reads and database writes use the Metrix API at `http://188.5.127.115:18271`. SSH remains the deployment and operational channel for uploading files, starting runs, and reading logs.
 
 ## CLI options
 
@@ -93,9 +91,7 @@ Configure only the Metrix source settings in the project environment:
 --output-dir PATH      Output root, default output or INTERFERENCE_OUTPUT_DIR
 --window WINDOW        Require an exact 16-digit source window
 --lookback-days N      Number of newest date directories scanned, default 3
---api-base URL         Metrix API base URL
---api-token TOKEN      Metrix API token
 --storage-id ID        Metrix storage connection ID
 --root PATH            Source directory in Metrix storage
---no-database          Generate CSV files without writing MySQL
+--no-database          Generate CSV files without writing the database
 ```

aidocs/project_context.md

@@ -8,9 +8,9 @@
 - NR CGI uses `{gNBplmn}-{gNBId}-{cellId}`. All 4G sources use `460-00-{node}-{cell}`, mapped to the actual node and cell column names in each workbook schema.
 - Runs are idempotent at the output-window directory level. Generation happens in a scoped temporary directory, then replaces only the same window below the configured output root. Source storage is never modified.
 - `manifest.json` records input paths, sizes, SHA-256 hashes, row counts, warnings, generated files, and the database write result for ingestion auditing.
-- The Metrix script container needs `openpyxl==3.1.5`, `PyMySQL==1.1.2`, bridge networking, and `python main.py`. Only Metrix Storage API settings are injected through project environment settings.
+- The Metrix script container needs `openpyxl==3.1.5`, bridge networking, and `python main.py`. Runtime-specific source and output settings may be injected through project environment settings.
 - Read-only validation against the current Metrix storage selected window `2026073110001100`, processed all seven source ZIP files, and produced 1,831 summary rows. Per-source row counts were `7 / 804 / 45 / 86 / 700 / 164 / 25` in `EXPECTED_TYPES` order; sampled CGI, cell name, and interference values matched the source workbooks.
-- `interference-etl-runtime:1.1` is now a local build helper only. Script projects use the standard `python:3.13.11-slim` image and upload `dist/InterferenceETL-offline-1.1.zip`, which vendors `openpyxl`, `et_xmlfile`, and `PyMySQL` at the workspace root.
+- `interference-etl-runtime:1.1` is now a local build helper only. Script projects use the standard `python:3.13.11-slim` image and upload `dist/InterferenceETL-offline-1.1.zip`, which vendors `openpyxl` and `et_xmlfile` at the workspace root.
 - Six Mock tests pass on Windows Python and inside the runtime image. They cover complete-hour fallback, strict schema rejection, summary extraction, cross-midnight window parsing, transactional database replacement, and rollback protection.
 
 ## 2026-07-31: Latest-hour MySQL retention
@@ -46,3 +46,13 @@
 - CellData workbooks use sheet `小区信息表` and required columns `eNB/gNB`, `CI`, `经度`, and `纬度`. CellData CGI is always `460-00-{eNB/gNB}-{CI}` and conflicting coordinates for the same CGI stop the run instead of silently overwriting data.
 - The summary CSV now ends with `longitude,latitude`. The script-owned `interference_hourly_summary` table has nullable `DECIMAL(10,6)` columns with the same names; existing tables are migrated automatically and unmatched CGI values are stored as `NULL`.
 - Read-only validation of the five `20260728` workbooks produced 66,250 unique CGI coordinates with no conflicts. Against the sampled latest-hour summary, 1,529 of 1,539 rows matched (99.35%); the remaining 10 rows correctly stay empty.
+
+## 2026-08-03: Metrix API database output
+
+- Storage reads and database writes now share one API Token authenticated `MetrixApiClient`. Direct PyMySQL access, MySQL host/account/password constants, and the `PyMySQL` offline dependency were removed.
+- `ApiSummaryStore` creates the dedicated `interference_etl` database and `interference_hourly_summary` table through the Database API. It checks the latest stored hour, then uses `/run-script` with `single_session=true` and an explicit transaction to refresh one complete hour and delete all others.
+- API address, Token, and ShareMySQL `conn_id` are constants loaded from ignored `runtime_config.py`; the tracked `runtime_config.example.py` documents the required names. The current API address is `http://188.5.127.115:18271`.
+- Deployment remains SSH based. The Metrix Script Management API is not used for uploads, execution, or log inspection.
+- Eleven containerized unit tests cover the pipeline, CellData enrichment, API database transaction, newer-hour protection, decimal validation, and API failure propagation.
+- SSH deployment and a real Metrix runner execution succeeded with run `eead5c315fb04836a6327be201f561e6`. Window `2026080310001100` produced 1,991 rows, matched 1,977 coordinates, left 14 unmatched, and deleted 1,789 rows from the prior hour.
+- Final API verification found exactly one stored hour (`2026-08-03 10:00:00`) and exactly six columns: `metric_time`, `cgi`, `cell_name`, `interference_dbm`, `longitude`, and `latitude`. The obsolete online source backup and environment-level API URL/Token entries were removed after the successful run.

main.py

@@ -2,6 +2,7 @@ from __future__ import annotations
 
 import argparse
 import csv
+from decimal import Decimal, InvalidOperation
 import hashlib
 import io
 import json
@@ -14,7 +15,7 @@ import re
 import shutil
 import sys
 import time
-from typing import Callable, Protocol
+from typing import Protocol
 from urllib.error import HTTPError, URLError
 from urllib.parse import quote, urlencode
 from urllib.request import Request, urlopen
@@ -27,6 +28,13 @@ sys.path.insert(0, str(VENDOR_DIR))
 
 from openpyxl import load_workbook
 
+try:
+    from runtime_config import METRIX_API_BASE_URL, METRIX_API_TOKEN, METRIX_DATABASE_CONNECTION_ID
+except ImportError:
+    METRIX_API_BASE_URL = "http://188.5.127.115:18271"
+    METRIX_API_TOKEN = ""
+    METRIX_DATABASE_CONNECTION_ID = ""
+
 
 EXPECTED_TYPES = (
     "5G下FDD干扰监控",
@@ -51,10 +59,6 @@ FILE_RE = re.compile(r"^(?P<source_type>.+)_LWP_每小时_过滤110_(?P<window>\
 CELL_DATA_FILE_RE = re.compile(r"(?P<date>20\d{6})\.xlsx$", re.IGNORECASE)
 
 LTE_PLMN = "460-00"
-DATABASE_HOST = "172.17.0.1"
-DATABASE_PORT = 3306
-DATABASE_USER = "root"
-DATABASE_PASSWORD = "OSp!jmgm@26"
 DATABASE_NAME = "interference_etl"
 DATABASE_TABLE = "interference_hourly_summary"
 
@@ -182,10 +186,69 @@ class SummaryStore(Protocol):
     def replace_latest(self, rows: list[dict[str, str]]) -> dict[str, object]: ...
 
 
-class MySQLSummaryStore:
-    def __init__(self, connect_factory: Callable[[], object] | None = None) -> None:
+class MetrixApiClient:
+    def __init__(self, base_url: str, token: str) -> None:
+        if not token:
+            raise ProcessingError("METRIX_API_TOKEN is required for Metrix API access")
+        self.base_url = base_url.rstrip("/")
+        self.token = token
+
+    def get_bytes(self, endpoint: str, query: dict[str, object] | None = None, timeout: int = 30) -> bytes:
+        return self._request("GET", endpoint, query=query, timeout=timeout)
+
+    def get_json(self, endpoint: str, query: dict[str, object] | None = None, timeout: int = 30) -> object:
+        return self._decode_json(self.get_bytes(endpoint, query, timeout))
+
+    def post_json(self, endpoint: str, payload: dict[str, object], timeout: int = 30) -> object:
+        body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
+        return self._decode_json(self._request("POST", endpoint, body=body, timeout=timeout))
+
+    def _request(
+        self,
+        method: str,
+        endpoint: str,
+        query: dict[str, object] | None = None,
+        body: bytes | None = None,
+        timeout: int = 30,
+    ) -> bytes:
+        url = f"{self.base_url}{endpoint}"
+        if query:
+            url = f"{url}?{urlencode(query)}"
+        headers = {"Authorization": f"Bearer {self.token}", "Accept": "application/json"}
+        if body is not None:
+            headers["Content-Type"] = "application/json"
+        request = Request(url, data=body, headers=headers, method=method)
+        last_error: Exception | None = None
+        for attempt in range(3):
+            try:
+                with urlopen(request, timeout=timeout) as response:
+                    return response.read()
+            except HTTPError as exc:
+                detail = exc.read().decode("utf-8", "replace")[:1000]
+                if exc.code < 500:
+                    raise ProcessingError(f"Metrix API {exc.code}: {detail}") from exc
+                last_error = exc
+            except (URLError, TimeoutError) as exc:
+                last_error = exc
+            if attempt < 2:
+                time.sleep(2**attempt)
+        raise ProcessingError(f"Metrix API request failed: {last_error}")
+
+    @staticmethod
+    def _decode_json(payload: bytes) -> object:
+        try:
+            return json.loads(payload.decode("utf-8"))
+        except (UnicodeDecodeError, json.JSONDecodeError) as exc:
+            raise ProcessingError("Metrix API returned invalid JSON") from exc
+
+
+class ApiSummaryStore:
+    def __init__(self, client: MetrixApiClient, connection_id: str) -> None:
+        if not connection_id:
+            raise ProcessingError("METRIX_DATABASE_CONNECTION_ID is required for database output")
+        self.client = client
+        self.connection_id = connection_id
         self.table = DATABASE_TABLE
-        self.connect_factory = connect_factory
 
     def replace_latest(self, rows: list[dict[str, str]]) -> dict[str, object]:
         if not rows:
@@ -194,14 +257,13 @@ class MySQLSummaryStore:
         if len(metric_times) != 1:
             raise ProcessingError("Database batch must contain exactly one metric hour")
         metric_time = next(iter(metric_times))
-        table = f"`{self.table}`"
-        connection = self._connect()
-        cursor = None
-        try:
-            cursor = connection.cursor()
-            cursor.execute(
-                f"""
-                CREATE TABLE IF NOT EXISTS {table} (
+        self._query(
+            f"CREATE DATABASE IF NOT EXISTS `{DATABASE_NAME}` "
+            "CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
+        )
+        self._query(
+            f"""
+                CREATE TABLE IF NOT EXISTS `{self.table}` (
                     metric_time DATETIME NOT NULL COMMENT '指标开始时间',
                     cgi VARCHAR(128) NOT NULL,
                     cell_name VARCHAR(255) NOT NULL,
@@ -210,119 +272,105 @@ class MySQLSummaryStore:
                     latitude DECIMAL(10,6) NULL,
                     PRIMARY KEY (metric_time, cgi)
                 ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
-                """
-            )
-            cursor.execute(f"SHOW COLUMNS FROM {table}")
-            existing_columns = {column[0] for column in cursor.fetchall()}
-            for column in ("longitude", "latitude"):
-                if column not in existing_columns:
-                    cursor.execute(f"ALTER TABLE {table} ADD COLUMN `{column}` DECIMAL(10,6) NULL")
-            cursor.execute(f"SELECT MAX(metric_time) FROM {table}")
-            latest_row = cursor.fetchone()
-            latest_time = latest_row[0] if latest_row else None
-            if isinstance(latest_time, str):
-                latest_time = datetime.fromisoformat(latest_time)
-            if latest_time is not None and latest_time > metric_time:
-                raise ProcessingError(
-                    f"Database already contains newer metric time {latest_time:%Y-%m-%d %H:%M:%S}; "
-                    f"refusing to replace it with {metric_time:%Y-%m-%d %H:%M:%S}"
+            """,
+            database=DATABASE_NAME,
+        )
+        columns = self.client.get_json(
+            self._endpoint("columns"),
+            {"database": DATABASE_NAME, "table": self.table},
+        )
+        if not isinstance(columns, list):
+            raise ProcessingError("Metrix Database API returned invalid column metadata")
+        existing_columns = {str(item.get("name")) for item in columns if isinstance(item, dict)}
+        for column in ("longitude", "latitude"):
+            if column not in existing_columns:
+                self._query(
+                    f"ALTER TABLE `{self.table}` ADD COLUMN `{column}` DECIMAL(10,6) NULL",
+                    database=DATABASE_NAME,
                 )
 
-            cursor.execute(f"DELETE FROM {table} WHERE metric_time = %s", (metric_time,))
-            refreshed_rows = cursor.rowcount
-            cursor.executemany(
-                f"""
-                INSERT INTO {table}
-                    (metric_time, cgi, cell_name, interference_dbm, longitude, latitude)
-                VALUES (%s, %s, %s, %s, %s, %s)
-                """,
-                [
-                    (
-                        metric_time,
-                        row["cgi"],
-                        row["cell_name"],
-                        row["interference_dbm"],
-                        row["longitude"] or None,
-                        row["latitude"] or None,
-                    )
-                    for row in rows
-                ],
+        latest_payload = self._query(
+            f"SELECT MAX(metric_time) AS latest_time FROM `{self.table}`",
+            database=DATABASE_NAME,
+        )
+        latest_rows = latest_payload.get("rows", [])
+        latest_value = latest_rows[0].get("latest_time") if latest_rows else None
+        latest_time = datetime.fromisoformat(str(latest_value)) if latest_value else None
+        if latest_time is not None and latest_time > metric_time:
+            raise ProcessingError(
+                f"Database already contains newer metric time {latest_time:%Y-%m-%d %H:%M:%S}; "
+                f"refusing to replace it with {metric_time:%Y-%m-%d %H:%M:%S}"
             )
-            cursor.execute(f"DELETE FROM {table} WHERE metric_time <> %s", (metric_time,))
-            old_rows_deleted = cursor.rowcount
-            connection.commit()
-        except ProcessingError:
-            connection.rollback()
-            raise
-        except Exception as exc:
-            connection.rollback()
-            raise ProcessingError(f"MySQL write failed: {exc}") from exc
-        finally:
-            if cursor is not None:
-                cursor.close()
-            connection.close()
+
+        metric_literal = f"'{metric_time:%Y-%m-%d %H:%M:%S}'"
+        values = ",\n".join(self._row_values(metric_literal, row) for row in rows)
+        script_payload = self.client.post_json(
+            self._endpoint("run-script"),
+            {
+                "content": f"""
+                    START TRANSACTION;
+                    DELETE FROM `{self.table}` WHERE metric_time = {metric_literal};
+                    INSERT INTO `{self.table}`
+                        (metric_time, cgi, cell_name, interference_dbm, longitude, latitude)
+                    VALUES
+                        {values};
+                    DELETE FROM `{self.table}` WHERE metric_time <> {metric_literal};
+                    COMMIT;
+                """,
+                "database": DATABASE_NAME,
+                "stop_on_error": True,
+                "single_session": True,
+            },
+            timeout=120,
+        )
+        if not isinstance(script_payload, dict) or not isinstance(script_payload.get("results"), list):
+            raise ProcessingError("Metrix Database API returned an invalid script result")
+        results = script_payload["results"]
+        failed = next((item for item in results if not item.get("ok")), None)
+        if failed is not None:
+            raise ProcessingError(f"Database script failed at statement {failed.get('index')}: {failed.get('message', '')}")
+        if len(results) != 5:
+            raise ProcessingError(f"Database script returned {len(results)} results; expected 5")
 
         return {
             "enabled": True,
             "table": self.table,
             "metric_time": metric_time.strftime("%Y-%m-%d %H:%M:%S"),
-            "inserted_rows": len(rows),
-            "refreshed_rows": refreshed_rows,
-            "old_rows_deleted": old_rows_deleted,
+            "inserted_rows": int(results[2].get("affected_rows") or 0),
+            "refreshed_rows": int(results[1].get("affected_rows") or 0),
+            "old_rows_deleted": int(results[3].get("affected_rows") or 0),
         }
 
-    def _connect(self) -> object:
-        if self.connect_factory is not None:
-            return self.connect_factory()
-        try:
-            import pymysql
-        except ImportError as exc:
-            raise ProcessingError("PyMySQL is required for database output") from exc
-        try:
-            bootstrap = pymysql.connect(
-                host=DATABASE_HOST,
-                port=DATABASE_PORT,
-                user=DATABASE_USER,
-                password=DATABASE_PASSWORD,
-                charset="utf8mb4",
-                autocommit=True,
-                connect_timeout=15,
-                read_timeout=60,
-                write_timeout=60,
+    def _endpoint(self, action: str) -> str:
+        return f"/api/databases/{quote(self.connection_id, safe='')}/{action}"
+
+    def _query(self, sql: str, database: str = "") -> dict[str, object]:
+        payload = self.client.post_json(
+            self._endpoint("query"),
+            {"sql": sql, "database": database, "page": 1, "page_size": 100},
+            timeout=120,
+        )
+        if not isinstance(payload, dict):
+            raise ProcessingError("Metrix Database API returned an invalid query result")
+        return payload
+
+    @staticmethod
+    def _row_values(metric_literal: str, row: dict[str, str]) -> str:
+        return "(" + ", ".join(
+            (
+                metric_literal,
+                sql_text_literal(row["cgi"]),
+                sql_text_literal(row["cell_name"]),
+                sql_decimal_literal(row["interference_dbm"], "interference_dbm"),
+                sql_decimal_literal(row["longitude"], "longitude", nullable=True),
+                sql_decimal_literal(row["latitude"], "latitude", nullable=True),
             )
-            try:
-                cursor = bootstrap.cursor()
-                try:
-                    cursor.execute(
-                        f"CREATE DATABASE IF NOT EXISTS `{DATABASE_NAME}` "
-                        "CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
-                    )
-                finally:
-                    cursor.close()
-            finally:
-                bootstrap.close()
-            return pymysql.connect(
-                host=DATABASE_HOST,
-                port=DATABASE_PORT,
-                user=DATABASE_USER,
-                password=DATABASE_PASSWORD,
-                database=DATABASE_NAME,
-                charset="utf8mb4",
-                autocommit=False,
-                connect_timeout=15,
-                read_timeout=60,
-                write_timeout=60,
-            )
-        except Exception as exc:
-            raise ProcessingError(f"MySQL database setup failed: {exc}") from exc
+        ) + ")"
 
 
 class ApiSource:
-    def __init__(self, base_url: str, token: str, storage_id: str, root: str) -> None:
-        if not token:
-            raise ProcessingError("METRIX_API_TOKEN is required in API mode")
-        self.base_url = base_url.rstrip("/")
-        self.token = token
+    def __init__(self, client: MetrixApiClient, storage_id: str, root: str) -> None:
+        self.client = client
         self.storage_id = storage_id
         self.root = root.rstrip("/") or "/"
 
@@ -345,7 +393,7 @@ class ApiSource:
 
     def download(self, path: str) -> bytes:
         endpoint = f"/api/storages/{quote(self.storage_id, safe='')}/download"
-        return self._request(endpoint, {"path": path}, timeout=120)
+        return self.client.get_bytes(endpoint, {"path": path}, timeout=120)
 
     def cell_data_workbooks(self) -> list[tuple[str, bytes]]:
         workbooks: list[tuple[str, bytes]] = []
@@ -357,27 +405,10 @@ class ApiSource:
 
     def _list_dir(self, path: str) -> list[dict[str, object]]:
         endpoint = f"/api/storages/{quote(self.storage_id, safe='')}/files"
-        payload = self._request(endpoint, {"path": path, "recursive": "false"})
-        return json.loads(payload.decode("utf-8")).get("entries", [])
-
-    def _request(self, endpoint: str, query: dict[str, object], timeout: int = 30) -> bytes:
-        url = f"{self.base_url}{endpoint}?{urlencode(query)}"
-        request = Request(url, headers={"Authorization": f"Bearer {self.token}", "Accept": "application/json"})
-        last_error: Exception | None = None
-        for attempt in range(3):
-            try:
-                with urlopen(request, timeout=timeout) as response:
-                    return response.read()
-            except HTTPError as exc:
-                detail = exc.read().decode("utf-8", "replace")[:1000]
-                if exc.code < 500:
-                    raise ProcessingError(f"Metrix API {exc.code}: {detail}") from exc
-                last_error = exc
-            except (URLError, TimeoutError) as exc:
-                last_error = exc
-            if attempt < 2:
-                time.sleep(2**attempt)
-        raise ProcessingError(f"Metrix API request failed: {last_error}")
+        payload = self.client.get_json(endpoint, {"path": path, "recursive": "false"})
+        if not isinstance(payload, dict) or not isinstance(payload.get("entries"), list):
+            raise ProcessingError("Metrix Storage API returned an invalid file list")
+        return payload["entries"]
 
 
 class LocalSource:
@@ -724,6 +755,24 @@ def normalize_cell(value: object) -> str:
     return str(value).strip()
 
 
+def sql_text_literal(value: str) -> str:
+    encoded = value.encode("utf-8").hex()
+    return "''" if not encoded else f"CONVERT(0x{encoded} USING utf8mb4)"
+
+
+def sql_decimal_literal(value: str, field: str, nullable: bool = False) -> str:
+    normalized = value.strip()
+    if not normalized and nullable:
+        return "NULL"
+    try:
+        number = Decimal(normalized)
+    except InvalidOperation as exc:
+        raise ProcessingError(f"Invalid decimal value for {field}: {value}") from exc
+    if not number.is_finite():
+        raise ProcessingError(f"Invalid decimal value for {field}: {value}")
+    return format(number, "f")
+
+
 def write_csv(path: Path, header: tuple[str, ...], rows: list[tuple[object, ...]]) -> None:
     with path.open("w", encoding="utf-8-sig", newline="") as file:
         writer = csv.writer(file)
@@ -750,11 +799,9 @@ def build_parser() -> argparse.ArgumentParser:
     parser.add_argument("--output-dir", type=Path, default=Path(os.getenv("INTERFERENCE_OUTPUT_DIR", "output")))
     parser.add_argument("--window", default=os.getenv("INTERFERENCE_WINDOW", ""), help="Optional exact 16-digit source window")
     parser.add_argument("--lookback-days", type=int, default=int(os.getenv("INTERFERENCE_LOOKBACK_DAYS", "3")))
-    parser.add_argument("--api-base", default=os.getenv("METRIX_API_BASE_URL", "http://172.17.0.1:8000"))
-    parser.add_argument("--api-token", default=os.getenv("METRIX_API_TOKEN", ""))
     parser.add_argument("--storage-id", default=os.getenv("METRIX_STORAGE_ID", DEFAULT_STORAGE_ID))
     parser.add_argument("--root", default=os.getenv("INTERFERENCE_SOURCE_ROOT", DEFAULT_ROOT))
-    parser.add_argument("--no-database", action="store_true", help="Generate files without writing MySQL")
+    parser.add_argument("--no-database", action="store_true", help="Generate files without writing the database")
     return parser
 
 
@@ -762,8 +809,17 @@ def main(argv: list[str] | None = None) -> int:
     args = build_parser().parse_args(argv)
     if args.lookback_days < 1:
         raise ProcessingError("lookback-days must be at least 1")
-    source: Source = LocalSource(args.source_dir) if args.source_dir else ApiSource(args.api_base, args.api_token, args.storage_id, args.root)
-    store = None if args.no_database else MySQLSummaryStore()
+    client: MetrixApiClient | None = None
+    if args.source_dir:
+        source: Source = LocalSource(args.source_dir)
+    else:
+        client = MetrixApiClient(METRIX_API_BASE_URL, METRIX_API_TOKEN)
+        source = ApiSource(client, args.storage_id, args.root)
+    if args.no_database:
+        store = None
+    else:
+        client = client or MetrixApiClient(METRIX_API_BASE_URL, METRIX_API_TOKEN)
+        store = ApiSummaryStore(client, METRIX_DATABASE_CONNECTION_ID)
     process(source, args.output_dir, args.lookback_days, args.window, store=store)
     return 0
 

requirements.txt

@@ -1,2 +1 @@
 openpyxl==3.1.5
-PyMySQL==1.1.2

runtime_config.example.py

@@ -0,0 +1,3 @@
+METRIX_API_BASE_URL = "http://188.5.127.115:18271"
+METRIX_API_TOKEN = "replace-with-metrix-api-token"
+METRIX_DATABASE_CONNECTION_ID = "replace-with-database-connection-id"

scripts/build_offline_package.py

@@ -13,7 +13,7 @@ PROJECT_ROOT = Path(__file__).resolve().parents[1]
 DEFAULT_IMAGE = "interference-etl-runtime:1.1"
 DEFAULT_BASE_IMAGE = "python:3.13.11-slim"
 DEFAULT_OUTPUT = PROJECT_ROOT / "dist" / "InterferenceETL-offline-1.1.zip"
-PACKAGE_FILES = ("main.py", "requirements.txt", "README.md")
+PACKAGE_FILES = ("main.py", "requirements.txt", "README.md", "runtime_config.py")
 PACKAGE_ENTRYPOINT = "from main import main\nraise SystemExit(main())\n"
 
 COPY_DEPENDENCIES = r"""
@@ -22,7 +22,7 @@ from pathlib import Path
 import shutil
 
 target = Path('/package/vendor')
-for name in ('openpyxl', 'et-xmlfile', 'PyMySQL'):
+for name in ('openpyxl', 'et-xmlfile'):
     package = distribution(name)
     for item in package.files or ():
         if '__pycache__' in item.parts or item.suffix in ('.pyc', '.pyo'):
@@ -39,6 +39,9 @@ for name in ('openpyxl', 'et-xmlfile', 'PyMySQL'):
 def build(output: Path, runtime_image: str, base_image: str) -> Path:
     output = output.resolve()
     output.parent.mkdir(parents=True, exist_ok=True)
+    missing = [name for name in PACKAGE_FILES if not (PROJECT_ROOT / name).is_file()]
+    if missing:
+        raise FileNotFoundError(f"Missing package files: {', '.join(missing)}")
     with tempfile.TemporaryDirectory(prefix="interference-etl-package-") as temp:
         staging = Path(temp)
         subprocess.run(
@@ -72,7 +75,7 @@ def build(output: Path, runtime_image: str, base_image: str) -> Path:
                 base_image,
                 "python",
                 "-c",
-                "import main, openpyxl, pymysql; print('offline_package_imports=ok')",
+                "import main, openpyxl; print('offline_package_imports=ok')",
             ],
             check=True,
         )

tests/test_pipeline.py

@@ -6,9 +6,7 @@ import io
 import json
 from pathlib import Path
 import tempfile
-import types
 import unittest
-from unittest.mock import patch
 import zipfile
 
 import main
@@ -112,9 +110,9 @@ class PipelineTest(unittest.TestCase):
         with self.assertRaisesRegex(main.ProcessingError, "Conflicting CellData coordinates"):
             main.load_cell_coordinates([("first.xlsx", first), ("second.xlsx", second)])
 
-    def test_mysql_store_replaces_same_hour_and_deletes_other_hours(self) -> None:
-        connection = FakeConnection(datetime(2026, 7, 31, 10), same_hour_rows=7, old_rows=20)
-        store = main.MySQLSummaryStore(connect_factory=lambda: connection)
+    def test_api_store_replaces_same_hour_and_deletes_other_hours(self) -> None:
+        client = FakeApiClient("2026-07-31T10:00:00")
+        store = main.ApiSummaryStore(client, "db_share_mysql")
 
         result = store.replace_latest([database_row()])
 
@@ -122,47 +120,42 @@ class PipelineTest(unittest.TestCase):
         self.assertEqual(result["inserted_rows"], 1)
         self.assertEqual(result["refreshed_rows"], 7)
         self.assertEqual(result["old_rows_deleted"], 20)
-        self.assertEqual(connection.cursor_instance.inserted[0][0], datetime(2026, 7, 31, 10))
-        self.assertIsNone(connection.cursor_instance.inserted[0][4])
-        self.assertIsNone(connection.cursor_instance.inserted[0][5])
-        self.assertTrue(connection.committed)
-        self.assertFalse(connection.rolled_back)
-        self.assertTrue(connection.closed)
-        self.assertNotIn("source_type", " ".join(connection.cursor_instance.queries))
-        self.assertNotIn("source_path", " ".join(connection.cursor_instance.queries))
-        self.assertTrue(any("ADD COLUMN `longitude`" in query for query in connection.cursor_instance.queries))
-        self.assertTrue(any("ADD COLUMN `latitude`" in query for query in connection.cursor_instance.queries))
+        queries = [payload["sql"] for endpoint, payload in client.posts if endpoint.endswith("/query")]
+        self.assertTrue(any("CREATE DATABASE IF NOT EXISTS `interference_etl`" in query for query in queries))
+        self.assertTrue(any("ADD COLUMN `longitude`" in query for query in queries))
+        self.assertTrue(any("ADD COLUMN `latitude`" in query for query in queries))
+        script = next(payload for endpoint, payload in client.posts if endpoint.endswith("/run-script"))
+        self.assertTrue(script["single_session"])
+        self.assertIn("START TRANSACTION", script["content"])
+        self.assertIn("NULL, NULL)", script["content"])
+        self.assertIn("CONVERT(0x", script["content"])
+        self.assertNotIn("source_type", script["content"])
+        self.assertNotIn("source_path", script["content"])
 
-    def test_mysql_store_refuses_to_replace_a_newer_hour(self) -> None:
-        connection = FakeConnection(datetime(2026, 7, 31, 11))
-        store = main.MySQLSummaryStore(connect_factory=lambda: connection)
+    def test_api_store_refuses_to_replace_a_newer_hour(self) -> None:
+        client = FakeApiClient("2026-07-31T11:00:00")
+        store = main.ApiSummaryStore(client, "db_share_mysql")
 
         with self.assertRaisesRegex(main.ProcessingError, "already contains newer metric time"):
             store.replace_latest([database_row()])
 
-        self.assertFalse(connection.committed)
-        self.assertTrue(connection.rolled_back)
-        self.assertEqual(connection.cursor_instance.inserted, [])
-        self.assertTrue(connection.closed)
+        self.assertFalse(any(endpoint.endswith("/run-script") for endpoint, _ in client.posts))
 
-    def test_mysql_store_creates_and_connects_to_own_database(self) -> None:
-        bootstrap = BootstrapConnection()
-        target = object()
-        calls: list[dict[str, object]] = []
+    def test_api_store_propagates_script_failure(self) -> None:
+        client = FakeApiClient(None, fail_script=True)
+        store = main.ApiSummaryStore(client, "db_share_mysql")
 
-        def connect(**kwargs: object) -> object:
-            calls.append(kwargs)
-            return bootstrap if len(calls) == 1 else target
+        with self.assertRaisesRegex(main.ProcessingError, "statement 3: insert failed"):
+            store.replace_latest([database_row()])
 
-        with patch.dict("sys.modules", {"pymysql": types.SimpleNamespace(connect=connect)}):
-            connection = main.MySQLSummaryStore()._connect()
+    def test_api_store_rejects_invalid_decimal(self) -> None:
+        client = FakeApiClient(None)
+        store = main.ApiSummaryStore(client, "db_share_mysql")
+        row = database_row()
+        row["interference_dbm"] = "not-a-number"
 
-        self.assertIs(connection, target)
-        self.assertNotIn("database", calls[0])
-        self.assertEqual(calls[1]["database"], "interference_etl")
-        self.assertIn("CREATE DATABASE IF NOT EXISTS `interference_etl`", bootstrap.cursor_instance.executed[0])
-        self.assertTrue(bootstrap.cursor_instance.closed)
-        self.assertTrue(bootstrap.closed)
+        with self.assertRaisesRegex(main.ProcessingError, "Invalid decimal value for interference_dbm"):
+            store.replace_latest([row])
 
 
 class RecordingStore:
@@ -181,81 +174,47 @@ class RecordingStore:
         }
 
 
-class FakeCursor:
-    def __init__(self, latest_time: datetime | None, same_hour_rows: int, old_rows: int) -> None:
+class FakeApiClient:
+    def __init__(self, latest_time: str | None, fail_script: bool = False) -> None:
         self.latest_time = latest_time
-        self.same_hour_rows = same_hour_rows
-        self.old_rows = old_rows
-        self.rowcount = 0
-        self.inserted: list[tuple[object, ...]] = []
-        self.queries: list[str] = []
-        self.closed = False
+        self.fail_script = fail_script
+        self.posts: list[tuple[str, dict[str, object]]] = []
 
-    def execute(self, query: str, params: tuple[object, ...] | None = None) -> None:
-        normalized = " ".join(query.split())
-        self.queries.append(normalized)
-        if normalized.startswith("DELETE") and "metric_time = %s" in normalized:
-            self.rowcount = self.same_hour_rows
-        elif normalized.startswith("DELETE") and "metric_time <> %s" in normalized:
-            self.rowcount = self.old_rows
+    def get_json(self, endpoint: str, query: dict[str, object] | None = None, timeout: int = 30) -> object:
+        del endpoint, query, timeout
+        return [
+            {"name": "metric_time"},
+            {"name": "cgi"},
+            {"name": "cell_name"},
+            {"name": "interference_dbm"},
+        ]
 
-    def executemany(self, query: str, values: list[tuple[object, ...]]) -> None:
-        del query
-        self.inserted = list(values)
-        self.rowcount = len(values)
-
-    def fetchone(self) -> tuple[datetime | None]:
-        return (self.latest_time,)
-
-    def fetchall(self) -> list[tuple[str]]:
-        return [(name,) for name in ("metric_time", "cgi", "cell_name", "interference_dbm")]
-
-    def close(self) -> None:
-        self.closed = True
-
-
-class FakeConnection:
-    def __init__(self, latest_time: datetime | None, same_hour_rows: int = 0, old_rows: int = 0) -> None:
-        self.cursor_instance = FakeCursor(latest_time, same_hour_rows, old_rows)
-        self.committed = False
-        self.rolled_back = False
-        self.closed = False
-
-    def cursor(self) -> FakeCursor:
-        return self.cursor_instance
-
-    def commit(self) -> None:
-        self.committed = True
-
-    def rollback(self) -> None:
-        self.rolled_back = True
-
-    def close(self) -> None:
-        self.closed = True
-
-
-class BootstrapCursor:
-    def __init__(self) -> None:
-        self.executed: list[str] = []
-        self.closed = False
-
-    def execute(self, query: str) -> None:
-        self.executed.append(" ".join(query.split()))
-
-    def close(self) -> None:
-        self.closed = True
-
-
-class BootstrapConnection:
-    def __init__(self) -> None:
-        self.cursor_instance = BootstrapCursor()
-        self.closed = False
-
-    def cursor(self) -> BootstrapCursor:
-        return self.cursor_instance
-
-    def close(self) -> None:
-        self.closed = True
+    def post_json(self, endpoint: str, payload: dict[str, object], timeout: int = 30) -> object:
+        del timeout
+        self.posts.append((endpoint, payload))
+        if endpoint.endswith("/query"):
+            if str(payload["sql"]).lstrip().startswith("SELECT MAX"):
+                return {"rows": [{"latest_time": self.latest_time}]}
+            return {"affected_rows": 0}
+        if self.fail_script:
+            return {
+                "results": [
+                    {"index": 1, "ok": True, "affected_rows": 0},
+                    {"index": 2, "ok": True, "affected_rows": 7},
+                    {"index": 3, "ok": False, "message": "insert failed", "affected_rows": 0},
+                ],
+                "stopped": True,
+            }
+        return {
+            "results": [
+                {"index": 1, "ok": True, "affected_rows": 0},
+                {"index": 2, "ok": True, "affected_rows": 7},
+                {"index": 3, "ok": True, "affected_rows": 1},
+                {"index": 4, "ok": True, "affected_rows": 20},
+                {"index": 5, "ok": True, "affected_rows": 0},
+            ],
+            "stopped": False,
+        }
 
 
 def database_row() -> dict[str, str]: