feat: 支持脚本管理离线上传包

2026-07-31 16:57:59 +08:00
parent dbe2024b6e
commit ad50be3256
3 changed files with 113 additions and 8 deletions
@@ -38,23 +38,21 @@ Use `--window 2026073110001100` to require one exact source window. Without it,

 ## Metrix Script Management

-The target server is offline, so build the runtime image on a connected machine and import it through Metrix Container Management:
+The target server is offline. Build the dependency image locally, then generate a self-contained Script Management upload ZIP:

 ```powershell
 docker build -t interference-etl-runtime:1.1 .
-New-Item -ItemType Directory -Force dist | Out-Null
-docker save -o dist\interference-etl-runtime-1.1.tar interference-etl-runtime:1.1
+python scripts\build_offline_package.py
 ```

-The local `dist/` directory is intentionally excluded from Git. Transfer the generated TAR as a deployment artifact rather than source code.
-The current `1.1` TAR is 46,944,256 bytes with SHA-256 `BF92BF776D52865CA484D0B7D1B10336C998A07E8310D6F01576C74A65A1FF6A`.
+The generated `dist/InterferenceETL-offline-1.1.zip` contains `main.py` plus `openpyxl`, `et_xmlfile`, and `PyMySQL`. The builder verifies imports using the standard `python:3.13.11-slim` image, so the uploaded workspace does not need an online `pip install` or a custom runtime image. The ZIP is 325,932 bytes with SHA-256 `D57930E36B3C70DF7D209C4A83546FEA3B73F87022910D1FDB548D6BED71675D`.

-Create the Metrix script project with:
+Create the project, then upload and extract the ZIP in its Script Management workspace. Use these project settings:

 ```text
 Name: InterferenceETL
 Language: python
-Base image: interference-etl-runtime:1.1
+Base image: python:3.13.11-slim
 Network: bridge
 Run command: python main.py
 Timeout: 1800 seconds
@@ -10,7 +10,7 @@
 - `manifest.json` records input paths, sizes, SHA-256 hashes, row counts, warnings, generated files, and the database write result for ingestion auditing.
 - The Metrix script container needs `openpyxl==3.1.5`, `PyMySQL==1.1.2`, bridge networking, `python main.py`, reachable API/MySQL addresses, and credentials injected through project environment settings. Secrets must not be committed.
 - Read-only validation against the current Metrix storage selected window `2026073110001100`, processed all seven source ZIP files, and produced 1,831 summary rows. Per-source row counts were `7 / 804 / 45 / 86 / 700 / 164 / 25` in `EXPECTED_TYPES` order; sampled CGI, cell name, and interference values matched the source workbooks.
- The current Linux/amd64 runtime image is `interference-etl-runtime:1.1`. Its offline archive is generated locally at `dist/interference-etl-runtime-1.1.tar` (46,944,256 bytes, SHA-256 `BF92BF776D52865CA484D0B7D1B10336C998A07E8310D6F01576C74A65A1FF6A`) and remains excluded from Git.
+- `interference-etl-runtime:1.1` is now a local build helper only. Script projects use the standard `python:3.13.11-slim` image and upload `dist/InterferenceETL-offline-1.1.zip`, which vendors `openpyxl`, `et_xmlfile`, and `PyMySQL` at the workspace root.
 - Six Mock tests pass on Windows Python and inside the runtime image. They cover complete-hour fallback, strict schema rejection, summary extraction, cross-midnight window parsing, transactional database replacement, and rollback protection.

 ## 2026-07-31: Latest-hour MySQL retention
@@ -19,3 +19,9 @@
 - The default table is `interference_hourly_summary`. Its only time column is `metric_time DATETIME`, populated from the source KPI `开始时间`, so users can identify the hour represented by every row.
 - One transaction deletes the selected hour for idempotent refresh, inserts its complete batch, then deletes every other database hour. Any failure rolls back the data changes, and an older selected hour cannot replace a newer hour already stored.
 - Database retention never deletes or modifies Metrix Storage/SFTP source ZIP or XLSX files. Generated CSV retention remains a separate pending decision.
+
+## 2026-07-31: Self-contained Script Management package
+
+- `scripts/build_offline_package.py` copies the three runtime distributions from the verified dependency image, adds the application files, and creates a ZIP that Metrix can extract without preserving executable bits or symlinks.
+- The builder runs an import check inside `python:3.13.11-slim`; the package therefore needs neither a custom server image nor online package installation.
+- Current artifact: `dist/InterferenceETL-offline-1.1.zip`, 325,932 bytes, SHA-256 `D57930E36B3C70DF7D209C4A83546FEA3B73F87022910D1FDB548D6BED71675D`. The builder excludes bytecode caches, and the ignored `dist/` directory is the only location for this generated package.
@@ -0,0 +1,101 @@
+from __future__ import annotations
+
+import argparse
+import hashlib
+from pathlib import Path
+import shutil
+import subprocess
+import tempfile
+import zipfile
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[1]
+DEFAULT_IMAGE = "interference-etl-runtime:1.1"
+DEFAULT_BASE_IMAGE = "python:3.13.11-slim"
+DEFAULT_OUTPUT = PROJECT_ROOT / "dist" / "InterferenceETL-offline-1.1.zip"
+PACKAGE_FILES = ("main.py", "requirements.txt", "README.md")
+
+COPY_DEPENDENCIES = r"""
+from importlib.metadata import distribution
+from pathlib import Path
+import shutil
+
+target = Path('/package')
+for name in ('openpyxl', 'et-xmlfile', 'PyMySQL'):
+    package = distribution(name)
+    for item in package.files or ():
+        if '__pycache__' in item.parts or item.suffix in ('.pyc', '.pyo'):
+            continue
+        source = Path(package.locate_file(item))
+        if not source.is_file():
+            continue
+        destination = target / item
+        destination.parent.mkdir(parents=True, exist_ok=True)
+        shutil.copy2(source, destination)
+"""
+
+
+def build(output: Path, runtime_image: str, base_image: str) -> Path:
+    output = output.resolve()
+    output.parent.mkdir(parents=True, exist_ok=True)
+    with tempfile.TemporaryDirectory(prefix="interference-etl-package-") as temp:
+        staging = Path(temp)
+        subprocess.run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{staging}:/package",
+                runtime_image,
+                "python",
+                "-c",
+                COPY_DEPENDENCIES,
+            ],
+            check=True,
+        )
+        for name in PACKAGE_FILES:
+            shutil.copy2(PROJECT_ROOT / name, staging / name)
+        subprocess.run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-e",
+                "PYTHONDONTWRITEBYTECODE=1",
+                "-v",
+                f"{staging}:/workspace:ro",
+                "-w",
+                "/workspace",
+                base_image,
+                "python",
+                "-c",
+                "import main, openpyxl, pymysql; print('offline_package_imports=ok')",
+            ],
+            check=True,
+        )
+        if output.exists():
+            output.unlink()
+        with zipfile.ZipFile(output, "w", compression=zipfile.ZIP_DEFLATED, compresslevel=9) as archive:
+            for path in sorted(staging.rglob("*")):
+                if path.is_file():
+                    archive.write(path, path.relative_to(staging).as_posix())
+    return output
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Build a self-contained Metrix Script Management upload ZIP")
+    parser.add_argument("--runtime-image", default=DEFAULT_IMAGE)
+    parser.add_argument("--base-image", default=DEFAULT_BASE_IMAGE)
+    parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT)
+    args = parser.parse_args()
+    output = build(args.output, args.runtime_image, args.base_image)
+    digest = hashlib.sha256(output.read_bytes()).hexdigest().upper()
+    print(f"output={output}")
+    print(f"size={output.stat().st_size}")
+    print(f"sha256={digest}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())