Metrix/InterferenceETL
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: 仅对储存历史CSV使用GBK

Browse Source
This commit is contained in:
nixevol
2026-08-07 16:37:20 +08:00
parent 8a3c273a04
commit 8e96b3a24b
5 changed files with 36 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
aidocs/project_context.md
+4 -4
View File
@@ -100,8 +100,8 @@
- Offline package SHA-256 is `74EEA84058C5F14E7108B840561C070DC47752D67FEC68262F0CC6C78EE8EC0D`. SSH deployment run `984ffc6f98024a2d81b63d4359aabc71` processed `2026-08-07 07:00:00` with 820 rows and deleted 1,137 rows from the prior database time.
- Production verification matched 616 current CGI values to the previous period and left both previous fields NULL for the remaining 204 CGI values. `prev_nearby_count` matched exactly; `prev_interference_dbm` uses the table's existing `DECIMAL(10,3)` precision. Repeat run `6b38030f6d9e4ab1a0ee6a4ba0dc5d6e` returned `status=skipped`.
## 2026-08-07: GBK CSV output
## 2026-08-07: GBK Storage history output
- All generated CSV files now use GBK without a UTF-8 BOM: the seven full source conversions, the merged summary, repaired history exports, and uploaded history files. `manifest.json`, API JSON, and database character sets remain unchanged.
- The encoding is centralized in `CSV_ENCODING`; tests verify GBK Chinese bytes, absence of the UTF-8 BOM, summary parsing, and history parsing.
- Production data contains non-breaking spaces (`U+00A0`), which Python's GBK codec cannot encode. CSV-only normalization converts them to ordinary spaces while keeping strict encoding for every other unsupported character, so unexpected data still fails visibly instead of losing text silently.
- Only final history CSV files uploaded to Metrix Storage use GBK without a UTF-8 BOM, including database-based history repairs. The seven local source conversions and local merged summary remain UTF-8 with BOM; `manifest.json`, API JSON, and database character sets remain unchanged.
- Production data contains non-breaking spaces (`U+00A0`), which Python's GBK codec cannot encode. History-output normalization converts them to ordinary spaces while keeping strict encoding for every other unsupported character.
- Encoding behavior is separated through `LOCAL_CSV_ENCODING` and `HISTORY_CSV_ENCODING`; tests verify the local UTF-8 BOM and GBK history parsing independently.