22 lines
1.2 KiB
Markdown
22 lines
1.2 KiB
Markdown
---
|
||
title: 楼宇查询改为8协程和独占写线程即时CSV落盘
|
||
type: decision
|
||
permalink: main/projects/8e229c96-a83b-4a02-ac40-10a0a9da444e/楼宇查询改为-8-协程和独占写线程即时-csv-落盘
|
||
stable_id: b082f2d6-b0cc-480c-af8c-df747786ac97
|
||
scope: project
|
||
memory_type: decision
|
||
project_id: 8e229c96-a83b-4a02-ac40-10a0a9da444e
|
||
usage_profile_id: 7aee455b-24cc-4008-bdaf-443721e6f939
|
||
status: active
|
||
revision: 1
|
||
request_id: marketgrid-nr-async-plan-20260910-01
|
||
created_at: '2026-09-10T09:04:29.586141+00:00'
|
||
updated_at: '2026-09-10T09:04:29.586141+00:00'
|
||
tags:
|
||
- asyncio
|
||
- aiohttp
|
||
- csv
|
||
- concurrency
|
||
---
|
||
|
||
用户要求GetNRBuildingsData改成多协程异步版，允许异步库，变量控制数量默认8，写入单线程，每次结果返回即写，避免文件竞态。采用aiohttp连接池和固定数量worker，bounded asyncio.Queue传递结果，单独ThreadPoolExecutor(max_workers=1)独占CSV句柄。每次查询结果写入后flush；动态properties后续新增字段时仅写线程以临时CSV原子替换扩展表头，不再等所有查询结束。观察到用户已将本地LIMIT改为1000，需保留；本地URL/COOKIES有效默认也保持且不得提交或上传。继续保持独立脚本，不修改main.py。 |