feat: RIIN 商品拉取改用 goods/list 批量接口(200/批串行)替换逐单 order/goods
CI / test (pull_request) Successful in 1m11s
CI / test (pull_request) Successful in 1m11s
- api/riin/client.py: 新增 get_order_goods_batch(distribution/order/goods/list) - flows/shared/clients.py: fetch_goods_concurrent → fetch_goods_for_orders, 批量 200/批、串行、批间 2s,批级重试;移除逐单全局限速(340~500ms)机制 - flows/riin/_pipeline.py: 适配新批量拉取 - flows/shared/db.py: 存在性预查 unnest cast timestamp[] → timestamptz[] (入参为带时区偏移的 tz-aware datetime,timestamptz 才能稳定匹配) 背景:旧逐单接口高频被软限流(成功返回空 data),10-28 起日均缺口 13~33%; 批量接口调用量从日均 ~1 万次降到 ~50 次,已实测回补 7689 单/9717 条商品全量成功。
This commit is contained in:
+4
-2
@@ -460,9 +460,11 @@ class SyncDB:
|
||||
stats = {"ordersCreated": 0, "ordersUpdated": 0, "goodsCreated": 0, "goodsUpdated": 0, "goodsRemoved": 0}
|
||||
with conn.cursor() as cur:
|
||||
# 统计已存在 → created/updated
|
||||
# 注:必须 cast 成 timestamptz——入参是带时区偏移的 tz-aware datetime,
|
||||
# cast 成 timestamp 会丢弃偏移,在会话时区非 +08 时匹配不到既有行,导致 created/updated 统计全错。
|
||||
pairs = [(o.orderNo, o.createdAt) for o in orders]
|
||||
cur.execute(
|
||||
"SELECT t.order_no, t.created_at FROM unnest(%s::text[], %s::timestamp[]) AS t(order_no,created_at) "
|
||||
"SELECT t.order_no, t.created_at FROM unnest(%s::text[], %s::timestamptz[]) AS t(order_no,created_at) "
|
||||
"JOIN \"Orders\" o ON o.\"orderNo\"=t.order_no AND o.\"createdAt\"=t.created_at",
|
||||
_unnest_pairs(pairs),
|
||||
)
|
||||
@@ -623,6 +625,6 @@ class SyncDB:
|
||||
def _unnest_pairs(pairs: list[tuple]) -> tuple[list, list]:
|
||||
"""构造 psycopg2 unnest 双数组参数:返回 (col_a_list, col_b_list)。
|
||||
|
||||
配合 SQL:unnest(%s::text[], %s::timestamp[]) 或 unnest(%s::text[], %s::text[])。
|
||||
配合 SQL:unnest(%s::text[], %s::timestamptz[]) 或 unnest(%s::text[], %s::text[])。
|
||||
"""
|
||||
return [p[0] for p in pairs], [p[1] for p in pairs]
|
||||
|
||||
Reference in New Issue
Block a user