merge: refactor/public-capacity-10k-yeuimu into develop (P0 性能整改——public 读路径分域缓存+全写路径失效、nginx 承压调优、连接池/PG 参数,已部署验证)

This commit is contained in:
yeuimu
2026-09-03 09:32:45 +08:00
42 changed files with 2180 additions and 936 deletions
+2 -1
View File
@@ -98,7 +98,8 @@ Prisma Postgres 最佳实践: prisma-postgres 技能
- **连真实库的集成测试隔离**:jest 并行套件共用一个数据库时,(a) 夹具的天然键(sdsCategoryId、名称等)必须带运行时间戳唯一化,禁止跨运行共享字面量;(b) 全量型操作(如 auto-group 扫全库)会顺带扫到其他并行套件的夹具,其写入路径必须对"成员中途消失"宽容(跳过而非抛错),否则会随机挂测试。 - **连真实库的集成测试隔离**:jest 并行套件共用一个数据库时,(a) 夹具的天然键(sdsCategoryId、名称等)必须带运行时间戳唯一化,禁止跨运行共享字面量;(b) 全量型操作(如 auto-group 扫全库)会顺带扫到其他并行套件的夹具,其写入路径必须对"成员中途消失"宽容(跳过而非抛错),否则会随机挂测试。
- **跑全量 jest 前先停 dev server**`nest start --watch` 等常驻进程与测试共用数据库时,其重编译窗口/后台钩子会与测试写入竞争,造成"单跑绿、全量偶发红"的假阳性;验证基线前先停掉所有 watch 进程再跑。 - **跑全量 jest 前先停 dev server**`nest start --watch` 等常驻进程与测试共用数据库时,其重编译窗口/后台钩子会与测试写入竞争,造成"单跑绿、全量偶发红"的假阳性;验证基线前先停掉所有 watch 进程再跑。
- **中文断言勿手写字面量排序**:JS `Array.sort()` 对中文按 UTF-16 码位排(如 烫 U+70EB < 直 U+76F4),手写期望序列容易按拼音/习惯顺序写反;比较选项集合时用 `expect.arrayContaining` + 长度,或从同一排序函数生成期望。 - **中文断言勿手写字面量排序**:JS `Array.sort()` 对中文按 UTF-16 码位排(如 烫 U+70EB < 直 U+76F4),手写期望序列容易按拼音/习惯顺序写反;比较选项集合时用 `expect.arrayContaining` + 长度,或从同一排序函数生成期望。
- **pnpm 仓库容器化执行要挂仓库根**pnpm 的 node_modules 是相对符号链接指向根 `.pnpm` store,容器里只挂子包目录(如 `apps/api:/app`)会断链报 "Cannot find module";必须挂整个仓库根并 `-w` 到子包。另外 Prisma 引擎与系统 libssl 版本强绑定:node:20-alpine 缺 libssl1.1 会报 engine 加载失败,直接复用项目自身的运行镜像(如 deploy-v2-api)跑 prisma/jest 最稳 - **pnpm 仓库容器化执行要挂仓库根**pnpm 的 node_modules 是相对符号链接指向根 `.pnpm` store,容器里只挂子包目录(如 `apps/api:/app`)会断链报 "Cannot find module";必须挂整个仓库根并 `-w` 到子包。另外 Prisma 引擎与系统 libssl 版本强绑定:node:20-alpine 缺 libssl1.1 会报 engine 加载失败;仓库 `.pnpm` store 里生成的 client 是 openssl-1.1.x target**bookworm 系镜像(node:20-slim)同样跑不了**,用 `node:20-bullseye-slim` 挂仓库根 + `--network host`(连本机一次性测试库)跑 jest/prisma 即可,无需依赖项目运行镜像
- **缓存值里绝不能带请求级切片(分页/字段裁剪)**:把 `items.slice(page…)` 的结果整个塞进缓存后,缓存键不含 page → 所有页码命中同一条目、页页返回第一页(total 对得上更具迷惑性);正确做法是缓存"全量物化结果",切片在缓存命中后按请求执行。TDD 用例必须包含"同键翻页 + 页内容随页码变化"的断言才能抓住这类错。
- **Prisma 唯一查询用字段名而非列名**:`where: { id }` 而非 `@map("category_id")` 映射后的 `categoryId`schema `@map` 只影响 SQL 列名,Prisma Client 的唯一输入类型永远用 model 字段名。 - **Prisma 唯一查询用字段名而非列名**:`where: { id }` 而非 `@map("category_id")` 映射后的 `categoryId`schema `@map` 只影响 SQL 列名,Prisma Client 的唯一输入类型永远用 model 字段名。
- **跨套件分页断言要圈定夹具**:真实库上测"列表排序"时全库数据可能远超 pageSize,夹具根本进不了第一页;给夹具商品名加唯一前缀 + `keyword` 过滤圈定,断言既稳定又能看到完整顺序。 - **跨套件分页断言要圈定夹具**:真实库上测"列表排序"时全库数据可能远超 pageSize,夹具根本进不了第一页;给夹具商品名加唯一前缀 + `keyword` 过滤圈定,断言既稳定又能看到完整顺序。
- **"绝对排序键 + 子集过滤"模式**:需要"任意筛选组合下顺序都正确"时,给每条数据算好一组绝对排序键(如 国家→二级→款→priority,缺失沉底),筛选只做子集过滤不做特殊排序分支——比每个筛选组合写一套 orderBy 逻辑可靠得多。 - **"绝对排序键 + 子集过滤"模式**:需要"任意筛选组合下顺序都正确"时,给每条数据算好一组绝对排序键(如 国家→二级→款→priority,缺失沉底),筛选只做子集过滤不做特殊排序分支——比每个筛选组合写一套 orderBy 逻辑可靠得多。
+3 -2
View File
@@ -9,6 +9,7 @@
* 后台等价入口:POST /product-families/organize(「整理」按钮)。 * 后台等价入口:POST /product-families/organize(「整理」按钮)。
*/ */
import { PrismaService } from '../src/prisma/prisma.service'; import { PrismaService } from '../src/prisma/prisma.service';
import { PublicCacheService } from '../src/public/public-cache.service';
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service'; import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
import { ProductFamiliesService } from '../src/product-families/product-families.service'; import { ProductFamiliesService } from '../src/product-families/product-families.service';
import { OrganizeService } from '../src/product-families/organize.service'; import { OrganizeService } from '../src/product-families/organize.service';
@@ -16,9 +17,9 @@ import { OrganizeService } from '../src/product-families/organize.service';
async function main() { async function main() {
const prisma = new PrismaService(); const prisma = new PrismaService();
await prisma.onModuleInit(); await prisma.onModuleInit();
const recompute = new FamilyRecomputeService(prisma); const recompute = new FamilyRecomputeService(prisma, new PublicCacheService());
const families = new ProductFamiliesService(prisma, recompute); const families = new ProductFamiliesService(prisma, recompute);
const organize = new OrganizeService(prisma, recompute, families); const organize = new OrganizeService(prisma, recompute, families, new PublicCacheService());
const result = await organize.organize(); const result = await organize.organize();
console.log( console.log(
+3 -1
View File
@@ -8,6 +8,7 @@
* origin_goods.good_name 为 SDS 纯镜像(每小时同步覆盖),本脚本只改 goods.good_name。 * origin_goods.good_name 为 SDS 纯镜像(每小时同步覆盖),本脚本只改 goods.good_name。
*/ */
import { PrismaService } from '../src/prisma/prisma.service'; import { PrismaService } from '../src/prisma/prisma.service';
import { PublicCacheService } from '../src/public/public-cache.service';
import { SyncService } from '../src/sync/sync.service'; import { SyncService } from '../src/sync/sync.service';
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service'; import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
import { GoodsService } from '../src/goods/goods.service'; import { GoodsService } from '../src/goods/goods.service';
@@ -16,12 +17,13 @@ async function main() {
const dryRun = !process.argv.includes('--apply'); const dryRun = !process.argv.includes('--apply');
const prisma = new PrismaService(); const prisma = new PrismaService();
await prisma.onModuleInit(); await prisma.onModuleInit();
const recompute = new FamilyRecomputeService(prisma); const recompute = new FamilyRecomputeService(prisma, new PublicCacheService());
// 回填路径不触发详情同步,SyncService 仅作占位依赖 // 回填路径不触发详情同步,SyncService 仅作占位依赖
const goods = new GoodsService( const goods = new GoodsService(
prisma, prisma,
{ queueProductDetailSync: async () => undefined } as unknown as SyncService, { queueProductDetailSync: async () => undefined } as unknown as SyncService,
recompute, recompute,
new PublicCacheService(),
); );
const result = await goods.backfillPureSkuGoodNames({ dryRun }); const result = await goods.backfillPureSkuGoodNames({ dryRun });
+2 -1
View File
@@ -7,12 +7,13 @@
* 运行:pnpm --filter @inkreach/api recompute:families * 运行:pnpm --filter @inkreach/api recompute:families
*/ */
import { PrismaService } from '../src/prisma/prisma.service'; import { PrismaService } from '../src/prisma/prisma.service';
import { PublicCacheService } from '../src/public/public-cache.service';
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service'; import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
async function main() { async function main() {
const prisma = new PrismaService(); const prisma = new PrismaService();
await prisma.onModuleInit(); await prisma.onModuleInit();
const recompute = new FamilyRecomputeService(prisma); const recompute = new FamilyRecomputeService(prisma, new PublicCacheService());
const families = await prisma.productFamily.findMany({ const families = await prisma.productFamily.findMany({
select: { id: true }, select: { id: true },
+3
View File
@@ -15,6 +15,7 @@ import { ProductFamiliesModule } from './product-families/product-families.modul
import { GoodsModule } from './goods/goods.module'; import { GoodsModule } from './goods/goods.module';
import { SyncModule } from './sync/sync.module'; import { SyncModule } from './sync/sync.module';
import { PublicModule } from './public/public.module'; import { PublicModule } from './public/public.module';
import { PublicCacheModule } from './public/public-cache.module';
import { UploadModule } from './upload/upload.module'; import { UploadModule } from './upload/upload.module';
@Module({ @Module({
@@ -31,6 +32,8 @@ import { UploadModule } from './upload/upload.module';
}, },
]), ]),
PrismaModule, PrismaModule,
// public 读路径缓存(分域版本失效,@Global 供各写路径注入 bump 入口)
PublicCacheModule,
AuthModule, AuthModule,
CountriesModule, CountriesModule,
CategoriesModule, CategoriesModule,
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { import {
BadRequestException, BadRequestException,
@@ -13,7 +14,7 @@ describe('CategoriesService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [CategoriesService, PrismaService], providers: [CategoriesService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(CategoriesService); service = moduleRef.get(CategoriesService);
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
+14 -4
View File
@@ -5,13 +5,17 @@ import {
NotFoundException, NotFoundException,
} from '@nestjs/common'; } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from '../public/public-cache.service';
import { CreateCategoryDto } from './dto/create-category.dto'; import { CreateCategoryDto } from './dto/create-category.dto';
import { UpdateCategoryDto } from './dto/update-category.dto'; import { UpdateCategoryDto } from './dto/update-category.dto';
import { CategoryNodeDto } from './dto/category-node.dto'; import { CategoryNodeDto } from './dto/category-node.dto';
@Injectable() @Injectable()
export class CategoriesService { export class CategoriesService {
constructor(private readonly prisma: PrismaService) {} constructor(
private readonly prisma: PrismaService,
private readonly publicCache: PublicCacheService,
) {}
async findAll(): Promise<CategoryNodeDto[]> { async findAll(): Promise<CategoryNodeDto[]> {
const all = await this.prisma.category.findMany({ const all = await this.prisma.category.findMany({
@@ -37,7 +41,7 @@ export class CategoriesService {
// Validate the parent exists to produce a clean 404 instead of FK error. // Validate the parent exists to produce a clean 404 instead of FK error.
await this.findOne(BigInt(dto.parentCategoryId)); await this.findOne(BigInt(dto.parentCategoryId));
} }
return this.prisma.category.create({ const created = await this.prisma.category.create({
data: { data: {
categoryName: dto.categoryName, categoryName: dto.categoryName,
categoryIcon: dto.categoryIcon ?? null, categoryIcon: dto.categoryIcon ?? null,
@@ -47,6 +51,8 @@ export class CategoriesService {
: BigInt(dto.parentCategoryId), : BigInt(dto.parentCategoryId),
}, },
}); });
this.publicCache.bump('meta');
return created;
} }
async update(id: bigint, dto: UpdateCategoryDto) { async update(id: bigint, dto: UpdateCategoryDto) {
@@ -66,7 +72,9 @@ export class CategoriesService {
? { disconnect: true } ? { disconnect: true }
: { connect: { id: BigInt(dto.parentCategoryId) } }; : { connect: { id: BigInt(dto.parentCategoryId) } };
} }
return this.prisma.category.update({ where: { id }, data }); const updated = await this.prisma.category.update({ where: { id }, data });
this.publicCache.bump('meta');
return updated;
} }
async remove(id: bigint) { async remove(id: bigint) {
@@ -80,7 +88,9 @@ export class CategoriesService {
); );
} }
try { try {
return await this.prisma.category.delete({ where: { id } }); const removed = await this.prisma.category.delete({ where: { id } });
this.publicCache.bump('meta');
return removed;
} catch (err) { } catch (err) {
if (this.isForeignKeyViolation(err)) { if (this.isForeignKeyViolation(err)) {
throw new BadRequestException( throw new BadRequestException(
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { import {
BadRequestException, BadRequestException,
@@ -14,7 +15,7 @@ describe('CountriesService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [CountriesService, PrismaService], providers: [CountriesService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(CountriesService); service = moduleRef.get(CountriesService);
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
+15 -4
View File
@@ -6,13 +6,17 @@ import {
NotFoundException, NotFoundException,
} from '@nestjs/common'; } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from '../public/public-cache.service';
import { CreateCountryDto } from './dto/create-country.dto'; import { CreateCountryDto } from './dto/create-country.dto';
import { UpdateCountryDto } from './dto/update-country.dto'; import { UpdateCountryDto } from './dto/update-country.dto';
import { ReorderCountriesDto } from './dto/reorder-countries.dto'; import { ReorderCountriesDto } from './dto/reorder-countries.dto';
@Injectable() @Injectable()
export class CountriesService { export class CountriesService {
constructor(private readonly prisma: PrismaService) {} constructor(
private readonly prisma: PrismaService,
private readonly publicCache: PublicCacheService,
) {}
findAll() { findAll() {
return this.prisma.country.findMany({ return this.prisma.country.findMany({
@@ -30,6 +34,7 @@ export class CountriesService {
}), }),
), ),
); );
this.publicCache.bump('meta');
return this.findAll(); return this.findAll();
} }
@@ -43,12 +48,14 @@ export class CountriesService {
async create(dto: CreateCountryDto) { async create(dto: CreateCountryDto) {
try { try {
return await this.prisma.country.create({ const created = await this.prisma.country.create({
data: { data: {
countryName: dto.countryName, countryName: dto.countryName,
countryIcon: dto.countryIcon ?? null, countryIcon: dto.countryIcon ?? null,
}, },
}); });
this.publicCache.bump('meta');
return created;
} catch (err) { } catch (err) {
if ( if (
err instanceof Prisma.PrismaClientKnownRequestError && err instanceof Prisma.PrismaClientKnownRequestError &&
@@ -63,13 +70,15 @@ export class CountriesService {
async update(id: bigint, dto: UpdateCountryDto) { async update(id: bigint, dto: UpdateCountryDto) {
await this.findOne(id); await this.findOne(id);
try { try {
return await this.prisma.country.update({ const updated = await this.prisma.country.update({
where: { id }, where: { id },
data: { data: {
countryName: dto.countryName, countryName: dto.countryName,
countryIcon: dto.countryIcon === undefined ? undefined : dto.countryIcon, countryIcon: dto.countryIcon === undefined ? undefined : dto.countryIcon,
}, },
}); });
this.publicCache.bump('meta');
return updated;
} catch (err) { } catch (err) {
if ( if (
err instanceof Prisma.PrismaClientKnownRequestError && err instanceof Prisma.PrismaClientKnownRequestError &&
@@ -84,7 +93,9 @@ export class CountriesService {
async remove(id: bigint) { async remove(id: bigint) {
await this.findOne(id); await this.findOne(id);
try { try {
return await this.prisma.country.delete({ where: { id } }); const removed = await this.prisma.country.delete({ where: { id } });
this.publicCache.bump('meta');
return removed;
} catch (err) { } catch (err) {
if (this.isForeignKeyViolation(err)) { if (this.isForeignKeyViolation(err)) {
throw new BadRequestException( throw new BadRequestException(
+2
View File
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { import {
BadRequestException, BadRequestException,
@@ -30,6 +31,7 @@ describe('GoodsService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [ providers: [
PublicCacheService,
GoodsService, GoodsService,
PrismaService, PrismaService,
{ {
+9
View File
@@ -14,6 +14,7 @@ import { BatchPriorityDto } from './dto/batch-priority.dto';
import { GoodDetailDto, GoodDto, PaginatedGoods } from './dto/good.dto'; import { GoodDetailDto, GoodDto, PaginatedGoods } from './dto/good.dto';
import { SyncService } from '../sync/sync.service'; import { SyncService } from '../sync/sync.service';
import { FamilyRecomputeService } from '../product-families/family-recompute.service'; import { FamilyRecomputeService } from '../product-families/family-recompute.service';
import { PublicCacheService } from '../public/public-cache.service';
import { isAutoTagGroupName } from '../product-families/auto-tag-rules'; import { isAutoTagGroupName } from '../product-families/auto-tag-rules';
import { randomUUID } from 'crypto'; import { randomUUID } from 'crypto';
import { import {
@@ -120,6 +121,7 @@ export class GoodsService {
private readonly prisma: PrismaService, private readonly prisma: PrismaService,
private readonly syncService: SyncService, private readonly syncService: SyncService,
private readonly familyRecompute: FamilyRecomputeService, private readonly familyRecompute: FamilyRecomputeService,
private readonly publicCache: PublicCacheService,
) {} ) {}
async findAll(query: QueryGoodDto): Promise<PaginatedGoods> { async findAll(query: QueryGoodDto): Promise<PaginatedGoods> {
@@ -240,6 +242,7 @@ export class GoodsService {
// 建商品后把链接有效标签镜像到该商品(纯聚合,不派生——派生在整理流程) // 建商品后把链接有效标签镜像到该商品(纯聚合,不派生——派生在整理流程)
await this.familyRecompute.mirrorLinkTagsToGoods(BigInt(result.originGoodId)); await this.familyRecompute.mirrorLinkTagsToGoods(BigInt(result.originGoodId));
} }
this.publicCache.bump('goods');
if ( if (
result.originGood?.source === 'SDS' && result.originGood?.source === 'SDS' &&
result.originGood.sdsGoodId && result.originGood.sdsGoodId &&
@@ -314,6 +317,7 @@ export class GoodsService {
} }
return good.id; return good.id;
}); });
this.publicCache.bump('goods');
if (family) this.familyRecompute.enqueue(family.id); if (family) this.familyRecompute.enqueue(family.id);
return this.findOne(goodId); return this.findOne(goodId);
} }
@@ -369,6 +373,7 @@ export class GoodsService {
await tx.good.update({ where: { id }, data: goodData }); await tx.good.update({ where: { id }, data: goodData });
} }
}); });
this.publicCache.bump('goods');
return this.findOne(id); return this.findOne(id);
} }
@@ -495,6 +500,7 @@ export class GoodsService {
// 归族联动后重算矩阵(成员变化 → 自动重算,结构化聚合) // 归族联动后重算矩阵(成员变化 → 自动重算,结构化聚合)
await this.familyRecompute.recomputeFamily(familyIdForTags); await this.familyRecompute.recomputeFamily(familyIdForTags);
} }
this.publicCache.bump('goods');
if ( if (
result.originGood?.source === 'SDS' && result.originGood?.source === 'SDS' &&
result.originGood.sdsGoodId && result.originGood.sdsGoodId &&
@@ -522,6 +528,7 @@ export class GoodsService {
} }
} }
}); });
this.publicCache.bump('goods');
return { id: id.toString() }; return { id: id.toString() };
} }
@@ -539,6 +546,7 @@ export class GoodsService {
} }
return { count: dto.items.length }; return { count: dto.items.length };
}); });
this.publicCache.bump('goods');
return result; return result;
} }
@@ -634,6 +642,7 @@ export class GoodsService {
)) { )) {
this.syncService.queueProductDetailSync(goodId); this.syncService.queueProductDetailSync(goodId);
} }
this.publicCache.bump('goods');
return result; return result;
} }
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { OriginGoodsService } from './origin-goods.service'; import { OriginGoodsService } from './origin-goods.service';
import { FamilyRecomputeService } from '../product-families/family-recompute.service'; import { FamilyRecomputeService } from '../product-families/family-recompute.service';
@@ -15,7 +16,7 @@ describe('OriginGoodsService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [OriginGoodsService, FamilyRecomputeService, ProductFamiliesService, OrganizeService, PrismaService], providers: [OriginGoodsService, FamilyRecomputeService, ProductFamiliesService, OrganizeService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(OriginGoodsService); service = moduleRef.get(OriginGoodsService);
organize = moduleRef.get(OrganizeService); organize = moduleRef.get(OrganizeService);
@@ -8,6 +8,7 @@ import { PrismaService } from '../prisma/prisma.service';
import { FamilyRecomputeService } from '../product-families/family-recompute.service'; import { FamilyRecomputeService } from '../product-families/family-recompute.service';
import { OrganizeService } from '../product-families/organize.service'; import { OrganizeService } from '../product-families/organize.service';
import { ProductFamiliesService } from '../product-families/product-families.service'; import { ProductFamiliesService } from '../product-families/product-families.service';
import { PublicCacheService } from '../public/public-cache.service';
import { QueryOriginGoodDto } from './dto/query-origin-good.dto'; import { QueryOriginGoodDto } from './dto/query-origin-good.dto';
/** 链接标签(origin_good_tags 行,含人工/派生标记) */ /** 链接标签(origin_good_tags 行,含人工/派生标记) */
@@ -97,6 +98,7 @@ export class OriginGoodsService {
private readonly familyRecompute: FamilyRecomputeService, private readonly familyRecompute: FamilyRecomputeService,
private readonly organize: OrganizeService, private readonly organize: OrganizeService,
private readonly families: ProductFamiliesService, private readonly families: ProductFamiliesService,
private readonly publicCache: PublicCacheService,
) {} ) {}
/** 链接当前标签(含 manual 标记) */ /** 链接当前标签(含 manual 标记) */
@@ -171,6 +173,9 @@ export class OriginGoodsService {
this.prisma.originGood.update({ where: { id }, data: { tagsManual: true } }), this.prisma.originGood.update({ where: { id }, data: { tagsManual: true } }),
]); ]);
await this.familyRecompute.mirrorLinkTagsToGoods(id); await this.familyRecompute.mirrorLinkTagsToGoods(id);
// 标签镜像改变 good_tags(meta 域的标签组过滤);无族链接挂靠失败时
// 后续 recompute 不会执行,此处显式 bump 兜住全部路径(含 matrix 归因)
this.publicCache.bump('meta', 'goods', 'matrix');
// 人工改标签 → 归因维度可能变化,自动重算族矩阵(有族才重算) // 人工改标签 → 归因维度可能变化,自动重算族矩阵(有族才重算)
const ogFull = await this.prisma.originGood.findUnique({ const ogFull = await this.prisma.originGood.findUnique({
where: { id }, where: { id },
@@ -205,6 +210,7 @@ export class OriginGoodsService {
]); ]);
// 派生集中在整理服务(解析去运行时化);「恢复自动」本身是显式人工动作 // 派生集中在整理服务(解析去运行时化);「恢复自动」本身是显式人工动作
await this.organize.deriveTagsForOg(id); await this.organize.deriveTagsForOg(id);
this.publicCache.bump('meta', 'goods', 'matrix');
if (og.familyId) await this.familyRecompute.recomputeFamily(og.familyId); if (og.familyId) await this.familyRecompute.recomputeFamily(og.familyId);
return this.getTags(id); return this.getTags(id);
} }
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { NotFoundException } from '@nestjs/common'; import { NotFoundException } from '@nestjs/common';
import { PositionsService } from './positions.service'; import { PositionsService } from './positions.service';
@@ -12,7 +13,7 @@ describe('PositionsService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [PositionsService, PrismaService], providers: [PositionsService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(PositionsService); service = moduleRef.get(PositionsService);
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
+14 -4
View File
@@ -3,12 +3,16 @@ import {
NotFoundException, NotFoundException,
} from '@nestjs/common'; } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from '../public/public-cache.service';
import { CreatePositionDto } from './dto/create-position.dto'; import { CreatePositionDto } from './dto/create-position.dto';
import { UpdatePositionDto } from './dto/update-position.dto'; import { UpdatePositionDto } from './dto/update-position.dto';
@Injectable() @Injectable()
export class PositionsService { export class PositionsService {
constructor(private readonly prisma: PrismaService) {} constructor(
private readonly prisma: PrismaService,
private readonly publicCache: PublicCacheService,
) {}
findAll(filters?: { countryId?: bigint; categoryId?: bigint }) { findAll(filters?: { countryId?: bigint; categoryId?: bigint }) {
const where: { countryId?: bigint; categoryId?: bigint } = {}; const where: { countryId?: bigint; categoryId?: bigint } = {};
@@ -37,7 +41,7 @@ export class PositionsService {
if (dto.categoryId !== undefined) { if (dto.categoryId !== undefined) {
await this.ensureCategory(dto.categoryId); await this.ensureCategory(dto.categoryId);
} }
return this.prisma.position.create({ const created = await this.prisma.position.create({
data: { data: {
indexVal: dto.indexVal, indexVal: dto.indexVal,
countryId: dto.countryId === undefined ? null : BigInt(dto.countryId), countryId: dto.countryId === undefined ? null : BigInt(dto.countryId),
@@ -45,6 +49,8 @@ export class PositionsService {
}, },
include: { country: true, category: true }, include: { country: true, category: true },
}); });
this.publicCache.bump('goods');
return created;
} }
async update(id: bigint, dto: UpdatePositionDto) { async update(id: bigint, dto: UpdatePositionDto) {
@@ -55,7 +61,7 @@ export class PositionsService {
if (dto.categoryId !== undefined && dto.categoryId !== null) { if (dto.categoryId !== undefined && dto.categoryId !== null) {
await this.ensureCategory(dto.categoryId); await this.ensureCategory(dto.categoryId);
} }
return this.prisma.position.update({ const updated = await this.prisma.position.update({
where: { id }, where: { id },
data: { data: {
indexVal: dto.indexVal, indexVal: dto.indexVal,
@@ -74,11 +80,15 @@ export class PositionsService {
}, },
include: { country: true, category: true }, include: { country: true, category: true },
}); });
this.publicCache.bump('goods');
return updated;
} }
async remove(id: bigint) { async remove(id: bigint) {
await this.findOne(id); await this.findOne(id);
return this.prisma.position.delete({ where: { id } }); const removed = await this.prisma.position.delete({ where: { id } });
this.publicCache.bump('goods');
return removed;
} }
private async ensureCountry(id: number) { private async ensureCountry(id: number) {
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { Prisma } from '@prisma/client'; import { Prisma } from '@prisma/client';
import { import {
@@ -103,13 +104,14 @@ describe('FamilyRecomputeService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [FamilyRecomputeService, PrismaService], providers: [FamilyRecomputeService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(FamilyRecomputeService); service = moduleRef.get(FamilyRecomputeService);
organize = new OrganizeService( organize = new OrganizeService(
moduleRef.get(PrismaService), moduleRef.get(PrismaService),
service, service,
new ProductFamiliesService(moduleRef.get(PrismaService), service), new ProductFamiliesService(moduleRef.get(PrismaService), service),
moduleRef.get(PublicCacheService),
); );
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
await prisma.onModuleInit(); await prisma.onModuleInit();
@@ -1,6 +1,7 @@
import { Injectable, Logger } from '@nestjs/common'; import { Injectable, Logger } from '@nestjs/common';
import { Prisma } from '@prisma/client'; import { Prisma } from '@prisma/client';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from '../public/public-cache.service';
import { isAutoTagGroupName } from './auto-tag-rules'; import { isAutoTagGroupName } from './auto-tag-rules';
/** /**
@@ -326,7 +327,10 @@ export class FamilyRecomputeService {
private readonly logger = new Logger(FamilyRecomputeService.name); private readonly logger = new Logger(FamilyRecomputeService.name);
private readonly pending = new Map<string, Promise<void>>(); private readonly pending = new Map<string, Promise<void>>();
constructor(private readonly prisma: PrismaService) {} constructor(
private readonly prisma: PrismaService,
private readonly publicCache: PublicCacheService,
) {}
/** 进程内去重的异步重算入口(同步钩子用) */ /** 进程内去重的异步重算入口(同步钩子用) */
enqueue(familyId: bigint): void { enqueue(familyId: bigint): void {
@@ -405,7 +409,9 @@ export class FamilyRecomputeService {
data: { stale: true }, data: { stale: true },
}); });
} }
// 族物化 JSON(矩阵/尺码表)与成员口径变化 → public 列表价/详情族块失效。
// 人工接管族(autoManaged=false)虽未重写矩阵,但其成员/详情已变,goods 域同样失效。
this.publicCache.bump('goods', 'matrix');
} }
/** 把链接的有效标签镜像到其名下商品(good 标签 = 链接标签 ∪ 非自动组既有标签) */ /** 把链接的有效标签镜像到其名下商品(good 标签 = 链接标签 ∪ 非自动组既有标签) */
async mirrorLinkTagsToGoods(ogId: bigint): Promise<void> { async mirrorLinkTagsToGoods(ogId: bigint): Promise<void> {
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { FamilyRecomputeService } from './family-recompute.service'; import { FamilyRecomputeService } from './family-recompute.service';
import { ProductFamiliesService } from './product-families.service'; import { ProductFamiliesService } from './product-families.service';
@@ -29,7 +30,7 @@ describe('链接级标签:派生 / 人工接管 / 商品镜像', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [FamilyRecomputeService, ProductFamiliesService, OrganizeService, OriginGoodsService, PrismaService], providers: [FamilyRecomputeService, ProductFamiliesService, OrganizeService, OriginGoodsService, PrismaService, PublicCacheService],
}).compile(); }).compile();
recompute = moduleRef.get(FamilyRecomputeService); recompute = moduleRef.get(FamilyRecomputeService);
organize = moduleRef.get(OrganizeService); organize = moduleRef.get(OrganizeService);
@@ -2,6 +2,7 @@ import { Injectable, Logger } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { FamilyRecomputeService } from './family-recompute.service'; import { FamilyRecomputeService } from './family-recompute.service';
import { ProductFamiliesService } from './product-families.service'; import { ProductFamiliesService } from './product-families.service';
import { PublicCacheService } from '../public/public-cache.service';
import { parseOriginName } from './origin-name.parser'; import { parseOriginName } from './origin-name.parser';
import { import {
DERIVED_TAG_GROUP_SPECS, DERIVED_TAG_GROUP_SPECS,
@@ -28,6 +29,7 @@ export class OrganizeService {
private readonly prisma: PrismaService, private readonly prisma: PrismaService,
private readonly recompute: FamilyRecomputeService, private readonly recompute: FamilyRecomputeService,
private readonly families: ProductFamiliesService, private readonly families: ProductFamiliesService,
private readonly publicCache: PublicCacheService,
) {} ) {}
async organize() { async organize() {
@@ -52,6 +54,9 @@ export class OrganizeService {
familiesRecomputed: families.length, familiesRecomputed: families.length,
}; };
this.logger.log(`organize done: ${JSON.stringify(result)}`); this.logger.log(`organize done: ${JSON.stringify(result)}`);
// 整理是全量标签/归族/矩阵重写动作,三域全失效(末尾逐族重算只覆盖 goods+matrix
// 标签镜像与标签组创建落在 meta 域)
this.publicCache.bump('meta', 'goods', 'matrix');
return result; return result;
} }
@@ -190,6 +195,8 @@ export class OrganizeService {
await this.prisma.originGood.update({ where: { id: og.id }, data: next }); await this.prisma.originGood.update({ where: { id: og.id }, data: next });
parsed += 1; parsed += 1;
} }
// 结构化标签列是 CUSTOM 成员的矩阵归因来源(下次重算生效),保守失效 matrix+goods
this.publicCache.bump('goods', 'matrix');
return { parsed, unparsable }; return { parsed, unparsable };
} }
@@ -220,6 +227,8 @@ export class OrganizeService {
for (const fid of touchedFamilies) { for (const fid of touchedFamilies) {
await this.recompute.recomputeFamily(fid); await this.recompute.recomputeFamily(fid);
} }
// 标签镜像(good_tags)落 meta 域;散链接无族不会被重算覆盖,入口统一失效
this.publicCache.bump('meta', 'goods', 'matrix');
return { linksUpdated, goodsUpdated, familiesRecomputed: touchedFamilies.length }; return { linksUpdated, goodsUpdated, familiesRecomputed: touchedFamilies.length };
} }
@@ -339,6 +348,7 @@ export class OrganizeService {
]); ]);
goodsUpdated += 1; goodsUpdated += 1;
} }
this.publicCache.bump('meta', 'goods');
return { goodsUpdated, linksUpdated }; return { goodsUpdated, linksUpdated };
} }
@@ -375,6 +385,7 @@ export class OrganizeService {
linksUpdated = 1; linksUpdated = 1;
} }
await this.recompute.mirrorLinkTagsToGoods(og.id); await this.recompute.mirrorLinkTagsToGoods(og.id);
this.publicCache.bump('meta', 'goods');
return { linksUpdated }; return { linksUpdated };
} }
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { BadRequestException, NotFoundException } from '@nestjs/common'; import { BadRequestException, NotFoundException } from '@nestjs/common';
import { Prisma } from '@prisma/client'; import { Prisma } from '@prisma/client';
@@ -28,13 +29,14 @@ describe('ProductFamiliesService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [ProductFamiliesService, FamilyRecomputeService, PrismaService], providers: [ProductFamiliesService, FamilyRecomputeService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(ProductFamiliesService); service = moduleRef.get(ProductFamiliesService);
organize = new OrganizeService( organize = new OrganizeService(
moduleRef.get(PrismaService), moduleRef.get(PrismaService),
moduleRef.get(FamilyRecomputeService), moduleRef.get(FamilyRecomputeService),
service, service,
moduleRef.get(PublicCacheService),
); );
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
await prisma.onModuleInit(); await prisma.onModuleInit();
@@ -0,0 +1,283 @@
import { Test } from '@nestjs/testing';
import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from './public-cache.service';
import { PublicService } from './public.service';
import { CategoriesService } from '../categories/categories.service';
import { CountriesService } from '../countries/countries.service';
import { TagsService } from '../tags/tags.service';
import { TagGroupsService } from '../tag-groups/tag-groups.service';
import { PositionsService } from '../positions/positions.service';
import { GoodsService } from '../goods/goods.service';
import { SyncService } from '../sync/sync.service';
import { SdsClientService } from '../sync/sds-client.service';
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
import { ProductFamiliesService } from '../product-families/product-families.service';
import { OrganizeService } from '../product-families/organize.service';
import { OriginGoodsService } from '../origin-goods/origin-goods.service';
/**
* public 缓存失效链路(集成):
* 1. 读端点命中缓存(同筛选翻页/重复详情不再查库);
* 2. 「TTL 未到但写已发生 → 前台立即可见」端到端;
* 3. 每条写路径 bump 断言(全局约束 §2:等待 TTL 过期不是失效手段)。
* 夹具自包含:全部天然键带运行时间戳,不依赖库内既有数据。
*/
describe('public 缓存失效链路', () => {
const stamp = Date.now();
let prisma: PrismaService;
let cache: PublicCacheService;
let service: PublicService;
let categoriesService: CategoriesService;
let countriesService: CountriesService;
let tagsService: TagsService;
let tagGroupsService: TagGroupsService;
let positionsService: PositionsService;
let goodsService: GoodsService;
let syncService: SyncService;
let countryId: bigint;
let categoryId: bigint;
let tagId: bigint;
let tagGroupId: bigint;
let positionId: bigint;
let originGoodId: bigint;
let familyId: bigint;
let goodId: bigint;
let sdsGoodId: string;
let sdsCategoryId: string;
beforeAll(async () => {
const sdsMock: Partial<SdsClientService> = {
// 返回库内全部 SDS 分类 + 逻辑:保证 syncCategories 不误删并行套件的夹具
fetchCategoryTree: async () => {
const rows = await prisma.category.findMany({
where: { sdsCategoryId: { not: null } },
select: { sdsCategoryId: true, categoryName: true },
});
return rows.map((r) => ({ id: r.sdsCategoryId!, name: r.categoryName }));
},
fetchProductsPage: async () => ({ items: [] }),
fetchProductDetail: async (goodId: string | number) => ({ id: goodId }),
};
const moduleRef = await Test.createTestingModule({
providers: [
PublicService,
CategoriesService,
CountriesService,
TagsService,
TagGroupsService,
PositionsService,
GoodsService,
SyncService,
FamilyRecomputeService,
ProductFamiliesService,
OrganizeService,
OriginGoodsService,
{ provide: SdsClientService, useValue: sdsMock },
PrismaService,
PublicCacheService,
],
}).compile();
service = moduleRef.get(PublicService);
cache = moduleRef.get(PublicCacheService);
categoriesService = moduleRef.get(CategoriesService);
countriesService = moduleRef.get(CountriesService);
tagsService = moduleRef.get(TagsService);
tagGroupsService = moduleRef.get(TagGroupsService);
positionsService = moduleRef.get(PositionsService);
goodsService = moduleRef.get(GoodsService);
syncService = moduleRef.get(SyncService);
prisma = moduleRef.get(PrismaService);
await prisma.onModuleInit();
const country = await prisma.country.create({
data: { countryName: `失效测试国家 ${stamp}` },
});
countryId = country.id;
sdsCategoryId = `inv-sds-cat-${stamp}`;
const category = await prisma.category.create({
data: { categoryName: `失效测试分类 ${stamp}`, sdsCategoryId },
});
categoryId = category.id;
const tagGroup = await prisma.tagGroup.create({
data: { groupName: `失效测试标签组 ${stamp}` },
});
tagGroupId = tagGroup.id;
const tag = await prisma.tag.create({
data: { tagName: `失效测试标签 ${stamp}`, tagGroupId },
});
tagId = tag.id;
const position = await prisma.position.create({
data: { indexVal: 1, countryId, categoryId },
});
positionId = position.id;
sdsGoodId = `inv-sds-good-${stamp}`;
const og = await prisma.originGood.create({
data: { sdsGoodId, goodName: `失效测试链接 ${stamp}`, goodImage: 'http://img' },
});
originGoodId = og.id;
const family = await prisma.productFamily.create({
data: { familyName: `失效测试族 ${stamp}`, primaryOriginGoodId: og.id },
});
familyId = family.id;
await prisma.originGood.update({
where: { id: og.id },
data: { familyId },
});
const good = await prisma.good.create({
data: {
goodName: `失效测试商品 ${stamp}`,
originGoodId,
familyId,
countryId,
categoryId,
goodPriority: 5,
},
});
goodId = good.id;
}, 30_000);
afterAll(async () => {
await prisma.good.deleteMany({ where: { id: goodId } });
await prisma.originGoodTag.deleteMany({ where: { originGoodId } });
await prisma.originGoodVariant.deleteMany({ where: { originGoodId } });
await prisma.originGoodDetail.deleteMany({ where: { originGoodId } });
await prisma.originGood.deleteMany({ where: { id: originGoodId } });
await prisma.familyPriceOverride.deleteMany({ where: { familyId } });
await prisma.productFamily.deleteMany({ where: { id: familyId } });
await prisma.position.deleteMany({ where: { id: positionId } });
await prisma.tag.deleteMany({ where: { id: tagId } });
await prisma.tagGroup.deleteMany({ where: { id: tagGroupId } });
await prisma.category.deleteMany({
where: { OR: [{ id: categoryId }, { sdsCategoryId: { startsWith: `inv-sds-cat-${stamp}` } }] },
});
await prisma.country.deleteMany({ where: { id: countryId } });
await prisma.$disconnect();
});
it('列表读缓存:同筛选重复请求与翻页共享一份缓存,只查一次库', async () => {
const findManySpy = jest.spyOn(prisma.good, 'findMany');
const query = {
page: 1,
pageSize: 1,
countryId: countryId.toString(),
keyword: `失效测试商品 ${stamp}`,
sort: 'DEFAULT' as const,
};
const first = await service.getGoods(query);
const second = await service.getGoods(query);
const page2 = await service.getGoods({ ...query, page: 2 });
expect(first.items).toHaveLength(1);
expect(first.items[0].goodId).toBe(familyId.toString());
expect(second.items).toEqual(first.items);
expect(page2.items).toHaveLength(0); // total=1,第二页为空,但仍命中同一份缓存
expect(findManySpy).toHaveBeenCalledTimes(1);
findManySpy.mockRestore();
});
it('详情读缓存:重复请求不再查族表', async () => {
const familySpy = jest.spyOn(prisma.productFamily, 'findUnique');
const first = await service.getGood(familyId.toString());
const second = await service.getGood(familyId.toString());
expect(first.goodId).toBe(familyId.toString());
expect(second).toEqual(first);
expect(familySpy).toHaveBeenCalledTimes(1);
familySpy.mockRestore();
});
it('TTL 未到但分类改名 → 分类树立即返回新名(写后即失效,端到端)', async () => {
const oldName = `失效测试分类 ${stamp}`;
const newName = `失效后分类 ${stamp}`;
const categorySpy = jest.spyOn(prisma.category, 'findMany');
const before = await service.getCategoriesTree();
const flatBefore = JSON.stringify(before);
expect(flatBefore).toContain(oldName);
const callsAfterWarm = categorySpy.mock.calls.length;
await categoriesService.update(categoryId, { categoryName: newName });
const after = await service.getCategoriesTree();
expect(JSON.stringify(after)).toContain(newName);
expect(JSON.stringify(after)).not.toContain(oldName);
// 旧值来自缓存则不会再查库;失效正确时应产生新的分类查询
expect(categorySpy.mock.calls.length).toBeGreaterThan(callsAfterWarm);
categorySpy.mockRestore();
});
it('分类写路径 bump metaCountries/Tags/TagGroups', async () => {
const meta0 = cache.version('meta');
const goods0 = cache.version('goods');
await countriesService.update(countryId, {
countryName: `失效测试国家改 ${stamp}`,
countryIcon: null,
});
expect(cache.version('meta')).toBe(meta0 + 1);
await tagsService.update(tagId, { tagName: `失效测试标签改 ${stamp}` });
expect(cache.version('meta')).toBe(meta0 + 2);
await tagGroupsService.update(tagGroupId, { groupName: `失效测试标签组改 ${stamp}` });
expect(cache.version('meta')).toBe(meta0 + 3);
expect(cache.version('goods')).toBe(goods0); // meta 写路径不误伤 goods 域
});
it('positions/goods 写路径 bump goods', async () => {
const goods0 = cache.version('goods');
const meta0 = cache.version('meta');
await positionsService.update(positionId, { indexVal: 9 });
expect(cache.version('goods')).toBe(goods0 + 1);
await goodsService.update(goodId, { goodName: `失效测试商品改 ${stamp}` });
// update 内部联动 recomputeFamilygoods+matrix)后再显式 bump goods
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods0 + 2);
expect(cache.version('meta')).toBe(meta0);
});
it('族重算 recomputeFamily bump goods+matrix', async () => {
const goods0 = cache.version('goods');
const matrix0 = cache.version('matrix');
const recompute = new FamilyRecomputeService(prisma, cache);
await recompute.recomputeFamily(familyId);
// goods 用 ≥:goods.update 等前置操作可能触发异步详情同步(fire-and-forget
// persistProductDetail 也会 bump goods),不与本断言强耦合;matrix 仅重算写
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods0 + 1);
expect(cache.version('matrix')).toBe(matrix0 + 1);
});
it('链接人工改标签 bump meta+goods+matrix(含无族路径兜底)', async () => {
const meta0 = cache.version('meta');
const goods0 = cache.version('goods');
const matrix0 = cache.version('matrix');
const originGoodsService = new OriginGoodsService(
prisma,
new FamilyRecomputeService(prisma, cache),
new OrganizeService(prisma, new FamilyRecomputeService(prisma, cache), new ProductFamiliesService(prisma, new FamilyRecomputeService(prisma, cache)), cache),
new ProductFamiliesService(prisma, new FamilyRecomputeService(prisma, cache)),
cache,
);
await originGoodsService.updateTags(originGoodId, []);
expect(cache.version('meta')).toBeGreaterThanOrEqual(meta0 + 1);
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods0 + 1);
expect(cache.version('matrix')).toBeGreaterThanOrEqual(matrix0 + 1);
});
it('同步写路径:syncCategories bump meta+goodssyncProducts bump goodspersistProductDetail bump goods', async () => {
const meta0 = cache.version('meta');
const goods0 = cache.version('goods');
await syncService.syncCategories();
expect(cache.version('meta')).toBe(meta0 + 1);
expect(cache.version('goods')).toBe(goods0 + 1);
await syncService.syncProducts();
expect(cache.version('goods')).toBe(goods0 + 2);
const goods1 = cache.version('goods');
// persistProductDetail 是私有方法,直调以断言挂钩(上游最小对象,mapper 全容忍)
await (syncService as unknown as { persistProductDetail: (id: bigint, up: unknown) => Promise<void> }).persistProductDetail(
originGoodId,
{ id: sdsGoodId },
);
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods1 + 1);
}, 60_000);
});
@@ -0,0 +1,16 @@
import { Global, Module } from '@nestjs/common';
import { PublicCacheService } from './public-cache.service';
/**
* Global public 缓存模块,导出 {@link PublicCacheService}。
*
* 与 PrismaModule 同款 @Global 模式:public 读路径与各 admin/sync 写路径
* 都要注入失效入口(bump),逐一 import 太啰嗦。单进程部署下进程内即全局;
* 扩容多副本时缓存需换共享存储(见整改计划 P1-3 扩展阶梯)。
*/
@Global()
@Module({
providers: [PublicCacheService],
exports: [PublicCacheService],
})
export class PublicCacheModule {}
@@ -0,0 +1,165 @@
import { PublicCacheDomain, PublicCacheService } from './public-cache.service';
describe('PublicCacheService', () => {
let cache: PublicCacheService;
beforeEach(() => {
cache = new PublicCacheService();
delete process.env.PUBLIC_CACHE_DISABLED;
delete process.env.PUBLIC_CACHE_TTL_MS;
});
afterAll(() => {
delete process.env.PUBLIC_CACHE_DISABLED;
delete process.env.PUBLIC_CACHE_TTL_MS;
});
it('命中:相同 key 第二次调用不再执行 loader', async () => {
const loader = jest.fn(async () => ({ value: 1 }));
const first = await cache.wrap('k', ['goods'], loader);
const second = await cache.wrap('k', ['goods'], loader);
expect(first).toEqual({ value: 1 });
expect(second).toEqual({ value: 1 });
expect(loader).toHaveBeenCalledTimes(1);
});
it('不同依赖域组合的相同 key 互不串缓存', async () => {
const a = await cache.wrap('k', ['goods'], async () => 'A');
const b = await cache.wrap('k', ['goods', 'matrix'], async () => 'B');
expect(a).toBe('A');
expect(b).toBe('B');
});
it('TTL 过期后重新执行 loader(TTL 只是兜底,不是失效手段)', async () => {
jest.useFakeTimers();
jest.setSystemTime(1_700_000_000_000);
try {
process.env.PUBLIC_CACHE_TTL_MS = '1000';
const loader = jest.fn(async () => ({ v: 1 }));
await cache.wrap('k', ['goods'], loader);
jest.setSystemTime(1_700_000_000_000 + 999);
await cache.wrap('k', ['goods'], loader);
expect(loader).toHaveBeenCalledTimes(1);
jest.setSystemTime(1_700_000_000_000 + 1001);
await cache.wrap('k', ['goods'], loader);
expect(loader).toHaveBeenCalledTimes(2);
} finally {
jest.useRealTimers();
}
});
it('bump 域版本后,依赖该域的缓存立即作废', async () => {
const loader = jest.fn(async () => ({ v: 1 }));
await cache.wrap('list', ['goods', 'matrix'], loader);
cache.bump('goods');
await cache.wrap('list', ['goods', 'matrix'], loader);
expect(loader).toHaveBeenCalledTimes(2);
});
it('bump 某域不影响不依赖该域的条目(域间隔离)', async () => {
const metaLoader = jest.fn(async () => 'meta-data');
const goodsLoader = jest.fn(async () => 'goods-data');
await cache.wrap('meta-key', ['meta'], metaLoader);
cache.bump('goods');
await cache.wrap('meta-key', ['meta'], metaLoader);
expect(metaLoader).toHaveBeenCalledTimes(1);
await cache.wrap('goods-key', ['goods'], goodsLoader);
expect(goodsLoader).toHaveBeenCalledTimes(1);
});
it('跨域条目:任一依赖域 bump 都使其作废', async () => {
const loader = jest.fn(async () => 'x');
await cache.wrap('combo', ['goods', 'matrix', 'meta'], loader);
cache.bump('matrix');
await cache.wrap('combo', ['goods', 'matrix', 'meta'], loader);
expect(loader).toHaveBeenCalledTimes(2);
});
it('竞态防护:loader 执行期间发生 bump → 结果照常返回但不入缓存', async () => {
let releaseLoader!: (v: string) => void;
const loader = jest.fn(
() =>
new Promise<string>((resolve) => {
releaseLoader = resolve;
}),
);
const inflight = cache.wrap('k', ['goods'], loader);
cache.bump('goods'); // 加载期间数据变了
releaseLoader('stale-value');
await expect(inflight).resolves.toBe('stale-value');
const loader2 = jest.fn(async () => 'fresh-value');
await expect(cache.wrap('k', ['goods'], loader2)).resolves.toBe('fresh-value');
expect(loader2).toHaveBeenCalledTimes(1); // 旧值没有写回缓存
});
it('并发合并:并发 miss 只触发一次 loader(防击穿)', async () => {
let release!: () => void;
const loader = jest.fn(
() =>
new Promise<number>((resolve) => {
release = () => resolve(42);
}),
);
const p1 = cache.wrap('hot', ['goods'], loader);
const p2 = cache.wrap('hot', ['goods'], loader);
const p3 = cache.wrap('hot', ['goods'], loader);
release();
expect(await Promise.all([p1, p2, p3])).toEqual([42, 42, 42]);
expect(loader).toHaveBeenCalledTimes(1);
});
it('PUBLIC_CACHE_DISABLED=true 时直通 loader、不缓存', async () => {
process.env.PUBLIC_CACHE_DISABLED = 'true';
const loader = jest.fn(async () => ({ v: 1 }));
await cache.wrap('k', ['goods'], loader);
await cache.wrap('k', ['goods'], loader);
expect(loader).toHaveBeenCalledTimes(2);
});
it('loader 抛错不缓存、不残留 pending', async () => {
const failing = jest.fn(async () => {
throw new Error('boom');
});
await expect(cache.wrap('k', ['goods'], failing)).rejects.toThrow('boom');
const ok = jest.fn(async () => 'good');
await expect(cache.wrap('k', ['goods'], ok)).resolves.toBe('good');
});
it('条目上限:先清过期条目,仍超限则按插入序淘汰最旧的', async () => {
jest.useFakeTimers();
jest.setSystemTime(1_700_000_000_000);
try {
process.env.PUBLIC_CACHE_TTL_MS = '1000';
process.env.PUBLIC_CACHE_MAX_ENTRIES = '3';
await cache.wrap('old', ['goods'], async () => 'old');
jest.setSystemTime(1_700_000_000_000 + 2000);
// old 已过期;再写 4 条活跃条目,第一条应顺带清掉过期的 old
for (let i = 0; i < 4; i++) {
await cache.wrap(`k${i}`, ['goods'], async () => i);
}
const loader = jest.fn(async () => 'reloaded');
// k0 应已被容量淘汰,重新加载;k3 仍命中
await cache.wrap('k0', ['goods'], loader);
expect(loader).toHaveBeenCalledTimes(1);
const hit = jest.fn(async () => -1);
await cache.wrap('k3', ['goods'], hit);
expect(hit).toHaveBeenCalledTimes(0);
} finally {
delete process.env.PUBLIC_CACHE_MAX_ENTRIES;
jest.useRealTimers();
}
});
it('version() 暴露域版本用于诊断,bump 递增', () => {
const before = cache.version('goods');
cache.bump('goods');
expect(cache.version('goods')).toBe(before + 1);
expect(cache.version('matrix')).toBe(before === 0 ? 0 : cache.version('matrix'));
});
it('未知域名直接抛错(编程错误早暴露)', async () => {
expect(() => cache.bump('nope' as PublicCacheDomain)).toThrow();
await expect(cache.wrap('k', ['nope' as PublicCacheDomain], async () => 1)).rejects.toThrow();
});
});
+171
View File
@@ -0,0 +1,171 @@
import { Injectable, Logger } from '@nestjs/common';
/**
* public 读路径的进程内缓存(性能整改 P0-1,详见
* docs/references/performance-review-public-port.md 与
* plans/refactor/public-capacity-10k-refactor.md)。
*
* 语义要点(全局约束 §2 的落地):
* - 分域版本号失效:写路径在事务提交成功后 bump 依赖域,依赖该域的缓存
* 条目立即作废——「等 TTL 自然过期」只是内存回收兜底,不是失效手段;
* - 条目记录写入时全部依赖域的版本快照,读时逐一比对,跨域依赖
* (列表同时依赖 goods+matrix+meta)天然联动失效;
* - loader 完成后复核版本:bump 发生在加载期间 → 结果照常返回但不入缓存,
* 杜绝「失效瞬间的并发读把旧值写回」竞态;
* - in-flight 合并:并发 miss 只触发一次 loader,防瞬时洪峰击穿缓存;
* - 单进程部署下进程内即全局缓存;扩容多副本时需换共享存储(计划 P1-3 阶梯 c)。
*
* 环境开关:PUBLIC_CACHE_DISABLED=true 全直通(应急);
* PUBLIC_CACHE_TTL_MS / PUBLIC_CACHE_MAX_ENTRIES 可调(默认 10 分钟 / 512 条)。
*/
export const PUBLIC_CACHE_DOMAINS = ['meta', 'goods', 'matrix'] as const;
export type PublicCacheDomain = (typeof PUBLIC_CACHE_DOMAINS)[number];
const DEFAULT_TTL_MS = 10 * 60 * 1000;
const DEFAULT_MAX_ENTRIES = 512;
interface CacheEntry {
value: unknown;
expiresAt: number;
/** 写入时各依赖域的版本快照——任一域 bump 即作废 */
versions: ReadonlyMap<PublicCacheDomain, number>;
}
@Injectable()
export class PublicCacheService {
private readonly logger = new Logger(PublicCacheService.name);
private readonly versions = new Map<PublicCacheDomain, number>(
PUBLIC_CACHE_DOMAINS.map((domain) => [domain, 0]),
);
private readonly entries = new Map<string, CacheEntry>();
private readonly pending = new Map<string, Promise<unknown>>();
/**
* 读穿透封装:命中(未过期且全部依赖域版本未变)直接返回缓存值,
* 否则执行 loader 并缓存。domains 声明该值依赖的数据域——
* 写路径 bump 其中任一域都会让本条目作废。
*/
async wrap<T>(
key: string,
domains: readonly PublicCacheDomain[],
loader: () => Promise<T>,
): Promise<T> {
this.assertDomains(domains);
if (this.disabled()) return loader();
const storeKey = `${[...domains].sort().join('+')}|${key}`;
const hit = this.readHit(storeKey);
if (hit !== undefined) return hit as T;
const inflight = this.pending.get(storeKey);
if (inflight) return inflight as Promise<T>;
const snapshot = this.snapshotVersions(domains);
const run: Promise<T> = (async () => {
const value = await loader();
if (this.versionsMatch(snapshot)) {
this.writeEntry(storeKey, value, snapshot);
} else {
this.logger.debug(`cache skip (version bumped during load): ${storeKey}`);
}
return value;
})();
this.pending.set(storeKey, run as Promise<unknown>);
// 失败与成功都要摘掉 pending,异常不得残留在途槽位
void run.catch(() => undefined).finally(() => this.pending.delete(storeKey));
return run;
}
/**
* 失效入口:写事务提交成功后调用。递增域版本并清除依赖该域的条目
* (版本号本身已保证正确性,清条目只为及时回收内存)。
*/
bump(...domains: PublicCacheDomain[]): void {
this.assertDomains(domains);
for (const domain of domains) {
this.versions.set(domain, (this.versions.get(domain) ?? 0) + 1);
}
if (this.entries.size === 0) return;
for (const [key, entry] of this.entries) {
if (domains.some((domain) => entry.versions.has(domain))) {
this.entries.delete(key);
}
}
}
/** 当前域版本(诊断/测试用) */
version(domain: PublicCacheDomain): number {
this.assertDomains([domain]);
return this.versions.get(domain) ?? 0;
}
private readHit(storeKey: string): unknown {
const entry = this.entries.get(storeKey);
if (!entry) return undefined;
if (Date.now() >= entry.expiresAt) {
this.entries.delete(storeKey);
return undefined;
}
if (!this.versionsMatch(entry.versions)) {
this.entries.delete(storeKey);
return undefined;
}
return entry.value;
}
private versionsMatch(snapshot: ReadonlyMap<PublicCacheDomain, number>): boolean {
for (const [domain, version] of snapshot) {
if ((this.versions.get(domain) ?? 0) !== version) return false;
}
return true;
}
private snapshotVersions(
domains: readonly PublicCacheDomain[],
): Map<PublicCacheDomain, number> {
return new Map(domains.map((domain) => [domain, this.versions.get(domain) ?? 0]));
}
private writeEntry(
storeKey: string,
value: unknown,
versions: ReadonlyMap<PublicCacheDomain, number>,
): void {
const maxEntries = this.maxEntries();
if (this.entries.size >= maxEntries) {
const now = Date.now();
for (const [key, entry] of this.entries) {
if (entry.expiresAt <= now) this.entries.delete(key);
}
while (this.entries.size >= maxEntries) {
const oldest = this.entries.keys().next().value;
if (oldest === undefined) break;
this.entries.delete(oldest);
}
}
this.entries.set(storeKey, { value, expiresAt: Date.now() + this.ttlMs(), versions });
}
private ttlMs(): number {
const parsed = Number(process.env.PUBLIC_CACHE_TTL_MS);
return Number.isFinite(parsed) && parsed > 0 ? parsed : DEFAULT_TTL_MS;
}
private maxEntries(): number {
const parsed = Number(process.env.PUBLIC_CACHE_MAX_ENTRIES);
return Number.isFinite(parsed) && parsed > 0 ? parsed : DEFAULT_MAX_ENTRIES;
}
private disabled(): boolean {
return process.env.PUBLIC_CACHE_DISABLED === 'true';
}
private assertDomains(domains: readonly PublicCacheDomain[]): void {
for (const domain of domains) {
if (!this.versions.has(domain)) {
throw new Error(`未知的 public 缓存域: ${String(domain)}`);
}
}
}
}
@@ -1,3 +1,4 @@
import { PublicCacheService } from './public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { Prisma } from '@prisma/client'; import { Prisma } from '@prisma/client';
import { PublicService } from './public.service'; import { PublicService } from './public.service';
@@ -24,8 +25,11 @@ describe('PublicService family block (PUBLIC_DETAIL_FROM_FAMILY)', () => {
let familyId = 0n; let familyId = 0n;
beforeAll(async () => { beforeAll(async () => {
// 本套件测数据语义(裸 prisma 改数后立即读公开端点),不走带 bump 的写路径——
// 禁用 public 缓存;缓存命中/失效语义由 public-cache.invalidation.spec.ts 覆盖
process.env.PUBLIC_CACHE_DISABLED = 'true';
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [PublicService, PrismaService], providers: [PublicService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(PublicService); service = moduleRef.get(PublicService);
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
@@ -71,11 +75,12 @@ describe('PublicService family block (PUBLIC_DETAIL_FROM_FAMILY)', () => {
createdFamilyIds.push(family.id); createdFamilyIds.push(family.id);
await prisma.originGood.update({ where: { id: og.id }, data: { familyId: family.id } }); await prisma.originGood.update({ where: { id: og.id }, data: { familyId: family.id } });
// 解析去运行时化:先整理派生标签,再重算(重算只聚合不解析) // 解析去运行时化:先整理派生标签,再重算(重算只聚合不解析)
const recomputeSvc = new FamilyRecomputeService(prisma); const recomputeSvc = new FamilyRecomputeService(prisma, new PublicCacheService());
const organizeSvc = new OrganizeService( const organizeSvc = new OrganizeService(
prisma, prisma,
recomputeSvc, recomputeSvc,
new ProductFamiliesService(prisma, recomputeSvc), new ProductFamiliesService(prisma, recomputeSvc),
new PublicCacheService(),
); );
await organizeSvc.deriveFamilyTags(family.id); await organizeSvc.deriveFamilyTags(family.id);
await recomputeSvc.recomputeFamily(family.id); await recomputeSvc.recomputeFamily(family.id);
@@ -196,13 +201,14 @@ describe('Good familyId derivation', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [PrismaService], providers: [PrismaService, PublicCacheService],
}).compile(); }).compile();
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
await prisma.onModuleInit(); await prisma.onModuleInit();
familiesService = new (require('../product-families/product-families.service').ProductFamiliesService)( familiesService = new (require('../product-families/product-families.service').ProductFamiliesService)(
prisma, prisma,
new FamilyRecomputeService(prisma), new FamilyRecomputeService(prisma, new PublicCacheService()),
new PublicCacheService(),
); );
}); });
+7 -3
View File
@@ -1,3 +1,4 @@
import { PublicCacheService } from './public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { BadRequestException, NotFoundException } from '@nestjs/common'; import { BadRequestException, NotFoundException } from '@nestjs/common';
import { PublicService } from './public.service'; import { PublicService } from './public.service';
@@ -20,8 +21,11 @@ describe('PublicService', () => {
let goodIds: bigint[] = []; let goodIds: bigint[] = [];
beforeAll(async () => { beforeAll(async () => {
// 本套件测数据语义(裸 prisma 改数后立即读公开端点),不走带 bump 的写路径——
// 禁用 public 缓存;缓存命中/失效语义由 public-cache.invalidation.spec.ts 覆盖
process.env.PUBLIC_CACHE_DISABLED = 'true';
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [PublicService, PrismaService], providers: [PublicService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(PublicService); service = moduleRef.get(PublicService);
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
@@ -188,7 +192,7 @@ describe('PublicService', () => {
}, },
}); });
// 物化族矩阵(公开详情 family 块依赖 priceMatrix 已重算) // 物化族矩阵(公开详情 family 块依赖 priceMatrix 已重算)
await new FamilyRecomputeService(prisma).recomputeFamily(family.id); await new FamilyRecomputeService(prisma, new PublicCacheService()).recomputeFamily(family.id);
// Seed a good in `otherCategory` so the "onlyHaveGoods" filter // Seed a good in `otherCategory` so the "onlyHaveGoods" filter
// returns more than one category. // returns more than one category.
@@ -406,7 +410,7 @@ describe('PublicService', () => {
expect(before.items).toHaveLength(1); expect(before.items).toHaveLength(1);
expect(before.items[0].price).toBe('50'); expect(before.items[0].price).toBe('50');
await new FamilyRecomputeService(prisma).recomputeFamily(family.id); await new FamilyRecomputeService(prisma, new PublicCacheService()).recomputeFamily(family.id);
const after = await service.getGoods({ const after = await service.getGoods({
page: 1, page: 1,
pageSize: 50, pageSize: 50,
+97 -11
View File
@@ -1,6 +1,7 @@
import { BadRequestException, Injectable, NotFoundException } from '@nestjs/common'; import { BadRequestException, Injectable, NotFoundException } from '@nestjs/common';
import { Category as PrismaCategory, Prisma } from '@prisma/client'; import { Category as PrismaCategory, Prisma } from '@prisma/client';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from './public-cache.service';
import { import {
PublicHomeGoodsQueryDto, PublicHomeGoodsQueryDto,
PublicQueryGoodDto, PublicQueryGoodDto,
@@ -86,9 +87,28 @@ interface TreeOrderMeta {
@Injectable() @Injectable()
export class PublicService { export class PublicService {
constructor(private readonly prisma: PrismaService) {} constructor(
private readonly prisma: PrismaService,
private readonly cache: PublicCacheService,
) {}
/**
* 以下读端点统一走 PublicCacheService(性能整改 P0-1TTL 只是内存回收
* 兜底,数据新鲜度由写路径 bump 保证,见 public-cache.service.ts)。
* 依赖域声明:
* - meta:分类/国家/标签组等低熵元数据(含 DEFAULT 排序的树序元数据)
* - goodsgoods 行/关联展示数据(含 position)
* - matrix:族 price_matrix 物化 JSON(族最低价聚合)
* 列表/详情同时展示元数据名与族价格 → 三域并依赖,任一写路径 bump 即失效。
*/
async getCategoriesTree(countryId?: string): Promise<PublicCategoryNodeDto[]> { async getCategoriesTree(countryId?: string): Promise<PublicCategoryNodeDto[]> {
return this.cache.wrap(`cat-tree:${countryId ?? 'all'}`, ['meta', 'goods'], () =>
this.loadCategoriesTree(countryId),
);
}
private async loadCategoriesTree(countryId?: string): Promise<PublicCategoryNodeDto[]> {
const goodsWhere: Prisma.GoodWhereInput = { const goodsWhere: Prisma.GoodWhereInput = {
familyId: { not: null }, familyId: { not: null },
originGood: { delisted: false }, originGood: { delisted: false },
@@ -133,6 +153,10 @@ export class PublicService {
} }
async getCountries(): Promise<PublicCountryDto[]> { async getCountries(): Promise<PublicCountryDto[]> {
return this.cache.wrap('countries', ['meta', 'goods'], () => this.loadCountries());
}
private async loadCountries(): Promise<PublicCountryDto[]> {
const rows = await this.prisma.country.findMany({ const rows = await this.prisma.country.findMany({
where: { goods: { some: { familyId: { not: null }, originGood: { delisted: false } } } }, where: { goods: { some: { familyId: { not: null }, originGood: { delisted: false } } } },
orderBy: [{ sortOrder: 'asc' }, { id: 'asc' }], orderBy: [{ sortOrder: 'asc' }, { id: 'asc' }],
@@ -141,6 +165,10 @@ export class PublicService {
} }
async getTags(): Promise<PublicTagDto[]> { async getTags(): Promise<PublicTagDto[]> {
return this.cache.wrap('tags', ['meta', 'goods'], () => this.loadTags());
}
private async loadTags(): Promise<PublicTagDto[]> {
const rows = await this.prisma.tag.findMany({ const rows = await this.prisma.tag.findMany({
where: { where: {
goodTags: { goodTags: {
@@ -158,6 +186,12 @@ export class PublicService {
} }
async getTagGroups(countryId?: string): Promise<PublicTagGroupFilterDto[]> { async getTagGroups(countryId?: string): Promise<PublicTagGroupFilterDto[]> {
return this.cache.wrap(`tag-groups:${countryId ?? 'all'}`, ['meta', 'goods'], () =>
this.loadTagGroups(countryId),
);
}
private async loadTagGroups(countryId?: string): Promise<PublicTagGroupFilterDto[]> {
const goodWhere: Prisma.GoodWhereInput = { const goodWhere: Prisma.GoodWhereInput = {
familyId: { not: null }, familyId: { not: null },
originGood: { delisted: false }, originGood: { delisted: false },
@@ -189,7 +223,44 @@ export class PublicService {
})); }));
} }
/**
* 缓存键 = 筛选参数(不含 page/pageSize):缓存的是已排序已分组的全量
* items,翻页在缓存命中后内存切片——所有页码共享同一份物化结果。
*/
private goodsListCacheKey(query: PublicQueryGoodDto): string {
return [
'goods-list',
query.countryId ?? '',
query.keyword ?? '',
query.categoryId ?? '',
query.minPrice ?? '',
query.maxPrice ?? '',
query.sort ?? 'DEFAULT',
JSON.stringify(query.tags ?? []),
].join('|');
}
async getGoods(query: PublicQueryGoodDto): Promise<PublicPaginatedGoods> { async getGoods(query: PublicQueryGoodDto): Promise<PublicPaginatedGoods> {
// 缓存的是「已排序已分组的全量 items」;分页切片必须在缓存外按请求执行,
// 否则后续页会拿到第一页的切片(缓存值会被多个页码共享)
const materialized = await this.cache.wrap(
this.goodsListCacheKey(query),
['goods', 'matrix', 'meta'],
() => this.loadGoodsMaterialized(query),
);
const start = (query.page - 1) * query.pageSize;
return {
items: materialized.items.slice(start, start + query.pageSize),
total: materialized.total,
page: query.page,
pageSize: query.pageSize,
};
}
private async loadGoodsMaterialized(query: PublicQueryGoodDto): Promise<{
items: PublicGoodDto[];
total: number;
}> {
// 无族商品(自定义)不进公开列表:只认族 // 无族商品(自定义)不进公开列表:只认族
const where: Prisma.GoodWhereInput = { const where: Prisma.GoodWhereInput = {
familyId: { not: null }, familyId: { not: null },
@@ -270,21 +341,18 @@ export class PublicService {
query.sort === 'PRICE_ASC' ? num(a.price) - num(b.price) : num(b.price) - num(a.price), query.sort === 'PRICE_ASC' ? num(a.price) - num(b.price) : num(b.price) - num(a.price),
); );
} }
const total = items.length; return { items, total: items.length };
const start = (query.page - 1) * query.pageSize;
return {
items: items.slice(start, start + query.pageSize),
total,
page: query.page,
pageSize: query.pageSize,
};
} }
/** /**
* 款序元数据:countries.sort_order(一级)+ 新树二/三级 categories.sort_order * 款序元数据:countries.sort_order(一级)+ 新树二/三级 categories.sort_order
* (款顺序,回填自排序表)。key 用 origin_goods.sds_category_id 关联商品→款。 * (款顺序,回填自排序表)。key 用 origin_goods.sds_category_id 关联商品→款。
*/ */
private async loadTreeOrderMeta(): Promise<TreeOrderMeta> { private loadTreeOrderMeta(): Promise<TreeOrderMeta> {
return this.cache.wrap('tree-order-meta', ['meta'], () => this.queryTreeOrderMeta());
}
private async queryTreeOrderMeta(): Promise<TreeOrderMeta> {
const [countries, leaves] = await Promise.all([ const [countries, leaves] = await Promise.all([
this.prisma.country.findMany({ select: { id: true, sortOrder: true } }), this.prisma.country.findMany({ select: { id: true, sortOrder: true } }),
this.prisma.$queryRaw< this.prisma.$queryRaw<
@@ -331,7 +399,11 @@ export class PublicService {
* 会让每个商品都携带整份矩阵(实测全量 ~330ms);PG 端展开聚合只回传 * 会让每个商品都携带整份矩阵(实测全量 ~330ms);PG 端展开聚合只回传
* 每族一个数字。非数字/缺失 price 的行跳过,与旧内存版过滤语义一致。 * 每族一个数字。非数字/缺失 price 的行跳过,与旧内存版过滤语义一致。
*/ */
private async loadFamilyMinPrices(): Promise<Map<string, string>> { private loadFamilyMinPrices(): Promise<Map<string, string>> {
return this.cache.wrap('family-min-prices', ['matrix'], () => this.queryFamilyMinPrices());
}
private async queryFamilyMinPrices(): Promise<Map<string, string>> {
const rows = await this.prisma.$queryRaw< const rows = await this.prisma.$queryRaw<
Array<{ family_id: bigint | string; min_price: Prisma.Decimal | null }> Array<{ family_id: bigint | string; min_price: Prisma.Decimal | null }>
>` >`
@@ -365,6 +437,12 @@ export class PublicService {
/** 族视角详情:代表 Good 提供公共字段(名称/主图/国家/分类),变体取全体成员并集 */ /** 族视角详情:代表 Good 提供公共字段(名称/主图/国家/分类),变体取全体成员并集 */
private async getGoodByFamilyId(familyId: bigint): Promise<PublicGoodDetailDto | null> { private async getGoodByFamilyId(familyId: bigint): Promise<PublicGoodDetailDto | null> {
return this.cache.wrap(`family-detail:${familyId.toString()}`, ['goods', 'matrix', 'meta'], () =>
this.loadGoodByFamilyId(familyId),
);
}
private async loadGoodByFamilyId(familyId: bigint): Promise<PublicGoodDetailDto | null> {
const [family, goods] = await Promise.all([ const [family, goods] = await Promise.all([
this.prisma.productFamily.findUnique({ where: { id: familyId } }), this.prisma.productFamily.findUnique({ where: { id: familyId } }),
this.prisma.good.findMany({ this.prisma.good.findMany({
@@ -405,6 +483,14 @@ export class PublicService {
} }
async getHomeGoods(query: PublicHomeGoodsQueryDto): Promise<PublicGoodDto[]> { async getHomeGoods(query: PublicHomeGoodsQueryDto): Promise<PublicGoodDto[]> {
return this.cache.wrap(
`home:${query.countryId ?? 'all'}:${query.limit}`,
['goods', 'matrix', 'meta'],
() => this.loadHomeGoods(query),
);
}
private async loadHomeGoods(query: PublicHomeGoodsQueryDto): Promise<PublicGoodDto[]> {
const rows = await this.prisma.good.findMany({ const rows = await this.prisma.good.findMany({
where: { where: {
positionId: { not: null }, positionId: { not: null },
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { Prisma } from '@prisma/client'; import { Prisma } from '@prisma/client';
import { SyncService } from './sync.service'; import { SyncService } from './sync.service';
@@ -27,6 +28,7 @@ describe('SyncService family hooks', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [ providers: [
PublicCacheService,
SyncService, SyncService,
{ {
provide: SdsClientService, provide: SdsClientService,
+3 -1
View File
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { ConfigModule } from '@nestjs/config'; import { ConfigModule } from '@nestjs/config';
import { import {
@@ -25,6 +26,7 @@ describe('SyncService', () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
imports: [ConfigModule.forRoot({ isGlobal: true })], imports: [ConfigModule.forRoot({ isGlobal: true })],
providers: [ providers: [
PublicCacheService,
SyncService, SyncService,
{ provide: SdsClientService, useValue: sdsMock }, { provide: SdsClientService, useValue: sdsMock },
{ provide: FamilyRecomputeService, useValue: { enqueue: jest.fn() } }, { provide: FamilyRecomputeService, useValue: { enqueue: jest.fn() } },
@@ -295,7 +297,7 @@ describe('SyncService product detail scopes', () => {
const familyRecompute = { const familyRecompute = {
enqueue: jest.fn(), enqueue: jest.fn(),
} as unknown as FamilyRecomputeService; } as unknown as FamilyRecomputeService;
const scopedService = new SyncService(prisma, sds, familyRecompute); const scopedService = new SyncService(prisma, sds, familyRecompute, new PublicCacheService());
jest jest
.spyOn(scopedService as any, 'persistProductDetail') .spyOn(scopedService as any, 'persistProductDetail')
.mockResolvedValue(undefined); .mockResolvedValue(undefined);
+8
View File
@@ -15,6 +15,7 @@ import {
} from './sds-client.service'; } from './sds-client.service';
import { normalizeProductDetail } from './sds-product-detail.mapper'; import { normalizeProductDetail } from './sds-product-detail.mapper';
import { FamilyRecomputeService } from '../product-families/family-recompute.service'; import { FamilyRecomputeService } from '../product-families/family-recompute.service';
import { PublicCacheService } from '../public/public-cache.service';
export interface CategorySyncResult { export interface CategorySyncResult {
@@ -96,6 +97,7 @@ export class SyncService {
private readonly prisma: PrismaService, private readonly prisma: PrismaService,
private readonly sds: SdsClientService, private readonly sds: SdsClientService,
private readonly familyRecompute: FamilyRecomputeService, private readonly familyRecompute: FamilyRecomputeService,
private readonly publicCache: PublicCacheService,
) {} ) {}
/** /**
@@ -298,6 +300,8 @@ export class SyncService {
message: `inserted=${inserted} updated=${updated} total=${flat.length} staleDeleted=${deletedStale}`, message: `inserted=${inserted} updated=${updated} total=${flat.length} staleDeleted=${deletedStale}`,
}, },
}); });
// 分类树/树序元数据变化 → public 缓存失效(全局约束 §2:不等 TTL)
this.publicCache.bump('meta', 'goods');
return { inserted, updated, total: flat.length, deletedStale }; return { inserted, updated, total: flat.length, deletedStale };
} catch (err) { } catch (err) {
const message = err instanceof Error ? err.message : String(err); const message = err instanceof Error ? err.message : String(err);
@@ -407,6 +411,8 @@ export class SyncService {
message: `inserted=${inserted} updated=${updated} total=${total} delisted=${delistedCount} reactivated=${reactivatedCount} leafCategories=${leafRows.length}`, message: `inserted=${inserted} updated=${updated} total=${total} delisted=${delistedCount} reactivated=${reactivatedCount} leafCategories=${leafRows.length}`,
}, },
}); });
// 链接镜像(名称/图/价/上下架)变化 → public 商品列表/详情失效
this.publicCache.bump('goods');
return { return {
inserted, inserted,
updated, updated,
@@ -677,6 +683,8 @@ export class SyncService {
}, },
}); });
}); });
// 详情/变体/价格落库 → public 详情缓存失效;族矩阵变化由重算钩子 bump matrix
this.publicCache.bump('goods');
// 族成员的详情/变体变化 → 异步重算该族(进程内去重) // 族成员的详情/变体变化 → 异步重算该族(进程内去重)
await this.maybeEnqueueFamilyRecompute(originGoodId); await this.maybeEnqueueFamilyRecompute(originGoodId);
} }
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { ConflictException, NotFoundException } from '@nestjs/common'; import { ConflictException, NotFoundException } from '@nestjs/common';
import { TagGroupsService } from './tag-groups.service'; import { TagGroupsService } from './tag-groups.service';
@@ -10,7 +11,7 @@ describe('TagGroupsService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [TagGroupsService, PrismaService], providers: [TagGroupsService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(TagGroupsService); service = moduleRef.get(TagGroupsService);
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
+17 -5
View File
@@ -5,13 +5,17 @@ import {
NotFoundException, NotFoundException,
} from '@nestjs/common'; } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from '../public/public-cache.service';
import { CreateTagGroupDto } from './dto/create-tag-group.dto'; import { CreateTagGroupDto } from './dto/create-tag-group.dto';
import { UpdateTagGroupDto } from './dto/update-tag-group.dto'; import { UpdateTagGroupDto } from './dto/update-tag-group.dto';
import { ReorderTagGroupsDto } from './dto/reorder-tag-groups.dto'; import { ReorderTagGroupsDto } from './dto/reorder-tag-groups.dto';
@Injectable() @Injectable()
export class TagGroupsService { export class TagGroupsService {
constructor(private readonly prisma: PrismaService) {} constructor(
private readonly prisma: PrismaService,
private readonly publicCache: PublicCacheService,
) {}
findAll() { findAll() {
return this.prisma.tagGroup.findMany({ return this.prisma.tagGroup.findMany({
@@ -31,7 +35,7 @@ export class TagGroupsService {
async create(dto: CreateTagGroupDto) { async create(dto: CreateTagGroupDto) {
try { try {
return await this.prisma.tagGroup.create({ const created = await this.prisma.tagGroup.create({
data: { data: {
groupName: dto.groupName, groupName: dto.groupName,
groupIcon: dto.groupIcon ?? null, groupIcon: dto.groupIcon ?? null,
@@ -39,6 +43,8 @@ export class TagGroupsService {
sortOrder: dto.sortOrder ?? 0, sortOrder: dto.sortOrder ?? 0,
}, },
}); });
this.publicCache.bump('meta');
return created;
} catch (err) { } catch (err) {
if ( if (
err instanceof Prisma.PrismaClientKnownRequestError && err instanceof Prisma.PrismaClientKnownRequestError &&
@@ -58,7 +64,9 @@ export class TagGroupsService {
if (dto.groupColor !== undefined) data.groupColor = dto.groupColor; if (dto.groupColor !== undefined) data.groupColor = dto.groupColor;
if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder; if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder;
try { try {
return await this.prisma.tagGroup.update({ where: { id }, data }); const updated = await this.prisma.tagGroup.update({ where: { id }, data });
this.publicCache.bump('meta');
return updated;
} catch (err) { } catch (err) {
if ( if (
err instanceof Prisma.PrismaClientKnownRequestError && err instanceof Prisma.PrismaClientKnownRequestError &&
@@ -72,11 +80,13 @@ export class TagGroupsService {
async remove(id: bigint) { async remove(id: bigint) {
await this.findOne(id); await this.findOne(id);
return this.prisma.tagGroup.delete({ where: { id } }); const removed = await this.prisma.tagGroup.delete({ where: { id } });
this.publicCache.bump('meta');
return removed;
} }
async reorder(dto: ReorderTagGroupsDto) { async reorder(dto: ReorderTagGroupsDto) {
return this.prisma.$transaction( const result = await this.prisma.$transaction(
dto.items.map((item) => dto.items.map((item) =>
this.prisma.tagGroup.update({ this.prisma.tagGroup.update({
where: { id: BigInt(item.id) }, where: { id: BigInt(item.id) },
@@ -84,5 +94,7 @@ export class TagGroupsService {
}), }),
), ),
); );
this.publicCache.bump('meta');
return result;
} }
} }
+2 -1
View File
@@ -1,3 +1,4 @@
import { PublicCacheService } from '../public/public-cache.service';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { ConflictException } from '@nestjs/common'; import { ConflictException } from '@nestjs/common';
import { validate } from 'class-validator'; import { validate } from 'class-validator';
@@ -13,7 +14,7 @@ describe('TagsService', () => {
beforeAll(async () => { beforeAll(async () => {
const moduleRef = await Test.createTestingModule({ const moduleRef = await Test.createTestingModule({
providers: [TagsService, PrismaService], providers: [TagsService, PrismaService, PublicCacheService],
}).compile(); }).compile();
service = moduleRef.get(TagsService); service = moduleRef.get(TagsService);
prisma = moduleRef.get(PrismaService); prisma = moduleRef.get(PrismaService);
+17 -5
View File
@@ -5,13 +5,17 @@ import {
NotFoundException, NotFoundException,
} from '@nestjs/common'; } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { PublicCacheService } from '../public/public-cache.service';
import { CreateTagDto } from './dto/create-tag.dto'; import { CreateTagDto } from './dto/create-tag.dto';
import { UpdateTagDto } from './dto/update-tag.dto'; import { UpdateTagDto } from './dto/update-tag.dto';
import { ReorderTagsDto } from './dto/reorder-tags.dto'; import { ReorderTagsDto } from './dto/reorder-tags.dto';
@Injectable() @Injectable()
export class TagsService { export class TagsService {
constructor(private readonly prisma: PrismaService) {} constructor(
private readonly prisma: PrismaService,
private readonly publicCache: PublicCacheService,
) {}
findAll() { findAll() {
return this.prisma.tag.findMany({ return this.prisma.tag.findMany({
@@ -31,7 +35,7 @@ export class TagsService {
async create(dto: CreateTagDto) { async create(dto: CreateTagDto) {
try { try {
return await this.prisma.tag.create({ const created = await this.prisma.tag.create({
data: { data: {
tagName: dto.tagName, tagName: dto.tagName,
tagColor: dto.tagColor ?? null, tagColor: dto.tagColor ?? null,
@@ -41,6 +45,8 @@ export class TagsService {
sortOrder: dto.sortOrder ?? 0, sortOrder: dto.sortOrder ?? 0,
}, },
}); });
this.publicCache.bump('meta');
return created;
} catch (err) { } catch (err) {
if ( if (
err instanceof Prisma.PrismaClientKnownRequestError && err instanceof Prisma.PrismaClientKnownRequestError &&
@@ -66,7 +72,9 @@ export class TagsService {
} }
if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder; if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder;
try { try {
return await this.prisma.tag.update({ where: { id }, data }); const updated = await this.prisma.tag.update({ where: { id }, data });
this.publicCache.bump('meta');
return updated;
} catch (err) { } catch (err) {
if ( if (
err instanceof Prisma.PrismaClientKnownRequestError && err instanceof Prisma.PrismaClientKnownRequestError &&
@@ -80,11 +88,13 @@ export class TagsService {
async remove(id: bigint) { async remove(id: bigint) {
await this.findOne(id); await this.findOne(id);
return this.prisma.tag.delete({ where: { id } }); const removed = await this.prisma.tag.delete({ where: { id } });
this.publicCache.bump('meta');
return removed;
} }
async reorder(dto: ReorderTagsDto) { async reorder(dto: ReorderTagsDto) {
return this.prisma.$transaction( const result = await this.prisma.$transaction(
dto.items.map((item) => dto.items.map((item) =>
this.prisma.tag.update({ this.prisma.tag.update({
where: { id: BigInt(item.id) }, where: { id: BigInt(item.id) },
@@ -100,5 +110,7 @@ export class TagsService {
}), }),
), ),
); );
this.publicCache.bump('meta');
return result;
} }
} }
+3
View File
@@ -17,6 +17,9 @@ COPY apps/admin apps/admin
RUN pnpm --filter @inkreach/admin exec vite build RUN pnpm --filter @inkreach/admin exec vite build
FROM nginx:1.27-alpine FROM nginx:1.27-alpine
# 性能整改 P0-2:替换官方主配置(worker_connections 1024 → 16384、gzip、访问日志);
# conf.dadmin.conf / admin.v2.conf)仍按服务挂载/拷贝
COPY deploy/nginx/nginx.conf /etc/nginx/nginx.conf
COPY --from=build /app/apps/admin/dist /usr/share/nginx/html/v2/admin COPY --from=build /app/apps/admin/dist /usr/share/nginx/html/v2/admin
COPY deploy/nginx/admin.conf /etc/nginx/conf.d/default.conf COPY deploy/nginx/admin.conf /etc/nginx/conf.d/default.conf
COPY deploy/h5 /usr/share/nginx/html/v2/h5 COPY deploy/h5 /usr/share/nginx/html/v2/h5
+23 -1
View File
@@ -9,6 +9,20 @@ services:
v2-postgres: v2-postgres:
image: postgres:16-alpine image: postgres:16-alpine
restart: unless-stopped restart: unless-stopped
# 性能整改 P0-3plans/refactor/public-capacity-10k-refactor.md):
# - statement_timeout=10s:慢查询不无限期占住连接(池排队由此可控);
# - max_connections=200:为未来 api 副本/同步 worker 预留(单 api 池 50);
# - shared_buffers/effective_cache_size14GB 内存机的工作集调优(数据 ~几十 MB,512MB 充裕)。
# 部署注意(全局约束 §1):改参数需重启容器(秒级中断)——先 pg_dump -Fc + 配置快照
# 落 deploy/backups/<时间戳>/ 再低峰 force-recreate。
command:
[
"postgres",
"-c", "statement_timeout=10000",
"-c", "max_connections=200",
"-c", "shared_buffers=512MB",
"-c", "effective_cache_size=1536MB",
]
environment: environment:
POSTGRES_USER: inkreach POSTGRES_USER: inkreach
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
@@ -32,7 +46,10 @@ services:
environment: environment:
NODE_ENV: production NODE_ENV: production
PORT: 3001 PORT: 3001
DATABASE_URL: postgresql://inkreach:${POSTGRES_PASSWORD}@v2-postgres:5432/inkreach # 性能整改 P0-3:池上限 50 < PG max_connections 200(同步/运维连接留余量);
# pool_timeout 单位是秒(Prisma 默认 10)——池耗尽 3s 快速失败而非挂 10s。
# 不要加 statement_cache_size=0pgbouncer 专用,直连禁 prepared statement 反而拖慢)。
DATABASE_URL: postgresql://inkreach:${POSTGRES_PASSWORD}@v2-postgres:5432/inkreach?connection_limit=50&pool_timeout=3
JWT_SECRET: ${JWT_SECRET} JWT_SECRET: ${JWT_SECRET}
CORS_ORIGINS: "*" CORS_ORIGINS: "*"
# 激活 product-family 聚合公开读路径 # 激活 product-family 聚合公开读路径
@@ -65,6 +82,11 @@ services:
ports: ports:
- "80:80" - "80:80"
- "443:443" - "443:443"
# 性能整改 P0-24 worker × 16384 连接 + 上游 keepalive 需要容器 fd 预算匹配
ulimits:
nofile:
soft: 65536
hard: 65536
volumes: volumes:
- ./nginx/admin.conf:/etc/nginx/conf.d/default.conf:ro - ./nginx/admin.conf:/etc/nginx/conf.d/default.conf:ro
- ./certbot/www:/var/www/certbot:ro - ./certbot/www:/var/www/certbot:ro
+38 -6
View File
@@ -1,3 +1,15 @@
# 上游长连接池(性能整改 P0-2):与 v2-api 复用连接,消除每请求 TCP 建连;
# keepalive 连接需配合 proxy_http_version 1.1 + proxy_set_header Connection ""
upstream v2_api {
server v2-api:3001;
keepalive 64;
}
# 防洪峰兜底限流(app 层 120 req/min/IP 更严,这里只挡瞬时洪峰/扫描;
# 默认 503 会误判服务故障,显式 429)
limit_req_zone $binary_remote_addr zone=public_api:10m rate=50r/s;
limit_req_status 429;
server { server {
listen 80; listen 80;
server_name official.inkreach.cc; server_name official.inkreach.cc;
@@ -27,6 +39,7 @@ server {
root /usr/share/nginx/html; root /usr/share/nginx/html;
# ACME challenge for cert renewals
location /.well-known/acme-challenge/ { location /.well-known/acme-challenge/ {
root /var/www/certbot; root /var/www/certbot;
} }
@@ -60,22 +73,32 @@ server {
} }
location /v2-api/ { location /v2-api/ {
proxy_pass http://v2-api:3001/; proxy_pass http://v2_api/;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 5s;
proxy_read_timeout 30s;
proxy_send_timeout 30s;
limit_req zone=public_api burst=200 nodelay;
} }
# H5 历史构建的 API 基址是同源相对路径 /public/*BASE_URL="/"),直通 v2 api(路径原样透传) # H5 历史构建的 API 基址是同源相对路径 /public/*BASE_URL="/"),直通 v2 api(路径原样透传)
location /public/ { location /public/ {
proxy_pass http://v2-api:3001; proxy_pass http://v2_api;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 5s;
proxy_read_timeout 30s;
proxy_send_timeout 30s;
limit_req zone=public_api burst=200 nodelay;
} }
location = /h5 { location = /h5 {
@@ -86,16 +109,25 @@ server {
try_files $uri $uri/ /h5/index.html; try_files $uri $uri/ /h5/index.html;
} }
# Uploaded files served by the API # Uploaded files served by the API.
# 缓存头注意:若运营有「同名替换图片」操作请改短 max-age 或改用版本化 URL
location /uploads/ { location /uploads/ {
proxy_pass http://v2-api:3001/uploads/; proxy_pass http://v2_api/uploads/;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host; proxy_set_header Host $host;
expires 30d;
add_header Cache-Control "public";
} }
# Static product assets served by the API # Static product assets served by the API(哈希文件名,可长缓存)
location /assets/ { location /assets/ {
proxy_pass http://v2-api:3001/assets/; proxy_pass http://v2_api/assets/;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host; proxy_set_header Host $host;
expires 30d;
add_header Cache-Control "public";
} }
# Website placeholder until the public site is deployed # Website placeholder until the public site is deployed
+6
View File
@@ -25,8 +25,14 @@ server {
add_header Cache-Control "no-cache"; add_header Cache-Control "no-cache";
} }
# v2 版 H5(构建时 router.base 必须为 /v2/h5/,产物放 /usr/share/nginx/html/v2/h5
# 性能整改 P0-2:哈希文件名的构建产物长缓存,index.html 永远 no-cache(上面精确匹配优先)
location /v2/h5/ { location /v2/h5/ {
try_files $uri $uri/ /v2/h5/index.html; try_files $uri $uri/ /v2/h5/index.html;
location ~* ^/v2/h5/(static|assets|img|fonts?)/ {
expires 30d;
add_header Cache-Control "public";
}
} }
location / { location / {
+44
View File
@@ -0,0 +1,44 @@
# 边缘 nginx 主配置(性能整改 P0-2plans/refactor/public-capacity-10k-refactor.md
# 由 deploy/admin.Dockerfile COPY 进镜像替换官方默认——官方默认 worker_connections 1024
# × 4 worker = 4096 连接硬顶,扛不住 1w 并发口径;events{} 只存在于主配置,conf.d 覆盖不了。
# 注意:admin 与 v2-admin 共用本 Dockerfile,两侧都会得到本主配置(upstream/gzip 对
# v2-admin 无害;worker_connections 抬升对静态服务同样受益)。
user nginx;
worker_processes auto;
# 每 worker 连接预算上限(4 × 16384 客户端连接 + 上游 keepalive 连接)
worker_rlimit_nofile 65536;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 16384;
multi_accept on;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
# 1w 并发下无访问日志无从排障:带上游耗时的结构化日志(rt=总耗时 urt=上游耗时)
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" "$http_user_agent" '
'rt=$request_time uct=$upstream_connect_time urt=$upstream_response_time';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# 压缩:API JSON(详情含 ~11KB price_matrix)与 SPA 静态资源,带宽降 5~10x
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 5;
gzip_min_length 1024;
gzip_types application/json application/javascript text/css text/plain text/xml image/svg+xml;
include /etc/nginx/conf.d/*.conf;
}
@@ -0,0 +1,122 @@
# public 端口性能审查报告(1w 瞬时并发请求口径)
- 审查日期:2026-09-03
- 审查对象:https://official.inkreach.cc 的公开访问链路(public 端口 = 边缘 nginx 80/443 → v2-api:3001 → v2-postgres
- 审查口径:**1w 瞬时并发请求**——同一时刻 1 万个请求同时到达边缘(压力测试式,万 QPS 级)
- 结论:**当前形态不能承压,差距约 2~3 个数量级**。瓶颈全部位于「配置与架构层」,与商品数据规模无关(goods 仅 240 行),改造后单机有希望在口径内达标,需压测实证。
---
## 1. 现状架构
单机(4 核 / 14GB / 无资源限额)docker compose 单栈 4 容器,全部单副本:
```
公网:80/443 (official.inkreach.cc)
└─ deploy-admin-1 (边缘 nginx 1.27-alpine,官方默认主配置)
├─ /public/ /v2-api/ /uploads/ /assets/ ──► deploy-v2-api-1 (NestJS :3001)
├─ /v2/admin/ /v2/h5/ ──► deploy-v2-admin-1 (SPA 静态)
└─ /admin/ /h5/ ──► try_files 本地静态
deploy-v2-api-1 ──► deploy-v2-postgres-1 (postgres:16-alpine)
```
关键事实(均已核对源码/配置原文):
| 项 | 现状 | 出处 |
| --- | --- | --- |
| nginx worker | `worker_processes auto`4 worker)× `worker_connections 1024`**最多 4096 条客户端连接** | 容器内官方默认 nginx.conf`deploy/nginx/admin.conf` 仅覆盖 conf.d |
| 边缘路由 | `/public/` 路径原样透传至 v2-api:3001 | `deploy/nginx/admin.conf` L72-79 |
| 压缩 | 无 gzipJSON/JS/图片全裸传) | nginx 默认 gzip 注释关闭;main.ts 无 compression 中间件 |
| 上游 keepalive | 无(每请求新建到 v2-api 的 TCP 连接) | admin.conf 无 upstream keepalive |
| 限流 | 全局限流 120 req/min/IP | `apps/api/src/app.module.ts` L27-32 |
| API 连接池 | Prisma 零配置 → 默认池 = 4核×2+1 = **9 条连接**,排队默认 10s 后报错 | `apps/api/src/prisma/prisma.service.ts` L12DATABASE_URL 无参数(compose L35 |
| PG | `max_connections=100`(默认)、无 statement_timeout、shared_buffers 默认 | docker-compose.yml 未覆盖;postgres:16 默认 |
| 缓存 | 全链路零缓存(无 Redis/cache-manager/nginx proxy_cache | 全仓 grep 零命中 |
| 同步任务 | 整点分类同步(长事务串行 ~226 分类)、03:30 详情全量同步 + 逐单族重算,与 public **同进程同池** | `apps/api/src/sync/sync.service.ts` L106-114、L138-145 |
| 数据规模 | goods 240 / origin_goods 542 / variants 12,155 / categories 243 / countries 12 | 生产导出 data-dump.json2026-08-22 |
## 2. 逐层瓶颈证据
### 2.1 边缘 nginx:第 0 层就爆(连接数硬顶 4096)
`worker_connections 1024` × 4 worker = **4096 条并发客户端连接**。HTTP/2 下每个浏览器占 1 条连接;1 万并发请求到场时,约 **六成连接进不了 accept 队列**(内核 backlog 默认 511,其余直接 reset/拒绝)。这是与后端优化无关的硬天花板。
### 2.2 API 层:单进程 + 9 连接池,直连 DB 吞吐 ~20~50 req/s
- **`GET /public/goods` 无 SQL 分页**`findMany` 不带 take/skip 全量拉取匹配行(`public.service.ts` L234),再内存分组、内存分页(L249-280)。数据量百级时可行,但每次请求的成本是「全表扫描 + 多表联表 + JS 排序」。
- **每次请求两个全表聚合**
- `loadFamilyMinPrices()`L334-351):`CROSS JOIN LATERAL jsonb_array_elements(price_matrix->'rows')` 展开**全部**族(每族 ~11KB JSONB),`/public/goods` 每页与 `/public/home-goods` 每次都跑;注释实测同类全量扫描 ~330ms(L329-333)。
- DEFAULT 排序额外 `loadTreeOrderMeta()`L287-307):全 countries + 全 categories 联表 raw SQL。
- **N+1 与串行链路**
- 分类树 `getCategoriesTree`:逐祖先串行 `findUnique`L102-113)。
- 详情 `getGoodByFamilyId`:3 条查询并发 + 响应内嵌整份 ~11KB price_matrix、全部变体 designData`resolveCategoryIcon` 串行向上最多 10 次 findUniqueL552-565)。
- **`keyword``ILIKE '%kw%'`**L199),无 pg_trgm 索引 → 必 seq scan(当前 240 行无感,量级放大后失效)。
- 单请求 DB 成本合计 ~300~500ms(列表)与 ~100~300ms(详情),9 条连接串行化后:**列表接口吞吐 ≈ 9 ÷ 0.4s ≈ 20~25 req/s**,全端点混合 ~30~60 req/s。
### 2.3 数据库层:无防护,池耗尽即排队超时
- Prisma 池 9 < PG max_connections 100,但**无任何超时参数**:慢查询可无限期占住连接;同步长事务(分类同步单事务串行 ~226 分类、详情同步逐商品 upsert)与 public 抢同一池。
- 池排队默认 10s 超时(P2028):1 万瞬时请求直连 DB 时,绝大多数请求排队 >10s 直接报错。
- 无 statement_timeout,无连接池水位监控。
### 2.4 传输层:无 gzip、无缓存头
- 详情响应内嵌 ~11KB price_matrix + 全部变体 designData;列表响应平均 ~2-5KB。按 ~20KB/响应粗算,1 万 req/s ≈ **1.6Gbps**,可打满常见 1Gbps 出口带宽;gzip 后 JSON 可压 5~10 倍。
- `/uploads/` `/assets/` 静态资源无任何 Cache-Control`main.ts` L73-81),每次访问重复下载。
### 2.5 全链路零缓存(杠杆最高的修复点)
分类树、国家、标签组、首页商品、商品列表、详情全部每次实时查库。**数据变更频率为小时级同步**syncCategories/syncProducts 整点跑),这意味着对 public 读路径引入分钟级 TTL 缓存几乎无损数据新鲜度,却能把 DB 热路径负载降 95%+。
### 2.6 同步任务竞争
整点分类同步长事务、03:30 详情全量同步 + 逐单族重算(fire-and-forget 进程内任务)与 public 共享同一 9 连接池。同步窗口内池被长事务/串行往返占满时,在线请求直接排队甚至超时。
### 2.7 压测陷阱:Throttler 429
全局限流 120 req/min/IP`app.module.ts` L27-32)。真实 1 万用户分布在不同 IP 无影响;但**单机/少 IP 压测流量会被 429 打满**,压测必须使用分布式源(≥数十个出口 IP)或临时抬高 `THROTTLE_LIMIT`,否则测出来的是限流器而不是系统容量。
## 3. 容量估算与结论
| 层 | 上限 | 1w 瞬时并发需求 | 差距 |
| --- | --- | --- | --- |
| nginx 连接数 | 4096 | 10000 | ~2.4x |
| API 直连 DB 吞吐 | ~30-60 req/s | ~10000 req/s | ~200-300x |
| 出口带宽(无 gzip | ~1 Gbps | ~1.6 Gbps | ~1.6x |
| PG 连接 | 100 | 9 池上限前置 | — |
**结论:当前形态不能承压 1w 瞬时并发请求**。即便并发被 nginx 放进来,DB 直连吞吐差两个数量级以上,池排队 10s 超时会把瞬时洪峰转为大面积 5xx。
**方向**:瓶颈 100% 属于「配置与架构层」——进程内存缓存(TTL 5~10min + 同步后失效)可把 DB 热路径负载降 ~95-99%nginx 连接数/gzip/keepalive/限流补齐后边缘可达 1w+ 连接;池参数与 statement_timeout 防排队超时与慢查询占池。改造后单机 4 核全缓存热路径(万级 req/s 简单 JSON 响应)有达标可能,但必须用分布式压测实证,达标口径见下。
## 4. 整改方向(摘要)
> **实施进度(2026-09-03**P0-1/P0-2/P0-3 已在 `refactor/public-capacity-10k-yeuimu` 分支完成并全量测试通过(238/238)——进程内分域缓存 + 全写路径即时失效、nginx 主配置/keepalive/gzip/限流/缓存头、连接池与 PG 参数。**已于 2026-09-03 08:38 部署到生产**(备份存档 `deploy/backups/20260903-083800-p0-perf-deploy/`,切换停机 23s):部署后实测——home-goods 首次 90ms/缓存命中 12ms;详情 247KB → gzip 23KB10.7x);静态资源 30d 缓存头生效;nginx 洪峰限流 429 生效且随窗口恢复(注意:单 IP 高频实测同时会打满 app 层 120 req/min 限流,属双层按设计工作);PG statement_timeout=10s/max_connections=200/shared_buffers=512MB 生效;行数 diff 零变化。**P1-3 压测基线(镜像栈 + 分布式源)为验收前置,未达标按计划扩展阶梯升级。**
完整任务分解见 [plans/refactor/public-capacity-10k-refactor.md](../plans/refactor/public-capacity-10k-refactor.md)
- **P0-1** 进程内存缓存层(NestJS cache-manager 或等价实现):categories/countries/tags/tag-groups/home-goods/goods(全量物化+内存分页)/goods/:id/族最低价聚合/树序元数据;**所有写路径(sync/族重算/admin 写/上传)提交后按域版本号主动失效**(全局约束:等待 TTL 过期不是兜底)
- **P0-2** nginx 调优:`worker_connections 16384``worker_rlimit_nofile`、upstream `keepalive 64`、gzip on、/uploads /assets 缓存头、limit_req 兜底、proxy 超时显式化
- **P0-3** 连接与超时:`DATABASE_URL?connection_limit=50&pool_timeout=3`pool_timeout 单位为秒)、PG `statement_timeout=10s`、shared_buffers/effective_cache_size 调参
- **P1-1** `/public/goods` SQL 分页(take/skip + count
- **P1-2** 同步任务与 public 隔离(错峰/批量化/独立 worker 开关)
- **P1-3** 压测基线:一次性 docker postgres + 生产 dump 测关键 SQL 真实耗时;k6 分布式压测
- **P2** 静态资源边缘直出、pg_trgm 索引、连接池水位监控告警
## 5. 验收口径(改造后)
- 分布式压测(≥数十个出口 IP,或 `THROTTLE_LIMIT` 临时抬高):**1 万并发瞬时请求**下:
- p95 延迟 < 300msCPU 侧),p99 < 1s
- 错误率 < 0.1%,无 nginx 连接拒绝/重置
- PG 连接水位不触顶(< 80%),无池排队超时
- 公布达标数字(req/s、p95/p99、错误率)后视为验收通过;不达标则加 v2-api 副本(上游 keepalive 已就位,扩容仅需加容器)。
## 附:相关文件索引
- 边缘路由:`deploy/nginx/admin.conf`
- 编排:`deploy/docker-compose.yml`
- 公开 API 实现:`apps/api/src/public/public.controller.ts``apps/api/src/public/public.service.ts`
- 入口中间件:`apps/api/src/main.ts`;限流:`apps/api/src/app.module.ts`
- 连接池:`apps/api/src/prisma/prisma.service.ts`
- 同步任务:`apps/api/src/sync/sync.service.ts``apps/api/src/product-families/family-recompute.service.ts`
- 数据规模:`deploy/data-dump.json`2026-08-22 导出)
+8 -1
View File
@@ -47,6 +47,13 @@ inkreach-official/
`v2-postgres`(数据卷 `deploy-v2_pgdata` external 引用,v2 数据未迁移)。 `v2-postgres`(数据卷 `deploy-v2_pgdata` external 引用,v2 数据未迁移)。
- 域名路由:`/v2/admin/`SPA)、`/v2-api/`(小程序/后台 API,去前缀转发)、`/public/` - 域名路由:`/v2/admin/`SPA)、`/v2-api/`(小程序/后台 API,去前缀转发)、`/public/`
H5 同源 API)、`/uploads/``/assets/` 均由 v2-api 服务;`/` 302 到 `/v2/admin/` H5 同源 API)、`/uploads/``/assets/` 均由 v2-api 服务;`/` 302 到 `/v2/admin/`
- 性能整改 P02026-09-03plans/refactor/public-capacity-10k-refactor.md):边缘 nginx
主配置改为仓库文件 `deploy/nginx/nginx.conf`worker_connections 16384 + gzip + 访问日志,
经 admin.Dockerfile COPY,改后须重建镜像);admin.conf 含 upstream keepalive/限流(429)/
代理超时/静态缓存头;v2-postgres 显式参数(statement_timeout=10s、max_connections=200、
shared_buffers=512MB);v2-api 的 DATABASE_URL 带 `connection_limit=50&pool_timeout=3`
pool_timeout 单位为秒);admin 服务 ulimits nofile 65536。改 PG 参数/重启前按全局约束
先 pg_dump + 配置快照落 `deploy/backups/<时间戳>/`
- v1(旧 api+postgres,曾支撑 `/public/` 与旧后台)已退役:切换时容器停用未删, - v1(旧 api+postgres,曾支撑 `/public/` 与旧后台)已退役:切换时容器停用未删,
库终档与全部配置快照及回滚手册见 `deploy/backups/consolidation-*/RESTORE.md` 库终档与全部配置快照及回滚手册见 `deploy/backups/consolidation-*/RESTORE.md`
- 原 v2 独立栈(`docker-compose.v2.yml``/opt/inkreach-v2` 检出)已并入单栈并删除, - 原 v2 独立栈(`docker-compose.v2.yml``/opt/inkreach-v2` 检出)已并入单栈并删除,
@@ -76,7 +83,7 @@ apps/api/
│ ├── product-families/ # 产品族(SPU 层):CRUD / auto-group(并入已有族优先,familyNameKey 族语义键)/ attachToMatchingFamily(单链接自动归族)/ consolidateFragments(碎片族合并)/ 成员管理 / 自定义成员 / 价格覆盖 / 重算 / 按链接名称派生标签(auto-tag-rules,含 热转印→烫画 别名)。数据约定(2026-09-03 起):光板/不打印链接不入族、矩阵不计算不打印工艺;auto-group/organize/人工改标签会把散链接回挂,运行前须排查(备份见 deploy/backups/20260903-noprint-removal/ │ ├── product-families/ # 产品族(SPU 层):CRUD / auto-group(并入已有族优先,familyNameKey 族语义键)/ attachToMatchingFamily(单链接自动归族)/ consolidateFragments(碎片族合并)/ 成员管理 / 自定义成员 / 价格覆盖 / 重算 / 按链接名称派生标签(auto-tag-rules,含 热转印→烫画 别名)。数据约定(2026-09-03 起):光板/不打印链接不入族、矩阵不计算不打印工艺;auto-group/organize/人工改标签会把散链接回挂,运行前须排查(备份见 deploy/backups/20260903-noprint-removal/
│ ├── goods/ # 商品 CRUD + 批量优先级 + 批量创建 + 展示名规范化(品名 SKU,剥 ASCII 型号限定词,纯款号按 SDS 分类名补描述) │ ├── goods/ # 商品 CRUD + 批量优先级 + 批量创建 + 展示名规范化(品名 SKU,剥 ASCII 型号限定词,纯款号按 SDS 分类名补描述)
│ ├── sync/ # SDS 同步:分类 / 商品 / 同步日志 │ ├── sync/ # SDS 同步:分类 / 商品 / 同步日志
│ ├── public/ # 公开 API:分类树 / 国家 / 商品分页 / 商品详情 │ ├── public/ # 公开 API:分类树 / 国家 / 商品分页 / 商品详情public-cache.service.ts 为进程内分域缓存(meta/goods/matrix 版本域 + TTL 兜底 + in-flight 合并,P0-1 性能整改),全部写路径(admin CRUD/sync/族重算/整理)事务提交后 bump 对应域即时失效;env 开关 PUBLIC_CACHE_DISABLED / PUBLIC_CACHE_TTL_MS / PUBLIC_CACHE_MAX_ENTRIES
│ └── common/ # 全局装饰器 / 过滤器 / 拦截器 │ └── common/ # 全局装饰器 / 过滤器 / 拦截器
│ ├── decorators/current-user.decorator.ts │ ├── decorators/current-user.decorator.ts
│ ├── filters/http-exception.filter.ts │ ├── filters/http-exception.filter.ts
@@ -0,0 +1,157 @@
# 整改计划:public 端口 1w 瞬时并发承压(public-capacity-10k-refactor
前置审查:[docs/references/performance-review-public-port.md](../../docs/references/performance-review-public-port.md)
## 目标
- 目标口径:**1w 瞬时并发请求**(同一时刻 1 万请求到达边缘,万 QPS 级)。
- 验收:p95 < 300ms、p99 < 1s、错误率 < 0.1%、无 nginx 连接拒绝/重置、PG 连接水位 < 80%、无池排队超时(详见报告 §5)。
- 原则:本次改造只动「配置与架构层」,不改对外 API 契约(响应结构不变),数据语义零变化。
## 全局约束(执行本计划所有任务强制遵守)
1. **数据库变更先行备份存档**:凡是涉及数据库更改的行动(schema 迁移、数据回填、族重算、同步行为变更、PG 参数调整与重启),执行前必须先完成风险回滚存档:
- 受影响库 `pg_dump -Fc` 全量 dump + 相关配置快照,落 `deploy/backups/<时间戳>/` 并写 `RESTORE.md`(沿用 AGENTS.md「生产栈收敛三件套」);
- 变更完成并验证通过前,备份不得清理;验收通过后按 RESTORE.md 归档流程归档。
- 本计划内的触发点:P0-3(PG 参数 + 重启)、P1-1(物化列迁移)、P1-2(重算/同步行为变更)、P2-1(索引/扩展)。
2. **写路径必失效缓存**:凡引入缓存(P0-1),所有数据写路径(admin 侧 CRUD、sync 同步、族重算、上传替换)在写事务提交成功后必须同步失效对应缓存域;「等待 TTL 自然过期」不作为可接受的兜底;每条写路径都必须有「写后缓存已失效」的测试覆盖。
## 执行顺序与依赖
```
P0-3(池参数,防排队超时)→ P0-1(缓存,DB 负载降 95%+)→ P0-2nginx,边缘承压)
→ P1-3(压测基线,验证 P0 是否达标)
→ P1-1(SQL 分页,缓存兜底后放宽量级)→ P1-2(同步隔离)→ P2(收尾项)
```
P0 三项互相独立可并行;P1-3 必须在 P0 全部上线后进行,否则测的是改造前基线(可先测一次作对照)。
---
## P0-1 进程内存缓存层(DB 热路径负载降 95%+,杠杆最高)
**原则**:public 读路径数据变更频率 = 小时级同步 → 分钟级 TTL 缓存几乎无损新鲜度。缓存键必须包含影响结果的全部入参;失效锚点为同步任务(见 P0-1.3)。
### P0-1.1 低熵全量数据缓存(categories / countries / tags / tag-groups / 树序元数据 / 族最低价)
- 涉及:`apps/api/src/public/public.service.ts``apps/api/src/app.module.ts`(若用 `@nestjs/cache-manager`
- 做法:引入进程内存缓存(`@nestjs/cache-manager` 或自研 Map+TTL 包装,单进程部署下内存缓存即全局缓存;不引入 Redis,避免新增运维依赖)。
- `loadTreeOrderMeta()``loadFamilyMinPrices()` 两个私有聚合结果整体缓存(TTL 10min)——它们被列表/首页每次请求复用,是最大的固定成本。
- `getCategoriesTree` / `getCountries` / `getTags` / `getTagGroups``countryId`(若有)作键缓存结果(TTL 10min)。
- 测试:TDD 先写——缓存命中返回等价结构、TTL 过期后重新查询、countryId 不同键不串数据。
- 回归风险:同步窗口内缓存有最多 10min 滞后 → P0-1.3 主动失效可缩短到秒级。
### P0-1.2 商品列表缓存:全量物化 + 内存分页(各 page/筛选组合共享一份缓存)
- 涉及:`apps/api/src/public/public.service.ts``getGoods`
- 做法:**不要**缓存「page 组合」——对筛选结果(countryId/keyword/category/tags/价格区间 + 排序)物化出 `{ rows: PublicGoodDto[](已排序已分组), total }` 整体缓存(TTL 10min,键 = 筛选参数序列化,不含 page/pageSize),`items.slice(start, start+pageSize)` 在缓存命中后执行。默认排序下全部 240 行物化成本一次,所有页共享。
- 注意:价格筛选/价格排序依赖族最低价 → 与 P0-1.1 的族最低价缓存同源;族最低价变化(族重算/同步)后必须连带失效列表缓存。
- 测试:TDD——同筛选不同 page 只触发一次底层查询(spy findMany 计数);筛选参数不同键不同;失效后重建。
- 回归风险:内存占用 = 缓存条目 × 240 DTO(每条 ~1KB)→ 筛选组合多时需设条目上限(LRU,如 64 条)与单条目 TTL。
### P0-1.3 详情缓存
- 涉及:`public.service.ts``getGoodByFamilyId`
- 做法:`goods/:id` 详情按 familyId 缓存(TTL 10min;响应含 11KB price_matrix,缓存收益尤其大)。
- 测试:命中返回等价结构、TTL 过期重建、失效后重建。
- 回归风险:多进程部署时内存缓存不相通 → 当前单进程无此问题;扩容多副本时需换 Redis 或接受短滞后(文档注明)。
### P0-1.4 缓存失效统一机制(全局约束 §2 的落地,必须先于/同步于 P0-1.1~1.3 交付)
- 涉及:新增 `apps/api/src/public/public-cache.service.ts`(或等价),`apps/api/src/sync/sync.service.ts``family-recompute.service.ts`、admin 侧全部写服务
- 做法:
1. **版本域设计**:每个缓存键带版本前缀,分域管理——`v:categories`(分类树/国家/标签组/树序元数据)、`v:goods`(列表/首页/详情)、`v:matrix`(族最低价);读路径取版本号拼键,写路径 `bump(域)` 使整个域失效。域粒度避免「任意写导致全量缓存清空」,实现简单且不漏键(不做逐个 `del` 键级清理,易漏易错)。
2. **统一入口**`PublicCacheService` 只暴露 `get/bump/set` 三个方法;所有写路径收口到 `bump`,禁止在业务代码里散落 cache 键。
3. **写路径挂钩清单**(缓存上线时必须全部挂上,缺一不可):
- `sync.service.ts``syncCategories` 提交后 → `v:categories``syncProducts` 提交后 → `v:goods`;详情同步 `persistProductDetail` 提交后 → `v:goods`
- `family-recompute.service.ts`:族重算 `persist` 提交后 → `v:matrix` + `v:goods`
- admin 侧写服务:goods CRUD/排序/标签挂接、categories、tags/tag-groups、positions、product-familiesoverride)→ 按影响域 `bump`;注意单纯「上传文件」若不改库行则无需 bump,挂钩点是**引用该图的写操作**(改 goods/detail/media 行 → `v:goods`)。
4. **写后即失效**write-through 语义):`bump` 必须在写事务提交**成功之后**执行(事务回滚不得 bump);`bump` 本身用版本号递增(防止「失效瞬间的并发读又把旧值写回缓存」的竞态——读路径读版本号后写入,版本号已变则丢弃写入或重查)。
- 测试(每个写路径至少一条):
- 「写后缓存已失效」:写入数据 → 断言对应缓存域读不到旧值;
- 「TTL 未到期但写已发生 → 前台立即可见新数据」的端到端用例(同步/族重算/admin 修改各一);
- 「事务回滚不 bump」:构造回滚路径断言版本号未变;
- 并发竞态:并发读+写下不出现陈旧值(版本号检查路径)。
- 回归风险:挂接漏写路径 = 前台最长 10min 陈旧(比报错更隐蔽)→ 挂钩清单作为自查 checklist 纳入 PR 模板与代码评审。
---
## P0-2 边缘 nginx 调优(连接数硬顶 4096 → 16384+,压缩与限流补齐)
- 涉及:`deploy/nginx/admin.conf`bind mount,改后 `docker compose up -d --force-recreate admin`,文件型挂载 inode 不随 restart 生效)、**新增** `deploy/nginx/nginx.conf` + `deploy/admin.Dockerfile`(主配置 COPY 进镜像,需 `docker compose build` 重建——只 force-recreate 不生效)、`deploy/nginx/admin.v2.conf`/v2/h5/ /v2/admin/ 静态缓存头落点)、`deploy/docker-compose.yml`admin 服务 ulimits
- 做法(`/etc/nginx/nginx.conf` 主配置无法覆盖时,在 conf.d 里用 `worker_rlimit_nofile` 不行——主配置的 workers 属 http 级不可段内覆盖——需在镜像层覆盖主配置或确认官方镜像默认即可):
1. **连接数**(已核实:`deploy/admin.Dockerfile` 最终镜像只 COPY 了 `conf.d/default.conf`,主 nginx.conf 为官方原版 `worker_connections 1024``events{}` 只存在于主配置,conf.d 无法覆盖):新增 `deploy/nginx/nginx.conf` 并在 Dockerfile 最终阶段加 `COPY deploy/nginx/nginx.conf /etc/nginx/nginx.conf`——内容 `worker_processes auto; worker_rlimit_nofile 65536; events { worker_connections 16384; multi_accept on; }`;同时在 compose `admin` 服务加 `ulimits: { nofile: { soft: 65536, hard: 65536 } }`4 worker × 16384 连接 + 上游 keepalive 连接需落在容器 fd 预算内,Docker 默认值需实测确认,低于预算必须显式设置)。
2. **upstream keepalive**`/v2-api/``/public/``/uploads/``/assets/` 四个 proxy location 统一加 `upstream v2_api { server v2-api:3001; keepalive 64; }` + `proxy_http_version 1.1`(已有)+ `proxy_set_header Connection ""`keepalive 必需)。
3. **gzip on**`gzip on; gzip_types application/json application/javascript text/css image/svg+xml; gzip_min_length 1k;`JSON API + SPA 静态资源,带宽降 5~10x)。
4. **静态资源缓存头**(注意配置落点):`/uploads/``/assets/` 在边缘 admin.conf 加 `expires 30d; add_header Cache-Control "public, immutable";`;但 `/v2/h5/``/v2/admin/` 的静态文件实际由 **v2-admin 容器的 `deploy/nginx/admin.v2.conf`** 提供(边缘只做代理),哈希文件名的缓存头要加在那边,`index.html` 保持 `no-cache`——边缘不要对代理响应统一加 expires,避免覆盖源头的 no-cache 语义。
5. **limit_req 兜底**(定位:只挡瞬时洪峰/扫描,不做业务限流——app 层 120 req/min/IP(≈2 r/s 持续)比它严得多):`limit_req_zone $binary_remote_addr zone=public:10m rate=50r/s;` + `limit_req zone=public burst=200 nodelay;` + 显式 `limit_req_status 429;`(默认 503 会让前端误判服务故障),仅挂 `/public/``/v2-api/` 两个 location。
6. **proxy 超时显式化**`proxy_connect_timeout 5s; proxy_read_timeout 30s; proxy_send_timeout 30s;`(默认 60s 过长,同步长事务窗口内挂死连接)。
7. **client_header_buffer 大请求头**(1 万连接下避免默认 1k 头部缓冲爆),并按 AGENTS.md 先做配置快照(`deploy/backups/<时间戳>/`)。
- 测试/验证:改后 `docker compose config` 校验、`nginx -t`(容器内)、`curl -I` 断言 gzip 头/Cache-Control;压测脚本断言无 connection refused。
- 回归风险:`/uploads/` 长缓存头会让「替换图片同名不同图」被浏览器缓存 → 上传路径需确认图片 URL 含版本参数或降级 `max-age=1d`(此点实施时与业务确认)。
## P0-3 连接池与超时参数(防排队超时与慢查询占池)
- 涉及:`deploy/docker-compose.yml`(生产 `DATABASE_URL`
- 做法:
1. `DATABASE_URL` 追加 `?connection_limit=50&pool_timeout=3`。注意 `pool_timeout` 单位是**秒**(Prisma 文档默认 10s),不要写成毫秒;不要加 `statement_cache_size=0`(那是 pgbouncer 事务模式的做法,直连场景禁用 prepared statement 反而拖慢)。50 < PG 上限,为同步与运维连接留余量。
2. PG 容器 `command: ["postgres", "-c", "statement_timeout=10000", "-c", "shared_buffers=512MB", "-c", "effective_cache_size=1536MB", "-c", "max_connections=200"]`14GB 内存机,shared_buffers 512MB 够工作集;statement_timeout 10s 兜住慢查询不占池)。
3. 同步任务自身查库量大(长事务串行循环)—— statement_timeout 10s 需确认不误伤同步批量 upsert(单条语句都不应超 10s;若误伤,同步侧改批量化见 P1-2)。
4. 变更 PG 参数需重启容器(秒级中断):按全局约束 §1 先做配置快照 + `pg_dump -Fc` 存档,低峰窗口执行,`docker compose up -d --force-recreate v2-postgres` 后立即验证健康与连接水位。
- 测试:入参级别——用一次性 docker postgresAGENTS.md 同款 54329 套路)跑 `prisma migrate deploy` + 全量 jest 确认无超时回归;生产以 `SHOW statement_timeout`/`pg_stat_activity` 验证。
- 回归风险:`pool_timeout=3000` 让洪峰期排队 >3s 快速失败(429/500)而非挂 10s——配合 P0-1 缓存后池压力极小;这是有意的快速失败语义,需在压测时观察错误率符合验收口径。
---
## P1-1 `/public/goods` SQL 分页(量级放宽后的正解)
- 涉及:`apps/api/src/public/public.service.ts``getGoods`
- 背景:P0-1.2 缓存已接住当前百级量级;SQL 分页是为「商品量级上万」预埋(代码注释 L231-233 已留方向)。
- 做法:族表物化视图方向——`product_families` 加「族代表行」物化列(代表 Good 的 goodName/主图/分类/countryId/排序键)与**族最低价物化列**P0-1.1 的 LATERAL 聚合结果届时改为由族重算路径写入该列,内存缓存退化为直读列),列表查询改为 `findMany({ take, skip, include })` + 独立 `count`,去掉内存分组与全量拉取。物化列的写入统一挂在族重算 `recomputeFamily` 与 admin 写路径上(写后 bump 对应缓存域,遵守全局约束 §2)。
- 测试:TDD——分页正确性(边界页、total 与 items 长度)、排序与现内存版全等(用 AGENTS.md「同一排序函数生成期望」防中文/键序手写错误)、打乱输入断言输出全等。
- 风险:改动面大(DTO/排序语义),**必须**与现实现做差分对照(同一数据集新老实现结果全等),放 P1 不与 P0 抢时间窗口;涉及 schema 变更(族表物化列)时按全局约束 §1 先备份存档并交付回滚迁移。
## P1-2 同步任务与 public 隔离
- 涉及:`apps/api/src/sync/sync.service.ts``apps/api/src/product-families/family-recompute.service.ts`
- 做法(按成本升序,实施时选一):
1. **错峰**:整点分类同步窗口(~分钟级长事务)移到低峰(如 04:00 后紧邻详情同步);至少避开业务高峰。
2. **批量化**syncCategories 单事务串行 → 分批事务(每 50 分类一提交);syncProducts 逐商品 upsert → `createMany`/并行化(注意 SDS 依赖逐条校验)。
3. **独立 worker**`SYNC_WORKER=true` 环境开关,第二个 v2-api 容器只跑 schedule`app.listen` 前 return / NestFactory disable listen),与 public 进程池物理隔离——唯一彻底方案,代价是 +1 容器与连接预算(已 P0-3 预留 50/池 ×2)。
- 测试:同步结果幂等与现行为全等(对比 sync_logs 行数与库行数);并发压测与同步同时进行的窗口场景;行为变更(批量化/重算)执行前按全局约束 §1 备份存档。
- 风险:族重算 fire-and-forget 队列在独立 worker 下需确认 enqueue 侧(public 进程也会 trigger?——现状 enqueue 在同步与族变更路径,若拆分需统一入口)。
## P1-3 压测基线(验收依据,必须分布式源)
- 涉及:k6(或 wrk)脚本放 `scripts/loadtest/`;目标环境与压力机要求见步骤 0
- 步骤:
0. **目标环境(硬性)**:1 万并发口径压测**不得直打生产栈**——用最近 `pg_dump` 在一次性镜像栈恢复(AGENTS.md 同款 `docker run … postgres:16-alpine` 起库 + `prisma migrate deploy` + 单独 compose 起 api/nginx 副本,参数与生产一致),`THROTTLE_LIMIT` 只在镜像栈抬高;若最终必须在生产验证,只允许业务书面确认的低峰窗口 + 只读场景 + 限流不动的保守梯度。**压力机必须是独立机器**(与被测机同机跑 10k VU 会互相抢 CPU,测出来的是压测机瓶颈)。
1. SQL 单测基线:dump 恢复后逐条跑 `loadFamilyMinPrices` / `loadTreeOrderMeta` / 列表全量 / 详情 / ILIKE keyword,记录真实耗时(填空报告 §2 的估算值)。
2. 功能压测:`THROTTLE_LIMIT` 临时抬到 1e6(或分布式 ≥ 数十 IP)避免 429 干扰;场景 = 混合读(home-goods / goods / 详情 / 分类树按 3:4:2:1)。
3. 口径压测:逐步加并发至 1 万瞬时请求(如 k6 10000 VU,分布式源),记录 p95/p99/错误率/nginx 连接拒绝/PG 水位,并记录 Node 进程 CPU 与事件循环延迟(`--max-old-space-size` 与 GC 停顿)。
4. **达标失败的扩展阶梯**(按成本升序,逐级尝试后再进下一级):
a. **响应字节级缓存**:P0-1 缓存的是 DTO 对象,命中后每请求仍要 `JSON.stringify`——1 万瞬时 burst 下单进程事件循环的序列化 CPU 可能成为新瓶颈(每响应 ~10-20KB × 1 万次)。若压测显示 stringify 是热点,把缓存值改为**已序列化的最终 JSON 字符串**(含 TransformInterceptor 包裹结构),命中路径直接回写字节,绕过逐请求序列化;
b. **nginx micro-cache**`proxy_cache``/public/` GET 设 10~30s 短 TTL,洪峰完全由边缘吸收。⚠️ 与全局约束 §2 冲突:proxy_cache 无法主动 purge,只能靠短 TTL——需业务确认接受该陈旧窗口,不接受则跳过此项;
c. **Redis + 多副本**:内存缓存换共享存储(版本域失效天然跨进程),加 v2-api 副本(nginx upstream 已 keepalive,扩容仅加容器 + PG max_connections 预算)。
- 交付:压测报告(数字+结论)追加到性能审查报告 §5 验收结果。
---
## P2 收尾项(非阻塞,可后置)
1. **静态资源边缘直出**`/uploads/` 数据卷直挂 edge nginx`deploy/docker-compose.yml``v2-uploads` 卷 + try_files),省一跳代理;注意 upload 与 nginx 读同卷的一致性(nginx 缓存/直读无冲突)。
2. **pg_trgm 索引**`CREATE EXTENSION pg_trgm; CREATE INDEX ... ON goods USING GIN (good_name gin_trgm_ops);`admin 侧 origin_goods.good_name 同)——goods 上万后 keyword 才需要;迁移文件 + 回滚脚本,执行前按全局约束 §1 备份存档(CREATE EXTENSION 为 DDL,回滚与重放需按迁移流程管理)。
3. **监控告警**sync_logs 失败数、`pg_stat_activity` 连接水位、Prisma 池排队(日志关键字 `Timed out fetching a new connection`)→ 告警;nginx access log 结构化落盘(当前无 access 日志,1w 并发下调试无从下手)。
4. **README/docs 更新**`docs/references/structs.md`(部署拓扑段补充性能配置说明)按 AGENTS.md 第 6/7 节执行。
## 全局回归策略
- 每项任务独立分支提交(Conventional Commits`perf(...)`),合并回 `develop`
- 全量 jest 前停 dev serverAGENTS.md 经验);连共享库前 `prisma migrate status`;集成测试夹具自包含(一次性 pg 跑套件)。
- 缓存相关改动重点回归:列表/详情/首页三端点输出与改造前全等(差分测试)、同步后新鲜度、TTL 失效路径。
## 验收(全部完成后)
- 分布式压测 1 万并发瞬时请求:p95 < 300ms、p99 < 1s、错误率 < 0.1%、无连接拒绝、PG 水位 < 80%、无池排队超时。
- 全局约束核查:涉及 DB 变更的任务全部留有 `deploy/backups/<时间戳>/` 存档与 RESTORE.md(可 dry-run 恢复验证);全部写路径(sync/族重算/admin CRUD/上传)的「写后缓存即失效」用例通过(含事务回滚不 bump、并发读不写回旧值)。
- 结果数字回填报告 §5 并关闭本计划。