merge: refactor/public-capacity-10k-yeuimu into develop (P0 性能整改——public 读路径分域缓存+全写路径失效、nginx 承压调优、连接池/PG 参数,已部署验证)
This commit is contained in:
@@ -98,7 +98,8 @@ Prisma Postgres 最佳实践: prisma-postgres 技能
|
|||||||
- **连真实库的集成测试隔离**:jest 并行套件共用一个数据库时,(a) 夹具的天然键(sdsCategoryId、名称等)必须带运行时间戳唯一化,禁止跨运行共享字面量;(b) 全量型操作(如 auto-group 扫全库)会顺带扫到其他并行套件的夹具,其写入路径必须对"成员中途消失"宽容(跳过而非抛错),否则会随机挂测试。
|
- **连真实库的集成测试隔离**:jest 并行套件共用一个数据库时,(a) 夹具的天然键(sdsCategoryId、名称等)必须带运行时间戳唯一化,禁止跨运行共享字面量;(b) 全量型操作(如 auto-group 扫全库)会顺带扫到其他并行套件的夹具,其写入路径必须对"成员中途消失"宽容(跳过而非抛错),否则会随机挂测试。
|
||||||
- **跑全量 jest 前先停 dev server**:`nest start --watch` 等常驻进程与测试共用数据库时,其重编译窗口/后台钩子会与测试写入竞争,造成"单跑绿、全量偶发红"的假阳性;验证基线前先停掉所有 watch 进程再跑。
|
- **跑全量 jest 前先停 dev server**:`nest start --watch` 等常驻进程与测试共用数据库时,其重编译窗口/后台钩子会与测试写入竞争,造成"单跑绿、全量偶发红"的假阳性;验证基线前先停掉所有 watch 进程再跑。
|
||||||
- **中文断言勿手写字面量排序**:JS `Array.sort()` 对中文按 UTF-16 码位排(如 烫 U+70EB < 直 U+76F4),手写期望序列容易按拼音/习惯顺序写反;比较选项集合时用 `expect.arrayContaining` + 长度,或从同一排序函数生成期望。
|
- **中文断言勿手写字面量排序**:JS `Array.sort()` 对中文按 UTF-16 码位排(如 烫 U+70EB < 直 U+76F4),手写期望序列容易按拼音/习惯顺序写反;比较选项集合时用 `expect.arrayContaining` + 长度,或从同一排序函数生成期望。
|
||||||
- **pnpm 仓库容器化执行要挂仓库根**:pnpm 的 node_modules 是相对符号链接指向根 `.pnpm` store,容器里只挂子包目录(如 `apps/api:/app`)会断链报 "Cannot find module";必须挂整个仓库根并 `-w` 到子包。另外 Prisma 引擎与系统 libssl 版本强绑定:node:20-alpine 缺 libssl1.1 会报 engine 加载失败,直接复用项目自身的运行镜像(如 deploy-v2-api)跑 prisma/jest 最稳。
|
- **pnpm 仓库容器化执行要挂仓库根**:pnpm 的 node_modules 是相对符号链接指向根 `.pnpm` store,容器里只挂子包目录(如 `apps/api:/app`)会断链报 "Cannot find module";必须挂整个仓库根并 `-w` 到子包。另外 Prisma 引擎与系统 libssl 版本强绑定:node:20-alpine 缺 libssl1.1 会报 engine 加载失败;仓库 `.pnpm` store 里生成的 client 是 openssl-1.1.x target,**bookworm 系镜像(node:20-slim)同样跑不了**,用 `node:20-bullseye-slim` 挂仓库根 + `--network host`(连本机一次性测试库)跑 jest/prisma 即可,无需依赖项目运行镜像。
|
||||||
|
- **缓存值里绝不能带请求级切片(分页/字段裁剪)**:把 `items.slice(page…)` 的结果整个塞进缓存后,缓存键不含 page → 所有页码命中同一条目、页页返回第一页(total 对得上更具迷惑性);正确做法是缓存"全量物化结果",切片在缓存命中后按请求执行。TDD 用例必须包含"同键翻页 + 页内容随页码变化"的断言才能抓住这类错。
|
||||||
- **Prisma 唯一查询用字段名而非列名**:`where: { id }` 而非 `@map("category_id")` 映射后的 `categoryId`;schema `@map` 只影响 SQL 列名,Prisma Client 的唯一输入类型永远用 model 字段名。
|
- **Prisma 唯一查询用字段名而非列名**:`where: { id }` 而非 `@map("category_id")` 映射后的 `categoryId`;schema `@map` 只影响 SQL 列名,Prisma Client 的唯一输入类型永远用 model 字段名。
|
||||||
- **跨套件分页断言要圈定夹具**:真实库上测"列表排序"时全库数据可能远超 pageSize,夹具根本进不了第一页;给夹具商品名加唯一前缀 + `keyword` 过滤圈定,断言既稳定又能看到完整顺序。
|
- **跨套件分页断言要圈定夹具**:真实库上测"列表排序"时全库数据可能远超 pageSize,夹具根本进不了第一页;给夹具商品名加唯一前缀 + `keyword` 过滤圈定,断言既稳定又能看到完整顺序。
|
||||||
- **"绝对排序键 + 子集过滤"模式**:需要"任意筛选组合下顺序都正确"时,给每条数据算好一组绝对排序键(如 国家→二级→款→priority,缺失沉底),筛选只做子集过滤不做特殊排序分支——比每个筛选组合写一套 orderBy 逻辑可靠得多。
|
- **"绝对排序键 + 子集过滤"模式**:需要"任意筛选组合下顺序都正确"时,给每条数据算好一组绝对排序键(如 国家→二级→款→priority,缺失沉底),筛选只做子集过滤不做特殊排序分支——比每个筛选组合写一套 orderBy 逻辑可靠得多。
|
||||||
|
|||||||
@@ -9,6 +9,7 @@
|
|||||||
* 后台等价入口:POST /product-families/organize(「整理」按钮)。
|
* 后台等价入口:POST /product-families/organize(「整理」按钮)。
|
||||||
*/
|
*/
|
||||||
import { PrismaService } from '../src/prisma/prisma.service';
|
import { PrismaService } from '../src/prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../src/public/public-cache.service';
|
||||||
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
|
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
|
||||||
import { ProductFamiliesService } from '../src/product-families/product-families.service';
|
import { ProductFamiliesService } from '../src/product-families/product-families.service';
|
||||||
import { OrganizeService } from '../src/product-families/organize.service';
|
import { OrganizeService } from '../src/product-families/organize.service';
|
||||||
@@ -16,9 +17,9 @@ import { OrganizeService } from '../src/product-families/organize.service';
|
|||||||
async function main() {
|
async function main() {
|
||||||
const prisma = new PrismaService();
|
const prisma = new PrismaService();
|
||||||
await prisma.onModuleInit();
|
await prisma.onModuleInit();
|
||||||
const recompute = new FamilyRecomputeService(prisma);
|
const recompute = new FamilyRecomputeService(prisma, new PublicCacheService());
|
||||||
const families = new ProductFamiliesService(prisma, recompute);
|
const families = new ProductFamiliesService(prisma, recompute);
|
||||||
const organize = new OrganizeService(prisma, recompute, families);
|
const organize = new OrganizeService(prisma, recompute, families, new PublicCacheService());
|
||||||
|
|
||||||
const result = await organize.organize();
|
const result = await organize.organize();
|
||||||
console.log(
|
console.log(
|
||||||
|
|||||||
@@ -8,6 +8,7 @@
|
|||||||
* origin_goods.good_name 为 SDS 纯镜像(每小时同步覆盖),本脚本只改 goods.good_name。
|
* origin_goods.good_name 为 SDS 纯镜像(每小时同步覆盖),本脚本只改 goods.good_name。
|
||||||
*/
|
*/
|
||||||
import { PrismaService } from '../src/prisma/prisma.service';
|
import { PrismaService } from '../src/prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../src/public/public-cache.service';
|
||||||
import { SyncService } from '../src/sync/sync.service';
|
import { SyncService } from '../src/sync/sync.service';
|
||||||
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
|
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
|
||||||
import { GoodsService } from '../src/goods/goods.service';
|
import { GoodsService } from '../src/goods/goods.service';
|
||||||
@@ -16,12 +17,13 @@ async function main() {
|
|||||||
const dryRun = !process.argv.includes('--apply');
|
const dryRun = !process.argv.includes('--apply');
|
||||||
const prisma = new PrismaService();
|
const prisma = new PrismaService();
|
||||||
await prisma.onModuleInit();
|
await prisma.onModuleInit();
|
||||||
const recompute = new FamilyRecomputeService(prisma);
|
const recompute = new FamilyRecomputeService(prisma, new PublicCacheService());
|
||||||
// 回填路径不触发详情同步,SyncService 仅作占位依赖
|
// 回填路径不触发详情同步,SyncService 仅作占位依赖
|
||||||
const goods = new GoodsService(
|
const goods = new GoodsService(
|
||||||
prisma,
|
prisma,
|
||||||
{ queueProductDetailSync: async () => undefined } as unknown as SyncService,
|
{ queueProductDetailSync: async () => undefined } as unknown as SyncService,
|
||||||
recompute,
|
recompute,
|
||||||
|
new PublicCacheService(),
|
||||||
);
|
);
|
||||||
|
|
||||||
const result = await goods.backfillPureSkuGoodNames({ dryRun });
|
const result = await goods.backfillPureSkuGoodNames({ dryRun });
|
||||||
|
|||||||
@@ -7,12 +7,13 @@
|
|||||||
* 运行:pnpm --filter @inkreach/api recompute:families
|
* 运行:pnpm --filter @inkreach/api recompute:families
|
||||||
*/
|
*/
|
||||||
import { PrismaService } from '../src/prisma/prisma.service';
|
import { PrismaService } from '../src/prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../src/public/public-cache.service';
|
||||||
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
|
import { FamilyRecomputeService } from '../src/product-families/family-recompute.service';
|
||||||
|
|
||||||
async function main() {
|
async function main() {
|
||||||
const prisma = new PrismaService();
|
const prisma = new PrismaService();
|
||||||
await prisma.onModuleInit();
|
await prisma.onModuleInit();
|
||||||
const recompute = new FamilyRecomputeService(prisma);
|
const recompute = new FamilyRecomputeService(prisma, new PublicCacheService());
|
||||||
|
|
||||||
const families = await prisma.productFamily.findMany({
|
const families = await prisma.productFamily.findMany({
|
||||||
select: { id: true },
|
select: { id: true },
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ import { ProductFamiliesModule } from './product-families/product-families.modul
|
|||||||
import { GoodsModule } from './goods/goods.module';
|
import { GoodsModule } from './goods/goods.module';
|
||||||
import { SyncModule } from './sync/sync.module';
|
import { SyncModule } from './sync/sync.module';
|
||||||
import { PublicModule } from './public/public.module';
|
import { PublicModule } from './public/public.module';
|
||||||
|
import { PublicCacheModule } from './public/public-cache.module';
|
||||||
import { UploadModule } from './upload/upload.module';
|
import { UploadModule } from './upload/upload.module';
|
||||||
|
|
||||||
@Module({
|
@Module({
|
||||||
@@ -31,6 +32,8 @@ import { UploadModule } from './upload/upload.module';
|
|||||||
},
|
},
|
||||||
]),
|
]),
|
||||||
PrismaModule,
|
PrismaModule,
|
||||||
|
// public 读路径缓存(分域版本失效,@Global 供各写路径注入 bump 入口)
|
||||||
|
PublicCacheModule,
|
||||||
AuthModule,
|
AuthModule,
|
||||||
CountriesModule,
|
CountriesModule,
|
||||||
CategoriesModule,
|
CategoriesModule,
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import {
|
import {
|
||||||
BadRequestException,
|
BadRequestException,
|
||||||
@@ -13,7 +14,7 @@ describe('CategoriesService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [CategoriesService, PrismaService],
|
providers: [CategoriesService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(CategoriesService);
|
service = moduleRef.get(CategoriesService);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
|
|||||||
@@ -5,13 +5,17 @@ import {
|
|||||||
NotFoundException,
|
NotFoundException,
|
||||||
} from '@nestjs/common';
|
} from '@nestjs/common';
|
||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { CreateCategoryDto } from './dto/create-category.dto';
|
import { CreateCategoryDto } from './dto/create-category.dto';
|
||||||
import { UpdateCategoryDto } from './dto/update-category.dto';
|
import { UpdateCategoryDto } from './dto/update-category.dto';
|
||||||
import { CategoryNodeDto } from './dto/category-node.dto';
|
import { CategoryNodeDto } from './dto/category-node.dto';
|
||||||
|
|
||||||
@Injectable()
|
@Injectable()
|
||||||
export class CategoriesService {
|
export class CategoriesService {
|
||||||
constructor(private readonly prisma: PrismaService) {}
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
|
) {}
|
||||||
|
|
||||||
async findAll(): Promise<CategoryNodeDto[]> {
|
async findAll(): Promise<CategoryNodeDto[]> {
|
||||||
const all = await this.prisma.category.findMany({
|
const all = await this.prisma.category.findMany({
|
||||||
@@ -37,7 +41,7 @@ export class CategoriesService {
|
|||||||
// Validate the parent exists to produce a clean 404 instead of FK error.
|
// Validate the parent exists to produce a clean 404 instead of FK error.
|
||||||
await this.findOne(BigInt(dto.parentCategoryId));
|
await this.findOne(BigInt(dto.parentCategoryId));
|
||||||
}
|
}
|
||||||
return this.prisma.category.create({
|
const created = await this.prisma.category.create({
|
||||||
data: {
|
data: {
|
||||||
categoryName: dto.categoryName,
|
categoryName: dto.categoryName,
|
||||||
categoryIcon: dto.categoryIcon ?? null,
|
categoryIcon: dto.categoryIcon ?? null,
|
||||||
@@ -47,6 +51,8 @@ export class CategoriesService {
|
|||||||
: BigInt(dto.parentCategoryId),
|
: BigInt(dto.parentCategoryId),
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return created;
|
||||||
}
|
}
|
||||||
|
|
||||||
async update(id: bigint, dto: UpdateCategoryDto) {
|
async update(id: bigint, dto: UpdateCategoryDto) {
|
||||||
@@ -66,7 +72,9 @@ export class CategoriesService {
|
|||||||
? { disconnect: true }
|
? { disconnect: true }
|
||||||
: { connect: { id: BigInt(dto.parentCategoryId) } };
|
: { connect: { id: BigInt(dto.parentCategoryId) } };
|
||||||
}
|
}
|
||||||
return this.prisma.category.update({ where: { id }, data });
|
const updated = await this.prisma.category.update({ where: { id }, data });
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return updated;
|
||||||
}
|
}
|
||||||
|
|
||||||
async remove(id: bigint) {
|
async remove(id: bigint) {
|
||||||
@@ -80,7 +88,9 @@ export class CategoriesService {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
return await this.prisma.category.delete({ where: { id } });
|
const removed = await this.prisma.category.delete({ where: { id } });
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return removed;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (this.isForeignKeyViolation(err)) {
|
if (this.isForeignKeyViolation(err)) {
|
||||||
throw new BadRequestException(
|
throw new BadRequestException(
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import {
|
import {
|
||||||
BadRequestException,
|
BadRequestException,
|
||||||
@@ -14,7 +15,7 @@ describe('CountriesService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [CountriesService, PrismaService],
|
providers: [CountriesService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(CountriesService);
|
service = moduleRef.get(CountriesService);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
|
|||||||
@@ -6,13 +6,17 @@ import {
|
|||||||
NotFoundException,
|
NotFoundException,
|
||||||
} from '@nestjs/common';
|
} from '@nestjs/common';
|
||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { CreateCountryDto } from './dto/create-country.dto';
|
import { CreateCountryDto } from './dto/create-country.dto';
|
||||||
import { UpdateCountryDto } from './dto/update-country.dto';
|
import { UpdateCountryDto } from './dto/update-country.dto';
|
||||||
import { ReorderCountriesDto } from './dto/reorder-countries.dto';
|
import { ReorderCountriesDto } from './dto/reorder-countries.dto';
|
||||||
|
|
||||||
@Injectable()
|
@Injectable()
|
||||||
export class CountriesService {
|
export class CountriesService {
|
||||||
constructor(private readonly prisma: PrismaService) {}
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
|
) {}
|
||||||
|
|
||||||
findAll() {
|
findAll() {
|
||||||
return this.prisma.country.findMany({
|
return this.prisma.country.findMany({
|
||||||
@@ -30,6 +34,7 @@ export class CountriesService {
|
|||||||
}),
|
}),
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
|
this.publicCache.bump('meta');
|
||||||
return this.findAll();
|
return this.findAll();
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -43,12 +48,14 @@ export class CountriesService {
|
|||||||
|
|
||||||
async create(dto: CreateCountryDto) {
|
async create(dto: CreateCountryDto) {
|
||||||
try {
|
try {
|
||||||
return await this.prisma.country.create({
|
const created = await this.prisma.country.create({
|
||||||
data: {
|
data: {
|
||||||
countryName: dto.countryName,
|
countryName: dto.countryName,
|
||||||
countryIcon: dto.countryIcon ?? null,
|
countryIcon: dto.countryIcon ?? null,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return created;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (
|
if (
|
||||||
err instanceof Prisma.PrismaClientKnownRequestError &&
|
err instanceof Prisma.PrismaClientKnownRequestError &&
|
||||||
@@ -63,13 +70,15 @@ export class CountriesService {
|
|||||||
async update(id: bigint, dto: UpdateCountryDto) {
|
async update(id: bigint, dto: UpdateCountryDto) {
|
||||||
await this.findOne(id);
|
await this.findOne(id);
|
||||||
try {
|
try {
|
||||||
return await this.prisma.country.update({
|
const updated = await this.prisma.country.update({
|
||||||
where: { id },
|
where: { id },
|
||||||
data: {
|
data: {
|
||||||
countryName: dto.countryName,
|
countryName: dto.countryName,
|
||||||
countryIcon: dto.countryIcon === undefined ? undefined : dto.countryIcon,
|
countryIcon: dto.countryIcon === undefined ? undefined : dto.countryIcon,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return updated;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (
|
if (
|
||||||
err instanceof Prisma.PrismaClientKnownRequestError &&
|
err instanceof Prisma.PrismaClientKnownRequestError &&
|
||||||
@@ -84,7 +93,9 @@ export class CountriesService {
|
|||||||
async remove(id: bigint) {
|
async remove(id: bigint) {
|
||||||
await this.findOne(id);
|
await this.findOne(id);
|
||||||
try {
|
try {
|
||||||
return await this.prisma.country.delete({ where: { id } });
|
const removed = await this.prisma.country.delete({ where: { id } });
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return removed;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (this.isForeignKeyViolation(err)) {
|
if (this.isForeignKeyViolation(err)) {
|
||||||
throw new BadRequestException(
|
throw new BadRequestException(
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import {
|
import {
|
||||||
BadRequestException,
|
BadRequestException,
|
||||||
@@ -30,6 +31,7 @@ describe('GoodsService', () => {
|
|||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [
|
providers: [
|
||||||
|
PublicCacheService,
|
||||||
GoodsService,
|
GoodsService,
|
||||||
PrismaService,
|
PrismaService,
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import { BatchPriorityDto } from './dto/batch-priority.dto';
|
|||||||
import { GoodDetailDto, GoodDto, PaginatedGoods } from './dto/good.dto';
|
import { GoodDetailDto, GoodDto, PaginatedGoods } from './dto/good.dto';
|
||||||
import { SyncService } from '../sync/sync.service';
|
import { SyncService } from '../sync/sync.service';
|
||||||
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { isAutoTagGroupName } from '../product-families/auto-tag-rules';
|
import { isAutoTagGroupName } from '../product-families/auto-tag-rules';
|
||||||
import { randomUUID } from 'crypto';
|
import { randomUUID } from 'crypto';
|
||||||
import {
|
import {
|
||||||
@@ -120,6 +121,7 @@ export class GoodsService {
|
|||||||
private readonly prisma: PrismaService,
|
private readonly prisma: PrismaService,
|
||||||
private readonly syncService: SyncService,
|
private readonly syncService: SyncService,
|
||||||
private readonly familyRecompute: FamilyRecomputeService,
|
private readonly familyRecompute: FamilyRecomputeService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
) {}
|
) {}
|
||||||
|
|
||||||
async findAll(query: QueryGoodDto): Promise<PaginatedGoods> {
|
async findAll(query: QueryGoodDto): Promise<PaginatedGoods> {
|
||||||
@@ -240,6 +242,7 @@ export class GoodsService {
|
|||||||
// 建商品后把链接有效标签镜像到该商品(纯聚合,不派生——派生在整理流程)
|
// 建商品后把链接有效标签镜像到该商品(纯聚合,不派生——派生在整理流程)
|
||||||
await this.familyRecompute.mirrorLinkTagsToGoods(BigInt(result.originGoodId));
|
await this.familyRecompute.mirrorLinkTagsToGoods(BigInt(result.originGoodId));
|
||||||
}
|
}
|
||||||
|
this.publicCache.bump('goods');
|
||||||
if (
|
if (
|
||||||
result.originGood?.source === 'SDS' &&
|
result.originGood?.source === 'SDS' &&
|
||||||
result.originGood.sdsGoodId &&
|
result.originGood.sdsGoodId &&
|
||||||
@@ -314,6 +317,7 @@ export class GoodsService {
|
|||||||
}
|
}
|
||||||
return good.id;
|
return good.id;
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('goods');
|
||||||
if (family) this.familyRecompute.enqueue(family.id);
|
if (family) this.familyRecompute.enqueue(family.id);
|
||||||
return this.findOne(goodId);
|
return this.findOne(goodId);
|
||||||
}
|
}
|
||||||
@@ -369,6 +373,7 @@ export class GoodsService {
|
|||||||
await tx.good.update({ where: { id }, data: goodData });
|
await tx.good.update({ where: { id }, data: goodData });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('goods');
|
||||||
return this.findOne(id);
|
return this.findOne(id);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -495,6 +500,7 @@ export class GoodsService {
|
|||||||
// 归族联动后重算矩阵(成员变化 → 自动重算,结构化聚合)
|
// 归族联动后重算矩阵(成员变化 → 自动重算,结构化聚合)
|
||||||
await this.familyRecompute.recomputeFamily(familyIdForTags);
|
await this.familyRecompute.recomputeFamily(familyIdForTags);
|
||||||
}
|
}
|
||||||
|
this.publicCache.bump('goods');
|
||||||
if (
|
if (
|
||||||
result.originGood?.source === 'SDS' &&
|
result.originGood?.source === 'SDS' &&
|
||||||
result.originGood.sdsGoodId &&
|
result.originGood.sdsGoodId &&
|
||||||
@@ -522,6 +528,7 @@ export class GoodsService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('goods');
|
||||||
return { id: id.toString() };
|
return { id: id.toString() };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -539,6 +546,7 @@ export class GoodsService {
|
|||||||
}
|
}
|
||||||
return { count: dto.items.length };
|
return { count: dto.items.length };
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('goods');
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -634,6 +642,7 @@ export class GoodsService {
|
|||||||
)) {
|
)) {
|
||||||
this.syncService.queueProductDetailSync(goodId);
|
this.syncService.queueProductDetailSync(goodId);
|
||||||
}
|
}
|
||||||
|
this.publicCache.bump('goods');
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { OriginGoodsService } from './origin-goods.service';
|
import { OriginGoodsService } from './origin-goods.service';
|
||||||
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
||||||
@@ -15,7 +16,7 @@ describe('OriginGoodsService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [OriginGoodsService, FamilyRecomputeService, ProductFamiliesService, OrganizeService, PrismaService],
|
providers: [OriginGoodsService, FamilyRecomputeService, ProductFamiliesService, OrganizeService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(OriginGoodsService);
|
service = moduleRef.get(OriginGoodsService);
|
||||||
organize = moduleRef.get(OrganizeService);
|
organize = moduleRef.get(OrganizeService);
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import { PrismaService } from '../prisma/prisma.service';
|
|||||||
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
||||||
import { OrganizeService } from '../product-families/organize.service';
|
import { OrganizeService } from '../product-families/organize.service';
|
||||||
import { ProductFamiliesService } from '../product-families/product-families.service';
|
import { ProductFamiliesService } from '../product-families/product-families.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { QueryOriginGoodDto } from './dto/query-origin-good.dto';
|
import { QueryOriginGoodDto } from './dto/query-origin-good.dto';
|
||||||
|
|
||||||
/** 链接标签(origin_good_tags 行,含人工/派生标记) */
|
/** 链接标签(origin_good_tags 行,含人工/派生标记) */
|
||||||
@@ -97,6 +98,7 @@ export class OriginGoodsService {
|
|||||||
private readonly familyRecompute: FamilyRecomputeService,
|
private readonly familyRecompute: FamilyRecomputeService,
|
||||||
private readonly organize: OrganizeService,
|
private readonly organize: OrganizeService,
|
||||||
private readonly families: ProductFamiliesService,
|
private readonly families: ProductFamiliesService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
) {}
|
) {}
|
||||||
|
|
||||||
/** 链接当前标签(含 manual 标记) */
|
/** 链接当前标签(含 manual 标记) */
|
||||||
@@ -171,6 +173,9 @@ export class OriginGoodsService {
|
|||||||
this.prisma.originGood.update({ where: { id }, data: { tagsManual: true } }),
|
this.prisma.originGood.update({ where: { id }, data: { tagsManual: true } }),
|
||||||
]);
|
]);
|
||||||
await this.familyRecompute.mirrorLinkTagsToGoods(id);
|
await this.familyRecompute.mirrorLinkTagsToGoods(id);
|
||||||
|
// 标签镜像改变 good_tags(meta 域的标签组过滤);无族链接挂靠失败时
|
||||||
|
// 后续 recompute 不会执行,此处显式 bump 兜住全部路径(含 matrix 归因)
|
||||||
|
this.publicCache.bump('meta', 'goods', 'matrix');
|
||||||
// 人工改标签 → 归因维度可能变化,自动重算族矩阵(有族才重算)
|
// 人工改标签 → 归因维度可能变化,自动重算族矩阵(有族才重算)
|
||||||
const ogFull = await this.prisma.originGood.findUnique({
|
const ogFull = await this.prisma.originGood.findUnique({
|
||||||
where: { id },
|
where: { id },
|
||||||
@@ -205,6 +210,7 @@ export class OriginGoodsService {
|
|||||||
]);
|
]);
|
||||||
// 派生集中在整理服务(解析去运行时化);「恢复自动」本身是显式人工动作
|
// 派生集中在整理服务(解析去运行时化);「恢复自动」本身是显式人工动作
|
||||||
await this.organize.deriveTagsForOg(id);
|
await this.organize.deriveTagsForOg(id);
|
||||||
|
this.publicCache.bump('meta', 'goods', 'matrix');
|
||||||
if (og.familyId) await this.familyRecompute.recomputeFamily(og.familyId);
|
if (og.familyId) await this.familyRecompute.recomputeFamily(og.familyId);
|
||||||
return this.getTags(id);
|
return this.getTags(id);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { NotFoundException } from '@nestjs/common';
|
import { NotFoundException } from '@nestjs/common';
|
||||||
import { PositionsService } from './positions.service';
|
import { PositionsService } from './positions.service';
|
||||||
@@ -12,7 +13,7 @@ describe('PositionsService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [PositionsService, PrismaService],
|
providers: [PositionsService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(PositionsService);
|
service = moduleRef.get(PositionsService);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
|
|||||||
@@ -3,12 +3,16 @@ import {
|
|||||||
NotFoundException,
|
NotFoundException,
|
||||||
} from '@nestjs/common';
|
} from '@nestjs/common';
|
||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { CreatePositionDto } from './dto/create-position.dto';
|
import { CreatePositionDto } from './dto/create-position.dto';
|
||||||
import { UpdatePositionDto } from './dto/update-position.dto';
|
import { UpdatePositionDto } from './dto/update-position.dto';
|
||||||
|
|
||||||
@Injectable()
|
@Injectable()
|
||||||
export class PositionsService {
|
export class PositionsService {
|
||||||
constructor(private readonly prisma: PrismaService) {}
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
|
) {}
|
||||||
|
|
||||||
findAll(filters?: { countryId?: bigint; categoryId?: bigint }) {
|
findAll(filters?: { countryId?: bigint; categoryId?: bigint }) {
|
||||||
const where: { countryId?: bigint; categoryId?: bigint } = {};
|
const where: { countryId?: bigint; categoryId?: bigint } = {};
|
||||||
@@ -37,7 +41,7 @@ export class PositionsService {
|
|||||||
if (dto.categoryId !== undefined) {
|
if (dto.categoryId !== undefined) {
|
||||||
await this.ensureCategory(dto.categoryId);
|
await this.ensureCategory(dto.categoryId);
|
||||||
}
|
}
|
||||||
return this.prisma.position.create({
|
const created = await this.prisma.position.create({
|
||||||
data: {
|
data: {
|
||||||
indexVal: dto.indexVal,
|
indexVal: dto.indexVal,
|
||||||
countryId: dto.countryId === undefined ? null : BigInt(dto.countryId),
|
countryId: dto.countryId === undefined ? null : BigInt(dto.countryId),
|
||||||
@@ -45,6 +49,8 @@ export class PositionsService {
|
|||||||
},
|
},
|
||||||
include: { country: true, category: true },
|
include: { country: true, category: true },
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('goods');
|
||||||
|
return created;
|
||||||
}
|
}
|
||||||
|
|
||||||
async update(id: bigint, dto: UpdatePositionDto) {
|
async update(id: bigint, dto: UpdatePositionDto) {
|
||||||
@@ -55,7 +61,7 @@ export class PositionsService {
|
|||||||
if (dto.categoryId !== undefined && dto.categoryId !== null) {
|
if (dto.categoryId !== undefined && dto.categoryId !== null) {
|
||||||
await this.ensureCategory(dto.categoryId);
|
await this.ensureCategory(dto.categoryId);
|
||||||
}
|
}
|
||||||
return this.prisma.position.update({
|
const updated = await this.prisma.position.update({
|
||||||
where: { id },
|
where: { id },
|
||||||
data: {
|
data: {
|
||||||
indexVal: dto.indexVal,
|
indexVal: dto.indexVal,
|
||||||
@@ -74,11 +80,15 @@ export class PositionsService {
|
|||||||
},
|
},
|
||||||
include: { country: true, category: true },
|
include: { country: true, category: true },
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('goods');
|
||||||
|
return updated;
|
||||||
}
|
}
|
||||||
|
|
||||||
async remove(id: bigint) {
|
async remove(id: bigint) {
|
||||||
await this.findOne(id);
|
await this.findOne(id);
|
||||||
return this.prisma.position.delete({ where: { id } });
|
const removed = await this.prisma.position.delete({ where: { id } });
|
||||||
|
this.publicCache.bump('goods');
|
||||||
|
return removed;
|
||||||
}
|
}
|
||||||
|
|
||||||
private async ensureCountry(id: number) {
|
private async ensureCountry(id: number) {
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { Prisma } from '@prisma/client';
|
import { Prisma } from '@prisma/client';
|
||||||
import {
|
import {
|
||||||
@@ -103,13 +104,14 @@ describe('FamilyRecomputeService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [FamilyRecomputeService, PrismaService],
|
providers: [FamilyRecomputeService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(FamilyRecomputeService);
|
service = moduleRef.get(FamilyRecomputeService);
|
||||||
organize = new OrganizeService(
|
organize = new OrganizeService(
|
||||||
moduleRef.get(PrismaService),
|
moduleRef.get(PrismaService),
|
||||||
service,
|
service,
|
||||||
new ProductFamiliesService(moduleRef.get(PrismaService), service),
|
new ProductFamiliesService(moduleRef.get(PrismaService), service),
|
||||||
|
moduleRef.get(PublicCacheService),
|
||||||
);
|
);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
await prisma.onModuleInit();
|
await prisma.onModuleInit();
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { Injectable, Logger } from '@nestjs/common';
|
import { Injectable, Logger } from '@nestjs/common';
|
||||||
import { Prisma } from '@prisma/client';
|
import { Prisma } from '@prisma/client';
|
||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { isAutoTagGroupName } from './auto-tag-rules';
|
import { isAutoTagGroupName } from './auto-tag-rules';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -326,7 +327,10 @@ export class FamilyRecomputeService {
|
|||||||
private readonly logger = new Logger(FamilyRecomputeService.name);
|
private readonly logger = new Logger(FamilyRecomputeService.name);
|
||||||
private readonly pending = new Map<string, Promise<void>>();
|
private readonly pending = new Map<string, Promise<void>>();
|
||||||
|
|
||||||
constructor(private readonly prisma: PrismaService) {}
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
|
) {}
|
||||||
|
|
||||||
/** 进程内去重的异步重算入口(同步钩子用) */
|
/** 进程内去重的异步重算入口(同步钩子用) */
|
||||||
enqueue(familyId: bigint): void {
|
enqueue(familyId: bigint): void {
|
||||||
@@ -405,7 +409,9 @@ export class FamilyRecomputeService {
|
|||||||
data: { stale: true },
|
data: { stale: true },
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
// 族物化 JSON(矩阵/尺码表)与成员口径变化 → public 列表价/详情族块失效。
|
||||||
|
// 人工接管族(autoManaged=false)虽未重写矩阵,但其成员/详情已变,goods 域同样失效。
|
||||||
|
this.publicCache.bump('goods', 'matrix');
|
||||||
}
|
}
|
||||||
/** 把链接的有效标签镜像到其名下商品(good 标签 = 链接标签 ∪ 非自动组既有标签) */
|
/** 把链接的有效标签镜像到其名下商品(good 标签 = 链接标签 ∪ 非自动组既有标签) */
|
||||||
async mirrorLinkTagsToGoods(ogId: bigint): Promise<void> {
|
async mirrorLinkTagsToGoods(ogId: bigint): Promise<void> {
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { FamilyRecomputeService } from './family-recompute.service';
|
import { FamilyRecomputeService } from './family-recompute.service';
|
||||||
import { ProductFamiliesService } from './product-families.service';
|
import { ProductFamiliesService } from './product-families.service';
|
||||||
@@ -29,7 +30,7 @@ describe('链接级标签:派生 / 人工接管 / 商品镜像', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [FamilyRecomputeService, ProductFamiliesService, OrganizeService, OriginGoodsService, PrismaService],
|
providers: [FamilyRecomputeService, ProductFamiliesService, OrganizeService, OriginGoodsService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
recompute = moduleRef.get(FamilyRecomputeService);
|
recompute = moduleRef.get(FamilyRecomputeService);
|
||||||
organize = moduleRef.get(OrganizeService);
|
organize = moduleRef.get(OrganizeService);
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import { Injectable, Logger } from '@nestjs/common';
|
|||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
import { FamilyRecomputeService } from './family-recompute.service';
|
import { FamilyRecomputeService } from './family-recompute.service';
|
||||||
import { ProductFamiliesService } from './product-families.service';
|
import { ProductFamiliesService } from './product-families.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { parseOriginName } from './origin-name.parser';
|
import { parseOriginName } from './origin-name.parser';
|
||||||
import {
|
import {
|
||||||
DERIVED_TAG_GROUP_SPECS,
|
DERIVED_TAG_GROUP_SPECS,
|
||||||
@@ -28,6 +29,7 @@ export class OrganizeService {
|
|||||||
private readonly prisma: PrismaService,
|
private readonly prisma: PrismaService,
|
||||||
private readonly recompute: FamilyRecomputeService,
|
private readonly recompute: FamilyRecomputeService,
|
||||||
private readonly families: ProductFamiliesService,
|
private readonly families: ProductFamiliesService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
) {}
|
) {}
|
||||||
|
|
||||||
async organize() {
|
async organize() {
|
||||||
@@ -52,6 +54,9 @@ export class OrganizeService {
|
|||||||
familiesRecomputed: families.length,
|
familiesRecomputed: families.length,
|
||||||
};
|
};
|
||||||
this.logger.log(`organize done: ${JSON.stringify(result)}`);
|
this.logger.log(`organize done: ${JSON.stringify(result)}`);
|
||||||
|
// 整理是全量标签/归族/矩阵重写动作,三域全失效(末尾逐族重算只覆盖 goods+matrix,
|
||||||
|
// 标签镜像与标签组创建落在 meta 域)
|
||||||
|
this.publicCache.bump('meta', 'goods', 'matrix');
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -190,6 +195,8 @@ export class OrganizeService {
|
|||||||
await this.prisma.originGood.update({ where: { id: og.id }, data: next });
|
await this.prisma.originGood.update({ where: { id: og.id }, data: next });
|
||||||
parsed += 1;
|
parsed += 1;
|
||||||
}
|
}
|
||||||
|
// 结构化标签列是 CUSTOM 成员的矩阵归因来源(下次重算生效),保守失效 matrix+goods
|
||||||
|
this.publicCache.bump('goods', 'matrix');
|
||||||
return { parsed, unparsable };
|
return { parsed, unparsable };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -220,6 +227,8 @@ export class OrganizeService {
|
|||||||
for (const fid of touchedFamilies) {
|
for (const fid of touchedFamilies) {
|
||||||
await this.recompute.recomputeFamily(fid);
|
await this.recompute.recomputeFamily(fid);
|
||||||
}
|
}
|
||||||
|
// 标签镜像(good_tags)落 meta 域;散链接无族不会被重算覆盖,入口统一失效
|
||||||
|
this.publicCache.bump('meta', 'goods', 'matrix');
|
||||||
return { linksUpdated, goodsUpdated, familiesRecomputed: touchedFamilies.length };
|
return { linksUpdated, goodsUpdated, familiesRecomputed: touchedFamilies.length };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -339,6 +348,7 @@ export class OrganizeService {
|
|||||||
]);
|
]);
|
||||||
goodsUpdated += 1;
|
goodsUpdated += 1;
|
||||||
}
|
}
|
||||||
|
this.publicCache.bump('meta', 'goods');
|
||||||
return { goodsUpdated, linksUpdated };
|
return { goodsUpdated, linksUpdated };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -375,6 +385,7 @@ export class OrganizeService {
|
|||||||
linksUpdated = 1;
|
linksUpdated = 1;
|
||||||
}
|
}
|
||||||
await this.recompute.mirrorLinkTagsToGoods(og.id);
|
await this.recompute.mirrorLinkTagsToGoods(og.id);
|
||||||
|
this.publicCache.bump('meta', 'goods');
|
||||||
return { linksUpdated };
|
return { linksUpdated };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { BadRequestException, NotFoundException } from '@nestjs/common';
|
import { BadRequestException, NotFoundException } from '@nestjs/common';
|
||||||
import { Prisma } from '@prisma/client';
|
import { Prisma } from '@prisma/client';
|
||||||
@@ -28,13 +29,14 @@ describe('ProductFamiliesService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [ProductFamiliesService, FamilyRecomputeService, PrismaService],
|
providers: [ProductFamiliesService, FamilyRecomputeService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(ProductFamiliesService);
|
service = moduleRef.get(ProductFamiliesService);
|
||||||
organize = new OrganizeService(
|
organize = new OrganizeService(
|
||||||
moduleRef.get(PrismaService),
|
moduleRef.get(PrismaService),
|
||||||
moduleRef.get(FamilyRecomputeService),
|
moduleRef.get(FamilyRecomputeService),
|
||||||
service,
|
service,
|
||||||
|
moduleRef.get(PublicCacheService),
|
||||||
);
|
);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
await prisma.onModuleInit();
|
await prisma.onModuleInit();
|
||||||
|
|||||||
@@ -0,0 +1,283 @@
|
|||||||
|
import { Test } from '@nestjs/testing';
|
||||||
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from './public-cache.service';
|
||||||
|
import { PublicService } from './public.service';
|
||||||
|
import { CategoriesService } from '../categories/categories.service';
|
||||||
|
import { CountriesService } from '../countries/countries.service';
|
||||||
|
import { TagsService } from '../tags/tags.service';
|
||||||
|
import { TagGroupsService } from '../tag-groups/tag-groups.service';
|
||||||
|
import { PositionsService } from '../positions/positions.service';
|
||||||
|
import { GoodsService } from '../goods/goods.service';
|
||||||
|
import { SyncService } from '../sync/sync.service';
|
||||||
|
import { SdsClientService } from '../sync/sds-client.service';
|
||||||
|
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
||||||
|
import { ProductFamiliesService } from '../product-families/product-families.service';
|
||||||
|
import { OrganizeService } from '../product-families/organize.service';
|
||||||
|
import { OriginGoodsService } from '../origin-goods/origin-goods.service';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* public 缓存失效链路(集成):
|
||||||
|
* 1. 读端点命中缓存(同筛选翻页/重复详情不再查库);
|
||||||
|
* 2. 「TTL 未到但写已发生 → 前台立即可见」端到端;
|
||||||
|
* 3. 每条写路径 bump 断言(全局约束 §2:等待 TTL 过期不是失效手段)。
|
||||||
|
* 夹具自包含:全部天然键带运行时间戳,不依赖库内既有数据。
|
||||||
|
*/
|
||||||
|
describe('public 缓存失效链路', () => {
|
||||||
|
const stamp = Date.now();
|
||||||
|
let prisma: PrismaService;
|
||||||
|
let cache: PublicCacheService;
|
||||||
|
let service: PublicService;
|
||||||
|
let categoriesService: CategoriesService;
|
||||||
|
let countriesService: CountriesService;
|
||||||
|
let tagsService: TagsService;
|
||||||
|
let tagGroupsService: TagGroupsService;
|
||||||
|
let positionsService: PositionsService;
|
||||||
|
let goodsService: GoodsService;
|
||||||
|
let syncService: SyncService;
|
||||||
|
|
||||||
|
let countryId: bigint;
|
||||||
|
let categoryId: bigint;
|
||||||
|
let tagId: bigint;
|
||||||
|
let tagGroupId: bigint;
|
||||||
|
let positionId: bigint;
|
||||||
|
let originGoodId: bigint;
|
||||||
|
let familyId: bigint;
|
||||||
|
let goodId: bigint;
|
||||||
|
let sdsGoodId: string;
|
||||||
|
let sdsCategoryId: string;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
const sdsMock: Partial<SdsClientService> = {
|
||||||
|
// 返回库内全部 SDS 分类 + 逻辑:保证 syncCategories 不误删并行套件的夹具
|
||||||
|
fetchCategoryTree: async () => {
|
||||||
|
const rows = await prisma.category.findMany({
|
||||||
|
where: { sdsCategoryId: { not: null } },
|
||||||
|
select: { sdsCategoryId: true, categoryName: true },
|
||||||
|
});
|
||||||
|
return rows.map((r) => ({ id: r.sdsCategoryId!, name: r.categoryName }));
|
||||||
|
},
|
||||||
|
fetchProductsPage: async () => ({ items: [] }),
|
||||||
|
fetchProductDetail: async (goodId: string | number) => ({ id: goodId }),
|
||||||
|
};
|
||||||
|
const moduleRef = await Test.createTestingModule({
|
||||||
|
providers: [
|
||||||
|
PublicService,
|
||||||
|
CategoriesService,
|
||||||
|
CountriesService,
|
||||||
|
TagsService,
|
||||||
|
TagGroupsService,
|
||||||
|
PositionsService,
|
||||||
|
GoodsService,
|
||||||
|
SyncService,
|
||||||
|
FamilyRecomputeService,
|
||||||
|
ProductFamiliesService,
|
||||||
|
OrganizeService,
|
||||||
|
OriginGoodsService,
|
||||||
|
{ provide: SdsClientService, useValue: sdsMock },
|
||||||
|
PrismaService,
|
||||||
|
PublicCacheService,
|
||||||
|
],
|
||||||
|
}).compile();
|
||||||
|
service = moduleRef.get(PublicService);
|
||||||
|
cache = moduleRef.get(PublicCacheService);
|
||||||
|
categoriesService = moduleRef.get(CategoriesService);
|
||||||
|
countriesService = moduleRef.get(CountriesService);
|
||||||
|
tagsService = moduleRef.get(TagsService);
|
||||||
|
tagGroupsService = moduleRef.get(TagGroupsService);
|
||||||
|
positionsService = moduleRef.get(PositionsService);
|
||||||
|
goodsService = moduleRef.get(GoodsService);
|
||||||
|
syncService = moduleRef.get(SyncService);
|
||||||
|
prisma = moduleRef.get(PrismaService);
|
||||||
|
await prisma.onModuleInit();
|
||||||
|
|
||||||
|
const country = await prisma.country.create({
|
||||||
|
data: { countryName: `失效测试国家 ${stamp}` },
|
||||||
|
});
|
||||||
|
countryId = country.id;
|
||||||
|
sdsCategoryId = `inv-sds-cat-${stamp}`;
|
||||||
|
const category = await prisma.category.create({
|
||||||
|
data: { categoryName: `失效测试分类 ${stamp}`, sdsCategoryId },
|
||||||
|
});
|
||||||
|
categoryId = category.id;
|
||||||
|
const tagGroup = await prisma.tagGroup.create({
|
||||||
|
data: { groupName: `失效测试标签组 ${stamp}` },
|
||||||
|
});
|
||||||
|
tagGroupId = tagGroup.id;
|
||||||
|
const tag = await prisma.tag.create({
|
||||||
|
data: { tagName: `失效测试标签 ${stamp}`, tagGroupId },
|
||||||
|
});
|
||||||
|
tagId = tag.id;
|
||||||
|
const position = await prisma.position.create({
|
||||||
|
data: { indexVal: 1, countryId, categoryId },
|
||||||
|
});
|
||||||
|
positionId = position.id;
|
||||||
|
|
||||||
|
sdsGoodId = `inv-sds-good-${stamp}`;
|
||||||
|
const og = await prisma.originGood.create({
|
||||||
|
data: { sdsGoodId, goodName: `失效测试链接 ${stamp}`, goodImage: 'http://img' },
|
||||||
|
});
|
||||||
|
originGoodId = og.id;
|
||||||
|
const family = await prisma.productFamily.create({
|
||||||
|
data: { familyName: `失效测试族 ${stamp}`, primaryOriginGoodId: og.id },
|
||||||
|
});
|
||||||
|
familyId = family.id;
|
||||||
|
await prisma.originGood.update({
|
||||||
|
where: { id: og.id },
|
||||||
|
data: { familyId },
|
||||||
|
});
|
||||||
|
const good = await prisma.good.create({
|
||||||
|
data: {
|
||||||
|
goodName: `失效测试商品 ${stamp}`,
|
||||||
|
originGoodId,
|
||||||
|
familyId,
|
||||||
|
countryId,
|
||||||
|
categoryId,
|
||||||
|
goodPriority: 5,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
goodId = good.id;
|
||||||
|
}, 30_000);
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await prisma.good.deleteMany({ where: { id: goodId } });
|
||||||
|
await prisma.originGoodTag.deleteMany({ where: { originGoodId } });
|
||||||
|
await prisma.originGoodVariant.deleteMany({ where: { originGoodId } });
|
||||||
|
await prisma.originGoodDetail.deleteMany({ where: { originGoodId } });
|
||||||
|
await prisma.originGood.deleteMany({ where: { id: originGoodId } });
|
||||||
|
await prisma.familyPriceOverride.deleteMany({ where: { familyId } });
|
||||||
|
await prisma.productFamily.deleteMany({ where: { id: familyId } });
|
||||||
|
await prisma.position.deleteMany({ where: { id: positionId } });
|
||||||
|
await prisma.tag.deleteMany({ where: { id: tagId } });
|
||||||
|
await prisma.tagGroup.deleteMany({ where: { id: tagGroupId } });
|
||||||
|
await prisma.category.deleteMany({
|
||||||
|
where: { OR: [{ id: categoryId }, { sdsCategoryId: { startsWith: `inv-sds-cat-${stamp}` } }] },
|
||||||
|
});
|
||||||
|
await prisma.country.deleteMany({ where: { id: countryId } });
|
||||||
|
await prisma.$disconnect();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('列表读缓存:同筛选重复请求与翻页共享一份缓存,只查一次库', async () => {
|
||||||
|
const findManySpy = jest.spyOn(prisma.good, 'findMany');
|
||||||
|
const query = {
|
||||||
|
page: 1,
|
||||||
|
pageSize: 1,
|
||||||
|
countryId: countryId.toString(),
|
||||||
|
keyword: `失效测试商品 ${stamp}`,
|
||||||
|
sort: 'DEFAULT' as const,
|
||||||
|
};
|
||||||
|
const first = await service.getGoods(query);
|
||||||
|
const second = await service.getGoods(query);
|
||||||
|
const page2 = await service.getGoods({ ...query, page: 2 });
|
||||||
|
expect(first.items).toHaveLength(1);
|
||||||
|
expect(first.items[0].goodId).toBe(familyId.toString());
|
||||||
|
expect(second.items).toEqual(first.items);
|
||||||
|
expect(page2.items).toHaveLength(0); // total=1,第二页为空,但仍命中同一份缓存
|
||||||
|
expect(findManySpy).toHaveBeenCalledTimes(1);
|
||||||
|
findManySpy.mockRestore();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('详情读缓存:重复请求不再查族表', async () => {
|
||||||
|
const familySpy = jest.spyOn(prisma.productFamily, 'findUnique');
|
||||||
|
const first = await service.getGood(familyId.toString());
|
||||||
|
const second = await service.getGood(familyId.toString());
|
||||||
|
expect(first.goodId).toBe(familyId.toString());
|
||||||
|
expect(second).toEqual(first);
|
||||||
|
expect(familySpy).toHaveBeenCalledTimes(1);
|
||||||
|
familySpy.mockRestore();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('TTL 未到但分类改名 → 分类树立即返回新名(写后即失效,端到端)', async () => {
|
||||||
|
const oldName = `失效测试分类 ${stamp}`;
|
||||||
|
const newName = `失效后分类 ${stamp}`;
|
||||||
|
const categorySpy = jest.spyOn(prisma.category, 'findMany');
|
||||||
|
const before = await service.getCategoriesTree();
|
||||||
|
const flatBefore = JSON.stringify(before);
|
||||||
|
expect(flatBefore).toContain(oldName);
|
||||||
|
const callsAfterWarm = categorySpy.mock.calls.length;
|
||||||
|
|
||||||
|
await categoriesService.update(categoryId, { categoryName: newName });
|
||||||
|
|
||||||
|
const after = await service.getCategoriesTree();
|
||||||
|
expect(JSON.stringify(after)).toContain(newName);
|
||||||
|
expect(JSON.stringify(after)).not.toContain(oldName);
|
||||||
|
// 旧值来自缓存则不会再查库;失效正确时应产生新的分类查询
|
||||||
|
expect(categorySpy.mock.calls.length).toBeGreaterThan(callsAfterWarm);
|
||||||
|
categorySpy.mockRestore();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('分类写路径 bump meta(Countries/Tags/TagGroups)', async () => {
|
||||||
|
const meta0 = cache.version('meta');
|
||||||
|
const goods0 = cache.version('goods');
|
||||||
|
await countriesService.update(countryId, {
|
||||||
|
countryName: `失效测试国家改 ${stamp}`,
|
||||||
|
countryIcon: null,
|
||||||
|
});
|
||||||
|
expect(cache.version('meta')).toBe(meta0 + 1);
|
||||||
|
|
||||||
|
await tagsService.update(tagId, { tagName: `失效测试标签改 ${stamp}` });
|
||||||
|
expect(cache.version('meta')).toBe(meta0 + 2);
|
||||||
|
|
||||||
|
await tagGroupsService.update(tagGroupId, { groupName: `失效测试标签组改 ${stamp}` });
|
||||||
|
expect(cache.version('meta')).toBe(meta0 + 3);
|
||||||
|
expect(cache.version('goods')).toBe(goods0); // meta 写路径不误伤 goods 域
|
||||||
|
});
|
||||||
|
|
||||||
|
it('positions/goods 写路径 bump goods', async () => {
|
||||||
|
const goods0 = cache.version('goods');
|
||||||
|
const meta0 = cache.version('meta');
|
||||||
|
await positionsService.update(positionId, { indexVal: 9 });
|
||||||
|
expect(cache.version('goods')).toBe(goods0 + 1);
|
||||||
|
|
||||||
|
await goodsService.update(goodId, { goodName: `失效测试商品改 ${stamp}` });
|
||||||
|
// update 内部联动 recomputeFamily(goods+matrix)后再显式 bump goods
|
||||||
|
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods0 + 2);
|
||||||
|
expect(cache.version('meta')).toBe(meta0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('族重算 recomputeFamily bump goods+matrix', async () => {
|
||||||
|
const goods0 = cache.version('goods');
|
||||||
|
const matrix0 = cache.version('matrix');
|
||||||
|
const recompute = new FamilyRecomputeService(prisma, cache);
|
||||||
|
await recompute.recomputeFamily(familyId);
|
||||||
|
// goods 用 ≥:goods.update 等前置操作可能触发异步详情同步(fire-and-forget
|
||||||
|
// persistProductDetail 也会 bump goods),不与本断言强耦合;matrix 仅重算写
|
||||||
|
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods0 + 1);
|
||||||
|
expect(cache.version('matrix')).toBe(matrix0 + 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('链接人工改标签 bump meta+goods+matrix(含无族路径兜底)', async () => {
|
||||||
|
const meta0 = cache.version('meta');
|
||||||
|
const goods0 = cache.version('goods');
|
||||||
|
const matrix0 = cache.version('matrix');
|
||||||
|
const originGoodsService = new OriginGoodsService(
|
||||||
|
prisma,
|
||||||
|
new FamilyRecomputeService(prisma, cache),
|
||||||
|
new OrganizeService(prisma, new FamilyRecomputeService(prisma, cache), new ProductFamiliesService(prisma, new FamilyRecomputeService(prisma, cache)), cache),
|
||||||
|
new ProductFamiliesService(prisma, new FamilyRecomputeService(prisma, cache)),
|
||||||
|
cache,
|
||||||
|
);
|
||||||
|
await originGoodsService.updateTags(originGoodId, []);
|
||||||
|
expect(cache.version('meta')).toBeGreaterThanOrEqual(meta0 + 1);
|
||||||
|
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods0 + 1);
|
||||||
|
expect(cache.version('matrix')).toBeGreaterThanOrEqual(matrix0 + 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('同步写路径:syncCategories bump meta+goods,syncProducts bump goods,persistProductDetail bump goods', async () => {
|
||||||
|
const meta0 = cache.version('meta');
|
||||||
|
const goods0 = cache.version('goods');
|
||||||
|
await syncService.syncCategories();
|
||||||
|
expect(cache.version('meta')).toBe(meta0 + 1);
|
||||||
|
expect(cache.version('goods')).toBe(goods0 + 1);
|
||||||
|
|
||||||
|
await syncService.syncProducts();
|
||||||
|
expect(cache.version('goods')).toBe(goods0 + 2);
|
||||||
|
|
||||||
|
const goods1 = cache.version('goods');
|
||||||
|
// persistProductDetail 是私有方法,直调以断言挂钩(上游最小对象,mapper 全容忍)
|
||||||
|
await (syncService as unknown as { persistProductDetail: (id: bigint, up: unknown) => Promise<void> }).persistProductDetail(
|
||||||
|
originGoodId,
|
||||||
|
{ id: sdsGoodId },
|
||||||
|
);
|
||||||
|
expect(cache.version('goods')).toBeGreaterThanOrEqual(goods1 + 1);
|
||||||
|
}, 60_000);
|
||||||
|
});
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
import { Global, Module } from '@nestjs/common';
|
||||||
|
import { PublicCacheService } from './public-cache.service';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Global public 缓存模块,导出 {@link PublicCacheService}。
|
||||||
|
*
|
||||||
|
* 与 PrismaModule 同款 @Global 模式:public 读路径与各 admin/sync 写路径
|
||||||
|
* 都要注入失效入口(bump),逐一 import 太啰嗦。单进程部署下进程内即全局;
|
||||||
|
* 扩容多副本时缓存需换共享存储(见整改计划 P1-3 扩展阶梯)。
|
||||||
|
*/
|
||||||
|
@Global()
|
||||||
|
@Module({
|
||||||
|
providers: [PublicCacheService],
|
||||||
|
exports: [PublicCacheService],
|
||||||
|
})
|
||||||
|
export class PublicCacheModule {}
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
import { PublicCacheDomain, PublicCacheService } from './public-cache.service';
|
||||||
|
|
||||||
|
describe('PublicCacheService', () => {
|
||||||
|
let cache: PublicCacheService;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
cache = new PublicCacheService();
|
||||||
|
delete process.env.PUBLIC_CACHE_DISABLED;
|
||||||
|
delete process.env.PUBLIC_CACHE_TTL_MS;
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(() => {
|
||||||
|
delete process.env.PUBLIC_CACHE_DISABLED;
|
||||||
|
delete process.env.PUBLIC_CACHE_TTL_MS;
|
||||||
|
});
|
||||||
|
|
||||||
|
it('命中:相同 key 第二次调用不再执行 loader', async () => {
|
||||||
|
const loader = jest.fn(async () => ({ value: 1 }));
|
||||||
|
const first = await cache.wrap('k', ['goods'], loader);
|
||||||
|
const second = await cache.wrap('k', ['goods'], loader);
|
||||||
|
expect(first).toEqual({ value: 1 });
|
||||||
|
expect(second).toEqual({ value: 1 });
|
||||||
|
expect(loader).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('不同依赖域组合的相同 key 互不串缓存', async () => {
|
||||||
|
const a = await cache.wrap('k', ['goods'], async () => 'A');
|
||||||
|
const b = await cache.wrap('k', ['goods', 'matrix'], async () => 'B');
|
||||||
|
expect(a).toBe('A');
|
||||||
|
expect(b).toBe('B');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('TTL 过期后重新执行 loader(TTL 只是兜底,不是失效手段)', async () => {
|
||||||
|
jest.useFakeTimers();
|
||||||
|
jest.setSystemTime(1_700_000_000_000);
|
||||||
|
try {
|
||||||
|
process.env.PUBLIC_CACHE_TTL_MS = '1000';
|
||||||
|
const loader = jest.fn(async () => ({ v: 1 }));
|
||||||
|
await cache.wrap('k', ['goods'], loader);
|
||||||
|
jest.setSystemTime(1_700_000_000_000 + 999);
|
||||||
|
await cache.wrap('k', ['goods'], loader);
|
||||||
|
expect(loader).toHaveBeenCalledTimes(1);
|
||||||
|
jest.setSystemTime(1_700_000_000_000 + 1001);
|
||||||
|
await cache.wrap('k', ['goods'], loader);
|
||||||
|
expect(loader).toHaveBeenCalledTimes(2);
|
||||||
|
} finally {
|
||||||
|
jest.useRealTimers();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('bump 域版本后,依赖该域的缓存立即作废', async () => {
|
||||||
|
const loader = jest.fn(async () => ({ v: 1 }));
|
||||||
|
await cache.wrap('list', ['goods', 'matrix'], loader);
|
||||||
|
cache.bump('goods');
|
||||||
|
await cache.wrap('list', ['goods', 'matrix'], loader);
|
||||||
|
expect(loader).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('bump 某域不影响不依赖该域的条目(域间隔离)', async () => {
|
||||||
|
const metaLoader = jest.fn(async () => 'meta-data');
|
||||||
|
const goodsLoader = jest.fn(async () => 'goods-data');
|
||||||
|
await cache.wrap('meta-key', ['meta'], metaLoader);
|
||||||
|
cache.bump('goods');
|
||||||
|
await cache.wrap('meta-key', ['meta'], metaLoader);
|
||||||
|
expect(metaLoader).toHaveBeenCalledTimes(1);
|
||||||
|
await cache.wrap('goods-key', ['goods'], goodsLoader);
|
||||||
|
expect(goodsLoader).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('跨域条目:任一依赖域 bump 都使其作废', async () => {
|
||||||
|
const loader = jest.fn(async () => 'x');
|
||||||
|
await cache.wrap('combo', ['goods', 'matrix', 'meta'], loader);
|
||||||
|
cache.bump('matrix');
|
||||||
|
await cache.wrap('combo', ['goods', 'matrix', 'meta'], loader);
|
||||||
|
expect(loader).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('竞态防护:loader 执行期间发生 bump → 结果照常返回但不入缓存', async () => {
|
||||||
|
let releaseLoader!: (v: string) => void;
|
||||||
|
const loader = jest.fn(
|
||||||
|
() =>
|
||||||
|
new Promise<string>((resolve) => {
|
||||||
|
releaseLoader = resolve;
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const inflight = cache.wrap('k', ['goods'], loader);
|
||||||
|
cache.bump('goods'); // 加载期间数据变了
|
||||||
|
releaseLoader('stale-value');
|
||||||
|
await expect(inflight).resolves.toBe('stale-value');
|
||||||
|
|
||||||
|
const loader2 = jest.fn(async () => 'fresh-value');
|
||||||
|
await expect(cache.wrap('k', ['goods'], loader2)).resolves.toBe('fresh-value');
|
||||||
|
expect(loader2).toHaveBeenCalledTimes(1); // 旧值没有写回缓存
|
||||||
|
});
|
||||||
|
|
||||||
|
it('并发合并:并发 miss 只触发一次 loader(防击穿)', async () => {
|
||||||
|
let release!: () => void;
|
||||||
|
const loader = jest.fn(
|
||||||
|
() =>
|
||||||
|
new Promise<number>((resolve) => {
|
||||||
|
release = () => resolve(42);
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const p1 = cache.wrap('hot', ['goods'], loader);
|
||||||
|
const p2 = cache.wrap('hot', ['goods'], loader);
|
||||||
|
const p3 = cache.wrap('hot', ['goods'], loader);
|
||||||
|
release();
|
||||||
|
expect(await Promise.all([p1, p2, p3])).toEqual([42, 42, 42]);
|
||||||
|
expect(loader).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('PUBLIC_CACHE_DISABLED=true 时直通 loader、不缓存', async () => {
|
||||||
|
process.env.PUBLIC_CACHE_DISABLED = 'true';
|
||||||
|
const loader = jest.fn(async () => ({ v: 1 }));
|
||||||
|
await cache.wrap('k', ['goods'], loader);
|
||||||
|
await cache.wrap('k', ['goods'], loader);
|
||||||
|
expect(loader).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('loader 抛错不缓存、不残留 pending', async () => {
|
||||||
|
const failing = jest.fn(async () => {
|
||||||
|
throw new Error('boom');
|
||||||
|
});
|
||||||
|
await expect(cache.wrap('k', ['goods'], failing)).rejects.toThrow('boom');
|
||||||
|
const ok = jest.fn(async () => 'good');
|
||||||
|
await expect(cache.wrap('k', ['goods'], ok)).resolves.toBe('good');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('条目上限:先清过期条目,仍超限则按插入序淘汰最旧的', async () => {
|
||||||
|
jest.useFakeTimers();
|
||||||
|
jest.setSystemTime(1_700_000_000_000);
|
||||||
|
try {
|
||||||
|
process.env.PUBLIC_CACHE_TTL_MS = '1000';
|
||||||
|
process.env.PUBLIC_CACHE_MAX_ENTRIES = '3';
|
||||||
|
await cache.wrap('old', ['goods'], async () => 'old');
|
||||||
|
jest.setSystemTime(1_700_000_000_000 + 2000);
|
||||||
|
// old 已过期;再写 4 条活跃条目,第一条应顺带清掉过期的 old
|
||||||
|
for (let i = 0; i < 4; i++) {
|
||||||
|
await cache.wrap(`k${i}`, ['goods'], async () => i);
|
||||||
|
}
|
||||||
|
const loader = jest.fn(async () => 'reloaded');
|
||||||
|
// k0 应已被容量淘汰,重新加载;k3 仍命中
|
||||||
|
await cache.wrap('k0', ['goods'], loader);
|
||||||
|
expect(loader).toHaveBeenCalledTimes(1);
|
||||||
|
const hit = jest.fn(async () => -1);
|
||||||
|
await cache.wrap('k3', ['goods'], hit);
|
||||||
|
expect(hit).toHaveBeenCalledTimes(0);
|
||||||
|
} finally {
|
||||||
|
delete process.env.PUBLIC_CACHE_MAX_ENTRIES;
|
||||||
|
jest.useRealTimers();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('version() 暴露域版本用于诊断,bump 递增', () => {
|
||||||
|
const before = cache.version('goods');
|
||||||
|
cache.bump('goods');
|
||||||
|
expect(cache.version('goods')).toBe(before + 1);
|
||||||
|
expect(cache.version('matrix')).toBe(before === 0 ? 0 : cache.version('matrix'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('未知域名直接抛错(编程错误早暴露)', async () => {
|
||||||
|
expect(() => cache.bump('nope' as PublicCacheDomain)).toThrow();
|
||||||
|
await expect(cache.wrap('k', ['nope' as PublicCacheDomain], async () => 1)).rejects.toThrow();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,171 @@
|
|||||||
|
import { Injectable, Logger } from '@nestjs/common';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* public 读路径的进程内缓存(性能整改 P0-1,详见
|
||||||
|
* docs/references/performance-review-public-port.md 与
|
||||||
|
* plans/refactor/public-capacity-10k-refactor.md)。
|
||||||
|
*
|
||||||
|
* 语义要点(全局约束 §2 的落地):
|
||||||
|
* - 分域版本号失效:写路径在事务提交成功后 bump 依赖域,依赖该域的缓存
|
||||||
|
* 条目立即作废——「等 TTL 自然过期」只是内存回收兜底,不是失效手段;
|
||||||
|
* - 条目记录写入时全部依赖域的版本快照,读时逐一比对,跨域依赖
|
||||||
|
* (列表同时依赖 goods+matrix+meta)天然联动失效;
|
||||||
|
* - loader 完成后复核版本:bump 发生在加载期间 → 结果照常返回但不入缓存,
|
||||||
|
* 杜绝「失效瞬间的并发读把旧值写回」竞态;
|
||||||
|
* - in-flight 合并:并发 miss 只触发一次 loader,防瞬时洪峰击穿缓存;
|
||||||
|
* - 单进程部署下进程内即全局缓存;扩容多副本时需换共享存储(计划 P1-3 阶梯 c)。
|
||||||
|
*
|
||||||
|
* 环境开关:PUBLIC_CACHE_DISABLED=true 全直通(应急);
|
||||||
|
* PUBLIC_CACHE_TTL_MS / PUBLIC_CACHE_MAX_ENTRIES 可调(默认 10 分钟 / 512 条)。
|
||||||
|
*/
|
||||||
|
|
||||||
|
export const PUBLIC_CACHE_DOMAINS = ['meta', 'goods', 'matrix'] as const;
|
||||||
|
export type PublicCacheDomain = (typeof PUBLIC_CACHE_DOMAINS)[number];
|
||||||
|
|
||||||
|
const DEFAULT_TTL_MS = 10 * 60 * 1000;
|
||||||
|
const DEFAULT_MAX_ENTRIES = 512;
|
||||||
|
|
||||||
|
interface CacheEntry {
|
||||||
|
value: unknown;
|
||||||
|
expiresAt: number;
|
||||||
|
/** 写入时各依赖域的版本快照——任一域 bump 即作废 */
|
||||||
|
versions: ReadonlyMap<PublicCacheDomain, number>;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Injectable()
|
||||||
|
export class PublicCacheService {
|
||||||
|
private readonly logger = new Logger(PublicCacheService.name);
|
||||||
|
private readonly versions = new Map<PublicCacheDomain, number>(
|
||||||
|
PUBLIC_CACHE_DOMAINS.map((domain) => [domain, 0]),
|
||||||
|
);
|
||||||
|
private readonly entries = new Map<string, CacheEntry>();
|
||||||
|
private readonly pending = new Map<string, Promise<unknown>>();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 读穿透封装:命中(未过期且全部依赖域版本未变)直接返回缓存值,
|
||||||
|
* 否则执行 loader 并缓存。domains 声明该值依赖的数据域——
|
||||||
|
* 写路径 bump 其中任一域都会让本条目作废。
|
||||||
|
*/
|
||||||
|
async wrap<T>(
|
||||||
|
key: string,
|
||||||
|
domains: readonly PublicCacheDomain[],
|
||||||
|
loader: () => Promise<T>,
|
||||||
|
): Promise<T> {
|
||||||
|
this.assertDomains(domains);
|
||||||
|
if (this.disabled()) return loader();
|
||||||
|
|
||||||
|
const storeKey = `${[...domains].sort().join('+')}|${key}`;
|
||||||
|
const hit = this.readHit(storeKey);
|
||||||
|
if (hit !== undefined) return hit as T;
|
||||||
|
|
||||||
|
const inflight = this.pending.get(storeKey);
|
||||||
|
if (inflight) return inflight as Promise<T>;
|
||||||
|
|
||||||
|
const snapshot = this.snapshotVersions(domains);
|
||||||
|
const run: Promise<T> = (async () => {
|
||||||
|
const value = await loader();
|
||||||
|
if (this.versionsMatch(snapshot)) {
|
||||||
|
this.writeEntry(storeKey, value, snapshot);
|
||||||
|
} else {
|
||||||
|
this.logger.debug(`cache skip (version bumped during load): ${storeKey}`);
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
})();
|
||||||
|
this.pending.set(storeKey, run as Promise<unknown>);
|
||||||
|
// 失败与成功都要摘掉 pending,异常不得残留在途槽位
|
||||||
|
void run.catch(() => undefined).finally(() => this.pending.delete(storeKey));
|
||||||
|
return run;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 失效入口:写事务提交成功后调用。递增域版本并清除依赖该域的条目
|
||||||
|
* (版本号本身已保证正确性,清条目只为及时回收内存)。
|
||||||
|
*/
|
||||||
|
bump(...domains: PublicCacheDomain[]): void {
|
||||||
|
this.assertDomains(domains);
|
||||||
|
for (const domain of domains) {
|
||||||
|
this.versions.set(domain, (this.versions.get(domain) ?? 0) + 1);
|
||||||
|
}
|
||||||
|
if (this.entries.size === 0) return;
|
||||||
|
for (const [key, entry] of this.entries) {
|
||||||
|
if (domains.some((domain) => entry.versions.has(domain))) {
|
||||||
|
this.entries.delete(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 当前域版本(诊断/测试用) */
|
||||||
|
version(domain: PublicCacheDomain): number {
|
||||||
|
this.assertDomains([domain]);
|
||||||
|
return this.versions.get(domain) ?? 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
private readHit(storeKey: string): unknown {
|
||||||
|
const entry = this.entries.get(storeKey);
|
||||||
|
if (!entry) return undefined;
|
||||||
|
if (Date.now() >= entry.expiresAt) {
|
||||||
|
this.entries.delete(storeKey);
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (!this.versionsMatch(entry.versions)) {
|
||||||
|
this.entries.delete(storeKey);
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
return entry.value;
|
||||||
|
}
|
||||||
|
|
||||||
|
private versionsMatch(snapshot: ReadonlyMap<PublicCacheDomain, number>): boolean {
|
||||||
|
for (const [domain, version] of snapshot) {
|
||||||
|
if ((this.versions.get(domain) ?? 0) !== version) return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private snapshotVersions(
|
||||||
|
domains: readonly PublicCacheDomain[],
|
||||||
|
): Map<PublicCacheDomain, number> {
|
||||||
|
return new Map(domains.map((domain) => [domain, this.versions.get(domain) ?? 0]));
|
||||||
|
}
|
||||||
|
|
||||||
|
private writeEntry(
|
||||||
|
storeKey: string,
|
||||||
|
value: unknown,
|
||||||
|
versions: ReadonlyMap<PublicCacheDomain, number>,
|
||||||
|
): void {
|
||||||
|
const maxEntries = this.maxEntries();
|
||||||
|
if (this.entries.size >= maxEntries) {
|
||||||
|
const now = Date.now();
|
||||||
|
for (const [key, entry] of this.entries) {
|
||||||
|
if (entry.expiresAt <= now) this.entries.delete(key);
|
||||||
|
}
|
||||||
|
while (this.entries.size >= maxEntries) {
|
||||||
|
const oldest = this.entries.keys().next().value;
|
||||||
|
if (oldest === undefined) break;
|
||||||
|
this.entries.delete(oldest);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.entries.set(storeKey, { value, expiresAt: Date.now() + this.ttlMs(), versions });
|
||||||
|
}
|
||||||
|
|
||||||
|
private ttlMs(): number {
|
||||||
|
const parsed = Number(process.env.PUBLIC_CACHE_TTL_MS);
|
||||||
|
return Number.isFinite(parsed) && parsed > 0 ? parsed : DEFAULT_TTL_MS;
|
||||||
|
}
|
||||||
|
|
||||||
|
private maxEntries(): number {
|
||||||
|
const parsed = Number(process.env.PUBLIC_CACHE_MAX_ENTRIES);
|
||||||
|
return Number.isFinite(parsed) && parsed > 0 ? parsed : DEFAULT_MAX_ENTRIES;
|
||||||
|
}
|
||||||
|
|
||||||
|
private disabled(): boolean {
|
||||||
|
return process.env.PUBLIC_CACHE_DISABLED === 'true';
|
||||||
|
}
|
||||||
|
|
||||||
|
private assertDomains(domains: readonly PublicCacheDomain[]): void {
|
||||||
|
for (const domain of domains) {
|
||||||
|
if (!this.versions.has(domain)) {
|
||||||
|
throw new Error(`未知的 public 缓存域: ${String(domain)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from './public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { Prisma } from '@prisma/client';
|
import { Prisma } from '@prisma/client';
|
||||||
import { PublicService } from './public.service';
|
import { PublicService } from './public.service';
|
||||||
@@ -24,8 +25,11 @@ describe('PublicService family block (PUBLIC_DETAIL_FROM_FAMILY)', () => {
|
|||||||
let familyId = 0n;
|
let familyId = 0n;
|
||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
|
// 本套件测数据语义(裸 prisma 改数后立即读公开端点),不走带 bump 的写路径——
|
||||||
|
// 禁用 public 缓存;缓存命中/失效语义由 public-cache.invalidation.spec.ts 覆盖
|
||||||
|
process.env.PUBLIC_CACHE_DISABLED = 'true';
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [PublicService, PrismaService],
|
providers: [PublicService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(PublicService);
|
service = moduleRef.get(PublicService);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
@@ -71,11 +75,12 @@ describe('PublicService family block (PUBLIC_DETAIL_FROM_FAMILY)', () => {
|
|||||||
createdFamilyIds.push(family.id);
|
createdFamilyIds.push(family.id);
|
||||||
await prisma.originGood.update({ where: { id: og.id }, data: { familyId: family.id } });
|
await prisma.originGood.update({ where: { id: og.id }, data: { familyId: family.id } });
|
||||||
// 解析去运行时化:先整理派生标签,再重算(重算只聚合不解析)
|
// 解析去运行时化:先整理派生标签,再重算(重算只聚合不解析)
|
||||||
const recomputeSvc = new FamilyRecomputeService(prisma);
|
const recomputeSvc = new FamilyRecomputeService(prisma, new PublicCacheService());
|
||||||
const organizeSvc = new OrganizeService(
|
const organizeSvc = new OrganizeService(
|
||||||
prisma,
|
prisma,
|
||||||
recomputeSvc,
|
recomputeSvc,
|
||||||
new ProductFamiliesService(prisma, recomputeSvc),
|
new ProductFamiliesService(prisma, recomputeSvc),
|
||||||
|
new PublicCacheService(),
|
||||||
);
|
);
|
||||||
await organizeSvc.deriveFamilyTags(family.id);
|
await organizeSvc.deriveFamilyTags(family.id);
|
||||||
await recomputeSvc.recomputeFamily(family.id);
|
await recomputeSvc.recomputeFamily(family.id);
|
||||||
@@ -196,13 +201,14 @@ describe('Good familyId derivation', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [PrismaService],
|
providers: [PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
await prisma.onModuleInit();
|
await prisma.onModuleInit();
|
||||||
familiesService = new (require('../product-families/product-families.service').ProductFamiliesService)(
|
familiesService = new (require('../product-families/product-families.service').ProductFamiliesService)(
|
||||||
prisma,
|
prisma,
|
||||||
new FamilyRecomputeService(prisma),
|
new FamilyRecomputeService(prisma, new PublicCacheService()),
|
||||||
|
new PublicCacheService(),
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from './public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { BadRequestException, NotFoundException } from '@nestjs/common';
|
import { BadRequestException, NotFoundException } from '@nestjs/common';
|
||||||
import { PublicService } from './public.service';
|
import { PublicService } from './public.service';
|
||||||
@@ -20,8 +21,11 @@ describe('PublicService', () => {
|
|||||||
let goodIds: bigint[] = [];
|
let goodIds: bigint[] = [];
|
||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
|
// 本套件测数据语义(裸 prisma 改数后立即读公开端点),不走带 bump 的写路径——
|
||||||
|
// 禁用 public 缓存;缓存命中/失效语义由 public-cache.invalidation.spec.ts 覆盖
|
||||||
|
process.env.PUBLIC_CACHE_DISABLED = 'true';
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [PublicService, PrismaService],
|
providers: [PublicService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(PublicService);
|
service = moduleRef.get(PublicService);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
@@ -188,7 +192,7 @@ describe('PublicService', () => {
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
// 物化族矩阵(公开详情 family 块依赖 priceMatrix 已重算)
|
// 物化族矩阵(公开详情 family 块依赖 priceMatrix 已重算)
|
||||||
await new FamilyRecomputeService(prisma).recomputeFamily(family.id);
|
await new FamilyRecomputeService(prisma, new PublicCacheService()).recomputeFamily(family.id);
|
||||||
|
|
||||||
// Seed a good in `otherCategory` so the "onlyHaveGoods" filter
|
// Seed a good in `otherCategory` so the "onlyHaveGoods" filter
|
||||||
// returns more than one category.
|
// returns more than one category.
|
||||||
@@ -406,7 +410,7 @@ describe('PublicService', () => {
|
|||||||
expect(before.items).toHaveLength(1);
|
expect(before.items).toHaveLength(1);
|
||||||
expect(before.items[0].price).toBe('50');
|
expect(before.items[0].price).toBe('50');
|
||||||
|
|
||||||
await new FamilyRecomputeService(prisma).recomputeFamily(family.id);
|
await new FamilyRecomputeService(prisma, new PublicCacheService()).recomputeFamily(family.id);
|
||||||
const after = await service.getGoods({
|
const after = await service.getGoods({
|
||||||
page: 1,
|
page: 1,
|
||||||
pageSize: 50,
|
pageSize: 50,
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { BadRequestException, Injectable, NotFoundException } from '@nestjs/common';
|
import { BadRequestException, Injectable, NotFoundException } from '@nestjs/common';
|
||||||
import { Category as PrismaCategory, Prisma } from '@prisma/client';
|
import { Category as PrismaCategory, Prisma } from '@prisma/client';
|
||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from './public-cache.service';
|
||||||
import {
|
import {
|
||||||
PublicHomeGoodsQueryDto,
|
PublicHomeGoodsQueryDto,
|
||||||
PublicQueryGoodDto,
|
PublicQueryGoodDto,
|
||||||
@@ -86,9 +87,28 @@ interface TreeOrderMeta {
|
|||||||
|
|
||||||
@Injectable()
|
@Injectable()
|
||||||
export class PublicService {
|
export class PublicService {
|
||||||
constructor(private readonly prisma: PrismaService) {}
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly cache: PublicCacheService,
|
||||||
|
) {}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 以下读端点统一走 PublicCacheService(性能整改 P0-1,TTL 只是内存回收
|
||||||
|
* 兜底,数据新鲜度由写路径 bump 保证,见 public-cache.service.ts)。
|
||||||
|
* 依赖域声明:
|
||||||
|
* - meta:分类/国家/标签组等低熵元数据(含 DEFAULT 排序的树序元数据)
|
||||||
|
* - goods:goods 行/关联展示数据(含 position)
|
||||||
|
* - matrix:族 price_matrix 物化 JSON(族最低价聚合)
|
||||||
|
* 列表/详情同时展示元数据名与族价格 → 三域并依赖,任一写路径 bump 即失效。
|
||||||
|
*/
|
||||||
|
|
||||||
async getCategoriesTree(countryId?: string): Promise<PublicCategoryNodeDto[]> {
|
async getCategoriesTree(countryId?: string): Promise<PublicCategoryNodeDto[]> {
|
||||||
|
return this.cache.wrap(`cat-tree:${countryId ?? 'all'}`, ['meta', 'goods'], () =>
|
||||||
|
this.loadCategoriesTree(countryId),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadCategoriesTree(countryId?: string): Promise<PublicCategoryNodeDto[]> {
|
||||||
const goodsWhere: Prisma.GoodWhereInput = {
|
const goodsWhere: Prisma.GoodWhereInput = {
|
||||||
familyId: { not: null },
|
familyId: { not: null },
|
||||||
originGood: { delisted: false },
|
originGood: { delisted: false },
|
||||||
@@ -133,6 +153,10 @@ export class PublicService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async getCountries(): Promise<PublicCountryDto[]> {
|
async getCountries(): Promise<PublicCountryDto[]> {
|
||||||
|
return this.cache.wrap('countries', ['meta', 'goods'], () => this.loadCountries());
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadCountries(): Promise<PublicCountryDto[]> {
|
||||||
const rows = await this.prisma.country.findMany({
|
const rows = await this.prisma.country.findMany({
|
||||||
where: { goods: { some: { familyId: { not: null }, originGood: { delisted: false } } } },
|
where: { goods: { some: { familyId: { not: null }, originGood: { delisted: false } } } },
|
||||||
orderBy: [{ sortOrder: 'asc' }, { id: 'asc' }],
|
orderBy: [{ sortOrder: 'asc' }, { id: 'asc' }],
|
||||||
@@ -141,6 +165,10 @@ export class PublicService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async getTags(): Promise<PublicTagDto[]> {
|
async getTags(): Promise<PublicTagDto[]> {
|
||||||
|
return this.cache.wrap('tags', ['meta', 'goods'], () => this.loadTags());
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadTags(): Promise<PublicTagDto[]> {
|
||||||
const rows = await this.prisma.tag.findMany({
|
const rows = await this.prisma.tag.findMany({
|
||||||
where: {
|
where: {
|
||||||
goodTags: {
|
goodTags: {
|
||||||
@@ -158,6 +186,12 @@ export class PublicService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async getTagGroups(countryId?: string): Promise<PublicTagGroupFilterDto[]> {
|
async getTagGroups(countryId?: string): Promise<PublicTagGroupFilterDto[]> {
|
||||||
|
return this.cache.wrap(`tag-groups:${countryId ?? 'all'}`, ['meta', 'goods'], () =>
|
||||||
|
this.loadTagGroups(countryId),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadTagGroups(countryId?: string): Promise<PublicTagGroupFilterDto[]> {
|
||||||
const goodWhere: Prisma.GoodWhereInput = {
|
const goodWhere: Prisma.GoodWhereInput = {
|
||||||
familyId: { not: null },
|
familyId: { not: null },
|
||||||
originGood: { delisted: false },
|
originGood: { delisted: false },
|
||||||
@@ -189,7 +223,44 @@ export class PublicService {
|
|||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 缓存键 = 筛选参数(不含 page/pageSize):缓存的是已排序已分组的全量
|
||||||
|
* items,翻页在缓存命中后内存切片——所有页码共享同一份物化结果。
|
||||||
|
*/
|
||||||
|
private goodsListCacheKey(query: PublicQueryGoodDto): string {
|
||||||
|
return [
|
||||||
|
'goods-list',
|
||||||
|
query.countryId ?? '',
|
||||||
|
query.keyword ?? '',
|
||||||
|
query.categoryId ?? '',
|
||||||
|
query.minPrice ?? '',
|
||||||
|
query.maxPrice ?? '',
|
||||||
|
query.sort ?? 'DEFAULT',
|
||||||
|
JSON.stringify(query.tags ?? []),
|
||||||
|
].join('|');
|
||||||
|
}
|
||||||
|
|
||||||
async getGoods(query: PublicQueryGoodDto): Promise<PublicPaginatedGoods> {
|
async getGoods(query: PublicQueryGoodDto): Promise<PublicPaginatedGoods> {
|
||||||
|
// 缓存的是「已排序已分组的全量 items」;分页切片必须在缓存外按请求执行,
|
||||||
|
// 否则后续页会拿到第一页的切片(缓存值会被多个页码共享)
|
||||||
|
const materialized = await this.cache.wrap(
|
||||||
|
this.goodsListCacheKey(query),
|
||||||
|
['goods', 'matrix', 'meta'],
|
||||||
|
() => this.loadGoodsMaterialized(query),
|
||||||
|
);
|
||||||
|
const start = (query.page - 1) * query.pageSize;
|
||||||
|
return {
|
||||||
|
items: materialized.items.slice(start, start + query.pageSize),
|
||||||
|
total: materialized.total,
|
||||||
|
page: query.page,
|
||||||
|
pageSize: query.pageSize,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadGoodsMaterialized(query: PublicQueryGoodDto): Promise<{
|
||||||
|
items: PublicGoodDto[];
|
||||||
|
total: number;
|
||||||
|
}> {
|
||||||
// 无族商品(自定义)不进公开列表:只认族
|
// 无族商品(自定义)不进公开列表:只认族
|
||||||
const where: Prisma.GoodWhereInput = {
|
const where: Prisma.GoodWhereInput = {
|
||||||
familyId: { not: null },
|
familyId: { not: null },
|
||||||
@@ -270,21 +341,18 @@ export class PublicService {
|
|||||||
query.sort === 'PRICE_ASC' ? num(a.price) - num(b.price) : num(b.price) - num(a.price),
|
query.sort === 'PRICE_ASC' ? num(a.price) - num(b.price) : num(b.price) - num(a.price),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
const total = items.length;
|
return { items, total: items.length };
|
||||||
const start = (query.page - 1) * query.pageSize;
|
|
||||||
return {
|
|
||||||
items: items.slice(start, start + query.pageSize),
|
|
||||||
total,
|
|
||||||
page: query.page,
|
|
||||||
pageSize: query.pageSize,
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 款序元数据:countries.sort_order(一级)+ 新树二/三级 categories.sort_order
|
* 款序元数据:countries.sort_order(一级)+ 新树二/三级 categories.sort_order
|
||||||
* (款顺序,回填自排序表)。key 用 origin_goods.sds_category_id 关联商品→款。
|
* (款顺序,回填自排序表)。key 用 origin_goods.sds_category_id 关联商品→款。
|
||||||
*/
|
*/
|
||||||
private async loadTreeOrderMeta(): Promise<TreeOrderMeta> {
|
private loadTreeOrderMeta(): Promise<TreeOrderMeta> {
|
||||||
|
return this.cache.wrap('tree-order-meta', ['meta'], () => this.queryTreeOrderMeta());
|
||||||
|
}
|
||||||
|
|
||||||
|
private async queryTreeOrderMeta(): Promise<TreeOrderMeta> {
|
||||||
const [countries, leaves] = await Promise.all([
|
const [countries, leaves] = await Promise.all([
|
||||||
this.prisma.country.findMany({ select: { id: true, sortOrder: true } }),
|
this.prisma.country.findMany({ select: { id: true, sortOrder: true } }),
|
||||||
this.prisma.$queryRaw<
|
this.prisma.$queryRaw<
|
||||||
@@ -331,7 +399,11 @@ export class PublicService {
|
|||||||
* 会让每个商品都携带整份矩阵(实测全量 ~330ms);PG 端展开聚合只回传
|
* 会让每个商品都携带整份矩阵(实测全量 ~330ms);PG 端展开聚合只回传
|
||||||
* 每族一个数字。非数字/缺失 price 的行跳过,与旧内存版过滤语义一致。
|
* 每族一个数字。非数字/缺失 price 的行跳过,与旧内存版过滤语义一致。
|
||||||
*/
|
*/
|
||||||
private async loadFamilyMinPrices(): Promise<Map<string, string>> {
|
private loadFamilyMinPrices(): Promise<Map<string, string>> {
|
||||||
|
return this.cache.wrap('family-min-prices', ['matrix'], () => this.queryFamilyMinPrices());
|
||||||
|
}
|
||||||
|
|
||||||
|
private async queryFamilyMinPrices(): Promise<Map<string, string>> {
|
||||||
const rows = await this.prisma.$queryRaw<
|
const rows = await this.prisma.$queryRaw<
|
||||||
Array<{ family_id: bigint | string; min_price: Prisma.Decimal | null }>
|
Array<{ family_id: bigint | string; min_price: Prisma.Decimal | null }>
|
||||||
>`
|
>`
|
||||||
@@ -365,6 +437,12 @@ export class PublicService {
|
|||||||
|
|
||||||
/** 族视角详情:代表 Good 提供公共字段(名称/主图/国家/分类),变体取全体成员并集 */
|
/** 族视角详情:代表 Good 提供公共字段(名称/主图/国家/分类),变体取全体成员并集 */
|
||||||
private async getGoodByFamilyId(familyId: bigint): Promise<PublicGoodDetailDto | null> {
|
private async getGoodByFamilyId(familyId: bigint): Promise<PublicGoodDetailDto | null> {
|
||||||
|
return this.cache.wrap(`family-detail:${familyId.toString()}`, ['goods', 'matrix', 'meta'], () =>
|
||||||
|
this.loadGoodByFamilyId(familyId),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadGoodByFamilyId(familyId: bigint): Promise<PublicGoodDetailDto | null> {
|
||||||
const [family, goods] = await Promise.all([
|
const [family, goods] = await Promise.all([
|
||||||
this.prisma.productFamily.findUnique({ where: { id: familyId } }),
|
this.prisma.productFamily.findUnique({ where: { id: familyId } }),
|
||||||
this.prisma.good.findMany({
|
this.prisma.good.findMany({
|
||||||
@@ -405,6 +483,14 @@ export class PublicService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async getHomeGoods(query: PublicHomeGoodsQueryDto): Promise<PublicGoodDto[]> {
|
async getHomeGoods(query: PublicHomeGoodsQueryDto): Promise<PublicGoodDto[]> {
|
||||||
|
return this.cache.wrap(
|
||||||
|
`home:${query.countryId ?? 'all'}:${query.limit}`,
|
||||||
|
['goods', 'matrix', 'meta'],
|
||||||
|
() => this.loadHomeGoods(query),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadHomeGoods(query: PublicHomeGoodsQueryDto): Promise<PublicGoodDto[]> {
|
||||||
const rows = await this.prisma.good.findMany({
|
const rows = await this.prisma.good.findMany({
|
||||||
where: {
|
where: {
|
||||||
positionId: { not: null },
|
positionId: { not: null },
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { Prisma } from '@prisma/client';
|
import { Prisma } from '@prisma/client';
|
||||||
import { SyncService } from './sync.service';
|
import { SyncService } from './sync.service';
|
||||||
@@ -27,6 +28,7 @@ describe('SyncService family hooks', () => {
|
|||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [
|
providers: [
|
||||||
|
PublicCacheService,
|
||||||
SyncService,
|
SyncService,
|
||||||
{
|
{
|
||||||
provide: SdsClientService,
|
provide: SdsClientService,
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { ConfigModule } from '@nestjs/config';
|
import { ConfigModule } from '@nestjs/config';
|
||||||
import {
|
import {
|
||||||
@@ -25,6 +26,7 @@ describe('SyncService', () => {
|
|||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
imports: [ConfigModule.forRoot({ isGlobal: true })],
|
imports: [ConfigModule.forRoot({ isGlobal: true })],
|
||||||
providers: [
|
providers: [
|
||||||
|
PublicCacheService,
|
||||||
SyncService,
|
SyncService,
|
||||||
{ provide: SdsClientService, useValue: sdsMock },
|
{ provide: SdsClientService, useValue: sdsMock },
|
||||||
{ provide: FamilyRecomputeService, useValue: { enqueue: jest.fn() } },
|
{ provide: FamilyRecomputeService, useValue: { enqueue: jest.fn() } },
|
||||||
@@ -295,7 +297,7 @@ describe('SyncService product detail scopes', () => {
|
|||||||
const familyRecompute = {
|
const familyRecompute = {
|
||||||
enqueue: jest.fn(),
|
enqueue: jest.fn(),
|
||||||
} as unknown as FamilyRecomputeService;
|
} as unknown as FamilyRecomputeService;
|
||||||
const scopedService = new SyncService(prisma, sds, familyRecompute);
|
const scopedService = new SyncService(prisma, sds, familyRecompute, new PublicCacheService());
|
||||||
jest
|
jest
|
||||||
.spyOn(scopedService as any, 'persistProductDetail')
|
.spyOn(scopedService as any, 'persistProductDetail')
|
||||||
.mockResolvedValue(undefined);
|
.mockResolvedValue(undefined);
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ import {
|
|||||||
} from './sds-client.service';
|
} from './sds-client.service';
|
||||||
import { normalizeProductDetail } from './sds-product-detail.mapper';
|
import { normalizeProductDetail } from './sds-product-detail.mapper';
|
||||||
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
import { FamilyRecomputeService } from '../product-families/family-recompute.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
|
|
||||||
|
|
||||||
export interface CategorySyncResult {
|
export interface CategorySyncResult {
|
||||||
@@ -96,6 +97,7 @@ export class SyncService {
|
|||||||
private readonly prisma: PrismaService,
|
private readonly prisma: PrismaService,
|
||||||
private readonly sds: SdsClientService,
|
private readonly sds: SdsClientService,
|
||||||
private readonly familyRecompute: FamilyRecomputeService,
|
private readonly familyRecompute: FamilyRecomputeService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
) {}
|
) {}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -298,6 +300,8 @@ export class SyncService {
|
|||||||
message: `inserted=${inserted} updated=${updated} total=${flat.length} staleDeleted=${deletedStale}`,
|
message: `inserted=${inserted} updated=${updated} total=${flat.length} staleDeleted=${deletedStale}`,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
// 分类树/树序元数据变化 → public 缓存失效(全局约束 §2:不等 TTL)
|
||||||
|
this.publicCache.bump('meta', 'goods');
|
||||||
return { inserted, updated, total: flat.length, deletedStale };
|
return { inserted, updated, total: flat.length, deletedStale };
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const message = err instanceof Error ? err.message : String(err);
|
const message = err instanceof Error ? err.message : String(err);
|
||||||
@@ -407,6 +411,8 @@ export class SyncService {
|
|||||||
message: `inserted=${inserted} updated=${updated} total=${total} delisted=${delistedCount} reactivated=${reactivatedCount} leafCategories=${leafRows.length}`,
|
message: `inserted=${inserted} updated=${updated} total=${total} delisted=${delistedCount} reactivated=${reactivatedCount} leafCategories=${leafRows.length}`,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
// 链接镜像(名称/图/价/上下架)变化 → public 商品列表/详情失效
|
||||||
|
this.publicCache.bump('goods');
|
||||||
return {
|
return {
|
||||||
inserted,
|
inserted,
|
||||||
updated,
|
updated,
|
||||||
@@ -677,6 +683,8 @@ export class SyncService {
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
// 详情/变体/价格落库 → public 详情缓存失效;族矩阵变化由重算钩子 bump matrix
|
||||||
|
this.publicCache.bump('goods');
|
||||||
// 族成员的详情/变体变化 → 异步重算该族(进程内去重)
|
// 族成员的详情/变体变化 → 异步重算该族(进程内去重)
|
||||||
await this.maybeEnqueueFamilyRecompute(originGoodId);
|
await this.maybeEnqueueFamilyRecompute(originGoodId);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { ConflictException, NotFoundException } from '@nestjs/common';
|
import { ConflictException, NotFoundException } from '@nestjs/common';
|
||||||
import { TagGroupsService } from './tag-groups.service';
|
import { TagGroupsService } from './tag-groups.service';
|
||||||
@@ -10,7 +11,7 @@ describe('TagGroupsService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [TagGroupsService, PrismaService],
|
providers: [TagGroupsService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(TagGroupsService);
|
service = moduleRef.get(TagGroupsService);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
|
|||||||
@@ -5,13 +5,17 @@ import {
|
|||||||
NotFoundException,
|
NotFoundException,
|
||||||
} from '@nestjs/common';
|
} from '@nestjs/common';
|
||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { CreateTagGroupDto } from './dto/create-tag-group.dto';
|
import { CreateTagGroupDto } from './dto/create-tag-group.dto';
|
||||||
import { UpdateTagGroupDto } from './dto/update-tag-group.dto';
|
import { UpdateTagGroupDto } from './dto/update-tag-group.dto';
|
||||||
import { ReorderTagGroupsDto } from './dto/reorder-tag-groups.dto';
|
import { ReorderTagGroupsDto } from './dto/reorder-tag-groups.dto';
|
||||||
|
|
||||||
@Injectable()
|
@Injectable()
|
||||||
export class TagGroupsService {
|
export class TagGroupsService {
|
||||||
constructor(private readonly prisma: PrismaService) {}
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
|
) {}
|
||||||
|
|
||||||
findAll() {
|
findAll() {
|
||||||
return this.prisma.tagGroup.findMany({
|
return this.prisma.tagGroup.findMany({
|
||||||
@@ -31,7 +35,7 @@ export class TagGroupsService {
|
|||||||
|
|
||||||
async create(dto: CreateTagGroupDto) {
|
async create(dto: CreateTagGroupDto) {
|
||||||
try {
|
try {
|
||||||
return await this.prisma.tagGroup.create({
|
const created = await this.prisma.tagGroup.create({
|
||||||
data: {
|
data: {
|
||||||
groupName: dto.groupName,
|
groupName: dto.groupName,
|
||||||
groupIcon: dto.groupIcon ?? null,
|
groupIcon: dto.groupIcon ?? null,
|
||||||
@@ -39,6 +43,8 @@ export class TagGroupsService {
|
|||||||
sortOrder: dto.sortOrder ?? 0,
|
sortOrder: dto.sortOrder ?? 0,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return created;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (
|
if (
|
||||||
err instanceof Prisma.PrismaClientKnownRequestError &&
|
err instanceof Prisma.PrismaClientKnownRequestError &&
|
||||||
@@ -58,7 +64,9 @@ export class TagGroupsService {
|
|||||||
if (dto.groupColor !== undefined) data.groupColor = dto.groupColor;
|
if (dto.groupColor !== undefined) data.groupColor = dto.groupColor;
|
||||||
if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder;
|
if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder;
|
||||||
try {
|
try {
|
||||||
return await this.prisma.tagGroup.update({ where: { id }, data });
|
const updated = await this.prisma.tagGroup.update({ where: { id }, data });
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return updated;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (
|
if (
|
||||||
err instanceof Prisma.PrismaClientKnownRequestError &&
|
err instanceof Prisma.PrismaClientKnownRequestError &&
|
||||||
@@ -72,11 +80,13 @@ export class TagGroupsService {
|
|||||||
|
|
||||||
async remove(id: bigint) {
|
async remove(id: bigint) {
|
||||||
await this.findOne(id);
|
await this.findOne(id);
|
||||||
return this.prisma.tagGroup.delete({ where: { id } });
|
const removed = await this.prisma.tagGroup.delete({ where: { id } });
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return removed;
|
||||||
}
|
}
|
||||||
|
|
||||||
async reorder(dto: ReorderTagGroupsDto) {
|
async reorder(dto: ReorderTagGroupsDto) {
|
||||||
return this.prisma.$transaction(
|
const result = await this.prisma.$transaction(
|
||||||
dto.items.map((item) =>
|
dto.items.map((item) =>
|
||||||
this.prisma.tagGroup.update({
|
this.prisma.tagGroup.update({
|
||||||
where: { id: BigInt(item.id) },
|
where: { id: BigInt(item.id) },
|
||||||
@@ -84,5 +94,7 @@ export class TagGroupsService {
|
|||||||
}),
|
}),
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return result;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { Test } from '@nestjs/testing';
|
import { Test } from '@nestjs/testing';
|
||||||
import { ConflictException } from '@nestjs/common';
|
import { ConflictException } from '@nestjs/common';
|
||||||
import { validate } from 'class-validator';
|
import { validate } from 'class-validator';
|
||||||
@@ -13,7 +14,7 @@ describe('TagsService', () => {
|
|||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
const moduleRef = await Test.createTestingModule({
|
const moduleRef = await Test.createTestingModule({
|
||||||
providers: [TagsService, PrismaService],
|
providers: [TagsService, PrismaService, PublicCacheService],
|
||||||
}).compile();
|
}).compile();
|
||||||
service = moduleRef.get(TagsService);
|
service = moduleRef.get(TagsService);
|
||||||
prisma = moduleRef.get(PrismaService);
|
prisma = moduleRef.get(PrismaService);
|
||||||
|
|||||||
@@ -5,13 +5,17 @@ import {
|
|||||||
NotFoundException,
|
NotFoundException,
|
||||||
} from '@nestjs/common';
|
} from '@nestjs/common';
|
||||||
import { PrismaService } from '../prisma/prisma.service';
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { PublicCacheService } from '../public/public-cache.service';
|
||||||
import { CreateTagDto } from './dto/create-tag.dto';
|
import { CreateTagDto } from './dto/create-tag.dto';
|
||||||
import { UpdateTagDto } from './dto/update-tag.dto';
|
import { UpdateTagDto } from './dto/update-tag.dto';
|
||||||
import { ReorderTagsDto } from './dto/reorder-tags.dto';
|
import { ReorderTagsDto } from './dto/reorder-tags.dto';
|
||||||
|
|
||||||
@Injectable()
|
@Injectable()
|
||||||
export class TagsService {
|
export class TagsService {
|
||||||
constructor(private readonly prisma: PrismaService) {}
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly publicCache: PublicCacheService,
|
||||||
|
) {}
|
||||||
|
|
||||||
findAll() {
|
findAll() {
|
||||||
return this.prisma.tag.findMany({
|
return this.prisma.tag.findMany({
|
||||||
@@ -31,7 +35,7 @@ export class TagsService {
|
|||||||
|
|
||||||
async create(dto: CreateTagDto) {
|
async create(dto: CreateTagDto) {
|
||||||
try {
|
try {
|
||||||
return await this.prisma.tag.create({
|
const created = await this.prisma.tag.create({
|
||||||
data: {
|
data: {
|
||||||
tagName: dto.tagName,
|
tagName: dto.tagName,
|
||||||
tagColor: dto.tagColor ?? null,
|
tagColor: dto.tagColor ?? null,
|
||||||
@@ -41,6 +45,8 @@ export class TagsService {
|
|||||||
sortOrder: dto.sortOrder ?? 0,
|
sortOrder: dto.sortOrder ?? 0,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return created;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (
|
if (
|
||||||
err instanceof Prisma.PrismaClientKnownRequestError &&
|
err instanceof Prisma.PrismaClientKnownRequestError &&
|
||||||
@@ -66,7 +72,9 @@ export class TagsService {
|
|||||||
}
|
}
|
||||||
if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder;
|
if (dto.sortOrder !== undefined) data.sortOrder = dto.sortOrder;
|
||||||
try {
|
try {
|
||||||
return await this.prisma.tag.update({ where: { id }, data });
|
const updated = await this.prisma.tag.update({ where: { id }, data });
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return updated;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (
|
if (
|
||||||
err instanceof Prisma.PrismaClientKnownRequestError &&
|
err instanceof Prisma.PrismaClientKnownRequestError &&
|
||||||
@@ -80,11 +88,13 @@ export class TagsService {
|
|||||||
|
|
||||||
async remove(id: bigint) {
|
async remove(id: bigint) {
|
||||||
await this.findOne(id);
|
await this.findOne(id);
|
||||||
return this.prisma.tag.delete({ where: { id } });
|
const removed = await this.prisma.tag.delete({ where: { id } });
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return removed;
|
||||||
}
|
}
|
||||||
|
|
||||||
async reorder(dto: ReorderTagsDto) {
|
async reorder(dto: ReorderTagsDto) {
|
||||||
return this.prisma.$transaction(
|
const result = await this.prisma.$transaction(
|
||||||
dto.items.map((item) =>
|
dto.items.map((item) =>
|
||||||
this.prisma.tag.update({
|
this.prisma.tag.update({
|
||||||
where: { id: BigInt(item.id) },
|
where: { id: BigInt(item.id) },
|
||||||
@@ -100,5 +110,7 @@ export class TagsService {
|
|||||||
}),
|
}),
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
|
this.publicCache.bump('meta');
|
||||||
|
return result;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,6 +17,9 @@ COPY apps/admin apps/admin
|
|||||||
RUN pnpm --filter @inkreach/admin exec vite build
|
RUN pnpm --filter @inkreach/admin exec vite build
|
||||||
|
|
||||||
FROM nginx:1.27-alpine
|
FROM nginx:1.27-alpine
|
||||||
|
# 性能整改 P0-2:替换官方主配置(worker_connections 1024 → 16384、gzip、访问日志);
|
||||||
|
# conf.d(admin.conf / admin.v2.conf)仍按服务挂载/拷贝
|
||||||
|
COPY deploy/nginx/nginx.conf /etc/nginx/nginx.conf
|
||||||
COPY --from=build /app/apps/admin/dist /usr/share/nginx/html/v2/admin
|
COPY --from=build /app/apps/admin/dist /usr/share/nginx/html/v2/admin
|
||||||
COPY deploy/nginx/admin.conf /etc/nginx/conf.d/default.conf
|
COPY deploy/nginx/admin.conf /etc/nginx/conf.d/default.conf
|
||||||
COPY deploy/h5 /usr/share/nginx/html/v2/h5
|
COPY deploy/h5 /usr/share/nginx/html/v2/h5
|
||||||
|
|||||||
@@ -9,6 +9,20 @@ services:
|
|||||||
v2-postgres:
|
v2-postgres:
|
||||||
image: postgres:16-alpine
|
image: postgres:16-alpine
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
# 性能整改 P0-3(plans/refactor/public-capacity-10k-refactor.md):
|
||||||
|
# - statement_timeout=10s:慢查询不无限期占住连接(池排队由此可控);
|
||||||
|
# - max_connections=200:为未来 api 副本/同步 worker 预留(单 api 池 50);
|
||||||
|
# - shared_buffers/effective_cache_size:14GB 内存机的工作集调优(数据 ~几十 MB,512MB 充裕)。
|
||||||
|
# 部署注意(全局约束 §1):改参数需重启容器(秒级中断)——先 pg_dump -Fc + 配置快照
|
||||||
|
# 落 deploy/backups/<时间戳>/ 再低峰 force-recreate。
|
||||||
|
command:
|
||||||
|
[
|
||||||
|
"postgres",
|
||||||
|
"-c", "statement_timeout=10000",
|
||||||
|
"-c", "max_connections=200",
|
||||||
|
"-c", "shared_buffers=512MB",
|
||||||
|
"-c", "effective_cache_size=1536MB",
|
||||||
|
]
|
||||||
environment:
|
environment:
|
||||||
POSTGRES_USER: inkreach
|
POSTGRES_USER: inkreach
|
||||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||||
@@ -32,7 +46,10 @@ services:
|
|||||||
environment:
|
environment:
|
||||||
NODE_ENV: production
|
NODE_ENV: production
|
||||||
PORT: 3001
|
PORT: 3001
|
||||||
DATABASE_URL: postgresql://inkreach:${POSTGRES_PASSWORD}@v2-postgres:5432/inkreach
|
# 性能整改 P0-3:池上限 50 < PG max_connections 200(同步/运维连接留余量);
|
||||||
|
# pool_timeout 单位是秒(Prisma 默认 10)——池耗尽 3s 快速失败而非挂 10s。
|
||||||
|
# 不要加 statement_cache_size=0(pgbouncer 专用,直连禁 prepared statement 反而拖慢)。
|
||||||
|
DATABASE_URL: postgresql://inkreach:${POSTGRES_PASSWORD}@v2-postgres:5432/inkreach?connection_limit=50&pool_timeout=3
|
||||||
JWT_SECRET: ${JWT_SECRET}
|
JWT_SECRET: ${JWT_SECRET}
|
||||||
CORS_ORIGINS: "*"
|
CORS_ORIGINS: "*"
|
||||||
# 激活 product-family 聚合公开读路径
|
# 激活 product-family 聚合公开读路径
|
||||||
@@ -65,6 +82,11 @@ services:
|
|||||||
ports:
|
ports:
|
||||||
- "80:80"
|
- "80:80"
|
||||||
- "443:443"
|
- "443:443"
|
||||||
|
# 性能整改 P0-2:4 worker × 16384 连接 + 上游 keepalive 需要容器 fd 预算匹配
|
||||||
|
ulimits:
|
||||||
|
nofile:
|
||||||
|
soft: 65536
|
||||||
|
hard: 65536
|
||||||
volumes:
|
volumes:
|
||||||
- ./nginx/admin.conf:/etc/nginx/conf.d/default.conf:ro
|
- ./nginx/admin.conf:/etc/nginx/conf.d/default.conf:ro
|
||||||
- ./certbot/www:/var/www/certbot:ro
|
- ./certbot/www:/var/www/certbot:ro
|
||||||
|
|||||||
+38
-6
@@ -1,3 +1,15 @@
|
|||||||
|
# 上游长连接池(性能整改 P0-2):与 v2-api 复用连接,消除每请求 TCP 建连;
|
||||||
|
# keepalive 连接需配合 proxy_http_version 1.1 + proxy_set_header Connection ""
|
||||||
|
upstream v2_api {
|
||||||
|
server v2-api:3001;
|
||||||
|
keepalive 64;
|
||||||
|
}
|
||||||
|
|
||||||
|
# 防洪峰兜底限流(app 层 120 req/min/IP 更严,这里只挡瞬时洪峰/扫描;
|
||||||
|
# 默认 503 会误判服务故障,显式 429)
|
||||||
|
limit_req_zone $binary_remote_addr zone=public_api:10m rate=50r/s;
|
||||||
|
limit_req_status 429;
|
||||||
|
|
||||||
server {
|
server {
|
||||||
listen 80;
|
listen 80;
|
||||||
server_name official.inkreach.cc;
|
server_name official.inkreach.cc;
|
||||||
@@ -27,6 +39,7 @@ server {
|
|||||||
|
|
||||||
root /usr/share/nginx/html;
|
root /usr/share/nginx/html;
|
||||||
|
|
||||||
|
# ACME challenge for cert renewals
|
||||||
location /.well-known/acme-challenge/ {
|
location /.well-known/acme-challenge/ {
|
||||||
root /var/www/certbot;
|
root /var/www/certbot;
|
||||||
}
|
}
|
||||||
@@ -60,22 +73,32 @@ server {
|
|||||||
}
|
}
|
||||||
|
|
||||||
location /v2-api/ {
|
location /v2-api/ {
|
||||||
proxy_pass http://v2-api:3001/;
|
proxy_pass http://v2_api/;
|
||||||
proxy_http_version 1.1;
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Connection "";
|
||||||
proxy_set_header Host $host;
|
proxy_set_header Host $host;
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
proxy_connect_timeout 5s;
|
||||||
|
proxy_read_timeout 30s;
|
||||||
|
proxy_send_timeout 30s;
|
||||||
|
limit_req zone=public_api burst=200 nodelay;
|
||||||
}
|
}
|
||||||
|
|
||||||
# H5 历史构建的 API 基址是同源相对路径 /public/*(BASE_URL="/"),直通 v2 api(路径原样透传)
|
# H5 历史构建的 API 基址是同源相对路径 /public/*(BASE_URL="/"),直通 v2 api(路径原样透传)
|
||||||
location /public/ {
|
location /public/ {
|
||||||
proxy_pass http://v2-api:3001;
|
proxy_pass http://v2_api;
|
||||||
proxy_http_version 1.1;
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Connection "";
|
||||||
proxy_set_header Host $host;
|
proxy_set_header Host $host;
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
proxy_connect_timeout 5s;
|
||||||
|
proxy_read_timeout 30s;
|
||||||
|
proxy_send_timeout 30s;
|
||||||
|
limit_req zone=public_api burst=200 nodelay;
|
||||||
}
|
}
|
||||||
|
|
||||||
location = /h5 {
|
location = /h5 {
|
||||||
@@ -86,16 +109,25 @@ server {
|
|||||||
try_files $uri $uri/ /h5/index.html;
|
try_files $uri $uri/ /h5/index.html;
|
||||||
}
|
}
|
||||||
|
|
||||||
# Uploaded files served by the API
|
# Uploaded files served by the API.
|
||||||
|
# 缓存头注意:若运营有「同名替换图片」操作请改短 max-age 或改用版本化 URL
|
||||||
location /uploads/ {
|
location /uploads/ {
|
||||||
proxy_pass http://v2-api:3001/uploads/;
|
proxy_pass http://v2_api/uploads/;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Connection "";
|
||||||
proxy_set_header Host $host;
|
proxy_set_header Host $host;
|
||||||
|
expires 30d;
|
||||||
|
add_header Cache-Control "public";
|
||||||
}
|
}
|
||||||
|
|
||||||
# Static product assets served by the API
|
# Static product assets served by the API(哈希文件名,可长缓存)
|
||||||
location /assets/ {
|
location /assets/ {
|
||||||
proxy_pass http://v2-api:3001/assets/;
|
proxy_pass http://v2_api/assets/;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Connection "";
|
||||||
proxy_set_header Host $host;
|
proxy_set_header Host $host;
|
||||||
|
expires 30d;
|
||||||
|
add_header Cache-Control "public";
|
||||||
}
|
}
|
||||||
|
|
||||||
# Website placeholder until the public site is deployed
|
# Website placeholder until the public site is deployed
|
||||||
|
|||||||
@@ -25,8 +25,14 @@ server {
|
|||||||
add_header Cache-Control "no-cache";
|
add_header Cache-Control "no-cache";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# v2 版 H5(构建时 router.base 必须为 /v2/h5/,产物放 /usr/share/nginx/html/v2/h5)
|
||||||
|
# 性能整改 P0-2:哈希文件名的构建产物长缓存,index.html 永远 no-cache(上面精确匹配优先)
|
||||||
location /v2/h5/ {
|
location /v2/h5/ {
|
||||||
try_files $uri $uri/ /v2/h5/index.html;
|
try_files $uri $uri/ /v2/h5/index.html;
|
||||||
|
location ~* ^/v2/h5/(static|assets|img|fonts?)/ {
|
||||||
|
expires 30d;
|
||||||
|
add_header Cache-Control "public";
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
location / {
|
location / {
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
# 边缘 nginx 主配置(性能整改 P0-2,plans/refactor/public-capacity-10k-refactor.md)
|
||||||
|
# 由 deploy/admin.Dockerfile COPY 进镜像替换官方默认——官方默认 worker_connections 1024
|
||||||
|
# × 4 worker = 4096 连接硬顶,扛不住 1w 并发口径;events{} 只存在于主配置,conf.d 覆盖不了。
|
||||||
|
# 注意:admin 与 v2-admin 共用本 Dockerfile,两侧都会得到本主配置(upstream/gzip 对
|
||||||
|
# v2-admin 无害;worker_connections 抬升对静态服务同样受益)。
|
||||||
|
user nginx;
|
||||||
|
worker_processes auto;
|
||||||
|
# 每 worker 连接预算上限(4 × 16384 客户端连接 + 上游 keepalive 连接)
|
||||||
|
worker_rlimit_nofile 65536;
|
||||||
|
|
||||||
|
error_log /var/log/nginx/error.log warn;
|
||||||
|
pid /var/run/nginx.pid;
|
||||||
|
|
||||||
|
events {
|
||||||
|
worker_connections 16384;
|
||||||
|
multi_accept on;
|
||||||
|
}
|
||||||
|
|
||||||
|
http {
|
||||||
|
include /etc/nginx/mime.types;
|
||||||
|
default_type application/octet-stream;
|
||||||
|
|
||||||
|
# 1w 并发下无访问日志无从排障:带上游耗时的结构化日志(rt=总耗时 urt=上游耗时)
|
||||||
|
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
|
||||||
|
'$status $body_bytes_sent "$http_referer" "$http_user_agent" '
|
||||||
|
'rt=$request_time uct=$upstream_connect_time urt=$upstream_response_time';
|
||||||
|
access_log /var/log/nginx/access.log main;
|
||||||
|
|
||||||
|
sendfile on;
|
||||||
|
tcp_nopush on;
|
||||||
|
tcp_nodelay on;
|
||||||
|
keepalive_timeout 65;
|
||||||
|
types_hash_max_size 2048;
|
||||||
|
|
||||||
|
# 压缩:API JSON(详情含 ~11KB price_matrix)与 SPA 静态资源,带宽降 5~10x
|
||||||
|
gzip on;
|
||||||
|
gzip_vary on;
|
||||||
|
gzip_proxied any;
|
||||||
|
gzip_comp_level 5;
|
||||||
|
gzip_min_length 1024;
|
||||||
|
gzip_types application/json application/javascript text/css text/plain text/xml image/svg+xml;
|
||||||
|
|
||||||
|
include /etc/nginx/conf.d/*.conf;
|
||||||
|
}
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
# public 端口性能审查报告(1w 瞬时并发请求口径)
|
||||||
|
|
||||||
|
- 审查日期:2026-09-03
|
||||||
|
- 审查对象:https://official.inkreach.cc 的公开访问链路(public 端口 = 边缘 nginx 80/443 → v2-api:3001 → v2-postgres)
|
||||||
|
- 审查口径:**1w 瞬时并发请求**——同一时刻 1 万个请求同时到达边缘(压力测试式,万 QPS 级)
|
||||||
|
- 结论:**当前形态不能承压,差距约 2~3 个数量级**。瓶颈全部位于「配置与架构层」,与商品数据规模无关(goods 仅 240 行),改造后单机有希望在口径内达标,需压测实证。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 现状架构
|
||||||
|
|
||||||
|
单机(4 核 / 14GB / 无资源限额)docker compose 单栈 4 容器,全部单副本:
|
||||||
|
|
||||||
|
```
|
||||||
|
公网:80/443 (official.inkreach.cc)
|
||||||
|
└─ deploy-admin-1 (边缘 nginx 1.27-alpine,官方默认主配置)
|
||||||
|
├─ /public/ /v2-api/ /uploads/ /assets/ ──► deploy-v2-api-1 (NestJS :3001)
|
||||||
|
├─ /v2/admin/ /v2/h5/ ──► deploy-v2-admin-1 (SPA 静态)
|
||||||
|
└─ /admin/ /h5/ ──► try_files 本地静态
|
||||||
|
deploy-v2-api-1 ──► deploy-v2-postgres-1 (postgres:16-alpine)
|
||||||
|
```
|
||||||
|
|
||||||
|
关键事实(均已核对源码/配置原文):
|
||||||
|
|
||||||
|
| 项 | 现状 | 出处 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| nginx worker | `worker_processes auto`(4 worker)× `worker_connections 1024` → **最多 4096 条客户端连接** | 容器内官方默认 nginx.conf(`deploy/nginx/admin.conf` 仅覆盖 conf.d) |
|
||||||
|
| 边缘路由 | `/public/` 路径原样透传至 v2-api:3001 | `deploy/nginx/admin.conf` L72-79 |
|
||||||
|
| 压缩 | 无 gzip(JSON/JS/图片全裸传) | nginx 默认 gzip 注释关闭;main.ts 无 compression 中间件 |
|
||||||
|
| 上游 keepalive | 无(每请求新建到 v2-api 的 TCP 连接) | admin.conf 无 upstream keepalive |
|
||||||
|
| 限流 | 全局限流 120 req/min/IP | `apps/api/src/app.module.ts` L27-32 |
|
||||||
|
| API 连接池 | Prisma 零配置 → 默认池 = 4核×2+1 = **9 条连接**,排队默认 10s 后报错 | `apps/api/src/prisma/prisma.service.ts` L12;DATABASE_URL 无参数(compose L35) |
|
||||||
|
| PG | `max_connections=100`(默认)、无 statement_timeout、shared_buffers 默认 | docker-compose.yml 未覆盖;postgres:16 默认 |
|
||||||
|
| 缓存 | 全链路零缓存(无 Redis/cache-manager/nginx proxy_cache) | 全仓 grep 零命中 |
|
||||||
|
| 同步任务 | 整点分类同步(长事务串行 ~226 分类)、03:30 详情全量同步 + 逐单族重算,与 public **同进程同池** | `apps/api/src/sync/sync.service.ts` L106-114、L138-145 |
|
||||||
|
| 数据规模 | goods 240 / origin_goods 542 / variants 12,155 / categories 243 / countries 12 | 生产导出 data-dump.json(2026-08-22) |
|
||||||
|
|
||||||
|
## 2. 逐层瓶颈证据
|
||||||
|
|
||||||
|
### 2.1 边缘 nginx:第 0 层就爆(连接数硬顶 4096)
|
||||||
|
|
||||||
|
`worker_connections 1024` × 4 worker = **4096 条并发客户端连接**。HTTP/2 下每个浏览器占 1 条连接;1 万并发请求到场时,约 **六成连接进不了 accept 队列**(内核 backlog 默认 511,其余直接 reset/拒绝)。这是与后端优化无关的硬天花板。
|
||||||
|
|
||||||
|
### 2.2 API 层:单进程 + 9 连接池,直连 DB 吞吐 ~20~50 req/s
|
||||||
|
|
||||||
|
- **`GET /public/goods` 无 SQL 分页**:`findMany` 不带 take/skip 全量拉取匹配行(`public.service.ts` L234),再内存分组、内存分页(L249-280)。数据量百级时可行,但每次请求的成本是「全表扫描 + 多表联表 + JS 排序」。
|
||||||
|
- **每次请求两个全表聚合**:
|
||||||
|
- `loadFamilyMinPrices()`(L334-351):`CROSS JOIN LATERAL jsonb_array_elements(price_matrix->'rows')` 展开**全部**族(每族 ~11KB JSONB),`/public/goods` 每页与 `/public/home-goods` 每次都跑;注释实测同类全量扫描 ~330ms(L329-333)。
|
||||||
|
- DEFAULT 排序额外 `loadTreeOrderMeta()`(L287-307):全 countries + 全 categories 联表 raw SQL。
|
||||||
|
- **N+1 与串行链路**:
|
||||||
|
- 分类树 `getCategoriesTree`:逐祖先串行 `findUnique`(L102-113)。
|
||||||
|
- 详情 `getGoodByFamilyId`:3 条查询并发 + 响应内嵌整份 ~11KB price_matrix、全部变体 designData;`resolveCategoryIcon` 串行向上最多 10 次 findUnique(L552-565)。
|
||||||
|
- **`keyword` 走 `ILIKE '%kw%'`**(L199),无 pg_trgm 索引 → 必 seq scan(当前 240 行无感,量级放大后失效)。
|
||||||
|
- 单请求 DB 成本合计 ~300~500ms(列表)与 ~100~300ms(详情),9 条连接串行化后:**列表接口吞吐 ≈ 9 ÷ 0.4s ≈ 20~25 req/s**,全端点混合 ~30~60 req/s。
|
||||||
|
|
||||||
|
### 2.3 数据库层:无防护,池耗尽即排队超时
|
||||||
|
|
||||||
|
- Prisma 池 9 < PG max_connections 100,但**无任何超时参数**:慢查询可无限期占住连接;同步长事务(分类同步单事务串行 ~226 分类、详情同步逐商品 upsert)与 public 抢同一池。
|
||||||
|
- 池排队默认 10s 超时(P2028):1 万瞬时请求直连 DB 时,绝大多数请求排队 >10s 直接报错。
|
||||||
|
- 无 statement_timeout,无连接池水位监控。
|
||||||
|
|
||||||
|
### 2.4 传输层:无 gzip、无缓存头
|
||||||
|
|
||||||
|
- 详情响应内嵌 ~11KB price_matrix + 全部变体 designData;列表响应平均 ~2-5KB。按 ~20KB/响应粗算,1 万 req/s ≈ **1.6Gbps**,可打满常见 1Gbps 出口带宽;gzip 后 JSON 可压 5~10 倍。
|
||||||
|
- `/uploads/` `/assets/` 静态资源无任何 Cache-Control(`main.ts` L73-81),每次访问重复下载。
|
||||||
|
|
||||||
|
### 2.5 全链路零缓存(杠杆最高的修复点)
|
||||||
|
|
||||||
|
分类树、国家、标签组、首页商品、商品列表、详情全部每次实时查库。**数据变更频率为小时级同步**(syncCategories/syncProducts 整点跑),这意味着对 public 读路径引入分钟级 TTL 缓存几乎无损数据新鲜度,却能把 DB 热路径负载降 95%+。
|
||||||
|
|
||||||
|
### 2.6 同步任务竞争
|
||||||
|
|
||||||
|
整点分类同步长事务、03:30 详情全量同步 + 逐单族重算(fire-and-forget 进程内任务)与 public 共享同一 9 连接池。同步窗口内池被长事务/串行往返占满时,在线请求直接排队甚至超时。
|
||||||
|
|
||||||
|
### 2.7 压测陷阱:Throttler 429
|
||||||
|
|
||||||
|
全局限流 120 req/min/IP(`app.module.ts` L27-32)。真实 1 万用户分布在不同 IP 无影响;但**单机/少 IP 压测流量会被 429 打满**,压测必须使用分布式源(≥数十个出口 IP)或临时抬高 `THROTTLE_LIMIT`,否则测出来的是限流器而不是系统容量。
|
||||||
|
|
||||||
|
## 3. 容量估算与结论
|
||||||
|
|
||||||
|
| 层 | 上限 | 1w 瞬时并发需求 | 差距 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| nginx 连接数 | 4096 | 10000 | ~2.4x |
|
||||||
|
| API 直连 DB 吞吐 | ~30-60 req/s | ~10000 req/s | ~200-300x |
|
||||||
|
| 出口带宽(无 gzip) | ~1 Gbps | ~1.6 Gbps | ~1.6x |
|
||||||
|
| PG 连接 | 100 | 9 池上限前置 | — |
|
||||||
|
|
||||||
|
**结论:当前形态不能承压 1w 瞬时并发请求**。即便并发被 nginx 放进来,DB 直连吞吐差两个数量级以上,池排队 10s 超时会把瞬时洪峰转为大面积 5xx。
|
||||||
|
|
||||||
|
**方向**:瓶颈 100% 属于「配置与架构层」——进程内存缓存(TTL 5~10min + 同步后失效)可把 DB 热路径负载降 ~95-99%;nginx 连接数/gzip/keepalive/限流补齐后边缘可达 1w+ 连接;池参数与 statement_timeout 防排队超时与慢查询占池。改造后单机 4 核全缓存热路径(万级 req/s 简单 JSON 响应)有达标可能,但必须用分布式压测实证,达标口径见下。
|
||||||
|
|
||||||
|
## 4. 整改方向(摘要)
|
||||||
|
|
||||||
|
> **实施进度(2026-09-03)**:P0-1/P0-2/P0-3 已在 `refactor/public-capacity-10k-yeuimu` 分支完成并全量测试通过(238/238)——进程内分域缓存 + 全写路径即时失效、nginx 主配置/keepalive/gzip/限流/缓存头、连接池与 PG 参数。**已于 2026-09-03 08:38 部署到生产**(备份存档 `deploy/backups/20260903-083800-p0-perf-deploy/`,切换停机 23s):部署后实测——home-goods 首次 90ms/缓存命中 12ms;详情 247KB → gzip 23KB(10.7x);静态资源 30d 缓存头生效;nginx 洪峰限流 429 生效且随窗口恢复(注意:单 IP 高频实测同时会打满 app 层 120 req/min 限流,属双层按设计工作);PG statement_timeout=10s/max_connections=200/shared_buffers=512MB 生效;行数 diff 零变化。**P1-3 压测基线(镜像栈 + 分布式源)为验收前置,未达标按计划扩展阶梯升级。**
|
||||||
|
|
||||||
|
完整任务分解见 [plans/refactor/public-capacity-10k-refactor.md](../plans/refactor/public-capacity-10k-refactor.md):
|
||||||
|
|
||||||
|
- **P0-1** 进程内存缓存层(NestJS cache-manager 或等价实现):categories/countries/tags/tag-groups/home-goods/goods(全量物化+内存分页)/goods/:id/族最低价聚合/树序元数据;**所有写路径(sync/族重算/admin 写/上传)提交后按域版本号主动失效**(全局约束:等待 TTL 过期不是兜底)
|
||||||
|
- **P0-2** nginx 调优:`worker_connections 16384`、`worker_rlimit_nofile`、upstream `keepalive 64`、gzip on、/uploads /assets 缓存头、limit_req 兜底、proxy 超时显式化
|
||||||
|
- **P0-3** 连接与超时:`DATABASE_URL?connection_limit=50&pool_timeout=3`(pool_timeout 单位为秒)、PG `statement_timeout=10s`、shared_buffers/effective_cache_size 调参
|
||||||
|
- **P1-1** `/public/goods` SQL 分页(take/skip + count)
|
||||||
|
- **P1-2** 同步任务与 public 隔离(错峰/批量化/独立 worker 开关)
|
||||||
|
- **P1-3** 压测基线:一次性 docker postgres + 生产 dump 测关键 SQL 真实耗时;k6 分布式压测
|
||||||
|
- **P2** 静态资源边缘直出、pg_trgm 索引、连接池水位监控告警
|
||||||
|
|
||||||
|
## 5. 验收口径(改造后)
|
||||||
|
|
||||||
|
- 分布式压测(≥数十个出口 IP,或 `THROTTLE_LIMIT` 临时抬高):**1 万并发瞬时请求**下:
|
||||||
|
- p95 延迟 < 300ms(CPU 侧),p99 < 1s
|
||||||
|
- 错误率 < 0.1%,无 nginx 连接拒绝/重置
|
||||||
|
- PG 连接水位不触顶(< 80%),无池排队超时
|
||||||
|
- 公布达标数字(req/s、p95/p99、错误率)后视为验收通过;不达标则加 v2-api 副本(上游 keepalive 已就位,扩容仅需加容器)。
|
||||||
|
|
||||||
|
## 附:相关文件索引
|
||||||
|
|
||||||
|
- 边缘路由:`deploy/nginx/admin.conf`
|
||||||
|
- 编排:`deploy/docker-compose.yml`
|
||||||
|
- 公开 API 实现:`apps/api/src/public/public.controller.ts`、`apps/api/src/public/public.service.ts`
|
||||||
|
- 入口中间件:`apps/api/src/main.ts`;限流:`apps/api/src/app.module.ts`
|
||||||
|
- 连接池:`apps/api/src/prisma/prisma.service.ts`
|
||||||
|
- 同步任务:`apps/api/src/sync/sync.service.ts`、`apps/api/src/product-families/family-recompute.service.ts`
|
||||||
|
- 数据规模:`deploy/data-dump.json`(2026-08-22 导出)
|
||||||
@@ -47,6 +47,13 @@ inkreach-official/
|
|||||||
`v2-postgres`(数据卷 `deploy-v2_pgdata` external 引用,v2 数据未迁移)。
|
`v2-postgres`(数据卷 `deploy-v2_pgdata` external 引用,v2 数据未迁移)。
|
||||||
- 域名路由:`/v2/admin/`(SPA)、`/v2-api/`(小程序/后台 API,去前缀转发)、`/public/`
|
- 域名路由:`/v2/admin/`(SPA)、`/v2-api/`(小程序/后台 API,去前缀转发)、`/public/`
|
||||||
(H5 同源 API)、`/uploads/`、`/assets/` 均由 v2-api 服务;`/` 302 到 `/v2/admin/`。
|
(H5 同源 API)、`/uploads/`、`/assets/` 均由 v2-api 服务;`/` 302 到 `/v2/admin/`。
|
||||||
|
- 性能整改 P0(2026-09-03,plans/refactor/public-capacity-10k-refactor.md):边缘 nginx
|
||||||
|
主配置改为仓库文件 `deploy/nginx/nginx.conf`(worker_connections 16384 + gzip + 访问日志,
|
||||||
|
经 admin.Dockerfile COPY,改后须重建镜像);admin.conf 含 upstream keepalive/限流(429)/
|
||||||
|
代理超时/静态缓存头;v2-postgres 显式参数(statement_timeout=10s、max_connections=200、
|
||||||
|
shared_buffers=512MB);v2-api 的 DATABASE_URL 带 `connection_limit=50&pool_timeout=3`
|
||||||
|
(pool_timeout 单位为秒);admin 服务 ulimits nofile 65536。改 PG 参数/重启前按全局约束
|
||||||
|
先 pg_dump + 配置快照落 `deploy/backups/<时间戳>/`。
|
||||||
- v1(旧 api+postgres,曾支撑 `/public/` 与旧后台)已退役:切换时容器停用未删,
|
- v1(旧 api+postgres,曾支撑 `/public/` 与旧后台)已退役:切换时容器停用未删,
|
||||||
库终档与全部配置快照及回滚手册见 `deploy/backups/consolidation-*/RESTORE.md`。
|
库终档与全部配置快照及回滚手册见 `deploy/backups/consolidation-*/RESTORE.md`。
|
||||||
- 原 v2 独立栈(`docker-compose.v2.yml`、`/opt/inkreach-v2` 检出)已并入单栈并删除,
|
- 原 v2 独立栈(`docker-compose.v2.yml`、`/opt/inkreach-v2` 检出)已并入单栈并删除,
|
||||||
@@ -76,7 +83,7 @@ apps/api/
|
|||||||
│ ├── product-families/ # 产品族(SPU 层):CRUD / auto-group(并入已有族优先,familyNameKey 族语义键)/ attachToMatchingFamily(单链接自动归族)/ consolidateFragments(碎片族合并)/ 成员管理 / 自定义成员 / 价格覆盖 / 重算 / 按链接名称派生标签(auto-tag-rules,含 热转印→烫画 别名)。数据约定(2026-09-03 起):光板/不打印链接不入族、矩阵不计算不打印工艺;auto-group/organize/人工改标签会把散链接回挂,运行前须排查(备份见 deploy/backups/20260903-noprint-removal/)
|
│ ├── product-families/ # 产品族(SPU 层):CRUD / auto-group(并入已有族优先,familyNameKey 族语义键)/ attachToMatchingFamily(单链接自动归族)/ consolidateFragments(碎片族合并)/ 成员管理 / 自定义成员 / 价格覆盖 / 重算 / 按链接名称派生标签(auto-tag-rules,含 热转印→烫画 别名)。数据约定(2026-09-03 起):光板/不打印链接不入族、矩阵不计算不打印工艺;auto-group/organize/人工改标签会把散链接回挂,运行前须排查(备份见 deploy/backups/20260903-noprint-removal/)
|
||||||
│ ├── goods/ # 商品 CRUD + 批量优先级 + 批量创建 + 展示名规范化(品名 SKU,剥 ASCII 型号限定词,纯款号按 SDS 分类名补描述)
|
│ ├── goods/ # 商品 CRUD + 批量优先级 + 批量创建 + 展示名规范化(品名 SKU,剥 ASCII 型号限定词,纯款号按 SDS 分类名补描述)
|
||||||
│ ├── sync/ # SDS 同步:分类 / 商品 / 同步日志
|
│ ├── sync/ # SDS 同步:分类 / 商品 / 同步日志
|
||||||
│ ├── public/ # 公开 API:分类树 / 国家 / 商品分页 / 商品详情
|
│ ├── public/ # 公开 API:分类树 / 国家 / 商品分页 / 商品详情;public-cache.service.ts 为进程内分域缓存(meta/goods/matrix 版本域 + TTL 兜底 + in-flight 合并,P0-1 性能整改),全部写路径(admin CRUD/sync/族重算/整理)事务提交后 bump 对应域即时失效;env 开关 PUBLIC_CACHE_DISABLED / PUBLIC_CACHE_TTL_MS / PUBLIC_CACHE_MAX_ENTRIES
|
||||||
│ └── common/ # 全局装饰器 / 过滤器 / 拦截器
|
│ └── common/ # 全局装饰器 / 过滤器 / 拦截器
|
||||||
│ ├── decorators/current-user.decorator.ts
|
│ ├── decorators/current-user.decorator.ts
|
||||||
│ ├── filters/http-exception.filter.ts
|
│ ├── filters/http-exception.filter.ts
|
||||||
|
|||||||
@@ -0,0 +1,157 @@
|
|||||||
|
# 整改计划:public 端口 1w 瞬时并发承压(public-capacity-10k-refactor)
|
||||||
|
|
||||||
|
前置审查:[docs/references/performance-review-public-port.md](../../docs/references/performance-review-public-port.md)
|
||||||
|
|
||||||
|
## 目标
|
||||||
|
|
||||||
|
- 目标口径:**1w 瞬时并发请求**(同一时刻 1 万请求到达边缘,万 QPS 级)。
|
||||||
|
- 验收:p95 < 300ms、p99 < 1s、错误率 < 0.1%、无 nginx 连接拒绝/重置、PG 连接水位 < 80%、无池排队超时(详见报告 §5)。
|
||||||
|
- 原则:本次改造只动「配置与架构层」,不改对外 API 契约(响应结构不变),数据语义零变化。
|
||||||
|
|
||||||
|
## 全局约束(执行本计划所有任务强制遵守)
|
||||||
|
|
||||||
|
1. **数据库变更先行备份存档**:凡是涉及数据库更改的行动(schema 迁移、数据回填、族重算、同步行为变更、PG 参数调整与重启),执行前必须先完成风险回滚存档:
|
||||||
|
- 受影响库 `pg_dump -Fc` 全量 dump + 相关配置快照,落 `deploy/backups/<时间戳>/` 并写 `RESTORE.md`(沿用 AGENTS.md「生产栈收敛三件套」);
|
||||||
|
- 变更完成并验证通过前,备份不得清理;验收通过后按 RESTORE.md 归档流程归档。
|
||||||
|
- 本计划内的触发点:P0-3(PG 参数 + 重启)、P1-1(物化列迁移)、P1-2(重算/同步行为变更)、P2-1(索引/扩展)。
|
||||||
|
2. **写路径必失效缓存**:凡引入缓存(P0-1),所有数据写路径(admin 侧 CRUD、sync 同步、族重算、上传替换)在写事务提交成功后必须同步失效对应缓存域;「等待 TTL 自然过期」不作为可接受的兜底;每条写路径都必须有「写后缓存已失效」的测试覆盖。
|
||||||
|
|
||||||
|
## 执行顺序与依赖
|
||||||
|
|
||||||
|
```
|
||||||
|
P0-3(池参数,防排队超时)→ P0-1(缓存,DB 负载降 95%+)→ P0-2(nginx,边缘承压)
|
||||||
|
→ P1-3(压测基线,验证 P0 是否达标)
|
||||||
|
→ P1-1(SQL 分页,缓存兜底后放宽量级)→ P1-2(同步隔离)→ P2(收尾项)
|
||||||
|
```
|
||||||
|
|
||||||
|
P0 三项互相独立可并行;P1-3 必须在 P0 全部上线后进行,否则测的是改造前基线(可先测一次作对照)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P0-1 进程内存缓存层(DB 热路径负载降 95%+,杠杆最高)
|
||||||
|
|
||||||
|
**原则**:public 读路径数据变更频率 = 小时级同步 → 分钟级 TTL 缓存几乎无损新鲜度。缓存键必须包含影响结果的全部入参;失效锚点为同步任务(见 P0-1.3)。
|
||||||
|
|
||||||
|
### P0-1.1 低熵全量数据缓存(categories / countries / tags / tag-groups / 树序元数据 / 族最低价)
|
||||||
|
|
||||||
|
- 涉及:`apps/api/src/public/public.service.ts`、`apps/api/src/app.module.ts`(若用 `@nestjs/cache-manager`)
|
||||||
|
- 做法:引入进程内存缓存(`@nestjs/cache-manager` 或自研 Map+TTL 包装,单进程部署下内存缓存即全局缓存;不引入 Redis,避免新增运维依赖)。
|
||||||
|
- `loadTreeOrderMeta()`、`loadFamilyMinPrices()` 两个私有聚合结果整体缓存(TTL 10min)——它们被列表/首页每次请求复用,是最大的固定成本。
|
||||||
|
- `getCategoriesTree` / `getCountries` / `getTags` / `getTagGroups` 按 `countryId`(若有)作键缓存结果(TTL 10min)。
|
||||||
|
- 测试:TDD 先写——缓存命中返回等价结构、TTL 过期后重新查询、countryId 不同键不串数据。
|
||||||
|
- 回归风险:同步窗口内缓存有最多 10min 滞后 → P0-1.3 主动失效可缩短到秒级。
|
||||||
|
|
||||||
|
### P0-1.2 商品列表缓存:全量物化 + 内存分页(各 page/筛选组合共享一份缓存)
|
||||||
|
|
||||||
|
- 涉及:`apps/api/src/public/public.service.ts` 的 `getGoods`
|
||||||
|
- 做法:**不要**缓存「page 组合」——对筛选结果(countryId/keyword/category/tags/价格区间 + 排序)物化出 `{ rows: PublicGoodDto[](已排序已分组), total }` 整体缓存(TTL 10min,键 = 筛选参数序列化,不含 page/pageSize),`items.slice(start, start+pageSize)` 在缓存命中后执行。默认排序下全部 240 行物化成本一次,所有页共享。
|
||||||
|
- 注意:价格筛选/价格排序依赖族最低价 → 与 P0-1.1 的族最低价缓存同源;族最低价变化(族重算/同步)后必须连带失效列表缓存。
|
||||||
|
- 测试:TDD——同筛选不同 page 只触发一次底层查询(spy findMany 计数);筛选参数不同键不同;失效后重建。
|
||||||
|
- 回归风险:内存占用 = 缓存条目 × 240 DTO(每条 ~1KB)→ 筛选组合多时需设条目上限(LRU,如 64 条)与单条目 TTL。
|
||||||
|
|
||||||
|
### P0-1.3 详情缓存
|
||||||
|
|
||||||
|
- 涉及:`public.service.ts` 的 `getGoodByFamilyId`
|
||||||
|
- 做法:`goods/:id` 详情按 familyId 缓存(TTL 10min;响应含 11KB price_matrix,缓存收益尤其大)。
|
||||||
|
- 测试:命中返回等价结构、TTL 过期重建、失效后重建。
|
||||||
|
- 回归风险:多进程部署时内存缓存不相通 → 当前单进程无此问题;扩容多副本时需换 Redis 或接受短滞后(文档注明)。
|
||||||
|
|
||||||
|
### P0-1.4 缓存失效统一机制(全局约束 §2 的落地,必须先于/同步于 P0-1.1~1.3 交付)
|
||||||
|
|
||||||
|
- 涉及:新增 `apps/api/src/public/public-cache.service.ts`(或等价),`apps/api/src/sync/sync.service.ts`、`family-recompute.service.ts`、admin 侧全部写服务
|
||||||
|
- 做法:
|
||||||
|
1. **版本域设计**:每个缓存键带版本前缀,分域管理——`v:categories`(分类树/国家/标签组/树序元数据)、`v:goods`(列表/首页/详情)、`v:matrix`(族最低价);读路径取版本号拼键,写路径 `bump(域)` 使整个域失效。域粒度避免「任意写导致全量缓存清空」,实现简单且不漏键(不做逐个 `del` 键级清理,易漏易错)。
|
||||||
|
2. **统一入口**:`PublicCacheService` 只暴露 `get/bump/set` 三个方法;所有写路径收口到 `bump`,禁止在业务代码里散落 cache 键。
|
||||||
|
3. **写路径挂钩清单**(缓存上线时必须全部挂上,缺一不可):
|
||||||
|
- `sync.service.ts`:`syncCategories` 提交后 → `v:categories`;`syncProducts` 提交后 → `v:goods`;详情同步 `persistProductDetail` 提交后 → `v:goods`;
|
||||||
|
- `family-recompute.service.ts`:族重算 `persist` 提交后 → `v:matrix` + `v:goods`;
|
||||||
|
- admin 侧写服务:goods CRUD/排序/标签挂接、categories、tags/tag-groups、positions、product-families(override)→ 按影响域 `bump`;注意单纯「上传文件」若不改库行则无需 bump,挂钩点是**引用该图的写操作**(改 goods/detail/media 行 → `v:goods`)。
|
||||||
|
4. **写后即失效**(write-through 语义):`bump` 必须在写事务提交**成功之后**执行(事务回滚不得 bump);`bump` 本身用版本号递增(防止「失效瞬间的并发读又把旧值写回缓存」的竞态——读路径读版本号后写入,版本号已变则丢弃写入或重查)。
|
||||||
|
- 测试(每个写路径至少一条):
|
||||||
|
- 「写后缓存已失效」:写入数据 → 断言对应缓存域读不到旧值;
|
||||||
|
- 「TTL 未到期但写已发生 → 前台立即可见新数据」的端到端用例(同步/族重算/admin 修改各一);
|
||||||
|
- 「事务回滚不 bump」:构造回滚路径断言版本号未变;
|
||||||
|
- 并发竞态:并发读+写下不出现陈旧值(版本号检查路径)。
|
||||||
|
- 回归风险:挂接漏写路径 = 前台最长 10min 陈旧(比报错更隐蔽)→ 挂钩清单作为自查 checklist 纳入 PR 模板与代码评审。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P0-2 边缘 nginx 调优(连接数硬顶 4096 → 16384+,压缩与限流补齐)
|
||||||
|
|
||||||
|
- 涉及:`deploy/nginx/admin.conf`(bind mount,改后 `docker compose up -d --force-recreate admin`,文件型挂载 inode 不随 restart 生效)、**新增** `deploy/nginx/nginx.conf` + `deploy/admin.Dockerfile`(主配置 COPY 进镜像,需 `docker compose build` 重建——只 force-recreate 不生效)、`deploy/nginx/admin.v2.conf`(/v2/h5/ /v2/admin/ 静态缓存头落点)、`deploy/docker-compose.yml`(admin 服务 ulimits)
|
||||||
|
- 做法(`/etc/nginx/nginx.conf` 主配置无法覆盖时,在 conf.d 里用 `worker_rlimit_nofile` 不行——主配置的 workers 属 http 级不可段内覆盖——需在镜像层覆盖主配置或确认官方镜像默认即可):
|
||||||
|
1. **连接数**(已核实:`deploy/admin.Dockerfile` 最终镜像只 COPY 了 `conf.d/default.conf`,主 nginx.conf 为官方原版 `worker_connections 1024`;`events{}` 只存在于主配置,conf.d 无法覆盖):新增 `deploy/nginx/nginx.conf` 并在 Dockerfile 最终阶段加 `COPY deploy/nginx/nginx.conf /etc/nginx/nginx.conf`——内容 `worker_processes auto; worker_rlimit_nofile 65536; events { worker_connections 16384; multi_accept on; }`;同时在 compose `admin` 服务加 `ulimits: { nofile: { soft: 65536, hard: 65536 } }`(4 worker × 16384 连接 + 上游 keepalive 连接需落在容器 fd 预算内,Docker 默认值需实测确认,低于预算必须显式设置)。
|
||||||
|
2. **upstream keepalive**:`/v2-api/`、`/public/`、`/uploads/`、`/assets/` 四个 proxy location 统一加 `upstream v2_api { server v2-api:3001; keepalive 64; }` + `proxy_http_version 1.1`(已有)+ `proxy_set_header Connection ""`(keepalive 必需)。
|
||||||
|
3. **gzip on**:`gzip on; gzip_types application/json application/javascript text/css image/svg+xml; gzip_min_length 1k;`(JSON API + SPA 静态资源,带宽降 5~10x)。
|
||||||
|
4. **静态资源缓存头**(注意配置落点):`/uploads/`、`/assets/` 在边缘 admin.conf 加 `expires 30d; add_header Cache-Control "public, immutable";`;但 `/v2/h5/`、`/v2/admin/` 的静态文件实际由 **v2-admin 容器的 `deploy/nginx/admin.v2.conf`** 提供(边缘只做代理),哈希文件名的缓存头要加在那边,`index.html` 保持 `no-cache`——边缘不要对代理响应统一加 expires,避免覆盖源头的 no-cache 语义。
|
||||||
|
5. **limit_req 兜底**(定位:只挡瞬时洪峰/扫描,不做业务限流——app 层 120 req/min/IP(≈2 r/s 持续)比它严得多):`limit_req_zone $binary_remote_addr zone=public:10m rate=50r/s;` + `limit_req zone=public burst=200 nodelay;` + 显式 `limit_req_status 429;`(默认 503 会让前端误判服务故障),仅挂 `/public/`、`/v2-api/` 两个 location。
|
||||||
|
6. **proxy 超时显式化**:`proxy_connect_timeout 5s; proxy_read_timeout 30s; proxy_send_timeout 30s;`(默认 60s 过长,同步长事务窗口内挂死连接)。
|
||||||
|
7. **client_header_buffer 大请求头**(1 万连接下避免默认 1k 头部缓冲爆),并按 AGENTS.md 先做配置快照(`deploy/backups/<时间戳>/`)。
|
||||||
|
- 测试/验证:改后 `docker compose config` 校验、`nginx -t`(容器内)、`curl -I` 断言 gzip 头/Cache-Control;压测脚本断言无 connection refused。
|
||||||
|
- 回归风险:`/uploads/` 长缓存头会让「替换图片同名不同图」被浏览器缓存 → 上传路径需确认图片 URL 含版本参数或降级 `max-age=1d`(此点实施时与业务确认)。
|
||||||
|
|
||||||
|
## P0-3 连接池与超时参数(防排队超时与慢查询占池)
|
||||||
|
|
||||||
|
- 涉及:`deploy/docker-compose.yml`(生产 `DATABASE_URL`)
|
||||||
|
- 做法:
|
||||||
|
1. `DATABASE_URL` 追加 `?connection_limit=50&pool_timeout=3`。注意 `pool_timeout` 单位是**秒**(Prisma 文档默认 10s),不要写成毫秒;不要加 `statement_cache_size=0`(那是 pgbouncer 事务模式的做法,直连场景禁用 prepared statement 反而拖慢)。50 < PG 上限,为同步与运维连接留余量。
|
||||||
|
2. PG 容器 `command: ["postgres", "-c", "statement_timeout=10000", "-c", "shared_buffers=512MB", "-c", "effective_cache_size=1536MB", "-c", "max_connections=200"]`(14GB 内存机,shared_buffers 512MB 够工作集;statement_timeout 10s 兜住慢查询不占池)。
|
||||||
|
3. 同步任务自身查库量大(长事务串行循环)—— statement_timeout 10s 需确认不误伤同步批量 upsert(单条语句都不应超 10s;若误伤,同步侧改批量化见 P1-2)。
|
||||||
|
4. 变更 PG 参数需重启容器(秒级中断):按全局约束 §1 先做配置快照 + `pg_dump -Fc` 存档,低峰窗口执行,`docker compose up -d --force-recreate v2-postgres` 后立即验证健康与连接水位。
|
||||||
|
- 测试:入参级别——用一次性 docker postgres(AGENTS.md 同款 54329 套路)跑 `prisma migrate deploy` + 全量 jest 确认无超时回归;生产以 `SHOW statement_timeout`/`pg_stat_activity` 验证。
|
||||||
|
- 回归风险:`pool_timeout=3000` 让洪峰期排队 >3s 快速失败(429/500)而非挂 10s——配合 P0-1 缓存后池压力极小;这是有意的快速失败语义,需在压测时观察错误率符合验收口径。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P1-1 `/public/goods` SQL 分页(量级放宽后的正解)
|
||||||
|
|
||||||
|
- 涉及:`apps/api/src/public/public.service.ts`(`getGoods`)
|
||||||
|
- 背景:P0-1.2 缓存已接住当前百级量级;SQL 分页是为「商品量级上万」预埋(代码注释 L231-233 已留方向)。
|
||||||
|
- 做法:族表物化视图方向——`product_families` 加「族代表行」物化列(代表 Good 的 goodName/主图/分类/countryId/排序键)与**族最低价物化列**(P0-1.1 的 LATERAL 聚合结果届时改为由族重算路径写入该列,内存缓存退化为直读列),列表查询改为 `findMany({ take, skip, include })` + 独立 `count`,去掉内存分组与全量拉取。物化列的写入统一挂在族重算 `recomputeFamily` 与 admin 写路径上(写后 bump 对应缓存域,遵守全局约束 §2)。
|
||||||
|
- 测试:TDD——分页正确性(边界页、total 与 items 长度)、排序与现内存版全等(用 AGENTS.md「同一排序函数生成期望」防中文/键序手写错误)、打乱输入断言输出全等。
|
||||||
|
- 风险:改动面大(DTO/排序语义),**必须**与现实现做差分对照(同一数据集新老实现结果全等),放 P1 不与 P0 抢时间窗口;涉及 schema 变更(族表物化列)时按全局约束 §1 先备份存档并交付回滚迁移。
|
||||||
|
|
||||||
|
## P1-2 同步任务与 public 隔离
|
||||||
|
|
||||||
|
- 涉及:`apps/api/src/sync/sync.service.ts`、`apps/api/src/product-families/family-recompute.service.ts`
|
||||||
|
- 做法(按成本升序,实施时选一):
|
||||||
|
1. **错峰**:整点分类同步窗口(~分钟级长事务)移到低峰(如 04:00 后紧邻详情同步);至少避开业务高峰。
|
||||||
|
2. **批量化**:syncCategories 单事务串行 → 分批事务(每 50 分类一提交);syncProducts 逐商品 upsert → `createMany`/并行化(注意 SDS 依赖逐条校验)。
|
||||||
|
3. **独立 worker**:`SYNC_WORKER=true` 环境开关,第二个 v2-api 容器只跑 schedule(`app.listen` 前 return / NestFactory disable listen),与 public 进程池物理隔离——唯一彻底方案,代价是 +1 容器与连接预算(已 P0-3 预留 50/池 ×2)。
|
||||||
|
- 测试:同步结果幂等与现行为全等(对比 sync_logs 行数与库行数);并发压测与同步同时进行的窗口场景;行为变更(批量化/重算)执行前按全局约束 §1 备份存档。
|
||||||
|
- 风险:族重算 fire-and-forget 队列在独立 worker 下需确认 enqueue 侧(public 进程也会 trigger?——现状 enqueue 在同步与族变更路径,若拆分需统一入口)。
|
||||||
|
|
||||||
|
## P1-3 压测基线(验收依据,必须分布式源)
|
||||||
|
|
||||||
|
- 涉及:k6(或 wrk)脚本放 `scripts/loadtest/`;目标环境与压力机要求见步骤 0
|
||||||
|
- 步骤:
|
||||||
|
0. **目标环境(硬性)**:1 万并发口径压测**不得直打生产栈**——用最近 `pg_dump` 在一次性镜像栈恢复(AGENTS.md 同款 `docker run … postgres:16-alpine` 起库 + `prisma migrate deploy` + 单独 compose 起 api/nginx 副本,参数与生产一致),`THROTTLE_LIMIT` 只在镜像栈抬高;若最终必须在生产验证,只允许业务书面确认的低峰窗口 + 只读场景 + 限流不动的保守梯度。**压力机必须是独立机器**(与被测机同机跑 10k VU 会互相抢 CPU,测出来的是压测机瓶颈)。
|
||||||
|
1. SQL 单测基线:dump 恢复后逐条跑 `loadFamilyMinPrices` / `loadTreeOrderMeta` / 列表全量 / 详情 / ILIKE keyword,记录真实耗时(填空报告 §2 的估算值)。
|
||||||
|
2. 功能压测:`THROTTLE_LIMIT` 临时抬到 1e6(或分布式 ≥ 数十 IP)避免 429 干扰;场景 = 混合读(home-goods / goods / 详情 / 分类树按 3:4:2:1)。
|
||||||
|
3. 口径压测:逐步加并发至 1 万瞬时请求(如 k6 10000 VU,分布式源),记录 p95/p99/错误率/nginx 连接拒绝/PG 水位,并记录 Node 进程 CPU 与事件循环延迟(`--max-old-space-size` 与 GC 停顿)。
|
||||||
|
4. **达标失败的扩展阶梯**(按成本升序,逐级尝试后再进下一级):
|
||||||
|
a. **响应字节级缓存**:P0-1 缓存的是 DTO 对象,命中后每请求仍要 `JSON.stringify`——1 万瞬时 burst 下单进程事件循环的序列化 CPU 可能成为新瓶颈(每响应 ~10-20KB × 1 万次)。若压测显示 stringify 是热点,把缓存值改为**已序列化的最终 JSON 字符串**(含 TransformInterceptor 包裹结构),命中路径直接回写字节,绕过逐请求序列化;
|
||||||
|
b. **nginx micro-cache**:`proxy_cache` 对 `/public/` GET 设 10~30s 短 TTL,洪峰完全由边缘吸收。⚠️ 与全局约束 §2 冲突:proxy_cache 无法主动 purge,只能靠短 TTL——需业务确认接受该陈旧窗口,不接受则跳过此项;
|
||||||
|
c. **Redis + 多副本**:内存缓存换共享存储(版本域失效天然跨进程),加 v2-api 副本(nginx upstream 已 keepalive,扩容仅加容器 + PG max_connections 预算)。
|
||||||
|
- 交付:压测报告(数字+结论)追加到性能审查报告 §5 验收结果。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P2 收尾项(非阻塞,可后置)
|
||||||
|
|
||||||
|
1. **静态资源边缘直出**:`/uploads/` 数据卷直挂 edge nginx(`deploy/docker-compose.yml` 挂 `v2-uploads` 卷 + try_files),省一跳代理;注意 upload 与 nginx 读同卷的一致性(nginx 缓存/直读无冲突)。
|
||||||
|
2. **pg_trgm 索引**:`CREATE EXTENSION pg_trgm; CREATE INDEX ... ON goods USING GIN (good_name gin_trgm_ops);`(admin 侧 origin_goods.good_name 同)——goods 上万后 keyword 才需要;迁移文件 + 回滚脚本,执行前按全局约束 §1 备份存档(CREATE EXTENSION 为 DDL,回滚与重放需按迁移流程管理)。
|
||||||
|
3. **监控告警**:sync_logs 失败数、`pg_stat_activity` 连接水位、Prisma 池排队(日志关键字 `Timed out fetching a new connection`)→ 告警;nginx access log 结构化落盘(当前无 access 日志,1w 并发下调试无从下手)。
|
||||||
|
4. **README/docs 更新**:`docs/references/structs.md`(部署拓扑段补充性能配置说明)按 AGENTS.md 第 6/7 节执行。
|
||||||
|
|
||||||
|
## 全局回归策略
|
||||||
|
|
||||||
|
- 每项任务独立分支提交(Conventional Commits,`perf(...)`),合并回 `develop`。
|
||||||
|
- 全量 jest 前停 dev server(AGENTS.md 经验);连共享库前 `prisma migrate status`;集成测试夹具自包含(一次性 pg 跑套件)。
|
||||||
|
- 缓存相关改动重点回归:列表/详情/首页三端点输出与改造前全等(差分测试)、同步后新鲜度、TTL 失效路径。
|
||||||
|
|
||||||
|
## 验收(全部完成后)
|
||||||
|
|
||||||
|
- 分布式压测 1 万并发瞬时请求:p95 < 300ms、p99 < 1s、错误率 < 0.1%、无连接拒绝、PG 水位 < 80%、无池排队超时。
|
||||||
|
- 全局约束核查:涉及 DB 变更的任务全部留有 `deploy/backups/<时间戳>/` 存档与 RESTORE.md(可 dry-run 恢复验证);全部写路径(sync/族重算/admin CRUD/上传)的「写后缓存即失效」用例通过(含事务回滚不 bump、并发读不写回旧值)。
|
||||||
|
- 结果数字回填报告 §5 并关闭本计划。
|
||||||
Reference in New Issue
Block a user