test: add k6 works list load test
This commit is contained in:
3
.gitignore
vendored
3
.gitignore
vendored
@@ -32,3 +32,6 @@ temp*build*/
|
|||||||
/logs
|
/logs
|
||||||
.worktrees/
|
.worktrees/
|
||||||
.env.secrets.local
|
.env.secrets.local
|
||||||
|
|
||||||
|
# Local load-test data extracted from private migration files
|
||||||
|
scripts/loadtest/data/*.local.json
|
||||||
|
|||||||
310
.hermes/plans/2026-05-11_195214-k6-works-list-load-test-plan.md
Normal file
310
.hermes/plans/2026-05-11_195214-k6-works-list-load-test-plan.md
Normal file
@@ -0,0 +1,310 @@
|
|||||||
|
# K6 作品列表压测计划（使用 spacetime-migration-7.json 作为数据源）
|
||||||
|
|
||||||
|
## 目标
|
||||||
|
|
||||||
|
使用 K6 对 Genarrative 的“作品列表”相关接口进行压测，并将用户提供的 `spacetime-migration-7.json` 作为压测数据源；数据处理时**只导入作品列表相关数据**，不导入用户、会话、钱包、埋点、运行存档等非作品表，避免把敏感或无关数据带入压测环境。
|
||||||
|
|
||||||
|
## 当前上下文
|
||||||
|
|
||||||
|
- 工作区：`/c/proj/Genarrative`
|
||||||
|
- 原始迁移文件：`C:\Users\DSK\AppData\Local\hermes\cache\documents\doc_150e84029b2d_spacetime-migration-7.json`
|
||||||
|
- 已确认原始迁移文件结构：
|
||||||
|
- `schema_version = 1`
|
||||||
|
- `tables = 53`
|
||||||
|
- 作品相关表中当前有数据的重点表：
|
||||||
|
- `puzzle_work_profile`：80 行
|
||||||
|
- `custom_world_profile`：1 行
|
||||||
|
- `match3d_work_profile`：0 行
|
||||||
|
- `big_fish_*`：当前样本中相关表为 0 行
|
||||||
|
- 原始文件还包含 `user_account`、`auth_identity`、`refresh_session`、`profile_wallet_ledger`、`asset_object`、运行记录等数据，压测导入时必须过滤。
|
||||||
|
- 当前仓库未发现现成 K6 脚本或 `k6` 相关文件，需要新增压测脚本与数据提取脚本。
|
||||||
|
- `package.json` 当前有 `dev/dev:rust/test/check` 等脚本，未发现 K6 npm script。
|
||||||
|
|
||||||
|
## 范围约束
|
||||||
|
|
||||||
|
### 本次只导入/使用
|
||||||
|
|
||||||
|
1. 作品列表表：
|
||||||
|
- `puzzle_work_profile`
|
||||||
|
- `custom_world_profile`
|
||||||
|
- 后续若接口覆盖其他玩法，可扩展：
|
||||||
|
- `match3d_work_profile`
|
||||||
|
- `square_hole_work_profile`（以实际 SpacetimeDB 表名为准）
|
||||||
|
- `big_fish_work_profile`（以实际 SpacetimeDB 表名为准）
|
||||||
|
- `visual_novel_work_profile`（以实际 SpacetimeDB 表名为准）
|
||||||
|
2. 为作品列表卡片展示所需的最小字段：
|
||||||
|
- 稳定 ID：`profile_id`、`work_id` 或 `public_work_code`
|
||||||
|
- 标题：`work_title` / `level_name` / `world_name`
|
||||||
|
- 描述：`work_description` / `summary` / `summary_text` / `subtitle`
|
||||||
|
- 作者：`owner_user_id`、`author_display_name`、`author_public_user_code`
|
||||||
|
- 封面：`cover_image_src`、`cover_asset_id`（如果接口只返回 asset id，则压测阶段不额外导入二进制 asset）
|
||||||
|
- 状态与计数：`publication_status`、`published_at`、`play_count`、`like_count`、`remix_count`
|
||||||
|
- 作品内容摘要：`levels_json`、`profile_payload_json`、`theme_tags_json` 等列表渲染或进入作品详情可能需要的 JSON 字段
|
||||||
|
|
||||||
|
### 本次不导入/不使用
|
||||||
|
|
||||||
|
- 认证与账号：`user_account`、`auth_identity`、`refresh_session`、`auth_store_snapshot`
|
||||||
|
- 用户资产与钱包：`profile_wallet_ledger`、`profile_dashboard_state`、`profile_redeem_*`、`profile_invite_*`
|
||||||
|
- 游玩历史/存档/运行态：`profile_played_world`、`public_work_play_daily_stat`、`puzzle_runtime_run`、`profile_save_archive`、`runtime_snapshot` 等
|
||||||
|
- AI 任务过程：`ai_task`、`ai_task_stage`、`ai_text_chunk`
|
||||||
|
- asset 二进制与绑定：`asset_object`、`asset_entity_binding`，除非后续确认作品列表接口强依赖它们；即便需要，也只导入作品列表封面所需的最小 metadata，不导入原始大对象。
|
||||||
|
|
||||||
|
## 推荐目录与文件
|
||||||
|
|
||||||
|
建议新增：
|
||||||
|
|
||||||
|
```text
|
||||||
|
.hermes/plans/2026-05-11_195214-k6-works-list-load-test-plan.md # 本计划
|
||||||
|
scripts/loadtest/extract-works-list-data.mjs # 从迁移文件提取作品列表数据
|
||||||
|
scripts/loadtest/k6-works-list.js # K6 压测脚本
|
||||||
|
scripts/loadtest/data/works-list.sample.json # 过滤后的样例数据（不要提交敏感原始迁移全量）
|
||||||
|
scripts/loadtest/README.md # 执行说明与指标阈值
|
||||||
|
```
|
||||||
|
|
||||||
|
可选新增 npm scripts：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"loadtest:extract-works": "node scripts/loadtest/extract-works-list-data.mjs",
|
||||||
|
"loadtest:k6:works": "k6 run scripts/loadtest/k6-works-list.js"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 数据提取方案
|
||||||
|
|
||||||
|
### 输入
|
||||||
|
|
||||||
|
默认读取：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node scripts/loadtest/extract-works-list-data.mjs \
|
||||||
|
--input "C:/Users/DSK/AppData/Local/hermes/cache/documents/doc_150e84029b2d_spacetime-migration-7.json" \
|
||||||
|
--output scripts/loadtest/data/works-list.local.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### 输出结构
|
||||||
|
|
||||||
|
建议输出为 K6 直接可读的 JSON：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"source": "spacetime-migration-7.json",
|
||||||
|
"generatedAt": "<iso datetime>",
|
||||||
|
"tables": {
|
||||||
|
"puzzle_work_profile": [
|
||||||
|
{
|
||||||
|
"profile_id": "...",
|
||||||
|
"work_id": "...",
|
||||||
|
"owner_user_id": "...",
|
||||||
|
"work_title": "...",
|
||||||
|
"work_description": "...",
|
||||||
|
"publication_status": "Published",
|
||||||
|
"published_at": { "__timestamp_micros_since_unix_epoch__": 0 },
|
||||||
|
"play_count": 0,
|
||||||
|
"like_count": 0,
|
||||||
|
"levels_json": "..."
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"custom_world_profile": []
|
||||||
|
},
|
||||||
|
"workIds": {
|
||||||
|
"puzzle": ["<profile_id>"],
|
||||||
|
"customWorld": ["<profile_id>"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 过滤原则
|
||||||
|
|
||||||
|
1. 按 `tables[].name` 白名单过滤，只保留作品 profile 表。
|
||||||
|
2. 对每个 row 再按字段白名单过滤，避免误带账号、手机号、token、钱包流水等字段。
|
||||||
|
3. 对特别大的字段进行处理：
|
||||||
|
- `cover_image_src` 如果是 `data:image/...base64`，默认替换为占位符或截断，避免压测数据文件过大。
|
||||||
|
- `levels_json`、`profile_payload_json` 保留原文，但可以记录大小；如果过大，再提供 `--compact` 选项只保留摘要。
|
||||||
|
4. 输出 `.local.json` 默认加入 `.gitignore`；如果要提交样例数据，只提交脱敏/裁剪后的 `works-list.sample.json`。
|
||||||
|
|
||||||
|
## K6 压测接口矩阵
|
||||||
|
|
||||||
|
需要先确认本地 api-server 实际端口。默认以 `http://127.0.0.1:8787` 为例，实际运行时通过环境变量覆盖：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:<actual-api-port> k6 run scripts/loadtest/k6-works-list.js
|
||||||
|
```
|
||||||
|
|
||||||
|
初版建议覆盖以下“作品列表”读接口，具体路径以仓库服务端路由为准，实施时需要通过搜索 api-server 路由确认：
|
||||||
|
|
||||||
|
| 场景 | 目的 | 候选路径 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 拼图作品列表 | 作品列表主场景之一，当前数据量最多 | `/api/creation/puzzle/works` 或实际 puzzle works list route |
|
||||||
|
| RPG/自定义世界作品列表 | 使用 `custom_world_profile` 数据 | `/api/creation/custom-world/works` 或实际 custom world works route |
|
||||||
|
| 作品详情/启动前读取 | 模拟用户从列表点进作品 | `/api/creation/*/works/:profileId` 或 `/api/runtime/*/works/:profileId` |
|
||||||
|
| 公开作品库 | 如果首页/发现页依赖 | `/api/runtime/*/works` 或 gallery/list route |
|
||||||
|
|
||||||
|
> 注意：不要凭空固定 endpoint。实施阶段先用 `search_files` / 路由源码确认真实路径，再写入 K6 脚本。
|
||||||
|
|
||||||
|
## K6 场景设计
|
||||||
|
|
||||||
|
### 阶段 1：基线 smoke
|
||||||
|
|
||||||
|
目的：确认脚本、数据和目标服务可用。
|
||||||
|
|
||||||
|
```js
|
||||||
|
export const options = {
|
||||||
|
scenarios: {
|
||||||
|
smoke: {
|
||||||
|
executor: 'constant-vus',
|
||||||
|
vus: 1,
|
||||||
|
duration: '30s'
|
||||||
|
}
|
||||||
|
},
|
||||||
|
thresholds: {
|
||||||
|
http_req_failed: ['rate<0.01'],
|
||||||
|
http_req_duration: ['p(95)<800']
|
||||||
|
}
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### 阶段 2：常规读压
|
||||||
|
|
||||||
|
目的：模拟日常列表浏览。
|
||||||
|
|
||||||
|
- `constant-vus`: 10/25/50 三档
|
||||||
|
- 每个 VU 随机选择作品类型和列表分页参数
|
||||||
|
- `sleep(0.5~2s)` 模拟用户停留
|
||||||
|
- 阈值建议：
|
||||||
|
- `http_req_failed < 1%`
|
||||||
|
- `p95 < 800ms`
|
||||||
|
- `p99 < 1500ms`
|
||||||
|
|
||||||
|
### 阶段 3：峰值/突刺
|
||||||
|
|
||||||
|
目的：模拟首页入口或活动导致的作品列表突增。
|
||||||
|
|
||||||
|
- `ramping-arrival-rate`
|
||||||
|
- 从 5 RPS 增长到 100 RPS，维持 2~5 分钟，再降回
|
||||||
|
- 单独输出 `checks`：列表接口状态码、响应 JSON shape、items 数量
|
||||||
|
|
||||||
|
### 阶段 4：容量探索
|
||||||
|
|
||||||
|
目的：找瓶颈，不作为每次回归必跑。
|
||||||
|
|
||||||
|
- 每轮提升 RPS 或 VU
|
||||||
|
- 观察：api-server CPU/内存、SpacetimeDB 日志、错误率、p95/p99
|
||||||
|
- 一旦 `http_req_failed >= 5%` 或 p95 持续超过 2s，停止继续加压并记录容量点。
|
||||||
|
|
||||||
|
## K6 脚本设计要点
|
||||||
|
|
||||||
|
1. 使用 `SharedArray` 加载 `works-list.local.json`，避免每个 VU 重复解析大 JSON。
|
||||||
|
2. 基于数据源里的 `profile_id` / `work_id` 随机抽样，保证请求覆盖真实作品 ID。
|
||||||
|
3. 对列表接口添加分页/排序 query，例如：
|
||||||
|
- `?limit=20&offset=0`
|
||||||
|
- `?pageSize=20&cursor=...`（以真实 API 为准）
|
||||||
|
4. 使用 `check()` 验证：
|
||||||
|
- HTTP 200
|
||||||
|
- 响应体是 JSON
|
||||||
|
- `items` 或 `works` 是数组
|
||||||
|
- 列表项包含 `profileId/profile_id`、标题字段、状态字段
|
||||||
|
5. 使用 `Trend` / `Rate` 细分指标：
|
||||||
|
- `works_list_duration`
|
||||||
|
- `works_detail_duration`
|
||||||
|
- `works_list_shape_error_rate`
|
||||||
|
6. 支持环境变量：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:8787 \
|
||||||
|
WORKS_DATA=scripts/loadtest/data/works-list.local.json \
|
||||||
|
SCENARIO=baseline \
|
||||||
|
k6 run scripts/loadtest/k6-works-list.js
|
||||||
|
```
|
||||||
|
|
||||||
|
## 实施步骤
|
||||||
|
|
||||||
|
1. **确认路由**
|
||||||
|
- 搜索 api-server / BFF 的作品列表路由。
|
||||||
|
- 明确各玩法对应 endpoint、鉴权要求、分页参数、返回字段。
|
||||||
|
2. **实现数据提取脚本**
|
||||||
|
- 新增 `scripts/loadtest/extract-works-list-data.mjs`。
|
||||||
|
- 只按表白名单读取作品列表 profile 表。
|
||||||
|
- 对字段做白名单与脱敏/截断。
|
||||||
|
- 输出 `works-list.local.json`。
|
||||||
|
3. **生成本地压测数据**
|
||||||
|
- 用用户提供的迁移文件生成 `scripts/loadtest/data/works-list.local.json`。
|
||||||
|
- 验证输出只包含作品表和作品字段。
|
||||||
|
4. **实现 K6 脚本**
|
||||||
|
- 新增 `scripts/loadtest/k6-works-list.js`。
|
||||||
|
- 支持 `BASE_URL`、`WORKS_DATA`、`SCENARIO`。
|
||||||
|
- 覆盖列表接口，必要时增加详情/启动前读取接口。
|
||||||
|
5. **新增执行说明**
|
||||||
|
- 在 `scripts/loadtest/README.md` 写明：安装 K6、启动本地 dev 栈、提取数据、运行 smoke/baseline/spike、查看结果。
|
||||||
|
6. **本地验证**
|
||||||
|
- 启动 Genarrative dev 栈；注意端口可能漂移，使用实际 api-server 端口。
|
||||||
|
- 跑 smoke：`SCENARIO=smoke`。
|
||||||
|
- 确认失败率、p95、响应 shape。
|
||||||
|
7. **可选集成 npm scripts**
|
||||||
|
- 如果团队希望标准化入口，再加入 `package.json` scripts。
|
||||||
|
8. **记录结果**
|
||||||
|
- 将 smoke/baseline/spike 的结果摘要追加到 `scripts/loadtest/README.md` 或单独保存到 `.hermes/plans/` 的结果文档中。
|
||||||
|
|
||||||
|
## 启动与运行建议
|
||||||
|
|
||||||
|
本地服务启动按当前 Genarrative dev 栈约定：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
如果 SpacetimeDB/API/Vite 端口被占用，项目脚本会寻找可用端口；压测时必须从启动日志中读取实际 api-server 地址，并传给 K6：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:<actual-api-port> \
|
||||||
|
WORKS_DATA=scripts/loadtest/data/works-list.local.json \
|
||||||
|
SCENARIO=smoke \
|
||||||
|
k6 run scripts/loadtest/k6-works-list.js
|
||||||
|
```
|
||||||
|
|
||||||
|
## 验证标准
|
||||||
|
|
||||||
|
### 数据源验证
|
||||||
|
|
||||||
|
- `works-list.local.json` 中只出现作品 profile 表。
|
||||||
|
- 不出现以下字段或内容：
|
||||||
|
- `password_hash`
|
||||||
|
- `refresh_token_hash`
|
||||||
|
- `phone_number_e164`
|
||||||
|
- `phone_number_masked`
|
||||||
|
- `wallet_ledger_id`
|
||||||
|
- `auth_identity`
|
||||||
|
- `user_account`
|
||||||
|
- `puzzle_work_profile` 行数应接近原始文件中的 80 行。
|
||||||
|
- `custom_world_profile` 行数应接近原始文件中的 1 行。
|
||||||
|
|
||||||
|
### K6 smoke 验证
|
||||||
|
|
||||||
|
- 所有目标接口返回 2xx。
|
||||||
|
- `http_req_failed < 1%`。
|
||||||
|
- 响应 JSON shape 与 shared contracts 对齐：`items` 或 `works` 数组。
|
||||||
|
- K6 输出中能区分不同 endpoint 的耗时。
|
||||||
|
|
||||||
|
### 性能阈值初稿
|
||||||
|
|
||||||
|
- Smoke：`p95 < 800ms`，失败率 `< 1%`
|
||||||
|
- Baseline：`p95 < 1000ms`，`p99 < 2000ms`，失败率 `< 1%`
|
||||||
|
- Spike：允许短暂 p95 抖动，但 1 分钟内应恢复；失败率 `< 5%`
|
||||||
|
|
||||||
|
阈值后续需要结合本地机器性能、SpacetimeDB 本地模式和正式部署规格调整。
|
||||||
|
|
||||||
|
## 风险与注意事项
|
||||||
|
|
||||||
|
1. **原始迁移文件包含敏感数据。** 必须只提取作品列表白名单字段，禁止把原始 JSON 全量提交到仓库。
|
||||||
|
2. **base64 封面可能导致压测数据膨胀。** 默认截断或替换为占位符，除非本次明确要测封面 payload 对响应体积的影响。
|
||||||
|
3. **本地 SpacetimeDB 与 api-server 端口会漂移。** 不要硬编码端口，运行时通过 `BASE_URL` 注入。
|
||||||
|
4. **列表接口可能需要鉴权。** 若实际接口要求登录，不要导入真实 refresh session；应使用本地测试账号或专门的压测 token 生成流程。
|
||||||
|
5. **作品表名/接口路径可能与候选名称不完全一致。** 实施前必须以源码路由为准。
|
||||||
|
6. **本计划仅保存压测方案，不执行实际压测。** 后续执行时再创建/修改脚本、导出过滤数据、跑 K6 并记录结果。
|
||||||
|
|
||||||
|
## 开放问题
|
||||||
|
|
||||||
|
1. 压测目标是本地 dev 栈、测试环境，还是预发/生产只读接口？不同环境阈值和安全边界不同。
|
||||||
|
2. “作品列表”是否只包含拼图和自定义世界，还是要覆盖 match3d、square-hole、big-fish、visual-novel 的统一列表入口？
|
||||||
|
3. 是否允许使用专门压测账号/token？如果接口无鉴权则无需处理。
|
||||||
|
4. 是否需要测封面/asset 加载，还是只测作品列表 JSON API？
|
||||||
@@ -36,6 +36,8 @@
|
|||||||
"format:check": "prettier --check .",
|
"format:check": "prettier --check .",
|
||||||
"test": "vitest run",
|
"test": "vitest run",
|
||||||
"test:watch": "vitest",
|
"test:watch": "vitest",
|
||||||
|
"loadtest:extract-works": "node scripts/loadtest/extract-works-list-data.mjs",
|
||||||
|
"loadtest:k6:works": "k6 run scripts/loadtest/k6-works-list.js",
|
||||||
"check": "npm run lint && npm run test && npm run build && npm run check:content",
|
"check": "npm run lint && npm run test && npm run build && npm run check:content",
|
||||||
"check:data": "node scripts/run-tsx.cjs scripts/validate-content.ts",
|
"check:data": "node scripts/run-tsx.cjs scripts/validate-content.ts",
|
||||||
"check:overrides": "node scripts/run-tsx.cjs scripts/validate-overrides.ts",
|
"check:overrides": "node scripts/run-tsx.cjs scripts/validate-overrides.ts",
|
||||||
|
|||||||
193
scripts/loadtest/README.md
Normal file
193
scripts/loadtest/README.md
Normal file
@@ -0,0 +1,193 @@
|
|||||||
|
# Genarrative 作品列表 K6 压测
|
||||||
|
|
||||||
|
本目录用于对“作品列表/公开广场”读接口做本地压测。数据源来自私有 SpacetimeDB migration，但提取脚本只输出作品 profile 白名单表，并对用户、作者、作品号、asset id 等标识做稳定映射。
|
||||||
|
|
||||||
|
## 文件
|
||||||
|
|
||||||
|
- `extract-works-list-data.mjs`：从 migration JSON 提取作品列表压测数据；本地输出也会脱敏路由 ID，因此默认用于列表接口压测，详情接口需先把同一份脱敏数据导入目标环境。
|
||||||
|
- `k6-works-list.js`：K6 压测脚本。
|
||||||
|
- `data/spacetime-migration-7.local.json`：本地私有原始数据副本，已被 `.gitignore` 忽略，不要提交。
|
||||||
|
- `data/works-list.local.json`：本地脱敏压测数据，已被 `.gitignore` 忽略，不要提交。
|
||||||
|
- `data/works-list.sample.json`：可提交的少量脱敏样例。
|
||||||
|
|
||||||
|
## 数据边界
|
||||||
|
|
||||||
|
允许导入的表：
|
||||||
|
|
||||||
|
- `puzzle_work_profile`
|
||||||
|
- `custom_world_profile`
|
||||||
|
- `match3d_work_profile`
|
||||||
|
- `square_hole_work_profile`
|
||||||
|
- `big_fish_work_profile`
|
||||||
|
- `visual_novel_work_profile`
|
||||||
|
|
||||||
|
明确不导入：
|
||||||
|
|
||||||
|
- 账号/认证：`user_account`、`auth_identity`、`refresh_session`、`auth_store_snapshot`
|
||||||
|
- 钱包/邀请：`profile_wallet_ledger`、`profile_redeem_*`、`profile_invite_*`
|
||||||
|
- 游玩历史/埋点/存档：`public_work_play_daily_stat`、`profile_played_world`、`puzzle_runtime_run`、`profile_save_archive`、`runtime_snapshot`
|
||||||
|
- AI 任务过程：`ai_task`、`ai_task_stage`、`ai_text_chunk`
|
||||||
|
- asset 二进制：`asset_object`、`asset_entity_binding`
|
||||||
|
|
||||||
|
提取脚本会移除 `source_session_id` / `source_agent_session_id` 等会话派生字段；这些字段不属于作品列表卡片压测必要字段。
|
||||||
|
|
||||||
|
## 重新提取数据
|
||||||
|
|
||||||
|
从仓库根目录执行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run loadtest:extract-works -- \
|
||||||
|
--input scripts/loadtest/data/spacetime-migration-7.local.json \
|
||||||
|
--output scripts/loadtest/data/works-list.local.json \
|
||||||
|
--sample-output scripts/loadtest/data/works-list.sample.json
|
||||||
|
```
|
||||||
|
|
||||||
|
也可以直接执行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node scripts/loadtest/extract-works-list-data.mjs \
|
||||||
|
--input scripts/loadtest/data/spacetime-migration-7.local.json \
|
||||||
|
--output scripts/loadtest/data/works-list.local.json \
|
||||||
|
--sample-output scripts/loadtest/data/works-list.sample.json
|
||||||
|
```
|
||||||
|
|
||||||
|
当前 local 全量提取结果：
|
||||||
|
|
||||||
|
- `puzzle_work_profile`: 80
|
||||||
|
- `custom_world_profile`: 1
|
||||||
|
- `match3d_work_profile`: 0
|
||||||
|
- `normalizedWorks`: 81
|
||||||
|
|
||||||
|
当前可提交 sample 结果：
|
||||||
|
|
||||||
|
- `puzzle_work_profile`: 3
|
||||||
|
- `custom_world_profile`: 1
|
||||||
|
- `match3d_work_profile`: 0
|
||||||
|
- `normalizedWorks`: 4
|
||||||
|
|
||||||
|
## 真实接口
|
||||||
|
|
||||||
|
已从 `server-rs/crates/api-server/src/app.rs` 确认的读接口：
|
||||||
|
|
||||||
|
公开接口，无需 Bearer token：
|
||||||
|
|
||||||
|
- `GET /api/runtime/puzzle/gallery`
|
||||||
|
- `GET /api/runtime/puzzle/gallery/{profile_id}`
|
||||||
|
- `GET /api/runtime/custom-world-gallery`
|
||||||
|
- `GET /api/runtime/custom-world-gallery/{owner_user_id}/{profile_id}`
|
||||||
|
- `GET /api/runtime/custom-world-gallery/by-code/{code}`
|
||||||
|
|
||||||
|
需要 Bearer token 的个人作品列表接口：
|
||||||
|
|
||||||
|
- `GET /api/runtime/puzzle/works`
|
||||||
|
- `GET /api/runtime/puzzle/works/{profile_id}`
|
||||||
|
- `GET /api/runtime/custom-world/works`
|
||||||
|
|
||||||
|
K6 脚本默认只跑公开列表接口；传入 `AUTH_TOKEN` 后会额外跑需要登录态的个人作品列表接口。当前真实列表 handler 未暴露分页/排序 query 参数，因此脚本不追加 `limit/offset`；若后续接口增加分页参数，再在 K6 中补随机分页。
|
||||||
|
|
||||||
|
详情接口默认不压测，因为本地数据中的 `profile_id` / `owner_user_id` 已脱敏，直接请求未导入脱敏数据的目标服务会 404。只有在目标环境已导入同一份脱敏数据，或改用真实 ID 本地文件时，才设置 `DETAIL_RATIO` 大于 0；详情请求不把 404 视为成功。
|
||||||
|
|
||||||
|
## 启动服务
|
||||||
|
|
||||||
|
按项目约定启动本地 dev 栈：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
注意端口可能漂移。以启动日志中的实际 api-server 端口为准，然后传给 K6：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:<actual-api-port> npm run loadtest:k6:works -- --summary-trend-stats="avg,min,med,p(90),p(95),p(99),max"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Smoke
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:8787 \
|
||||||
|
WORKS_DATA=scripts/loadtest/data/works-list.local.json \
|
||||||
|
SCENARIO=smoke \
|
||||||
|
DETAIL_RATIO=0 \
|
||||||
|
npm run loadtest:k6:works
|
||||||
|
```
|
||||||
|
|
||||||
|
默认：1 VU / 30s。
|
||||||
|
|
||||||
|
## Baseline
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:8787 \
|
||||||
|
WORKS_DATA=scripts/loadtest/data/works-list.local.json \
|
||||||
|
SCENARIO=baseline \
|
||||||
|
VUS=10 \
|
||||||
|
DURATION=3m \
|
||||||
|
DETAIL_RATIO=0 \
|
||||||
|
npm run loadtest:k6:works
|
||||||
|
```
|
||||||
|
|
||||||
|
默认阈值：
|
||||||
|
|
||||||
|
- `http_req_failed < 1%`
|
||||||
|
- `http_req_duration p95 < 800ms`
|
||||||
|
- `http_req_duration p99 < 1500ms`
|
||||||
|
- `works_list_shape_error_rate < 1%`
|
||||||
|
|
||||||
|
## Spike
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:8787 \
|
||||||
|
WORKS_DATA=scripts/loadtest/data/works-list.local.json \
|
||||||
|
SCENARIO=spike \
|
||||||
|
START_RPS=5 \
|
||||||
|
PEAK_RPS=100 \
|
||||||
|
HOLD=2m \
|
||||||
|
DETAIL_RATIO=0 \
|
||||||
|
npm run loadtest:k6:works
|
||||||
|
```
|
||||||
|
|
||||||
|
默认阈值：
|
||||||
|
|
||||||
|
- `http_req_failed < 5%`
|
||||||
|
- `http_req_duration p95 < 2000ms`
|
||||||
|
- `works_list_shape_error_rate < 5%`
|
||||||
|
|
||||||
|
## 带登录态压测个人作品列表
|
||||||
|
|
||||||
|
先通过本地登录或接口获取 access token，然后传入：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:8787 \
|
||||||
|
AUTH_TOKEN='<access-token>' \
|
||||||
|
SCENARIO=smoke \
|
||||||
|
DETAIL_RATIO=0 \
|
||||||
|
npm run loadtest:k6:works
|
||||||
|
```
|
||||||
|
|
||||||
|
不要把 token 写入仓库文件、README 或 shell history 中可共享的位置。
|
||||||
|
|
||||||
|
## 详情接口压测
|
||||||
|
|
||||||
|
仅当目标环境存在 `WORKS_DATA` 中的同一批 `profileId/ownerUserId` 时启用：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BASE_URL=http://127.0.0.1:8787 \
|
||||||
|
WORKS_DATA=scripts/loadtest/data/works-list.local.json \
|
||||||
|
SCENARIO=smoke \
|
||||||
|
DETAIL_RATIO=0.35 \
|
||||||
|
npm run loadtest:k6:works
|
||||||
|
```
|
||||||
|
|
||||||
|
如果详情请求返回 404，说明压测数据 ID 未导入目标环境或目标服务数据不一致，应先修正数据源，不要把 404 当成功。
|
||||||
|
|
||||||
|
## 排障
|
||||||
|
|
||||||
|
- 如果公开 gallery 返回 `creation_entry_disabled` 或 503，检查本地 creation entry 配置是否禁用了对应入口。
|
||||||
|
- 如果个人作品列表返回 401，确认 `AUTH_TOKEN` 是当前 api-server 可识别的 access token。
|
||||||
|
- 如果详情全部 404，确认是否已向目标环境导入与 `WORKS_DATA` 一致的数据。
|
||||||
|
|
||||||
|
## 验证命令
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx vitest run scripts/loadtest/extract-works-list-data.test.ts
|
||||||
|
npx eslint scripts/loadtest/extract-works-list-data.mjs scripts/loadtest/extract-works-list-data.test.ts scripts/loadtest/k6-works-list.js
|
||||||
|
```
|
||||||
214
scripts/loadtest/data/works-list.sample.json
Normal file
214
scripts/loadtest/data/works-list.sample.json
Normal file
@@ -0,0 +1,214 @@
|
|||||||
|
{
|
||||||
|
"source": "spacetime-migration-7.local.json",
|
||||||
|
"generatedAt": "2026-05-11T13:09:51.569Z",
|
||||||
|
"counts": {
|
||||||
|
"puzzle_work_profile": 3,
|
||||||
|
"custom_world_profile": 1,
|
||||||
|
"match3d_work_profile": 0
|
||||||
|
},
|
||||||
|
"tables": {
|
||||||
|
"puzzle_work_profile": [
|
||||||
|
{
|
||||||
|
"profile_id": "profile-001",
|
||||||
|
"work_id": "work-001",
|
||||||
|
"owner_user_id": "user-001",
|
||||||
|
"author_display_name": "author-001",
|
||||||
|
"cover_asset_id": "asset-001",
|
||||||
|
"cover_image_src": "/generated-puzzle-assets/puzzle-session-f38101d7277040fcb6fbc41fea8b714a/puzzle-session-f38101d7277040fcb6fbc41fea8b714a-candidate-2/asset-1777649330373133/image.png",
|
||||||
|
"work_title": "化学家",
|
||||||
|
"level_name": "文学家",
|
||||||
|
"summary": "几个文学家正站在山上面对着瀑布侃侃而谈",
|
||||||
|
"work_description": "一个穿着白大褂的化学家正在做酷炫的化学实验，背景是化学实验室",
|
||||||
|
"levels_json": "[{\"level_id\":\"puzzle-level-1777649242577-7\",\"level_name\":\"文学家\",\"picture_description\":\"几个文学家正站在山上面对着瀑布侃侃而谈\",\"candidates\":[{\"candidate_id\":\"puzzle-session-f38101d7277040fcb6fbc41fea8b714a-candidate-2\",\"image_src\":\"/generated-puzzle-assets/puzzle-session-f38101d7277040fcb6fbc41fea8b714a/puzzle-session-f38101d7277040fcb6fbc41fea8b714a-candidate-2/asset-1777649330373133/image.png\",\"asset_id\":\"asset-1777649330373133\",\"prompt\":\"几个文学家正站在山上面对着瀑布侃侃而谈\",\"actual_prompt\":\"请生成一张高清插画。画面主体：几个文学家正站在山上面对着瀑布侃侃而谈。画面…",
|
||||||
|
"anchor_pack_json": "{\"theme_promise\":{\"key\":\"themePromise\",\"label\":\"题材承诺\",\"value\":\"化学家\",\"status\":\"Locked\"},\"visual_subject\":{\"key\":\"visualSubject\",\"label\":\"画面主体\",\"value\":\"一个穿着白大褂的化学家正在做酷炫的化学实验，背景是化学实验室\",\"status\":\"Locked\"},\"visual_mood\":{\"key\":\"visualMood\",\"label\":\"视觉气质\",\"value\":\"清晰、适合拼图切块\",\"status\":\"Inferred\"},\"composition_hooks\":{\"key\":\"compositionHooks\",\"label\":\"拼图记忆点\",\"value\":\"主体轮廓、色块分区、局部细节\",\"status\":\"Inferred\"},\"tags_and_forbidden\":{\"key\":\"tagsAndForbidden\",\"label\":\"标签与禁忌\",\"value\":\"化学家、拼图、插画；禁止标题字\",\"status\":\"I…",
|
||||||
|
"theme_tags_json": "[\"化学家\",\"拼图\",\"插画\",\"禁止标题字\"]",
|
||||||
|
"publication_status": {
|
||||||
|
"Published": []
|
||||||
|
},
|
||||||
|
"play_count": 1,
|
||||||
|
"like_count": 0,
|
||||||
|
"remix_count": 1,
|
||||||
|
"updated_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777703338322544
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777648804043558
|
||||||
|
},
|
||||||
|
"published_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777649364112270
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"profile_id": "profile-002",
|
||||||
|
"work_id": "work-002",
|
||||||
|
"owner_user_id": "user-002",
|
||||||
|
"author_display_name": "author-002",
|
||||||
|
"work_title": "我不知道",
|
||||||
|
"level_name": "",
|
||||||
|
"summary": "你猜我是谁",
|
||||||
|
"work_description": "你猜我是谁",
|
||||||
|
"levels_json": "[{\"level_id\":\"puzzle-level-1\",\"level_name\":\"\",\"picture_description\":\"真不知道\",\"candidates\":[],\"selected_candidate_id\":null,\"cover_image_src\":null,\"cover_asset_id\":null,\"generation_status\":\"idle\"}]",
|
||||||
|
"anchor_pack_json": "{\"theme_promise\":{\"key\":\"themePromise\",\"label\":\"题材承诺\",\"value\":\"我不知道\",\"status\":\"Locked\"},\"visual_subject\":{\"key\":\"visualSubject\",\"label\":\"画面主体\",\"value\":\"真不知道\",\"status\":\"Locked\"},\"visual_mood\":{\"key\":\"visualMood\",\"label\":\"视觉气质\",\"value\":\"清晰、适合拼图切块\",\"status\":\"Inferred\"},\"composition_hooks\":{\"key\":\"compositionHooks\",\"label\":\"拼图记忆点\",\"value\":\"主体轮廓、色块分区、局部细节\",\"status\":\"Inferred\"},\"tags_and_forbidden\":{\"key\":\"tagsAndForbidden\",\"label\":\"标签与禁忌\",\"value\":\"我不知道、拼图、插画；禁止标题字\",\"status\":\"Inferred\"}}",
|
||||||
|
"theme_tags_json": "[\"我不知道\"]",
|
||||||
|
"publication_status": {
|
||||||
|
"Draft": []
|
||||||
|
},
|
||||||
|
"play_count": 0,
|
||||||
|
"like_count": 0,
|
||||||
|
"remix_count": 0,
|
||||||
|
"updated_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777619351714201
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777619336673245
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"profile_id": "profile-003",
|
||||||
|
"work_id": "work-003",
|
||||||
|
"owner_user_id": "user-003",
|
||||||
|
"author_display_name": "author-002",
|
||||||
|
"work_title": "",
|
||||||
|
"level_name": "",
|
||||||
|
"summary": "",
|
||||||
|
"work_description": "",
|
||||||
|
"levels_json": "[{\"level_id\":\"puzzle-level-1\",\"level_name\":\"\",\"picture_description\":\"\",\"candidates\":[],\"selected_candidate_id\":null,\"cover_image_src\":null,\"cover_asset_id\":null,\"generation_status\":\"idle\"}]",
|
||||||
|
"anchor_pack_json": "{\"theme_promise\":{\"key\":\"themePromise\",\"label\":\"题材承诺\",\"value\":\"\",\"status\":\"Missing\"},\"visual_subject\":{\"key\":\"visualSubject\",\"label\":\"画面主体\",\"value\":\"\",\"status\":\"Missing\"},\"visual_mood\":{\"key\":\"visualMood\",\"label\":\"视觉气质\",\"value\":\"\",\"status\":\"Missing\"},\"composition_hooks\":{\"key\":\"compositionHooks\",\"label\":\"拼图记忆点\",\"value\":\"\",\"status\":\"Missing\"},\"tags_and_forbidden\":{\"key\":\"tagsAndForbidden\",\"label\":\"标签与禁忌\",\"value\":\"\",\"status\":\"Missing\"}}",
|
||||||
|
"theme_tags_json": "[\"拼图\",\"插画\",\"清晰构图\"]",
|
||||||
|
"publication_status": {
|
||||||
|
"Draft": []
|
||||||
|
},
|
||||||
|
"play_count": 0,
|
||||||
|
"like_count": 0,
|
||||||
|
"remix_count": 0,
|
||||||
|
"updated_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777622285252380
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777622285252380
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"custom_world_profile": [
|
||||||
|
{
|
||||||
|
"profile_id": "profile-081",
|
||||||
|
"owner_user_id": "user-002",
|
||||||
|
"author_display_name": "author-012",
|
||||||
|
"author_public_user_code": "author-code-001",
|
||||||
|
"world_name": "青春飞扬校园",
|
||||||
|
"summary_text": "在现代校园中，玩家摆脱内卷，追求真实成长",
|
||||||
|
"subtitle": "反内卷的自由学习之旅",
|
||||||
|
"profile_payload_json": "{\"anchorContent\":null,\"anchorPack\":null,\"attributeSchema\":{\"generatedFrom\":{\"conflictCore\":\"与传统教育模式的冲突\",\"settingSummary\":\"在现代校园中，玩家摆脱内卷，追求真实成长\",\"tone\":\"积极向上，充满活力与创新\",\"worldName\":\"青春飞扬校园\",\"worldType\":\"CUSTOM\"},\"id\":\"schema:rpg-agent:1e15b44d:v1\",\"schemaVersion\":1,\"slots\":[{\"name\":\"知识储备\",\"slotId\":\"axis_a\"},{\"name\":\"创新思维\",\"slotId\":\"axis_b\"},{\"name\":\"社交能力\",\"slotId\":\"axis_c\"},{\"name\":\"抗压能力\",\"slotId\":\"axis_d\"},{\"name\":\"自我认知\",\"slotId\":\"axis_e\"},{\"name\":\"团队协作\",\"slotId\":\"axis_f\"}],\"worldId\":\"custom:青春飞扬校…",
|
||||||
|
"publication_status": {
|
||||||
|
"Draft": []
|
||||||
|
},
|
||||||
|
"play_count": 0,
|
||||||
|
"like_count": 0,
|
||||||
|
"remix_count": 0,
|
||||||
|
"updated_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777532006629209
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777531745887256
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"match3d_work_profile": []
|
||||||
|
},
|
||||||
|
"profileIds": {
|
||||||
|
"puzzle": [
|
||||||
|
"profile-001",
|
||||||
|
"profile-002",
|
||||||
|
"profile-003"
|
||||||
|
],
|
||||||
|
"customWorld": [
|
||||||
|
"profile-081"
|
||||||
|
],
|
||||||
|
"match3d": [],
|
||||||
|
"squareHole": [],
|
||||||
|
"bigFish": [],
|
||||||
|
"visualNovel": []
|
||||||
|
},
|
||||||
|
"workIds": {
|
||||||
|
"puzzle": [
|
||||||
|
"work-001",
|
||||||
|
"work-002",
|
||||||
|
"work-003"
|
||||||
|
],
|
||||||
|
"customWorld": [],
|
||||||
|
"match3d": [],
|
||||||
|
"squareHole": [],
|
||||||
|
"bigFish": [],
|
||||||
|
"visualNovel": []
|
||||||
|
},
|
||||||
|
"normalizedWorks": [
|
||||||
|
{
|
||||||
|
"type": "puzzle",
|
||||||
|
"workId": "work-001",
|
||||||
|
"profileId": "profile-001",
|
||||||
|
"ownerUserId": "user-001",
|
||||||
|
"title": "化学家",
|
||||||
|
"subtitle": "几个文学家正站在山上面对着瀑布侃侃而谈",
|
||||||
|
"publicationStatus": {
|
||||||
|
"Published": []
|
||||||
|
},
|
||||||
|
"playCount": 1,
|
||||||
|
"likeCount": 0,
|
||||||
|
"remixCount": 1,
|
||||||
|
"coverImageSrc": "/generated-puzzle-assets/puzzle-session-f38101d7277040fcb6fbc41fea8b714a/puzzle-session-f38101d7277040fcb6fbc41fea8b714a-candidate-2/asset-1777649330373133/image.png",
|
||||||
|
"updatedAt": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777703338322544
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "puzzle",
|
||||||
|
"workId": "work-002",
|
||||||
|
"profileId": "profile-002",
|
||||||
|
"ownerUserId": "user-002",
|
||||||
|
"title": "我不知道",
|
||||||
|
"subtitle": "你猜我是谁",
|
||||||
|
"publicationStatus": {
|
||||||
|
"Draft": []
|
||||||
|
},
|
||||||
|
"playCount": 0,
|
||||||
|
"likeCount": 0,
|
||||||
|
"remixCount": 0,
|
||||||
|
"updatedAt": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777619351714201
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "puzzle",
|
||||||
|
"workId": "work-003",
|
||||||
|
"profileId": "profile-003",
|
||||||
|
"ownerUserId": "user-003",
|
||||||
|
"title": "",
|
||||||
|
"subtitle": "",
|
||||||
|
"publicationStatus": {
|
||||||
|
"Draft": []
|
||||||
|
},
|
||||||
|
"playCount": 0,
|
||||||
|
"likeCount": 0,
|
||||||
|
"remixCount": 0,
|
||||||
|
"updatedAt": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777622285252380
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "customWorld",
|
||||||
|
"profileId": "profile-081",
|
||||||
|
"ownerUserId": "user-002",
|
||||||
|
"title": "青春飞扬校园",
|
||||||
|
"subtitle": "反内卷的自由学习之旅",
|
||||||
|
"publicationStatus": {
|
||||||
|
"Draft": []
|
||||||
|
},
|
||||||
|
"playCount": 0,
|
||||||
|
"likeCount": 0,
|
||||||
|
"remixCount": 0,
|
||||||
|
"updatedAt": {
|
||||||
|
"__timestamp_micros_since_unix_epoch__": 1777532006629209
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
370
scripts/loadtest/extract-works-list-data.mjs
Normal file
370
scripts/loadtest/extract-works-list-data.mjs
Normal file
@@ -0,0 +1,370 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
import { readFile, writeFile } from 'node:fs/promises';
|
||||||
|
import { basename } from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
const ALLOWED_TABLES = new Set([
|
||||||
|
'puzzle_work_profile',
|
||||||
|
'custom_world_profile',
|
||||||
|
'match3d_work_profile',
|
||||||
|
'square_hole_work_profile',
|
||||||
|
'big_fish_work_profile',
|
||||||
|
'visual_novel_work_profile',
|
||||||
|
]);
|
||||||
|
|
||||||
|
const WORK_TABLE_TYPES = {
|
||||||
|
puzzle_work_profile: 'puzzle',
|
||||||
|
custom_world_profile: 'customWorld',
|
||||||
|
match3d_work_profile: 'match3d',
|
||||||
|
square_hole_work_profile: 'squareHole',
|
||||||
|
big_fish_work_profile: 'bigFish',
|
||||||
|
visual_novel_work_profile: 'visualNovel',
|
||||||
|
};
|
||||||
|
|
||||||
|
const TABLE_OUTPUT_ORDER = [
|
||||||
|
'puzzle_work_profile',
|
||||||
|
'custom_world_profile',
|
||||||
|
'match3d_work_profile',
|
||||||
|
'square_hole_work_profile',
|
||||||
|
'big_fish_work_profile',
|
||||||
|
'visual_novel_work_profile',
|
||||||
|
];
|
||||||
|
|
||||||
|
const WORK_TYPES = ['puzzle', 'customWorld', 'match3d', 'squareHole', 'bigFish', 'visualNovel'];
|
||||||
|
const SHORT_TEXT_LIMIT = 120;
|
||||||
|
const LONG_TEXT_LIMIT = 500;
|
||||||
|
const SENSITIVE_PATTERN = /(token|secret|password|passwd|phone|wallet|credential|authorization|auth[_-]?key|api[_-]?key)/giu;
|
||||||
|
|
||||||
|
class StableMapper {
|
||||||
|
constructor(prefix) {
|
||||||
|
this.prefix = prefix;
|
||||||
|
this.values = new Map();
|
||||||
|
}
|
||||||
|
|
||||||
|
map(value) {
|
||||||
|
if (value === undefined || value === null || value === '') return value;
|
||||||
|
const key = String(value);
|
||||||
|
if (!this.values.has(key)) {
|
||||||
|
this.values.set(
|
||||||
|
key,
|
||||||
|
`${this.prefix}-${String(this.values.size + 1).padStart(3, '0')}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return this.values.get(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function createContext() {
|
||||||
|
return {
|
||||||
|
user: new StableMapper('user'),
|
||||||
|
session: new StableMapper('session'),
|
||||||
|
author: new StableMapper('author'),
|
||||||
|
authorCode: new StableMapper('author-code'),
|
||||||
|
publicWorkCode: new StableMapper('public-work-code'),
|
||||||
|
coverAsset: new StableMapper('asset'),
|
||||||
|
work: new StableMapper('work'),
|
||||||
|
profile: new StableMapper('profile'),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function createWorkTypeBuckets() {
|
||||||
|
return Object.fromEntries(WORK_TYPES.map((type) => [type, []]));
|
||||||
|
}
|
||||||
|
|
||||||
|
function unwrapSpacetimeOption(value) {
|
||||||
|
if (
|
||||||
|
value &&
|
||||||
|
typeof value === 'object' &&
|
||||||
|
!Array.isArray(value) &&
|
||||||
|
Object.keys(value).length === 1
|
||||||
|
) {
|
||||||
|
if (Object.prototype.hasOwnProperty.call(value, 'some')) return value.some;
|
||||||
|
if (Object.prototype.hasOwnProperty.call(value, 'none')) return undefined;
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function truncateText(value, limit) {
|
||||||
|
if (value === undefined || value === null) return value;
|
||||||
|
const text = String(value).replace(/\s+/g, ' ').trim();
|
||||||
|
if (text.length <= limit) return text;
|
||||||
|
return `${text.slice(0, limit)}…`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function redactSensitiveText(value) {
|
||||||
|
if (value === undefined || value === null) return value;
|
||||||
|
return String(value).replace(SENSITIVE_PATTERN, '[redacted]');
|
||||||
|
}
|
||||||
|
|
||||||
|
function sanitizeCoverImageSrc(value) {
|
||||||
|
const unwrapped = unwrapSpacetimeOption(value);
|
||||||
|
if (unwrapped === undefined || unwrapped === null || unwrapped === '') return unwrapped;
|
||||||
|
const text = String(unwrapped);
|
||||||
|
if (text.startsWith('data:image/')) return '[redacted-data-image]';
|
||||||
|
let withoutQuery = text.split('?')[0].split('#')[0];
|
||||||
|
if (withoutQuery.length > 180) withoutQuery = `${withoutQuery.slice(0, 180)}…`;
|
||||||
|
return withoutQuery;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sanitizeLargeJson(value) {
|
||||||
|
const unwrapped = unwrapSpacetimeOption(value);
|
||||||
|
if (unwrapped === undefined || unwrapped === null) return unwrapped;
|
||||||
|
if (typeof unwrapped === 'string') {
|
||||||
|
return truncateText(redactSensitiveText(unwrapped), LONG_TEXT_LIMIT);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return truncateText(redactSensitiveText(JSON.stringify(unwrapped)), LONG_TEXT_LIMIT);
|
||||||
|
} catch {
|
||||||
|
return truncateText(redactSensitiveText(String(unwrapped)), LONG_TEXT_LIMIT);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function firstDefined(row, keys) {
|
||||||
|
for (const key of keys) {
|
||||||
|
if (row[key] !== undefined && row[key] !== null) return row[key];
|
||||||
|
}
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sanitizeShortField(row, sanitized, key) {
|
||||||
|
if (row[key] !== undefined) {
|
||||||
|
sanitized[key] = truncateText(unwrapSpacetimeOption(row[key]), SHORT_TEXT_LIMIT);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function sanitizeWorkRow(row, ctx) {
|
||||||
|
const sanitized = {};
|
||||||
|
const profileId = unwrapSpacetimeOption(firstDefined(row, ['profile_id', 'profileId']));
|
||||||
|
const workId = unwrapSpacetimeOption(firstDefined(row, ['work_id', 'workId']));
|
||||||
|
|
||||||
|
if (profileId !== undefined) sanitized.profile_id = ctx.profile.map(profileId);
|
||||||
|
if (workId !== undefined) sanitized.work_id = ctx.work.map(workId);
|
||||||
|
if (row.owner_user_id !== undefined) {
|
||||||
|
sanitized.owner_user_id = ctx.user.map(unwrapSpacetimeOption(row.owner_user_id));
|
||||||
|
}
|
||||||
|
if (row.user_id !== undefined) sanitized.user_id = ctx.user.map(unwrapSpacetimeOption(row.user_id));
|
||||||
|
|
||||||
|
if (row.author_display_name !== undefined) {
|
||||||
|
sanitized.author_display_name = ctx.author.map(unwrapSpacetimeOption(row.author_display_name));
|
||||||
|
}
|
||||||
|
if (row.public_work_code !== undefined) {
|
||||||
|
sanitized.public_work_code = ctx.publicWorkCode.map(unwrapSpacetimeOption(row.public_work_code));
|
||||||
|
}
|
||||||
|
if (row.author_public_user_code !== undefined) {
|
||||||
|
sanitized.author_public_user_code = ctx.authorCode.map(
|
||||||
|
unwrapSpacetimeOption(row.author_public_user_code),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (row.cover_asset_id !== undefined) {
|
||||||
|
sanitized.cover_asset_id = ctx.coverAsset.map(unwrapSpacetimeOption(row.cover_asset_id));
|
||||||
|
}
|
||||||
|
if (row.cover_image_src !== undefined) sanitized.cover_image_src = sanitizeCoverImageSrc(row.cover_image_src);
|
||||||
|
|
||||||
|
for (const key of [
|
||||||
|
'title',
|
||||||
|
'work_title',
|
||||||
|
'level_name',
|
||||||
|
'world_name',
|
||||||
|
'summary',
|
||||||
|
'summary_text',
|
||||||
|
'description',
|
||||||
|
'work_description',
|
||||||
|
'subtitle',
|
||||||
|
]) {
|
||||||
|
sanitizeShortField(row, sanitized, key);
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const key of ['levels_json', 'profile_payload_json', 'anchor_pack_json', 'theme_tags_json']) {
|
||||||
|
if (row[key] !== undefined) sanitized[key] = sanitizeLargeJson(row[key]);
|
||||||
|
}
|
||||||
|
|
||||||
|
const passthroughKeys = [
|
||||||
|
'publication_status',
|
||||||
|
'publicationStatus',
|
||||||
|
'play_count',
|
||||||
|
'playCount',
|
||||||
|
'like_count',
|
||||||
|
'likeCount',
|
||||||
|
'remix_count',
|
||||||
|
'remixCount',
|
||||||
|
'updated_at',
|
||||||
|
'created_at',
|
||||||
|
'published_at',
|
||||||
|
'visibility',
|
||||||
|
'status',
|
||||||
|
'category',
|
||||||
|
'tags',
|
||||||
|
];
|
||||||
|
for (const key of passthroughKeys) {
|
||||||
|
if (row[key] !== undefined) sanitized[key] = unwrapSpacetimeOption(row[key]);
|
||||||
|
}
|
||||||
|
|
||||||
|
return sanitized;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeWork(tableName, row) {
|
||||||
|
const type = WORK_TABLE_TYPES[tableName];
|
||||||
|
return {
|
||||||
|
type,
|
||||||
|
workId: row.work_id,
|
||||||
|
profileId: row.profile_id,
|
||||||
|
ownerUserId: row.owner_user_id,
|
||||||
|
publicWorkCode: row.public_work_code,
|
||||||
|
title: row.title ?? row.work_title ?? row.level_name ?? row.world_name,
|
||||||
|
subtitle: row.subtitle ?? row.summary_text ?? row.summary ?? row.work_description ?? row.description,
|
||||||
|
publicationStatus: row.publicationStatus ?? row.publication_status ?? row.status,
|
||||||
|
playCount: row.playCount ?? row.play_count ?? 0,
|
||||||
|
likeCount: row.likeCount ?? row.like_count ?? 0,
|
||||||
|
remixCount: row.remixCount ?? row.remix_count ?? 0,
|
||||||
|
coverImageSrc: row.cover_image_src,
|
||||||
|
updatedAt: row.updated_at,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function toRowsByTable(input) {
|
||||||
|
const tables = Array.isArray(input?.tables) ? input.tables : [];
|
||||||
|
const result = new Map();
|
||||||
|
for (const table of tables) {
|
||||||
|
if (!ALLOWED_TABLES.has(table?.name)) continue;
|
||||||
|
result.set(table.name, Array.isArray(table.rows) ? table.rows : []);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function extractWorksListData(input, options = {}) {
|
||||||
|
const ctx = createContext();
|
||||||
|
const rowsByTable = toRowsByTable(input);
|
||||||
|
const outputTables = {};
|
||||||
|
const counts = {};
|
||||||
|
const profileIds = createWorkTypeBuckets();
|
||||||
|
const workIds = createWorkTypeBuckets();
|
||||||
|
const normalizedWorks = [];
|
||||||
|
|
||||||
|
for (const tableName of TABLE_OUTPUT_ORDER) {
|
||||||
|
const sourceRows = rowsByTable.get(tableName);
|
||||||
|
if (!sourceRows) continue;
|
||||||
|
const sanitizedRows = sourceRows.map((row) => sanitizeWorkRow(row, ctx));
|
||||||
|
outputTables[tableName] = sanitizedRows;
|
||||||
|
counts[tableName] = sanitizedRows.length;
|
||||||
|
|
||||||
|
const type = WORK_TABLE_TYPES[tableName];
|
||||||
|
if (type) {
|
||||||
|
for (const row of sanitizedRows) {
|
||||||
|
if (row.profile_id) profileIds[type].push(row.profile_id);
|
||||||
|
if (row.work_id) workIds[type].push(row.work_id);
|
||||||
|
normalizedWorks.push(normalizeWork(tableName, row));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
source: options.source ?? 'unknown',
|
||||||
|
generatedAt: options.generatedAt ?? new Date().toISOString(),
|
||||||
|
counts,
|
||||||
|
tables: outputTables,
|
||||||
|
profileIds,
|
||||||
|
workIds,
|
||||||
|
normalizedWorks,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function createSampleOutput(output, maxRowsPerTable = 3) {
|
||||||
|
const tables = {};
|
||||||
|
const counts = {};
|
||||||
|
const allowedWorkIds = new Set();
|
||||||
|
const allowedProfileIds = new Set();
|
||||||
|
|
||||||
|
for (const [tableName, rows] of Object.entries(output.tables)) {
|
||||||
|
tables[tableName] = rows.slice(0, maxRowsPerTable);
|
||||||
|
counts[tableName] = tables[tableName].length;
|
||||||
|
const type = WORK_TABLE_TYPES[tableName];
|
||||||
|
if (type) {
|
||||||
|
for (const row of tables[tableName]) {
|
||||||
|
if (row.work_id) allowedWorkIds.add(row.work_id);
|
||||||
|
if (row.profile_id) allowedProfileIds.add(row.profile_id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const profileIds = Object.fromEntries(
|
||||||
|
Object.entries(output.profileIds).map(([type, ids]) => [
|
||||||
|
type,
|
||||||
|
ids.filter((id) => allowedProfileIds.has(id)).slice(0, maxRowsPerTable),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
const workIds = Object.fromEntries(
|
||||||
|
Object.entries(output.workIds).map(([type, ids]) => [
|
||||||
|
type,
|
||||||
|
ids.filter((id) => allowedWorkIds.has(id)).slice(0, maxRowsPerTable),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
const normalizedWorks = output.normalizedWorks
|
||||||
|
.filter((work) => allowedWorkIds.has(work.workId) || allowedProfileIds.has(work.profileId))
|
||||||
|
.slice(0, maxRowsPerTable * 6);
|
||||||
|
|
||||||
|
return {
|
||||||
|
...output,
|
||||||
|
counts,
|
||||||
|
tables,
|
||||||
|
profileIds,
|
||||||
|
workIds,
|
||||||
|
normalizedWorks,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseArgs(argv) {
|
||||||
|
const args = {};
|
||||||
|
for (let index = 0; index < argv.length; index += 1) {
|
||||||
|
const arg = argv[index];
|
||||||
|
if (arg === '--input' || arg === '--output' || arg === '--sample-output') {
|
||||||
|
const value = argv[index + 1];
|
||||||
|
if (!value || value.startsWith('--')) throw new Error(`${arg} requires a value`);
|
||||||
|
args[arg.slice(2)] = value;
|
||||||
|
index += 1;
|
||||||
|
} else if (arg === '--help' || arg === '-h') {
|
||||||
|
args.help = true;
|
||||||
|
} else {
|
||||||
|
throw new Error(`Unknown argument: ${arg}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return args;
|
||||||
|
}
|
||||||
|
|
||||||
|
function usage() {
|
||||||
|
return 'Usage: node scripts/loadtest/extract-works-list-data.mjs --input <migration.json> --output <works-list.local.json> [--sample-output <works-list.sample.json>]';
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function runCli(argv = process.argv.slice(2)) {
|
||||||
|
const args = parseArgs(argv);
|
||||||
|
if (args.help) {
|
||||||
|
console.log(usage());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!args.input) throw new Error('Missing required --input. ' + usage());
|
||||||
|
if (!args.output) throw new Error('Missing required --output. ' + usage());
|
||||||
|
|
||||||
|
const raw = await readFile(args.input, 'utf8');
|
||||||
|
const migration = JSON.parse(raw);
|
||||||
|
const output = extractWorksListData(migration, { source: basename(args.input) });
|
||||||
|
await writeFile(args.output, `${JSON.stringify(output, null, 2)}\n`, 'utf8');
|
||||||
|
|
||||||
|
if (args['sample-output']) {
|
||||||
|
const sample = createSampleOutput(output);
|
||||||
|
await writeFile(args['sample-output'], `${JSON.stringify(sample, null, 2)}\n`, 'utf8');
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
`works-list extracted: source=${output.source}, tables=${Object.keys(output.tables).length}, normalizedWorks=${output.normalizedWorks.length}`,
|
||||||
|
);
|
||||||
|
for (const [tableName, count] of Object.entries(output.counts)) {
|
||||||
|
console.log(` ${tableName}: ${count}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const isDirectRun = process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1];
|
||||||
|
if (isDirectRun) {
|
||||||
|
runCli().catch((error) => {
|
||||||
|
console.error(error instanceof Error ? error.message : String(error));
|
||||||
|
process.exitCode = 1;
|
||||||
|
});
|
||||||
|
}
|
||||||
247
scripts/loadtest/extract-works-list-data.test.ts
Normal file
247
scripts/loadtest/extract-works-list-data.test.ts
Normal file
@@ -0,0 +1,247 @@
|
|||||||
|
import { execFile } from 'node:child_process';
|
||||||
|
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import { promisify } from 'node:util';
|
||||||
|
|
||||||
|
import { describe, expect, it } from 'vitest';
|
||||||
|
|
||||||
|
import { extractWorksListData } from './extract-works-list-data.mjs';
|
||||||
|
|
||||||
|
const execFileAsync = promisify(execFile);
|
||||||
|
const scriptPath = fileURLToPath(new URL('./extract-works-list-data.mjs', import.meta.url));
|
||||||
|
|
||||||
|
const fixtureMigration = {
|
||||||
|
schema_version: 7,
|
||||||
|
tables: [
|
||||||
|
{
|
||||||
|
name: 'puzzle_work_profile',
|
||||||
|
rows: [
|
||||||
|
{
|
||||||
|
profile_id: 'profile-real-aaa',
|
||||||
|
work_id: 'work-real-aaa',
|
||||||
|
owner_user_id: 'owner-secret-123',
|
||||||
|
author_display_name: 'Alice Secret',
|
||||||
|
author_public_user_code: 'author-code-secret',
|
||||||
|
public_work_code: 'public-code-secret',
|
||||||
|
title: '超长标题'.repeat(20),
|
||||||
|
summary: 'summary '.repeat(80),
|
||||||
|
description: 'description '.repeat(120),
|
||||||
|
publication_status: 'published',
|
||||||
|
play_count: 42,
|
||||||
|
like_count: 7,
|
||||||
|
cover_asset_id: { some: 'asset-secret-cover' },
|
||||||
|
cover_image_src: { some: 'https://cdn.example.test/cover.png?token=***&sig=abc' },
|
||||||
|
levels_json: JSON.stringify({ secret: 'level-token-value', data: 'x'.repeat(2000) }),
|
||||||
|
theme_tags_json: JSON.stringify(['化学家', '实验室']),
|
||||||
|
remix_count: 2,
|
||||||
|
updated_at: '2026-05-01T00:00:00Z',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
profile_id: 'profile-real-bbb',
|
||||||
|
work_id: 'work-real-bbb',
|
||||||
|
owner_user_id: 'owner-secret-123',
|
||||||
|
author_display_name: 'Alice Secret',
|
||||||
|
publication_status: 'draft',
|
||||||
|
play_count: 3,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'custom_world_profile',
|
||||||
|
rows: [
|
||||||
|
{
|
||||||
|
profile_id: 'world-profile-secret',
|
||||||
|
work_id: 'world-work-secret',
|
||||||
|
owner_user_id: 'world-owner-secret',
|
||||||
|
title: '世界作品',
|
||||||
|
profile_payload_json: '{"large":"' + 'y'.repeat(2000) + '"}',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'public_work_play_daily_stat',
|
||||||
|
rows: [
|
||||||
|
{
|
||||||
|
source_type: 'puzzle',
|
||||||
|
profile_id: 'profile-real-aaa',
|
||||||
|
owner_user_id: 'owner-secret-123',
|
||||||
|
user_id: 'player-secret-456',
|
||||||
|
source_session_id: 'session-secret-789',
|
||||||
|
played_day: '2026-05-01',
|
||||||
|
play_count: 12,
|
||||||
|
updated_at: '2026-05-02T00:00:00Z',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'user_account',
|
||||||
|
rows: [
|
||||||
|
{
|
||||||
|
user_id: 'owner-secret-123',
|
||||||
|
phone: '+8613800138000',
|
||||||
|
auth_token: 'auth-token-secret',
|
||||||
|
wallet_balance: 999,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'refresh_session',
|
||||||
|
rows: [{ token: 'refresh-token-secret', source_session_id: 'session-secret-789' }],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'profile_wallet_ledger',
|
||||||
|
rows: [{ wallet_id: 'wallet-secret', amount: 100 }],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
async function withTempDir(fn) {
|
||||||
|
const dir = await mkdtemp(path.join(tmpdir(), 'works-list-test-'));
|
||||||
|
try {
|
||||||
|
return await fn(dir);
|
||||||
|
} finally {
|
||||||
|
await rm(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('extractWorksListData', () => {
|
||||||
|
it('只保留作品 profile 白名单表，禁用的行为/敏感表不会出现在输出 JSON 字符串中', () => {
|
||||||
|
const output = extractWorksListData(fixtureMigration, { source: 'fixture.local.json' });
|
||||||
|
const serialized = JSON.stringify(output);
|
||||||
|
|
||||||
|
expect(Object.keys(output.tables).sort()).toEqual([
|
||||||
|
'custom_world_profile',
|
||||||
|
'puzzle_work_profile',
|
||||||
|
]);
|
||||||
|
expect(serialized).not.toContain('public_work_play_daily_stat');
|
||||||
|
expect(serialized).not.toContain('user_account');
|
||||||
|
expect(serialized).not.toContain('refresh_session');
|
||||||
|
expect(serialized).not.toContain('profile_wallet_ledger');
|
||||||
|
expect(serialized).not.toContain('+8613800138000');
|
||||||
|
expect(serialized).not.toContain('auth-token-secret');
|
||||||
|
expect(serialized).not.toContain('wallet-secret');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('不会输出 owner/user/session/auth/token/phone/wallet 等敏感原值，owner 稳定映射', () => {
|
||||||
|
const output = extractWorksListData(fixtureMigration, { source: 'fixture.local.json' });
|
||||||
|
const serialized = JSON.stringify(output);
|
||||||
|
|
||||||
|
for (const secret of [
|
||||||
|
'owner-secret-123',
|
||||||
|
'player-secret-456',
|
||||||
|
'session-secret-789',
|
||||||
|
'Alice Secret',
|
||||||
|
'author-code-secret',
|
||||||
|
'public-code-secret',
|
||||||
|
'asset-secret-cover',
|
||||||
|
'SECRET_TOKEN',
|
||||||
|
]) {
|
||||||
|
expect(serialized).not.toContain(secret);
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(output.tables.puzzle_work_profile[0].owner_user_id).toBe('user-001');
|
||||||
|
expect(output.tables.puzzle_work_profile[1].owner_user_id).toBe('user-001');
|
||||||
|
expect(output.tables.puzzle_work_profile[0].author_display_name).toBe('author-001');
|
||||||
|
expect(serialized).not.toContain('level-token-value');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('puzzle 数据生成 profileIds/workIds 和 normalizedWorks，并保留列表展示字段', () => {
|
||||||
|
const output = extractWorksListData(fixtureMigration, { source: 'fixture.local.json' });
|
||||||
|
|
||||||
|
expect(output.source).toBe('fixture.local.json');
|
||||||
|
expect(output.generatedAt).toEqual(expect.any(String));
|
||||||
|
expect(output.counts.puzzle_work_profile).toBe(2);
|
||||||
|
expect(output.profileIds.puzzle).toEqual(['profile-001', 'profile-002']);
|
||||||
|
expect(output.workIds.puzzle).toEqual(['work-001', 'work-002']);
|
||||||
|
expect(output.normalizedWorks).toEqual(
|
||||||
|
expect.arrayContaining([
|
||||||
|
expect.objectContaining({
|
||||||
|
type: 'puzzle',
|
||||||
|
workId: 'work-001',
|
||||||
|
profileId: 'profile-001',
|
||||||
|
publicationStatus: 'published',
|
||||||
|
playCount: 42,
|
||||||
|
title: expect.any(String),
|
||||||
|
remixCount: 2,
|
||||||
|
}),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
expect(output.tables.puzzle_work_profile[0].cover_image_src).toBe('https://cdn.example.test/cover.png');
|
||||||
|
expect(output.tables.puzzle_work_profile[0].theme_tags_json).toBe('["化学家","实验室"]');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('data image、URL token 和绝对输入路径不会泄露到输出', async () => {
|
||||||
|
await withTempDir(async (dir) => {
|
||||||
|
const input = path.join(dir, 'migration.local.json');
|
||||||
|
const output = path.join(dir, 'works-list.local.json');
|
||||||
|
await writeFile(
|
||||||
|
input,
|
||||||
|
JSON.stringify({
|
||||||
|
tables: [
|
||||||
|
{
|
||||||
|
name: 'puzzle_work_profile',
|
||||||
|
rows: [
|
||||||
|
{
|
||||||
|
profile_id: 'profile-real',
|
||||||
|
work_id: 'work-real',
|
||||||
|
cover_image_src: { some: 'data:image/png;base64,SECRET_IMAGE_BYTES' },
|
||||||
|
levels_json: JSON.stringify({ token: 'SECRET_TOKEN_VALUE', title: 'safe' }),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
'utf8',
|
||||||
|
);
|
||||||
|
|
||||||
|
await execFileAsync(process.execPath, [scriptPath, '--input', input, '--output', output]);
|
||||||
|
const extracted = JSON.parse(await readFile(output, 'utf8'));
|
||||||
|
const serialized = JSON.stringify(extracted);
|
||||||
|
|
||||||
|
expect(extracted.source).toBe('migration.local.json');
|
||||||
|
expect(serialized).not.toContain(dir);
|
||||||
|
expect(serialized).not.toContain('SECRET_IMAGE_BYTES');
|
||||||
|
expect(serialized).not.toContain('SECRET_TOKEN_VALUE');
|
||||||
|
expect(extracted.tables.puzzle_work_profile[0].cover_image_src).toBe('[redacted-data-image]');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('sample-output 只输出少量脱敏样例', async () => {
|
||||||
|
await withTempDir(async (dir) => {
|
||||||
|
const input = path.join(dir, 'migration.local.json');
|
||||||
|
const output = path.join(dir, 'works-list.local.json');
|
||||||
|
const sampleOutput = path.join(dir, 'works-list.sample.json');
|
||||||
|
const manyRows = Array.from({ length: 5 }, (_, index) => ({
|
||||||
|
profile_id: `profile-real-${index}`,
|
||||||
|
work_id: `work-real-${index}`,
|
||||||
|
owner_user_id: `owner-secret-${index}`,
|
||||||
|
title: `作品 ${index}`,
|
||||||
|
publication_status: 'published',
|
||||||
|
play_count: index,
|
||||||
|
}));
|
||||||
|
await writeFile(
|
||||||
|
input,
|
||||||
|
JSON.stringify({ tables: [{ name: 'puzzle_work_profile', rows: manyRows }] }),
|
||||||
|
'utf8',
|
||||||
|
);
|
||||||
|
|
||||||
|
await execFileAsync(process.execPath, [scriptPath, '--input', input, '--output', output, '--sample-output', sampleOutput]);
|
||||||
|
const sample = JSON.parse(await readFile(sampleOutput, 'utf8'));
|
||||||
|
const serialized = JSON.stringify(sample);
|
||||||
|
|
||||||
|
expect(sample.tables.puzzle_work_profile).toHaveLength(3);
|
||||||
|
expect(sample.normalizedWorks).toHaveLength(3);
|
||||||
|
expect(serialized).not.toContain('owner-secret-0');
|
||||||
|
expect(serialized).not.toContain('work-real-0');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('CLI 参数缺失时退出非 0 并输出清晰错误', async () => {
|
||||||
|
await expect(execFileAsync(process.execPath, [scriptPath, '--input', 'missing.json'])).rejects.toMatchObject({
|
||||||
|
code: 1,
|
||||||
|
stderr: expect.stringContaining('--output'),
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
228
scripts/loadtest/k6-works-list.js
Normal file
228
scripts/loadtest/k6-works-list.js
Normal file
@@ -0,0 +1,228 @@
|
|||||||
|
/* global __ENV */
|
||||||
|
import { check, sleep } from 'k6';
|
||||||
|
import { SharedArray } from 'k6/data';
|
||||||
|
import http from 'k6/http';
|
||||||
|
import { Rate, Trend } from 'k6/metrics';
|
||||||
|
|
||||||
|
const DEFAULT_WORKS_DATA = 'scripts/loadtest/data/works-list.local.json';
|
||||||
|
const WORKS_DATA = __ENV.WORKS_DATA || DEFAULT_WORKS_DATA;
|
||||||
|
const BASE_URL = (__ENV.BASE_URL || 'http://127.0.0.1:8787').replace(/\/+$/u, '');
|
||||||
|
const AUTH_TOKEN = __ENV.AUTH_TOKEN || '';
|
||||||
|
const SCENARIO = __ENV.SCENARIO || 'smoke';
|
||||||
|
const REQUEST_TIMEOUT = __ENV.REQUEST_TIMEOUT || '30s';
|
||||||
|
const SLEEP_MIN_SECONDS = Number(__ENV.SLEEP_MIN_SECONDS || '0.5');
|
||||||
|
const SLEEP_MAX_SECONDS = Number(__ENV.SLEEP_MAX_SECONDS || '2');
|
||||||
|
const DETAIL_RATIO = Number(__ENV.DETAIL_RATIO || '0');
|
||||||
|
|
||||||
|
const worksListShapeErrorRate = new Rate('works_list_shape_error_rate');
|
||||||
|
const worksDetailShapeErrorRate = new Rate('works_detail_shape_error_rate');
|
||||||
|
const worksListDuration = new Trend('works_list_duration');
|
||||||
|
const worksDetailDuration = new Trend('works_detail_duration');
|
||||||
|
|
||||||
|
const data = new SharedArray('works-list-data', () => [JSON.parse(open(WORKS_DATA))])[0];
|
||||||
|
const normalizedWorks = Array.isArray(data.normalizedWorks) ? data.normalizedWorks : [];
|
||||||
|
|
||||||
|
const scenarioOptions = {
|
||||||
|
smoke: {
|
||||||
|
scenarios: {
|
||||||
|
smoke: {
|
||||||
|
executor: 'constant-vus',
|
||||||
|
vus: Number(__ENV.VUS || 1),
|
||||||
|
duration: __ENV.DURATION || '30s',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
thresholds: {
|
||||||
|
http_req_failed: ['rate<0.01'],
|
||||||
|
http_req_duration: ['p(95)<800'],
|
||||||
|
works_list_shape_error_rate: ['rate<0.01'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
baseline: {
|
||||||
|
scenarios: {
|
||||||
|
baseline: {
|
||||||
|
executor: 'constant-vus',
|
||||||
|
vus: Number(__ENV.VUS || 10),
|
||||||
|
duration: __ENV.DURATION || '3m',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
thresholds: {
|
||||||
|
http_req_failed: ['rate<0.01'],
|
||||||
|
http_req_duration: ['p(95)<800', 'p(99)<1500'],
|
||||||
|
works_list_shape_error_rate: ['rate<0.01'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
spike: {
|
||||||
|
scenarios: {
|
||||||
|
spike: {
|
||||||
|
executor: 'ramping-arrival-rate',
|
||||||
|
preAllocatedVUs: Number(__ENV.PREALLOCATED_VUS || 50),
|
||||||
|
maxVUs: Number(__ENV.MAX_VUS || 200),
|
||||||
|
timeUnit: '1s',
|
||||||
|
stages: [
|
||||||
|
{ target: Number(__ENV.START_RPS || 5), duration: __ENV.RAMP_UP || '30s' },
|
||||||
|
{ target: Number(__ENV.PEAK_RPS || 100), duration: __ENV.HOLD || '2m' },
|
||||||
|
{ target: Number(__ENV.END_RPS || 5), duration: __ENV.RAMP_DOWN || '30s' },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
thresholds: {
|
||||||
|
http_req_failed: ['rate<0.05'],
|
||||||
|
http_req_duration: ['p(95)<2000'],
|
||||||
|
works_list_shape_error_rate: ['rate<0.05'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
export const options = scenarioOptions[SCENARIO] || scenarioOptions.smoke;
|
||||||
|
|
||||||
|
const PUBLIC_ENDPOINTS = [
|
||||||
|
{
|
||||||
|
name: 'puzzle_gallery_list',
|
||||||
|
method: 'GET',
|
||||||
|
path: '/api/runtime/puzzle/gallery',
|
||||||
|
expectCollectionKeys: ['items', 'works', 'entries'],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'custom_world_gallery_list',
|
||||||
|
method: 'GET',
|
||||||
|
path: '/api/runtime/custom-world-gallery',
|
||||||
|
expectCollectionKeys: ['entries', 'items', 'works'],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
const AUTH_ENDPOINTS = [
|
||||||
|
{
|
||||||
|
name: 'puzzle_works_list',
|
||||||
|
method: 'GET',
|
||||||
|
path: '/api/runtime/puzzle/works',
|
||||||
|
expectCollectionKeys: ['items', 'works'],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'custom_world_works_list',
|
||||||
|
method: 'GET',
|
||||||
|
path: '/api/runtime/custom-world/works',
|
||||||
|
expectCollectionKeys: ['items', 'entries', 'works'],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
function requestParams(endpointName) {
|
||||||
|
const headers = { 'x-genarrative-response-envelope': 'v1' };
|
||||||
|
if (AUTH_TOKEN) headers.Authorization = `Bearer ${AUTH_TOKEN}`;
|
||||||
|
return {
|
||||||
|
headers,
|
||||||
|
timeout: REQUEST_TIMEOUT,
|
||||||
|
tags: { endpoint: endpointName },
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildUrl(path) {
|
||||||
|
return `${BASE_URL}${path}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseJson(response) {
|
||||||
|
try {
|
||||||
|
return response.json();
|
||||||
|
} catch (_) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function unwrapPayload(json) {
|
||||||
|
if (!json || typeof json !== 'object') return null;
|
||||||
|
if (json.data && typeof json.data === 'object') return json.data;
|
||||||
|
return json;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasCollection(payload, keys) {
|
||||||
|
return keys.some((key) => Array.isArray(payload?.[key]));
|
||||||
|
}
|
||||||
|
|
||||||
|
function firstCollection(payload, keys) {
|
||||||
|
for (const key of keys) {
|
||||||
|
if (Array.isArray(payload?.[key])) return payload[key];
|
||||||
|
}
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasListItemShape(payload, keys) {
|
||||||
|
const collection = firstCollection(payload, keys);
|
||||||
|
if (collection.length === 0) return true;
|
||||||
|
const item = collection[0];
|
||||||
|
const hasId = Boolean(
|
||||||
|
item?.profileId || item?.profile_id || item?.workId || item?.work_id || item?.publicWorkCode,
|
||||||
|
);
|
||||||
|
const hasTitle = Boolean(
|
||||||
|
item?.title || item?.workTitle || item?.work_title || item?.levelName || item?.worldName,
|
||||||
|
);
|
||||||
|
return hasId && hasTitle;
|
||||||
|
}
|
||||||
|
|
||||||
|
function randomItem(items) {
|
||||||
|
if (!items.length) return null;
|
||||||
|
return items[Math.floor(Math.random() * items.length)];
|
||||||
|
}
|
||||||
|
|
||||||
|
function listEndpoints() {
|
||||||
|
return AUTH_TOKEN ? PUBLIC_ENDPOINTS.concat(AUTH_ENDPOINTS) : PUBLIC_ENDPOINTS;
|
||||||
|
}
|
||||||
|
|
||||||
|
function detailEndpointFor(work) {
|
||||||
|
if (!work || !work.profileId) return null;
|
||||||
|
if (work.type === 'puzzle') {
|
||||||
|
return {
|
||||||
|
name: 'puzzle_gallery_detail',
|
||||||
|
path: `/api/runtime/puzzle/gallery/${encodeURIComponent(work.profileId)}`,
|
||||||
|
expectKeys: ['item', 'work', 'entry'],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (work.type === 'customWorld' && work.profileId && work.ownerUserId) {
|
||||||
|
return {
|
||||||
|
name: 'custom_world_gallery_detail',
|
||||||
|
path: `/api/runtime/custom-world-gallery/${encodeURIComponent(work.ownerUserId)}/${encodeURIComponent(work.profileId)}`,
|
||||||
|
expectKeys: ['entry', 'item', 'work'],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function performListRequest(endpoint) {
|
||||||
|
const url = buildUrl(endpoint.path);
|
||||||
|
const response = http.request(endpoint.method, url, null, requestParams(endpoint.name));
|
||||||
|
worksListDuration.add(response.timings.duration, { endpoint: endpoint.name });
|
||||||
|
const json = parseJson(response);
|
||||||
|
const payload = unwrapPayload(json);
|
||||||
|
const ok = check(response, {
|
||||||
|
[`${endpoint.name} status is 200`]: (res) => res.status === 200,
|
||||||
|
[`${endpoint.name} returns json object`]: () => Boolean(payload),
|
||||||
|
[`${endpoint.name} has collection`]: () => hasCollection(payload, endpoint.expectCollectionKeys),
|
||||||
|
[`${endpoint.name} list item shape`]: () => hasListItemShape(payload, endpoint.expectCollectionKeys),
|
||||||
|
});
|
||||||
|
worksListShapeErrorRate.add(!ok, { endpoint: endpoint.name });
|
||||||
|
}
|
||||||
|
|
||||||
|
function performDetailRequest() {
|
||||||
|
const endpoint = detailEndpointFor(randomItem(normalizedWorks));
|
||||||
|
if (!endpoint) return;
|
||||||
|
|
||||||
|
const response = http.get(buildUrl(endpoint.path), requestParams(endpoint.name));
|
||||||
|
worksDetailDuration.add(response.timings.duration, { endpoint: endpoint.name });
|
||||||
|
const json = parseJson(response);
|
||||||
|
const payload = unwrapPayload(json);
|
||||||
|
const ok = check(response, {
|
||||||
|
[`${endpoint.name} status is 200`]: (res) => res.status === 200,
|
||||||
|
[`${endpoint.name} has detail payload`]: () => endpoint.expectKeys.some((key) => payload?.[key]),
|
||||||
|
});
|
||||||
|
worksDetailShapeErrorRate.add(!ok, { endpoint: endpoint.name });
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function () {
|
||||||
|
for (const endpoint of listEndpoints()) {
|
||||||
|
performListRequest(endpoint);
|
||||||
|
}
|
||||||
|
if (normalizedWorks.length && DETAIL_RATIO > 0 && Math.random() < DETAIL_RATIO) {
|
||||||
|
performDetailRequest();
|
||||||
|
}
|
||||||
|
|
||||||
|
const jitter = SLEEP_MIN_SECONDS + Math.random() * Math.max(0, SLEEP_MAX_SECONDS - SLEEP_MIN_SECONDS);
|
||||||
|
sleep(jitter);
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user