合并 master 并保留角色动作抠图修复
保留 master 的单次 FFmpeg filter graph 批量抽帧。 保留 BgFilter 前 RGB8 无补边输入与抠图后 RGBA8 透明补边。 合并并保留对应权威文档规则。
This commit is contained in:
@@ -503,6 +503,9 @@
|
||||
},
|
||||
"404": {
|
||||
"$ref": "#/components/responses/NotFound"
|
||||
},
|
||||
"409": {
|
||||
"$ref": "#/components/responses/Conflict"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1327,6 +1330,16 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"Conflict": {
|
||||
"description": "画布 revision 冲突,需重新读取最新快照后合并或重试",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/ErrorResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"UpstreamError": {
|
||||
"description": "上游生成服务失败",
|
||||
"content": {
|
||||
@@ -1755,8 +1768,17 @@
|
||||
"$ref": "#/components/schemas/EditorCanvasViewport"
|
||||
},
|
||||
"layers": {
|
||||
"type": "object",
|
||||
"description": "画布图层 JSON,最大约 256KB。"
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
},
|
||||
"description": "兼容画布布局 JSON,最大 2 MiB;服务端会拆分为结构化图层与生成对话框行。"
|
||||
},
|
||||
"expectedRevision": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"description": "可选的画布 revision CAS;不匹配时返回 409。"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
@@ -2110,7 +2132,11 @@
|
||||
"$ref": "#/components/schemas/EditorCanvasViewport"
|
||||
},
|
||||
"layers": {
|
||||
"type": "object"
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
},
|
||||
"resources": {
|
||||
"type": "array",
|
||||
@@ -2132,6 +2158,8 @@
|
||||
"title",
|
||||
"viewport",
|
||||
"layers",
|
||||
"revision",
|
||||
"layoutStorageVersion",
|
||||
"createdAt",
|
||||
"updatedAt"
|
||||
],
|
||||
@@ -2149,7 +2177,25 @@
|
||||
"$ref": "#/components/schemas/EditorCanvasViewport"
|
||||
},
|
||||
"layers": {
|
||||
"type": "object"
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
},
|
||||
"revision": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"layoutStorageVersion": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"backgroundColor": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"createdAt": {
|
||||
"type": "string",
|
||||
|
||||
@@ -37,6 +37,16 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-19 角色动作视频使用单进程批量抽帧
|
||||
|
||||
- 背景:角色动作生成在拿到预览视频后,原实现会为 `32 / 40 / 48` 个采样点分别启动一次 FFmpeg、重复解码同一视频。release 的 2 vCPU 主机在一次 32 帧任务中因此出现约 10 秒的 CPU 尖刺,且进程启动和重复解码都不是业务必需开销。
|
||||
- 决策:角色动作抽帧必须先沿用 `compute_sample_time_seconds()` 计算全部采样点,再通过一个 FFmpeg filter graph 对输入统一 `setpts`、`split`,各分支按精确 `select=gte(t\,<target>)` 输出一帧。不得改用会漂移现有采样时刻的粗粒度 `fps` 抽帧。单次命令完成后逐一确认全部目标文件存在,任一缺帧继续使用原有用户错误文案,并在 details 中保留首个缺帧的 `targetSeconds / outputPath`、整批 `missingFrames` 和 stdout/stderr。视频封面等单帧调用保留兼容 helper,但内部复用同一批量实现。
|
||||
- 影响范围:`server-rs/crates/api-server/src/character_animation_assets.rs` 的角色动作视频本地抽帧和单帧视频封面抽取;不改变尾帧安全步长、BgFilter 并发、OSS 路径、帧编号、透明化后处理或前后端结果契约。
|
||||
- 验证方式:运行 `cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`,真实短视频回归必须由一次 FFmpeg 命令产出整批帧,并继续断言 `32帧·4秒` 最后一帧为 `3.875s`;追加 `cargo check -p api-server --manifest-path server-rs/Cargo.toml`、Rust 格式、编码和 diff 门禁。
|
||||
- 关联文档:`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`、`docs/project-memory/shared-memory/pitfalls.md`。
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-18 图片生成 K 档由 provider 直接生成
|
||||
|
||||
- 背景:旧 gpt-image-2 尺寸表会把 2K 竖版回落到 `1024x1536`,图标入口又使用固定 `360x360 / 512x512` 占位;角色去背景结果变小时还会直接放大整张透明成品,导致 UI 显示的 2K 与模型实际生成清晰度不一致。
|
||||
@@ -4351,3 +4361,7 @@
|
||||
- Vite 全量边界补充:`src/games/**`、`src/data/**`、`src/prompts/**`、旧顶层 App / Playground、旧路由和 `services/ai.ts` 必须由 pre-transform 门禁直接拒绝;所有同源 `/generated-*` 裸读在 dev 与生产统一为空 `404`,历史对象只经现役签名读取接口兼容,不允许 SPA fallback 伪装成资产成功响应。
|
||||
- 前端混合根目录补充:`src/components`、`src/hooks`、`src/persistence`、`src/routing`、`src/services` 的根级文件实行现役白名单,Vite 与 ESLint 使用同一口径阻断旧 RPG / 玩法根文件;子目录仍按现役目录和退役目录分别管理,新增公共根文件必须显式登记。
|
||||
- 影响范围:`PlatformEntryActiveFlowShell`、`PlatformActiveProfileView`、编辑器 / 项目搜索、平台 profile clients、`module-runtime`、`platform-llm`、Cargo workspace / resolve graph、Vite / ESLint / Rust 产物门禁及旧业务退役方案。
|
||||
|
||||
## 2026-07-20 VectorEngine 图片任务预算收口到 worker deadline
|
||||
|
||||
- 决策:`editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation` 和 `editor_ui_design_asset_extraction` 使用默认 `1800s` long job 预算。worker 从同一起点计算绝对 job deadline,并向 provider 提前保留 `min(60s, job 预算 / 2)` 作为审计、OSS 和终态写回窗口。deadline 只经进程内 `RequestContext` 传递;VectorEngine 单 attempt 取配置 timeout 与剩余预算的较小值,退避加下一次 attempt 无法落在同一 deadline 内时停止重试,参考图和响应图片下载也受同一 deadline 限制。普通 HTTP / `inline` 保持无 deadline 行为;`VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 默认仍为 `1000000`,配置加载层允许显式值更低。lease 续租 / fencing、迟到写回仲裁、attempt 耗尽和原子退款语义不变。
|
||||
|
||||
@@ -288,12 +288,12 @@
|
||||
- 验证:`npm run test -- src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx`;`cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`。
|
||||
- 关联:`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`server-rs/crates/api-server/src/app.rs`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。
|
||||
|
||||
## 图片编辑器角色动画抽帧不要采到视频尾点
|
||||
## 图片编辑器角色动画抽帧不要采到视频尾点或逐帧重启 FFmpeg
|
||||
|
||||
- 现象:画板角色图点击 `生成动画` 后,Ark 视频已生成并上传 OSS,但后端返回 `ffmpeg 已执行但未产出动作帧文件(requestId:...)`。
|
||||
- 原因:FFmpeg 在 `-ss` 采样时间落到视频尾点附近时可能退出码仍为 `0`,但实际输出 `0` 帧;如果后端按 `duration - 0.001` 抽最后一帧,低帧率或短视频很容易踩到不可解码尾点。
|
||||
- 处理:角色动画抽帧按目标帧数预留一个采样步长,例如 `32帧·4秒` 最后一帧采 `3.875s`,不要采 `3.999s`;`ffmpeg` 返回成功但无输出文件时,错误 details 保留 `targetSeconds`、`stdout`、`stderr` 和输出路径,用户主文案保持简短。
|
||||
- 验证:`cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`,其中 `editor_character_animation_extracts_final_sample_from_short_video` 应覆盖本机 FFmpeg 8 的 0 帧回归。
|
||||
- 原因:FFmpeg 在采样时间落到视频尾点附近时可能退出码仍为 `0`,但实际输出 `0` 帧;如果后端按 `duration - 0.001` 抽最后一帧,低帧率或短视频很容易踩到不可解码尾点。旧实现还会为 `32 / 40 / 48` 个采样点分别启动 FFmpeg、重复解码同一视频,在低配 worker 上形成不必要的多秒 CPU 尖刺。
|
||||
- 处理:角色动画先按目标帧数计算全部安全采样时刻,例如 `32帧·4秒` 最后一帧采 `3.875s`,不要采 `3.999s`;随后使用单个 `setpts + split + select` filter graph 批量输出全部帧,不改用粗粒度 `fps` 抽帧。命令返回后逐一检查输出,缺帧时用户主文案保持简短,details 保留首个缺帧的 `targetSeconds / outputPath`、整批 `missingFrames` 和 stdout/stderr。
|
||||
- 验证:`cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`;`editor_character_animation_batch_extracts_all_samples_from_short_video` 必须用一次 FFmpeg 产出整批短视频帧,尾帧测试继续锁定 `3.875s`。
|
||||
- 关联:`server-rs/crates/api-server/src/character_animation_assets.rs`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。
|
||||
|
||||
## Windows 本地角色动画抽帧找不到 ffmpeg 先查 dev 子进程环境
|
||||
@@ -376,6 +376,14 @@
|
||||
- 验证:测试断言菜单不包含在底部工具栏 / 参考图行里,并且生成面板打开时底部 `AI画布工具栏` 仍存在;规范参考图来源菜单应能通过 portal 点击“从画布中选择 / 上传图片”并写回规范参考图。
|
||||
- 关联:`src/components/common/PlatformFloatingMenu.tsx`、`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx`。
|
||||
|
||||
## 图片编辑器 portal 菜单必须显式继承画板主题 token
|
||||
|
||||
- 现象:生成视频参数面板里点击“静音”后,开关轨道和白色滑块一起消失;如果直接把轨道改成 `#00ff00`,虽然重新可见,却变成与画板主题不一致的荧光绿。相同比例、清晰度、slider、时长文字和模型选中勾选也可能丢失选中态主题。
|
||||
- 原因:`renderEditorPortal(...)` 把 `.image-canvas-editor__portal-menu` 挂到 `document.body`,它不再是 `.image-canvas-editor` 的后代,无法继承只定义在编辑器根节点上的 `--image-canvas-brand-*` 自定义属性。浏览器会把依赖缺失变量且没有 fallback 的声明按无效值处理,轨道背景最终为透明。
|
||||
- 处理:portal 继续挂到 `document.body` 以避免局部 `overflow` 裁切,但外层必须通过 `.image-canvas-editor__portal-theme` 同步当前 `platform-theme--light / platform-theme--dark`;画板品牌 token 由 `.image-canvas-editor`、主题桥接层与 `.image-canvas-editor__portal-menu` 共用同一组声明。控件继续消费主题变量,不使用单点硬编码颜色,也不要只给静音轨道补 fallback 而遗漏同一 portal 内其它 token 消费者。
|
||||
- 验证:`scripts/image-canvas-portal-theme.test.ts` 应锁定编辑器根节点、portal 主题桥接层与 portal 菜单共享完整品牌 token,静音 pressed 轨道仍使用 `var(--image-canvas-brand-accent)` 且不出现 `#00ff00`;`useImageCanvasGenerationSurface.test.tsx` 应覆盖暗色主题 class 被桥接到 `document.body` 下的 portal。真实浏览器从 `生成视频 -> 视频参数 -> 静音` 点击后,轨道 computed background 应为非透明当前主题色,portal 内 `--image-canvas-brand-accent`、`--image-canvas-brand-border-strong` 和 `--image-canvas-brand-soft` 均应有值。
|
||||
- 关联:`src/index.css`、`scripts/image-canvas-portal-theme.test.ts`、`src/components/image-editor/ImageCanvasEditorPortal.tsx`、`src/components/image-editor/useImageCanvasGenerationSurface.tsx`、`src/components/image-editor/ImageCanvasGenerationComposerView.tsx`。
|
||||
|
||||
## 图片编辑器规范图片面板不要脱离统一生成 shell
|
||||
|
||||
- 现象:生成 UI 设计图或新建图标规范时,面板参考图、输入区和底部生成按钮相对生成图片 / 生成角色 / 生成视频错位;图标规范甚至可能缺少首行参考图入口。
|
||||
@@ -3232,3 +3240,9 @@
|
||||
- 原因:现役模块的静态 import 会让 Vite、TypeScript 和打包器递归解析依赖;watch ignore 只停止监听,Tailwind source 只控制 class 扫描,tree-shaking 也发生在模块已经加载之后。经 barrel 只取一个公共函数尤其容易把同文件的旧导出一起带回图中。
|
||||
- 处理:把仍在用的公共账号 / 钱包 / 设置能力迁到明确的现役 client 与 presentation model;Vite `pre` transform 对退役模块真实路径直接失败,ESLint 在现役源上增加 restricted imports。每次恢复公共 UI 后用 `tsc --listFilesOnly` 和全新浏览器 context 复核,不能用已有 HMR 会话判绿。
|
||||
- 关联:`vite.config.ts`、`.eslintrc.cjs`、`src/services/platform-entry/`、`docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md`。
|
||||
|
||||
## VectorEngine 请求超时不能脱离 worker 绝对预算(2026-07-20)
|
||||
|
||||
- 现象:VectorEngine 单次请求超时大于 worker job 执行预算时,worker 已停止续租,provider 才超时或开始重试;最终 lease 过期、任务失败并退款,上游却可能继续消耗资源或迟到成功。
|
||||
- 原因:单 attempt timeout、重试退避、图片下载与 worker / lease 分别使用独立的相对计时,没有共享同一绝对 deadline;只抬高 worker timeout 或单独压低 provider timeout 都无法保证留出终态写回窗口。
|
||||
- 处理:实际调用 VectorEngine 的四类图片 job 使用 `1800s` long 预算;从 job 开始的同一起点派生 provider deadline,常规提前 `60s`、短预算提前一半。每次 attempt、退避、下一次 attempt 和图片下载都必须在该 deadline 内;普通 HTTP / `inline` 不伪造 worker deadline。修复时不改动 lease fencing、迟到写回仲裁和原子退款语义。
|
||||
|
||||
@@ -125,6 +125,7 @@
|
||||
- 生成器快照刷新后必须恢复;待生成、生成中、失败和已生成后跟随成品图层的生成器都不能因为刷新丢失输入、参数、参考图或占位框位置。宣发素材生成器刷新后必须继续显示正确的卡片类型、游戏名、分类、描述和已绑定参考图。
|
||||
- 画布多选语义必须同时覆盖普通图层和仍显示占位框的生成器对象:Shift 点选或框选可把生成器加入当前选择;拖动任一已选图层或生成器时,所有已选普通图层和生成器占位框同步移动;删除 / Backspace / Delete 作用于完整选择集合,移除所有已选图层和生成器对象。生成器对象在选择集合中使用稳定 `generation-dialog:<id>` 目标 ID,不把生成器伪装成普通图层,也不新增后端表。
|
||||
- 生成类入口打开画布内面板时,底部 AI 工具栏必须保持可见;`生成规范`、角色 / 图标规范来源、角色常规参考图来源这类轻量菜单通过页面级 fixed portal 渲染,不能留在底部工具栏或参考图横向滚动容器内部,避免被局部 `overflow` 裁切。角色规范和常规参考图来源菜单必须向上弹出;常规参考图点击后先选择“从画布中选择”或“上传图片”,从画布取图时只绑定参考图,不触发普通画布图层选中、聚焦、面板隐藏或拖拽逻辑,绑定后退出画布选择状态。所有生成面板参考图槽位统一为方形图标组件;角色规范槽位只显示规范 logo 和 `角色规范` 四字,绑定来源标题只保留给可访问名称、悬浮 title 和图片信息。已有参考图槽位只有在 hover / focus 时显示右上角 `×`,点击后只解绑对应参考图。角色形象生成面板每次成功绑定角色规范后,在当前编辑器生命周期内缓存为上一张角色规范;再次新建角色形象时自动带入该缓存。图标素材和 UI 设计图面板每次成功绑定图标规范后,同样缓存为上一张图标规范;再次新建需要图标规范的素材时自动带入该缓存。生成规范菜单里的图标规范对象自身只把首行参考图作为可选参考,不要求必须先绑定图标规范。
|
||||
- 所有挂到 `document.body` 的 `.image-canvas-editor__portal-menu` 必须放在同步当前 `platform-theme--light / platform-theme--dark` 的 `.image-canvas-editor__portal-theme` 桥接层下,并与 `.image-canvas-editor` 共用完整的 `--image-canvas-brand-*` token 声明。portal 内的比例 / 清晰度选中态、视频和音频 slider、静音开关、时长文字与模型选中勾选继续消费同一组主题变量;不得用 `#00ff00` 等硬编码颜色绕过变量作用域,也不得只修单个控件而让其它 portal 选中态继续退回无效声明。
|
||||
- 生成规范类图片面板底部必须以禁用态参数按钮显示 `16:9·2K` 和 `gpt-image-2`,视觉对齐可编辑面板参数控件,提交到 `/api/editor/images/generations` 时也固定携带这些参数。
|
||||
- 快速编辑面板底部只显示模型选择和 `修改` 按钮;打开时视口聚焦必须预留底部面板空间,面板位于素材下方,不得遮挡原素材,且素材在当前屏幕内完整可见。快速编辑请求只把原图或红框序号标注图作为 `sourceImageSrc` 直接提交,信息面板输入快照只展示用户填写的快速编辑提示词。
|
||||
- 点击生成、生成规范、生成角色形象或生成图标素材后创建的占位图可继续保留;点击画布空白区域让当前图片或占位图失焦时,关闭当前生成面板并移除图片选中样式,但不删除占位图本身。
|
||||
|
||||
@@ -124,7 +124,11 @@ worker 配置:
|
||||
- `GENARRATIVE_EXTERNAL_GENERATION_WORKER_POLL_INTERVAL_MS`:空队列轮询间隔。
|
||||
- `GENARRATIVE_EXTERNAL_GENERATION_WORKER_LEASE_SECONDS`:任务 lease 时长,默认 `600`;worker 会按约三分之一 lease、最长 30 秒的间隔续租。该值应覆盖一次心跳网络抖动窗口,不需要大于完整外部生成链路耗时。
|
||||
- `GENARRATIVE_EXTERNAL_GENERATION_WORKER_JOB_TIMEOUT_SECONDS`:普通外部生成 job 的执行预算,默认 `900`。超过预算后当前 worker 停止续租并释放 worker 槽位,但不取消已启动的业务 future,也不主动写入失败 / 重试状态;在途执行交由 lease fencing 仲裁:写回在租约有效期内到达则照常完成,否则被拒绝,租约过期后任务可被重新认领,attempt 耗尽时由认领事务原子标记失败并结算退款。
|
||||
- `GENARRATIVE_EXTERNAL_GENERATION_WORKER_LONG_JOB_TIMEOUT_SECONDS`:视频、角色动作等长耗时 job 的执行预算,默认 `1800`。
|
||||
- `GENARRATIVE_EXTERNAL_GENERATION_WORKER_LONG_JOB_TIMEOUT_SECONDS`:VectorEngine 图片生成 / 编辑、图标 spritesheet 生成、UI 素材提取以及角色动作、视频等长耗时 job 的执行预算,默认 `1800`。其中四类 VectorEngine 图片 job 固定为 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation` 和 `editor_ui_design_asset_extraction`;手动去背景等不直接调用 VectorEngine 的 job 继续使用普通预算。
|
||||
|
||||
worker 在单次 job 开始执行时从同一个单调时钟起点计算绝对 `job deadline` 和更早的 `provider deadline`:常规情况下为终态审计、OSS 持久化及 `complete/fail` 回写保留 `60` 秒;当整个 job 预算小于 `120` 秒时,保留其一半,避免 provider 预算被全部吃掉。该 deadline 只通过进程内 `RequestContext` 传给 VectorEngine 图片调用,不写入 HTTP DTO、队列 payload 或 SpacetimeDB;普通 HTTP / `inline` 上下文没有 deadline,保持原有行为。
|
||||
|
||||
VectorEngine 每次发送的实际 timeout 取 `min(VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS, provider 剩余预算)`。遇到可重试传输错误或 408 / 429 / 5xx 时,只有当剩余预算还容得下本次退避和下一次 attempt 才继续;否则立即停止重试并返回当前 provider 错误,deadline 耗尽时返回 timeout。同一绝对 deadline 同时覆盖参考图下载、provider 请求 / 响应和响应图片 URL 下载,不允许请求已返回后的图片下载越过 provider 预算。`VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 默认仍为 `1000000`;配置加载层允许显式值低于默认值,不再在读取环境变量时强制抬升。
|
||||
|
||||
controller 配置:
|
||||
|
||||
@@ -136,7 +140,7 @@ controller 配置:
|
||||
- `GENARRATIVE_EXTERNAL_GENERATION_CONTROLLER_SERVICE_TEMPLATE`:systemd worker 模板,默认 `genarrative-external-generation-worker@{}.service`。
|
||||
- `GENARRATIVE_EXTERNAL_GENERATION_CONTROLLER_DRY_RUN`:只记录决策不执行 systemctl,默认 `false`。
|
||||
|
||||
动态缩扩容方式:生产默认由 `deploy/systemd/genarrative-external-generation-controller.service` 启动 `GENARRATIVE_PROCESS_ROLE=external-generation-controller`,controller 读取 `get_external_generation_queue_stats_and_return` 后对 `genarrative-external-generation-worker@N.service` 执行精确 `systemctl start/stop`;无需改变 HTTP 进程数。controller 只操作 `@1..@MAX` 中的缺口或最高编号多余实例,保留 `@1` 作为保底 worker。缩容或发布重启 worker 时,进程收到 SIGINT/SIGTERM 后会停止 claim 新任务并等待当前任务完成;若进程被硬杀、机器断电或超过 systemd `TimeoutStopSec`,未完成任务会在 lease 过期后被其它 worker 重新领取。若 worker 内业务 future 长时间无返回,执行预算到期后 worker 会停止续租并释放槽位,在途 future 继续运行至租约仲裁窗口;有效租约内的写回仍可完成,租约过期后才会由其它 worker 重新认领,避免客户端取消与服务端写回竞态。容器链路已有独立 `external-generation-worker` compose service;扩 worker 必须扩这个 worker service,不能只扩 `api-server` HTTP service。
|
||||
动态缩扩容方式:生产默认由 `deploy/systemd/genarrative-external-generation-controller.service` 启动 `GENARRATIVE_PROCESS_ROLE=external-generation-controller`,controller 读取 `get_external_generation_queue_stats_and_return` 后对 `genarrative-external-generation-worker@N.service` 执行精确 `systemctl start/stop`;无需改变 HTTP 进程数。controller 只操作 `@1..@MAX` 中的缺口或最高编号多余实例,保留 `@1` 作为保底 worker。缩容或发布重启 worker 时,进程收到 SIGINT/SIGTERM 后会停止 claim 新任务并等待当前任务完成;若进程被硬杀、机器断电或超过 systemd `TimeoutStopSec`,未完成任务会在 lease 过期后被其它 worker 重新领取。VectorEngine 图片链路会先于整个 job 执行预算停止 provider 发送 / 重试,以便 worker 在有效 lease 内完成终态写回;若其他业务 future 仍长时间无返回,执行预算到期后 worker 会停止续租并释放槽位,在途 future 继续运行至租约仲裁窗口;有效租约内的写回仍可完成,租约过期后才会由其它 worker 重新认领,避免客户端取消与服务端写回竞态。本次预算收口不改变 lease 续租 / fencing、迟到写回仲裁、attempt 耗尽收口和原子退款语义。容器链路已有独立 `external-generation-worker` compose service;扩 worker 必须扩这个 worker service,不能只扩 `api-server` HTTP service。
|
||||
|
||||
## 已接入的拼图纵切
|
||||
|
||||
|
||||
@@ -80,7 +80,7 @@ npm run check:server-rs-ddd
|
||||
- 画布 Agent 工具复用既有编辑器图片生成 / 修改 / 图标 spritesheet BFF,并继续使用后端模型定价和 `execute_billable_asset_operation_with_cost`;前端不提交 `priceMudPoints`。
|
||||
- `/messages/{messageId}/confirm` 与 `/messages/{messageId}/cancel` 只返回成功确认;前端成功后立即重新读取整个会话,以会话详情中的权威消息状态和 `externalJobId` 驱动气泡展示与任务轮询。
|
||||
- 会话详情的终态懒回填必须在单次 GET 和同一 conversation lock 内完成有界重试:任务结果读取、completed payload 解析或工具 formatter 首次失败后最多重试 3 次,每次等待 100ms 并重新读取主任务。任务读取失败或 completed 任务暂缺 `result_payload_json` 时,本次重试耗尽后仍保留 OSS 工具消息的 `not_completed + externalJobId`,由下次会话读取继续 reconcile;JSON 损坏、结果结构不兼容或 formatter 失败等确定性致命错误在重试耗尽后原子写为 `failed`,保存“重试 3 次后仍失败”的最后错误,避免永久循环。
|
||||
- 画布 Agent 是“正式任务 payload 不进入通用用户 read model”规则的窄例外消费者:`GET /api/editor/agent-conversations/{conversationId}` 只按会话中已有的 `externalJobId` 定向读取主任务,完成后由对应工具 formatter 从 `result_payload_json` 提取并归一有界的图片 / 视频 / 音频引用,写入 OSS 工具消息后返回。前端仍不得通过通用任务列表 / 状态接口读取或解析 `request_payload_json` / `result_payload_json`;OSS 轻量媒体引用只是会话展示与后续 Agent 上下文,不替代 `editor_project_resource`、`editor_asset`、`editor_canvas.layers_json` 或 `external_generation_job` 的业务真相。
|
||||
- 画布 Agent 是“正式任务 payload 不进入通用用户 read model”规则的窄例外消费者:`GET /api/editor/agent-conversations/{conversationId}` 只按会话中已有的 `externalJobId` 定向读取主任务,完成后由对应工具 formatter 从 `result_payload_json` 提取并归一有界的图片 / 视频 / 音频引用,写入 OSS 工具消息后返回。前端仍不得通过通用任务列表 / 状态接口读取或解析 `request_payload_json` / `result_payload_json`;OSS 轻量媒体引用只是会话展示与后续 Agent 上下文,不替代 `editor_project_resource`、`editor_asset`、结构化画布表或 `external_generation_job` 的业务真相。未激活结构化存储的 canvas 才继续以 `editor_canvas.layers_json` 作为 legacy 布局真相。
|
||||
|
||||
### 创作 / 游玩统一流程主干
|
||||
|
||||
@@ -284,7 +284,8 @@ npm run check:server-rs-ddd
|
||||
- Rust 结构体:`ExternalGenerationJob`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/external_generation.rs`
|
||||
- 现役覆盖:worker claim 只允许 `source_module = editor-canvas`;下述逐玩法生成和写回描述均为退役前历史。历史 pending / running 行继续保留原状态,不得领取、失败收口或改写 payload。
|
||||
- 用途:外部生成 worker 的内部持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写阶段、完成 / 失败。队列行继续保存 worker 执行、计费与滚动发布兼容所需字段,末尾可选 `phase` 只取 `generating / processing`;claim 写 `generating`,真实进入抠图处理时由受 `job_id + worker_id + lease_token` 保护的 procedure 写 `processing`。phase procedure 以结构化结果区分 `LeaseFencingRejected` 与 `OtherRejected`;`LeaseFencingRejected` 立即终止,`OtherRejected` 以及 SDK 的 `Procedure` / `Runtime` 错误不重试,只有 `Build` / `ConnectDropped` / `Timeout` 在同一个 job attempt 内重试一次。该重试只重新上报 phase,不把任务写回 `pending`,也不重新调用 provider;编辑器 job 入队固定 `max_attempts=1`,第二次传输失败后任务进入 `failed`,不会回到 `pending` 或从 provider 生成起点重跑。用户可见任务列表、价格、状态、阶段、未确认终态数量和通知确认时间的正式读取事实源已经迁到 `external_generation_job_summary`;BFF 不得再为列表 / 详情 / acknowledge 读取该大表。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile;图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表,worker 成功后经 `api-server` facade 写入 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`,前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
|
||||
- 用途:外部生成 worker 的内部持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写阶段、完成 / 失败。队列行继续保存 worker 执行、计费与滚动发布兼容所需字段,末尾可选 `phase` 只取 `generating / processing`;claim 写 `generating`,真实进入抠图处理时由受 `job_id + worker_id + lease_token` 保护的 procedure 写 `processing`。phase procedure 以结构化结果区分 `LeaseFencingRejected` 与 `OtherRejected`;`LeaseFencingRejected` 立即终止,`OtherRejected` 以及 SDK 的 `Procedure` / `Runtime` 错误不重试,只有 `Build` / `ConnectDropped` / `Timeout` 在同一个 job attempt 内重试一次。该重试只重新上报 phase,不把任务写回 `pending`,也不重新调用 provider;编辑器 job 入队固定 `max_attempts=1`,第二次传输失败后任务进入 `failed`,不会回到 `pending` 或从 provider 生成起点重跑。用户可见任务列表、价格、状态、阶段、未确认终态数量和通知确认时间的正式读取事实源已经迁到 `external_generation_job_summary`;BFF 不得再为列表 / 详情 / acknowledge 读取该大表。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile;图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表。结构化 canvas 激活后,当前 worker completion 先以读取时 canvas revision 执行 CAS,并发冲突时拒绝覆盖并保留可诊断失败;目标是进一步收口为受 lease 栅栏保护的单事务幂等写入 `editor_project_resource`、结果 `editor_canvas_layer`、`editor_canvas_generation_dialog` 终态和 canvas revision。未激活 canvas 在 2 MiB 上限内继续走 legacy `editor_canvas.layers_json` 兼容写回。前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
|
||||
- 素材写回:worker 成功后仍经 `api-server` facade 写入 `editor_project_resource` / `editor_asset`;结构化 canvas 的 layer / dialog / revision 与未激活 canvas 的 legacy `layers_json` 分流按上一条执行,前端不直接发明正式完成态。
|
||||
- 载荷约束:本次先对 `source_module = editor-canvas` 的 `request_payload_json` / `result_payload_json` 实施有限大小合法 JSON、任意层级禁止 `data:` / `blob:` 的双层门禁,只保存 worker 执行必需的普通参数和已登记媒体引用。画布 Agent 来源的任务可在 `result_payload_json.editor-agent-tool-call-result` 中保存有界的轻量结果和已登记媒体引用,供后端按已有 `externalJobId + owner_user_id` 定向懒回填;其它编辑器任务保持元数据结果,并可保存有界的 `warning.code/reason`。其它玩法在完成各自参考图资源化之前不由本次门禁静默改变既有请求契约。该主表只供 worker claim / 执行、受控维护以及画布 Agent 的定向结果回填读取;正式用户任务列表、单任务状态、队列概览与 acknowledge 不得返回或解析这两个 payload。画布 Agent 懒回填必须经对应工具 formatter 归一为有界轻量媒体引用后写入 OSS 会话,不能把原始 payload 直接透传前端。
|
||||
- 非阻断告警:角色形象、图标图集和 UI 素材提取已保存 provider 原图、但透明背景处理最终失败时,以原图唯一主图完成任务;透明图和切片不写入画布。这个 source-only 降级只包住透明背景处理的最终失败,phase 上报、provider 原图持久化、透明处理图持久化或画布写回失败仍按任务错误传播。图标 / UI 透明图集成功但自动拆分降级时仍保留透明图集;通用 `warning` 与 `sliceWarning` 互斥。两类成功降级都以既有 `completed` 状态收口,不新增状态值:source-only 的 inline / external v1 响应使用结构化 `warning.code/reason`,仅拆分失败的 inline / external v1 响应继续使用既有 `sliceWarning.code/reason`,其 `reason` 保留原始诊断;queue worker 才把两者归一为有界的 `result_payload_json.warning`,且通用 `warning` 优先并原样保留完整 `reason`,只有 `sliceWarning.reason` 由 worker 添加“图集已生成,但自动拆分未完成:”前缀。除上述画布 Agent 定向回填的轻量结果外,队列结果不保存图片、切片列表或媒体 URL。
|
||||
|
||||
@@ -528,21 +529,42 @@ npm run check:server-rs-ddd
|
||||
|
||||
- Rust 结构体:`EditorProject`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
|
||||
- 说明:图片画布工程真相表,保存 owner、标题和工程时间戳;viewport 与图层布局已拆到 `editor_canvas`,旧 layout columns 暂作为兼容列保留,不再作为权威数据源。只通过 `/api/editor/projects*`、`/api/editor/projects/{projectId}/agent-conversations`、`/api/editor/agent-conversations/{conversationId}*` BFF 和 `spacetime-client` facade 读写;项目页列表、重命名和删除也使用该能力,删除工程时级联清理默认画布和资源元数据。
|
||||
- 说明:图片画布工程真相表,保存 owner、标题和工程时间戳;viewport 与画布数据已拆到 `editor_canvas` 及其结构化子表,旧 project layout columns 暂作为兼容列保留,不再作为权威数据源。只通过 `/api/editor/projects*`、`/api/editor/projects/{projectId}/agent-conversations`、`/api/editor/agent-conversations/{conversationId}*` BFF 和 `spacetime-client` facade 读写;项目页列表、重命名和删除也使用该能力,删除工程时级联清理默认画布、结构化画布行、迁移状态和资源元数据。
|
||||
- 索引:`by_editor_project_owner_user_id` 用于读取当前用户最近编辑工程和项目页工程列表。
|
||||
|
||||
### `editor_canvas`
|
||||
|
||||
- Rust 结构体:`EditorCanvas`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
|
||||
- 说明:图片画布数据表,归属于 `editor_project`,保存默认画布的 viewport typed columns、图层布局 JSON、owner 和时间戳;当前编辑器读取 / 保存 project 的默认 canvas,后续支持一个工程多个 canvas。
|
||||
- 说明:图片画布根表,归属于 `editor_project`,保存默认画布的 viewport typed columns、owner、时间戳与当前 revision;当前编辑器读取 / 保存 project 的默认 canvas,后续支持一个工程多个 canvas。`layers_json` 继续保留,legacy canvas 在统一 **2 MiB** 上限内以它作为布局真相;`editor_canvas_layout_migration` 激活 structured 后,图层和生成对话框分别以 `editor_canvas_layer`、`editor_canvas_generation_dialog` 为权威,`layers_json` 只作为存量迁移输入和受限回滚载体。所有结构化 mutation 必须以 `expected_revision` 做 CAS,成功事务只递增一次 revision。
|
||||
- 索引:`by_editor_canvas_project_id`、`by_editor_canvas_owner_user_id`。
|
||||
|
||||
### `editor_canvas_layer`
|
||||
|
||||
- Rust 结构体:`EditorCanvasLayer`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
|
||||
- 说明:图片画布结构化图层表,一行保存一个 layer 的 canvas / project / owner 归属、几何、层级顺序、分组、hidden / locked / flip 状态与 `resource_id`。V1 的 `item_json` 只保留未结构化扩展字段,单行最大 512 KiB;快照以 typed 列重组,不得把完整图层 JSON 当作平行真相。媒体业务真相仍由 `editor_project_resource` / `editor_asset` 持有。历史 layout 的 `sourceResourceId == resourceId` 属于无意义自引用,迁移时按资源表真相剥离;其他资源字段冲突继续 fail-closed。唯一存量缺资源例外是已缺资源行、但帧与预览均为稳定站内对象路径的 `local-* + generated + image-sequence` 历史角色动作图层:迁移保留其有界媒体扩展并纳入 canonical hash;active 后只能续存同一行且扩展不可变,不能新增或篡改。其他缺资源图层继续 fail-closed。当前站内写入通过 revision CAS 把兼容布局事务性拆成行;后续再将新增、移动、缩放、删除、重排和分组收窄为有界 batch mutation。
|
||||
- 索引:按 canvas 读取完整结构化快照,按 project 做级联清理;owner 保留在行内用于归属校验。同一 canvas 的 layer id 必须稳定且唯一。
|
||||
|
||||
### `editor_canvas_generation_dialog`
|
||||
|
||||
- Rust 结构体:`EditorCanvasGenerationDialog`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
|
||||
- 说明:图片画布生成对话框表,保存 canvas / project / owner、生成模式、状态、可选 source / generated layer、占位几何和有界扩展字段;typed 列为快照真相,`dialog_json` 只保留未结构化参数。当前 worker completion 以读取时 revision 调用 CAS 保存,冲突时拒绝覆盖;`job_id + worker_id + lease_token` 栅栏下的资源 / layer / dialog / job 单事务完成仍是后续收口。
|
||||
- 索引:按 canvas 和 project 读取结构化行;当前未建立 external job 二级索引。
|
||||
|
||||
### `editor_canvas_layout_migration`
|
||||
|
||||
- Rust 结构体:`EditorCanvasLayoutMigration`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
|
||||
- 说明:每个 canvas 一行的结构化持久化控制与迁移审计表,保存 schema version、`backfilled / active / rolled_back` 状态、最后校验 revision、legacy / structured canonical hash、layer / dialog 数量、资源引用集合 hash 和激活 / 回滚时间。存量迁移固定执行幂等 `backfill → hash / 数量 / 资源引用核对 → revision CAS activate`;active 重入和 rollback 同样重新核对,不只按状态早返。回滚结果还必须不超过 2 MiB。三个 procedure 只允许 database migration operator 调用;`npm run spacetime:editor-canvas-layout:migrate` 默认 dry-run,显式 `--apply` 才写入。
|
||||
- 索引:`canvas_id` 唯一,一切模式切换仅允许受限迁移 procedure 执行;前端不得提交或推断存储模式。
|
||||
|
||||
### `editor_project_resource`
|
||||
|
||||
- Rust 结构体:`EditorProjectResource`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
|
||||
- 说明:图片画布工程资源元数据表,保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task、源资源关系、`asset_kind`、`generation_inputs_json` 和历史 `public_showcase_enabled`。`asset_kind` 标记角色、图标、UI 设计图、视频、音频等素材类别;`generation_inputs_json` 保存用户可见生成输入快照,供图片信息页刷新后恢复。`public_showcase_enabled` 只保留旧接口兼容,不再作为 `/creation` 的 `陶泥儿精选` 事实源;精选公开改由账号级生成素材提交 `editor_showcase_asset` 审核决定。图片 / 图标 / UI 提取等生成 BFF 在请求携带 `project_id` 时负责创建该表记录并把 resource 快照返回前端;前端只保存布局引用,不能把同一生成结果再次作为正式业务真相写入。项目封面快照也落在该表,使用 `asset_kind = project-cover-snapshot`、`source_type = uploaded` 和私有 OSS / asset object 引用,代表画布当前视口栅格化后的静态封面;项目列表和创作主页最近项目只读取最新封面快照资源,不在列表页根据 `editor_canvas.layers_json` 临时拼画布。从账号级素材库把同一生成素材拖回同一项目画布时,后端优先复用同项目内同源同媒体资源,避免每个图层实例都插入新的资源行。账号级素材删除不级联删除该表,避免历史画布丢图。`editor_canvas.layers_json` 只保存图层几何、层级、分组、资源引用和生成器对象;新写入不再把素材生成输入快照作为图层布局真相保存,旧布局字段只作为兼容兜底读取。
|
||||
- 说明:图片画布工程资源元数据表,保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task、源资源关系、`asset_kind`、`generation_inputs_json` 和历史 `public_showcase_enabled`。`asset_kind` 标记角色、图标、UI 设计图、视频、音频等素材类别;`generation_inputs_json` 保存用户可见生成输入快照,供图片信息页刷新后恢复。`public_showcase_enabled` 只保留旧接口兼容,不再作为 `/creation` 的 `陶泥儿精选` 事实源;精选公开改由账号级生成素材提交 `editor_showcase_asset` 审核决定。图片 / 图标 / UI 提取等生成 BFF 在请求携带 `project_id` 时负责创建该表记录并把 resource 快照返回前端;前端只保存稳定 `resource_id` 布局引用,不能把同一生成结果再次作为正式业务真相写入。项目封面快照也落在该表,使用 `asset_kind = project-cover-snapshot`、`source_type = uploaded` 和私有 OSS / asset object 引用,代表画布当前视口栅格化后的静态封面;项目列表和创作主页最近项目只读取最新封面快照资源,不在列表页根据 layout 临时拼画布。从账号级素材库把同一生成素材拖回同一项目画布时,后端优先复用同项目内同源同媒体资源,避免每个图层实例都插入新的资源行。账号级素材删除不级联删除该表,避免历史画布丢图。结构化 canvas 的几何、层级、分组和资源引用以 `editor_canvas_layer` 为权威,生成器对象以 `editor_canvas_generation_dialog` 为权威;legacy canvas 才在 2 MiB 上限内从 `editor_canvas.layers_json` 兼容读取。新写入不再把素材生成输入快照作为图层布局真相保存。历史普通图层缺资源只能由 migration operator 调用 `repair_editor_canvas_resources_and_return` 定向修复:procedure 每次只处理一个尚无迁移记录的 legacy canvas,校验 owner/project、revision、canvas/project 两份 raw layout SHA-256、精确 layer/resource/sourceResourceId、同工程替换资源与 private asset_object 谱系;图片只替换引用,音频只恢复经核验的 `420x120` 项目资源行。运维入口 `npm run spacetime:editor-canvas-resources:repair` 默认 dry-run,apply 必须绑定 plan SHA-256 并在成功后自动复核 already-repaired,禁止手工 SQL 绕过事务 guard。
|
||||
- 索引:`by_editor_project_resource_project_id`、`by_editor_project_resource_owner_user_id`。
|
||||
|
||||
### `editor_asset_folder`
|
||||
|
||||
@@ -63,7 +63,7 @@ HTTP 角色的 `GENARRATIVE_SPACETIME_POOL_SIZE` 只表示 procedure / reducer
|
||||
|
||||
生产拆分角色时,`external-generation-worker` 和 `external-generation-controller` 的专属 env 示例会把 `GENARRATIVE_SPACETIME_POOL_SIZE` 覆盖为 `1`;非 HTTP 角色不创建 API 缓存读连接,只保留 `external_generation_job` 队列窄订阅作为响应式唤醒信号,实际抢占和扩缩容判断仍走 SpacetimeDB procedure。worker / controller 不执行模型定价 seed,启动时先调用受 runtime writer 鉴权的 queue-stats procedure 做只读预检,身份不匹配时 fail-fast;当前正式 systemd unit 通过共同加载 `/etc/genarrative/api-server.env` 继承同一 `GENARRATIVE_SPACETIME_TOKEN`,专属角色 env 示例不重复配置该 token。`GENARRATIVE_EXTERNAL_GENERATION_WORKER_POLL_INTERVAL_MS` 与 controller poll interval 只作为订阅失效、漏事件和 lease 过期这类时间条件的兜底,不作为正常领取任务的主路径。
|
||||
|
||||
生产 worker 默认 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_LEASE_SECONDS=600`,只覆盖 worker 心跳抖动和短暂断连窗口,不再把 lease 当成完整任务时长;默认 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_JOB_TIMEOUT_SECONDS=900`,角色动画 / 视频类长任务使用 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_LONG_JOB_TIMEOUT_SECONDS=1800`。worker 在单次尝试超过执行预算后会停止续租并释放 worker 槽位,但不会取消已启动的业务 future 或主动写入失败 / 重试状态;在途执行由 lease fencing 仲裁,有效租约内写回仍可完成,租约过期后任务才可重新领取,attempt 耗尽时由认领事务标记失败并结算退款。生产部署和 provision 脚本会给 `/etc/genarrative/api-server.env` 与 `/etc/genarrative/external-generation-worker.env` 补齐这些变量;已有自定义值不覆盖,只会把历史旧默认 `3600` 迁移为 `600`。
|
||||
生产 worker 默认 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_LEASE_SECONDS=600`,只覆盖 worker 心跳抖动和短暂断连窗口,不再把 lease 当成完整任务时长;默认 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_JOB_TIMEOUT_SECONDS=900`。`editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction` 四类 VectorEngine 图片任务与角色动画 / 视频类长任务使用 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_LONG_JOB_TIMEOUT_SECONDS=1800`,手动去背景、音效和背景音乐继续使用普通预算。worker 在单次尝试超过执行预算后会停止续租并释放 worker 槽位,但不会取消已启动的业务 future 或主动写入失败 / 重试状态;在途执行由 lease fencing 仲裁,有效租约内写回仍可完成,租约过期后任务才可重新领取,attempt 耗尽时由认领事务标记失败并结算退款。生产部署和 provision 脚本会给 `/etc/genarrative/api-server.env` 与 `/etc/genarrative/external-generation-worker.env` 补齐这些变量;已有自定义值不覆盖,只会把历史旧默认 `3600` 迁移为 `600`。
|
||||
|
||||
lease 过期后不代表任务一定再次执行:claim transaction 只有在 `attempt < max_attempts` 时才会递增 attempt 并返回 worker;如果过期的是最终 attempt,则直接把 job 收口为 `failed`、清理 lease,并按入队冻结价格为当前 attempt 原子退款或写 cancellation intent。该终态任务不会再次进入 provider executor,迟到 consume 会被 settlement intent 拒绝。
|
||||
|
||||
@@ -153,9 +153,9 @@ spacetime sql <database> "SELECT * FROM runtime_setting LIMIT 1" --server http:/
|
||||
|
||||
本地 `spacetime` CLI / standalone 版本必须和 `server-rs/Cargo.toml` 里锁定的 `spacetimedb` 版本一致;当前统一版本为 `2.6.1`。若版本错配,procedure 返回值可能在宿主侧触发 `Failed to BSATN deserialize procedure return value`,api-server 最终表现为现役 settings、editor project 或 profile procedure 超时。排障时先运行 `spacetime --version`,再对照 `server-rs/Cargo.toml` 的 `spacetimedb = "..."`;遇到版本不匹配时直接执行 `spacetime version install <version> && spacetime version use <version>`,或在目标就是最新版本时执行 `spacetime version upgrade`,升级后重启 `npm run dev:spacetime` 再重试。当前 `scripts/dev.mjs` 会在启动和复用本地 SpacetimeDB 前写入并校验 `dev-spacetime-tool-version`。2.6.1 修复了 procedure context 中调用者 `Identity` / `ConnectionId` 始终为空的回归,依赖 `ctx.sender` 鉴权时必须同时确认宿主已升级。
|
||||
|
||||
本地 `.env`、`.env.local` 或 `.env.secrets.local` 修改后必须重启 `api-server` 才会生效;若已经通过 `npm run dev` 启动完整联调,可在该终端输入 `rs api-server`。排查图片编辑器 VectorEngine 生成链路时,确认 `VECTOR_ENGINE_BASE_URL`、`VECTOR_ENGINE_API_KEY` 和 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 只在本地或服务器密钥文件中配置,不能写入 Git。VectorEngine `gpt-image-2` 图片协议、URL / base64 响应解析、远端图片下载和 provider 侧结构化日志在 `server-rs/crates/platform-image`;`api-server` 只做编辑器请求编排、OSS / asset 持久化、计费和失败审计落库。`platform-image` 会在 JSON 生成和 multipart 编辑请求发送前归一显式像素尺寸;若请求发送失败,先按同一 `request_id` 查看 provider 日志与 `external_api_call_failure.metadata_json.errorSource`,当前 multipart `/v1/images/edits` 单独强制 HTTP/1.1。
|
||||
本地 `.env`、`.env.local` 或 `.env.secrets.local` 修改后必须重启 `api-server` 才会生效;若已经通过 `npm run dev` 启动完整联调,可在该终端输入 `rs api-server`。排查图片编辑器 VectorEngine 生成链路时,确认 `VECTOR_ENGINE_BASE_URL`、`VECTOR_ENGINE_API_KEY` 和 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 只在本地或服务器密钥文件中配置,不能写入 Git。`VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 是单次 attempt 的配置上限,默认 `1000000`;配置加载层允许显式值低于该默认值,不再在读取环境变量时强制抬高。VectorEngine `gpt-image-2` 图片协议、URL / base64 响应解析、远端图片下载和 provider 侧结构化日志在 `server-rs/crates/platform-image`;`api-server` 只做编辑器请求编排、OSS / asset 持久化、计费和失败审计落库。`platform-image` 会在 JSON 生成和 multipart 编辑请求发送前归一显式像素尺寸;若请求发送失败,先按同一 `request_id` 查看 provider 日志与 `external_api_call_failure.metadata_json.errorSource`,当前 multipart `/v1/images/edits` 单独强制 HTTP/1.1。
|
||||
|
||||
VectorEngine 图片生成 / 编辑在 `request_send` 阶段出现 `timeout`、`connect`、libcurl 35 SSL connect reset、libcurl 56 receive error / `unexpected eof while reading`、recv failure 等临时传输错误,或在 `upstream_status` 阶段收到 408 / 429 / 5xx(例如 Nginx HTML `502 Bad Gateway`)时,`platform-image` 会对同一请求最多发送 5 次;multipart 图片编辑每次重试都会重新构造 form,避免复用已消费的 body。日志中 `VectorEngine 图片请求发送失败,准备重试` 或 `VectorEngine 图片上游状态可重试,准备重试` 表示本次失败已进入下一次尝试;最终仍失败时才会写入 `external_api_call_failure` 并返回 504 / 502。排查生产失败时应同时统计 retry 前的尝试日志和最终 audit,避免把一次用户请求内的多次发送误判成多个用户请求。
|
||||
VectorEngine 图片生成 / 编辑在 `request_send` 阶段出现 `timeout`、`connect`、libcurl 35 SSL connect reset、libcurl 56 receive error / `unexpected eof while reading`、recv failure 等临时传输错误,或在 `upstream_status` 阶段收到 408 / 429 / 5xx(例如 Nginx HTML `502 Bad Gateway`)时,`platform-image` 会对同一请求最多发送 5 次;multipart 图片编辑每次重试都会重新构造 form,避免复用已消费的 body。worker 从 job 开始的同一时钟起点计算绝对 deadline,常规保留最后 `60` 秒给审计、OSS 和终态写回;job 预算小于 `120` 秒时保留一半。VectorEngine 单次 attempt timeout 取配置值和剩余 provider 预算的较小值;退避后已没有下一次 attempt 的预算时立即停止重试。该 deadline 覆盖参考图、provider 请求 / 响应和响应图片下载的整次 provider future,但只在 worker 进程内通过 `RequestContext` 传递;普通 HTTP / `inline` 没有该 deadline,继续保持原有 timeout 和重试行为。日志中 `VectorEngine 图片请求发送失败,准备重试` 或 `VectorEngine 图片上游状态可重试,准备重试` 表示本次失败确有预算进入下一次尝试;预算耗尽或最终仍失败时才会写入 `external_api_call_failure` 并返回 504 / 502。排查生产失败时应同时统计 retry 前的尝试日志和最终 audit,避免把一次用户请求内的多次发送误判成多个用户请求。这项收口不修改 lease 续租 / fencing、迟到写回仲裁、attempt 耗尽与原子退款语义。
|
||||
|
||||
图片编辑器生成属于持久队列长任务:提交接口返回 job 后,前端通过 `/api/runtime/external-generation/jobs/{jobId}` 与编辑器项目资源状态收敛。生产排查小程序或 WebView `Failed to fetch` 时,若 Nginx access log 为 `499`、`upstream_status=-`,先按提交请求的 `request_id`、job id、worker 日志和 `external_api_call_failure` 对齐真实任务,不把客户端断开直接判定为 provider 失败。
|
||||
|
||||
@@ -524,7 +524,7 @@ Pingora current release 自审脚本 `scripts/ops/pingora-current-release-audit.
|
||||
|
||||
同一 API release 随包依赖还必须包含 `scripts/check-pingora-release-readiness.mjs` 与 `scripts/check-pingora-canary-live.mjs`。前者在 current release 上以 `--release-runtime-only` 汇总运行时复核,后者支撑目标 Nginx canary live smoke;缺少任一脚本时不能进入直连切换窗口。
|
||||
|
||||
`Genarrative-Stdb-Module-Build` 的 Jenkins 归档产物必须包含 `build/<version>/spacetime_module.wasm`、`spacetime_module.wasm.sha256`、`release-manifest.json`、`scripts/deploy/production-stdb-publish.sh`、`scripts/deploy/production-runtime-writer-identity-rotate.mjs`、`scripts/deploy/maintenance-on.sh`、`scripts/deploy/maintenance-off.sh`、`scripts/spacetime-migration-common.mjs` 和 `scripts/database-backup-to-oss.mjs`,不得包含 `migration-bootstrap-secret.txt` 或任何原始 bootstrap secret。`Genarrative-Stdb-Module-Build` 只接受 `MIGRATION_BOOTSTRAP_SECRET_CREDENTIAL_ID` 指向的受保护 Jenkins Secret File:构建 shell 从临时文件读取原始值,强制校验为 64 位十六进制,计算 SHA-256,随后只通过 `GENARRATIVE_SPACETIME_MIGRATION_BOOTSTRAP_SECRET_SHA256` 注入 Rust 编译;WASM 因而只包含摘要,不包含可下载的原文,Stdb `release-manifest.json` 以 `migration_bootstrap_secret_sha256` 记录该非敏感摘要。`Genarrative-Stdb-Module-Publish` 只通过 `copyArtifacts` 复制上述非敏感产物,不在目标机器 checkout Git,并在发布阶段用同一个凭据 ID 再次挂载 Secret File;publish 必须再次校验 64 位十六进制、重算 SHA-256,并与 manifest 的 `migration_bootstrap_secret_sha256` 强制匹配后才可发布。Full Build 必须保证 Stdb Build / Publish 的 `MIGRATION_BOOTSTRAP_SECRET_CREDENTIAL_ID` 完全相同并把同一个 ID 同时透传,不能从构建 artifact 传 secret;ID 不同、manifest 缺摘要或摘要不匹配都必须在发布前失败。
|
||||
`Genarrative-Stdb-Module-Build` 的 Jenkins 归档产物必须包含 `build/<version>/spacetime_module.wasm`、`spacetime_module.wasm.sha256`、`release-manifest.json`、`scripts/deploy/production-stdb-publish.sh`、`scripts/deploy/production-runtime-writer-identity-rotate.mjs`、`scripts/deploy/maintenance-on.sh`、`scripts/deploy/maintenance-off.sh`、`scripts/spacetime-migration-common.mjs`、`scripts/spacetime-maintain-external-generation-jobs.mjs`、`scripts/spacetime-migrate-editor-canvas-layout.mjs` 和 `scripts/database-backup-to-oss.mjs`,不得包含 `migration-bootstrap-secret.txt` 或任何原始 bootstrap secret。`Genarrative-Stdb-Module-Build` 只接受 `MIGRATION_BOOTSTRAP_SECRET_CREDENTIAL_ID` 指向的受保护 Jenkins Secret File:构建 shell 从临时文件读取原始值,强制校验为 64 位十六进制,计算 SHA-256,随后只通过 `GENARRATIVE_SPACETIME_MIGRATION_BOOTSTRAP_SECRET_SHA256` 注入 Rust 编译;WASM 因而只包含摘要,不包含可下载的原文,Stdb `release-manifest.json` 以 `migration_bootstrap_secret_sha256` 记录该非敏感摘要。`Genarrative-Stdb-Module-Publish` 只通过 `copyArtifacts` 复制上述非敏感产物,不在目标机器 checkout Git,并在发布阶段用同一个凭据 ID 再次挂载 Secret File;publish 必须再次校验 64 位十六进制、重算 SHA-256,并与 manifest 的 `migration_bootstrap_secret_sha256` 强制匹配后才可发布。Full Build 必须保证 Stdb Build / Publish 的 `MIGRATION_BOOTSTRAP_SECRET_CREDENTIAL_ID` 完全相同并把同一个 ID 同时透传,不能从构建 artifact 传 secret;ID 不同、manifest 缺摘要或摘要不匹配都必须在发布前失败。
|
||||
|
||||
三个 SCM Jenkinsfile 将 `MIGRATION_BOOTSTRAP_SECRET_CREDENTIAL_ID` 默认固定为 `genarrative-spacetime-bootstrap-secret-dev-file`。Secret File 的原文只存在于 Jenkins Credentials;credential ID、参数默认值和定时 / 发布行为以仓库 Jenkinsfile 为事实源,不能只改 Job UI,因为 Declarative Pipeline 下一次载入会重写参数定义。旧 Secret Text `genarrative-spacetime-bootstrap-secret-dev` 继续保留给 Database Import / Export,不得原地改类型或删除。
|
||||
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
# 图片画布结构化持久化与迁移回滚方案
|
||||
|
||||
## 1. 背景与目标
|
||||
|
||||
图片画布当前把全部图层和生成对话框整体序列化到 `editor_canvas.layers_json`。该字段接近原 256 KiB 上限时,资源登记仍可成功,而布局保存会返回 `413`;任务完成后重新读取后端旧快照,会把前端尚未持久化的参考图和生成结果从画布移除。
|
||||
|
||||
本方案分两步处理:
|
||||
|
||||
1. 先把 `api-server` 与 SpacetimeDB 的 legacy layout 校验统一提高到 **2 MiB**,为存量画布止血。HTTP 路由的 body envelope 必须覆盖 2 MiB payload 及 JSON 包装开销,不能只修改领域常量。
|
||||
2. 再把画布拆为结构化的 layer、generation dialog、layout migration/state 三张表;`editor_canvas.layers_json` 保留为旧数据、迁移输入和受限回滚载体,不再作为激活结构化存储后的权威读取来源。
|
||||
|
||||
2 MiB 是迁移窗口内的临时兼容上限,不是继续扩大整体 JSON 的长期容量方案。媒体正文、Data URL、生成输入快照仍不得进入画布布局;媒体和生成元数据继续以 `editor_project_resource` / `editor_asset` 为真相源。
|
||||
|
||||
## 2. 权威边界
|
||||
|
||||
结构化模式激活后,画布事实分工如下:
|
||||
|
||||
| 数据 | 权威位置 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| 工程归属、默认画布、viewport、当前 revision | `editor_project` / `editor_canvas` | `editor_canvas` 保留 `layers_json` legacy 列,但其内容不参与结构化模式下的正常写入仲裁 |
|
||||
| 图层实例、几何、层级、分组、显示与锁定状态、资源引用 | `editor_canvas_layer` | 一行一个图层;模式专属的有界扩展字段可放 JSON,不能复制媒体正文或完整资源元数据 |
|
||||
| 生成对话框、占位层、来源层、结果层、任务与状态 | `editor_canvas_generation_dialog` | 生成任务和 UI 对话框的持久关联,不再嵌入全量 layout JSON |
|
||||
| 存储模式、迁移阶段、校验 hash、激活与回滚审计 | `editor_canvas_layout_migration` | 每个 canvas 一行,控制 legacy / structured 读取,不允许前端自行切换 |
|
||||
| 图片、视频、音频等媒体及生成元数据 | `editor_project_resource` / `editor_asset` | layer/dialog 只保存稳定 ID 引用;资源创建成功不能被当作布局保存成功 |
|
||||
|
||||
结构化快照由后端在同一 revision 下读取 `editor_canvas`、layer、dialog 和 layout migration state 后组装。前端只消费 BFF 快照,不直接订阅表、做 join 或推断迁移状态。
|
||||
|
||||
## 3. 表与写入契约
|
||||
|
||||
### 3.1 `editor_canvas_layer`
|
||||
|
||||
保存 `layer_id`、`canvas_id`、`project_id`、`owner_user_id`、坐标、宽高、原始尺寸、层级顺序、可选 `group_id`、hidden / locked / flip 状态、`resource_id`、有界的未结构化扩展 JSON、创建与更新时间。查询有 canvas / project 索引;同一 canvas 的层级顺序由 `sort_order` 决定。
|
||||
|
||||
layer 只表达“某个资源怎样放在画布上”。`src / prompt / actualPrompt / model / provider / taskId / objectKey / assetObjectId / sourceResourceId / assetKind / generationInputs / sourceType` 不得为方便展示而重复进入 `item_json` 或成为 layer 真相;迁移或结构化保存时先逐字段核对 `editor_project_resource`,缺少资源、字段冲突或无法无损重组时必须 fail-closed。历史 layout 若把 `sourceResourceId` 错写成当前图层自己的 `resourceId`,这是无意义的自引用,不作为 A/B 来源冲突:迁移时删除该重复字段并以项目资源表为真相;其他非空且不一致的来源 ID 继续拒绝。唯一存量缺资源例外是历史角色动作产生的自包含本地图层:`resourceId` 必须以 `local-` 开头、`sourceType=generated`、`mediaType=image-sequence`,至少包含一帧;帧序号必须从 1 连续递增,宽高必须是有限正数;所有 `imageSequenceFrames[].imageSrc` 及可选 `thumbnailSrc / previewVideoPath` 都必须是无 query / fragment、无路径回退段的站内根路径,且不能含 `data:`、`blob:`、HTTP 或签名 URL;可选帧 `objectKey` 必须与 `imageSrc` 去掉首斜杠后完全一致,图层级 `imageSrc / objectKey / assetObjectId` 必须为空,图层级 `src` 只允许为空或与首帧 `imageSrc` 完全一致。满足这些条件但资源行已不存在时,保留其有界媒体扩展和生成元数据以便前端从首帧恢复,不把这些字段从 canonical hash 中剥离;`sourceResourceId` 目标仍存在时必须属于同工程和 owner,目标已删除时保留原引用参与 hash,不据此伪造资源行。该例外不适用于普通图片、视频、音频、非本地 ID、空帧序列或已有资源字段冲突。存量 `assetKind / generationInputs` 仅允许在资源行尚未记录时由 apply 事务补入资源表,dry-run 只计算预览而不写库。前端读取项目快照时继续使用随项目返回的 resources 按 `resource_id` hydrate,兼容现有画布快照语义。
|
||||
|
||||
### 3.2 `editor_canvas_generation_dialog`
|
||||
|
||||
保存 `dialog_id`、`canvas_id`、`project_id`、`owner_user_id`、生成模式、状态、可选 `source_layer_id`、`generated_layer_id`、占位几何、有界扩展 JSON、创建与更新时间。扩展 JSON 只保留未结构化的生成参数,`mode / status / source / generated / placeholder.x/y/width/height` 以 typed 列为真相;`placeholder.originalWidth / originalHeight`、其他未知 placeholder 字段、dialog 内未知字段及 generation-dialog 顶层未知字段必须原样保存在扩展 JSON。`canvas-settings` 暂无扩展列,出现未识别字段时直接拒绝迁移,不能静默丢弃。当前 external job 关联仍以 worker 任务为准,未在本表重复建立任务真相。
|
||||
|
||||
### 3.3 `editor_canvas_layout_migration`
|
||||
|
||||
保存 `canvas_id`、schema version、`backfilled / active / rolled_back` 状态、最后校验 revision、legacy / structured canonical hash、layer / dialog 数量、资源引用集合 hash、迁移 / 激活 / 回滚时间。状态变更只允许已授权 database migration operator 调用 procedure 执行。procedure 错误作为调用结果和运维日志保留,不另外写入可能与失败事务脱节的 `last_error` 列。
|
||||
|
||||
### 3.4 revision CAS
|
||||
|
||||
所有用户布局写入必须携带读取快照时获得的 `expectedRevision`。procedure 在事务中校验 canvas 当前 revision;不一致返回 `409`,前端应重载后端最新快照,不能只换上新 revision 就原样重放冲突前的整包布局。
|
||||
|
||||
前端保存队列只对无 HTTP 响应的传输失败以及 `408 / 425 / 429 / 502 / 503 / 504` 做有界退避重试,`400 / 403 / 404 / 413` 等确定性错误不重复提交。若旧请求执行期间已有更新布局排队,旧请求失败后必须继续发送最新布局;`409` 后权威快照暂时加载失败时保留 pending save 并定时重新进入冲突恢复,不能等待用户再次拖动画布才恢复保存。
|
||||
|
||||
本次结构化 V1 先保留旧 `{ viewport, layers }` PATCH 作为兼容输入。legacy canvas 即使携带 `expectedRevision` 也只做 CAS legacy 保存,不允许用户写入绕过 migration operator 直接激活 structured;只有已完成 backfill / activate、且 active 迁移记录的 revision / hash / 数量 / 资源引用校验均通过时,后端才在单个事务内把兼容输入拆成 layer / dialog 行并递增一次 revision。旧无 CAS procedure 不得写 structured canvas。V1 快照从 typed 列重组,`item_json / dialog_json` 只保留最大 512 KiB 的未结构化扩展字段。自包含本地图片序列在 active canvas 中只能继续保存已回填且 `layerId / resourceId / sourceType / item_json` 语义完全一致的原行;允许修改几何、层级、分组、显隐等 typed 布局字段。前端序列化按正常资源真相边界省略 `assetKind / generationInputs` 时,后端只从既有结构化行恢复这两个冻结字段再校验;显式修改仍拒绝。active 路径不再经过 legacy 元数据清洗,拒绝新增缺资源序列或改写既有帧、预览、prompt 和生成扩展。后续将新增、移动、缩放、删除、重排和分组收窄为有界 batch mutation;在此之前 2 MiB 仍是兼容整包入口的上限。
|
||||
|
||||
### 3.5 worker 原子完成
|
||||
|
||||
worker 完成生成任务时,本次先用读取时 revision 调用 CAS 保存;发生并发变更时拒绝覆盖并让任务保留可诊断失败,不再静默覆盖用户布局。最终收口仍是受 `job_id + worker_id + lease_token` 栅栏保护的后端 procedure 在同一事务内:
|
||||
|
||||
1. 校验 job、owner、project、canvas、dialog 和租约;
|
||||
2. 幂等创建或确认 `editor_project_resource`;
|
||||
3. 创建 / 替换结果 layer,并删除或更新占位 layer;
|
||||
4. 把 dialog 更新为终态并关联 `generated_layer_id`;
|
||||
5. 递增 canvas revision,最后才允许完成 external job。
|
||||
|
||||
重复 completion 必须返回同一资源、layer 和 dialog 终态,不得重复插入,也不能因 dialog 暂时缺失而返回 `changed=false` 后仍把任务标记完成。任一步失败时整笔业务写回回滚,任务保留可诊断的失败或可重试状态。
|
||||
|
||||
## 4. 存量迁移
|
||||
|
||||
迁移按 canvas 执行 `backfill → hash 核对 → activate`,并保持幂等:
|
||||
|
||||
1. **Backfill**:在短事务中读取 `editor_canvas.layers_json`、canvas revision 和更新时间;解析 legacy items,把普通图层与 generation dialog 分别归一到两张结构化表。按稳定 `canvas_id + item id` upsert,重复运行不能产生新 ID 或重复行。记录本次基准 revision 和 legacy canonical hash。
|
||||
2. **Hash 核对**:按固定字段顺序、数值归一规则和稳定 item 排序,把结构化行重组成 canonical legacy 语义;分别计算 layer 数、dialog 数、资源引用集合以及 SHA-256 canonical hash。raw JSON 的空白、对象 key 顺序和无意义默认值差异不作为不一致;任何不可识别字段必须保存在有界扩展字段中或使迁移失败,不能静默丢弃。项目资源存在时继续逐字段核对;除 `sourceResourceId == resourceId` 的历史自引用按资源表真相剥离外,其他 `sourceResourceId` 等冲突都拒绝迁移。只有上一节定义的缺资源自包含本地图片序列保留 layout 内元数据并参与 hash。
|
||||
3. **Activate**:只有 structured hash、数量和资源引用集合均匹配,且 `editor_canvas.revision` 仍等于 backfill 基准 revision 时,才以 CAS 把 layout migration state 切为 `active` 并把 canvas storage version 切到 structured。revision 已变化时丢弃本轮验证结果并重新 backfill;不得覆盖迁移期间的用户更新。
|
||||
|
||||
当前提供单项目、可重跑的运维入口:`npm run spacetime:editor-canvas-layout:migrate -- --database <db> --server <server> --project-id <id> --owner-user-id <id> --action <backfill|activate|rollback>`。默认 dry-run,只有显式追加 `--apply` 才写入;backfill dry-run 即使遇到尚无 `editor_canvas` 的旧工程,也只用 `editor_project` 构造内存预览,不创建 canvas、不补资源元数据。每个项目依次执行 backfill dry-run / apply 和 activate dry-run / apply,不在一个长事务中扫全表。脚本只调用受 migration operator 保护的 procedure,不直接修改生产表。
|
||||
|
||||
若全量 backfill 审计发现普通图层缺少 `editor_project_resource`,先使用定向资源修复入口:`npm run spacetime:editor-canvas-resources:repair -- --database <db> --server <server> --plan-file </absolute/outside-repo/repair-plan.json>`。plan 必须是仓库外、当前用户持有、权限严格为 `0600` 的普通文件;每个 canvas 绑定 owner / project、expected revision、`editor_canvas.layers_json` 与 legacy `editor_project.layers_json` 两份原始字节 SHA-256,以及精确 layer/resource/sourceResourceId 或 asset_object 修复动作。默认逐 canvas dry-run;apply 必须追加 `--apply --confirm-plan-sha256 <dry-run 输出>`,成功后脚本自动以同一 plan 再 dry-run,并要求全部返回 `already_repaired`。
|
||||
|
||||
`repair_editor_canvas_resources_and_return` 只允许 migration operator 调用,并且只修尚无迁移记录的 legacy canvas。图片动作仅在缺失旧 resourceId、精确旧 sourceResourceId、同工程唯一现存资源、layout 资源元数据与 private asset_object owner/objectKey/task 谱系全部一致时,替换 `resourceId` 并删除重复 `sourceResourceId`。音频动作仅在原 resourceId 全局不存在、图层与 private asset_object 的 owner/objectKey/source job/entity/content type/长度全部匹配时,恢复 `420x120` 的 `sound-effect` / `background-music` 项目资源行;资源 `asset_kind` 分别映射 asset_object 的 `editor_sound_effect` / `editor_background_music`。apply 在一个事务内插入资源、同步两份 legacy layout、递增一次 canvas revision;任一 guard 失败整画布回滚。plan、脚本输出、测试和文档均不得包含生产真实 ID。
|
||||
|
||||
## 5. 兼容与回滚
|
||||
|
||||
- 结构化代码上线后先保持 `legacy` 主读写,允许 schema 与 procedure 先行发布;确认新 API 可用后才开始 backfill 和逐 canvas 激活。
|
||||
- `layers_json` 在迁移和观察期内保留原值,不删除、不改名、不重排现有字段。激活结构化模式不立即清空 legacy JSON。
|
||||
- 回滚前由后端在一个一致 revision 下读取 layer 与 dialog,按 legacy schema 重组并计算 canonical hash。重组结果必须是合法 JSON 且 UTF-8 大小 **不超过 2 MiB**,随后在事务内写回 `editor_canvas.layers_json`、递增 revision、记录回滚审计并把 state 切回 `legacy`。对已处于 `rolled_back` 的重复回滚请求,也必须重新复核 revision、structured hash、layer/dialog 数量、资源引用集合以及 legacy/structured 一致性,任一漂移都 fail-closed。
|
||||
- 超过 2 MiB、包含无法降级字段、hash / 资源引用核对失败或 revision CAS 冲突时,回滚必须拒绝并保持 structured,禁止截断图层、丢弃 dialog 或只切换读取标志。生产二进制回滚应保留能读取 structured 的前向兼容版本,不能把超限 canvas 强行交给旧二进制。
|
||||
- 观察期结束前不删除三张结构化表或 legacy 列。后续是否停止生成 legacy 回滚快照、是否清理旧 JSON,需另行评审和迁移窗口,不随本次结构化上线自动执行。
|
||||
|
||||
## 6. 发布次序
|
||||
|
||||
1. 发布 2 MiB hotfix:先发布不变更 schema 的 SpacetimeDB 模块校验,再部署同一上限和足够 body envelope 的 `api-server`;确认 release manifest、服务状态和健康检查。
|
||||
2. 发布新增三表、revision / CAS、typed 快照重组与迁移 procedure 的 SpacetimeDB 模块;同步 migration table 白名单和生成绑定。此时保持 legacy 主读写。
|
||||
3. 部署可同时读取 legacy / structured 的 `spacetime-client` 与 `api-server`,再部署携带 revision 的前端。混合版本期间未激活 canvas 必须仍可正常编辑,旧 API 写 structured canvas 必须失败而不是无 CAS 覆盖。
|
||||
4. 先对测试账号和单个真实 canvas dry-run、backfill、核对、activate,再按批次扩大;持续观测 CAS conflict、迁移失败、快照缺资源和 worker completion 指标。
|
||||
5. 验收完成并经过观察期后,才把新 canvas 默认设为 structured。legacy 回滚能力继续保留。
|
||||
|
||||
SpacetimeDB 必须先于依赖新 procedure / bindings 的 API 发布;前端必须最后发布。任何阶段失败都停留在当前可读模式,不做跨版本破坏性清理。
|
||||
|
||||
## 7. 验收清单
|
||||
|
||||
- 2 MiB hotfix 后,原先约 254 KiB 的受影响画布和大于 256 KiB 的测试布局可保存并刷新恢复;超过 2 MiB 的 legacy layout 在 API 与 SpacetimeDB 两层都稳定拒绝,错误可观测。
|
||||
- backfill 重跑不会增加 layer / dialog 行,canonical hash、数量和资源引用集合一致;迁移中发生用户写入时 activate CAS 失败并安全重跑。
|
||||
- release 存量抽样中的缺资源 `local-*` 角色动作序列可无损 round-trip;同形状但空帧、相对路径、HTTP / 签名 URL、`data:` / `blob:` 引用必须拒绝。已有资源的 `sourceResourceId == resourceId` 历史自引用应按资源表真相安全剥离,其他来源 ID 或资源字段冲突仍必须拒绝。
|
||||
- release 全量审计暴露的普通缺资源行必须先通过定向 repair dry-run;图片只能复用同工程唯一资源,音频只能从已登记 private asset_object 恢复。修复后同一 plan 全部命中 `already_repaired`,再重跑全量 backfill dry-run,要求所有 canvas 均通过。
|
||||
- structured 模式下 typed 列而非扩展 JSON 决定几何、层级、分组、显示 / 锁定、资源引用和 dialog 状态;两个客户端基于同一 revision 写入时只允许一个成功,冲突方重载后端最新快照,不换上新 revision 原样重放旧整包。细粒度 batch mutation 是取消 2 MiB 兼容入口的后续项,不冒充为本次已完成。
|
||||
- worker completion 当前以读取时 revision 做 CAS,冲突时拒绝覆盖;V2 保存和保存后快照在同一 procedure 结果内返回,避免“已提交但后续 GET 失败”的不确定结果。lease-fenced 资源 / layer / dialog / job 单事务 completion 仍是后续收口项。
|
||||
- structured 快照刷新后,上传参考图、生成结果、占位与 dialog 状态均可恢复;资源存在但布局写入失败时不会伪装为保存成功。
|
||||
- 回滚重组结果经 schema 校验、canonical hash / 资源引用核对且不超过 2 MiB;超限或不一致时明确拒绝且 structured 快照仍可读取。
|
||||
- 完成 `npm run spacetime:generate`、`npm run check:spacetime-runtime-access`、`npm run check:spacetime-schema`、相关 Rust / API / 前端定向测试、`npm run check:encoding` 和 `git diff --check`。
|
||||
@@ -165,6 +165,7 @@
|
||||
- 视频生成完成后,后端先把带纯色背景的预览视频登记为 OSS 私有对象、`asset_object`、项目资源和账号素材,再按面板选择抽取对应帧数:`32`、`40` 或 `48`。未传 `assetFolderId` 时进入默认“项目”素材文件夹;后续抽帧或抠图失败不能抹掉这份已经生成成功的可恢复视频。
|
||||
- 抽帧采样必须按目标帧数预留视频尾部安全步长,例如 `32帧·4秒` 最后一帧采 `3.875s`,避免 FFmpeg 在尾点附近返回成功但输出 `0` 帧。
|
||||
- 图片画布角色动作的 FFmpeg 原始帧在上传 OSS 前必须转为 RGB8,并按最终帧宽高的 contain 比例使用 `Triangle` 只缩放到内容尺寸;不得提前创建最终目标尺寸 RGBA 画布,不得引入 Alpha 通道或透明 padding。以 `560×752` 原始帧、`323×480` 最终目标为例,上传给抠图链路的源帧必须是 `323×434 RGB8 PNG`,没有上下补边。
|
||||
- 后端先计算整批精确采样时刻,再用单个 FFmpeg filter graph 统一解码预览视频并输出 `32 / 40 / 48` 张源帧;不得为每帧重新启动 FFmpeg、重复解码同一视频,也不得用会改变现有尾帧安全时刻的粗粒度 `fps` 抽帧替代。批量命令成功后必须逐一确认全部目标帧文件存在,缺少任一帧都按整批失败处理并保留缺帧编号、目标时刻和输出路径诊断。
|
||||
- 每帧绿幕源图字节由上传 owned 消费(`frame.bytes` 移入 put,上传完成后释放原帧缓冲,不克隆保留);后续只持 object key。每次 BgFilter attempt 重新签发 600 秒 GET URL,multipart 仅传 `image_url`(加 `background_mode=flat`、`seg_model=birefnet`、`cross_check=on` 与同一次生成已选定的 `screenColor`),不传 `file`。BgFilter 主路径不重新下载原帧;失败后走 `阿里云通用抠图(按签名 URL 单独下载)→ 本地 editor_green_screen(再按 object key 独立下载一次并在产出后释放)`。BgFilter 每一次 HTTP attempt 的 timeout 使用“`GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS` 基准值 + `2000ms × 本次实际帧数`”,默认 `32 / 40 / 48` 帧分别为 `244000 / 260000 / 276000ms`;首次失败后重试 `1` 次。
|
||||
- BgFilter、阿里云或本地键色返回透明结果后,后端继续通过现有最终帧 finalizer 转为 RGBA8,按宽高比居中放入最终目标尺寸,并使用 `RGBA(0,0,0,0)` 补边。上述样例最终输出必须为 `323×480 RGBA8 PNG`,顶部和底部各 `23px` 透明 padding,内容区域完整保留抠图结果。
|
||||
- 全部 `32 / 40 / 48` 帧以覆盖本次所有帧的无序在途集合连续发射,允许乱序完成并最终按 `frameIndex` 排序;任一帧最终失败时先排空全部已启动 Future,再让整项任务失败退款,不发布缺帧动画。
|
||||
|
||||
@@ -153,7 +153,7 @@ pipeline {
|
||||
|
||||
stage('Archive') {
|
||||
steps {
|
||||
archiveArtifacts artifacts: "build/${env.EFFECTIVE_BUILD_VERSION}/spacetime_module.wasm,build/${env.EFFECTIVE_BUILD_VERSION}/spacetime_module.wasm.sha256,build/${env.EFFECTIVE_BUILD_VERSION}/release-manifest.json,scripts/deploy/production-stdb-publish.sh,scripts/deploy/production-runtime-writer-identity-rotate.mjs,scripts/deploy/maintenance-on.sh,scripts/deploy/maintenance-off.sh,scripts/spacetime-migration-common.mjs,scripts/spacetime-maintain-external-generation-jobs.mjs,scripts/database-backup-to-oss.mjs", fingerprint: true
|
||||
archiveArtifacts artifacts: "build/${env.EFFECTIVE_BUILD_VERSION}/spacetime_module.wasm,build/${env.EFFECTIVE_BUILD_VERSION}/spacetime_module.wasm.sha256,build/${env.EFFECTIVE_BUILD_VERSION}/release-manifest.json,scripts/deploy/production-stdb-publish.sh,scripts/deploy/production-runtime-writer-identity-rotate.mjs,scripts/deploy/maintenance-on.sh,scripts/deploy/maintenance-off.sh,scripts/spacetime-migration-common.mjs,scripts/spacetime-maintain-external-generation-jobs.mjs,scripts/spacetime-migrate-editor-canvas-layout.mjs,scripts/spacetime-repair-editor-canvas-resources.mjs,scripts/database-backup-to-oss.mjs", fingerprint: true
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -93,7 +93,7 @@ pipeline {
|
||||
copyArtifacts(
|
||||
projectName: params.BUILD_JOB_NAME,
|
||||
selector: specific(params.BUILD_NUMBER_TO_DEPLOY),
|
||||
filter: "build/${params.BUILD_VERSION}/spacetime_module.wasm,build/${params.BUILD_VERSION}/spacetime_module.wasm.sha256,build/${params.BUILD_VERSION}/release-manifest.json,scripts/deploy/production-stdb-publish.sh,scripts/deploy/production-runtime-writer-identity-rotate.mjs,scripts/deploy/maintenance-on.sh,scripts/deploy/maintenance-off.sh,scripts/spacetime-migration-common.mjs,scripts/spacetime-maintain-external-generation-jobs.mjs,scripts/database-backup-to-oss.mjs",
|
||||
filter: "build/${params.BUILD_VERSION}/spacetime_module.wasm,build/${params.BUILD_VERSION}/spacetime_module.wasm.sha256,build/${params.BUILD_VERSION}/release-manifest.json,scripts/deploy/production-stdb-publish.sh,scripts/deploy/production-runtime-writer-identity-rotate.mjs,scripts/deploy/maintenance-on.sh,scripts/deploy/maintenance-off.sh,scripts/spacetime-migration-common.mjs,scripts/spacetime-maintain-external-generation-jobs.mjs,scripts/spacetime-migrate-editor-canvas-layout.mjs,scripts/spacetime-repair-editor-canvas-resources.mjs,scripts/database-backup-to-oss.mjs",
|
||||
target: '.',
|
||||
fingerprintArtifacts: true
|
||||
)
|
||||
|
||||
@@ -18,6 +18,8 @@
|
||||
"admin-web:preview": "npm --prefix apps/admin-web run preview --",
|
||||
"spacetime:generate": "node scripts/generate-spacetime-bindings.mjs",
|
||||
"spacetime:external-generation:maintain": "node scripts/spacetime-maintain-external-generation-jobs.mjs",
|
||||
"spacetime:editor-canvas-layout:migrate": "node scripts/spacetime-migrate-editor-canvas-layout.mjs",
|
||||
"spacetime:editor-canvas-resources:repair": "node scripts/spacetime-repair-editor-canvas-resources.mjs",
|
||||
"spacetime:wechat-virtual-payment:reconcile": "node scripts/reconcile-wechat-virtual-payment-order.mjs",
|
||||
"check:wechat-virtual-payment-reconcile": "node scripts/check-wechat-virtual-payment-reconcile.mjs",
|
||||
"check:api-server-env": "node scripts/check-api-server-env.mjs",
|
||||
|
||||
@@ -521,6 +521,46 @@ const checks = [
|
||||
includes: 'completed_before_micros: encodeSpacetimeCliOption(',
|
||||
reason: '历史 payload 截止时间必须编码为 CLI SATS Option,确保事故时间过滤可调用。',
|
||||
},
|
||||
{
|
||||
file: 'scripts/spacetime-migrate-editor-canvas-layout.mjs',
|
||||
includes: 'dry_run: !options.apply',
|
||||
reason: '图片画布存量迁移必须默认 dry-run,只有显式 --apply 才写入。',
|
||||
},
|
||||
{
|
||||
file: 'scripts/spacetime-repair-editor-canvas-resources.mjs',
|
||||
includes: 'buildProcedureInput(canvas, updatedAtMicros, !options.apply)',
|
||||
reason: '图片画布资源修复必须默认 dry-run,只有显式 --apply 才写入。',
|
||||
},
|
||||
{
|
||||
file: 'scripts/spacetime-repair-editor-canvas-resources.mjs',
|
||||
includes: '--confirm-plan-sha256',
|
||||
reason: '图片画布资源修复 apply 必须绑定同一份 dry-run plan 摘要。',
|
||||
},
|
||||
{
|
||||
file: 'scripts/spacetime-repair-editor-canvas-resources.mjs',
|
||||
includes: '(metadata.mode & 0o777) !== 0o600',
|
||||
reason: '包含真实资源 ID 的修复 plan 必须使用仓库外 0600 文件。',
|
||||
},
|
||||
{
|
||||
file: 'jenkins/Jenkinsfile.production-stdb-module-build',
|
||||
includes: 'scripts/spacetime-migrate-editor-canvas-layout.mjs',
|
||||
reason: 'Stdb Build 必须归档图片画布存量迁移脚本。',
|
||||
},
|
||||
{
|
||||
file: 'jenkins/Jenkinsfile.production-stdb-module-publish',
|
||||
includes: 'scripts/spacetime-migrate-editor-canvas-layout.mjs',
|
||||
reason: 'Stdb Publish 必须从同一上游制品复制图片画布存量迁移脚本。',
|
||||
},
|
||||
{
|
||||
file: 'jenkins/Jenkinsfile.production-stdb-module-build',
|
||||
includes: 'scripts/spacetime-repair-editor-canvas-resources.mjs',
|
||||
reason: 'Stdb Build 必须归档图片画布资源修复脚本。',
|
||||
},
|
||||
{
|
||||
file: 'jenkins/Jenkinsfile.production-stdb-module-publish',
|
||||
includes: 'scripts/spacetime-repair-editor-canvas-resources.mjs',
|
||||
reason: 'Stdb Publish 必须从同一上游制品复制图片画布资源修复脚本。',
|
||||
},
|
||||
{
|
||||
file: 'scripts/deploy-rust-remote.sh',
|
||||
excludes:
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
function readIndexCss() {
|
||||
return fs.readFileSync(path.resolve(process.cwd(), 'src/index.css'), 'utf8');
|
||||
}
|
||||
|
||||
function getCssBlock(source: string, selector: string) {
|
||||
const selectorIndex = source.indexOf(selector);
|
||||
expect(selectorIndex, `${selector} should exist`).toBeGreaterThanOrEqual(0);
|
||||
|
||||
const openBraceIndex = source.indexOf('{', selectorIndex);
|
||||
expect(
|
||||
openBraceIndex,
|
||||
`${selector} should open a CSS block`,
|
||||
).toBeGreaterThanOrEqual(0);
|
||||
|
||||
let depth = 0;
|
||||
for (let index = openBraceIndex; index < source.length; index += 1) {
|
||||
const char = source[index];
|
||||
if (char === '{') {
|
||||
depth += 1;
|
||||
} else if (char === '}') {
|
||||
depth -= 1;
|
||||
if (depth === 0) {
|
||||
return source.slice(openBraceIndex + 1, index);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`${selector} block is not closed`);
|
||||
}
|
||||
|
||||
describe('image canvas portal theme contract', () => {
|
||||
it('exposes every image canvas brand token to body-level portal menus', () => {
|
||||
const css = readIndexCss();
|
||||
const sharedTokenBlock = getCssBlock(
|
||||
css,
|
||||
'.image-canvas-editor,\n.image-canvas-editor__portal-theme,\n.image-canvas-editor__portal-menu',
|
||||
);
|
||||
const editorBlock = getCssBlock(css, '.image-canvas-editor {');
|
||||
const pressedToggleTrackBlock = getCssBlock(
|
||||
css,
|
||||
".image-canvas-editor__video-toggle[aria-pressed='true'] span",
|
||||
);
|
||||
|
||||
for (const token of [
|
||||
'--image-canvas-brand-accent:',
|
||||
'--image-canvas-brand-accent-strong:',
|
||||
'--image-canvas-brand-fill:',
|
||||
'--image-canvas-brand-text-on-fill:',
|
||||
'--image-canvas-brand-border:',
|
||||
'--image-canvas-brand-border-strong:',
|
||||
'--image-canvas-brand-border-soft:',
|
||||
'--image-canvas-brand-soft:',
|
||||
'--image-canvas-brand-soft-strong:',
|
||||
'--image-canvas-brand-shadow:',
|
||||
'--image-canvas-brand-focus-ring:',
|
||||
]) {
|
||||
expect(sharedTokenBlock).toContain(token);
|
||||
}
|
||||
|
||||
for (const editorOnlyDeclaration of [
|
||||
'position:',
|
||||
'display:',
|
||||
'height:',
|
||||
'overflow:',
|
||||
'background:',
|
||||
'color:',
|
||||
'user-select:',
|
||||
]) {
|
||||
expect(sharedTokenBlock).not.toContain(editorOnlyDeclaration);
|
||||
}
|
||||
|
||||
expect(editorBlock).not.toContain('--image-canvas-brand-accent:');
|
||||
expect(pressedToggleTrackBlock).toContain(
|
||||
'background: var(--image-canvas-brand-accent);',
|
||||
);
|
||||
expect(pressedToggleTrackBlock).not.toContain('#00ff00');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import {
|
||||
callSpacetimeProcedureViaCli,
|
||||
ensureProcedureOk,
|
||||
} from './spacetime-migration-common.mjs';
|
||||
|
||||
const ACTIONS = new Map([
|
||||
['backfill', 'backfill_editor_canvas_layout_and_return'],
|
||||
['activate', 'activate_editor_canvas_layout_and_return'],
|
||||
['rollback', 'rollback_editor_canvas_layout_and_return'],
|
||||
]);
|
||||
|
||||
function usage() {
|
||||
return `用法:
|
||||
node scripts/spacetime-migrate-editor-canvas-layout.mjs \\
|
||||
--database <name> --project-id <projectId> --owner-user-id <userId> \\
|
||||
--action <backfill|activate|rollback> [--server <name-or-url>] [--apply]
|
||||
|
||||
默认只执行 dry-run 校验,不修改数据;追加 --apply 才写入。
|
||||
必须使用已授权 database migration operator 的 spacetime CLI 登录态,
|
||||
并显式传入 --server 或设置 GENARRATIVE_SPACETIME_SERVER。
|
||||
|
||||
存量项目的安全顺序:
|
||||
1. backfill dry-run
|
||||
2. backfill --apply
|
||||
3. activate dry-run
|
||||
4. activate --apply
|
||||
需要回滚时先执行 rollback dry-run,确认 hash / revision / 2 MiB 门禁通过后再 --apply。`;
|
||||
}
|
||||
|
||||
function parseOptions(argv) {
|
||||
const options = {
|
||||
action: '',
|
||||
apply: false,
|
||||
database: process.env.GENARRATIVE_SPACETIME_DATABASE || '',
|
||||
ownerUserId: '',
|
||||
passthrough: [],
|
||||
projectId: '',
|
||||
server: process.env.GENARRATIVE_SPACETIME_SERVER || '',
|
||||
serverUrl: process.env.GENARRATIVE_SPACETIME_SERVER_URL || '',
|
||||
};
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const arg = argv[index];
|
||||
const readValue = () => {
|
||||
const value = argv[index + 1];
|
||||
if (!value || value.startsWith('--')) {
|
||||
throw new Error(`${arg} 缺少参数值。`);
|
||||
}
|
||||
index += 1;
|
||||
return value.trim();
|
||||
};
|
||||
if (arg === '--action') {
|
||||
options.action = readValue();
|
||||
} else if (arg === '--database') {
|
||||
options.database = readValue();
|
||||
} else if (arg === '--project-id') {
|
||||
options.projectId = readValue();
|
||||
} else if (arg === '--owner-user-id') {
|
||||
options.ownerUserId = readValue();
|
||||
} else if (arg === '--server') {
|
||||
options.server = readValue();
|
||||
} else if (arg === '--server-url') {
|
||||
options.serverUrl = readValue();
|
||||
} else if (arg === '--apply') {
|
||||
options.apply = true;
|
||||
} else if (arg === '--help' || arg === '-h') {
|
||||
options.help = true;
|
||||
} else if (arg === '--no-config' || arg === '--anonymous') {
|
||||
options.passthrough.push(arg);
|
||||
} else {
|
||||
throw new Error(`未知参数: ${arg}`);
|
||||
}
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
try {
|
||||
const options = parseOptions(process.argv.slice(2));
|
||||
if (options.help) {
|
||||
console.log(usage());
|
||||
process.exit(0);
|
||||
}
|
||||
if (!options.database || !options.projectId || !options.ownerUserId) {
|
||||
throw new Error('--database、--project-id 和 --owner-user-id 均为必填。');
|
||||
}
|
||||
if (!options.server && !options.serverUrl) {
|
||||
throw new Error('必须显式传入 --server / --server-url,不使用默认 cloud target。');
|
||||
}
|
||||
const procedureName = ACTIONS.get(options.action);
|
||||
if (!procedureName) {
|
||||
throw new Error('--action 只支持 backfill、activate 或 rollback。');
|
||||
}
|
||||
const result = await callSpacetimeProcedureViaCli(options, procedureName, {
|
||||
project_id: options.projectId,
|
||||
owner_user_id: options.ownerUserId,
|
||||
updated_at_micros: Date.now() * 1_000,
|
||||
dry_run: !options.apply,
|
||||
});
|
||||
ensureProcedureOk(result);
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{
|
||||
action: options.action,
|
||||
applied: options.apply,
|
||||
procedure: procedureName,
|
||||
...result,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
if (!options.apply) {
|
||||
console.log('dry-run 已通过;确认输出后追加 --apply 重跑同一动作。');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`[spacetime:editor-canvas-layout:migrate] ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
@@ -292,6 +292,36 @@ function normalizeProcedureResult(value, procedureName) {
|
||||
}
|
||||
|
||||
function normalizeSatsProduct(value, procedureName) {
|
||||
if (
|
||||
procedureName === 'repair_editor_canvas_resources_and_return' &&
|
||||
value.length === 3
|
||||
) {
|
||||
return {
|
||||
ok: normalizeSatsValue(value[0]),
|
||||
repair: normalizeEditorCanvasResourceRepairSnapshot(
|
||||
normalizeSatsOption(value[1]),
|
||||
),
|
||||
error_message: normalizeSatsOption(value[2]),
|
||||
};
|
||||
}
|
||||
|
||||
if (
|
||||
[
|
||||
'backfill_editor_canvas_layout_and_return',
|
||||
'activate_editor_canvas_layout_and_return',
|
||||
'rollback_editor_canvas_layout_and_return',
|
||||
].includes(procedureName) &&
|
||||
value.length === 3
|
||||
) {
|
||||
return {
|
||||
ok: normalizeSatsValue(value[0]),
|
||||
migration: normalizeEditorCanvasLayoutMigrationSnapshot(
|
||||
normalizeSatsOption(value[1]),
|
||||
),
|
||||
error_message: normalizeSatsOption(value[2]),
|
||||
};
|
||||
}
|
||||
|
||||
if (
|
||||
procedureName === 'backfill_external_generation_job_summaries_and_return' &&
|
||||
value.length === 8
|
||||
@@ -357,6 +387,40 @@ function normalizeSatsProduct(value, procedureName) {
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeEditorCanvasLayoutMigrationSnapshot(value) {
|
||||
if (!Array.isArray(value) || value.length !== 11) {
|
||||
return value;
|
||||
}
|
||||
return {
|
||||
canvas_id: normalizeSatsValue(value[0]),
|
||||
project_id: normalizeSatsValue(value[1]),
|
||||
source_layout_sha256: normalizeSatsValue(value[2]),
|
||||
structured_layout_sha256: normalizeSatsValue(value[3]),
|
||||
verified_revision: normalizeSatsValue(value[4]),
|
||||
layer_count: normalizeSatsValue(value[5]),
|
||||
dialog_count: normalizeSatsValue(value[6]),
|
||||
resource_refs_sha256: normalizeSatsValue(value[7]),
|
||||
migration_version: normalizeSatsValue(value[8]),
|
||||
status: normalizeSatsValue(value[9]),
|
||||
revision: normalizeSatsValue(value[10]),
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeEditorCanvasResourceRepairSnapshot(value) {
|
||||
if (!Array.isArray(value) || value.length !== 7) {
|
||||
return value;
|
||||
}
|
||||
return {
|
||||
dry_run: normalizeSatsValue(value[0]),
|
||||
already_repaired: normalizeSatsValue(value[1]),
|
||||
matched_layer_count: normalizeSatsValue(value[2]),
|
||||
remapped_layer_count: normalizeSatsValue(value[3]),
|
||||
restored_resource_count: normalizeSatsValue(value[4]),
|
||||
removed_source_resource_id_count: normalizeSatsValue(value[5]),
|
||||
revision: normalizeSatsValue(value[6]),
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeSatsValue(value) {
|
||||
if (Array.isArray(value)) {
|
||||
return value.map((item) => normalizeSatsValue(item));
|
||||
|
||||
@@ -3,6 +3,7 @@ import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
buildSpacetimeCallArgs,
|
||||
encodeSpacetimeCliOption,
|
||||
parseProcedureResult,
|
||||
} from './spacetime-migration-common.mjs';
|
||||
|
||||
describe('SpacetimeDB CLI SATS option encoding', () => {
|
||||
@@ -33,4 +34,69 @@ describe('SpacetimeDB CLI SATS option encoding', () => {
|
||||
expect(input.cursor_job_id).toEqual([0, 'job-2']);
|
||||
expect(input.completed_before_micros).toEqual([0, 123]);
|
||||
});
|
||||
|
||||
it('normalizes editor canvas migration procedure results', () => {
|
||||
const result = parseProcedureResult(
|
||||
JSON.stringify([
|
||||
true,
|
||||
[
|
||||
0,
|
||||
[
|
||||
'canvas-1',
|
||||
'project-1',
|
||||
'source-hash',
|
||||
'structured-hash',
|
||||
7,
|
||||
2,
|
||||
1,
|
||||
'resource-hash',
|
||||
1,
|
||||
'backfilled',
|
||||
7,
|
||||
],
|
||||
],
|
||||
[1],
|
||||
]),
|
||||
'backfill_editor_canvas_layout_and_return',
|
||||
);
|
||||
|
||||
expect(result).toEqual({
|
||||
ok: true,
|
||||
migration: {
|
||||
canvas_id: 'canvas-1',
|
||||
project_id: 'project-1',
|
||||
source_layout_sha256: 'source-hash',
|
||||
structured_layout_sha256: 'structured-hash',
|
||||
verified_revision: 7,
|
||||
layer_count: 2,
|
||||
dialog_count: 1,
|
||||
resource_refs_sha256: 'resource-hash',
|
||||
migration_version: 1,
|
||||
status: 'backfilled',
|
||||
revision: 7,
|
||||
},
|
||||
error_message: null,
|
||||
});
|
||||
});
|
||||
|
||||
it('normalizes editor canvas resource repair procedure results', () => {
|
||||
const result = parseProcedureResult(
|
||||
JSON.stringify([true, [0, [true, false, 2, 0, 0, 0, 7]], [1]]),
|
||||
'repair_editor_canvas_resources_and_return',
|
||||
);
|
||||
|
||||
expect(result).toEqual({
|
||||
ok: true,
|
||||
repair: {
|
||||
dry_run: true,
|
||||
already_repaired: false,
|
||||
matched_layer_count: 2,
|
||||
remapped_layer_count: 0,
|
||||
restored_resource_count: 0,
|
||||
removed_source_resource_id_count: 0,
|
||||
revision: 7,
|
||||
},
|
||||
error_message: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,477 @@
|
||||
import {
|
||||
chmod,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
rm,
|
||||
symlink,
|
||||
writeFile,
|
||||
} from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
|
||||
import {
|
||||
buildProcedureInput,
|
||||
executeRepairPlan,
|
||||
parseOptions,
|
||||
readRepairPlan,
|
||||
runCli,
|
||||
validateRepairPlan,
|
||||
} from './spacetime-repair-editor-canvas-resources.mjs';
|
||||
|
||||
const SHA_A = 'a'.repeat(64);
|
||||
const SHA_B = 'b'.repeat(64);
|
||||
const PLAN_SHA = 'c'.repeat(64);
|
||||
const temporaryRoots: string[] = [];
|
||||
|
||||
afterEach(async () => {
|
||||
process.exitCode = undefined;
|
||||
await Promise.all(
|
||||
temporaryRoots
|
||||
.splice(0)
|
||||
.map((root) => rm(root, { recursive: true, force: true })),
|
||||
);
|
||||
});
|
||||
|
||||
function action(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
action_kind: 'reuse_project_resource',
|
||||
layer_id: 'private-layer-id',
|
||||
expected_resource_id: 'missing-private-resource-id',
|
||||
replacement_resource_id: 'replacement-private-resource-id',
|
||||
expected_layout_source_resource_id: 'source-private-resource-id',
|
||||
asset_object_id: null,
|
||||
asset_kind: null,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function canvas(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
canvas_id: 'private-canvas-id',
|
||||
project_id: 'private-project-id',
|
||||
owner_user_id: 'private-owner-id',
|
||||
expected_revision: 7,
|
||||
expected_canvas_layout_sha256: SHA_A,
|
||||
expected_project_layout_sha256: SHA_B,
|
||||
actions: [action()],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function planValue(canvases = [canvas()]) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
expected_canvas_count: canvases.length,
|
||||
expected_action_count: canvases.reduce(
|
||||
(sum, item) =>
|
||||
sum + (Array.isArray(item.actions) ? item.actions.length : 0),
|
||||
0,
|
||||
),
|
||||
canvases,
|
||||
};
|
||||
}
|
||||
|
||||
function executionOptions(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
apply: false,
|
||||
confirmPlanSha256: '',
|
||||
database: 'genarrative-prod',
|
||||
passthrough: [],
|
||||
planFile: '/outside/repair-plan.json',
|
||||
server: 'http://127.0.0.1:3101',
|
||||
serverUrl: '',
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function procedureResult(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
ok: true,
|
||||
repair: {
|
||||
dry_run: true,
|
||||
already_repaired: false,
|
||||
matched_layer_count: 1,
|
||||
remapped_layer_count: 1,
|
||||
restored_resource_count: 0,
|
||||
removed_source_resource_id_count: 1,
|
||||
revision: 7,
|
||||
},
|
||||
error_message: null,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe('spacetime editor canvas resource repair options', () => {
|
||||
it('defaults to dry-run and requires an explicit apply flag', () => {
|
||||
const options = parseOptions([
|
||||
'--database',
|
||||
'genarrative-prod',
|
||||
'--server-url',
|
||||
'http://127.0.0.1:3101',
|
||||
'--plan-file',
|
||||
'/tmp/repair-plan.json',
|
||||
]);
|
||||
|
||||
expect(options.apply).toBe(false);
|
||||
expect(options.confirmPlanSha256).toBe('');
|
||||
expect(options.planFile).toBe('/tmp/repair-plan.json');
|
||||
});
|
||||
|
||||
it('records explicit apply and lowercases the confirmed plan hash', () => {
|
||||
const options = parseOptions([
|
||||
'--apply',
|
||||
'--confirm-plan-sha256',
|
||||
SHA_A.toUpperCase(),
|
||||
]);
|
||||
|
||||
expect(options.apply).toBe(true);
|
||||
expect(options.confirmPlanSha256).toBe(SHA_A);
|
||||
});
|
||||
|
||||
it('rejects anonymous procedure calls', () => {
|
||||
expect(() => parseOptions(['--anonymous'])).toThrow('未知参数');
|
||||
});
|
||||
});
|
||||
|
||||
describe('spacetime editor canvas resource repair plan', () => {
|
||||
it('validates and normalizes the two supported action kinds', () => {
|
||||
const plan = validateRepairPlan(
|
||||
planValue([
|
||||
canvas({
|
||||
actions: [
|
||||
action(),
|
||||
action({
|
||||
action_kind: 'restore_project_resource',
|
||||
layer_id: 'private-audio-layer-id',
|
||||
expected_resource_id: 'missing-private-audio-resource-id',
|
||||
replacement_resource_id: null,
|
||||
expected_layout_source_resource_id: null,
|
||||
asset_object_id: 'private-asset-object-id',
|
||||
asset_kind: 'background-music',
|
||||
}),
|
||||
],
|
||||
}),
|
||||
]),
|
||||
);
|
||||
|
||||
expect(plan.expected_action_count).toBe(2);
|
||||
expect(plan.canvases[0].actions).toMatchObject([
|
||||
{ action_kind: 'reuse_project_resource' },
|
||||
{
|
||||
action_kind: 'restore_project_resource',
|
||||
asset_kind: 'background-music',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('rejects count mismatches, duplicate layers, and invalid kind-specific fields', () => {
|
||||
expect(() =>
|
||||
validateRepairPlan({ ...planValue(), expected_action_count: 2 }),
|
||||
).toThrow('expected_action_count');
|
||||
expect(() =>
|
||||
validateRepairPlan(
|
||||
planValue([
|
||||
canvas({
|
||||
actions: [action(), action({ replacement_resource_id: 'other' })],
|
||||
}),
|
||||
]),
|
||||
),
|
||||
).toThrow('layer_id 在同一画布内重复');
|
||||
expect(() =>
|
||||
validateRepairPlan(
|
||||
planValue([
|
||||
canvas({
|
||||
actions: [
|
||||
action({
|
||||
action_kind: 'restore_project_resource',
|
||||
replacement_resource_id: null,
|
||||
expected_layout_source_resource_id: null,
|
||||
asset_object_id: 'private-asset-object-id',
|
||||
asset_kind: 'image',
|
||||
}),
|
||||
],
|
||||
}),
|
||||
]),
|
||||
),
|
||||
).toThrow('asset_kind 只支持');
|
||||
});
|
||||
|
||||
it('reads only an external current-user 0600 regular plan and returns its hash', async () => {
|
||||
const root = await makeTemporaryRoot();
|
||||
const planPath = path.join(root, 'repair-plan.json');
|
||||
await writeFile(planPath, `${JSON.stringify(planValue())}\n`, 'utf8');
|
||||
await chmod(planPath, 0o600);
|
||||
|
||||
const result = await readRepairPlan(planPath);
|
||||
|
||||
expect(result.plan.expected_canvas_count).toBe(1);
|
||||
expect(result.planSha256).toMatch(/^[0-9a-f]{64}$/u);
|
||||
});
|
||||
|
||||
it('rejects permissive modes, repository paths, and symlink path components', async () => {
|
||||
const root = await makeTemporaryRoot();
|
||||
const planPath = path.join(root, 'repair-plan.json');
|
||||
await writeFile(planPath, JSON.stringify(planValue()), 'utf8');
|
||||
await chmod(planPath, 0o644);
|
||||
await expect(readRepairPlan(planPath)).rejects.toThrow('0600');
|
||||
|
||||
await chmod(planPath, 0o600);
|
||||
await expect(readRepairPlan(planPath, { repoRoot: root })).rejects.toThrow(
|
||||
'必须位于仓库外',
|
||||
);
|
||||
|
||||
const realDirectory = path.join(root, 'real');
|
||||
const linkedDirectory = path.join(root, 'linked');
|
||||
await mkdir(realDirectory);
|
||||
const linkedPlanPath = path.join(realDirectory, 'linked-plan.json');
|
||||
await writeFile(linkedPlanPath, JSON.stringify(planValue()), 'utf8');
|
||||
await chmod(linkedPlanPath, 0o600);
|
||||
await symlink(realDirectory, linkedDirectory);
|
||||
await expect(
|
||||
readRepairPlan(path.join(linkedDirectory, 'linked-plan.json')),
|
||||
).rejects.toThrow('路径链不能包含符号链接');
|
||||
});
|
||||
});
|
||||
|
||||
describe('spacetime editor canvas resource repair execution', () => {
|
||||
it('splits the two action kinds into typed procedure inputs', () => {
|
||||
const plan = validateRepairPlan(
|
||||
planValue([
|
||||
canvas({
|
||||
actions: [
|
||||
action(),
|
||||
action({
|
||||
action_kind: 'restore_project_resource',
|
||||
layer_id: 'private-audio-layer-id',
|
||||
expected_resource_id: 'missing-private-audio-resource-id',
|
||||
replacement_resource_id: null,
|
||||
expected_layout_source_resource_id: null,
|
||||
asset_object_id: 'private-asset-object-id',
|
||||
asset_kind: 'sound-effect',
|
||||
}),
|
||||
],
|
||||
}),
|
||||
]),
|
||||
);
|
||||
|
||||
const input = buildProcedureInput(plan.canvases[0], 123, true);
|
||||
|
||||
expect(input.remaps).toEqual([
|
||||
{
|
||||
layer_id: 'private-layer-id',
|
||||
expected_missing_resource_id: 'missing-private-resource-id',
|
||||
replacement_resource_id: 'replacement-private-resource-id',
|
||||
expected_layout_source_resource_id: 'source-private-resource-id',
|
||||
},
|
||||
]);
|
||||
expect(input.restores).toEqual([
|
||||
{
|
||||
layer_id: 'private-audio-layer-id',
|
||||
resource_id: 'missing-private-audio-resource-id',
|
||||
asset_object_id: 'private-asset-object-id',
|
||||
asset_kind: 'sound-effect',
|
||||
},
|
||||
]);
|
||||
expect(input.canvas_id).toBe('private-canvas-id');
|
||||
expect(input.updated_at_micros).toBe(123);
|
||||
expect(input.dry_run).toBe(true);
|
||||
});
|
||||
|
||||
it('calls one procedure per canvas and emits only anonymous aggregate output', async () => {
|
||||
const plan = validateRepairPlan(
|
||||
planValue([
|
||||
canvas(),
|
||||
canvas({
|
||||
canvas_id: 'second-private-canvas-id',
|
||||
project_id: 'second-private-project-id',
|
||||
actions: [
|
||||
action({
|
||||
layer_id: 'second-private-layer-id',
|
||||
expected_resource_id: 'second-private-resource-id',
|
||||
replacement_resource_id: 'second-private-replacement-id',
|
||||
}),
|
||||
],
|
||||
}),
|
||||
]),
|
||||
);
|
||||
const calls: unknown[] = [];
|
||||
|
||||
const summary = await executeRepairPlan({
|
||||
options: executionOptions(),
|
||||
plan,
|
||||
planSha256: PLAN_SHA,
|
||||
updatedAtMicros: 123,
|
||||
callProcedure: async (options, procedureName, input) => {
|
||||
calls.push({ options, procedureName, input });
|
||||
return procedureResult();
|
||||
},
|
||||
});
|
||||
|
||||
expect(calls).toHaveLength(2);
|
||||
expect(calls).toMatchObject([
|
||||
{ procedureName: 'repair_editor_canvas_resources_and_return' },
|
||||
{ procedureName: 'repair_editor_canvas_resources_and_return' },
|
||||
]);
|
||||
expect(summary).toMatchObject({
|
||||
ok: true,
|
||||
dry_run: true,
|
||||
canvas_count: 2,
|
||||
action_count: 2,
|
||||
processed_canvas_count: 2,
|
||||
matched_layer_count: 2,
|
||||
updated_layer_count: 2,
|
||||
inserted_resource_count: 0,
|
||||
});
|
||||
const publicOutput = JSON.stringify(summary);
|
||||
for (const privateValue of [
|
||||
'private-project-id',
|
||||
'private-layer-id',
|
||||
'missing-private-resource-id',
|
||||
'replacement-private-resource-id',
|
||||
]) {
|
||||
expect(publicOutput).not.toContain(privateValue);
|
||||
}
|
||||
});
|
||||
|
||||
it('requires apply to match the dry-run plan hash', async () => {
|
||||
const plan = validateRepairPlan(planValue());
|
||||
const callProcedure = async () => procedureResult();
|
||||
|
||||
await expect(
|
||||
executeRepairPlan({
|
||||
options: executionOptions({ apply: true, confirmPlanSha256: SHA_A }),
|
||||
plan,
|
||||
planSha256: PLAN_SHA,
|
||||
callProcedure,
|
||||
}),
|
||||
).rejects.toThrow('与当前 plan 不一致');
|
||||
});
|
||||
|
||||
it('stops apply at the first failed canvas without exposing procedure messages', async () => {
|
||||
const plan = validateRepairPlan(
|
||||
planValue([
|
||||
canvas(),
|
||||
canvas({
|
||||
canvas_id: 'second-private-canvas-id',
|
||||
project_id: 'second-private-project-id',
|
||||
actions: [action({ layer_id: 'second-private-layer-id' })],
|
||||
}),
|
||||
]),
|
||||
);
|
||||
let callCount = 0;
|
||||
|
||||
await expect(
|
||||
executeRepairPlan({
|
||||
options: executionOptions({
|
||||
apply: true,
|
||||
confirmPlanSha256: PLAN_SHA,
|
||||
}),
|
||||
plan,
|
||||
planSha256: PLAN_SHA,
|
||||
callProcedure: async () => {
|
||||
callCount += 1;
|
||||
return procedureResult({
|
||||
ok: false,
|
||||
repair: null,
|
||||
error_message:
|
||||
'contains private-project-id and private-resource-id',
|
||||
});
|
||||
},
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
'第 1 个画布修复失败,error_code=repair_rejected;此前成功处理 0 个画布',
|
||||
);
|
||||
expect(callCount).toBe(1);
|
||||
});
|
||||
|
||||
it('continues dry-run validation and reports only ordinal-safe failure codes', async () => {
|
||||
const plan = validateRepairPlan(
|
||||
planValue([
|
||||
canvas(),
|
||||
canvas({
|
||||
canvas_id: 'second-private-canvas-id',
|
||||
project_id: 'second-private-project-id',
|
||||
actions: [action({ layer_id: 'second-private-layer-id' })],
|
||||
}),
|
||||
]),
|
||||
);
|
||||
let callCount = 0;
|
||||
|
||||
const summary = await executeRepairPlan({
|
||||
options: executionOptions(),
|
||||
plan,
|
||||
planSha256: PLAN_SHA,
|
||||
callProcedure: async () => {
|
||||
callCount += 1;
|
||||
return callCount === 1
|
||||
? procedureResult({
|
||||
ok: false,
|
||||
repair: null,
|
||||
error_message: 'contains private-project-id',
|
||||
})
|
||||
: procedureResult();
|
||||
},
|
||||
});
|
||||
|
||||
expect(callCount).toBe(2);
|
||||
expect(summary).toMatchObject({
|
||||
ok: false,
|
||||
processed_canvas_count: 1,
|
||||
failures: [{ canvas_ordinal: 1, error_code: 'repair_rejected' }],
|
||||
});
|
||||
expect(JSON.stringify(summary)).not.toContain('private-project-id');
|
||||
});
|
||||
|
||||
it('automatically dry-runs the same plan after apply and requires already repaired', async () => {
|
||||
const plan = validateRepairPlan(planValue());
|
||||
const originalLog = console.log;
|
||||
console.log = () => {};
|
||||
let callCount = 0;
|
||||
try {
|
||||
const summary = await runCli(
|
||||
[
|
||||
'--database',
|
||||
'genarrative-prod',
|
||||
'--server-url',
|
||||
'http://127.0.0.1:3101',
|
||||
'--plan-file',
|
||||
'/outside/repair-plan.json',
|
||||
'--apply',
|
||||
'--confirm-plan-sha256',
|
||||
PLAN_SHA,
|
||||
],
|
||||
{
|
||||
readPlan: async () => ({ plan, planSha256: PLAN_SHA }),
|
||||
callProcedure: async (_options, _procedureName, input) => {
|
||||
callCount += 1;
|
||||
const dryRun = input.dry_run === true;
|
||||
return procedureResult({
|
||||
repair: {
|
||||
...procedureResult().repair,
|
||||
dry_run: dryRun,
|
||||
already_repaired: dryRun,
|
||||
remapped_layer_count: dryRun ? 0 : 1,
|
||||
removed_source_resource_id_count: dryRun ? 0 : 1,
|
||||
revision: 8,
|
||||
},
|
||||
});
|
||||
},
|
||||
updatedAtMicros: 123,
|
||||
},
|
||||
);
|
||||
|
||||
expect(callCount).toBe(2);
|
||||
expect(summary).toMatchObject({ ok: true, dry_run: false });
|
||||
} finally {
|
||||
console.log = originalLog;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
async function makeTemporaryRoot() {
|
||||
const root = await mkdtemp(path.join(tmpdir(), 'editor-canvas-repair-'));
|
||||
temporaryRoots.push(root);
|
||||
return root;
|
||||
}
|
||||
@@ -4128,23 +4128,29 @@ async fn extract_animation_frames_from_preview_video(
|
||||
known_duration_seconds,
|
||||
)?
|
||||
.max(0.001);
|
||||
let mut finalized_frames = Vec::with_capacity(plan.frame_count as usize);
|
||||
let mut target_seconds = Vec::with_capacity(plan.frame_count as usize);
|
||||
let mut raw_frame_paths = Vec::with_capacity(plan.frame_count as usize);
|
||||
for frame_index in 0..plan.frame_count {
|
||||
let target_seconds = compute_sample_time_seconds(
|
||||
target_seconds.push(compute_sample_time_seconds(
|
||||
duration_seconds,
|
||||
frame_index,
|
||||
plan.frame_count,
|
||||
plan.sample_start_ratio,
|
||||
plan.sample_end_ratio,
|
||||
plan.frame_count > 1 && plan.sample_end_ratio < 1.0,
|
||||
);
|
||||
let raw_frame_path = temp_dir.join(format!("raw-frame-{:02}.png", frame_index + 1));
|
||||
extract_video_frame_to_png(
|
||||
&input_path,
|
||||
&raw_frame_path,
|
||||
target_seconds,
|
||||
extraction_settings,
|
||||
)?;
|
||||
));
|
||||
raw_frame_paths.push(temp_dir.join(format!("raw-frame-{:02}.png", frame_index + 1)));
|
||||
}
|
||||
// 中文注释:整批采样点只进入一个 FFmpeg filter graph,避免逐帧重启进程并重复解码同一视频。
|
||||
extract_video_frames_to_png(
|
||||
&input_path,
|
||||
raw_frame_paths.as_slice(),
|
||||
target_seconds.as_slice(),
|
||||
extraction_settings,
|
||||
)?;
|
||||
|
||||
let mut finalized_frames = Vec::with_capacity(raw_frame_paths.len());
|
||||
for raw_frame_path in raw_frame_paths {
|
||||
let frame_bytes = fs::read(&raw_frame_path).map_err(|error| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "character-animation",
|
||||
@@ -4271,41 +4277,108 @@ fn compute_sample_time_seconds(
|
||||
(sample_start + sample_window * progress).min(safe_sample_end)
|
||||
}
|
||||
|
||||
fn extract_video_frame_to_png(
|
||||
fn build_batch_frame_filter_graph(target_seconds: &[f64]) -> String {
|
||||
let split_outputs = (0..target_seconds.len())
|
||||
.map(|index| format!("[s{index}]"))
|
||||
.collect::<String>();
|
||||
let selectors = target_seconds
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(index, target_seconds)| {
|
||||
format!("[s{index}]select=gte(t\\,{target_seconds:.3})[f{index}]")
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join(";");
|
||||
format!(
|
||||
"[0:v]setpts=PTS-STARTPTS,split={}{split_outputs};{selectors}",
|
||||
target_seconds.len()
|
||||
)
|
||||
}
|
||||
|
||||
fn extract_video_frames_to_png(
|
||||
input_path: &Path,
|
||||
output_path: &Path,
|
||||
target_seconds: f64,
|
||||
output_paths: &[PathBuf],
|
||||
target_seconds: &[f64],
|
||||
extraction_settings: &BackendFrameExtractionSettings,
|
||||
) -> Result<(), AppError> {
|
||||
if output_paths.is_empty()
|
||||
|| output_paths.len() != target_seconds.len()
|
||||
|| target_seconds
|
||||
.iter()
|
||||
.any(|target_seconds| !target_seconds.is_finite() || *target_seconds < 0.0)
|
||||
{
|
||||
return Err(
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
|
||||
"provider": "character-animation",
|
||||
"message": "动作视频批量抽帧计划无效。",
|
||||
"outputCount": output_paths.len(),
|
||||
"targetCount": target_seconds.len(),
|
||||
})),
|
||||
);
|
||||
}
|
||||
|
||||
let filter_graph = build_batch_frame_filter_graph(target_seconds);
|
||||
let mut owned_args = vec![
|
||||
"-hide_banner".to_string(),
|
||||
"-loglevel".to_string(),
|
||||
"error".to_string(),
|
||||
"-y".to_string(),
|
||||
"-i".to_string(),
|
||||
input_path.to_string_lossy().into_owned(),
|
||||
"-filter_complex".to_string(),
|
||||
filter_graph,
|
||||
];
|
||||
for (frame_index, output_path) in output_paths.iter().enumerate() {
|
||||
owned_args.extend([
|
||||
"-map".to_string(),
|
||||
format!("[f{frame_index}]"),
|
||||
"-frames:v".to_string(),
|
||||
"1".to_string(),
|
||||
"-pix_fmt".to_string(),
|
||||
"rgb24".to_string(),
|
||||
"-f".to_string(),
|
||||
"image2".to_string(),
|
||||
output_path.to_string_lossy().into_owned(),
|
||||
]);
|
||||
}
|
||||
let args = owned_args.iter().map(String::as_str).collect::<Vec<_>>();
|
||||
let fallback_message = if output_paths.len() == 1 {
|
||||
"抽取动作视频帧失败"
|
||||
} else {
|
||||
"批量抽取动作视频帧失败"
|
||||
};
|
||||
let output = run_process_with_timeout(
|
||||
&extraction_settings.ffmpeg_path,
|
||||
&[
|
||||
"-y",
|
||||
"-ss",
|
||||
&format!("{target_seconds:.3}"),
|
||||
"-i",
|
||||
input_path.to_string_lossy().as_ref(),
|
||||
"-frames:v",
|
||||
"1",
|
||||
"-pix_fmt",
|
||||
"rgb24",
|
||||
"-f",
|
||||
"image2",
|
||||
output_path.to_string_lossy().as_ref(),
|
||||
],
|
||||
args.as_slice(),
|
||||
extraction_settings.timeout_ms,
|
||||
"抽取动作视频帧失败",
|
||||
fallback_message,
|
||||
)?;
|
||||
|
||||
if !output_path.is_file() {
|
||||
let missing_frames = output_paths
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, output_path)| !output_path.is_file())
|
||||
.collect::<Vec<_>>();
|
||||
if let Some((first_missing_index, first_missing_path)) = missing_frames.first().copied() {
|
||||
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
||||
let stdout = String::from_utf8_lossy(&output.stdout).trim().to_string();
|
||||
let missing_frame_details = missing_frames
|
||||
.into_iter()
|
||||
.map(|(frame_index, output_path)| {
|
||||
json!({
|
||||
"frameIndex": frame_index + 1,
|
||||
"targetSeconds": target_seconds[frame_index],
|
||||
"outputPath": output_path.to_string_lossy(),
|
||||
})
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
return Err(
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "character-animation",
|
||||
"message": "ffmpeg 已执行但未产出动作帧文件。",
|
||||
"targetSeconds": target_seconds,
|
||||
"outputPath": output_path.to_string_lossy(),
|
||||
"targetSeconds": target_seconds[first_missing_index],
|
||||
"outputPath": first_missing_path.to_string_lossy(),
|
||||
"missingFrames": missing_frame_details,
|
||||
"stderr": stderr,
|
||||
"stdout": stdout,
|
||||
})),
|
||||
@@ -4314,6 +4387,22 @@ fn extract_video_frame_to_png(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn extract_video_frame_to_png(
|
||||
input_path: &Path,
|
||||
output_path: &Path,
|
||||
target_seconds: f64,
|
||||
extraction_settings: &BackendFrameExtractionSettings,
|
||||
) -> Result<(), AppError> {
|
||||
let output_paths = [output_path.to_path_buf()];
|
||||
let target_seconds = [target_seconds];
|
||||
extract_video_frames_to_png(
|
||||
input_path,
|
||||
output_paths.as_slice(),
|
||||
target_seconds.as_slice(),
|
||||
extraction_settings,
|
||||
)
|
||||
}
|
||||
|
||||
fn run_process_with_timeout(
|
||||
program: &str,
|
||||
args: &[&str],
|
||||
@@ -6630,21 +6719,30 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_extracts_final_sample_from_short_video() {
|
||||
fn editor_character_animation_builds_one_filter_graph_for_all_samples() {
|
||||
let filter_graph = build_batch_frame_filter_graph(&[0.0, 1.0, 2.0, 3.875]);
|
||||
|
||||
assert_eq!(
|
||||
filter_graph,
|
||||
"[0:v]setpts=PTS-STARTPTS,split=4[s0][s1][s2][s3];[s0]select=gte(t\\,0.000)[f0];[s1]select=gte(t\\,1.000)[f1];[s2]select=gte(t\\,2.000)[f2];[s3]select=gte(t\\,3.875)[f3]"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_batch_extracts_all_samples_from_short_video() {
|
||||
if Command::new("ffmpeg").arg("-version").output().is_err() {
|
||||
return;
|
||||
}
|
||||
|
||||
let temp_dir = create_animation_temp_dir().expect("temp dir should be created");
|
||||
let input_path = temp_dir.join("short-preview.mp4");
|
||||
let output_path = temp_dir.join("raw-frame-32.png");
|
||||
let make_video_status = Command::new("ffmpeg")
|
||||
.args([
|
||||
"-y",
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc=size=160x120:rate=8:duration=4",
|
||||
"testsrc=size=160x120:rate=8:duration=6",
|
||||
"-pix_fmt",
|
||||
"yuv420p",
|
||||
input_path.to_string_lossy().as_ref(),
|
||||
@@ -6658,18 +6756,55 @@ mod tests {
|
||||
let _ = fs::remove_dir_all(&temp_dir);
|
||||
return;
|
||||
}
|
||||
let target_seconds = compute_sample_time_seconds(4.0, 31, 32, 0.0, 1.0, false);
|
||||
let target_seconds = (0..48)
|
||||
.map(|frame_index| compute_sample_time_seconds(6.0, frame_index, 48, 0.0, 1.0, false))
|
||||
.collect::<Vec<_>>();
|
||||
let output_paths = (0..48)
|
||||
.map(|frame_index| temp_dir.join(format!("raw-frame-{:02}.png", frame_index + 1)))
|
||||
.collect::<Vec<_>>();
|
||||
let settings = BackendFrameExtractionSettings {
|
||||
ffmpeg_path: "ffmpeg".to_string(),
|
||||
ffprobe_path: "ffprobe".to_string(),
|
||||
timeout_ms: 10_000,
|
||||
};
|
||||
|
||||
let result =
|
||||
extract_video_frame_to_png(&input_path, &output_path, target_seconds, &settings);
|
||||
let result = extract_video_frames_to_png(
|
||||
&input_path,
|
||||
output_paths.as_slice(),
|
||||
target_seconds.as_slice(),
|
||||
&settings,
|
||||
);
|
||||
let output_count = output_paths
|
||||
.iter()
|
||||
.filter(|output_path| output_path.is_file())
|
||||
.count();
|
||||
let missing_output_paths = [
|
||||
temp_dir.join("missing-check-frame-01.png"),
|
||||
temp_dir.join("missing-check-frame-02.png"),
|
||||
];
|
||||
let missing_target_seconds = [0.0, 60.0];
|
||||
let missing_error = extract_video_frames_to_png(
|
||||
&input_path,
|
||||
missing_output_paths.as_slice(),
|
||||
missing_target_seconds.as_slice(),
|
||||
&settings,
|
||||
)
|
||||
.expect_err("a successful ffmpeg exit with a missing frame should still fail the batch");
|
||||
let missing_details = missing_error
|
||||
.details()
|
||||
.cloned()
|
||||
.expect("missing frame error should keep diagnostics");
|
||||
|
||||
let _ = fs::remove_dir_all(&temp_dir);
|
||||
result.expect("safe final sample should produce a frame");
|
||||
result.expect("one ffmpeg process should extract every safe sample");
|
||||
assert_eq!(output_count, 48);
|
||||
assert_eq!(
|
||||
missing_error.body_text(),
|
||||
"ffmpeg 已执行但未产出动作帧文件。"
|
||||
);
|
||||
assert_eq!(missing_details["targetSeconds"], 60.0);
|
||||
assert_eq!(missing_details["missingFrames"][0]["frameIndex"], 2);
|
||||
assert_eq!(missing_details["missingFrames"][0]["targetSeconds"], 60.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -1090,9 +1090,8 @@ impl AppConfig {
|
||||
if let Some(vector_engine_image_request_timeout_ms) =
|
||||
read_first_positive_u64_env(&["VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS"])
|
||||
{
|
||||
// 中文注释:VectorEngine image-2 实测可能超过 500 秒;旧环境文件中常见的 180 秒值不能再提前截断真实生图。
|
||||
config.vector_engine_image_request_timeout_ms = vector_engine_image_request_timeout_ms
|
||||
.max(DEFAULT_VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS);
|
||||
// 单次 attempt 上限允许按环境收短;worker 调用还会受整次任务的绝对 deadline 约束。
|
||||
config.vector_engine_image_request_timeout_ms = vector_engine_image_request_timeout_ms;
|
||||
}
|
||||
|
||||
if let Some(vector_engine_audio_request_timeout_ms) =
|
||||
@@ -1530,9 +1529,8 @@ mod tests {
|
||||
DEFAULT_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS,
|
||||
DEFAULT_EXTERNAL_GENERATION_WORKER_JOB_TIMEOUT_SECONDS,
|
||||
DEFAULT_EXTERNAL_GENERATION_WORKER_LEASE_SECONDS,
|
||||
DEFAULT_EXTERNAL_GENERATION_WORKER_LONG_JOB_TIMEOUT_SECONDS,
|
||||
DEFAULT_VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS, ExternalGenerationMode, LlmProvider,
|
||||
ProcessRole, parse_bool, parse_external_generation_mode, parse_process_role,
|
||||
DEFAULT_EXTERNAL_GENERATION_WORKER_LONG_JOB_TIMEOUT_SECONDS, ExternalGenerationMode,
|
||||
LlmProvider, ProcessRole, parse_bool, parse_external_generation_mode, parse_process_role,
|
||||
};
|
||||
use std::{
|
||||
fs,
|
||||
@@ -1773,10 +1771,7 @@ mod tests {
|
||||
config.vector_engine_base_url,
|
||||
"https://vector.internal.example"
|
||||
);
|
||||
assert_eq!(
|
||||
config.vector_engine_image_request_timeout_ms,
|
||||
DEFAULT_VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS
|
||||
);
|
||||
assert_eq!(config.vector_engine_image_request_timeout_ms, 210_000);
|
||||
assert_eq!(
|
||||
config.hyper3d_base_url,
|
||||
"https://model.internal.example/api/v2"
|
||||
|
||||
@@ -42,14 +42,14 @@ use spacetime_client::{
|
||||
EditorAssetMediaRepairRecordInput, EditorAssetRecord, EditorAssetUpdateRecordInput,
|
||||
EditorCanvasRecord, EditorCanvasViewportRecord, EditorProjectCreateRecordInput,
|
||||
EditorProjectDeleteRecordInput, EditorProjectGetRecordInput,
|
||||
EditorProjectLayoutSaveRecordInput, EditorProjectRecord, EditorProjectRenameRecordInput,
|
||||
EditorProjectResourceCreateRecordInput, EditorProjectResourceMediaRepairRecordInput,
|
||||
EditorProjectResourceRecord, EditorProjectResourceShowcaseUpdateRecordInput,
|
||||
EditorShowcaseAssetLikeToggleRecordInput, EditorShowcaseAssetPublicListRecordInput,
|
||||
EditorShowcaseAssetRecord, EditorShowcaseAssetSubmitRecordInput,
|
||||
EditorShowcaseCampaignConfigGetRecordInput, EditorShowcaseCampaignConfigRecord,
|
||||
ExternalGenerationJobPhaseUpdateError, ExternalGenerationJobPhaseUpdateRecordInput,
|
||||
SpacetimeClientError,
|
||||
EditorProjectLayoutSaveRecordInput, EditorProjectLayoutSaveV2RecordInput, EditorProjectRecord,
|
||||
EditorProjectRenameRecordInput, EditorProjectResourceCreateRecordInput,
|
||||
EditorProjectResourceMediaRepairRecordInput, EditorProjectResourceRecord,
|
||||
EditorProjectResourceShowcaseUpdateRecordInput, EditorShowcaseAssetLikeToggleRecordInput,
|
||||
EditorShowcaseAssetPublicListRecordInput, EditorShowcaseAssetRecord,
|
||||
EditorShowcaseAssetSubmitRecordInput, EditorShowcaseCampaignConfigGetRecordInput,
|
||||
EditorShowcaseCampaignConfigRecord, ExternalGenerationJobPhaseUpdateError,
|
||||
ExternalGenerationJobPhaseUpdateRecordInput, SpacetimeClientError,
|
||||
};
|
||||
|
||||
use crate::{
|
||||
@@ -96,7 +96,8 @@ pub(crate) const EDITOR_PROJECT_ID_PREFIX: &str = "proj-";
|
||||
pub(crate) const EDITOR_RESOURCE_ID_PREFIX: &str = "editor-resource-";
|
||||
pub(crate) const EDITOR_ASSET_FOLDER_ID_PREFIX: &str = "editor-asset-folder-";
|
||||
pub(crate) const EDITOR_ASSET_ID_PREFIX: &str = "editor-asset-";
|
||||
const EDITOR_LAYOUT_MAX_BYTES: usize = 256 * 1024;
|
||||
const EDITOR_LAYOUT_MAX_BYTES: usize = 2 * 1024 * 1024;
|
||||
pub(crate) const EDITOR_LAYOUT_REQUEST_BODY_MAX_BYTES: usize = 4 * 1024 * 1024;
|
||||
pub(crate) const EDITOR_PROJECT_DEFAULT_TITLE: &str = "未命名画布";
|
||||
const EDITOR_ASSET_METADATA_MAX_BYTES: usize = 64 * 1024;
|
||||
const EDITOR_ASSET_DEFAULT_FOLDER_ID: &str = "project";
|
||||
@@ -165,6 +166,7 @@ pub struct EditorCanvasViewportPayload {
|
||||
pub struct EditorProjectLayoutSaveRequest {
|
||||
pub(crate) viewport: EditorCanvasViewportPayload,
|
||||
pub(crate) layers: Value,
|
||||
pub(crate) expected_revision: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
@@ -453,6 +455,8 @@ pub struct EditorProjectResponse {
|
||||
pub struct EditorProjectLayoutSaveResponse {
|
||||
project_id: String,
|
||||
canvas_id: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
revision: Option<u64>,
|
||||
updated_at: String,
|
||||
}
|
||||
|
||||
@@ -742,6 +746,10 @@ pub struct EditorCanvasPayload {
|
||||
title: String,
|
||||
viewport: EditorCanvasViewportPayload,
|
||||
layers: Value,
|
||||
revision: u64,
|
||||
layout_storage_version: u32,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
background_color: Option<String>,
|
||||
created_at: String,
|
||||
updated_at: String,
|
||||
}
|
||||
@@ -1047,25 +1055,53 @@ pub async fn save_editor_project_layout(
|
||||
Extension(authenticated): Extension<AuthenticatedAccessToken>,
|
||||
Json(payload): Json<EditorProjectLayoutSaveRequest>,
|
||||
) -> Result<Json<Value>, AppError> {
|
||||
let expected_revision = payload.expected_revision;
|
||||
let viewport = payload.viewport.into_record();
|
||||
let layers_json = serialize_editor_layers(payload.layers)?;
|
||||
let ack = state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout_ack(EditorProjectLayoutSaveRecordInput {
|
||||
project_id,
|
||||
owner_user_id: authenticated.claims().user_id().to_string(),
|
||||
viewport: payload.viewport.into_record(),
|
||||
layers_json,
|
||||
updated_at_micros: current_utc_micros(),
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
let owner_user_id = authenticated.claims().user_id().to_string();
|
||||
let updated_at_micros = current_utc_micros();
|
||||
let (project_id, canvas_id, revision, updated_at) =
|
||||
if let Some(expected_revision) = expected_revision {
|
||||
let ack = state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout_v2_ack(EditorProjectLayoutSaveV2RecordInput {
|
||||
project_id,
|
||||
owner_user_id,
|
||||
viewport,
|
||||
layers_json,
|
||||
expected_revision,
|
||||
updated_at_micros,
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
(
|
||||
ack.project_id,
|
||||
ack.canvas_id,
|
||||
Some(ack.revision),
|
||||
ack.updated_at,
|
||||
)
|
||||
} else {
|
||||
let ack = state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout_ack(EditorProjectLayoutSaveRecordInput {
|
||||
project_id,
|
||||
owner_user_id,
|
||||
viewport,
|
||||
layers_json,
|
||||
updated_at_micros,
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
(ack.project_id, ack.canvas_id, None, ack.updated_at)
|
||||
};
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorProjectLayoutSaveResponse {
|
||||
project_id: ack.project_id,
|
||||
canvas_id: ack.canvas_id,
|
||||
updated_at: ack.updated_at,
|
||||
project_id,
|
||||
canvas_id,
|
||||
revision,
|
||||
updated_at,
|
||||
},
|
||||
))
|
||||
}
|
||||
@@ -5729,6 +5765,9 @@ fn editor_canvas_payload_from_record(
|
||||
scale: record.viewport.scale,
|
||||
},
|
||||
layers: sanitize_editor_payload_media(record.layers, resources),
|
||||
revision: record.revision,
|
||||
layout_storage_version: record.layout_storage_version,
|
||||
background_color: record.background_color,
|
||||
created_at: record.created_at,
|
||||
updated_at: record.updated_at,
|
||||
}
|
||||
@@ -6205,6 +6244,7 @@ pub(crate) async fn complete_editor_canvas_generation(
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
let project = repair_editor_project_record_inline_media(state, project).await;
|
||||
let expected_revision = project.canvas.revision;
|
||||
let viewport = project.viewport.clone();
|
||||
let project_payload = editor_project_payload_from_record(project);
|
||||
let Some(placeholder) =
|
||||
@@ -6225,17 +6265,15 @@ pub(crate) async fn complete_editor_canvas_generation(
|
||||
if !completion_result.changed {
|
||||
return Ok(None);
|
||||
}
|
||||
let saved = state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout(EditorProjectLayoutSaveRecordInput {
|
||||
project_id: project_id.to_string(),
|
||||
owner_user_id: owner_user_id.to_string(),
|
||||
viewport,
|
||||
layers_json: serialize_editor_layers(completion_result.layers)?,
|
||||
updated_at_micros: current_utc_micros(),
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
let saved = save_editor_project_layout_with_revision_and_get(
|
||||
state,
|
||||
project_id,
|
||||
owner_user_id,
|
||||
viewport,
|
||||
completion_result.layers,
|
||||
expected_revision,
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok(Some(editor_project_payload_from_record(saved)))
|
||||
}
|
||||
@@ -6265,6 +6303,7 @@ pub(crate) async fn complete_editor_canvas_generation_with_items(
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
let project = repair_editor_project_record_inline_media(state, project).await;
|
||||
let expected_revision = project.canvas.revision;
|
||||
let viewport = project.viewport.clone();
|
||||
let project_payload = editor_project_payload_from_record(project);
|
||||
let Some(placeholder) =
|
||||
@@ -6284,17 +6323,15 @@ pub(crate) async fn complete_editor_canvas_generation_with_items(
|
||||
if !completion_result.changed {
|
||||
return Ok(None);
|
||||
}
|
||||
let saved = state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout(EditorProjectLayoutSaveRecordInput {
|
||||
project_id: project_id.to_string(),
|
||||
owner_user_id: owner_user_id.to_string(),
|
||||
viewport,
|
||||
layers_json: serialize_editor_layers(completion_result.layers)?,
|
||||
updated_at_micros: current_utc_micros(),
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
let saved = save_editor_project_layout_with_revision_and_get(
|
||||
state,
|
||||
project_id,
|
||||
owner_user_id,
|
||||
viewport,
|
||||
completion_result.layers,
|
||||
expected_revision,
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok(Some(editor_project_payload_from_record(saved)))
|
||||
}
|
||||
@@ -6328,6 +6365,7 @@ pub(crate) async fn complete_editor_canvas_background_removal(
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
let project = repair_editor_project_record_inline_media(state, project).await;
|
||||
let expected_revision = project.canvas.revision;
|
||||
let viewport = project.viewport.clone();
|
||||
let project_payload = editor_project_payload_from_record(project);
|
||||
let Some(layers) = replace_editor_canvas_layer_with_resource(
|
||||
@@ -6340,19 +6378,39 @@ pub(crate) async fn complete_editor_canvas_background_removal(
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
let saved = state
|
||||
let saved = save_editor_project_layout_with_revision_and_get(
|
||||
state,
|
||||
project_id,
|
||||
owner_user_id,
|
||||
viewport,
|
||||
layers,
|
||||
expected_revision,
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok(Some(editor_project_payload_from_record(saved)))
|
||||
}
|
||||
|
||||
pub(crate) async fn save_editor_project_layout_with_revision_and_get(
|
||||
state: &AppState,
|
||||
project_id: &str,
|
||||
owner_user_id: &str,
|
||||
viewport: EditorCanvasViewportRecord,
|
||||
layers: Value,
|
||||
expected_revision: u64,
|
||||
) -> Result<EditorProjectRecord, AppError> {
|
||||
state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout(EditorProjectLayoutSaveRecordInput {
|
||||
.save_editor_project_layout_v2(EditorProjectLayoutSaveV2RecordInput {
|
||||
project_id: project_id.to_string(),
|
||||
owner_user_id: owner_user_id.to_string(),
|
||||
viewport,
|
||||
layers_json: serialize_editor_layers(layers)?,
|
||||
expected_revision,
|
||||
updated_at_micros: current_utc_micros(),
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
|
||||
Ok(Some(editor_project_payload_from_record(saved)))
|
||||
.map_err(map_editor_project_error)
|
||||
}
|
||||
|
||||
fn replace_editor_canvas_layer_with_resource(
|
||||
@@ -7953,6 +8011,12 @@ pub(crate) fn map_editor_project_error(error: SpacetimeClientError) -> AppError
|
||||
"message": message,
|
||||
}))
|
||||
}
|
||||
SpacetimeClientError::Procedure(message) if message.contains("版本冲突") => {
|
||||
AppError::from_status(StatusCode::CONFLICT).with_details(json!({
|
||||
"provider": "editor-project",
|
||||
"message": message,
|
||||
}))
|
||||
}
|
||||
SpacetimeClientError::Procedure(message) => AppError::from_status(StatusCode::BAD_REQUEST)
|
||||
.with_details(json!({
|
||||
"provider": "editor-project",
|
||||
@@ -8148,6 +8212,15 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_project_revision_conflicts_map_to_http_conflict() {
|
||||
let error = map_editor_project_error(SpacetimeClientError::Procedure(
|
||||
"图片画布版本冲突:期望 revision 4,当前为 5".to_string(),
|
||||
));
|
||||
|
||||
assert_eq!(error.status_code(), StatusCode::CONFLICT);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bgfilter_body_read_failure_audits_as_transport_not_5xx() {
|
||||
// HTTP 状态已成功、但读 body 时链路断裂:外部 API 失败审计必须归类为 transport,
|
||||
@@ -8750,6 +8823,24 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_layout_serialization_accepts_payload_up_to_two_mib() {
|
||||
let payload = json!("x".repeat(EDITOR_LAYOUT_MAX_BYTES - 2));
|
||||
|
||||
let serialized = serialize_editor_layers(payload).expect("two MiB layout should save");
|
||||
|
||||
assert_eq!(serialized.len(), EDITOR_LAYOUT_MAX_BYTES);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_layout_serialization_rejects_payload_over_two_mib() {
|
||||
let payload = json!("x".repeat(EDITOR_LAYOUT_MAX_BYTES - 1));
|
||||
|
||||
let error = serialize_editor_layers(payload).expect_err("oversized layout should fail");
|
||||
|
||||
assert!(error.body_text().contains("图层布局过大"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_asset_metadata_serialization_rejects_nested_inline_media() {
|
||||
let error = serialize_editor_asset_metadata(Some(json!({
|
||||
|
||||
@@ -1049,6 +1049,7 @@ mod tests {
|
||||
base_url: "https://vector.example".to_string(),
|
||||
api_key: "secret".to_string(),
|
||||
request_timeout_ms: 180_000,
|
||||
request_deadline: None,
|
||||
external_api_audit_state: None,
|
||||
external_api_audit_user_id: None,
|
||||
external_api_audit_profile_id: None,
|
||||
|
||||
@@ -32,7 +32,8 @@ use crate::{
|
||||
editor_project_resource_payload_from_record, extract_editor_ui_design_assets_for_owner,
|
||||
generate_editor_icon_spritesheet_for_owner, generate_editor_image_for_owner,
|
||||
map_editor_project_error, normalize_editor_persisted_media_src, normalize_optional_string,
|
||||
serialize_editor_asset_metadata, serialize_editor_layers,
|
||||
save_editor_project_layout_with_revision_and_get, serialize_editor_asset_metadata,
|
||||
serialize_editor_layers,
|
||||
},
|
||||
external_api_auth::ExternalApiPrincipal,
|
||||
http_error::AppError,
|
||||
@@ -62,6 +63,7 @@ pub struct ExternalEditorProjectCreateRequest {
|
||||
pub struct ExternalEditorCanvasSaveRequest {
|
||||
viewport: EditorCanvasViewportPayload,
|
||||
layers: Value,
|
||||
expected_revision: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
@@ -339,17 +341,29 @@ pub async fn save_external_editor_canvas(
|
||||
Json(payload): Json<ExternalEditorCanvasSaveRequest>,
|
||||
) -> Result<Json<Value>, AppError> {
|
||||
require_scope(&principal, SCOPE_EDITOR_CANVAS)?;
|
||||
let project = state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout(EditorProjectLayoutSaveRecordInput {
|
||||
project_id,
|
||||
owner_user_id: principal.owner_user_id().to_string(),
|
||||
viewport: payload.viewport.into_record(),
|
||||
layers_json: serialize_editor_layers(payload.layers)?,
|
||||
updated_at_micros: current_utc_micros(),
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?;
|
||||
let project = if let Some(expected_revision) = payload.expected_revision {
|
||||
save_editor_project_layout_with_revision_and_get(
|
||||
&state,
|
||||
project_id.as_str(),
|
||||
principal.owner_user_id(),
|
||||
payload.viewport.into_record(),
|
||||
payload.layers,
|
||||
expected_revision,
|
||||
)
|
||||
.await?
|
||||
} else {
|
||||
state
|
||||
.spacetime_client()
|
||||
.save_editor_project_layout(EditorProjectLayoutSaveRecordInput {
|
||||
project_id,
|
||||
owner_user_id: principal.owner_user_id().to_string(),
|
||||
viewport: payload.viewport.into_record(),
|
||||
layers_json: serialize_editor_layers(payload.layers)?,
|
||||
updated_at_micros: current_utc_micros(),
|
||||
})
|
||||
.await
|
||||
.map_err(map_editor_project_error)?
|
||||
};
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
@@ -854,6 +868,14 @@ mod tests {
|
||||
.get("priceMudPoints")
|
||||
.is_none()
|
||||
);
|
||||
assert_eq!(
|
||||
parsed["components"]["schemas"]["EditorProject"]["properties"]["layers"]["type"],
|
||||
"array"
|
||||
);
|
||||
assert_eq!(
|
||||
parsed["components"]["schemas"]["EditorCanvas"]["properties"]["layers"]["type"],
|
||||
"array"
|
||||
);
|
||||
assert!(
|
||||
parsed["paths"]
|
||||
.get("/api/external/v1/editor/images/edits")
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
use std::{future::Future, io, pin::Pin, time::Duration};
|
||||
use std::{
|
||||
future::Future,
|
||||
io,
|
||||
pin::Pin,
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
|
||||
use axum::Json;
|
||||
use serde_json::{Value, json};
|
||||
@@ -15,6 +20,8 @@ use tokio::{
|
||||
use tracing::{error, info, warn};
|
||||
|
||||
const MAX_EDITOR_GENERATION_WARNING_CHARS: usize = 2_048;
|
||||
// provider 必须先结束,给失败审计、计费结算和队列终态写回保留有效 lease 内的收尾窗口。
|
||||
const EXTERNAL_GENERATION_WORKER_TERMINAL_WRITE_RESERVE: Duration = Duration::from_secs(60);
|
||||
const EDITOR_GENERATION_SLICE_WARNING_PREFIX: &str = "图集已生成,但自动拆分未完成:";
|
||||
const EDITOR_GENERATION_WARNING_REDACTED_MESSAGE: &str =
|
||||
"自动拆分未完成(告警详情含内联媒体引用,已省略)";
|
||||
@@ -315,6 +322,8 @@ async fn process_external_generation_job(
|
||||
) -> Result<(), String> {
|
||||
let heartbeat_interval = external_generation_worker_heartbeat_interval(lease);
|
||||
let job_timeout = external_generation_worker_job_timeout(&state.config, job.job_kind.as_str());
|
||||
let (job_deadline, provider_deadline) =
|
||||
external_generation_worker_deadlines(Instant::now(), job_timeout);
|
||||
let billing_price_mud_points = match external_generation_billing_price_mud_points(&job) {
|
||||
Ok(price_mud_points) => price_mud_points,
|
||||
Err(message) => {
|
||||
@@ -338,14 +347,19 @@ async fn process_external_generation_job(
|
||||
job.job_id.clone(),
|
||||
billing_claim_attempt,
|
||||
billing_price_mud_points,
|
||||
process_external_generation_job_once(state.clone(), worker_id.clone(), job.clone()),
|
||||
process_external_generation_job_once(
|
||||
state.clone(),
|
||||
worker_id.clone(),
|
||||
job.clone(),
|
||||
provider_deadline,
|
||||
),
|
||||
));
|
||||
let heartbeat =
|
||||
maintain_external_generation_job_lease(&state, &worker_id, &job, lease, heartbeat_interval);
|
||||
match await_external_generation_job_execution(
|
||||
join_external_generation_work(&mut work_handle),
|
||||
heartbeat,
|
||||
job_timeout,
|
||||
job_deadline,
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -446,7 +460,7 @@ enum ExternalGenerationJobExecutionOutcome {
|
||||
async fn await_external_generation_job_execution<W, H>(
|
||||
work: W,
|
||||
heartbeat: H,
|
||||
job_timeout: Duration,
|
||||
job_deadline: Instant,
|
||||
) -> ExternalGenerationJobExecutionOutcome
|
||||
where
|
||||
W: Future<Output = Result<(), String>>,
|
||||
@@ -454,13 +468,13 @@ where
|
||||
{
|
||||
tokio::pin!(work);
|
||||
tokio::pin!(heartbeat);
|
||||
let job_deadline = sleep(job_timeout);
|
||||
tokio::pin!(job_deadline);
|
||||
let job_deadline_sleep = tokio::time::sleep_until(job_deadline.into());
|
||||
tokio::pin!(job_deadline_sleep);
|
||||
|
||||
tokio::select! {
|
||||
biased;
|
||||
result = &mut work => ExternalGenerationJobExecutionOutcome::Finished(result),
|
||||
_ = &mut job_deadline => ExternalGenerationJobExecutionOutcome::TimedOut,
|
||||
_ = &mut job_deadline_sleep => ExternalGenerationJobExecutionOutcome::TimedOut,
|
||||
result = &mut heartbeat => match result {
|
||||
Ok(()) => unreachable!("external generation heartbeat monitor should not finish"),
|
||||
Err(error) => ExternalGenerationJobExecutionOutcome::LeaseRenewalFailed(error),
|
||||
@@ -485,6 +499,7 @@ async fn process_external_generation_job_once(
|
||||
state: AppState,
|
||||
worker_id: String,
|
||||
job: ExternalGenerationJobRecord,
|
||||
provider_deadline: Instant,
|
||||
) -> Result<(), String> {
|
||||
match job.job_kind.as_str() {
|
||||
#[cfg(any())]
|
||||
@@ -752,7 +767,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match generate_editor_image_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
@@ -788,7 +803,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match edit_editor_image_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
@@ -819,7 +834,7 @@ async fn process_external_generation_job_once(
|
||||
}
|
||||
};
|
||||
payload.task_id.get_or_insert_with(|| job.job_id.clone());
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match remove_editor_image_background_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
@@ -849,7 +864,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match generate_editor_icon_spritesheet_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
@@ -885,7 +900,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match extract_editor_ui_design_assets_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
@@ -922,7 +937,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match generate_editor_character_animation_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
@@ -954,7 +969,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match generate_editor_video_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
@@ -985,7 +1000,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match generate_editor_sound_effect_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
@@ -1016,7 +1031,7 @@ async fn process_external_generation_job_once(
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
match generate_editor_background_music_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
@@ -1112,13 +1127,17 @@ async fn fail_queue_job_after_worker_error(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn worker_request_context(job: &ExternalGenerationJobRecord) -> RequestContext {
|
||||
fn worker_request_context(
|
||||
job: &ExternalGenerationJobRecord,
|
||||
provider_deadline: Instant,
|
||||
) -> RequestContext {
|
||||
RequestContext::new(
|
||||
format!("external-generation-worker-{}", job.job_id),
|
||||
format!("external-generation-worker {}", job.job_kind),
|
||||
std::time::Duration::ZERO,
|
||||
false,
|
||||
)
|
||||
.with_external_call_deadline(provider_deadline)
|
||||
}
|
||||
|
||||
fn editor_generation_worker_caller(
|
||||
@@ -1409,11 +1428,29 @@ fn external_generation_worker_heartbeat_interval(lease: Duration) -> Duration {
|
||||
Duration::from_millis(heartbeat_millis)
|
||||
}
|
||||
|
||||
fn external_generation_worker_deadlines(
|
||||
started_at: Instant,
|
||||
job_timeout: Duration,
|
||||
) -> (Instant, Instant) {
|
||||
let terminal_write_reserve =
|
||||
EXTERNAL_GENERATION_WORKER_TERMINAL_WRITE_RESERVE.min(job_timeout / 2);
|
||||
let Some(job_deadline) = started_at.checked_add(job_timeout) else {
|
||||
return (started_at, started_at);
|
||||
};
|
||||
let provider_deadline = started_at
|
||||
.checked_add(job_timeout.saturating_sub(terminal_write_reserve))
|
||||
.unwrap_or(started_at);
|
||||
(job_deadline, provider_deadline)
|
||||
}
|
||||
|
||||
fn external_generation_worker_job_timeout(config: &AppConfig, job_kind: &str) -> Duration {
|
||||
match job_kind {
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND | EDITOR_VIDEO_GENERATION_JOB_KIND => {
|
||||
config.external_generation_worker_long_job_timeout
|
||||
}
|
||||
EDITOR_IMAGE_GENERATION_JOB_KIND
|
||||
| EDITOR_IMAGE_EDIT_JOB_KIND
|
||||
| EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND
|
||||
| EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND
|
||||
| EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND
|
||||
| EDITOR_VIDEO_GENERATION_JOB_KIND => config.external_generation_worker_long_job_timeout,
|
||||
_ => config.external_generation_worker_job_timeout,
|
||||
}
|
||||
}
|
||||
@@ -1556,8 +1593,12 @@ mod tests {
|
||||
std::future::pending::<Result<(), String>>().await
|
||||
};
|
||||
|
||||
let outcome =
|
||||
await_external_generation_job_execution(work, heartbeat, Duration::from_secs(1)).await;
|
||||
let outcome = await_external_generation_job_execution(
|
||||
work,
|
||||
heartbeat,
|
||||
Instant::now() + Duration::from_secs(1),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_eq!(
|
||||
outcome,
|
||||
@@ -1578,9 +1619,12 @@ mod tests {
|
||||
};
|
||||
let heartbeat = std::future::pending::<Result<(), String>>();
|
||||
|
||||
let outcome =
|
||||
await_external_generation_job_execution(work, heartbeat, Duration::from_millis(25))
|
||||
.await;
|
||||
let outcome = await_external_generation_job_execution(
|
||||
work,
|
||||
heartbeat,
|
||||
Instant::now() + Duration::from_millis(25),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_eq!(outcome, ExternalGenerationJobExecutionOutcome::TimedOut);
|
||||
assert!(
|
||||
@@ -1884,7 +1928,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn worker_job_timeout_uses_long_budget_for_video_jobs() {
|
||||
fn worker_job_timeout_uses_long_budget_for_image_and_video_jobs() {
|
||||
let config = AppConfig {
|
||||
external_generation_worker_job_timeout: Duration::from_secs(60),
|
||||
external_generation_worker_long_job_timeout: Duration::from_secs(600),
|
||||
@@ -1893,7 +1937,25 @@ mod tests {
|
||||
|
||||
assert_eq!(
|
||||
external_generation_worker_job_timeout(&config, EDITOR_IMAGE_GENERATION_JOB_KIND),
|
||||
Duration::from_secs(60)
|
||||
Duration::from_secs(600)
|
||||
);
|
||||
assert_eq!(
|
||||
external_generation_worker_job_timeout(&config, EDITOR_IMAGE_EDIT_JOB_KIND),
|
||||
Duration::from_secs(600)
|
||||
);
|
||||
assert_eq!(
|
||||
external_generation_worker_job_timeout(
|
||||
&config,
|
||||
EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND
|
||||
),
|
||||
Duration::from_secs(600)
|
||||
);
|
||||
assert_eq!(
|
||||
external_generation_worker_job_timeout(
|
||||
&config,
|
||||
EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND
|
||||
),
|
||||
Duration::from_secs(600)
|
||||
);
|
||||
assert_eq!(
|
||||
external_generation_worker_job_timeout(
|
||||
@@ -1906,6 +1968,61 @@ mod tests {
|
||||
external_generation_worker_job_timeout(&config, EDITOR_VIDEO_GENERATION_JOB_KIND),
|
||||
Duration::from_secs(600)
|
||||
);
|
||||
assert_eq!(
|
||||
external_generation_worker_job_timeout(&config, EDITOR_BACKGROUND_REMOVAL_JOB_KIND),
|
||||
Duration::from_secs(60)
|
||||
);
|
||||
assert_eq!(
|
||||
external_generation_worker_job_timeout(
|
||||
&config,
|
||||
EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND
|
||||
),
|
||||
Duration::from_secs(60)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn worker_provider_deadline_reserves_terminal_write_budget() {
|
||||
let started_at = Instant::now();
|
||||
let (job_deadline, provider_deadline) =
|
||||
external_generation_worker_deadlines(started_at, Duration::from_secs(1_800));
|
||||
|
||||
assert_eq!(
|
||||
job_deadline.duration_since(started_at),
|
||||
Duration::from_secs(1_800)
|
||||
);
|
||||
assert_eq!(
|
||||
provider_deadline.duration_since(started_at),
|
||||
Duration::from_secs(1_740)
|
||||
);
|
||||
assert_eq!(
|
||||
job_deadline.duration_since(provider_deadline),
|
||||
Duration::from_secs(60)
|
||||
);
|
||||
|
||||
let (short_job_deadline, short_provider_deadline) =
|
||||
external_generation_worker_deadlines(started_at, Duration::from_secs(30));
|
||||
assert_eq!(
|
||||
short_job_deadline.duration_since(started_at),
|
||||
Duration::from_secs(30)
|
||||
);
|
||||
assert_eq!(
|
||||
short_provider_deadline.duration_since(started_at),
|
||||
Duration::from_secs(15)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn worker_request_context_carries_provider_deadline() {
|
||||
let job = external_generation_job_record_fixture(Some("lease-1"));
|
||||
let provider_deadline = Instant::now() + Duration::from_secs(30);
|
||||
|
||||
let request_context = worker_request_context(&job, provider_deadline);
|
||||
|
||||
assert_eq!(
|
||||
request_context.external_call_deadline(),
|
||||
Some(provider_deadline)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user