功能:完成BGM正式提交链路

前端接入规范化提示词、同步排他锁及账号与项目分层回调门禁

关闭BGM正式POST自动重试并补充真实传输层单次请求回归

后端统一站内BGM规范载荷并保留External v1原始载荷幂等语义

补齐队列、直返、平台音频及生命周期边界测试

新增并更新T5独立实施记录
This commit is contained in:
2026-08-05 09:40:02 +00:00
parent 07eb59fb6d
commit 3b69b48860
16 changed files with 1894 additions and 123 deletions
@@ -0,0 +1,259 @@
# BGM 生成提示词优化 T5 正式提交与方案 A 锁实施方案
日期:`2026-08-05`
状态:实施完成,待提交
开发分支:`codex/bgm-generation-opt-v1`
## 一、文档定位
本文承接 T5 的具体工程规划、现役链路调研、改动边界、测试设计和后续实施记录。
- 阶段级目标、依赖关系和总体完成定义继续保留在[总任务拆解](./【实施计划】BGM生成提示词优化V1.0任务拆解-2026-08-04.md)。
- 产品与技术口径以[画板音乐生成入口设计](../docs/【编辑器】画板音乐生成入口设计-2026-06-18.md)为权威。本次修订只补齐该既有口径在现役 queue / inline、scope 所有权和动态价格链路上的实现边界,不新增产品能力。
- T4 的组件与交互细节留在[T4 界面与预设交互实施方案](./【实施方案】BGM生成提示词优化T4界面与预设交互-2026-08-04.md);该文第十二节列出的交接风险由本文承接并给出解法。
- 本文不得反向修改需求文件,也不得用实现便利覆盖权威设计。
## 二、实施基线
已完成提交:
- T3`825b0ccf6`
- T3-R2`3fbdf210b`
- T0 补差:`0dc1eed38`
- T4`d825d8181`
### 2.1 现役正式提交链路调研结论
以下为实施前的实测事实,不是推测:
1. `submitImageGeneration``useImageCanvasGenerationSubmissionWorkflow.ts`)服务 `edit` / `image` / `character` / `video` / `audio-sound-effect` / `audio-background-music` 六种 mode,入口没有任何同步锁。现有防重依赖“重渲染后生成按钮 `disabled`、composer 卸载”,属于渲染后生效的状态防护,不满足方案 A 要求的同步互斥。
2. 入口第一行是 `dialog.prompt.trim() || 默认回退`。它同时踩三个问题:JavaScript `String.trim()` 会额外删除首尾 `U+FEFF`,与 Rust 侧不一致;空值回退为“游戏背景音乐”形成用户不可见 Prompt;紧接着**同步**把 dialog 置为 `status:'generating'``composerOpen:false`
3. 第 3 点会直接打死提交锁:Prompt 助手 hook 的 dialog 清理 effect 看到 `composerOpen === false``clearActiveOperation(abort:true)`,把刚 claim 的 `submitting` operation reject 掉;随后 `finishSubmission` 找不到 active operation 返回 `false`,锁形同虚设,dialog 重新打开后还能再次提交。
4. `buildImageGenerationSubmissionPlan` 内部还会独立调用一次 `getDialogDefaultPrompt``ImageCanvasGenerationSubmissionModel.ts`)。只删除入口那一处回退不够,默认文案会从这里回到请求体。
5. 后端生成记录 `prompt` / `actual_prompt`、canvas layer 和内部完成响应已经取自 `normalized.gpt_description_prompt`;但 queue 入口虽然先计算了 `normalized`,实际传给 `enqueue_editor_generation_job_for_caller` 的仍是原始 `payload`。因此“只替换 normalize 即可让队列等值”的旧判断错误,站内 BGM handler 必须在入队前把 canonical Prompt 和固定 `make_instrumental:true` 写回本次入队 payload。
6. Rust 的 `str::trim()` 语义与本需求 canonicalizer 一致(都按 `char::is_whitespace` 删除两端)。真正的跨语言不一致只存在于 TypeScript 侧;BFF 和 `platform-audio` 仍需防御性调用已有 BGM 专用 validator,以拒绝空值和超限值。
7. BGM composer 的提交锁完全来自 `assistState.status === 'submitting'`,与 `dialog.status` 无关。因此 T5 不需要新增 dialog 状态:提交期间 `dialog.status` 保持 `idle`,面板由助手状态锁定。
8. 状态模型对 `submitting``rejectOperation` 会保留 `undoPromptSnapshot`。“后端拒绝时保留提交前撤销快照”已经具备,T5 不需要修改纯状态模型。
9. `generateEditorBackgroundMusic` 当前给正式 POST 配置了 `retryUnsafeMethods:true`、最多两次 transport retry,而站内 queue 每次请求都会生成新的 job ID。UI 同步锁无法阻止 client 内部重发;T5 必须仅对 BGM 正式 POST关闭这项既有自动重试,不能为此新增幂等协议或改动其它生成 mode。
10. `finishSubmission` 返回 `false` 既可能表示 dialog 被删除,也可能表示账号 / 项目 scope 已变化。正式任务不能因此被取消,但本地 dialog、Prompt、composer 和结果层也不能继续只按 `generation-dialog-N` 写回,因为该 ID 不是跨项目全局唯一。
11. 生产和默认配置走 queue:首次成功响应只携带 `queueState`,此时即可判定站内任务已创建。兼容 inline 模式不会暴露中间接受响应,HTTP Promise 要到生成完成后才返回,不能把两种模式写成同一个可观察时序。
12. `vector_engine_audio_generation/tests.rs` 当前没有被父模块挂载,不能作为 T5 验收依据。新增 Rust 用例必须放进实际编译的模块内测试或真正挂载的精简测试模块。
## 三、既有口径下的实现收口
### 3.1 提交锁与 composer 生命周期
T4 交接要求“后端接受后才隐藏 composer”,同时要求普通 `composerOpen=false` 不得提前 reject `submitting`。两者分别处理正常接受时序和用户主动归档 / 切面板,不是新增交互:
- 当前 BGM 面板保持打开时,提交期间继续可见并全锁;queue 模式收到任务已创建的成功响应后才结束 `submitting` 并切入现有占位。
- 用户在等待期间归档或切走面板时,只把原 dialog 变为 inactive / closed,不取消已经发出的正式请求,也不主动重新激活该 dialog。
- dialog 被删除或账号 / 项目 scope 改变时,可以移除临时助手状态;这不代表正式请求已取消。
### 3.2 正式任务处理与本地 UI 写回必须分开
点击时冻结本次 `{账号、项目、dialogId、operation、canonical Prompt、submission plan}`。后续异步阶段遵循两条独立规则:
1. 已发出的正式请求和后端已经创建的任务继续按现役链路处理,不能因 `finishSubmission` 返回 `false` 被误判为取消。
2. `status``composerOpen`、Prompt、错误信息、本地 `applyProjectSnapshot` / asset upsert 和结果层都属于 UI 写回;每次跨 `await` 后写入前都必须重新确认仍是原账号、原项目和原 dialog。scope 已变化或 dialog 已删除时跳过本地 UI 写回,不能按同名 ID 修改新 scope。
普通 `composerOpen=false` 且 scope / dialog 仍匹配时,`finishSubmission` 仍可结算。若随后失败:
- 原 dialog 仍是当前打开面板:恢复 `failed + composerOpen:true`
- 原 dialog 已被用户归档或切走:写入失败状态和 canonical Prompt,但保留关闭 / inactive,不抢回焦点。
- 原 scope 已失效或 dialog 已删除:不再写本地 UI。
因此,`finishSubmission` 的返回值不能取消正式任务,但也不能被解释成“所有本地结果都可无条件写回”。
### 3.3 可观察提交时点与 transport 边界
```text
点击生成
→ 同步 beginSubmissioncanonicalize、写回、资格校验、冻结 scope 与请求值
→ 只发送一次 BGM 正式 POST,不执行 unsafe 自动重试
├─ queue 接受 → finishSubmission({ operation, accepted: true }) → 原 UI 目标仍有效时切 generating / 隐藏 composer → 现有轮询
├─ 首次 POST 非成功或抛错 → finishSubmission({ operation, accepted: false }) → 原 UI 目标仍有效时记录失败
└─ inline 兼容模式 → 无中间接受响应,保持 submitting 到现有终态响应,再结算并处理结果
```
- T5 不新增 Idempotency-Key、持久提交账本、服务端 dedupe 或新的响应阶段;“一次点击一次请求”通过同步 UI 锁和关闭 BGM 正式 POST 的既有 unsafe retry 实现。
- 请求返回 transport 错误时,客户端不能证明服务端一定未接受;T5 不自动重发。用户之后再次明确点击属于新的提交,不在本切片引入跨页面、跨 scope 的 durable exactly-once 设计。
- 保留现有 BGM 正式请求超时;不新增重试、超时或限流。
## 四、改动边界
- 只改 `audio-background-music``edit` / `image` / `character` / `video` / `audio-sound-effect` 的提交代码路径、文案、默认 Prompt 和错误展示一字不动。
- 实现方式为在 `submitImageGeneration` 内增加 BGM 前置分支,分支结束后复用现有 audio 结果处理(队列、占位、素材库、项目刷新、`addAudioResultLayer`),不复制这段逻辑,也不新建平行提交函数。
- 不新增 dialog 状态、不修改纯状态模型、不新增持久提交锁或全局画布锁。
- 只移除 BGM 正式 POST 的 `EDITOR_REQUEST_RETRY_OPTIONS`;不修改共享 retry 配置或其它请求。
- 不新增 Idempotency-Key、dedupe、请求账本、响应字段或 queue 状态。
- 站内 BGM handler 的 queue payload 执行 canonical 写回;不改变 External v1 的请求、幂等键或 payload 等值语义。
- 不修改 Suno 三字段结构、固定模型、现役动态价格配置和计费包装;客户端不提交价格,服务端继续在入队时冻结价格。
- “正式请求不含助手隐藏元数据”只约束本功能前端构造的 submission payload 和固定三字段 Suno body;不在 T5 新增通用 `generationInputs` allowlist 或恶意客户端清洗器。
- 不修改 SpacetimeDB schema、External v1、External OpenAPI 或 LLM 日志策略。
- 不增加新的重试、超时或限流。
## 五、目标文件结构
|文件|T5 职责|
|---|---|
|`useImageCanvasBackgroundMusicPromptAssist.ts`|清理 effect 豁免 `submitting``finishSubmission` 不再要求 composer 打开|
|`useImageCanvasGenerationWorkflow.ts`|把现有完整 Prompt 助手 controller 和当前 scope 接入 submission workflow|
|`useImageCanvasGenerationSubmissionWorkflow.ts`|BGM 前置分支:同步取锁、冻结 scope / 请求值;分离正式任务处理与 scope-safe UI 写回|
|`ImageCanvasGenerationSubmissionModel.ts`|BGM 分支使用冻结的 canonical Prompt,不再经过默认回退|
|`editorProjectClient.ts`|仅移除 BGM 正式 POST 的 unsafe 自动重试,保留现有超时|
|`server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs`|BGM 入站改用 canonical 校验;站内 handler 用 canonical payload 进入 queue|
|`server-rs/crates/platform-audio/src/request.rs`|BGM Suno body builder 防御性 canonicalization|
|对应现役测试文件|覆盖 client 单次发送、scope 所有权、queue payload、body 与持久化等值;不依赖未挂载的 `vector_engine_audio_generation/tests.rs`|
## 六、T5A:助手生命周期豁免
### 6.1 修改
- 清理 effect 的第二个循环:active operation 为 `submitting` 时跳过,不 abort、不 reject、不移除记录。
- `finishSubmission`:移除“composer 仍打开”的前置条件;operation、scope 和 BGM dialog 仍匹配即可结算。
- 返回值只表示这次助手 operation 是否由原 UI 目标成功结算。正式请求已经发出后,该值不负责取消任务;submission workflow 仍需在后续每个 UI 写回点重新检查冻结 scope。
- 其余 6 处 `composerOpen` 判断保持不变。AI 操作入口在 `submitting` 期间已由 `isSubmissionLocked` 挡住,迟到响应、预设、撤销和 `beginSubmission` 的语义不受影响。
### 6.2 验收
- 提交期间把 `composerOpen``false`(选中其它图层、归档、切面板),`submitting` 不被 reject`finishSubmission` 仍能结算。
- 提交期间删除 dialog:UI 状态移除,`finishSubmission` 返回 `false`,不抛错。
- 提交期间切换账号或项目:助手状态被 reset,`finishSubmission` 返回 `false`;正式任务继续,但旧回调不能写入新 scope 的同 ID dialog。
- T3 / T3-R2 既有的 operation ID、跨 dialog、迟到响应、超时和 scope 用例继续通过。
## 七、T5B:前端 BGM 提交分支
### 7.1 同步阶段(必须早于任何 `await`)
1. 用类型收窄取得带稳定 ID 的 BGM dialog;不是 BGM 则走现有通用路径。
2. `beginSubmission(dialog.id)` 在同一个同步边界内完成 canonicalize、输入框写回和正式生成资格校验;状态模型继续是唯一资格判定源,不在 workflow 再写第二套 0 / 200 校验。
3. 返回 `null` 表示重复点击、scope / dialog 不合格或 Prompt 无正式生成资格,直接结束;canonical 写回按现有 controller 结果保留,不发送请求。
4. claim 成功后冻结 `{账号、项目、dialogId、operation、canonical Prompt}`,并以该 Prompt 构造 submission planBGM 分支不得再经过原生 `trim``getDialogDefaultPrompt`
5. 不在同步阶段切换 `status` / `composerOpen`
### 7.2 异步阶段
- 调用 `generateEditorBackgroundMusic` 一次;该 client 不再配置 unsafe retry。
- queue 首次成功响应代表任务已创建:先调用 `finishSubmission({ operation, accepted: true })`;原 UI 目标仍有效时再设 `status:'generating'``composerOpen:false`,随后复用现有 queue 轮询、项目刷新、素材库和结果处理。
- 首次 POST 非成功或抛错:先调用 `finishSubmission({ operation, accepted: false })`。原 dialog 仍是当前面板时恢复 `failed + composerOpen:true`;已归档 / 切走时保留关闭状态并记录失败;scope 已变化或 dialog 已删除时不写本地 UI。
- queue 已接受后的终态成功 / 失败继续走现有正式链路;每次应用 project snapshot、upsert asset、写 dialog 或本地结果层前重新检查冻结 scope 和 dialog。scope 已失效时跳过这些前端后续写回,后端已经接受的任务继续处理;T5 不新增跨 scope 的恢复或刷新机制。
- inline 兼容模式没有 `queueState`:面板保持 `submitting` 到现有 HTTP 终态响应;成功后结算并直接处理现有结果,失败按上一条恢复,不新增中间响应或占位协议。
### 7.3 验收
- 同一 dialog 快速重复点击只产生一次正式 POST;queue 模式只创建一个 queue job 并沿用该 job 的现有计费链路,inline 模式沿用现有终态处理;retryable HTTP 状态或 transport error 也不会由 client 自动重发。
- 空 Prompt、全 Unicode 空白 Prompt 不产生正式请求。
- 1 个有效字符可以提交;201 个 code point 不能提交。
- 提交期间输入框、预设、展开收起、箭头、AI 补全、一键简化、撤销和生成全部锁定,面板保持可见。
- 后端拒绝后 canonical Prompt 和撤销快照保留;当前面板恢复,用户已经归档 / 切走的面板不被自动激活。
- 后端接受后,原 UI 目标仍有效时才进入占位;画布其它功能全程可用。
- 账号 / 项目 A 的迟到成功或失败不能修改账号 / 项目 B 的同 ID dialogqueue 已接受后再切 scope 也遵守该规则。
- inline 模式在终态响应前保持 `submitting`,不为兼容模式新增协议。
- 其余 5 种 mode 的提交行为无差异。
## 八、T5C:后端 canonicalization
### 8.1 修改
- `normalize_editor_background_music_request_with_pricing` 改用 BGM canonical 校验函数:canonicalize、至少 1 个有效字符、不超过 200 个 code point,返回 canonical Prompt。
- 登录态站内 `generate_editor_background_music` handler 在 queue / inline 分流前,用上述结果构造本次 canonical payload:覆盖 `gpt_description_prompt`,并固定 `make_instrumental:true`。queue 分支必须把这份 payload 交给现有 enqueue helper,不能继续序列化原始请求。
- 不在共享 External v1 调用边界新增 payload 改写;External v1 路由、Idempotency-Key、OpenAPI 和现有 payload 等值判断保持不变。
- `build_editor_background_music_task_body` 执行同一套幂等 canonicalization 后再构造 Suno body。
- BGM 请求不接收客户端价格。服务端继续按现役动态定价解析,并把本次价格冻结到 queue job;后续计费 target、资产成本和内部完成响应继续复用该冻结价格,不硬编码 `5`
- SFX 继续使用现有 `normalize_limited_text`Vidu body、默认 Prompt、时长与 1500 字限制不变。
### 8.2 验收
- 空 Prompt、全空白 Prompt 返回 `400`,不创建任务、不扣费。
- 201 个 code point 拒绝;200 个通过。
- 站内 queue 的实际 `request_payload_json.gptDescriptionPrompt` 与 canonical Prompt 逐 code point 等值,`makeInstrumental``true`;不能只断言 normalize 返回值。
- 首尾 `U+0085` 等 Unicode `White_Space` 被删除;`U+200B``U+FEFF`、组合字符和 ZWJ emoji 不被误删;内部空格与 LF / CRLF 原样保留。
- 连续两次 canonicalization 结果一致。
- Suno body 仍只含 `mv``gpt_description_prompt``make_instrumental`
- queue job 的 `price_mud_points`、计费 target 和内部完成响应使用同一次服务端冻结价格;前端请求中没有 `priceMudPoints`
- External v1 契约与幂等回归无差异。
- SFX 相关 Rust 测试无回归。
## 九、T5D:端到端等值
- 逐层断言写回后的输入框、前端 BFF 请求、站内 generation queue payload、Suno body、`editor_project_resource``editor_asset`、canvas layer 以及内部 `EditorAudioGenerateResponse` 的 Prompt 字段逐 code point 完全一致。
- queue 首次响应仍只断言现有 `queueState`;普通站内 job 的 `result_payload_json` 继续使用现有 metadata-only 形态,不为 T5 增加 `prompt` / `actualPrompt` 字段。`prompt` / `actualPrompt` 的完成响应断言落在 inline 或 worker 内部生成响应边界。
- 断言本功能前端构造的正式 submission payload 不含预设 ID、分组、颜色、助手系统模板、内部引导、AI 快照或格式 / 完整性 / 残句判断字段;Suno body 的 key 集合精确等于三个固定字段。
- 不把上条扩展成 BFF 对任意 `generationInputs` 的通用 allowlist / sanitizer,本切片只保证本功能客户端不发送这些隐藏元数据。
- 用例至少覆盖:含首尾边界空白、内部换行和零宽字符的 Prompt;仅 1 个有效字符的 Prompt;恰好 200 个 code point 的 Prompt。
## 十、测试计划
|层级|场景|期望|
|---|---|---|
|助手 hook|提交期间 `composerOpen=false`|`submitting` 不被 reject`finishSubmission` 可结算|
|助手 hook|提交期间删除 dialog|移除 UI 状态,`finishSubmission` 返回 `false`,不抛错|
|助手 hook|提交期间切换账号 / 项目|状态 reset;正式任务继续,旧 UI 不写入新 scope|
|BGM client|retryable HTTP 状态 / transport error|一次调用只发送一次 POST,不执行 unsafe retry|
|前端提交|双击生成|一次 client 调用、一次正式请求|
|前端提交|空 / 全空白 Prompt|不发请求,保留 canonical 文本|
|前端提交|1 个有效字符 / 200 / 201|前两者可提交,201 拒绝|
|前端提交|当前面板后端拒绝|面板恢复,canonical Prompt 与撤销快照保留|
|前端提交|归档 / 切面板后拒绝|记录失败但保持关闭,不抢焦点|
|前端提交|queue 接受|先结算再切占位,画布其它功能可用|
|前端提交|queue 接受后切 scope,再返回终态|后端已创建任务不被视为取消;新 scope 同 ID dialog 与本地画布不被旧回调修改|
|前端提交|inline 成功 / 失败|终态响应前保持 `submitting`,随后走现有结果 / 失败链路|
|后端|空、全空白、201|`400`,不创建任务|
|后端|`U+200B` / `U+FEFF` / 组合字符 / ZWJ emoji|不被误删|
|后端|幂等性|两次 canonicalization 结果一致|
|后端 queue|站内边界空白 Prompt|实际 `request_payload_json` 使用 canonical Prompt,固定 instrumental,冻结动态价格|
|后端 queue|一次站内正式 POST|创建一个 queue job,并沿用该 job 的单次计费链路|
|平台 body|边界空白 + 内部换行 + 零宽字符|三个固定字段,Prompt 逐 code point 等值|
|端到端|边界空白 + 内部换行 + 零宽字符|前端、queue、Suno、resource、asset、canvas 与内部完成响应等值|
|回归|External v1|路由、OpenAPI、Idempotency-Key 与 payload 等值语义无差异|
|回归|SFX|Vidu body、默认 Prompt、时长与 1500 字限制不变|
|回归|图片 / 视频 / 角色 / edit|提交路径与文案无差异|
Rust 测试必须落入 `cargo test -p api-server` 实际会编译执行的位置,例如 `generation.rs` / `publish.rs` 的内联测试或新挂载的精简测试模块;不得把未挂载的 `vector_engine_audio_generation/tests.rs` 当作通过证据。T5 不借此清理整份历史测试文件,也不扩展计费系统。
## 十一、建议实施顺序
1. T5A:助手生命周期豁免(可与 T5C 并行)。
2. T5C:站内 canonical queue payload 与 `platform-audio` 防御性 canonicalization。
3. BGM client:移除正式 POST 的 unsafe retry;补 controller / scope 接线。
4. T5B:前端 BGM 提交分支和 scope-safe UI 写回(依赖 T5A)。
5. T5Dqueue / inline 分流与端到端等值测试。
6. 定向测试、类型检查、ESLint、Rust 定向测试、编码检查与差异检查。
7. 真实浏览器验证:以默认 queue 模式走通“提交成功进占位”“当前面板失败恢复”“归档后失败不抢焦点”三条路径;inline 兼容语义由定向测试覆盖,不新增联调协议。
建议一个提交:`前后端:收紧BGM生成提交链路`
## 十二、风险与交接
- 本切片是发布切点。T4 与 T5 之间不可发布,因为 T4 面板已按 canonical 口径展示,而正式提交仍走旧 `trim` 与默认回退。
- 空 Prompt 收紧后,历史上依赖默认回退成功的空提交会被前端拦截并被后端拒绝。这是需求明确要求的行为变化。
- 提交期间面板保持可见是既定口径,但与当前“点击立即切占位”的手感不同,需要在浏览器验证中确认无违和。
- 正式请求没有可取消的 `AbortController`。请求发出后即使 dialog 被删除或 scope 切换,也可能已经被后端接受并扣费;实现不得把这类情况当作“已取消”。
- BGM 正式 POST 关闭 unsafe retry 后,transport 结果未知时不会自动重发;T5 不引入 durable 幂等或对账系统。用户随后再次明确点击是新提交,这是本切片保留的既有边界。
- scope 切换会清理临时助手锁。正式任务继续由后端处理,但旧异步回调必须停止本地 UI 写回;T5 不新增跨页面持久锁。
- inline 兼容模式无法观察“上游任务已创建但尚未完成”的中间时刻,因此锁会保持到现有 HTTP 终态响应;生产和默认 queue 行为不受该兼容边界影响。
- 现役 BGM 价格由服务端配置解析并在入队时冻结;T5 不修改价格、计费规则或钱包账本。
- T6 需要汇总本切片测试,并复核完成定义中与提交链路相关的条目。
## 十三、实施记录
|日期|阶段|状态|记录|
|---|---|---|---|
|2026-08-05|T5 初始规划|已被本轮修订取代|完成正式提交链路初查;后续评审确认“queue 已使用 normalized”“权威设计无需修改”两项判断不成立。|
|2026-08-05|T5 规划修订|完成|收口 unsafe POST retry、scope-safe UI 写回、站内 canonical queue payload、queue / inline 可观察时序、动态价格和有效测试落点;明确不新增幂等协议、持久锁、响应字段、通用 sanitizer 或计费设计。|
|2026-08-05|T5A 助手生命周期|完成|修改 `useImageCanvasBackgroundMusicPromptAssist.ts` 及定向测试:`submitting` 不再因普通 composer 关闭被清理,dialog 删除和 scope 切换仍清理,`finishSubmission` 继续校验 operation、scope 与 BGM dialog。|
|2026-08-05|T5B / T5D 正式提交与等值链路|完成|修改 BGM submission model、submission workflow、workflow 接线、BGM `generationInputs` helper、正式 client 及定向测试:同步 claim 早于首个 `await`canonical Prompt 是唯一请求值,关闭 unsafe POST retryqueue / inline 按既定时序结算;冻结 scope 版本和 UI 所有权,覆盖首次响应及 queue 终态前的 `A → B → A` 往返、当前面板失败恢复和归档后不抢焦点;其它生成 mode 保持原路径。|
|2026-08-05|T5C 后端 canonicalization|完成|修改站内 BGM handler、queue 准备边界、`platform-audio` Suno body builder 及现役测试:登录态 queue / inline 使用 canonical Prompt 并固定 `makeInstrumental=true`queue 准备边界绑定 canonical payload 与动态冻结价格,实际 queue serializer 对该 payload 序列化;External v1 继续保留原始 payload 和既有幂等语义,SFX 未改。|
|2026-08-05|T5 修复前验证|完成|前端 5 个定向测试文件 `193/193` 通过,`npm run typecheck` 与变更文件 ESLint 通过;`platform-audio` `31/31`、窄口径 `api-server` BGM 定向 `28/28`、登录态 queue 准备 `1/1`、External v1 `7/7` 通过。该记录保留当时命令口径,后续验收以本表最新记录为准。|
|2026-08-05|T5 评审修复|完成|拆分钱包账号所有权、任务列表账号内项目所有权(同时校验 `currentUserId + projectId`)与 dialog/canvas 完整所有权;真实 transport 层覆盖 BGM 正式 POST 遇到 `429``503``TypeError` 均只发送一次;补齐 External v1 BGM 原始 payload 保留、同原值幂等重放及 canonical 等价值冲突回归。未修改 SFX、其它生成 mode、通用 retry、计费、Schema、External 路由 / DTO / OpenAPI 或 worker。|
|2026-08-05|T5 修复后验证|完成|前端 6 个定向测试文件 `199/199` 通过,其中真实 transport `3/3``npm run typecheck` 与变更文件 ESLint 通过;`platform-audio` `31/31`、广义 `cargo test -p api-server background_music` `35/35`、External v1 既有回归 `7/7` 通过;Rust 格式、编码与 `git diff --check` 通过。真实 queue 浏览器 smoke 会创建正式任务并可能扣费,本切片不执行。|
后续每次实施只在本节追加:
- 完成的切片;
- 实际修改文件;
- 验证命令与结果;
- 尚存风险;
- 对下一阶段的交接。
@@ -132,6 +132,21 @@ where
)
.await?;
ensure_external_api_generation_job_matches_request(
&job,
owner_user_id,
job_kind,
&request_payload_json,
)?;
Ok(job)
}
fn ensure_external_api_generation_job_matches_request(
job: &ExternalGenerationJobRecord,
owner_user_id: &str,
job_kind: &str,
request_payload_json: &str,
) -> Result<(), AppError> {
if job.job_kind != job_kind
|| job.owner_user_id != owner_user_id
|| job.request_payload_json != request_payload_json
@@ -143,7 +158,7 @@ where
})),
);
}
Ok(job)
Ok(())
}
#[allow(clippy::too_many_arguments)]
@@ -264,6 +279,16 @@ where
Ok(request_payload_json)
}
#[cfg(test)]
pub(crate) fn serialize_editor_generation_job_payload_for_test<T>(
payload: &T,
) -> Result<String, AppError>
where
T: Serialize + ?Sized,
{
serialize_editor_generation_job_payload(payload)
}
fn payload_serialization_error(error: serde_json::Error) -> AppError {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
"provider": EDITOR_GENERATION_QUEUE_PROVIDER,
@@ -423,6 +448,54 @@ mod tests {
assert!(error.body_text().contains("超过持久化上限"));
}
#[test]
fn external_background_music_idempotency_matches_the_raw_payload_exactly() {
let raw_prompt = "\u{0085}\u{2003}森林音乐\u{00a0}";
let raw_payload = shared_contracts::assets::EditorBackgroundMusicGenerateRequest {
gpt_description_prompt: raw_prompt.to_string(),
make_instrumental: false,
project_id: Some("project-1".to_string()),
canvas_completion: None,
generation_inputs: None,
asset_folder_id: None,
asset_label: None,
};
let raw_payload_json = serialize_editor_generation_job_payload(&raw_payload)
.expect("External BGM 原始 payload 应可序列化");
let mut persisted_job = queue_job_fixture("pending", None);
persisted_job.job_kind = EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND.to_string();
persisted_job.request_payload_json = raw_payload_json.clone();
ensure_external_api_generation_job_matches_request(
&persisted_job,
"user-1",
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
&raw_payload_json,
)
.expect("同一原始 payload 的幂等重放应复用既有任务");
let canonical_payload = shared_contracts::assets::EditorBackgroundMusicGenerateRequest {
gpt_description_prompt: "森林音乐".to_string(),
..raw_payload
};
let canonical_payload_json = serialize_editor_generation_job_payload(&canonical_payload)
.expect("canonical 等价值应可序列化");
let error = ensure_external_api_generation_job_matches_request(
&persisted_job,
"user-1",
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
&canonical_payload_json,
)
.expect_err("同一幂等键改用 canonical 等价值必须按既有精确语义冲突");
assert_eq!(error.status_code(), StatusCode::CONFLICT);
assert!(
error
.body_text()
.contains("Idempotency-Key 已用于不同的生成请求")
);
}
#[test]
fn queue_state_maps_idempotent_replays_to_the_persisted_status() {
let cases = [
@@ -55,6 +55,36 @@ pub(super) struct NormalizedEditorBackgroundMusicRequest {
pub(super) price_mud_points: u32,
}
#[derive(Debug)]
struct PreparedEditorBackgroundMusicQueueJob {
payload: assets::EditorBackgroundMusicGenerateRequest,
price_mud_points: u32,
}
#[derive(Debug)]
struct CanonicalEditorBackgroundMusicSubmissionPayload {
payload: assets::EditorBackgroundMusicGenerateRequest,
}
impl CanonicalEditorBackgroundMusicSubmissionPayload {
fn new(payload: assets::EditorBackgroundMusicGenerateRequest) -> Result<Self, AppError> {
Ok(Self {
payload: canonicalize_editor_background_music_submission_payload(payload)?,
})
}
fn into_queue_job(
self,
pricing: &EditorGenerationPricingConfig,
) -> Result<PreparedEditorBackgroundMusicQueueJob, AppError> {
prepare_editor_background_music_queue_job(self.payload, pricing)
}
fn into_payload(self) -> assets::EditorBackgroundMusicGenerateRequest {
self.payload
}
}
#[cfg(test)]
pub(super) fn normalize_editor_sound_effect_request(
payload: assets::EditorSoundEffectGenerateRequest,
@@ -120,20 +150,40 @@ pub(super) fn normalize_editor_background_music_request_with_pricing(
payload: assets::EditorBackgroundMusicGenerateRequest,
pricing: &EditorGenerationPricingConfig,
) -> Result<NormalizedEditorBackgroundMusicRequest, AppError> {
let payload = canonicalize_editor_background_music_submission_payload(payload)?;
let expected_price_mud_points =
pricing.background_music_model_mud_points(Some(platform_audio::SUNO_DEFAULT_MODEL));
Ok(NormalizedEditorBackgroundMusicRequest {
gpt_description_prompt: platform_audio::normalize_limited_text(
&payload.gpt_description_prompt,
"gpt_description_prompt",
platform_audio::SUNO_GPT_DESCRIPTION_PROMPT_MAX_CHARS,
)
.map_err(map_platform_audio_error)?,
make_instrumental: true,
gpt_description_prompt: payload.gpt_description_prompt,
make_instrumental: payload.make_instrumental,
price_mud_points: expected_price_mud_points,
})
}
fn canonicalize_editor_background_music_submission_payload(
mut payload: assets::EditorBackgroundMusicGenerateRequest,
) -> Result<assets::EditorBackgroundMusicGenerateRequest, AppError> {
let validated = platform_audio::validate_background_music_generation_prompt(
&payload.gpt_description_prompt,
)
.map_err(map_platform_audio_error)?;
payload.gpt_description_prompt = validated.prompt.to_string();
payload.make_instrumental = true;
Ok(payload)
}
fn prepare_editor_background_music_queue_job(
payload: assets::EditorBackgroundMusicGenerateRequest,
pricing: &EditorGenerationPricingConfig,
) -> Result<PreparedEditorBackgroundMusicQueueJob, AppError> {
let normalized =
normalize_editor_background_music_request_with_pricing(payload.clone(), pricing)?;
Ok(PreparedEditorBackgroundMusicQueueJob {
payload,
price_mud_points: normalized.price_mud_points,
})
}
pub async fn generate_editor_sound_effect(
State(state): State<AppState>,
Extension(request_context): Extension<RequestContext>,
@@ -372,14 +422,15 @@ pub async fn generate_editor_background_music(
payload: Result<Json<assets::EditorBackgroundMusicGenerateRequest>, JsonRejection>,
) -> Result<Json<Value>, Response> {
let Json(payload) = parse_json_payload(&request_context, payload)?;
let payload = CanonicalEditorBackgroundMusicSubmissionPayload::new(payload)
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
let owner_user_id = authenticated.claims().user_id().to_string();
if !state.config.external_generation_mode.is_inline() {
let queue_job = enqueue_editor_background_music_generation_for_owner(
let queue_job = enqueue_logged_in_editor_background_music_generation_for_owner(
&state,
&request_context,
owner_user_id.as_str(),
payload,
None,
)
.await?;
return Ok(json_success_body(
@@ -393,7 +444,67 @@ pub async fn generate_editor_background_music(
state,
request_context,
owner_user_id,
Ok(Json(payload)),
Ok(Json(payload.into_payload())),
)
.await
}
async fn load_editor_background_music_queue_pricing(
state: &AppState,
request_context: &RequestContext,
) -> Result<EditorGenerationPricingConfig, Response> {
state.editor_generation_pricing().await.map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR)
.with_details(json!({
"provider": "editor-generation-pricing",
"message": error.to_string(),
}))
.into_response_with_context(Some(request_context))
})
}
async fn enqueue_prepared_editor_background_music_generation_for_owner(
state: &AppState,
request_context: &RequestContext,
owner_user_id: &str,
prepared: PreparedEditorBackgroundMusicQueueJob,
external_idempotency_key: Option<&str>,
) -> Result<ExternalGenerationJobRecord, Response> {
let source_entity_id = editor_generation_source_entity_id(
prepared.payload.project_id.as_deref(),
"editor-background-music",
);
enqueue_editor_generation_job_for_caller(
state,
request_context,
owner_user_id,
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
source_entity_id,
"图片画布生成背景音乐",
u64::from(prepared.price_mud_points),
&prepared.payload,
external_idempotency_key,
)
.await
.map_err(|error| error.into_response_with_context(Some(request_context)))
}
async fn enqueue_logged_in_editor_background_music_generation_for_owner(
state: &AppState,
request_context: &RequestContext,
owner_user_id: &str,
payload: CanonicalEditorBackgroundMusicSubmissionPayload,
) -> Result<ExternalGenerationJobRecord, Response> {
let pricing = load_editor_background_music_queue_pricing(state, request_context).await?;
let prepared = payload
.into_queue_job(&pricing)
.map_err(|error| error.into_response_with_context(Some(request_context)))?;
enqueue_prepared_editor_background_music_generation_for_owner(
state,
request_context,
owner_user_id,
prepared,
None,
)
.await
}
@@ -405,34 +516,17 @@ pub(crate) async fn enqueue_editor_background_music_generation_for_owner(
payload: assets::EditorBackgroundMusicGenerateRequest,
external_idempotency_key: Option<&str>,
) -> Result<ExternalGenerationJobRecord, Response> {
let pricing = state.editor_generation_pricing().await.map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR)
.with_details(json!({
"provider": "editor-generation-pricing",
"message": error.to_string(),
}))
.into_response_with_context(Some(request_context))
})?;
let normalized =
normalize_editor_background_music_request_with_pricing(payload.clone(), &pricing)
.map_err(|error| error.into_response_with_context(Some(request_context)))?;
let source_entity_id = editor_generation_source_entity_id(
payload.project_id.as_deref(),
"editor-background-music",
);
enqueue_editor_generation_job_for_caller(
let pricing = load_editor_background_music_queue_pricing(state, request_context).await?;
let prepared = prepare_editor_background_music_queue_job(payload, &pricing)
.map_err(|error| error.into_response_with_context(Some(request_context)))?;
enqueue_prepared_editor_background_music_generation_for_owner(
state,
request_context,
owner_user_id,
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
source_entity_id,
"图片画布生成背景音乐",
u64::from(normalized.price_mud_points),
&payload,
prepared,
external_idempotency_key,
)
.await
.map_err(|error| error.into_response_with_context(Some(request_context)))
}
pub(crate) async fn generate_editor_background_music_for_owner(
@@ -630,3 +724,110 @@ fn editor_audio_bad_request(message: impl Into<String>) -> AppError {
"message": message.into(),
}))
}
#[cfg(test)]
mod tests {
use serde_json::Value;
use shared_contracts::assets;
use super::{
CanonicalEditorBackgroundMusicSubmissionPayload, normalize_editor_background_music_request,
prepare_editor_background_music_queue_job,
};
use crate::editor_generation_queue::serialize_editor_generation_job_payload_for_test;
fn background_music_payload(
prompt: impl Into<String>,
make_instrumental: bool,
) -> assets::EditorBackgroundMusicGenerateRequest {
assets::EditorBackgroundMusicGenerateRequest {
gpt_description_prompt: prompt.into(),
make_instrumental,
project_id: Some("project-1".to_string()),
canvas_completion: None,
generation_inputs: None,
asset_folder_id: None,
asset_label: None,
}
}
#[test]
fn editor_background_music_normalization_uses_canonical_generation_prompt() {
let prompt = "\u{0085} \u{2003}森林 \r\n\u{200b}\u{feff}音乐 \u{00a0}";
let normalized =
normalize_editor_background_music_request(background_music_payload(prompt, false))
.expect("canonical BGM prompt should be accepted");
assert_eq!(
normalized.gpt_description_prompt,
"森林 \r\n\u{200b}\u{feff}音乐"
);
assert!(normalized.make_instrumental);
for invalid_prompt in [" \r\n\u{0085}", &"".repeat(201)] {
let error = normalize_editor_background_music_request(background_music_payload(
invalid_prompt,
true,
))
.expect_err("invalid canonical BGM prompt should be rejected");
assert_eq!(error.status_code(), axum::http::StatusCode::BAD_REQUEST);
}
}
#[test]
fn logged_in_background_music_queue_preparation_uses_canonical_prompt_and_frozen_price() {
let pricing =
crate::editor_generation_config::load_editor_generation_pricing_from_paths(None)
.expect("default model pricing should be valid");
let prepared =
CanonicalEditorBackgroundMusicSubmissionPayload::new(background_music_payload(
"\u{0085}\u{2003}森林 \r\n\u{200b}\u{feff}音乐\u{00a0}",
false,
))
.expect("logged-in submission should canonicalize")
.into_queue_job(&pricing)
.expect("logged-in queue job should prepare");
let request_payload_json =
serialize_editor_generation_job_payload_for_test(&prepared.payload)
.expect("the actual queue serializer should accept the canonical payload");
let request_payload: Value =
serde_json::from_str(&request_payload_json).expect("queue payload should be JSON");
assert_eq!(
request_payload["gptDescriptionPrompt"],
"森林 \r\n\u{200b}\u{feff}音乐"
);
assert_eq!(request_payload["makeInstrumental"], true);
assert!(request_payload.get("priceMudPoints").is_none());
assert_eq!(
prepared.price_mud_points,
pricing.background_music_model_mud_points(Some(platform_audio::SUNO_DEFAULT_MODEL))
);
}
#[test]
fn external_background_music_queue_preparation_preserves_the_raw_payload() {
let pricing =
crate::editor_generation_config::load_editor_generation_pricing_from_paths(None)
.expect("default model pricing should be valid");
let raw_prompt = "\u{0085}\u{2003}森林 \r\n\u{200b}\u{feff}音乐\u{00a0}";
let prepared = prepare_editor_background_music_queue_job(
background_music_payload(raw_prompt, false),
&pricing,
)
.expect("External BGM queue job should validate without rewriting its payload");
let request_payload_json =
serialize_editor_generation_job_payload_for_test(&prepared.payload)
.expect("the actual queue serializer should accept the raw External payload");
let request_payload: Value =
serde_json::from_str(&request_payload_json).expect("queue payload should be JSON");
assert_eq!(request_payload["gptDescriptionPrompt"], raw_prompt);
assert_eq!(request_payload["makeInstrumental"], false);
assert!(request_payload.get("priceMudPoints").is_none());
assert_eq!(
prepared.price_mud_points,
pricing.background_music_model_mud_points(Some(platform_audio::SUNO_DEFAULT_MODEL))
);
}
}
@@ -2,9 +2,8 @@ use serde_json::{Map, Value, json};
use crate::{
AudioError, BackgroundMusicTaskRequest, EditorBackgroundMusicTaskRequest,
EditorSoundEffectTaskRequest, SUNO_DEFAULT_MODEL, SUNO_GPT_DESCRIPTION_PROMPT_MAX_CHARS,
SUNO_PROMPT_MAX_CHARS, SUNO_TAGS_MAX_CHARS, SUNO_TITLE_MAX_CHARS, SoundEffectTaskRequest,
VIDU_AUDIO_MODEL, VIDU_PROMPT_MAX_CHARS,
EditorSoundEffectTaskRequest, SUNO_DEFAULT_MODEL, SUNO_PROMPT_MAX_CHARS, SUNO_TAGS_MAX_CHARS,
SUNO_TITLE_MAX_CHARS, SoundEffectTaskRequest, VIDU_AUDIO_MODEL, VIDU_PROMPT_MAX_CHARS,
};
pub fn build_background_music_task_body(
@@ -62,16 +61,13 @@ pub fn build_sound_effect_task_body(request: SoundEffectTaskRequest) -> Result<V
pub fn build_editor_background_music_task_body(
request: EditorBackgroundMusicTaskRequest,
) -> Result<Value, AudioError> {
let prompt = normalize_limited_text(
&request.gpt_description_prompt,
"gpt_description_prompt",
SUNO_GPT_DESCRIPTION_PROMPT_MAX_CHARS,
)?;
let prompt =
crate::validate_background_music_generation_prompt(&request.gpt_description_prompt)?;
let model = normalize_optional_text(request.model.as_deref())
.unwrap_or_else(|| SUNO_DEFAULT_MODEL.to_string());
Ok(json!({
"mv": model,
"gpt_description_prompt": prompt,
"gpt_description_prompt": prompt.prompt,
"make_instrumental": request.make_instrumental,
}))
}
@@ -287,15 +287,27 @@ fn sound_effect_request_rejects_overlong_prompt() {
#[test]
fn editor_background_music_request_body_uses_gpt_description_prompt_and_instrumental() {
let body = build_editor_background_music_task_body(EditorBackgroundMusicTaskRequest {
gpt_description_prompt: " 轻快森林冒险背景音乐 ".to_string(),
gpt_description_prompt: "\u{0085}\u{2003}轻快森林 \r\n\u{200b}\u{feff}冒险背景音乐\u{00a0}"
.to_string(),
make_instrumental: true,
model: None,
})
.expect("editor background music body should be valid");
assert_eq!(body["mv"], SUNO_DEFAULT_MODEL);
assert_eq!(body["gpt_description_prompt"], "轻快森林冒险背景音乐");
assert_eq!(
body["gpt_description_prompt"],
"轻快森林 \r\n\u{200b}\u{feff}冒险背景音乐"
);
assert_eq!(body["make_instrumental"], true);
assert_eq!(
body.as_object()
.expect("editor background music body should be an object")
.keys()
.map(String::as_str)
.collect::<std::collections::BTreeSet<_>>(),
std::collections::BTreeSet::from(["gpt_description_prompt", "make_instrumental", "mv",])
);
assert!(body.get("prompt").is_none());
}
@@ -325,6 +337,18 @@ fn editor_background_music_request_body_rejects_overlong_gpt_description_prompt(
assert!(error.message().contains("gpt_description_prompt 超过"));
}
#[test]
fn editor_background_music_request_body_rejects_unicode_whitespace_only_prompt() {
let error = build_editor_background_music_task_body(EditorBackgroundMusicTaskRequest {
gpt_description_prompt: "\u{0085}\u{2003}\r\n".to_string(),
make_instrumental: true,
model: None,
})
.expect_err("Unicode-whitespace-only prompt should fail");
assert!(error.message().contains("至少需要 1 个有效字符"));
}
#[test]
fn vidu_sound_effect_request_body_uses_text2audio_contract() {
let body = build_sound_effect_task_body(platform_audio::SoundEffectTaskRequest {
@@ -1071,10 +1071,12 @@ export function buildBackgroundMusicGenerationInputs(
gptDescriptionPrompt: string,
): CanvasGenerationInputs {
return {
fields: createGenerationInputField(
'gpt_description_prompt',
gptDescriptionPrompt,
),
fields: [
{
title: 'gpt_description_prompt',
value: gptDescriptionPrompt,
},
],
references: [],
};
}
@@ -1074,19 +1074,20 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
const plan = buildImageGenerationSubmissionPlan({
dialog: {
mode: 'audio-background-music',
prompt: ' 温暖轻快的森林冒险背景音乐 ',
prompt: '不会进入正式请求的旧值',
status: 'idle',
},
layers: [],
nextGeneratedIndex: 5,
canonicalBackgroundMusicPrompt: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
});
expect(plan).toEqual({
kind: 'audio',
audioKind: 'background-music',
normalizedPrompt: '温暖轻快的森林冒险背景音乐',
normalizedPrompt: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
input: {
gptDescriptionPrompt: '温暖轻快的森林冒险背景音乐',
gptDescriptionPrompt: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
makeInstrumental: true,
},
result: {
@@ -1095,7 +1096,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
fields: [
{
title: 'gpt_description_prompt',
value: '温暖轻快的森林冒险背景音乐',
value: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
},
],
references: [],
@@ -1103,4 +1104,18 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
},
});
});
it('does not build a background music plan without a claimed canonical prompt', () => {
expect(() =>
buildImageGenerationSubmissionPlan({
dialog: {
mode: 'audio-background-music',
prompt: ' 不应原生 trim 或回退默认值 ',
status: 'idle',
},
layers: [],
nextGeneratedIndex: 1,
}),
).toThrow('背景音乐提交缺少已确认的提示词');
});
});
@@ -55,6 +55,7 @@ type ImageGenerationSubmissionOptions = {
dialog: GenerateDialogState;
layers: CanvasLayer[];
nextGeneratedIndex: number;
canonicalBackgroundMusicPrompt?: string;
};
export const EDITOR_GENERATED_ASSET_LABEL_MAX_CHARS = 80;
@@ -137,9 +138,6 @@ function getDialogDefaultPrompt(mode: GenerateDialogState['mode']) {
if (mode === 'audio-sound-effect') {
return '游戏音效';
}
if (mode === 'audio-background-music') {
return '游戏背景音乐';
}
return 'AI 生成图片';
}
@@ -228,7 +226,32 @@ export function buildImageGenerationSubmissionPlan({
dialog,
layers,
nextGeneratedIndex,
canonicalBackgroundMusicPrompt,
}: ImageGenerationSubmissionOptions): ImageGenerationSubmissionPlan {
if (dialog.mode === 'audio-background-music') {
if (canonicalBackgroundMusicPrompt === undefined) {
throw new Error('背景音乐提交缺少已确认的提示词');
}
return {
kind: 'audio',
audioKind: 'background-music',
normalizedPrompt: canonicalBackgroundMusicPrompt,
input: {
gptDescriptionPrompt: canonicalBackgroundMusicPrompt,
makeInstrumental: true,
},
result: {
title: resolveGenerationAssetLabel(
dialog.assetLabel,
`游戏背景音乐 ${nextGeneratedIndex}`,
),
generationInputs: buildBackgroundMusicGenerationInputs(
canonicalBackgroundMusicPrompt,
),
},
};
}
const normalizedPrompt =
dialog.prompt.trim() || getDialogDefaultPrompt(dialog.mode);
@@ -537,26 +560,6 @@ export function buildImageGenerationSubmissionPlan({
};
}
if (dialog.mode === 'audio-background-music') {
return {
kind: 'audio',
audioKind: 'background-music',
normalizedPrompt,
input: {
gptDescriptionPrompt: normalizedPrompt,
makeInstrumental: true,
},
result: {
title: resolveGenerationAssetLabel(
dialog.assetLabel,
`游戏背景音乐 ${nextGeneratedIndex}`,
),
generationInputs:
buildBackgroundMusicGenerationInputs(normalizedPrompt),
},
};
}
const imageModel = normalizeEditorImageModel(dialog.imageModel);
return {
kind: 'image',
@@ -1004,7 +1004,7 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
});
});
it('preserves undo on submission rejection and ignores finishes for closed or deleted dialogs', async () => {
it('preserves undo on rejection, settles a closed composer, and ignores a deleted dialog', async () => {
promptAssistClientMocks.complete.mockResolvedValueOnce(
promptAssistResponse('完整的森林冒险背景音乐'),
);
@@ -1106,7 +1106,7 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
await waitFor(() => {
expect(
result.current.promptAssist.getDialogState('dialog-close').status,
).toBe('idle');
).toBe('submitting');
expect(
Object.prototype.hasOwnProperty.call(
result.current.promptAssist.dialogStates,
@@ -1115,13 +1115,12 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
).toBe(false);
});
const statesBeforeFinish = result.current.promptAssist.dialogStates;
let closeFinished = true;
let closeFinished = false;
let deleteFinished = true;
act(() => {
closeFinished = result.current.promptAssist.finishSubmission({
operation: closeClaim.operation,
accepted: false,
accepted: true,
});
deleteFinished = result.current.promptAssist.finishSubmission({
operation: deleteClaim.operation,
@@ -1129,9 +1128,20 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
});
});
expect(closeFinished).toBe(false);
expect(closeFinished).toBe(true);
expect(deleteFinished).toBe(false);
expect(result.current.promptAssist.dialogStates).toBe(statesBeforeFinish);
expect(
result.current.promptAssist.getDialogState('dialog-close'),
).toMatchObject({
status: 'idle',
operationId: null,
});
expect(
Object.prototype.hasOwnProperty.call(
result.current.promptAssist.dialogStates,
'dialog-delete',
),
).toBe(false);
expect(
requireDialog(result.current.dialogs, 'dialog-close').composerOpen,
).toBe(false);
@@ -1139,4 +1149,53 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
result.current.dialogs.some((dialog) => dialog.id === 'dialog-delete'),
).toBe(false);
});
it('rejects a submission finish after the committed scope changes', () => {
const initialDialogs = [createDialog('dialog-a', '森林冒险')];
const { result, rerender } = renderHook(
({ currentUserId, projectId }: Omit<HarnessProps, 'initialDialogs'>) =>
usePromptAssistHarness({
initialDialogs,
currentUserId,
projectId,
}),
{
initialProps: {
currentUserId: 'user-a',
projectId: 'project-a',
},
},
);
let claim!: NonNullable<
ReturnType<typeof result.current.promptAssist.beginSubmission>
>;
act(() => {
claim = result.current.promptAssist.beginSubmission('dialog-a')!;
});
rerender({
currentUserId: 'user-a',
projectId: 'project-b',
});
let finished = true;
act(() => {
finished = result.current.promptAssist.finishSubmission({
operation: claim.operation,
accepted: true,
});
});
expect(finished).toBe(false);
expect(result.current.promptAssist.getDialogState('dialog-a')).toMatchObject(
{
status: 'idle',
operationId: null,
},
);
expect(requireDialog(result.current.dialogs, 'dialog-a').prompt).toBe(
'森林冒险',
);
});
});
@@ -265,7 +265,10 @@ export function useImageCanvasBackgroundMusicPromptAssist({
}
}
for (const [dialogId] of activeOperationsRef.current) {
for (const [dialogId, activeOperation] of activeOperationsRef.current) {
if (activeOperation.operation.status === 'submitting') {
continue;
}
const dialog = getCanvasGenerationDialogById(dialogId);
if (!isBackgroundMusicDialog(dialog) || dialog.composerOpen === false) {
clearActiveOperation(dialogId, {
@@ -650,8 +653,7 @@ export function useImageCanvasBackgroundMusicPromptAssist({
const latestDialog = getCanvasGenerationDialogById(operation.dialogId);
if (
!Object.is(activeOperation.scopeKey, committedScopeKeyRef.current) ||
!isBackgroundMusicDialog(latestDialog) ||
latestDialog.composerOpen === false
!isBackgroundMusicDialog(latestDialog)
) {
clearActiveOperation(operation.dialogId, {
abort: false,
File diff suppressed because it is too large Load Diff
@@ -3,7 +3,9 @@ import {
type MutableRefObject,
type SetStateAction,
useCallback,
useLayoutEffect,
useMemo,
useRef,
} from 'react';
import type { ExternalGenerationJobStatusRecord } from '../../../packages/shared/src/contracts/externalGeneration';
@@ -83,6 +85,7 @@ import type {
} from './ImageCanvasUiAssetExtractionModel';
import { resolveUiAssetExtractionGenerationPlan } from './ImageCanvasUiAssetExtractionModel';
import { renderUiDesignAssetExtractionMarkedImage } from './ImageCanvasUiAssetExtractionRasterModel';
import type { BackgroundMusicPromptAssistController } from './useImageCanvasBackgroundMusicPromptAssist';
type CanvasSize = { width: number; height: number };
@@ -282,6 +285,11 @@ type GenerationSubmissionWorkflowOptions = {
updater: CanvasGenerationDialogUpdater,
) => void;
hasCanvasGenerationDialogById: (dialogId: string) => boolean;
getCanvasGenerationDialogById: (
dialogId: string,
) => CanvasGenerationDialogState | undefined;
activeCanvasGenerationDialogId?: string | null;
backgroundMusicPromptAssist: BackgroundMusicPromptAssistController;
getGeneratingDialogPlaceholder: (
dialog: GenerateDialogState,
) => GenerateDialogState['placeholder'];
@@ -301,6 +309,7 @@ type GenerationSubmissionWorkflowOptions = {
setActiveSidebarPanel: Dispatch<SetStateAction<SidebarPanel | null>>;
rememberImageModel: (imageModel: string) => void;
projectId?: string | null;
currentUserId?: string | null;
assetFolderId?: string | null;
upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void;
applyProjectSnapshot?: (project: EditorProjectSnapshot) => void;
@@ -606,6 +615,9 @@ export function useImageCanvasGenerationSubmissionWorkflow({
openCanvasGenerationDialog,
updateCanvasGenerationDialogById,
hasCanvasGenerationDialogById,
getCanvasGenerationDialogById,
activeCanvasGenerationDialogId,
backgroundMusicPromptAssist,
getGeneratingDialogPlaceholder,
appendCanvasLayersWithResources,
captureCanvasHistory,
@@ -616,6 +628,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
setActiveSidebarPanel,
rememberImageModel,
projectId,
currentUserId,
assetFolderId,
upsertGeneratedAsset,
applyProjectSnapshot,
@@ -623,6 +636,76 @@ export function useImageCanvasGenerationSubmissionWorkflow({
onWalletBalanceMayHaveChanged,
onGenerationWarning,
}: GenerationSubmissionWorkflowOptions) {
const currentBackgroundMusicSubmissionScopeRef = useRef({
currentUserId: currentUserId ?? null,
projectId: projectId ?? null,
activeCanvasGenerationDialogId,
version: 0,
mounted: false,
});
useLayoutEffect(() => {
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
const scopeChanged =
!Object.is(currentScope.currentUserId, currentUserId ?? null) ||
!Object.is(currentScope.projectId, projectId ?? null);
currentBackgroundMusicSubmissionScopeRef.current = {
currentUserId: currentUserId ?? null,
projectId: projectId ?? null,
activeCanvasGenerationDialogId,
version: scopeChanged ? currentScope.version + 1 : currentScope.version,
mounted: true,
};
return () => {
currentBackgroundMusicSubmissionScopeRef.current.mounted = false;
};
}, [activeCanvasGenerationDialogId, currentUserId, projectId]);
const isBackgroundMusicSubmissionUiTargetCurrent = useCallback(
(scope: {
currentUserId: string | null;
projectId: string | null;
dialogId: string;
scopeVersion: number;
}) => {
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
const currentDialog = getCanvasGenerationDialogById(scope.dialogId);
return (
currentScope.mounted &&
currentScope.version === scope.scopeVersion &&
Object.is(scope.currentUserId, currentScope.currentUserId) &&
Object.is(scope.projectId, currentScope.projectId) &&
currentDialog?.mode === 'audio-background-music'
);
},
[getCanvasGenerationDialogById],
);
const isBackgroundMusicSubmissionAccountCurrent = useCallback(
(scope: { currentUserId: string | null }) => {
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
return (
currentScope.mounted &&
Object.is(scope.currentUserId, currentScope.currentUserId)
);
},
[],
);
const isBackgroundMusicSubmissionProjectCurrent = useCallback(
(scope: {
currentUserId: string | null;
projectId: string | null;
}) => {
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
return (
currentScope.mounted &&
Object.is(scope.currentUserId, currentScope.currentUserId) &&
Object.is(scope.projectId, currentScope.projectId)
);
},
[],
);
const addGeneratedLayersToCanvas = useCallback(
(nextLayers: CanvasLayer[]) => {
if (!nextLayers.length) {
@@ -1751,24 +1834,46 @@ export function useImageCanvasGenerationSubmissionWorkflow({
const submitImageGeneration = useCallback(
async (dialog: GenerateDialogState) => {
const normalizedPrompt =
dialog.prompt.trim() ||
(dialog.mode === 'edit'
? '修改当前图片'
: dialog.mode === 'audio-sound-effect'
? '游戏音效'
: dialog.mode === 'audio-background-music'
? '游戏背景音乐'
: 'AI 生成图片');
const canvasDialog = isCanvasGenerationDialog(dialog) ? dialog : null;
if (canvasDialog) {
const backgroundMusicDialog =
canvasDialog?.mode === 'audio-background-music' ? canvasDialog : null;
if (dialog.mode === 'audio-background-music' && !backgroundMusicDialog) {
return;
}
const backgroundMusicClaim = backgroundMusicDialog
? backgroundMusicPromptAssist.beginSubmission(backgroundMusicDialog.id)
: null;
if (backgroundMusicDialog && !backgroundMusicClaim) {
return;
}
const backgroundMusicSubmission =
backgroundMusicDialog && backgroundMusicClaim
? {
currentUserId: currentUserId ?? null,
projectId: projectId ?? null,
dialogId: backgroundMusicDialog.id,
operation: backgroundMusicClaim.operation,
prompt: backgroundMusicClaim.prompt,
scopeVersion:
currentBackgroundMusicSubmissionScopeRef.current.version,
}
: null;
const normalizedPrompt =
backgroundMusicSubmission?.prompt ??
(dialog.prompt.trim() ||
(dialog.mode === 'edit'
? '修改当前图片'
: dialog.mode === 'audio-sound-effect'
? '游戏音效'
: 'AI 生成图片'));
if (!backgroundMusicSubmission && canvasDialog) {
updateCanvasGenerationDialogById(canvasDialog.id, (currentDialog) => ({
...currentDialog,
prompt: normalizedPrompt,
status: 'generating',
composerOpen: false,
}));
} else {
} else if (!backgroundMusicSubmission) {
setGenerateDialog({
...dialog,
prompt: normalizedPrompt,
@@ -1777,11 +1882,15 @@ export function useImageCanvasGenerationSubmissionWorkflow({
});
}
let backgroundMusicSubmissionAccepted = false;
let backgroundMusicSubmissionUiOwned = false;
let backgroundMusicQueueAcceptedWhileCurrentAndOpen = false;
try {
const submissionPlan = buildImageGenerationSubmissionPlan({
dialog,
layers,
nextGeneratedIndex: layerCounterRef.current + 1,
canonicalBackgroundMusicPrompt: backgroundMusicSubmission?.prompt,
});
if (submissionPlan.kind === 'edit') {
const referenceImageSrc = await resolveEditorGenerationMediaReference(
@@ -1971,9 +2080,28 @@ export function useImageCanvasGenerationSubmissionWorkflow({
} else if (submissionPlan.kind === 'audio') {
const canvasCompletionPlaceholder =
getGeneratingDialogPlaceholder(dialog);
const generated = await runEditorGenerationWithWalletRefresh(
const generated =
submissionPlan.audioKind === 'sound-effect'
? generateEditorSoundEffect({
? await runEditorGenerationWithWalletRefresh(
generateEditorSoundEffect({
...submissionPlan.input,
projectId,
generationInputs: submissionPlan.result.generationInputs,
assetFolderId,
assetLabel: submissionPlan.result.title,
...(projectId && canvasCompletionPlaceholder
? {
canvasCompletion: {
dialogId: canvasDialog?.id,
title: submissionPlan.result.title,
placeholder: canvasCompletionPlaceholder,
},
}
: {}),
}),
onWalletBalanceMayHaveChanged,
)
: await generateEditorBackgroundMusic({
...submissionPlan.input,
projectId,
generationInputs: submissionPlan.result.generationInputs,
@@ -1988,36 +2116,110 @@ export function useImageCanvasGenerationSubmissionWorkflow({
},
}
: {}),
})
: generateEditorBackgroundMusic({
...submissionPlan.input,
projectId,
generationInputs: submissionPlan.result.generationInputs,
assetFolderId,
assetLabel: submissionPlan.result.title,
...(projectId && canvasCompletionPlaceholder
});
const isBackgroundMusic =
submissionPlan.audioKind === 'background-music';
const backgroundMusicQueueState = isBackgroundMusic
? queuedStateFromResponse(generated)
: null;
if (isBackgroundMusic && backgroundMusicSubmission) {
backgroundMusicSubmissionAccepted = true;
const currentScope =
currentBackgroundMusicSubmissionScopeRef.current;
const currentDialog = getCanvasGenerationDialogById(
backgroundMusicSubmission.dialogId,
);
backgroundMusicQueueAcceptedWhileCurrentAndOpen =
Boolean(backgroundMusicQueueState) &&
currentScope.activeCanvasGenerationDialogId ===
backgroundMusicSubmission.dialogId &&
currentDialog?.composerOpen !== false;
backgroundMusicSubmissionUiOwned =
backgroundMusicPromptAssist.finishSubmission({
operation: backgroundMusicSubmission.operation,
accepted: true,
});
if (
backgroundMusicQueueState &&
backgroundMusicSubmissionUiOwned &&
isBackgroundMusicSubmissionUiTargetCurrent(
backgroundMusicSubmission,
)
) {
updateCanvasGenerationDialogById(
backgroundMusicSubmission.dialogId,
(currentDialog) =>
currentDialog.mode === 'audio-background-music'
? {
canvasCompletion: {
dialogId: canvasDialog?.id,
title: submissionPlan.result.title,
placeholder: canvasCompletionPlaceholder,
},
...currentDialog,
prompt: backgroundMusicSubmission.prompt,
status: 'generating',
composerOpen: false,
errorMessage: undefined,
}
: {}),
}),
onWalletBalanceMayHaveChanged,
);
: currentDialog,
);
}
if (
!backgroundMusicQueueState &&
isBackgroundMusicSubmissionAccountCurrent(
backgroundMusicSubmission,
)
) {
notifyWalletBalanceMayHaveChanged(onWalletBalanceMayHaveChanged);
}
}
if (
await applyQueuedEditorGenerationProject(
generated,
projectId,
applyProjectSnapshot,
onQueuedGenerationTask,
onWalletBalanceMayHaveChanged,
backgroundMusicSubmission && applyProjectSnapshot
? (projectSnapshot) => {
if (
backgroundMusicSubmissionUiOwned &&
isBackgroundMusicSubmissionUiTargetCurrent(
backgroundMusicSubmission,
)
) {
applyProjectSnapshot(projectSnapshot);
}
}
: applyProjectSnapshot,
backgroundMusicSubmission && onQueuedGenerationTask
? () => {
if (
isBackgroundMusicSubmissionProjectCurrent(
backgroundMusicSubmission,
)
) {
onQueuedGenerationTask();
}
}
: onQueuedGenerationTask,
backgroundMusicSubmission && onWalletBalanceMayHaveChanged
? () => {
if (
isBackgroundMusicSubmissionAccountCurrent(
backgroundMusicSubmission,
)
) {
onWalletBalanceMayHaveChanged();
}
}
: onWalletBalanceMayHaveChanged,
)
) {
return;
}
if (
backgroundMusicSubmission &&
(!backgroundMusicSubmissionUiOwned ||
!isBackgroundMusicSubmissionUiTargetCurrent(
backgroundMusicSubmission,
))
) {
return;
}
if (generated.project && applyProjectSnapshot) {
applyProjectSnapshot(generated.project);
if (generated.asset) {
@@ -2114,7 +2316,56 @@ export function useImageCanvasGenerationSubmissionWorkflow({
});
}
} catch (error) {
if (canvasDialog) {
if (backgroundMusicSubmission) {
if (!backgroundMusicSubmissionAccepted) {
backgroundMusicSubmissionUiOwned =
backgroundMusicPromptAssist.finishSubmission({
operation: backgroundMusicSubmission.operation,
accepted: false,
});
if (
isBackgroundMusicSubmissionAccountCurrent(
backgroundMusicSubmission,
)
) {
notifyWalletBalanceMayHaveChanged(
onWalletBalanceMayHaveChanged,
);
}
}
if (
backgroundMusicSubmissionUiOwned &&
isBackgroundMusicSubmissionUiTargetCurrent(
backgroundMusicSubmission,
)
) {
const currentScope =
currentBackgroundMusicSubmissionScopeRef.current;
const currentDialog = getCanvasGenerationDialogById(
backgroundMusicSubmission.dialogId,
);
const shouldReopenComposer =
currentScope.activeCanvasGenerationDialogId ===
backgroundMusicSubmission.dialogId &&
(currentDialog?.composerOpen !== false ||
backgroundMusicQueueAcceptedWhileCurrentAndOpen);
updateCanvasGenerationDialogById(
backgroundMusicSubmission.dialogId,
(latestDialog) =>
latestDialog.mode === 'audio-background-music'
? {
...latestDialog,
prompt: backgroundMusicSubmission.prompt,
status: 'failed',
composerOpen: shouldReopenComposer
? true
: latestDialog.composerOpen,
errorMessage: resolveImageGenerationErrorMessage(error),
}
: latestDialog,
);
}
} else if (canvasDialog) {
updateCanvasGenerationDialogById(canvasDialog.id, () => ({
...canvasDialog,
prompt: normalizedPrompt,
@@ -2138,7 +2389,13 @@ export function useImageCanvasGenerationSubmissionWorkflow({
addAudioResultLayer,
addVideoResultLayer,
applyQuickEditResultToSourceLayer,
backgroundMusicPromptAssist,
currentUserId,
getCanvasGenerationDialogById,
getGeneratingDialogPlaceholder,
isBackgroundMusicSubmissionAccountCurrent,
isBackgroundMusicSubmissionProjectCurrent,
isBackgroundMusicSubmissionUiTargetCurrent,
layerCounterRef,
layers,
quickEditSelectionState,
@@ -1988,6 +1988,11 @@ export function useImageCanvasGenerationWorkflow({
openCanvasGenerationDialog,
updateCanvasGenerationDialogById,
hasCanvasGenerationDialogById,
getCanvasGenerationDialogById,
activeCanvasGenerationDialogId: isCanvasGenerationDialog(generateDialog)
? generateDialog.id
: null,
backgroundMusicPromptAssist,
getGeneratingDialogPlaceholder,
appendCanvasLayersWithResources,
captureCanvasHistory,
@@ -1998,6 +2003,7 @@ export function useImageCanvasGenerationWorkflow({
setActiveSidebarPanel,
rememberImageModel,
projectId,
currentUserId,
assetFolderId,
upsertGeneratedAsset,
applyProjectSnapshot,
@@ -1612,13 +1612,27 @@ describe('editorProjectClient', () => {
}),
}),
'生成游戏背景音乐失败',
expect.objectContaining({
{
timeoutMs: 1_200_000,
retry: editorRetryOptionsExpectation,
}),
},
);
});
it('forwards background music transport failures without configuring retries', async () => {
requestJsonMock.mockRejectedValueOnce(new Error('transport failed'));
await expect(
generateEditorBackgroundMusic({
gptDescriptionPrompt: '森林冒险背景音乐',
makeInstrumental: true,
}),
).rejects.toThrow('transport failed');
expect(requestJsonMock.mock.calls[0]?.[3]).toEqual({
timeoutMs: 1_200_000,
});
});
it('completes a background music prompt through the authenticated internal BFF', async () => {
requestJsonMock.mockResolvedValueOnce({
prompt: '轻快明亮的森林冒险背景音乐',
@@ -0,0 +1,77 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { setStoredAccessToken } from '../apiClient';
import { generateEditorBackgroundMusic } from './editorProjectClient';
function createLocalStorageMock() {
const store = new Map<string, string>();
return {
getItem(key: string) {
return store.has(key) ? store.get(key)! : null;
},
setItem(key: string, value: string) {
store.set(key, String(value));
},
removeItem(key: string) {
store.delete(key);
},
clear() {
store.clear();
},
};
}
describe('editorProjectClient background music transport', () => {
const fetchMock = vi.fn();
const requestBody = {
gptDescriptionPrompt: '森林冒险背景音乐',
makeInstrumental: true as const,
projectId: 'editor-project-1',
};
beforeEach(() => {
fetchMock.mockReset();
vi.stubGlobal('fetch', fetchMock);
vi.stubGlobal('window', {
dispatchEvent: vi.fn(),
localStorage: createLocalStorageMock(),
});
setStoredAccessToken('background-music-test-token', { emit: false });
});
afterEach(() => {
vi.unstubAllGlobals();
});
function expectSingleBackgroundMusicPost() {
expect(fetchMock).toHaveBeenCalledTimes(1);
const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit];
expect(url).toBe('/api/editor/audios/background-music/generations');
expect(init.method).toBe('POST');
expect(JSON.parse(String(init.body))).toEqual(requestBody);
}
it.each([429, 503])(
'does not retry the formal POST after an HTTP %i response',
async (status) => {
fetchMock.mockResolvedValueOnce(new Response('', { status }));
await expect(
generateEditorBackgroundMusic(requestBody),
).rejects.toMatchObject({ status });
expectSingleBackgroundMusicPost();
},
);
it('does not retry the formal POST after a transport error', async () => {
fetchMock.mockRejectedValueOnce(new TypeError('Failed to fetch'));
await expect(
generateEditorBackgroundMusic(requestBody),
).rejects.toBeInstanceOf(TypeError);
expectSingleBackgroundMusicPost();
});
});
@@ -1291,7 +1291,6 @@ export async function generateEditorBackgroundMusic(
'生成游戏背景音乐失败',
{
timeoutMs: 1_200_000,
retry: EDITOR_REQUEST_RETRY_OPTIONS,
},
);
}