功能:完成BGM正式提交链路
前端接入规范化提示词、同步排他锁及账号与项目分层回调门禁 关闭BGM正式POST自动重试并补充真实传输层单次请求回归 后端统一站内BGM规范载荷并保留External v1原始载荷幂等语义 补齐队列、直返、平台音频及生命周期边界测试 新增并更新T5独立实施记录
This commit is contained in:
@@ -0,0 +1,259 @@
|
||||
# BGM 生成提示词优化 T5 正式提交与方案 A 锁实施方案
|
||||
|
||||
日期:`2026-08-05`
|
||||
|
||||
状态:实施完成,待提交
|
||||
|
||||
开发分支:`codex/bgm-generation-opt-v1`
|
||||
|
||||
## 一、文档定位
|
||||
|
||||
本文承接 T5 的具体工程规划、现役链路调研、改动边界、测试设计和后续实施记录。
|
||||
|
||||
- 阶段级目标、依赖关系和总体完成定义继续保留在[总任务拆解](./【实施计划】BGM生成提示词优化V1.0任务拆解-2026-08-04.md)。
|
||||
- 产品与技术口径以[画板音乐生成入口设计](../docs/【编辑器】画板音乐生成入口设计-2026-06-18.md)为权威。本次修订只补齐该既有口径在现役 queue / inline、scope 所有权和动态价格链路上的实现边界,不新增产品能力。
|
||||
- T4 的组件与交互细节留在[T4 界面与预设交互实施方案](./【实施方案】BGM生成提示词优化T4界面与预设交互-2026-08-04.md);该文第十二节列出的交接风险由本文承接并给出解法。
|
||||
- 本文不得反向修改需求文件,也不得用实现便利覆盖权威设计。
|
||||
|
||||
## 二、实施基线
|
||||
|
||||
已完成提交:
|
||||
|
||||
- T3:`825b0ccf6`
|
||||
- T3-R2:`3fbdf210b`
|
||||
- T0 补差:`0dc1eed38`
|
||||
- T4:`d825d8181`
|
||||
|
||||
### 2.1 现役正式提交链路调研结论
|
||||
|
||||
以下为实施前的实测事实,不是推测:
|
||||
|
||||
1. `submitImageGeneration`(`useImageCanvasGenerationSubmissionWorkflow.ts`)服务 `edit` / `image` / `character` / `video` / `audio-sound-effect` / `audio-background-music` 六种 mode,入口没有任何同步锁。现有防重依赖“重渲染后生成按钮 `disabled`、composer 卸载”,属于渲染后生效的状态防护,不满足方案 A 要求的同步互斥。
|
||||
2. 入口第一行是 `dialog.prompt.trim() || 默认回退`。它同时踩三个问题:JavaScript `String.trim()` 会额外删除首尾 `U+FEFF`,与 Rust 侧不一致;空值回退为“游戏背景音乐”形成用户不可见 Prompt;紧接着**同步**把 dialog 置为 `status:'generating'`、`composerOpen:false`。
|
||||
3. 第 3 点会直接打死提交锁:Prompt 助手 hook 的 dialog 清理 effect 看到 `composerOpen === false` 就 `clearActiveOperation(abort:true)`,把刚 claim 的 `submitting` operation reject 掉;随后 `finishSubmission` 找不到 active operation 返回 `false`,锁形同虚设,dialog 重新打开后还能再次提交。
|
||||
4. `buildImageGenerationSubmissionPlan` 内部还会独立调用一次 `getDialogDefaultPrompt`(`ImageCanvasGenerationSubmissionModel.ts`)。只删除入口那一处回退不够,默认文案会从这里回到请求体。
|
||||
5. 后端生成记录 `prompt` / `actual_prompt`、canvas layer 和内部完成响应已经取自 `normalized.gpt_description_prompt`;但 queue 入口虽然先计算了 `normalized`,实际传给 `enqueue_editor_generation_job_for_caller` 的仍是原始 `payload`。因此“只替换 normalize 即可让队列等值”的旧判断错误,站内 BGM handler 必须在入队前把 canonical Prompt 和固定 `make_instrumental:true` 写回本次入队 payload。
|
||||
6. Rust 的 `str::trim()` 语义与本需求 canonicalizer 一致(都按 `char::is_whitespace` 删除两端)。真正的跨语言不一致只存在于 TypeScript 侧;BFF 和 `platform-audio` 仍需防御性调用已有 BGM 专用 validator,以拒绝空值和超限值。
|
||||
7. BGM composer 的提交锁完全来自 `assistState.status === 'submitting'`,与 `dialog.status` 无关。因此 T5 不需要新增 dialog 状态:提交期间 `dialog.status` 保持 `idle`,面板由助手状态锁定。
|
||||
8. 状态模型对 `submitting` 的 `rejectOperation` 会保留 `undoPromptSnapshot`。“后端拒绝时保留提交前撤销快照”已经具备,T5 不需要修改纯状态模型。
|
||||
9. `generateEditorBackgroundMusic` 当前给正式 POST 配置了 `retryUnsafeMethods:true`、最多两次 transport retry,而站内 queue 每次请求都会生成新的 job ID。UI 同步锁无法阻止 client 内部重发;T5 必须仅对 BGM 正式 POST关闭这项既有自动重试,不能为此新增幂等协议或改动其它生成 mode。
|
||||
10. `finishSubmission` 返回 `false` 既可能表示 dialog 被删除,也可能表示账号 / 项目 scope 已变化。正式任务不能因此被取消,但本地 dialog、Prompt、composer 和结果层也不能继续只按 `generation-dialog-N` 写回,因为该 ID 不是跨项目全局唯一。
|
||||
11. 生产和默认配置走 queue:首次成功响应只携带 `queueState`,此时即可判定站内任务已创建。兼容 inline 模式不会暴露中间接受响应,HTTP Promise 要到生成完成后才返回,不能把两种模式写成同一个可观察时序。
|
||||
12. `vector_engine_audio_generation/tests.rs` 当前没有被父模块挂载,不能作为 T5 验收依据。新增 Rust 用例必须放进实际编译的模块内测试或真正挂载的精简测试模块。
|
||||
|
||||
## 三、既有口径下的实现收口
|
||||
|
||||
### 3.1 提交锁与 composer 生命周期
|
||||
|
||||
T4 交接要求“后端接受后才隐藏 composer”,同时要求普通 `composerOpen=false` 不得提前 reject `submitting`。两者分别处理正常接受时序和用户主动归档 / 切面板,不是新增交互:
|
||||
|
||||
- 当前 BGM 面板保持打开时,提交期间继续可见并全锁;queue 模式收到任务已创建的成功响应后才结束 `submitting` 并切入现有占位。
|
||||
- 用户在等待期间归档或切走面板时,只把原 dialog 变为 inactive / closed,不取消已经发出的正式请求,也不主动重新激活该 dialog。
|
||||
- dialog 被删除或账号 / 项目 scope 改变时,可以移除临时助手状态;这不代表正式请求已取消。
|
||||
|
||||
### 3.2 正式任务处理与本地 UI 写回必须分开
|
||||
|
||||
点击时冻结本次 `{账号、项目、dialogId、operation、canonical Prompt、submission plan}`。后续异步阶段遵循两条独立规则:
|
||||
|
||||
1. 已发出的正式请求和后端已经创建的任务继续按现役链路处理,不能因 `finishSubmission` 返回 `false` 被误判为取消。
|
||||
2. `status`、`composerOpen`、Prompt、错误信息、本地 `applyProjectSnapshot` / asset upsert 和结果层都属于 UI 写回;每次跨 `await` 后写入前都必须重新确认仍是原账号、原项目和原 dialog。scope 已变化或 dialog 已删除时跳过本地 UI 写回,不能按同名 ID 修改新 scope。
|
||||
|
||||
普通 `composerOpen=false` 且 scope / dialog 仍匹配时,`finishSubmission` 仍可结算。若随后失败:
|
||||
|
||||
- 原 dialog 仍是当前打开面板:恢复 `failed + composerOpen:true`。
|
||||
- 原 dialog 已被用户归档或切走:写入失败状态和 canonical Prompt,但保留关闭 / inactive,不抢回焦点。
|
||||
- 原 scope 已失效或 dialog 已删除:不再写本地 UI。
|
||||
|
||||
因此,`finishSubmission` 的返回值不能取消正式任务,但也不能被解释成“所有本地结果都可无条件写回”。
|
||||
|
||||
### 3.3 可观察提交时点与 transport 边界
|
||||
|
||||
```text
|
||||
点击生成
|
||||
→ 同步 beginSubmission:canonicalize、写回、资格校验、冻结 scope 与请求值
|
||||
→ 只发送一次 BGM 正式 POST,不执行 unsafe 自动重试
|
||||
├─ queue 接受 → finishSubmission({ operation, accepted: true }) → 原 UI 目标仍有效时切 generating / 隐藏 composer → 现有轮询
|
||||
├─ 首次 POST 非成功或抛错 → finishSubmission({ operation, accepted: false }) → 原 UI 目标仍有效时记录失败
|
||||
└─ inline 兼容模式 → 无中间接受响应,保持 submitting 到现有终态响应,再结算并处理结果
|
||||
```
|
||||
|
||||
- T5 不新增 Idempotency-Key、持久提交账本、服务端 dedupe 或新的响应阶段;“一次点击一次请求”通过同步 UI 锁和关闭 BGM 正式 POST 的既有 unsafe retry 实现。
|
||||
- 请求返回 transport 错误时,客户端不能证明服务端一定未接受;T5 不自动重发。用户之后再次明确点击属于新的提交,不在本切片引入跨页面、跨 scope 的 durable exactly-once 设计。
|
||||
- 保留现有 BGM 正式请求超时;不新增重试、超时或限流。
|
||||
|
||||
## 四、改动边界
|
||||
|
||||
- 只改 `audio-background-music`。`edit` / `image` / `character` / `video` / `audio-sound-effect` 的提交代码路径、文案、默认 Prompt 和错误展示一字不动。
|
||||
- 实现方式为在 `submitImageGeneration` 内增加 BGM 前置分支,分支结束后复用现有 audio 结果处理(队列、占位、素材库、项目刷新、`addAudioResultLayer`),不复制这段逻辑,也不新建平行提交函数。
|
||||
- 不新增 dialog 状态、不修改纯状态模型、不新增持久提交锁或全局画布锁。
|
||||
- 只移除 BGM 正式 POST 的 `EDITOR_REQUEST_RETRY_OPTIONS`;不修改共享 retry 配置或其它请求。
|
||||
- 不新增 Idempotency-Key、dedupe、请求账本、响应字段或 queue 状态。
|
||||
- 站内 BGM handler 的 queue payload 执行 canonical 写回;不改变 External v1 的请求、幂等键或 payload 等值语义。
|
||||
- 不修改 Suno 三字段结构、固定模型、现役动态价格配置和计费包装;客户端不提交价格,服务端继续在入队时冻结价格。
|
||||
- “正式请求不含助手隐藏元数据”只约束本功能前端构造的 submission payload 和固定三字段 Suno body;不在 T5 新增通用 `generationInputs` allowlist 或恶意客户端清洗器。
|
||||
- 不修改 SpacetimeDB schema、External v1、External OpenAPI 或 LLM 日志策略。
|
||||
- 不增加新的重试、超时或限流。
|
||||
|
||||
## 五、目标文件结构
|
||||
|
||||
|文件|T5 职责|
|
||||
|---|---|
|
||||
|`useImageCanvasBackgroundMusicPromptAssist.ts`|清理 effect 豁免 `submitting`;`finishSubmission` 不再要求 composer 打开|
|
||||
|`useImageCanvasGenerationWorkflow.ts`|把现有完整 Prompt 助手 controller 和当前 scope 接入 submission workflow|
|
||||
|`useImageCanvasGenerationSubmissionWorkflow.ts`|BGM 前置分支:同步取锁、冻结 scope / 请求值;分离正式任务处理与 scope-safe UI 写回|
|
||||
|`ImageCanvasGenerationSubmissionModel.ts`|BGM 分支使用冻结的 canonical Prompt,不再经过默认回退|
|
||||
|`editorProjectClient.ts`|仅移除 BGM 正式 POST 的 unsafe 自动重试,保留现有超时|
|
||||
|`server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs`|BGM 入站改用 canonical 校验;站内 handler 用 canonical payload 进入 queue|
|
||||
|`server-rs/crates/platform-audio/src/request.rs`|BGM Suno body builder 防御性 canonicalization|
|
||||
|对应现役测试文件|覆盖 client 单次发送、scope 所有权、queue payload、body 与持久化等值;不依赖未挂载的 `vector_engine_audio_generation/tests.rs`|
|
||||
|
||||
## 六、T5A:助手生命周期豁免
|
||||
|
||||
### 6.1 修改
|
||||
|
||||
- 清理 effect 的第二个循环:active operation 为 `submitting` 时跳过,不 abort、不 reject、不移除记录。
|
||||
- `finishSubmission`:移除“composer 仍打开”的前置条件;operation、scope 和 BGM dialog 仍匹配即可结算。
|
||||
- 返回值只表示这次助手 operation 是否由原 UI 目标成功结算。正式请求已经发出后,该值不负责取消任务;submission workflow 仍需在后续每个 UI 写回点重新检查冻结 scope。
|
||||
- 其余 6 处 `composerOpen` 判断保持不变。AI 操作入口在 `submitting` 期间已由 `isSubmissionLocked` 挡住,迟到响应、预设、撤销和 `beginSubmission` 的语义不受影响。
|
||||
|
||||
### 6.2 验收
|
||||
|
||||
- 提交期间把 `composerOpen` 置 `false`(选中其它图层、归档、切面板),`submitting` 不被 reject,`finishSubmission` 仍能结算。
|
||||
- 提交期间删除 dialog:UI 状态移除,`finishSubmission` 返回 `false`,不抛错。
|
||||
- 提交期间切换账号或项目:助手状态被 reset,`finishSubmission` 返回 `false`;正式任务继续,但旧回调不能写入新 scope 的同 ID dialog。
|
||||
- T3 / T3-R2 既有的 operation ID、跨 dialog、迟到响应、超时和 scope 用例继续通过。
|
||||
|
||||
## 七、T5B:前端 BGM 提交分支
|
||||
|
||||
### 7.1 同步阶段(必须早于任何 `await`)
|
||||
|
||||
1. 用类型收窄取得带稳定 ID 的 BGM dialog;不是 BGM 则走现有通用路径。
|
||||
2. `beginSubmission(dialog.id)` 在同一个同步边界内完成 canonicalize、输入框写回和正式生成资格校验;状态模型继续是唯一资格判定源,不在 workflow 再写第二套 0 / 200 校验。
|
||||
3. 返回 `null` 表示重复点击、scope / dialog 不合格或 Prompt 无正式生成资格,直接结束;canonical 写回按现有 controller 结果保留,不发送请求。
|
||||
4. claim 成功后冻结 `{账号、项目、dialogId、operation、canonical Prompt}`,并以该 Prompt 构造 submission plan;BGM 分支不得再经过原生 `trim` 或 `getDialogDefaultPrompt`。
|
||||
5. 不在同步阶段切换 `status` / `composerOpen`。
|
||||
|
||||
### 7.2 异步阶段
|
||||
|
||||
- 调用 `generateEditorBackgroundMusic` 一次;该 client 不再配置 unsafe retry。
|
||||
- queue 首次成功响应代表任务已创建:先调用 `finishSubmission({ operation, accepted: true })`;原 UI 目标仍有效时再设 `status:'generating'`、`composerOpen:false`,随后复用现有 queue 轮询、项目刷新、素材库和结果处理。
|
||||
- 首次 POST 非成功或抛错:先调用 `finishSubmission({ operation, accepted: false })`。原 dialog 仍是当前面板时恢复 `failed + composerOpen:true`;已归档 / 切走时保留关闭状态并记录失败;scope 已变化或 dialog 已删除时不写本地 UI。
|
||||
- queue 已接受后的终态成功 / 失败继续走现有正式链路;每次应用 project snapshot、upsert asset、写 dialog 或本地结果层前重新检查冻结 scope 和 dialog。scope 已失效时跳过这些前端后续写回,后端已经接受的任务继续处理;T5 不新增跨 scope 的恢复或刷新机制。
|
||||
- inline 兼容模式没有 `queueState`:面板保持 `submitting` 到现有 HTTP 终态响应;成功后结算并直接处理现有结果,失败按上一条恢复,不新增中间响应或占位协议。
|
||||
|
||||
### 7.3 验收
|
||||
|
||||
- 同一 dialog 快速重复点击只产生一次正式 POST;queue 模式只创建一个 queue job 并沿用该 job 的现有计费链路,inline 模式沿用现有终态处理;retryable HTTP 状态或 transport error 也不会由 client 自动重发。
|
||||
- 空 Prompt、全 Unicode 空白 Prompt 不产生正式请求。
|
||||
- 1 个有效字符可以提交;201 个 code point 不能提交。
|
||||
- 提交期间输入框、预设、展开收起、箭头、AI 补全、一键简化、撤销和生成全部锁定,面板保持可见。
|
||||
- 后端拒绝后 canonical Prompt 和撤销快照保留;当前面板恢复,用户已经归档 / 切走的面板不被自动激活。
|
||||
- 后端接受后,原 UI 目标仍有效时才进入占位;画布其它功能全程可用。
|
||||
- 账号 / 项目 A 的迟到成功或失败不能修改账号 / 项目 B 的同 ID dialog;queue 已接受后再切 scope 也遵守该规则。
|
||||
- inline 模式在终态响应前保持 `submitting`,不为兼容模式新增协议。
|
||||
- 其余 5 种 mode 的提交行为无差异。
|
||||
|
||||
## 八、T5C:后端 canonicalization
|
||||
|
||||
### 8.1 修改
|
||||
|
||||
- `normalize_editor_background_music_request_with_pricing` 改用 BGM canonical 校验函数:canonicalize、至少 1 个有效字符、不超过 200 个 code point,返回 canonical Prompt。
|
||||
- 登录态站内 `generate_editor_background_music` handler 在 queue / inline 分流前,用上述结果构造本次 canonical payload:覆盖 `gpt_description_prompt`,并固定 `make_instrumental:true`。queue 分支必须把这份 payload 交给现有 enqueue helper,不能继续序列化原始请求。
|
||||
- 不在共享 External v1 调用边界新增 payload 改写;External v1 路由、Idempotency-Key、OpenAPI 和现有 payload 等值判断保持不变。
|
||||
- `build_editor_background_music_task_body` 执行同一套幂等 canonicalization 后再构造 Suno body。
|
||||
- BGM 请求不接收客户端价格。服务端继续按现役动态定价解析,并把本次价格冻结到 queue job;后续计费 target、资产成本和内部完成响应继续复用该冻结价格,不硬编码 `5`。
|
||||
- SFX 继续使用现有 `normalize_limited_text`,Vidu body、默认 Prompt、时长与 1500 字限制不变。
|
||||
|
||||
### 8.2 验收
|
||||
|
||||
- 空 Prompt、全空白 Prompt 返回 `400`,不创建任务、不扣费。
|
||||
- 201 个 code point 拒绝;200 个通过。
|
||||
- 站内 queue 的实际 `request_payload_json.gptDescriptionPrompt` 与 canonical Prompt 逐 code point 等值,`makeInstrumental` 为 `true`;不能只断言 normalize 返回值。
|
||||
- 首尾 `U+0085` 等 Unicode `White_Space` 被删除;`U+200B`、`U+FEFF`、组合字符和 ZWJ emoji 不被误删;内部空格与 LF / CRLF 原样保留。
|
||||
- 连续两次 canonicalization 结果一致。
|
||||
- Suno body 仍只含 `mv`、`gpt_description_prompt`、`make_instrumental`。
|
||||
- queue job 的 `price_mud_points`、计费 target 和内部完成响应使用同一次服务端冻结价格;前端请求中没有 `priceMudPoints`。
|
||||
- External v1 契约与幂等回归无差异。
|
||||
- SFX 相关 Rust 测试无回归。
|
||||
|
||||
## 九、T5D:端到端等值
|
||||
|
||||
- 逐层断言写回后的输入框、前端 BFF 请求、站内 generation queue payload、Suno body、`editor_project_resource`、`editor_asset`、canvas layer 以及内部 `EditorAudioGenerateResponse` 的 Prompt 字段逐 code point 完全一致。
|
||||
- queue 首次响应仍只断言现有 `queueState`;普通站内 job 的 `result_payload_json` 继续使用现有 metadata-only 形态,不为 T5 增加 `prompt` / `actualPrompt` 字段。`prompt` / `actualPrompt` 的完成响应断言落在 inline 或 worker 内部生成响应边界。
|
||||
- 断言本功能前端构造的正式 submission payload 不含预设 ID、分组、颜色、助手系统模板、内部引导、AI 快照或格式 / 完整性 / 残句判断字段;Suno body 的 key 集合精确等于三个固定字段。
|
||||
- 不把上条扩展成 BFF 对任意 `generationInputs` 的通用 allowlist / sanitizer,本切片只保证本功能客户端不发送这些隐藏元数据。
|
||||
- 用例至少覆盖:含首尾边界空白、内部换行和零宽字符的 Prompt;仅 1 个有效字符的 Prompt;恰好 200 个 code point 的 Prompt。
|
||||
|
||||
## 十、测试计划
|
||||
|
||||
|层级|场景|期望|
|
||||
|---|---|---|
|
||||
|助手 hook|提交期间 `composerOpen=false`|`submitting` 不被 reject,`finishSubmission` 可结算|
|
||||
|助手 hook|提交期间删除 dialog|移除 UI 状态,`finishSubmission` 返回 `false`,不抛错|
|
||||
|助手 hook|提交期间切换账号 / 项目|状态 reset;正式任务继续,旧 UI 不写入新 scope|
|
||||
|BGM client|retryable HTTP 状态 / transport error|一次调用只发送一次 POST,不执行 unsafe retry|
|
||||
|前端提交|双击生成|一次 client 调用、一次正式请求|
|
||||
|前端提交|空 / 全空白 Prompt|不发请求,保留 canonical 文本|
|
||||
|前端提交|1 个有效字符 / 200 / 201|前两者可提交,201 拒绝|
|
||||
|前端提交|当前面板后端拒绝|面板恢复,canonical Prompt 与撤销快照保留|
|
||||
|前端提交|归档 / 切面板后拒绝|记录失败但保持关闭,不抢焦点|
|
||||
|前端提交|queue 接受|先结算再切占位,画布其它功能可用|
|
||||
|前端提交|queue 接受后切 scope,再返回终态|后端已创建任务不被视为取消;新 scope 同 ID dialog 与本地画布不被旧回调修改|
|
||||
|前端提交|inline 成功 / 失败|终态响应前保持 `submitting`,随后走现有结果 / 失败链路|
|
||||
|后端|空、全空白、201|`400`,不创建任务|
|
||||
|后端|`U+200B` / `U+FEFF` / 组合字符 / ZWJ emoji|不被误删|
|
||||
|后端|幂等性|两次 canonicalization 结果一致|
|
||||
|后端 queue|站内边界空白 Prompt|实际 `request_payload_json` 使用 canonical Prompt,固定 instrumental,冻结动态价格|
|
||||
|后端 queue|一次站内正式 POST|创建一个 queue job,并沿用该 job 的单次计费链路|
|
||||
|平台 body|边界空白 + 内部换行 + 零宽字符|三个固定字段,Prompt 逐 code point 等值|
|
||||
|端到端|边界空白 + 内部换行 + 零宽字符|前端、queue、Suno、resource、asset、canvas 与内部完成响应等值|
|
||||
|回归|External v1|路由、OpenAPI、Idempotency-Key 与 payload 等值语义无差异|
|
||||
|回归|SFX|Vidu body、默认 Prompt、时长与 1500 字限制不变|
|
||||
|回归|图片 / 视频 / 角色 / edit|提交路径与文案无差异|
|
||||
|
||||
Rust 测试必须落入 `cargo test -p api-server` 实际会编译执行的位置,例如 `generation.rs` / `publish.rs` 的内联测试或新挂载的精简测试模块;不得把未挂载的 `vector_engine_audio_generation/tests.rs` 当作通过证据。T5 不借此清理整份历史测试文件,也不扩展计费系统。
|
||||
|
||||
## 十一、建议实施顺序
|
||||
|
||||
1. T5A:助手生命周期豁免(可与 T5C 并行)。
|
||||
2. T5C:站内 canonical queue payload 与 `platform-audio` 防御性 canonicalization。
|
||||
3. BGM client:移除正式 POST 的 unsafe retry;补 controller / scope 接线。
|
||||
4. T5B:前端 BGM 提交分支和 scope-safe UI 写回(依赖 T5A)。
|
||||
5. T5D:queue / inline 分流与端到端等值测试。
|
||||
6. 定向测试、类型检查、ESLint、Rust 定向测试、编码检查与差异检查。
|
||||
7. 真实浏览器验证:以默认 queue 模式走通“提交成功进占位”“当前面板失败恢复”“归档后失败不抢焦点”三条路径;inline 兼容语义由定向测试覆盖,不新增联调协议。
|
||||
|
||||
建议一个提交:`前后端:收紧BGM生成提交链路`。
|
||||
|
||||
## 十二、风险与交接
|
||||
|
||||
- 本切片是发布切点。T4 与 T5 之间不可发布,因为 T4 面板已按 canonical 口径展示,而正式提交仍走旧 `trim` 与默认回退。
|
||||
- 空 Prompt 收紧后,历史上依赖默认回退成功的空提交会被前端拦截并被后端拒绝。这是需求明确要求的行为变化。
|
||||
- 提交期间面板保持可见是既定口径,但与当前“点击立即切占位”的手感不同,需要在浏览器验证中确认无违和。
|
||||
- 正式请求没有可取消的 `AbortController`。请求发出后即使 dialog 被删除或 scope 切换,也可能已经被后端接受并扣费;实现不得把这类情况当作“已取消”。
|
||||
- BGM 正式 POST 关闭 unsafe retry 后,transport 结果未知时不会自动重发;T5 不引入 durable 幂等或对账系统。用户随后再次明确点击是新提交,这是本切片保留的既有边界。
|
||||
- scope 切换会清理临时助手锁。正式任务继续由后端处理,但旧异步回调必须停止本地 UI 写回;T5 不新增跨页面持久锁。
|
||||
- inline 兼容模式无法观察“上游任务已创建但尚未完成”的中间时刻,因此锁会保持到现有 HTTP 终态响应;生产和默认 queue 行为不受该兼容边界影响。
|
||||
- 现役 BGM 价格由服务端配置解析并在入队时冻结;T5 不修改价格、计费规则或钱包账本。
|
||||
- T6 需要汇总本切片测试,并复核完成定义中与提交链路相关的条目。
|
||||
|
||||
## 十三、实施记录
|
||||
|
||||
|日期|阶段|状态|记录|
|
||||
|---|---|---|---|
|
||||
|2026-08-05|T5 初始规划|已被本轮修订取代|完成正式提交链路初查;后续评审确认“queue 已使用 normalized”“权威设计无需修改”两项判断不成立。|
|
||||
|2026-08-05|T5 规划修订|完成|收口 unsafe POST retry、scope-safe UI 写回、站内 canonical queue payload、queue / inline 可观察时序、动态价格和有效测试落点;明确不新增幂等协议、持久锁、响应字段、通用 sanitizer 或计费设计。|
|
||||
|2026-08-05|T5A 助手生命周期|完成|修改 `useImageCanvasBackgroundMusicPromptAssist.ts` 及定向测试:`submitting` 不再因普通 composer 关闭被清理,dialog 删除和 scope 切换仍清理,`finishSubmission` 继续校验 operation、scope 与 BGM dialog。|
|
||||
|2026-08-05|T5B / T5D 正式提交与等值链路|完成|修改 BGM submission model、submission workflow、workflow 接线、BGM `generationInputs` helper、正式 client 及定向测试:同步 claim 早于首个 `await`,canonical Prompt 是唯一请求值,关闭 unsafe POST retry;queue / inline 按既定时序结算;冻结 scope 版本和 UI 所有权,覆盖首次响应及 queue 终态前的 `A → B → A` 往返、当前面板失败恢复和归档后不抢焦点;其它生成 mode 保持原路径。|
|
||||
|2026-08-05|T5C 后端 canonicalization|完成|修改站内 BGM handler、queue 准备边界、`platform-audio` Suno body builder 及现役测试:登录态 queue / inline 使用 canonical Prompt 并固定 `makeInstrumental=true`,queue 准备边界绑定 canonical payload 与动态冻结价格,实际 queue serializer 对该 payload 序列化;External v1 继续保留原始 payload 和既有幂等语义,SFX 未改。|
|
||||
|2026-08-05|T5 修复前验证|完成|前端 5 个定向测试文件 `193/193` 通过,`npm run typecheck` 与变更文件 ESLint 通过;`platform-audio` `31/31`、窄口径 `api-server` BGM 定向 `28/28`、登录态 queue 准备 `1/1`、External v1 `7/7` 通过。该记录保留当时命令口径,后续验收以本表最新记录为准。|
|
||||
|2026-08-05|T5 评审修复|完成|拆分钱包账号所有权、任务列表账号内项目所有权(同时校验 `currentUserId + projectId`)与 dialog/canvas 完整所有权;真实 transport 层覆盖 BGM 正式 POST 遇到 `429`、`503` 和 `TypeError` 均只发送一次;补齐 External v1 BGM 原始 payload 保留、同原值幂等重放及 canonical 等价值冲突回归。未修改 SFX、其它生成 mode、通用 retry、计费、Schema、External 路由 / DTO / OpenAPI 或 worker。|
|
||||
|2026-08-05|T5 修复后验证|完成|前端 6 个定向测试文件 `199/199` 通过,其中真实 transport `3/3`;`npm run typecheck` 与变更文件 ESLint 通过;`platform-audio` `31/31`、广义 `cargo test -p api-server background_music` `35/35`、External v1 既有回归 `7/7` 通过;Rust 格式、编码与 `git diff --check` 通过。真实 queue 浏览器 smoke 会创建正式任务并可能扣费,本切片不执行。|
|
||||
|
||||
后续每次实施只在本节追加:
|
||||
|
||||
- 完成的切片;
|
||||
- 实际修改文件;
|
||||
- 验证命令与结果;
|
||||
- 尚存风险;
|
||||
- 对下一阶段的交接。
|
||||
@@ -132,6 +132,21 @@ where
|
||||
)
|
||||
.await?;
|
||||
|
||||
ensure_external_api_generation_job_matches_request(
|
||||
&job,
|
||||
owner_user_id,
|
||||
job_kind,
|
||||
&request_payload_json,
|
||||
)?;
|
||||
Ok(job)
|
||||
}
|
||||
|
||||
fn ensure_external_api_generation_job_matches_request(
|
||||
job: &ExternalGenerationJobRecord,
|
||||
owner_user_id: &str,
|
||||
job_kind: &str,
|
||||
request_payload_json: &str,
|
||||
) -> Result<(), AppError> {
|
||||
if job.job_kind != job_kind
|
||||
|| job.owner_user_id != owner_user_id
|
||||
|| job.request_payload_json != request_payload_json
|
||||
@@ -143,7 +158,7 @@ where
|
||||
})),
|
||||
);
|
||||
}
|
||||
Ok(job)
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
@@ -264,6 +279,16 @@ where
|
||||
Ok(request_payload_json)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) fn serialize_editor_generation_job_payload_for_test<T>(
|
||||
payload: &T,
|
||||
) -> Result<String, AppError>
|
||||
where
|
||||
T: Serialize + ?Sized,
|
||||
{
|
||||
serialize_editor_generation_job_payload(payload)
|
||||
}
|
||||
|
||||
fn payload_serialization_error(error: serde_json::Error) -> AppError {
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
|
||||
"provider": EDITOR_GENERATION_QUEUE_PROVIDER,
|
||||
@@ -423,6 +448,54 @@ mod tests {
|
||||
assert!(error.body_text().contains("超过持久化上限"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn external_background_music_idempotency_matches_the_raw_payload_exactly() {
|
||||
let raw_prompt = "\u{0085}\u{2003}森林音乐\u{00a0}";
|
||||
let raw_payload = shared_contracts::assets::EditorBackgroundMusicGenerateRequest {
|
||||
gpt_description_prompt: raw_prompt.to_string(),
|
||||
make_instrumental: false,
|
||||
project_id: Some("project-1".to_string()),
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
asset_folder_id: None,
|
||||
asset_label: None,
|
||||
};
|
||||
let raw_payload_json = serialize_editor_generation_job_payload(&raw_payload)
|
||||
.expect("External BGM 原始 payload 应可序列化");
|
||||
let mut persisted_job = queue_job_fixture("pending", None);
|
||||
persisted_job.job_kind = EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND.to_string();
|
||||
persisted_job.request_payload_json = raw_payload_json.clone();
|
||||
|
||||
ensure_external_api_generation_job_matches_request(
|
||||
&persisted_job,
|
||||
"user-1",
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
|
||||
&raw_payload_json,
|
||||
)
|
||||
.expect("同一原始 payload 的幂等重放应复用既有任务");
|
||||
|
||||
let canonical_payload = shared_contracts::assets::EditorBackgroundMusicGenerateRequest {
|
||||
gpt_description_prompt: "森林音乐".to_string(),
|
||||
..raw_payload
|
||||
};
|
||||
let canonical_payload_json = serialize_editor_generation_job_payload(&canonical_payload)
|
||||
.expect("canonical 等价值应可序列化");
|
||||
let error = ensure_external_api_generation_job_matches_request(
|
||||
&persisted_job,
|
||||
"user-1",
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
|
||||
&canonical_payload_json,
|
||||
)
|
||||
.expect_err("同一幂等键改用 canonical 等价值必须按既有精确语义冲突");
|
||||
|
||||
assert_eq!(error.status_code(), StatusCode::CONFLICT);
|
||||
assert!(
|
||||
error
|
||||
.body_text()
|
||||
.contains("Idempotency-Key 已用于不同的生成请求")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn queue_state_maps_idempotent_replays_to_the_persisted_status() {
|
||||
let cases = [
|
||||
|
||||
@@ -55,6 +55,36 @@ pub(super) struct NormalizedEditorBackgroundMusicRequest {
|
||||
pub(super) price_mud_points: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct PreparedEditorBackgroundMusicQueueJob {
|
||||
payload: assets::EditorBackgroundMusicGenerateRequest,
|
||||
price_mud_points: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct CanonicalEditorBackgroundMusicSubmissionPayload {
|
||||
payload: assets::EditorBackgroundMusicGenerateRequest,
|
||||
}
|
||||
|
||||
impl CanonicalEditorBackgroundMusicSubmissionPayload {
|
||||
fn new(payload: assets::EditorBackgroundMusicGenerateRequest) -> Result<Self, AppError> {
|
||||
Ok(Self {
|
||||
payload: canonicalize_editor_background_music_submission_payload(payload)?,
|
||||
})
|
||||
}
|
||||
|
||||
fn into_queue_job(
|
||||
self,
|
||||
pricing: &EditorGenerationPricingConfig,
|
||||
) -> Result<PreparedEditorBackgroundMusicQueueJob, AppError> {
|
||||
prepare_editor_background_music_queue_job(self.payload, pricing)
|
||||
}
|
||||
|
||||
fn into_payload(self) -> assets::EditorBackgroundMusicGenerateRequest {
|
||||
self.payload
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(super) fn normalize_editor_sound_effect_request(
|
||||
payload: assets::EditorSoundEffectGenerateRequest,
|
||||
@@ -120,20 +150,40 @@ pub(super) fn normalize_editor_background_music_request_with_pricing(
|
||||
payload: assets::EditorBackgroundMusicGenerateRequest,
|
||||
pricing: &EditorGenerationPricingConfig,
|
||||
) -> Result<NormalizedEditorBackgroundMusicRequest, AppError> {
|
||||
let payload = canonicalize_editor_background_music_submission_payload(payload)?;
|
||||
let expected_price_mud_points =
|
||||
pricing.background_music_model_mud_points(Some(platform_audio::SUNO_DEFAULT_MODEL));
|
||||
Ok(NormalizedEditorBackgroundMusicRequest {
|
||||
gpt_description_prompt: platform_audio::normalize_limited_text(
|
||||
&payload.gpt_description_prompt,
|
||||
"gpt_description_prompt",
|
||||
platform_audio::SUNO_GPT_DESCRIPTION_PROMPT_MAX_CHARS,
|
||||
)
|
||||
.map_err(map_platform_audio_error)?,
|
||||
make_instrumental: true,
|
||||
gpt_description_prompt: payload.gpt_description_prompt,
|
||||
make_instrumental: payload.make_instrumental,
|
||||
price_mud_points: expected_price_mud_points,
|
||||
})
|
||||
}
|
||||
|
||||
fn canonicalize_editor_background_music_submission_payload(
|
||||
mut payload: assets::EditorBackgroundMusicGenerateRequest,
|
||||
) -> Result<assets::EditorBackgroundMusicGenerateRequest, AppError> {
|
||||
let validated = platform_audio::validate_background_music_generation_prompt(
|
||||
&payload.gpt_description_prompt,
|
||||
)
|
||||
.map_err(map_platform_audio_error)?;
|
||||
payload.gpt_description_prompt = validated.prompt.to_string();
|
||||
payload.make_instrumental = true;
|
||||
Ok(payload)
|
||||
}
|
||||
|
||||
fn prepare_editor_background_music_queue_job(
|
||||
payload: assets::EditorBackgroundMusicGenerateRequest,
|
||||
pricing: &EditorGenerationPricingConfig,
|
||||
) -> Result<PreparedEditorBackgroundMusicQueueJob, AppError> {
|
||||
let normalized =
|
||||
normalize_editor_background_music_request_with_pricing(payload.clone(), pricing)?;
|
||||
Ok(PreparedEditorBackgroundMusicQueueJob {
|
||||
payload,
|
||||
price_mud_points: normalized.price_mud_points,
|
||||
})
|
||||
}
|
||||
|
||||
pub async fn generate_editor_sound_effect(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
@@ -372,14 +422,15 @@ pub async fn generate_editor_background_music(
|
||||
payload: Result<Json<assets::EditorBackgroundMusicGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
let Json(payload) = parse_json_payload(&request_context, payload)?;
|
||||
let payload = CanonicalEditorBackgroundMusicSubmissionPayload::new(payload)
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
|
||||
let owner_user_id = authenticated.claims().user_id().to_string();
|
||||
if !state.config.external_generation_mode.is_inline() {
|
||||
let queue_job = enqueue_editor_background_music_generation_for_owner(
|
||||
let queue_job = enqueue_logged_in_editor_background_music_generation_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
owner_user_id.as_str(),
|
||||
payload,
|
||||
None,
|
||||
)
|
||||
.await?;
|
||||
return Ok(json_success_body(
|
||||
@@ -393,7 +444,67 @@ pub async fn generate_editor_background_music(
|
||||
state,
|
||||
request_context,
|
||||
owner_user_id,
|
||||
Ok(Json(payload)),
|
||||
Ok(Json(payload.into_payload())),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn load_editor_background_music_queue_pricing(
|
||||
state: &AppState,
|
||||
request_context: &RequestContext,
|
||||
) -> Result<EditorGenerationPricingConfig, Response> {
|
||||
state.editor_generation_pricing().await.map_err(|error| {
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR)
|
||||
.with_details(json!({
|
||||
"provider": "editor-generation-pricing",
|
||||
"message": error.to_string(),
|
||||
}))
|
||||
.into_response_with_context(Some(request_context))
|
||||
})
|
||||
}
|
||||
|
||||
async fn enqueue_prepared_editor_background_music_generation_for_owner(
|
||||
state: &AppState,
|
||||
request_context: &RequestContext,
|
||||
owner_user_id: &str,
|
||||
prepared: PreparedEditorBackgroundMusicQueueJob,
|
||||
external_idempotency_key: Option<&str>,
|
||||
) -> Result<ExternalGenerationJobRecord, Response> {
|
||||
let source_entity_id = editor_generation_source_entity_id(
|
||||
prepared.payload.project_id.as_deref(),
|
||||
"editor-background-music",
|
||||
);
|
||||
enqueue_editor_generation_job_for_caller(
|
||||
state,
|
||||
request_context,
|
||||
owner_user_id,
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
|
||||
source_entity_id,
|
||||
"图片画布生成背景音乐",
|
||||
u64::from(prepared.price_mud_points),
|
||||
&prepared.payload,
|
||||
external_idempotency_key,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(request_context)))
|
||||
}
|
||||
|
||||
async fn enqueue_logged_in_editor_background_music_generation_for_owner(
|
||||
state: &AppState,
|
||||
request_context: &RequestContext,
|
||||
owner_user_id: &str,
|
||||
payload: CanonicalEditorBackgroundMusicSubmissionPayload,
|
||||
) -> Result<ExternalGenerationJobRecord, Response> {
|
||||
let pricing = load_editor_background_music_queue_pricing(state, request_context).await?;
|
||||
let prepared = payload
|
||||
.into_queue_job(&pricing)
|
||||
.map_err(|error| error.into_response_with_context(Some(request_context)))?;
|
||||
enqueue_prepared_editor_background_music_generation_for_owner(
|
||||
state,
|
||||
request_context,
|
||||
owner_user_id,
|
||||
prepared,
|
||||
None,
|
||||
)
|
||||
.await
|
||||
}
|
||||
@@ -405,34 +516,17 @@ pub(crate) async fn enqueue_editor_background_music_generation_for_owner(
|
||||
payload: assets::EditorBackgroundMusicGenerateRequest,
|
||||
external_idempotency_key: Option<&str>,
|
||||
) -> Result<ExternalGenerationJobRecord, Response> {
|
||||
let pricing = state.editor_generation_pricing().await.map_err(|error| {
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR)
|
||||
.with_details(json!({
|
||||
"provider": "editor-generation-pricing",
|
||||
"message": error.to_string(),
|
||||
}))
|
||||
.into_response_with_context(Some(request_context))
|
||||
})?;
|
||||
let normalized =
|
||||
normalize_editor_background_music_request_with_pricing(payload.clone(), &pricing)
|
||||
.map_err(|error| error.into_response_with_context(Some(request_context)))?;
|
||||
let source_entity_id = editor_generation_source_entity_id(
|
||||
payload.project_id.as_deref(),
|
||||
"editor-background-music",
|
||||
);
|
||||
enqueue_editor_generation_job_for_caller(
|
||||
let pricing = load_editor_background_music_queue_pricing(state, request_context).await?;
|
||||
let prepared = prepare_editor_background_music_queue_job(payload, &pricing)
|
||||
.map_err(|error| error.into_response_with_context(Some(request_context)))?;
|
||||
enqueue_prepared_editor_background_music_generation_for_owner(
|
||||
state,
|
||||
request_context,
|
||||
owner_user_id,
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
|
||||
source_entity_id,
|
||||
"图片画布生成背景音乐",
|
||||
u64::from(normalized.price_mud_points),
|
||||
&payload,
|
||||
prepared,
|
||||
external_idempotency_key,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(request_context)))
|
||||
}
|
||||
|
||||
pub(crate) async fn generate_editor_background_music_for_owner(
|
||||
@@ -630,3 +724,110 @@ fn editor_audio_bad_request(message: impl Into<String>) -> AppError {
|
||||
"message": message.into(),
|
||||
}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use serde_json::Value;
|
||||
use shared_contracts::assets;
|
||||
|
||||
use super::{
|
||||
CanonicalEditorBackgroundMusicSubmissionPayload, normalize_editor_background_music_request,
|
||||
prepare_editor_background_music_queue_job,
|
||||
};
|
||||
use crate::editor_generation_queue::serialize_editor_generation_job_payload_for_test;
|
||||
|
||||
fn background_music_payload(
|
||||
prompt: impl Into<String>,
|
||||
make_instrumental: bool,
|
||||
) -> assets::EditorBackgroundMusicGenerateRequest {
|
||||
assets::EditorBackgroundMusicGenerateRequest {
|
||||
gpt_description_prompt: prompt.into(),
|
||||
make_instrumental,
|
||||
project_id: Some("project-1".to_string()),
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
asset_folder_id: None,
|
||||
asset_label: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_background_music_normalization_uses_canonical_generation_prompt() {
|
||||
let prompt = "\u{0085} \u{2003}森林 \r\n\u{200b}\u{feff}音乐 \u{00a0}";
|
||||
let normalized =
|
||||
normalize_editor_background_music_request(background_music_payload(prompt, false))
|
||||
.expect("canonical BGM prompt should be accepted");
|
||||
|
||||
assert_eq!(
|
||||
normalized.gpt_description_prompt,
|
||||
"森林 \r\n\u{200b}\u{feff}音乐"
|
||||
);
|
||||
assert!(normalized.make_instrumental);
|
||||
|
||||
for invalid_prompt in [" \r\n\u{0085}", &"乐".repeat(201)] {
|
||||
let error = normalize_editor_background_music_request(background_music_payload(
|
||||
invalid_prompt,
|
||||
true,
|
||||
))
|
||||
.expect_err("invalid canonical BGM prompt should be rejected");
|
||||
assert_eq!(error.status_code(), axum::http::StatusCode::BAD_REQUEST);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn logged_in_background_music_queue_preparation_uses_canonical_prompt_and_frozen_price() {
|
||||
let pricing =
|
||||
crate::editor_generation_config::load_editor_generation_pricing_from_paths(None)
|
||||
.expect("default model pricing should be valid");
|
||||
let prepared =
|
||||
CanonicalEditorBackgroundMusicSubmissionPayload::new(background_music_payload(
|
||||
"\u{0085}\u{2003}森林 \r\n\u{200b}\u{feff}音乐\u{00a0}",
|
||||
false,
|
||||
))
|
||||
.expect("logged-in submission should canonicalize")
|
||||
.into_queue_job(&pricing)
|
||||
.expect("logged-in queue job should prepare");
|
||||
let request_payload_json =
|
||||
serialize_editor_generation_job_payload_for_test(&prepared.payload)
|
||||
.expect("the actual queue serializer should accept the canonical payload");
|
||||
let request_payload: Value =
|
||||
serde_json::from_str(&request_payload_json).expect("queue payload should be JSON");
|
||||
|
||||
assert_eq!(
|
||||
request_payload["gptDescriptionPrompt"],
|
||||
"森林 \r\n\u{200b}\u{feff}音乐"
|
||||
);
|
||||
assert_eq!(request_payload["makeInstrumental"], true);
|
||||
assert!(request_payload.get("priceMudPoints").is_none());
|
||||
assert_eq!(
|
||||
prepared.price_mud_points,
|
||||
pricing.background_music_model_mud_points(Some(platform_audio::SUNO_DEFAULT_MODEL))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn external_background_music_queue_preparation_preserves_the_raw_payload() {
|
||||
let pricing =
|
||||
crate::editor_generation_config::load_editor_generation_pricing_from_paths(None)
|
||||
.expect("default model pricing should be valid");
|
||||
let raw_prompt = "\u{0085}\u{2003}森林 \r\n\u{200b}\u{feff}音乐\u{00a0}";
|
||||
let prepared = prepare_editor_background_music_queue_job(
|
||||
background_music_payload(raw_prompt, false),
|
||||
&pricing,
|
||||
)
|
||||
.expect("External BGM queue job should validate without rewriting its payload");
|
||||
let request_payload_json =
|
||||
serialize_editor_generation_job_payload_for_test(&prepared.payload)
|
||||
.expect("the actual queue serializer should accept the raw External payload");
|
||||
let request_payload: Value =
|
||||
serde_json::from_str(&request_payload_json).expect("queue payload should be JSON");
|
||||
|
||||
assert_eq!(request_payload["gptDescriptionPrompt"], raw_prompt);
|
||||
assert_eq!(request_payload["makeInstrumental"], false);
|
||||
assert!(request_payload.get("priceMudPoints").is_none());
|
||||
assert_eq!(
|
||||
prepared.price_mud_points,
|
||||
pricing.background_music_model_mud_points(Some(platform_audio::SUNO_DEFAULT_MODEL))
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,9 +2,8 @@ use serde_json::{Map, Value, json};
|
||||
|
||||
use crate::{
|
||||
AudioError, BackgroundMusicTaskRequest, EditorBackgroundMusicTaskRequest,
|
||||
EditorSoundEffectTaskRequest, SUNO_DEFAULT_MODEL, SUNO_GPT_DESCRIPTION_PROMPT_MAX_CHARS,
|
||||
SUNO_PROMPT_MAX_CHARS, SUNO_TAGS_MAX_CHARS, SUNO_TITLE_MAX_CHARS, SoundEffectTaskRequest,
|
||||
VIDU_AUDIO_MODEL, VIDU_PROMPT_MAX_CHARS,
|
||||
EditorSoundEffectTaskRequest, SUNO_DEFAULT_MODEL, SUNO_PROMPT_MAX_CHARS, SUNO_TAGS_MAX_CHARS,
|
||||
SUNO_TITLE_MAX_CHARS, SoundEffectTaskRequest, VIDU_AUDIO_MODEL, VIDU_PROMPT_MAX_CHARS,
|
||||
};
|
||||
|
||||
pub fn build_background_music_task_body(
|
||||
@@ -62,16 +61,13 @@ pub fn build_sound_effect_task_body(request: SoundEffectTaskRequest) -> Result<V
|
||||
pub fn build_editor_background_music_task_body(
|
||||
request: EditorBackgroundMusicTaskRequest,
|
||||
) -> Result<Value, AudioError> {
|
||||
let prompt = normalize_limited_text(
|
||||
&request.gpt_description_prompt,
|
||||
"gpt_description_prompt",
|
||||
SUNO_GPT_DESCRIPTION_PROMPT_MAX_CHARS,
|
||||
)?;
|
||||
let prompt =
|
||||
crate::validate_background_music_generation_prompt(&request.gpt_description_prompt)?;
|
||||
let model = normalize_optional_text(request.model.as_deref())
|
||||
.unwrap_or_else(|| SUNO_DEFAULT_MODEL.to_string());
|
||||
Ok(json!({
|
||||
"mv": model,
|
||||
"gpt_description_prompt": prompt,
|
||||
"gpt_description_prompt": prompt.prompt,
|
||||
"make_instrumental": request.make_instrumental,
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -287,15 +287,27 @@ fn sound_effect_request_rejects_overlong_prompt() {
|
||||
#[test]
|
||||
fn editor_background_music_request_body_uses_gpt_description_prompt_and_instrumental() {
|
||||
let body = build_editor_background_music_task_body(EditorBackgroundMusicTaskRequest {
|
||||
gpt_description_prompt: " 轻快森林冒险背景音乐 ".to_string(),
|
||||
gpt_description_prompt: "\u{0085}\u{2003}轻快森林 \r\n\u{200b}\u{feff}冒险背景音乐\u{00a0}"
|
||||
.to_string(),
|
||||
make_instrumental: true,
|
||||
model: None,
|
||||
})
|
||||
.expect("editor background music body should be valid");
|
||||
|
||||
assert_eq!(body["mv"], SUNO_DEFAULT_MODEL);
|
||||
assert_eq!(body["gpt_description_prompt"], "轻快森林冒险背景音乐");
|
||||
assert_eq!(
|
||||
body["gpt_description_prompt"],
|
||||
"轻快森林 \r\n\u{200b}\u{feff}冒险背景音乐"
|
||||
);
|
||||
assert_eq!(body["make_instrumental"], true);
|
||||
assert_eq!(
|
||||
body.as_object()
|
||||
.expect("editor background music body should be an object")
|
||||
.keys()
|
||||
.map(String::as_str)
|
||||
.collect::<std::collections::BTreeSet<_>>(),
|
||||
std::collections::BTreeSet::from(["gpt_description_prompt", "make_instrumental", "mv",])
|
||||
);
|
||||
assert!(body.get("prompt").is_none());
|
||||
}
|
||||
|
||||
@@ -325,6 +337,18 @@ fn editor_background_music_request_body_rejects_overlong_gpt_description_prompt(
|
||||
assert!(error.message().contains("gpt_description_prompt 超过"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_background_music_request_body_rejects_unicode_whitespace_only_prompt() {
|
||||
let error = build_editor_background_music_task_body(EditorBackgroundMusicTaskRequest {
|
||||
gpt_description_prompt: "\u{0085}\u{2003}\r\n".to_string(),
|
||||
make_instrumental: true,
|
||||
model: None,
|
||||
})
|
||||
.expect_err("Unicode-whitespace-only prompt should fail");
|
||||
|
||||
assert!(error.message().contains("至少需要 1 个有效字符"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vidu_sound_effect_request_body_uses_text2audio_contract() {
|
||||
let body = build_sound_effect_task_body(platform_audio::SoundEffectTaskRequest {
|
||||
|
||||
@@ -1071,10 +1071,12 @@ export function buildBackgroundMusicGenerationInputs(
|
||||
gptDescriptionPrompt: string,
|
||||
): CanvasGenerationInputs {
|
||||
return {
|
||||
fields: createGenerationInputField(
|
||||
'gpt_description_prompt',
|
||||
gptDescriptionPrompt,
|
||||
),
|
||||
fields: [
|
||||
{
|
||||
title: 'gpt_description_prompt',
|
||||
value: gptDescriptionPrompt,
|
||||
},
|
||||
],
|
||||
references: [],
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1074,19 +1074,20 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
|
||||
const plan = buildImageGenerationSubmissionPlan({
|
||||
dialog: {
|
||||
mode: 'audio-background-music',
|
||||
prompt: ' 温暖轻快的森林冒险背景音乐 ',
|
||||
prompt: '不会进入正式请求的旧值',
|
||||
status: 'idle',
|
||||
},
|
||||
layers: [],
|
||||
nextGeneratedIndex: 5,
|
||||
canonicalBackgroundMusicPrompt: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
|
||||
});
|
||||
|
||||
expect(plan).toEqual({
|
||||
kind: 'audio',
|
||||
audioKind: 'background-music',
|
||||
normalizedPrompt: '温暖轻快的森林冒险背景音乐',
|
||||
normalizedPrompt: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
|
||||
input: {
|
||||
gptDescriptionPrompt: '温暖轻快的森林冒险背景音乐',
|
||||
gptDescriptionPrompt: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
|
||||
makeInstrumental: true,
|
||||
},
|
||||
result: {
|
||||
@@ -1095,7 +1096,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
|
||||
fields: [
|
||||
{
|
||||
title: 'gpt_description_prompt',
|
||||
value: '温暖轻快的森林冒险背景音乐',
|
||||
value: '\uFEFF温暖轻快的森林冒险背景音乐\uFEFF',
|
||||
},
|
||||
],
|
||||
references: [],
|
||||
@@ -1103,4 +1104,18 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('does not build a background music plan without a claimed canonical prompt', () => {
|
||||
expect(() =>
|
||||
buildImageGenerationSubmissionPlan({
|
||||
dialog: {
|
||||
mode: 'audio-background-music',
|
||||
prompt: ' 不应原生 trim 或回退默认值 ',
|
||||
status: 'idle',
|
||||
},
|
||||
layers: [],
|
||||
nextGeneratedIndex: 1,
|
||||
}),
|
||||
).toThrow('背景音乐提交缺少已确认的提示词');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -55,6 +55,7 @@ type ImageGenerationSubmissionOptions = {
|
||||
dialog: GenerateDialogState;
|
||||
layers: CanvasLayer[];
|
||||
nextGeneratedIndex: number;
|
||||
canonicalBackgroundMusicPrompt?: string;
|
||||
};
|
||||
|
||||
export const EDITOR_GENERATED_ASSET_LABEL_MAX_CHARS = 80;
|
||||
@@ -137,9 +138,6 @@ function getDialogDefaultPrompt(mode: GenerateDialogState['mode']) {
|
||||
if (mode === 'audio-sound-effect') {
|
||||
return '游戏音效';
|
||||
}
|
||||
if (mode === 'audio-background-music') {
|
||||
return '游戏背景音乐';
|
||||
}
|
||||
return 'AI 生成图片';
|
||||
}
|
||||
|
||||
@@ -228,7 +226,32 @@ export function buildImageGenerationSubmissionPlan({
|
||||
dialog,
|
||||
layers,
|
||||
nextGeneratedIndex,
|
||||
canonicalBackgroundMusicPrompt,
|
||||
}: ImageGenerationSubmissionOptions): ImageGenerationSubmissionPlan {
|
||||
if (dialog.mode === 'audio-background-music') {
|
||||
if (canonicalBackgroundMusicPrompt === undefined) {
|
||||
throw new Error('背景音乐提交缺少已确认的提示词');
|
||||
}
|
||||
return {
|
||||
kind: 'audio',
|
||||
audioKind: 'background-music',
|
||||
normalizedPrompt: canonicalBackgroundMusicPrompt,
|
||||
input: {
|
||||
gptDescriptionPrompt: canonicalBackgroundMusicPrompt,
|
||||
makeInstrumental: true,
|
||||
},
|
||||
result: {
|
||||
title: resolveGenerationAssetLabel(
|
||||
dialog.assetLabel,
|
||||
`游戏背景音乐 ${nextGeneratedIndex}`,
|
||||
),
|
||||
generationInputs: buildBackgroundMusicGenerationInputs(
|
||||
canonicalBackgroundMusicPrompt,
|
||||
),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const normalizedPrompt =
|
||||
dialog.prompt.trim() || getDialogDefaultPrompt(dialog.mode);
|
||||
|
||||
@@ -537,26 +560,6 @@ export function buildImageGenerationSubmissionPlan({
|
||||
};
|
||||
}
|
||||
|
||||
if (dialog.mode === 'audio-background-music') {
|
||||
return {
|
||||
kind: 'audio',
|
||||
audioKind: 'background-music',
|
||||
normalizedPrompt,
|
||||
input: {
|
||||
gptDescriptionPrompt: normalizedPrompt,
|
||||
makeInstrumental: true,
|
||||
},
|
||||
result: {
|
||||
title: resolveGenerationAssetLabel(
|
||||
dialog.assetLabel,
|
||||
`游戏背景音乐 ${nextGeneratedIndex}`,
|
||||
),
|
||||
generationInputs:
|
||||
buildBackgroundMusicGenerationInputs(normalizedPrompt),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const imageModel = normalizeEditorImageModel(dialog.imageModel);
|
||||
return {
|
||||
kind: 'image',
|
||||
|
||||
@@ -1004,7 +1004,7 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('preserves undo on submission rejection and ignores finishes for closed or deleted dialogs', async () => {
|
||||
it('preserves undo on rejection, settles a closed composer, and ignores a deleted dialog', async () => {
|
||||
promptAssistClientMocks.complete.mockResolvedValueOnce(
|
||||
promptAssistResponse('完整的森林冒险背景音乐'),
|
||||
);
|
||||
@@ -1106,7 +1106,7 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
|
||||
await waitFor(() => {
|
||||
expect(
|
||||
result.current.promptAssist.getDialogState('dialog-close').status,
|
||||
).toBe('idle');
|
||||
).toBe('submitting');
|
||||
expect(
|
||||
Object.prototype.hasOwnProperty.call(
|
||||
result.current.promptAssist.dialogStates,
|
||||
@@ -1115,13 +1115,12 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
const statesBeforeFinish = result.current.promptAssist.dialogStates;
|
||||
let closeFinished = true;
|
||||
let closeFinished = false;
|
||||
let deleteFinished = true;
|
||||
act(() => {
|
||||
closeFinished = result.current.promptAssist.finishSubmission({
|
||||
operation: closeClaim.operation,
|
||||
accepted: false,
|
||||
accepted: true,
|
||||
});
|
||||
deleteFinished = result.current.promptAssist.finishSubmission({
|
||||
operation: deleteClaim.operation,
|
||||
@@ -1129,9 +1128,20 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
|
||||
});
|
||||
});
|
||||
|
||||
expect(closeFinished).toBe(false);
|
||||
expect(closeFinished).toBe(true);
|
||||
expect(deleteFinished).toBe(false);
|
||||
expect(result.current.promptAssist.dialogStates).toBe(statesBeforeFinish);
|
||||
expect(
|
||||
result.current.promptAssist.getDialogState('dialog-close'),
|
||||
).toMatchObject({
|
||||
status: 'idle',
|
||||
operationId: null,
|
||||
});
|
||||
expect(
|
||||
Object.prototype.hasOwnProperty.call(
|
||||
result.current.promptAssist.dialogStates,
|
||||
'dialog-delete',
|
||||
),
|
||||
).toBe(false);
|
||||
expect(
|
||||
requireDialog(result.current.dialogs, 'dialog-close').composerOpen,
|
||||
).toBe(false);
|
||||
@@ -1139,4 +1149,53 @@ describe('useImageCanvasBackgroundMusicPromptAssist', () => {
|
||||
result.current.dialogs.some((dialog) => dialog.id === 'dialog-delete'),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('rejects a submission finish after the committed scope changes', () => {
|
||||
const initialDialogs = [createDialog('dialog-a', '森林冒险')];
|
||||
const { result, rerender } = renderHook(
|
||||
({ currentUserId, projectId }: Omit<HarnessProps, 'initialDialogs'>) =>
|
||||
usePromptAssistHarness({
|
||||
initialDialogs,
|
||||
currentUserId,
|
||||
projectId,
|
||||
}),
|
||||
{
|
||||
initialProps: {
|
||||
currentUserId: 'user-a',
|
||||
projectId: 'project-a',
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
let claim!: NonNullable<
|
||||
ReturnType<typeof result.current.promptAssist.beginSubmission>
|
||||
>;
|
||||
act(() => {
|
||||
claim = result.current.promptAssist.beginSubmission('dialog-a')!;
|
||||
});
|
||||
|
||||
rerender({
|
||||
currentUserId: 'user-a',
|
||||
projectId: 'project-b',
|
||||
});
|
||||
|
||||
let finished = true;
|
||||
act(() => {
|
||||
finished = result.current.promptAssist.finishSubmission({
|
||||
operation: claim.operation,
|
||||
accepted: true,
|
||||
});
|
||||
});
|
||||
|
||||
expect(finished).toBe(false);
|
||||
expect(result.current.promptAssist.getDialogState('dialog-a')).toMatchObject(
|
||||
{
|
||||
status: 'idle',
|
||||
operationId: null,
|
||||
},
|
||||
);
|
||||
expect(requireDialog(result.current.dialogs, 'dialog-a').prompt).toBe(
|
||||
'森林冒险',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -265,7 +265,10 @@ export function useImageCanvasBackgroundMusicPromptAssist({
|
||||
}
|
||||
}
|
||||
|
||||
for (const [dialogId] of activeOperationsRef.current) {
|
||||
for (const [dialogId, activeOperation] of activeOperationsRef.current) {
|
||||
if (activeOperation.operation.status === 'submitting') {
|
||||
continue;
|
||||
}
|
||||
const dialog = getCanvasGenerationDialogById(dialogId);
|
||||
if (!isBackgroundMusicDialog(dialog) || dialog.composerOpen === false) {
|
||||
clearActiveOperation(dialogId, {
|
||||
@@ -650,8 +653,7 @@ export function useImageCanvasBackgroundMusicPromptAssist({
|
||||
const latestDialog = getCanvasGenerationDialogById(operation.dialogId);
|
||||
if (
|
||||
!Object.is(activeOperation.scopeKey, committedScopeKeyRef.current) ||
|
||||
!isBackgroundMusicDialog(latestDialog) ||
|
||||
latestDialog.composerOpen === false
|
||||
!isBackgroundMusicDialog(latestDialog)
|
||||
) {
|
||||
clearActiveOperation(operation.dialogId, {
|
||||
abort: false,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -3,7 +3,9 @@ import {
|
||||
type MutableRefObject,
|
||||
type SetStateAction,
|
||||
useCallback,
|
||||
useLayoutEffect,
|
||||
useMemo,
|
||||
useRef,
|
||||
} from 'react';
|
||||
|
||||
import type { ExternalGenerationJobStatusRecord } from '../../../packages/shared/src/contracts/externalGeneration';
|
||||
@@ -83,6 +85,7 @@ import type {
|
||||
} from './ImageCanvasUiAssetExtractionModel';
|
||||
import { resolveUiAssetExtractionGenerationPlan } from './ImageCanvasUiAssetExtractionModel';
|
||||
import { renderUiDesignAssetExtractionMarkedImage } from './ImageCanvasUiAssetExtractionRasterModel';
|
||||
import type { BackgroundMusicPromptAssistController } from './useImageCanvasBackgroundMusicPromptAssist';
|
||||
|
||||
type CanvasSize = { width: number; height: number };
|
||||
|
||||
@@ -282,6 +285,11 @@ type GenerationSubmissionWorkflowOptions = {
|
||||
updater: CanvasGenerationDialogUpdater,
|
||||
) => void;
|
||||
hasCanvasGenerationDialogById: (dialogId: string) => boolean;
|
||||
getCanvasGenerationDialogById: (
|
||||
dialogId: string,
|
||||
) => CanvasGenerationDialogState | undefined;
|
||||
activeCanvasGenerationDialogId?: string | null;
|
||||
backgroundMusicPromptAssist: BackgroundMusicPromptAssistController;
|
||||
getGeneratingDialogPlaceholder: (
|
||||
dialog: GenerateDialogState,
|
||||
) => GenerateDialogState['placeholder'];
|
||||
@@ -301,6 +309,7 @@ type GenerationSubmissionWorkflowOptions = {
|
||||
setActiveSidebarPanel: Dispatch<SetStateAction<SidebarPanel | null>>;
|
||||
rememberImageModel: (imageModel: string) => void;
|
||||
projectId?: string | null;
|
||||
currentUserId?: string | null;
|
||||
assetFolderId?: string | null;
|
||||
upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void;
|
||||
applyProjectSnapshot?: (project: EditorProjectSnapshot) => void;
|
||||
@@ -606,6 +615,9 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
openCanvasGenerationDialog,
|
||||
updateCanvasGenerationDialogById,
|
||||
hasCanvasGenerationDialogById,
|
||||
getCanvasGenerationDialogById,
|
||||
activeCanvasGenerationDialogId,
|
||||
backgroundMusicPromptAssist,
|
||||
getGeneratingDialogPlaceholder,
|
||||
appendCanvasLayersWithResources,
|
||||
captureCanvasHistory,
|
||||
@@ -616,6 +628,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
setActiveSidebarPanel,
|
||||
rememberImageModel,
|
||||
projectId,
|
||||
currentUserId,
|
||||
assetFolderId,
|
||||
upsertGeneratedAsset,
|
||||
applyProjectSnapshot,
|
||||
@@ -623,6 +636,76 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
onWalletBalanceMayHaveChanged,
|
||||
onGenerationWarning,
|
||||
}: GenerationSubmissionWorkflowOptions) {
|
||||
const currentBackgroundMusicSubmissionScopeRef = useRef({
|
||||
currentUserId: currentUserId ?? null,
|
||||
projectId: projectId ?? null,
|
||||
activeCanvasGenerationDialogId,
|
||||
version: 0,
|
||||
mounted: false,
|
||||
});
|
||||
useLayoutEffect(() => {
|
||||
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
|
||||
const scopeChanged =
|
||||
!Object.is(currentScope.currentUserId, currentUserId ?? null) ||
|
||||
!Object.is(currentScope.projectId, projectId ?? null);
|
||||
currentBackgroundMusicSubmissionScopeRef.current = {
|
||||
currentUserId: currentUserId ?? null,
|
||||
projectId: projectId ?? null,
|
||||
activeCanvasGenerationDialogId,
|
||||
version: scopeChanged ? currentScope.version + 1 : currentScope.version,
|
||||
mounted: true,
|
||||
};
|
||||
return () => {
|
||||
currentBackgroundMusicSubmissionScopeRef.current.mounted = false;
|
||||
};
|
||||
}, [activeCanvasGenerationDialogId, currentUserId, projectId]);
|
||||
|
||||
const isBackgroundMusicSubmissionUiTargetCurrent = useCallback(
|
||||
(scope: {
|
||||
currentUserId: string | null;
|
||||
projectId: string | null;
|
||||
dialogId: string;
|
||||
scopeVersion: number;
|
||||
}) => {
|
||||
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
|
||||
const currentDialog = getCanvasGenerationDialogById(scope.dialogId);
|
||||
return (
|
||||
currentScope.mounted &&
|
||||
currentScope.version === scope.scopeVersion &&
|
||||
Object.is(scope.currentUserId, currentScope.currentUserId) &&
|
||||
Object.is(scope.projectId, currentScope.projectId) &&
|
||||
currentDialog?.mode === 'audio-background-music'
|
||||
);
|
||||
},
|
||||
[getCanvasGenerationDialogById],
|
||||
);
|
||||
|
||||
const isBackgroundMusicSubmissionAccountCurrent = useCallback(
|
||||
(scope: { currentUserId: string | null }) => {
|
||||
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
|
||||
return (
|
||||
currentScope.mounted &&
|
||||
Object.is(scope.currentUserId, currentScope.currentUserId)
|
||||
);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
const isBackgroundMusicSubmissionProjectCurrent = useCallback(
|
||||
(scope: {
|
||||
currentUserId: string | null;
|
||||
projectId: string | null;
|
||||
}) => {
|
||||
const currentScope = currentBackgroundMusicSubmissionScopeRef.current;
|
||||
return (
|
||||
currentScope.mounted &&
|
||||
Object.is(scope.currentUserId, currentScope.currentUserId) &&
|
||||
Object.is(scope.projectId, currentScope.projectId)
|
||||
);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
const addGeneratedLayersToCanvas = useCallback(
|
||||
(nextLayers: CanvasLayer[]) => {
|
||||
if (!nextLayers.length) {
|
||||
@@ -1751,24 +1834,46 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
|
||||
const submitImageGeneration = useCallback(
|
||||
async (dialog: GenerateDialogState) => {
|
||||
const normalizedPrompt =
|
||||
dialog.prompt.trim() ||
|
||||
(dialog.mode === 'edit'
|
||||
? '修改当前图片'
|
||||
: dialog.mode === 'audio-sound-effect'
|
||||
? '游戏音效'
|
||||
: dialog.mode === 'audio-background-music'
|
||||
? '游戏背景音乐'
|
||||
: 'AI 生成图片');
|
||||
const canvasDialog = isCanvasGenerationDialog(dialog) ? dialog : null;
|
||||
if (canvasDialog) {
|
||||
const backgroundMusicDialog =
|
||||
canvasDialog?.mode === 'audio-background-music' ? canvasDialog : null;
|
||||
if (dialog.mode === 'audio-background-music' && !backgroundMusicDialog) {
|
||||
return;
|
||||
}
|
||||
const backgroundMusicClaim = backgroundMusicDialog
|
||||
? backgroundMusicPromptAssist.beginSubmission(backgroundMusicDialog.id)
|
||||
: null;
|
||||
if (backgroundMusicDialog && !backgroundMusicClaim) {
|
||||
return;
|
||||
}
|
||||
const backgroundMusicSubmission =
|
||||
backgroundMusicDialog && backgroundMusicClaim
|
||||
? {
|
||||
currentUserId: currentUserId ?? null,
|
||||
projectId: projectId ?? null,
|
||||
dialogId: backgroundMusicDialog.id,
|
||||
operation: backgroundMusicClaim.operation,
|
||||
prompt: backgroundMusicClaim.prompt,
|
||||
scopeVersion:
|
||||
currentBackgroundMusicSubmissionScopeRef.current.version,
|
||||
}
|
||||
: null;
|
||||
const normalizedPrompt =
|
||||
backgroundMusicSubmission?.prompt ??
|
||||
(dialog.prompt.trim() ||
|
||||
(dialog.mode === 'edit'
|
||||
? '修改当前图片'
|
||||
: dialog.mode === 'audio-sound-effect'
|
||||
? '游戏音效'
|
||||
: 'AI 生成图片'));
|
||||
if (!backgroundMusicSubmission && canvasDialog) {
|
||||
updateCanvasGenerationDialogById(canvasDialog.id, (currentDialog) => ({
|
||||
...currentDialog,
|
||||
prompt: normalizedPrompt,
|
||||
status: 'generating',
|
||||
composerOpen: false,
|
||||
}));
|
||||
} else {
|
||||
} else if (!backgroundMusicSubmission) {
|
||||
setGenerateDialog({
|
||||
...dialog,
|
||||
prompt: normalizedPrompt,
|
||||
@@ -1777,11 +1882,15 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
});
|
||||
}
|
||||
|
||||
let backgroundMusicSubmissionAccepted = false;
|
||||
let backgroundMusicSubmissionUiOwned = false;
|
||||
let backgroundMusicQueueAcceptedWhileCurrentAndOpen = false;
|
||||
try {
|
||||
const submissionPlan = buildImageGenerationSubmissionPlan({
|
||||
dialog,
|
||||
layers,
|
||||
nextGeneratedIndex: layerCounterRef.current + 1,
|
||||
canonicalBackgroundMusicPrompt: backgroundMusicSubmission?.prompt,
|
||||
});
|
||||
if (submissionPlan.kind === 'edit') {
|
||||
const referenceImageSrc = await resolveEditorGenerationMediaReference(
|
||||
@@ -1971,9 +2080,28 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
} else if (submissionPlan.kind === 'audio') {
|
||||
const canvasCompletionPlaceholder =
|
||||
getGeneratingDialogPlaceholder(dialog);
|
||||
const generated = await runEditorGenerationWithWalletRefresh(
|
||||
const generated =
|
||||
submissionPlan.audioKind === 'sound-effect'
|
||||
? generateEditorSoundEffect({
|
||||
? await runEditorGenerationWithWalletRefresh(
|
||||
generateEditorSoundEffect({
|
||||
...submissionPlan.input,
|
||||
projectId,
|
||||
generationInputs: submissionPlan.result.generationInputs,
|
||||
assetFolderId,
|
||||
assetLabel: submissionPlan.result.title,
|
||||
...(projectId && canvasCompletionPlaceholder
|
||||
? {
|
||||
canvasCompletion: {
|
||||
dialogId: canvasDialog?.id,
|
||||
title: submissionPlan.result.title,
|
||||
placeholder: canvasCompletionPlaceholder,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
}),
|
||||
onWalletBalanceMayHaveChanged,
|
||||
)
|
||||
: await generateEditorBackgroundMusic({
|
||||
...submissionPlan.input,
|
||||
projectId,
|
||||
generationInputs: submissionPlan.result.generationInputs,
|
||||
@@ -1988,36 +2116,110 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
})
|
||||
: generateEditorBackgroundMusic({
|
||||
...submissionPlan.input,
|
||||
projectId,
|
||||
generationInputs: submissionPlan.result.generationInputs,
|
||||
assetFolderId,
|
||||
assetLabel: submissionPlan.result.title,
|
||||
...(projectId && canvasCompletionPlaceholder
|
||||
});
|
||||
const isBackgroundMusic =
|
||||
submissionPlan.audioKind === 'background-music';
|
||||
const backgroundMusicQueueState = isBackgroundMusic
|
||||
? queuedStateFromResponse(generated)
|
||||
: null;
|
||||
if (isBackgroundMusic && backgroundMusicSubmission) {
|
||||
backgroundMusicSubmissionAccepted = true;
|
||||
const currentScope =
|
||||
currentBackgroundMusicSubmissionScopeRef.current;
|
||||
const currentDialog = getCanvasGenerationDialogById(
|
||||
backgroundMusicSubmission.dialogId,
|
||||
);
|
||||
backgroundMusicQueueAcceptedWhileCurrentAndOpen =
|
||||
Boolean(backgroundMusicQueueState) &&
|
||||
currentScope.activeCanvasGenerationDialogId ===
|
||||
backgroundMusicSubmission.dialogId &&
|
||||
currentDialog?.composerOpen !== false;
|
||||
backgroundMusicSubmissionUiOwned =
|
||||
backgroundMusicPromptAssist.finishSubmission({
|
||||
operation: backgroundMusicSubmission.operation,
|
||||
accepted: true,
|
||||
});
|
||||
if (
|
||||
backgroundMusicQueueState &&
|
||||
backgroundMusicSubmissionUiOwned &&
|
||||
isBackgroundMusicSubmissionUiTargetCurrent(
|
||||
backgroundMusicSubmission,
|
||||
)
|
||||
) {
|
||||
updateCanvasGenerationDialogById(
|
||||
backgroundMusicSubmission.dialogId,
|
||||
(currentDialog) =>
|
||||
currentDialog.mode === 'audio-background-music'
|
||||
? {
|
||||
canvasCompletion: {
|
||||
dialogId: canvasDialog?.id,
|
||||
title: submissionPlan.result.title,
|
||||
placeholder: canvasCompletionPlaceholder,
|
||||
},
|
||||
...currentDialog,
|
||||
prompt: backgroundMusicSubmission.prompt,
|
||||
status: 'generating',
|
||||
composerOpen: false,
|
||||
errorMessage: undefined,
|
||||
}
|
||||
: {}),
|
||||
}),
|
||||
onWalletBalanceMayHaveChanged,
|
||||
);
|
||||
: currentDialog,
|
||||
);
|
||||
}
|
||||
if (
|
||||
!backgroundMusicQueueState &&
|
||||
isBackgroundMusicSubmissionAccountCurrent(
|
||||
backgroundMusicSubmission,
|
||||
)
|
||||
) {
|
||||
notifyWalletBalanceMayHaveChanged(onWalletBalanceMayHaveChanged);
|
||||
}
|
||||
}
|
||||
if (
|
||||
await applyQueuedEditorGenerationProject(
|
||||
generated,
|
||||
projectId,
|
||||
applyProjectSnapshot,
|
||||
onQueuedGenerationTask,
|
||||
onWalletBalanceMayHaveChanged,
|
||||
backgroundMusicSubmission && applyProjectSnapshot
|
||||
? (projectSnapshot) => {
|
||||
if (
|
||||
backgroundMusicSubmissionUiOwned &&
|
||||
isBackgroundMusicSubmissionUiTargetCurrent(
|
||||
backgroundMusicSubmission,
|
||||
)
|
||||
) {
|
||||
applyProjectSnapshot(projectSnapshot);
|
||||
}
|
||||
}
|
||||
: applyProjectSnapshot,
|
||||
backgroundMusicSubmission && onQueuedGenerationTask
|
||||
? () => {
|
||||
if (
|
||||
isBackgroundMusicSubmissionProjectCurrent(
|
||||
backgroundMusicSubmission,
|
||||
)
|
||||
) {
|
||||
onQueuedGenerationTask();
|
||||
}
|
||||
}
|
||||
: onQueuedGenerationTask,
|
||||
backgroundMusicSubmission && onWalletBalanceMayHaveChanged
|
||||
? () => {
|
||||
if (
|
||||
isBackgroundMusicSubmissionAccountCurrent(
|
||||
backgroundMusicSubmission,
|
||||
)
|
||||
) {
|
||||
onWalletBalanceMayHaveChanged();
|
||||
}
|
||||
}
|
||||
: onWalletBalanceMayHaveChanged,
|
||||
)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
if (
|
||||
backgroundMusicSubmission &&
|
||||
(!backgroundMusicSubmissionUiOwned ||
|
||||
!isBackgroundMusicSubmissionUiTargetCurrent(
|
||||
backgroundMusicSubmission,
|
||||
))
|
||||
) {
|
||||
return;
|
||||
}
|
||||
if (generated.project && applyProjectSnapshot) {
|
||||
applyProjectSnapshot(generated.project);
|
||||
if (generated.asset) {
|
||||
@@ -2114,7 +2316,56 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
if (canvasDialog) {
|
||||
if (backgroundMusicSubmission) {
|
||||
if (!backgroundMusicSubmissionAccepted) {
|
||||
backgroundMusicSubmissionUiOwned =
|
||||
backgroundMusicPromptAssist.finishSubmission({
|
||||
operation: backgroundMusicSubmission.operation,
|
||||
accepted: false,
|
||||
});
|
||||
if (
|
||||
isBackgroundMusicSubmissionAccountCurrent(
|
||||
backgroundMusicSubmission,
|
||||
)
|
||||
) {
|
||||
notifyWalletBalanceMayHaveChanged(
|
||||
onWalletBalanceMayHaveChanged,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (
|
||||
backgroundMusicSubmissionUiOwned &&
|
||||
isBackgroundMusicSubmissionUiTargetCurrent(
|
||||
backgroundMusicSubmission,
|
||||
)
|
||||
) {
|
||||
const currentScope =
|
||||
currentBackgroundMusicSubmissionScopeRef.current;
|
||||
const currentDialog = getCanvasGenerationDialogById(
|
||||
backgroundMusicSubmission.dialogId,
|
||||
);
|
||||
const shouldReopenComposer =
|
||||
currentScope.activeCanvasGenerationDialogId ===
|
||||
backgroundMusicSubmission.dialogId &&
|
||||
(currentDialog?.composerOpen !== false ||
|
||||
backgroundMusicQueueAcceptedWhileCurrentAndOpen);
|
||||
updateCanvasGenerationDialogById(
|
||||
backgroundMusicSubmission.dialogId,
|
||||
(latestDialog) =>
|
||||
latestDialog.mode === 'audio-background-music'
|
||||
? {
|
||||
...latestDialog,
|
||||
prompt: backgroundMusicSubmission.prompt,
|
||||
status: 'failed',
|
||||
composerOpen: shouldReopenComposer
|
||||
? true
|
||||
: latestDialog.composerOpen,
|
||||
errorMessage: resolveImageGenerationErrorMessage(error),
|
||||
}
|
||||
: latestDialog,
|
||||
);
|
||||
}
|
||||
} else if (canvasDialog) {
|
||||
updateCanvasGenerationDialogById(canvasDialog.id, () => ({
|
||||
...canvasDialog,
|
||||
prompt: normalizedPrompt,
|
||||
@@ -2138,7 +2389,13 @@ export function useImageCanvasGenerationSubmissionWorkflow({
|
||||
addAudioResultLayer,
|
||||
addVideoResultLayer,
|
||||
applyQuickEditResultToSourceLayer,
|
||||
backgroundMusicPromptAssist,
|
||||
currentUserId,
|
||||
getCanvasGenerationDialogById,
|
||||
getGeneratingDialogPlaceholder,
|
||||
isBackgroundMusicSubmissionAccountCurrent,
|
||||
isBackgroundMusicSubmissionProjectCurrent,
|
||||
isBackgroundMusicSubmissionUiTargetCurrent,
|
||||
layerCounterRef,
|
||||
layers,
|
||||
quickEditSelectionState,
|
||||
|
||||
@@ -1988,6 +1988,11 @@ export function useImageCanvasGenerationWorkflow({
|
||||
openCanvasGenerationDialog,
|
||||
updateCanvasGenerationDialogById,
|
||||
hasCanvasGenerationDialogById,
|
||||
getCanvasGenerationDialogById,
|
||||
activeCanvasGenerationDialogId: isCanvasGenerationDialog(generateDialog)
|
||||
? generateDialog.id
|
||||
: null,
|
||||
backgroundMusicPromptAssist,
|
||||
getGeneratingDialogPlaceholder,
|
||||
appendCanvasLayersWithResources,
|
||||
captureCanvasHistory,
|
||||
@@ -1998,6 +2003,7 @@ export function useImageCanvasGenerationWorkflow({
|
||||
setActiveSidebarPanel,
|
||||
rememberImageModel,
|
||||
projectId,
|
||||
currentUserId,
|
||||
assetFolderId,
|
||||
upsertGeneratedAsset,
|
||||
applyProjectSnapshot,
|
||||
|
||||
@@ -1612,13 +1612,27 @@ describe('editorProjectClient', () => {
|
||||
}),
|
||||
}),
|
||||
'生成游戏背景音乐失败',
|
||||
expect.objectContaining({
|
||||
{
|
||||
timeoutMs: 1_200_000,
|
||||
retry: editorRetryOptionsExpectation,
|
||||
}),
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('forwards background music transport failures without configuring retries', async () => {
|
||||
requestJsonMock.mockRejectedValueOnce(new Error('transport failed'));
|
||||
|
||||
await expect(
|
||||
generateEditorBackgroundMusic({
|
||||
gptDescriptionPrompt: '森林冒险背景音乐',
|
||||
makeInstrumental: true,
|
||||
}),
|
||||
).rejects.toThrow('transport failed');
|
||||
|
||||
expect(requestJsonMock.mock.calls[0]?.[3]).toEqual({
|
||||
timeoutMs: 1_200_000,
|
||||
});
|
||||
});
|
||||
|
||||
it('completes a background music prompt through the authenticated internal BFF', async () => {
|
||||
requestJsonMock.mockResolvedValueOnce({
|
||||
prompt: '轻快明亮的森林冒险背景音乐',
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { setStoredAccessToken } from '../apiClient';
|
||||
import { generateEditorBackgroundMusic } from './editorProjectClient';
|
||||
|
||||
function createLocalStorageMock() {
|
||||
const store = new Map<string, string>();
|
||||
|
||||
return {
|
||||
getItem(key: string) {
|
||||
return store.has(key) ? store.get(key)! : null;
|
||||
},
|
||||
setItem(key: string, value: string) {
|
||||
store.set(key, String(value));
|
||||
},
|
||||
removeItem(key: string) {
|
||||
store.delete(key);
|
||||
},
|
||||
clear() {
|
||||
store.clear();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe('editorProjectClient background music transport', () => {
|
||||
const fetchMock = vi.fn();
|
||||
const requestBody = {
|
||||
gptDescriptionPrompt: '森林冒险背景音乐',
|
||||
makeInstrumental: true as const,
|
||||
projectId: 'editor-project-1',
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
fetchMock.mockReset();
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
vi.stubGlobal('window', {
|
||||
dispatchEvent: vi.fn(),
|
||||
localStorage: createLocalStorageMock(),
|
||||
});
|
||||
setStoredAccessToken('background-music-test-token', { emit: false });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
function expectSingleBackgroundMusicPost() {
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit];
|
||||
expect(url).toBe('/api/editor/audios/background-music/generations');
|
||||
expect(init.method).toBe('POST');
|
||||
expect(JSON.parse(String(init.body))).toEqual(requestBody);
|
||||
}
|
||||
|
||||
it.each([429, 503])(
|
||||
'does not retry the formal POST after an HTTP %i response',
|
||||
async (status) => {
|
||||
fetchMock.mockResolvedValueOnce(new Response('', { status }));
|
||||
|
||||
await expect(
|
||||
generateEditorBackgroundMusic(requestBody),
|
||||
).rejects.toMatchObject({ status });
|
||||
|
||||
expectSingleBackgroundMusicPost();
|
||||
},
|
||||
);
|
||||
|
||||
it('does not retry the formal POST after a transport error', async () => {
|
||||
fetchMock.mockRejectedValueOnce(new TypeError('Failed to fetch'));
|
||||
|
||||
await expect(
|
||||
generateEditorBackgroundMusic(requestBody),
|
||||
).rejects.toBeInstanceOf(TypeError);
|
||||
|
||||
expectSingleBackgroundMusicPost();
|
||||
});
|
||||
});
|
||||
@@ -1291,7 +1291,6 @@ export async function generateEditorBackgroundMusic(
|
||||
'生成游戏背景音乐失败',
|
||||
{
|
||||
timeoutMs: 1_200_000,
|
||||
retry: EDITOR_REQUEST_RETRY_OPTIONS,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user