diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index d6fa72fc3..1b4d69ebb 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -6948,3 +6948,10 @@ - Windows 锁文件决策：提升权限进程新建 `.agent/.manifest.json.lock` 时，Windows 可能把 owner 设为 `Administrators`。仅在固定锁路径已取得不共享独占句柄并确认是普通、非 reparse、单链接文件后，才初始化为当前 `TokenUser`；随后再次复核句柄并执行原有 owner/DACL 校验，不放宽既有异常对象的安全规则。 - Provider Schema 决策：`agent.route_manifest.missingAssetSlots` 不再广告 OpenAI-compatible 代理拒绝的 `uniqueItems`；Runtime 继续排序去重，Schema 子集门禁新增该关键字，真实 Provider smoke 必须在发布前证明工具目录可被接受。 - 运行决策：Godot 项目提交给 Project Supervisor 时使用 `standard` Run Profile，避免触发 Web 专用 `game/index.html`、HTTP preview 与自主 Web 完成门。Godot 编辑器启动和内嵌运行预览不在本切片范围。 + +## 2026-08-13 Spine 序列帧多模态与视频直接转换 + +- 同一 Spine 序列帧面板固定分为“AI生成/改造”和“视频直接转换”两个标签，旧项目默认恢复 AI。AI 视频以 `motion` 表示动作视频、`appearance` 表示可选角色外观图；上传序列帧 PNG 只作为一张完整参考图，不做网格识别。 +- 视频直接转换走内部 `editor_character_animation_video_conversion` durable job，FFprobe + FFmpeg 覆盖完整源时长，采样率最高 8 FPS、最低 1 FPS，产出 2–120 帧；480p/720p 是只缩不放的最大长边。该路径固定免费、保留背景，不调用 Seedance 或 BgFilter，也不复制保存源视频。 +- 直接转换仍落 `character-animation` / `image-sequence`，配方 action 为 `character-animation.convert`，来源槽位为 `source`；最多 120 帧时首帧 item 同时承载最终 resource/asset。原子生成上限和角色动画拆帧上限扩为 120，但图标图集公开切片继续保持 64，既有 JSON 大小门禁不放宽。 +- `/api/external/v1` 不开放上述内部字段或直接转换路由，不修改 OpenAPI；本次不改 SpacetimeDB 表结构、迁移或生成 bindings。 diff --git a/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md b/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md index 71e79dc82..f0a127682 100644 --- a/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md +++ b/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md @@ -1,7 +1,17 @@ # Art Agent 多模态 Spine 序列帧动画生成方案 状态：已实施，待合并 -更新时间：`2026-08-11` +更新时间：`2026-08-13` + +## 2026-08-13 多模态输入与直接转换补充 + +- 同一面板分为“AI生成/改造”和“视频直接转换”两个标签，旧配方默认恢复 AI；两个标签独立保留草稿；标签为分段按钮样式（明显可点击、带选中态），不再像标题。 +- AI 视频以 `motion` 表示动作视频、`appearance` 表示可选角色外观图（可多张）；Provider 的 `reference_video` 负责动作、`reference_image` 负责外观。动作视频已选中后，外观图是可选的普通图片参考，不要求画布图层标记为 `character`；上传序列帧 PNG 作为一张完整参考图，不识别网格或拆帧。 +- 直接转换使用内部 `/api/editor/character-animations/video-conversions` 与 durable job `editor_character_animation_video_conversion`，固定免费、保留背景，不调用 Seedance、BgFilter 或其它去背景模型，也不重复保存源视频。**去背景暂缓**：直接转换的源视频背景非纯色，本地键色抠图不适用，待确定方案后再做。 +- 源视频仅接受 owner-scoped MP4/MOV，限制 50MB、2–15 秒。FFprobe 读取旋转后的显示尺寸、精确时长与平均 FPS；FFmpeg 覆盖完整时间轴，采样率为 `clamp(min(源平均 FPS, 8), 1, 8)`，产出 2–120 帧。 +- 480p/720p 表示最大长边，只缩小不放大。正式 `imageSequenceDurationMs` 等于探测时长，播放器、普通 ZIP 与 Spine JSON 的单帧时长统一按 `imageSequenceDurationMs ÷ frameCount`。 +- 直接转换仍落 `character-animation` / `image-sequence`，配方 action 为 `character-animation.convert`，来源槽位为 `source`。最多 120 帧时第一帧同时承载最终 resource/asset；角色动画拆帧允许 120 项，图标图集公开切片继续限制 64 项，既有 512 KiB/2 MiB JSON 门禁不放宽。 +- `/api/external/v1` 保持原图片模式，不暴露内部新增字段或直接转换接口，不修改 OpenAPI。 ## 1. 文档目的 @@ -25,12 +35,13 @@ 2. **不要新增新的序列帧媒体类型。**继续使用： - `assetKind = "character-animation"`； - `mediaType = "image-sequence"`； - - `generationInputs.action = "character-animation.generate"`。 + - AI 生成使用 `generationInputs.action = "character-animation.generate"`； + - 视频直接转换使用 `generationInputs.action = "character-animation.convert"`。 3. **扩展现有角色动作生成链路，而不是重写。**现有链路已经覆盖图片首帧驱动、Seedance 预览视频、抽帧、逐帧透明化、OSS、原子落库、队列恢复和失败退款。 4. **附件 ZIP 导出已经实现。**当前 `buildSpineImageSequenceJson` 与 `useImageCanvasAssetExportWorkflow` 生成的目录、`metadata.json`、`skeleton.json`、`README.md` 和 `manifest.txt` 与附件示例一致，只需要回归验证和入口文案调整。 5. **本期不应承诺“真正的 Spine 骨骼动画”。**UI 和文案应写“Spine 序列帧动画”或“Spine JSON 序列帧包”，避免用户误认为会自动拆件和蒙皮。 6. **1080p 暂不开放。**当前服务端明确拒绝 `seedance2.0-fast + 1080p`；应等供应商能力、成本和逐帧处理资源验证后再开放。 -7. **关键需求口径已经冻结并完成实施。**视频模式采用“参考视频 + 文本生成新视频后抽帧”，默认 480p × 4 秒为 40 泥点，最终帧自动透明化，拆帧只进入素材库；完整决策见第 13、16 节。 +7. **关键需求口径已经冻结并完成实施。**AI 视频模式采用“动作视频 + 可选角色外观图 + 文本”生成新视频后抽帧，按现有规格计费且最终帧自动透明化；视频直接转换固定免费、保留背景；拆帧只进入素材库。完整决策见第 13、16 节。 ### 2.2 为什么不是新玩法接入 @@ -82,7 +93,7 @@ manifest.txt 现有角色动作能力已经包含： -- 角色图片右键或浮动工具栏“生成动画”； +- 统一从底部工具栏“Spine序列帧动画”进入；不再从图层右键菜单或选中图层浮动工具栏暴露独立入口；当前选中视频（或带预览视频的序列帧）时，底部入口默认打开“视频直接转换”，其它情况默认打开“AI生成/改造”； - `character-animation` 生成占位框； - 动作描述； - 待机、行走、奔跑、跳跃、攻击、受击、倒下预设； @@ -509,10 +520,12 @@ flowchart TD `ImageCanvasBottomToolbarView.tsx`： -- 在“生成视频”后新增“Spine 序列帧动画”； -- 点击后在视口中心创建独立 `character-animation` generation dialog； -- 不在当前面板下方追加内容； -- 移动端保持横向滚动或现有工具栏收纳规则。 +- 在“生成视频”后保留唯一的“Spine 序列帧动画”入口； +- 点击后在视口中心创建统一的 `character-animation` generation dialog； +- 统一面板内同时承载“AI生成/改造”和“视频直接转换”两个标签； +- 当前选中视频或带预览视频的序列帧时默认进入“视频直接转换”，当前选中角色图时默认进入“AI生成/改造”，无适用选中图层时默认进入“AI生成/改造”； +- 右键菜单和选中图层浮动工具栏不再提供“生成动画”或“视频转换为Spine序列帧动画”平行入口； +- 不在当前面板下方追加内容；移动端保持横向滚动或现有工具栏收纳规则。 `CanvasTool` 增加 `character-animation`，但不新增页面路由。 @@ -755,19 +768,20 @@ idle 在 P0/P1 决策冻结后，本功能至少满足： -1. 底部工具栏最右侧出现“Spine 序列帧动画”，与前一个工具之间不增加分割线。 -2. 无参考、角色图片参考、视频参考三种合法提交都由后端权威校验。 -3. 画布选择非角色图片时提示“请选择角色图”；上传图片规则按最终确认口径执行。 -4. 请求只携带稳定对象引用，不携带 Data URL / blob URL。 -5. 生成中刷新页面后任务和占位框可恢复。 -6. 失败不产生残缺序列，且按现有 durable billing 规则退款。 -7. 完成后项目 resource 和账号 asset 均为 `character-animation`，帧顺序稳定。 -8. 序列图层可循环播放并显示当前帧/总帧数。 -9. 元数据展示类型、输入、Model、Resolution、Task。 -10. 改造可恢复原输入模式、参考、动作、背景和规格，并生成新资源。 -11. 拆帧不重复上传已有 PNG。 -12. Spine ZIP 与附件目录结构一致，`skeleton.json` 能通过现有 validator。 -13. 存量角色动作项目、External v1 图片模式和普通视频生成不回归。 +1. 底部工具栏最右侧出现“Spine 序列帧动画”，与前一个工具之间不增加分割线；Spine 序列帧不再从右键菜单或选中图层浮动工具栏提供平行入口。 +2. 点击底部入口后，统一面板同时提供“AI生成/改造”和“视频直接转换”；选中视频默认进入直接转换，选中角色图默认进入 AI，无适用选中图层时默认进入 AI。 +3. 无参考、角色图片参考、视频参考三种合法提交都由后端权威校验。 +4. 画布选择非角色图片时按当前多模态参考规则处理；上传图片规则按最终确认口径执行。 +5. 请求只携带稳定对象引用，不携带 Data URL / blob URL。 +6. 生成中刷新页面后任务和占位框可恢复。 +7. 失败不产生残缺序列，且按现有 durable billing 规则退款。 +8. 完成后项目 resource 和账号 asset 均为 `character-animation`，帧顺序稳定。 +9. 序列图层可循环播放并显示当前帧/总帧数。 +10. 元数据展示类型、输入、Model、Resolution、Task。 +11. 改造可恢复原输入模式、参考、动作、背景和规格，并生成新资源。 +12. 拆帧不重复上传已有 PNG。 +13. Spine ZIP 与附件目录结构一致，`skeleton.json` 能通过现有 validator。 +14. 存量角色动作项目、External v1 图片模式和普通视频生成不回归。 ## 15. 验证命令建议 @@ -808,7 +822,7 @@ git diff --check 本方案已按推荐 v1 口径落地： -- 底部工具栏已新增 `Spine序列帧动画` 入口；入口使用“播放器 + 帧格”图标，位于同一底栏的最右侧，与前一个工具之间不增加分割线，也不使用独立按钮容器；画布中的序列帧生成占位卡片复用同一个图标组件，不再显示旧的小人图标；底栏桌面宽度按完整内容自适应，不再因第十三个按钮出现横向滚动条，窄屏仍可横向滑动但不显示 scrollbar； +- 底部工具栏保留唯一的 `Spine序列帧动画` 入口；选中视频或带预览视频的序列帧时默认进入“视频直接转换”，选中角色图或无适用选中图层时默认进入“AI生成/改造”；右键菜单和选中图层浮动工具栏不再提供平行入口；入口使用“播放器 + 帧格”图标，位于同一底栏的最右侧，与前一个工具之间不增加分割线，也不使用独立按钮容器；画布中的序列帧生成占位卡片复用同一个图标组件，不再显示旧的小人图标；底栏桌面宽度按完整内容自适应，不再因第十三个按钮出现横向滚动条，窄屏仍可横向滑动但不显示 scrollbar； - 文本、角色图片、参考视频三种模式统一使用 `character-animation` generation dialog；单参考限制按当前 dialog 语义判断而不是依赖工具栏 `activeTool`，上传前只接收一个文件；图片、视频、音频参考的异步探测、OSS 上传、持久化和回填均绑定发起时的 dialog 身份与分媒体容量 reservation，切换面板后不把结果写入新面板；其他多参考生成面板仍保持原能力； - Seedance 2.0 Fast 支持 480p / 720p，不开放 1080p； - 背景键色冻结为白 `#FFFFFF`、绿 `#00FF00`、蓝 `#0000FF`、紫 `#8000FF`，最终产物继续自动透明化；新独立面板使用显式键色，旧角色图层“生成动画”入口继续由后端自动配色，前端在没有显式键色时不再伪装成绿色已选； diff --git a/docs/technical/【技术方案】角色规范到多视角图集拆分与动画生成流程-2026-08-12.md b/docs/technical/【技术方案】角色规范到多视角图集拆分与动画生成流程-2026-08-12.md index 04114540d..fbf65a63d 100644 --- a/docs/technical/【技术方案】角色规范到多视角图集拆分与动画生成流程-2026-08-12.md +++ b/docs/technical/【技术方案】角色规范到多视角图集拆分与动画生成流程-2026-08-12.md @@ -3,7 +3,8 @@ - 日期：2026-08-12 - 状态：设计稿 v3（简化：唯一新增能力是"从角色设定生成多向角色图"，拆分与动画全走现有能力） - 实现状态：**核心已实现**（多向图生成：后端 `character-sheet` kind + 前端角色面板开关，2026-08-12） - - 拆分集成：角色多方向图（action=`character-sheet.generate`）经既有「拆分图集」切出的单视角切片按 `character` 落库，可直接作为动画参考（`editor_spritesheet_slice_record_asset_kind`，`editor_project_icon.rs`） + - 方向固定为**八方向**（上、下、左、右 + 四角，围绕一圈排布，3×3 九宫格中心留空）；提示词按八向围绕一圈约束排布，降低拆分粘连概率。 + - 拆分：复用既有「拆分图集」的「连通块定位 + 外接矩形截图」算法（`slice_generated_icon_spritesheet_all_by_connected_components`），不新增像素级连通块掩码；切出的单视角切片按 `icon` 落库（`split_editor_icon_spritesheet`），不改 `character`。 - 关联：`【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md` ## 1. 背景 @@ -14,11 +15,11 @@ ## 2. 范围 -**唯一新增能力**：从角色设定（角色规范图 / 已有单视角角色图）生成一张**多方向角色图**（四视图默认）。 +**唯一新增能力**：从角色设定（角色规范图 / 已有单视角角色图）生成一张**多方向角色图**（八方向：上、下、左、右 + 四角，围绕一圈排布）。 **其余全部复用现有能力，不做额外保障：** - 图集类型：复用现有 `icon-spritesheet`（「图集」），不新增资产类型。 -- 拆分：复用现有「拆分图集」（连通块拆分，`split_editor_icon_spritesheet`），**手动触发**，不自动拆分。 +- 拆分：复用现有「拆分图集」（`split_editor_icon_spritesheet`，连通块定位 + 外接矩形截图），**手动触发**，不自动拆分。 - 方向/数量：**不保证**。生成错了用户重新抽卡；方向标签用户自己看。 - 动画：对拆分出的单视角图，按现有动画面板逐个生成。 @@ -31,7 +32,7 @@ 【新增】生成多方向角色图 → 产出 icon-spritesheet 图集 │ ▼ -【现有】用户手动「拆分图集」（连通块）→ N 张单视角图 +【现有】用户手动「拆分图集」→ N 张单视角图（按 icon 落库） │ ▼ 【现有】选单视角图 → 图生视频 → 抽帧 → Spine序列帧 @@ -39,17 +40,18 @@ ## 4. 新增能力：生成多方向角色图 -- 入口：在「生成角色形象」面板加一个按钮「生成多方向角色图」；或复用「生成图标素材」面板切到角色多向模式。建议前者。 +- 入口：在「生成角色形象」面板底部，作为「多方向图」可勾选项，与「像素艺术」并列；勾选后本次生成走 `character-sheet` 图集模式。 - 链路：复用图集生成链路（`generate_editor_icon_spritesheet_for_owner`），模型 `nanobanana2`。 -- 输出：一张 `icon-spritesheet` 图集，四个方向（前/后/左/右），同一角色、方向之间纯色底间隔、彼此分离。 +- 输出：一张 `icon-spritesheet` 图集，八个方向（正面、右前、右侧、右后、背面、左后、左侧、左前），按 3×3 九宫格围绕一圈排布、中心留空，同一角色、方向之间纯色底充分间隔、彼此分离。 - 提示词是质量关键（草案）： ``` -根据角色设定生成游戏角色的多方向视图图集。 -画面中只有同一个角色，按四个方向排布：正面、背面、左侧、右侧，每个方向一个完整角色，方向之间相互分离、互不重叠、互不接触。 -四个方向必须是同一个角色，脸型、发型、服装、配色、武器、头身比、体型全程一致，禁止漂移、换装、变形。 -每个方向角色完整居中，不裁切头顶、手脚或武器；四个方向底部基线对齐，画面占比一致。 -背景为单一纯色（绿幕 #00FF00 / 白底 #FFFFFF），平整无纹理、无渐变、无阴影、无地面、无道具；角色之间由纯色背景间隔，便于后续拆分。 +根据角色设定生成游戏角色的八方向视图图集。 +画面中只有同一个角色，共八个方向：正面、右前、右侧、右后、背面、左后、左侧、左前，每个方向一个完整角色。 +八个方向必须是同一个角色，脸型、发型、服装、配色、武器、头身比、体型全程一致，禁止漂移、换装、变形。 +八个方向按三乘三九宫格围绕一圈排布：上、下、左、右以及四个角落各放一个方向，九宫格中心留空，不放置角色。 +每个方向角色完整居中，不裁切头顶、手脚或武器；各方向底部基线对齐，画面占比一致。 +背景为单一纯色（绿幕 #00FF00），平整无纹理、无渐变、无阴影、无地面、无道具；角色之间由纯色背景充分间隔，互不接触、互不重叠，便于后续拆分。 禁止额外角色、分身、宠物、文字、水印、边框、编号、网格线。 角色设定：{} ``` @@ -65,7 +67,7 @@ ## 6. 边界 - 生成质量靠提示词；图不对 → 用户重新抽卡，**不做数量/方向校验**。 -- 连通块若粘连（方向贴太近）→ 拆分数量不对 → 重新抽卡或人工处理，不自动兜底。 +- 外接矩形若粘连（方向贴太近）→ 拆分数量不对 → 重新抽卡或人工处理，不自动兜底；提示词以「围绕一圈 + 纯色背景充分间隔」降低粘连概率。 - 成本：多向图一次图像生成；拆分 0 provider 成本；每个方向动画各一次（现有计价）。 - 旧链路（直接拿单视角角色图做动画）不变。 @@ -77,12 +79,12 @@ ## 8. 决策点（留给产品） -- 四视图 vs 八方向（当前建议四视图：前/后/左/右）。 -- 入口位置（「生成角色形象」面板按钮 vs 复用图标图集面板）。 -- 多向图背景色（绿幕/白底）。 +- 方向数已定**八方向**（上、下、左、右 + 四角，围绕一圈；不再提供四视角选项）。 +- 入口已定：角色面板底部「多方向图」可勾选项，与「像素艺术」并列。 +- 多向图背景色（当前固定绿幕 #00FF00）。 ## 9. 改动文件 -- 前端：`ImageCanvasGenerationModel.ts`（多向图提示词/常量）、`ImageCanvasGenerationDialogModel.ts`、`ImageCanvasGenerationSubmissionModel.ts`、`ImageCanvasGenerationComposerView.tsx`（入口按钮）。 -- 后端：`editor_project_icon.rs`（多向图生成提示词分支）或 `editor_project.rs`；`shared-contracts`（如需 DTO 字段）。 -- 无需动拆分逻辑、动画逻辑、存储层。 +- 前端：`ImageCanvasGenerationModel.ts`（多向图提示词/常量）、`ImageCanvasGenerationDialogModel.ts`、`ImageCanvasGenerationSubmissionModel.ts`、`ImageCanvasCharacterGenerationComposerView.tsx`（底部「多方向图」勾选项）、`ImageCanvasGenerationImageOptionsView.tsx`（`extraToggles` 插槽）。 +- 后端：`editor_project.rs`（`build_editor_character_multi_view_sheet_prompt` 八向围绕一圈提示词）。 +- 无需动拆分逻辑、动画逻辑、存储层；切片按 `icon` 落库，不加 `character` 分支。 diff --git a/server-rs/crates/api-server/src/app.rs b/server-rs/crates/api-server/src/app.rs index 510018922..7846a328b 100644 --- a/server-rs/crates/api-server/src/app.rs +++ b/server-rs/crates/api-server/src/app.rs @@ -1823,6 +1823,29 @@ mod tests { assert_eq!(response.status(), StatusCode::UNAUTHORIZED); } + #[tokio::test] + async fn editor_character_animation_video_conversion_requires_bearer_auth() { + let app = build_router(AppState::new(AppConfig::default()).expect("state should build")); + let response = app + .oneshot( + Request::builder() + .method("POST") + .uri("/api/editor/character-animations/video-conversions") + .header("content-type", "application/json") + .body(Body::from( + serde_json::json!({ + "sourceVideoSrc": "users/owner/source.mp4", + "resolution": "480p" + }) + .to_string(), + )) + .expect("request should build"), + ) + .await + .expect("request should succeed"); + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); + } + #[tokio::test] async fn editor_character_animation_frame_split_requires_bearer_auth() { let app = build_router(AppState::new(AppConfig::default()).expect("state should build")); diff --git a/server-rs/crates/api-server/src/asset_billing.rs b/server-rs/crates/api-server/src/asset_billing.rs index 11e7369eb..55410ec08 100644 --- a/server-rs/crates/api-server/src/asset_billing.rs +++ b/server-rs/crates/api-server/src/asset_billing.rs @@ -930,7 +930,7 @@ mod tests { for (source, expected_calls) in [ (include_str!("editor_project.rs"), 4), (include_str!("editor_project_icon.rs"), 2), - (include_str!("character_animation_assets.rs"), 2), + (include_str!("character_animation_assets.rs"), 3), ( include_str!("vector_engine_audio_generation/generation.rs"), 2, diff --git a/server-rs/crates/api-server/src/character_animation_assets.rs b/server-rs/crates/api-server/src/character_animation_assets.rs index 65454fc86..cba181a48 100644 --- a/server-rs/crates/api-server/src/character_animation_assets.rs +++ b/server-rs/crates/api-server/src/character_animation_assets.rs @@ -29,8 +29,8 @@ use module_assets::{ generate_asset_object_id, }; use platform_oss::{ - LegacyAssetPrefix, OssHeadObjectRequest, OssObjectAccess, OssPutObjectRequest, - OssRequestAttemptContext, OssSignedGetObjectUrlRequest, + LegacyAssetPrefix, OssDeleteObjectRequest, OssHeadObjectRequest, OssObjectAccess, + OssPutObjectRequest, OssRequestAttemptContext, OssSignedGetObjectUrlRequest, }; use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; @@ -45,7 +45,9 @@ use shared_contracts::assets::{ CharacterWorkflowCacheSaveRequest, CharacterWorkflowCacheSaveResponse, EditorCharacterAnimationFramePayload, EditorCharacterAnimationFrameSplitRequest, EditorCharacterAnimationFrameSplitResponse, EditorCharacterAnimationGenerateRequest, - EditorCharacterAnimationGenerateResponse, EditorVideoGenerateRequest, + EditorCharacterAnimationGenerateResponse, EditorCharacterAnimationBackgroundRemovalRequest, + EditorCharacterAnimationBackgroundRemovalResponse, EditorCharacterAnimationVideoConversionRequest, + EditorCharacterAnimationVideoConversionResponse, EditorVideoGenerateRequest, EditorVideoGenerateResponse, }; #[cfg(any())] @@ -75,7 +77,9 @@ use crate::{ }, editor_generation_config::EditorGenerationPricingConfig, editor_generation_queue::{ - EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND, + EDITOR_CHARACTER_ANIMATION_BACKGROUND_REMOVAL_JOB_KIND, + EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, + EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND, EditorGenerationQueuedResponse, editor_generation_queue_state, editor_generation_source_entity_id, enqueue_editor_generation_job_for_caller, }, @@ -91,9 +95,10 @@ use crate::{ normalize_editor_record_object_key, persist_editor_generation_result_atomically, preflight_editor_billable_generation_target, rebuild_editor_generation_input_references, remove_editor_generated_screen_background_with_bgfilter, + request_editor_background_removal_image_with_bgfilter_worker, resolve_editor_reference_object_key_for_owner, sanitize_editor_client_generation_inputs, serialize_editor_generation_inputs, serialize_editor_image_sequence_frames, - with_editor_media_duration_generation_input, + validate_editor_generation_operation_id, with_editor_media_duration_generation_input, }, editor_screen_background_decision::{ EditorScreenBackgroundDecisionInput, EditorScreenBackgroundDecisionKind, @@ -105,8 +110,10 @@ use crate::{ NormalizedEditorCharacterAnimationRequest, apply_editor_character_animation_authoritative_reference_dimensions, build_authoritative_editor_character_animation_generation_inputs, - build_editor_character_animation_frame_asset_label, editor_character_animation_bad_request, - editor_character_animation_source_asset_label, + build_authoritative_editor_character_animation_video_conversion_inputs, + build_editor_character_animation_frame_asset_label, + build_editor_character_animation_video_conversion_plan, + editor_character_animation_bad_request, editor_character_animation_source_asset_label, normalize_editor_character_animation_request_with_pricing, parse_editor_character_animation_background_color, resolve_editor_character_animation_asset_label, @@ -631,6 +638,10 @@ pub async fn generate_editor_character_animation( .reference_src .as_deref() .is_some_and(matches_inline_media_source) + || payload + .appearance_reference_srcs + .iter() + .any(|source| matches_inline_media_source(source.as_str())) { return Err(character_animation_error_response( &request_context, @@ -671,6 +682,58 @@ pub async fn generate_editor_character_animation( .await } +pub async fn convert_editor_character_animation_video( + State(state): State, + Extension(request_context): Extension, + Extension(authenticated): Extension, + payload: Result, JsonRejection>, +) -> Result, Response> { + let Json(payload) = payload.map_err(|error| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request(error.body_text()), + ) + })?; + if matches_inline_media_source(payload.source_video_src.as_str()) { + return Err(character_animation_error_response( + &request_context, + editor_character_animation_bad_request("源视频必须先上传 OSS，并使用稳定媒体引用。"), + )); + } + let owner_user_id = authenticated.claims().user_id().to_string(); + if !state.config.external_generation_mode.is_inline() { + let queue_job = enqueue_editor_character_animation_video_conversion_for_owner( + &state, + &request_context, + owner_user_id.as_str(), + payload, + ) + .await?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + let caller = editor_inline_generation_caller( + owner_user_id, + &request_context, + EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_JOB_KIND, + &payload, + ) + .map_err(|error| character_animation_error_response(&request_context, error))?; + with_editor_generation_durable_billing_boundary( + convert_editor_character_animation_video_for_owner( + state, + request_context, + caller, + Ok(Json(payload)), + ), + ) + .await +} + pub async fn split_editor_character_animation_frames( State(state): State, Extension(request_context): Extension, @@ -747,11 +810,11 @@ pub async fn split_editor_character_animation_frames( "message": "序列帧资源缺少完整的正式帧列表。", })) })?; - if frames.len() > 64 { + if frames.len() > 120 { return Err( AppError::from_status(StatusCode::UNPROCESSABLE_ENTITY).with_details(json!({ "provider": "editor-character-animation-frame-split", - "message": "单次最多拆分64帧。", + "message": "单次最多拆分120帧。", })), ); } @@ -980,6 +1043,371 @@ pub async fn split_editor_character_animation_frames( )) } +pub async fn remove_editor_character_animation_background( + State(state): State, + Extension(request_context): Extension, + Extension(authenticated): Extension, + Json(payload): Json, +) -> Result, Response> { + let owner_user_id = authenticated.claims().user_id().to_string(); + if !state.config.external_generation_mode.is_inline() { + let queue_job = enqueue_editor_character_animation_background_removal_for_owner( + &state, + &request_context, + owner_user_id.as_str(), + payload, + ) + .await?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + let caller = editor_inline_generation_caller( + owner_user_id, + &request_context, + EDITOR_CHARACTER_ANIMATION_BACKGROUND_REMOVAL_JOB_KIND, + &payload, + ) + .map_err(|error| character_animation_error_response(&request_context, error))?; + with_editor_generation_durable_billing_boundary( + remove_editor_character_animation_background_for_owner( + state, + request_context, + caller, + Ok(Json(payload)), + ), + ) + .await +} + +pub(crate) async fn enqueue_editor_character_animation_background_removal_for_owner( + state: &AppState, + request_context: &RequestContext, + owner_user_id: &str, + mut payload: EditorCharacterAnimationBackgroundRemovalRequest, +) -> Result { + let target = preflight_editor_billable_generation_target( + state, + owner_user_id, + Some(payload.project_id.clone()), + payload + .asset_folder_id + .clone() + .or_else(|| Some("project".to_string())), + ) + .await + .map_err(|error| character_animation_error_response(request_context, error))?; + payload.project_id = target.project_id.unwrap_or(payload.project_id); + payload.asset_folder_id = target.asset_folder_id; + let source_entity_id = editor_generation_source_entity_id( + Some(payload.project_id.as_str()), + "character-animation-background-removal", + ); + enqueue_editor_generation_job_for_caller( + state, + request_context, + owner_user_id, + EDITOR_CHARACTER_ANIMATION_BACKGROUND_REMOVAL_JOB_KIND, + source_entity_id, + "Spine序列帧去背景", + 0, + &payload, + None, + ) + .await + .map_err(|error| error.into_response_with_context(Some(request_context))) +} + +pub(crate) async fn remove_editor_character_animation_background_for_owner( + state: AppState, + request_context: RequestContext, + caller: EditorGenerationCaller, + payload: Result, JsonRejection>, +) -> Result, Response> { + let Json(payload) = payload.map_err(|error| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request(error.body_text()), + ) + })?; + let owner_user_id = caller.owner_user_id.clone(); + let project_id = payload.project_id.trim().to_string(); + let source_layer_id = payload.source_layer_id.trim().to_string(); + let source_resource_id = payload.source_resource_id.trim().to_string(); + if project_id.is_empty() || source_layer_id.is_empty() || source_resource_id.is_empty() { + return Err(character_animation_error_response( + &request_context, + editor_character_animation_bad_request( + "projectId、sourceLayerId 和 sourceResourceId 不能为空。", + ), + )); + } + let project = state + .spacetime_client() + .get_editor_project(EditorProjectGetRecordInput { + project_id: project_id.clone(), + owner_user_id: owner_user_id.clone(), + }) + .await + .map_err(map_editor_project_error) + .map_err(|error| character_animation_error_response(&request_context, error))?; + let source_resource = project + .resources + .iter() + .find(|resource| resource.resource_id == source_resource_id) + .cloned() + .ok_or_else(|| { + character_animation_error_response( + &request_context, + AppError::from_status(StatusCode::NOT_FOUND).with_details(json!({ + "provider": "editor-character-animation-background-removal", + "message": "未找到要去背景的Spine序列帧资源。", + })), + ) + })?; + if project.owner_user_id != owner_user_id + || source_resource.owner_user_id != owner_user_id + || source_resource.project_id != project_id + { + return Err(character_animation_error_response( + &request_context, + AppError::from_status(StatusCode::FORBIDDEN).with_details(json!({ + "provider": "editor-character-animation-background-removal", + "message": "序列帧资源不属于当前账号或当前项目。", + })), + )); + } + if source_resource.asset_kind.as_deref() + != Some(EDITOR_CHARACTER_ANIMATION_RESOURCE_ASSET_KIND) + { + return Err(character_animation_error_response( + &request_context, + AppError::from_status(StatusCode::UNPROCESSABLE_ENTITY).with_details(json!({ + "provider": "editor-character-animation-background-removal", + "message": "只有Spine序列帧动画可以去除背景。", + })), + )); + } + let frames: Vec = source_resource + .image_sequence_frames + .clone() + .and_then(|value| serde_json::from_value(value).ok()) + .filter(|frames: &Vec| !frames.is_empty()) + .ok_or_else(|| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request("序列帧资源缺少完整的正式帧列表。"), + ) + })?; + if frames.len() > 120 { + return Err(character_animation_error_response( + &request_context, + editor_character_animation_bad_request("单次最多处理120帧。"), + )); + } + let frame_width = source_resource.width; + let frame_height = source_resource.height; + let duration_ms = source_resource.image_sequence_duration_ms.unwrap_or(0); + let plan = crate::editor_sequence_animation::EditorCharacterAnimationVideoConversionPlan { + duration_seconds: duration_ms as f64 / 1000.0, + duration_ms, + source_fps: 0.0, + sampling_fps: 0.0, + frame_count: frames.len() as u32, + frame_width, + frame_height, + }; + let asset_label = + resolve_editor_character_animation_asset_label(payload.asset_label.as_deref()); + let operation = editor_generation_operation(&caller) + .map_err(|error| character_animation_error_response(&request_context, error))?; + let task_id = operation.operation_id.clone(); + let storage_task_id = format!( + "{}-{}", + sanitize_storage_segment(owner_user_id.as_str(), "owner"), + operation.operation_fingerprint + ); + if let Some(reporter) = caller.phase_reporter.as_ref() { + reporter + .report_processing(&state) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + } + let matting_audit = crate::external_api_audit::ExternalApiAuditContext { + user_id: Some(owner_user_id.clone()), + profile_id: Some(project_id.clone()), + request_id: Some(request_context.request_id().to_string()), + external_call_deadline: request_context.external_call_deadline(), + }; + let persisted_frames = extract_and_prepare_background_removal_frames( + &state, + &caller, + &storage_task_id, + &frames, + frame_width, + frame_height, + &matting_audit, + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + let result_frames = persisted_frames.frames.clone(); + let first_frame = result_frames.first().ok_or_else(|| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request("去背景没有产生序列帧。"), + ) + })?; + let frame_payload_json = serde_json::to_value(&result_frames).map_err(|error| { + character_animation_error_response( + &request_context, + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "editor-character-animation", + "message": format!("序列化去背景帧失败：{error}"), + })), + ) + })?; + let generation_inputs = rebuild_editor_generation_input_references( + &state, + owner_user_id.as_str(), + payload.generation_inputs.clone(), + vec![("源序列帧".to_string(), source_resource_id.clone())], + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + let completed_at_micros = current_utc_micros(); + let prepared = match prepare_editor_character_animation_sequence_result( + &caller, + Some(project_id.as_str()), + payload.asset_folder_id.as_deref(), + asset_label.as_str(), + "BgFilter", + Some(source_resource_id.clone()), + generation_inputs, + plan, + &persisted_frames, + frame_payload_json, + completed_at_micros, + ) { + Ok(prepared) => prepared, + Err(error) => { + cleanup_uncommitted_character_animation_objects( + &state, + &persisted_frames.uploaded_object_keys, + ) + .await; + return Err(character_animation_error_response(&request_context, error)); + } + }; + let layer_id = format!("layer-editor-character-animation-{task_id}"); + let canvas_item = payload.canvas_completion.as_ref().map(|completion| { + build_editor_character_animation_canvas_placement_item( + completion, + prepared.final_resource_id.as_str(), + first_frame.width, + first_frame.height, + layer_id.as_str(), + ) + }); + let canvas_layout_plan = EditorCanvasLayoutPlan::generation( + owner_user_id.as_str(), + Some(project_id.as_str()), + payload.canvas_completion.as_ref(), + canvas_item.into_iter().collect(), + Some(layer_id), + ); + let final_candidate = prepared + .items + .iter() + .find(|item| item.slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .ok_or_else(|| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request("去背景缺少最终候选记录。"), + ) + })?; + let compact_resource = final_candidate + .project_resource + .as_ref() + .map(compact_editor_generation_candidate_resource); + let compact_asset = final_candidate + .asset + .as_ref() + .map(compact_editor_generation_candidate_asset); + let job_result_payload_json = editor_generation_job_result_payload_json( + &caller, + json!({ + "ok": true, + "taskId": task_id, + "frames": &result_frames, + "frameCount": result_frames.len(), + "durationMs": duration_ms, + "frameWidth": frame_width, + "frameHeight": frame_height, + "priceMudPoints": 0, + "resource": compact_resource, + "asset": compact_asset, + }), + ); + let persisted = match persist_editor_generation_result_atomically( + &state, + &caller, + prepared.items, + canvas_layout_plan, + job_result_payload_json, + completed_at_micros, + ) + .await + { + Ok(persisted) => persisted, + Err(error) => { + cleanup_uncommitted_character_animation_objects( + &state, + &persisted_frames.uploaded_object_keys, + ) + .await; + return Err(character_animation_error_response(&request_context, error)); + } + }; + let final_resource = persisted + .items + .iter() + .find(|item| item.slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .and_then(|item| item.project_resource.clone()) + .map(editor_project_resource_payload_from_record) + .and_then(|resource| serde_json::to_value(resource).ok()); + let final_asset = persisted + .items + .iter() + .find(|item| item.slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .and_then(|item| item.asset.clone()) + .map(editor_asset_payload_from_record) + .and_then(|asset| serde_json::to_value(asset).ok()); + let completed_project = persisted + .project + .map(editor_project_payload_from_record) + .and_then(|project| serde_json::to_value(project).ok()); + Ok(json_success_body( + Some(&request_context), + EditorCharacterAnimationBackgroundRemovalResponse { + ok: true, + task_id, + frames: result_frames, + frame_count: plan.frame_count, + duration_ms, + frame_width, + frame_height, + price_mud_points: 0, + project: completed_project, + resource: final_resource, + asset: final_asset, + queue_state: None, + }, + )) +} + pub(crate) async fn enqueue_editor_character_animation_for_owner( state: &AppState, request_context: &RequestContext, @@ -992,6 +1420,10 @@ pub(crate) async fn enqueue_editor_character_animation_for_owner( .reference_src .as_deref() .is_some_and(matches_inline_media_source) + || payload + .appearance_reference_srcs + .iter() + .any(|source| matches_inline_media_source(source.as_str())) { return Err(character_animation_error_response( request_context, @@ -1051,6 +1483,54 @@ pub(crate) async fn enqueue_editor_character_animation_for_owner( .map_err(|error| error.into_response_with_context(Some(request_context))) } +pub(crate) async fn enqueue_editor_character_animation_video_conversion_for_owner( + state: &AppState, + request_context: &RequestContext, + owner_user_id: &str, + mut payload: EditorCharacterAnimationVideoConversionRequest, +) -> Result { + if matches_inline_media_source(payload.source_video_src.as_str()) { + return Err(character_animation_error_response( + request_context, + editor_character_animation_bad_request("源视频必须先上传 OSS，并使用稳定媒体引用。"), + )); + } + crate::editor_sequence_animation::normalize_editor_character_animation_video_conversion_resolution( + payload.resolution.as_str(), + ) + .map_err(|error| character_animation_error_response(request_context, error))?; + let target = preflight_editor_billable_generation_target( + state, + owner_user_id, + payload.project_id.clone(), + payload + .asset_folder_id + .clone() + .or_else(|| Some("project".to_string())), + ) + .await + .map_err(|error| character_animation_error_response(request_context, error))?; + payload.project_id = target.project_id; + payload.asset_folder_id = target.asset_folder_id; + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "character-animation-video-conversion", + ); + enqueue_editor_generation_job_for_caller( + state, + request_context, + owner_user_id, + EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_JOB_KIND, + source_entity_id, + "视频直接转换序列帧", + 0, + &payload, + None, + ) + .await + .map_err(|error| error.into_response_with_context(Some(request_context))) +} + pub(crate) async fn generate_editor_character_animation_for_owner( state: AppState, request_context: RequestContext, @@ -1085,6 +1565,10 @@ pub(crate) async fn generate_editor_character_animation_for_owner( .reference_src .as_deref() .is_some_and(matches_inline_media_source) + || payload + .appearance_reference_srcs + .iter() + .any(|source| matches_inline_media_source(source.as_str())) { return Err(character_animation_error_response( &request_context, @@ -1214,6 +1698,39 @@ pub(crate) async fn generate_editor_character_animation_for_owner( (Some(object_key), None) } }; + let mut appearance_object_keys = Vec::new(); + if !payload.appearance_reference_srcs.is_empty() { + if provisional_normalized.input_mode != EditorCharacterAnimationInputMode::Video { + return Err(character_animation_error_response( + &request_context, + editor_character_animation_bad_request("角色外观图仅支持视频模式。"), + )); + } + for appearance_src in &payload.appearance_reference_srcs { + let appearance_src = appearance_src.trim(); + if appearance_src.is_empty() { + continue; + } + let object_key = resolve_editor_reference_object_key_for_owner( + &state, + owner_user_id.as_str(), + appearance_src, + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + // 先下载并解码图片完成权威 MIME、尺寸和内容校验；Provider 阶段必须继续使用 + // 稳定 objectKey，由统一视频参考解析器签发可访问 URL，不能把内部 data URL 传下去。 + resolve_editor_character_animation_reference_image( + &state, + &source_resolve_client, + object_key.as_str(), + "appearanceReferenceSrcs", + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + appearance_object_keys.push(object_key); + } + } let priced = normalize_editor_character_animation_request_with_pricing( payload.clone(), &pricing, @@ -1230,15 +1747,33 @@ pub(crate) async fn generate_editor_character_animation_for_owner( ) .await .map_err(|error| character_animation_error_response(&request_context, error))?; + let mut appearance_source_resource_ids = Vec::with_capacity(appearance_object_keys.len()); + for (index, appearance_object_key) in appearance_object_keys.iter().enumerate() { + let appearance_source_resource_id = validate_editor_character_animation_source_resource( + &state, + owner_user_id.as_str(), + project_id.as_deref(), + EditorCharacterAnimationInputMode::Image, + payload + .appearance_source_resource_ids + .get(index) + .map(String::as_str), + Some(appearance_object_key.as_str()), + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + appearance_source_resource_ids.push(appearance_source_resource_id); + } let authoritative_generation_inputs = build_authoritative_editor_character_animation_generation_inputs( client_generation_inputs, &payload, priced.input_mode, reference_object_key.as_deref(), + &appearance_object_keys, ) .map_err(|error| character_animation_error_response(&request_context, error))?; - let generation_reference_sources = reference_object_key + let mut generation_reference_sources: Vec<(String, String)> = reference_object_key .as_ref() .map(|object_key| { ( @@ -1257,6 +1792,16 @@ pub(crate) async fn generate_editor_character_animation_for_owner( }) .into_iter() .collect(); + for (index, object_key) in appearance_object_keys.iter().enumerate() { + generation_reference_sources.push(( + "参考图片".to_string(), + appearance_source_resource_ids + .get(index) + .cloned() + .flatten() + .unwrap_or_else(|| object_key.clone()), + )); + } let generation_inputs = rebuild_editor_generation_input_references( &state, owner_user_id.as_str(), @@ -1361,6 +1906,7 @@ pub(crate) async fn generate_editor_character_animation_for_owner( task_id.as_str(), &normalized, reference_object_key.as_deref(), + &appearance_object_keys, ) .await? } @@ -1371,32 +1917,71 @@ pub(crate) async fn generate_editor_character_animation_for_owner( // 都走 execute_billable 的失败退款。数据库事实稍后与 canvas/job 一次性原子提交。 // 有别于角色生图 / 编辑器视频（provider 产物即交付物，provider 成功即收口扣费）。 if let Some(reporter) = caller.phase_reporter.as_ref() { - reporter.report_processing(&state).await?; + if let Err(error) = reporter.report_processing(&state).await { + cleanup_generated_editor_character_animation_preview( + &state, + &generated, + "editor_animation_preview_cleanup_failed", + ) + .await; + return Err(error); + } } - let persisted_frames = extract_and_prepare_editor_character_animation_frames( - &state, - &caller, - normalized.source_layer_id.as_str(), - task_id.as_str(), - generated.preview_video_path.as_str(), - &normalized, - &extraction_settings, - &matting_audit, - ) - .await?; + let persisted_frames = + match extract_and_prepare_editor_character_animation_frames( + &state, + &caller, + normalized.source_layer_id.as_str(), + task_id.as_str(), + generated.preview_video_path.as_str(), + &normalized, + &extraction_settings, + &matting_audit, + ) + .await + { + Ok(persisted_frames) => persisted_frames, + Err(error) => { + cleanup_generated_editor_character_animation_preview( + &state, + &generated, + "editor_animation_preview_cleanup_failed", + ) + .await; + return Err(error); + } + }; let frames = persisted_frames.frames.clone(); - frames.first().ok_or_else(|| { - AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ - "provider": "editor-character-animation", - "message": "角色动作抽帧完成但没有可保存的序列帧。", - })) - })?; - let frame_payload_json = serde_json::to_value(&frames).map_err(|error| { - AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ - "provider": "editor-character-animation", - "message": format!("角色动作序列帧无法持久化：{error}"), - })) - })?; + if frames.first().is_none() { + cleanup_generated_editor_character_animation_preview( + &state, + &generated, + "editor_animation_preview_cleanup_failed", + ) + .await; + return Err(AppError::from_status(StatusCode::BAD_GATEWAY).with_details( + json!({ + "provider": "editor-character-animation", + "message": "角色动作抽帧完成但没有可保存的序列帧。", + }), + )); + } + let frame_payload_json = match serde_json::to_value(&frames) { + Ok(frame_payload_json) => frame_payload_json, + Err(error) => { + cleanup_generated_editor_character_animation_preview( + &state, + &generated, + "editor_animation_preview_cleanup_failed", + ) + .await; + return Err(AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR) + .with_details(json!({ + "provider": "editor-character-animation", + "message": format!("角色动作序列帧无法持久化：{error}"), + }))); + } + }; Ok::<_, AppError>((generated, frames, normalized, persisted_frames, frame_payload_json)) }, ) @@ -1414,7 +1999,7 @@ pub(crate) async fn generate_editor_character_animation_for_owner( ) })?; let completed_at_micros = current_utc_micros(); - let prepared = prepare_editor_character_animation_result( + let prepared = match prepare_editor_character_animation_result( &caller, project_id.as_deref(), asset_folder_id.as_deref().or(Some("project")), @@ -1426,8 +2011,17 @@ pub(crate) async fn generate_editor_character_animation_for_owner( &persisted_frames, frame_payload_json, completed_at_micros, - ) - .map_err(|error| character_animation_error_response(&request_context, error))?; + ) { + Ok(prepared) => prepared, + Err(error) => { + let mut object_keys = persisted_frames.uploaded_object_keys.clone(); + if let Some(preview_object_key) = generated.object_key.as_deref() { + object_keys.push(preview_object_key.to_string()); + } + cleanup_uncommitted_character_animation_objects(&state, &object_keys).await; + return Err(character_animation_error_response(&request_context, error)); + } + }; let layer_id = format!("layer-editor-character-animation-{task_id}"); let canvas_item = canvas_completion.as_ref().map(|completion| { build_editor_character_animation_canvas_placement_item( @@ -1485,7 +2079,7 @@ pub(crate) async fn generate_editor_character_animation_for_owner( "asset": compact_asset, }), ); - let persisted = persist_editor_generation_result_atomically( + let persisted = match persist_editor_generation_result_atomically( &state, &caller, prepared.items, @@ -1494,7 +2088,17 @@ pub(crate) async fn generate_editor_character_animation_for_owner( completed_at_micros, ) .await - .map_err(|error| character_animation_error_response(&request_context, error))?; + { + Ok(persisted) => persisted, + Err(error) => { + let mut object_keys = persisted_frames.uploaded_object_keys.clone(); + if let Some(preview_object_key) = generated.object_key.as_deref() { + object_keys.push(preview_object_key.to_string()); + } + cleanup_uncommitted_character_animation_objects(&state, &object_keys).await; + return Err(character_animation_error_response(&request_context, error)); + } + }; let final_resource = persisted .items .iter() @@ -1554,6 +2158,303 @@ pub(crate) async fn generate_editor_character_animation_for_owner( )) } +pub(crate) async fn convert_editor_character_animation_video_for_owner( + state: AppState, + request_context: RequestContext, + caller: EditorGenerationCaller, + payload: Result, JsonRejection>, +) -> Result, Response> { + let Json(mut payload) = payload.map_err(|error| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request(error.body_text()), + ) + })?; + if matches_inline_media_source(payload.source_video_src.as_str()) { + return Err(character_animation_error_response( + &request_context, + editor_character_animation_bad_request("源视频必须先上传 OSS，并使用稳定媒体引用。"), + )); + } + if payload.canvas_completion.is_some() + && payload + .project_id + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()) + .is_none() + { + return Err(character_animation_error_response( + &request_context, + editor_character_animation_bad_request("视频直接转换画布完成需要有效的 projectId。"), + )); + } + let owner_user_id = caller.owner_user_id.clone(); + let target = preflight_editor_billable_generation_target( + &state, + owner_user_id.as_str(), + payload.project_id.clone(), + payload + .asset_folder_id + .clone() + .or_else(|| Some("project".to_string())), + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + payload.project_id = target.project_id; + payload.asset_folder_id = target.asset_folder_id; + let project_id = payload.project_id.clone(); + let canvas_completion = payload.canvas_completion.clone(); + let asset_label = + resolve_editor_character_animation_asset_label(payload.asset_label.as_deref()); + let source_client = build_upstream_http_client(SOURCE_IMAGE_RESOLVE_TIMEOUT_MS) + .map_err(|error| character_animation_error_response(&request_context, error))?; + let source_object_key = resolve_editor_reference_object_key_for_owner( + &state, + owner_user_id.as_str(), + payload.source_video_src.as_str(), + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + let extraction_settings = resolve_backend_frame_extraction_settings(&state); + let metadata = validate_editor_character_animation_reference_video_object( + &state, + &source_client, + owner_user_id.as_str(), + source_object_key.as_str(), + "sourceVideoSrc", + &extraction_settings, + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + let plan = build_editor_character_animation_video_conversion_plan( + metadata.duration_seconds, + metadata.source_fps, + metadata.width, + metadata.height, + payload.resolution.as_str(), + ) + .map_err(|error| character_animation_error_response(&request_context, error))?; + let source_resource_id = validate_editor_character_animation_source_resource( + &state, + owner_user_id.as_str(), + project_id.as_deref(), + EditorCharacterAnimationInputMode::Video, + payload.source_resource_id.as_deref(), + Some(source_object_key.as_str()), + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + let source_resource_id = + source_resource_id.or(resolve_editor_character_animation_source_asset_resource( + &state, + owner_user_id.as_str(), + project_id.as_deref(), + payload.generation_inputs.as_ref(), + source_object_key.as_str(), + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?); + let authoritative_inputs = + build_authoritative_editor_character_animation_video_conversion_inputs( + payload.generation_inputs.clone(), + &payload, + source_object_key.as_str(), + plan, + ) + .map_err(|error| character_animation_error_response(&request_context, error))?; + let generation_inputs = rebuild_editor_generation_input_references( + &state, + owner_user_id.as_str(), + Some(authoritative_inputs), + vec![( + "源视频".to_string(), + source_resource_id + .clone() + .unwrap_or_else(|| source_object_key.clone()), + )], + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + let operation = editor_generation_operation(&caller) + .map_err(|error| character_animation_error_response(&request_context, error))?; + let task_id = operation.operation_id.clone(); + // operation_id 用于持久化幂等；帧对象路径还绑定请求 fingerprint。相同 request id + // 的不同请求不会互相覆盖，重试同一请求仍能重用路径。 + let storage_task_id = format!( + "{}-{}", + sanitize_storage_segment(owner_user_id.as_str(), "owner"), + operation.operation_fingerprint + ); + if let Some(reporter) = caller.phase_reporter.as_ref() { + reporter + .report_processing(&state) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + } + let persisted_frames = extract_and_prepare_direct_video_conversion_frames( + &state, + &caller, + storage_task_id.as_str(), + source_object_key.as_str(), + plan, + &extraction_settings, + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))?; + let frames = persisted_frames.frames.clone(); + let first_frame = frames.first().ok_or_else(|| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request("视频直接转换没有产生序列帧。"), + ) + })?; + let frame_payload_json = serde_json::to_value(&frames).map_err(|error| { + character_animation_error_response( + &request_context, + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "editor-character-animation", + "message": format!("序列化直接转换帧失败：{error}"), + })), + ) + })?; + let completed_at_micros = current_utc_micros(); + let prepared = match prepare_editor_character_animation_sequence_result( + &caller, + project_id.as_deref(), + payload.asset_folder_id.as_deref(), + asset_label.as_str(), + "FFmpeg", + source_resource_id, + generation_inputs, + plan, + &persisted_frames, + frame_payload_json, + completed_at_micros, + ) { + Ok(prepared) => prepared, + Err(error) => { + cleanup_uncommitted_character_animation_objects( + &state, + &persisted_frames.uploaded_object_keys, + ) + .await; + return Err(character_animation_error_response(&request_context, error)); + } + }; + let layer_id = format!("layer-editor-character-animation-{task_id}"); + let canvas_item = canvas_completion.as_ref().map(|completion| { + build_editor_character_animation_canvas_placement_item( + completion, + prepared.final_resource_id.as_str(), + first_frame.width, + first_frame.height, + layer_id.as_str(), + ) + }); + let canvas_layout_plan = EditorCanvasLayoutPlan::generation( + owner_user_id.as_str(), + project_id.as_deref(), + canvas_completion.as_ref(), + canvas_item.into_iter().collect(), + Some(layer_id), + ); + let final_candidate = prepared + .items + .iter() + .find(|item| item.slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .ok_or_else(|| { + character_animation_error_response( + &request_context, + editor_character_animation_bad_request("视频直接转换缺少最终候选记录。"), + ) + })?; + let compact_resource = final_candidate + .project_resource + .as_ref() + .map(compact_editor_generation_candidate_resource); + let compact_asset = final_candidate + .asset + .as_ref() + .map(compact_editor_generation_candidate_asset); + let job_result_payload_json = editor_generation_job_result_payload_json( + &caller, + json!({ + "ok": true, + "taskId": task_id, + "processingMethod": "视频直接转换", + "frames": &frames, + "frameCount": frames.len(), + "durationMs": plan.duration_ms, + "frameWidth": plan.frame_width, + "frameHeight": plan.frame_height, + "sourceFps": plan.source_fps, + "samplingFps": plan.sampling_fps, + "priceMudPoints": 0, + "resource": compact_resource, + "asset": compact_asset, + }), + ); + let persisted = match persist_editor_generation_result_atomically( + &state, + &caller, + prepared.items, + canvas_layout_plan, + job_result_payload_json, + completed_at_micros, + ) + .await + { + Ok(persisted) => persisted, + Err(error) => { + cleanup_uncommitted_character_animation_objects( + &state, + &persisted_frames.uploaded_object_keys, + ) + .await; + return Err(character_animation_error_response(&request_context, error)); + } + }; + let final_resource = persisted + .items + .iter() + .find(|item| item.slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .and_then(|item| item.project_resource.clone()) + .map(editor_project_resource_payload_from_record) + .and_then(|resource| serde_json::to_value(resource).ok()); + let final_asset = persisted + .items + .iter() + .find(|item| item.slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .and_then(|item| item.asset.clone()) + .map(editor_asset_payload_from_record) + .and_then(|asset| serde_json::to_value(asset).ok()); + let completed_project = persisted + .project + .map(editor_project_payload_from_record) + .and_then(|project| serde_json::to_value(project).ok()); + Ok(json_success_body( + Some(&request_context), + EditorCharacterAnimationVideoConversionResponse { + ok: true, + task_id, + processing_method: "视频直接转换".to_string(), + frames, + frame_count: plan.frame_count, + duration_ms: plan.duration_ms, + frame_width: plan.frame_width, + frame_height: plan.frame_height, + source_fps: plan.source_fps, + sampling_fps: plan.sampling_fps, + price_mud_points: 0, + project: completed_project, + resource: final_resource, + asset: final_asset, + queue_state: None, + }, + )) +} + pub async fn generate_editor_video( State(state): State, Extension(request_context): Extension, @@ -2338,6 +3239,87 @@ async fn resolve_media_source_as_data_url( Ok(media_payload_as_data_url(&payload)) } +async fn resolve_editor_character_animation_source_asset_resource( + state: &AppState, + owner_user_id: &str, + project_id: Option<&str>, + generation_inputs: Option<&Value>, + source_object_key: &str, +) -> Result, AppError> { + let Some(reference) = generation_inputs + .and_then(Value::as_object) + .and_then(|inputs| inputs.get("references")) + .and_then(Value::as_array) + .and_then(|references| { + references.iter().find(|reference| { + reference.get("id").and_then(Value::as_str) == Some("source") + || reference.get("id").and_then(Value::as_str) == Some("motion") + }) + }) + else { + return Ok(None); + }; + if reference.get("refType").and_then(Value::as_str) != Some("asset") { + return Ok(None); + } + let Some(asset_id) = reference.get("refId").and_then(Value::as_str) else { + return Ok(None); + }; + let library = state + .spacetime_client() + .get_editor_asset_library(owner_user_id.to_string(), current_utc_micros()) + .await + .map_err(map_editor_project_error)?; + let Some(asset) = library + .assets + .iter() + .find(|asset| asset.asset_id.trim() == asset_id.trim()) + else { + return Err(editor_character_animation_bad_request( + "源视频素材不属于当前账号。", + )); + }; + let asset_object_key = normalize_editor_record_object_key( + asset.object_key.as_deref(), + asset.image_src.as_str(), + ) + .ok_or_else(|| editor_character_animation_bad_request("源视频素材缺少稳定 objectKey。"))?; + if asset_object_key != source_object_key.trim().trim_start_matches('/') { + return Err(editor_character_animation_bad_request( + "源视频素材与本次转换对象不一致。", + )); + } + let Some(source_resource_id) = asset.source_resource_id.as_deref() else { + return Ok(None); + }; + let Some(project_id) = project_id.map(str::trim).filter(|id| !id.is_empty()) else { + return Ok(None); + }; + let project = state + .spacetime_client() + .get_editor_project(EditorProjectGetRecordInput { + project_id: project_id.to_string(), + owner_user_id: owner_user_id.to_string(), + }) + .await + .map_err(map_editor_project_error)?; + let resource = project + .resources + .iter() + .find(|resource| resource.resource_id.trim() == source_resource_id.trim()) + .ok_or_else(|| { + editor_character_animation_bad_request("源视频素材来源资源不属于当前画布项目。") + })?; + if resource.owner_user_id.trim() != owner_user_id.trim() + || resource.project_id.trim() != project_id + { + return Err(editor_character_animation_bad_request( + "源视频素材来源资源不属于当前账号或当前画布项目。", + )); + } + Ok(Some(source_resource_id.trim().to_string())) +} + async fn validate_editor_character_animation_source_resource( state: &AppState, owner_user_id: &str, @@ -2459,8 +3441,13 @@ async fn resolve_editor_character_animation_reference_image( "参考图片缺少可识别的媒体宽高。", )); } + // 数据 URL 必须与声明的权威宽高一致：带 EXIF 旋转（如手机竖拍）的源图在 + // apply_orientation 后宽高已交换，若仍返回原始字节，`ratio="same"` 会用声明尺寸 + // 去算帧尺寸而 provider 实际拿到的是未旋转图像，导致 letterbox 与预览错位。 + let data_url = encode_rgba_image_as_png_data_url(&decoded.to_rgba8()) + .ok_or_else(|| editor_character_animation_bad_request("参考图片重新编码失败。"))?; Ok(ResolvedEditorCharacterAnimationReferenceImage { - data_url: media_payload_as_data_url(&payload), + data_url, width: decoded.width(), height: decoded.height(), }) @@ -2575,7 +3562,7 @@ fn probe_editor_character_animation_reference_video_metadata( "-select_streams", "v:0", "-show_entries", - "stream=width,height:stream_tags=rotate:stream_side_data=rotation:format=duration", + "stream=width,height,avg_frame_rate,r_frame_rate:stream_tags=rotate:stream_side_data=rotation:format=duration", "-of", "json", input_path.to_string_lossy().as_ref(), @@ -2633,10 +3620,22 @@ fn parse_editor_character_animation_reference_video_probe( { std::mem::swap(&mut width, &mut height); } + let source_fps = stream + .get("avg_frame_rate") + .and_then(Value::as_str) + .and_then(crate::editor_sequence_animation::parse_editor_character_animation_ffprobe_frame_rate) + .or_else(|| { + stream + .get("r_frame_rate") + .and_then(Value::as_str) + .and_then(crate::editor_sequence_animation::parse_editor_character_animation_ffprobe_frame_rate) + }) + .unwrap_or(1.0); Ok(EditorCharacterAnimationReferenceVideoMetadata { duration_seconds, width, height, + source_fps, }) } @@ -3007,14 +4006,26 @@ async fn put_generated_editor_character_animation_preview( ), ) .await?; - let prepared_asset_object = prepare_editor_character_animation_source_asset_object( + let prepared_asset_object = match prepare_editor_character_animation_source_asset_object( state, caller, source_layer_id, put_result.object_key.clone(), content_type, ) - .await?; + .await + { + Ok(prepared_asset_object) => prepared_asset_object, + Err(error) => { + cleanup_uncommitted_character_animation_object( + state, + put_result.object_key.as_str(), + "editor_animation_preview_prepare_cleanup_failed", + ) + .await; + return Err(error); + } + }; Ok(PersistedEditorCharacterAnimationPreview { legacy_public_path: put_result.legacy_public_path, @@ -3079,9 +4090,14 @@ async fn request_editor_sequence_animation_text_or_video_preview( task_id: &str, request: &NormalizedEditorCharacterAnimationRequest, reference_object_key: Option<&str>, + appearance_reference_srcs: &[String], ) -> Result { - let video_request = - build_editor_sequence_animation_video_request(settings, request, reference_object_key)?; + let video_request = build_editor_sequence_animation_video_request( + settings, + request, + reference_object_key, + appearance_reference_srcs, + )?; let video_settings = EditorVideoSettings { ark: settings.ark.clone(), }; @@ -3130,6 +4146,7 @@ fn build_editor_sequence_animation_video_request( settings: &EditorCharacterAnimationSettings, request: &NormalizedEditorCharacterAnimationRequest, reference_object_key: Option<&str>, + appearance_reference_srcs: &[String], ) -> Result { let reference_video_srcs = match request.input_mode { EditorCharacterAnimationInputMode::Text => Vec::new(), @@ -3145,7 +4162,14 @@ fn build_editor_sequence_animation_video_request( } }; Ok(NormalizedEditorVideoRequest { - prompt: request.prompt.clone(), + prompt: if !appearance_reference_srcs.is_empty() { + format!( + "{}\n严格使用参考视频的动作、节奏与镜头；严格使用参考图片的角色外观、服装和身份，不交换两种参考的职责。", + request.prompt + ) + } else { + request.prompt.clone() + }, model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(), provider_model: settings.ark.model.clone(), aspect_ratio: request.ratio.clone(), @@ -3155,7 +4179,7 @@ fn build_editor_sequence_animation_video_request( price_mud_points: request.price_mud_points, width: request.frame_width, height: request.frame_height, - reference_image_srcs: Vec::new(), + reference_image_srcs: appearance_reference_srcs.to_vec(), reference_video_srcs, reference_audio_srcs: Vec::new(), }) @@ -3532,6 +4556,11 @@ async fn extract_and_prepare_editor_character_animation_frames( error = %error, "editor_animation_frame_pipeline_failed_after_drain" ); + let uploaded_object_keys = frame_payloads + .iter() + .map(|frame| frame.object_key.clone()) + .collect::>(); + cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await; return Err(error); } @@ -3554,9 +4583,14 @@ fn build_persisted_editor_character_animation_frame_set( .iter() .map(|frame| (frame.slot.clone(), frame.prepared_asset_object.clone())) .collect(); + let uploaded_object_keys = frame_payloads + .iter() + .map(|frame| frame.object_key.clone()) + .collect(); Ok(PersistedEditorCharacterAnimationFrameSet { first_frame_object_key, prepared_asset_objects, + uploaded_object_keys, frames: frame_payloads .into_iter() .map(|frame| frame.payload) @@ -3564,6 +4598,334 @@ fn build_persisted_editor_character_animation_frame_set( }) } +async fn cleanup_generated_editor_character_animation_preview( + state: &AppState, + generated: &GeneratedAnimationPreview, + reason: &'static str, +) { + let Some(object_key) = generated.object_key.as_deref() else { + return; + }; + cleanup_uncommitted_character_animation_object(state, object_key, reason).await; +} + +async fn cleanup_uncommitted_character_animation_object( + state: &AppState, + object_key: &str, + reason: &'static str, +) { + let Ok(oss_client) = require_oss_client(state) else { + return; + }; + // 清理前必须先查正式 asset_object；首次提交成功但响应丢失后，重试失败不能把 + // 已提交结果引用的对象删掉。查询失败时采取 fail-closed，宁可留下孤儿对象，也不能 + // 破坏正式资产。 + let registered = match state + .spacetime_client() + .get_asset_object_by_location(module_assets::AssetObjectLocationInput { + bucket: oss_client.config_bucket().to_string(), + object_key: object_key.to_string(), + }) + .await + { + Ok(registered) => registered, + Err(error) => { + tracing::warn!( + object_key, + error = %error, + reason, + "character_animation_orphan_cleanup_registration_lookup_failed" + ); + return; + } + }; + if registered.is_some() { + tracing::debug!( + object_key, + reason, + "character_animation_orphan_cleanup_skipped_registered_object" + ); + return; + } + if let Err(error) = oss_client + .delete_object( + state.character_animation_oss_http_client(), + OssDeleteObjectRequest { + object_key: object_key.to_string(), + }, + ) + .await + { + tracing::warn!( + object_key, + error = %error, + reason, + "character_animation_orphan_cleanup_failed" + ); + } +} + +async fn cleanup_uncommitted_character_animation_objects(state: &AppState, object_keys: &[String]) { + for object_key in object_keys { + cleanup_uncommitted_character_animation_object( + state, + object_key.as_str(), + "direct_video_conversion_orphan_cleanup_failed", + ) + .await; + } +} + +async fn extract_and_prepare_direct_video_conversion_frames( + state: &AppState, + caller: &EditorGenerationCaller, + task_id: &str, + source_object_key: &str, + plan: crate::editor_sequence_animation::EditorCharacterAnimationVideoConversionPlan, + extraction_settings: &BackendFrameExtractionSettings, +) -> Result { + let payload = load_media_source_payload_with_client( + state, + state.character_animation_oss_http_client(), + source_object_key, + Some(EDITOR_CHARACTER_ANIMATION_REFERENCE_VIDEO_MAX_BYTES), + ) + .await?; + let temp_dir = create_animation_temp_dir()?; + let input_path = temp_dir.join(format!("source.{}", payload.extension)); + fs::write(&input_path, payload.bytes).map_err(|error| { + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "editor-character-animation", + "message": format!("写入直接转换源视频失败：{error}"), + })) + })?; + let extraction = (|| { + let target_seconds = (0..plan.frame_count) + .map(|index| { + compute_sample_time_seconds( + plan.duration_seconds, + index, + plan.frame_count, + 0.0, + 1.0, + true, + ) + }) + .collect::>(); + let output_paths = (0..plan.frame_count) + .map(|index| temp_dir.join(format!("frame-{:03}.png", index + 1))) + .collect::>(); + extract_video_frames_to_png( + &input_path, + output_paths.as_slice(), + target_seconds.as_slice(), + extraction_settings, + )?; + output_paths + .into_iter() + .map(|path| { + let bytes = fs::read(path).map_err(|error| { + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": "editor-character-animation", + "message": format!("读取直接转换帧失败：{error}"), + })) + })?; + finalize_animation_frame_payload( + bytes.as_slice(), + "image/png", + plan.frame_width, + plan.frame_height, + false, + ) + }) + .collect::, _>>() + })(); + let _ = fs::remove_dir_all(&temp_dir); + let frames = extraction?; + let mut processed = Vec::with_capacity(frames.len()); + let mut uploaded_object_keys = Vec::with_capacity(frames.len()); + for (frame_index, frame) in frames.into_iter().enumerate() { + let content_type = frame.mime_type.clone(); + let put_result = match put_character_animation_frame_object( + state, + frame_index + 1, + "conversion_frame_put", + LegacyAssetPrefix::Animations, + vec![ + "editor".to_string(), + "video-conversions".to_string(), + task_id.to_string(), + ], + format!("frame{:03}.png", frame_index + 1), + frame.mime_type, + frame.bytes, + build_asset_metadata( + EDITOR_CHARACTER_ANIMATION_ASSET_KIND, + caller.owner_user_id.as_str(), + "editor_video_conversion", + task_id, + "animation_frame", + "video-direct-conversion", + ), + ) + .await + { + Ok(result) => result, + Err(error) => { + cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await; + return Err(error); + } + }; + uploaded_object_keys.push(put_result.object_key.clone()); + let slot = editor_character_animation_frame_slot(frame_index); + let prepared_asset_object = match prepare_editor_character_animation_frame_asset_object( + state, + caller, + task_id, + slot.as_str(), + put_result.object_key.clone(), + content_type, + frame_index + 1, + ) + .await + { + Ok(result) => result, + Err(error) => { + cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await; + return Err(error); + } + }; + processed.push(ProcessedEditorCharacterAnimationFrame { + frame_index, + payload: build_editor_character_animation_frame_payload( + put_result.legacy_public_path, + put_result.object_key.clone(), + prepared_asset_object.asset_object_id.clone(), + plan.frame_width, + plan.frame_height, + ), + object_key: put_result.object_key, + prepared_asset_object, + slot, + }); + } + match build_persisted_editor_character_animation_frame_set(processed) { + Ok(result) => Ok(result), + Err(error) => { + cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await; + Err(error) + } + } +} + +/// 对已有序列帧逐帧做通用抠图（BgFilter complex，无键色），生成透明序列帧对象。 +/// 输入是源序列帧资源的稳定 objectKey 列表；任一帧失败即整批清理，不发布残缺结果。 +async fn extract_and_prepare_background_removal_frames( + state: &AppState, + caller: &EditorGenerationCaller, + task_id: &str, + source_frames: &[EditorCharacterAnimationFramePayload], + frame_width: u32, + frame_height: u32, + audit: &crate::external_api_audit::ExternalApiAuditContext, +) -> Result { + let mut processed = Vec::with_capacity(source_frames.len()); + let mut uploaded_object_keys = Vec::with_capacity(source_frames.len()); + for (frame_index, source_frame) in source_frames.iter().enumerate() { + let source_object_key = source_frame + .object_key + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()) + .ok_or_else(|| { + editor_character_animation_bad_request("序列帧缺少稳定 objectKey，无法去背景。") + })?; + let removed = request_editor_background_removal_image_with_bgfilter_worker( + state, + source_object_key, + audit, + ) + .await?; + let finalized = finalize_animation_frame_payload( + removed.image.bytes.as_slice(), + removed.image.mime_type.as_str(), + frame_width, + frame_height, + false, + )?; + let content_type = finalized.mime_type.clone(); + let put_result = match put_character_animation_frame_object( + state, + frame_index + 1, + "background_removal_frame_put", + LegacyAssetPrefix::Animations, + vec![ + "editor".to_string(), + "background-removals".to_string(), + task_id.to_string(), + ], + format!("frame{:03}.png", frame_index + 1), + finalized.mime_type, + finalized.bytes, + build_asset_metadata( + EDITOR_CHARACTER_ANIMATION_ASSET_KIND, + caller.owner_user_id.as_str(), + "editor_background_removal", + task_id, + "animation_frame", + "background-removal", + ), + ) + .await + { + Ok(result) => result, + Err(error) => { + cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await; + return Err(error); + } + }; + uploaded_object_keys.push(put_result.object_key.clone()); + let slot = editor_character_animation_frame_slot(frame_index); + let prepared_asset_object = match prepare_editor_character_animation_frame_asset_object( + state, + caller, + task_id, + slot.as_str(), + put_result.object_key.clone(), + content_type, + frame_index + 1, + ) + .await + { + Ok(result) => result, + Err(error) => { + cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await; + return Err(error); + } + }; + processed.push(ProcessedEditorCharacterAnimationFrame { + frame_index, + payload: build_editor_character_animation_frame_payload( + put_result.legacy_public_path, + put_result.object_key.clone(), + prepared_asset_object.asset_object_id.clone(), + frame_width, + frame_height, + ), + object_key: put_result.object_key, + prepared_asset_object, + slot, + }); + } + match build_persisted_editor_character_animation_frame_set(processed) { + Ok(result) => Ok(result), + Err(error) => { + cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await; + Err(error) + } + } +} + async fn process_and_persist_editor_character_animation_frame( state: &AppState, caller: &EditorGenerationCaller, @@ -3576,7 +4938,8 @@ async fn process_and_persist_editor_character_animation_frame( screen_color: EditorScreenBackgroundColor, audit: &crate::external_api_audit::ExternalApiAuditContext, ) -> Result { - // 中文注释：每一帧只要求自己的绿幕源图先落 OSS，不再等待整批源图全部上传完成。 + // 每一帧的绿幕源图仅作为 BgFilter 输入，不是正式产物；无论该帧后续哪一步失败， + // 都必须清理它。最终 PNG 只有在整批原子登记成功后才成为正式资产对象。 let source_put = put_character_animation_frame_object( state, frame_index + 1, @@ -3604,10 +4967,9 @@ async fn process_and_persist_editor_character_animation_frame( ), ) .await?; + let source_object_key = source_put.object_key.clone(); - // 排队与调用预算由共享 helper 按 N/est 公式和父剩余预算派生，动画帧不再携带 - // 任何按帧数放大的 timeout。 - let removed = remove_editor_generated_screen_background_with_bgfilter( + let removed = match remove_editor_generated_screen_background_with_bgfilter( state, source_put.object_key.as_str(), screen_color, @@ -3615,21 +4977,44 @@ async fn process_and_persist_editor_character_animation_frame( EDITOR_BGFILTER_CROSS_CHECK_ENABLED, audit, ) - .await?; + .await + { + Ok(removed) => removed, + Err(error) => { + cleanup_uncommitted_character_animation_object( + state, + source_object_key.as_str(), + "editor_animation_source_cleanup_failed", + ) + .await; + return Err(error); + } + }; tracing::debug!( frame_index, screen_color = screen_color.hex, "editor_animation_frame_background_removed" ); - let finalized = finalize_animation_frame_payload( + let finalized = match finalize_animation_frame_payload( removed.image.bytes.as_slice(), removed.image.mime_type.as_str(), frame_width, frame_height, false, - )?; + ) { + Ok(finalized) => finalized, + Err(error) => { + cleanup_uncommitted_character_animation_object( + state, + source_object_key.as_str(), + "editor_animation_source_cleanup_failed", + ) + .await; + return Err(error); + } + }; let content_type = finalized.mime_type.clone(); - let put_result = put_character_animation_frame_object( + let put_result = match put_character_animation_frame_object( state, frame_index + 1, "final_put", @@ -3651,9 +5036,21 @@ async fn process_and_persist_editor_character_animation_frame( "editor-character-animation", ), ) - .await?; + .await + { + Ok(result) => result, + Err(error) => { + cleanup_uncommitted_character_animation_object( + state, + source_object_key.as_str(), + "editor_animation_source_cleanup_failed", + ) + .await; + return Err(error); + } + }; let slot = editor_character_animation_frame_slot(frame_index); - let prepared_asset_object = prepare_editor_character_animation_frame_asset_object( + let prepared_asset_object = match prepare_editor_character_animation_frame_asset_object( state, caller, source_layer_id, @@ -3662,7 +5059,24 @@ async fn process_and_persist_editor_character_animation_frame( content_type, frame_index + 1, ) - .await?; + .await + { + Ok(result) => result, + Err(error) => { + cleanup_uncommitted_character_animation_objects( + state, + &[source_object_key.clone(), put_result.object_key.clone()], + ) + .await; + return Err(error); + } + }; + cleanup_uncommitted_character_animation_object( + state, + source_object_key.as_str(), + "editor_animation_source_cleanup_failed", + ) + .await; Ok(ProcessedEditorCharacterAnimationFrame { frame_index, @@ -4189,6 +5603,7 @@ fn editor_inline_generation_caller( "message": format!("序列化编辑器生成幂等请求失败：{error}"), })) })?; + let operation_id = validate_editor_generation_operation_id(request_context.request_id())?; Ok(EditorGenerationCaller { audit_subject_user_id: Some(owner_user_id.clone()), audit_project_id: None, @@ -4196,7 +5611,7 @@ fn editor_inline_generation_caller( phase_reporter: None, operation: Some(EditorGenerationOperationContext { operation_kind: operation_kind.to_string(), - operation_id: request_context.request_id().trim().to_string(), + operation_id, operation_fingerprint: editor_generation_request_fingerprint( operation_kind, request_payload_json.as_str(), @@ -4448,6 +5863,132 @@ fn prepare_editor_character_animation_result( }) } +#[allow(clippy::too_many_arguments)] +fn prepare_editor_character_animation_sequence_result( + caller: &EditorGenerationCaller, + project_id: Option<&str>, + asset_folder_id: Option<&str>, + asset_label: &str, + provider: &str, + source_resource_id: Option, + generation_inputs: Option, + plan: crate::editor_sequence_animation::EditorCharacterAnimationVideoConversionPlan, + frames: &PersistedEditorCharacterAnimationFrameSet, + frame_payload_json: Value, + completed_at_micros: i64, +) -> Result { + let operation = editor_generation_operation(caller)?; + let first_frame = frames.frames.first().ok_or_else(|| { + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": "editor-character-animation", + "message": "序列帧生成没有产生可保存的帧。", + })) + })?; + let final_object = frames + .prepared_asset_objects + .iter() + .find(|(slot, _)| slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .map(|(_, object)| object.clone()) + .ok_or_else(|| editor_character_animation_bad_request("序列帧生成缺少首帧对象。"))?; + let final_resource_id = editor_generation_stable_resource_id( + caller.owner_user_id.as_str(), + operation.operation_kind.as_str(), + operation.operation_id.as_str(), + EDITOR_CHARACTER_ANIMATION_FINAL_SLOT, + ); + let generation_inputs_json = serialize_editor_generation_inputs( + Some(EDITOR_CHARACTER_ANIMATION_RESOURCE_ASSET_KIND), + sanitize_editor_client_generation_inputs(generation_inputs), + )?; + let image_sequence_frames_json = + serialize_editor_image_sequence_frames(Some(frame_payload_json))?; + let project_id = project_id + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(str::to_string); + let final_resource = + project_id + .as_ref() + .map(|project_id| EditorProjectResourceCreateRecordInput { + resource_id: final_resource_id.clone(), + project_id: project_id.clone(), + owner_user_id: caller.owner_user_id.clone(), + asset_object_id: Some(final_object.asset_object_id.clone()), + image_src: first_frame.image_src.clone(), + object_key: Some(frames.first_frame_object_key.clone()), + width: first_frame.width, + height: first_frame.height, + source_type: "generated".to_string(), + prompt: None, + actual_prompt: None, + model: None, + provider: Some(provider.to_string()), + task_id: Some(operation.operation_id.clone()), + source_resource_id: source_resource_id.clone(), + asset_kind: Some(EDITOR_CHARACTER_ANIMATION_RESOURCE_ASSET_KIND.to_string()), + generation_inputs_json: generation_inputs_json.clone(), + updated_at_micros: completed_at_micros, + image_sequence_frames_json: image_sequence_frames_json.clone(), + image_sequence_duration_ms: Some(plan.duration_ms), + }); + let final_asset = asset_folder_id + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(|folder_id| EditorAssetCreateRecordInput { + asset_id: editor_generation_stable_asset_id( + caller.owner_user_id.as_str(), + operation.operation_kind.as_str(), + operation.operation_id.as_str(), + EDITOR_CHARACTER_ANIMATION_FINAL_SLOT, + ), + owner_user_id: caller.owner_user_id.clone(), + folder_id: folder_id.to_string(), + label: asset_label.to_string(), + asset_object_id: Some(final_object.asset_object_id.clone()), + image_src: first_frame.image_src.clone(), + object_key: Some(frames.first_frame_object_key.clone()), + width: first_frame.width, + height: first_frame.height, + source_type: "generated".to_string(), + prompt: None, + actual_prompt: None, + model: None, + provider: Some(provider.to_string()), + task_id: Some(operation.operation_id.clone()), + asset_kind: Some(EDITOR_CHARACTER_ANIMATION_RESOURCE_ASSET_KIND.to_string()), + generation_inputs_json, + source_resource_id: final_resource + .as_ref() + .map(|resource| resource.resource_id.clone()), + now_micros: completed_at_micros, + thumbnail_src: Some(first_frame.image_src.clone()), + generation_cost_mud_points: 0, + group_task_id: None, + group_task_expected_asset_count: None, + image_sequence_frames_json, + image_sequence_duration_ms: Some(plan.duration_ms), + }); + let items = frames + .prepared_asset_objects + .iter() + .map(|(slot, object)| PreparedEditorGenerationResultItem { + slot: slot.clone(), + asset_object: Some(object.clone()), + project_resource: (slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .then(|| final_resource.clone()) + .flatten(), + asset: (slot == EDITOR_CHARACTER_ANIMATION_FINAL_SLOT) + .then(|| final_asset.clone()) + .flatten(), + binding: None, + }) + .collect(); + Ok(PreparedEditorCharacterAnimationResult { + items, + final_resource_id, + }) +} + #[allow(clippy::too_many_arguments)] fn prepare_editor_video_result( caller: &EditorGenerationCaller, @@ -6206,7 +7747,11 @@ async fn load_media_source_payload_with_client( validate_media_payload_download_size(next_len, max_bytes)?; bytes.extend_from_slice(chunk.as_ref()); } - if head.content_length != u64::try_from(bytes.len()).unwrap_or(u64::MAX) { + // 有界下载（视频参考）才做严格的 HEAD/GET 长度一致性校验：视频路径依赖该一致性 + // 拒绝截断或异常流。旧图片路径（max_bytes=None）不要求 HEAD 与 chunked GET 完全一致， + // 避免 OSS 编码 / 缓存差异导致原本可用的图片被误判失败。 + if max_bytes.is_some() && head.content_length != u64::try_from(bytes.len()).unwrap_or(u64::MAX) + { return Err( AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ "provider": "aliyun-oss", @@ -6254,6 +7799,25 @@ fn media_payload_as_data_url(payload: &MediaPayload) -> String { ) } +fn encode_rgba_image_as_png_data_url(rgba: &RgbaImage) -> Option { + let mut bytes = Vec::new(); + if PngEncoder::new(&mut bytes) + .write_image( + rgba.as_raw(), + rgba.width(), + rgba.height(), + ColorType::Rgba8.into(), + ) + .is_err() + { + return None; + } + Some(format!( + "data:image/png;base64,{}", + encode_base64(bytes.as_slice()) + )) +} + fn parse_media_data_url(value: &str) -> Option { let body = value.trim().strip_prefix("data:")?; let (mime_type, data) = body.split_once(";base64,")?; @@ -7479,6 +9043,7 @@ struct PersistedEditorCharacterAnimationFrameSet { frames: Vec, first_frame_object_key: String, prepared_asset_objects: Vec<(String, AssetObjectUpsertInput)>, + uploaded_object_keys: Vec, } struct ProcessedEditorCharacterAnimationFrame { @@ -7513,6 +9078,7 @@ struct EditorCharacterAnimationReferenceVideoMetadata { duration_seconds: f64, width: u32, height: u32, + source_fps: f64, } // 发布整套动作后，当前阶段只需要把旧前端依赖的 animationMap 返回出去。 @@ -7540,6 +9106,7 @@ mod tests { input_mode: None, reference_media_type: None, reference_src: None, + appearance_reference_srcs: Vec::new(), prompt_text: "待机呼吸".to_string(), background_color: None, screen_color: None, @@ -7552,6 +9119,7 @@ mod tests { canvas_completion: None, generation_inputs: None, source_resource_id: None, + appearance_source_resource_ids: Vec::new(), asset_folder_id: None, asset_label: None, } @@ -7812,6 +9380,7 @@ mod tests { input_mode: None, reference_media_type: None, reference_src: None, + appearance_reference_srcs: Vec::new(), prompt_text: "待机呼吸".to_string(), background_color: None, screen_color: None, @@ -7824,6 +9393,7 @@ mod tests { canvas_completion: None, generation_inputs: None, source_resource_id: None, + appearance_source_resource_ids: Vec::new(), asset_folder_id: None, asset_label: None, }) @@ -7844,6 +9414,7 @@ mod tests { input_mode: Some("video".to_string()), reference_media_type: Some("video".to_string()), reference_src: Some("users/owner/reference/action.mp4".to_string()), + appearance_reference_srcs: Vec::new(), prompt_text: "复刻参考视频的奔跑节奏".to_string(), background_color: Some("blue".to_string()), ratio: "16:9".to_string(), @@ -7864,6 +9435,7 @@ mod tests { &settings, &normalized, Some("users/owner/reference/action.mp4"), + &[], ) .expect("video request should build"); @@ -7883,6 +9455,36 @@ mod tests { ); assert!(video_request.reference_image_srcs.is_empty()); assert!(video_request.reference_audio_srcs.is_empty()); + + let video_request_with_appearance = build_editor_sequence_animation_video_request( + &settings, + &normalized, + Some("users/owner/reference/action.mp4"), + &["generated-character-drafts/editor/seedance-references/image/appearance.png" + .to_string()], + ) + .expect("video request with appearance reference should build"); + assert_eq!( + video_request_with_appearance.reference_video_srcs, + vec!["users/owner/reference/action.mp4".to_string()] + ); + assert_eq!( + video_request_with_appearance.reference_image_srcs, + vec![ + "generated-character-drafts/editor/seedance-references/image/appearance.png" + .to_string() + ] + ); + assert!( + video_request_with_appearance + .reference_image_srcs + .iter() + .all(|source| !source.starts_with("data:")) + ); + + let prompt = video_request_with_appearance.prompt; + assert!(prompt.contains("参考视频的动作")); + assert!(prompt.contains("参考图片的角色外观")); } #[test] @@ -8064,7 +9666,7 @@ mod tests { &[ "execute_billable_asset_operation_with_cost", "request_editor_character_animation_preview", - "reporter.report_processing(&state).await?", + "reporter.report_processing(&state).await", "extract_and_prepare_editor_character_animation_frames", "Ok::<_, AppError>((", "prepare_editor_character_animation_result", @@ -8214,6 +9816,46 @@ mod tests { } } + #[test] + fn editor_character_animation_direct_video_conversion_bypasses_seedance_and_background_removal() + { + let source = include_str!("character_animation_assets.rs"); + assert_function_contains_in_order( + source, + "pub(crate) async fn convert_editor_character_animation_video_for_owner", + "pub async fn generate_editor_video", + &[ + "validate_editor_character_animation_reference_video_object", + "build_editor_character_animation_video_conversion_plan", + "extract_and_prepare_direct_video_conversion_frames", + "prepare_editor_character_animation_sequence_result", + "persist_editor_generation_result_atomically", + ], + ); + let body = source + .split_once("pub(crate) async fn convert_editor_character_animation_video_for_owner") + .and_then(|(_, tail)| { + tail.split_once("pub async fn generate_editor_video") + .map(|(body, _)| body) + }) + .expect("direct conversion function should exist"); + for forbidden in [ + "create_ark_content_generation_task", + "request_seedance", + "remove_editor_generated_screen_background_with_bgfilter", + "execute_billable_asset_operation_with_cost", + ] { + assert!( + !body.contains(forbidden), + "direct conversion must not call {forbidden}" + ); + } + assert!(body.contains("price_mud_points: 0")); + assert!(body.contains("let storage_task_id = format!")); + assert!(body.contains("storage_task_id.as_str(),")); + assert!(body.contains("cleanup_uncommitted_character_animation_objects")); + } + #[test] fn editor_character_animation_frames_use_three_stage_matting_fallback() { let source = include_str!("character_animation_assets.rs"); @@ -8292,8 +9934,8 @@ mod tests { "pub(crate) async fn generate_editor_character_animation_for_owner", "pub async fn generate_editor_video", &[ - "let persisted_frames = extract_and_prepare_editor_character_animation_frames", - "let frame_payload_json = serde_json::to_value(&frames)", + "extract_and_prepare_editor_character_animation_frames", + "serde_json::to_value(&frames)", "prepare_editor_character_animation_result", "persist_editor_generation_result_atomically", "EDITOR_CHARACTER_ANIMATION_FINAL_SLOT", @@ -8593,7 +10235,7 @@ mod tests { fn editor_character_animation_parses_authoritative_video_probe_metadata() { let metadata = parse_editor_character_animation_reference_video_probe( br#"{ - "streams": [{"width": 1080, "height": 1920}], + "streams": [{"width": 1080, "height": 1920, "avg_frame_rate": "30000/1000"}], "format": {"duration": "6.250000"} }"#, ) @@ -8605,6 +10247,7 @@ mod tests { duration_seconds: 6.25, width: 1080, height: 1920, + source_fps: 30.0, } ); diff --git a/server-rs/crates/api-server/src/editor_agent/tool.rs b/server-rs/crates/api-server/src/editor_agent/tool.rs index f94f1678b..cef77e705 100644 --- a/server-rs/crates/api-server/src/editor_agent/tool.rs +++ b/server-rs/crates/api-server/src/editor_agent/tool.rs @@ -371,7 +371,6 @@ impl EditorAgentTool for GenerateImageTool { seg_model: None, aspect_ratio: Some(args.aspect_ratio), image_size: Some(args.image_size), - character_sheet_views: None, reference_image_srcs: Some(reference_image_srcs), project_id: Some(context.conversation.project_id.clone()), asset_kind: Some("editor_agent_generated_image".to_string()), @@ -480,7 +479,6 @@ impl EditorAgentTool for GenerateCharacterTool { seg_model: Some("birefnet".to_string()), aspect_ratio: Some(args.aspect_ratio), image_size: Some(args.image_size), - character_sheet_views: None, reference_image_srcs: Some(reference_image_srcs), project_id: Some(context.conversation.project_id.clone()), asset_kind: Some("character".to_string()), @@ -591,7 +589,6 @@ impl EditorAgentTool for GenerateUiDesignTool { seg_model: None, aspect_ratio: Some(args.aspect_ratio), image_size: Some(args.image_size), - character_sheet_views: None, reference_image_srcs: Some(reference_image_srcs), project_id: Some(context.conversation.project_id.clone()), asset_kind: Some("ui-design".to_string()), diff --git a/server-rs/crates/api-server/src/editor_generation_queue.rs b/server-rs/crates/api-server/src/editor_generation_queue.rs index 3d0388147..06d2740c0 100644 --- a/server-rs/crates/api-server/src/editor_generation_queue.rs +++ b/server-rs/crates/api-server/src/editor_generation_queue.rs @@ -20,6 +20,10 @@ pub(crate) const EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND: &str = "editor_ui_design_asset_extraction"; pub(crate) const EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND: &str = "editor_character_animation_generation"; +pub(crate) const EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_JOB_KIND: &str = + "editor_character_animation_video_conversion"; +pub(crate) const EDITOR_CHARACTER_ANIMATION_BACKGROUND_REMOVAL_JOB_KIND: &str = + "editor_character_animation_background_removal"; pub(crate) const EDITOR_VIDEO_GENERATION_JOB_KIND: &str = "editor_video_generation"; pub(crate) const EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND: &str = "editor_sound_effect_generation"; pub(crate) const EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND: &str = diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index 5828ec721..1a2fd889a 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -350,9 +350,6 @@ pub struct EditorImageGenerationRequest { pub(crate) seg_model: Option, pub(crate) aspect_ratio: Option, pub(crate) image_size: Option, - /// 角色多方向图集的视角数：4 或 8；仅 kind=character-sheet 时生效，缺省 4。 - #[serde(default, skip_serializing_if = "Option::is_none")] - pub(crate) character_sheet_views: Option, pub(crate) reference_image_srcs: Option>, pub(crate) project_id: Option, pub(crate) asset_kind: Option, @@ -491,6 +488,46 @@ pub struct EditorUiDesignAssetExtractionRequest { pub(crate) canvas_completion: Option, } +const EDITOR_GENERATION_OPERATION_ID_MAX_BYTES: usize = 128; + +/// 将请求级 request id 收敛为可持久化的编辑器 operation id。 +/// +/// request id 会进入 SpacetimeDB 幂等键、资产对象标识和画布层标识，不能把任意 +/// header 内容直接带入这些边界。这里只允许客户端和队列实际使用的稳定 ID 字符集， +/// 超限或包含路径/控制字符时拒绝请求，而不是静默改写导致幂等键碰撞。 +pub(crate) fn validate_editor_generation_operation_id( + operation_id: &str, +) -> Result { + let operation_id = operation_id.trim(); + if operation_id.is_empty() { + return Err( + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "editor-generation-operation", + "message": "编辑器生成 request id 不能为空。", + })), + ); + } + if operation_id.len() > EDITOR_GENERATION_OPERATION_ID_MAX_BYTES { + return Err(AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "editor-generation-operation", + "message": format!( + "编辑器生成 request id 不能超过 {EDITOR_GENERATION_OPERATION_ID_MAX_BYTES} 个字节。" + ), + }))); + } + if !operation_id.chars().all(|character| { + character.is_ascii_alphanumeric() || matches!(character, '-' | '_' | '.' | ':') + }) { + return Err( + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "editor-generation-operation", + "message": "编辑器生成 request id 含有不支持的字符。", + })), + ); + } + Ok(operation_id.to_string()) +} + #[derive(Clone, Debug)] pub(crate) struct EditorGenerationOperationContext { pub(crate) operation_kind: String, @@ -588,9 +625,10 @@ impl EditorGenerationCaller { "message": format!("序列化编辑器生成幂等请求失败：{error}"), })) })?; + let operation_id = validate_editor_generation_operation_id(request_context.request_id())?; self.operation = Some(EditorGenerationOperationContext { operation_kind: operation_kind.to_string(), - operation_id: request_context.request_id().trim().to_string(), + operation_id, operation_fingerprint: editor_generation_request_fingerprint( operation_kind, request_payload_json.as_str(), @@ -2521,7 +2559,6 @@ pub async fn generate_editor_scene( seg_model: None, aspect_ratio: Some(generation_options.aspect_ratio.to_string()), image_size: Some(generation_options.image_size.to_string()), - character_sheet_views: None, reference_image_srcs: (!payload.reference_image_srcs.is_empty()) .then_some(payload.reference_image_srcs), project_id: payload.project_id, @@ -2821,16 +2858,6 @@ where let normalized_kind = payload.kind.as_deref().map(str::trim); let is_character_generation = matches!(normalized_kind, Some("character")); let is_character_sheet_generation = matches!(normalized_kind, Some("character-sheet")); - if is_character_sheet_generation - && !matches!(payload.character_sheet_views.unwrap_or(4), 4 | 8) - { - return Err( - AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ - "provider": "editor-image-generation", - "message": "characterSheetViews 只支持 4 或 8。", - })), - ); - } let pixel_art_supported = matches!(normalized_kind, None | Some("") | Some("character")); let (image_style, mut generation_warning) = normalize_editor_image_generation_style(payload.style.as_deref(), pixel_art_supported); @@ -3039,7 +3066,7 @@ where screen_color.expect("character generation should have screen color"), ) } else if is_character_sheet_generation { - // 多方向图集以纯绿底间隔各方向，供连通块拆分；不参与编辑器的背景自动选色。 + // 多方向图集以纯绿底间隔各方向，便于后续拆分；不参与编辑器的背景自动选色。 build_editor_character_multi_view_sheet_prompt( role_setting.as_str(), crate::editor_green_screen::EditorScreenBackgroundColor { @@ -3049,7 +3076,6 @@ where green: 255, blue: 0, }, - payload.character_sheet_views.unwrap_or(4), ) } else if is_ui_design_generation { build_editor_ui_design_prompt(role_setting.as_str(), ui_has_references) @@ -5919,10 +5945,10 @@ pub(crate) async fn remove_editor_image_background_for_owner( )) } -struct EditorBackgroundRemovalImage { - image: DownloadedOpenAiImage, - width: u32, - height: u32, +pub(crate) struct EditorBackgroundRemovalImage { + pub(crate) image: DownloadedOpenAiImage, + pub(crate) width: u32, + pub(crate) height: u32, } fn validate_editor_pixel_art_snap_canvas_completion( @@ -7022,7 +7048,7 @@ fn editor_bgfilter_flat_deadline_reserve(aliyun_timeout_ms: u64) -> Duration { .saturating_add(EDITOR_BGFILTER_PARENT_TRANSPORT_WINDOW) } -async fn request_editor_background_removal_image_with_bgfilter_worker( +pub(crate) async fn request_editor_background_removal_image_with_bgfilter_worker( state: &AppState, source_object_key: &str, audit: &crate::external_api_audit::ExternalApiAuditContext, @@ -7833,7 +7859,6 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner( ), path_kind: "ui-design-assets", asset_kind: EDITOR_UI_DESIGN_ASSET_IMAGE_KIND, - record_asset_kind: "icon", persistence_provider: "vector-engine", }, ) @@ -10894,17 +10919,16 @@ fn build_editor_character_image_prompt( fn build_editor_character_multi_view_sheet_prompt( role_setting: &str, screen_color: EditorScreenBackgroundColor, - view_count: u32, ) -> String { - let (view_count_text, view_layout) = match view_count { - 8 => ("八", "正面、右前、右侧、右后、背面、左后、左侧、左前"), - _ => ("四", "正面、背面、左侧、右侧"), - }; + // 八方向围绕一圈：正面、右前、右侧、右后、背面、左后、左侧、左前，按 3×3 九宫格外圈 + // 排布、中心留空，方向之间由纯色背景充分间隔，使后续外接矩形拆分不易粘连。 + const VIEW_LAYOUT: &str = "正面、右前、右侧、右后、背面、左后、左侧、左前"; vec![ format!( - "根据角色设定生成游戏角色的多方向视图图集。画面中只有同一个角色，按{view_count_text}个方向排布：{view_layout}，每个方向一个完整角色，方向之间相互分离、互不重叠、互不接触。\ -{view_count_text}个方向必须是同一个角色，脸型、发型、服装、配色、武器、头身比、体型全程一致，禁止漂移、换装、变形。\ -每个方向角色完整居中，不裁切头顶、手脚或武器；各方向底部基线对齐，画面占比一致。{}；角色之间由纯色背景间隔，便于后续拆分。\ + "根据角色设定生成游戏角色的八方向视图图集。画面中只有同一个角色，共八个方向：{VIEW_LAYOUT}，每个方向一个完整角色。\ +八个方向必须是同一个角色，脸型、发型、服装、配色、武器、头身比、体型全程一致，禁止漂移、换装、变形。\ +八个方向按三乘三九宫格围绕一圈排布：上、下、左、右以及四个角落各放一个方向，九宫格中心留空，不放置角色。\ +每个方向角色完整居中，不裁切头顶、手脚或武器；各方向底部基线对齐，画面占比一致。{}；角色之间由纯色背景充分间隔，互不接触、互不重叠，便于后续拆分。\ 禁止额外角色、分身、宠物、文字、水印、边框、编号、网格线。", editor_green_screen_character_prompt_clause(screen_color) ), @@ -12351,6 +12375,25 @@ mod tests { thread, }; + #[test] + fn editor_generation_operation_id_accepts_stable_request_id_characters() { + assert_eq!( + validate_editor_generation_operation_id(" web-abc_123.v1:retry ").unwrap(), + "web-abc_123.v1:retry" + ); + } + + #[test] + fn editor_generation_operation_id_rejects_unsafe_or_oversized_request_ids() { + for request_id in ["", " ", "request/id", "request id", "request\n id"] { + assert!( + validate_editor_generation_operation_id(request_id).is_err(), + "request id should be rejected: {request_id:?}" + ); + } + assert!(validate_editor_generation_operation_id(&"x".repeat(129)).is_err()); + } + #[test] fn ordinary_image_prompt_validation_stays_outside_prompt_builder_execution() { assert!(validate_editor_image_generation_prompt("生成森林场景").is_ok()); @@ -16618,38 +16661,20 @@ mod tests { green: 255, blue: 0, }, - 4, ); - assert!(prompt.contains("多方向视图图集")); - assert!(prompt.contains("四个方向排布：正面、背面、左侧、右侧")); - assert!(prompt.contains("方向之间相互分离、互不重叠、互不接触")); + assert!(prompt.contains("八方向视图图集")); + assert!(prompt.contains("共八个方向：正面、右前、右侧、右后、背面、左后、左侧、左前")); + assert!(prompt.contains("八个方向必须是同一个角色")); + assert!(prompt.contains("按三乘三九宫格围绕一圈排布")); + assert!(prompt.contains("九宫格中心留空")); assert!(prompt.contains("脸型、发型、服装、配色、武器、头身比、体型全程一致")); assert!(prompt.contains("背景固定为单一纯色背景 纯绿色 #00FF00")); - assert!(prompt.contains("角色之间由纯色背景间隔，便于后续拆分")); + assert!(prompt.contains("角色之间由纯色背景充分间隔，互不接触、互不重叠")); assert!(prompt.contains("禁止额外角色、分身、宠物、文字、水印、边框、编号、网格线")); assert!(prompt.contains("角色设定：日系魔法少女")); } - #[test] - fn editor_character_multi_view_sheet_prompt_supports_eight_directions() { - let prompt = build_editor_character_multi_view_sheet_prompt( - "日系魔法少女", - crate::editor_green_screen::EditorScreenBackgroundColor { - label: "纯绿色", - hex: "#00FF00", - red: 0, - green: 255, - blue: 0, - }, - 8, - ); - - assert!(prompt.contains("八个方向排布：正面、右前、右侧、右后、背面、左后、左侧、左前")); - assert!(prompt.contains("八个方向必须是同一个角色")); - assert!(prompt.contains("每个方向角色完整居中")); - } - #[test] fn editor_character_sheet_kind_routes_to_sheet_prompt_and_icon_spritesheet_asset_kind() { assert_eq!( diff --git a/server-rs/crates/api-server/src/editor_project_icon.rs b/server-rs/crates/api-server/src/editor_project_icon.rs index 5126d4b39..78aa5dd26 100644 --- a/server-rs/crates/api-server/src/editor_project_icon.rs +++ b/server-rs/crates/api-server/src/editor_project_icon.rs @@ -638,7 +638,6 @@ fn build_icon_spec_image_request( seg_model: None, aspect_ratio: Some(ICON_SPEC_ASPECT_RATIO.to_string()), image_size: Some(ICON_SPEC_IMAGE_SIZE.to_string()), - character_sheet_views: None, reference_image_srcs: payload.reference_id.map(|reference_id| vec![reference_id]), project_id: payload.project_id, asset_kind: Some("icon-spec".to_string()), @@ -841,9 +840,6 @@ pub(crate) struct PersistEditorSpritesheetSlicesInput { pub(crate) generation_inputs: Option, pub(crate) path_kind: &'static str, pub(crate) asset_kind: &'static str, - /// 切片资源/素材记录的 asset_kind（前端据此判断素材语义）。图标图集默认 "icon"； - /// 角色多方向图集拆分传 "character"，使单视角切片可直接作为角色动画参考。 - pub(crate) record_asset_kind: &'static str, pub(crate) persistence_provider: &'static str, } @@ -964,7 +960,7 @@ pub(crate) async fn persist_editor_spritesheet_slices( provider: Some(input.provider.clone()), task_id: Some(input.task_id.clone()), source_resource_id: normalize_optional_string(input.source_resource_id.clone()), - asset_kind: Some(input.record_asset_kind.to_string()), + asset_kind: Some("icon".to_string()), generation_inputs_json: generation_inputs_json.clone(), updated_at_micros: now_micros, image_sequence_frames_json: None, @@ -994,7 +990,7 @@ pub(crate) async fn persist_editor_spritesheet_slices( model: Some(input.model.clone()), provider: Some(input.provider.clone()), task_id: Some(input.task_id.clone()), - asset_kind: Some(input.record_asset_kind.to_string()), + asset_kind: Some("icon".to_string()), generation_inputs_json: generation_inputs_json.clone(), source_resource_id: resource_id.clone(), generation_cost_mud_points: 0, @@ -1967,7 +1963,6 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( ), path_kind: "icon-spritesheet-assets", asset_kind: EDITOR_ICON_SPRITESHEET_SLICE_ASSET_KIND, - record_asset_kind: "icon", persistence_provider: "vector-engine", }, ) @@ -2175,17 +2170,6 @@ pub(crate) fn resolve_editor_icon_spritesheet_source_layer_asset_kind<'a>( } } -/// 图集切片记录 asset_kind：角色多方向图集（action=character-sheet.generate）拆出的 -/// 单视角切片按 character 落库（可直接作动画参考），其余图集按 icon。 -fn editor_spritesheet_slice_record_asset_kind(source_generation_inputs: Option<&Value>) -> &'static str { - source_generation_inputs - .and_then(|inputs| inputs.get("action")) - .and_then(Value::as_str) - .is_some_and(|action| action == "character-sheet.generate") - .then_some("character") - .unwrap_or("icon") -} - pub async fn split_editor_icon_spritesheet( State(state): State, Extension(request_context): Extension, @@ -2325,10 +2309,6 @@ pub async fn split_editor_icon_spritesheet( .clone() .unwrap_or_else(|| "Genarrative".to_string()); let generation_inputs = source_resource.generation_inputs.clone(); - // 中文注释：角色多方向图集（action=character-sheet.generate）拆分出的单视角切片按 - // character 落库，使其可直接作为角色动画参考；普通图标图集仍按 icon。 - let slice_record_asset_kind = - editor_spritesheet_slice_record_asset_kind(source_resource.generation_inputs.as_ref()); let icon_image_srcs = persist_editor_spritesheet_slices( &state, slices, @@ -2348,7 +2328,6 @@ pub async fn split_editor_icon_spritesheet( generation_inputs, path_kind: "icon-spritesheet-slices", asset_kind: EDITOR_ICON_SPRITESHEET_SLICE_ASSET_KIND, - record_asset_kind: slice_record_asset_kind, persistence_provider: "genarrative", }, ) @@ -2698,32 +2677,6 @@ pub(crate) fn validate_editor_icon_spritesheet_dimensions( mod tests { use super::*; - #[test] - fn sprite_slice_record_asset_kind_marks_character_sheet_slices_as_character() { - assert_eq!( - editor_spritesheet_slice_record_asset_kind(None), - "icon" - ); - assert_eq!( - editor_spritesheet_slice_record_asset_kind(Some(&json!({ - "action": "icon.generate" - }))), - "icon" - ); - assert_eq!( - editor_spritesheet_slice_record_asset_kind(Some(&json!({ - "action": "character-sheet.generate" - }))), - "character" - ); - assert_eq!( - editor_spritesheet_slice_record_asset_kind(Some(&json!({ - "version": 2 - }))), - "icon" - ); - } - #[test] fn game_genre_serde_uses_chinese_slugs() { for genre in GameGenre::ALL { diff --git a/server-rs/crates/api-server/src/editor_sequence_animation.rs b/server-rs/crates/api-server/src/editor_sequence_animation.rs index ab72ddd32..dff23869a 100644 --- a/server-rs/crates/api-server/src/editor_sequence_animation.rs +++ b/server-rs/crates/api-server/src/editor_sequence_animation.rs @@ -2,6 +2,7 @@ use axum::http::StatusCode; use serde_json::{Value, json}; use shared_contracts::assets::{ EDITOR_ASSET_LABEL_MAX_CHARS, EditorCharacterAnimationGenerateRequest, + EditorCharacterAnimationVideoConversionRequest, }; use crate::{ @@ -16,6 +17,131 @@ pub(crate) const EDITOR_CHARACTER_ANIMATION_MODEL: &str = "seedance2.0-fast"; pub(crate) const EDITOR_CHARACTER_ANIMATION_REFERENCE_VIDEO_MAX_BYTES: u64 = 50 * 1024 * 1024; const EDITOR_CHARACTER_ANIMATION_REFERENCE_VIDEO_MIN_DURATION_SECONDS: f64 = 2.0; const EDITOR_CHARACTER_ANIMATION_REFERENCE_VIDEO_MAX_DURATION_SECONDS: f64 = 15.0; +pub(crate) const EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_MAX_FRAMES: u32 = 120; + +#[derive(Clone, Copy, Debug, PartialEq)] +pub(crate) struct EditorCharacterAnimationVideoConversionPlan { + pub(crate) duration_seconds: f64, + pub(crate) duration_ms: u64, + pub(crate) source_fps: f64, + pub(crate) sampling_fps: f64, + pub(crate) frame_count: u32, + pub(crate) frame_width: u32, + pub(crate) frame_height: u32, +} + +pub(crate) fn normalize_editor_character_animation_video_conversion_resolution( + value: &str, +) -> Result<&'static str, AppError> { + match value.trim() { + "480p" => Ok("480p"), + "720p" => Ok("720p"), + _ => Err(editor_character_animation_bad_request( + "resolution 只支持 480p 或 720p。", + )), + } +} + +pub(crate) fn build_editor_character_animation_video_conversion_plan( + duration_seconds: f64, + source_fps: f64, + source_width: u32, + source_height: u32, + resolution: &str, +) -> Result { + validate_editor_character_animation_reference_video_duration(duration_seconds)?; + if !source_fps.is_finite() || source_fps <= 0.0 { + return Err(editor_character_animation_bad_request( + "源视频缺少有效平均帧率。", + )); + } + if source_width == 0 || source_height == 0 { + return Err(editor_character_animation_bad_request( + "源视频缺少有效显示尺寸。", + )); + } + let max_long_edge = + match normalize_editor_character_animation_video_conversion_resolution(resolution)? { + "480p" => 480_u32, + "720p" => 720_u32, + _ => unreachable!(), + }; + let source_long_edge = source_width.max(source_height); + let scale = (f64::from(max_long_edge) / f64::from(source_long_edge)).min(1.0); + let frame_width = (f64::from(source_width) * scale).round().max(1.0) as u32; + let frame_height = (f64::from(source_height) * scale).round().max(1.0) as u32; + let sampling_fps = source_fps.min(8.0).max(1.0); + let frame_count = (duration_seconds * sampling_fps).round().clamp( + 2.0, + f64::from(EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_MAX_FRAMES), + ) as u32; + Ok(EditorCharacterAnimationVideoConversionPlan { + duration_seconds, + duration_ms: (duration_seconds * 1_000.0).round().max(1.0) as u64, + source_fps, + sampling_fps, + frame_count, + frame_width, + frame_height, + }) +} + +pub(crate) fn parse_editor_character_animation_ffprobe_frame_rate(value: &str) -> Option { + let value = value.trim(); + if let Some((numerator, denominator)) = value.split_once('/') { + let numerator = numerator.trim().parse::().ok()?; + let denominator = denominator.trim().parse::().ok()?; + return (numerator.is_finite() + && denominator.is_finite() + && numerator > 0.0 + && denominator > 0.0) + .then_some(numerator / denominator); + } + value + .parse::() + .ok() + .filter(|value| value.is_finite() && *value > 0.0) +} + +pub(crate) fn build_authoritative_editor_character_animation_video_conversion_inputs( + client_generation_inputs: Option, + payload: &EditorCharacterAnimationVideoConversionRequest, + source_object_key: &str, + plan: EditorCharacterAnimationVideoConversionPlan, +) -> Result { + let resolution = + normalize_editor_character_animation_video_conversion_resolution(&payload.resolution)?; + let source_object_key = source_object_key.trim().trim_start_matches('/').to_string(); + if source_object_key.is_empty() { + return Err(editor_character_animation_bad_request( + "源视频缺少稳定 objectKey。", + )); + } + let mut result = client_generation_inputs + .and_then(|value| value.as_object().cloned()) + .unwrap_or_default(); + result.insert("version".to_string(), json!(2)); + result.insert("action".to_string(), json!("character-animation.convert")); + result.insert( + "fields".to_string(), + json!([ + { "id": "sourceVideoSrc", "title": "源视频", "value": source_object_key }, + { "id": "resolution", "title": "最大长边", "value": resolution }, + { "id": "actualWidth", "title": "实际宽度", "value": plan.frame_width }, + { "id": "actualHeight", "title": "实际高度", "value": plan.frame_height }, + { "id": "sourceDurationMs", "title": "源时长", "value": plan.duration_ms }, + { "id": "sourceFps", "title": "源帧率", "value": plan.source_fps }, + { "id": "frameCount", "title": "实际帧数", "value": plan.frame_count }, + { "id": "samplingFps", "title": "采样率", "value": plan.sampling_fps }, + { "id": "processingMethod", "title": "处理方式", "value": "视频直接转换" } + ]), + ); + result.insert( + "references".to_string(), + json!([{ "id": "source", "title": "源视频" }]), + ); + Ok(Value::Object(result)) +} #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub(crate) enum EditorCharacterAnimationInputMode { @@ -80,6 +206,10 @@ pub(crate) fn normalize_editor_character_animation_request_with_pricing( ); let reference_src = trim_optional_text(payload.reference_src.as_deref()) .or_else(|| trim_optional_text(Some(payload.source_image_src.as_str()))); + let has_appearance_references = payload + .appearance_reference_srcs + .iter() + .any(|src| !src.trim().is_empty()); if input_mode == EditorCharacterAnimationInputMode::Text && reference_src.is_some() { return Err(editor_character_animation_bad_request( "文本模式不能携带参考素材。", @@ -90,14 +220,21 @@ pub(crate) fn normalize_editor_character_animation_request_with_pricing( "图片或视频模式必须提供参考素材。", )); } + if has_appearance_references && input_mode != EditorCharacterAnimationInputMode::Video { + return Err(editor_character_animation_bad_request( + "appearanceReferenceSrcs 仅支持视频模式。", + )); + } let prompt_text = payload .prompt_text .trim() .chars() .take(4000) .collect::(); - if prompt_text.is_empty() { - return Err(editor_character_animation_bad_request("动画描述不能为空。")); + if prompt_text.is_empty() && reference_src.is_none() { + return Err(editor_character_animation_bad_request( + "动画描述和参考素材不能同时为空。", + )); } let resolution = normalize_editor_character_animation_resolution(payload.resolution.as_str())?; let ratio = normalize_editor_character_animation_ratio(payload.ratio.as_str())?; @@ -250,14 +387,18 @@ pub(crate) fn validate_editor_character_animation_reference_content_type( .to_ascii_lowercase(); let matches = match expected_kind { EditorCharacterAnimationReferenceMediaKind::Image => normalized.starts_with("image/"), - EditorCharacterAnimationReferenceMediaKind::Video => normalized.starts_with("video/"), + EditorCharacterAnimationReferenceMediaKind::Video => { + normalized == "video/mp4" || normalized == "video/quicktime" + } }; if matches { return Ok(()); } let message = match expected_kind { EditorCharacterAnimationReferenceMediaKind::Image => "图片模式只能使用真实图片素材。", - EditorCharacterAnimationReferenceMediaKind::Video => "视频模式只能使用真实视频素材。", + EditorCharacterAnimationReferenceMediaKind::Video => { + "视频模式只能使用真实 MP4 或 MOV 视频素材。" + } }; Err( AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ @@ -324,13 +465,15 @@ pub(crate) fn build_authoritative_editor_character_animation_generation_inputs( payload: &EditorCharacterAnimationGenerateRequest, input_mode: EditorCharacterAnimationInputMode, reference_object_key: Option<&str>, + appearance_object_keys: &[String], ) -> Result { - const AUTHORITATIVE_FIELD_IDS: [&str; 11] = [ + const AUTHORITATIVE_FIELD_IDS: [&str; 12] = [ "inputMode", "referenceSrc", "referenceMediaType", "referenceWidth", "referenceHeight", + "appearanceReferenceSrc", "prompt", "backgroundColor", "resolution", @@ -345,8 +488,10 @@ pub(crate) fn build_authoritative_editor_character_animation_generation_inputs( .chars() .take(4000) .collect::(); - if prompt.is_empty() { - return Err(editor_character_animation_bad_request("动画描述不能为空。")); + if prompt.is_empty() && reference_object_key.is_none() && appearance_object_keys.is_empty() { + return Err(editor_character_animation_bad_request( + "动画描述和参考素材不能同时为空。", + )); } let resolution = normalize_editor_character_animation_resolution(payload.resolution.as_str())?; // generationInputs 保存用户请求语义；same 不能被 provider 使用的具体比例覆盖。 @@ -390,7 +535,30 @@ pub(crate) fn build_authoritative_editor_character_animation_generation_inputs( json!({ "id": "referenceWidth", "title": "参考宽度", "value": payload.source_width }), json!({ "id": "referenceHeight", "title": "参考高度", "value": payload.source_height }), ]); - reference_slots.push(json!({ "id": "source", "title": reference_title })); + reference_slots.push(json!({ + "id": if input_mode == EditorCharacterAnimationInputMode::Video { "motion" } else { "source" }, + "title": reference_title + })); + } + if !appearance_object_keys.is_empty() { + if input_mode != EditorCharacterAnimationInputMode::Video { + return Err(editor_character_animation_bad_request( + "角色外观图仅支持视频模式。", + )); + } + for appearance_object_key in appearance_object_keys { + let appearance_object_key = + appearance_object_key.trim().trim_start_matches('/'); + if appearance_object_key.is_empty() { + continue; + } + fields.push(json!({ + "id": "appearanceReferenceSrc", + "title": "角色外观", + "value": appearance_object_key, + })); + reference_slots.push(json!({ "id": "appearance", "title": "角色外观" })); + } } fields.push(json!({ "id": "prompt", "title": "动作描述", "value": prompt })); @@ -705,6 +873,7 @@ mod tests { input_mode: None, reference_media_type: None, reference_src: None, + appearance_reference_srcs: Vec::new(), prompt_text: "待机呼吸".to_string(), background_color: None, screen_color: None, @@ -717,11 +886,52 @@ mod tests { canvas_completion: None, generation_inputs: None, source_resource_id: None, + appearance_source_resource_ids: Vec::new(), asset_folder_id: None, asset_label: None, } } + #[test] + fn video_conversion_plan_honors_fractional_fps_rotation_size_and_no_upscale() { + assert!( + (parse_editor_character_animation_ffprobe_frame_rate("24000/1001").unwrap() - 23.976) + .abs() + < 0.001 + ); + let plan = build_editor_character_animation_video_conversion_plan( + 2.5, + 24_000.0 / 1_001.0, + 360, + 640, + "720p", + ) + .unwrap(); + assert_eq!((plan.frame_width, plan.frame_height), (360, 640)); + assert_eq!(plan.frame_count, 20); + assert_eq!(plan.duration_ms, 2_500); + assert_eq!(plan.sampling_fps, 8.0); + } + + #[test] + fn video_conversion_plan_downscales_and_caps_at_120_frames() { + let plan = + build_editor_character_animation_video_conversion_plan(15.0, 60.0, 1920, 1080, "480p") + .unwrap(); + assert_eq!((plan.frame_width, plan.frame_height), (480, 270)); + assert_eq!(plan.frame_count, 120); + assert_eq!(plan.sampling_fps, 8.0); + } + + #[test] + fn video_conversion_plan_uses_one_fps_floor_and_two_frame_minimum() { + let plan = + build_editor_character_animation_video_conversion_plan(2.0, 0.5, 320, 180, "480p") + .unwrap(); + assert_eq!(plan.sampling_fps, 1.0); + assert_eq!(plan.frame_count, 2); + } + fn generation_field<'a>(inputs: &'a Value, id: &str) -> Option<&'a Value> { inputs .get("fields") @@ -893,7 +1103,7 @@ mod tests { assert!( video_error .body_text() - .contains("视频模式只能使用真实视频素材") + .contains("视频模式只能使用真实 MP4 或 MOV 视频素材") ); let image_error = validate_editor_character_animation_reference_content_type( @@ -1017,6 +1227,7 @@ mod tests { &request, EditorCharacterAnimationInputMode::Text, None, + &[], ) .expect("text request should always produce a recoverable recipe"); @@ -1072,6 +1283,7 @@ mod tests { &request, EditorCharacterAnimationInputMode::Image, Some("/users/owner/hero.png"), + &[], ) .expect("authoritative image recipe should build"); @@ -1131,6 +1343,7 @@ mod tests { &request, EditorCharacterAnimationInputMode::Image, Some("users/owner/hero.png"), + &[], ) .expect("legacy image recipe should preserve automatic background semantics"); @@ -1151,6 +1364,7 @@ mod tests { &video_request, EditorCharacterAnimationInputMode::Video, Some("users/owner/action.mp4"), + &[], ) .expect("video recipe should persist the default green background"); assert_eq!( diff --git a/server-rs/crates/api-server/src/external_editor_api.rs b/server-rs/crates/api-server/src/external_editor_api.rs index ac44a43ca..9ef70320c 100644 --- a/server-rs/crates/api-server/src/external_editor_api.rs +++ b/server-rs/crates/api-server/src/external_editor_api.rs @@ -152,6 +152,7 @@ impl From input_mode: None, reference_media_type: None, reference_src: None, + appearance_reference_srcs: Vec::new(), prompt_text: payload.prompt_text, background_color: None, screen_color: payload.screen_color, @@ -164,6 +165,7 @@ impl From canvas_completion: payload.canvas_completion, generation_inputs: payload.generation_inputs, source_resource_id: payload.source_resource_id, + appearance_source_resource_ids: Vec::new(), asset_folder_id: payload.asset_folder_id, asset_label: payload.asset_label, } @@ -1553,6 +1555,27 @@ mod tests { .expect_err("External v1 不应接收内部三模态扩展字段"); assert!(error.to_string().contains("inputMode")); + + let appearance_error = + serde_json::from_value::(json!({ + "sourceLayerId": "layer-character", + "sourceImageSrc": "editor/uploads/character.png", + "sourceWidth": 512, + "sourceHeight": 512, + "promptText": "原地待机", + "resolution": "480p", + "ratio": "same", + "frameCount": 32, + "durationSeconds": 4, + "model": "seedance2.0-fast", + "appearanceReferenceSrc": "editor/uploads/appearance.png" + })) + .expect_err("External v1 不应接收内部角色外观字段"); + assert!( + appearance_error + .to_string() + .contains("appearanceReferenceSrc") + ); } #[test] diff --git a/server-rs/crates/api-server/src/external_generation_worker.rs b/server-rs/crates/api-server/src/external_generation_worker.rs index f46f9c314..97945d8f3 100644 --- a/server-rs/crates/api-server/src/external_generation_worker.rs +++ b/server-rs/crates/api-server/src/external_generation_worker.rs @@ -29,12 +29,16 @@ const EDITOR_GENERATION_WARNING_REDACTED_MESSAGE: &str = use crate::{ asset_billing::with_external_generation_billing_attempt_context, character_animation_assets::{ + convert_editor_character_animation_video_for_owner, generate_editor_character_animation_for_owner, generate_editor_video_for_owner, + remove_editor_character_animation_background_for_owner, }, config::AppConfig, editor_generation_queue::{ EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, EDITOR_BACKGROUND_REMOVAL_JOB_KIND, - EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, EDITOR_ICON_SPEC_GENERATION_JOB_KIND, + EDITOR_CHARACTER_ANIMATION_BACKGROUND_REMOVAL_JOB_KIND, + EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, + EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_JOB_KIND, EDITOR_ICON_SPEC_GENERATION_JOB_KIND, EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, EDITOR_IMAGE_EDIT_JOB_KIND, EDITOR_IMAGE_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND, @@ -958,6 +962,64 @@ async fn process_external_generation_job_once( } } } + EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_JOB_KIND => { + let payload = match serde_json::from_str::< + shared_contracts::assets::EditorCharacterAnimationVideoConversionRequest, + >(job.request_payload_json.as_str()) + { + Ok(payload) => payload, + Err(error) => { + let message = format!("视频直接转换任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job, provider_deadline); + match convert_editor_character_animation_video_for_owner( + state.clone(), + request_context, + editor_generation_worker_caller(&worker_id, &job)?, + Ok(Json(payload)), + ) + .await + { + Ok(_) => Ok(()), + Err(response) => { + let message = response_error_message(response).await; + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_CHARACTER_ANIMATION_BACKGROUND_REMOVAL_JOB_KIND => { + let payload = match serde_json::from_str::< + shared_contracts::assets::EditorCharacterAnimationBackgroundRemovalRequest, + >(job.request_payload_json.as_str()) + { + Ok(payload) => payload, + Err(error) => { + let message = format!("Spine序列帧去背景任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job, provider_deadline); + match remove_editor_character_animation_background_for_owner( + state.clone(), + request_context, + editor_generation_worker_caller(&worker_id, &job)?, + Ok(Json(payload)), + ) + .await + { + Ok(_) => Ok(()), + Err(response) => { + let message = response_error_message(response).await; + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } EDITOR_VIDEO_GENERATION_JOB_KIND => { let payload = match serde_json::from_str::< shared_contracts::assets::EditorVideoGenerateRequest, @@ -1633,6 +1695,8 @@ fn external_generation_worker_job_timeout(config: &AppConfig, job_kind: &str) -> | EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND | EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND | EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND + | EDITOR_CHARACTER_ANIMATION_VIDEO_CONVERSION_JOB_KIND + | EDITOR_CHARACTER_ANIMATION_BACKGROUND_REMOVAL_JOB_KIND | EDITOR_VIDEO_GENERATION_JOB_KIND => config.external_generation_worker_long_job_timeout, _ => config.external_generation_worker_job_timeout, } diff --git a/server-rs/crates/api-server/src/modules/platform_support.rs b/server-rs/crates/api-server/src/modules/platform_support.rs index 9b1d4d813..7777207dd 100644 --- a/server-rs/crates/api-server/src/modules/platform_support.rs +++ b/server-rs/crates/api-server/src/modules/platform_support.rs @@ -13,10 +13,11 @@ use crate::{ assets::get_asset_history, auth::require_bearer_auth, character_animation_assets::{ - generate_character_animation, generate_editor_character_animation, generate_editor_video, - get_character_animation_job, get_character_workflow_cache, - import_character_animation_video, list_character_animation_templates, - publish_character_animation, save_character_workflow_cache, + convert_editor_character_animation_video, generate_character_animation, + generate_editor_character_animation, generate_editor_video, get_character_animation_job, + get_character_workflow_cache, import_character_animation_video, + list_character_animation_templates, publish_character_animation, + remove_editor_character_animation_background, save_character_workflow_cache, split_editor_character_animation_frames, }, character_visual_assets::{ @@ -129,12 +130,24 @@ pub fn router(state: AppState) -> Router { require_bearer_auth, )), ) + .route( + "/api/editor/character-animations/video-conversions", + post(convert_editor_character_animation_video).route_layer( + middleware::from_fn_with_state(state.clone(), require_bearer_auth), + ), + ) .route( "/api/editor/character-animations/frame-splits", post(split_editor_character_animation_frames).route_layer( middleware::from_fn_with_state(state.clone(), require_bearer_auth), ), ) + .route( + "/api/editor/character-animations/background-removals", + post(remove_editor_character_animation_background).route_layer( + middleware::from_fn_with_state(state.clone(), require_bearer_auth), + ), + ) .route( "/api/editor/videos/generations", post(generate_editor_video).route_layer(middleware::from_fn_with_state( diff --git a/server-rs/crates/platform-oss/src/lib.rs b/server-rs/crates/platform-oss/src/lib.rs index c563ee158..15a1cea2b 100644 --- a/server-rs/crates/platform-oss/src/lib.rs +++ b/server-rs/crates/platform-oss/src/lib.rs @@ -104,6 +104,11 @@ pub struct OssHeadObjectRequest { pub object_key: String, } +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct OssDeleteObjectRequest { + pub object_key: String, +} + #[derive(Clone, Debug, PartialEq, Eq)] pub struct OssPutObjectRequest { pub prefix: LegacyAssetPrefix, @@ -218,6 +223,7 @@ pub struct OssClient { pub enum OssRequestOperation { Put, Head, + Delete, } #[derive(Clone, Debug, PartialEq, Eq)] @@ -841,6 +847,51 @@ impl OssClient { result } + /// 删除服务端生成但尚未登记为正式资产的对象。404 视为幂等成功。 + pub async fn delete_object( + &self, + client: &reqwest::Client, + request: OssDeleteObjectRequest, + ) -> Result<(), OssError> { + let started_at = Instant::now(); + let object_key = normalize_object_key(&request.object_key)?; + let target_url = build_object_url(&self.config.bucket, &self.config.endpoint, &object_key) + .map_err(|error| { + request_error( + OssRequestOperation::Delete, + &format!("构造 OSS 对象 URL 失败：{error}"), + ) + })?; + let response = send_signed_request( + client, + &self.config, + Method::DELETE, + Some(&object_key), + target_url, + OssRequestOperation::Delete, + ) + .await?; + let status = response.status(); + if !status.is_success() && status != reqwest::StatusCode::NOT_FOUND { + return Err(request_status_error( + OssRequestOperation::Delete, + status.as_u16(), + format!("OSS DeleteObject 失败，状态码：{status}"), + )); + } + info!( + provider = OSS_PROVIDER, + operation = "delete_object", + bucket = %self.config.bucket(), + endpoint = %self.config.endpoint(), + object_key, + status = status.as_u16(), + elapsed_ms = elapsed_ms(started_at), + "OSS DeleteObject 删除完成" + ); + Ok(()) + } + // AI 生成资源默认由服务端上传 OSS，Web 端只拿签名读地址，不直接持有写权限。 pub async fn put_object( &self, diff --git a/server-rs/crates/shared-contracts/src/assets.rs b/server-rs/crates/shared-contracts/src/assets.rs index 99827e657..2d0534012 100644 --- a/server-rs/crates/shared-contracts/src/assets.rs +++ b/server-rs/crates/shared-contracts/src/assets.rs @@ -325,6 +325,9 @@ pub struct EditorCharacterAnimationGenerateRequest { pub reference_media_type: Option, #[serde(default, skip_serializing_if = "Option::is_none")] pub reference_src: Option, + /// 视频动作参考之外的可选角色外观图片。仅 inputMode=video 时允许，可多张。 + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub appearance_reference_srcs: Vec, pub prompt_text: String, #[serde(default, skip_serializing_if = "Option::is_none")] pub background_color: Option, @@ -344,12 +347,57 @@ pub struct EditorCharacterAnimationGenerateRequest { pub generation_inputs: Option, #[serde(default, skip_serializing_if = "Option::is_none")] pub source_resource_id: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub appearance_source_resource_ids: Vec, #[serde(default, skip_serializing_if = "Option::is_none")] pub asset_folder_id: Option, #[serde(default, skip_serializing_if = "Option::is_none")] pub asset_label: Option, } +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct EditorCharacterAnimationVideoConversionRequest { + pub source_video_src: String, + pub resolution: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub canvas_completion: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub generation_inputs: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub source_resource_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asset_folder_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asset_label: Option, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct EditorCharacterAnimationVideoConversionResponse { + pub ok: bool, + pub task_id: String, + pub processing_method: String, + pub frames: Vec, + pub frame_count: u32, + pub duration_ms: u64, + pub frame_width: u32, + pub frame_height: u32, + pub source_fps: f64, + pub sampling_fps: f64, + pub price_mud_points: u32, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub resource: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asset: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub queue_state: Option, +} + #[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] #[serde(rename_all = "camelCase")] pub struct EditorCharacterAnimationFramePayload { @@ -406,6 +454,43 @@ pub struct EditorCharacterAnimationFrameSplitResponse { pub assets: Vec, } +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct EditorCharacterAnimationBackgroundRemovalRequest { + pub project_id: String, + pub source_layer_id: String, + pub source_resource_id: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asset_folder_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asset_label: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub canvas_completion: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub generation_inputs: Option, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct EditorCharacterAnimationBackgroundRemovalResponse { + pub ok: bool, + pub task_id: String, + pub frames: Vec, + pub frame_count: u32, + pub duration_ms: u64, + pub frame_width: u32, + pub frame_height: u32, + pub price_mud_points: u32, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub resource: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asset: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub queue_state: Option, +} + #[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorAssetPayload { @@ -1376,6 +1461,7 @@ mod tests { input_mode: Some("video".to_string()), reference_media_type: Some("video".to_string()), reference_src: Some("generated/editor/reference.mp4".to_string()), + appearance_reference_srcs: vec!["generated/editor/appearance.png".to_string()], prompt_text: "待机呼吸".to_string(), background_color: Some("purple".to_string()), screen_color: None, @@ -1399,6 +1485,7 @@ mod tests { }), generation_inputs: None, source_resource_id: Some("resource-source".to_string()), + appearance_source_resource_ids: vec!["resource-appearance".to_string()], asset_folder_id: Some("project".to_string()), asset_label: Some("勇者奔跑".to_string()), }) @@ -1412,6 +1499,10 @@ mod tests { json!("generated/editor/reference.mp4") ); assert_eq!(payload["backgroundColor"], json!("purple")); + assert_eq!( + payload["appearanceReferenceSrcs"], + json!(["generated/editor/appearance.png"]) + ); assert_eq!( payload["sourceImageSrc"], json!("/generated-characters/hero/master.png") @@ -1421,6 +1512,10 @@ mod tests { assert_eq!(payload["projectId"], json!("editor-project-1")); assert_eq!(payload["canvasCompletion"]["dialogId"], json!("dialog-1")); assert_eq!(payload["sourceResourceId"], json!("resource-source")); + assert_eq!( + payload["appearanceSourceResourceIds"], + json!(["resource-appearance"]) + ); assert_eq!(payload["assetFolderId"], json!("project")); assert_eq!(payload["assetLabel"], json!("勇者奔跑")); } @@ -1443,6 +1538,66 @@ mod tests { assert_eq!(payload["assetLabel"], json!("奔跑动作")); } + #[test] + fn editor_character_animation_video_conversion_uses_discriminated_shape() { + let payload = serde_json::to_value(EditorCharacterAnimationVideoConversionRequest { + source_video_src: "users/owner/source.mov".to_string(), + resolution: "720p".to_string(), + project_id: Some("project-1".to_string()), + canvas_completion: None, + generation_inputs: None, + source_resource_id: Some("resource-video".to_string()), + asset_folder_id: Some("project".to_string()), + asset_label: Some("跑步视频转换".to_string()), + }) + .expect("request should serialize"); + assert_eq!(payload["sourceVideoSrc"], "users/owner/source.mov"); + assert_eq!(payload["resolution"], "720p"); + assert!(payload.get("promptText").is_none()); + assert!(payload.get("model").is_none()); + } + + #[test] + fn editor_character_animation_120_frame_response_stays_within_queue_json_gate() { + let frames = (0..120) + .map(|index| EditorCharacterAnimationFramePayload { + image_src: format!( + "/generated-character-animations/editor/video-conversions/task/frame{:03}.png", + index + 1 + ), + object_key: Some(format!( + "generated-character-animations/editor/video-conversions/task/frame{:03}.png", + index + 1 + )), + asset_object_id: Some(format!("assetobj_{:064}", index + 1)), + width: 720, + height: 405, + }) + .collect(); + let payload = EditorCharacterAnimationVideoConversionResponse { + ok: true, + task_id: "task-max-frames".to_string(), + processing_method: "视频直接转换".to_string(), + frames, + frame_count: 120, + duration_ms: 15_000, + frame_width: 720, + frame_height: 405, + source_fps: 29.970_029_97, + sampling_fps: 8.0, + price_mud_points: 0, + project: None, + resource: None, + asset: None, + queue_state: None, + }; + let bytes = serde_json::to_vec(&payload).expect("response should serialize"); + assert!( + bytes.len() < 512 * 1024, + "120 frame payload must fit queue gate" + ); + } + #[test] fn editor_character_animation_response_includes_frames_and_preview_video() { let payload = serde_json::to_value(EditorCharacterAnimationGenerateResponse { diff --git a/server-rs/crates/spacetime-module/src/editor_project_storage.rs b/server-rs/crates/spacetime-module/src/editor_project_storage.rs index 8c8e69e68..bc3dbe2bb 100644 --- a/server-rs/crates/spacetime-module/src/editor_project_storage.rs +++ b/server-rs/crates/spacetime-module/src/editor_project_storage.rs @@ -17,13 +17,14 @@ const EDITOR_CANVAS_LAYOUT_MIGRATION_STATUS_ACTIVE: &str = "active"; const EDITOR_CANVAS_LAYOUT_MIGRATION_STATUS_ROLLED_BACK: &str = "rolled_back"; const EDITOR_CANVAS_RESOURCE_REPAIR_MAX_ACTIONS: usize = 16; const EDITOR_SPRITESHEET_SLICE_BATCH_MAX_ITEMS: usize = 64; +const EDITOR_CHARACTER_ANIMATION_SLICE_BATCH_MAX_ITEMS: usize = 120; const EDITOR_CANVAS_AUDIO_RESOURCE_WIDTH: u32 = 420; const EDITOR_CANVAS_AUDIO_RESOURCE_HEIGHT: u32 = 120; const EDITOR_CHARACTER_ANIMATION_NORMALIZATION_MAX_BATCH_SIZE: u32 = 25; const EDITOR_CHARACTER_ANIMATION_CANVAS_NORMALIZATION_MAX_BATCH_SIZE: u32 = 5; -// 透明图集最多产出 64 个切片，另有 provider 原图和透明整图两个正式 item。 -const EDITOR_GENERATION_RESULT_MAX_ITEMS: usize = 66; -const EDITOR_GENERATION_OPERATION_KINDS: [&str; 10] = [ +// 角色动画视频转换最多 120 帧，第一帧同时承载最终 resource/asset，事务正好 120 项。 +const EDITOR_GENERATION_RESULT_MAX_ITEMS: usize = 120; +const EDITOR_GENERATION_OPERATION_KINDS: [&str; 11] = [ "editor_image_generation", "editor_icon_spec_generation", "editor_image_edit", @@ -31,6 +32,7 @@ const EDITOR_GENERATION_OPERATION_KINDS: [&str; 10] = [ "editor_icon_spritesheet_generation", "editor_ui_design_asset_extraction", "editor_character_animation_generation", + "editor_character_animation_video_conversion", "editor_video_generation", "editor_sound_effect_generation", "editor_background_music_generation", @@ -3449,7 +3451,9 @@ fn validate_editor_generation_result_shape( return Err("生成结果 completed_at_micros 必须为正数".to_string()); } if !(1..=EDITOR_GENERATION_RESULT_MAX_ITEMS).contains(&input.items.len()) { - return Err("生成结果 item 数量必须在 1 到 66 之间".to_string()); + return Err(format!( + "生成结果 item 数量必须在 1 到 {EDITOR_GENERATION_RESULT_MAX_ITEMS} 之间" + )); } let result_project_id = editor_generation_result_project_id(input)?; let mut slots = BTreeSet::new(); @@ -5103,8 +5107,19 @@ fn validate_editor_spritesheet_slice_batch( group_task_id: Option<&str>, ) -> Result<(), String> { let item_count = input.items.len(); - if !(1..=EDITOR_SPRITESHEET_SLICE_BATCH_MAX_ITEMS).contains(&item_count) { - return Err("图集切片批次产物数量必须在 1 到 64 之间".to_string()); + let is_character_animation_batch = input.items.iter().all(|item| { + matches!( + item.asset_object.asset_kind.trim(), + EDITOR_CHARACTER_ANIMATION_ASSET_KIND | EDITOR_CHARACTER_ANIMATION_OBJECT_KIND + ) + }); + let max_items = if is_character_animation_batch { + EDITOR_CHARACTER_ANIMATION_SLICE_BATCH_MAX_ITEMS + } else { + EDITOR_SPRITESHEET_SLICE_BATCH_MAX_ITEMS + }; + if !(1..=max_items).contains(&item_count) { + return Err(format!("图集切片批次产物数量必须在 1 到 {max_items} 之间")); } if usize::try_from(input.expected_asset_count).ok() != Some(item_count) { return Err("图集切片批次产物数量与预期不一致".to_string()); @@ -14084,7 +14099,7 @@ mod tests { } #[test] - fn editor_generation_result_shape_accepts_sixty_four_slices_plus_two_source_items() { + fn editor_generation_result_shape_accepts_exactly_one_hundred_twenty_items() { let mut input = editor_generation_result_input(); input.items.clear(); for index in 0..EDITOR_GENERATION_RESULT_MAX_ITEMS { @@ -14138,8 +14153,8 @@ mod tests { "editor_image_generation", "job-1", ) - .expect_err("more than the full spritesheet bundle must fail") - .contains("1 到 66") + .expect_err("more than the maximum generation bundle must fail") + .contains("1 到 120") ); } @@ -15685,6 +15700,45 @@ mod tests { .expect("complete batch should pass before transaction writes"); } + #[test] + fn character_animation_split_accepts_120_but_icon_split_remains_64() { + let mut character_batch = spritesheet_slice_batch(); + character_batch.items = (0..120).map(spritesheet_slice_item).collect(); + character_batch.expected_asset_count = 120; + for item in &mut character_batch.items { + item.asset_object.asset_kind = EDITOR_CHARACTER_ANIMATION_OBJECT_KIND.to_string(); + let asset = item.asset.as_mut().expect("asset"); + asset.group_task_expected_asset_count = Some(120); + } + validate_editor_spritesheet_slice_batch( + &character_batch, + "user-1", + "task-1", + Some("group-task-1"), + ) + .expect("character animation should allow 120 frames"); + + let mut icon_batch = spritesheet_slice_batch(); + icon_batch.items = (0..65).map(spritesheet_slice_item).collect(); + icon_batch.expected_asset_count = 65; + for item in &mut icon_batch.items { + item.asset + .as_mut() + .expect("asset") + .group_task_expected_asset_count = Some(65); + } + assert!( + validate_editor_spritesheet_slice_batch( + &icon_batch, + "user-1", + "task-1", + Some("group-task-1"), + ) + .expect_err("icon spritesheet must remain capped at 64") + .contains("1 到 64") + ); + } + #[test] fn spritesheet_slice_batch_accepts_reused_source_objects_under_split_operation() { let mut batch = spritesheet_slice_batch(); diff --git a/src/components/image-editor/ImageCanvasCharacterAnimationPanelView.test.tsx b/src/components/image-editor/ImageCanvasCharacterAnimationPanelView.test.tsx index ddc7fd156..e51a10fd8 100644 --- a/src/components/image-editor/ImageCanvasCharacterAnimationPanelView.test.tsx +++ b/src/components/image-editor/ImageCanvasCharacterAnimationPanelView.test.tsx @@ -227,6 +227,7 @@ describe('ImageCanvasCharacterAnimationPanelView', () => { it('renders completed frame count and failed error state', () => { const result = { + resultType: 'ai' as const, taskId: 'task-a', model: 'seedance2.0-fast' as const, prompt: '行走动作', diff --git a/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.test.tsx b/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.test.tsx index f19664276..01bd849f2 100644 --- a/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.test.tsx +++ b/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.test.tsx @@ -93,9 +93,6 @@ function CharacterGenerationHarness({ {String(dialog.characterSheetEnabled === true)} - - {dialog.characterSheetViewSet ?? '4'} - {String(isPickingSpec)} {String(isPickingReference)} @@ -135,57 +132,21 @@ describe('ImageCanvasCharacterGenerationComposerView', () => { />, ); - const toggle = screen.getByRole('button', { name: '多方向角色图' }); - expect(toggle.getAttribute('aria-pressed')).toBe('false'); + const toggle = screen.getByRole('checkbox', { name: '多方向图' }); + expect((toggle as HTMLInputElement).checked).toBe(false); fireEvent.click(toggle); expect(screen.getByLabelText('当前多方向角色图').textContent).toBe('true'); expect( - screen.getByRole('button', { name: '多方向角色图' }).getAttribute( - 'aria-pressed', - ), - ).toBe('true'); + (screen.getByRole('checkbox', { name: '多方向图' }) as HTMLInputElement) + .checked, + ).toBe(true); - fireEvent.click(screen.getByRole('button', { name: '多方向角色图' })); + fireEvent.click(screen.getByRole('checkbox', { name: '多方向图' })); expect(screen.getByLabelText('当前多方向角色图').textContent).toBe('false'); }); - it('shows the view-set dropdown only while the sheet is enabled', () => { - render( - , - ); - - expect(screen.queryByRole('button', { name: /多方向图视角/ })).toBeNull(); - - fireEvent.click(screen.getByRole('button', { name: '多方向角色图' })); - - expect( - screen.getByRole('button', { name: '多方向图视角 四方向' }), - ).toBeTruthy(); - }); - - it('switches the character sheet view set between four and eight', () => { - render( - , - ); - - fireEvent.click(screen.getByRole('button', { name: '多方向图视角 四方向' })); - fireEvent.click(screen.getByRole('button', { name: '八方向' })); - - expect(screen.getByLabelText('当前多方向图视角数').textContent).toBe('8'); - expect( - screen.getByRole('button', { name: '多方向图视角 八方向' }), - ).toBeTruthy(); - }); - it('updates character prompt, clears failed state and submits', () => { const submitCharacter = vi.fn(); render( diff --git a/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx b/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx index b77d8ef1c..03e61c913 100644 --- a/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx +++ b/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx @@ -1,12 +1,10 @@ -import { ChevronDown, ClipboardList, ImagePlus, LayoutGrid } from 'lucide-react'; +import { ClipboardList, ImagePlus } from 'lucide-react'; import { type CSSProperties, type Dispatch, type ReactNode, type RefObject, type SetStateAction, - useRef, - useState, } from 'react'; import { AutoGrowTextArea } from '../common/AutoGrowTextArea'; @@ -14,7 +12,6 @@ import { PlatformFloatingMenu, PlatformFloatingMenuItem, } from '../common/PlatformFloatingMenu'; -import { PlatformInlineOptionButton } from '../common/PlatformInlineOptionButton'; import { PlatformStatusMessage } from '../common/PlatformStatusMessage'; import type { CharacterReferenceImage, @@ -104,8 +101,6 @@ export function ImageCanvasCharacterGenerationComposerView({ hasPendingImageReferenceUploads = false, onSubmit, }: ImageCanvasCharacterGenerationComposerViewProps) { - const [isViewSetMenuOpen, setIsViewSetMenuOpen] = useState(false); - const viewSetButtonRef = useRef(null); useImageCanvasFloatingOptionDismiss({ isOpen: isCharacterSpecMenuOpen || isCharacterReferenceMenuOpen, boundaryRefs: [characterSpecButtonRef, characterReferenceButtonRef], @@ -114,25 +109,6 @@ export function ImageCanvasCharacterGenerationComposerView({ setIsCharacterReferenceMenuOpen(false); }, }); - useImageCanvasFloatingOptionDismiss({ - isOpen: isViewSetMenuOpen, - boundaryRefs: [viewSetButtonRef], - onDismiss: () => setIsViewSetMenuOpen(false), - restoreFocusRef: viewSetButtonRef, - }); - const setViewSet = (value: '4' | '8') => { - setIsViewSetMenuOpen(false); - setGenerateDialog((currentDialog) => - currentDialog?.mode === 'character' - ? { - ...resetFailedDialogStatus(currentDialog), - characterSheetViewSet: value, - } - : currentDialog, - ); - }; - const selectedViewSetLabel = - (dialog.characterSheetViewSet ?? '4') === '8' ? '八方向' : '四方向'; return (
setIsCharacterReferenceMenuOpen((open) => !open)} /> -
diff --git a/src/components/image-editor/ImageCanvasContextMenusView.test.tsx b/src/components/image-editor/ImageCanvasContextMenusView.test.tsx index eedc9fba6..d136ba461 100644 --- a/src/components/image-editor/ImageCanvasContextMenusView.test.tsx +++ b/src/components/image-editor/ImageCanvasContextMenusView.test.tsx @@ -56,7 +56,6 @@ function renderContextMenus( onFitLayers: vi.fn(), onOpenQuickEditPanel: vi.fn(), onOpenLayerMetadata: vi.fn(), - onOpenCharacterAnimationPanel: vi.fn(), ...overrides, }; @@ -160,7 +159,6 @@ describe('ImageCanvasContextMenusView', () => { screen.getByRole('menuitem', { name: 'Spine 导出（zip）' }), ); fireEvent.click(screen.getByRole('menuitem', { name: '快速编辑' })); - fireEvent.click(screen.getByRole('menuitem', { name: '生成动画' })); fireEvent.click(screen.getByRole('menuitem', { name: '删除' })); expect(props.onCopyContextLayers).toHaveBeenNthCalledWith(1); @@ -179,9 +177,8 @@ describe('ImageCanvasContextMenusView', () => { expect.objectContaining({ mode: 'spine-json' }), ); expect(props.onOpenQuickEditPanel).toHaveBeenCalledWith(layer); - expect(props.onOpenCharacterAnimationPanel).toHaveBeenCalledWith(layer); - expect(props.onCloseContextMenu).toHaveBeenCalledTimes(2); - expect(props.onCloseImageContextMenu).toHaveBeenCalledTimes(2); + expect(props.onCloseContextMenu).toHaveBeenCalledTimes(1); + expect(props.onCloseImageContextMenu).toHaveBeenCalledTimes(1); expect(props.onDeleteContextLayers).toHaveBeenCalledTimes(1); }); @@ -319,19 +316,6 @@ describe('ImageCanvasContextMenusView', () => { } }); - it('closes the standalone image menu after opening character animation', () => { - const layer = createLayer({ assetKind: 'character' }); - const props = renderContextMenus({ - imageContextMenu: { layerId: layer.id, x: 20, y: 22 }, - imageContextMenuLayer: layer, - }); - - fireEvent.click(screen.getByRole('menuitem', { name: '生成动画' })); - - expect(props.onOpenCharacterAnimationPanel).toHaveBeenCalledWith(layer); - expect(props.onCloseImageContextMenu).toHaveBeenCalledTimes(1); - }); - it('renders the image context menu when no canvas context menu is open', () => { const layer = createLayer({ assetKind: 'character' }); const props = renderContextMenus({ diff --git a/src/components/image-editor/ImageCanvasContextMenusView.tsx b/src/components/image-editor/ImageCanvasContextMenusView.tsx index e8702e976..da08d5aed 100644 --- a/src/components/image-editor/ImageCanvasContextMenusView.tsx +++ b/src/components/image-editor/ImageCanvasContextMenusView.tsx @@ -53,7 +53,6 @@ type ImageCanvasContextMenusViewProps = { onFitLayers: () => void; onOpenQuickEditPanel: (layer: CanvasLayer) => void; onOpenLayerMetadata: (layer: CanvasLayer) => void; - onOpenCharacterAnimationPanel: (layer: CanvasLayer) => void; }; type MeasuredMenuLayout = { @@ -94,7 +93,6 @@ export function ImageCanvasContextMenusView({ onFitLayers, onOpenQuickEditPanel, onOpenLayerMetadata, - onOpenCharacterAnimationPanel, }: ImageCanvasContextMenusViewProps) { const isImageSequenceLayer = contextMenuLayer?.mediaType === 'image-sequence'; const layerCopyBlockedTitle = @@ -525,19 +523,6 @@ export function ImageCanvasContextMenusView({ > 查看图片信息 - {imageContextMenuLayer.assetKind === 'character' ? ( - - ) : null}
) : null} @@ -585,17 +570,6 @@ export function ImageCanvasContextMenusView({ > 查看图片信息 - {imageContextMenuLayer.assetKind === 'character' ? ( - { - onOpenCharacterAnimationPanel(imageContextMenuLayer); - onCloseImageContextMenu(); - }} - > - 生成动画 - - ) : null} onDeleteLayerById(imageContextMenuLayer.id)} diff --git a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx index 9f28be8b8..1ab91da71 100644 --- a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx @@ -3452,20 +3452,14 @@ describe('ImageCanvasEditorView generation integration', () => { const characterLayer = await screen.findByAltText('画布图片：市场老妇人'); fireEvent.click(characterLayer.closest('button')!); expect(screen.getByText('角色')).toBeTruthy(); - expect(screen.getByRole('button', { name: '生成动画' })).toBeTruthy(); + expect(screen.queryByRole('button', { name: '生成动画' })).toBeNull(); expect(screen.queryByRole('dialog', { name: '重绘图片' })).toBeNull(); expect( screen.queryByRole('dialog', { name: 'Spine序列帧动画生成' }), ).toBeNull(); - fireEvent.contextMenu(characterLayer.closest('button')!, { - clientX: 260, - clientY: 220, - }); - expect(screen.getByRole('menuitem', { name: '生成动画' })).toBeTruthy(); - expect(screen.queryByRole('dialog', { name: '重绘图片' })).toBeNull(); - fireEvent.click(screen.getByRole('button', { name: '生成动画' })); + fireEvent.click(screen.getByRole('button', { name: 'Spine序列帧动画' })); expect( screen.getByRole('button', { name: '角色动作生成占位图' }), ).toBeTruthy(); @@ -3540,21 +3534,24 @@ describe('ImageCanvasEditorView generation integration', () => { }), ); fireEvent.click( - within( - screen.getByRole('group', { name: '序列帧参数选项' }), - ).getByRole('button', { name: '720p' }), + within(screen.getByRole('group', { name: '序列帧参数选项' })).getByRole( + 'button', + { name: '720p' }, + ), ); fireEvent.click( - within( - screen.getByRole('group', { name: '序列帧参数选项' }), - ).getByRole('button', { name: '16:9' }), + within(screen.getByRole('group', { name: '序列帧参数选项' })).getByRole( + 'button', + { name: '16:9' }, + ), ); fireEvent.click( - within( - screen.getByRole('group', { name: '序列帧参数选项' }), - ).getByRole('button', { - name: '48帧·6秒', - }), + within(screen.getByRole('group', { name: '序列帧参数选项' })).getByRole( + 'button', + { + name: '48帧·6秒', + }, + ), ); const resizedPlaceholder = screen.getByRole('button', { name: '角色动作生成占位图', diff --git a/src/components/image-editor/ImageCanvasEditorModel.test.ts b/src/components/image-editor/ImageCanvasEditorModel.test.ts index 4f160f6c4..696ee5196 100644 --- a/src/components/image-editor/ImageCanvasEditorModel.test.ts +++ b/src/components/image-editor/ImageCanvasEditorModel.test.ts @@ -1339,6 +1339,103 @@ describe('ImageCanvasEditorModel', () => { }); }); + it('hydrates a character-animation preview video through its source resource', () => { + const frames = [ + { + imageSrc: '/generated/sequence/frame-1.png', + objectKey: 'generated/sequence/frame-1.png', + width: 480, + height: 480, + }, + { + imageSrc: '/generated/sequence/frame-2.png', + objectKey: 'generated/sequence/frame-2.png', + width: 480, + height: 480, + }, + ]; + const hydrated = hydrateLayer( + { + layerId: 'layer-sequence', + resourceId: 'resource-sequence', + title: '角色动作', + sourceType: 'generated', + }, + new Map([ + [ + 'resource-sequence', + { + resourceId: 'resource-sequence', + imageSrc: frames[0]!.imageSrc, + objectKey: frames[0]!.objectKey, + assetKind: 'character-animation', + imageSequenceFrames: frames, + imageSequenceDurationMs: 4_000, + taskId: 'animation-task-1', + sourceResourceId: 'resource-preview-video', + }, + ], + [ + 'resource-preview-video', + { + resourceId: 'resource-preview-video', + imageSrc: '/generated/video/preview.mp4', + objectKey: 'generated/video/preview.mp4', + assetKind: 'video', + taskId: 'animation-task-1', + }, + ], + ]), + ); + + expect(hydrated).toMatchObject({ + assetKind: 'character-animation', + mediaType: 'image-sequence', + previewVideoPath: 'generated/video/preview.mp4', + imageSequenceFrames: frames, + imageSequenceDurationMs: 4_000, + }); + }); + + it('does not hydrate an older action reference as a generated preview video', () => { + const frames = [ + { imageSrc: '/generated/sequence/frame-1.png', width: 480, height: 480 }, + { imageSrc: '/generated/sequence/frame-2.png', width: 480, height: 480 }, + ]; + const hydrated = hydrateLayer( + { + layerId: 'layer-sequence', + resourceId: 'resource-sequence', + title: '角色动作', + sourceType: 'generated', + }, + new Map([ + [ + 'resource-sequence', + { + imageSrc: frames[0]!.imageSrc, + assetKind: 'character-animation', + imageSequenceFrames: frames, + imageSequenceDurationMs: 4_000, + taskId: 'animation-task-2', + sourceResourceId: 'resource-action-video', + }, + ], + [ + 'resource-action-video', + { + imageSrc: '/generated/source/action.mp4', + objectKey: 'generated/source/action.mp4', + assetKind: 'video', + taskId: 'source-task-1', + }, + ], + ]), + ); + + expect(hydrated?.previewVideoPath).toBeNull(); + }); + it('rejects an action layer when its project resource lacks formal sequence fields', () => { const frames = [ { @@ -1637,6 +1734,100 @@ describe('ImageCanvasEditorModel', () => { }); }); + it('round-trips independent character animation AI and conversion drafts', () => { + const dialog: CanvasGenerationDialogState = { + id: 'dialog-character-animation-drafts', + mode: 'character-animation', + prompt: '保留 AI 提示词', + status: 'idle', + characterAnimationWorkflow: 'video-conversion', + characterAnimationBackgroundColor: 'green', + characterAnimationResolution: '480p', + characterAnimationConversionResolution: '720p', + generationReferences: [ + { + id: 'motion', + label: '动作视频', + src: 'users/user-1/motion.mp4', + objectKey: 'users/user-1/motion.mp4', + mediaType: 'video', + resourceId: 'resource-motion', + }, + ], + characterAnimationAppearanceReferences: [ + { + id: 'appearance', + label: '角色外观', + src: 'users/user-1/appearance.png', + objectKey: 'users/user-1/appearance.png', + resourceId: 'resource-appearance', + }, + ], + characterAnimationConversionVideoReference: { + id: 'conversion-source', + label: '直接转换源视频', + src: 'users/user-1/source.mov', + objectKey: 'users/user-1/source.mov', + mediaType: 'video', + resourceId: 'resource-conversion-source', + }, + }; + + const layout = serializeCanvasLayout({ + layers: [], + canvasGenerationDialogs: [dialog], + }); + const { generationDialogs } = splitCanvasLayoutItems( + layout, + new Map([ + [ + 'resource-motion', + { + imageSrc: 'users/user-1/motion.mp4', + objectKey: 'users/user-1/motion.mp4', + }, + ], + [ + 'resource-appearance', + { + imageSrc: 'users/user-1/appearance.png', + objectKey: 'users/user-1/appearance.png', + }, + ], + [ + 'resource-conversion-source', + { + imageSrc: 'users/user-1/source.mov', + objectKey: 'users/user-1/source.mov', + }, + ], + ]), + ); + + expect(generationDialogs[0]).toMatchObject({ + prompt: '保留 AI 提示词', + characterAnimationWorkflow: 'video-conversion', + characterAnimationBackgroundColor: 'green', + characterAnimationResolution: '480p', + characterAnimationConversionResolution: '720p', + generationReferences: [ + expect.objectContaining({ + objectKey: 'users/user-1/motion.mp4', + mediaType: 'video', + }), + ], + characterAnimationAppearanceReferences: [ + expect.objectContaining({ + objectKey: 'users/user-1/appearance.png', + }), + ], + characterAnimationConversionVideoReference: expect.objectContaining({ + objectKey: 'users/user-1/source.mov', + mediaType: 'video', + }), + }); + }); + it('keeps the operation ledger out of the layout and restores it from the local ledger', () => { const dialogId = 'dialog-perfect-pixel-round-trip'; const operation = buildPerfectPixelOperation(dialogId); diff --git a/src/components/image-editor/ImageCanvasEditorModel.ts b/src/components/image-editor/ImageCanvasEditorModel.ts index d063fac44..a44fa39e4 100644 --- a/src/components/image-editor/ImageCanvasEditorModel.ts +++ b/src/components/image-editor/ImageCanvasEditorModel.ts @@ -7,7 +7,7 @@ import { isEditorSceneStylePreset } from '../../../packages/shared/src/contracts import type { EditorAssetGenerationInputs, EditorAssetLibrarySnapshot, - EditorCharacterAnimationGenerationResult, + EditorCharacterAnimationResult, EditorPixelArtSnapInput, EditorProjectLayerSnapshot, EditorProjectSnapshot, @@ -722,6 +722,12 @@ function serializeDialogReferences( generationReferences: serializeGenerationReferences( dialog.generationReferences, ), + characterAnimationAppearanceReferences: serializeGenerationReferences( + dialog.characterAnimationAppearanceReferences, + ), + characterAnimationConversionVideoReference: serializeGenerationReference( + dialog.characterAnimationConversionVideoReference, + ), characterSpecReference: serializeGenerationReference( dialog.characterSpecReference, ), @@ -1274,6 +1280,34 @@ export function hydrateCanvasGenerationDialog( typeof snapshot.videoWebSearchEnabled === 'boolean' ? snapshot.videoWebSearchEnabled : undefined, + characterAnimationWorkflow: + snapshot.characterAnimationWorkflow === 'video-conversion' + ? 'video-conversion' + : snapshot.characterAnimationWorkflow === 'ai' + ? 'ai' + : undefined, + characterAnimationAppearanceReferences: hydrateCharacterReferences( + snapshot.characterAnimationAppearanceReferences, + resourcesById, + currentUserId, + ), + characterAnimationConversionVideoReference: hydrateCharacterReference( + snapshot.characterAnimationConversionVideoReference, + resourcesById, + currentUserId, + ), + characterAnimationConversionResolution: + snapshot.characterAnimationConversionResolution === '480p' || + snapshot.characterAnimationConversionResolution === '720p' + ? snapshot.characterAnimationConversionResolution + : undefined, + characterAnimationBackgroundColor: + snapshot.characterAnimationBackgroundColor === 'white' || + snapshot.characterAnimationBackgroundColor === 'green' || + snapshot.characterAnimationBackgroundColor === 'blue' || + snapshot.characterAnimationBackgroundColor === 'purple' + ? snapshot.characterAnimationBackgroundColor + : undefined, characterAnimationResolution: snapshot.characterAnimationResolution === '480p' || snapshot.characterAnimationResolution === '720p' @@ -1409,6 +1443,33 @@ export function hydrateLayer( const src = isFormalCharacterAnimation ? firstSequenceFrame || resource?.imageSrc || '' : snapshotSrc || firstSequenceFrame || resource?.imageSrc || ''; + const sequencePreviewVideoPath = isFormalCharacterAnimation + ? (() => { + const previewResourceId = stringOrNull(resource?.sourceResourceId); + const previewResource = previewResourceId + ? resourcesById.get(previewResourceId) + : undefined; + if (previewResource?.assetKind !== 'video') { + return undefined; + } + // 正式序列帧的 sourceResourceId 应指向同一次操作登记的预览视频。 + // 如果它实际指向更早的动作参考视频，不能把参考视频当成生成结果播放； + // 这也避免历史错误 lineage 在刷新后悄悄改变“底层视频”。 + const animationTaskId = stringOrNull(resource?.taskId); + const previewTaskId = stringOrNull(previewResource.taskId); + if ( + animationTaskId && + previewTaskId && + animationTaskId !== previewTaskId + ) { + return undefined; + } + return ( + stringOrNull(previewResource.objectKey) ?? + stringOrNull(previewResource.imageSrc) + ); + })() + : undefined; const title = typeof snapshot.title === 'string' ? snapshot.title : '画布图片'; if (!resourceId || !layerId || !src) { @@ -1503,7 +1564,7 @@ export function hydrateLayer( ? imageSequenceFrames : undefined, previewVideoPath: isFormalCharacterAnimation - ? null + ? (sequencePreviewVideoPath ?? null) : stringOrNull(snapshot.previewVideoPath), prompt: isFormalCharacterAnimation ? stringOrNull(resource?.prompt) @@ -2023,25 +2084,43 @@ function videoAspectRatioOrUndefined(value: unknown) { function hydrateCharacterAnimationResult( value: unknown, -): EditorCharacterAnimationGenerationResult | undefined { +): EditorCharacterAnimationResult | undefined { if (!value || typeof value !== 'object') { return undefined; } - const result = value as Partial; + const result = value as Record & + Partial; if ( typeof result.taskId !== 'string' || - result.model !== 'seedance2.0-fast' || - typeof result.prompt !== 'string' || - typeof result.previewVideoPath !== 'string' || !Array.isArray(result.frames) || typeof result.frameCount !== 'number' || - typeof result.durationSeconds !== 'number' || - typeof result.fps !== 'number' || typeof result.priceMudPoints !== 'number' ) { return undefined; } - return result as EditorCharacterAnimationGenerationResult; + if (result.resultType === 'video-conversion') { + return result.processingMethod === '视频直接转换' && + typeof result.durationMs === 'number' && + typeof result.frameWidth === 'number' && + typeof result.frameHeight === 'number' && + typeof result.sourceFps === 'number' && + typeof result.samplingFps === 'number' && + result.priceMudPoints === 0 + ? (result as EditorCharacterAnimationResult) + : undefined; + } + // 旧项目在判别字段引入前只持久化 AI 结果；按原契约恢复并补上判别字段。 + if ( + (result.resultType === undefined || result.resultType === 'ai') && + result.model === 'seedance2.0-fast' && + typeof result.prompt === 'string' && + typeof result.previewVideoPath === 'string' && + typeof result.durationSeconds === 'number' && + typeof result.fps === 'number' + ) { + return { ...result, resultType: 'ai' } as EditorCharacterAnimationResult; + } + return undefined; } function hydrateImageSequenceFrames( diff --git a/src/components/image-editor/ImageCanvasEditorShellView.test.tsx b/src/components/image-editor/ImageCanvasEditorShellView.test.tsx index 3d0602a4e..71b3671fd 100644 --- a/src/components/image-editor/ImageCanvasEditorShellView.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorShellView.test.tsx @@ -221,6 +221,7 @@ function createStageProps(): ImageCanvasStageViewProps { onPerfectPixel: vi.fn(), onSplitIconSpritesheet: vi.fn(), onSplitCharacterAnimation: vi.fn(), + onRemoveCharacterAnimationBackground: vi.fn(), onExtractUiDesignAssets: vi.fn(), onUiAssetExtractionToolChange: vi.fn(), onUiAssetExtractionModelChange: vi.fn(), @@ -234,7 +235,6 @@ function createStageProps(): ImageCanvasStageViewProps { onQuickEditSelectionPointerStart: vi.fn(), onQuickEditSelectionPointerMove: vi.fn(), onQuickEditSelectionPointerEnd: vi.fn(), - onOpenCharacterAnimationPanel: vi.fn(), onDownloadLayer: vi.fn(), onPasteCanvasClipboard: vi.fn(), onCopyContextLayers: vi.fn(), diff --git a/src/components/image-editor/ImageCanvasEditorTypes.ts b/src/components/image-editor/ImageCanvasEditorTypes.ts index 92bbb03e0..be70fb385 100644 --- a/src/components/image-editor/ImageCanvasEditorTypes.ts +++ b/src/components/image-editor/ImageCanvasEditorTypes.ts @@ -8,7 +8,7 @@ import type { EditorAssetSnapshot, EditorCharacterAnimationBackgroundColor, EditorCharacterAnimationFrameCount, - EditorCharacterAnimationGenerationResult, + EditorCharacterAnimationResult, EditorCharacterAnimationRatio, EditorCharacterAnimationResolution, EditorImageGenerationStyle, @@ -90,6 +90,8 @@ export type CanvasGenerationAction = | 'audio.sound-effect.generate' | 'audio.background-music.generate' | 'character-animation.generate' + | 'character-animation.convert' + | 'character-animation.remove-background' | 'image.edit' | 'ui-design.extract-assets' | 'image.perfect-pixel' @@ -219,6 +221,8 @@ export type CharacterReferenceImage = { label: string; src: string; mediaType?: CanvasMediaType; + /** 参考来源的画布资产类型；用于区分序列帧底层视频与独立源视频。 */ + assetKind?: CanvasAssetKind | null; mimeType?: string; sizeBytes?: number; durationSeconds?: number; @@ -279,11 +283,9 @@ export type GenerateDialogState = { generationReferences?: CharacterReferenceImage[]; characterSpecReference?: CharacterReferenceImage | null; characterReferences?: CharacterReferenceImage[]; - // 中文注释：角色面板的"多方向角色图"开关——开启时生成一张四方向角色图集 + // 中文注释：角色面板的"多方向角色图"开关——开启时生成一张八方向角色图集 // （后端 kind=character-sheet，产物按 icon-spritesheet 落库，可走既有拆分图集）。 characterSheetEnabled?: boolean; - // 中文注释：多方向角色图的视角数（4 或 8），仅 characterSheetEnabled 时生效。 - characterSheetViewSet?: '4' | '8'; iconSpecReference?: CharacterReferenceImage | null; iconDescriptions?: string[]; publicationWorkflowId?: PublicationMaterialsWorkflowId; @@ -306,7 +308,11 @@ export type GenerateDialogState = { characterAnimationFrameCount?: EditorCharacterAnimationFrameCount; characterAnimationDurationSeconds?: 4 | 5 | 6; characterAnimationBackgroundColor?: EditorCharacterAnimationBackgroundColor; - characterAnimationResult?: EditorCharacterAnimationGenerationResult; + characterAnimationWorkflow?: 'ai' | 'video-conversion'; + characterAnimationAppearanceReferences?: CharacterReferenceImage[]; + characterAnimationConversionVideoReference?: CharacterReferenceImage | null; + characterAnimationConversionResolution?: EditorCharacterAnimationResolution; + characterAnimationResult?: EditorCharacterAnimationResult; soundModel?: EditorSoundEffectModel | 'audio1.0'; soundDurationMode?: EditorSoundEffectDurationMode; soundDurationSeconds?: number; @@ -501,6 +507,10 @@ export type CropExpandResizeHandle = export type CharacterAnimationPanelState = { sourceLayerId?: string; references?: CharacterReferenceImage[]; + workflow?: 'ai' | 'video-conversion'; + appearanceReferences?: CharacterReferenceImage[]; + conversionVideoReference?: CharacterReferenceImage | null; + conversionResolution?: EditorCharacterAnimationResolution; promptText: string; assetLabel?: string; backgroundColor?: EditorCharacterAnimationBackgroundColor; @@ -510,7 +520,7 @@ export type CharacterAnimationPanelState = { durationSeconds: 4 | 5 | 6; status: 'idle' | 'generating' | 'completed' | 'failed'; errorMessage?: string; - result?: EditorCharacterAnimationGenerationResult; + result?: EditorCharacterAnimationResult; }; export type UploadTarget = diff --git a/src/components/image-editor/ImageCanvasEditorView.tsx b/src/components/image-editor/ImageCanvasEditorView.tsx index aa4403734..86d610b7f 100644 --- a/src/components/image-editor/ImageCanvasEditorView.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.tsx @@ -236,7 +236,8 @@ function findSourceAnimationLayer( return ( layers.find( (layer) => - (layer.resourceId === sourceResourceId || layer.id === sourceResourceId) && + (layer.resourceId === sourceResourceId || + layer.id === sourceResourceId) && layer.assetKind === 'character', ) ?? null ); @@ -1431,6 +1432,9 @@ export function ImageCanvasEditorView({ pendingImageReferenceUploadCount; const generationSurface = useImageCanvasGenerationSurface({ layers, + selectedLayer: selectedLayerId + ? (layers.find((layer) => layer.id === selectedLayerId) ?? null) + : null, canvasSize, viewport, setViewport, @@ -1603,6 +1607,8 @@ export function ImageCanvasEditorView({ splittingIconSpritesheetLayerIds, splitSelectedCharacterAnimationFrames, splittingCharacterAnimationLayerIds, + removeSelectedCharacterAnimationBackground, + removingBackgroundCharacterAnimationLayerIds, extractUiDesignAssets, pickCharacterSpecFromLayer, pickGenerationReferenceFromLayer, @@ -2598,6 +2604,7 @@ export function ImageCanvasEditorView({ pendingPerfectPixelLayerIds, splittingIconSpritesheetLayerIds, splittingCharacterAnimationLayerIds, + removingBackgroundCharacterAnimationLayerIds, uploadDropTarget, contextMenu, canvasClipboard, @@ -2664,6 +2671,11 @@ export function ImageCanvasEditorView({ splitSelectedCharacterAnimationFrames(layer), ); }, + onRemoveCharacterAnimationBackground: (layer: CanvasLayer) => { + void flushProjectPersistence().then(() => + removeSelectedCharacterAnimationBackground(layer), + ); + }, onExtractUiDesignAssets: extractUiDesignAssets, onUiAssetExtractionToolChange: changeUiAssetExtractionTool, onUiAssetExtractionModelChange: changeUiAssetExtractionModel, @@ -2679,7 +2691,6 @@ export function ImageCanvasEditorView({ onQuickEditSelectionPointerStart: startQuickEditSelectionPointer, onQuickEditSelectionPointerMove: moveQuickEditSelectionPointer, onQuickEditSelectionPointerEnd: endQuickEditSelectionPointer, - onOpenCharacterAnimationPanel: openCharacterAnimationPanel, onDownloadLayer: exportLayerImage, onPasteCanvasClipboard: pasteCanvasOrSystemClipboard, onCopyContextLayers: copyContextLayers, diff --git a/src/components/image-editor/ImageCanvasExportModel.ts b/src/components/image-editor/ImageCanvasExportModel.ts index 020bb627f..c9e350877 100644 --- a/src/components/image-editor/ImageCanvasExportModel.ts +++ b/src/components/image-editor/ImageCanvasExportModel.ts @@ -456,7 +456,7 @@ export function buildSpineImageSequenceJson({ skeleton: { spine: DEFAULT_SPINE_VERSION, images: './frames/', - fps: Math.max(1, Math.round(frames.length / durationSeconds)), + fps: Number((frames.length / durationSeconds).toFixed(6)), width: skeletonWidth, height: skeletonHeight, }, diff --git a/src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx b/src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx index 68b1bec1a..1a20c5d40 100644 --- a/src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx +++ b/src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx @@ -2119,6 +2119,190 @@ describe('ImageCanvasGenerationComposerView', () => { ).toBeTruthy(); }); + it('AI 视频从画布选中外观图后展示已选参考并支持移除', () => { + renderComposer({ + id: 'dialog-character-animation-video-appearance', + mode: 'character-animation', + prompt: '原地奔跑', + status: 'idle', + composerOpen: true, + generationReferences: [ + { + id: 'motion-video', + label: '动作视频', + src: 'generated/motion.mp4', + mediaType: 'video', + }, + ], + characterAnimationAppearanceReferences: [ + { + id: 'appearance-image', + label: '骑士角色', + src: 'generated/knight.png', + mediaType: 'image', + }, + ], + characterAnimationWorkflow: 'ai', + }); + + const panel = screen.getByRole('dialog', { + name: 'Spine序列帧动画生成', + }); + expect(within(panel).getByText('骑士角色')).toBeTruthy(); + expect( + within(panel).getByRole('button', { name: '删除骑士角色' }), + ).toBeTruthy(); + }); + + it('删除动作视频后清空动作并保留外观图片参考', () => { + const dialog: GenerateDialogState = { + id: 'dialog-character-animation-video-remove-motion', + mode: 'character-animation', + prompt: '原地奔跑', + status: 'idle', + composerOpen: true, + generationReferences: [ + { + id: 'motion-video', + label: '动作视频', + src: 'generated/motion.mp4', + mediaType: 'video', + }, + ], + characterAnimationAppearanceReferences: [ + { + id: 'appearance-image', + label: '骑士角色', + src: 'generated/knight.png', + mediaType: 'image', + }, + ], + characterAnimationWorkflow: 'ai', + }; + const setGenerateDialog = vi.fn(); + renderComposer(dialog, { setGenerateDialog }); + + const panel = screen.getByRole('dialog', { + name: 'Spine序列帧动画生成', + }); + fireEvent.click( + within(panel).getByRole('button', { name: '删除动作视频' }), + ); + + const updater = setGenerateDialog.mock.calls[0]?.[0] as ( + currentDialog: GenerateDialogState | null, + ) => GenerateDialogState | null; + const nextDialog = updater(dialog); + expect(nextDialog?.generationReferences).toEqual([]); + expect(nextDialog?.characterAnimationAppearanceReferences).toEqual([ + expect.objectContaining({ id: 'appearance-image', mediaType: 'image' }), + ]); + }); + + it('视频直接转换标签不展示 AI 外观图草稿', () => { + renderComposer({ + id: 'dialog-character-animation-conversion-hidden-appearance', + mode: 'character-animation', + prompt: '', + status: 'idle', + composerOpen: true, + characterAnimationWorkflow: 'video-conversion', + characterAnimationConversionVideoReference: { + id: 'conversion-video', + label: '源视频', + src: 'generated/source.mp4', + mediaType: 'video', + }, + generationReferences: [ + { + id: 'motion-video', + label: '动作视频', + src: 'generated/motion.mp4', + mediaType: 'video', + }, + ], + characterAnimationAppearanceReferences: [ + { + id: 'appearance-image', + label: '骑士角色', + src: 'generated/knight.png', + mediaType: 'image', + }, + ], + }); + + const panel = screen.getByRole('dialog', { + name: 'Spine序列帧动画生成', + }); + expect(within(panel).queryByText('骑士角色')).toBeNull(); + expect(within(panel).getAllByText('源视频')).toHaveLength(2); + expect( + within(panel).getByRole('button', { name: '删除源视频' }), + ).toBeTruthy(); + }); + + it('视频作为 AI 参考时不展示角色外观占位符，但仍保留统一参考入口', () => { + renderComposer({ + id: 'dialog-character-animation-video-reference', + mode: 'character-animation', + prompt: '原地奔跑', + status: 'idle', + composerOpen: true, + generationReferences: [ + { + id: 'motion-video', + label: '动作视频', + src: 'generated/motion.mp4', + mediaType: 'video', + }, + ], + characterAnimationWorkflow: 'ai', + }); + + const panel = screen.getByRole('dialog', { + name: 'Spine序列帧动画生成', + }); + expect(within(panel).queryByText('角色外观')).toBeNull(); + expect( + within(panel).getByRole('button', { name: '添加序列帧参考素材' }), + ).toBeTruthy(); + }); + + it('defaults to AI workflow and keeps independent drafts while switching tabs', () => { + render(); + + const panel = screen.getByRole('dialog', { + name: 'Spine序列帧动画生成', + }); + const aiTab = within(panel).getByRole('tab', { name: 'AI生成/改造' }); + const conversionTab = within(panel).getByRole('tab', { + name: '视频直接转换', + }); + expect(aiTab.getAttribute('aria-selected')).toBe('true'); + expect( + within(panel).getByRole('textbox', { name: '动画描述' }).textContent, + ).toBe('原地奔跑'); + + fireEvent.click(conversionTab); + expect(conversionTab.getAttribute('aria-selected')).toBe('true'); + expect( + within(panel).queryByRole('textbox', { name: '动画描述' }), + ).toBeNull(); + expect(within(panel).getByText('免费转换')).toBeTruthy(); + expect( + ( + within(panel).getByRole('button', { + name: '免费转换为Spine序列帧动画', + }) as HTMLButtonElement + ).disabled, + ).toBe(true); + + fireEvent.click(aiTab); + expect( + within(panel).getByRole('textbox', { name: '动画描述' }).textContent, + ).toBe('原地奔跑'); + }); + it('序列帧背景、比例、帧数时长和清晰度可独立选中并保留其他参数', () => { render(); diff --git a/src/components/image-editor/ImageCanvasGenerationComposerView.tsx b/src/components/image-editor/ImageCanvasGenerationComposerView.tsx index b6a07d5dd..c9a848bd5 100644 --- a/src/components/image-editor/ImageCanvasGenerationComposerView.tsx +++ b/src/components/image-editor/ImageCanvasGenerationComposerView.tsx @@ -417,6 +417,9 @@ function ImageCanvasSequenceAnimationComposerView({ const pendingActionFocusIndexRef = useRef(null); const pendingActionFocusTargetRef = useRef(null); const reference = dialog.generationReferences?.[0]; + const workflow = dialog.characterAnimationWorkflow ?? 'ai'; + const conversionReference = + dialog.characterAnimationConversionVideoReference ?? null; const isGenerating = dialog.status === 'generating'; const resolution = dialog.characterAnimationResolution ?? '480p'; const ratio = dialog.characterAnimationRatio ?? '1:1'; @@ -480,179 +483,316 @@ function ImageCanvasSequenceAnimationComposerView({ aria-label="Spine序列帧动画生成" onPointerDown={(event) => event.stopPropagation()} > -
- {reference ? ( - { - // 仅当比例是“同图尺寸”时才回落到 1:1；用户显式选过 4:3/16:9 等 - // 比例时移除参考应保留原选择。 - const isSameRatio = dialog.characterAnimationRatio === 'same'; - updateDialog({ - sourceLayerId: undefined, - generationReferences: [], - ...(isSameRatio ? { characterAnimationRatio: '1:1' } : {}), - }); - }} - /> - ) : null} -
- updateDialog({ prompt })} - />
- {CHARACTER_ANIMATION_BACKGROUND_OPTIONS.map((option) => ( - - ))} -
-
- {CHARACTER_ANIMATION_ACTION_PROMPTS.map((preset) => ( - - ))} - {visibleMoreActions.map((label, index) => ( - - ))} - - {actionPresetAnnouncement} - -
-
- } - onClick={() => { - setIsGenerationReferenceMenuOpen(false); - setIsParameterMenuOpen((open) => !open); - }} - > - {ratio === 'same' ? '同图尺寸' : ratio} · {durationSeconds}秒 ·{' '} - {resolution} - - { - if (!isGenerating) { - onSubmit(); - } - }} + role="tab" + className="image-canvas-editor__workflow-tab" + aria-selected={workflow === 'video-conversion'} + disabled={isGenerating} + onClick={() => + updateDialog({ characterAnimationWorkflow: 'video-conversion' }) + } > - {isGenerating ? '生成中' : `生成${price}泥点`} - + 视频直接转换 +
- {dialog.status === 'failed' ? ( - - {dialog.errorMessage ?? '生成Spine序列帧动画失败'} - - ) : null} + {workflow === 'video-conversion' ? ( + <> +
+ {conversionReference ? ( + + updateDialog({ + characterAnimationConversionVideoReference: null, + }) + } + /> + ) : null} +
+
+ + 清晰度 + +
+ {CHARACTER_ANIMATION_RESOLUTION_OPTIONS.map((value) => ( + + ))} +
+
+ !isGenerating && onSubmit()} + > + {isGenerating ? '转换中' : '免费转换'} + + + ) : ( + <> +
+ {reference ? ( + { + // 仅当比例是“同图尺寸”时才回落到 1:1；用户显式选过 4:3/16:9 等 + // 比例时移除参考应保留原选择。 + const isSameRatio = + dialog.characterAnimationRatio === 'same'; + updateDialog({ + sourceLayerId: undefined, + generationReferences: [], + ...(isSameRatio + ? { characterAnimationRatio: '1:1' } + : {}), + }); + }} + /> + ) : null} + {reference?.mediaType === 'video' + ? (dialog.characterAnimationAppearanceReferences ?? []).map( + (appearanceReference, index) => ( + + updateDialog({ + characterAnimationAppearanceReferences: ( + dialog.characterAnimationAppearanceReferences ?? [] + ).filter( + (item) => item.id !== appearanceReference.id, + ), + }) + } + /> + ), + ) + : null} +
+ updateDialog({ prompt })} + /> +
+ + 背景色 + +
+ {CHARACTER_ANIMATION_BACKGROUND_OPTIONS.map((option) => ( + + ))} +
+
+
+ + 动作 + +
+ {CHARACTER_ANIMATION_ACTION_PROMPTS.map((preset) => ( + + ))} + {visibleMoreActions.map((label, index) => ( + + ))} + + + {actionPresetAnnouncement} + +
+
+
+ } + onClick={() => { + setIsGenerationReferenceMenuOpen(false); + setIsParameterMenuOpen((open) => !open); + }} + > + {ratio === 'same' ? '同图尺寸' : ratio} · {durationSeconds}秒 ·{' '} + {resolution} + + { + if (!isGenerating) { + onSubmit(); + } + }} + > + {isGenerating ? '生成中' : `生成${price}泥点`} + +
+ {dialog.status === 'failed' ? ( + + {dialog.errorMessage ?? '生成Spine序列帧动画失败'} + + ) : null} + + )} {isParameterMenuOpen ? renderEditorPortal( @@ -781,24 +921,29 @@ function ImageCanvasSequenceAnimationComposerView({ > 从画布中选择 - { - setIsGenerationReferenceMenuOpen(false); - setIsPickingGenerationReferenceFromCanvas(false); - onRequestUpload('video-reference-image'); - }} - > - 上传图片 - - { - setIsGenerationReferenceMenuOpen(false); - setIsPickingGenerationReferenceFromCanvas(false); - onRequestUpload('video-reference-video'); - }} - > - 上传视频 - + {workflow !== 'video-conversion' ? ( + { + setIsGenerationReferenceMenuOpen(false); + setIsPickingGenerationReferenceFromCanvas(false); + onRequestUpload('video-reference-image'); + }} + > + 上传图片 + + ) : null} + {workflow === 'video-conversion' || + reference?.mediaType !== 'video' ? ( + { + setIsGenerationReferenceMenuOpen(false); + setIsPickingGenerationReferenceFromCanvas(false); + onRequestUpload('video-reference-video'); + }} + > + 上传视频 + + ) : null} , ) : null} @@ -2224,7 +2369,11 @@ export function ImageCanvasGenerationComposerView({ {isPickingGenerationReferenceFromCanvas ? (
{generateDialog?.mode === 'character-animation' - ? '请选择画布中的角色图或视频作为参考，按 Esc 退出' + ? generateDialog.characterAnimationWorkflow === 'video-conversion' + ? '请选择画布中的视频作为源视频，按 Esc 退出' + : generateDialog.generationReferences?.[0]?.mediaType === 'video' + ? '请选择画布中的图片作为外观参考，按 Esc 退出' + : '请选择画布中的角色图或视频作为参考，按 Esc 退出' : '请选择画布中的图片作为参考图，按 Esc 退出'}
) : null} diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts index 962021fab..ad3a9f28e 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts @@ -1242,7 +1242,6 @@ describe('ImageCanvasGenerationDialogModel', () => { fields: [ { id: 'prompt', title: '角色设定', value: '日系魔法少女' }, { id: 'model', title: '模型', value: 'nanobanana2' }, - { id: 'viewSet', title: '视角集合', value: '八视图' }, { id: 'aspectRatio', title: '比例', value: '1:1' }, { id: 'imageSize', title: '尺寸', value: '1K' }, ], @@ -1259,7 +1258,6 @@ describe('ImageCanvasGenerationDialogModel', () => { sourceLayerId: 'layer-sheet-result', style: 'none', characterSheetEnabled: true, - characterSheetViewSet: '8', }); }); @@ -1484,6 +1482,136 @@ describe('ImageCanvasGenerationDialogModel', () => { }); }); + it('restores video AI motion and appearance from named reference slots', () => { + const motionLayer = createLayer({ + id: 'layer-motion', + resourceId: 'resource-motion', + title: '奔跑动作', + mediaType: 'video', + assetKind: 'video', + src: '/media/motion.mp4', + }); + const appearanceLayer = createLayer({ + id: 'layer-appearance', + resourceId: 'resource-appearance', + title: '骑士外观', + assetKind: 'character', + src: '/images/appearance.png', + }); + const draft = createSameSourceGenerationDialogDraft({ + sourceLayer: createLayer({ + id: 'layer-animation-result', + sourceType: 'generated', + assetKind: 'character-animation', + generationInputs: { + version: 2, + action: 'character-animation.generate', + fields: [ + { id: 'inputMode', title: '输入模式', value: 'video' }, + { id: 'prompt', title: '动作描述', value: '复刻奔跑节奏' }, + { + id: 'referenceSrc', + title: '参考素材', + value: 'users/user-1/motion.mp4', + }, + { + id: 'referenceMediaType', + title: '参考类型', + value: 'video', + }, + { + id: 'appearanceReferenceSrc', + title: '角色外观', + value: 'users/user-1/appearance.png', + }, + ], + references: [ + { + id: 'motion', + title: '动作视频', + refType: 'project-resource', + refId: 'resource-motion', + }, + { + id: 'appearance', + title: '角色外观', + refType: 'project-resource', + refId: 'resource-appearance', + }, + ], + }, + }), + canvasSize: { width: 960, height: 720 }, + viewport: { x: 0, y: 0, scale: 1 }, + mode: 'redraw', + availableLayers: [motionLayer, appearanceLayer], + }); + + expect(draft).toMatchObject({ + mode: 'character-animation', + characterAnimationWorkflow: 'ai', + generationReferences: [ + expect.objectContaining({ + resourceId: 'resource-motion', + mediaType: 'video', + }), + ], + characterAnimationAppearanceReferences: [ + expect.objectContaining({ + resourceId: 'resource-appearance', + label: '骑士外观', + }), + ], + }); + }); + + it('restores video conversion results to the conversion workflow', () => { + const draft = createSameSourceGenerationDialogDraft({ + sourceLayer: createLayer({ + id: 'layer-converted-animation', + sourceType: 'generated', + assetKind: 'character-animation', + generationInputs: { + version: 2, + action: 'character-animation.convert', + fields: [ + { + id: 'sourceVideoSrc', + title: '源视频', + value: 'users/user-1/source.mov', + }, + { id: 'resolution', title: '最大长边', value: '720p' }, + ], + references: [ + { + id: 'source', + title: '源视频', + label: '源动作.mov', + refType: 'asset', + refId: 'asset-source-video', + }, + ], + }, + }), + canvasSize: { width: 960, height: 720 }, + viewport: { x: 0, y: 0, scale: 1 }, + mode: 'redraw', + availableLayers: [], + }); + + expect(draft).toMatchObject({ + mode: 'character-animation', + characterAnimationWorkflow: 'video-conversion', + characterAnimationConversionResolution: '720p', + characterAnimationConversionVideoReference: { + label: '源动作.mov', + src: 'users/user-1/source.mov', + sourceAssetId: 'asset-source-video', + mediaType: 'video', + }, + }); + }); + it('does not restore a project resource id that is unavailable in the current canvas', () => { const draft = createSameSourceGenerationDialogDraft({ sourceLayer: createLayer({ @@ -1875,6 +2003,40 @@ describe('ImageCanvasGenerationDialogModel', () => { }); }); + it('allows an ordinary canvas image as appearance after a video motion reference', () => { + const dialog: GenerateDialogState = { + mode: 'character-animation', + prompt: '原地奔跑', + status: 'idle', + characterAnimationWorkflow: 'ai', + generationReferences: [ + { + id: 'motion', + label: '动作视频', + src: 'generated/motion.mp4', + mediaType: 'video', + }, + ], + }; + const ordinaryImage = createLayer({ + id: 'asset-image', + title: '素材1', + src: 'generated/asset-1.png', + }); + + expect(appendGenerationReference(dialog, ordinaryImage)).toMatchObject({ + generationReferences: [ + expect.objectContaining({ label: '动作视频', mediaType: 'video' }), + ], + characterAnimationAppearanceReferences: [ + { + label: '素材1', + src: 'generated/asset-1.png', + }, + ], + }); + }); + it('caps image references before a sixth reference can enter submission state', () => { const existingReferences = Array.from({ length: 5 }, (_, index) => ({ id: `reference-${index}`, diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts index 51ef28256..51d484269 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts @@ -116,6 +116,12 @@ function getReferenceMediaType(layer: CanvasLayer) { : 'image'; } +function hasCharacterAnimationVideoReference(layer: CanvasLayer) { + return Boolean( + layer.assetKind === 'character-animation' && layer.previewVideoPath?.trim(), + ); +} + export function isIconSpecReferenceLayer(layer: CanvasLayer) { if (layer.assetKind === 'icon-spec') { return true; @@ -317,7 +323,6 @@ export function createCharacterGenerationDialogDraft({ characterSpecReference: null, characterReferences: [], characterSheetEnabled: false, - characterSheetViewSet: '4', imageModel: normalizedImageModel, aspectRatio: dimensionDefaults.aspectRatio, imageSize: dimensionDefaults.imageSize, @@ -861,6 +866,14 @@ function getNormalizedString( return typeof value === 'string' ? value : fallback; } +function getNormalizedStringArray(sourceLayer: CanvasLayer, id: string) { + return (sourceLayer.generationInputs?.fields ?? []) + .filter((field) => field.id === id) + .flatMap((field) => + typeof field.value === 'string' ? [field.value] : [], + ); +} + function getNormalizedNumber( sourceLayer: CanvasLayer, id: string, @@ -1064,7 +1077,7 @@ function createNormalizedGenerationDialogDraft({ } if (inputs.action === 'character-sheet.generate') { - // 多方向角色图回弹到角色面板并恢复图集模式与视角数。 + // 多方向角色图回弹到角色面板并恢复图集模式（固定八方向）。 return placeDraftBesideSourceLayer( resizeGenerationPlaceholderToImageSelection({ ...createCharacterGenerationDialogDraft({ @@ -1082,10 +1095,6 @@ function createNormalizedGenerationDialogDraft({ aspectRatio: getNormalizedString(sourceLayer, 'aspectRatio', '1:1'), imageSize: getNormalizedString(sourceLayer, 'imageSize', '1K'), characterSheetEnabled: true, - characterSheetViewSet: - getNormalizedString(sourceLayer, 'viewSet') === '八视图' - ? '8' - : '4', characterSpecReference: resolveNormalizedReferences( sourceLayer, @@ -1302,12 +1311,19 @@ function createNormalizedGenerationDialogDraft({ } if (inputs.action === 'character-animation.generate') { - const canvasReferences = resolveNormalizedReferences( - sourceLayer, - 'source', - availableLayers, - ); const inputMode = getNormalizedString(sourceLayer, 'inputMode', 'image'); + const primaryReferenceSlot = inputMode === 'video' ? 'motion' : 'source'; + const canvasReferences = [ + ...resolveNormalizedReferences( + sourceLayer, + primaryReferenceSlot, + availableLayers, + ), + // 新槽位上线前视频 AI 也使用 source；仅在 motion 缺失时兼容旧配方。 + ...(inputMode === 'video' && primaryReferenceSlot === 'motion' + ? resolveNormalizedReferences(sourceLayer, 'source', availableLayers) + : []), + ]; const storedReferenceSrc = getNormalizedString( sourceLayer, 'referenceSrc', @@ -1318,7 +1334,9 @@ function createNormalizedGenerationDialogDraft({ inputMode === 'video' ? 'video' : 'image', ); const storedReferenceInput = sourceLayer.generationInputs?.references.find( - (reference) => reference.id === 'source', + (reference) => + reference.id === primaryReferenceSlot || + (inputMode === 'video' && reference.id === 'source'), ); const storedReferenceWidth = getNormalizedNumber( sourceLayer, @@ -1337,7 +1355,7 @@ function createNormalizedGenerationDialogDraft({ id: `restored-character-animation-reference:${sourceLayer.id}`, label: storedReferenceInput?.label ?? - (storedReferenceMediaType === 'video' ? '参考视频' : '角色图片'), + (storedReferenceMediaType === 'video' ? '动作视频' : '角色图片'), src: storedReferenceSrc, objectKey: storedReferenceSrc, ...(storedReferenceInput?.refType === 'asset' @@ -1355,6 +1373,33 @@ function createNormalizedGenerationDialogDraft({ : ('image' as const), } : undefined); + const appearanceCanvasReferences = resolveNormalizedReferences( + sourceLayer, + 'appearance', + availableLayers, + ); + const storedAppearanceSrcs = getNormalizedStringArray( + sourceLayer, + 'appearanceReferenceSrc', + ) + .map((src) => src.trim()) + .filter((src) => src.length > 0); + const storedAppearanceInput = sourceLayer.generationInputs?.references.find( + (reference) => reference.id === 'appearance', + ); + const restoredAppearanceReferences = + appearanceCanvasReferences.length > 0 + ? appearanceCanvasReferences + : storedAppearanceSrcs.map((storedAppearanceSrc, index) => ({ + id: `restored-character-animation-appearance:${sourceLayer.id}:${index}`, + label: storedAppearanceInput?.label ?? '参考图', + src: storedAppearanceSrc, + objectKey: storedAppearanceSrc, + mediaType: 'image' as const, + ...(storedAppearanceInput?.refType === 'asset' + ? { sourceAssetId: storedAppearanceInput.refId } + : {}), + })); const animationSource = restoredReference ? (availableLayers ?? []).find( (layer) => @@ -1373,7 +1418,11 @@ function createNormalizedGenerationDialogDraft({ }); // 帧数/时长必须匹配合法配对（32/4、40/5、48/6），否则 UI 会同时高亮 // 冲突的选项而提交被后端拒绝。这里按合法配对校验，损坏数据回退到默认 32/4。 - const restoredFrameCount = getNormalizedNumber(sourceLayer, 'frameCount', 32); + const restoredFrameCount = getNormalizedNumber( + sourceLayer, + 'frameCount', + 32, + ); const restoredDurationSeconds = getNormalizedNumber( sourceLayer, 'durationSeconds', @@ -1395,6 +1444,10 @@ function createNormalizedGenerationDialogDraft({ prompt, sourceLayerId: animationSource?.id, generationReferences: restoredReference ? [restoredReference] : [], + characterAnimationAppearanceReferences: + restoredAppearanceReferences.length + ? restoredAppearanceReferences + : undefined, characterAnimationBackgroundColor: (getNormalizedString( sourceLayer, 'backgroundColor', @@ -1410,14 +1463,68 @@ function createNormalizedGenerationDialogDraft({ 'ratio', animationSource ? 'same' : '1:1', ) as GenerateDialogState['characterAnimationRatio'], - characterAnimationFrameCount: pairedFrameCount as GenerateDialogState['characterAnimationFrameCount'], - characterAnimationDurationSeconds: pairedDurationSeconds as GenerateDialogState['characterAnimationDurationSeconds'], + characterAnimationFrameCount: + pairedFrameCount as GenerateDialogState['characterAnimationFrameCount'], + characterAnimationDurationSeconds: + pairedDurationSeconds as GenerateDialogState['characterAnimationDurationSeconds'], }), sourceLayer, ) : null; } + if (inputs.action === 'character-animation.convert') { + const sourceReferences = resolveNormalizedReferences( + sourceLayer, + 'source', + availableLayers, + ); + const storedSource = getNormalizedString( + sourceLayer, + 'sourceVideoSrc', + ).trim(); + const storedReferenceInput = sourceLayer.generationInputs?.references.find( + (reference) => reference.id === 'source', + ); + const restoredSource = + sourceReferences[0] ?? + (storedSource + ? { + id: `restored-character-animation-conversion:${sourceLayer.id}`, + label: storedReferenceInput?.label ?? '源视频', + src: storedSource, + objectKey: storedSource, + mediaType: 'video' as const, + ...(storedReferenceInput?.refType === 'asset' + ? { sourceAssetId: storedReferenceInput.refId } + : {}), + } + : undefined); + if (!restoredSource) { + return null; + } + const draft = createCharacterAnimationGenerationDialogDraft({ + canvasSize, + viewport, + }); + return draft + ? placeDraftBesideSourceLayer( + { + ...draft, + sourceLayerId: undefined, + characterAnimationWorkflow: 'video-conversion', + characterAnimationConversionVideoReference: restoredSource, + characterAnimationConversionResolution: getNormalizedString( + sourceLayer, + 'resolution', + '480p', + ) as GenerateDialogState['characterAnimationConversionResolution'], + }, + sourceLayer, + ) + : null; + } + return null; } @@ -2136,24 +2243,52 @@ export function createCharacterAnimationGenerationDialogDraft({ canvasSize, viewport, layer, + workflow = 'ai', }: { canvasSize: CanvasSize; viewport: CanvasViewport; layer?: CanvasLayer; + workflow?: 'ai' | 'video-conversion'; }): Omit | null { - if (layer && layer.assetKind !== 'character') { + if ( + layer && + (workflow === 'video-conversion' + ? layer.mediaType !== 'video' && + !( + layer.assetKind === 'character-animation' && + Boolean(layer.previewVideoPath?.trim()) + ) + : layer.assetKind !== 'character') + ) { return null; } const worldCenter = getViewportWorldCenter({ canvasSize, viewport }); const draft: Omit = { mode: 'character-animation', ...(layer ? { sourceLayerId: layer.id } : {}), - generationReferences: layer ? [createCanvasLayerReference(layer)] : [], + generationReferences: + layer && workflow === 'ai' + ? [ + createCanvasLayerReference(layer, { + preferSequenceVideo: true, + }), + ] + : [], + ...(layer && workflow === 'video-conversion' + ? { + characterAnimationConversionVideoReference: + createCanvasLayerReference(layer, { + preferSequenceVideo: true, + }), + } + : {}), prompt: '', status: 'idle', composerOpen: true, characterAnimationResolution: '480p', - characterAnimationRatio: layer ? 'same' : '1:1', + characterAnimationWorkflow: workflow, + characterAnimationConversionResolution: '480p', + characterAnimationRatio: layer && workflow === 'ai' ? 'same' : '1:1', ...(layer ? {} : { characterAnimationBackgroundColor: 'green' as const }), characterAnimationFrameCount: 32, characterAnimationDurationSeconds: 4, @@ -2220,18 +2355,42 @@ export function appendGenerationReference( } if (dialog?.mode === 'character-animation') { const mediaType = getReferenceMediaType(layer); - if ( - mediaType === 'audio' || - (mediaType === 'image' && - layer.assetKind !== 'character' && - layer.assetKind !== 'character-animation') - ) { + if (dialog.characterAnimationWorkflow === 'video-conversion') { + return mediaType === 'video' || hasCharacterAnimationVideoReference(layer) + ? { + ...resetFailedGenerationDialog(dialog), + sourceLayerId: layer.id, + characterAnimationConversionVideoReference: + createCanvasLayerReference(layer, { + preferSequenceVideo: true, + }), + composerOpen: true, + } + : dialog; + } + const hasVideoMotionReference = + dialog.generationReferences?.[0]?.mediaType === 'video'; + if (mediaType === 'audio') { return dialog; } return resizeGenerationPlaceholderToCharacterAnimationSelection({ ...resetFailedGenerationDialog(dialog), - sourceLayerId: layer.id, - generationReferences: [createCanvasLayerReference(layer)], + ...(mediaType === 'image' && + dialog.generationReferences?.[0]?.mediaType === 'video' + ? { + characterAnimationAppearanceReferences: [ + ...(dialog.characterAnimationAppearanceReferences ?? []), + createCanvasLayerReference(layer), + ], + } + : { + sourceLayerId: layer.id, + generationReferences: [ + createCanvasLayerReference(layer, { + preferSequenceVideo: true, + }), + ], + }), composerOpen: true, }); } diff --git a/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx b/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx index 800eef79a..48459717c 100644 --- a/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx +++ b/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx @@ -44,6 +44,7 @@ type ImageCanvasGenerationImageOptionsViewProps = { onSubmitRequest: () => void; lockedModel?: string; styleControl?: ReactNode; + extraToggles?: ReactNode; renderEditorPortal?: (node: ReactNode) => ReactNode; buildPortalMenuStyle?: ( anchor: HTMLElement | null, @@ -138,6 +139,7 @@ export function ImageCanvasGenerationImageOptionsView({ onSubmitRequest, lockedModel, styleControl, + extraToggles, renderEditorPortal = (node) => node, buildPortalMenuStyle = () => ({}), }: ImageCanvasGenerationImageOptionsViewProps) { @@ -331,20 +333,25 @@ export function ImageCanvasGenerationImageOptionsView({ ) : null} {styleControl ?? - (supportsImageStyle ? ( - + (supportsImageStyle || extraToggles ? ( +
+ {supportsImageStyle ? ( + + ) : null} + {extraToggles} +
) : null)} {includeModel ? (
([ 'audio.sound-effect.generate', 'audio.background-music.generate', 'character-animation.generate', + 'character-animation.convert', 'ui-design.extract-assets', ]); diff --git a/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts b/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts index 9d5e8a4ef..39eba3382 100644 --- a/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts @@ -558,6 +558,7 @@ describe('ImageCanvasGenerationLayerModel', () => { ]; const layer = createCharacterAnimationResultLayer({ generated: { + resultType: 'ai', taskId: 'character-animation-task-1', model: 'seedance2.0-fast', prompt: '待机动作', @@ -605,7 +606,7 @@ describe('ImageCanvasGenerationLayerModel', () => { mediaType: 'image-sequence', thumbnailSrc: '/generated-character-drafts/editor/frame1.png', imageSequenceFrames: frames, - previewVideoPath: null, + previewVideoPath: '/generated-character-drafts/editor/preview.mp4', x: -222, y: -242, width: 1024, @@ -625,6 +626,61 @@ describe('ImageCanvasGenerationLayerModel', () => { expect(layer?.provider).toBeUndefined(); }); + it('creates direct video conversion layers without fake prompt or model metadata', () => { + const frames = [ + { imageSrc: '/converted/frame-1.png', width: 640, height: 360 }, + { imageSrc: '/converted/frame-2.png', width: 640, height: 360 }, + ]; + const layer = createCharacterAnimationResultLayer({ + generated: { + resultType: 'video-conversion', + taskId: 'conversion-task-1', + processingMethod: '视频直接转换', + frames, + frameCount: 2, + durationMs: 2_500, + frameWidth: 640, + frameHeight: 360, + sourceFps: 29.97, + samplingFps: 0.8, + priceMudPoints: 0, + resource: { + resourceId: 'resource-converted-animation', + projectId: 'project-1', + imageSrc: frames[0]!.imageSrc, + width: 640, + height: 360, + sourceType: 'generated', + assetKind: 'character-animation', + imageSequenceFrames: frames, + imageSequenceDurationMs: 2_500, + }, + }, + generatedIndex: 18, + title: '视频转换序列帧', + canvasSize: { width: 900, height: 640 }, + viewport: { x: 0, y: 0, scale: 1 }, + generationInputs: { + version: 2, + action: 'character-animation.convert', + fields: [], + references: [], + }, + }); + + expect(layer).toMatchObject({ + mediaType: 'image-sequence', + assetKind: 'character-animation', + taskId: 'conversion-task-1', + provider: 'FFmpeg', + imageSequenceDurationMs: 2_500, + }); + expect(layer?.prompt).toBeUndefined(); + expect(layer?.actualPrompt).toBeUndefined(); + expect(layer?.model).toBeUndefined(); + expect(layer?.previewVideoPath).toBeNull(); + }); + it('links a character animation result layer to its persisted project resource', () => { const frames = [ { @@ -644,6 +700,7 @@ describe('ImageCanvasGenerationLayerModel', () => { ]; const layer = createCharacterAnimationResultLayer({ generated: { + resultType: 'ai', taskId: 'character-animation-task-persisted', model: 'seedance2.0-fast', prompt: '奔跑动作', @@ -704,6 +761,7 @@ describe('ImageCanvasGenerationLayerModel', () => { ]; const layer = createCharacterAnimationResultLayer({ generated: { + resultType: 'ai', taskId: 'character-animation-task-asset-only', model: 'seedance2.0-fast', prompt: '跳跃动作', diff --git a/src/components/image-editor/ImageCanvasGenerationLayerModel.ts b/src/components/image-editor/ImageCanvasGenerationLayerModel.ts index 5dc1e1a31..9302447e5 100644 --- a/src/components/image-editor/ImageCanvasGenerationLayerModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationLayerModel.ts @@ -1,6 +1,6 @@ import type { EditorAudioGenerationResult, - EditorCharacterAnimationGenerationResult, + EditorCharacterAnimationResult, EditorIconSpritesheetGenerationResult, EditorIconSpritesheetIconResult, EditorImageGenerationResult, @@ -66,7 +66,7 @@ type AudioResultLayerOptions = { }; type CharacterAnimationResultLayerOptions = { - generated: EditorCharacterAnimationGenerationResult; + generated: EditorCharacterAnimationResult; generatedIndex: number; title: string; canvasSize: CanvasSize; @@ -497,7 +497,16 @@ export function createCharacterAnimationResultLayer({ mediaType: 'image-sequence', thumbnailSrc: previewFrame.imageSrc, imageSequenceFrames, - previewVideoPath: null, + previewVideoPath: + generated.resultType === 'ai' + ? generated.previewVideoPath + : sourceLayer?.mediaType === 'video' + ? sourceLayer.objectKey?.trim() || + sourceLayer.previewVideoPath || + sourceLayer.src + : sourceLayer?.assetKind === 'character-animation' + ? (sourceLayer.previewVideoPath ?? null) + : null, x: frameX ?? worldCenter.x - width / 2, y: frameY ?? worldCenter.y - height / 2, width, @@ -506,9 +515,11 @@ export function createCharacterAnimationResultLayer({ originalHeight, zIndex: generatedIndex + 10, sourceType: 'generated', - prompt: generated.prompt, - actualPrompt: generated.prompt, - model: generated.model, + prompt: generated.resultType === 'ai' ? generated.prompt : undefined, + actualPrompt: generated.resultType === 'ai' ? generated.prompt : undefined, + model: generated.resultType === 'ai' ? generated.model : undefined, + provider: + generated.resultType === 'video-conversion' ? 'FFmpeg' : undefined, taskId: generated.taskId, assetKind: 'character-animation', imageSequenceDurationMs, diff --git a/src/components/image-editor/ImageCanvasGenerationModel.test.ts b/src/components/image-editor/ImageCanvasGenerationModel.test.ts index a7e77c300..f7219d1e9 100644 --- a/src/components/image-editor/ImageCanvasGenerationModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationModel.test.ts @@ -31,6 +31,7 @@ import { calculateEditorUiDesignPrice, calculateEditorVideoPrice, canOpenRedrawPanel, + createCanvasLayerReference, CANVAS_GENERATION_PARAMETER_FALLBACK_WARNING, CHARACTER_ANIMATION_MODEL, decodeCanvasGenerationInputs, @@ -64,6 +65,37 @@ import { } from './ImageCanvasGenerationModel'; describe('ImageCanvasGenerationModel', () => { + it('uses a character animation preview video when a video reference is requested', () => { + const reference = createCanvasLayerReference( + { + id: 'layer-sequence', + resourceId: 'resource-sequence', + sourceResourceId: 'resource-motion-video', + title: '序列帧动作', + src: '/frames/frame-001.png', + previewVideoPath: '/motion/preview.mp4', + mediaType: 'image-sequence', + assetKind: 'character-animation', + x: 0, + y: 0, + width: 480, + height: 480, + originalWidth: 480, + originalHeight: 480, + zIndex: 1, + sourceType: 'generated', + }, + { preferSequenceVideo: true }, + ); + + expect(reference).toMatchObject({ + mediaType: 'video', + src: '/motion/preview.mp4', + objectKey: '/motion/preview.mp4', + resourceId: 'resource-motion-video', + }); + }); + it('resolves character animation pixel and display sizes from ratio and resolution', () => { expect( resolveEditorCharacterAnimationFramePixelSize({ @@ -117,6 +149,59 @@ describe('ImageCanvasGenerationModel', () => { ).toBeUndefined(); }); + it('persists motion and appearance slots for video-guided character animation', () => { + const inputs = buildCharacterAnimationGenerationInputs( + '复刻动作并替换外观', + null, + { + id: 'motion-video', + label: '奔跑动作.mov', + src: 'users/user-1/motion.mov', + mediaType: 'video', + resourceId: 'resource-motion', + width: 1920, + height: 1080, + }, + { + referenceSrc: 'users/user-1/motion.mov', + referenceMediaType: 'video', + appearanceReferences: [ + { + id: 'appearance-image', + label: '骑士外观.png', + src: 'users/user-1/appearance.png', + mediaType: 'image', + sourceAssetId: 'asset-appearance', + }, + ], + appearanceReferenceSrcs: ['users/user-1/appearance.png'], + }, + ); + + expect(inputs.references).toEqual([ + expect.objectContaining({ + id: 'motion', + refType: 'project-resource', + refId: 'resource-motion', + }), + expect.objectContaining({ + id: 'appearance', + refType: 'asset', + refId: 'asset-appearance', + }), + ]); + expect(inputs.fields).toEqual( + expect.arrayContaining([ + { id: 'inputMode', title: '输入模式', value: 'video' }, + { + id: 'appearanceReferenceSrc', + title: '参考图', + value: 'users/user-1/appearance.png', + }, + ]), + ); + }); + afterEach(() => { applyEditorGenerationPricingConfig({ models: EDITOR_MODEL_MUD_POINT_CONFIG, diff --git a/src/components/image-editor/ImageCanvasGenerationModel.ts b/src/components/image-editor/ImageCanvasGenerationModel.ts index 0edba945b..381b84022 100644 --- a/src/components/image-editor/ImageCanvasGenerationModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationModel.ts @@ -913,6 +913,8 @@ export function buildCharacterAnimationGenerationInputs( backgroundColor?: string; referenceSrc?: string; referenceMediaType?: 'image' | 'video'; + appearanceReferences?: CharacterReferenceImage[]; + appearanceReferenceSrcs?: string[]; } = {}, ): CanvasGenerationInputs { const inputMode = @@ -967,6 +969,15 @@ export function buildCharacterAnimationGenerationInputs( 'referenceHeight', ) : []), + ...(options.appearanceReferenceSrcs?.length + ? options.appearanceReferenceSrcs.flatMap((src) => + createGenerationInputField( + '参考图', + src, + 'appearanceReferenceSrc', + ), + ) + : []), ...createGenerationInputField('动作描述', promptText, 'prompt'), ...(options.backgroundColor ? createGenerationInputField( @@ -992,19 +1003,28 @@ export function buildCharacterAnimationGenerationInputs( 'durationSeconds', ), ], - references: sourceLayer - ? createLayerGenerationInputReference( - reference?.mediaType === 'video' ? '参考视频' : '角色图片', - sourceLayer, - { id: 'source' }, - ) - : reference + references: [ + ...(reference ? createGenerationInputReference( - reference.mediaType === 'video' ? '参考视频' : '角色图片', + reference.mediaType === 'video' ? '参考视频' : '参考图片', reference, - { id: 'source' }, + { id: reference.mediaType === 'video' ? 'motion' : 'source' }, ) - : [], + : sourceLayer + ? createLayerGenerationInputReference('角色图片', sourceLayer, { + id: 'source', + }) + : []), + ...(options.appearanceReferences?.length + ? options.appearanceReferences.flatMap((appearanceReference) => + createGenerationInputReference( + '参考图', + appearanceReference, + { id: 'appearance' }, + ), + ) + : []), + ], }; } @@ -1264,19 +1284,33 @@ export function resolveCharacterAnimationSourceImageSrc(layer: CanvasLayer) { export function createCanvasLayerReference( layer: CanvasLayer, + options: { preferSequenceVideo?: boolean } = {}, ): CharacterReferenceImage { - const mediaType = - layer.mediaType === 'video' || layer.mediaType === 'audio' - ? layer.mediaType + // 正式序列帧资源的 PNG 是最终交付物；在动作视频/直接转换槽位中，若仍保留底层视频， + // 则沿用视频引用。未指定时保持旧的整张序列帧图片语义。 + const sequenceVideoSrc = + options.preferSequenceVideo && layer.assetKind === 'character-animation' + ? layer.previewVideoPath?.trim() : undefined; + const isVideoReference = + Boolean(sequenceVideoSrc) || layer.mediaType === 'video'; + const mediaType = isVideoReference + ? 'video' + : layer.mediaType === 'audio' + ? 'audio' + : undefined; + const src = sequenceVideoSrc || layer.objectKey?.trim() || layer.src; return { id: `canvas-${layer.id}`, label: layer.title, - src: layer.objectKey?.trim() || layer.src, + src, ...(mediaType ? { mediaType } : {}), - objectKey: layer.objectKey ?? undefined, + assetKind: layer.assetKind ?? null, + objectKey: sequenceVideoSrc || (layer.objectKey ?? undefined), assetObjectId: layer.assetObjectId ?? undefined, - resourceId: layer.resourceId, + resourceId: sequenceVideoSrc + ? (layer.sourceResourceId ?? layer.resourceId) + : layer.resourceId, sourceAssetId: layer.sourceAssetId ?? undefined, width: layer.originalWidth, height: layer.originalHeight, @@ -1340,7 +1374,8 @@ export function buildDeterministicGenerationInputs( | 'image.perfect-pixel' | 'spritesheet.split' | 'image.remove-background' - | 'image.crop-expand', + | 'image.crop-expand' + | 'character-animation.remove-background', sourceLayer: CanvasLayer, sourceTitle = '原图', ): CanvasGenerationInputs { @@ -2182,7 +2217,6 @@ export function buildCharacterSheetGenerationInputs( model?: string; aspectRatio?: string; imageSize?: string; - viewSet?: '4' | '8'; } = {}, ): CanvasGenerationInputs { return { @@ -2195,11 +2229,6 @@ export function buildCharacterSheetGenerationInputs( options.model ?? DEFAULT_IMAGE_MODEL, 'model', ), - ...createGenerationInputField( - '视角集合', - options.viewSet === '8' ? '八视图' : '四视图', - 'viewSet', - ), ...createGenerationInputField( '比例', options.aspectRatio ?? '1:1', diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts index bcd432090..53cbcb1ec 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts @@ -680,12 +680,12 @@ describe('ImageCanvasGenerationSubmissionModel', () => { action: 'character-sheet.generate', fields: expect.arrayContaining([ { id: 'prompt', title: '角色设定', value: '日系魔法少女' }, - { id: 'viewSet', title: '视角集合', value: '四视图' }, ]), }), ); expect(plan.kind === 'image' ? plan.input : null).toMatchObject({ - characterSheetViews: 4, + kind: 'character-sheet', + assetKind: 'icon-spritesheet', }); // 图集不做像素化、不去背景，screenColor/segModel 不携带。 expect(plan.kind === 'image' ? plan.input : null).not.toHaveProperty( @@ -696,33 +696,6 @@ describe('ImageCanvasGenerationSubmissionModel', () => { ); }); - it('builds eight-direction character-sheet plans when view set is eight', () => { - const plan = buildImageGenerationSubmissionPlan({ - dialog: { - mode: 'character', - prompt: '日系魔法少女', - status: 'idle', - characterSheetEnabled: true, - characterSheetViewSet: '8', - }, - layers: [], - nextGeneratedIndex: 4, - }); - - expect(plan.kind === 'image' ? plan.input : null).toMatchObject({ - kind: 'character-sheet', - characterSheetViews: 8, - }); - expect(plan.kind === 'image' ? plan.result.generationInputs : null).toEqual( - expect.objectContaining({ - action: 'character-sheet.generate', - fields: expect.arrayContaining([ - { id: 'viewSet', title: '视角集合', value: '八视图' }, - ]), - }), - ); - }); - it('locks UI design generation submissions to gpt-image-2', () => { const plan = buildImageGenerationSubmissionPlan({ dialog: { diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts index a825ef917..73a8f3be7 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts @@ -802,8 +802,7 @@ export function buildImageGenerationSubmissionPlan({ const screenColor = DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR; const segModel = DEFAULT_EDITOR_BGFILTER_SEG_MODEL; const sheetEnabled = dialog.characterSheetEnabled === true; - const sheetViewSet = dialog.characterSheetViewSet ?? '4'; - // 中文注释：多方向角色图走后端 kind=character-sheet（绿色间隔底，便于连通块拆分）， + // 中文注释：多方向角色图走后端 kind=character-sheet（绿色间隔底，便于后续拆分）， // 产物按 icon-spritesheet 落库，复用既有「拆分图集」能力；不做像素化、不去背景。 return { kind: 'image', @@ -823,7 +822,6 @@ export function buildImageGenerationSubmissionPlan({ ...(sheetEnabled ? { assetKind: 'icon-spritesheet', - characterSheetViews: sheetViewSet === '8' ? 8 : 4, } : {}), ...(referenceImageSrcs.length ? { referenceImageSrcs } : {}), @@ -845,7 +843,6 @@ export function buildImageGenerationSubmissionPlan({ model: imageModel, aspectRatio: dialog.aspectRatio ?? '1:1', imageSize: dialog.imageSize ?? '1K', - viewSet: sheetViewSet, }, ) : buildCharacterGenerationInputs( @@ -1183,8 +1180,8 @@ export function buildCharacterAnimationSubmissionPlan({ }): CharacterAnimationSubmissionPlan { const promptText = panel.promptText.trim(); const resultTitle = resolveGenerationAssetLabel(panel.assetLabel, '角色动作'); - if (!promptText) { - throw new Error('动画描述不能为空。'); + if (!promptText && !reference && !sourceLayer) { + throw new Error('动画描述和参考素材不能同时为空。'); } const inputMode = reference?.mediaType === 'video' diff --git a/src/components/image-editor/ImageCanvasReferenceSlot.tsx b/src/components/image-editor/ImageCanvasReferenceSlot.tsx index 849f48311..f0304f6e2 100644 --- a/src/components/image-editor/ImageCanvasReferenceSlot.tsx +++ b/src/components/image-editor/ImageCanvasReferenceSlot.tsx @@ -11,8 +11,7 @@ type ImageCanvasReferenceSlotTone = | 'character' | 'quick-edit' | 'video' - | 'audio' - | 'character-sheet'; + | 'audio'; type ImageCanvasReferenceSlotProps = { buttonRef?: RefObject; @@ -26,8 +25,6 @@ type ImageCanvasReferenceSlotProps = { objectKey?: string | null; disabled?: boolean; isAdd?: boolean; - /** 选中态（用于模式开关类入口）：置位时高亮边框并带 aria-pressed。 */ - pressed?: boolean; onClick?: () => void; onRemove?: () => void; removeLabel?: string; @@ -45,7 +42,6 @@ export function ImageCanvasReferenceSlot({ objectKey, disabled = false, isAdd = false, - pressed, onClick, onRemove, removeLabel, @@ -58,11 +54,11 @@ export function ImageCanvasReferenceSlot({ ? 'image-canvas-editor__reference-chip--has-image' : 'image-canvas-editor__reference-chip--icon-only', isAdd ? 'image-canvas-editor__reference-chip--upload' : null, - pressed ? 'image-canvas-editor__reference-chip--active' : null, className, ] .filter(Boolean) - .join(' '); const slotContent = ( + .join(' '); + const slotContent = ( <> {hasPreviewImage ? ( @@ -93,7 +89,6 @@ export function ImageCanvasReferenceSlot({ className={slotClassName} disabled={disabled} aria-label={ariaLabel} - aria-pressed={pressed === undefined ? undefined : pressed} title={title ?? ariaLabel} onClick={onClick} > diff --git a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx index 35f08ecec..5cd3caa5c 100644 --- a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx +++ b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx @@ -54,11 +54,12 @@ function renderSelectedToolbar( isPerfectPixelPendingConfirmation: false, isSplittingIconSpritesheet: false, isSplittingCharacterAnimation: false, + isRemovingCharacterAnimationBackground: false, isPersistingAssetKind: false, onSplitIconSpritesheet: vi.fn(), onSplitCharacterAnimation: vi.fn(), + onRemoveCharacterAnimationBackground: vi.fn(), onExtractUiDesignAssets: vi.fn(), - onOpenCharacterAnimationPanel: vi.fn(), onDownloadLayer: vi.fn(), ...overrides, }; @@ -478,8 +479,10 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { within(actionToolbar) .getAllByRole('button') .map((button) => button.getAttribute('aria-label')), - ).toEqual(['拆帧', '改造', '下载按钮']); - fireEvent.click(within(actionToolbar).getByRole('button', { name: '拆帧' })); + ).toEqual(['改造', '去背景', '拆帧', '下载按钮']); + fireEvent.click( + within(actionToolbar).getByRole('button', { name: '拆帧' }), + ); expect(props.onSplitCharacterAnimation).toHaveBeenCalledWith( expect.objectContaining({ assetKind: 'character-animation' }), ); @@ -557,28 +560,6 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { expect(uiProps.onOpenRedrawPanel).toHaveBeenCalledWith(uiDesignLayer); }); - it('renders character animation only for character layers', () => { - const layer = createLayer({ - sourceType: 'generated', - assetKind: 'character', - }); - const props = renderSelectedToolbar({ selectedLayer: layer }); - const animationButton = screen.getByRole('button', { name: '生成动画' }); - - expect(animationButton.textContent).toContain('生成动画'); - expect( - animationButton.querySelector('.lucide-person-standing'), - ).toBeTruthy(); - - fireEvent.click(animationButton); - - expect(props.onOpenCharacterAnimationPanel).toHaveBeenCalledWith(layer); - - cleanup(); - renderSelectedToolbar({ selectedLayer: createLayer({ id: 'layer-2' }) }); - expect(screen.queryByRole('button', { name: '生成动画' })).toBeNull(); - }); - it('hides quick edit only for individual icon assets', () => { renderSelectedToolbar({ selectedLayer: createLayer({ assetKind: 'icon' }), @@ -618,11 +599,12 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { isPerfectPixelPendingConfirmation={false} isSplittingIconSpritesheet={false} isSplittingCharacterAnimation={false} + isRemovingCharacterAnimationBackground={false} isPersistingAssetKind={false} onSplitIconSpritesheet={vi.fn()} onSplitCharacterAnimation={vi.fn()} + onRemoveCharacterAnimationBackground={vi.fn()} onExtractUiDesignAssets={vi.fn()} - onOpenCharacterAnimationPanel={vi.fn()} onDownloadLayer={vi.fn()} />, ); @@ -642,11 +624,12 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { isPerfectPixelPendingConfirmation={false} isSplittingIconSpritesheet={false} isSplittingCharacterAnimation={false} + isRemovingCharacterAnimationBackground={false} isPersistingAssetKind={false} onSplitIconSpritesheet={vi.fn()} onSplitCharacterAnimation={vi.fn()} + onRemoveCharacterAnimationBackground={vi.fn()} onExtractUiDesignAssets={vi.fn()} - onOpenCharacterAnimationPanel={vi.fn()} onDownloadLayer={vi.fn()} />, ); diff --git a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx index 63bde75c4..352731275 100644 --- a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx +++ b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx @@ -4,7 +4,6 @@ import { Grid2X2, ImageOff, Loader2, - PersonStanding, Scissors, Sparkles, WandSparkles, @@ -32,11 +31,12 @@ type ImageCanvasSelectedLayerToolbarViewProps = { isPerfectPixelPendingConfirmation: boolean; isSplittingIconSpritesheet: boolean; isSplittingCharacterAnimation: boolean; + isRemovingCharacterAnimationBackground: boolean; isPersistingAssetKind: boolean; onSplitIconSpritesheet: (layer: CanvasLayer) => void; onSplitCharacterAnimation: (layer: CanvasLayer) => void; + onRemoveCharacterAnimationBackground: (layer: CanvasLayer) => void; onExtractUiDesignAssets: (layer: CanvasLayer) => void; - onOpenCharacterAnimationPanel: (layer: CanvasLayer) => void; onDownloadLayer: (layer: CanvasLayer) => void; }; @@ -52,11 +52,12 @@ export function ImageCanvasSelectedLayerToolbarView({ isPerfectPixelPendingConfirmation, isSplittingIconSpritesheet, isSplittingCharacterAnimation, + isRemovingCharacterAnimationBackground, isPersistingAssetKind, onSplitIconSpritesheet, onSplitCharacterAnimation, + onRemoveCharacterAnimationBackground, onExtractUiDesignAssets, - onOpenCharacterAnimationPanel, onDownloadLayer, }: ImageCanvasSelectedLayerToolbarViewProps) { if (!selectedLayer || !selectedToolbarStyle) { @@ -157,7 +158,7 @@ export function ImageCanvasSelectedLayerToolbarView({ ? '完美像素处理中' : isPerfectPixelPendingConfirmation ? '完美像素结果待确认' - : '完美像素' + : '完美像素' } title={ isPersistingAssetKind @@ -166,7 +167,7 @@ export function ImageCanvasSelectedLayerToolbarView({ ? '完美像素处理中' : isPerfectPixelPendingConfirmation ? '结果待确认，双击原占位继续核对或重试' - : '自动识别并规整像素网格' + : '自动识别并规整像素网格' } icon={ isPersistingAssetKind || isPerfectPixelProcessing ? ( @@ -194,7 +195,7 @@ export function ImageCanvasSelectedLayerToolbarView({ ? '处理中' : isPerfectPixelPendingConfirmation ? '待确认' - : '完美像素'} + : '完美像素'} @@ -247,55 +248,71 @@ export function ImageCanvasSelectedLayerToolbarView({ 提取素材 ) : null} - {selectedLayer.assetKind === 'character-animation' ? ( - - ) : ( - - ) - } - disabled={ - isCharacterAnimationResourcePending || isSplittingCharacterAnimation - } - aria-busy={ - isCharacterAnimationResourcePending || isSplittingCharacterAnimation - } - onClick={() => onSplitCharacterAnimation(selectedLayer)} - > - {characterAnimationSplitLabel} - - ) : null} - {selectedLayer.assetKind === 'character' ? ( - } - onClick={() => onOpenCharacterAnimationPanel(selectedLayer)} - > - 生成动画 - - ) : null} {canRedraw ? ( + } + onClick={() => onOpenRedrawPanel(selectedLayer)} + > + 改造 + + ) : null} + {selectedLayer.assetKind === 'character-animation' ? ( <> -