From 1575afc92c7d8e27c718aa9df82b6270e30c68ef Mon Sep 17 00:00:00 2001 From: suzmii Date: Mon, 10 Aug 2026 20:14:09 +0800 Subject: [PATCH 01/15] =?UTF-8?q?Spine=E5=BA=8F=E5=88=97=E5=B8=A7=E5=8A=A8?= =?UTF-8?q?=E7=94=BB=E5=A4=9A=E6=A8=A1=E6=80=81=E7=94=9F=E6=88=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/README.md | 1 + docs/project-memory/shared-memory/pitfalls.md | 9 + ...ArtAgent多模态Spine序列帧动画生成-2026-08-10.md | 829 ++++++++ .../src/contracts/editorCharacterAnimation.ts | 12 + packages/shared/src/index.ts | 1 + server-rs/Cargo.lock | 1 + server-rs/crates/api-server/src/app.rs | 25 + .../src/character_animation_assets.rs | 1784 +++++++++++++---- .../crates/api-server/src/editor_project.rs | 38 +- .../src/editor_sequence_animation.rs | 1201 +++++++++++ .../api-server/src/external_editor_api.rs | 123 +- server-rs/crates/api-server/src/main.rs | 1 + .../src/modules/platform_support.rs | 7 + server-rs/crates/module-assets/Cargo.toml | 1 + .../module-assets/src/character_animation.rs | 95 + server-rs/crates/module-assets/src/lib.rs | 5 + .../crates/shared-contracts/src/assets.rs | 97 + .../src/editor_project_storage.rs | 79 +- .../ImageCanvasBottomToolbarView.test.tsx | 37 + .../ImageCanvasBottomToolbarView.tsx | 19 +- ...CanvasEditorGenerationIntegration.test.tsx | 94 +- .../ImageCanvasEditorModel.test.ts | 4 + .../image-editor/ImageCanvasEditorModel.ts | 8 + .../ImageCanvasEditorShellView.test.tsx | 1 + .../image-editor/ImageCanvasEditorTypes.ts | 9 +- .../image-editor/ImageCanvasEditorView.tsx | 33 +- ...ImageCanvasGenerationComposerView.test.tsx | 407 +++- .../ImageCanvasGenerationComposerView.tsx | 460 ++++- .../ImageCanvasGenerationDialogModel.test.ts | 161 +- .../ImageCanvasGenerationDialogModel.ts | 145 +- .../ImageCanvasGenerationModel.test.ts | 55 + .../ImageCanvasGenerationModel.ts | 260 ++- ...ageCanvasGenerationSubmissionModel.test.ts | 94 +- .../ImageCanvasGenerationSubmissionModel.ts | 57 +- ...ageCanvasSelectedLayerToolbarView.test.tsx | 33 +- .../ImageCanvasSelectedLayerToolbarView.tsx | 37 + .../ImageCanvasSequenceAnimationIcon.tsx | 34 + .../image-editor/ImageCanvasStageView.tsx | 9 + .../ImageCanvasUploadModel.test.ts | 42 + .../image-editor/ImageCanvasUploadModel.ts | 36 +- .../ImageCanvasWorldView.test.tsx | 33 +- .../image-editor/ImageCanvasWorldView.tsx | 10 +- ...useImageCanvasAssetExportWorkflow.test.tsx | 6 +- ...anvasGenerationSubmissionWorkflow.test.tsx | 54 +- ...ImageCanvasGenerationSubmissionWorkflow.ts | 94 +- .../useImageCanvasGenerationSurface.test.tsx | 17 + .../useImageCanvasGenerationSurface.tsx | 4 + .../useImageCanvasGenerationWorkflow.test.tsx | 303 ++- .../useImageCanvasGenerationWorkflow.ts | 178 +- .../useImageCanvasStageInteractions.test.tsx | 41 +- .../useImageCanvasStageInteractions.ts | 3 +- .../useImageCanvasUploadWorkflow.test.tsx | 402 +++- .../useImageCanvasUploadWorkflow.ts | 355 +++- src/index.css | 299 ++- .../image-editor/editorProjectClient.ts | 50 +- 55 files changed, 7398 insertions(+), 795 deletions(-) create mode 100644 docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md create mode 100644 packages/shared/src/contracts/editorCharacterAnimation.ts create mode 100644 server-rs/crates/api-server/src/editor_sequence_animation.rs create mode 100644 server-rs/crates/module-assets/src/character_animation.rs create mode 100644 src/components/image-editor/ImageCanvasSequenceAnimationIcon.tsx diff --git a/docs/README.md b/docs/README.md index e571dde87..9cfb5eead 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,6 +20,7 @@ - [图片画布编辑器 MVP 接入方案](./technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md) - [图片画布编辑器前端拆分计划](./technical/【前端架构】图片画布编辑器前端拆分计划-2026-06-17.md) - [图片画布游戏场景生成链路](./technical/【技术方案】图片画布游戏场景生成链路-2026-08-04.md) +- [Art Agent 多模态 Spine 序列帧动画生成](./technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md) - [画板音乐生成入口设计](./【编辑器】画板音乐生成入口设计-2026-06-18.md) - [SFX 生成优化 V2.0 任务拆解](./project-memory/plans/【实施计划】SFX生成优化V2.0任务拆解-2026-08-06.md) - [SFX 生成优化 V2.0 T6 测试与发布门禁](./【实施记录】SFX生成优化V2.0T6测试与发布门禁-2026-08-07.md) diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index f3a638797..189c1a34b 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -472,6 +472,8 @@ - 现象:产品要求画板 `生成角色动作` 返回后按透明序列帧播放和下载,但旧实现或旧测试可能继续把结果当作预览视频处理。 - 原因:后端仍需要先生成 `previewVideoPath` 再抽帧、绿幕去背和落 OSS;如果前端把预览视频当主媒体,就会绕过已经扣绿幕的 PNG 帧,也无法按序列帧打包下载。 - 处理:角色动作结果图层主 `src` 使用 `frames[0].imageSrc`,`mediaType` 固定为 `image-sequence`,`assetKind` 固定为 `character-animation`,完整帧列表写入 `imageSequenceFrames`,`previewVideoPath` 只作为来源信息保留。生成端确认每帧对象后必须把该帧 `objectKey` 与 `assetObjectId` 一起写入正式 payload 和 `generation_inputs_json.characterAnimation.frames`。单图层下载必须生成序列帧 ZIP;画布素材 ZIP 中角色动作写入 `sequences/<编号-标题>/frames/`。不得移除后端原有视频生成、抽帧、绿幕去背和帧落盘流程。 +- 拆帧:拆帧只把正式帧对象登记成独立项目资源和账号素材,必须复用每帧已有的 `assetObjectId/objectKey`,禁止重新下载再上传 PNG。批次继续复用现有 spritesheet slice 批量事务,原子写入全部帧并完成 cohort;拆帧 task/cohort 必须按 owner + `sourceResourceId` + 规范化目标目录与标签稳定隔离,不能直接复用原生成 task,否则同一素材进入多个项目后会把多套帧归入一个错误 cohort。原帧 `asset_object.source_job_id` 继续保留原生成来源,不能为了匹配拆帧 task 伪造血缘。结果只进入素材库,不自动创建几十个画布图层。 +- 多模态参考:`inputMode/referenceMediaType` 只是客户端意图,不是媒体事实。图片或视频参考必须先按 owner 解析到权威 objectKey,再在扣费和 provider 调用前使用 OSS HEAD、权威 `asset_object.content_type` 或实际媒体探测验证 `image/*` / `video/*`;不能把上游报错和退款当作后端媒体校验。 - 验证:`ImageCanvasGenerationLayerModel` 应断言动作结果 `src` 为首帧且 `mediaType="image-sequence"`;画布集成测试应出现 `画布序列帧:角色动作` 图片播放器,不应出现角色动作 `