From 097170836d184ded166bbe2e51faa06bf31905e5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E9=AB=98=E7=89=A9?= <253518756@qq.com> Date: Fri, 19 Jun 2026 10:09:19 +0800 Subject: [PATCH] Add UI-design asset extraction & spritesheet support MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Introduce a new UI design asset extraction flow: add POST /api/editor/ui-designs/assets/extractions handler that always uses gpt-image-2 with the fixed prompt "提取画面中的所有独立并整理成spritesheet"; parse Data URL references, call OpenAI image edits, return a spritesheet and sliced icon assets. Increase request body limit for image Data URLs and wire the route with bearer auth. Add platform-image slicing: slice_generated_icon_spritesheet_all_by_connected_components (with auto-naming "素材 1..N", foreground estimation and heuristics) and export it. Add Rust tests for the handler, prompt constant, and slicing behavior. Update frontend client and integration tests to call the new extractEditorUiDesignAssets API and verify spritesheet + icon layers are added to the canvas and persisted. Update multiple docs to describe the extraction UI, toolbar placement, and related audio/playback pitfalls. Minor related updates: toolbar button ordering and audio duration / signed read-url notes. --- .../shared-memory/decision-log.md | 8 + docs/project-memory/shared-memory/pitfalls.md | 8 + ...架构】图片画布编辑器MVP接入方案-2026-06-11.md | 3 +- ...】生成类面板Lovart统一改造方案-2026-06-17.md | 3 +- ...辑器】画板UI设计图生成入口设计-2026-06-17.md | 14 ++ ...辑器】画板图标素材生成入口设计-2026-06-15.md | 10 +- ...编辑器】画板音乐生成入口设计-2026-06-18.md | 11 +- server-rs/crates/api-server/src/app.rs | 43 +++++ .../crates/api-server/src/editor_project.rs | 98 +++++++++- .../api-server/src/modules/editor_project.rs | 40 +++- .../src/generated_asset_sheets/mod.rs | 1 + .../src/generated_asset_sheets/sheet.rs | 182 ++++++++++++++++-- ...CanvasEditorGenerationIntegration.test.tsx | 99 ++++++++++ .../ImageCanvasEditorModel.test.ts | 35 ++-- .../image-editor/ImageCanvasEditorModel.ts | 23 +++ .../ImageCanvasEditorShellView.test.tsx | 1 + .../image-editor/ImageCanvasEditorTypes.ts | 2 + .../ImageCanvasEditorView.test-utils.ts | 3 + .../image-editor/ImageCanvasEditorView.tsx | 2 + .../ImageCanvasGenerationLayerModel.test.ts | 31 ++- .../ImageCanvasGenerationLayerModel.ts | 51 ++++- .../ImageCanvasGenerationModel.ts | 26 +++ .../ImageCanvasMetadataModalView.test.tsx | 34 ++++ .../ImageCanvasMetadataModalView.tsx | 23 ++- ...ageCanvasSelectedLayerToolbarView.test.tsx | 29 +++ .../ImageCanvasSelectedLayerToolbarView.tsx | 14 ++ .../image-editor/ImageCanvasStageView.tsx | 3 + .../ImageCanvasWorldView.test.tsx | 10 +- .../image-editor/ImageCanvasWorldView.tsx | 118 +++++++++--- ...anvasGenerationSubmissionWorkflow.test.tsx | 81 +++++++- ...ImageCanvasGenerationSubmissionWorkflow.ts | 60 +++++- .../useImageCanvasGenerationWorkflow.ts | 26 +++ .../useImageCanvasProjectPersistence.ts | 6 +- src/index.css | 36 +++- .../image-editor/editorProjectClient.test.ts | 44 +++++ .../image-editor/editorProjectClient.ts | 24 +++ 36 files changed, 1092 insertions(+), 110 deletions(-) diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index dfd601fc4..10c4f2756 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -16,6 +16,14 @@ --- +## 2026-06-18 图片画布 UI 设计图提取素材保留图集 + +- 背景:UI 设计图需要从成图中继续抽取可复用独立素材;原图标素材生成只把拆分后的图标放入画布,spritesheet 原图没有保留,后续追溯和二次切图不方便。 +- 决策:`assetKind="ui-design"` 图层浮动工具栏新增 `提取素材`,固定调用 `/api/editor/ui-designs/assets/extractions`,后端固定 `gpt-image-2` 和提示词 `提取画面中的所有独立并整理成spritesheet`,返回结构复用图标 spritesheet 响应。图标生成与 UI 提取都必须把 spritesheet 图集作为 `assetKind="icon-spritesheet"` 图层放到画布,再放拆分后的 `assetKind="icon"` 素材。 +- 影响范围:图片画布浮动工具栏、编辑器图片生成 BFF、`platform-image` 图集连通域拆分、画布图层类型和编辑器文档。 +- 验证方式:运行图片画布工具栏 / 图集落层 / 生成提交相关前端测试,`cargo test -p platform-image generated_asset_sheets --manifest-path server-rs/Cargo.toml`,以及 `cargo test -p api-server editor_ui_design_asset_extraction_prompt_is_fixed --manifest-path server-rs/Cargo.toml`。 +- 关联文档:`docs/【编辑器】画板UI设计图生成入口设计-2026-06-17.md`、`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`。 + ## 2026-06-18 图片画布 Seedance 2.0 参考媒体提交边界 - 背景:`/editor/canvas` 生成视频需要严格对齐火山 Seedance 2.0 多模态参考输入;参考视频若继续走 Base64 / `data:video` 会超过请求体并被上游拒绝,参考音频单独输入和非 Seedance 模型携带参考字段也会违反文档契约。 diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index f2ee9b825..0e965d443 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -23,6 +23,14 @@ - 验证:`cargo test -p platform-audio --manifest-path server-rs/Cargo.toml`;`cargo test -p api-server vector_engine_audio_generation --manifest-path server-rs/Cargo.toml`。 - 关联:`server-rs/crates/platform-audio/src/client.rs`、`server-rs/crates/platform-audio/src/response.rs`、`docs/【编辑器】画板音乐生成入口设计-2026-06-18.md`。 +## 图片画布音频卡播放条 0:00 要优先查签名 URL 和嵌套交互 + +- 现象:画板音效或背景音乐已经生成成功,但卡片里的播放条显示 `0:00`,点击无法预览。 +- 原因:generated 音频资源通常是私有 OSS 路径,直接把 `/generated-*` 或 generated OSS 地址交给 `