diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index aabef48b6..48f5d97a8 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -24,6 +24,14 @@ - 验证方式：`cargo test -p api-server external_api --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server external_editor_api --manifest-path server-rs/Cargo.toml`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:spacetime-schema`、`npm run check:encoding`、`git diff --check`。 - 关联文档：`docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md`。 +## 2026-06-19 图片画布素材生成元数据上移到资源和素材 + +- 背景：角色、图标、UI 设计图、视频和音频等生成结果会在图片信息页展示用户可见输入快照；此前这些 `assetKind/generationInputs` 主要保存在画布 layer JSON 中，素材进入账号级素材库后跨项目复用和刷新恢复都依赖画布布局，不符合素材库作为账号级事实源的边界。 +- 决策：普通图层的新保存不再把 `assetKind/generationInputs` 写入 `editor_canvas.layers_json`。`editor_project_resource` 保存项目画布资源快照的 `asset_kind/generation_inputs_json`，`editor_asset` 保存账号级素材的同名元数据；图片 / 图标 / UI 提取等生成 BFF 在请求携带 `projectId` / `assetFolderId` 时由后端创建新 resource / asset 并把快照回传前端，前端只用回包更新画布图层和素材栏，不再把同一生成结果二次调用保存接口。前端加载时优先从 resource / asset 恢复素材类别和生成输入快照，旧 layout 中的同名字段只作为历史兼容兜底。生成器对象本身仍作为 `itemType="generation-dialog"` 保存在画布布局中。 +- 影响范围：`server-rs/crates/spacetime-module/src/editor_project_storage.rs`、`server-rs/crates/spacetime-client/src/mapper/editor_project.rs`、`server-rs/crates/api-server/src/editor_project.rs`、`src/services/image-editor/editorProjectClient.ts`、图片画布 hydrate / serialize / project persistence / asset library 代码和后端数据契约文档。 +- 验证方式：运行 `npm run spacetime:generate`、`npm run check:spacetime-schema`、`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`，并按需补充 `cargo check -p spacetime-client -p api-server --manifest-path server-rs/Cargo.toml`。 +- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`。 + ## 2026-06-19 图片画布生成按钮价格统一绑定模型定价配置 - 背景：图片画布的生成图片、生成视频、生成规范、生成角色、生成素材、生成 UI、宣发素材、快速编辑、重绘和音频生成入口都在按钮内显示泥点；如果按钮文案、前端提交和后端校验各自写固定数值，后续调整模型价格会出现展示价、提交价和扣费价不一致。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 707959d24..368850adc 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -39,12 +39,12 @@ - 新增 `editor_project` 表保存图片画布工程：`projectId`、`ownerUserId`、标题、创建时间和更新时间；历史 layout 字段暂保留为兼容列，不再作为权威画布数据。 - 新增 `editor_canvas` 表保存工程下的画布：`canvasId`、`projectId`、`ownerUserId`、标题、viewport、图层布局 JSON、创建时间和更新时间。当前编辑器使用项目默认画布，后续可扩展为一个 project 下多个 canvas。 - 新增 `editor_asset_folder` 表保存账号级素材文件夹：`folderId`、`ownerUserId`、名称、排序、折叠状态、系统默认标记、创建时间和更新时间。素材文件夹不归属于 project，同一个账号进入任一项目都能看到。 -- 新增 `editor_asset` 表保存账号级素材：`assetId`、`ownerUserId`、`folderId`、名称、图片读取地址、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、创建时间和更新时间。素材只跟账号走，不跟 project 走。 -- `editor_project_resource` 表保存工程画布引用过的资源快照：`resourceId`、`projectId`、`ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource，图层只引用 resourceId。 +- 新增 `editor_asset` 表保存账号级素材：`assetId`、`ownerUserId`、`folderId`、名称、图片读取地址、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、`assetKind`、`generationInputs`、创建时间和更新时间。素材只跟账号走，不跟 project 走；角色、图标、UI 设计图、视频和音频等生成结果的用户可见输入快照随素材保存。 +- `editor_project_resource` 表保存工程画布引用过的资源快照：`resourceId`、`projectId`、`ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind`、`generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource，图层只引用 resourceId；图片、图标和 UI 素材生成 BFF 在请求携带 `projectId` 时由后端直接创建新 resource，并把 `resourceId` 随生成响应返回给前端。 - 图片文件本体继续走 OSS，浏览器读取私有 generated 对象仍经 `/api/assets/read-url` 换签。 - 当前 MVP 的本地上传先以 data URL 持久化在素材记录中，保证刷新和跨项目可见；后续接入正式 OSS 上传时，只替换 `imageSrc/objectKey/assetObjectId` 的写入方式，账号级素材表和画布资源表不变。 -- 资源表只保存资源元数据；图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组：普通图层按 `layerId/resourceId` 保存，生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存，不新增单独表。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`；生成成功后仍保存该快照，只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。图层展示尺寸不再作为独立 `Size` 真相保存，刷新与新建图层均按 `Resolution`（`originalWidth/originalHeight`）原分辨率显示。图层组第一版是画布内布局语义，不单独建表。 -- 图片类生成结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；视频结果当前只保存为画布视频资源，不进入图片素材库。 +- 资源表保存资源和素材级元数据；图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组：普通图层按 `layerId/resourceId` 保存，生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存，不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON；刷新时优先从 `editor_project_resource` 恢复，旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`；生成成功后仍保存该快照，只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。图层展示尺寸不再作为独立 `Size` 真相保存，刷新与新建图层均按 `Resolution`（`originalWidth/originalHeight`）原分辨率显示。图层组第一版是画布内布局语义，不单独建表。 +- 图片类生成结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；该写入由生成 BFF 在请求携带 `assetFolderId` 时完成，前端只把响应中的 resource / asset 快照用于落画布和更新素材栏，不再把同一生成结果二次调用素材创建接口。视频结果当前只保存为画布视频资源，不进入图片素材库。 - 前端不直接订阅 SpacetimeDB，统一通过 api-server 的 `/api/editor/projects*` BFF 读写。 - 未登录用户可以使用本地演示态，但不触发工程自动保存；真实图片生成 / 修改需要登录。编辑器 API 请求允许使用 refresh cookie 静默补 access token，但 401 / 403 只在编辑器局部提示登录，不清空整站登录态，也不把后端 requestId 直接作为生图弹窗主文案。 @@ -65,10 +65,10 @@ - `POST /api/editor/assets`：批量或单个创建账号级素材，支持按钮上传和拖拽上传后的 data URL / 后续 OSS 元数据。 - `PATCH /api/editor/assets/{assetId}`：重命名素材或移动素材到文件夹。 - `DELETE /api/editor/assets/{assetId}`：删除素材。已放入画布的 project resource 不被级联删除，避免旧画布丢图。 -- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；角色生成可携带 `model`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits。携带参考图的快速编辑也走该接口，前端必须把参考图源预读成图片 Data URL 后放入 `referenceImageSrcs`；接口返回 data URL、尺寸、prompt、model、provider 和 taskId。 -- `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet，再由后端切分为独立透明图标。请求支持 `model`、`aspectRatio`、`imageSize` 和 `priceMudPoints`；`priceMudPoints` 必须来自编辑器生成计费配置的 `icon` 档位（首版 12 泥点），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`，后端把 UI 尺寸归一为文档支持的 `1024x1024`、`1024x1536`、`1536x1024`、`2048x2048`、`2048x1152` 等 `size` 字符串。 -- `POST /api/editor/ui-designs/assets/extractions`：以 UI 设计图 Data URL 作为参考图，固定 `gpt-image-2` 和提示词 `提取画面中的所有独立并整理成spritesheet` 生成素材 spritesheet，再按连通域自动拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。前端必须把 spritesheet 原图与拆分素材都加入画布。 -- `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据。 +- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；角色生成可携带 `model`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits。携带参考图的快速编辑也走该接口，前端必须把参考图源预读成图片 Data URL 后放入 `referenceImageSrcs`；请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成成功后创建 project resource / 账号素材并在响应中返回 resource / asset 快照。 +- `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet，再由后端切分为独立透明图标。请求支持 `model`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置的 `icon` 档位（首版 12 泥点），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。后端把 spritesheet 和拆分后的 icon 都保存为 project resource / 账号素材，并随响应返回对应快照。 +- `POST /api/editor/ui-designs/assets/extractions`：以 UI 设计图 Data URL 作为参考图，固定 `gpt-image-2` 和提示词 `提取画面中的所有独立并整理成spritesheet` 生成素材 spritesheet，再按连通域自动拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，后端保存 spritesheet / 拆分素材并返回对应 resource / asset 快照；前端必须把 spritesheet 原图与拆分素材都加入画布。 +- `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据；请求携带 project / asset 上下文时由后端创建新 resource / asset，前端只消费响应快照。 - `POST /api/editor/videos/generations`：按视频描述、模型、比例、时长、分辨率、模式、声音和泥点价格生成视频。前端可选模型为 `seedance2.0-fast`、`seedance2.0`、`kling3.0`、`kling3.0-omni`，默认 `seedance2.0-fast`；后端必须将 `seedance2.0-fast` 映射到 `doubao-seedance-2-0-fast-260128`，将 `seedance2.0` 映射到 `doubao-seedance-2-0-260128`，两者不得混用。后端复用 Ark / VectorEngine content generation task 轮询链路，下载最终视频并持久化到 OSS，返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`。 所有写接口都必须校验 Bearer 登录态和 owner；接口只返回当前用户有权读取的工程与资源。 diff --git a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md index 553c2f3bd..922ac32b9 100644 --- a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md +++ b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md @@ -454,7 +454,7 @@ npm run check:server-rs-ddd - Rust 结构体：`EditorProjectResource` - 源码：`server-rs/crates/spacetime-module/src/editor_project_storage.rs` -- 说明：图片画布工程资源元数据表，保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task 和源资源关系；账号级素材删除不级联删除该表，避免历史画布丢图。 +- 说明：图片画布工程资源元数据表，保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task、源资源关系、`asset_kind` 和 `generation_inputs_json`。`asset_kind` 标记角色、图标、UI 设计图、视频、音频等素材类别；`generation_inputs_json` 保存用户可见生成输入快照，供图片信息页刷新后恢复。图片 / 图标 / UI 提取等生成 BFF 在请求携带 `project_id` 时负责创建该表记录并把 resource 快照返回前端；前端只保存布局引用，不能把同一生成结果再次作为正式业务真相写入。账号级素材删除不级联删除该表，避免历史画布丢图。`editor_canvas.layers_json` 只保存图层几何、层级、分组、资源引用和生成器对象；新写入不再把素材生成输入快照作为图层布局真相保存，旧布局字段只作为兼容兜底读取。 - 索引：`by_editor_project_resource_project_id`、`by_editor_project_resource_owner_user_id`。 ### `editor_asset_folder` @@ -468,7 +468,7 @@ npm run check:server-rs-ddd - Rust 结构体：`EditorAsset` - 源码：`server-rs/crates/spacetime-module/src/editor_project_storage.rs` -- 说明：图片画布账号级素材表，保存用户上传 / 生成素材的名称、文件夹、图片读取地址、OSS 引用、尺寸、来源类型和生成元数据；素材在同一账号的所有项目中可见。素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId。 +- 说明：图片画布账号级素材表，保存用户上传 / 生成素材的名称、文件夹、图片读取地址、OSS 引用、尺寸、来源类型、prompt、provider、task、`asset_kind` 和 `generation_inputs_json`。素材在同一账号的所有项目中可见；图片 / 图标 / UI 提取等生成 BFF 在请求携带 `asset_folder_id` 时负责创建账号级生成素材并返回 asset 快照，前端只用该快照更新素材栏。素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId，画布从 resource / asset 级元数据恢复素材类别和用户可见生成输入快照。 - 索引：`by_editor_asset_owner_user_id`、`by_editor_asset_folder_id`。 ### `inventory_slot` diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index e0d8eed19..a4253cb80 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -60,6 +60,8 @@ const EDITOR_ASSET_FOLDER_ID_PREFIX: &str = "editor-asset-folder-"; pub(crate) const EDITOR_ASSET_ID_PREFIX: &str = "editor-asset-"; const EDITOR_LAYOUT_MAX_BYTES: usize = 256 * 1024; pub(crate) const EDITOR_PROJECT_DEFAULT_TITLE: &str = "未命名画布"; +const EDITOR_ASSET_METADATA_MAX_BYTES: usize = 64 * 1024; +const EDITOR_ASSET_DEFAULT_FOLDER_ID: &str = "project"; const EDITOR_IMAGE_GENERATION_SIZE: &str = "1024x1024"; const EDITOR_IMAGE_MODEL_NANOBANANA2: &str = "gemini-3.1-flash-image-preview"; const EDITOR_IMAGE_MODEL_NANOBANANA2_DISPLAY_ALIAS: &str = "nanobanana2"; @@ -113,6 +115,8 @@ pub struct EditorProjectResourceCreateRequest { provider: Option, task_id: Option, source_resource_id: Option, + asset_kind: Option, + generation_inputs: Option, } #[derive(Debug, Deserialize)] @@ -145,6 +149,8 @@ pub struct EditorAssetCreateRequest { model: Option, provider: Option, task_id: Option, + asset_kind: Option, + generation_inputs: Option, } #[derive(Debug, Deserialize)] @@ -164,6 +170,12 @@ pub struct EditorImageGenerationRequest { aspect_ratio: Option, image_size: Option, reference_image_srcs: Option>, + project_id: Option, + asset_kind: Option, + generation_inputs: Option, + asset_folder_id: Option, + asset_label: Option, + source_resource_id: Option, } #[derive(Debug, Deserialize)] @@ -171,6 +183,12 @@ pub struct EditorImageGenerationRequest { pub struct EditorImageEditRequest { prompt: String, source_image_src: String, + project_id: Option, + asset_kind: Option, + generation_inputs: Option, + asset_folder_id: Option, + asset_label: Option, + source_resource_id: Option, } #[derive(Debug, Deserialize)] @@ -182,12 +200,19 @@ pub struct EditorIconSpritesheetGenerationRequest { aspect_ratio: Option, image_size: Option, price_mud_points: u32, + project_id: Option, + generation_inputs: Option, + asset_folder_id: Option, } #[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] pub struct EditorUiDesignAssetExtractionRequest { source_image_src: String, + project_id: Option, + generation_inputs: Option, + asset_folder_id: Option, + spritesheet_label: Option, } #[derive(Debug, Serialize)] @@ -244,6 +269,33 @@ pub struct EditorAssetResponse { asset: EditorAssetPayload, } +#[derive(Debug)] +struct EditorGeneratedAssetRecord { + resource: Option, + asset: Option, +} + +#[derive(Debug)] +struct PersistEditorGeneratedAssetInput { + project_id: Option, + owner_user_id: String, + folder_id: Option, + label: String, + image_src: String, + object_key: Option, + asset_object_id: Option, + width: u32, + height: u32, + prompt: String, + actual_prompt: Option, + model: String, + provider: String, + task_id: String, + source_resource_id: Option, + asset_kind: Option, + generation_inputs: Option, +} + #[derive(Debug, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorImageGenerationResponse { @@ -258,6 +310,8 @@ pub struct EditorImageGenerationResponse { model: String, provider: &'static str, task_id: String, + resource: Option, + asset: Option, } #[derive(Debug, Serialize)] @@ -267,6 +321,8 @@ pub struct EditorIconSpritesheetIconResponse { image_src: String, width: u32, height: u32, + resource: Option, + asset: Option, } #[derive(Debug, Serialize)] @@ -282,6 +338,8 @@ pub struct EditorIconSpritesheetGenerationResponse { provider: &'static str, task_id: String, price_mud_points: u32, + spritesheet_resource: Option, + spritesheet_asset: Option, } #[derive(Debug, Serialize)] @@ -325,6 +383,8 @@ pub struct EditorProjectResourcePayload { provider: Option, task_id: Option, source_resource_id: Option, + asset_kind: Option, + generation_inputs: Option, created_at: String, updated_at: String, } @@ -365,6 +425,8 @@ pub struct EditorAssetPayload { model: Option, provider: Option, task_id: Option, + asset_kind: Option, + generation_inputs: Option, created_at: String, updated_at: String, } @@ -545,6 +607,8 @@ pub async fn create_editor_project_resource( Extension(authenticated): Extension, Json(payload): Json, ) -> Result, AppError> { + let generation_inputs_json = + serialize_editor_asset_metadata(payload.generation_inputs.clone())?; let resource = state .spacetime_client() .create_editor_project_resource(EditorProjectResourceCreateRecordInput { @@ -563,6 +627,8 @@ pub async fn create_editor_project_resource( provider: normalize_optional_string(payload.provider), task_id: normalize_optional_string(payload.task_id), source_resource_id: normalize_optional_string(payload.source_resource_id), + asset_kind: normalize_optional_string(payload.asset_kind), + generation_inputs_json, updated_at_micros: current_utc_micros(), }) .await @@ -678,6 +744,8 @@ pub async fn create_editor_asset( Extension(authenticated): Extension, Json(payload): Json, ) -> Result, AppError> { + let generation_inputs_json = + serialize_editor_asset_metadata(payload.generation_inputs.clone())?; let asset = state .spacetime_client() .create_editor_asset(EditorAssetCreateRecordInput { @@ -696,6 +764,8 @@ pub async fn create_editor_asset( model: normalize_optional_string(payload.model), provider: normalize_optional_string(payload.provider), task_id: normalize_optional_string(payload.task_id), + asset_kind: normalize_optional_string(payload.asset_kind), + generation_inputs_json, now_micros: current_utc_micros(), }) .await @@ -953,6 +1023,32 @@ pub async fn generate_editor_image( image.mime_type, BASE64_STANDARD.encode(image.bytes.as_slice()) ); + let generated_asset = persist_editor_generated_asset( + &state, + PersistEditorGeneratedAssetInput { + project_id: payload.project_id, + owner_user_id: current_owner_user_id(&authenticated), + folder_id: payload.asset_folder_id, + label: normalize_optional_string(payload.asset_label) + .unwrap_or_else(|| role_setting.clone()), + image_src: image_src.clone(), + object_key: persisted.as_ref().map(|asset| asset.object_key.clone()), + asset_object_id: persisted + .as_ref() + .map(|asset| asset.asset_object_id.clone()), + width, + height, + prompt: role_setting.clone(), + actual_prompt: generated.actual_prompt.clone(), + model: generation_options.model.to_string(), + provider: "VectorEngine".to_string(), + task_id: generated.task_id.clone(), + source_resource_id: payload.source_resource_id, + asset_kind: payload.asset_kind, + generation_inputs: payload.generation_inputs, + }, + ) + .await?; Ok(json_success_body( Some(&request_context), @@ -970,6 +1066,8 @@ pub async fn generate_editor_image( model: generation_options.model.to_string(), provider: "VectorEngine", task_id: generated.task_id, + resource: generated_asset.resource, + asset: generated_asset.asset, }, )) } @@ -1141,6 +1239,30 @@ pub async fn edit_editor_image( image.mime_type, BASE64_STANDARD.encode(image.bytes.as_slice()) ); + let generated_asset = persist_editor_generated_asset( + &state, + PersistEditorGeneratedAssetInput { + project_id: payload.project_id, + owner_user_id: current_owner_user_id(&authenticated), + folder_id: payload.asset_folder_id, + label: normalize_optional_string(payload.asset_label) + .unwrap_or_else(|| "图片修改结果".to_string()), + image_src: image_src.clone(), + object_key: None, + asset_object_id: None, + width, + height, + prompt: prompt.clone(), + actual_prompt: generated.actual_prompt.clone(), + model: GPT_IMAGE_2_MODEL.to_string(), + provider: "VectorEngine".to_string(), + task_id: generated.task_id.clone(), + source_resource_id: payload.source_resource_id, + asset_kind: payload.asset_kind, + generation_inputs: payload.generation_inputs, + }, + ) + .await?; Ok(json_success_body( Some(&request_context), @@ -1156,6 +1278,8 @@ pub async fn edit_editor_image( model: GPT_IMAGE_2_MODEL.to_string(), provider: "VectorEngine", task_id: generated.task_id, + resource: generated_asset.resource, + asset: generated_asset.asset, }, )) } @@ -1246,10 +1370,63 @@ pub async fn generate_editor_icon_spritesheet( })?; let spritesheet_image_src = data_url_from_image_bytes(image.mime_type.as_str(), image.bytes.as_slice()); - let icon_image_srcs = icon_slices - .into_iter() - .map(editor_icon_response_from_slice) - .collect(); + let generation_inputs = payload.generation_inputs.clone(); + let owner_user_id = current_owner_user_id(&authenticated); + let spritesheet_record = persist_editor_generated_asset( + &state, + PersistEditorGeneratedAssetInput { + project_id: payload.project_id.clone(), + owner_user_id: owner_user_id.clone(), + folder_id: payload.asset_folder_id.clone(), + label: "图标素材图集".to_string(), + image_src: spritesheet_image_src.clone(), + object_key: None, + asset_object_id: None, + width: spritesheet_width, + height: spritesheet_height, + prompt: prompt.clone(), + actual_prompt: generated.actual_prompt.clone(), + model: generation_options.model.to_string(), + provider: "VectorEngine".to_string(), + task_id: generated.task_id.clone(), + source_resource_id: None, + asset_kind: Some("icon-spritesheet".to_string()), + generation_inputs: generation_inputs.clone(), + }, + ) + .await?; + let mut icon_image_srcs = Vec::new(); + for icon in icon_slices { + let icon_response = editor_icon_response_from_slice(icon); + let icon_record = persist_editor_generated_asset( + &state, + PersistEditorGeneratedAssetInput { + project_id: payload.project_id.clone(), + owner_user_id: owner_user_id.clone(), + folder_id: payload.asset_folder_id.clone(), + label: icon_response.name.clone(), + image_src: icon_response.image_src.clone(), + object_key: None, + asset_object_id: None, + width: icon_response.width, + height: icon_response.height, + prompt: prompt.clone(), + actual_prompt: generated.actual_prompt.clone(), + model: generation_options.model.to_string(), + provider: "VectorEngine".to_string(), + task_id: generated.task_id.clone(), + source_resource_id: None, + asset_kind: Some("icon".to_string()), + generation_inputs: generation_inputs.clone(), + }, + ) + .await?; + icon_image_srcs.push(EditorIconSpritesheetIconResponse { + resource: icon_record.resource, + asset: icon_record.asset, + ..icon_response + }); + } Ok(json_success_body( Some(&request_context), @@ -1264,6 +1441,8 @@ pub async fn generate_editor_icon_spritesheet( provider: "VectorEngine", task_id: generated.task_id, price_mud_points: expected_price_mud_points, + spritesheet_resource: spritesheet_record.resource, + spritesheet_asset: spritesheet_record.asset, }, )) } @@ -1323,10 +1502,65 @@ pub async fn extract_editor_ui_design_assets( })?; let spritesheet_image_src = data_url_from_image_bytes(image.mime_type.as_str(), image.bytes.as_slice()); - let icon_image_srcs = icon_slices - .into_iter() - .map(editor_icon_response_from_slice) - .collect(); + let prompt = EDITOR_UI_DESIGN_ASSET_EXTRACTION_PROMPT.to_string(); + let generation_inputs = payload.generation_inputs.clone(); + let owner_user_id = current_owner_user_id(&authenticated); + let spritesheet_record = persist_editor_generated_asset( + &state, + PersistEditorGeneratedAssetInput { + project_id: payload.project_id.clone(), + owner_user_id: owner_user_id.clone(), + folder_id: payload.asset_folder_id.clone(), + label: normalize_optional_string(payload.spritesheet_label) + .unwrap_or_else(|| "UI设计图素材图集".to_string()), + image_src: spritesheet_image_src.clone(), + object_key: None, + asset_object_id: None, + width: spritesheet_width, + height: spritesheet_height, + prompt: prompt.clone(), + actual_prompt: generated.actual_prompt.clone(), + model: GPT_IMAGE_2_MODEL.to_string(), + provider: "VectorEngine".to_string(), + task_id: generated.task_id.clone(), + source_resource_id: None, + asset_kind: Some("icon-spritesheet".to_string()), + generation_inputs: generation_inputs.clone(), + }, + ) + .await?; + let mut icon_image_srcs = Vec::new(); + for icon in icon_slices { + let icon_response = editor_icon_response_from_slice(icon); + let icon_record = persist_editor_generated_asset( + &state, + PersistEditorGeneratedAssetInput { + project_id: payload.project_id.clone(), + owner_user_id: owner_user_id.clone(), + folder_id: payload.asset_folder_id.clone(), + label: icon_response.name.clone(), + image_src: icon_response.image_src.clone(), + object_key: None, + asset_object_id: None, + width: icon_response.width, + height: icon_response.height, + prompt: prompt.clone(), + actual_prompt: generated.actual_prompt.clone(), + model: GPT_IMAGE_2_MODEL.to_string(), + provider: "VectorEngine".to_string(), + task_id: generated.task_id.clone(), + source_resource_id: None, + asset_kind: Some("icon".to_string()), + generation_inputs: generation_inputs.clone(), + }, + ) + .await?; + icon_image_srcs.push(EditorIconSpritesheetIconResponse { + resource: icon_record.resource, + asset: icon_record.asset, + ..icon_response + }); + } Ok(json_success_body( Some(&request_context), @@ -1335,7 +1569,7 @@ pub async fn extract_editor_ui_design_assets( spritesheet_width, spritesheet_height, icon_image_srcs, - prompt: EDITOR_UI_DESIGN_ASSET_EXTRACTION_PROMPT.to_string(), + prompt, actual_prompt: generated.actual_prompt, model: GPT_IMAGE_2_MODEL.to_string(), provider: "VectorEngine", @@ -1344,6 +1578,8 @@ pub async fn extract_editor_ui_design_assets( Some("icon"), Some(GPT_IMAGE_2_MODEL), ), + spritesheet_resource: spritesheet_record.resource, + spritesheet_asset: spritesheet_record.asset, }, )) } @@ -1405,6 +1641,8 @@ pub(crate) fn editor_project_resource_payload_from_record( provider: record.provider, task_id: record.task_id, source_resource_id: record.source_resource_id, + asset_kind: record.asset_kind, + generation_inputs: record.generation_inputs, created_at: record.created_at, updated_at: record.updated_at, } @@ -1457,11 +1695,103 @@ pub(crate) fn editor_asset_payload_from_record(record: EditorAssetRecord) -> Edi model: record.model, provider: record.provider, task_id: record.task_id, + asset_kind: record.asset_kind, + generation_inputs: record.generation_inputs, created_at: record.created_at, updated_at: record.updated_at, } } +async fn persist_editor_generated_asset( + state: &AppState, + input: PersistEditorGeneratedAssetInput, +) -> Result { + let generation_inputs_json = serialize_editor_asset_metadata(input.generation_inputs.clone())?; + let now_micros = current_utc_micros(); + let resource = if let Some(project_id) = normalize_optional_string(input.project_id.clone()) { + let record = state + .spacetime_client() + .create_editor_project_resource(EditorProjectResourceCreateRecordInput { + resource_id: build_prefixed_uuid_id(EDITOR_RESOURCE_ID_PREFIX), + project_id, + owner_user_id: input.owner_user_id.clone(), + asset_object_id: input.asset_object_id.clone(), + image_src: input.image_src.clone(), + object_key: input.object_key.clone(), + width: input.width, + height: input.height, + source_type: "generated".to_string(), + prompt: Some(input.prompt.clone()), + actual_prompt: input.actual_prompt.clone(), + model: Some(input.model.clone()), + provider: Some(input.provider.clone()), + task_id: Some(input.task_id.clone()), + source_resource_id: normalize_optional_string(input.source_resource_id.clone()), + asset_kind: normalize_optional_string(input.asset_kind.clone()), + generation_inputs_json: generation_inputs_json.clone(), + updated_at_micros: now_micros, + }) + .await + .map_err(map_editor_project_error)?; + Some(editor_project_resource_payload_from_record(record)) + } else { + None + }; + + let folder_id = + normalize_generated_asset_folder_id(input.folder_id, input.owner_user_id.as_str()); + let asset = if let Some(folder_id) = folder_id { + let record = state + .spacetime_client() + .create_editor_asset(EditorAssetCreateRecordInput { + asset_id: build_prefixed_uuid_id(EDITOR_ASSET_ID_PREFIX), + owner_user_id: input.owner_user_id.clone(), + folder_id, + label: input.label, + asset_object_id: input.asset_object_id, + image_src: input.image_src, + object_key: input.object_key, + width: input.width, + height: input.height, + source_type: "generated".to_string(), + prompt: Some(input.prompt), + actual_prompt: input.actual_prompt, + model: Some(input.model), + provider: Some(input.provider), + task_id: Some(input.task_id), + asset_kind: normalize_optional_string(input.asset_kind), + generation_inputs_json, + now_micros, + }) + .await + .map_err(map_editor_project_error)?; + Some(editor_asset_payload_from_record(record)) + } else { + None + }; + + Ok(EditorGeneratedAssetRecord { resource, asset }) +} + +fn normalize_generated_asset_folder_id( + folder_id: Option, + owner_user_id: &str, +) -> Option { + let normalized = normalize_optional_string(folder_id); + match normalized.as_deref() { + None => None, + Some("project") => Some(default_editor_asset_folder_id(owner_user_id)), + Some(value) if value.starts_with("folder-") => { + Some(default_editor_asset_folder_id(owner_user_id)) + } + Some(_) => normalized, + } +} + +fn default_editor_asset_folder_id(owner_user_id: &str) -> String { + format!("{owner_user_id}:asset-folder:{EDITOR_ASSET_DEFAULT_FOLDER_ID}") +} + impl EditorCanvasViewportPayload { pub(crate) fn into_record(self) -> EditorCanvasViewportRecord { EditorCanvasViewportRecord { @@ -1494,6 +1824,30 @@ pub(crate) fn serialize_editor_layers(layers: Value) -> Result Ok(payload) } +fn serialize_editor_asset_metadata(metadata: Option) -> Result, AppError> { + let Some(metadata) = metadata else { + return Ok(None); + }; + if metadata.is_null() { + return Ok(None); + } + let payload = serde_json::to_string(&metadata).map_err(|error| { + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "editor-project", + "message": format!("资产元数据无法序列化：{error}"), + })) + })?; + if payload.len() > EDITOR_ASSET_METADATA_MAX_BYTES { + return Err( + AppError::from_status(StatusCode::PAYLOAD_TOO_LARGE).with_details(json!({ + "provider": "editor-project", + "message": "资产元数据过大", + })), + ); + } + Ok(Some(payload)) +} + pub(crate) fn normalize_optional_string(value: Option) -> Option { value .map(|item| item.trim().to_string()) @@ -1641,6 +1995,8 @@ fn editor_icon_response_from_slice( image_src: data_url_from_image_bytes("image/png", icon.bytes.as_slice()), width: icon.width, height: icon.height, + resource: None, + asset: None, } } @@ -2091,6 +2447,8 @@ mod tests { image_src: "data:image/png;base64,back".to_string(), width: 128, height: 128, + resource: None, + asset: None, }], prompt: "图标 prompt".to_string(), actual_prompt: Some("图标 prompt".to_string()), @@ -2101,9 +2459,35 @@ mod tests { Some("icon"), None, ), + spritesheet_resource: None, + spritesheet_asset: None, }; let payload = serde_json::to_value(response).expect("response should serialize"); assert_eq!(payload["priceMudPoints"], json!(12)); } + + #[test] + fn generated_asset_folder_id_normalizes_project_and_legacy_local_folder() { + assert_eq!( + normalize_generated_asset_folder_id(Some("project".to_string()), "user-1"), + Some("user-1:asset-folder:project".to_string()) + ); + assert_eq!( + normalize_generated_asset_folder_id(Some(" folder-temporary ".to_string()), "user-1"), + Some("user-1:asset-folder:project".to_string()) + ); + assert_eq!( + normalize_generated_asset_folder_id(Some("folder-real".to_string()), "user-1"), + Some("user-1:asset-folder:project".to_string()) + ); + assert_eq!( + normalize_generated_asset_folder_id( + Some("user-1:asset-folder:project".to_string()), + "user-1", + ), + Some("user-1:asset-folder:project".to_string()) + ); + assert_eq!(normalize_generated_asset_folder_id(None, "user-1"), None); + } } diff --git a/server-rs/crates/api-server/src/external_editor_api.rs b/server-rs/crates/api-server/src/external_editor_api.rs index e47f36dcc..ec82a9b0b 100644 --- a/server-rs/crates/api-server/src/external_editor_api.rs +++ b/server-rs/crates/api-server/src/external_editor_api.rs @@ -283,6 +283,8 @@ pub async fn generate_external_editor_image( model: Some(GPT_IMAGE_2_MODEL.to_string()), provider: Some(EXTERNAL_EDITOR_IMAGE_PROVIDER.to_string()), task_id: Some(generated.task_id.clone()), + asset_kind: Some(EXTERNAL_EDITOR_IMAGE_ASSET_KIND.to_string()), + generation_inputs_json: None, now_micros, }) .await @@ -307,6 +309,8 @@ pub async fn generate_external_editor_image( provider: Some(EXTERNAL_EDITOR_IMAGE_PROVIDER.to_string()), task_id: Some(generated.task_id.clone()), source_resource_id: None, + asset_kind: Some(EXTERNAL_EDITOR_IMAGE_ASSET_KIND.to_string()), + generation_inputs_json: None, updated_at_micros: now_micros, }) .await diff --git a/server-rs/crates/spacetime-client/src/mapper/editor_project.rs b/server-rs/crates/spacetime-client/src/mapper/editor_project.rs index 6f977bf42..b53925b53 100644 --- a/server-rs/crates/spacetime-client/src/mapper/editor_project.rs +++ b/server-rs/crates/spacetime-client/src/mapper/editor_project.rs @@ -47,6 +47,8 @@ pub struct EditorProjectResourceRecord { pub provider: Option, pub task_id: Option, pub source_resource_id: Option, + pub asset_kind: Option, + pub generation_inputs: Option, pub created_at: String, pub updated_at: String, } @@ -78,6 +80,8 @@ pub struct EditorAssetRecord { pub model: Option, pub provider: Option, pub task_id: Option, + pub asset_kind: Option, + pub generation_inputs: Option, pub created_at: String, pub updated_at: String, } @@ -142,6 +146,8 @@ pub struct EditorProjectResourceCreateRecordInput { pub provider: Option, pub task_id: Option, pub source_resource_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub updated_at_micros: i64, } @@ -187,6 +193,8 @@ pub struct EditorAssetCreateRecordInput { pub model: Option, pub provider: Option, pub task_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub now_micros: i64, } @@ -283,6 +291,8 @@ impl From provider: input.provider, task_id: input.task_id, source_resource_id: input.source_resource_id, + asset_kind: input.asset_kind, + generation_inputs_json: input.generation_inputs_json, updated_at_micros: input.updated_at_micros, } } @@ -346,6 +356,8 @@ impl From for crate::module_bindings::EditorAssetC model: input.model, provider: input.provider, task_id: input.task_id, + asset_kind: input.asset_kind, + generation_inputs_json: input.generation_inputs_json, now_micros: input.now_micros, } } @@ -426,6 +438,7 @@ pub(crate) fn map_editor_project_resource_procedure_result( result .resource .map(map_editor_project_resource_snapshot) + .transpose()? .ok_or_else(|| SpacetimeClientError::missing_snapshot("图片画布资源快照")) } @@ -439,6 +452,7 @@ pub(crate) fn map_editor_asset_library_procedure_result( result .library .map(map_editor_asset_library_snapshot) + .transpose()? .ok_or_else(|| SpacetimeClientError::missing_snapshot("图片画布素材库快照")) } @@ -465,6 +479,7 @@ pub(crate) fn map_editor_asset_folder_library_procedure_result( result .library .map(map_editor_asset_library_snapshot) + .transpose()? .ok_or_else(|| SpacetimeClientError::missing_snapshot("图片画布素材库快照")) } @@ -478,6 +493,7 @@ pub(crate) fn map_editor_asset_procedure_result( result .asset .map(map_editor_asset_snapshot) + .transpose()? .ok_or_else(|| SpacetimeClientError::missing_snapshot("图片画布素材快照")) } @@ -516,7 +532,7 @@ fn map_editor_project_snapshot( .resources .into_iter() .map(map_editor_project_resource_snapshot) - .collect(), + .collect::, _>>()?, created_at: format_timestamp_micros(snapshot.created_at_micros), updated_at: format_timestamp_micros(snapshot.updated_at_micros), }) @@ -524,8 +540,12 @@ fn map_editor_project_snapshot( fn map_editor_project_resource_snapshot( snapshot: EditorProjectResourceSnapshot, -) -> EditorProjectResourceRecord { - EditorProjectResourceRecord { +) -> Result { + let generation_inputs = parse_optional_generation_inputs_json( + snapshot.generation_inputs_json.as_deref(), + "图片画布资源生成输入 JSON", + )?; + Ok(EditorProjectResourceRecord { resource_id: snapshot.resource_id, project_id: snapshot.project_id, asset_object_id: snapshot.asset_object_id, @@ -540,15 +560,17 @@ fn map_editor_project_resource_snapshot( provider: snapshot.provider, task_id: snapshot.task_id, source_resource_id: snapshot.source_resource_id, + asset_kind: snapshot.asset_kind, + generation_inputs, created_at: format_timestamp_micros(snapshot.created_at_micros), updated_at: format_timestamp_micros(snapshot.updated_at_micros), - } + }) } fn map_editor_asset_library_snapshot( snapshot: EditorAssetLibrarySnapshot, -) -> EditorAssetLibraryRecord { - EditorAssetLibraryRecord { +) -> Result { + Ok(EditorAssetLibraryRecord { folders: snapshot .folders .into_iter() @@ -558,8 +580,8 @@ fn map_editor_asset_library_snapshot( .assets .into_iter() .map(map_editor_asset_snapshot) - .collect(), - } + .collect::, _>>()?, + }) } fn map_editor_asset_folder_snapshot( @@ -576,8 +598,14 @@ fn map_editor_asset_folder_snapshot( } } -fn map_editor_asset_snapshot(snapshot: EditorAssetSnapshot) -> EditorAssetRecord { - EditorAssetRecord { +fn map_editor_asset_snapshot( + snapshot: EditorAssetSnapshot, +) -> Result { + let generation_inputs = parse_optional_generation_inputs_json( + snapshot.generation_inputs_json.as_deref(), + "图片画布素材生成输入 JSON", + )?; + Ok(EditorAssetRecord { asset_id: snapshot.asset_id, folder_id: snapshot.folder_id, label: snapshot.label, @@ -592,7 +620,21 @@ fn map_editor_asset_snapshot(snapshot: EditorAssetSnapshot) -> EditorAssetRecord model: snapshot.model, provider: snapshot.provider, task_id: snapshot.task_id, + asset_kind: snapshot.asset_kind, + generation_inputs, created_at: format_timestamp_micros(snapshot.created_at_micros), updated_at: format_timestamp_micros(snapshot.updated_at_micros), - } + }) +} + +fn parse_optional_generation_inputs_json( + raw_json: Option<&str>, + label: &str, +) -> Result, SpacetimeClientError> { + let Some(raw_json) = raw_json.map(str::trim).filter(|value| !value.is_empty()) else { + return Ok(None); + }; + serde_json::from_str(raw_json).map(Some).map_err(|error| { + SpacetimeClientError::validation_failed(format!("{label} 无法解析：{error}")) + }) } diff --git a/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_create_input_type.rs b/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_create_input_type.rs index 5c5df7547..f37faf5d1 100644 --- a/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_create_input_type.rs +++ b/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_create_input_type.rs @@ -22,6 +22,8 @@ pub struct EditorAssetCreateInput { pub model: Option, pub provider: Option, pub task_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub now_micros: i64, } diff --git a/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_snapshot_type.rs b/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_snapshot_type.rs index 2bcb7cedc..85cb1de4a 100644 --- a/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_snapshot_type.rs +++ b/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_snapshot_type.rs @@ -21,6 +21,8 @@ pub struct EditorAssetSnapshot { pub model: Option, pub provider: Option, pub task_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub created_at_micros: i64, pub updated_at_micros: i64, } diff --git a/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_type.rs b/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_type.rs index f824530ec..04e959907 100644 --- a/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_type.rs +++ b/server-rs/crates/spacetime-client/src/module_bindings/editor_asset_type.rs @@ -24,6 +24,8 @@ pub struct EditorAsset { pub task_id: Option, pub created_at: __sdk::Timestamp, pub updated_at: __sdk::Timestamp, + pub asset_kind: Option, + pub generation_inputs_json: Option, } impl __sdk::InModule for EditorAsset { @@ -51,6 +53,8 @@ pub struct EditorAssetCols { pub task_id: __sdk::__query_builder::Col>, pub created_at: __sdk::__query_builder::Col, pub updated_at: __sdk::__query_builder::Col, + pub asset_kind: __sdk::__query_builder::Col>, + pub generation_inputs_json: __sdk::__query_builder::Col>, } impl __sdk::__query_builder::HasCols for EditorAsset { @@ -74,6 +78,11 @@ impl __sdk::__query_builder::HasCols for EditorAsset { task_id: __sdk::__query_builder::Col::new(table_name, "task_id"), created_at: __sdk::__query_builder::Col::new(table_name, "created_at"), updated_at: __sdk::__query_builder::Col::new(table_name, "updated_at"), + asset_kind: __sdk::__query_builder::Col::new(table_name, "asset_kind"), + generation_inputs_json: __sdk::__query_builder::Col::new( + table_name, + "generation_inputs_json", + ), } } } diff --git a/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_create_input_type.rs b/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_create_input_type.rs index f0576ddbb..cac8c3376 100644 --- a/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_create_input_type.rs +++ b/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_create_input_type.rs @@ -22,6 +22,8 @@ pub struct EditorProjectResourceCreateInput { pub provider: Option, pub task_id: Option, pub source_resource_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub updated_at_micros: i64, } diff --git a/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_snapshot_type.rs b/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_snapshot_type.rs index 5f812b246..085ff2b4c 100644 --- a/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_snapshot_type.rs +++ b/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_snapshot_type.rs @@ -21,6 +21,8 @@ pub struct EditorProjectResourceSnapshot { pub provider: Option, pub task_id: Option, pub source_resource_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub created_at_micros: i64, pub updated_at_micros: i64, } diff --git a/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_type.rs b/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_type.rs index d635e0dcd..a47526ce4 100644 --- a/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_type.rs +++ b/server-rs/crates/spacetime-client/src/module_bindings/editor_project_resource_type.rs @@ -24,6 +24,8 @@ pub struct EditorProjectResource { pub source_resource_id: Option, pub created_at: __sdk::Timestamp, pub updated_at: __sdk::Timestamp, + pub asset_kind: Option, + pub generation_inputs_json: Option, } impl __sdk::InModule for EditorProjectResource { @@ -51,6 +53,8 @@ pub struct EditorProjectResourceCols { pub source_resource_id: __sdk::__query_builder::Col>, pub created_at: __sdk::__query_builder::Col, pub updated_at: __sdk::__query_builder::Col, + pub asset_kind: __sdk::__query_builder::Col>, + pub generation_inputs_json: __sdk::__query_builder::Col>, } impl __sdk::__query_builder::HasCols for EditorProjectResource { @@ -74,6 +78,11 @@ impl __sdk::__query_builder::HasCols for EditorProjectResource { source_resource_id: __sdk::__query_builder::Col::new(table_name, "source_resource_id"), created_at: __sdk::__query_builder::Col::new(table_name, "created_at"), updated_at: __sdk::__query_builder::Col::new(table_name, "updated_at"), + asset_kind: __sdk::__query_builder::Col::new(table_name, "asset_kind"), + generation_inputs_json: __sdk::__query_builder::Col::new( + table_name, + "generation_inputs_json", + ), } } } diff --git a/server-rs/crates/spacetime-module/src/editor_project_storage.rs b/server-rs/crates/spacetime-module/src/editor_project_storage.rs index aacc3e55b..0195c9da9 100644 --- a/server-rs/crates/spacetime-module/src/editor_project_storage.rs +++ b/server-rs/crates/spacetime-module/src/editor_project_storage.rs @@ -69,6 +69,10 @@ pub struct EditorProjectResource { source_resource_id: Option, created_at: Timestamp, updated_at: Timestamp, + #[default(None::)] + asset_kind: Option, + #[default(None::)] + generation_inputs_json: Option, } #[spacetimedb::table( @@ -111,6 +115,10 @@ pub struct EditorAsset { task_id: Option, created_at: Timestamp, updated_at: Timestamp, + #[default(None::)] + asset_kind: Option, + #[default(None::)] + generation_inputs_json: Option, } #[derive(Clone, Debug, PartialEq, SpacetimeType)] @@ -184,6 +192,8 @@ pub struct EditorProjectResourceCreateInput { pub provider: Option, pub task_id: Option, pub source_resource_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub updated_at_micros: i64, } @@ -203,6 +213,8 @@ pub struct EditorProjectResourceSnapshot { pub provider: Option, pub task_id: Option, pub source_resource_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub created_at_micros: i64, pub updated_at_micros: i64, } @@ -234,6 +246,8 @@ pub struct EditorAssetSnapshot { pub model: Option, pub provider: Option, pub task_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub created_at_micros: i64, pub updated_at_micros: i64, } @@ -292,6 +306,8 @@ pub struct EditorAssetCreateInput { pub model: Option, pub provider: Option, pub task_id: Option, + pub asset_kind: Option, + pub generation_inputs_json: Option, pub now_micros: i64, } @@ -736,7 +752,7 @@ fn save_editor_project_layout( ) -> Result { let project_id = normalize_required(&input.project_id, "editor_project.project_id")?; let owner_user_id = normalize_required(&input.owner_user_id, "editor_project.owner_user_id")?; - let layers_json = normalize_layout_json(input.layers_json)?; + let input_layers_json = normalize_layout_json(input.layers_json)?; let project = require_owned_project(ctx, project_id.as_str(), owner_user_id.as_str())?; let now = Timestamp::from_micros_since_unix_epoch(input.updated_at_micros); let canvas = ensure_default_canvas( @@ -746,6 +762,20 @@ fn save_editor_project_layout( Some(&project), now, )?; + let _ = migrate_canvas_layer_asset_metadata( + ctx, + project.project_id.as_str(), + owner_user_id.as_str(), + canvas.layers_json.clone(), + now, + )?; + let layers_json = migrate_canvas_layer_asset_metadata( + ctx, + project.project_id.as_str(), + owner_user_id.as_str(), + input_layers_json, + now, + )?; ctx.db.editor_canvas().canvas_id().delete(&canvas.canvas_id); ctx.db.editor_canvas().insert(EditorCanvas { @@ -828,6 +858,8 @@ fn create_editor_project_resource( source_resource_id: normalize_optional(input.source_resource_id), created_at: now, updated_at: now, + asset_kind: normalize_optional(input.asset_kind), + generation_inputs_json: normalize_optional(input.generation_inputs_json), }); ctx.db @@ -986,6 +1018,8 @@ fn create_editor_asset( task_id: normalize_optional(input.task_id), created_at: now, updated_at: now, + asset_kind: normalize_optional(input.asset_kind), + generation_inputs_json: normalize_optional(input.generation_inputs_json), }); ctx.db .editor_asset() @@ -1031,6 +1065,8 @@ fn update_editor_asset( task_id: asset.task_id, created_at: asset.created_at, updated_at: now, + asset_kind: asset.asset_kind, + generation_inputs_json: asset.generation_inputs_json, }); ctx.db .editor_asset() @@ -1302,6 +1338,8 @@ fn asset_snapshot_from_row(row: EditorAsset) -> EditorAssetSnapshot { model: row.model, provider: row.provider, task_id: row.task_id, + asset_kind: row.asset_kind, + generation_inputs_json: row.generation_inputs_json, created_at_micros: row.created_at.to_micros_since_unix_epoch(), updated_at_micros: row.updated_at.to_micros_since_unix_epoch(), } @@ -1323,6 +1361,8 @@ fn resource_snapshot_from_row(row: EditorProjectResource) -> EditorProjectResour provider: row.provider, task_id: row.task_id, source_resource_id: row.source_resource_id, + asset_kind: row.asset_kind, + generation_inputs_json: row.generation_inputs_json, created_at_micros: row.created_at.to_micros_since_unix_epoch(), updated_at_micros: row.updated_at.to_micros_since_unix_epoch(), } @@ -1367,6 +1407,123 @@ fn normalize_layout_json(value: String) -> Result { Ok(value) } +fn migrate_canvas_layer_asset_metadata( + ctx: &ReducerContext, + project_id: &str, + owner_user_id: &str, + layers_json: String, + now: Timestamp, +) -> Result { + let mut layout = serde_json::from_str::(&layers_json) + .map_err(|_| "图片画布图层布局不是合法 JSON".to_string())?; + let Some(items) = layout.as_array_mut() else { + return Ok(layers_json); + }; + let mut changed = false; + for item in items { + let Some(object) = item.as_object_mut() else { + continue; + }; + if object.get("itemType").and_then(JsonValue::as_str).is_some() { + continue; + } + let resource_id = object + .get("resourceId") + .and_then(JsonValue::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned); + let asset_kind = object + .remove("assetKind") + .and_then(|value| normalize_legacy_layout_string(value.as_str())); + let generation_inputs_json = object + .remove("generationInputs") + .and_then(|value| serialize_non_empty_json_value(&value)); + changed |= asset_kind.is_some() || generation_inputs_json.is_some(); + let Some(resource_id) = resource_id else { + continue; + }; + if asset_kind.is_some() || generation_inputs_json.is_some() { + migrate_project_resource_asset_metadata( + ctx, + project_id, + owner_user_id, + resource_id, + asset_kind, + generation_inputs_json, + now, + ); + } + } + if !changed { + return Ok(layers_json); + } + serde_json::to_string(&layout).map_err(|_| "图片画布图层布局无法序列化".to_string()) +} + +fn migrate_project_resource_asset_metadata( + ctx: &ReducerContext, + project_id: &str, + owner_user_id: &str, + resource_id: String, + asset_kind: Option, + generation_inputs_json: Option, + now: Timestamp, +) { + let Some(resource) = ctx + .db + .editor_project_resource() + .resource_id() + .find(&resource_id) + .filter(|resource| { + resource.project_id == project_id && resource.owner_user_id == owner_user_id + }) + else { + return; + }; + let next_asset_kind = resource.asset_kind.clone().or(asset_kind); + let next_generation_inputs_json = resource + .generation_inputs_json + .clone() + .or(generation_inputs_json); + if next_asset_kind == resource.asset_kind + && next_generation_inputs_json == resource.generation_inputs_json + { + return; + } + ctx.db + .editor_project_resource() + .resource_id() + .delete(&resource.resource_id); + ctx.db + .editor_project_resource() + .insert(EditorProjectResource { + updated_at: now, + asset_kind: next_asset_kind, + generation_inputs_json: next_generation_inputs_json, + ..resource + }); +} + +fn normalize_legacy_layout_string(value: Option<&str>) -> Option { + value + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned) +} + +fn serialize_non_empty_json_value(value: &JsonValue) -> Option { + if value.is_null() { + return None; + } + if value.as_object().is_some_and(JsonMap::is_empty) + || value.as_array().is_some_and(Vec::is_empty) + { + return None; + } + serde_json::to_string(value).ok() +} + fn editor_project_ok(project: Option) -> EditorProjectProcedureResult { EditorProjectProcedureResult { ok: true, diff --git a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx index 89de97128..304584f11 100644 --- a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx @@ -134,6 +134,79 @@ describe('ImageCanvasEditorView generation integration', () => { ); }; + const createGeneratedResource = ({ + resourceId, + projectId = 'editor-project-default', + imageSrc = 'data:image/png;base64,generated', + width = 1024, + height = 1024, + objectKey, + assetObjectId, + assetKind, + generationInputs, + }: { + resourceId: string; + projectId?: string; + imageSrc?: string; + width?: number; + height?: number; + objectKey?: string; + assetObjectId?: string; + assetKind?: string; + generationInputs?: unknown; + }) => ({ + resourceId, + projectId, + imageSrc, + objectKey, + assetObjectId, + width, + height, + sourceType: 'generated', + assetKind, + generationInputs, + createdAt: '2026-06-19T00:00:00.000Z', + updatedAt: '2026-06-19T00:00:00.000Z', + }); + + const createGeneratedAsset = ({ + assetId, + folderId = 'folder-project', + label = '生成素材', + imageSrc = 'data:image/png;base64,generated', + width = 1024, + height = 1024, + objectKey, + assetObjectId, + assetKind, + generationInputs, + }: { + assetId: string; + folderId?: string; + label?: string; + imageSrc?: string; + width?: number; + height?: number; + objectKey?: string; + assetObjectId?: string; + assetKind?: string; + generationInputs?: unknown; + }) => ({ + assetId, + folderId, + label, + imageSrc, + objectKey, + assetObjectId, + width, + height, + sourceType: 'generated', + assetKind, + generationInputs, + createdAt: '2026-06-19T00:00:00.000Z', + updatedAt: '2026-06-19T00:00:00.000Z', + }); + it('restores generated image composer from saved generator snapshots', async () => { loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({ projectId: 'editor-project-generated-dialog', @@ -214,6 +287,17 @@ describe('ImageCanvasEditorView generation integration', () => { model: 'gpt-image-2', provider: 'VectorEngine', taskId: 'editor-real-task-1', + resource: createGeneratedResource({ + resourceId: 'resource-generated-image-1', + imageSrc: 'data:image/png;base64,ZmFrZS1pbWFnZQ==', + assetKind: 'character', + }), + asset: createGeneratedAsset({ + assetId: 'asset-generated-image-1', + label: '生成图片 1', + imageSrc: 'data:image/png;base64,ZmFrZS1pbWFnZQ==', + assetKind: 'character', + }), }); render(); @@ -267,32 +351,23 @@ describe('ImageCanvasEditorView generation integration', () => { ); expect(screen.getByRole('status').textContent).toContain('生成中'); - expect(generateEditorImageMock).toHaveBeenCalledWith({ - prompt: '一张明亮的拼图主视觉', - model: 'gemini-3.1-flash-image-preview', - aspectRatio: '1:1', - imageSize: '1K', - }); + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + prompt: '一张明亮的拼图主视觉', + model: 'gemini-3.1-flash-image-preview', + aspectRatio: '1:1', + imageSize: '1K', + projectId: 'editor-project-default', + assetFolderId: 'project', + assetLabel: expect.stringMatching(/生成图片/u), + }), + ); await waitFor(() => { expect(screen.getByAltText(/画布图片：生成图片/)).toBeTruthy(); }); - await waitFor(() => { - expect(createEditorAssetMock).toHaveBeenCalledWith( - expect.objectContaining({ - folderId: 'project', - label: expect.stringMatching(/生成图片/u), - imageSrc: 'data:image/png;base64,ZmFrZS1pbWFnZQ==', - width: 1024, - height: 1024, - sourceType: 'generated', - prompt: '一张明亮的拼图主视觉', - model: 'gpt-image-2', - provider: 'VectorEngine', - taskId: 'editor-real-task-1', - }), - ); - }); + expect(createEditorAssetMock).not.toHaveBeenCalled(); + expect(createEditorProjectResourceMock).not.toHaveBeenCalled(); fireEvent.click(screen.getByRole('button', { name: '打开素材' })); expect( screen.getByRole('button', { name: /添加生成图片/u }), @@ -942,14 +1017,27 @@ describe('ImageCanvasEditorView generation integration', () => { within(specDialog).getByRole('button', { name: '提交生成规范' }), ); - expect(generateEditorImageMock).toHaveBeenCalledWith({ - kind: 'spec', - model: 'gpt-image-2', - size: '2048x1152', - aspectRatio: '16:9', - imageSize: '2K', - prompt: expect.stringContaining('玩法设计：平台跳跃玩法'), - }); + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'spec', + model: 'gpt-image-2', + size: '2048x1152', + aspectRatio: '16:9', + imageSize: '2K', + prompt: expect.stringContaining('玩法设计：平台跳跃玩法'), + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'spec', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '玩法设定', + value: '平台跳跃玩法', + }), + ]), + }), + }), + ); const prompt = generateEditorImageMock.mock.calls[0]?.[0]?.prompt ?? ''; expect(prompt).toContain('生成2D 角色美术视觉规范设定图'); expect(prompt).toContain('美术风格：低多边形卡通'); @@ -960,16 +1048,6 @@ describe('ImageCanvasEditorView generation integration', () => { expect(screen.getByAltText(/画布图片：角色规范/)).toBeTruthy(); }); expect(screen.getByText('规范')).toBeTruthy(); - await waitFor(() => { - expect(createEditorProjectResourceMock).toHaveBeenCalledWith( - 'editor-project-default', - expect.objectContaining({ - sourceType: 'generated', - width: 2048, - height: 1152, - }), - ); - }); await waitFor(() => { expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( 'editor-project-default', @@ -977,7 +1055,6 @@ describe('ImageCanvasEditorView generation integration', () => { layers: expect.arrayContaining([ expect.objectContaining({ title: expect.stringMatching(/角色规范/u), - assetKind: 'spec', }), ]), }), @@ -1573,14 +1650,19 @@ describe('ImageCanvasEditorView generation integration', () => { ), ); - expect(generateEditorImageMock).toHaveBeenCalledWith({ - kind: 'spec', - model: 'gpt-image-2', - size: '2048x1152', - aspectRatio: '16:9', - imageSize: '2K', - prompt: expect.stringContaining('生成一张完整游戏UI规范汇总设定展板'), - }); + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'spec', + model: 'gpt-image-2', + size: '2048x1152', + aspectRatio: '16:9', + imageSize: '2K', + prompt: expect.stringContaining('生成一张完整游戏UI规范汇总设定展板'), + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'icon-spec', + }), + ); const prompt = generateEditorImageMock.mock.calls[0]?.[0]?.prompt ?? ''; expect(prompt).toContain('玩法设定：消除类派对玩法'); expect(prompt).toContain('美术风格：糖果玻璃拟物'); @@ -1622,14 +1704,19 @@ describe('ImageCanvasEditorView generation integration', () => { ), ); - expect(generateEditorImageMock).toHaveBeenCalledWith({ - kind: 'spec', - model: 'gpt-image-2', - size: '2048x1152', - aspectRatio: '16:9', - imageSize: '2K', - prompt: '生成一张武器图标规范展板', - }); + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'spec', + model: 'gpt-image-2', + size: '2048x1152', + aspectRatio: '16:9', + imageSize: '2K', + prompt: '生成一张武器图标规范展板', + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'spec', + }), + ); await waitFor(() => { expect(screen.getByAltText(/画布图片：自定义规范/)).toBeTruthy(); }); @@ -1808,18 +1895,32 @@ describe('ImageCanvasEditorView generation integration', () => { within(characterPanel).getByRole('button', { name: '生成' }), ); - expect(generateEditorImageMock).toHaveBeenCalledWith({ - kind: 'character', - prompt: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', - model: 'gemini-3.1-flash-image-preview', - aspectRatio: '1:1', - imageSize: '1K', - referenceImageSrcs: [ - '/creation-type-references/puzzle.webp', - '/creation-type-references/big-fish.webp', - expect.stringMatching(/^data:image\/png;base64,/u), - ], - }); + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'character', + prompt: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', + model: 'gemini-3.1-flash-image-preview', + aspectRatio: '1:1', + imageSize: '1K', + referenceImageSrcs: [ + '/creation-type-references/puzzle.webp', + '/creation-type-references/big-fish.webp', + expect.stringMatching(/^data:image\/png;base64,/u), + ], + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'character', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '角色设定', + value: + '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', + }), + ]), + }), + }), + ); await waitFor(() => { expect(screen.getByAltText(/画布图片：角色形象/u)).toBeTruthy(); @@ -1854,7 +1955,6 @@ describe('ImageCanvasEditorView generation integration', () => { layers: expect.arrayContaining([ expect.objectContaining({ title: expect.stringMatching(/角色形象/u), - assetKind: 'character', objectKey: 'generated-character-drafts/editor/character-images/editor-character-1/image.png', assetObjectId: 'asset-object-editor-character-1', @@ -1863,16 +1963,6 @@ describe('ImageCanvasEditorView generation integration', () => { }), ); }); - await waitFor(() => { - expect(createEditorProjectResourceMock).toHaveBeenCalledWith( - 'editor-project-default', - expect.objectContaining({ - objectKey: - 'generated-character-drafts/editor/character-images/editor-character-1/image.png', - assetObjectId: 'asset-object-editor-character-1', - }), - ); - }); }); it('removes the active character generation placeholder with Backspace', async () => { @@ -2023,14 +2113,26 @@ describe('ImageCanvasEditorView generation integration', () => { ); fireEvent.click(within(iconPanel).getByRole('button', { name: '生成' })); - expect(generateEditorIconSpritesheetMock).toHaveBeenCalledWith({ - referenceImageSrc: 'data:image/png;base64,icon-spec', - iconDescriptions: ['返回按钮', '设置按钮'], - model: 'gemini-3.1-flash-image-preview', - aspectRatio: '1:1', - imageSize: '1K', - priceMudPoints: 12, - }); + expect(generateEditorIconSpritesheetMock).toHaveBeenCalledWith( + expect.objectContaining({ + referenceImageSrc: 'data:image/png;base64,icon-spec', + iconDescriptions: ['返回按钮', '设置按钮'], + model: 'gemini-3.1-flash-image-preview', + aspectRatio: '1:1', + imageSize: '1K', + priceMudPoints: 12, + projectId: 'editor-project-icons', + assetFolderId: 'project', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '素材描述', + value: '返回按钮\n设置按钮', + }), + ]), + }), + }), + ); await waitFor(() => { expect(screen.getByAltText('画布图片：返回按钮')).toBeTruthy(); @@ -2052,23 +2154,6 @@ describe('ImageCanvasEditorView generation integration', () => { ).toBeTruthy(); expect(within(iconInfoPanel).getByText('图标规范')).toBeTruthy(); expect(within(iconInfoPanel).getByText('清爽按钮图标规范')).toBeTruthy(); - await waitFor(() => { - expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( - 'editor-project-icons', - expect.objectContaining({ - layers: expect.arrayContaining([ - expect.objectContaining({ - title: '返回按钮', - assetKind: 'icon', - }), - expect.objectContaining({ - title: '设置按钮', - assetKind: 'icon', - }), - ]), - }), - ); - }); }); it('allows icon asset generation to reference icon specs created from the renamed spec entry', async () => { @@ -2124,21 +2209,19 @@ describe('ImageCanvasEditorView generation integration', () => { expect.objectContaining({ kind: 'spec', prompt: expect.stringContaining('背包整理玩法'), - }), - ); - await waitFor(() => { - expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( - 'editor-project-default', - expect.objectContaining({ - layers: expect.arrayContaining([ + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'icon-spec', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ expect.objectContaining({ - title: expect.stringMatching(/图标规范/u), - assetKind: 'icon-spec', + title: '玩法设定', + value: '背包整理玩法', }), ]), }), - ); - }); + }), + ); fireEvent.click(screen.getByRole('button', { name: '生成图标素材' })); const iconPanel = screen.getByRole('dialog', { name: '生成图标素材' }); @@ -2168,14 +2251,18 @@ describe('ImageCanvasEditorView generation integration', () => { ); fireEvent.click(within(iconPanel).getByRole('button', { name: '生成' })); - expect(generateEditorIconSpritesheetMock).toHaveBeenCalledWith({ - referenceImageSrc: 'data:image/png;base64,cmVuYW1lZC1pY29uLXNwZWM=', - iconDescriptions: ['背包按钮'], - model: 'gemini-3.1-flash-image-preview', - aspectRatio: '1:1', - imageSize: '1K', - priceMudPoints: 12, - }); + expect(generateEditorIconSpritesheetMock).toHaveBeenCalledWith( + expect.objectContaining({ + referenceImageSrc: 'data:image/png;base64,cmVuYW1lZC1pY29uLXNwZWM=', + iconDescriptions: ['背包按钮'], + model: 'gemini-3.1-flash-image-preview', + aspectRatio: '1:1', + imageSize: '1K', + priceMudPoints: 12, + projectId: 'editor-project-default', + assetFolderId: 'project', + }), + ); await waitFor(() => { expect(screen.getByAltText('画布图片：背包按钮')).toBeTruthy(); }); @@ -2460,15 +2547,51 @@ describe('ImageCanvasEditorView generation integration', () => { fireEvent.click(screen.getByRole('button', { name: '提取素材' })); await waitFor(() => { - expect(extractEditorUiDesignAssetsMock).toHaveBeenCalledWith({ - sourceImageSrc: 'data:image/png;base64,dWktZGVzaWdu', - }); + expect(extractEditorUiDesignAssetsMock).toHaveBeenCalledWith( + expect.objectContaining({ + sourceImageSrc: 'data:image/png;base64,dWktZGVzaWdu', + projectId: 'editor-project-ui-extract', + assetFolderId: 'project', + spritesheetLabel: '战斗UI设计图 素材图集', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '提取提示词', + value: '提取画面中的所有独立并整理成spritesheet', + }), + ]), + references: expect.arrayContaining([ + expect.objectContaining({ + title: 'UI设计图', + label: '战斗UI设计图', + }), + ]), + }), + }), + ); }); expect( await screen.findByAltText('画布图片：战斗UI设计图 素材图集'), ).toBeTruthy(); expect(screen.getByAltText('画布图片：素材 1')).toBeTruthy(); expect(screen.getByText('图集')).toBeTruthy(); + await waitFor(() => { + expect(createEditorProjectResourceMock).toHaveBeenCalledWith( + 'editor-project-ui-extract', + expect.objectContaining({ + sourceType: 'generated', + assetKind: 'icon', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '提取提示词', + value: '提取画面中的所有独立并整理成spritesheet', + }), + ]), + }), + }), + ); + }); await waitFor(() => { expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( 'editor-project-ui-extract', @@ -2476,11 +2599,11 @@ describe('ImageCanvasEditorView generation integration', () => { layers: expect.arrayContaining([ expect.objectContaining({ title: '战斗UI设计图 素材图集', - assetKind: 'icon-spritesheet', + resourceId: 'resource-editor-project-ui-extract-512', }), expect.objectContaining({ title: '素材 1', - assetKind: 'icon', + resourceId: 'resource-editor-project-ui-extract-96', }), ]), }), @@ -2614,16 +2737,31 @@ describe('ImageCanvasEditorView generation integration', () => { ); await waitFor(() => { - expect(generateEditorImageMock).toHaveBeenCalledWith({ - prompt: '参考图2增加萤火虫', - size: '1536x1024', - kind: 'quick-edit', - model: 'gpt-image-2', - referenceImageSrcs: [ - 'data:image/png;base64,ZXh0cmEtcmVm', - 'data:image/png;base64,c291cmNl', - ], - }); + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + prompt: '参考图2增加萤火虫', + size: '1536x1024', + kind: 'quick-edit', + model: 'gpt-image-2', + referenceImageSrcs: [ + 'data:image/png;base64,ZXh0cmEtcmVm', + 'data:image/png;base64,c291cmNl', + ], + projectId: 'editor-project-quick-edit', + assetKind: 'spec', + assetFolderId: 'project', + assetLabel: '魔法森林 快速编辑', + sourceResourceId: 'resource-quick-source', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '快速编辑提示词', + value: '参考图2增加萤火虫', + }), + ]), + }), + }), + ); }); await waitFor(() => { expect(screen.getByAltText('画布图片：魔法森林 快速编辑')).toBeTruthy(); @@ -2642,7 +2780,7 @@ describe('ImageCanvasEditorView generation integration', () => { layers: expect.arrayContaining([ expect.objectContaining({ title: '魔法森林 快速编辑', - assetKind: 'spec', + resourceId: 'resource-editor-project-quick-edit-1536', width: 1536, height: 1024, originalWidth: 1536, @@ -2721,6 +2859,10 @@ describe('ImageCanvasEditorView generation integration', () => { size: '1024x1024', model: 'gpt-image-2', kind: 'quick-edit', + projectId: 'editor-project-context-quick-edit', + assetFolderId: 'project', + assetLabel: '右键图片 快速编辑', + sourceResourceId: 'resource-context-source', }), ); }); @@ -2800,6 +2942,10 @@ describe('ImageCanvasEditorView generation integration', () => { prompt: '改成陶泥风格', kind: 'quick-edit', referenceImageSrcs: ['data:image/webp;base64,aGVsbG8='], + projectId: 'editor-project-public-quick-edit', + assetFolderId: 'project', + assetLabel: '公开拼图素材 快速编辑', + sourceResourceId: 'resource-public-source', }), ); }); diff --git a/src/components/image-editor/ImageCanvasEditorModel.test.ts b/src/components/image-editor/ImageCanvasEditorModel.test.ts index 63524aa1e..c1efa3dd5 100644 --- a/src/components/image-editor/ImageCanvasEditorModel.test.ts +++ b/src/components/image-editor/ImageCanvasEditorModel.test.ts @@ -164,10 +164,24 @@ describe('ImageCanvasEditorModel', () => { const snapshot = serializeLayer(layer); expect(snapshot).not.toHaveProperty('src'); + expect(snapshot).not.toHaveProperty('assetKind'); + expect(snapshot).not.toHaveProperty('generationInputs'); const hydrated = hydrateLayer( snapshot, - new Map([['resource-generated', { imageSrc: '/read/generated.png' }]]), + new Map([ + [ + 'resource-generated', + { + imageSrc: '/read/generated.png', + assetKind: 'character', + generationInputs: { + fields: [{ title: '角色设定', value: '骑士' }], + references: [], + }, + }, + ], + ]), ); expect(hydrated).toMatchObject({ @@ -178,6 +192,7 @@ describe('ImageCanvasEditorModel', () => { objectKey: 'generated/object.png', locked: true, }); + expect(hydrated?.generationInputs?.fields[0]?.value).toBe('骑士'); }); it('hydrates manually selected publication material tags from saved layout', () => { diff --git a/src/components/image-editor/ImageCanvasEditorModel.ts b/src/components/image-editor/ImageCanvasEditorModel.ts index 72f8aec97..cc81f791c 100644 --- a/src/components/image-editor/ImageCanvasEditorModel.ts +++ b/src/components/image-editor/ImageCanvasEditorModel.ts @@ -144,7 +144,8 @@ export function createLayerFromAsset( assetObjectId: asset.assetObjectId, durationSeconds: asset.durationSeconds, sourceAssetId: asset.id, - assetKind, + assetKind: asset.assetKind ?? assetKind, + generationInputs: asset.generationInputs, } satisfies CanvasLayer; } @@ -175,8 +176,6 @@ export function serializeLayer( sourceResourceId: layer.sourceResourceId, sourceAssetId: layer.sourceAssetId, groupId: layer.groupId, - assetKind: layer.assetKind, - generationInputs: layer.generationInputs, hidden: layer.hidden, locked: layer.locked, flipX: layer.flipX, @@ -364,13 +363,14 @@ export function hydrateCanvasGenerationDialog( export function hydrateLayer( snapshot: EditorProjectLayerSnapshot, - resourcesById: Map, + resourcesById: Map, ): CanvasLayer | null { const resourceId = typeof snapshot.resourceId === 'string' ? snapshot.resourceId : ''; const layerId = typeof snapshot.layerId === 'string' ? snapshot.layerId : ''; + const resource = resourcesById.get(resourceId); const snapshotSrc = typeof snapshot.src === 'string' ? snapshot.src : ''; - const src = snapshotSrc || resourcesById.get(resourceId)?.imageSrc || ''; + const src = snapshotSrc || resource?.imageSrc || ''; const title = typeof snapshot.title === 'string' ? snapshot.title : '画布图片'; if (!resourceId || !layerId || !src) { @@ -413,16 +413,16 @@ export function hydrateLayer( assetObjectId: stringOrNull(snapshot.assetObjectId), durationSeconds: audioDurationOrNull(snapshot.durationSeconds) ?? - audioDurationOrNull( - resourcesById.get(resourceId) as - | ({ imageSrc: string } & { durationSeconds?: unknown }) - | undefined, - ), + audioDurationOrNull(resource?.durationSeconds), sourceResourceId: stringOrNull(snapshot.sourceResourceId), sourceAssetId: stringOrNull(snapshot.sourceAssetId), groupId: stringOrNull(snapshot.groupId), - assetKind: canvasAssetKindOrNull(snapshot.assetKind), - generationInputs: generationInputsOrNull(snapshot.generationInputs), + assetKind: + canvasAssetKindOrNull(resource?.assetKind) ?? + canvasAssetKindOrNull(snapshot.assetKind), + generationInputs: + generationInputsOrNull(resource?.generationInputs) ?? + generationInputsOrNull(snapshot.generationInputs), hidden: booleanFromSnapshot(snapshot.hidden), locked: booleanFromSnapshot(snapshot.locked), flipX: booleanFromSnapshot(snapshot.flipX), @@ -467,6 +467,8 @@ export function mapAssetLibrarySnapshot( taskId: asset.taskId ?? undefined, objectKey: asset.objectKey ?? undefined, assetObjectId: asset.assetObjectId ?? undefined, + assetKind: canvasAssetKindOrNull(asset.assetKind), + generationInputs: generationInputsOrNull(asset.generationInputs), durationSeconds: asset.durationSeconds ?? undefined, }; }), @@ -494,6 +496,13 @@ function inferAudioAssetKindFromLabel(label: string): CanvasAssetKind { : 'sound-effect'; } +export type CanvasLayerResourceMetadata = { + imageSrc: string; + durationSeconds?: number | null; + assetKind?: string | null; + generationInputs?: unknown; +}; + export function normalizeAssetLibrary(library: EditorAssetLibrarySnapshot) { const mapped = mapAssetLibrarySnapshot(library); let hasDefaultFolder = false; diff --git a/src/components/image-editor/ImageCanvasEditorTypes.ts b/src/components/image-editor/ImageCanvasEditorTypes.ts index a49bff48d..f009b3c7f 100644 --- a/src/components/image-editor/ImageCanvasEditorTypes.ts +++ b/src/components/image-editor/ImageCanvasEditorTypes.ts @@ -1,4 +1,5 @@ import type { + EditorAssetSnapshot, EditorCharacterAnimationFrameCount, EditorCharacterAnimationGenerationResult, EditorCharacterAnimationRatio, @@ -41,6 +42,8 @@ export type EditorAsset = { taskId?: string; objectKey?: string; assetObjectId?: string; + assetKind?: CanvasAssetKind | null; + generationInputs?: CanvasGenerationInputs | null; durationSeconds?: number; uploadStatus?: 'uploading' | 'failed'; uploadProgress?: number; @@ -86,6 +89,7 @@ export type CanvasLayer = { assetObjectId?: string | null; sourceResourceId?: string | null; sourceAssetId?: string | null; + generatedAssetSnapshot?: EditorAssetSnapshot | null; groupId?: string | null; assetKind?: CanvasAssetKind | null; generationInputs?: CanvasGenerationInputs | null; diff --git a/src/components/image-editor/ImageCanvasEditorView.test-utils.ts b/src/components/image-editor/ImageCanvasEditorView.test-utils.ts index 06c680a77..6338a1ceb 100644 --- a/src/components/image-editor/ImageCanvasEditorView.test-utils.ts +++ b/src/components/image-editor/ImageCanvasEditorView.test-utils.ts @@ -266,6 +266,8 @@ export function setupImageCanvasEditorViewTestLifecycle({ width: input.width, height: input.height, sourceType: input.sourceType, + assetKind: input.assetKind, + generationInputs: input.generationInputs, durationSeconds: input.durationSeconds, })); createEditorAssetFolderMock.mockResolvedValue({ diff --git a/src/components/image-editor/ImageCanvasEditorView.tsx b/src/components/image-editor/ImageCanvasEditorView.tsx index bd669a4f7..b48ab6846 100644 --- a/src/components/image-editor/ImageCanvasEditorView.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.tsx @@ -11,10 +11,15 @@ import { import { createEditorAsset, createEditorProjectResource, + type EditorAssetSnapshot, } from '../../services/image-editor/editorProjectClient'; import { useAuthUi } from '../auth/AuthUiContext'; import { PlatformDangerConfirmDialog } from '../common/PlatformDangerConfirmDialog'; -import { resolveContextMenuPosition } from './ImageCanvasEditorModel'; +import { + canvasAssetKindOrNull, + generationInputsOrNull, + resolveContextMenuPosition, +} from './ImageCanvasEditorModel'; import { ImageCanvasEditorShellView } from './ImageCanvasEditorShellView'; import type { AssetPointerDragState, @@ -343,6 +348,8 @@ export function ImageCanvasEditorView() { provider: layer.provider, taskId: layer.taskId, durationSeconds: layer.durationSeconds, + assetKind: layer.assetKind, + generationInputs: layer.generationInputs, }) .then((asset) => { setAssets((currentAssets) => [ @@ -368,6 +375,8 @@ export function ImageCanvasEditorView() { objectKey: asset.objectKey ?? undefined, assetObjectId: asset.assetObjectId ?? undefined, durationSeconds: asset.durationSeconds ?? layer.durationSeconds, + assetKind: canvasAssetKindOrNull(asset.assetKind), + generationInputs: generationInputsOrNull(asset.generationInputs), }, ]); setAssetFolders((currentFolders) => @@ -406,6 +415,55 @@ export function ImageCanvasEditorView() { }, [activeUploadFolderId, openEditorLoginModal, setAssetFolders, setAssets], ); + const upsertGeneratedAsset = useCallback( + (asset: EditorAssetSnapshot) => { + setAssets((currentAssets) => [ + ...currentAssets.filter( + (currentAsset) => currentAsset.id !== asset.assetId, + ), + { + id: asset.assetId, + label: asset.label, + src: asset.imageSrc, + width: asset.width, + height: asset.height, + folderId: asset.folderId, + sourceKind: 'uploaded', + sourceType: asset.sourceType, + persisted: true, + prompt: asset.prompt ?? undefined, + actualPrompt: asset.actualPrompt ?? undefined, + model: asset.model ?? undefined, + provider: asset.provider ?? undefined, + taskId: asset.taskId ?? undefined, + objectKey: asset.objectKey ?? undefined, + assetObjectId: asset.assetObjectId ?? undefined, + durationSeconds: asset.durationSeconds ?? undefined, + assetKind: canvasAssetKindOrNull(asset.assetKind), + generationInputs: generationInputsOrNull(asset.generationInputs), + }, + ]); + setAssetFolders((currentFolders) => + currentFolders.some((folder) => folder.id === asset.folderId) + ? currentFolders.map((folder) => + folder.id === asset.folderId + ? { ...folder, collapsed: false } + : folder, + ) + : [ + ...currentFolders, + { + id: asset.folderId, + label: '项目素材', + collapsed: false, + systemDefault: true, + persisted: true, + }, + ], + ); + }, + [setAssetFolders, setAssets], + ); const projectPersistenceRefs = useMemo( () => ({ layersRef, @@ -548,6 +606,9 @@ export function ImageCanvasEditorView() { requestUpload, persistGeneratedAsset, persistUpdatedLayerResource, + projectId, + assetFolderId: activeUploadFolderId, + upsertGeneratedAsset, }); const { setQuickEditPanel, diff --git a/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts b/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts index 83db30a25..3e963fde9 100644 --- a/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts @@ -98,6 +98,57 @@ describe('ImageCanvasGenerationLayerModel', () => { expect(layer.generationInputs?.fields[0]?.value).toBe('生成提示词'); }); + it('uses backend-persisted resource and asset snapshots for generated layers', () => { + const generationInputs = createGenerationInputs(); + const layer = createGeneratedResultLayer({ + generated: createGenerated({ + resource: { + resourceId: 'resource-generated-bff', + projectId: 'editor-project-1', + imageSrc: 'data:image/png;base64,generated', + objectKey: 'generated/from-resource.png', + assetObjectId: 'asset-object-from-resource', + width: 1024, + height: 1024, + sourceType: 'generated', + assetKind: 'character', + generationInputs, + }, + asset: { + assetId: 'asset-generated-bff', + folderId: 'user-1:asset-folder:project', + label: '角色形象', + imageSrc: 'data:image/png;base64,generated', + objectKey: 'generated/from-asset.png', + assetObjectId: 'asset-object-from-asset', + width: 1024, + height: 1024, + sourceType: 'generated', + assetKind: 'character', + generationInputs, + }, + }), + generatedIndex: 9, + canvasSize: { width: 900, height: 640 }, + viewport: { x: 0, y: 0, scale: 1 }, + assetKind: 'spec', + generationInputs: { + fields: [{ title: '旧输入', value: '本地兜底' }], + references: [], + }, + }); + + expect(layer).toMatchObject({ + resourceId: 'resource-generated-bff', + sourceAssetId: 'asset-generated-bff', + objectKey: 'generated/from-resource.png', + assetObjectId: 'asset-object-from-resource', + assetKind: 'character', + }); + expect(layer.generationInputs?.fields[0]?.value).toBe('生成提示词'); + expect(layer.generatedAssetSnapshot?.assetId).toBe('asset-generated-bff'); + }); + it('creates edit result layers beside the source image', () => { const sourceLayer = createSourceLayer(); const layer = createGeneratedResultLayer({ diff --git a/src/components/image-editor/ImageCanvasGenerationLayerModel.ts b/src/components/image-editor/ImageCanvasGenerationLayerModel.ts index db2b19e04..3f1c17cc1 100644 --- a/src/components/image-editor/ImageCanvasGenerationLayerModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationLayerModel.ts @@ -101,16 +101,24 @@ function applyGeneratedMetadata( generated: EditorImageGenerationResult, generationInputs: CanvasGenerationInputs | undefined, ): CanvasLayer { + const resource = generated.resource; + const asset = generated.asset; return { ...layer, + resourceId: resource?.resourceId ?? layer.resourceId, + sourceAssetId: asset?.assetId ?? layer.sourceAssetId, prompt: generated.prompt, actualPrompt: generated.actualPrompt ?? generated.prompt, model: generated.model, provider: generated.provider, taskId: generated.taskId, - objectKey: generated.objectKey, - assetObjectId: generated.assetObjectId, - generationInputs, + objectKey: resource?.objectKey ?? generated.objectKey, + assetObjectId: resource?.assetObjectId ?? generated.assetObjectId, + assetKind: + (resource?.assetKind as CanvasLayer['assetKind'] | undefined) ?? + layer.assetKind, + generationInputs: resource?.generationInputs ?? generationInputs, + generatedAssetSnapshot: asset, }; } @@ -232,6 +240,8 @@ export function createIconSpritesheetResultLayers({ frame, spritesheetTitle, }: IconSpritesheetResultLayerOptions): CanvasLayer[] { + const spritesheetResource = generated.spritesheetResource; + const spritesheetAsset = generated.spritesheetAsset; const worldCenter = getViewportWorldCenter({ canvasSize, viewport }); const startX = frame?.x ?? worldCenter.x - ICON_FRAME_DISPLAY_SIZE.width / 2; @@ -250,7 +260,9 @@ export function createIconSpritesheetResultLayers({ ); const spritesheetLayer: CanvasLayer = { id: `layer-icon-spritesheet-${startIndex}`, - resourceId: `local-resource-icon-spritesheet-${startIndex}`, + resourceId: + spritesheetResource?.resourceId ?? + `local-resource-icon-spritesheet-${startIndex}`, title: spritesheetTitle ?? `图标素材图集 ${startIndex}`, src: generated.spritesheetImageSrc, x: startX, @@ -267,7 +279,12 @@ export function createIconSpritesheetResultLayers({ provider: generated.provider, taskId: generated.taskId, assetKind: 'icon-spritesheet', - generationInputs, + sourceAssetId: spritesheetAsset?.assetId, + objectKey: spritesheetResource?.objectKey ?? undefined, + assetObjectId: spritesheetResource?.assetObjectId ?? undefined, + generationInputs: + spritesheetResource?.generationInputs ?? generationInputs, + generatedAssetSnapshot: spritesheetAsset, }; const iconStartX = startX + spritesheetDisplaySize.width + 32; let cursorX = iconStartX; @@ -275,6 +292,8 @@ export function createIconSpritesheetResultLayers({ let rowHeight = 0; const iconLayers = iconResults.map((icon, index) => { + const resource = icon.resource; + const asset = icon.asset; const originalWidth = icon.width || 128; const originalHeight = icon.height || 128; const { width, height } = resolveLayerResolutionSize( @@ -291,7 +310,7 @@ export function createIconSpritesheetResultLayers({ const generatedIndex = startIndex + index + 1; const layer: CanvasLayer = { id: `layer-icon-${generatedIndex}`, - resourceId: `local-resource-icon-${generatedIndex}`, + resourceId: resource?.resourceId ?? `local-resource-icon-${generatedIndex}`, title: icon.name, src: icon.imageSrc, x: cursorX, @@ -308,7 +327,11 @@ export function createIconSpritesheetResultLayers({ provider: generated.provider, taskId: generated.taskId, assetKind: 'icon', - generationInputs, + sourceAssetId: asset?.assetId, + objectKey: resource?.objectKey ?? undefined, + assetObjectId: resource?.assetObjectId ?? undefined, + generationInputs: resource?.generationInputs ?? generationInputs, + generatedAssetSnapshot: asset, }; cursorX += width + spacing; diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts index 70b2697ed..9a41257a0 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts @@ -58,6 +58,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { imageSize: '1K', }, result: { + title: '生成图片 3', generationInputs: { fields: [{ title: '生成提示词', value: '一张发光主视觉' }], references: [], diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts index 755dbf9e4..b9c507bf5 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts @@ -453,6 +453,7 @@ export function buildImageGenerationSubmissionPlan({ : {}), }, result: { + title: `生成图片 ${nextGeneratedIndex}`, generationInputs: buildImageGenerationInputs( normalizedPrompt, dialog.generationReferences, diff --git a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts index 4a59ede00..35bdfe346 100644 --- a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts +++ b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts @@ -20,6 +20,7 @@ import { generateEditorSoundEffect, generateEditorVideo, } from '../../services/image-editor/editorProjectClient'; +import type { EditorAssetSnapshot } from '../../services/image-editor/editorProjectClient'; import type { CanvasGenerationDialogState, CanvasGenerationInputs, @@ -96,7 +97,9 @@ type GenerationSubmissionWorkflowOptions = { setActiveTool: Dispatch>; setActiveSidebarPanel: Dispatch>; rememberImageModel: (imageModel: string) => void; - persistGeneratedAsset?: (layer: CanvasLayer) => void; + projectId?: string | null; + assetFolderId?: string | null; + upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void; }; async function normalizePublicationReferenceImages( @@ -142,14 +145,21 @@ export function useImageCanvasGenerationSubmissionWorkflow({ setActiveTool, setActiveSidebarPanel, rememberImageModel, - persistGeneratedAsset, + projectId, + assetFolderId, + upsertGeneratedAsset, }: GenerationSubmissionWorkflowOptions) { const addGeneratedLayersToCanvas = useCallback( (nextLayers: CanvasLayer[]) => { appendCanvasLayersWithResources(nextLayers); - nextLayers.forEach((layer) => persistGeneratedAsset?.(layer)); + nextLayers.forEach((layer) => { + const asset = layer.generatedAssetSnapshot; + if (asset) { + upsertGeneratedAsset?.(asset); + } + }); }, - [appendCanvasLayersWithResources, persistGeneratedAsset], + [appendCanvasLayersWithResources, upsertGeneratedAsset], ); const addGeneratedResultLayer = useCallback( @@ -510,8 +520,14 @@ export function useImageCanvasGenerationSubmissionWorkflow({ const sourceImageSrc = await resolveEditorImageReferenceDataUrl( sourceLayer.objectKey?.trim() || sourceLayer.src, ); + const generationInputs = + buildUiDesignAssetExtractionGenerationInputs(sourceLayer); const generated = await extractEditorUiDesignAssets({ sourceImageSrc, + projectId, + generationInputs, + assetFolderId, + spritesheetLabel: `${sourceLayer.title} 素材图集`, }); const extractionFrame = { x: sourceLayer.x + sourceLayer.width + 32, @@ -524,7 +540,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({ const nextLayers = addIconSpritesheetResultLayers( generated, generated.iconImageSrcs, - buildUiDesignAssetExtractionGenerationInputs(sourceLayer), + generationInputs, extractionFrame, undefined, { @@ -542,7 +558,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({ ); } }, - [addIconSpritesheetResultLayers, fitLayers], + [addIconSpritesheetResultLayers, fitLayers, projectId, assetFolderId], ); const submitIconSpritesheetGeneration = useCallback( @@ -583,9 +599,12 @@ export function useImageCanvasGenerationSubmissionWorkflow({ }); try { - const generated = await generateEditorIconSpritesheet( - submissionPlan.input, - ); + const generated = await generateEditorIconSpritesheet({ + ...submissionPlan.input, + projectId, + generationInputs: submissionPlan.generationInputs, + assetFolderId, + }); rememberImageModel(submissionPlan.rememberImageModel); addIconSpritesheetResultLayers( generated, @@ -608,6 +627,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({ addIconSpritesheetResultLayers, getGeneratingDialogPlaceholder, rememberImageModel, + projectId, + assetFolderId, updateCanvasGenerationDialogById, ], ); @@ -663,19 +684,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({ const sourceReferenceImageSrc = await resolveEditorImageReferenceDataUrl( quickEditSourceLayer.objectKey?.trim() || quickEditSourceLayer.src, ); - const generated = await generateEditorImage({ - prompt: normalizedPrompt, - size: quickEditPanel.size, - kind: 'quick-edit', - model: normalizeEditorImageModel(quickEditPanel.model), - referenceImageSrcs: [ - ...extraReferenceImageSrcs, - sourceReferenceImageSrc, - ], - }); - addQuickEditResultLayer( - generated, - quickEditSourceLayer, + const generationInputs = panelMode === 'redraw' ? buildEditGenerationInputs( '重绘提示词', @@ -687,7 +696,29 @@ export function useImageCanvasGenerationSubmissionWorkflow({ normalizedPrompt, quickEditSourceLayer, quickEditReferences, - ), + ); + const generated = await generateEditorImage({ + prompt: normalizedPrompt, + size: quickEditPanel.size, + kind: 'quick-edit', + model: normalizeEditorImageModel(quickEditPanel.model), + referenceImageSrcs: [ + ...extraReferenceImageSrcs, + sourceReferenceImageSrc, + ], + projectId, + assetKind: quickEditSourceLayer.assetKind, + generationInputs, + assetFolderId, + assetLabel: `${quickEditSourceLayer.title} ${ + panelMode === 'redraw' ? '重绘' : '快速编辑' + }`, + sourceResourceId: quickEditSourceLayer.resourceId, + }); + addQuickEditResultLayer( + generated, + quickEditSourceLayer, + generationInputs, panelMode, quickEditDialogId ? { @@ -722,6 +753,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({ addQuickEditResultLayer, getGeneratingDialogPlaceholder, openCanvasGenerationDialog, + projectId, + assetFolderId, quickEditPanel, quickEditSourceLayer, setQuickEditPanel, @@ -764,11 +797,18 @@ export function useImageCanvasGenerationSubmissionWorkflow({ }); if (submissionPlan.kind === 'edit') { const referenceImageSrc = await resolveEditorImageReferenceDataUrl( - submissionPlan.sourceLayer.src, + submissionPlan.sourceLayer.objectKey?.trim() || + submissionPlan.sourceLayer.src, ); const generated = await editEditorImage({ prompt: submissionPlan.normalizedPrompt, sourceImageSrc: referenceImageSrc, + projectId, + assetKind: submissionPlan.sourceLayer.assetKind, + generationInputs: submissionPlan.generationInputs, + assetFolderId, + assetLabel: `${submissionPlan.sourceLayer.title} 修改结果`, + sourceResourceId: submissionPlan.sourceLayer.resourceId, }); addGeneratedResultLayer(generated, { sourceLayer: submissionPlan.sourceLayer, @@ -800,7 +840,14 @@ export function useImageCanvasGenerationSubmissionWorkflow({ submissionPlan.input.kind === 'publication-material' ? await normalizePublicationReferenceImages(submissionPlan.input) : submissionPlan.input; - const generated = await generateEditorImage(imageGenerationInput); + const generated = await generateEditorImage({ + ...imageGenerationInput, + projectId, + assetKind: submissionPlan.result.assetKind, + generationInputs: submissionPlan.result.generationInputs, + assetFolderId, + assetLabel: submissionPlan.result.title, + }); if (submissionPlan.rememberImageModel) { rememberImageModel(submissionPlan.rememberImageModel); } @@ -840,6 +887,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({ layerCounterRef, layers, rememberImageModel, + projectId, + assetFolderId, setGenerateDialog, updateCanvasGenerationDialogById, ], diff --git a/src/components/image-editor/useImageCanvasGenerationSurface.tsx b/src/components/image-editor/useImageCanvasGenerationSurface.tsx index d7ad480a1..38d7dc666 100644 --- a/src/components/image-editor/useImageCanvasGenerationSurface.tsx +++ b/src/components/image-editor/useImageCanvasGenerationSurface.tsx @@ -15,6 +15,7 @@ import { PlatformFloatingMenu, PlatformFloatingMenuItem, } from '../common/PlatformFloatingMenu'; +import type { EditorAssetSnapshot } from '../../services/image-editor/editorProjectClient'; import type { CanvasGenerationDialogState, CanvasLayer, @@ -87,6 +88,9 @@ type ImageCanvasGenerationSurfaceOptions = { requestUpload: (target: UploadTarget) => void; persistGeneratedAsset?: (layer: CanvasLayer) => void; persistUpdatedLayerResource?: (layer: CanvasLayer) => void; + projectId?: string | null; + assetFolderId?: string | null; + upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void; }; function renderEditorPortal(node: ReactNode) { @@ -160,6 +164,9 @@ export function useImageCanvasGenerationSurface({ requestUpload, persistGeneratedAsset, persistUpdatedLayerResource, + projectId, + assetFolderId, + upsertGeneratedAsset, }: ImageCanvasGenerationSurfaceOptions) { const toolbarOptionCloseTimerRef = useRef { model: 'gemini-3.1-flash-image-preview', aspectRatio: '1:1', imageSize: '1K', + projectId: undefined, + assetFolderId: undefined, + assetKind: undefined, + assetLabel: '生成图片 1', + generationInputs: { + fields: [{ title: '生成提示词', value: '一张生成图' }], + references: [], + }, }); expect(screen.getByTestId('dialog').textContent).toBe( 'generate:generating:closed:-:placeholder', @@ -644,7 +652,21 @@ describe('useImageCanvasGenerationWorkflow', () => { model: 'gemini-3.1-flash-image-preview', aspectRatio: '1:1', imageSize: '1K', + projectId: undefined, + assetFolderId: undefined, + assetKind: undefined, + assetLabel: '生成图片 1', referenceImageSrcs: ['data:image/png;base64,source'], + generationInputs: { + fields: [{ title: '生成提示词', value: '一张生成图' }], + references: [ + { + title: '参考图 1', + label: '画布参考图', + src: 'data:image/png;base64,source', + }, + ], + }, }); }); }); @@ -706,6 +728,21 @@ describe('useImageCanvasGenerationWorkflow', () => { kind: 'quick-edit', model: 'gemini-3.1-flash-image-preview', referenceImageSrcs: ['data:image/png;base64,source'], + projectId: undefined, + assetFolderId: undefined, + assetKind: undefined, + assetLabel: '源图 快速编辑', + sourceResourceId: 'resource-source', + generationInputs: { + fields: [{ title: '快速编辑提示词', value: '快速修图' }], + references: [ + { + title: '参考图 1', + label: '源图', + src: 'data:image/png;base64,source', + }, + ], + }, }); }); diff --git a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts index abd6348af..7ddaadcfa 100644 --- a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts +++ b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts @@ -10,6 +10,7 @@ import { useState, } from 'react'; +import type { EditorAssetSnapshot } from '../../services/image-editor/editorProjectClient'; import type { CanvasGenerationDialogState, CanvasLayer, @@ -183,6 +184,9 @@ type GenerationWorkflowOptions = { setImageContextMenu: Dispatch>; persistGeneratedAsset?: (layer: CanvasLayer) => void; persistUpdatedLayerResource?: (layer: CanvasLayer) => void; + projectId?: string | null; + assetFolderId?: string | null; + upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void; }; export function useImageCanvasGenerationWorkflow({ @@ -211,6 +215,9 @@ export function useImageCanvasGenerationWorkflow({ setImageContextMenu, persistGeneratedAsset, persistUpdatedLayerResource, + projectId, + assetFolderId, + upsertGeneratedAsset, }: GenerationWorkflowOptions) { const [isSpecMenuOpen, setIsSpecMenuOpen] = useState(false); const [isGenerationReferenceMenuOpen, setIsGenerationReferenceMenuOpen] = @@ -966,7 +973,9 @@ export function useImageCanvasGenerationWorkflow({ setActiveTool, setActiveSidebarPanel, rememberImageModel: setLastImageModel, - persistGeneratedAsset, + projectId, + assetFolderId, + upsertGeneratedAsset, }); const setEffectiveCharacterAnimationPanel: Dispatch< diff --git a/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx b/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx index b916c92be..ff9566569 100644 --- a/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx +++ b/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx @@ -29,7 +29,10 @@ vi.mock('../../services/image-editor/editorProjectClient', async () => { }; }); -function createLayer(id: string): CanvasLayer { +function createLayer( + id: string, + overrides: Partial = {}, +): CanvasLayer { return { id, resourceId: `local-${id}`, @@ -44,6 +47,7 @@ function createLayer(id: string): CanvasLayer { zIndex: 3, sourceType: 'uploaded', sourceAssetId: 'asset-a', + ...overrides, }; } @@ -118,7 +122,14 @@ function ProjectPersistenceHarness({ {projectTitle} {projectRenameValue} - {layers.map((layer) => `${layer.id}:${layer.resourceId}`).join(',')} + {layers + .map( + (layer) => + `${layer.id}:${layer.resourceId}:${layer.assetKind ?? '-'}:${ + layer.generationInputs?.fields[0]?.value ?? '-' + }`, + ) + .join(',')} {selectedLayerRef.current ?? '-'} {layerCounterRef.current} @@ -135,6 +146,38 @@ function ProjectPersistenceHarness({ > append + +