GenarrativeAI/Genarrative
11
0
Fork 0
Code Issues 19 Pull Requests 12 Actions Packages Projects 1 Releases Wiki Activity

后端保存画布生成素材

Browse Source 
生成接口接收项目与素材上下文并返回 resource/asset 快照

画布前端使用后端回包落层并避免重复创建资源和素材

补充生成元数据保存边界的测试与文档
This commit is contained in:
段舒康
2026-06-19 16:30:56 +08:00
parent 76445da9b4
commit e644d7c99f
17 changed files with 1029 additions and 231 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/project-memory/shared-memory/decision-log.md
+1 -1
View File
@@ -19,7 +19,7 @@
## 2026-06-19 图片画布素材生成元数据上移到资源和素材
- 背景：角色、图标、UI 设计图、视频和音频等生成结果会在图片信息页展示用户可见输入快照；此前这些 `assetKind/generationInputs` 主要保存在画布 layer JSON 中，素材进入账号级素材库后跨项目复用和刷新恢复都依赖画布布局，不符合素材库作为账号级事实源的边界。
- 决策：普通图层的新保存不再把 `assetKind/generationInputs` 写入 `editor_canvas.layers_json`。`editor_project_resource` 保存项目画布资源快照的 `asset_kind/generation_inputs_json`，`editor_asset` 保存账号级素材的同名元数据；前端加载时优先从 resource / asset 恢复素材类别和生成输入快照，旧 layout 中的同名字段只作为历史兼容兜底。生成器对象本身仍作为 `itemType="generation-dialog"` 保存在画布布局中。
- 决策：普通图层的新保存不再把 `assetKind/generationInputs` 写入 `editor_canvas.layers_json`。`editor_project_resource` 保存项目画布资源快照的 `asset_kind/generation_inputs_json`，`editor_asset` 保存账号级素材的同名元数据；图片 / 图标 / UI 提取等生成 BFF 在请求携带 `projectId` / `assetFolderId` 时由后端创建新 resource / asset 并把快照回传前端，前端只用回包更新画布图层和素材栏，不再把同一生成结果二次调用保存接口。前端加载时优先从 resource / asset 恢复素材类别和生成输入快照，旧 layout 中的同名字段只作为历史兼容兜底。生成器对象本身仍作为 `itemType="generation-dialog"` 保存在画布布局中。
- 影响范围：`server-rs/crates/spacetime-module/src/editor_project_storage.rs`、`server-rs/crates/spacetime-client/src/mapper/editor_project.rs`、`server-rs/crates/api-server/src/editor_project.rs`、`src/services/image-editor/editorProjectClient.ts`、图片画布 hydrate / serialize / project persistence / asset library 代码和后端数据契约文档。
- 验证方式：运行 `npm run spacetime:generate`、`npm run check:spacetime-schema`、`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`，并按需补充 `cargo check -p spacetime-client -p api-server --manifest-path server-rs/Cargo.toml`。
- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`。
docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md
+6 -6
View File
@@ -40,11 +40,11 @@
- 新增 `editor_canvas` 表保存工程下的画布：`canvasId`、`projectId`、`ownerUserId`、标题、viewport、图层布局 JSON、创建时间和更新时间。当前编辑器使用项目默认画布，后续可扩展为一个 project 下多个 canvas。
- 新增 `editor_asset_folder` 表保存账号级素材文件夹：`folderId`、`ownerUserId`、名称、排序、折叠状态、系统默认标记、创建时间和更新时间。素材文件夹不归属于 project，同一个账号进入任一项目都能看到。
- 新增 `editor_asset` 表保存账号级素材：`assetId`、`ownerUserId`、`folderId`、名称、图片读取地址、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、`assetKind`、`generationInputs`、创建时间和更新时间。素材只跟账号走，不跟 project 走；角色、图标、UI 设计图、视频和音频等生成结果的用户可见输入快照随素材保存。
- `editor_project_resource` 表保存工程画布引用过的资源快照：`resourceId`、`projectId`、`ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind`、`generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource，图层只引用 resourceId。
- `editor_project_resource` 表保存工程画布引用过的资源快照：`resourceId`、`projectId`、`ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind`、`generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource，图层只引用 resourceId；图片、图标和 UI 素材生成 BFF 在请求携带 `projectId` 时由后端直接创建新 resource，并把 `resourceId` 随生成响应返回给前端。
- 图片文件本体继续走 OSS，浏览器读取私有 generated 对象仍经 `/api/assets/read-url` 换签。
- 当前 MVP 的本地上传先以 data URL 持久化在素材记录中，保证刷新和跨项目可见；后续接入正式 OSS 上传时，只替换 `imageSrc/objectKey/assetObjectId` 的写入方式，账号级素材表和画布资源表不变。
- 资源表保存资源和素材级元数据；图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组：普通图层按 `layerId/resourceId` 保存，生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存，不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON；刷新时优先从 `editor_project_resource` 恢复，旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`；生成成功后仍保存该快照，只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。图层展示尺寸不再作为独立 `Size` 真相保存，刷新与新建图层均按 `Resolution`（`originalWidth/originalHeight`）原分辨率显示。图层组第一版是画布内布局语义，不单独建表。
- 图片类生成结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；视频结果当前只保存为画布视频资源，不进入图片素材库。
- 图片类生成结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；该写入由生成 BFF 在请求携带 `assetFolderId` 时完成，前端只把响应中的 resource / asset 快照用于落画布和更新素材栏，不再把同一生成结果二次调用素材创建接口。视频结果当前只保存为画布视频资源，不进入图片素材库。
- 前端不直接订阅 SpacetimeDB，统一通过 api-server 的 `/api/editor/projects*` BFF 读写。
- 未登录用户可以使用本地演示态，但不触发工程自动保存；真实图片生成 / 修改需要登录。编辑器 API 请求允许使用 refresh cookie 静默补 access token，但 401 / 403 只在编辑器局部提示登录，不清空整站登录态，也不把后端 requestId 直接作为生图弹窗主文案。
@@ -65,10 +65,10 @@
- `POST /api/editor/assets`：批量或单个创建账号级素材，支持按钮上传和拖拽上传后的 data URL / 后续 OSS 元数据。
- `PATCH /api/editor/assets/{assetId}`：重命名素材或移动素材到文件夹。
- `DELETE /api/editor/assets/{assetId}`：删除素材。已放入画布的 project resource 不被级联删除，避免旧画布丢图。
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；角色生成可携带 `model`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits。携带参考图的快速编辑也走该接口，前端必须把参考图源预读成图片 Data URL 后放入 `referenceImageSrcs`；接口返回 data URL、尺寸、prompt、model、provider 和 taskId。
- `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet，再由后端切分为独立透明图标。请求支持 `model`、`aspectRatio`、`imageSize` 和 `priceMudPoints`；`priceMudPoints` 必须来自编辑器生成计费配置的 `icon` 档位（首版 12 泥点），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`，后端把 UI 尺寸归一为文档支持的 `1024x1024`、`1024x1536`、`1536x1024`、`2048x2048`、`2048x1152` 等 `size` 字符串。
- `POST /api/editor/ui-designs/assets/extractions`：以 UI 设计图 Data URL 作为参考图，固定 `gpt-image-2` 和提示词 `提取画面中的所有独立并整理成spritesheet` 生成素材 spritesheet，再按连通域自动拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。前端必须把 spritesheet 原图与拆分素材都加入画布。
- `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据。
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；角色生成可携带 `model`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits。携带参考图的快速编辑也走该接口，前端必须把参考图源预读成图片 Data URL 后放入 `referenceImageSrcs`；请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成成功后创建 project resource / 账号素材并在响应中返回 resource / asset 快照。
- `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet，再由后端切分为独立透明图标。请求支持 `model`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置的 `icon` 档位（首版 12 泥点），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。后端把 spritesheet 和拆分后的 icon 都保存为 project resource / 账号素材，并随响应返回对应快照。
- `POST /api/editor/ui-designs/assets/extractions`：以 UI 设计图 Data URL 作为参考图，固定 `gpt-image-2` 和提示词 `提取画面中的所有独立并整理成spritesheet` 生成素材 spritesheet，再按连通域自动拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，后端保存 spritesheet / 拆分素材并返回对应 resource / asset 快照；前端必须把 spritesheet 原图与拆分素材都加入画布。
- `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据；请求携带 project / asset 上下文时由后端创建新 resource / asset，前端只消费响应快照。
- `POST /api/editor/videos/generations`：按视频描述、模型、比例、时长、分辨率、模式、声音和泥点价格生成视频。前端可选模型为 `seedance2.0-fast`、`seedance2.0`、`kling3.0`、`kling3.0-omni`，默认 `seedance2.0-fast`；后端必须将 `seedance2.0-fast` 映射到 `doubao-seedance-2-0-fast-260128`，将 `seedance2.0` 映射到 `doubao-seedance-2-0-260128`，两者不得混用。后端复用 Ark / VectorEngine content generation task 轮询链路，下载最终视频并持久化到 OSS，返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`。
所有写接口都必须校验 Bearer 登录态和 owner；接口只返回当前用户有权读取的工程与资源。
docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md
+2 -2
View File
@@ -447,7 +447,7 @@ npm run check:server-rs-ddd
- Rust 结构体：`EditorProjectResource`
- 源码：`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
- 说明：图片画布工程资源元数据表，保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task、源资源关系、`asset_kind` 和 `generation_inputs_json`。`asset_kind` 标记角色、图标、UI 设计图、视频、音频等素材类别；`generation_inputs_json` 保存用户可见生成输入快照，供图片信息页刷新后恢复。账号级素材删除不级联删除该表，避免历史画布丢图。`editor_canvas.layers_json` 只保存图层几何、层级、分组、资源引用和生成器对象；新写入不再把素材生成输入快照作为图层布局真相保存，旧布局字段只作为兼容兜底读取。
- 说明：图片画布工程资源元数据表，保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task、源资源关系、`asset_kind` 和 `generation_inputs_json`。`asset_kind` 标记角色、图标、UI 设计图、视频、音频等素材类别；`generation_inputs_json` 保存用户可见生成输入快照，供图片信息页刷新后恢复。图片 / 图标 / UI 提取等生成 BFF 在请求携带 `project_id` 时负责创建该表记录并把 resource 快照返回前端；前端只保存布局引用，不能把同一生成结果再次作为正式业务真相写入。账号级素材删除不级联删除该表，避免历史画布丢图。`editor_canvas.layers_json` 只保存图层几何、层级、分组、资源引用和生成器对象；新写入不再把素材生成输入快照作为图层布局真相保存，旧布局字段只作为兼容兜底读取。
- 索引：`by_editor_project_resource_project_id`、`by_editor_project_resource_owner_user_id`。
### `editor_asset_folder`
@@ -461,7 +461,7 @@ npm run check:server-rs-ddd
- Rust 结构体：`EditorAsset`
- 源码：`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
- 说明：图片画布账号级素材表，保存用户上传 / 生成素材的名称、文件夹、图片读取地址、OSS 引用、尺寸、来源类型、prompt、provider、task、`asset_kind` 和 `generation_inputs_json`。素材在同一账号的所有项目中可见；素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId，画布从 resource / asset 级元数据恢复素材类别和用户可见生成输入快照。
- 说明：图片画布账号级素材表，保存用户上传 / 生成素材的名称、文件夹、图片读取地址、OSS 引用、尺寸、来源类型、prompt、provider、task、`asset_kind` 和 `generation_inputs_json`。素材在同一账号的所有项目中可见；图片 / 图标 / UI 提取等生成 BFF 在请求携带 `asset_folder_id` 时负责创建账号级生成素材并返回 asset 快照，前端只用该快照更新素材栏。素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId，画布从 resource / asset 级元数据恢复素材类别和用户可见生成输入快照。
- 索引：`by_editor_asset_owner_user_id`、`by_editor_asset_folder_id`。
### `inventory_slot`
server-rs/crates/api-server/src/editor_project.rs
+348 -9
View File
@@ -61,6 +61,7 @@ const EDITOR_ASSET_ID_PREFIX: &str = "editor-asset-";
const EDITOR_LAYOUT_MAX_BYTES: usize = 256 * 1024;
const EDITOR_ASSET_METADATA_MAX_BYTES: usize = 64 * 1024;
const EDITOR_PROJECT_DEFAULT_TITLE: &str = "未命名画布";
const EDITOR_ASSET_DEFAULT_FOLDER_ID: &str = "project";
const EDITOR_IMAGE_GENERATION_SIZE: &str = "1024x1024";
const EDITOR_IMAGE_MODEL_NANOBANANA2: &str = "gemini-3.1-flash-image-preview";
const EDITOR_IMAGE_MODEL_NANOBANANA2_DISPLAY_ALIAS: &str = "nanobanana2";
@@ -169,6 +170,12 @@ pub struct EditorImageGenerationRequest {
aspect_ratio: Option<String>,
image_size: Option<String>,
reference_image_srcs: Option<Vec<String>>,
project_id: Option<String>,
asset_kind: Option<String>,
generation_inputs: Option<Value>,
asset_folder_id: Option<String>,
asset_label: Option<String>,
source_resource_id: Option<String>,
}
#[derive(Debug, Deserialize)]
@@ -176,6 +183,12 @@ pub struct EditorImageGenerationRequest {
pub struct EditorImageEditRequest {
prompt: String,
source_image_src: String,
project_id: Option<String>,
asset_kind: Option<String>,
generation_inputs: Option<Value>,
asset_folder_id: Option<String>,
asset_label: Option<String>,
source_resource_id: Option<String>,
}
#[derive(Debug, Deserialize)]
@@ -187,12 +200,19 @@ pub struct EditorIconSpritesheetGenerationRequest {
aspect_ratio: Option<String>,
image_size: Option<String>,
price_mud_points: u32,
project_id: Option<String>,
generation_inputs: Option<Value>,
asset_folder_id: Option<String>,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct EditorUiDesignAssetExtractionRequest {
source_image_src: String,
project_id: Option<String>,
generation_inputs: Option<Value>,
asset_folder_id: Option<String>,
spritesheet_label: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -249,6 +269,33 @@ pub struct EditorAssetResponse {
asset: EditorAssetPayload,
}
#[derive(Debug)]
struct EditorGeneratedAssetRecord {
resource: Option<EditorProjectResourcePayload>,
asset: Option<EditorAssetPayload>,
}
#[derive(Debug)]
struct PersistEditorGeneratedAssetInput {
project_id: Option<String>,
owner_user_id: String,
folder_id: Option<String>,
label: String,
image_src: String,
object_key: Option<String>,
asset_object_id: Option<String>,
width: u32,
height: u32,
prompt: String,
actual_prompt: Option<String>,
model: String,
provider: String,
task_id: String,
source_resource_id: Option<String>,
asset_kind: Option<String>,
generation_inputs: Option<Value>,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct EditorImageGenerationResponse {
@@ -263,6 +310,8 @@ pub struct EditorImageGenerationResponse {
model: String,
provider: &'static str,
task_id: String,
resource: Option<EditorProjectResourcePayload>,
asset: Option<EditorAssetPayload>,
}
#[derive(Debug, Serialize)]
@@ -272,6 +321,8 @@ pub struct EditorIconSpritesheetIconResponse {
image_src: String,
width: u32,
height: u32,
resource: Option<EditorProjectResourcePayload>,
asset: Option<EditorAssetPayload>,
}
#[derive(Debug, Serialize)]
@@ -287,6 +338,8 @@ pub struct EditorIconSpritesheetGenerationResponse {
provider: &'static str,
task_id: String,
price_mud_points: u32,
spritesheet_resource: Option<EditorProjectResourcePayload>,
spritesheet_asset: Option<EditorAssetPayload>,
}
#[derive(Debug, Serialize)]
@@ -967,6 +1020,32 @@ pub async fn generate_editor_image(
image.mime_type,
BASE64_STANDARD.encode(image.bytes.as_slice())
);
let generated_asset = persist_editor_generated_asset(
&state,
PersistEditorGeneratedAssetInput {
project_id: payload.project_id,
owner_user_id: current_owner_user_id(&authenticated),
folder_id: payload.asset_folder_id,
label: normalize_optional_string(payload.asset_label)
.unwrap_or_else(|| role_setting.clone()),
image_src: image_src.clone(),
object_key: persisted.as_ref().map(|asset| asset.object_key.clone()),
asset_object_id: persisted
.as_ref()
.map(|asset| asset.asset_object_id.clone()),
width,
height,
prompt: role_setting.clone(),
actual_prompt: generated.actual_prompt.clone(),
model: generation_options.model.to_string(),
provider: "VectorEngine".to_string(),
task_id: generated.task_id.clone(),
source_resource_id: payload.source_resource_id,
asset_kind: payload.asset_kind,
generation_inputs: payload.generation_inputs,
},
)
.await?;
Ok(json_success_body(
Some(&request_context),
@@ -984,6 +1063,8 @@ pub async fn generate_editor_image(
model: generation_options.model.to_string(),
provider: "VectorEngine",
task_id: generated.task_id,
resource: generated_asset.resource,
asset: generated_asset.asset,
},
))
}
@@ -1155,6 +1236,30 @@ pub async fn edit_editor_image(
image.mime_type,
BASE64_STANDARD.encode(image.bytes.as_slice())
);
let generated_asset = persist_editor_generated_asset(
&state,
PersistEditorGeneratedAssetInput {
project_id: payload.project_id,
owner_user_id: current_owner_user_id(&authenticated),
folder_id: payload.asset_folder_id,
label: normalize_optional_string(payload.asset_label)
.unwrap_or_else(|| "图片修改结果".to_string()),
image_src: image_src.clone(),
object_key: None,
asset_object_id: None,
width,
height,
prompt: prompt.clone(),
actual_prompt: generated.actual_prompt.clone(),
model: GPT_IMAGE_2_MODEL.to_string(),
provider: "VectorEngine".to_string(),
task_id: generated.task_id.clone(),
source_resource_id: payload.source_resource_id,
asset_kind: payload.asset_kind,
generation_inputs: payload.generation_inputs,
},
)
.await?;
Ok(json_success_body(
Some(&request_context),
@@ -1170,6 +1275,8 @@ pub async fn edit_editor_image(
model: GPT_IMAGE_2_MODEL.to_string(),
provider: "VectorEngine",
task_id: generated.task_id,
resource: generated_asset.resource,
asset: generated_asset.asset,
},
))
}
@@ -1260,10 +1367,63 @@ pub async fn generate_editor_icon_spritesheet(
})?;
let spritesheet_image_src =
data_url_from_image_bytes(image.mime_type.as_str(), image.bytes.as_slice());
let icon_image_srcs = icon_slices
.into_iter()
.map(editor_icon_response_from_slice)
.collect();
let generation_inputs = payload.generation_inputs.clone();
let owner_user_id = current_owner_user_id(&authenticated);
let spritesheet_record = persist_editor_generated_asset(
&state,
PersistEditorGeneratedAssetInput {
project_id: payload.project_id.clone(),
owner_user_id: owner_user_id.clone(),
folder_id: payload.asset_folder_id.clone(),
label: "图标素材图集".to_string(),
image_src: spritesheet_image_src.clone(),
object_key: None,
asset_object_id: None,
width: spritesheet_width,
height: spritesheet_height,
prompt: prompt.clone(),
actual_prompt: generated.actual_prompt.clone(),
model: generation_options.model.to_string(),
provider: "VectorEngine".to_string(),
task_id: generated.task_id.clone(),
source_resource_id: None,
asset_kind: Some("icon-spritesheet".to_string()),
generation_inputs: generation_inputs.clone(),
},
)
.await?;
let mut icon_image_srcs = Vec::new();
for icon in icon_slices {
let icon_response = editor_icon_response_from_slice(icon);
let icon_record = persist_editor_generated_asset(
&state,
PersistEditorGeneratedAssetInput {
project_id: payload.project_id.clone(),
owner_user_id: owner_user_id.clone(),
folder_id: payload.asset_folder_id.clone(),
label: icon_response.name.clone(),
image_src: icon_response.image_src.clone(),
object_key: None,
asset_object_id: None,
width: icon_response.width,
height: icon_response.height,
prompt: prompt.clone(),
actual_prompt: generated.actual_prompt.clone(),
model: generation_options.model.to_string(),
provider: "VectorEngine".to_string(),
task_id: generated.task_id.clone(),
source_resource_id: None,
asset_kind: Some("icon".to_string()),
generation_inputs: generation_inputs.clone(),
},
)
.await?;
icon_image_srcs.push(EditorIconSpritesheetIconResponse {
resource: icon_record.resource,
asset: icon_record.asset,
..icon_response
});
}
Ok(json_success_body(
Some(&request_context),
@@ -1278,6 +1438,8 @@ pub async fn generate_editor_icon_spritesheet(
provider: "VectorEngine",
task_id: generated.task_id,
price_mud_points: expected_price_mud_points,
spritesheet_resource: spritesheet_record.resource,
spritesheet_asset: spritesheet_record.asset,
},
))
}
@@ -1337,10 +1499,65 @@ pub async fn extract_editor_ui_design_assets(
})?;
let spritesheet_image_src =
data_url_from_image_bytes(image.mime_type.as_str(), image.bytes.as_slice());
let icon_image_srcs = icon_slices
.into_iter()
.map(editor_icon_response_from_slice)
.collect();
let prompt = EDITOR_UI_DESIGN_ASSET_EXTRACTION_PROMPT.to_string();
let generation_inputs = payload.generation_inputs.clone();
let owner_user_id = current_owner_user_id(&authenticated);
let spritesheet_record = persist_editor_generated_asset(
&state,
PersistEditorGeneratedAssetInput {
project_id: payload.project_id.clone(),
owner_user_id: owner_user_id.clone(),
folder_id: payload.asset_folder_id.clone(),
label: normalize_optional_string(payload.spritesheet_label)
.unwrap_or_else(|| "UI设计图素材图集".to_string()),
image_src: spritesheet_image_src.clone(),
object_key: None,
asset_object_id: None,
width: spritesheet_width,
height: spritesheet_height,
prompt: prompt.clone(),
actual_prompt: generated.actual_prompt.clone(),
model: GPT_IMAGE_2_MODEL.to_string(),
provider: "VectorEngine".to_string(),
task_id: generated.task_id.clone(),
source_resource_id: None,
asset_kind: Some("icon-spritesheet".to_string()),
generation_inputs: generation_inputs.clone(),
},
)
.await?;
let mut icon_image_srcs = Vec::new();
for icon in icon_slices {
let icon_response = editor_icon_response_from_slice(icon);
let icon_record = persist_editor_generated_asset(
&state,
PersistEditorGeneratedAssetInput {
project_id: payload.project_id.clone(),
owner_user_id: owner_user_id.clone(),
folder_id: payload.asset_folder_id.clone(),
label: icon_response.name.clone(),
image_src: icon_response.image_src.clone(),
object_key: None,
asset_object_id: None,
width: icon_response.width,
height: icon_response.height,
prompt: prompt.clone(),
actual_prompt: generated.actual_prompt.clone(),
model: GPT_IMAGE_2_MODEL.to_string(),
provider: "VectorEngine".to_string(),
task_id: generated.task_id.clone(),
source_resource_id: None,
asset_kind: Some("icon".to_string()),
generation_inputs: generation_inputs.clone(),
},
)
.await?;
icon_image_srcs.push(EditorIconSpritesheetIconResponse {
resource: icon_record.resource,
asset: icon_record.asset,
..icon_response
});
}
Ok(json_success_body(
Some(&request_context),
@@ -1349,7 +1566,7 @@ pub async fn extract_editor_ui_design_assets(
spritesheet_width,
spritesheet_height,
icon_image_srcs,
prompt: EDITOR_UI_DESIGN_ASSET_EXTRACTION_PROMPT.to_string(),
prompt,
actual_prompt: generated.actual_prompt,
model: GPT_IMAGE_2_MODEL.to_string(),
provider: "VectorEngine",
@@ -1358,6 +1575,8 @@ pub async fn extract_editor_ui_design_assets(
Some("icon"),
Some(GPT_IMAGE_2_MODEL),
),
spritesheet_resource: spritesheet_record.resource,
spritesheet_asset: spritesheet_record.asset,
},
))
}
@@ -1478,6 +1697,96 @@ fn editor_asset_payload_from_record(record: EditorAssetRecord) -> EditorAssetPay
}
}
async fn persist_editor_generated_asset(
state: &AppState,
input: PersistEditorGeneratedAssetInput,
) -> Result<EditorGeneratedAssetRecord, AppError> {
let generation_inputs_json = serialize_editor_asset_metadata(input.generation_inputs.clone())?;
let now_micros = current_utc_micros();
let resource = if let Some(project_id) = normalize_optional_string(input.project_id.clone()) {
let record = state
.spacetime_client()
.create_editor_project_resource(EditorProjectResourceCreateRecordInput {
resource_id: build_prefixed_uuid_id(EDITOR_RESOURCE_ID_PREFIX),
project_id,
owner_user_id: input.owner_user_id.clone(),
asset_object_id: input.asset_object_id.clone(),
image_src: input.image_src.clone(),
object_key: input.object_key.clone(),
width: input.width,
height: input.height,
source_type: "generated".to_string(),
prompt: Some(input.prompt.clone()),
actual_prompt: input.actual_prompt.clone(),
model: Some(input.model.clone()),
provider: Some(input.provider.clone()),
task_id: Some(input.task_id.clone()),
source_resource_id: normalize_optional_string(input.source_resource_id.clone()),
asset_kind: normalize_optional_string(input.asset_kind.clone()),
generation_inputs_json: generation_inputs_json.clone(),
updated_at_micros: now_micros,
})
.await
.map_err(map_editor_project_error)?;
Some(editor_project_resource_payload_from_record(record))
} else {
None
};
let folder_id =
normalize_generated_asset_folder_id(input.folder_id, input.owner_user_id.as_str());
let asset = if let Some(folder_id) = folder_id {
let record = state
.spacetime_client()
.create_editor_asset(EditorAssetCreateRecordInput {
asset_id: build_prefixed_uuid_id(EDITOR_ASSET_ID_PREFIX),
owner_user_id: input.owner_user_id.clone(),
folder_id,
label: input.label,
asset_object_id: input.asset_object_id,
image_src: input.image_src,
object_key: input.object_key,
width: input.width,
height: input.height,
source_type: "generated".to_string(),
prompt: Some(input.prompt),
actual_prompt: input.actual_prompt,
model: Some(input.model),
provider: Some(input.provider),
task_id: Some(input.task_id),
asset_kind: normalize_optional_string(input.asset_kind),
generation_inputs_json,
now_micros,
})
.await
.map_err(map_editor_project_error)?;
Some(editor_asset_payload_from_record(record))
} else {
None
};
Ok(EditorGeneratedAssetRecord { resource, asset })
}
fn normalize_generated_asset_folder_id(
folder_id: Option<String>,
owner_user_id: &str,
) -> Option<String> {
let normalized = normalize_optional_string(folder_id);
match normalized.as_deref() {
None => None,
Some("project") => Some(default_editor_asset_folder_id(owner_user_id)),
Some(value) if value.starts_with("folder-") => {
Some(default_editor_asset_folder_id(owner_user_id))
}
Some(_) => normalized,
}
}
fn default_editor_asset_folder_id(owner_user_id: &str) -> String {
format!("{owner_user_id}:asset-folder:{EDITOR_ASSET_DEFAULT_FOLDER_ID}")
}
impl EditorCanvasViewportPayload {
fn into_record(self) -> EditorCanvasViewportRecord {
EditorCanvasViewportRecord {
@@ -1681,6 +1990,8 @@ fn editor_icon_response_from_slice(
image_src: data_url_from_image_bytes("image/png", icon.bytes.as_slice()),
width: icon.width,
height: icon.height,
resource: None,
asset: None,
}
}
@@ -2128,6 +2439,8 @@ mod tests {
image_src: "data:image/png;base64,back".to_string(),
width: 128,
height: 128,
resource: None,
asset: None,
}],
prompt: "图标 prompt".to_string(),
actual_prompt: Some("图标 prompt".to_string()),
@@ -2138,9 +2451,35 @@ mod tests {
Some("icon"),
None,
),
spritesheet_resource: None,
spritesheet_asset: None,
};
let payload = serde_json::to_value(response).expect("response should serialize");
assert_eq!(payload["priceMudPoints"], json!(12));
}
#[test]
fn generated_asset_folder_id_normalizes_project_and_legacy_local_folder() {
assert_eq!(
normalize_generated_asset_folder_id(Some("project".to_string()), "user-1"),
Some("user-1:asset-folder:project".to_string())
);
assert_eq!(
normalize_generated_asset_folder_id(Some(" folder-temporary ".to_string()), "user-1"),
Some("user-1:asset-folder:project".to_string())
);
assert_eq!(
normalize_generated_asset_folder_id(Some("folder-real".to_string()), "user-1"),
Some("user-1:asset-folder:project".to_string())
);
assert_eq!(
normalize_generated_asset_folder_id(
Some("user-1:asset-folder:project".to_string()),
"user-1",
),
Some("user-1:asset-folder:project".to_string())
);
assert_eq!(normalize_generated_asset_folder_id(None, "user-1"), None);
}
}
server-rs/crates/spacetime-module/src/editor_project_storage.rs
+9 -11
View File
@@ -1424,11 +1424,7 @@ fn migrate_canvas_layer_asset_metadata(
let Some(object) = item.as_object_mut() else {
continue;
};
if object
.get("itemType")
.and_then(JsonValue::as_str)
.is_some()
{
if object.get("itemType").and_then(JsonValue::as_str).is_some() {
continue;
}
let resource_id = object
@@ -1499,12 +1495,14 @@ fn migrate_project_resource_asset_metadata(
.editor_project_resource()
.resource_id()
.delete(&resource.resource_id);
ctx.db.editor_project_resource().insert(EditorProjectResource {
updated_at: now,
asset_kind: next_asset_kind,
generation_inputs_json: next_generation_inputs_json,
..resource
});
ctx.db
.editor_project_resource()
.insert(EditorProjectResource {
updated_at: now,
asset_kind: next_asset_kind,
generation_inputs_json: next_generation_inputs_json,
..resource
});
}
fn normalize_legacy_layout_string(value: Option<&str>) -> Option<String> {
src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx
+268 -164
View File
File diff suppressed because it is too large Load Diff
src/components/image-editor/ImageCanvasEditorTypes.ts
+2
View File
@@ -1,4 +1,5 @@
import type {
EditorAssetSnapshot,
EditorCharacterAnimationFrameCount,
EditorCharacterAnimationGenerationResult,
EditorCharacterAnimationRatio,
@@ -88,6 +89,7 @@ export type CanvasLayer = {
assetObjectId?: string | null;
sourceResourceId?: string | null;
sourceAssetId?: string | null;
generatedAssetSnapshot?: EditorAssetSnapshot | null;
groupId?: string | null;
assetKind?: CanvasAssetKind | null;
generationInputs?: CanvasGenerationInputs | null;
src/components/image-editor/ImageCanvasEditorView.tsx
+53
View File
@@ -11,6 +11,7 @@ import {
import {
createEditorAsset,
createEditorProjectResource,
type EditorAssetSnapshot,
} from '../../services/image-editor/editorProjectClient';
import { useAuthUi } from '../auth/AuthUiContext';
import { PlatformDangerConfirmDialog } from '../common/PlatformDangerConfirmDialog';
@@ -411,6 +412,55 @@ export function ImageCanvasEditorView() {
},
[activeUploadFolderId, openEditorLoginModal, setAssetFolders, setAssets],
);
const upsertGeneratedAsset = useCallback(
(asset: EditorAssetSnapshot) => {
setAssets((currentAssets) => [
...currentAssets.filter(
(currentAsset) => currentAsset.id !== asset.assetId,
),
{
id: asset.assetId,
label: asset.label,
src: asset.imageSrc,
width: asset.width,
height: asset.height,
folderId: asset.folderId,
sourceKind: 'uploaded',
sourceType: asset.sourceType,
persisted: true,
prompt: asset.prompt ?? undefined,
actualPrompt: asset.actualPrompt ?? undefined,
model: asset.model ?? undefined,
provider: asset.provider ?? undefined,
taskId: asset.taskId ?? undefined,
objectKey: asset.objectKey ?? undefined,
assetObjectId: asset.assetObjectId ?? undefined,
durationSeconds: asset.durationSeconds ?? undefined,
assetKind: canvasAssetKindOrNull(asset.assetKind),
generationInputs: generationInputsOrNull(asset.generationInputs),
},
]);
setAssetFolders((currentFolders) =>
currentFolders.some((folder) => folder.id === asset.folderId)
? currentFolders.map((folder) =>
folder.id === asset.folderId
? { ...folder, collapsed: false }
: folder,
)
: [
...currentFolders,
{
id: asset.folderId,
label: '项目素材',
collapsed: false,
systemDefault: true,
persisted: true,
},
],
);
},
[setAssetFolders, setAssets],
);
const projectPersistenceRefs = useMemo(
() => ({
layersRef,
@@ -552,6 +602,9 @@ export function ImageCanvasEditorView() {
requestUpload,
persistGeneratedAsset,
persistUpdatedLayerResource,
projectId,
assetFolderId: activeUploadFolderId,
upsertGeneratedAsset,
});
const {
setQuickEditPanel,
src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts
+51
View File
@@ -97,6 +97,57 @@ describe('ImageCanvasGenerationLayerModel', () => {
expect(layer.generationInputs?.fields[0]?.value).toBe('生成提示词');
});
it('uses backend-persisted resource and asset snapshots for generated layers', () => {
const generationInputs = createGenerationInputs();
const layer = createGeneratedResultLayer({
generated: createGenerated({
resource: {
resourceId: 'resource-generated-bff',
projectId: 'editor-project-1',
imageSrc: 'data:image/png;base64,generated',
objectKey: 'generated/from-resource.png',
assetObjectId: 'asset-object-from-resource',
width: 1024,
height: 1024,
sourceType: 'generated',
assetKind: 'character',
generationInputs,
},
asset: {
assetId: 'asset-generated-bff',
folderId: 'user-1:asset-folder:project',
label: '角色形象',
imageSrc: 'data:image/png;base64,generated',
objectKey: 'generated/from-asset.png',
assetObjectId: 'asset-object-from-asset',
width: 1024,
height: 1024,
sourceType: 'generated',
assetKind: 'character',
generationInputs,
},
}),
generatedIndex: 9,
canvasSize: { width: 900, height: 640 },
viewport: { x: 0, y: 0, scale: 1 },
assetKind: 'spec',
generationInputs: {
fields: [{ title: '旧输入', value: '本地兜底' }],
references: [],
},
});
expect(layer).toMatchObject({
resourceId: 'resource-generated-bff',
sourceAssetId: 'asset-generated-bff',
objectKey: 'generated/from-resource.png',
assetObjectId: 'asset-object-from-resource',
assetKind: 'character',
});
expect(layer.generationInputs?.fields[0]?.value).toBe('生成提示词');
expect(layer.generatedAssetSnapshot?.assetId).toBe('asset-generated-bff');
});
it('creates edit result layers beside the source image', () => {
const sourceLayer = createSourceLayer();
const layer = createGeneratedResultLayer({
src/components/image-editor/ImageCanvasGenerationLayerModel.ts
+30 -7
View File
@@ -101,16 +101,24 @@ function applyGeneratedMetadata(
generated: EditorImageGenerationResult,
generationInputs: CanvasGenerationInputs | undefined,
): CanvasLayer {
const resource = generated.resource;
const asset = generated.asset;
return {
...layer,
resourceId: resource?.resourceId ?? layer.resourceId,
sourceAssetId: asset?.assetId ?? layer.sourceAssetId,
prompt: generated.prompt,
actualPrompt: generated.actualPrompt ?? generated.prompt,
model: generated.model,
provider: generated.provider,
taskId: generated.taskId,
objectKey: generated.objectKey,
assetObjectId: generated.assetObjectId,
generationInputs,
objectKey: resource?.objectKey ?? generated.objectKey,
assetObjectId: resource?.assetObjectId ?? generated.assetObjectId,
assetKind:
(resource?.assetKind as CanvasLayer['assetKind'] | undefined) ??
layer.assetKind,
generationInputs: resource?.generationInputs ?? generationInputs,
generatedAssetSnapshot: asset,
};
}
@@ -232,6 +240,8 @@ export function createIconSpritesheetResultLayers({
frame,
spritesheetTitle,
}: IconSpritesheetResultLayerOptions): CanvasLayer[] {
const spritesheetResource = generated.spritesheetResource;
const spritesheetAsset = generated.spritesheetAsset;
const worldCenter = getViewportWorldCenter({ canvasSize, viewport });
const startX =
frame?.x ?? worldCenter.x - ICON_FRAME_DISPLAY_SIZE.width / 2;
@@ -250,7 +260,9 @@ export function createIconSpritesheetResultLayers({
);
const spritesheetLayer: CanvasLayer = {
id: `layer-icon-spritesheet-${startIndex}`,
resourceId: `local-resource-icon-spritesheet-${startIndex}`,
resourceId:
spritesheetResource?.resourceId ??
`local-resource-icon-spritesheet-${startIndex}`,
title: spritesheetTitle ?? `图标素材图集 ${startIndex}`,
src: generated.spritesheetImageSrc,
x: startX,
@@ -267,7 +279,12 @@ export function createIconSpritesheetResultLayers({
provider: generated.provider,
taskId: generated.taskId,
assetKind: 'icon-spritesheet',
generationInputs,
sourceAssetId: spritesheetAsset?.assetId,
objectKey: spritesheetResource?.objectKey ?? undefined,
assetObjectId: spritesheetResource?.assetObjectId ?? undefined,
generationInputs:
spritesheetResource?.generationInputs ?? generationInputs,
generatedAssetSnapshot: spritesheetAsset,
};
const iconStartX = startX + spritesheetDisplaySize.width + 32;
let cursorX = iconStartX;
@@ -275,6 +292,8 @@ export function createIconSpritesheetResultLayers({
let rowHeight = 0;
const iconLayers = iconResults.map((icon, index) => {
const resource = icon.resource;
const asset = icon.asset;
const originalWidth = icon.width || 128;
const originalHeight = icon.height || 128;
const { width, height } = resolveLayerResolutionSize(
@@ -291,7 +310,7 @@ export function createIconSpritesheetResultLayers({
const generatedIndex = startIndex + index + 1;
const layer: CanvasLayer = {
id: `layer-icon-${generatedIndex}`,
resourceId: `local-resource-icon-${generatedIndex}`,
resourceId: resource?.resourceId ?? `local-resource-icon-${generatedIndex}`,
title: icon.name,
src: icon.imageSrc,
x: cursorX,
@@ -308,7 +327,11 @@ export function createIconSpritesheetResultLayers({
provider: generated.provider,
taskId: generated.taskId,
assetKind: 'icon',
generationInputs,
sourceAssetId: asset?.assetId,
objectKey: resource?.objectKey ?? undefined,
assetObjectId: resource?.assetObjectId ?? undefined,
generationInputs: resource?.generationInputs ?? generationInputs,
generatedAssetSnapshot: asset,
};
cursorX += width + spacing;
src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts
+74 -29
View File
@@ -20,6 +20,7 @@ import {
generateEditorSoundEffect,
generateEditorVideo,
} from '../../services/image-editor/editorProjectClient';
import type { EditorAssetSnapshot } from '../../services/image-editor/editorProjectClient';
import type {
CanvasGenerationDialogState,
CanvasGenerationInputs,
@@ -96,7 +97,9 @@ type GenerationSubmissionWorkflowOptions = {
setActiveTool: Dispatch<SetStateAction<CanvasTool>>;
setActiveSidebarPanel: Dispatch<SetStateAction<SidebarPanel | null>>;
rememberImageModel: (imageModel: string) => void;
persistGeneratedAsset?: (layer: CanvasLayer) => void;
projectId?: string | null;
assetFolderId?: string | null;
upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void;
};
async function normalizePublicationReferenceImages(
@@ -142,18 +145,21 @@ export function useImageCanvasGenerationSubmissionWorkflow({
setActiveTool,
setActiveSidebarPanel,
rememberImageModel,
persistGeneratedAsset,
projectId,
assetFolderId,
upsertGeneratedAsset,
}: GenerationSubmissionWorkflowOptions) {
const addGeneratedLayersToCanvas = useCallback(
(nextLayers: CanvasLayer[]) => {
appendCanvasLayersWithResources(nextLayers);
nextLayers
.filter(
(layer) => layer.mediaType !== 'video' && layer.mediaType !== 'audio',
)
.forEach((layer) => persistGeneratedAsset?.(layer));
nextLayers.forEach((layer) => {
const asset = layer.generatedAssetSnapshot;
if (asset) {
upsertGeneratedAsset?.(asset);
}
});
},
[appendCanvasLayersWithResources, persistGeneratedAsset],
[appendCanvasLayersWithResources, upsertGeneratedAsset],
);
const addGeneratedResultLayer = useCallback(
@@ -514,8 +520,14 @@ export function useImageCanvasGenerationSubmissionWorkflow({
const sourceImageSrc = await resolveEditorImageReferenceDataUrl(
sourceLayer.objectKey?.trim() || sourceLayer.src,
);
const generationInputs =
buildUiDesignAssetExtractionGenerationInputs(sourceLayer);
const generated = await extractEditorUiDesignAssets({
sourceImageSrc,
projectId,
generationInputs,
assetFolderId,
spritesheetLabel: `${sourceLayer.title} 素材图集`,
});
const extractionFrame = {
x: sourceLayer.x + sourceLayer.width + 32,
@@ -528,7 +540,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
const nextLayers = addIconSpritesheetResultLayers(
generated,
generated.iconImageSrcs,
buildUiDesignAssetExtractionGenerationInputs(sourceLayer),
generationInputs,
extractionFrame,
undefined,
{
@@ -546,7 +558,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
);
}
},
[addIconSpritesheetResultLayers, fitLayers],
[addIconSpritesheetResultLayers, fitLayers, projectId, assetFolderId],
);
const submitIconSpritesheetGeneration = useCallback(
@@ -587,9 +599,12 @@ export function useImageCanvasGenerationSubmissionWorkflow({
});
try {
const generated = await generateEditorIconSpritesheet(
submissionPlan.input,
);
const generated = await generateEditorIconSpritesheet({
...submissionPlan.input,
projectId,
generationInputs: submissionPlan.generationInputs,
assetFolderId,
});
rememberImageModel(submissionPlan.rememberImageModel);
addIconSpritesheetResultLayers(
generated,
@@ -612,6 +627,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({
addIconSpritesheetResultLayers,
getGeneratingDialogPlaceholder,
rememberImageModel,
projectId,
assetFolderId,
updateCanvasGenerationDialogById,
],
);
@@ -667,19 +684,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
const sourceReferenceImageSrc = await resolveEditorImageReferenceDataUrl(
quickEditSourceLayer.objectKey?.trim() || quickEditSourceLayer.src,
);
const generated = await generateEditorImage({
prompt: normalizedPrompt,
size: quickEditPanel.size,
kind: 'quick-edit',
model: normalizeEditorImageModel(quickEditPanel.model),
referenceImageSrcs: [
...extraReferenceImageSrcs,
sourceReferenceImageSrc,
],
});
addQuickEditResultLayer(
generated,
quickEditSourceLayer,
const generationInputs =
panelMode === 'redraw'
? buildEditGenerationInputs(
'重绘提示词',
@@ -691,7 +696,29 @@ export function useImageCanvasGenerationSubmissionWorkflow({
normalizedPrompt,
quickEditSourceLayer,
quickEditReferences,
),
);
const generated = await generateEditorImage({
prompt: normalizedPrompt,
size: quickEditPanel.size,
kind: 'quick-edit',
model: normalizeEditorImageModel(quickEditPanel.model),
referenceImageSrcs: [
...extraReferenceImageSrcs,
sourceReferenceImageSrc,
],
projectId,
assetKind: quickEditSourceLayer.assetKind,
generationInputs,
assetFolderId,
assetLabel: `${quickEditSourceLayer.title} ${
panelMode === 'redraw' ? '重绘' : '快速编辑'
}`,
sourceResourceId: quickEditSourceLayer.resourceId,
});
addQuickEditResultLayer(
generated,
quickEditSourceLayer,
generationInputs,
panelMode,
quickEditDialogId
? {
@@ -726,6 +753,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({
addQuickEditResultLayer,
getGeneratingDialogPlaceholder,
openCanvasGenerationDialog,
projectId,
assetFolderId,
quickEditPanel,
quickEditSourceLayer,
setQuickEditPanel,
@@ -768,11 +797,18 @@ export function useImageCanvasGenerationSubmissionWorkflow({
});
if (submissionPlan.kind === 'edit') {
const referenceImageSrc = await resolveEditorImageReferenceDataUrl(
submissionPlan.sourceLayer.src,
submissionPlan.sourceLayer.objectKey?.trim() ||
submissionPlan.sourceLayer.src,
);
const generated = await editEditorImage({
prompt: submissionPlan.normalizedPrompt,
sourceImageSrc: referenceImageSrc,
projectId,
assetKind: submissionPlan.sourceLayer.assetKind,
generationInputs: submissionPlan.generationInputs,
assetFolderId,
assetLabel: `${submissionPlan.sourceLayer.title} 修改结果`,
sourceResourceId: submissionPlan.sourceLayer.resourceId,
});
addGeneratedResultLayer(generated, {
sourceLayer: submissionPlan.sourceLayer,
@@ -804,7 +840,14 @@ export function useImageCanvasGenerationSubmissionWorkflow({
submissionPlan.input.kind === 'publication-material'
? await normalizePublicationReferenceImages(submissionPlan.input)
: submissionPlan.input;
const generated = await generateEditorImage(imageGenerationInput);
const generated = await generateEditorImage({
...imageGenerationInput,
projectId,
assetKind: submissionPlan.result.assetKind,
generationInputs: submissionPlan.result.generationInputs,
assetFolderId,
assetLabel: submissionPlan.result.title,
});
if (submissionPlan.rememberImageModel) {
rememberImageModel(submissionPlan.rememberImageModel);
}
@@ -844,6 +887,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({
layerCounterRef,
layers,
rememberImageModel,
projectId,
assetFolderId,
setGenerateDialog,
updateCanvasGenerationDialogById,
],
src/components/image-editor/useImageCanvasGenerationSurface.tsx
+10
View File
@@ -15,6 +15,7 @@ import {
PlatformFloatingMenu,
PlatformFloatingMenuItem,
} from '../common/PlatformFloatingMenu';
import type { EditorAssetSnapshot } from '../../services/image-editor/editorProjectClient';
import type {
CanvasGenerationDialogState,
CanvasLayer,
@@ -86,6 +87,9 @@ type ImageCanvasGenerationSurfaceOptions = {
requestUpload: (target: UploadTarget) => void;
persistGeneratedAsset?: (layer: CanvasLayer) => void;
persistUpdatedLayerResource?: (layer: CanvasLayer) => void;
projectId?: string | null;
assetFolderId?: string | null;
upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void;
};
function renderEditorPortal(node: ReactNode) {
@@ -158,6 +162,9 @@ export function useImageCanvasGenerationSurface({
requestUpload,
persistGeneratedAsset,
persistUpdatedLayerResource,
projectId,
assetFolderId,
upsertGeneratedAsset,
}: ImageCanvasGenerationSurfaceOptions) {
const toolbarOptionCloseTimerRef = useRef<ReturnType<
typeof setTimeout
@@ -187,6 +194,9 @@ export function useImageCanvasGenerationSurface({
setImageContextMenu,
persistGeneratedAsset,
persistUpdatedLayerResource,
projectId,
assetFolderId,
upsertGeneratedAsset,
});
const activeGenerationLayer = activeCanvasGenerationDialog?.generatedLayerId
src/components/image-editor/useImageCanvasGenerationWorkflow.ts
+10 -1
View File
@@ -10,6 +10,7 @@ import {
useState,
} from 'react';
import type { EditorAssetSnapshot } from '../../services/image-editor/editorProjectClient';
import type {
CanvasGenerationDialogState,
CanvasLayer,
@@ -182,6 +183,9 @@ type GenerationWorkflowOptions = {
setImageContextMenu: Dispatch<SetStateAction<ImageContextMenuState | null>>;
persistGeneratedAsset?: (layer: CanvasLayer) => void;
persistUpdatedLayerResource?: (layer: CanvasLayer) => void;
projectId?: string | null;
assetFolderId?: string | null;
upsertGeneratedAsset?: (asset: EditorAssetSnapshot) => void;
};
export function useImageCanvasGenerationWorkflow({
@@ -209,6 +213,9 @@ export function useImageCanvasGenerationWorkflow({
setImageContextMenu,
persistGeneratedAsset,
persistUpdatedLayerResource,
projectId,
assetFolderId,
upsertGeneratedAsset,
}: GenerationWorkflowOptions) {
const [isSpecMenuOpen, setIsSpecMenuOpen] = useState(false);
const [isGenerationReferenceMenuOpen, setIsGenerationReferenceMenuOpen] =
@@ -951,7 +958,9 @@ export function useImageCanvasGenerationWorkflow({
setActiveTool,
setActiveSidebarPanel,
rememberImageModel: setLastImageModel,
persistGeneratedAsset,
projectId,
assetFolderId,
upsertGeneratedAsset,
});
const setEffectiveCharacterAnimationPanel: Dispatch<
src/components/image-editor/useImageCanvasProjectPersistence.test.tsx
+42
View File
@@ -146,6 +146,20 @@ function ProjectPersistenceHarness({
>
append
</button>
<button
type="button"
onClick={() => {
persistence.appendCanvasLayersWithResources([
createLayer('layer-server-resource', {
resourceId: 'resource-from-generation-bff',
title: '后端已保存图片',
sourceType: 'generated',
}),
]);
}}
>
append server resource
</button>
<button
type="button"
onClick={() => {
@@ -270,6 +284,34 @@ describe('useImageCanvasProjectPersistence', () => {
});
});
it('does not create a project resource again for backend-persisted generated layers', async () => {
render(<ProjectPersistenceHarness />);
expect(await screen.findByText('editor-project-default')).toBeTruthy();
act(() => {
screen.getByRole('button', { name: 'append server resource' }).click();
});
expect(screen.getByTestId('layers').textContent).toBe(
'layer-server-resource:resource-from-generation-bff:-:-',
);
expect(createEditorProjectResourceMock).not.toHaveBeenCalled();
await waitFor(() => {
expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith(
'editor-project-default',
expect.objectContaining({
layers: expect.arrayContaining([
expect.objectContaining({
layerId: 'layer-server-resource',
resourceId: 'resource-from-generation-bff',
sourceType: 'generated',
}),
]),
}),
);
});
});
it('persists generated layer metadata on the project resource and restores it from resources', async () => {
loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({
projectId: 'editor-project-default',
src/components/image-editor/useImageCanvasProjectPersistence.ts
+3 -1
View File
@@ -171,7 +171,9 @@ export function useImageCanvasProjectPersistence({
refs.layersRef.current = snapshotLayers;
setLayers(snapshotLayers);
nextLayers.forEach((layer) =>
createProjectResourceForLayer(layer, { snapshotLayers }),
layer.resourceId.startsWith('local-resource-')
? createProjectResourceForLayer(layer, { snapshotLayers })
: undefined,
);
},
[createProjectResourceForLayer, refs, setLayers],
src/services/image-editor/editorProjectClient.test.ts
+62
View File
@@ -571,6 +571,15 @@ describe('editorProjectClient', () => {
model: 'gpt-image-2',
aspectRatio: '2:3',
imageSize: '1K',
projectId: 'editor-project-1',
assetKind: 'character',
assetFolderId: 'project',
assetLabel: '角色形象',
sourceResourceId: 'resource-spec-1',
generationInputs: {
fields: [{ title: '角色设定', value: '角色设定' }],
references: [],
},
});
expect(requestJsonMock).toHaveBeenCalledWith(
@@ -584,6 +593,15 @@ describe('editorProjectClient', () => {
model: 'gpt-image-2',
aspectRatio: '2:3',
imageSize: '1K',
projectId: 'editor-project-1',
assetKind: 'character',
generationInputs: {
fields: [{ title: '角色设定', value: '角色设定' }],
references: [],
},
assetFolderId: 'project',
assetLabel: '角色形象',
sourceResourceId: 'resource-spec-1',
}),
}),
'生成图片失败',
@@ -663,6 +681,12 @@ describe('editorProjectClient', () => {
aspectRatio: '1:1',
imageSize: '2K',
priceMudPoints: 12,
projectId: 'editor-project-1',
assetFolderId: 'project',
generationInputs: {
fields: [{ title: '素材描述', value: '返回按钮' }],
references: [],
},
});
expect(requestJsonMock).toHaveBeenCalledWith(
@@ -677,6 +701,12 @@ describe('editorProjectClient', () => {
aspectRatio: '1:1',
imageSize: '2K',
priceMudPoints: 12,
projectId: 'editor-project-1',
generationInputs: {
fields: [{ title: '素材描述', value: '返回按钮' }],
references: [],
},
assetFolderId: 'project',
}),
}),
'生成图标素材失败',
@@ -710,6 +740,13 @@ describe('editorProjectClient', () => {
const result = await extractEditorUiDesignAssets({
sourceImageSrc: 'data:image/png;base64,ui-design',
projectId: 'editor-project-1',
assetFolderId: 'project',
spritesheetLabel: '战斗UI设计图 素材图集',
generationInputs: {
fields: [{ title: '来源', value: '战斗UI设计图' }],
references: [],
},
});
expect(result.taskId).toBe('ui-asset-extraction-1');
@@ -720,6 +757,13 @@ describe('editorProjectClient', () => {
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
sourceImageSrc: 'data:image/png;base64,ui-design',
projectId: 'editor-project-1',
generationInputs: {
fields: [{ title: '来源', value: '战斗UI设计图' }],
references: [],
},
assetFolderId: 'project',
spritesheetLabel: '战斗UI设计图 素材图集',
}),
}),
'提取UI设计图素材失败',
@@ -1128,6 +1172,15 @@ describe('editorProjectClient', () => {
sourceImageSrc: 'data:image/png;base64,source',
size: '1024x1024',
model: 'gpt-image-2',
projectId: 'editor-project-1',
assetKind: 'character',
assetFolderId: 'project',
assetLabel: '角色形象 修改结果',
sourceResourceId: 'resource-character-1',
generationInputs: {
fields: [{ title: '修改提示词', value: '把画面改成黄昏光线' }],
references: [],
},
});
expect(result.taskId).toBe('vector-edit-1');
@@ -1141,6 +1194,15 @@ describe('editorProjectClient', () => {
sourceImageSrc: 'data:image/png;base64,source',
size: '1024x1024',
model: 'gpt-image-2',
projectId: 'editor-project-1',
assetKind: 'character',
generationInputs: {
fields: [{ title: '修改提示词', value: '把画面改成黄昏光线' }],
references: [],
},
assetFolderId: 'project',
assetLabel: '角色形象 修改结果',
sourceResourceId: 'resource-character-1',
}),
}),
'修改图片失败',
src/services/image-editor/editorProjectClient.ts
+58
View File
@@ -122,6 +122,12 @@ export type EditorImageGenerationInput = {
aspectRatio?: string;
imageSize?: string;
referenceImageSrcs?: string[];
projectId?: string | null;
assetKind?: string | null;
generationInputs?: EditorAssetGenerationInputs | null;
assetFolderId?: string | null;
assetLabel?: string | null;
sourceResourceId?: string | null;
};
export type EditorIconSpritesheetGenerationInput = {
@@ -131,10 +137,17 @@ export type EditorIconSpritesheetGenerationInput = {
aspectRatio?: string;
imageSize?: string;
priceMudPoints: number;
projectId?: string | null;
generationInputs?: EditorAssetGenerationInputs | null;
assetFolderId?: string | null;
};
export type EditorUiDesignAssetExtractionInput = {
sourceImageSrc: string;
projectId?: string | null;
generationInputs?: EditorAssetGenerationInputs | null;
assetFolderId?: string | null;
spritesheetLabel?: string | null;
};
export type EditorImageEditInput = {
@@ -142,6 +155,12 @@ export type EditorImageEditInput = {
sourceImageSrc: string;
size?: string;
model?: string;
projectId?: string | null;
assetKind?: string | null;
generationInputs?: EditorAssetGenerationInputs | null;
assetFolderId?: string | null;
assetLabel?: string | null;
sourceResourceId?: string | null;
};
export type EditorImageGenerationResult = {
@@ -156,6 +175,8 @@ export type EditorImageGenerationResult = {
model: string;
provider: string;
taskId: string;
resource?: EditorProjectResourceSnapshot | null;
asset?: EditorAssetSnapshot | null;
};
export type EditorIconSpritesheetIconResult = {
@@ -163,6 +184,8 @@ export type EditorIconSpritesheetIconResult = {
imageSrc: string;
width: number;
height: number;
resource?: EditorProjectResourceSnapshot | null;
asset?: EditorAssetSnapshot | null;
};
export type EditorIconSpritesheetGenerationResult = {
@@ -176,6 +199,8 @@ export type EditorIconSpritesheetGenerationResult = {
provider: string;
taskId: string;
priceMudPoints: number;
spritesheetResource?: EditorProjectResourceSnapshot | null;
spritesheetAsset?: EditorAssetSnapshot | null;
};
export type EditorCharacterAnimationResolution = '480p' | '720p';
@@ -578,6 +603,16 @@ export async function generateEditorImage(input: EditorImageGenerationInput) {
...(input.referenceImageSrcs?.length
? { referenceImageSrcs: input.referenceImageSrcs }
: {}),
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.assetKind ? { assetKind: input.assetKind } : {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
...(input.assetFolderId ? { assetFolderId: input.assetFolderId } : {}),
...(input.assetLabel ? { assetLabel: input.assetLabel } : {}),
...(input.sourceResourceId
? { sourceResourceId: input.sourceResourceId }
: {}),
}),
'生成图片失败',
{
@@ -601,6 +636,11 @@ export async function generateEditorIconSpritesheet(
...(input.aspectRatio ? { aspectRatio: input.aspectRatio } : {}),
...(input.imageSize ? { imageSize: input.imageSize } : {}),
priceMudPoints: input.priceMudPoints,
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
...(input.assetFolderId ? { assetFolderId: input.assetFolderId } : {}),
}),
'生成图标素材失败',
{
@@ -619,6 +659,14 @@ export async function extractEditorUiDesignAssets(
EDITOR_UI_DESIGN_ASSET_EXTRACTION_API,
jsonRequest('POST', {
sourceImageSrc: input.sourceImageSrc,
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
...(input.assetFolderId ? { assetFolderId: input.assetFolderId } : {}),
...(input.spritesheetLabel
? { spritesheetLabel: input.spritesheetLabel }
: {}),
}),
'提取UI设计图素材失败',
{
@@ -638,6 +686,16 @@ export async function editEditorImage(input: EditorImageEditInput) {
sourceImageSrc: input.sourceImageSrc,
...(input.size ? { size: input.size } : {}),
...(input.model ? { model: input.model } : {}),
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.assetKind ? { assetKind: input.assetKind } : {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
...(input.assetFolderId ? { assetFolderId: input.assetFolderId } : {}),
...(input.assetLabel ? { assetLabel: input.assetLabel } : {}),
...(input.sourceResourceId
? { sourceResourceId: input.sourceResourceId }
: {}),
}),
'修改图片失败',
{