From da3aac825734380da65493f8afceef71dca5cb9f Mon Sep 17 00:00:00 2001 From: MengHao Date: Fri, 17 Jul 2026 19:43:49 +0800 Subject: [PATCH] =?UTF-8?q?=E4=BF=AE=E5=A4=8D=E5=9B=BE=E7=89=87=E6=94=B9?= =?UTF-8?q?=E9=80=A0=E5=90=8E=E4=BA=8C=E7=BB=B4=E6=B8=85=E6=99=B0=E5=BA=A6?= =?UTF-8?q?=E9=99=8D=E7=BA=A7=E9=97=AE=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 前端按图层真实尺寸恢复图片改造比例与清晰度 后端在角色去背景后恢复生成原图像素尺寸 补充前后端二千像素尺寸回归测试 同步更新图片画布技术文档与项目决策记录 --- .../shared-memory/decision-log.md | 8 ++ ...架构】图片画布编辑器MVP接入方案-2026-06-11.md | 2 +- .../crates/api-server/src/editor_project.rs | 106 +++++++++++++++++- .../ImageCanvasGenerationDialogModel.test.ts | 27 +++++ .../ImageCanvasGenerationDialogModel.ts | 56 ++++++--- 5 files changed, 177 insertions(+), 22 deletions(-) diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index 74aa4db36..1858b259e 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -16,6 +16,14 @@ --- +## 2026-07-17 图片改造保持源图与所选清晰度 + +- 背景：图片画布从已生成的 2K 角色图重新打开生成器时，面板恢复逻辑会优先采用新建面板的 1K 默认值；即使用户重新选择 2K，角色透明化链路也可能接受 BgFilter / 阿里云返回的 1K 后处理图，并因 `nanobanana2` 使用标量清晰度档位而跳过几何尺寸恢复，最终把 2K provider 原图降为 1K 透明图。 +- 决策：从既有图片重新打开普通图片、角色、UI 或宣发生成器时，在没有仍存活的生成对话框快照时按当前图层真实 `originalWidth / originalHeight` 恢复比例与清晰度，并按目标模型支持范围归一；角色形象去背景完成后必须把透明结果恢复到去背景前 provider 原图的像素尺寸，再进入既有交付尺寸恢复和持久化，去背景供应商不得改变最终素材分辨率。 +- 影响范围：图片画布生成对话框恢复、角色形象 BgFilter / 阿里云 / 本地去背后处理、项目资源与账号素材尺寸元数据。 +- 验证方式：覆盖“持久化 2K 角色图重开仍为 2K”和“1K 去背结果恢复到 2K provider 原图尺寸”的前后端定向测试，并运行前端类型检查、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:encoding` 和 `git diff --check`。 +- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。 + ## 2026-07-15 角色动作 BgFilter 请求超时按帧数扩展 - 背景：角色动作全部序列帧会并发进入 BgFilter，而服务端可能在自身进程内排队；固定 `180000ms` 会把排队时间和单帧推理共用同一预算，靠后的请求可能在服务仍正常处理时被 api-server 提前取消。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 84c136ca2..d46c5efa0 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -87,7 +87,7 @@ - `POST /api/editor/assets`：批量或单个创建账号级素材，登录态上传必须写入 OSS / asset object 引用和 `/` 轻量路径，不允许把 Data URL / signed URL 写入素材库。 - `PATCH /api/editor/assets/{assetId}`：重命名素材或移动素材到文件夹。 - `DELETE /api/editor/assets/{assetId}`：删除素材。已放入画布的 project resource 不被级联删除，避免旧画布丢图。 -- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；普通图片的 provider 回图先留在内存，尺寸变换成功后只上传变换结果，变换失败则只上传 provider 原图，主结果只写一次 OSS 且不额外创建“原始输出”。角色生成可携带 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`；api-server 先保存带纯色背景源图，再调用 BgFilter 并传入 `screen_color=`、`seg_model=`，透明处理成功时生成透明 PNG，最终失败时按前述多产物降级规则以原图主结果和通用 `warning` 收口。宣发素材携带 `kind: "publication-material"` 时固定归一为 `gpt-image-2`，不支持 `nanobanana2`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits；`nanobanana2` 的 `512 / 1024 / 2K` 是标量清晰度档位，后端保留 provider 输出几何尺寸，不按 `宽x高` 解析。普通重绘继续走该接口并把当前图层图片作为参考图；图片快速编辑不走该接口。请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成完成后在响应中返回实际产物的 project / resource / asset 快照。 +- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；普通图片的 provider 回图先留在内存，尺寸变换成功后只上传变换结果，变换失败则只上传 provider 原图，主结果只写一次 OSS 且不额外创建“原始输出”。角色生成可携带 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`；api-server 先保存带纯色背景源图，再调用 BgFilter 并传入 `screen_color=`、`seg_model=`，透明处理成功时生成透明 PNG，最终失败时按前述多产物降级规则以原图主结果和通用 `warning` 收口。透明处理正常成功但返回尺寸与 provider 原图不同时，必须先把透明 PNG 恢复到 provider 原图像素尺寸，再执行交付尺寸恢复和持久化，避免 2K 角色图在去背景后降为 1K。宣发素材携带 `kind: "publication-material"` 时固定归一为 `gpt-image-2`，不支持 `nanobanana2`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits；`nanobanana2` 的 `512 / 1024 / 2K` 是标量清晰度档位，后端保留 provider 输出几何尺寸，不按 `宽x高` 解析。从既有图层重新打开生成器且没有仍存活的对话框快照时，前端按该图层真实 `originalWidth / originalHeight` 恢复比例和清晰度，不得回落到新建面板的 1K 默认值。普通重绘继续走该接口并把当前图层图片作为参考图；图片快速编辑不走该接口。请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成完成后在响应中返回实际产物的 project / resource / asset 快照。 - `POST /api/editor/images/background-removals`：接收当前图片源，校验登录态后无条件创建外部生成任务，响应只返回 `queueState`。worker 由 api-server 解析图片文件，并通过共享 BgFilter HTTP client 调用 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL/remove-background`；multipart 固定为 `file + background_mode=complex + seg_model=birefnet + cross_check=off`，不包含 `screen_color`，首次失败立即重试 `1` 次，两次都失败返回最终错误。请求可携带 `projectId`、`targetLayerId`、`assetFolderId`、`assetLabel`、`sourceResourceId` 和 `canvasCompletion`，有 `canvasCompletion` 时完成后按生成占位写入结果图层，否则沿用旧的目标图层替换路径。令牌只在服务端通过 `GENARRATIVE_EDITOR_BGFILTER_TOKEN` 注入，未配置时兼容回退旧 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`。 - `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet；api-server 先保存带纯色背景 spritesheet 源图，透明处理成功后再保存透明 spritesheet 并尝试拆分。请求支持 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置中对应生图模型的尺寸档位（如 `nanobanana2` 的 `0.5K / 1K / 2K` 或 `gpt-image-2` 的 `1K / 2K`），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。透明处理最终失败时只保存并返回原图主结果，不生成透明图或切片；透明图成功但拆分失败时保留整张透明图并返回 `sliceWarning`。响应只返回实际产物对应的 project / resource / asset 快照及可选通用 `warning`。 - `POST /api/editor/ui-designs/assets/extractions`：前端把红色框选轮廓绘入本地临时图后，先将该图上传 OSS 并确认 asset object，再以返回的 `objectKey` 作为参考图入队；Data URL / Blob URL 只允许停留在上传前的浏览器临时态。接口固定 `gpt-image-2` 和自动决策纯色背景素材提取提示词生成素材 spritesheet；api-server 先保存带纯色背景 spritesheet 源图，透明处理成功后再保存透明 spritesheet 并按连通域尝试拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求必须携带 `screenColor`、`segModel`、`aspectRatio: "1:1"`、`imageSize: "1K" | "2K"` 和 `priceMudPoints`；框选数量不超过 6 个时前端按 `1:1·1K` 与 gpt-image-2 1K 价格提交，超过 6 个时按 `1:1·2K` 与 2K 价格提交。后端必须在调用上游前校验比例、尺寸和泥点价格，只允许 `1:1 / 1K / 2K`。透明处理最终失败时只保存并返回原图主结果，不生成透明图或切片；透明图成功但拆分失败时保留整张透明图并返回 `sliceWarning`。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，响应只返回实际产物对应的 project / resource / asset 快照及可选通用 `warning`；前端按后端快照落画布，不补造缺失产物。 diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index 4d5e82c72..227c8ad84 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -1852,11 +1852,28 @@ pub(crate) async fn generate_editor_image_for_owner( )); } }; - image = removal.image; + let removal_provider = removal.provider; + let (restored_removal_image, postprocess_dimension_error) = + restore_editor_postprocessed_image_output_dimensions_or_original( + removal.image, + provider_width, + provider_height, + ); + if let Some(error) = postprocess_dimension_error { + tracing::warn!( + task_id = %generated.task_id, + provider = removal_provider, + provider_width, + provider_height, + error = ?error, + "角色透明图尺寸恢复失败，保留去背景服务原始输出" + ); + } + image = restored_removal_image; output_prompt = "去除纯色背景".to_string(); output_actual_prompt = None; output_model = removal.model; - output_provider = removal.provider.to_string(); + output_provider = removal_provider.to_string(); output_generation_inputs = build_editor_derived_asset_generation_inputs("角色抠图", &output_model, &source_record); Some(source_record) @@ -2418,9 +2435,10 @@ async fn ensure_editor_image_edit_source_allowed( ensure_editor_image_edit_asset_kind_allowed(asset_kind) } -fn encode_editor_image_edit_png( +fn encode_editor_image_png( image: image::DynamicImage, failure_status: StatusCode, + provider: &str, failure_message: &str, ) -> Result, AppError> { let mut bytes = Cursor::new(Vec::new()); @@ -2428,13 +2446,21 @@ fn encode_editor_image_edit_png( .write_to(&mut bytes, image::ImageFormat::Png) .map_err(|error| { AppError::from_status(failure_status).with_details(json!({ - "provider": "editor-image-edit", + "provider": provider, "message": format!("{failure_message}：{error}"), })) })?; Ok(bytes.into_inner()) } +fn encode_editor_image_edit_png( + image: image::DynamicImage, + failure_status: StatusCode, + failure_message: &str, +) -> Result, AppError> { + encode_editor_image_png(image, failure_status, "editor-image-edit", failure_message) +} + fn prepare_editor_image_edit_references( reference_images: &mut [OpenAiReferenceImage], target_size: &str, @@ -2571,6 +2597,51 @@ fn restore_editor_generated_image_output_dimensions_or_original( } } +fn restore_editor_postprocessed_image_output_dimensions( + output: &DownloadedOpenAiImage, + target_width: u32, + target_height: u32, +) -> Result, AppError> { + let decoded = image::load_from_memory(output.bytes.as_slice()).map_err(|error| { + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": "editor-character-image", + "message": format!("角色透明图不是有效图片：{error}"), + })) + })?; + if decoded.width() == target_width && decoded.height() == target_height { + return Ok(None); + } + + let restored = decoded.resize_exact( + target_width, + target_height, + image::imageops::FilterType::Lanczos3, + ); + Ok(Some(DownloadedOpenAiImage { + bytes: encode_editor_image_png( + restored, + StatusCode::BAD_GATEWAY, + "editor-character-image", + "恢复角色透明图尺寸失败", + )?, + mime_type: "image/png".to_string(), + extension: "png".to_string(), + })) +} + +fn restore_editor_postprocessed_image_output_dimensions_or_original( + output: DownloadedOpenAiImage, + target_width: u32, + target_height: u32, +) -> (DownloadedOpenAiImage, Option) { + match restore_editor_postprocessed_image_output_dimensions(&output, target_width, target_height) + { + Ok(Some(restored)) => (restored, None), + Ok(None) => (output, None), + Err(error) => (output, Some(error)), + } +} + fn restore_editor_generated_image_output_dimensions( output: &DownloadedOpenAiImage, model: &str, @@ -8469,6 +8540,33 @@ mod tests { assert_eq!(restored.extension, "png"); } + #[test] + fn character_postprocess_restores_provider_dimensions() { + let image = image::DynamicImage::new_rgba8(1024, 1024); + let mut bytes = Cursor::new(Vec::new()); + image + .write_to(&mut bytes, image::ImageFormat::Png) + .expect("test image should encode"); + let postprocessed = DownloadedOpenAiImage { + bytes: bytes.into_inner(), + mime_type: "image/png".to_string(), + extension: "png".to_string(), + }; + + let restored = + restore_editor_postprocessed_image_output_dimensions(&postprocessed, 2048, 2048) + .expect("postprocessed image should restore provider dimensions") + .expect("1K postprocessed image should be resized back to 2K"); + let restored_image = image::load_from_memory(restored.bytes.as_slice()).unwrap(); + + assert_eq!( + (restored_image.width(), restored_image.height()), + (2048, 2048) + ); + assert_eq!(restored.mime_type, "image/png"); + assert_eq!(restored.extension, "png"); + } + #[test] fn publication_material_generation_restores_provider_output_to_workflow_dimensions() { let image = image::DynamicImage::new_rgba8(944, 704); diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts index b2300817f..b6bc63582 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts @@ -672,6 +672,33 @@ describe('ImageCanvasGenerationDialogModel', () => { }); }); + it('preserves 2K dimensions when reopening a persisted character layer', () => { + expect( + createLayerGenerationDialogDraft({ + sourceLayer: createLayer({ + sourceType: 'generated', + assetKind: 'character', + model: IMAGE_MODEL_NANOBANANA2, + originalWidth: 2048, + originalHeight: 2048, + generationInputs: { + fields: [{ title: '角色设定', value: '香水师角色' }], + references: [], + }, + }), + canvasSize: { width: 960, height: 720 }, + viewport: { x: 0, y: 0, scale: 1 }, + }), + ).toMatchObject({ + mode: 'character', + prompt: '香水师角色', + imageModel: IMAGE_MODEL_NANOBANANA2, + aspectRatio: '1:1', + imageSize: '2K', + generatedLayerId: 'layer-source', + }); + }); + it('inherits and normalizes quick edit model parameters from the source', () => { expect( createQuickEditPanelDraft( diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts index 7e7339a8f..9467a86ab 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts @@ -691,8 +691,7 @@ export function createLayerGenerationDialogDraft({ return { ...draft, prompt: draft.prompt || sourceLayer.prompt?.trim() || '', - imageModel: - sourceDialog?.imageModel ?? sourceLayer.model ?? draft.imageModel, + imageModel: sourceDialog?.imageModel ?? draft.imageModel, status: 'idle', composerOpen: true, generatedLayerId: sourceLayer.id, @@ -740,23 +739,46 @@ function restoreSharedImageOptions( sourceLayer: CanvasLayer, sourceDialog?: CanvasGenerationDialogState | null, ): Omit { + const requestedModel = normalizeEditorImageModel( + sourceDialog?.imageModel ?? sourceLayer.model ?? draft.imageModel, + ); + const imageModel = EDITOR_IMAGE_MODEL_OPTIONS.some( + (option) => option.value === requestedModel, + ) + ? requestedModel + : DEFAULT_IMAGE_MODEL; + const dimensionOptions = + EDITOR_IMAGE_DIMENSION_OPTIONS[ + imageModel as keyof typeof EDITOR_IMAGE_DIMENSION_OPTIONS + ] ?? EDITOR_IMAGE_DIMENSION_OPTIONS[DEFAULT_IMAGE_MODEL]; + const supportedAspectRatios = + dimensionOptions.aspectRatios as readonly string[]; + const supportedImageSizes = dimensionOptions.imageSizes as readonly string[]; + const inferredAspectRatio = inferEditorImageAspectRatio( + sourceLayer.originalWidth, + sourceLayer.originalHeight, + ); + const inferredImageSize = inferEditorImageSizeLabel( + sourceLayer.originalWidth, + sourceLayer.originalHeight, + ); + const preferredAspectRatio = sourceDialog?.aspectRatio ?? inferredAspectRatio; + const preferredImageSize = sourceDialog?.imageSize ?? inferredImageSize; return { ...draft, - imageModel: sourceDialog?.imageModel ?? draft.imageModel, - aspectRatio: - sourceDialog?.aspectRatio ?? - draft.aspectRatio ?? - inferEditorImageAspectRatio( - sourceLayer.originalWidth, - sourceLayer.originalHeight, - ), - imageSize: - sourceDialog?.imageSize ?? - draft.imageSize ?? - inferEditorImageSizeLabel( - sourceLayer.originalWidth, - sourceLayer.originalHeight, - ), + imageModel, + aspectRatio: supportedAspectRatios.includes(preferredAspectRatio) + ? preferredAspectRatio + : draft.aspectRatio && supportedAspectRatios.includes(draft.aspectRatio) + ? draft.aspectRatio + : (dimensionOptions.aspectRatios[0] ?? '1:1'), + imageSize: supportedImageSizes.includes(preferredImageSize) + ? preferredImageSize + : draft.imageSize && supportedImageSizes.includes(draft.imageSize) + ? draft.imageSize + : (dimensionOptions.imageSizes.find((size) => size === '1K') ?? + dimensionOptions.imageSizes[0] ?? + '1K'), }; }