GenarrativeAI/Genarrative
11
0
Fork 0
Code Issues 21 Pull Requests 4 Actions Packages Projects 3 Releases Wiki Activity

修复图片改造后二维清晰度降级问题

Browse Source 
前端按图层真实尺寸恢复图片改造比例与清晰度

后端在角色去背景后恢复生成原图像素尺寸

补充前后端二千像素尺寸回归测试

同步更新图片画布技术文档与项目决策记录
This commit is contained in:
MengHao
2026-07-17 19:43:49 +08:00
parent 387a1c26e3
commit da3aac8257
5 changed files with 177 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/project-memory/shared-memory/decision-log.md
+8
View File
@@ -16,6 +16,14 @@
---
## 2026-07-17 图片改造保持源图与所选清晰度
- 背景：图片画布从已生成的 2K 角色图重新打开生成器时，面板恢复逻辑会优先采用新建面板的 1K 默认值；即使用户重新选择 2K，角色透明化链路也可能接受 BgFilter / 阿里云返回的 1K 后处理图，并因 `nanobanana2` 使用标量清晰度档位而跳过几何尺寸恢复，最终把 2K provider 原图降为 1K 透明图。
- 决策：从既有图片重新打开普通图片、角色、UI 或宣发生成器时，在没有仍存活的生成对话框快照时按当前图层真实 `originalWidth / originalHeight` 恢复比例与清晰度，并按目标模型支持范围归一；角色形象去背景完成后必须把透明结果恢复到去背景前 provider 原图的像素尺寸，再进入既有交付尺寸恢复和持久化，去背景供应商不得改变最终素材分辨率。
- 影响范围：图片画布生成对话框恢复、角色形象 BgFilter / 阿里云 / 本地去背后处理、项目资源与账号素材尺寸元数据。
- 验证方式：覆盖“持久化 2K 角色图重开仍为 2K”和“1K 去背结果恢复到 2K provider 原图尺寸”的前后端定向测试，并运行前端类型检查、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:encoding` 和 `git diff --check`。
- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。
## 2026-07-15 角色动作 BgFilter 请求超时按帧数扩展
- 背景：角色动作全部序列帧会并发进入 BgFilter，而服务端可能在自身进程内排队；固定 `180000ms` 会把排队时间和单帧推理共用同一预算，靠后的请求可能在服务仍正常处理时被 api-server 提前取消。
docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md
+1 -1
View File
@@ -87,7 +87,7 @@
- `POST /api/editor/assets`：批量或单个创建账号级素材，登录态上传必须写入 OSS / asset object 引用和 `/<objectKey>` 轻量路径，不允许把 Data URL / signed URL 写入素材库。
- `PATCH /api/editor/assets/{assetId}`：重命名素材或移动素材到文件夹。
- `DELETE /api/editor/assets/{assetId}`：删除素材。已放入画布的 project resource 不被级联删除，避免旧画布丢图。
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；普通图片的 provider 回图先留在内存，尺寸变换成功后只上传变换结果，变换失败则只上传 provider 原图，主结果只写一次 OSS 且不额外创建“原始输出”。角色生成可携带 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`；api-server 先保存带纯色背景源图，再调用 BgFilter 并传入 `screen_color=<screenColor>`、`seg_model=<segModel>`，透明处理成功时生成透明 PNG，最终失败时按前述多产物降级规则以原图主结果和通用 `warning` 收口。宣发素材携带 `kind: "publication-material"` 时固定归一为 `gpt-image-2`，不支持 `nanobanana2`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits；`nanobanana2` 的 `512 / 1024 / 2K` 是标量清晰度档位，后端保留 provider 输出几何尺寸，不按 `宽x高` 解析。普通重绘继续走该接口并把当前图层图片作为参考图；图片快速编辑不走该接口。请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成完成后在响应中返回实际产物的 project / resource / asset 快照。
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；普通图片的 provider 回图先留在内存，尺寸变换成功后只上传变换结果，变换失败则只上传 provider 原图，主结果只写一次 OSS 且不额外创建“原始输出”。角色生成可携带 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`；api-server 先保存带纯色背景源图，再调用 BgFilter 并传入 `screen_color=<screenColor>`、`seg_model=<segModel>`，透明处理成功时生成透明 PNG，最终失败时按前述多产物降级规则以原图主结果和通用 `warning` 收口。透明处理正常成功但返回尺寸与 provider 原图不同时，必须先把透明 PNG 恢复到 provider 原图像素尺寸，再执行交付尺寸恢复和持久化，避免 2K 角色图在去背景后降为 1K。宣发素材携带 `kind: "publication-material"` 时固定归一为 `gpt-image-2`，不支持 `nanobanana2`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits；`nanobanana2` 的 `512 / 1024 / 2K` 是标量清晰度档位，后端保留 provider 输出几何尺寸，不按 `宽x高` 解析。从既有图层重新打开生成器且没有仍存活的对话框快照时，前端按该图层真实 `originalWidth / originalHeight` 恢复比例和清晰度，不得回落到新建面板的 1K 默认值。普通重绘继续走该接口并把当前图层图片作为参考图；图片快速编辑不走该接口。请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成完成后在响应中返回实际产物的 project / resource / asset 快照。
- `POST /api/editor/images/background-removals`：接收当前图片源，校验登录态后无条件创建外部生成任务，响应只返回 `queueState`。worker 由 api-server 解析图片文件，并通过共享 BgFilter HTTP client 调用 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL/remove-background`；multipart 固定为 `file + background_mode=complex + seg_model=birefnet + cross_check=off`，不包含 `screen_color`，首次失败立即重试 `1` 次，两次都失败返回最终错误。请求可携带 `projectId`、`targetLayerId`、`assetFolderId`、`assetLabel`、`sourceResourceId` 和 `canvasCompletion`，有 `canvasCompletion` 时完成后按生成占位写入结果图层，否则沿用旧的目标图层替换路径。令牌只在服务端通过 `GENARRATIVE_EDITOR_BGFILTER_TOKEN` 注入，未配置时兼容回退旧 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`。
- `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet；api-server 先保存带纯色背景 spritesheet 源图，透明处理成功后再保存透明 spritesheet 并尝试拆分。请求支持 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置中对应生图模型的尺寸档位（如 `nanobanana2` 的 `0.5K / 1K / 2K` 或 `gpt-image-2` 的 `1K / 2K`），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。透明处理最终失败时只保存并返回原图主结果，不生成透明图或切片；透明图成功但拆分失败时保留整张透明图并返回 `sliceWarning`。响应只返回实际产物对应的 project / resource / asset 快照及可选通用 `warning`。
- `POST /api/editor/ui-designs/assets/extractions`：前端把红色框选轮廓绘入本地临时图后，先将该图上传 OSS 并确认 asset object，再以返回的 `objectKey` 作为参考图入队；Data URL / Blob URL 只允许停留在上传前的浏览器临时态。接口固定 `gpt-image-2` 和自动决策纯色背景素材提取提示词生成素材 spritesheet；api-server 先保存带纯色背景 spritesheet 源图，透明处理成功后再保存透明 spritesheet 并按连通域尝试拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求必须携带 `screenColor`、`segModel`、`aspectRatio: "1:1"`、`imageSize: "1K" | "2K"` 和 `priceMudPoints`；框选数量不超过 6 个时前端按 `1:1·1K` 与 gpt-image-2 1K 价格提交，超过 6 个时按 `1:1·2K` 与 2K 价格提交。后端必须在调用上游前校验比例、尺寸和泥点价格，只允许 `1:1 / 1K / 2K`。透明处理最终失败时只保存并返回原图主结果，不生成透明图或切片；透明图成功但拆分失败时保留整张透明图并返回 `sliceWarning`。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，响应只返回实际产物对应的 project / resource / asset 快照及可选通用 `warning`；前端按后端快照落画布，不补造缺失产物。
server-rs/crates/api-server/src/editor_project.rs
+102 -4
View File
@@ -1852,11 +1852,28 @@ pub(crate) async fn generate_editor_image_for_owner(
));
}
};
image = removal.image;
let removal_provider = removal.provider;
let (restored_removal_image, postprocess_dimension_error) =
restore_editor_postprocessed_image_output_dimensions_or_original(
removal.image,
provider_width,
provider_height,
);
if let Some(error) = postprocess_dimension_error {
tracing::warn!(
task_id = %generated.task_id,
provider = removal_provider,
provider_width,
provider_height,
error = ?error,
"角色透明图尺寸恢复失败，保留去背景服务原始输出"
);
}
image = restored_removal_image;
output_prompt = "去除纯色背景".to_string();
output_actual_prompt = None;
output_model = removal.model;
output_provider = removal.provider.to_string();
output_provider = removal_provider.to_string();
output_generation_inputs =
build_editor_derived_asset_generation_inputs("角色抠图", &output_model, &source_record);
Some(source_record)
@@ -2418,9 +2435,10 @@ async fn ensure_editor_image_edit_source_allowed(
ensure_editor_image_edit_asset_kind_allowed(asset_kind)
}
fn encode_editor_image_edit_png(
fn encode_editor_image_png(
image: image::DynamicImage,
failure_status: StatusCode,
provider: &str,
failure_message: &str,
) -> Result<Vec<u8>, AppError> {
let mut bytes = Cursor::new(Vec::new());
@@ -2428,13 +2446,21 @@ fn encode_editor_image_edit_png(
.write_to(&mut bytes, image::ImageFormat::Png)
.map_err(|error| {
AppError::from_status(failure_status).with_details(json!({
"provider": "editor-image-edit",
"provider": provider,
"message": format!("{failure_message}：{error}"),
}))
})?;
Ok(bytes.into_inner())
}
fn encode_editor_image_edit_png(
image: image::DynamicImage,
failure_status: StatusCode,
failure_message: &str,
) -> Result<Vec<u8>, AppError> {
encode_editor_image_png(image, failure_status, "editor-image-edit", failure_message)
}
fn prepare_editor_image_edit_references(
reference_images: &mut [OpenAiReferenceImage],
target_size: &str,
@@ -2571,6 +2597,51 @@ fn restore_editor_generated_image_output_dimensions_or_original(
}
}
fn restore_editor_postprocessed_image_output_dimensions(
output: &DownloadedOpenAiImage,
target_width: u32,
target_height: u32,
) -> Result<Option<DownloadedOpenAiImage>, AppError> {
let decoded = image::load_from_memory(output.bytes.as_slice()).map_err(|error| {
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": "editor-character-image",
"message": format!("角色透明图不是有效图片：{error}"),
}))
})?;
if decoded.width() == target_width && decoded.height() == target_height {
return Ok(None);
}
let restored = decoded.resize_exact(
target_width,
target_height,
image::imageops::FilterType::Lanczos3,
);
Ok(Some(DownloadedOpenAiImage {
bytes: encode_editor_image_png(
restored,
StatusCode::BAD_GATEWAY,
"editor-character-image",
"恢复角色透明图尺寸失败",
)?,
mime_type: "image/png".to_string(),
extension: "png".to_string(),
}))
}
fn restore_editor_postprocessed_image_output_dimensions_or_original(
output: DownloadedOpenAiImage,
target_width: u32,
target_height: u32,
) -> (DownloadedOpenAiImage, Option<AppError>) {
match restore_editor_postprocessed_image_output_dimensions(&output, target_width, target_height)
{
Ok(Some(restored)) => (restored, None),
Ok(None) => (output, None),
Err(error) => (output, Some(error)),
}
}
fn restore_editor_generated_image_output_dimensions(
output: &DownloadedOpenAiImage,
model: &str,
@@ -8469,6 +8540,33 @@ mod tests {
assert_eq!(restored.extension, "png");
}
#[test]
fn character_postprocess_restores_provider_dimensions() {
let image = image::DynamicImage::new_rgba8(1024, 1024);
let mut bytes = Cursor::new(Vec::new());
image
.write_to(&mut bytes, image::ImageFormat::Png)
.expect("test image should encode");
let postprocessed = DownloadedOpenAiImage {
bytes: bytes.into_inner(),
mime_type: "image/png".to_string(),
extension: "png".to_string(),
};
let restored =
restore_editor_postprocessed_image_output_dimensions(&postprocessed, 2048, 2048)
.expect("postprocessed image should restore provider dimensions")
.expect("1K postprocessed image should be resized back to 2K");
let restored_image = image::load_from_memory(restored.bytes.as_slice()).unwrap();
assert_eq!(
(restored_image.width(), restored_image.height()),
(2048, 2048)
);
assert_eq!(restored.mime_type, "image/png");
assert_eq!(restored.extension, "png");
}
#[test]
fn publication_material_generation_restores_provider_output_to_workflow_dimensions() {
let image = image::DynamicImage::new_rgba8(944, 704);
src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts
+27
View File
@@ -672,6 +672,33 @@ describe('ImageCanvasGenerationDialogModel', () => {
});
});
it('preserves 2K dimensions when reopening a persisted character layer', () => {
expect(
createLayerGenerationDialogDraft({
sourceLayer: createLayer({
sourceType: 'generated',
assetKind: 'character',
model: IMAGE_MODEL_NANOBANANA2,
originalWidth: 2048,
originalHeight: 2048,
generationInputs: {
fields: [{ title: '角色设定', value: '香水师角色' }],
references: [],
},
}),
canvasSize: { width: 960, height: 720 },
viewport: { x: 0, y: 0, scale: 1 },
}),
).toMatchObject({
mode: 'character',
prompt: '香水师角色',
imageModel: IMAGE_MODEL_NANOBANANA2,
aspectRatio: '1:1',
imageSize: '2K',
generatedLayerId: 'layer-source',
});
});
it('inherits and normalizes quick edit model parameters from the source', () => {
expect(
createQuickEditPanelDraft(
src/components/image-editor/ImageCanvasGenerationDialogModel.ts
+39 -17
View File
@@ -691,8 +691,7 @@ export function createLayerGenerationDialogDraft({
return {
...draft,
prompt: draft.prompt || sourceLayer.prompt?.trim() || '',
imageModel:
sourceDialog?.imageModel ?? sourceLayer.model ?? draft.imageModel,
imageModel: sourceDialog?.imageModel ?? draft.imageModel,
status: 'idle',
composerOpen: true,
generatedLayerId: sourceLayer.id,
@@ -740,23 +739,46 @@ function restoreSharedImageOptions(
sourceLayer: CanvasLayer,
sourceDialog?: CanvasGenerationDialogState | null,
): Omit<CanvasGenerationDialogState, 'id'> {
const requestedModel = normalizeEditorImageModel(
sourceDialog?.imageModel ?? sourceLayer.model ?? draft.imageModel,
);
const imageModel = EDITOR_IMAGE_MODEL_OPTIONS.some(
(option) => option.value === requestedModel,
)
? requestedModel
: DEFAULT_IMAGE_MODEL;
const dimensionOptions =
EDITOR_IMAGE_DIMENSION_OPTIONS[
imageModel as keyof typeof EDITOR_IMAGE_DIMENSION_OPTIONS
] ?? EDITOR_IMAGE_DIMENSION_OPTIONS[DEFAULT_IMAGE_MODEL];
const supportedAspectRatios =
dimensionOptions.aspectRatios as readonly string[];
const supportedImageSizes = dimensionOptions.imageSizes as readonly string[];
const inferredAspectRatio = inferEditorImageAspectRatio(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
);
const inferredImageSize = inferEditorImageSizeLabel(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
);
const preferredAspectRatio = sourceDialog?.aspectRatio ?? inferredAspectRatio;
const preferredImageSize = sourceDialog?.imageSize ?? inferredImageSize;
return {
...draft,
imageModel: sourceDialog?.imageModel ?? draft.imageModel,
aspectRatio:
sourceDialog?.aspectRatio ??
draft.aspectRatio ??
inferEditorImageAspectRatio(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
),
imageSize:
sourceDialog?.imageSize ??
draft.imageSize ??
inferEditorImageSizeLabel(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
),
imageModel,
aspectRatio: supportedAspectRatios.includes(preferredAspectRatio)
? preferredAspectRatio
: draft.aspectRatio && supportedAspectRatios.includes(draft.aspectRatio)
? draft.aspectRatio
: (dimensionOptions.aspectRatios[0] ?? '1:1'),
imageSize: supportedImageSizes.includes(preferredImageSize)
? preferredImageSize
: draft.imageSize && supportedImageSizes.includes(draft.imageSize)
? draft.imageSize
: (dimensionOptions.imageSizes.find((size) => size === '1K') ??
dimensionOptions.imageSizes[0] ??
'1K'),
};
}