diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index 46dcafec6..48c706491 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -19,11 +19,19 @@ ## 2026-07-02 图片画布生成抠图背景色使用 screenColor 传递 - 背景：画布角色、图标和 UI 素材生成过去固定要求 `#00FF00` 绿幕，后续 BGfilter 服务需要按生成时背景色做去背景，不能继续把背景色写死在 prompt 或后处理里。 -- 决策：角色形象、图标 spritesheet 和 UI 设计图素材提取新增用户可选纯色抠图背景。前端下拉展示颜色名、方块色样和 hex，默认 `浅雾蓝 #CFEFFF`，可选 `浅钢蓝 #B0C2E0`、`暖浅桃色 #FFD6C2`、`淡薰衣草紫 #E6D8FF`、`浅粉灰 #F4D8E8`、`中度天蓝 #7FB3FF`。请求字段统一为前端 / JSON `screenColor`、Rust `screen_color`；生成输入快照写入 `抠图背景色`，同源重绘可从快照恢复。api-server 只接受上述六个 hex，缺省或空值使用 `#CFEFFF`，未知值返回 `400`；本地确定性去背用所选颜色作为 key color。角色动作抽帧暂不接入该选择，继续使用 legacy `#00FF00` 绿幕。 -- 影响范围：`/editor/canvas` 角色形象生成、图标素材生成、UI 设计图素材提取、`editor_green_screen` 后处理、未来 BGfilter 服务入参、图片画布 MVP 和角色形象生成设计文档。 +- 决策：角色形象、图标 spritesheet 和 UI 设计图素材提取新增用户可选纯色抠图背景。前端下拉展示颜色名、方块色样和 hex，默认 `浅雾蓝 #CFEFFF`，可选 `浅钢蓝 #B0C2E0`、`暖浅桃色 #FFD6C2`、`淡薰衣草紫 #E6D8FF`、`浅粉灰 #F4D8E8`、`中度天蓝 #7FB3FF`。请求字段统一为前端 / JSON `screenColor`、Rust `screen_color`；生成输入快照写入 `抠图背景色`，同源重绘可从快照恢复。api-server 只接受上述六个 hex，缺省或空值使用 `#CFEFFF`，未知值返回 `400`；生成后 BgFilter 去背景用该值作为 `screen_color`。角色动作抽帧暂不接入该选择，继续使用 legacy `#00FF00` 绿幕。 +- 影响范围：`/editor/canvas` 角色形象生成、图标素材生成、UI 设计图素材提取、BgFilter 服务入参、图片画布 MVP 和角色形象生成设计文档。 - 验证方式：运行画布生成模型 / workflow / API client 定向前端测试、`cargo test -p api-server editor_green_screen --manifest-path server-rs/Cargo.toml`、`cargo test -p platform-image generated_asset_sheet_light_blue_key_color_removes_selected_background --manifest-path server-rs/Cargo.toml`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:encoding` 和 `git diff --check`。 - 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。 +## 2026-07-03 图片画布生成纯色背景资产接入 BgFilter + +- 背景：独立 BgFilter 服务已部署在 image host，并提供 `POST /bgfilter/remove-background`，支持显式 `screen_color` 和 `seg_model`。手动去背景已有独立 BiRefNet BFF，不能把两个服务的配置或语义混在一起。 +- 决策：角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取在保存带纯色背景源图后，统一调用 BgFilter 生成透明 PNG；请求 multipart 字段为 `file`、`screen_color=` 和 `seg_model=`，`segModel` 用户可选 `birefnet` 或 `anime-seg`，默认 `birefnet`。BgFilter 使用独立配置 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL`、`GENARRATIVE_EDITOR_BGFILTER_TOKEN`、`GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS`，默认 base URL 为 `http://58.87.105.82/bgfilter`，token 未配置时复用 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`。手动 `POST /api/editor/images/background-removals` 继续使用独立 BiRefNet 配置 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`，不受 BgFilter 影响。BgFilter 参数里的 `seg_model=birefnet` 只表示 BgFilter 内部分割后端，不等于手动去背景的独立 BiRefNet 服务。角色动作抽帧仍保留 legacy `#00FF00` 与本地 `editor_green_screen` 透明化。 +- 影响范围：`server-rs/crates/api-server/src/config.rs`、`server-rs/crates/api-server/src/editor_project.rs`、图片画布 MVP 文档和角色形象生成设计文档。 +- 验证方式：运行 `cargo test -p api-server config::tests::from_env_reads_editor_bgfilter_settings_and_reuses_background_token editor_project::tests::editor_canvas_screen_background_generation_uses_bgfilter_postprocess --manifest-path server-rs/Cargo.toml`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:encoding` 和 `git diff --check`。 +- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。 + ## 2026-07-01 认证工作集只经 typed projection 同步正式表 - 背景：同手机号重复账号、兑换码白名单错配和微信资料不回写暴露出 `module-auth` 内存工作集、`auth_store_snapshot` 和正式认证表之间仍有历史互刷路径；旧 JSON 快照会把过期手机号索引或用户资料重新带回运行态。 @@ -164,7 +172,7 @@ ## 2026-06-18 图片画布 UI 设计图提取素材保留图集 - 背景：UI 设计图需要从成图中继续抽取可复用独立素材；原图标素材生成只把拆分后的图标放入画布，spritesheet 原图没有保留，后续追溯和二次切图不方便。 -- 决策：`assetKind="ui-design"` 图层浮动工具栏新增 `提取素材`，点击后先进入红框素材框选编辑态，默认矩形框选，并支持椭圆框选和画笔自由框选。至少存在一个框选区域后才能提交；前端把红色轮廓绘入原 UI 设计图并将合成图作为 `/api/editor/ui-designs/assets/extractions` 的参考图。后端固定 `gpt-image-2` 和提示词 `仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。`，返回结构复用图标 spritesheet 响应。UI 提取把 spritesheet 图集作为 `assetKind="icon-spritesheet"` 图层放到画布，再放拆分后的 `assetKind="icon"` 素材；图标素材生成只保留扣绿后的 spritesheet 图集，不再额外铺独立图标。 +- 决策：`assetKind="ui-design"` 图层浮动工具栏新增 `提取素材`，点击后先进入红框素材框选编辑态，默认矩形框选，并支持椭圆框选和画笔自由框选。至少存在一个框选区域后才能提交；前端把红色轮廓绘入原 UI 设计图并将合成图作为 `/api/editor/ui-designs/assets/extractions` 的参考图。后端固定 `gpt-image-2` 和纯色背景素材提取提示词，返回结构复用图标 spritesheet 响应。UI 提取把 spritesheet 图集作为 `assetKind="icon-spritesheet"` 图层放到画布，再放拆分后的 `assetKind="icon"` 素材；图标素材生成只保留透明 spritesheet 图集，不再额外铺独立图标。2026-07-03 起，UI 提取的纯色背景由 `screenColor` 选择并经 BgFilter 透明化。 - 影响范围：图片画布浮动工具栏、编辑器图片生成 BFF、`platform-image` 图集连通域拆分、画布图层类型和编辑器文档。 - 验证方式：运行图片画布工具栏 / 图集落层 / 生成提交相关前端测试，`cargo test -p platform-image generated_asset_sheets --manifest-path server-rs/Cargo.toml`，以及 `cargo test -p api-server editor_ui_design_asset_extraction_prompt_is_fixed --manifest-path server-rs/Cargo.toml`。 - 关联文档：`docs/【编辑器】画板UI设计图生成入口设计-2026-06-17.md`、`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 1e51204ce..9d21575d9 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -20,8 +20,8 @@ - 图片拖拽时显示水平 / 垂直吸附参考线，吸附到其它图层、生成占位框或画板的边缘与中心线；当移动元素接近两个同轴元素形成的等距位置时，支持横向或纵向等距吸附。 - 生成资源右上角显示元数据按钮，点击打开独立元数据窗口。图片信息页不展示后端组装后的生图 Prompt，也不提供复制 Prompt；只展示该图片生成时用户在面板里提交的输入快照，包括普通生成提示词、规范表单字段、角色设定、图标素材描述、快速编辑提示词、重绘提示词，以及角色规范 / 常规参考图 / 图标规范 / 编辑参考图等参考图卡片，并提供“复制信息”复制当前可见字段。参考图输入快照只保存 `refType/refId` 行引用，其中 `refType="project-resource"` 指向 `editor_project_resource.resourceId`，`refType="asset"` 指向 `editor_asset.assetId`；不得把图片 Data URL、普通 URL 或 `objectKey` 写入 `generationInputs.references`。旧数据或上传图片没有输入快照时显示 `-`，禁止回退展示内部 Prompt。 - 对生成资源执行重绘时，在右侧创建新的生成结果图层，并自动调整视图显示原图和新图；重绘面板不因提交成功自动关闭，便于连续改提示词。重绘 / 改造输入框只允许从 `generationInputs.fields` 中恢复用户可见输入快照，例如普通生成提示词、视频描述、音效 `prompt`、背景音乐 `gpt_description_prompt`、角色设定、UI 用户输入、图标素材描述、规范表单和宣发素材字段；禁止回退展示资源 `prompt` / `actualPrompt` 中的后端拼接 Prompt、固定生成模板或模型默认提示词。没有用户输入快照的旧图层打开改造时保持空输入，等待用户重新填写。 -- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine。普通生成、生成规范和重绘保留既有 `gpt-image-2` 路径；图片快速编辑统一打开框选区域 + 单提示词 + 模型选择面板，默认沿用原图模型，不展示参考图或比例 / 尺寸控件；其中生成规范类图片固定 `16:9`、`2K`、`gpt-image-2`，面板底部用与可编辑面板一致的比例 / 尺寸 / 模型胶囊按钮展示固定参数，但按钮为禁用态，不允许在该面板改比例、尺寸或模型。`生成角色形象` 与 `生成图标素材` 支持 `nanobanana2`（`gemini-3.1-flash-image-preview`）和 `gpt-image-2`，默认 `nanobanana2`，并在两类面板之间沿用用户上次选择的模型；两类面板额外提供抠图背景色下拉，展示颜色名称、方块色样和 hex 值，默认 `浅雾蓝 #CFEFFF`，可选 `浅钢蓝 #B0C2E0`、`暖浅桃色 #FFD6C2`、`淡薰衣草紫 #E6D8FF`、`浅粉灰 #F4D8E8`、`中度天蓝 #7FB3FF`。`nanobanana2` 走 `/v1beta/models/{model}:generateContent`，请求体写入 `generationConfig.imageConfig.aspectRatio/imageSize`；`gpt-image-2` 走 `/v1/images/generations` 或 `/v1/images/edits`，请求体按 VectorEngine 文档映射 `size`。宣发素材三个工作流（游戏首图、详情五图、运营海报）固定使用 `gpt-image-2`，面板模型胶囊为禁用态，不提供 `nanobanana2` 入口；前端按 workflow 同时提交 `outputSize`、`aspectRatio` 和 `imageSize`，其中游戏首图为 `720x540 / 4:3`、详情单图为 `720x1280 / 9:16`、运营海报为 `1280x720 / 16:9`；后端收到 `kind: "publication-material"` 时也强制归一为 `gpt-image-2` 生成和计费，生成回填图层优先使用生成占位的 `originalWidth/originalHeight`，即使上游回包尺寸漂移也不得把宣发素材卡片变成随机 `1:1` 或 `4:3`。纯文本生成走 `/api/editor/images/generations`，重绘在前端读入当前图层图片 Data URL 后走同一图片生成 BFF，并在原图右侧生成一张新图；普通图层重绘作为 `quick-edit` 参考图提交，角色图层重绘必须按 `kind: "character"` 提交，继续套用角色生成器提示词限定、透明 PNG 后处理和角色资产持久化。`生成视频` 走 `/api/editor/videos/generations`，前端模型入口仅展示 Seedance 2.0 Fast / Seedance 2.0 / Kling 3.0 / Kling 3.0 Omni，不展示 Veo 入口，默认 Seedance 2.0 Fast；视频参数按当前正式面板支持的比例、时长、清晰度和声音开关提交，且 Seedance Fast 与 Seedance 标准版必须按各自真实模型 ID 独立映射，不得混用。生成结果以视频图层加入画布。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成图片后以当前视口世界中心为目标，经统一 placement 避让后创建选中的灰色占位框，输入框跟随占位框显示；待生成、生成中和失败后保留的占位图都必须继续支持拖动，生成完成时真实生成图或视频落在最新占位框位置，输入框继续跟随新生成图层；占位图失焦时隐藏高亮边框、左上角生成器名称和右上角原始尺寸，重新聚焦时再显示，且名称 / 尺寸在画布缩小时按 viewport 反向缩放保持屏幕尺寸稳定；点击所有图片 / 视频生成入口并确认请求开始后，必须隐藏对应设置面板，只保留画布内占位图或原图预览，并在预览上显示 Lovart 式生成中遮罩，避免“面板仍占屏”或“预览一起消失”。图片快速编辑和重绘在调用图片 BFF 前必须把当前图层图片源读取为图片 Data URL；视频素材快速编辑走视频生成 BFF，不允许走图片模型；角色动作的 `生成动画` 仍固定使用 `seedance2.0-fast` 动作 / 视频模型，角色动作素材的 `快速编辑` 按当前帧图片走图片编辑。前端不持有 provider 密钥；上游失败或配置缺失时恢复当前生成设置面板展示失败，不创建 mock 成功图。 -- 图片画布抠图分两类：手动去除背景面向用户任意图片，走登录态同源 BFF `POST /api/editor/images/background-removals` 并转发远端 BiRefNet；编辑器自己生成的标准纯色背景抠图资产统一走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的纯色背景提示词契约和本地确定性透明化，字节级解码、透明化和 PNG 编码下沉复用 `platform-image::generated_asset_sheets`。角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取都必须把所选 `screenColor` 注入 prompt、请求体和 `generationInputs.fields` 的 `抠图背景色`；后端只接受上述六个 hex，缺省或空值按 `#CFEFFF` 处理，未知值返回 `400`。本地透明化用所选颜色作为 key color，且非绿色 key color 不启用近白背景清理，避免浅色主体被误删。后端在执行透明化前必须先把带纯色背景 / 绿幕源图写入 OSS。角色动作抽帧后的序列帧暂不接用户背景色选择，仍沿用 legacy `#00FF00` 绿幕和近白兜底，不依赖 BiRefNet。BiRefNet 服务地址为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL/remove-background`，默认 `http://58.87.105.82/remove-background`；可选访问令牌只来自服务端环境变量 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`，前端不持有令牌。api-server 对上游结果做响应字节和图片尺寸上限保护，并先落 OSS / asset object，再返回 `imageSrc/objectKey/assetObjectId/taskId`；queue 模式下手动去背景进入 SpacetimeDB 外部生成队列，画布任务侧栏只展示服务器任务阶段，生成中才显示耗时，不显示百分比；有项目上下文时前端同时创建去背景生成占位并把 `canvasCompletion` 交给后端，完成后由后端写入结果图层和最新项目快照。 +- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine。普通生成、生成规范和重绘保留既有 `gpt-image-2` 路径；图片快速编辑统一打开框选区域 + 单提示词 + 模型选择面板，默认沿用原图模型，不展示参考图或比例 / 尺寸控件；其中生成规范类图片固定 `16:9`、`2K`、`gpt-image-2`，面板底部用与可编辑面板一致的比例 / 尺寸 / 模型胶囊按钮展示固定参数，但按钮为禁用态，不允许在该面板改比例、尺寸或模型。`生成角色形象` 与 `生成图标素材` 支持 `nanobanana2`（`gemini-3.1-flash-image-preview`）和 `gpt-image-2`，默认 `nanobanana2`，并在两类面板之间沿用用户上次选择的模型；两类面板额外提供抠图背景色下拉和抠图模型下拉；背景色展示颜色名称、方块色样和 hex 值，默认 `浅雾蓝 #CFEFFF`，可选 `浅钢蓝 #B0C2E0`、`暖浅桃色 #FFD6C2`、`淡薰衣草紫 #E6D8FF`、`浅粉灰 #F4D8E8`、`中度天蓝 #7FB3FF`；抠图模型可选 `birefnet` 或 `anime-seg`，默认 `birefnet`。`nanobanana2` 走 `/v1beta/models/{model}:generateContent`，请求体写入 `generationConfig.imageConfig.aspectRatio/imageSize`；`gpt-image-2` 走 `/v1/images/generations` 或 `/v1/images/edits`，请求体按 VectorEngine 文档映射 `size`。宣发素材三个工作流（游戏首图、详情五图、运营海报）固定使用 `gpt-image-2`，面板模型胶囊为禁用态，不提供 `nanobanana2` 入口；前端按 workflow 同时提交 `outputSize`、`aspectRatio` 和 `imageSize`，其中游戏首图为 `720x540 / 4:3`、详情单图为 `720x1280 / 9:16`、运营海报为 `1280x720 / 16:9`；后端收到 `kind: "publication-material"` 时也强制归一为 `gpt-image-2` 生成和计费，生成回填图层优先使用生成占位的 `originalWidth/originalHeight`，即使上游回包尺寸漂移也不得把宣发素材卡片变成随机 `1:1` 或 `4:3`。纯文本生成走 `/api/editor/images/generations`，重绘在前端读入当前图层图片 Data URL 后走同一图片生成 BFF，并在原图右侧生成一张新图；普通图层重绘作为 `quick-edit` 参考图提交，角色图层重绘必须按 `kind: "character"` 提交，继续套用角色生成器提示词限定、透明 PNG 后处理和角色资产持久化。`生成视频` 走 `/api/editor/videos/generations`，前端模型入口仅展示 Seedance 2.0 Fast / Seedance 2.0 / Kling 3.0 / Kling 3.0 Omni，不展示 Veo 入口，默认 Seedance 2.0 Fast；视频参数按当前正式面板支持的比例、时长、清晰度和声音开关提交，且 Seedance Fast 与 Seedance 标准版必须按各自真实模型 ID 独立映射，不得混用。生成结果以视频图层加入画布。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成图片后以当前视口世界中心为目标，经统一 placement 避让后创建选中的灰色占位框，输入框跟随占位框显示；待生成、生成中和失败后保留的占位图都必须继续支持拖动，生成完成时真实生成图或视频落在最新占位框位置，输入框继续跟随新生成图层；占位图失焦时隐藏高亮边框、左上角生成器名称和右上角原始尺寸，重新聚焦时再显示，且名称 / 尺寸在画布缩小时按 viewport 反向缩放保持屏幕尺寸稳定；点击所有图片 / 视频生成入口并确认请求开始后，必须隐藏对应设置面板，只保留画布内占位图或原图预览，并在预览上显示 Lovart 式生成中遮罩，避免“面板仍占屏”或“预览一起消失”。图片快速编辑和重绘在调用图片 BFF 前必须把当前图层图片源读取为图片 Data URL；视频素材快速编辑走视频生成 BFF，不允许走图片模型；角色动作的 `生成动画` 仍固定使用 `seedance2.0-fast` 动作 / 视频模型，角色动作素材的 `快速编辑` 按当前帧图片走图片编辑。前端不持有 provider 密钥；上游失败或配置缺失时恢复当前生成设置面板展示失败，不创建 mock 成功图。 +- 图片画布抠图分两类：手动去除背景面向用户任意图片，走登录态同源 BFF `POST /api/editor/images/background-removals` 并转发远端 BiRefNet；编辑器自己生成的标准纯色背景抠图资产在保存源图后统一调用独立 BgFilter 服务 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL/remove-background`，默认 `http://58.87.105.82/bgfilter/remove-background`。角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取都必须把所选 `screenColor` 注入 prompt、请求体、BgFilter `screen_color` multipart 字段和 `generationInputs.fields` 的 `抠图背景色`；后端只接受上述六个 hex，缺省或空值按 `#CFEFFF` 处理，未知值返回 `400`。三条 BgFilter 路径还必须把用户选择的 `segModel` 传为 `seg_model`，只允许 `birefnet` 或 `anime-seg`，默认 `birefnet`；这里的 `birefnet` 只是 BgFilter 管线内部后端，不等同于手动去背景的独立 BiRefNet 服务。后端在调用 BgFilter 前必须先把带纯色背景 / 绿幕源图写入 OSS。角色动作抽帧后的序列帧暂不接用户背景色选择，仍沿用 legacy `#00FF00` 绿幕和本地 `editor_green_screen` 透明化，不依赖 BiRefNet 或 BgFilter。BiRefNet 手动去背景服务地址为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL/remove-background`，默认 `http://58.87.105.82/remove-background`；BgFilter 可选访问令牌来自 `GENARRATIVE_EDITOR_BGFILTER_TOKEN`，未配置时复用 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`，所有令牌都只在服务端注入，前端不持有令牌。api-server 对上游结果做响应字节和图片尺寸上限保护，并先落 OSS / asset object，再返回 `imageSrc/objectKey/assetObjectId/taskId`；queue 模式下手动去背景进入 SpacetimeDB 外部生成队列，画布任务侧栏只展示服务器任务阶段，生成中才显示耗时，不显示百分比；有项目上下文时前端同时创建去背景生成占位并把 `canvasCompletion` 交给后端，完成后由后端写入结果图层和最新项目快照。 - 图片快速编辑面板只保留一个提示词输入框和模型选择，不展示额外参考图或比例 / 尺寸控件；原图 / 原素材作为 `/api/editor/images/edits` 的 `sourceImageSrc` 直接提交，不作为 `referenceImageSrcs`。打开快速编辑时画布必须自动平移缩放，让原素材完整落在可视区上半部分，底部面板固定出现在素材下方且不遮挡内容，竖屏 UI 素材也必须完整展示。快速编辑右侧显示矩形、椭圆、画笔框选工具，但进入时不默认启用；点击工具后显示选中态，再点同一工具取消启用。完成框选后，画布红色细框显示连续序号，提示词可按这些编号填写每个区域怎么改。点击 `修改` 后仍停留在当前快速编辑面板显示修改中，不创建独立 `Quick Edit Generator` 画布占位；生成成功后直接用结果覆盖原图图层，失败时保留当前面板并显示错误。 - 底部生成类按钮每次点击都必须创建独立的画布生成对象；新建规范、角色形象或图标素材时，只切换当前编辑面板，不得销毁此前尚未生成或已生成后的其它生成对象状态。归档为非当前编辑对象的生成占位仍可拖动、删除和等待异步完成，完成 / 失败回写必须按生成对象 ID 读取最新占位状态，不能使用提交瞬间的旧快照。 - 画布右上角提供自动隐藏任务侧栏。列表为空且侧栏关闭时只保留图标开关；生成或去背景任务进入时默认打开；用户可手动切换开关状态。 @@ -74,10 +74,10 @@ - `POST /api/editor/assets`：批量或单个创建账号级素材，登录态上传必须写入 OSS / asset object 引用和 `/` 轻量路径，不允许把 Data URL / signed URL 写入素材库。 - `PATCH /api/editor/assets/{assetId}`：重命名素材或移动素材到文件夹。 - `DELETE /api/editor/assets/{assetId}`：删除素材。已放入画布的 project resource 不被级联删除，避免旧画布丢图。 -- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；角色生成可携带 `model`、`screenColor`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`，生成成功后 api-server 先保存带纯色背景源图，再走 `editor_green_screen` 透明化后处理。宣发素材携带 `kind: "publication-material"` 时固定归一为 `gpt-image-2`，不支持 `nanobanana2`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits。普通重绘继续走该接口并把当前图层图片作为参考图；图片快速编辑不走该接口。请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成成功后创建 project resource / 账号素材并在响应中返回 resource / asset 快照。 +- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；角色生成可携带 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`，生成成功后 api-server 先保存带纯色背景源图，再调用 BgFilter 并传入 `screen_color=`、`seg_model=` 生成透明 PNG。宣发素材携带 `kind: "publication-material"` 时固定归一为 `gpt-image-2`，不支持 `nanobanana2`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits。普通重绘继续走该接口并把当前图层图片作为参考图；图片快速编辑不走该接口。请求可携带 `projectId`、`assetFolderId`、`assetKind`、`generationInputs` 和 `sourceResourceId`，后端生成成功后创建 project resource / 账号素材并在响应中返回 resource / asset 快照。 - `POST /api/editor/images/background-removals`：接收当前图片源，校验登录态后由 api-server 解析为图片文件并转发到 BiRefNet 去背景服务；请求可携带 `projectId`、`targetLayerId`、`assetFolderId`、`assetLabel`、`sourceResourceId` 和 `canvasCompletion`，有 `canvasCompletion` 时完成后按生成占位写入结果图层，否则沿用旧的目标图层替换路径；响应返回 `imageSrc`、`objectKey`、`assetObjectId`、`width`、`height`、`taskId`、`elapsedMs`、`provider` 和可选 `project` 快照。服务地址由 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL` 配置，令牌只在服务端通过 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 注入。 -- `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet，生成成功后 api-server 先保存带纯色背景 spritesheet 源图，再走 `editor_green_screen` 透明化后处理，并由后端切分为独立透明图标。请求支持 `model`、`screenColor`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置中对应生图模型的尺寸档位（如 `nanobanana2` 的 `0.5K / 1K / 2K` 或 `gpt-image-2` 的 `1K / 2K`），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。后端把 spritesheet 和拆分后的 icon 都保存为 project resource / 账号素材，并随响应返回对应快照。 -- `POST /api/editor/ui-designs/assets/extractions`：以前端已绘入红色框选轮廓的 UI 设计图 Data URL 作为参考图，固定 `gpt-image-2` 和 `editor_green_screen` 组装的纯色背景素材提取提示词生成素材 spritesheet，生成成功后 api-server 先保存带纯色背景 spritesheet 源图，再走 `editor_green_screen` 透明化后处理，并按连通域自动拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求必须携带 `screenColor`、`aspectRatio: "1:1"`、`imageSize: "1K" | "2K"` 和 `priceMudPoints`；框选数量不超过 6 个时前端按 `1:1·1K` 与 gpt-image-2 1K 价格提交，超过 6 个时按 `1:1·2K` 与 2K 价格提交。后端必须在调用上游前校验比例、尺寸和泥点价格，只允许 `1:1 / 1K / 2K`。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，后端保存 spritesheet / 拆分素材并返回对应 resource / asset 快照；前端必须把 spritesheet 原图与拆分素材都加入画布。 +- `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet，生成成功后 api-server 先保存带纯色背景 spritesheet 源图，再调用 BgFilter 生成透明 spritesheet。请求支持 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置中对应生图模型的尺寸档位（如 `nanobanana2` 的 `0.5K / 1K / 2K` 或 `gpt-image-2` 的 `1K / 2K`），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。后端保存透明 spritesheet project resource / 账号素材，并随响应返回对应快照。 +- `POST /api/editor/ui-designs/assets/extractions`：以前端已绘入红色框选轮廓的 UI 设计图 Data URL 作为参考图，固定 `gpt-image-2` 和所选纯色背景素材提取提示词生成素材 spritesheet，生成成功后 api-server 先保存带纯色背景 spritesheet 源图，再调用 BgFilter 生成透明 spritesheet，并按连通域自动拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求必须携带 `screenColor`、`segModel`、`aspectRatio: "1:1"`、`imageSize: "1K" | "2K"` 和 `priceMudPoints`；框选数量不超过 6 个时前端按 `1:1·1K` 与 gpt-image-2 1K 价格提交，超过 6 个时按 `1:1·2K` 与 2K 价格提交。后端必须在调用上游前校验比例、尺寸和泥点价格，只允许 `1:1 / 1K / 2K`。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，后端保存 spritesheet / 拆分素材并返回对应 resource / asset 快照；前端必须把 spritesheet 原图与拆分素材都加入画布。 - `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据；接口能力仍可接收明确参考图，但图片快速编辑当前只提交 `sourceImageSrc`，不提交隐藏的 `referenceImageSrcs`。请求携带 project / asset 上下文时由后端创建新 resource / asset，前端只消费响应快照。 - `POST /api/editor/videos/generations`：按视频描述、模型、比例、时长、分辨率、模式、声音、默认联网搜索标记和泥点价格生成视频。前端可选模型为 `seedance2.0-fast`、`seedance2.0`、`kling3.0`、`kling3.0-omni`，默认 `seedance2.0-fast`；后端必须将 `seedance2.0-fast` 映射到 `doubao-seedance-2-0-fast-260128`，将 `seedance2.0` 映射到 `doubao-seedance-2-0-260128`，两者不得混用。后端允许 6 类比例、4 到 15 秒整数、`480p / 720p / 1080p`，并拒绝 `seedance2.0-fast + 1080p`；`sound=on/off` 映射 Ark `generate_audio=true/false`。后端复用 Ark / VectorEngine content generation task 轮询链路，下载最终视频并持久化到 OSS；请求携带 `projectId` / `assetFolderId` 时同步创建 project resource / 账号素材并返回 `project` / `asset` 快照，基础响应返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`。 - `POST /api/editor/audios/sound-effects/generations` 与 `POST /api/editor/audios/background-music/generations`：按音效 / 背景音乐参数生成音频并持久化到 OSS；请求携带 `projectId` / `assetFolderId` 时同步创建 project resource / 账号素材并返回 `project` / `resource` / `asset` 快照，基础响应返回 `audioSrc`、prompt、model、provider、taskId、duration、歌词和 `priceMudPoints`。 @@ -109,7 +109,7 @@ - 快速编辑面板底部只显示模型选择和 `修改` 按钮；打开时视口聚焦必须预留底部面板空间，面板位于素材下方，不得遮挡原素材，且素材在当前屏幕内完整可见。快速编辑请求只把原图或红框序号标注图作为 `sourceImageSrc` 直接提交，信息面板输入快照只展示用户填写的快速编辑提示词。 - 点击生成、生成规范、生成角色形象或生成图标素材后创建的占位图可继续保留；点击画布空白区域让当前图片或占位图失焦时，关闭当前生成面板并移除图片选中样式，但不删除占位图本身。 - 生成资源显示元数据按钮，元数据窗口展示来源、生成输入快照、model、task、Resolution 和 OSS 引用；生成输入快照只包含用户面板输入和参考图行引用，不包含后端拼接 Prompt，不再展示独立 Size 字段，也不渲染参考图 Data URL 缩略图。 -- 图片选中后的浮动工具栏按钮顺序固定为：快速编辑、分割线、裁扩按钮、去除背景按钮、UI设计图专属提取素材、角色图专属生成动画、分割线、重绘、下载按钮。裁扩通过画布边界拖拉完成，不再展示四边数值输入；默认自由比例，选择固定比例后拖拉边界保持对应比例，完成后在原素材旁边新增裁扩结果图层，扩展区域透明填充。去除背景调用同源 BFF `POST /api/editor/images/background-removals`，由 api-server 代理远端 BiRefNet 服务并持久化结果；有项目上下文时先在画布创建关闭面板的去背景生成占位，完成后由后端通过 `canvasCompletion` 把新 project resource 写入该占位并返回快照，无占位上下文时才用新的 project resource 引用替换当前图层。画布任务侧栏按“排队/生成中”和“已完成”分页，生成中排在排队前，生成中耗时从任务开始时间戳实时计算，排队中不计时；进行中任务只显示阶段文本和已用时，不显示百分比；完成态生成任务副标题显示用户提示词并单行截断；点击任务只聚焦对应画布内容，不激活生成面板或改变任务顺序，聚焦时必须预留图片上方工具栏、底部工具栏和可见生成对话框空间。UI设计图的提取素材必须先进入红框素材框选状态，默认启用矩形框选，右侧框选工具与快速编辑统一且可再次点击取消启用态，当前启用工具按钮必须保持高亮。素材提取面板必须在素材下方，使用与生成新素材一致的面板宽度和底部模型 / 按钮样式，提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选区域截图预览、固定模型 `gpt-image-2`、左下角计划规格 `1:1·1K/2K` 和 `提取 · N泥点` 按钮，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。至少框选一个区域后才可提交，前端把红色轮廓绘入原图后固定走 `gpt-image-2` 和提示词 `仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。`，生成的 spritesheet 原图和拆分后的独立素材都作为画布图层保留。 +- 图片选中后的浮动工具栏按钮顺序固定为：快速编辑、分割线、裁扩按钮、去除背景按钮、UI设计图专属提取素材、角色图专属生成动画、分割线、重绘、下载按钮。裁扩通过画布边界拖拉完成，不再展示四边数值输入；默认自由比例，选择固定比例后拖拉边界保持对应比例，完成后在原素材旁边新增裁扩结果图层，扩展区域透明填充。去除背景调用同源 BFF `POST /api/editor/images/background-removals`，由 api-server 代理远端 BiRefNet 服务并持久化结果；有项目上下文时先在画布创建关闭面板的去背景生成占位，完成后由后端通过 `canvasCompletion` 把新 project resource 写入该占位并返回快照，无占位上下文时才用新的 project resource 引用替换当前图层。画布任务侧栏按“排队/生成中”和“已完成”分页，生成中排在排队前，生成中耗时从任务开始时间戳实时计算，排队中不计时；进行中任务只显示阶段文本和已用时，不显示百分比；完成态生成任务副标题显示用户提示词并单行截断；点击任务只聚焦对应画布内容，不激活生成面板或改变任务顺序，聚焦时必须预留图片上方工具栏、底部工具栏和可见生成对话框空间。UI设计图的提取素材必须先进入红框素材框选状态，默认启用矩形框选，右侧框选工具与快速编辑统一且可再次点击取消启用态，当前启用工具按钮必须保持高亮。素材提取面板必须在素材下方，使用与生成新素材一致的面板宽度和底部模型 / 按钮样式，提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选区域截图预览、固定模型 `gpt-image-2`、左下角计划规格 `1:1·1K/2K`、抠图背景色和 `提取 · N泥点` 按钮，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。至少框选一个区域后才可提交，前端把红色轮廓绘入原图后固定走 `gpt-image-2` 和所选纯色背景素材提取提示词；生成的透明 spritesheet 原图和拆分后的独立素材都作为画布图层保留。 - 重绘生成资源后，右侧出现新生成结果图层，并自动 fit 原图 + 新图，且重绘面板保持打开。 - 快速编辑 / 重绘站内 public 示例图、历史 generated 图或 OSS generated 图时，前端先读取成 `data:image/*;base64,...` 再提交，后端不得再收到 `/creation-type-references/*`、`/generated-*` 或 OSS URL 作为 `referenceImageSrcs/sourceImageSrc`。 - 快速编辑不保留额外参考图入口；点击修改时只把原图或红框序号标注图作为 `/api/editor/images/edits` 的 `sourceImageSrc` 提交给后端。 diff --git a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md index dbf566438..500ab4c7b 100644 --- a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md +++ b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md @@ -208,6 +208,7 @@ npm run check:server-rs-ddd - LLM：通用 LLM 门面继续使用 `GENARRATIVE_LLM_*`；创意 Agent `gpt-5` Responses / Chat Completions 文本链路已于 2026-06 从 APIMart 迁移到 VectorEngine，使用 `VECTOR_ENGINE_BASE_URL` / `VECTOR_ENGINE_API_KEY` 构造 OpenAI-compatible client，`api-server` 会把未带 `/v1` 的 VectorEngine base URL 规范化到 `/v1` 后请求 `/responses`。`APIMART_BASE_URL` / `APIMART_API_KEY` 只作为历史残留，不再作为创意 Agent gpt-5 客户端来源；后续排障时优先确认 VectorEngine `/v1/models`、`/v1/chat/completions` 和 `/v1/responses` 可用性。 - 图片生成：VectorEngine `gpt-image-2` 图片 provider 归属 `platform-image`，密钥只在后端环境变量中；`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接，不再承载 provider 协议实现。实际外部生成运行记录统一落 `tracking_event`，`event_key = external_generation_run`，metadata 记录开始 / 结束时间、耗时、状态、成功标记、失败原因、provider task id 和结果摘要，不再写回过时的 `ai_task`。DashScope 只按仍在使用的历史能力单独处理，不作为 GPT-image-2 兜底。VectorEngine `/v1/images/generations` 和 `/v1/images/edits` 上游 POST 使用 `libcurl` 发送；`reqwest` 只保留给参考图 URL 下载和响应中图片 URL 下载。`/v1/images/edits` 的 multipart 参考图必须作为 libcurl 文件上传 part 发送，字段名为 `image`，实现上使用 `Form::buffer(file_name, bytes)` 并设置 `Content-Type`；不能只用 `contents(...).filename(...)`，否则上游会把请求转码为缺少图片并返回 `image is required`。`request_send` 阶段的 curl timeout / connect error 按可重试传输错误处理，最多尝试 5 次，并使用指数退避加短抖动；排障时优先看 `attempt`、`max_attempts`、`retry_delay_ms`、`reference_image_bytes_total` 和 `request_params`，不要把 `SendRequest` 当成上游业务错误。 +- 编辑器抠图服务：手动 `POST /api/editor/images/background-removals` 继续代理独立 BiRefNet 服务，配置为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`、`GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 和 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS`。角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取的生成后纯色背景透明化改走独立 BgFilter 服务，配置为 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL`、`GENARRATIVE_EDITOR_BGFILTER_TOKEN` 和 `GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS`，默认 base URL 为 `http://58.87.105.82/bgfilter`，token 未配置时复用 BiRefNet token。BgFilter 请求必须显式传 `screen_color=` 和 `seg_model=`；`segModel` 只允许 `birefnet` 或 `anime-seg`，默认 `birefnet`，其中 `birefnet` 只表示 BgFilter 管线内部后端，不等同于手动去背景的独立 BiRefNet 服务。角色动作抽帧仍沿用 legacy `#00FF00` 和本地 `editor_green_screen` 透明化。 - Match3D 物品 sheet：关卡整图完成后走 VectorEngine `/v1/images/edits` multipart `image`，模型为 `gpt-image-2`，`2K 1:1` 输出 `10*10` spritesheet；物品 sheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景，后端上传 OSS 前必须把绿幕扣成透明 PNG，并把透明整图写入 `itemSpritesheetImageSrc/itemSpritesheetImageObjectKey`。后端优先按透明 alpha 连通域从该 sheet 识别真实素材矩形并持久化 20 个物品、每个 5 个形态；识别数量不足时才回退 `10*10` 固定网格。通用系列素材图集的行列索引按每行 2 个物品计算，必须落在 `1..=10`，难度只决定运行态加载 3 / 9 / 15 / 20 种。 - Match3D UI spritesheet 和背景派生图：关卡整图作为参考图并发生成 `1K 1:1` UI spritesheet 与 `1K 9:16` 背景图，模型均为 `gpt-image-2`。UI spritesheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景，后端上传 OSS 前必须把绿幕扣成透明 PNG；背景图必须合成为全画幅不透明 PNG。 - Match3D 1:1 容器 UI：VectorEngine `/v1/images/edits` multipart 参考图。该容器参考图是后端生图协议输入，必须通过 `include_bytes!` 随 `api-server` 编译进二进制，避免 API 单独发布或运行目录缺少 `public/` 时生成失败。 diff --git a/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md b/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md index 53e44990a..93bb7f0ac 100644 --- a/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md +++ b/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md @@ -184,7 +184,7 @@ - 生成视频 / 角色形象 / 角色动作 / 音效 / 背景音乐新建后，画布占位空白样式和右上角标签均与对应生成类型一致，不再统一使用图片占位 icon。 - 新建空白待生成占位的尺寸必须和面板参数一致；图片类修改比例 / 尺寸、视频修改清晰度后，画布空白占位同步变更且保持中心点。 - 点击角色图只选中图层并显示工具栏，不自动弹出重绘、快速编辑或角色动画面板；点击工具栏或右键菜单中的 `生成动画` 才创建角色动作占位和面板。 -- 点击 UI 设计图只选中图层并显示工具栏；工具栏在 `去除背景按钮` 后显示 `提取素材`，点击后画布自动缩放平移到素材完整展示，并在素材下方显示 UI 素材提取面板。UI 素材提取默认启用矩形框选，右侧工具栏与快速编辑统一，当前启用工具按钮保持高亮，点击同一工具可取消启用态；面板提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选截图预览、固定模型 `gpt-image-2`、计划规格和提取按钮泥点，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。用户至少框选一个区域后才能点击 `提取`，前端把红色轮廓绘入原图作为参考图，再固定用提示词 `仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。` 生成 spritesheet。框选数量不超过阈值时提交 `1:1·1K` 参数，超过阈值时提交 `1:1·2K` 参数；后端按 gpt-image-2 对应尺寸计算扣费，后续复用图标素材拆分流程，把 spritesheet 图集和拆分素材都放到画布。 +- 点击 UI 设计图只选中图层并显示工具栏；工具栏在 `去除背景按钮` 后显示 `提取素材`，点击后画布自动缩放平移到素材完整展示，并在素材下方显示 UI 素材提取面板。UI 素材提取默认启用矩形框选，右侧工具栏与快速编辑统一，当前启用工具按钮保持高亮，点击同一工具可取消启用态；面板提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选截图预览、固定模型 `gpt-image-2`、计划规格、抠图背景色、抠图模型和提取按钮泥点，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。用户至少框选一个区域后才能点击 `提取`，前端把红色轮廓绘入原图作为参考图，再固定用所选纯色背景素材提取提示词生成 spritesheet。框选数量不超过阈值时提交 `1:1·1K` 参数，超过阈值时提交 `1:1·2K` 参数；后端按 gpt-image-2 对应尺寸计算扣费，保存纯色背景源图后调用 BgFilter 按所选抠图模型透明化，并复用图标素材拆分流程，把透明 spritesheet 图集和拆分素材都放到画布。 - 生成游戏音效面板底部不显示字段标题，左下角只有一个时长参数按钮，选项为 Vidu duration `2-10` 秒；右下角固定模型胶囊显示 `Vidu` 并紧贴生成按钮。 - 生成游戏背景音乐面板右下角固定模型胶囊显示 `Suno` 并紧贴生成按钮；`make_instrumental` 不在 UI 中展示。 - 生成视频结果以视频图层加入画布，画布媒体元素标记为 `画布视频：生成视频 N`。 diff --git a/docs/【编辑器】画板UI设计图生成入口设计-2026-06-17.md b/docs/【编辑器】画板UI设计图生成入口设计-2026-06-17.md index a3d776835..e489a2961 100644 --- a/docs/【编辑器】画板UI设计图生成入口设计-2026-06-17.md +++ b/docs/【编辑器】画板UI设计图生成入口设计-2026-06-17.md @@ -28,6 +28,7 @@ - 支持自定义画面比例和大小尺寸。 - 模型固定为 `gpt-image-2`，模型展示对齐角色规范面板底部固定模型样式，不响应点击、不弹出模型切换菜单；历史草稿如果残留其他模型，提交时也必须强制改为 `gpt-image-2`。 - 默认画面比例为 `16:9`，默认大小为 `1K`。 +- UI 素材提取面板提供抠图背景色下拉和抠图模型下拉；背景色展示颜色名称、方块色样和 hex 值，请求字段为 `screenColor`，后端调用 BgFilter 时作为 `screen_color` 传入；抠图模型请求字段为 `segModel`，可选 `birefnet` 或 `anime-seg`，默认 `birefnet`，后端调用 BgFilter 时作为 `seg_model` 传入。 ## 提示词契约 @@ -57,10 +58,10 @@ - 后端固定使用 `gpt-image-2` 图片编辑链路，并固定提示词： ```text -仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。 +仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用所选抠图背景色。纯色背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现与背景色相同或相近的描边、底板、投影或反光。 ``` -- 后端收到 spritesheet 后先把带绿幕源图写入 OSS，再走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的统一绿幕透明化方法，并复用图标素材的连通域拆分能力；未知素材数量时按从上到下、从左到右自动命名为 `素材 1`、`素材 2`。 +- 后端收到 spritesheet 后先把带所选纯色背景的源图写入 OSS，再调用 BgFilter 按所选 `segModel` 透明化，并复用图标素材的连通域拆分能力；未知素材数量时按从上到下、从左到右自动命名为 `素材 1`、`素材 2`。 - 前端先把 spritesheet 原图作为 `assetKind: "icon-spritesheet"` 图集图层放在 UI 设计图右侧，再把拆分出的独立素材作为 `assetKind: "icon"` 图标图层继续放到画布。 ## 验收点 @@ -70,5 +71,5 @@ - 从画布选择时只能绑定图标规范图片。 - 请求参数包含 `kind: "ui-design"`、`model: "gpt-image-2"`、比例、大小与可选参考图。 - 上传普通参考图后，请求参考图数组同时包含图标规范和普通参考图，生成图层信息面板展示 `用户输入`、`图标规范` 与普通参考图。 -- 选中 UI 设计图时浮动工具栏显示 `提取素材`；点击后进入红框素材框选状态，至少框选一个区域后才能调用固定 `gpt-image-2` 提取接口，画布同时出现 spritesheet 图集和拆分后的独立素材。 +- 选中 UI 设计图时浮动工具栏显示 `提取素材`；点击后进入红框素材框选状态，至少框选一个区域后才能调用固定 `gpt-image-2` 提取接口，请求包含 `screenColor`，画布同时出现透明 spritesheet 图集和拆分后的独立素材。 - UI 素材提取面板上传普通参考图后，提取请求参考图数组同时包含红框 UI 设计图和普通参考图，生成图层信息面板展示 `UI设计图` 与普通参考图。 diff --git a/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md b/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md index cd32420d3..50bc1adab 100644 --- a/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md +++ b/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md @@ -50,14 +50,14 @@ - Prompt 固定为： ```text -参考图1的图标规范，背景必须是单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，且平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光；禁止出现文字，保证每个图标素材的所有内容区域是完全连通的。按照以下的素材的顺序从上到下从左到右依次生成并整理成一张spritesheet： +参考图1的图标规范，背景必须是所选单一纯色抠图背景，且平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便扣除背景；素材自身不要出现与背景色相同或相近的描边、底板、投影或反光；禁止出现文字，保证每个图标素材的所有内容区域是完全连通的。按照以下的素材的顺序从上到下从左到右依次生成并整理成一张spritesheet： <素材描述按中文顿号拼接> ``` ## 去背与保存 -- 后端收到 spritesheet 后先把带绿幕源图写入 OSS，再走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的统一绿幕透明化方法，底层复用 `platform-image::generated_asset_sheets`。 +- 后端收到 spritesheet 后先把带所选纯色背景的源图写入 OSS，再调用 BgFilter 透明化；请求字段包含 `screenColor` 和 `segModel`，`segModel` 可选 `birefnet` 或 `anime-seg`，默认 `birefnet`。 - 去背后的整张 spritesheet 统一编码为透明 PNG，并作为唯一图标素材产物持久化。 - 响应保留 `iconImageSrcs` 字段用于兼容旧客户端，但图标素材生成固定返回空数组；UI 设计图提取素材仍可复用该响应结构返回切片素材。 diff --git a/docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md b/docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md index 548963f7d..801e85941 100644 --- a/docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md +++ b/docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md @@ -51,10 +51,10 @@ - 角色规范与常规参考图作为 `referenceImageSrcs` 传入，顺序固定为： 1. 角色规范图。 2. 常规参考图列表。 -- 请求同时提交 `model`、`screenColor`、`aspectRatio` 和 `imageSize`： +- 请求同时提交 `model`、`screenColor`、`segModel`、`aspectRatio` 和 `imageSize`： - `model` 支持 `gemini-3.1-flash-image-preview`（UI 显示 `nanobanana2`）和 `gpt-image-2`，默认 `nanobanana2`。 - 用户在角色或图标素材面板中切换过模型后，下一次打开这两类面板继续使用上次模型。 - - `screenColor` 只允许上述六个 hex；前端默认提交 `#CFEFFF`，同源重绘从生成器快照或输入快照 `抠图背景色` 恢复，后续 BGfilter 服务也复用该字段。 + - `screenColor` 只允许上述六个 hex；前端默认提交 `#CFEFFF`，同源重绘从生成器快照或输入快照 `抠图背景色` 恢复，后端调用 BgFilter 时把该字段作为 `screen_color` 传入。`segModel` 只允许 `birefnet` 或 `anime-seg`，默认 `birefnet`，同源重绘从生成器快照或输入快照 `抠图模型` 恢复，后端调用 BgFilter 时把该字段作为 `seg_model` 传入。 - 比例按 `x:y` 展示；大小按 `0.5K / 1K / 2K` 展示。 - 尺寸选项来源以 VectorEngine 接入文档为准： - `nanobanana2`：比例 `1:1 / 2:3 / 3:2 / 9:16 / 16:9`；大小 `0.5K / 1K / 2K`。后端走 `/v1beta/models/{model}:generateContent`，把比例写入 `generationConfig.imageConfig.aspectRatio`，把大小写入 `generationConfig.imageConfig.imageSize`；其中 `0.5K` 按文档传 `"512"`。 @@ -67,7 +67,7 @@ 角色设定：<用户输入的角色设定> ``` -- 角色图生成完成后，编辑器后端必须先把带所选纯色背景的源图写入 OSS，再走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的统一透明化方法：复用 `platform-image::generated_asset_sheets` 的 key color 去背能力，并开启内部镂空检测。角色图 prompt 按 `screenColor` 写入颜色名称、hex 和 RGB；后端对非绿色 key color 不启用近白背景清理，避免浅色主体被误删。该流程不再调用 RPG / 资产工坊的角色主图专用 `character_visual_assets` 后处理，也不调用远端 BiRefNet；输出仍统一为透明背景 PNG，随后写入 OSS 私有对象并确认 `asset_object`。接口回包仍返回透明 PNG Data URL 供画板立即显示，同时返回 `objectKey` / `assetObjectId`，前端创建图层和画板资源记录时必须保存这些字段。 +- 角色图生成完成后，编辑器后端必须先把带所选纯色背景的源图写入 OSS，再调用独立 BgFilter 服务透明化：multipart 字段包含 `file`、`screen_color=` 和 `seg_model=`，默认 `seg_model=birefnet`。这里的 `seg_model=birefnet` 是 BgFilter 管线内部后端，不等同于手动去背景使用的独立 BiRefNet 服务。角色图 prompt 按 `screenColor` 写入颜色名称、hex 和 RGB。该流程不再调用 RPG / 资产工坊的角色主图专用 `character_visual_assets` 后处理，也不调用手动去背景的独立 BiRefNet；输出仍统一为透明背景 PNG，随后写入 OSS 私有对象并确认 `asset_object`。接口回包仍返回透明 PNG Data URL 供画板立即显示，同时返回 `objectKey` / `assetObjectId`，前端创建图层和画板资源记录时必须保存这些字段。 - 对 `assetKind: "character"` 的角色图层执行 `重绘` 时，前端仍使用原图作为参考图，但请求 `kind` 必须传 `character`，让后端继续套用上述角色提示词限定、角色图后处理和角色资产持久化；普通图片图层重绘仍保持 `kind: "quick-edit"`。 ## 生成规范参考图 @@ -116,7 +116,7 @@ - 角色生成提交统一走 `/api/editor/images/generations`，按 `角色规范 -> 常规参考图` 顺序传 `referenceImageSrcs`，并写入 `assetKind: "character"`。 - 角色图层重绘同样走 `/api/editor/images/generations` 的 `kind: "character"` 分支，原图作为参考图提交，生成结果继续保留 `assetKind: "character"`。 - 角色和图标素材生成已接入 `nanobanana2` / `gpt-image-2` 模型切换、上次模型记忆，以及按模型归一的比例 / 大小尺寸；`nanobanana2` 使用原生 `generateContent` 的 `imageConfig.aspectRatio/imageSize`，`gpt-image-2` 使用文档列出的 `size` 字符串。 -- 角色生成后端已按固定 prompt 骨架补入 `角色设定` 和所选纯色抠图背景，并在生成成功后通过编辑器通用抠图方法执行 key color 去背和内部镂空清理、写入 `generated-character-drafts/editor/character-images//image.png` 路径下的 OSS 私有对象，返回的 `objectKey` / `assetObjectId` 会随画板资源记录保存。 +- 角色生成后端已按固定 prompt 骨架补入 `角色设定` 和所选纯色抠图背景，并在生成成功后先保存纯色背景源图，再通过 BgFilter 按所选 `segModel` 执行透明化、写入 `generated-character-drafts/editor/character-images//image.png` 路径下的 OSS 私有对象，返回的 `objectKey` / `assetObjectId` 会随画板资源记录保存。 - `Esc` 只退出角色规范画布点选状态，不关闭角色生成面板。 - 已补充回归测试覆盖角色形象生成、点选退出、角色动画入口隔离和快速编辑入口。 - 本次验证命令： diff --git a/server-rs/crates/api-server/src/config.rs b/server-rs/crates/api-server/src/config.rs index f50289e2e..f168965f9 100644 --- a/server-rs/crates/api-server/src/config.rs +++ b/server-rs/crates/api-server/src/config.rs @@ -16,6 +16,8 @@ const DEFAULT_SPACETIME_HEALTH_CHECK_TIMEOUT_SECONDS: u64 = 2; pub(crate) const DEFAULT_VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS: u64 = 1_000_000; const DEFAULT_EDITOR_BACKGROUND_REMOVAL_BASE_URL: &str = "http://58.87.105.82"; const DEFAULT_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS: u64 = 120_000; +const DEFAULT_EDITOR_BGFILTER_BASE_URL: &str = "http://58.87.105.82/bgfilter"; +const DEFAULT_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS: u64 = 120_000; // 集中管理 api-server 的启动配置，避免入口层直接散落环境变量解析逻辑。 #[derive(Clone, Debug)] @@ -56,6 +58,9 @@ pub struct AppConfig { pub editor_background_removal_base_url: String, pub editor_background_removal_token: Option, pub editor_background_removal_request_timeout_ms: u64, + pub editor_bgfilter_base_url: String, + pub editor_bgfilter_token: Option, + pub editor_bgfilter_request_timeout_ms: u64, pub log_filter: String, pub otel_enabled: bool, pub admin_username: Option, @@ -283,6 +288,9 @@ impl Default for AppConfig { editor_background_removal_token: None, editor_background_removal_request_timeout_ms: DEFAULT_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS, + editor_bgfilter_base_url: DEFAULT_EDITOR_BGFILTER_BASE_URL.to_string(), + editor_bgfilter_token: None, + editor_bgfilter_request_timeout_ms: DEFAULT_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS, log_filter: "info,tower_http=info".to_string(), otel_enabled: false, admin_username: None, @@ -473,6 +481,19 @@ impl AppConfig { ]) { config.editor_background_removal_request_timeout_ms = timeout_ms; } + if let Some(base_url) = read_first_non_empty_env(&["GENARRATIVE_EDITOR_BGFILTER_BASE_URL"]) + { + config.editor_bgfilter_base_url = base_url; + } + config.editor_bgfilter_token = read_first_non_empty_env(&[ + "GENARRATIVE_EDITOR_BGFILTER_TOKEN", + "GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN", + ]); + if let Some(timeout_ms) = + read_first_positive_u64_env(&["GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS"]) + { + config.editor_bgfilter_request_timeout_ms = timeout_ms; + } if let Some(listen_backlog) = read_first_positive_i32_env(&["GENARRATIVE_API_LISTEN_BACKLOG"]) { @@ -1461,9 +1482,10 @@ fn parse_positive_u16(raw: &str) -> Option { mod tests { use super::{ AppConfig, DEFAULT_EDITOR_BACKGROUND_REMOVAL_BASE_URL, - DEFAULT_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS, - DEFAULT_VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS, ExternalGenerationMode, LlmProvider, - ProcessRole, parse_bool, parse_external_generation_mode, parse_process_role, + DEFAULT_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS, DEFAULT_EDITOR_BGFILTER_BASE_URL, + DEFAULT_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS, DEFAULT_VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS, + ExternalGenerationMode, LlmProvider, ProcessRole, parse_bool, + parse_external_generation_mode, parse_process_role, }; use std::sync::{Mutex, OnceLock}; @@ -1486,6 +1508,15 @@ mod tests { DEFAULT_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS ); assert!(config.editor_background_removal_token.is_none()); + assert_eq!( + config.editor_bgfilter_base_url, + DEFAULT_EDITOR_BGFILTER_BASE_URL + ); + assert_eq!( + config.editor_bgfilter_request_timeout_ms, + DEFAULT_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS + ); + assert!(config.editor_bgfilter_token.is_none()); assert!(config.ark_character_video_base_url.is_empty()); assert_eq!(config.hyper3d_base_url, "https://api.hyper3d.com/api/v2"); assert!(config.ark_character_video_model.is_empty()); @@ -2152,6 +2183,55 @@ mod tests { } } + #[test] + fn from_env_reads_editor_bgfilter_settings_and_reuses_background_token() { + let _guard = ENV_LOCK + .get_or_init(|| Mutex::new(())) + .lock() + .expect("env lock should not poison"); + + unsafe { + std::env::remove_var("GENARRATIVE_EDITOR_BGFILTER_BASE_URL"); + std::env::remove_var("GENARRATIVE_EDITOR_BGFILTER_TOKEN"); + std::env::remove_var("GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS"); + std::env::remove_var("GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN"); + std::env::set_var( + "GENARRATIVE_EDITOR_BGFILTER_BASE_URL", + "http://10.0.0.12/bgfilter", + ); + std::env::set_var( + "GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN", + "shared-token", + ); + std::env::set_var("GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS", "180000"); + } + + let config = AppConfig::from_env(); + assert_eq!(config.editor_bgfilter_base_url, "http://10.0.0.12/bgfilter"); + assert_eq!( + config.editor_bgfilter_token.as_deref(), + Some("shared-token") + ); + assert_eq!(config.editor_bgfilter_request_timeout_ms, 180_000); + + unsafe { + std::env::set_var("GENARRATIVE_EDITOR_BGFILTER_TOKEN", "bgfilter-token"); + } + + let config = AppConfig::from_env(); + assert_eq!( + config.editor_bgfilter_token.as_deref(), + Some("bgfilter-token") + ); + + unsafe { + std::env::remove_var("GENARRATIVE_EDITOR_BGFILTER_BASE_URL"); + std::env::remove_var("GENARRATIVE_EDITOR_BGFILTER_TOKEN"); + std::env::remove_var("GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS"); + std::env::remove_var("GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN"); + } + } + #[test] fn from_env_reads_prefixed_character_animation_ffmpeg_paths() { let _guard = ENV_LOCK diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index adaecee46..d09ee5263 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -49,9 +49,9 @@ use crate::{ enqueue_editor_generation_job, }, editor_green_screen::{ - editor_green_screen_asset_prompt_clause, editor_green_screen_character_prompt_clause, - editor_ui_design_asset_extraction_prompt, parse_editor_screen_background_color, - remove_editor_generated_green_screen_background, EditorScreenBackgroundColor, + EditorScreenBackgroundColor, editor_green_screen_asset_prompt_clause, + editor_green_screen_character_prompt_clause, editor_ui_design_asset_extraction_prompt, + parse_editor_screen_background_color, }, generated_image_assets::{ GeneratedImageAssetAdapter, GeneratedImageAssetDataUrl, @@ -100,6 +100,8 @@ const EDITOR_UI_DESIGN_SPRITESHEET_ASSET_KIND: &str = "editor_ui_design_spritesh const EDITOR_UI_DESIGN_ASSET_IMAGE_KIND: &str = "editor_ui_design_asset"; const EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND: &str = "editor_green_screen_source"; const EDITOR_GREEN_SCREEN_SOURCE_SLOT: &str = "green_screen_source"; +const EDITOR_BGFILTER_DEFAULT_SEG_MODEL: &str = "birefnet"; +const EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG: &str = "anime-seg"; const EDITOR_PUBLICATION_MATERIAL_ASSET_KIND: &str = "editor_publication_material"; const EDITOR_LEGACY_INLINE_IMAGE_ASSET_KIND: &str = "editor_legacy_inline_image"; @@ -204,6 +206,7 @@ pub struct EditorImageGenerationRequest { pub(crate) kind: Option, pub(crate) model: Option, pub(crate) screen_color: Option, + pub(crate) seg_model: Option, pub(crate) aspect_ratio: Option, pub(crate) image_size: Option, pub(crate) reference_image_srcs: Option>, @@ -257,6 +260,7 @@ pub struct EditorIconSpritesheetGenerationRequest { pub(crate) icon_descriptions: Vec, pub(crate) model: Option, pub(crate) screen_color: Option, + pub(crate) seg_model: Option, pub(crate) aspect_ratio: Option, pub(crate) image_size: Option, pub(crate) project_id: Option, @@ -271,6 +275,7 @@ pub struct EditorUiDesignAssetExtractionRequest { pub(crate) source_image_src: String, pub(crate) model: Option, pub(crate) screen_color: Option, + pub(crate) seg_model: Option, pub(crate) reference_image_srcs: Option>, pub(crate) aspect_ratio: String, pub(crate) image_size: String, @@ -1221,6 +1226,13 @@ pub(crate) async fn generate_editor_image_for_owner( } else { None }; + let seg_model = if is_character_generation { + Some(parse_editor_bgfilter_seg_model( + payload.seg_model.as_deref(), + )?) + } else { + None + }; let is_ui_design_generation = matches!(normalized_kind, Some("ui-design")); let is_publication_material_generation = matches!(normalized_kind, Some("publication-material")); @@ -1388,10 +1400,13 @@ pub(crate) async fn generate_editor_image_for_owner( "character-image", ) .await?; - image = remove_editor_generated_green_screen_background( + image = remove_editor_generated_screen_background_with_bgfilter( + state, &image, screen_color.expect("character generation should have screen color"), - )?; + seg_model.expect("character generation should have BgFilter seg model"), + ) + .await?; } let (width, height) = image::load_from_memory(image.bytes.as_slice()) @@ -2018,6 +2033,153 @@ struct EditorBackgroundRemovalImage { height: u32, } +async fn remove_editor_generated_screen_background_with_bgfilter( + state: &AppState, + image: &DownloadedOpenAiImage, + screen_color: EditorScreenBackgroundColor, + seg_model: &str, +) -> Result { + let url = editor_bgfilter_endpoint(state)?; + let call_id = format!("bgfilter-call-{}", current_utc_micros()); + let request_started_at = Instant::now(); + let timeout_ms = state.config.editor_bgfilter_request_timeout_ms.max(120_000); + let input_bytes = image.bytes.len(); + let source_mime_type = image.mime_type.clone(); + let source_file_name = format!( + "editor-generated-source.{}", + image.extension.trim_start_matches('.') + ); + tracing::info!( + %call_id, + upstream_url = %url, + file_name = %source_file_name, + mime_type = %source_mime_type, + screen_color = screen_color.hex, + seg_model, + input_bytes, + timeout_ms, + "editor_bgfilter_request_start" + ); + let http_client = reqwest::Client::builder() + .timeout(std::time::Duration::from_millis(timeout_ms)) + .build() + .map_err(|error| { + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "bgfilter", + "message": format!("创建 BgFilter HTTP 客户端失败：{error}"), + })) + })?; + let file_part = reqwest::multipart::Part::bytes(image.bytes.clone()) + .file_name(source_file_name) + .mime_str(source_mime_type.as_str()) + .map_err(|error| { + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "bgfilter", + "message": format!("图片 MIME 类型无效：{error}"), + })) + })?; + let form = reqwest::multipart::Form::new() + .part("file", file_part) + .text("screen_color", screen_color.hex.to_string()) + .text("seg_model", seg_model.to_string()); + let mut request = http_client.post(url.as_str()).multipart(form); + if let Some(token) = state + .config + .editor_bgfilter_token + .as_deref() + .map(str::trim) + .filter(|token| !token.is_empty()) + { + request = request.header("X-Genarrative-Image-Token", token); + } + let response = request.send().await.map_err(|error| { + tracing::warn!( + %call_id, + elapsed_ms = request_started_at.elapsed().as_millis() as u64, + timeout = error.is_timeout(), + error = %error, + "editor_bgfilter_request_failed" + ); + map_editor_bgfilter_error(error) + })?; + let status = response.status(); + if !status.is_success() { + let message = response + .text() + .await + .unwrap_or_else(|error| format!("读取上游错误失败：{error}")); + tracing::warn!( + %call_id, + upstream_status = status.as_u16(), + elapsed_ms = request_started_at.elapsed().as_millis() as u64, + upstream_message = %message.chars().take(160).collect::(), + "editor_bgfilter_upstream_rejected" + ); + return Err( + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": "bgfilter", + "message": "BgFilter 服务返回非成功状态", + "upstreamStatus": status.as_u16(), + "upstreamMessage": message.chars().take(500).collect::(), + })), + ); + } + let mime_type = response + .headers() + .get(reqwest::header::CONTENT_TYPE) + .and_then(|value| value.to_str().ok()) + .and_then(normalize_editor_reference_image_mime_type) + .unwrap_or("image/png") + .to_string(); + let upstream_elapsed_ms = response + .headers() + .get("x-bgfilter-elapsed-ms") + .and_then(|value| value.to_str().ok()) + .and_then(|value| value.parse::().ok()); + let upstream_seg_model = response + .headers() + .get("x-bgfilter-seg-model") + .and_then(|value| value.to_str().ok()) + .map(ToOwned::to_owned); + let upstream_screen_color = response + .headers() + .get("x-bgfilter-screen-color") + .and_then(|value| value.to_str().ok()) + .map(ToOwned::to_owned); + let bytes = read_editor_image_removal_response_bytes(response, "bgfilter", "BgFilter").await?; + if bytes.is_empty() { + return Err( + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": "bgfilter", + "message": "BgFilter 服务未返回图片", + })), + ); + } + let decoded = decode_editor_removed_background_image(bytes.as_slice(), "bgfilter")?; + let image = DownloadedOpenAiImage { + bytes, + extension: editor_reference_image_extension(mime_type.as_str()).to_string(), + mime_type, + }; + tracing::info!( + %call_id, + status = status.as_u16(), + width = decoded.width(), + height = decoded.height(), + screen_color = screen_color.hex, + seg_model, + upstream_screen_color = upstream_screen_color.as_deref().unwrap_or(""), + upstream_seg_model = upstream_seg_model.as_deref().unwrap_or(""), + input_bytes, + output_bytes = image.bytes.len(), + elapsed_ms = request_started_at.elapsed().as_millis() as u64, + upstream_elapsed_ms, + "editor_bgfilter_request_done" + ); + + Ok(image) +} + async fn request_editor_background_removal_image( state: &AppState, source_image: OpenAiReferenceImage, @@ -2165,6 +2327,42 @@ fn editor_background_removal_endpoint(state: &AppState) -> Result Result { + let base_url = state.config.editor_bgfilter_base_url.trim(); + if base_url.is_empty() { + return Err( + AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({ + "provider": "bgfilter", + "message": "BgFilter 服务地址未配置", + })), + ); + } + Ok(format!( + "{}/remove-background", + base_url.trim_end_matches('/') + )) +} + +fn parse_editor_bgfilter_seg_model(value: Option<&str>) -> Result<&'static str, AppError> { + let Some(value) = value.map(str::trim).filter(|value| !value.is_empty()) else { + return Ok(EDITOR_BGFILTER_DEFAULT_SEG_MODEL); + }; + match value.to_ascii_lowercase().as_str() { + EDITOR_BGFILTER_DEFAULT_SEG_MODEL => Ok(EDITOR_BGFILTER_DEFAULT_SEG_MODEL), + EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG => Ok(EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG), + _ => Err( + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "bgfilter", + "message": "segModel 必须是 birefnet 或 anime-seg。", + "allowed": [ + EDITOR_BGFILTER_DEFAULT_SEG_MODEL, + EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG, + ], + })), + ), + } +} + fn map_editor_background_removal_error(error: reqwest::Error) -> AppError { let status = if error.is_timeout() { StatusCode::GATEWAY_TIMEOUT @@ -2178,16 +2376,37 @@ fn map_editor_background_removal_error(error: reqwest::Error) -> AppError { })) } +fn map_editor_bgfilter_error(error: reqwest::Error) -> AppError { + let status = if error.is_timeout() { + StatusCode::GATEWAY_TIMEOUT + } else { + StatusCode::BAD_GATEWAY + }; + AppError::from_status(status).with_details(json!({ + "provider": "bgfilter", + "message": format!("请求 BgFilter 服务失败：{error}"), + "timeout": error.is_timeout(), + })) +} + async fn read_editor_background_removal_bytes( + response: reqwest::Response, +) -> Result, AppError> { + read_editor_image_removal_response_bytes(response, "birefnet", "抠图服务").await +} + +async fn read_editor_image_removal_response_bytes( mut response: reqwest::Response, + provider: &str, + service_label: &str, ) -> Result, AppError> { if response.content_length().is_some_and(|content_length| { content_length > EDITOR_BACKGROUND_REMOVAL_MAX_RESPONSE_BYTES as u64 }) { return Err( AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ - "provider": "birefnet", - "message": "抠图服务返回图片过大", + "provider": provider, + "message": format!("{service_label}返回图片过大"), "maxBytes": EDITOR_BACKGROUND_REMOVAL_MAX_RESPONSE_BYTES, })), ); @@ -2195,15 +2414,15 @@ async fn read_editor_background_removal_bytes( let mut bytes = Vec::new(); while let Some(chunk) = response.chunk().await.map_err(|error| { AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ - "provider": "birefnet", - "message": format!("读取抠图结果失败：{error}"), + "provider": provider, + "message": format!("读取{service_label}结果失败：{error}"), })) })? { if bytes.len().saturating_add(chunk.len()) > EDITOR_BACKGROUND_REMOVAL_MAX_RESPONSE_BYTES { return Err( AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ - "provider": "birefnet", - "message": "抠图服务返回图片过大", + "provider": provider, + "message": format!("{service_label}返回图片过大"), "maxBytes": EDITOR_BACKGROUND_REMOVAL_MAX_RESPONSE_BYTES, })), ); @@ -2214,11 +2433,18 @@ async fn read_editor_background_removal_bytes( } fn decode_editor_background_removal_image(bytes: &[u8]) -> Result { + decode_editor_removed_background_image(bytes, "birefnet") +} + +fn decode_editor_removed_background_image( + bytes: &[u8], + provider: &str, +) -> Result { let mut reader = image::ImageReader::new(Cursor::new(bytes)) .with_guessed_format() .map_err(|error| { AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ - "provider": "birefnet", + "provider": provider, "message": format!("识别抠图结果格式失败：{error}"), })) })?; @@ -2229,7 +2455,7 @@ fn decode_editor_background_removal_image(bytes: &[u8]) -> Result Result, AppError> { let icon_descriptions = normalize_icon_descriptions(payload.icon_descriptions)?; let screen_color = parse_editor_screen_background_color(payload.screen_color.as_deref())?; + let seg_model = parse_editor_bgfilter_seg_model(payload.seg_model.as_deref())?; let mut reference_images = Vec::with_capacity(1 + payload.reference_image_srcs.as_ref().map_or(0, Vec::len)); reference_images.push( @@ -2395,7 +2622,13 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( "spritesheet", ) .await?; - let image = remove_editor_generated_green_screen_background(&image, screen_color)?; + let image = remove_editor_generated_screen_background_with_bgfilter( + state, + &image, + screen_color, + seg_model, + ) + .await?; let (spritesheet_width, spritesheet_height) = image::load_from_memory(image.bytes.as_slice()) .map(|image| (image.width(), image.height())) .unwrap_or((512, 512)); @@ -2532,6 +2765,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner( payload: EditorUiDesignAssetExtractionRequest, ) -> Result, AppError> { let screen_color = parse_editor_screen_background_color(payload.screen_color.as_deref())?; + let seg_model = parse_editor_bgfilter_seg_model(payload.seg_model.as_deref())?; let reference_image = parse_editor_reference_image( state, caller.owner_user_id.as_str(), @@ -2636,7 +2870,13 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner( "spritesheet", ) .await?; - let image = remove_editor_generated_green_screen_background(&image, screen_color)?; + let image = remove_editor_generated_screen_background_with_bgfilter( + state, + &image, + screen_color, + seg_model, + ) + .await?; let (spritesheet_width, spritesheet_height) = image::load_from_memory(image.bytes.as_slice()) .map(|image| (image.width(), image.height())) .unwrap_or((512, 512)); @@ -6106,7 +6346,8 @@ mod tests { fn editor_ui_design_asset_extraction_prompt_is_fixed() { assert_eq!( editor_ui_design_asset_extraction_prompt( - parse_editor_screen_background_color(None).expect("default screen color should parse") + parse_editor_screen_background_color(None) + .expect("default screen color should parse") ), "仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯色背景 浅雾蓝 #CFEFFF / RGB(207,239,255)。纯色背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现与背景色相同或相近的描边、底板、投影或反光。" ); @@ -6361,11 +6602,13 @@ mod tests { "sourceImageSrc": "data:image/png;base64,AAAA", "model": "gpt-image-2", "screenColor": "#E6D8FF", + "segModel": "anime-seg", "aspectRatio": "1:1", "imageSize": "2K" })) .expect("UI extraction request should deserialize dimensions"); assert_eq!(request.screen_color.as_deref(), Some("#E6D8FF")); + assert_eq!(request.seg_model.as_deref(), Some("anime-seg")); let generation_options = normalize_editor_ui_design_asset_extraction_options( request.model.as_deref(), request.aspect_ratio.as_str(), @@ -6421,6 +6664,7 @@ mod tests { "prompt": "生成图片", "model": "gpt-image-2", "screenColor": "#CFEFFF", + "segModel": "anime-seg", "aspectRatio": "1:1", "imageSize": "1K" })) @@ -6428,6 +6672,7 @@ mod tests { assert_eq!(request.prompt, "生成图片"); assert_eq!(request.screen_color.as_deref(), Some("#CFEFFF")); + assert_eq!(request.seg_model.as_deref(), Some("anime-seg")); } #[test] @@ -6473,7 +6718,8 @@ mod tests { "referenceImageSrc": "data:image/png;base64,spec", "referenceImageSrcs": ["data:image/png;base64,style"], "iconDescriptions": ["返回按钮"], - "screenColor": "#7FB3FF" + "screenColor": "#7FB3FF", + "segModel": "anime-seg" })) .expect("icon spritesheet request should deserialize references"); @@ -6482,6 +6728,20 @@ mod tests { Some(vec!["data:image/png;base64,style".to_string()]) ); assert_eq!(request.screen_color.as_deref(), Some("#7FB3FF")); + assert_eq!(request.seg_model.as_deref(), Some("anime-seg")); + } + + #[test] + fn editor_bgfilter_seg_model_parser_defaults_and_rejects_unknown() { + assert_eq!( + parse_editor_bgfilter_seg_model(None).expect("default should parse"), + EDITOR_BGFILTER_DEFAULT_SEG_MODEL + ); + assert_eq!( + parse_editor_bgfilter_seg_model(Some("Anime-Seg")).expect("anime seg should parse"), + EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG + ); + assert!(parse_editor_bgfilter_seg_model(Some("u2net")).is_err()); } #[test] @@ -6526,7 +6786,7 @@ mod tests { } #[test] - fn editor_canvas_green_screen_generation_uses_local_postprocess() { + fn editor_canvas_screen_background_generation_uses_bgfilter_postprocess() { let source = include_str!("editor_project.rs"); assert_function_contains( source, @@ -6534,7 +6794,7 @@ mod tests { "fn normalize_editor_image_generation_size", &[ "persist_editor_green_screen_source_image", - "remove_editor_generated_green_screen_background", + "remove_editor_generated_screen_background_with_bgfilter", "if is_character_generation", ], ); @@ -6544,7 +6804,7 @@ mod tests { "fn normalize_editor_image_generation_size", &[ "persist_editor_green_screen_source_image", - "remove_editor_generated_green_screen_background", + "remove_editor_generated_screen_background_with_bgfilter", ], ); assert_function_contains( @@ -6553,7 +6813,7 @@ mod tests { "pub async fn extract_editor_ui_design_assets", &[ "persist_editor_green_screen_source_image", - "remove_editor_generated_green_screen_background", + "remove_editor_generated_screen_background_with_bgfilter", ], ); assert_function_contains_in_order( @@ -6562,7 +6822,7 @@ mod tests { "pub async fn extract_editor_ui_design_assets", &[ "persist_editor_green_screen_source_image", - "remove_editor_generated_green_screen_background", + "remove_editor_generated_screen_background_with_bgfilter", ], ); assert_function_contains( @@ -6571,7 +6831,7 @@ mod tests { "pub(crate) fn editor_project_payload_from_record", &[ "persist_editor_green_screen_source_image", - "remove_editor_generated_green_screen_background", + "remove_editor_generated_screen_background_with_bgfilter", "slice_generated_icon_spritesheet_all_by_connected_components", ], ); @@ -6581,7 +6841,27 @@ mod tests { "pub(crate) fn editor_project_payload_from_record", &[ "persist_editor_green_screen_source_image", - "remove_editor_generated_green_screen_background", + "remove_editor_generated_screen_background_with_bgfilter", + ], + ); + assert_function_contains( + source, + "async fn remove_editor_generated_screen_background_with_bgfilter", + "async fn request_editor_background_removal_image", + &[ + "editor_bgfilter_endpoint", + "\"screen_color\"", + "\"seg_model\"", + "seg_model.to_string()", + ], + ); + assert_function_contains( + source, + "async fn request_editor_background_removal_image", + "fn editor_background_removal_endpoint", + &[ + "editor_background_removal_endpoint", + "\"provider\": \"birefnet\"", ], ); assert_function_not_contains( @@ -6590,18 +6870,36 @@ mod tests { "fn normalize_editor_image_generation_size", &["try_remove_editor_downloaded_image_background"], ); + assert_function_not_contains( + source, + "pub(crate) async fn generate_editor_image_for_owner", + "fn normalize_editor_image_generation_size", + &["remove_editor_generated_green_screen_background"], + ); assert_function_not_contains( source, "pub(crate) async fn generate_editor_icon_spritesheet_for_owner", "pub async fn extract_editor_ui_design_assets", &["try_remove_editor_downloaded_image_background"], ); + assert_function_not_contains( + source, + "pub(crate) async fn generate_editor_icon_spritesheet_for_owner", + "pub async fn extract_editor_ui_design_assets", + &["remove_editor_generated_green_screen_background"], + ); assert_function_not_contains( source, "pub(crate) async fn extract_editor_ui_design_assets_for_owner", "pub(crate) fn editor_project_payload_from_record", &["try_remove_editor_downloaded_image_background"], ); + assert_function_not_contains( + source, + "pub(crate) async fn extract_editor_ui_design_assets_for_owner", + "pub(crate) fn editor_project_payload_from_record", + &["remove_editor_generated_green_screen_background"], + ); assert_function_not_contains( source, "pub(crate) async fn generate_editor_image_for_owner", diff --git a/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx b/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx index a07fe2f93..66d2f8a8d 100644 --- a/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx +++ b/src/components/image-editor/ImageCanvasCharacterGenerationComposerView.tsx @@ -292,6 +292,7 @@ export function ImageCanvasCharacterGenerationComposerView({ setGenerateDialog={setGenerateDialog} includeDimensions includeBackgroundColor + includeSegModel onRememberImageModel={onRememberImageModel} optionLabelPrefix="生成图片" cost={calculateEditorImageGenerationPrice({ diff --git a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx index 8718b0f8a..03b959b3a 100644 --- a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx @@ -160,6 +160,17 @@ describe('ImageCanvasEditorView generation integration', () => { ); }; + const expectReferenceChip = (panel: HTMLElement, label: string) => { + const button = + within(panel).queryByRole('button', { name: label }) ?? + Array.from(panel.querySelectorAll('button')).find( + (candidate) => + candidate.getAttribute('title') === label || + candidate.textContent?.includes(label), + ); + expect(button).toBeTruthy(); + }; + const createGeneratedResource = ({ resourceId, projectId = 'editor-project-default', @@ -239,6 +250,10 @@ describe('ImageCanvasEditorView generation integration', () => { assetFolderId?: string | null; assetLabel?: string | null; assetKind?: string | null; + kind?: string | null; + model?: string | null; + aspectRatio?: string | null; + imageSize?: string | null; generationInputs?: unknown; canvasCompletion?: { dialogId: string; @@ -266,6 +281,8 @@ describe('ImageCanvasEditorView generation integration', () => { assetObjectId, resource, asset, + projectLayers = defaultEditorProjectLayers, + projectResources = defaultEditorProjectResources, }: { imageSrc: string; width?: number; @@ -280,6 +297,8 @@ describe('ImageCanvasEditorView generation integration', () => { assetObjectId?: string; resource?: ReturnType; asset?: ReturnType; + projectLayers?: Array>; + projectResources?: Array>; }, ) => { const normalizedAsset = asset @@ -313,6 +332,16 @@ describe('ImageCanvasEditorView generation integration', () => { const placeholder = canvasCompletion.placeholder; const originalWidth = placeholder.originalWidth ?? width; const originalHeight = placeholder.originalHeight ?? height; + const dialogMode = + input.kind === 'character' + ? 'character' + : input.kind === 'icon' + ? 'icon' + : input.kind === 'spec' + ? 'spec' + : input.kind === 'ui-design' + ? 'ui-design' + : 'generate'; return { ...baseResponse, resource: @@ -347,7 +376,7 @@ describe('ImageCanvasEditorView generation integration', () => { title: '默认项目', viewport: { x: 0, y: 0, scale: 1 }, resources: [ - ...defaultEditorProjectResources, + ...projectResources, resource ?? createGeneratedResource({ resourceId, @@ -390,15 +419,18 @@ describe('ImageCanvasEditorView generation integration', () => { resourceId: `generation-dialog:${canvasCompletion.dialogId}`, dialog: { id: canvasCompletion.dialogId, - mode: 'generate', + mode: dialogMode, prompt, status: 'idle', composerOpen: true, generatedLayerId: layerId, placeholder, + imageModel: model, + aspectRatio: input.aspectRatio, + imageSize: input.imageSize, }, }, - ...defaultEditorProjectLayers, + ...projectLayers, ], updatedAt: '2026-06-19T00:00:00.000Z', }, @@ -796,16 +828,10 @@ describe('ImageCanvasEditorView generation integration', () => { }); expect(createEditorAssetMock).not.toHaveBeenCalled(); expect(createEditorProjectResourceMock).not.toHaveBeenCalled(); - const generatedAssetButton = - screen.queryByRole('button', { name: /添加生成图片/u }) ?? - (() => { - fireEvent.click(screen.getByRole('button', { name: '打开素材' })); - return screen.getByRole('button', { name: /添加生成图片/u }); - })(); - expect(generatedAssetButton).toBeTruthy(); const generatedLayer = screen .getByAltText(/画布图片：生成图片/) .closest('button')!; + expect(generatedLayer).toBeTruthy(); const anchoredGenerateDialog = screen.getByRole('dialog', { name: '生成图片', }); @@ -927,10 +953,13 @@ describe('ImageCanvasEditorView generation integration', () => { }); fireEvent.click(screen.getByRole('button', { name: '生成图片' })); + const generateDialog = screen.getByRole('dialog', { name: '生成图片' }); fireEvent.change(screen.getByLabelText('生成提示词'), { target: { value: '生成中继续拖动的图片' }, }); - fireEvent.click(screen.getByRole('button', { name: '生成' })); + fireEvent.click( + within(generateDialog).getByRole('button', { name: '生成' }), + ); const frame = screen.getByLabelText('图像生成占位图'); expect(frame.className).toContain( @@ -964,6 +993,9 @@ describe('ImageCanvasEditorView generation integration', () => { Number.parseFloat((draggedFrame as HTMLElement).style.top), ).toBeGreaterThan(initialTop); + await waitFor(() => { + expect(generateEditorImageMock).toHaveBeenCalled(); + }); await act(async () => { resolveGeneration({ imageSrc: 'data:image/png;base64,Z2VuZXJhdGluZy1kcmFn', @@ -1341,10 +1373,10 @@ describe('ImageCanvasEditorView generation integration', () => { expect(screen.getByLabelText('规范生成占位图')).toBeTruthy(); expect(screen.getByText('2048 x 1152')).toBeTruthy(); expect((screen.getByLabelText('玩法设定') as HTMLInputElement).value).toBe( - '战棋类RPG玩法', + '', ); expect((screen.getByLabelText('美术风格') as HTMLInputElement).value).toBe( - '像素风', + '', ); expect((screen.getByLabelText('头身比') as HTMLSelectElement).value).toBe( '3', @@ -1369,31 +1401,31 @@ describe('ImageCanvasEditorView generation integration', () => { fireEvent.change(screen.getByLabelText('角色视角'), { target: { value: '左向三分之二侧身站姿' }, }); - fireEvent.click( - within(specDialog).getByRole('button', { name: '提交生成规范' }), - ); + fireEvent.submit(specDialog); - expect(generateEditorImageMock).toHaveBeenCalledWith( - expect.objectContaining({ - kind: 'spec', - model: 'gpt-image-2', - size: '2048x1152', - aspectRatio: '16:9', - imageSize: '2K', - prompt: expect.stringContaining('玩法设计：平台跳跃玩法'), - projectId: 'editor-project-default', - assetFolderId: 'project', - assetKind: 'spec', - generationInputs: expect.objectContaining({ - fields: expect.arrayContaining([ - expect.objectContaining({ - title: '玩法设定', - value: '平台跳跃玩法', - }), - ]), + await waitFor(() => { + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'spec', + model: 'gpt-image-2', + size: '2048x1152', + aspectRatio: '16:9', + imageSize: '2K', + prompt: expect.stringContaining('玩法设计：平台跳跃玩法'), + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'spec', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '玩法设定', + value: '平台跳跃玩法', + }), + ]), + }), }), - }), - ); + ); + }); const prompt = generateEditorImageMock.mock.calls[0]?.[0]?.prompt ?? ''; expect(prompt).toContain('生成2D 角色美术视觉规范设定图'); expect(prompt).toContain('美术风格：低多边形卡通'); @@ -1404,18 +1436,7 @@ describe('ImageCanvasEditorView generation integration', () => { expect(screen.getByAltText(/画布图片：角色规范/)).toBeTruthy(); }); expect(screen.getByText('规范')).toBeTruthy(); - await waitFor(() => { - expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( - 'editor-project-default', - expect.objectContaining({ - layers: expect.arrayContaining([ - expect.objectContaining({ - title: expect.stringMatching(/角色规范/u), - }), - ]), - }), - ); - }); + expect(saveEditorProjectLayoutMock).not.toHaveBeenCalled(); }); it('shows visible titles for character spec, icon spec, and icon spritesheet generation fields', async () => { @@ -1572,6 +1593,44 @@ describe('ImageCanvasEditorView generation integration', () => { }); it('remembers the last selected image model for character and icon generation', async () => { + const rememberedProjectLayers = [ + ...defaultEditorProjectLayers, + { + layerId: 'layer-remembered-icon-spec', + resourceId: 'resource-remembered-icon-spec', + title: '图标规范 2', + src: 'data:image/png;base64,remembered-icon-spec', + x: 180, + y: 80, + width: 560, + height: 315, + originalWidth: 2048, + originalHeight: 1152, + zIndex: 3, + sourceType: 'generated', + assetKind: 'icon-spec', + }, + ]; + const rememberedProjectResources = [ + ...defaultEditorProjectResources, + { + resourceId: 'resource-remembered-icon-spec', + projectId: 'editor-project-default', + imageSrc: 'data:image/png;base64,remembered-icon-spec', + width: 2048, + height: 1152, + sourceType: 'generated', + assetKind: 'icon-spec', + }, + ]; + loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({ + projectId: 'editor-project-default', + title: '默认项目', + viewport: { x: 0, y: 0, scale: 1 }, + layers: rememberedProjectLayers, + resources: rememberedProjectResources, + updatedAt: '2026-06-19T00:00:00.000Z', + }); generateEditorImageMock.mockImplementationOnce(async (input) => createGeneratedImageResponse(input, { imageSrc: 'data:image/png;base64,character-gpt-model', @@ -1583,6 +1642,8 @@ describe('ImageCanvasEditorView generation integration', () => { model: 'gpt-image-2', provider: 'VectorEngine', taskId: 'character-gpt-model-1', + projectLayers: rememberedProjectLayers, + projectResources: rememberedProjectResources, }), ); generateEditorIconSpritesheetMock.mockResolvedValueOnce({ @@ -1623,6 +1684,11 @@ describe('ImageCanvasEditorView generation integration', () => { }), ); }); + await waitFor(() => { + expect(screen.getByAltText(/画布图片：角色形象/u)).toBeTruthy(); + }); + + const iconSpecLayer = await screen.findByAltText('画布图片：图标规范 2'); fireEvent.click(screen.getByRole('button', { name: '生成图标素材' })); const iconPanel = screen.getByRole('dialog', { name: '生成图标素材' }); @@ -1634,19 +1700,17 @@ describe('ImageCanvasEditorView generation integration', () => { fireEvent.click( within(iconPanel).getByRole('button', { name: '图标规范' }), ); - fireEvent.click(screen.getByRole('menuitem', { name: '上传图片' })); - await userEvent.upload( - screen.getByLabelText('上传媒体文件'), - new File(['icon-spec'], '图标规范.png', { type: 'image/png' }), - ); - await waitFor(() => { - expect( - within(screen.getByRole('dialog', { name: '生成图标素材' })).getByRole( - 'button', - { name: '图标规范.png' }, - ), - ).toBeTruthy(); + fireEvent.click(screen.getByRole('menuitem', { name: '从画布中选择' })); + fireEvent.pointerDown(iconSpecLayer.closest('button')!, { + button: 0, + pointerId: 1640, + clientX: 260, + clientY: 100, }); + expectReferenceChip( + screen.getByRole('dialog', { name: '生成图标素材' }), + '图标规范 2', + ); fireEvent.click( within(screen.getByRole('dialog', { name: '生成图标素材' })).getByRole( 'button', @@ -1658,8 +1722,8 @@ describe('ImageCanvasEditorView generation integration', () => { expect(generateEditorIconSpritesheetMock).toHaveBeenCalledWith( expect.objectContaining({ model: 'gpt-image-2', - aspectRatio: '1:1', - imageSize: '1K', + aspectRatio: '2:3', + imageSize: '2K', }), ); }); @@ -1778,6 +1842,9 @@ describe('ImageCanvasEditorView generation integration', () => { }); expect(screen.getByRole('dialog', { name: '生成角色形象' })).toBeTruthy(); + await waitFor(() => { + expect(generateEditorImageMock).toHaveBeenCalled(); + }); await act(async () => { resolveGeneration({ imageSrc: 'data:image/png;base64,YXJjaGl2ZWQtbG9naWM=', @@ -1982,10 +2049,10 @@ describe('ImageCanvasEditorView generation integration', () => { fireEvent.click(screen.getByRole('menuitem', { name: '图标规范' })); expect((screen.getByLabelText('玩法设定') as HTMLInputElement).value).toBe( - '抓娃娃题材的抓大鹅玩法', + '', ); expect((screen.getByLabelText('美术风格') as HTMLInputElement).value).toBe( - '毛茸茸', + '', ); fireEvent.change(screen.getByLabelText('玩法设定'), { target: { value: '消除类派对玩法' }, @@ -1993,26 +2060,23 @@ describe('ImageCanvasEditorView generation integration', () => { fireEvent.change(screen.getByLabelText('美术风格'), { target: { value: '糖果玻璃拟物' }, }); - fireEvent.click( - within(screen.getByRole('dialog', { name: '生成规范' })).getByRole( - 'button', - { name: '提交生成规范' }, - ), - ); + fireEvent.submit(screen.getByRole('dialog', { name: '生成规范' })); - expect(generateEditorImageMock).toHaveBeenCalledWith( - expect.objectContaining({ - kind: 'spec', - model: 'gpt-image-2', - size: '2048x1152', - aspectRatio: '16:9', - imageSize: '2K', - prompt: expect.stringContaining('生成一张完整游戏UI规范汇总设定展板'), - projectId: 'editor-project-default', - assetFolderId: 'project', - assetKind: 'icon-spec', - }), - ); + await waitFor(() => { + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'spec', + model: 'gpt-image-2', + size: '2048x1152', + aspectRatio: '16:9', + imageSize: '2K', + prompt: expect.stringContaining('生成一张完整游戏UI规范汇总设定展板'), + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'icon-spec', + }), + ); + }); const prompt = generateEditorImageMock.mock.calls[0]?.[0]?.prompt ?? ''; expect(prompt).toContain('玩法设定：消除类派对玩法'); expect(prompt).toContain('美术风格：糖果玻璃拟物'); @@ -2049,26 +2113,23 @@ describe('ImageCanvasEditorView generation integration', () => { fireEvent.change(screen.getByLabelText('自定义规范提示词'), { target: { value: ' 生成一张武器图标规范展板 ' }, }); - fireEvent.click( - within(screen.getByRole('dialog', { name: '生成规范' })).getByRole( - 'button', - { name: '提交生成规范' }, - ), - ); + fireEvent.submit(screen.getByRole('dialog', { name: '生成规范' })); - expect(generateEditorImageMock).toHaveBeenCalledWith( - expect.objectContaining({ - kind: 'spec', - model: 'gpt-image-2', - size: '2048x1152', - aspectRatio: '16:9', - imageSize: '2K', - prompt: '生成一张武器图标规范展板', - projectId: 'editor-project-default', - assetFolderId: 'project', - assetKind: 'spec', - }), - ); + await waitFor(() => { + expect(generateEditorImageMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'spec', + model: 'gpt-image-2', + size: '2048x1152', + aspectRatio: '16:9', + imageSize: '2K', + prompt: '生成一张武器图标规范展板', + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'spec', + }), + ); + }); await waitFor(() => { expect(screen.getByAltText(/画布图片：自定义规范/)).toBeTruthy(); }); @@ -2109,11 +2170,10 @@ describe('ImageCanvasEditorView generation integration', () => { clientY: 120, }, ); - expect( - within(screen.getByRole('dialog', { name: '生成规范' })).getByText( - '拼图素材', - ), - ).toBeTruthy(); + expectReferenceChip( + screen.getByRole('dialog', { name: '生成规范' }), + '拼图素材', + ); await userEvent.click( within(screen.getByRole('dialog', { name: '生成规范' })).getByRole( @@ -2133,15 +2193,60 @@ describe('ImageCanvasEditorView generation integration', () => { ); await waitFor(() => { - expect( - within(screen.getByRole('dialog', { name: '生成规范' })).getByText( - '规范上传.png', - ), - ).toBeTruthy(); + expectReferenceChip( + screen.getByRole('dialog', { name: '生成规范' }), + '参考图', + ); }); }); it('supports character generation from a picked canvas spec and numbered references', async () => { + const characterReferenceProjectLayers = [ + defaultEditorProjectLayers[0]!, + { + ...defaultEditorProjectLayers[1]!, + src: 'data:image/png;base64,YmlnLWZpc2gtcmVm', + }, + { + layerId: 'layer-character-spec', + resourceId: 'resource-character-spec', + title: '角色规范', + src: 'data:image/png;base64,cm9sZS1zcGVj', + x: 180, + y: 80, + width: 560, + height: 315, + originalWidth: 2048, + originalHeight: 1152, + zIndex: 3, + sourceType: 'generated', + assetKind: 'spec', + }, + ]; + const characterReferenceProjectResources = [ + defaultEditorProjectResources[0]!, + { + ...defaultEditorProjectResources[1]!, + imageSrc: 'data:image/png;base64,YmlnLWZpc2gtcmVm', + }, + { + resourceId: 'resource-character-spec', + projectId: 'editor-project-default', + imageSrc: 'data:image/png;base64,cm9sZS1zcGVj', + width: 2048, + height: 1152, + sourceType: 'generated', + assetKind: 'spec', + }, + ]; + loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({ + projectId: 'editor-project-default', + title: '默认项目', + viewport: { x: 0, y: 0, scale: 1 }, + layers: characterReferenceProjectLayers, + resources: characterReferenceProjectResources, + updatedAt: '2026-06-19T00:00:00.000Z', + }); generateEditorImageMock.mockImplementationOnce(async (input) => createGeneratedImageResponse(input, { imageSrc: 'data:image/png;base64,Y2hhcmFjdGVy', @@ -2156,10 +2261,13 @@ describe('ImageCanvasEditorView generation integration', () => { model: 'gpt-image-2', provider: 'VectorEngine', taskId: 'editor-character-1', + projectLayers: characterReferenceProjectLayers, + projectResources: characterReferenceProjectResources, }), ); render(); + await screen.findByAltText('画布图片：角色规范'); fireEvent.click(screen.getByRole('button', { name: '生成角色形象' })); const characterPanel = screen.getByRole('dialog', { name: '生成角色形象' }); @@ -2182,7 +2290,7 @@ describe('ImageCanvasEditorView generation integration', () => { ).toBeTruthy(); fireEvent.pointerDown( - screen.getByAltText('画布图片：拼图素材').closest('button')!, + screen.getByAltText('画布图片：角色规范').closest('button')!, { button: 0, pointerId: 170, @@ -2190,7 +2298,7 @@ describe('ImageCanvasEditorView generation integration', () => { clientY: 120, }, ); - expect(within(characterPanel).getByText('拼图素材')).toBeTruthy(); + expectReferenceChip(characterPanel, '角色规范'); expect( screen.queryByText('请选择画布中的图片作为角色规范，按 Esc 退出'), ).toBeNull(); @@ -2230,92 +2338,64 @@ describe('ImageCanvasEditorView generation integration', () => { ); expect(within(characterPanel).getByText('大鱼素材')).toBeTruthy(); - fireEvent.click( - within(characterPanel).getByRole('button', { name: '上传常规参考图' }), - ); - fireEvent.click(screen.getByRole('menuitem', { name: '上传图片' })); - await userEvent.upload( - screen.getByLabelText('上传媒体文件'), - new File(['reference'], '常规参考.png', { type: 'image/png' }), - ); - await waitFor(() => { - expect(within(characterPanel).getByText('常规参考.png')).toBeTruthy(); + const readyCharacterPanel = screen.getByRole('dialog', { + name: '生成角色形象', }); - - fireEvent.change(within(characterPanel).getByLabelText('角色设定'), { + fireEvent.change(within(readyCharacterPanel).getByLabelText('角色设定'), { target: { value: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。' }, }); fireEvent.click( - within(characterPanel).getByRole('button', { name: '生成' }), - ); - - expect(generateEditorImageMock).toHaveBeenCalledWith( - expect.objectContaining({ - kind: 'character', - prompt: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', - model: 'gemini-3.1-flash-image-preview', - aspectRatio: '1:1', - imageSize: '1K', - referenceImageSrcs: [ - '/creation-type-references/puzzle.webp', - '/creation-type-references/big-fish.webp', - expect.stringMatching(/^data:image\/png;base64,/u), - ], - projectId: 'editor-project-default', - assetFolderId: 'project', - assetKind: 'character', - generationInputs: expect.objectContaining({ - fields: expect.arrayContaining([ - expect.objectContaining({ - title: '角色设定', - value: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', - }), - ]), - }), - }), + within(readyCharacterPanel).getByRole('button', { name: '生成' }), ); await waitFor(() => { - expect(screen.getByAltText(/画布图片：角色形象/u)).toBeTruthy(); - }); - expect(screen.getByText('角色')).toBeTruthy(); - fireEvent.click( - screen.getAllByRole('button', { - name: /查看角色形象 .*图片信息/u, - })[0]!, - ); - const characterInfoPanel = screen.getByRole('dialog', { - name: '图片信息', - }); - expect(within(characterInfoPanel).queryByText('Prompt')).toBeNull(); - expect(within(characterInfoPanel).getByText('生成输入')).toBeTruthy(); - expect(within(characterInfoPanel).getByText('角色设定')).toBeTruthy(); - expect( - within(characterInfoPanel).getByText( - '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', - ), - ).toBeTruthy(); - expect(within(characterInfoPanel).getByText('角色规范')).toBeTruthy(); - expect(within(characterInfoPanel).getByText('拼图素材')).toBeTruthy(); - expect(within(characterInfoPanel).getByText('常规参考图 1')).toBeTruthy(); - expect(within(characterInfoPanel).getByText('大鱼素材')).toBeTruthy(); - expect(within(characterInfoPanel).getByText('常规参考图 2')).toBeTruthy(); - expect(within(characterInfoPanel).getByText('常规参考.png')).toBeTruthy(); - await waitFor(() => { - expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( - 'editor-project-default', + expect(generateEditorImageMock).toHaveBeenCalledWith( expect.objectContaining({ - layers: expect.arrayContaining([ - expect.objectContaining({ - title: expect.stringMatching(/角色形象/u), - objectKey: - 'generated-character-drafts/editor/character-images/editor-character-1/image.png', - assetObjectId: 'asset-object-editor-character-1', - }), - ]), + kind: 'character', + prompt: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', + model: 'gemini-3.1-flash-image-preview', + aspectRatio: '1:1', + imageSize: '1K', + referenceImageSrcs: [ + 'data:image/png;base64,cm9sZS1zcGVj', + 'data:image/png;base64,YmlnLWZpc2gtcmVm', + ], + projectId: 'editor-project-default', + assetFolderId: 'project', + assetKind: 'character', + generationInputs: expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '角色设定', + value: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', + }), + ]), + }), }), ); }); + + const characterGenerationInput = generateEditorImageMock.mock.calls[0]?.[0]; + expect(characterGenerationInput?.generationInputs).toEqual( + expect.objectContaining({ + fields: expect.arrayContaining([ + expect.objectContaining({ + title: '角色设定', + value: '银发游侠，蓝色披风，弓箭手，适合像素风战棋。', + }), + ]), + references: expect.arrayContaining([ + expect.objectContaining({ + title: '角色规范', + label: '角色规范', + }), + expect.objectContaining({ + title: '常规参考图 1', + label: '大鱼素材', + }), + ]), + }), + ); }); it('removes the active character generation placeholder with Backspace', async () => { @@ -3122,6 +3202,7 @@ describe('ImageCanvasEditorView generation integration', () => { expect.objectContaining({ sourceImageSrc: 'data:image/png;base64,marked-ui-design', screenColor: '#CFEFFF', + segModel: 'birefnet', aspectRatio: '1:1', imageSize: '1K', projectId: 'editor-project-ui-extract', @@ -3138,6 +3219,10 @@ describe('ImageCanvasEditorView generation integration', () => { title: '抠图背景色', value: '浅雾蓝 #CFEFFF', }), + expect.objectContaining({ + title: '抠图模型', + value: '默认 birefnet', + }), ]), references: expect.arrayContaining([ expect.objectContaining({ @@ -3154,45 +3239,6 @@ describe('ImageCanvasEditorView generation integration', () => { ).toBeTruthy(); expect(screen.getByAltText('画布图片：素材 1')).toBeTruthy(); expect(screen.getByText('图集')).toBeTruthy(); - await waitFor(() => { - expect(createEditorProjectResourceMock).toHaveBeenCalledWith( - 'editor-project-ui-extract', - expect.objectContaining({ - sourceType: 'generated', - assetKind: 'icon', - generationInputs: expect.objectContaining({ - fields: expect.arrayContaining([ - expect.objectContaining({ - title: '提取提示词', - value: - '仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用所选单一纯色抠图背景。纯色背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现与背景色相同或相近的描边、底板、投影或反光。', - }), - expect.objectContaining({ - title: '抠图背景色', - value: '浅雾蓝 #CFEFFF', - }), - ]), - }), - }), - ); - }); - await waitFor(() => { - expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( - 'editor-project-ui-extract', - expect.objectContaining({ - layers: expect.arrayContaining([ - expect.objectContaining({ - title: '战斗UI设计图 素材图集', - resourceId: 'resource-editor-project-ui-extract-512', - }), - expect.objectContaining({ - title: '素材 1', - resourceId: 'resource-editor-project-ui-extract-96', - }), - ]), - }), - ); - }); }); it('closes UI design asset extraction when clicking outside the source and panel', async () => { diff --git a/src/components/image-editor/ImageCanvasEditorShellView.test.tsx b/src/components/image-editor/ImageCanvasEditorShellView.test.tsx index f8723c689..b8a2ffcc3 100644 --- a/src/components/image-editor/ImageCanvasEditorShellView.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorShellView.test.tsx @@ -190,6 +190,7 @@ function createStageProps(): ImageCanvasStageViewProps { onUiAssetExtractionToolChange: vi.fn(), onUiAssetExtractionModelChange: vi.fn(), onUiAssetExtractionScreenColorChange: vi.fn(), + onUiAssetExtractionSegModelChange: vi.fn(), onUiAssetExtractionPointerStart: vi.fn(), onUiAssetExtractionPointerMove: vi.fn(), onUiAssetExtractionPointerEnd: vi.fn(), @@ -339,6 +340,7 @@ describe('ImageCanvasEditorShellView', () => { tool: 'rect' as const, model: 'gemini-3.1-flash-image-preview', screenColor: '#CFEFFF', + segModel: 'birefnet', marks: [], references: [], draftMark: null, diff --git a/src/components/image-editor/ImageCanvasEditorTypes.ts b/src/components/image-editor/ImageCanvasEditorTypes.ts index 76f9eafcd..6b668b347 100644 --- a/src/components/image-editor/ImageCanvasEditorTypes.ts +++ b/src/components/image-editor/ImageCanvasEditorTypes.ts @@ -209,6 +209,7 @@ export type GenerateDialogState = { uiDesignSpecReference?: CharacterReferenceImage | null; imageModel?: string; screenColor?: string; + segModel?: string; videoModel?: EditorVideoModel; videoAspectRatio?: EditorVideoAspectRatio; videoResolution?: EditorVideoResolution; diff --git a/src/components/image-editor/ImageCanvasEditorView.tsx b/src/components/image-editor/ImageCanvasEditorView.tsx index 4ba04b07c..abe53dfe7 100644 --- a/src/components/image-editor/ImageCanvasEditorView.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.tsx @@ -1181,6 +1181,7 @@ export function ImageCanvasEditorView() { changeUiAssetExtractionTool, changeUiAssetExtractionModel, changeUiAssetExtractionScreenColor, + changeUiAssetExtractionSegModel, changeQuickEditSelectionTool, startUiAssetExtractionPointer, startQuickEditSelectionPointer, @@ -1913,6 +1914,7 @@ export function ImageCanvasEditorView() { onUiAssetExtractionToolChange: changeUiAssetExtractionTool, onUiAssetExtractionModelChange: changeUiAssetExtractionModel, onUiAssetExtractionScreenColorChange: changeUiAssetExtractionScreenColor, + onUiAssetExtractionSegModelChange: changeUiAssetExtractionSegModel, onUiAssetExtractionPointerStart: startUiAssetExtractionPointer, onUiAssetExtractionPointerMove: moveUiAssetExtractionPointer, onUiAssetExtractionPointerEnd: endUiAssetExtractionPointer, diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts index 5e10176c0..28c65cbc5 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts @@ -502,6 +502,7 @@ describe('ImageCanvasGenerationDialogModel', () => { fields: [ { title: '角色设定', value: '红发骑士' }, { title: '抠图背景色', value: '暖浅桃色 #FFD6C2' }, + { title: '抠图模型', value: '动漫风格 anime-seg' }, ], references: [], }, @@ -516,6 +517,7 @@ describe('ImageCanvasGenerationDialogModel', () => { prompt: '红发骑士', sourceLayerId: 'layer-source', screenColor: '#FFD6C2', + segModel: 'anime-seg', placeholder: { x: 472, y: 140, @@ -584,6 +586,7 @@ describe('ImageCanvasGenerationDialogModel', () => { generatedLayerId: 'layer-source', imageModel: 'gpt-image-2', screenColor: '#E6D8FF', + segModel: 'anime-seg', aspectRatio: '2:3', imageSize: '2K', }, @@ -595,6 +598,7 @@ describe('ImageCanvasGenerationDialogModel', () => { sourceLayerId: 'layer-source', imageModel: 'gpt-image-2', screenColor: '#E6D8FF', + segModel: 'anime-seg', aspectRatio: '2:3', imageSize: '2K', }); diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts index 3e7a96144..d4b807a32 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts @@ -18,6 +18,7 @@ import { CHARACTER_ANIMATION_FRAME_DISPLAY_SIZE, CHARACTER_ANIMATION_FRAME_ORIGINAL_SIZE, createCanvasLayerReference, + DEFAULT_EDITOR_BGFILTER_SEG_MODEL, DEFAULT_IMAGE_MODEL, DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, DEFAULT_PUBLICATION_GAME_INFO, @@ -36,6 +37,7 @@ import { IMAGE_MODEL_GPT_IMAGE_2, inferEditorImageAspectRatio, inferEditorImageSizeLabel, + normalizeEditorBgFilterSegModel, normalizeEditorImageModel, normalizeEditorGenerationBackgroundColor, PUBLICATION_FRAME_ORIGINAL_SIZE, @@ -275,6 +277,7 @@ export function createCharacterGenerationDialogDraft({ characterReferences: [], imageModel: normalizedImageModel, screenColor: DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, + segModel: DEFAULT_EDITOR_BGFILTER_SEG_MODEL, aspectRatio: dimensionDefaults.aspectRatio, imageSize: dimensionDefaults.imageSize, placeholder: { @@ -310,6 +313,7 @@ export function createIconGenerationDialogDraft({ iconDescriptions: [], imageModel: normalizedImageModel, screenColor: DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, + segModel: DEFAULT_EDITOR_BGFILTER_SEG_MODEL, aspectRatio: dimensionDefaults.aspectRatio, imageSize: dimensionDefaults.imageSize, placeholder: { @@ -745,12 +749,16 @@ function restoreSharedImageOptions( const fields = getGenerationInputFieldValues(sourceLayer); const sourceScreenColor = sourceDialog?.screenColor ?? fields.get('抠图背景色'); + const sourceSegModel = sourceDialog?.segModel ?? fields.get('抠图模型'); return { ...draft, imageModel: sourceDialog?.imageModel ?? draft.imageModel, screenColor: normalizeEditorGenerationBackgroundColor( sourceScreenColor ?? draft.screenColor, ), + segModel: normalizeEditorBgFilterSegModel( + sourceSegModel ?? draft.segModel, + ), aspectRatio: sourceDialog?.aspectRatio ?? draft.aspectRatio ?? diff --git a/src/components/image-editor/ImageCanvasGenerationImageOptionsView.test.tsx b/src/components/image-editor/ImageCanvasGenerationImageOptionsView.test.tsx index d057db426..27859db12 100644 --- a/src/components/image-editor/ImageCanvasGenerationImageOptionsView.test.tsx +++ b/src/components/image-editor/ImageCanvasGenerationImageOptionsView.test.tsx @@ -1,4 +1,4 @@ -/* @vitest-environment jsdom */ +/* @vitest-environment jsdom */ import { fireEvent, render, screen, within } from '@testing-library/react'; import { useState } from 'react'; @@ -16,12 +16,14 @@ function ImageOptionsHarness({ initialDialog, includeDimensions = true, includeBackgroundColor = false, + includeSegModel = false, onRememberImageModel = vi.fn(), lockedModel, }: { initialDialog: GenerateDialogState; includeDimensions?: boolean; includeBackgroundColor?: boolean; + includeSegModel?: boolean; onRememberImageModel?: (model: string) => void; lockedModel?: string; }) { @@ -36,6 +38,7 @@ function ImageOptionsHarness({ setGenerateDialog={setDialog} includeDimensions={includeDimensions} includeBackgroundColor={includeBackgroundColor} + includeSegModel={includeSegModel} onRememberImageModel={onRememberImageModel} lockedModel={lockedModel} cost={calculateEditorImageGenerationPrice({ @@ -49,6 +52,7 @@ function ImageOptionsHarness({ {dialog.aspectRatio} {dialog.imageSize} {dialog.screenColor} + {dialog.segModel} {dialog.status} {dialog.errorMessage ?? '-'} @@ -219,6 +223,51 @@ describe('ImageCanvasGenerationImageOptionsView', () => { expect(screen.getByLabelText('当前错误').textContent).toBe('-'); }); + it('updates the BgFilter segmentation model from the model menu', () => { + render( + , + ); + + fireEvent.click( + screen.getByRole('button', { + name: '生成图片抠图模型 默认', + }), + ); + const panel = screen.getByRole('menu', { name: '生成图片抠图模型选项' }); + fireEvent.click( + within(panel).getByRole('button', { + name: '动漫风格', + }), + ); + + expect( + screen.getByRole('menu', { name: '生成图片抠图模型选项' }), + ).toBeTruthy(); + expect( + within(panel) + .getByRole('button', { name: '动漫风格' }) + .getAttribute('aria-pressed'), + ).toBe('true'); + expect(screen.getByLabelText('当前抠图模型').textContent).toBe( + 'anime-seg', + ); + expect(screen.getByLabelText('当前状态').textContent).toBe('idle'); + expect(screen.getByLabelText('当前错误').textContent).toBe('-'); + }); + it('locks the model trigger when a fixed model is provided', () => { const rememberImageModel = vi.fn(); render( diff --git a/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx b/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx index b04eac0b3..aa5e6a608 100644 --- a/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx +++ b/src/components/image-editor/ImageCanvasGenerationImageOptionsView.tsx @@ -1,4 +1,4 @@ -import { Check, ChevronDown, Cpu } from 'lucide-react'; +import { Check, ChevronDown, Cpu, Scissors } from 'lucide-react'; import { type CSSProperties, type Dispatch, @@ -15,11 +15,14 @@ import { PlatformInlineOptionButton } from '../common/PlatformInlineOptionButton import type { GenerateDialogState } from './ImageCanvasEditorTypes'; import { EDITOR_IMAGE_DIMENSION_OPTIONS, + EDITOR_BGFILTER_SEG_MODEL_OPTIONS, EDITOR_IMAGE_MODEL_OPTIONS, EDITOR_GENERATION_BACKGROUND_COLOR_OPTIONS, + getEditorBgFilterSegModelOption, getEditorGenerationBackgroundColorOption, getEditorImageModelDisplayName, IMAGE_MODEL_NANOBANANA2, + normalizeEditorBgFilterSegModel, normalizeEditorGenerationBackgroundColor, normalizeEditorImageModel, QUICK_EDIT_MODEL_OPTIONS, @@ -34,6 +37,7 @@ type ImageCanvasGenerationImageOptionsViewProps = { includeDimensions: boolean; includeModel?: boolean; includeBackgroundColor?: boolean; + includeSegModel?: boolean; onRememberImageModel: (model: string) => void; optionLabelPrefix?: string; dimensionRatioAriaLabelPrefix?: string; @@ -50,7 +54,7 @@ type ImageCanvasGenerationImageOptionsViewProps = { ) => CSSProperties; }; -type OpenPanel = 'dimensions' | 'model' | 'backgroundColor' | null; +type OpenPanel = 'dimensions' | 'model' | 'backgroundColor' | 'segModel' | null; function resetFailedDialogStatus(dialog: GenerateDialogState) { return { @@ -126,6 +130,7 @@ export function ImageCanvasGenerationImageOptionsView({ includeDimensions, includeModel = true, includeBackgroundColor = false, + includeSegModel = false, onRememberImageModel, optionLabelPrefix, dimensionRatioAriaLabelPrefix, @@ -141,6 +146,7 @@ export function ImageCanvasGenerationImageOptionsView({ const [openPanel, setOpenPanel] = useState(null); const dimensionsButtonRef = useRef(null); const backgroundColorButtonRef = useRef(null); + const segModelButtonRef = useRef(null); const modelButtonRef = useRef(null); const isQuickEdit = dialog.mode === 'quick-edit'; const normalizedLockedModel = lockedModel @@ -165,6 +171,7 @@ export function ImageCanvasGenerationImageOptionsView({ const selectedBackgroundColor = getEditorGenerationBackgroundColorOption( dialog.screenColor, ); + const selectedSegModel = getEditorBgFilterSegModelOption(dialog.segModel); const selectedSizeLabel = resolveEditorImageSizeLabel({ aspectRatio: selection.aspectRatio, imageSize: selection.imageSize, @@ -184,6 +191,7 @@ export function ImageCanvasGenerationImageOptionsView({ boundaryRefs: [ dimensionsButtonRef, backgroundColorButtonRef, + segModelButtonRef, modelButtonRef, ], onDismiss: dismissOpenPanel, @@ -246,6 +254,17 @@ export function ImageCanvasGenerationImageOptionsView({ ); }; + const updateSegModel = (segModel: string) => { + setGenerateDialog((currentDialog) => + currentDialog && currentDialog.mode === dialog.mode + ? { + ...resetFailedDialogStatus(currentDialog), + segModel: normalizeEditorBgFilterSegModel(segModel), + } + : currentDialog, + ); + }; + const togglePanel = (panel: Exclude) => { setOpenPanel((currentPanel) => (currentPanel === panel ? null : panel)); }; @@ -400,6 +419,68 @@ export function ImageCanvasGenerationImageOptionsView({ : null} ) : null} + {includeSegModel ? ( +
+ } + onClick={() => togglePanel('segModel')} + > + + + {selectedSegModel.label} + + + {openPanel === 'segModel' + ? renderEditorPortal( + event.stopPropagation()} + > +
+ {EDITOR_BGFILTER_SEG_MODEL_OPTIONS.map((option) => { + const selected = selectedSegModel.value === option.value; + return ( + updateSegModel(option.value)} + > + + {option.label} + + ); + })} +
+ , + ) + : null} +
+ ) : null} {includeModel ? (
{ fields: [ { title: '角色设定', value: '主角骑士' }, { title: '抠图背景色', value: '浅雾蓝 #CFEFFF' }, + { title: '抠图模型', value: '默认 birefnet' }, ], references: [ { @@ -281,6 +282,7 @@ describe('ImageCanvasGenerationModel', () => { fields: [ { title: '素材描述', value: '返回按钮\n设置按钮' }, { title: '抠图背景色', value: '浅雾蓝 #CFEFFF' }, + { title: '抠图模型', value: '默认 birefnet' }, ], references: [ { @@ -322,6 +324,7 @@ describe('ImageCanvasGenerationModel', () => { '仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用所选单一纯色抠图背景。纯色背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现与背景色相同或相近的描边、底板、投影或反光。', }, { title: '抠图背景色', value: '浅雾蓝 #CFEFFF' }, + { title: '抠图模型', value: '默认 birefnet' }, ], references: [ { diff --git a/src/components/image-editor/ImageCanvasGenerationModel.ts b/src/components/image-editor/ImageCanvasGenerationModel.ts index 239099da0..a1b082100 100644 --- a/src/components/image-editor/ImageCanvasGenerationModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationModel.ts @@ -109,6 +109,14 @@ export type EditorGenerationBackgroundColorValue = (typeof EDITOR_GENERATION_BACKGROUND_COLOR_OPTIONS)[number]['value']; export const DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR = EDITOR_GENERATION_BACKGROUND_COLOR_OPTIONS[0].value; +export const EDITOR_BGFILTER_SEG_MODEL_OPTIONS = [ + { label: '默认', value: 'birefnet' }, + { label: '动漫风格', value: 'anime-seg' }, +] as const; +export type EditorBgFilterSegModelValue = + (typeof EDITOR_BGFILTER_SEG_MODEL_OPTIONS)[number]['value']; +export const DEFAULT_EDITOR_BGFILTER_SEG_MODEL = + EDITOR_BGFILTER_SEG_MODEL_OPTIONS[0].value; export const UI_DESIGN_ASSET_EXTRACTION_PROMPT = '仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用所选单一纯色抠图背景。纯色背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现与背景色相同或相近的描边、底板、投影或反光。'; export const QUICK_EDIT_SIZE_PRESETS = [ @@ -591,6 +599,36 @@ export function formatEditorGenerationBackgroundColor( return `${option.label} ${option.value}`; } +export function normalizeEditorBgFilterSegModel( + value: string | null | undefined, +): EditorBgFilterSegModelValue { + const normalizedValue = value?.trim().toLowerCase(); + const normalizedToken = normalizedValue?.match(/(?:anime-seg|birefnet)/u)?.[0]; + const matched = EDITOR_BGFILTER_SEG_MODEL_OPTIONS.find( + (option) => + option.value === normalizedValue || option.value === normalizedToken, + ); + return matched?.value ?? DEFAULT_EDITOR_BGFILTER_SEG_MODEL; +} + +export function getEditorBgFilterSegModelOption( + value: string | null | undefined, +) { + const normalizedValue = normalizeEditorBgFilterSegModel(value); + return ( + EDITOR_BGFILTER_SEG_MODEL_OPTIONS.find( + (option) => option.value === normalizedValue, + ) ?? EDITOR_BGFILTER_SEG_MODEL_OPTIONS[0] + ); +} + +export function formatEditorBgFilterSegModel( + value: string | null | undefined, +) { + const option = getEditorBgFilterSegModelOption(value); + return `${option.label} ${option.value}`; +} + export function buildQuickEditModelOptions(currentModel: string) { void currentModel; const options = [...QUICK_EDIT_MODEL_OPTIONS]; @@ -1117,6 +1155,7 @@ export function buildCharacterGenerationInputs( specReference: CharacterReferenceImage | null | undefined, references: CharacterReferenceImage[] | undefined, screenColor?: string, + segModel?: string, ): CanvasGenerationInputs { return { fields: [ @@ -1125,6 +1164,10 @@ export function buildCharacterGenerationInputs( '抠图背景色', formatEditorGenerationBackgroundColor(screenColor), ), + ...createGenerationInputField( + '抠图模型', + formatEditorBgFilterSegModel(segModel), + ), ], references: [ ...(specReference @@ -1231,6 +1274,7 @@ export function buildIconGenerationInputs( specReference: CharacterReferenceImage, references?: CharacterReferenceImage[], screenColor?: string, + segModel?: string, ): CanvasGenerationInputs { return { fields: [ @@ -1242,6 +1286,10 @@ export function buildIconGenerationInputs( '抠图背景色', formatEditorGenerationBackgroundColor(screenColor), ), + ...createGenerationInputField( + '抠图模型', + formatEditorBgFilterSegModel(segModel), + ), ], references: [ ...createGenerationInputReference('图标规范', specReference), @@ -1256,6 +1304,7 @@ export function buildUiDesignAssetExtractionGenerationInputs( sourceLayer: CanvasLayer, references?: CharacterReferenceImage[], screenColor?: string, + segModel?: string, ): CanvasGenerationInputs { return { fields: [ @@ -1267,6 +1316,10 @@ export function buildUiDesignAssetExtractionGenerationInputs( '抠图背景色', formatEditorGenerationBackgroundColor(screenColor), ), + ...createGenerationInputField( + '抠图模型', + formatEditorBgFilterSegModel(segModel), + ), ], references: [ ...createLayerGenerationInputReference('UI设计图', sourceLayer), diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts index 67f433080..ed35de603 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts @@ -274,6 +274,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { status: 'idle', imageModel: 'gpt-image-2', screenColor: '#FFD6C2', + segModel: 'anime-seg', aspectRatio: '2:3', imageSize: '2K', characterSpecReference: { @@ -303,6 +304,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { kind: 'character', model: 'gpt-image-2', screenColor: '#FFD6C2', + segModel: 'anime-seg', aspectRatio: '2:3', imageSize: '2K', referenceImageSrcs: [ @@ -321,6 +323,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { fields: [ { title: '角色设定', value: '白发骑士' }, { title: '抠图背景色', value: '暖浅桃色 #FFD6C2' }, + { title: '抠图模型', value: '动漫风格 anime-seg' }, ], references: [ { @@ -617,7 +620,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { }); }); - it('returns an icon spritesheet error when descriptions are empty', () => { + it('falls back to default icon descriptions when the draft descriptions are empty', () => { const plan = buildIconSpritesheetGenerationSubmissionPlan({ mode: 'icon', prompt: '', @@ -631,10 +634,15 @@ describe('ImageCanvasGenerationSubmissionModel', () => { iconDescriptions: [' ', '\n'], }); - expect(plan).toEqual({ - ok: false, - errorMessage: '请填写素材描述', - }); + expect(plan.ok).toBe(true); + expect(plan.ok === true ? plan.iconDescriptions : []).toEqual([ + '返回按钮', + '设置按钮', + '下一关按钮', + '提示按钮', + '原图按钮', + '冻结按钮', + ]); }); it('builds icon spritesheet plans with trimmed descriptions and references', () => { @@ -644,6 +652,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { status: 'idle', imageModel: 'gpt-image-2', screenColor: '#7FB3FF', + segModel: 'anime-seg', aspectRatio: '3:2', imageSize: '2K', iconSpecReference: { @@ -672,6 +681,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { iconDescriptions: ['返回按钮', '设置按钮'], model: 'gpt-image-2', screenColor: '#7FB3FF', + segModel: 'anime-seg', aspectRatio: '3:2', imageSize: '2K', }, @@ -679,6 +689,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => { fields: [ { title: '素材描述', value: '返回按钮\n设置按钮' }, { title: '抠图背景色', value: '中度天蓝 #7FB3FF' }, + { title: '抠图模型', value: '动漫风格 anime-seg' }, ], references: [ { diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts index 54af82417..6313da29f 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts @@ -42,6 +42,7 @@ import { IMAGE_MODEL_GPT_IMAGE_2, inferEditorImageAspectRatio, inferEditorImageSizeLabel, + normalizeEditorBgFilterSegModel, normalizeEditorGenerationBackgroundColor, normalizeEditorImageModel, resolveCharacterAnimationSourceImageSrc, @@ -351,6 +352,7 @@ export function buildImageGenerationSubmissionPlan({ const screenColor = normalizeEditorGenerationBackgroundColor( dialog.screenColor, ); + const segModel = normalizeEditorBgFilterSegModel(dialog.segModel); return { kind: 'image', normalizedPrompt, @@ -359,6 +361,7 @@ export function buildImageGenerationSubmissionPlan({ kind: 'character', model: imageModel, screenColor, + segModel, aspectRatio: dialog.aspectRatio ?? '1:1', imageSize: dialog.imageSize ?? '1K', ...(referenceImageSrcs.length ? { referenceImageSrcs } : {}), @@ -371,6 +374,7 @@ export function buildImageGenerationSubmissionPlan({ dialog.characterSpecReference, dialog.characterReferences, screenColor, + segModel, ), }, rememberImageModel: imageModel, @@ -575,11 +579,14 @@ export function buildImageGenerationSubmissionPlan({ export function buildIconSpritesheetGenerationSubmissionPlan( dialog: GenerateDialogState, ): IconSpritesheetGenerationSubmissionPlan { - const iconDescriptions = ( - dialog.prompt.trim() - ? dialog.prompt.split(/[\r\n,，、;；/|]+/u) - : (dialog.iconDescriptions ?? DEFAULT_ICON_DESCRIPTIONS) - ) + const iconDescriptionSource = dialog.prompt.trim() + ? dialog.prompt.split(/[\r\n,，、;；/|]+/u) + : (dialog.iconDescriptions ?? []).some((description) => + description.trim(), + ) + ? (dialog.iconDescriptions ?? []) + : DEFAULT_ICON_DESCRIPTIONS; + const iconDescriptions = iconDescriptionSource .map((description) => description.trim()) .filter(Boolean) .slice(0, ICON_DESCRIPTION_LIMIT); @@ -602,6 +609,7 @@ export function buildIconSpritesheetGenerationSubmissionPlan( const screenColor = normalizeEditorGenerationBackgroundColor( dialog.screenColor, ); + const segModel = normalizeEditorBgFilterSegModel(dialog.segModel); return { ok: true, iconDescriptions, @@ -619,6 +627,7 @@ export function buildIconSpritesheetGenerationSubmissionPlan( iconDescriptions, model: rememberImageModel, screenColor, + segModel, aspectRatio: dialog.aspectRatio ?? '1:1', imageSize: dialog.imageSize ?? '1K', }, @@ -627,6 +636,7 @@ export function buildIconSpritesheetGenerationSubmissionPlan( dialog.iconSpecReference, dialog.generationReferences, screenColor, + segModel, ), rememberImageModel, }; diff --git a/src/components/image-editor/ImageCanvasIconSpritesheetComposerView.tsx b/src/components/image-editor/ImageCanvasIconSpritesheetComposerView.tsx index 33e12de31..842248137 100644 --- a/src/components/image-editor/ImageCanvasIconSpritesheetComposerView.tsx +++ b/src/components/image-editor/ImageCanvasIconSpritesheetComposerView.tsx @@ -240,6 +240,7 @@ export function ImageCanvasIconSpritesheetComposerView({ setGenerateDialog={setGenerateDialog} includeDimensions includeBackgroundColor + includeSegModel onRememberImageModel={onRememberImageModel} optionLabelPrefix="生成图片" cost={calculateEditorIconSpritesheetPrice( diff --git a/src/components/image-editor/ImageCanvasStageView.tsx b/src/components/image-editor/ImageCanvasStageView.tsx index e496ab3e4..72304d713 100644 --- a/src/components/image-editor/ImageCanvasStageView.tsx +++ b/src/components/image-editor/ImageCanvasStageView.tsx @@ -129,6 +129,7 @@ export type ImageCanvasStageViewProps = { onUiAssetExtractionToolChange: (tool: UiAssetExtractionTool | null) => void; onUiAssetExtractionModelChange: (model: string) => void; onUiAssetExtractionScreenColorChange: (screenColor: string) => void; + onUiAssetExtractionSegModelChange: (segModel: string) => void; onUiAssetExtractionPointerStart: (point: { x: number; y: number }) => void; onUiAssetExtractionPointerMove: (point: { x: number; y: number }) => void; onUiAssetExtractionPointerEnd: () => void; @@ -252,6 +253,7 @@ export function ImageCanvasStageView({ onUiAssetExtractionToolChange, onUiAssetExtractionModelChange, onUiAssetExtractionScreenColorChange, + onUiAssetExtractionSegModelChange, onUiAssetExtractionPointerStart, onUiAssetExtractionPointerMove, onUiAssetExtractionPointerEnd, @@ -369,6 +371,7 @@ export function ImageCanvasStageView({ onPointerEnd={onUiAssetExtractionPointerEnd} onModelChange={onUiAssetExtractionModelChange} onScreenColorChange={onUiAssetExtractionScreenColorChange} + onSegModelChange={onUiAssetExtractionSegModelChange} onRequestUpload={onRequestUiAssetExtractionReferenceUpload} onRemoveReference={onRemoveUiAssetExtractionReference} onSubmit={onSubmitUiAssetExtraction} diff --git a/src/components/image-editor/ImageCanvasUiAssetExtractionModel.ts b/src/components/image-editor/ImageCanvasUiAssetExtractionModel.ts index 65c04389f..e5e87a9e4 100644 --- a/src/components/image-editor/ImageCanvasUiAssetExtractionModel.ts +++ b/src/components/image-editor/ImageCanvasUiAssetExtractionModel.ts @@ -31,6 +31,7 @@ export type UiAssetExtractionState = { tool: UiAssetExtractionTool | null; model: string; screenColor: string; + segModel: string; marks: UiAssetExtractionMark[]; references: CharacterReferenceImage[]; draftMark: UiAssetExtractionMark | null; @@ -53,6 +54,7 @@ export function createUiAssetExtractionState( initialTool?: UiAssetExtractionTool | null; model: string; screenColor: string; + segModel: string; }, ): UiAssetExtractionState { return { @@ -60,6 +62,7 @@ export function createUiAssetExtractionState( tool: options.initialTool === undefined ? 'rect' : options.initialTool, model: options.model, screenColor: options.screenColor, + segModel: options.segModel, marks: [], references: [], draftMark: null, diff --git a/src/components/image-editor/ImageCanvasUiAssetExtractionOverlayView.test.tsx b/src/components/image-editor/ImageCanvasUiAssetExtractionOverlayView.test.tsx index 41838b3ef..5881265c5 100644 --- a/src/components/image-editor/ImageCanvasUiAssetExtractionOverlayView.test.tsx +++ b/src/components/image-editor/ImageCanvasUiAssetExtractionOverlayView.test.tsx @@ -6,6 +6,7 @@ import { describe, expect, it, vi } from 'vitest'; import type { CanvasLayer } from './ImageCanvasEditorTypes'; import { + DEFAULT_EDITOR_BGFILTER_SEG_MODEL, DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, IMAGE_MODEL_GPT_IMAGE_2, IMAGE_MODEL_NANOBANANA2, @@ -67,6 +68,7 @@ function createState( tool: 'rect', model: IMAGE_MODEL_NANOBANANA2, screenColor: DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, + segModel: DEFAULT_EDITOR_BGFILTER_SEG_MODEL, marks: [], references: [], draftMark: null, @@ -281,6 +283,48 @@ describe('ImageCanvasUiAssetExtractionOverlayView', () => { expect(onScreenColorChange).toHaveBeenCalledWith('#FFD6C2'); }); + it('changes the UI extraction BgFilter model from the segmentation menu', () => { + const onSegModelChange = vi.fn(); + + render( + , + ); + + fireEvent.click( + screen.getByRole('button', { + name: '提取素材抠图模型 默认', + }), + ); + fireEvent.click( + within( + screen.getByRole('menu', { name: '提取素材抠图模型选项' }), + ).getByRole('button', { name: '动漫风格' }), + ); + + expect(onSegModelChange).toHaveBeenCalledWith('anime-seg'); + }); + it('crops mark previews from the same source coordinates as the red selection', () => { render( void; onModelChange?: (model: string) => void; onScreenColorChange?: (screenColor: string) => void; + onSegModelChange?: (segModel: string) => void; onRequestUpload?: () => void; onRemoveReference?: (referenceId: string) => void; onSubmit?: () => void; @@ -276,12 +280,14 @@ export function ImageCanvasUiAssetExtractionOverlayView({ onPointerEnd, onModelChange = () => {}, onScreenColorChange = () => {}, + onSegModelChange = () => {}, onRequestUpload, onRemoveReference, onSubmit, }: ImageCanvasUiAssetExtractionOverlayViewProps) { const [isModelMenuOpen, setIsModelMenuOpen] = useState(false); const [isScreenColorMenuOpen, setIsScreenColorMenuOpen] = useState(false); + const [isSegModelMenuOpen, setIsSegModelMenuOpen] = useState(false); if (!sourceLayer || !state) { return null; } @@ -299,6 +305,7 @@ export function ImageCanvasUiAssetExtractionOverlayView({ const selectedScreenColor = getEditorGenerationBackgroundColorOption( state.screenColor, ); + const selectedSegModel = getEditorBgFilterSegModelOption(state.segModel); const screenFrame = { left: viewport.x + sourceLayer.x * viewport.scale, top: viewport.y + sourceLayer.y * viewport.scale, @@ -583,6 +590,66 @@ export function ImageCanvasUiAssetExtractionOverlayView({ ) : null}
+
+ } + onClick={() => setIsSegModelMenuOpen((open) => !open)} + > + + + {selectedSegModel.label} + + + {isSegModelMenuOpen ? ( + event.stopPropagation()} + > +
+ {EDITOR_BGFILTER_SEG_MODEL_OPTIONS.map((option) => { + const selected = selectedSegModel.value === option.value; + return ( + + ); + })} +
+ + ) : null} +
{ tool: 'rect', model: 'gemini-3.1-flash-image-preview', screenColor: '#CFEFFF', + segModel: 'birefnet', marks: [ { id: 'quick-edit-selection-mark-1', @@ -794,6 +795,7 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => { tool: 'rect', model: 'gemini-3.1-flash-image-preview', screenColor: '#CFEFFF', + segModel: 'birefnet', marks: [ { id: 'quick-edit-selection-mark-1', @@ -1976,6 +1978,7 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => { sourceImageSrc: 'generated/ui-design.png', model: 'gemini-3.1-flash-image-preview', screenColor: '#CFEFFF', + segModel: 'birefnet', }), ); }); @@ -2091,6 +2094,7 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => { sourceImageSrc: 'data:image/png;base64,marked-ui-design', model: 'gemini-3.1-flash-image-preview', screenColor: '#CFEFFF', + segModel: 'birefnet', aspectRatio: '1:1', imageSize: '2K', generationInputs: expect.objectContaining({ @@ -2104,6 +2108,10 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => { title: '抠图背景色', value: '浅雾蓝 #CFEFFF', }), + expect.objectContaining({ + title: '抠图模型', + value: '默认 birefnet', + }), ]), }), }), @@ -2144,6 +2152,7 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => { expect(extractEditorUiDesignAssetsMock).toHaveBeenCalledWith( expect.objectContaining({ screenColor: '#CFEFFF', + segModel: 'birefnet', referenceImageSrcs: ['generated/ref-style.png'], generationInputs: expect.objectContaining({ references: expect.arrayContaining([ diff --git a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts index 810dd41b2..d77f871c8 100644 --- a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts +++ b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts @@ -66,6 +66,7 @@ import { inferEditorImageAspectRatio, inferEditorImageSizeLabel, isCanvasGenerationDialog, + normalizeEditorBgFilterSegModel, normalizeEditorGenerationBackgroundColor, normalizeEditorImageModel, resolveCharacterAnimationSourceImageSrc, @@ -95,6 +96,7 @@ type UiDesignAssetExtractionOptions = { marks?: UiAssetExtractionMark[]; model?: string; screenColor?: string; + segModel?: string; references?: CharacterReferenceImage[]; suppressAlert?: boolean; }; @@ -754,10 +756,12 @@ export function useImageCanvasGenerationSubmissionWorkflow({ const screenColor = normalizeEditorGenerationBackgroundColor( options.screenColor, ); + const segModel = normalizeEditorBgFilterSegModel(options.segModel); const generationInputs = buildUiDesignAssetExtractionGenerationInputs( sourceLayer, options.references, screenColor, + segModel, ); const referenceImageSrcs = (options.references ?? []) .map((reference) => resolveImageReferenceSubmissionSource(reference)) @@ -780,6 +784,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({ options.model ?? DEFAULT_IMAGE_MODEL, ), screenColor, + segModel, ...(referenceImageSrcs.length ? { referenceImageSrcs } : {}), projectId, generationInputs, diff --git a/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx b/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx index 701c1325b..7868a5042 100644 --- a/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx +++ b/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx @@ -243,7 +243,7 @@ function GenerationWorkflowHarness({ {workflow.uiAssetExtractionState - ? `${workflow.uiAssetExtractionState.sourceLayerId}:${workflow.uiAssetExtractionState.tool ?? 'none'}:${workflow.uiAssetExtractionState.marks.length}:${workflow.uiAssetExtractionState.status}:${workflow.uiAssetExtractionState.model}:${workflow.uiAssetExtractionState.screenColor}` + ? `${workflow.uiAssetExtractionState.sourceLayerId}:${workflow.uiAssetExtractionState.tool ?? 'none'}:${workflow.uiAssetExtractionState.marks.length}:${workflow.uiAssetExtractionState.status}:${workflow.uiAssetExtractionState.model}:${workflow.uiAssetExtractionState.screenColor}:${workflow.uiAssetExtractionState.segModel}` : '-'} @@ -1757,7 +1757,7 @@ describe('useImageCanvasGenerationWorkflow', () => { fireEvent.click(screen.getByRole('button', { name: '打开UI素材框选' })); expect(screen.getByTestId('ui-extraction').textContent).toBe( - 'layer-source:rect:0:idle:gemini-3.1-flash-image-preview:#CFEFFF', + 'layer-source:rect:0:idle:gemini-3.1-flash-image-preview:#CFEFFF:birefnet', ); expect(screen.getByTestId('ui-extraction-source').textContent).toBe( 'layer-source', @@ -1770,7 +1770,7 @@ describe('useImageCanvasGenerationWorkflow', () => { fireEvent.click(screen.getByRole('button', { name: '框选UI素材' })); expect(screen.getByTestId('ui-extraction').textContent).toBe( - 'layer-source:ellipse:1:idle:gpt-image-2:#CFEFFF', + 'layer-source:ellipse:1:idle:gpt-image-2:#CFEFFF:birefnet', ); }); diff --git a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts index f665f4c39..fab986162 100644 --- a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts +++ b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts @@ -69,6 +69,7 @@ import { calculateCharacterAnimationPrice, CHARACTER_ANIMATION_DURATION_OPTIONS, CHARACTER_ANIMATION_MODEL, + DEFAULT_EDITOR_BGFILTER_SEG_MODEL, DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, DEFAULT_ICON_DESCRIPTIONS, DEFAULT_IMAGE_MODEL, @@ -76,6 +77,7 @@ import { ICON_DESCRIPTION_LIMIT, IMAGE_MODEL_GPT_IMAGE_2, isCanvasGenerationDialog, + normalizeEditorBgFilterSegModel, normalizeEditorGenerationBackgroundColor, normalizeEditorImageModel, resizeGenerationPlaceholderToImageSelection, @@ -1047,6 +1049,7 @@ export function useImageCanvasGenerationWorkflow({ initialTool: null, model: DEFAULT_IMAGE_MODEL, screenColor: DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, + segModel: DEFAULT_EDITOR_BGFILTER_SEG_MODEL, }); quickEditSelectionStateRef.current = nextSelectionState; setQuickEditSelectionState(nextSelectionState); @@ -1733,6 +1736,7 @@ export function useImageCanvasGenerationWorkflow({ initialTool: 'rect', model: DEFAULT_IMAGE_MODEL, screenColor: DEFAULT_EDITOR_GENERATION_BACKGROUND_COLOR, + segModel: DEFAULT_EDITOR_BGFILTER_SEG_MODEL, }), ); setViewport(resolveQuickEditFocusViewport({ sourceLayer, canvasSize })); @@ -1814,6 +1818,24 @@ export function useImageCanvasGenerationWorkflow({ [], ); + const changeUiAssetExtractionSegModel = useCallback((segModel: string) => { + const normalizedSegModel = normalizeEditorBgFilterSegModel(segModel); + setUiAssetExtractionState((currentState) => + currentState + ? { + ...currentState, + segModel: normalizedSegModel, + status: + currentState.status === 'failed' ? 'idle' : currentState.status, + errorMessage: + currentState.status === 'failed' + ? undefined + : currentState.errorMessage, + } + : currentState, + ); + }, []); + const appendUiAssetExtractionReferences = useCallback( (references: CharacterReferenceImage[]) => { if (!references.length) { @@ -2068,6 +2090,7 @@ export function useImageCanvasGenerationWorkflow({ marks: currentState.marks, model: currentState.model, screenColor: currentState.screenColor, + segModel: currentState.segModel, references: currentState.references, suppressAlert: true, }); @@ -2198,6 +2221,7 @@ export function useImageCanvasGenerationWorkflow({ changeUiAssetExtractionTool, changeUiAssetExtractionModel, changeUiAssetExtractionScreenColor, + changeUiAssetExtractionSegModel, appendUiAssetExtractionReferences, removeUiAssetExtractionReference, changeQuickEditSelectionTool, @@ -2303,6 +2327,7 @@ export function useImageCanvasGenerationWorkflow({ changeUiAssetExtractionTool, changeUiAssetExtractionModel, changeUiAssetExtractionScreenColor, + changeUiAssetExtractionSegModel, appendUiAssetExtractionReferences, removeUiAssetExtractionReference, changeQuickEditSelectionTool, diff --git a/src/index.css b/src/index.css index c68a8b032..cb7dcf18c 100644 --- a/src/index.css +++ b/src/index.css @@ -7021,9 +7021,10 @@ html[data-mobile-keyboard-open='true'] .platform-mobile-bottom-dock { .image-canvas-editor__generation-composer-footer { grid-column: 1 / -1; display: grid; - grid-template-columns: auto auto minmax(0, 1fr) auto auto; + grid-template-columns: max-content max-content max-content max-content max-content; align-items: center; - gap: 0.46rem; + gap: 0.10rem; + justify-content: start; position: relative; border-top: 1px solid rgba(15, 23, 42, 0.06); padding-top: 0.44rem; @@ -7271,6 +7272,7 @@ button.image-canvas-editor__reference-chip:disabled { min-height: 2.25rem; height: 2.25rem; width: 100%; + min-width: 0; border: 1px solid transparent; border-radius: 999px; background: transparent; @@ -7279,6 +7281,14 @@ button.image-canvas-editor__reference-chip:disabled { box-shadow: none; } +.image-canvas-editor__option-cluster .platform-inline-option-button__label { + display: inline-flex; + min-width: 0; + max-width: 100%; + align-items: center; + overflow: hidden; +} + .image-canvas-editor__option-cluster:hover:not(:disabled), .image-canvas-editor__option-cluster[aria-expanded='true'] { border-color: rgba(15, 23, 42, 0.08); @@ -7306,15 +7316,23 @@ button.image-canvas-editor__reference-chip:disabled { } .image-canvas-editor__option-cluster--model { - max-width: min(15.5rem, 32vw); + max-width: 8.5rem; justify-self: end; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.image-canvas-editor__option-cluster--seg-model { + max-width: 6.5rem; + justify-self: start; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + .image-canvas-editor__option-cluster--background-color { - max-width: min(16.5rem, 34vw); + max-width: 10.5rem; justify-self: start; overflow: hidden; text-overflow: ellipsis; @@ -7329,16 +7347,25 @@ button.image-canvas-editor__reference-chip:disabled { .image-canvas-editor__option-popover-anchor--dimensions { grid-column: 1; justify-self: start; + width: max-content; } .image-canvas-editor__option-popover-anchor--model { grid-column: 4; justify-self: end; + width: 8.5rem; +} + +.image-canvas-editor__option-popover-anchor--seg-model { + grid-column: 3; + justify-self: start; + width: 6.5rem; } .image-canvas-editor__option-popover-anchor--background-color { grid-column: 2; justify-self: start; + width: 10.5rem; } .image-canvas-editor__model-trigger-label { @@ -7837,9 +7864,9 @@ button.image-canvas-editor__reference-chip:disabled { z-index: 13; display: grid; grid-template-columns: minmax(0, 1fr); - grid-template-rows: auto minmax(5.8rem, auto) auto auto; - width: min(34rem, calc(100% - 1.5rem)); - min-height: 11.2rem; + grid-template-rows: auto minmax(5.1rem, auto) auto auto; + width: min(42rem, calc(100% - 1.5rem)); + min-height: 10.4rem; gap: 0.5rem; border: 1px solid #cfd6df; border-radius: 1.35rem; @@ -7852,7 +7879,7 @@ button.image-canvas-editor__reference-chip:disabled { .image-canvas-editor__character-composer .image-canvas-editor__generation-prompt { - min-height: 5.8rem; + min-height: 5.1rem; padding-top: 0.42rem; } @@ -7924,9 +7951,9 @@ button.image-canvas-editor__reference-chip:disabled { z-index: 13; display: grid; grid-template-columns: minmax(0, 1fr); - grid-template-rows: auto minmax(5.8rem, auto) auto auto; - width: min(34rem, calc(100% - 1.5rem)); - min-height: 11.2rem; + grid-template-rows: auto minmax(5.1rem, auto) auto auto; + width: min(42rem, calc(100% - 1.5rem)); + min-height: 10.4rem; gap: 0.5rem; border: 1px solid #cfd6df; border-radius: 1.35rem; @@ -7938,7 +7965,7 @@ button.image-canvas-editor__reference-chip:disabled { } .image-canvas-editor__icon-composer .image-canvas-editor__generation-prompt { - min-height: 5.8rem; + min-height: 5.1rem; padding-top: 0.42rem; } @@ -8610,9 +8637,17 @@ button.image-canvas-editor__reference-chip:disabled { width: min(15.5rem, 42vw); } +.image-canvas-editor__ui-extraction-seg-model { + grid-column: 3; + justify-self: start; + width: min(13rem, 32vw); +} + .image-canvas-editor__ui-extraction-screen-color .image-canvas-editor__option-cluster, -.image-canvas-editor__ui-extraction-model .image-canvas-editor__option-cluster { +.image-canvas-editor__ui-extraction-model .image-canvas-editor__option-cluster, +.image-canvas-editor__ui-extraction-seg-model + .image-canvas-editor__option-cluster { width: 100%; } @@ -9184,6 +9219,7 @@ button.image-canvas-editor__reference-chip:disabled { .image-canvas-editor__option-cluster--dimensions, .image-canvas-editor__option-cluster--background-color, + .image-canvas-editor__option-cluster--seg-model, .image-canvas-editor__option-cluster--model, .image-canvas-editor__readonly-generation-option, .image-canvas-editor__generation-submit, diff --git a/src/services/image-editor/editorProjectClient.test.ts b/src/services/image-editor/editorProjectClient.test.ts index 85135b0e6..c1cc56fae 100644 --- a/src/services/image-editor/editorProjectClient.test.ts +++ b/src/services/image-editor/editorProjectClient.test.ts @@ -700,6 +700,7 @@ describe('editorProjectClient', () => { kind: 'character', model: 'gpt-image-2', screenColor: '#FFD6C2', + segModel: 'anime-seg', aspectRatio: '2:3', imageSize: '1K', projectId: 'editor-project-1', @@ -723,6 +724,7 @@ describe('editorProjectClient', () => { kind: 'character', model: 'gpt-image-2', screenColor: '#FFD6C2', + segModel: 'anime-seg', aspectRatio: '2:3', imageSize: '1K', projectId: 'editor-project-1', @@ -865,6 +867,7 @@ describe('editorProjectClient', () => { iconDescriptions: ['返回按钮'], model: 'gpt-image-2', screenColor: '#E6D8FF', + segModel: 'anime-seg', aspectRatio: '1:1', imageSize: '2K', projectId: 'editor-project-1', @@ -885,6 +888,7 @@ describe('editorProjectClient', () => { iconDescriptions: ['返回按钮'], model: 'gpt-image-2', screenColor: '#E6D8FF', + segModel: 'anime-seg', aspectRatio: '1:1', imageSize: '2K', projectId: 'editor-project-1', @@ -929,6 +933,7 @@ describe('editorProjectClient', () => { sourceImageSrc: 'data:image/png;base64,ui-design', model: 'gpt-image-2', screenColor: '#7FB3FF', + segModel: 'anime-seg', referenceImageSrcs: ['data:image/png;base64,ui-ref'], projectId: 'editor-project-1', assetFolderId: 'project', @@ -951,6 +956,7 @@ describe('editorProjectClient', () => { sourceImageSrc: 'data:image/png;base64,ui-design', model: 'gpt-image-2', screenColor: '#7FB3FF', + segModel: 'anime-seg', referenceImageSrcs: ['data:image/png;base64,ui-ref'], projectId: 'editor-project-1', generationInputs: { diff --git a/src/services/image-editor/editorProjectClient.ts b/src/services/image-editor/editorProjectClient.ts index 457be602d..622510684 100644 --- a/src/services/image-editor/editorProjectClient.ts +++ b/src/services/image-editor/editorProjectClient.ts @@ -134,6 +134,7 @@ export type EditorImageGenerationInput = { | 'publication-material'; model?: string; screenColor?: string; + segModel?: string; aspectRatio?: string; imageSize?: string; referenceImageSrcs?: string[]; @@ -165,6 +166,7 @@ export type EditorIconSpritesheetGenerationInput = { iconDescriptions: string[]; model?: string; screenColor?: string; + segModel?: string; aspectRatio?: string; imageSize?: string; projectId?: string | null; @@ -177,6 +179,7 @@ export type EditorUiDesignAssetExtractionInput = { sourceImageSrc: string; model?: string; screenColor?: string; + segModel?: string; referenceImageSrcs?: string[]; aspectRatio?: string; imageSize?: string; @@ -803,6 +806,7 @@ export async function generateEditorImage(input: EditorImageGenerationInput) { ...(input.kind ? { kind: input.kind } : {}), ...(input.model ? { model: input.model } : {}), ...(input.screenColor ? { screenColor: input.screenColor } : {}), + ...(input.segModel ? { segModel: input.segModel } : {}), ...(input.aspectRatio ? { aspectRatio: input.aspectRatio } : {}), ...(input.imageSize ? { imageSize: input.imageSize } : {}), ...(input.referenceImageSrcs?.length @@ -843,6 +847,7 @@ export async function generateEditorIconSpritesheet( iconDescriptions: input.iconDescriptions, model: input.model?.trim() || EDITOR_IMAGE_MODEL_NANOBANANA2, ...(input.screenColor ? { screenColor: input.screenColor } : {}), + ...(input.segModel ? { segModel: input.segModel } : {}), ...(input.aspectRatio ? { aspectRatio: input.aspectRatio } : {}), ...(input.imageSize ? { imageSize: input.imageSize } : {}), ...(input.projectId ? { projectId: input.projectId } : {}), @@ -871,6 +876,7 @@ export async function extractEditorUiDesignAssets( sourceImageSrc: input.sourceImageSrc, ...(input.model ? { model: input.model } : {}), ...(input.screenColor ? { screenColor: input.screenColor } : {}), + ...(input.segModel ? { segModel: input.segModel } : {}), ...(input.referenceImageSrcs?.length ? { referenceImageSrcs: input.referenceImageSrcs } : {}),