完善画布任务与标准绿幕抠除链路
统一编辑器标准绿幕提示词和本地绿幕透明化后处理 生成标准绿幕资产前保存源图到 OSS 并下沉字节级抠绿到 platform-image 将手动去背景接入外部生成队列与画布任务侧栏 修复任务侧栏分页、滚动、计时、焦点与完成信息展示 补充右键菜单关闭和素材复制下载交互 同步编辑器相关文档和项目记忆
This commit is contained in:
@@ -16,11 +16,11 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-06-29 图片画布抠图走远端 BiRefNet BFF
|
||||
## 2026-06-29 图片画布手动抠图走远端 BiRefNet BFF
|
||||
|
||||
- 背景：前端 `chromaKey` 和后端绿幕轻量去背对复杂人物、UI 图标边缘和非纯色背景质量不足，且远端 image host 已部署 BiRefNet 服务，需要让画布默认抠图走高质量模型，同时避免把服务令牌暴露到浏览器。
|
||||
- 决策：画布手动“去除背景”默认调用登录态同源 BFF `POST /api/editor/images/background-removals`；角色形象生成、图标 spritesheet 生成、UI 设计图素材提取生成后的透明化和角色动作抽帧后的序列帧透明化也默认走同一套 BiRefNet 上游，不再静默回退到本地绿幕抠图。api-server 解析当前图片或生成结果后代理到 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL/remove-background`，默认指向 `http://58.87.105.82/remove-background`，可选 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 只在服务端注入。api-server 对上游结果做字节和尺寸上限保护，并先落 OSS / asset object 再返回给前端。
|
||||
- 影响范围：api-server 编辑器图片接口、图片画布手动去背景、角色形象生成、图标素材生成、UI 素材提取、角色动作生成、画布右上角任务侧栏、图片画布 MVP 技术文档。
|
||||
- 背景：用户手动“去除背景”面对任意图片，前端 `chromaKey` 和标准绿幕后处理不适合复杂人物、自然背景或非纯色背景；远端 image host 已部署 BiRefNet 服务，需要让手动抠图走高质量模型，同时避免把服务令牌暴露到浏览器。
|
||||
- 决策：画布手动“去除背景”默认调用登录态同源 BFF `POST /api/editor/images/background-removals`。api-server 解析当前图片后代理到 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL/remove-background`，默认指向 `http://58.87.105.82/remove-background`，可选 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 只在服务端注入。api-server 对上游结果做字节和尺寸上限保护，并先落 OSS / asset object 再返回给前端。编辑器自己生成的标准绿幕资产不属于该决策，见 2026-06-30 绿幕契约收口。
|
||||
- 影响范围：api-server 编辑器图片接口、图片画布手动去背景、画布右上角任务侧栏、图片画布 MVP 技术文档。
|
||||
- 验证方式：运行 `cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server config::tests::from_env_reads_editor_background_removal_settings --manifest-path server-rs/Cargo.toml`、`npm run typecheck`、定向画布 workflow 测试、`npm run check:encoding` 和 `git diff --check`。
|
||||
- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
@@ -148,11 +148,18 @@
|
||||
## 2026-06-18 图片画布 UI 设计图提取素材保留图集
|
||||
|
||||
- 背景：UI 设计图需要从成图中继续抽取可复用独立素材；原图标素材生成只把拆分后的图标放入画布，spritesheet 原图没有保留，后续追溯和二次切图不方便。
|
||||
- 决策：`assetKind="ui-design"` 图层浮动工具栏新增 `提取素材`，点击后先进入红框素材框选编辑态，默认矩形框选，并支持椭圆框选和画笔自由框选。至少存在一个框选区域后才能提交；前端把红色轮廓绘入原 UI 设计图并将合成图作为 `/api/editor/ui-designs/assets/extractions` 的参考图。后端固定 `gpt-image-2` 和提示词 `仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。`，返回结构复用图标 spritesheet 响应。UI 提取把 spritesheet 图集作为 `assetKind="icon-spritesheet"` 图层放到画布，再放拆分后的 `assetKind="icon"` 素材；图标素材生成只保留扣绿后的 spritesheet 图集，不再额外铺独立图标。
|
||||
- 决策：`assetKind="ui-design"` 图层浮动工具栏新增 `提取素材`，点击后先进入红框素材框选编辑态，默认矩形框选，并支持椭圆框选和画笔自由框选。至少存在一个框选区域后才能提交；前端把红色轮廓绘入原 UI 设计图并将合成图作为 `/api/editor/ui-designs/assets/extractions` 的参考图。后端固定 `gpt-image-2` 和提示词 `仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。`，返回结构复用图标 spritesheet 响应。UI 提取把 spritesheet 图集作为 `assetKind="icon-spritesheet"` 图层放到画布，再放拆分后的 `assetKind="icon"` 素材；图标素材生成只保留扣绿后的 spritesheet 图集，不再额外铺独立图标。
|
||||
- 影响范围：图片画布浮动工具栏、编辑器图片生成 BFF、`platform-image` 图集连通域拆分、画布图层类型和编辑器文档。
|
||||
- 验证方式：运行图片画布工具栏 / 图集落层 / 生成提交相关前端测试，`cargo test -p platform-image generated_asset_sheets --manifest-path server-rs/Cargo.toml`，以及 `cargo test -p api-server editor_ui_design_asset_extraction_prompt_is_fixed --manifest-path server-rs/Cargo.toml`。
|
||||
- 关联文档：`docs/【编辑器】画板UI设计图生成入口设计-2026-06-17.md`、`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`。
|
||||
|
||||
## 2026-06-30 图片画布标准绿幕契约收口
|
||||
|
||||
- 背景：角色形象、图标 spritesheet、UI 提取 spritesheet 和角色动作帧都要求模型生成标准绿幕，但提示词片段和后处理入口分散在多个模块中，容易把标准绿幕资产误接到远端 BiRefNet。
|
||||
- 决策：编辑器标准绿幕提示词和本地确定性绿幕透明化统一收口到 `server-rs/crates/api-server/src/editor_green_screen.rs`。手动 `POST /api/editor/images/background-removals` 继续面向用户任意图片并走 BiRefNet；编辑器自己生成的标准绿幕资产统一复用 `platform-image::generated_asset_sheets` 的本地透明化能力，不再依赖 BiRefNet。角色图、图标 spritesheet、UI 提取 spritesheet 和角色动作抽帧源图必须在绿幕透明化前先保存一份带绿幕原图到 OSS，便于追溯和重处理。
|
||||
- 影响范围：`editor_project.rs` 的角色图 / 图标图集 / UI 提取图集、`character_animation_assets.rs` 的编辑器角色动作帧、编辑器绿幕相关文档。
|
||||
- 验证方式：运行 `cargo test -p api-server --manifest-path server-rs/Cargo.toml prompt`，并单独按需过滤 `editor_canvas_green_screen_generation_uses_local_postprocess`、`editor_character_animation_frames_use_local_green_screen_postprocess`；同时运行 `npm run check:encoding` 和 `git diff --check`。
|
||||
|
||||
## 2026-06-18 `/creation` 独立为陶泥儿创作工具主页
|
||||
|
||||
- 背景：图片画布项目已经成为独立项目资产，旧“创作”站内 Tab 和一级“草稿”入口不能清晰表达桌面端创作工具主页与项目管理入口。
|
||||
@@ -2935,7 +2942,8 @@
|
||||
## 2026-06-22 图片画布外部生成统一改走 worker 队列
|
||||
|
||||
- 背景：图片画布的图片、改图、图标素材、UI 素材提取、角色动作、视频和音频生成都可能长时间等待外部 provider；如果继续由 HTTP handler 同步执行，生产只能扩 API 进程，不能独立扩生成吞吐。
|
||||
- 决策：`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下，画板所有外部 provider 生成入口统一入 `external_generation_job`，job kind 使用 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation`。worker 成功后由后端写 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`；前端只轮询 BFF job 状态并重新读取项目快照，不从队列 payload 或本地临时状态重建完成图层。
|
||||
- 决策：`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下，画板所有外部 provider 生成入口统一入 `external_generation_job`，job kind 使用 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation`。worker 成功后由后端写 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`；前端只轮询 BFF job 状态并重新读取项目快照，不从队列 payload 或本地临时状态重建完成图层。
|
||||
- 2026-06-29 补充：手动点击图层“去除背景”也属于图片画布外部 provider 任务，`/api/editor/images/background-removals` 在 queue 模式只入队 `editor_background_removal`，worker 完成后用新 resource 原地替换目标 layer。任务列表只展示服务器 `external_generation_job` 返回的任务，禁止再用前端 local task 伪造抠图进度。
|
||||
- 补充：带 `dialogId` 的 `canvasCompletion` 必须读取后端当前 layout 中的最新 generation dialog placeholder；等待期间用户移动占位时，结果层要跟随最新占位。无 dialog 的重绘 / UI 素材提取等入口使用明确的右侧完成占位；生成器已删除时不把结果重新塞回画布。
|
||||
- 影响范围：`server-rs/crates/api-server/src/editor_generation_queue.rs`、`server-rs/crates/api-server/src/external_generation_worker.rs`、`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`src/services/image-editor/editorProjectClient.ts`。
|
||||
- 验证方式：`cargo test -p api-server external_generation_worker --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_canvas_generation --manifest-path server-rs/Cargo.toml`、`cargo test -p shared-contracts --manifest-path server-rs/Cargo.toml`、`npm run test -- src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -246,7 +246,7 @@ npm run check:server-rs-ddd
|
||||
|
||||
- Rust 结构体：`ExternalGenerationJob`
|
||||
- 源码：`server-rs/crates/spacetime-module/src/external_generation.rs`
|
||||
- 用途：外部生成 worker 的持久任务队列和用户可见生成任务列表；`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时，`api-server` HTTP 角色只入队，`external-generation-worker` 角色通过 claim lease 领取、续租、执行，并用 `lease_token` 栅栏回写完成 / 失败。队列行同时保存 `price_mud_points`、`refund_ledger_id` 和 `notification_acknowledged_at`，BFF 通过 `GET /api/runtime/external-generation/jobs` 返回当前账号的正式生成任务列表、价格、状态和未确认终态数量；前端只能展示该后端事实，完成 / 失败提示展示后后台调用 `POST /api/runtime/external-generation/jobs/acknowledge` 由后端写确认时间，关闭按钮只收起本地弹窗，未确认终态任务会在下次登录后再次集中弹出。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity，避免过期 worker 写 session / work profile；图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表，worker 成功后经 `api-server` facade 写入 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`，前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行，三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回，半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job；业务失败态写回成功后才能 fail job，失败态未写回时保留租约等待后续重领。
|
||||
- 用途：外部生成 worker 的持久任务队列和用户可见生成任务列表；`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时，`api-server` HTTP 角色只入队，`external-generation-worker` 角色通过 claim lease 领取、续租、执行，并用 `lease_token` 栅栏回写完成 / 失败。队列行同时保存 `price_mud_points`、`refund_ledger_id` 和 `notification_acknowledged_at`，BFF 通过 `GET /api/runtime/external-generation/jobs` 返回当前账号的正式生成任务列表、价格、状态和未确认终态数量；前端只能展示该后端事实，完成 / 失败提示展示后后台调用 `POST /api/runtime/external-generation/jobs/acknowledge` 由后端写确认时间，关闭按钮只收起本地弹窗，未确认终态任务会在下次登录后再次集中弹出。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity，避免过期 worker 写 session / work profile；图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表，worker 成功后经 `api-server` facade 写入 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`，前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行，三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回，半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job；业务失败态写回成功后才能 fail job，失败态未写回时保留租约等待后续重领。
|
||||
|
||||
### `external_generation_job_event`
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -57,10 +57,10 @@
|
||||
- 后端固定使用 `gpt-image-2` 图片编辑链路，并固定提示词：
|
||||
|
||||
```text
|
||||
仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。
|
||||
仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。
|
||||
```
|
||||
|
||||
- 后端收到 spritesheet 后复用图标素材的连通域拆分能力；未知素材数量时按从上到下、从左到右自动命名为 `素材 1`、`素材 2`。
|
||||
- 后端收到 spritesheet 后先把带绿幕源图写入 OSS，再走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的统一绿幕透明化方法，并复用图标素材的连通域拆分能力；未知素材数量时按从上到下、从左到右自动命名为 `素材 1`、`素材 2`。
|
||||
- 前端先把 spritesheet 原图作为 `assetKind: "icon-spritesheet"` 图集图层放在 UI 设计图右侧，再把拆分出的独立素材作为 `assetKind: "icon"` 图标图层继续放到画布。
|
||||
|
||||
## 验收点
|
||||
|
||||
@@ -50,14 +50,14 @@
|
||||
- Prompt 固定为：
|
||||
|
||||
```text
|
||||
参考图1的图标规范，单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕背景方便扣除背景，禁止出现文字，保证每个图标素材的所有内容区域是完全连通的。按照以下的素材的顺序从上到下从左到右依次生成并整理成一张spritesheet：
|
||||
参考图1的图标规范，背景必须是单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，且平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光；禁止出现文字，保证每个图标素材的所有内容区域是完全连通的。按照以下的素材的顺序从上到下从左到右依次生成并整理成一张spritesheet：
|
||||
|
||||
<素材描述按中文顿号拼接>
|
||||
```
|
||||
|
||||
## 去背与保存
|
||||
|
||||
- 后端收到 spritesheet 后先复用 `platform-image::generated_asset_sheets` 的绿幕去背能力。
|
||||
- 后端收到 spritesheet 后先把带绿幕源图写入 OSS，再走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的统一绿幕透明化方法，底层复用 `platform-image::generated_asset_sheets`。
|
||||
- 去背后的整张 spritesheet 统一编码为透明 PNG，并作为唯一图标素材产物持久化。
|
||||
- 响应保留 `iconImageSrcs` 字段用于兼容旧客户端，但图标素材生成固定返回空数组；UI 设计图提取素材仍可复用该响应结构返回切片素材。
|
||||
|
||||
|
||||
@@ -60,11 +60,11 @@
|
||||
- `kind = "character"` 时，后端不直接把前端文本当完整生图提示词，而是把文本作为 `角色设定` 填入固定提示词骨架：
|
||||
|
||||
```text
|
||||
基于图1的角色美术视觉规范指导生成游戏角色形象图。画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，只作为抠像底色，禁止生成美术视觉规范、出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。
|
||||
基于图1的角色美术视觉规范指导生成游戏角色形象图。画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，只作为抠像底色；绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具；角色主体不得带绿色描边、绿色投影或绿色反光；禁止生成美术视觉规范、出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。
|
||||
角色设定：<用户输入的角色设定>
|
||||
```
|
||||
|
||||
- 角色图生成完成后，编辑器后端必须走通用抠图方法：复用 `platform-image::generated_asset_sheets` 的绿幕 / 近白背景去背能力，并开启内部绿幕 / 近白镂空检测。角色图 prompt 固定要求标准 `#00FF00 / RGB(0,255,0)` 绿幕；后端路径仍兼容生成模型把标准绿幕压成暗绿 / 灰绿背景的情况，但这类宽松识别只用于从画布边缘连通扩散出的背景，不作为全图断开绿色区域删除依据。该流程不再调用 RPG / 资产工坊的角色主图专用 `character_visual_assets` 后处理；输出仍统一为透明背景 PNG，随后写入 OSS 私有对象并确认 `asset_object`。接口回包仍返回透明 PNG Data URL 供画板立即显示，同时返回 `objectKey` / `assetObjectId`，前端创建图层和画板资源记录时必须保存这两个字段。
|
||||
- 角色图生成完成后，编辑器后端必须先把带绿幕源图写入 OSS，再走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的统一绿幕透明化方法：复用 `platform-image::generated_asset_sheets` 的绿幕 / 近白背景去背能力，并开启内部绿幕 / 近白镂空检测。角色图 prompt 固定要求标准 `#00FF00 / RGB(0,255,0)` 绿幕；后端路径仍兼容生成模型把标准绿幕压成暗绿 / 灰绿背景的情况，但这类宽松识别只用于从画布边缘连通扩散出的背景，不作为全图断开绿色区域删除依据。该流程不再调用 RPG / 资产工坊的角色主图专用 `character_visual_assets` 后处理，也不调用远端 BiRefNet；输出仍统一为透明背景 PNG，随后写入 OSS 私有对象并确认 `asset_object`。接口回包仍返回透明 PNG Data URL 供画板立即显示，同时返回 `objectKey` / `assetObjectId`，前端创建图层和画板资源记录时必须保存这两个字段。
|
||||
- 对 `assetKind: "character"` 的角色图层执行 `重绘` 时，前端仍使用原图作为参考图，但请求 `kind` 必须传 `character`，让后端继续套用上述角色提示词限定、角色图后处理和角色资产持久化；普通图片图层重绘仍保持 `kind: "quick-edit"`。
|
||||
|
||||
## 生成规范参考图
|
||||
@@ -152,7 +152,7 @@
|
||||
- 后端 prompt 使用以下固定骨架，并把面板输入追加到 `动作描述：` 后：
|
||||
|
||||
```text
|
||||
生成游戏角色动画，参考图作为首帧和尾帧，画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，只作为抠像底色，禁止出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。
|
||||
生成游戏角色动画，参考图作为首帧和尾帧，画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，只作为抠像底色；绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具；角色主体不得带绿色描边、绿色投影或绿色反光；禁止出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。
|
||||
动作描述：
|
||||
<用户输入的动画描述>
|
||||
```
|
||||
@@ -161,7 +161,7 @@
|
||||
|
||||
- 视频生成完成后，后端按面板选择抽取对应帧数：`32`、`40` 或 `48`。
|
||||
- 抽帧采样必须按目标帧数预留视频尾部安全步长，例如 `32帧·4秒` 最后一帧采 `3.875s`，避免 FFmpeg 在尾点附近返回成功但输出 `0` 帧。
|
||||
- 每帧必须执行绿幕去背，输出透明背景 PNG。
|
||||
- 每帧必须先把带绿幕源图写入 OSS，再执行 `editor_green_screen` 统一绿幕去背，输出透明背景 PNG。
|
||||
- 抽帧结果写入 OSS，并返回帧路径、帧尺寸、帧数、fps、预览视频路径、模型、价格和实际 prompt。
|
||||
- 画板前端回填角色动作结果时，必须以 `frames[0].imageSrc` 创建 `mediaType: "image-sequence"`、`assetKind: "character-animation"` 图层，并把完整 `frames` 保存为图层 `imageSequenceFrames`；`previewVideoPath` 只保留为上游预览视频来源，不作为画布主媒体。
|
||||
- 角色动作图层在画布中使用序列帧播放器循环展示透明 PNG 帧；刷新恢复时必须继续读取 `imageSequenceFrames`，不能回退到 `<video>` 预览。
|
||||
|
||||
@@ -60,10 +60,14 @@ use crate::{
|
||||
EditorGenerationQueuedResponse, editor_generation_queue_state,
|
||||
editor_generation_source_entity_id, enqueue_editor_generation_job,
|
||||
},
|
||||
editor_green_screen::{
|
||||
editor_green_screen_character_prompt_clause,
|
||||
remove_editor_generated_green_screen_background,
|
||||
},
|
||||
editor_project::{
|
||||
EditorCanvasGeneratedLayerInput, PersistEditorGeneratedAssetRequest,
|
||||
build_editor_canvas_generated_layer_item, complete_editor_canvas_generation_with_items,
|
||||
persist_editor_generated_media_asset, try_remove_editor_downloaded_image_background,
|
||||
persist_editor_generated_media_asset,
|
||||
},
|
||||
http_error::AppError,
|
||||
openai_image_generation::DownloadedOpenAiImage,
|
||||
@@ -102,7 +106,8 @@ const FIXED_ARK_CHARACTER_VIDEO_DURATION_SECONDS: u32 = 4;
|
||||
const ARK_VIDEO_TASK_POLL_INTERVAL_MS: u64 = 5_000;
|
||||
const EDITOR_CHARACTER_ANIMATION_MODEL: &str = "seedance2.0-fast";
|
||||
const EDITOR_CHARACTER_ANIMATION_ASSET_KIND: &str = "editor_character_animation";
|
||||
const EDITOR_CHARACTER_ANIMATION_PROMPT_PREFIX: &str = "生成游戏角色动画，参考图作为首帧和尾帧，画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，只作为抠像底色，禁止出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。\n动作描述：";
|
||||
const EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND: &str = "editor_green_screen_source";
|
||||
const EDITOR_GREEN_SCREEN_SOURCE_SLOT: &str = "green_screen_source";
|
||||
const EDITOR_VIDEO_ASSET_KIND: &str = "editor_video";
|
||||
const EDITOR_VIDEO_ENTITY_KIND: &str = "editor_canvas";
|
||||
const EDITOR_VIDEO_SLOT: &str = "video_preview";
|
||||
@@ -2059,13 +2064,19 @@ async fn extract_and_persist_editor_character_animation_frames(
|
||||
Some(request.duration_seconds as f64),
|
||||
)
|
||||
.await?;
|
||||
let finalized_frames = remove_editor_character_animation_frame_backgrounds(
|
||||
persist_editor_character_animation_green_screen_source_frames(
|
||||
state,
|
||||
owner_user_id,
|
||||
source_layer_id,
|
||||
task_id,
|
||||
&finalized_frames,
|
||||
)
|
||||
.await?;
|
||||
let finalized_frames = remove_editor_character_animation_frame_backgrounds(
|
||||
finalized_frames,
|
||||
request.frame_width,
|
||||
request.frame_height,
|
||||
)
|
||||
.await?;
|
||||
)?;
|
||||
|
||||
let mut frame_payloads = Vec::with_capacity(finalized_frames.len());
|
||||
for (index, frame) in finalized_frames.into_iter().enumerate() {
|
||||
@@ -2101,25 +2112,51 @@ async fn extract_and_persist_editor_character_animation_frames(
|
||||
Ok(frame_payloads)
|
||||
}
|
||||
|
||||
async fn remove_editor_character_animation_frame_backgrounds(
|
||||
async fn persist_editor_character_animation_green_screen_source_frames(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
source_layer_id: &str,
|
||||
task_id: &str,
|
||||
frames: &[FinalizedAnimationFrame],
|
||||
) -> Result<(), AppError> {
|
||||
for (index, frame) in frames.iter().enumerate() {
|
||||
put_character_animation_object(
|
||||
state,
|
||||
LegacyAssetPrefix::Animations,
|
||||
vec![
|
||||
"editor".to_string(),
|
||||
sanitize_storage_segment(source_layer_id, "layer"),
|
||||
task_id.to_string(),
|
||||
],
|
||||
format!("green-screen-frame{:02}.{}", index + 1, frame.extension),
|
||||
frame.mime_type.clone(),
|
||||
frame.bytes.clone(),
|
||||
build_asset_metadata(
|
||||
EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND,
|
||||
owner_user_id,
|
||||
"editor_layer",
|
||||
source_layer_id,
|
||||
EDITOR_GREEN_SCREEN_SOURCE_SLOT,
|
||||
"editor-character-animation",
|
||||
),
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove_editor_character_animation_frame_backgrounds(
|
||||
frames: Vec<FinalizedAnimationFrame>,
|
||||
frame_width: u32,
|
||||
frame_height: u32,
|
||||
) -> Result<Vec<FinalizedAnimationFrame>, AppError> {
|
||||
let mut removed_frames = Vec::with_capacity(frames.len());
|
||||
for (index, frame) in frames.into_iter().enumerate() {
|
||||
let file_name = format!("character-animation-frame-{:02}.png", index + 1);
|
||||
let removed = try_remove_editor_downloaded_image_background(
|
||||
state,
|
||||
&DownloadedOpenAiImage {
|
||||
bytes: frame.bytes,
|
||||
mime_type: frame.mime_type,
|
||||
extension: frame.extension,
|
||||
},
|
||||
file_name.as_str(),
|
||||
)
|
||||
.await?;
|
||||
for frame in frames {
|
||||
let removed = remove_editor_generated_green_screen_background(&DownloadedOpenAiImage {
|
||||
bytes: frame.bytes,
|
||||
mime_type: frame.mime_type,
|
||||
extension: frame.extension,
|
||||
})?;
|
||||
removed_frames.push(finalize_animation_frame_payload(
|
||||
removed.bytes.as_slice(),
|
||||
removed.mime_type.as_str(),
|
||||
@@ -3345,8 +3382,8 @@ fn editor_video_bad_request(message: impl Into<String>) -> AppError {
|
||||
|
||||
fn build_editor_character_animation_prompt(prompt_text: &str) -> String {
|
||||
format!(
|
||||
"{}\n{}",
|
||||
EDITOR_CHARACTER_ANIMATION_PROMPT_PREFIX,
|
||||
"生成游戏角色动画，参考图作为首帧和尾帧，画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。{}；禁止出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。\n动作描述：\n{}",
|
||||
editor_green_screen_character_prompt_clause(),
|
||||
prompt_text.trim()
|
||||
)
|
||||
}
|
||||
@@ -5364,6 +5401,8 @@ mod tests {
|
||||
assert!(prompt.contains("生成游戏角色动画"));
|
||||
assert!(prompt.contains("参考图作为首帧和尾帧"));
|
||||
assert!(prompt.contains("背景固定为单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕"));
|
||||
assert!(prompt.contains("绿幕背景必须平整无纹理、无渐变、无阴影"));
|
||||
assert!(prompt.contains("角色主体不得带绿色描边、绿色投影或绿色反光"));
|
||||
assert!(prompt.contains("动作描述：\n行走两步后回到站姿。"));
|
||||
}
|
||||
|
||||
@@ -5413,7 +5452,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_frames_default_to_birefnet_cutout() {
|
||||
fn editor_character_animation_frames_use_local_green_screen_postprocess() {
|
||||
let source = include_str!("character_animation_assets.rs");
|
||||
assert_function_contains(
|
||||
source,
|
||||
@@ -5421,15 +5460,36 @@ mod tests {
|
||||
"async fn publish_animation_set",
|
||||
&[
|
||||
"apply_chroma_key: false",
|
||||
"persist_editor_character_animation_green_screen_source_frames",
|
||||
"remove_editor_character_animation_frame_backgrounds",
|
||||
],
|
||||
);
|
||||
assert_function_contains_in_order(
|
||||
source,
|
||||
"async fn extract_and_persist_editor_character_animation_frames",
|
||||
"async fn publish_animation_set",
|
||||
&[
|
||||
"persist_editor_character_animation_green_screen_source_frames",
|
||||
"remove_editor_character_animation_frame_backgrounds",
|
||||
],
|
||||
);
|
||||
assert_function_contains(
|
||||
source,
|
||||
"async fn remove_editor_character_animation_frame_backgrounds",
|
||||
"async fn persist_editor_character_animation_green_screen_source_frames",
|
||||
"fn remove_editor_character_animation_frame_backgrounds",
|
||||
&[
|
||||
"EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND",
|
||||
"EDITOR_GREEN_SCREEN_SOURCE_SLOT",
|
||||
"green-screen-frame",
|
||||
"put_character_animation_object",
|
||||
],
|
||||
);
|
||||
assert_function_contains(
|
||||
source,
|
||||
"fn remove_editor_character_animation_frame_backgrounds",
|
||||
"async fn publish_animation_set",
|
||||
&[
|
||||
"try_remove_editor_downloaded_image_background",
|
||||
"remove_editor_generated_green_screen_background",
|
||||
"finalize_animation_frame_payload",
|
||||
],
|
||||
);
|
||||
@@ -5898,4 +5958,22 @@ mod tests {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn assert_function_contains_in_order(source: &str, start: &str, end: &str, snippets: &[&str]) {
|
||||
let start_index = source
|
||||
.find(start)
|
||||
.unwrap_or_else(|| panic!("missing function start marker: {start}"));
|
||||
let tail = &source[start_index..];
|
||||
let end_index = tail
|
||||
.find(end)
|
||||
.unwrap_or_else(|| panic!("missing function end marker: {end}"));
|
||||
let body = &tail[..end_index];
|
||||
let mut cursor = 0;
|
||||
for snippet in snippets {
|
||||
let next = body[cursor..]
|
||||
.find(snippet)
|
||||
.unwrap_or_else(|| panic!("function body starting with {start} should contain {snippet} after offset {cursor}"));
|
||||
cursor += next + snippet.len();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ use crate::{http_error::AppError, request_context::RequestContext, state::AppSta
|
||||
|
||||
pub(crate) const EDITOR_IMAGE_GENERATION_JOB_KIND: &str = "editor_image_generation";
|
||||
pub(crate) const EDITOR_IMAGE_EDIT_JOB_KIND: &str = "editor_image_edit";
|
||||
pub(crate) const EDITOR_BACKGROUND_REMOVAL_JOB_KIND: &str = "editor_background_removal";
|
||||
pub(crate) const EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND: &str =
|
||||
"editor_icon_spritesheet_generation";
|
||||
pub(crate) const EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND: &str =
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
use axum::http::StatusCode;
|
||||
use platform_image::generated_asset_sheets::{
|
||||
GeneratedAssetSheetAlphaOptions, GeneratedAssetSheetError,
|
||||
remove_generated_asset_sheet_green_screen_background_bytes,
|
||||
};
|
||||
use serde_json::json;
|
||||
|
||||
use crate::{http_error::AppError, openai_image_generation::DownloadedOpenAiImage};
|
||||
|
||||
pub(crate) const EDITOR_GREEN_SCREEN_COLOR_PROMPT: &str = "单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕";
|
||||
pub(crate) const EDITOR_GREEN_SCREEN_BACKGROUND_GUARDRAILS: &str =
|
||||
"绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具";
|
||||
pub(crate) const EDITOR_GREEN_SCREEN_ASSET_GUARDRAILS: &str =
|
||||
"素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光";
|
||||
pub(crate) const EDITOR_GREEN_SCREEN_CHARACTER_GUARDRAILS: &str =
|
||||
"角色主体不得带绿色描边、绿色投影或绿色反光";
|
||||
|
||||
pub(crate) fn editor_green_screen_asset_prompt_clause() -> String {
|
||||
format!(
|
||||
"背景必须是{}，且{}，方便扣除背景；{}",
|
||||
EDITOR_GREEN_SCREEN_COLOR_PROMPT,
|
||||
EDITOR_GREEN_SCREEN_BACKGROUND_GUARDRAILS,
|
||||
EDITOR_GREEN_SCREEN_ASSET_GUARDRAILS
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn editor_green_screen_character_prompt_clause() -> String {
|
||||
format!(
|
||||
"背景固定为{}，只作为抠像底色；{}；{}",
|
||||
EDITOR_GREEN_SCREEN_COLOR_PROMPT,
|
||||
EDITOR_GREEN_SCREEN_BACKGROUND_GUARDRAILS,
|
||||
EDITOR_GREEN_SCREEN_CHARACTER_GUARDRAILS
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn editor_ui_design_asset_extraction_prompt() -> String {
|
||||
format!(
|
||||
"仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用{}。{}，方便后续扣除背景；{}。",
|
||||
EDITOR_GREEN_SCREEN_COLOR_PROMPT,
|
||||
EDITOR_GREEN_SCREEN_BACKGROUND_GUARDRAILS,
|
||||
EDITOR_GREEN_SCREEN_ASSET_GUARDRAILS
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn remove_editor_generated_green_screen_background(
|
||||
image: &DownloadedOpenAiImage,
|
||||
) -> Result<DownloadedOpenAiImage, AppError> {
|
||||
let bytes = remove_generated_asset_sheet_green_screen_background_bytes(
|
||||
image.bytes.as_slice(),
|
||||
GeneratedAssetSheetAlphaOptions {
|
||||
remove_muted_green_screen_background: true,
|
||||
detect_internal_holes: true,
|
||||
internal_hole_min_pixels: 16,
|
||||
..GeneratedAssetSheetAlphaOptions::green_screen()
|
||||
},
|
||||
)
|
||||
.map_err(map_editor_green_screen_error)?;
|
||||
|
||||
Ok(DownloadedOpenAiImage {
|
||||
bytes,
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
fn map_editor_green_screen_error(error: GeneratedAssetSheetError) -> AppError {
|
||||
let status = match error {
|
||||
GeneratedAssetSheetError::DecodeImage { .. } => StatusCode::BAD_GATEWAY,
|
||||
GeneratedAssetSheetError::InvalidRequest { .. } => StatusCode::BAD_REQUEST,
|
||||
GeneratedAssetSheetError::EncodeImage { .. }
|
||||
| GeneratedAssetSheetError::BuildHttpClient { .. }
|
||||
| GeneratedAssetSheetError::Oss(_) => StatusCode::INTERNAL_SERVER_ERROR,
|
||||
};
|
||||
AppError::from_status(status).with_details(json!({
|
||||
"provider": "editor-green-screen",
|
||||
"message": error.message(),
|
||||
}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::io::Cursor;
|
||||
|
||||
use image::{DynamicImage, ImageFormat, Rgba, RgbaImage};
|
||||
|
||||
#[test]
|
||||
fn editor_green_screen_postprocess_turns_background_transparent() {
|
||||
let mut source = RgbaImage::from_pixel(3, 3, Rgba([0, 255, 0, 255]));
|
||||
source.put_pixel(1, 1, Rgba([240, 40, 40, 255]));
|
||||
let mut bytes = Vec::new();
|
||||
DynamicImage::ImageRgba8(source)
|
||||
.write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png)
|
||||
.expect("test PNG should encode");
|
||||
|
||||
let output = remove_editor_generated_green_screen_background(&DownloadedOpenAiImage {
|
||||
bytes,
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
})
|
||||
.expect("green screen should be removed");
|
||||
let image = image::load_from_memory(output.bytes.as_slice())
|
||||
.expect("output PNG should decode")
|
||||
.to_rgba8();
|
||||
|
||||
assert_eq!(image.get_pixel(0, 0).0[3], 0);
|
||||
assert_eq!(image.get_pixel(1, 1).0[3], 255);
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -20,17 +20,19 @@ use crate::{
|
||||
generate_editor_character_animation_for_owner, generate_editor_video_for_owner,
|
||||
},
|
||||
editor_generation_queue::{
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, EDITOR_BACKGROUND_REMOVAL_JOB_KIND,
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND,
|
||||
EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, EDITOR_IMAGE_EDIT_JOB_KIND,
|
||||
EDITOR_IMAGE_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
|
||||
EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND,
|
||||
},
|
||||
editor_project::{
|
||||
EditorGenerationCaller, EditorIconSpritesheetGenerationRequest, EditorImageEditRequest,
|
||||
EditorBackgroundRemovalRequest, EditorGenerationCaller,
|
||||
EditorIconSpritesheetGenerationRequest, EditorImageEditRequest,
|
||||
EditorImageGenerationRequest, EditorUiDesignAssetExtractionRequest,
|
||||
edit_editor_image_for_owner, extract_editor_ui_design_assets_for_owner,
|
||||
generate_editor_icon_spritesheet_for_owner, generate_editor_image_for_owner,
|
||||
remove_editor_image_background_for_owner,
|
||||
},
|
||||
jump_hop::{
|
||||
JUMP_HOP_COMPILE_DRAFT_JOB_KIND, JumpHopCompileDraftWorkerPayload,
|
||||
@@ -615,6 +617,35 @@ async fn process_external_generation_job_once(
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_BACKGROUND_REMOVAL_JOB_KIND => {
|
||||
let mut payload = match serde_json::from_str::<EditorBackgroundRemovalRequest>(
|
||||
job.request_payload_json.as_str(),
|
||||
) {
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布去除背景任务参数解析失败：{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
payload.task_id.get_or_insert_with(|| job.job_id.clone());
|
||||
let request_context = worker_request_context(&job);
|
||||
match remove_editor_image_background_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
editor_generation_worker_caller(&job),
|
||||
payload,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(error) => {
|
||||
let message = error.body_text();
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<EditorIconSpritesheetGenerationRequest>(
|
||||
job.request_payload_json.as_str(),
|
||||
|
||||
@@ -38,6 +38,7 @@ mod custom_world_result_prompts;
|
||||
mod custom_world_rpg_draft_prompts;
|
||||
mod editor_generation_config;
|
||||
mod editor_generation_queue;
|
||||
mod editor_green_screen;
|
||||
mod editor_project;
|
||||
mod edutainment_baby_drawing;
|
||||
mod edutainment_baby_object;
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
use std::io::Cursor;
|
||||
|
||||
use super::color::{
|
||||
GENERATED_ASSET_SHEET_GREEN_SCREEN_MIN_SCORE, GENERATED_ASSET_SHEET_GREEN_SCREEN_SOFT_SCORE,
|
||||
GENERATED_ASSET_SHEET_HARD_GREEN_SCREEN_SCORE, clamp_generated_asset_sheet_unit,
|
||||
@@ -8,6 +10,8 @@ use super::color::{
|
||||
is_generated_asset_sheet_soft_green_matte_pixel, lerp_generated_asset_sheet_channel,
|
||||
touches_generated_asset_sheet_background_mask,
|
||||
};
|
||||
use super::error::GeneratedAssetSheetError;
|
||||
use image::ImageFormat;
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct GeneratedAssetSheetKeyColor {
|
||||
@@ -100,6 +104,24 @@ pub fn apply_generated_asset_sheet_alpha_with_options(
|
||||
image::DynamicImage::ImageRgba8(image)
|
||||
}
|
||||
|
||||
pub fn remove_generated_asset_sheet_green_screen_background_bytes(
|
||||
source: &[u8],
|
||||
options: GeneratedAssetSheetAlphaOptions,
|
||||
) -> Result<Vec<u8>, GeneratedAssetSheetError> {
|
||||
let source = image::load_from_memory(source).map_err(|error| {
|
||||
GeneratedAssetSheetError::decode_image(format!("绿幕图片解码失败：{error}"))
|
||||
})?;
|
||||
let transparent = apply_generated_asset_sheet_alpha_with_options(source, options);
|
||||
|
||||
let mut cursor = Cursor::new(Vec::new());
|
||||
transparent
|
||||
.write_to(&mut cursor, ImageFormat::Png)
|
||||
.map_err(|error| {
|
||||
GeneratedAssetSheetError::encode_image(format!("透明绿幕 PNG 编码失败：{error}"))
|
||||
})?;
|
||||
Ok(cursor.into_inner())
|
||||
}
|
||||
|
||||
fn remove_generated_asset_sheet_green_screen_background(
|
||||
pixels: &mut [u8],
|
||||
width: usize,
|
||||
|
||||
@@ -8,6 +8,7 @@ pub mod sheet;
|
||||
pub use alpha::{
|
||||
GeneratedAssetSheetAlphaOptions, GeneratedAssetSheetKeyColor,
|
||||
apply_generated_asset_sheet_alpha_with_options, apply_generated_asset_sheet_green_screen_alpha,
|
||||
remove_generated_asset_sheet_green_screen_background_bytes,
|
||||
};
|
||||
pub use error::GeneratedAssetSheetError;
|
||||
pub use persist::{
|
||||
|
||||
@@ -7,7 +7,8 @@ use platform_image::generated_asset_sheets::{
|
||||
apply_generated_asset_sheet_alpha_with_options, apply_generated_asset_sheet_green_screen_alpha,
|
||||
build_generated_asset_sheet_prompt, crop_generated_asset_sheet_view_edge_matte,
|
||||
crop_generated_asset_sheet_view_edge_matte_with_options,
|
||||
prepare_generated_asset_sheet_put_request, slice_generated_asset_sheet,
|
||||
prepare_generated_asset_sheet_put_request,
|
||||
remove_generated_asset_sheet_green_screen_background_bytes, slice_generated_asset_sheet,
|
||||
slice_generated_asset_sheet_two_items_per_row,
|
||||
};
|
||||
use platform_oss::LegacyAssetPrefix;
|
||||
@@ -144,6 +145,29 @@ fn generated_asset_sheet_green_screen_alpha_removes_green_background() {
|
||||
assert_eq!(cleaned.get_pixel(10, 10).0[3], 255);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generated_asset_sheet_green_screen_background_bytes_returns_png() {
|
||||
let mut sheet = RgbaImage::from_pixel(3, 3, Rgba([0, 255, 0, 255]));
|
||||
sheet.put_pixel(1, 1, Rgba([240, 40, 40, 255]));
|
||||
|
||||
let output = remove_generated_asset_sheet_green_screen_background_bytes(
|
||||
encode_image(sheet).as_slice(),
|
||||
GeneratedAssetSheetAlphaOptions {
|
||||
remove_muted_green_screen_background: true,
|
||||
detect_internal_holes: true,
|
||||
internal_hole_min_pixels: 16,
|
||||
..GeneratedAssetSheetAlphaOptions::green_screen()
|
||||
},
|
||||
)
|
||||
.expect("green screen bytes should process");
|
||||
let image = image::load_from_memory(output.as_slice())
|
||||
.expect("output PNG should decode")
|
||||
.to_rgba8();
|
||||
|
||||
assert_eq!(image.get_pixel(0, 0).0[3], 0);
|
||||
assert_eq!(image.get_pixel(1, 1).0[3], 255);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generated_asset_sheet_muted_green_alpha_requires_explicit_option() {
|
||||
let mut sheet = RgbaImage::from_pixel(20, 20, Rgba([59, 131, 67, 255]));
|
||||
|
||||
@@ -143,6 +143,34 @@ describe('ImageCanvasAssetRowView', () => {
|
||||
expect(onDownloadAsset).toHaveBeenCalledWith(asset);
|
||||
});
|
||||
|
||||
it('closes the right click menu when row actions run on the same asset', () => {
|
||||
const asset = createAsset({
|
||||
sourceResourceId: 'resource-asset-1',
|
||||
publicShowcaseEnabled: true,
|
||||
});
|
||||
const addAssetLayer = vi.fn();
|
||||
const onDownloadAsset = vi.fn();
|
||||
renderAssetRow({ asset, addAssetLayer, onDownloadAsset });
|
||||
|
||||
fireEvent.contextMenu(screen.getByRole('button', { name: '添加账号素材A' }), {
|
||||
clientX: 120,
|
||||
clientY: 80,
|
||||
});
|
||||
expect(screen.getByRole('menu')).toBeTruthy();
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '下载素材账号素材A' }));
|
||||
expect(onDownloadAsset).toHaveBeenCalledWith(asset);
|
||||
expect(screen.queryByRole('menu')).toBeNull();
|
||||
|
||||
fireEvent.contextMenu(screen.getByRole('button', { name: '添加账号素材A' }), {
|
||||
clientX: 120,
|
||||
clientY: 80,
|
||||
});
|
||||
fireEvent.click(screen.getByRole('button', { name: '添加账号素材A' }));
|
||||
expect(addAssetLayer).toHaveBeenCalledWith(asset);
|
||||
expect(screen.queryByRole('menu')).toBeNull();
|
||||
});
|
||||
|
||||
it('selects the asset instead of adding it in selection mode', () => {
|
||||
const addAssetLayer = vi.fn();
|
||||
const toggleAssetSelected = vi.fn();
|
||||
|
||||
@@ -210,13 +210,19 @@ export function ImageCanvasAssetRowView({
|
||||
label={`下载素材${asset.label}`}
|
||||
title="下载"
|
||||
icon={Download}
|
||||
onClick={() => onDownloadAsset(asset)}
|
||||
onClick={() => {
|
||||
closeMenu();
|
||||
onDownloadAsset(asset);
|
||||
}}
|
||||
/>
|
||||
<EditorIconButton
|
||||
label={`重命名素材${asset.label}`}
|
||||
title="重命名"
|
||||
icon={Pencil}
|
||||
onClick={() => startRenamingAsset(asset)}
|
||||
onClick={() => {
|
||||
closeMenu();
|
||||
startRenamingAsset(asset);
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
@@ -251,6 +257,7 @@ export function ImageCanvasAssetRowView({
|
||||
if (isUploadingAsset || isFailedUpload) {
|
||||
return;
|
||||
}
|
||||
closeMenu();
|
||||
if (suppressAssetClickRef.current) {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -82,6 +82,22 @@ describe('ImageCanvasContextMenusView', () => {
|
||||
expect(props.onCloseContextMenu).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('closes canvas context menus when another operation starts outside', () => {
|
||||
const props = renderContextMenus({
|
||||
contextMenu: {
|
||||
kind: 'blank',
|
||||
x: 10,
|
||||
y: 12,
|
||||
canvasPoint: { x: 18, y: 24 },
|
||||
},
|
||||
});
|
||||
|
||||
fireEvent.pointerDown(document.body);
|
||||
|
||||
expect(props.onCloseContextMenu).toHaveBeenCalledTimes(1);
|
||||
expect(props.onCloseImageContextMenu).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('renders layer context commands and forwards layer operations', () => {
|
||||
const layer = createLayer({
|
||||
assetKind: 'character',
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { useCallback, useEffect, useRef } from 'react';
|
||||
|
||||
import {
|
||||
PlatformFloatingMenu,
|
||||
PlatformFloatingMenuItem,
|
||||
@@ -72,11 +74,44 @@ export function ImageCanvasContextMenusView({
|
||||
onOpenCharacterAnimationPanel,
|
||||
}: ImageCanvasContextMenusViewProps) {
|
||||
const isImageSequenceLayer = contextMenuLayer?.mediaType === 'image-sequence';
|
||||
const menuRef = useRef<HTMLDivElement | null>(null);
|
||||
const closeMenus = useCallback(() => {
|
||||
onCloseContextMenu();
|
||||
onCloseImageContextMenu();
|
||||
}, [onCloseContextMenu, onCloseImageContextMenu]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!contextMenu && !imageContextMenu) {
|
||||
return undefined;
|
||||
}
|
||||
const handlePointerDown = (event: PointerEvent) => {
|
||||
if (menuRef.current?.contains(event.target as Node)) {
|
||||
return;
|
||||
}
|
||||
closeMenus();
|
||||
};
|
||||
const handleKeyDown = (event: KeyboardEvent) => {
|
||||
if (event.key === 'Escape') {
|
||||
closeMenus();
|
||||
}
|
||||
};
|
||||
window.addEventListener('pointerdown', handlePointerDown);
|
||||
window.addEventListener('keydown', handleKeyDown);
|
||||
window.addEventListener('scroll', closeMenus, true);
|
||||
window.addEventListener('resize', closeMenus);
|
||||
return () => {
|
||||
window.removeEventListener('pointerdown', handlePointerDown);
|
||||
window.removeEventListener('keydown', handleKeyDown);
|
||||
window.removeEventListener('scroll', closeMenus, true);
|
||||
window.removeEventListener('resize', closeMenus);
|
||||
};
|
||||
}, [closeMenus, contextMenu, imageContextMenu]);
|
||||
|
||||
return (
|
||||
<>
|
||||
{contextMenu ? (
|
||||
<div
|
||||
ref={menuRef}
|
||||
className="image-canvas-editor__context-menu"
|
||||
role="menu"
|
||||
aria-label={
|
||||
@@ -334,6 +369,7 @@ export function ImageCanvasContextMenusView({
|
||||
|
||||
{imageContextMenu && imageContextMenuLayer && !contextMenu ? (
|
||||
<div
|
||||
ref={menuRef}
|
||||
className="image-canvas-editor__context-menu"
|
||||
style={{
|
||||
left: imageContextMenu.x,
|
||||
|
||||
@@ -3029,9 +3029,9 @@ describe('ImageCanvasEditorView generation integration', () => {
|
||||
},
|
||||
],
|
||||
prompt:
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。',
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。',
|
||||
actualPrompt:
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。',
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。',
|
||||
model: 'gpt-image-2',
|
||||
provider: 'VectorEngine',
|
||||
taskId: 'ui-extract-task-1',
|
||||
@@ -3134,7 +3134,7 @@ describe('ImageCanvasEditorView generation integration', () => {
|
||||
expect.objectContaining({
|
||||
title: '提取提示词',
|
||||
value:
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。',
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。',
|
||||
}),
|
||||
]),
|
||||
references: expect.arrayContaining([
|
||||
@@ -3163,7 +3163,7 @@ describe('ImageCanvasEditorView generation integration', () => {
|
||||
expect.objectContaining({
|
||||
title: '提取提示词',
|
||||
value:
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。',
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。',
|
||||
}),
|
||||
]),
|
||||
}),
|
||||
|
||||
@@ -136,7 +136,7 @@ function createStageProps(): ImageCanvasStageViewProps {
|
||||
hoveredLayerId: null,
|
||||
canvasMarquee: null,
|
||||
canvasGenerationDialogs: [],
|
||||
localTaskItems: [],
|
||||
taskListRefreshKey: 0,
|
||||
isTaskSidebarOpen: false,
|
||||
generateDialog: null,
|
||||
cropExpandPanel: null,
|
||||
@@ -180,7 +180,6 @@ function createStageProps(): ImageCanvasStageViewProps {
|
||||
onGenerationFramePointerDown: vi.fn(),
|
||||
onActivateGenerationDialog: vi.fn(),
|
||||
onFocusExternalTask: vi.fn(),
|
||||
onFocusLocalTask: vi.fn(),
|
||||
onToggleTaskSidebar: vi.fn(),
|
||||
onCropExpandHandlePointerDown: vi.fn(),
|
||||
onOpenQuickEditPanel: vi.fn(),
|
||||
|
||||
@@ -252,19 +252,6 @@ export type CanvasGenerationDialogState = GenerateDialogState & {
|
||||
|
||||
export type CanvasTaskStatus = 'pending' | 'running' | 'done' | 'failed';
|
||||
|
||||
export type CanvasTaskListItem = {
|
||||
id: string;
|
||||
title: string;
|
||||
detail: string;
|
||||
status: CanvasTaskStatus;
|
||||
startedAt: number;
|
||||
finishedAt?: number;
|
||||
elapsedMs?: number;
|
||||
provider?: string;
|
||||
errorMessage?: string;
|
||||
focusLayerId?: string;
|
||||
};
|
||||
|
||||
export type ImageContextMenuState = {
|
||||
layerId: string;
|
||||
x: number;
|
||||
|
||||
@@ -42,7 +42,6 @@ import type {
|
||||
CanvasContextMenuState,
|
||||
CanvasGenerationDialogState,
|
||||
CanvasLayer,
|
||||
CanvasTaskListItem,
|
||||
CanvasTool,
|
||||
CanvasViewport,
|
||||
CharacterReferenceImage,
|
||||
@@ -647,7 +646,9 @@ export function ImageCanvasEditorView() {
|
||||
);
|
||||
const focusCanvasLayerById = useCallback(
|
||||
(layerId: string) => {
|
||||
const targetLayer = layersRef.current.find((layer) => layer.id === layerId);
|
||||
const targetLayer = layersRef.current.find(
|
||||
(layer) => layer.id === layerId,
|
||||
);
|
||||
if (!targetLayer) {
|
||||
return;
|
||||
}
|
||||
@@ -656,7 +657,12 @@ export function ImageCanvasEditorView() {
|
||||
setActiveSidebarPanel('layers');
|
||||
focusCanvasBounds(targetLayer);
|
||||
},
|
||||
[focusCanvasBounds, selectSingleLayer, setActiveSidebarPanel, setActiveTool],
|
||||
[
|
||||
focusCanvasBounds,
|
||||
selectSingleLayer,
|
||||
setActiveSidebarPanel,
|
||||
setActiveTool,
|
||||
],
|
||||
);
|
||||
const focusCanvasGenerationDialog = useCallback(
|
||||
(dialog: CanvasGenerationDialogState) => {
|
||||
@@ -673,7 +679,9 @@ export function ImageCanvasEditorView() {
|
||||
setSelectedLayerIds([sourceLayer.id]);
|
||||
setActiveTool('generate');
|
||||
setActiveSidebarPanel('layers');
|
||||
setViewport(resolveQuickEditFocusViewport({ sourceLayer, canvasSize }));
|
||||
setViewport(
|
||||
resolveQuickEditFocusViewport({ sourceLayer, canvasSize }),
|
||||
);
|
||||
return;
|
||||
}
|
||||
}
|
||||
@@ -725,14 +733,6 @@ export function ImageCanvasEditorView() {
|
||||
},
|
||||
[focusCanvasGenerationDialog, focusCanvasLayerById],
|
||||
);
|
||||
const focusLocalCanvasTask = useCallback(
|
||||
(task: CanvasTaskListItem) => {
|
||||
if (task.focusLayerId) {
|
||||
focusCanvasLayerById(task.focusLayerId);
|
||||
}
|
||||
},
|
||||
[focusCanvasLayerById],
|
||||
);
|
||||
const persistGeneratedAsset = useCallback(
|
||||
(layer: CanvasLayer) => {
|
||||
if (authUiRef.current && !authUiRef.current.canAccessProtectedData) {
|
||||
@@ -1735,7 +1735,7 @@ export function ImageCanvasEditorView() {
|
||||
hoveredLayerId,
|
||||
canvasMarquee,
|
||||
canvasGenerationDialogs,
|
||||
localTaskItems: generationSurface.backgroundRemovalTasks,
|
||||
taskListRefreshKey: generationSurface.taskListRefreshKey,
|
||||
isTaskSidebarOpen: generationSurface.isTaskSidebarOpen,
|
||||
generateDialog,
|
||||
cropExpandPanel: generationSurface.cropExpandPanel,
|
||||
@@ -1784,7 +1784,6 @@ export function ImageCanvasEditorView() {
|
||||
onGenerationFramePointerDown: handleGenerationFramePointerDown,
|
||||
onActivateGenerationDialog: activateCanvasGenerationDialog,
|
||||
onFocusExternalTask: focusExternalGenerationTask,
|
||||
onFocusLocalTask: focusLocalCanvasTask,
|
||||
onToggleTaskSidebar: generationSurface.toggleTaskSidebar,
|
||||
onCropExpandHandlePointerDown: generationSurface.startCropExpandFrameResize,
|
||||
onOpenQuickEditPanel: openLayerGenerationDialog,
|
||||
|
||||
@@ -481,7 +481,7 @@ describe('ImageCanvasGenerationDialogModel', () => {
|
||||
{
|
||||
title: '提取提示词',
|
||||
value:
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板或绿色阴影。',
|
||||
'仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。',
|
||||
},
|
||||
{ title: 'model', value: 'gpt-image-2' },
|
||||
],
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user