修复序列帧去背景队列与幂等发布

序列帧去背景改用本地绿幕算法并在 durable 父任务内并行处理。

将去背景操作加入生成结果白名单,并按 operation_id 隔离帧对象路径。

去背景结果发布为新的序列资源、素材和画布图层,同时保留原始序列。

同步前端生成占位、相关测试及队列与资源契约文档。
This commit is contained in:
2026-08-15 11:59:50 +08:00
parent 05f0e2db06
commit 13afdd7a3b
11 changed files with 404 additions and 125 deletions
@@ -1,5 +1,13 @@
# 决策记录
## 2026-08-15 Spine 序列帧去背景复用父队列并行处理
- 背景:Spine 序列帧去背景已经是 `editor_character_animation_background_removal` durable job,但父 job 内的 33 帧仍逐帧串行下载、绿幕处理、上传和帧对象准备,用户只能看到长时间的二态 loading。
- 决策:继续只使用一个现有 `external_generation_job` 父任务,不新增逐帧子任务、表、计费或公开 DTO;worker 在父 job 内用 `buffer_unordered(frame_count.max(1))` 并行处理所有帧,必须 drain 全部已发出的 future,再按 `frame_index` 排序并一次性发布。任一帧失败时清理已成功但尚未正式提交的对象;当前帧上传后帧对象准备失败时在该 future 内立即清理,避免并行化引入孤儿对象。
- 幂等边界:帧对象路径绑定本次 job 的 `operation_id`,同一 job 重放复用路径,不同 job 即使输入 fingerprint 相同也必须生成新的对象路径和新的 asset_object;结果作为新的 `character-animation` / `image-sequence` resource、素材和画布图层发布,原始序列保留。
- 去背景算法边界不变:只使用本地 `screen-color-keying` 绿幕扣除,不调用 LLM、BgFilter 或阿里云;普通图片去背景链路不受影响。
- 关联文档:`docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md`。
## 2026-08-12 Repository checks 采用 CI 与本地共用的单一门禁入口
- 背景:master run 1037 的 Backend/Frontend 已通过,但 `Repository checks` 因 3 个 `simple-import-sort/imports` 错误失败。原 pre-commit 只运行 Prettier,Prettier 不处理 ESLint import 排序;推送前又未运行完整仓库 lint,因此本地与 CI 的覆盖范围长期存在漂移。
@@ -14147,4 +14155,5 @@
- 视频直接转换走内部 `editor_character_animation_video_conversion` durable job,FFprobe + FFmpeg 覆盖完整源时长,采样率最高 8 FPS、最低 1 FPS,产出 2–66 帧;480p/720p 是只缩不放的最大长边。该路径固定免费、保留背景,不调用 Seedance 或 BgFilter,也不复制保存源视频。
- 直接转换仍落 `character-animation` / `image-sequence`,配方 action 为 `character-animation.convert`,来源槽位为 `source`;最多 66 帧时首帧 item 同时承载最终 resource/asset。原子生成上限和角色动画拆帧上限沿用 master 的 66,但图标图集公开切片继续保持 64,既有 JSON 大小门禁不放宽。
- merge master 后保留角色图层浮动工具栏和右键菜单中的 `生成动画` 快捷入口,统一进入同一个角色动作 dialog;`character-animation` 结果不支持快速编辑,仍保留 `改造 / 去背景 / 拆帧 / 下载`。快速编辑入口与提交门禁使用统一正向白名单,未知媒体或素材类型默认拒绝。
- Spine 序列帧工具栏的“去背景”暂定为纯算法绿幕扣除:只读取正式序列帧 objectKey,使用本地 `screen-color-keying` 处理约定的 `#00FF00` 绿幕,不调用 BgFilter、阿里云或其它模型;普通图片 `/api/editor/images/background-removals` 的通用去背景链路不受影响。
- `/api/external/v1` 不开放上述内部字段或直接转换路由,不修改 OpenAPI;本次不改 SpacetimeDB 表结构、迁移或生成 bindings。
@@ -465,6 +465,7 @@ flowchart TD
- 对同一 job kind 解析 v2;
- 对部署前旧 payload 做兼容转换;
- phase 继续使用 `generating -> processing`;
- `editor_character_animation_background_removal` 与 AI 图片生成一样先入现有 durable queue;其单个父 job 内使用受控的 `buffer_unordered(frame_count.max(1))` 并行帧处理,完成后统一排序和原子发布,不拆成逐帧队列任务;
- 不需要新增逐帧进度列。画布上的 `15/30` 是播放帧数,不是 worker 进度。
### 8.5 计费
@@ -604,9 +605,11 @@ flowchart TD
AI 生成链默认逐帧去背景,视频直接转换固定保留源背景。`character-animation` 浮动工具栏继续提供 `去背景`,与 `改造 / 拆帧 / 下载` 并列;点击后必须走现有 durable 派生任务,不得把序列第一帧当普通图片处理。派生任务保持以下边界:
- 原始不透明序列与透明序列是两个正式 resource;
- 去背景全帧成功后一次性发布新序列;
- 去背景全帧成功后一次性发布新的 `character-animation` / `image-sequence` 资源、素材和画布图层,保留原始序列图层不被替换;
- 任一帧失败不产生残缺结果;
- 当前费用固定为 0 泥点,失败不产生不完整正式结果。
- 当前费用固定为 0 泥点,失败不产生不完整正式结果;请求先进入现有 `external_generation_job` 队列,不能由 HTTP 请求同步执行。
- 队列 worker 保持一个父 job,在 job 内按帧并行执行下载、纯本地绿幕扣除、最终 PNG 上传和帧对象准备;所有已发出的帧必须 drain 完成后再按帧号排序,一帧失败即清理未提交对象并整批失败,不新增逐帧子任务表或独立计费。
- 帧对象路径绑定本次 job 的 `operationId`,同一 job 重放复用路径,不同 job 即使请求 fingerprint 相同也不得复用旧 object location,避免新 `asset_object` 与历史结果发生幂等冲突。
#### 拆帧
@@ -345,7 +345,7 @@ Responses 的终态载荷既是工具调用的恢复源,也是正文的恢复
- Rust 结构体:`ExternalGenerationJob`
- 源码:`server-rs/crates/spacetime-module/src/external_generation.rs`
- 现役覆盖:worker claim 只允许 `source_module = editor-canvas`;下述逐玩法生成和写回描述均为退役前历史。历史 pending / running 行继续保留原状态,不得领取、失败收口或改写 payload。
- 用途:外部生成 worker 的内部持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写阶段、完成 / 失败。队列行继续保存 worker 执行、计费与滚动发布兼容所需字段,末尾可选 `phase` 只取 `generating / processing`;claim 写 `generating`,真实进入抠图处理时由受 `job_id + worker_id + lease_token` 保护的 procedure 写 `processing`。phase procedure 以结构化结果区分 `LeaseFencingRejected` 与 `OtherRejected`;`LeaseFencingRejected` 立即终止,`OtherRejected` 以及 SDK 的 `Procedure` / `Runtime` 错误不重试,只有 `Build` / `ConnectDropped` / `Timeout` 在同一个 job attempt 内重试一次。该重试只重新上报 phase,不把任务写回 `pending`,也不重新调用 provider;编辑器 job 入队固定 `max_attempts=1`,第二次传输失败后任务进入 `failed`,不会回到 `pending` 或从 provider 生成起点重跑。用户可见任务列表、价格、状态、阶段、未确认终态数量和通知确认时间的正式读取事实源已经迁到 `external_generation_job_summary`;BFF 不得再为列表 / 详情 / acknowledge 读取该大表。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile;图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表。结构化 canvas 激活后,当前 worker completion 先以读取时 canvas revision 执行 CAS,并发冲突时拒绝覆盖并保留可诊断失败;目标是进一步收口为受 lease 栅栏保护的单事务幂等写入 `editor_project_resource`、结果 `editor_canvas_layer`、`editor_canvas_generation_dialog` 终态和 canvas revision。未激活 canvas 在 2 MiB 上限内继续走 legacy `editor_canvas.layers_json` 兼容写回。前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
- 用途:外部生成 worker 的内部持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写阶段、完成 / 失败。队列行继续保存 worker 执行、计费与滚动发布兼容所需字段,末尾可选 `phase` 只取 `generating / processing`;claim 写 `generating`,真实进入抠图处理时由受 `job_id + worker_id + lease_token` 保护的 procedure 写 `processing`。phase procedure 以结构化结果区分 `LeaseFencingRejected` 与 `OtherRejected`;`LeaseFencingRejected` 立即终止,`OtherRejected` 以及 SDK 的 `Procedure` / `Runtime` 错误不重试,只有 `Build` / `ConnectDropped` / `Timeout` 在同一个 job attempt 内重试一次。该重试只重新上报 phase,不把任务写回 `pending`,也不重新调用 provider;编辑器 job 入队固定 `max_attempts=1`,第二次传输失败后任务进入 `failed`,不会回到 `pending` 或从 provider 生成起点重跑。用户可见任务列表、价格、状态、阶段、未确认终态数量和通知确认时间的正式读取事实源已经迁到 `external_generation_job_summary`;BFF 不得再为列表 / 详情 / acknowledge 读取该大表。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile;图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_character_animation_video_conversion`、`editor_character_animation_background_removal`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表。结构化 canvas 激活后,当前 worker completion 先以读取时 canvas revision 执行 CAS,并发冲突时拒绝覆盖并保留可诊断失败;目标是进一步收口为受 lease 栅栏保护的单事务幂等写入 `editor_project_resource`、结果 `editor_canvas_layer`、`editor_canvas_generation_dialog` 终态和 canvas revision。未激活 canvas 在 2 MiB 上限内继续走 legacy `editor_canvas.layers_json` 兼容写回。前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
- 2026-08-06 收口覆盖:上一条用途描述中“当前先 CAS、单事务仍是目标”的旧句已作废。现役编辑器生成不再组合调用 object confirm、resource create、asset create、canvas save 和 job complete。`api-server` 只准备稳定候选,再经 `spacetime-client` 调用 `persist_editor_generation_result_and_return`;procedure 在同一 `try_with_tx` 内写入可选 `asset_object`、全部 `editor_project_resource`、`editor_asset`、可选 `asset_entity_binding`、可选 canvas V2 CAS、queue job 终态和 `editor_generation_operation` receipt。结构化 canvas 的 layer / dialog / revision 与未激活 canvas 的 legacy `layers_json` 仍经既有 V2 布局验证分流,前端不直接发明正式完成态。queue 首次提交在同一快照验证 owner、job kind、request fingerprint 和有效 `job_id + worker_id + lease_token`;统一 procedure 已完成 job 后 worker 不得再单独 complete。
- 载荷约束:本次先对 `source_module = editor-canvas` 的 `request_payload_json` / `result_payload_json` 实施有限大小合法 JSON、任意层级禁止 `data:` / `blob:` 的双层门禁,只保存 worker 执行必需的普通参数和已登记媒体引用。画布 Agent 来源的任务可在 `result_payload_json.editor-agent-tool-call-result` 中保存有界的轻量结果和已登记媒体引用,供后端按已有 `externalJobId + owner_user_id` 定向懒回填;其它编辑器任务保持元数据结果,并可保存有界的 `warning.code/reason`。其它玩法在完成各自参考图资源化之前不由本次门禁静默改变既有请求契约。该主表只供 worker claim / 执行、受控维护以及画布 Agent 的定向结果回填读取;正式用户任务列表、单任务状态、队列概览与 acknowledge 不得返回或解析这两个 payload。画布 Agent 懒回填必须经对应工具 formatter 归一为有界轻量媒体引用后写入 OSS 会话,不能把原始 payload 直接透传前端。
- 非阻断告警:角色形象、图标图集和 UI 素材提取已保存 provider 原图、但透明背景处理最终失败时,以原图唯一主图完成任务;透明图和切片不写入画布。这个 source-only 降级只包住透明背景处理的最终失败,phase 上报、provider 原图持久化、透明处理图持久化或画布写回失败仍按任务错误传播。图标 / UI 透明图集成功但自动拆分降级时仍保留透明图集;通用 `warning` 与 `sliceWarning` 只在「透明背景最终失败」这一条上互斥,风格归一化或像素规整产生的通用 `warning` 可与 `sliceWarning` 并存。两类成功降级都以既有 `completed` 状态收口,不新增状态值:source-only 的 inline / external v1 响应使用结构化 `warning.code/reason`,仅拆分失败的 inline / external v1 响应继续使用既有 `sliceWarning.code/reason`,其 `reason` 保留原始诊断;queue worker 才把两者归一为有界的 `result_payload_json.warning`:只有一条时原样保留完整 `reason`,两条并存时按“通用在前、拆分在后”拼接且 `code` 收敛为 `multiple-generation-warnings`(两条 `code` 相同则沿用原 `code`),不允许任何一条被丢弃;`sliceWarning.reason` 无论是否并存都由 worker 添加“图集已生成,但自动拆分未完成:”前缀,拼接结果最后统一做长度上界收敛。除上述画布 Agent 定向回填的轻量结果外,队列结果不保存图片、切片列表或媒体 URL。
@@ -68,7 +68,7 @@ Windows 本地如果已在 `%LOCALAPPDATA%\Genarrative\ffmpeg\bin` 安装 FFmpeg
开发态 `npm run dev` 与 `npm run dev:api-server` 都会注入 `GENARRATIVE_DEV_PASSWORD_ENTRY_AUTO_REGISTER_ENABLED=true`,因此密码登录在本地开发环境可直接注册未知手机号账号。完整 `npm run dev` 会强制父 API 使用 `GENARRATIVE_PROCESS_ROLE=all`,忽略外层显式角色,确保本地 `api-server` 同时监听 HTTP 并消费外部生成队列;只有单模块 `npm run dev:api-server` 会保留显式 `GENARRATIVE_PROCESS_ROLE`,未设置时默认为 `all`。`all` 不内嵌 BgFilter worker;启动器总是先启动并验活独立 `GENARRATIVE_PROCESS_ROLE=bgfilter-worker` 进程,再启动父 API,并向两者注入同一个内部 base URL / Token。Linux 本地默认 `all` 角色启动前,dev 脚本会停止当前仓库、同一个 SpacetimeDB server / database 下遗留的 `GENARRATIVE_PROCESS_ROLE=external-generation-worker` 进程,避免旧 worker 二进制继续抢同一条队列并在业务写回时制造 procedure 超时;显式拆分 `api` / `external-generation-worker` 做生产式验证时不会触发这项清理。生产环境仍按 `api-server` 配置默认关闭密码自动注册,并由独立 worker 进程消费队列。
本地排查外部内容生成 worker 队列时,默认同一 Rust 进程同时监听 HTTP 并消费 `external_generation_job` 队列;更接近生产的验证应分别启动 `api`、`external-generation-worker` 和 `external-generation-controller`。生产默认 `GENARRATIVE_PROCESS_ROLE=api`,外部生成任务由独立 `GENARRATIVE_PROCESS_ROLE=external-generation-worker` 进程消费;生产与容器扩缩容验证保持 `queue`。当前 worker 只领取 `source_module = editor-canvas` 的图片画布任务,包括 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation`。旧玩法历史任务即使仍为 pending / running 也不领取、不改状态;显式把本地进程角色设为 `api` 且没有 worker 时,现役编辑器生成请求只返回 queued/running,不会兜底执行外部 provider。
本地排查外部内容生成 worker 队列时,默认同一 Rust 进程同时监听 HTTP 并消费 `external_generation_job` 队列;更接近生产的验证应分别启动 `api`、`external-generation-worker` 和 `external-generation-controller`。生产默认 `GENARRATIVE_PROCESS_ROLE=api`,外部生成任务由独立 `GENARRATIVE_PROCESS_ROLE=external-generation-worker` 进程消费;生产与容器扩缩容验证保持 `queue`。当前 worker 只领取 `source_module = editor-canvas` 的图片画布任务,包括 `editor_image_generation`、`editor_image_edit`、`editor_background_removal`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_character_animation_video_conversion`、`editor_character_animation_background_removal`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation`。旧玩法历史任务即使仍为 pending / running 也不领取、不改状态;显式把本地进程角色设为 `api` 且没有 worker 时,现役编辑器生成请求只返回 queued/running,不会兜底执行外部 provider。
HTTP 角色的 `GENARRATIVE_SPACETIME_POOL_SIZE` 只表示 procedure / reducer 调用池大小;池连接不订阅 read model。HTTP 角色会额外创建 1 条共享缓存读连接,当前只保留可选的 `user_account` 读取;配置为 `8` 时基础连接拓扑是 8 条调用连接加 1 条缓存读连接。`/readyz` 同时检查调用池与缓存读连接,缓存连接未准备好时不能放量。
@@ -84,20 +84,22 @@ use crate::{
EditorGenerationQueuedResponse, editor_generation_queue_state,
editor_generation_source_entity_id, enqueue_editor_generation_job_for_caller,
},
editor_green_screen::EditorScreenBackgroundColor,
editor_green_screen::{
EditorScreenBackgroundColor, remove_editor_character_animation_green_screen_background,
},
editor_project::{
EDITOR_ASSET_ID_PREFIX, EDITOR_BGFILTER_CROSS_CHECK_ENABLED,
EDITOR_BGFILTER_DEFAULT_SEG_MODEL, EDITOR_RESOURCE_ID_PREFIX,
EditorCanvasGeneratedLayerInput, EditorCanvasLayoutPlan, EditorGenerationCaller,
EditorGenerationOperationContext, PreparedEditorGenerationResultItem,
build_editor_canvas_generated_layer_item, editor_asset_payload_from_record,
EDITOR_BGFILTER_DEFAULT_SEG_MODEL, EDITOR_LOCAL_MATTING_PROVIDER,
EDITOR_RESOURCE_ID_PREFIX, EditorCanvasGeneratedLayerInput, EditorCanvasLayoutPlan,
EditorGenerationCaller, EditorGenerationOperationContext,
PreparedEditorGenerationResultItem, build_editor_canvas_generated_layer_item,
download_editor_persisted_image_object, editor_asset_payload_from_record,
editor_media_src_from_object_key, editor_project_payload_from_record,
editor_project_resource_payload_from_record, map_editor_project_error,
normalize_editor_record_object_key, optional_editor_idempotency_key,
persist_editor_generation_result_atomically, preflight_editor_billable_generation_target,
rebuild_editor_generation_input_references,
remove_editor_generated_screen_background_with_bgfilter,
request_editor_background_removal_image_with_bgfilter_worker,
resolve_editor_reference_object_key_for_owner, sanitize_editor_client_generation_inputs,
serialize_editor_generation_inputs, serialize_editor_image_sequence_frames,
validate_editor_generation_operation_id, with_editor_media_duration_generation_input,
@@ -1230,10 +1232,12 @@ pub(crate) async fn remove_editor_character_animation_background_for_owner(
let operation = editor_generation_operation(&caller)
.map_err(|error| character_animation_error_response(&request_context, error))?;
let task_id = operation.operation_id.clone();
let storage_task_id = format!(
"{}-{}",
sanitize_storage_segment(owner_user_id.as_str(), "owner"),
operation.operation_fingerprint
// 同一 job 的重放必须复用对象路径;新的去背景 job 必须生成一套新序列对象。
// 不能使用 request fingerprint:用户再次对同一源序列执行去背景时 fingerprint 相同,
// 会复用上一任务已登记的 object location,并与新 job 的稳定 asset_object id 冲突。
let storage_task_id = editor_character_animation_background_removal_storage_task_id(
owner_user_id.as_str(),
task_id.as_str(),
);
if let Some(reporter) = caller.phase_reporter.as_ref() {
reporter
@@ -1241,12 +1245,6 @@ pub(crate) async fn remove_editor_character_animation_background_for_owner(
.await
.map_err(|error| character_animation_error_response(&request_context, error))?;
}
let matting_audit = crate::external_api_audit::ExternalApiAuditContext {
user_id: Some(owner_user_id.clone()),
profile_id: Some(project_id.clone()),
request_id: Some(request_context.request_id().to_string()),
external_call_deadline: request_context.external_call_deadline(),
};
let persisted_frames = extract_and_prepare_background_removal_frames(
&state,
&caller,
@@ -1254,7 +1252,6 @@ pub(crate) async fn remove_editor_character_animation_background_for_owner(
&frames,
frame_width,
frame_height,
&matting_audit,
)
.await
.map_err(|error| character_animation_error_response(&request_context, error))?;
@@ -1288,7 +1285,7 @@ pub(crate) async fn remove_editor_character_animation_background_for_owner(
Some(project_id.as_str()),
payload.asset_folder_id.as_deref(),
asset_label.as_str(),
"BgFilter",
EDITOR_LOCAL_MATTING_PROVIDER,
Some(source_resource_id.clone()),
generation_inputs,
plan,
@@ -4826,8 +4823,9 @@ async fn extract_and_prepare_direct_video_conversion_frames(
}
}
/// 对已有序列帧逐帧做通用抠图(BgFilter complex,无键色),生成透明序列帧对象。
/// 输入是源序列帧资源的稳定 objectKey 列表;任一帧失败即整批清理,不发布残缺结果。
/// 对已有序列帧逐帧执行纯算法绿幕扣除,生成透明序列帧对象。
/// 输入是源序列帧资源的稳定 objectKey 列表;不调用 BgFilter、阿里云或其它模型。
/// 任一帧失败即整批清理,不发布残缺结果。
async fn extract_and_prepare_background_removal_frames(
state: &AppState,
caller: &EditorGenerationCaller,
@@ -4835,103 +4833,139 @@ async fn extract_and_prepare_background_removal_frames(
source_frames: &[EditorCharacterAnimationFramePayload],
frame_width: u32,
frame_height: u32,
audit: &crate::external_api_audit::ExternalApiAuditContext,
) -> Result<PersistedEditorCharacterAnimationFrameSet, AppError> {
let mut processed = Vec::with_capacity(source_frames.len());
let mut uploaded_object_keys = Vec::with_capacity(source_frames.len());
for (frame_index, source_frame) in source_frames.iter().enumerate() {
let source_object_key = source_frame
.object_key
.as_deref()
.map(str::trim)
.filter(|value| !value.is_empty())
.ok_or_else(|| {
editor_character_animation_bad_request("序列帧缺少稳定 objectKey,无法去背景。")
})?;
let removed = request_editor_background_removal_image_with_bgfilter_worker(
state,
source_object_key,
audit,
)
.await?;
let finalized = finalize_animation_frame_payload(
removed.image.bytes.as_slice(),
removed.image.mime_type.as_str(),
frame_width,
frame_height,
false,
)?;
let content_type = finalized.mime_type.clone();
let put_result = match put_character_animation_frame_object(
state,
frame_index + 1,
"background_removal_frame_put",
LegacyAssetPrefix::Animations,
vec![
"editor".to_string(),
"background-removals".to_string(),
task_id.to_string(),
],
format!("frame{:03}.png", frame_index + 1),
finalized.mime_type,
finalized.bytes,
build_asset_metadata(
EDITOR_CHARACTER_ANIMATION_ASSET_KIND,
caller.owner_user_id.as_str(),
"editor_background_removal",
task_id,
"animation_frame",
"background-removal",
),
)
.await
{
Ok(result) => result,
Err(error) => {
cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await;
return Err(error);
use futures_util::StreamExt as _;
let frame_count = source_frames.len();
let frame_results = futures_util::stream::iter(source_frames.iter().cloned().enumerate().map(
|(frame_index, source_frame)| async move {
let result = async {
let source_object_key = source_frame
.object_key
.as_deref()
.map(str::trim)
.filter(|value| !value.is_empty())
.ok_or_else(|| {
editor_character_animation_bad_request(
"序列帧缺少稳定 objectKey,无法去背景。",
)
})?;
let source_image =
download_editor_persisted_image_object(state, source_object_key).await?;
let removed =
remove_editor_character_animation_green_screen_background(&source_image)?;
let finalized = finalize_animation_frame_payload(
removed.bytes.as_slice(),
removed.mime_type.as_str(),
frame_width,
frame_height,
false,
)?;
let content_type = finalized.mime_type.clone();
let put_result = put_character_animation_frame_object(
state,
frame_index + 1,
"background_removal_frame_put",
LegacyAssetPrefix::Animations,
vec![
"editor".to_string(),
"background-removals".to_string(),
task_id.to_string(),
],
format!("frame{:03}.png", frame_index + 1),
finalized.mime_type,
finalized.bytes,
build_asset_metadata(
EDITOR_CHARACTER_ANIMATION_ASSET_KIND,
caller.owner_user_id.as_str(),
"editor_background_removal",
task_id,
"animation_frame",
"background-removal",
),
)
.await?;
let output_object_key = put_result.object_key.clone();
let slot = editor_character_animation_frame_slot(frame_index);
let prepared_asset_object =
match prepare_editor_character_animation_frame_asset_object(
state,
caller,
task_id,
slot.as_str(),
output_object_key.clone(),
content_type,
frame_index + 1,
)
.await
{
Ok(result) => result,
Err(error) => {
// 当前帧已经上传但还没有进入成功结果集合;必须在本帧 future
// 内先清理,不能等全量 drain 后只清理其它成功帧。
cleanup_uncommitted_character_animation_object(
state,
output_object_key.as_str(),
"background_removal_frame_prepare_failed",
)
.await;
return Err(error);
}
};
Ok::<ProcessedEditorCharacterAnimationFrame, AppError>(
ProcessedEditorCharacterAnimationFrame {
frame_index,
payload: build_editor_character_animation_frame_payload(
put_result.legacy_public_path,
output_object_key.clone(),
prepared_asset_object.asset_object_id.clone(),
frame_width,
frame_height,
),
object_key: output_object_key,
prepared_asset_object,
slot,
},
)
}
};
uploaded_object_keys.push(put_result.object_key.clone());
let slot = editor_character_animation_frame_slot(frame_index);
let prepared_asset_object = match prepare_editor_character_animation_frame_asset_object(
state,
caller,
task_id,
slot.as_str(),
put_result.object_key.clone(),
content_type,
frame_index + 1,
)
.await
{
Ok(result) => result,
Err(error) => {
cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await;
return Err(error);
}
};
processed.push(ProcessedEditorCharacterAnimationFrame {
frame_index,
payload: build_editor_character_animation_frame_payload(
put_result.legacy_public_path,
put_result.object_key.clone(),
prepared_asset_object.asset_object_id.clone(),
frame_width,
frame_height,
),
object_key: put_result.object_key,
prepared_asset_object,
slot,
});
}
match build_persisted_editor_character_animation_frame_set(processed) {
Ok(result) => Ok(result),
Err(error) => {
cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await;
Err(error)
.await;
result.map_err(|error| (frame_index, error))
},
))
// 中文注释:去背景属于单个 durable job;帧在 job 内并行处理,且不能因首个
// 失败提前取消其它已经发出的帧,必须先 drain 完整批次,再统一清理并失败关闭。
.buffer_unordered(frame_count.max(1))
.collect::<Vec<_>>()
.await;
let mut processed = Vec::with_capacity(frame_count);
let mut frame_errors = Vec::new();
for frame_result in frame_results {
match frame_result {
Ok(frame_payload) => processed.push(frame_payload),
Err(frame_error) => frame_errors.push(frame_error),
}
}
processed.sort_by_key(|frame| frame.frame_index);
frame_errors.sort_by_key(|(frame_index, _)| *frame_index);
if let Some((frame_index, error)) = frame_errors.into_iter().next() {
tracing::warn!(
frame_index = frame_index + 1,
completed_frames = processed.len(),
expected_frames = frame_count,
error = %error,
"editor_animation_background_removal_failed_after_drain"
);
let uploaded_object_keys = processed
.iter()
.map(|frame| frame.object_key.clone())
.collect::<Vec<_>>();
cleanup_uncommitted_character_animation_objects(state, &uploaded_object_keys).await;
return Err(error);
}
build_persisted_editor_character_animation_frame_set(processed)
}
async fn process_and_persist_editor_character_animation_frame(
@@ -5631,6 +5665,17 @@ fn editor_inline_generation_caller<T: Serialize>(
})
}
fn editor_character_animation_background_removal_storage_task_id(
owner_user_id: &str,
operation_id: &str,
) -> String {
format!(
"{}-{}",
sanitize_storage_segment(owner_user_id, "owner"),
sanitize_storage_segment(operation_id, "task"),
)
}
fn editor_character_animation_frame_slot(frame_index: usize) -> String {
if frame_index == 0 {
EDITOR_CHARACTER_ANIMATION_FINAL_SLOT.to_string()
@@ -9936,6 +9981,42 @@ mod tests {
assert!(!frame_pipeline.contains("DownloadedOpenAiImage"));
}
#[test]
fn editor_character_animation_background_removal_storage_path_is_task_scoped() {
let first =
editor_character_animation_background_removal_storage_task_id("user-1", "task-1");
let replay =
editor_character_animation_background_removal_storage_task_id("user-1", "task-1");
let next_job =
editor_character_animation_background_removal_storage_task_id("user-1", "task-2");
assert_eq!(first, replay);
assert_ne!(first, next_job);
assert!(first.ends_with("-task-1"));
}
#[test]
fn editor_character_animation_background_removal_uses_local_green_screen_only() {
let source = include_str!("character_animation_assets.rs");
let body = source
.split_once("async fn extract_and_prepare_background_removal_frames")
.and_then(|(_, tail)| {
tail.split_once("async fn process_and_persist_editor_character_animation_frame")
})
.map(|(body, _)| body)
.expect("background removal frame function should exist");
assert!(body.contains("remove_editor_character_animation_green_screen_background"));
assert!(body.contains("download_editor_persisted_image_object"));
assert!(body.contains(".buffer_unordered(frame_count.max(1))"));
assert!(body.contains(".collect::<Vec<_>>()"));
assert!(body.contains("processed.sort_by_key"));
assert!(body.contains("frame_errors.sort_by_key"));
assert!(!body.contains("request_editor_background_removal_image_with_bgfilter_worker"));
assert!(!body.contains("remove_editor_generated_screen_background_with_bgfilter"));
assert!(!body.contains("ExternalApiAuditContext"));
}
#[test]
fn editor_character_animation_persists_one_final_asset_after_frame_extraction() {
let source = include_str!("character_animation_assets.rs");
@@ -208,6 +208,32 @@ pub(crate) fn remove_editor_generated_green_screen_background(
})
}
/// 角色动作手动去背景只处理约定的纯绿幕,不调用 BgFilter、阿里云或其它模型。
/// 不启用近白背景清理,避免把白色角色内容误当成背景;低饱和绿色仅作为绿幕边缘
/// 与压缩/抗锯齿造成的颜色变化处理。
pub(crate) fn remove_editor_character_animation_green_screen_background(
image: &DownloadedOpenAiImage,
) -> Result<DownloadedOpenAiImage, AppError> {
let bytes = remove_generated_asset_sheet_green_screen_background_bytes(
image.bytes.as_slice(),
GeneratedAssetSheetAlphaOptions {
key_color: GeneratedAssetSheetKeyColor::GREEN_SCREEN,
remove_near_white_background: false,
remove_disconnected_hard_key_background: true,
remove_muted_green_screen_background: true,
detect_internal_holes: true,
internal_hole_min_pixels: 16,
},
)
.map_err(map_editor_green_screen_error)?;
Ok(DownloadedOpenAiImage {
bytes,
mime_type: "image/png".to_string(),
extension: "png".to_string(),
})
}
fn map_editor_green_screen_error(error: GeneratedAssetSheetError) -> AppError {
let status = match error {
GeneratedAssetSheetError::DecodeImage { .. } => StatusCode::BAD_GATEWAY,
@@ -259,6 +285,30 @@ mod tests {
assert_eq!(image.get_pixel(1, 1).0[3], 255);
}
#[test]
fn character_animation_green_screen_removal_keeps_white_foreground() {
let mut source = RgbaImage::from_pixel(3, 3, Rgba([0, 255, 0, 255]));
source.put_pixel(1, 1, Rgba([255, 255, 255, 255]));
let mut bytes = Vec::new();
DynamicImage::ImageRgba8(source)
.write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png)
.expect("test PNG should encode");
let output =
remove_editor_character_animation_green_screen_background(&DownloadedOpenAiImage {
bytes,
mime_type: "image/png".to_string(),
extension: "png".to_string(),
})
.expect("green screen should be removed");
let image = image::load_from_memory(output.bytes.as_slice())
.expect("output PNG should decode")
.to_rgba8();
assert_eq!(image.get_pixel(0, 0).0[3], 0);
assert_eq!(image.get_pixel(1, 1).0, [255, 255, 255, 255]);
}
#[test]
fn editor_screen_background_color_parser_uses_default_and_rejects_unknown() {
assert_eq!(
@@ -231,7 +231,7 @@ const EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG: &str = "anime-seg";
const EDITOR_BGFILTER_MATTING_PROVIDER: &str = "BgFilter";
const EDITOR_ALIYUN_MATTING_PROVIDER: &str = "Aliyun Matting";
const EDITOR_ALIYUN_MATTING_MODEL: &str = "segment-common-image";
const EDITOR_LOCAL_MATTING_PROVIDER: &str = "Genarrative Local";
pub(crate) const EDITOR_LOCAL_MATTING_PROVIDER: &str = "Genarrative Local";
const EDITOR_LOCAL_MATTING_MODEL: &str = "screen-color-keying";
const EDITOR_MATTING_SOURCE_URL_EXPIRE_SECONDS: u64 = 600;
pub(crate) const EDITOR_BGFILTER_CROSS_CHECK_ENABLED: bool = true;
@@ -12721,7 +12721,7 @@ pub(crate) async fn read_editor_reference_image_object_with_client(
})
}
async fn download_editor_persisted_image_object(
pub(crate) async fn download_editor_persisted_image_object(
state: &AppState,
object_key: &str,
) -> Result<DownloadedOpenAiImage, AppError> {
@@ -27,7 +27,7 @@ const EDITOR_IMAGE_ASSET_KIND_CLEANUP_CANVAS_MAX_BATCH_SIZE: u32 = 5;
const EDITOR_LEGACY_IMAGE_ASSET_KIND: &str = "image";
// 正式生成结果沿用 master 的 66 项上限;图标拆分仍由独立的 64 项门禁约束。
const EDITOR_GENERATION_RESULT_MAX_ITEMS: usize = 66;
const EDITOR_GENERATION_OPERATION_KINDS: [&str; 11] = [
const EDITOR_GENERATION_OPERATION_KINDS: [&str; 12] = [
"editor_image_generation",
"editor_icon_spec_generation",
"editor_image_edit",
@@ -36,6 +36,7 @@ const EDITOR_GENERATION_OPERATION_KINDS: [&str; 11] = [
"editor_ui_design_asset_extraction",
"editor_character_animation_generation",
"editor_character_animation_video_conversion",
"editor_character_animation_background_removal",
"editor_video_generation",
"editor_sound_effect_generation",
"editor_background_music_generation",
@@ -15370,6 +15371,45 @@ mod tests {
}
}
#[test]
fn editor_generation_result_shape_accepts_character_animation_background_removal() {
let mut input = editor_generation_result_input();
let operation_kind = "editor_character_animation_background_removal";
let operation_id = "job-1";
input.operation_kind = operation_kind.to_string();
input.items[0]
.project_resource
.as_mut()
.expect("resource")
.resource_id = shared_contracts::editor_generation_stable_resource_id(
"user-1",
operation_kind,
operation_id,
"primary",
);
assert!(
validate_editor_generation_result_shape(
&input,
"user-1",
operation_kind,
operation_id,
)
.is_ok()
);
assert!(
validate_editor_generation_result_shape(
&input,
"user-1",
"editor_character_animation_background_removal_unknown",
operation_id,
)
.expect_err("unknown operation kinds must remain rejected")
.contains("不在编辑器生成白名单内")
);
}
#[test]
fn editor_generation_result_shape_requires_stable_ids_and_unique_slots() {
let input = editor_generation_result_input();
@@ -14,6 +14,7 @@ import {
closeGenerateComposerDialog,
createAudioRedrawGenerationDialogDraft,
createBackgroundMusicGenerationDialogDraft,
createCharacterAnimationBackgroundRemovalDialogDraft,
createCharacterAnimationGenerationDialogDraft,
createCharacterAnimationPanelDraft,
createCharacterGenerationDialogDraft,
@@ -559,6 +560,33 @@ describe('ImageCanvasGenerationDialogModel', () => {
});
});
it('creates a separate character animation placement for background removal', () => {
expect(
createCharacterAnimationBackgroundRemovalDialogDraft({
sourceLayer: createLayer({
id: 'layer-animation',
assetKind: 'character-animation',
}),
assetLabel: '源动画 去背景',
}),
).toMatchObject({
mode: 'character-animation',
prompt: '去除背景',
assetLabel: '源动画 去背景',
status: 'generating',
composerOpen: false,
sourceLayerId: 'layer-animation',
placeholder: {
x: 472,
y: 140,
width: 320,
height: 240,
originalWidth: 1024,
originalHeight: 768,
},
});
});
it('restores remodel prompts from user-visible generation input snapshots only', () => {
const cases: Array<{
fields: NonNullable<CanvasLayer['generationInputs']>['fields'];
@@ -869,9 +869,7 @@ function getNormalizedString(
function getNormalizedStringArray(sourceLayer: CanvasLayer, id: string) {
return (sourceLayer.generationInputs?.fields ?? [])
.filter((field) => field.id === id)
.flatMap((field) =>
typeof field.value === 'string' ? [field.value] : [],
);
.flatMap((field) => (typeof field.value === 'string' ? [field.value] : []));
}
function getNormalizedNumber(
@@ -2196,6 +2194,33 @@ export function createQuickEditGenerationDialogDraft({
};
}
export function createCharacterAnimationBackgroundRemovalDialogDraft({
sourceLayer,
assetLabel,
status = 'generating',
}: {
sourceLayer: CanvasLayer;
assetLabel?: string;
status?: CanvasGenerationDialogState['status'];
}): Omit<CanvasGenerationDialogState, 'id'> {
return {
mode: 'character-animation',
prompt: '去除背景',
assetLabel,
status,
composerOpen: false,
sourceLayerId: sourceLayer.id,
placeholder: {
x: sourceLayer.x + sourceLayer.width + 32,
y: sourceLayer.y,
width: sourceLayer.width,
height: sourceLayer.height,
originalWidth: sourceLayer.originalWidth || sourceLayer.width,
originalHeight: sourceLayer.originalHeight || sourceLayer.height,
},
};
}
export function createLegacyImageRedrawGenerationDialogDraft({
sourceLayer,
canvasSize,
@@ -67,6 +67,7 @@ import {
closeGenerateComposerDialog,
createAudioRedrawGenerationDialogDraft,
createBackgroundMusicGenerationDialogDraft,
createCharacterAnimationBackgroundRemovalDialogDraft,
createCharacterAnimationGenerationDialogDraft,
createCharacterGenerationDialogDraft,
createEditDialogDraft,
@@ -3697,13 +3698,33 @@ export function useImageCanvasGenerationWorkflow({
setMetadataLayer(null);
setCropExpandPanel(null);
setQuickEditPanel(null);
const assetLabel = `${sourceLayer.title} 去背景`;
const backgroundRemovalDialog = applyProjectSnapshot
? createCharacterAnimationBackgroundRemovalDialogDraft({
sourceLayer,
assetLabel,
})
: null;
const backgroundRemovalPlacement = backgroundRemovalDialog
? openPlacedCanvasGenerationDialog(backgroundRemovalDialog)
: undefined;
const backgroundRemovalDialogId = backgroundRemovalPlacement?.dialogId;
try {
const result = await removeEditorCharacterAnimationBackground({
projectId,
sourceLayerId: sourceLayer.id,
sourceResourceId: sourceLayer.resourceId,
assetFolderId,
assetLabel: `${sourceLayer.title} 去背景`,
assetLabel,
...(backgroundRemovalPlacement?.placeholder
? {
canvasCompletion: {
dialogId: backgroundRemovalDialogId,
title: assetLabel,
placeholder: backgroundRemovalPlacement.placeholder,
},
}
: {}),
generationInputs: buildDeterministicGenerationInputs(
'character-animation.remove-background',
sourceLayer,
@@ -3712,10 +3733,17 @@ export function useImageCanvasGenerationWorkflow({
await applyQueuedEditorGenerationProject(
result,
projectId,
applyProjectSnapshot,
(project) =>
applyProjectSnapshot?.(project, {
type: 'remove-background',
count: 1,
}),
refreshTaskListForQueuedGeneration,
onWalletBalanceMayHaveChanged,
showGenerationWarning,
backgroundRemovalDialogId,
(project) =>
preserveSourceLayerInProjectSnapshot(project, sourceLayer),
);
try {
await Promise.resolve(refreshAssetLibrary?.());
@@ -3723,6 +3751,19 @@ export function useImageCanvasGenerationWorkflow({
// 服务端已原子落库成功;列表刷新失败不能把成功事实伪装成去背景失败。
}
} catch (error) {
if (backgroundRemovalDialogId) {
updateCanvasGenerationDialogById(
backgroundRemovalDialogId,
(dialog) => ({
...dialog,
status: 'failed',
errorMessage:
error instanceof Error && error.message.trim()
? error.message
: '去除序列帧背景失败',
}),
);
}
window.alert(
error instanceof Error && error.message.trim()
? error.message
@@ -3747,6 +3788,7 @@ export function useImageCanvasGenerationWorkflow({
assetFolderId,
closeGenerationTransientState,
onWalletBalanceMayHaveChanged,
openPlacedCanvasGenerationDialog,
projectId,
refreshAssetLibrary,
refreshTaskListForQueuedGeneration,
@@ -3755,6 +3797,7 @@ export function useImageCanvasGenerationWorkflow({
setMetadataLayer,
setQuickEditPanel,
showGenerationWarning,
updateCanvasGenerationDialogById,
],
);