优化角色动作BgFilter并发抠图链路

复用BgFilter HTTP连接池并为单帧失败增加一次重试
将全部动作帧改为无序连续在途并在完成后恢复帧序
按单帧源图上传、抠图和透明帧上传组成流水线并排空失败批次
为角色动作开启cross-check并同步后端文档与决策记录
This commit is contained in:
2026-07-13 09:04:20 +00:00
parent a96f84be62
commit 20600c75c7
6 changed files with 295 additions and 189 deletions
@@ -16,6 +16,22 @@
---
## 2026-07-13 角色动作逐帧开启 BgFilter cross-check
- 背景:角色动作逐帧抠图此前为减少额外推理开销固定传 `cross_check=off`,但动作帧同样需要保留发丝、镂空和运动边缘质量。
- 决策:角色动作逐帧 BgFilter 请求固定显式传 `cross_check=on`,与角色形象保持一致;图标 spritesheet 和 UI 设计图素材提取继续固定传 `off`。该策略仍属于后端内部供应商参数,不进入前端或外部 OpenAPI。
- 影响范围:`server-rs/crates/api-server/src/editor_project.rs``server-rs/crates/api-server/src/character_animation_assets.rs`、后端架构文档和图片画布技术文档。
- 验证方式:运行 `cargo test -p api-server editor_bgfilter_cross_check --manifest-path server-rs/Cargo.toml``cargo test -p api-server editor_character_animation_frames_use_three_stage_matting_fallback --manifest-path server-rs/Cargo.toml``cargo check -p api-server --manifest-path server-rs/Cargo.toml``npm run check:encoding``git diff --check`
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
## 2026-07-13 角色动作 BgFilter 全帧流水线与单次重试
- 背景:角色动作抽帧后原先固定 `buffered(3)`,并在整批绿幕源帧串行落 OSS 后才开始抠图;每帧还单独创建 HTTP Client。公网 BgFilter 的网络等待会让服务端推理队列出现空档,且首个最终错误会通过 `try_collect` 提前取消 api-server 中其余已发 Future。
- 决策:BgFilter HTTP Client 在 `AppState` 中统一创建并复用 keep-alive 连接池;每次 BgFilter 调用失败后立即重试 `1` 次,两次都失败才进入既有“阿里云通用抠图 → 本地键色”降级链,每次已发失败调用都保留审计。角色动作全部 `32 / 40 / 48` 帧按“单帧绿幕源图落 OSS → BgFilter/降级 → 透明帧落 OSS”独立流水化,使用覆盖本次全部帧的 `buffer_unordered` 连续发射并携带原始帧序,完成后排序;不在 api-server 新增供应商进程锁或全局 Semaphore。任一帧最终失败时先排空全部已启动 Future,再让整个动作任务失败退款,不发布缺帧动画。
- 影响范围:`server-rs/crates/api-server/src/state.rs``server-rs/crates/api-server/src/editor_project.rs``server-rs/crates/api-server/src/character_animation_assets.rs`、后端架构文档和图片画布技术文档。
- 验证方式:运行 `cargo test -p api-server editor_bgfilter_retries_once_before_fallback --manifest-path server-rs/Cargo.toml``cargo test -p api-server editor_character_animation_frames_use_three_stage_matting_fallback --manifest-path server-rs/Cargo.toml``cargo test -p api-server editor_canvas_screen_background_generation_uses_bgfilter_postprocess --manifest-path server-rs/Cargo.toml``cargo check -p api-server --manifest-path server-rs/Cargo.toml``npm run check:encoding``git diff --check`
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
## 2026-07-12 泥点充值收敛为四档并统一资产入口
- 背景:主站与图片画板的泥点余额入口、余额明细和充值弹窗存在不同实现,旧充值口径仍展示六档泥点、首充双倍和会员购买 / 升级入口,容易让展示、商品资格与后端余额真相发生漂移。
@@ -32,10 +48,10 @@
- 验证方式:`npm run spacetime:generate``npm run check:spacetime-schema`、钱包定向 Rust 测试、个人中心定向前端测试、`npm run typecheck``npm run check:encoding``git diff --check`
- 关联文档:`docs/【项目基线】当前产品与工程约束-2026-05-15.md``docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`
## 2026-07-11 BgFilter 交叉模型否决用于角色形象
## 2026-07-11 BgFilter 交叉模型否决用于角色形象与角色动作
- 背景:新版 BgFilter 的 `cross_check` 默认开启,会额外运行 HR-matting 第二意见模型;角色形象需要保留发丝、镂空等复杂边缘质量,但角色动作序列帧、图标 spritesheet 和 UI 素材提取不需要承担这部分额外推理开销。
- 决策:api-server 调用 BgFilter 时必须显式发送 multipart 字段 `cross_check`,不依赖服务端默认值。角色形象生成固定传 `on`角色动作逐帧去背、图标 spritesheet 生成和 UI 设计图素材提取固定传 `off`。角色动作逐帧去背与三条静态生图路线复用同一条 `BgFilter → 阿里云通用抠图 → 本地键色` 降级链和同一 BgFilter 熔断器。该字段是后端内部供应商策略,不进入前端请求或外部 OpenAPI。
- 背景:新版 BgFilter 的 `cross_check` 默认开启,会额外运行 HR-matting 第二意见模型;角色形象与角色动作序列帧需要保留发丝、镂空和运动边缘质量图标 spritesheet 和 UI 素材提取不需要承担这部分额外推理开销。
- 决策:api-server 调用 BgFilter 时必须显式发送 multipart 字段 `cross_check`,不依赖服务端默认值。角色形象生成和角色动作逐帧去背固定传 `on`;图标 spritesheet 生成和 UI 设计图素材提取固定传 `off`。角色动作逐帧去背与三条静态生图路线复用同一条 `BgFilter → 阿里云通用抠图 → 本地键色` 降级链和同一 BgFilter 熔断器。该字段是后端内部供应商策略,不进入前端请求或外部 OpenAPI。
- 影响范围:`server-rs/crates/api-server/src/editor_project.rs``server-rs/crates/api-server/src/character_animation_assets.rs`、图片画布 BgFilter 调用文档。
- 验证方式:运行 `cargo test -p api-server editor_bgfilter_cross_check --manifest-path server-rs/Cargo.toml``cargo test -p api-server editor_canvas_screen_background_generation_uses_bgfilter_postprocess --manifest-path server-rs/Cargo.toml``cargo test -p api-server editor_character_animation_frames_use_three_stage_matting_fallback --manifest-path server-rs/Cargo.toml``cargo check -p api-server --manifest-path server-rs/Cargo.toml``npm run check:encoding``git diff --check`
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
@@ -75,7 +91,7 @@
## 2026-07-09 角色动作视频生成背景色统一为多色自动决策 + 阿里云抠帧
- 背景:角色动作视频抽帧过去固定 legacy `#00FF00` 绿幕 + 本地 `editor_green_screen`,与生图链路的多色自动决策不一致;实测出现背景色与前景 / 皮肤撞色(蓝撞蓝、桃 / 黄撞肤色)以及图生视频背景变白的问题。
- 决策:角色动作视频背景色与生图统一。`screenColor=auto` 时由视觉 LLM`gpt-5-mini`Responses 协议、`reasoning_effort=low``max_tokens=1024`)读源角色图自动决策,并经硬过滤器(Lab 危险质量 + 皮肤专属三判据:ΔE 距离 / 色调投影 / RGB 分离)剔除与前景及皮肤撞色的候选,手动 hex 仍尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景确定性等于抠图键色。抽帧后逐帧优先走 BgFilter(固定 `seg_model=birefnet``cross_check=off`),失败依次降级阿里云通用抠图和本地 `editor_green_screen` 键色兜底(按生成时选定的背景色,而非固定 `#00FF00`)。BgFilter 与阿里云抠图失败均写入 `external_api_call_failure` 失败审计。调色板新增中明度低饱和「灰竹绿 `#A0BBA0`」补齐冷区绿色段。
- 决策:角色动作视频背景色与生图统一。`screenColor=auto` 时由视觉 LLM`gpt-5-mini`Responses 协议、`reasoning_effort=low``max_tokens=1024`)读源角色图自动决策,并经硬过滤器(Lab 危险质量 + 皮肤专属三判据:ΔE 距离 / 色调投影 / RGB 分离)剔除与前景及皮肤撞色的候选,手动 hex 仍尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景确定性等于抠图键色。抽帧后逐帧优先走 BgFilter(固定 `seg_model=birefnet``cross_check=on`),失败依次降级阿里云通用抠图和本地 `editor_green_screen` 键色兜底(按生成时选定的背景色,而非固定 `#00FF00`)。BgFilter 与阿里云抠图失败均写入 `external_api_call_failure` 失败审计。调色板新增中明度低饱和「灰竹绿 `#A0BBA0`」补齐冷区绿色段。
- 影响范围:`server-rs/crates/api-server/src/character_animation_assets.rs``editor_screen_background_decision.rs``editor_screen_background_filter.rs`(新增硬过滤模块)、`editor_green_screen.rs`(调色板)、`external_api_audit.rs``llm_model_routing.rs`、图片画布 MVP 与后端数据契约文档。
- 验证方式:`cargo test -p api-server editor_screen_background character_animation --manifest-path server-rs/Cargo.toml``cargo check -p api-server --manifest-path server-rs/Cargo.toml`、真机对源角色图跑视觉决策与候选危险度表、抽帧后采样序列帧背景色确认落在冷区安全集。
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
File diff suppressed because one or more lines are too long
@@ -230,7 +230,8 @@ npm run check:server-rs-ddd
- LLM:通用 LLM 门面继续使用 `GENARRATIVE_LLM_*`;创意 Agent `gpt-5.4-mini` Chat Completions 文本链路已于 2026-06 从 APIMart 迁移到 VectorEngine,使用 `VECTOR_ENGINE_BASE_URL` / `VECTOR_ENGINE_API_KEY` 构造 OpenAI-compatible client`api-server` 会把未带 `/v1` 的 VectorEngine base URL 规范化到 `/v1` 后请求 `/chat/completions`。通用 `/api/llm/chat/completions` 代理使用 `GENARRATIVE_LLM_PROVIDER=openai-compatible``GENARRATIVE_LLM_BASE_URL=https://api.vectorengine.cn/v1``GENARRATIVE_LLM_MODEL=gpt-5.4-mini`;未单独配置 `GENARRATIVE_LLM_API_KEY` 时可复用 `VECTOR_ENGINE_API_KEY``APIMART_BASE_URL` / `APIMART_API_KEY` 只作为历史残留,不再作为创意 Agent gpt-5.4-mini 客户端来源;后续排障时优先确认 VectorEngine `/v1/models``/v1/chat/completions``/v1/responses` 可用性。
- 图片生成:VectorEngine `gpt-image-2` 图片 provider 归属 `platform-image`,密钥只在后端环境变量中;`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接,不再承载 provider 协议实现。实际外部生成运行记录统一落 `tracking_event``event_key = external_generation_run`metadata 记录开始 / 结束时间、耗时、状态、成功标记、失败原因、provider task id 和结果摘要,不再写回过时的 `ai_task`。DashScope 只按仍在使用的历史能力单独处理,不作为 GPT-image-2 兜底。VectorEngine `/v1/images/generations``/v1/images/edits` 上游 POST 使用 `libcurl` 发送;`reqwest` 只保留给参考图 URL 下载和响应中图片 URL 下载。`/v1/images/edits` 的 multipart 参考图必须作为 libcurl 文件上传 part 发送,字段名为 `image`,实现上使用 `Form::buffer(file_name, bytes)` 并设置 `Content-Type`;不能只用 `contents(...).filename(...)`,否则上游会把请求转码为缺少图片并返回 `image is required``request_send` 阶段的 curl timeout / connect error 按可重试传输错误处理,最多尝试 5 次,并使用指数退避加短抖动;排障时优先看 `attempt``max_attempts``retry_delay_ms``reference_image_bytes_total``request_params`,不要把 `SendRequest` 当成上游业务错误。
- 编辑器抠图服务:手动 `POST /api/editor/images/background-removals` 继续代理独立 BiRefNet 服务,配置为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL``GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN``GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS`。角色形象生成、图标 spritesheet 生成、UI 设计图素材提取和角色动作抽帧后的纯色背景透明化统一走独立 BgFilter 服务,配置为 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL``GENARRATIVE_EDITOR_BGFILTER_TOKEN``GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS`,默认 base URL 为 `http://58.87.105.82/bgfilter`,默认请求超时为 `180000ms`BgFilter 当前为 CPU 推理,单次抠图较慢,必须留足超时),token 未配置时复用 BiRefNet token。BgFilter 请求必须显式传 `screen_color=<screenColor>``seg_model=<segModel>``cross_check=<on|off>`;角色形象生成固定传 `cross_check=on`角色动作逐帧去背、图标 spritesheet 生成和 UI 设计图素材提取固定传 `cross_check=off`,不依赖服务端默认值。前端用户路径不展示抠图模型选择并固定提交默认 `birefnet`,后端仍识别内部保留的 `anime-seg`,其中 `birefnet` 只表示 BgFilter 管线内部后端,不等同于手动去背景的独立 BiRefNet 服务;`cross_check` 同样只属于后端内部供应商策略,不进入前端或外部 OpenAPI。BgFilter 调用失败,或连续失败达到 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_FAILURE_THRESHOLD`(默认 `3`)并在 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_COOLDOWN_SECONDS`(默认 `300`)内打开熔断时,四条路线均跳过或结束 BgFilter 调用后复用同一兜底链:先调用阿里云通用抠图,阿里云失败才使用本地 `editor_green_screen` 键色扣除;熔断期不得直接退化到本地兜底。角色动作视频生成的背景色已与生图链路统一:`screenColor=auto` 时由视觉 LLM`gpt-5-mini`Responses 协议、low 推理档)读源角色图自动决策,并经硬过滤器剔除与前景 / 皮肤撞色的候选,手动 hex 则尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景等于抠图键色;抽帧后逐帧固定使用 `seg_model=birefnet``cross_check=off` 进入上述三段式链路。阿里云通用抠图配置为 `GENARRATIVE_ALIYUN_MATTING_ENABLED``GENARRATIVE_ALIYUN_MATTING_ENDPOINT``GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_ID``GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_SECRET``GENARRATIVE_ALIYUN_MATTING_REQUEST_TIMEOUT_MS`;未配置专用 AK/SK 时可复用 `ALIBABA_CLOUD_ACCESS_KEY_ID` / `ALIBABA_CLOUD_ACCESS_KEY_SECRET`,默认 endpoint 为 `imageseg.cn-shanghai.aliyuncs.com`。BgFilter 与阿里云抠图失败都写入 `external_api_call_failure` 审计。
- 编辑器抠图服务:手动 `POST /api/editor/images/background-removals` 继续代理独立 BiRefNet 服务,配置为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL``GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN``GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS`。角色形象生成、图标 spritesheet 生成、UI 设计图素材提取和角色动作抽帧后的纯色背景透明化统一走独立 BgFilter 服务,配置为 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL``GENARRATIVE_EDITOR_BGFILTER_TOKEN``GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS`,默认 base URL 为 `http://58.87.105.82/bgfilter`,默认请求超时为 `180000ms`BgFilter 当前为 CPU 推理,单次抠图较慢,必须留足超时),token 未配置时复用 BiRefNet token。BgFilter 请求必须显式传 `screen_color=<screenColor>``seg_model=<segModel>``cross_check=<on|off>`;角色形象生成和角色动作逐帧去背固定传 `cross_check=on`,图标 spritesheet 生成和 UI 设计图素材提取固定传 `cross_check=off`,不依赖服务端默认值。前端用户路径不展示抠图模型选择并固定提交默认 `birefnet`,后端仍识别内部保留的 `anime-seg`,其中 `birefnet` 只表示 BgFilter 管线内部后端,不等同于手动去背景的独立 BiRefNet 服务;`cross_check` 同样只属于后端内部供应商策略,不进入前端或外部 OpenAPI。BgFilter 调用失败,或连续失败达到 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_FAILURE_THRESHOLD`(默认 `3`)并在 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_COOLDOWN_SECONDS`(默认 `300`)内打开熔断时,四条路线均跳过或结束 BgFilter 调用后复用同一兜底链:先调用阿里云通用抠图,阿里云失败才使用本地 `editor_green_screen` 键色扣除;熔断期不得直接退化到本地兜底。角色动作视频生成的背景色已与生图链路统一:`screenColor=auto` 时由视觉 LLM`gpt-5-mini`Responses 协议、low 推理档)读源角色图自动决策,并经硬过滤器剔除与前景 / 皮肤撞色的候选,手动 hex 则尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景等于抠图键色;抽帧后逐帧固定使用 `seg_model=birefnet``cross_check=on` 进入上述三段式链路。阿里云通用抠图配置为 `GENARRATIVE_ALIYUN_MATTING_ENABLED``GENARRATIVE_ALIYUN_MATTING_ENDPOINT``GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_ID``GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_SECRET``GENARRATIVE_ALIYUN_MATTING_REQUEST_TIMEOUT_MS`;未配置专用 AK/SK 时可复用 `ALIBABA_CLOUD_ACCESS_KEY_ID` / `ALIBABA_CLOUD_ACCESS_KEY_SECRET`,默认 endpoint 为 `imageseg.cn-shanghai.aliyuncs.com`。BgFilter 与阿里云抠图失败都写入 `external_api_call_failure` 审计。
- BgFilter 连接复用、重试与动作帧流水线:api-server 必须在 `AppState` 复用同一个 BgFilter HTTP Client 及 keep-alive 连接池。单次 BgFilter 调用失败后立即重试 `1` 次,第二次仍失败才进入既有“阿里云通用抠图 → 本地键色”降级链,每次已发出的失败调用都单独写审计。角色动作全部 `32 / 40 / 48` 帧按“单帧绿幕源图先落 OSS → BgFilter/降级 → 透明帧落 OSS”独立流水化,使用覆盖本次全部帧的无序在途集合连续发射,不在 api-server 增加供应商进程锁或固定小并发窗口;返回结果携带原始帧序并在收口时排序。任一帧最终失败时必须先排空全部已启动 Future,再让整个动作任务失败退款,不能发布缺帧动画。
- Match3D 物品 sheet:关卡整图完成后走 VectorEngine `/v1/images/edits` multipart `image`,模型为 `gpt-image-2``2K 1:1` 输出 `10*10` spritesheet;物品 sheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG,并把透明整图写入 `itemSpritesheetImageSrc/itemSpritesheetImageObjectKey`。后端优先按透明 alpha 连通域从该 sheet 识别真实素材矩形并持久化 20 个物品、每个 5 个形态;识别数量不足时才回退 `10*10` 固定网格。通用系列素材图集的行列索引按每行 2 个物品计算,必须落在 `1..=10`,难度只决定运行态加载 3 / 9 / 15 / 20 种。
- Match3D UI spritesheet 和背景派生图:关卡整图作为参考图并发生成 `1K 1:1` UI spritesheet 与 `1K 9:16` 背景图,模型均为 `gpt-image-2`。UI spritesheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG;背景图必须合成为全画幅不透明 PNG。
- Match3D 1:1 容器 UIVectorEngine `/v1/images/edits` multipart 参考图。该容器参考图是后端生图协议输入,必须通过 `include_bytes!``api-server` 编译进二进制,避免 API 单独发布或运行目录缺少 `public/` 时生成失败。
@@ -64,7 +64,7 @@ use crate::{
EditorScreenBackgroundColor, editor_green_screen_character_prompt_clause,
},
editor_project::{
EDITOR_BGFILTER_CROSS_CHECK_DISABLED, EDITOR_BGFILTER_DEFAULT_SEG_MODEL,
EDITOR_BGFILTER_CROSS_CHECK_ENABLED, EDITOR_BGFILTER_DEFAULT_SEG_MODEL,
EditorCanvasGeneratedLayerInput, PersistEditorGeneratedAssetRequest,
apply_editor_screen_background_decision_to_generation_inputs,
build_editor_canvas_generated_layer_item, complete_editor_canvas_generation_with_items,
@@ -2132,138 +2132,151 @@ async fn extract_and_persist_editor_character_animation_frames(
Some(request.duration_seconds as f64),
)
.await?;
persist_editor_character_animation_green_screen_source_frames(
state,
owner_user_id,
source_layer_id,
task_id,
&finalized_frames,
)
.await?;
let finalized_frames = remove_editor_character_animation_frame_backgrounds(
state,
finalized_frames,
request.frame_width,
request.frame_height,
request.screen_color,
audit,
)
.await?;
use futures_util::StreamExt as _;
let mut frame_payloads = Vec::with_capacity(finalized_frames.len());
for (index, frame) in finalized_frames.into_iter().enumerate() {
let put_result = put_character_animation_object(
state,
LegacyAssetPrefix::Animations,
vec![
"editor".to_string(),
sanitize_storage_segment(source_layer_id, "layer"),
task_id.to_string(),
],
format!("frame{:02}.{}", index + 1, frame.extension),
frame.mime_type,
frame.bytes,
build_asset_metadata(
EDITOR_CHARACTER_ANIMATION_ASSET_KIND,
let frame_count = finalized_frames.len();
let frame_results = futures_util::stream::iter(finalized_frames.into_iter().enumerate().map(
|(frame_index, frame)| async move {
process_and_persist_editor_character_animation_frame(
state,
owner_user_id,
"editor_layer",
source_layer_id,
"animation_frame",
"editor-character-animation",
),
)
.await?;
frame_payloads.push(EditorCharacterAnimationFramePayload {
frame_index: index as u32 + 1,
image_src: put_result.legacy_public_path,
width: request.frame_width,
height: request.frame_height,
});
task_id,
frame_index,
frame,
request.frame_width,
request.frame_height,
request.screen_color,
audit,
)
.await
.map_err(|error| (frame_index, error))
},
))
// 中文注释:动作帧上限固定为 48。全部帧连续进入在途集合,由 BgFilter 服务端既有进程锁自行排队;
// api-server 不再等待前一帧返回,也不因返回乱序产生队头阻塞。
.buffer_unordered(frame_count.max(1))
// 中文注释:不能 try_collect 提前取消。已经发出的请求必须全部排空,避免服务端完成计算后无人接收。
.collect::<Vec<_>>()
.await;
let mut frame_payloads = Vec::with_capacity(frame_count);
let mut frame_errors = Vec::new();
for frame_result in frame_results {
match frame_result {
Ok(frame_payload) => frame_payloads.push(frame_payload),
Err(frame_error) => frame_errors.push(frame_error),
}
}
frame_payloads.sort_by_key(|frame| frame.frame_index);
frame_errors.sort_by_key(|(frame_index, _)| *frame_index);
if let Some((frame_index, error)) = frame_errors.into_iter().next() {
tracing::warn!(
frame_index = frame_index + 1,
completed_frames = frame_payloads.len(),
expected_frames = frame_count,
error = %error,
"editor_animation_frame_pipeline_failed_after_drain"
);
return Err(error);
}
Ok(frame_payloads)
}
async fn persist_editor_character_animation_green_screen_source_frames(
async fn process_and_persist_editor_character_animation_frame(
state: &AppState,
owner_user_id: &str,
source_layer_id: &str,
task_id: &str,
frames: &[FinalizedAnimationFrame],
) -> Result<(), AppError> {
for (index, frame) in frames.iter().enumerate() {
put_character_animation_object(
state,
LegacyAssetPrefix::Animations,
vec![
"editor".to_string(),
sanitize_storage_segment(source_layer_id, "layer"),
task_id.to_string(),
],
format!("green-screen-frame{:02}.{}", index + 1, frame.extension),
frame.mime_type.clone(),
frame.bytes.clone(),
build_asset_metadata(
EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND,
owner_user_id,
"editor_layer",
source_layer_id,
EDITOR_GREEN_SCREEN_SOURCE_SLOT,
"editor-character-animation",
),
)
.await?;
}
Ok(())
}
/// 逐帧抠图:优先 BgFilter(关闭 cross-check),失败依次降级阿里云通用抠图和本地键色。
/// 小并发保序处理,单帧降级不影响其他帧。
async fn remove_editor_character_animation_frame_backgrounds(
state: &AppState,
frames: Vec<FinalizedAnimationFrame>,
frame_index: usize,
frame: FinalizedAnimationFrame,
frame_width: u32,
frame_height: u32,
screen_color: EditorScreenBackgroundColor,
audit: &crate::external_api_audit::ExternalApiAuditContext,
) -> Result<Vec<FinalizedAnimationFrame>, AppError> {
use futures_util::{StreamExt as _, TryStreamExt as _};
) -> Result<EditorCharacterAnimationFramePayload, AppError> {
// 中文注释:每一帧只要求自己的绿幕源图先落 OSS,不再等待整批源图全部上传完成。
put_character_animation_object(
state,
LegacyAssetPrefix::Animations,
vec![
"editor".to_string(),
sanitize_storage_segment(source_layer_id, "layer"),
task_id.to_string(),
],
format!(
"green-screen-frame{:02}.{}",
frame_index + 1,
frame.extension
),
frame.mime_type.clone(),
frame.bytes.clone(),
build_asset_metadata(
EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND,
owner_user_id,
"editor_layer",
source_layer_id,
EDITOR_GREEN_SCREEN_SOURCE_SLOT,
"editor-character-animation",
),
)
.await?;
const FRAME_MATTING_CONCURRENCY: usize = 3;
let image = DownloadedOpenAiImage {
bytes: frame.bytes,
mime_type: frame.mime_type,
extension: frame.extension,
};
let removed = remove_editor_generated_screen_background_with_bgfilter(
state,
&image,
screen_color,
EDITOR_BGFILTER_DEFAULT_SEG_MODEL,
EDITOR_BGFILTER_CROSS_CHECK_ENABLED,
audit,
)
.await?;
tracing::debug!(
frame_index,
screen_color = screen_color.hex,
"editor_animation_frame_background_removed"
);
let finalized = finalize_animation_frame_payload(
removed.bytes.as_slice(),
removed.mime_type.as_str(),
frame_width,
frame_height,
false,
)?;
let put_result = put_character_animation_object(
state,
LegacyAssetPrefix::Animations,
vec![
"editor".to_string(),
sanitize_storage_segment(source_layer_id, "layer"),
task_id.to_string(),
],
format!("frame{:02}.{}", frame_index + 1, finalized.extension),
finalized.mime_type,
finalized.bytes,
build_asset_metadata(
EDITOR_CHARACTER_ANIMATION_ASSET_KIND,
owner_user_id,
"editor_layer",
source_layer_id,
"animation_frame",
"editor-character-animation",
),
)
.await?;
futures_util::stream::iter(frames.into_iter().enumerate().map(
|(frame_index, frame)| async move {
let image = DownloadedOpenAiImage {
bytes: frame.bytes,
mime_type: frame.mime_type,
extension: frame.extension,
};
let removed = remove_editor_generated_screen_background_with_bgfilter(
state,
&image,
screen_color,
EDITOR_BGFILTER_DEFAULT_SEG_MODEL,
EDITOR_BGFILTER_CROSS_CHECK_DISABLED,
audit,
)
.await?;
tracing::debug!(
frame_index,
screen_color = screen_color.hex,
"editor_animation_frame_background_removed"
);
finalize_animation_frame_payload(
removed.bytes.as_slice(),
removed.mime_type.as_str(),
frame_width,
frame_height,
false,
)
},
))
.buffered(FRAME_MATTING_CONCURRENCY)
.try_collect::<Vec<_>>()
.await
Ok(EditorCharacterAnimationFramePayload {
frame_index: frame_index as u32 + 1,
image_src: put_result.legacy_public_path,
width: frame_width,
height: frame_height,
})
}
async fn publish_animation_set(
@@ -5685,38 +5698,37 @@ mod tests {
"async fn publish_animation_set",
&[
"apply_chroma_key: false",
"persist_editor_character_animation_green_screen_source_frames",
"remove_editor_character_animation_frame_backgrounds",
"process_and_persist_editor_character_animation_frame",
".buffer_unordered(frame_count.max(1))",
".collect::<Vec<_>>()",
"frame_payloads.sort_by_key",
"frame_errors.sort_by_key",
],
);
assert_function_contains_in_order(
source,
"async fn extract_and_persist_editor_character_animation_frames",
"async fn process_and_persist_editor_character_animation_frame",
"async fn publish_animation_set",
&[
"persist_editor_character_animation_green_screen_source_frames",
"remove_editor_character_animation_frame_backgrounds",
"put_character_animation_object",
"green-screen-frame",
"remove_editor_generated_screen_background_with_bgfilter",
"finalize_animation_frame_payload",
"put_character_animation_object",
"animation_frame",
],
);
assert_function_contains(
source,
"async fn persist_editor_character_animation_green_screen_source_frames",
"fn remove_editor_character_animation_frame_backgrounds",
"async fn process_and_persist_editor_character_animation_frame",
"async fn publish_animation_set",
&[
"EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND",
"EDITOR_GREEN_SCREEN_SOURCE_SLOT",
"green-screen-frame",
"put_character_animation_object",
],
);
assert_function_contains(
source,
"fn remove_editor_character_animation_frame_backgrounds",
"async fn publish_animation_set",
&[
"remove_editor_generated_screen_background_with_bgfilter",
"EDITOR_BGFILTER_DEFAULT_SEG_MODEL",
"EDITOR_BGFILTER_CROSS_CHECK_DISABLED",
"EDITOR_BGFILTER_CROSS_CHECK_ENABLED",
"finalize_animation_frame_payload",
],
);
+102 -52
View File
@@ -115,8 +115,9 @@ const EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND: &str = "editor_green_screen_source"
const EDITOR_GREEN_SCREEN_SOURCE_SLOT: &str = "green_screen_source";
pub(crate) const EDITOR_BGFILTER_DEFAULT_SEG_MODEL: &str = "birefnet";
const EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG: &str = "anime-seg";
const EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK: bool = true;
pub(crate) const EDITOR_BGFILTER_CROSS_CHECK_ENABLED: bool = true;
pub(crate) const EDITOR_BGFILTER_CROSS_CHECK_DISABLED: bool = false;
const EDITOR_BGFILTER_RETRY_COUNT: usize = 1;
const EDITOR_PUBLICATION_MATERIAL_ASSET_KIND: &str = "editor_publication_material";
const EDITOR_LEGACY_INLINE_IMAGE_ASSET_KIND: &str = "editor_legacy_inline_image";
@@ -1587,7 +1588,7 @@ pub(crate) async fn generate_editor_image_for_owner(
&image,
screen_color.expect("character generation should have screen color"),
seg_model.expect("character generation should have BgFilter seg model"),
EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK,
EDITOR_BGFILTER_CROSS_CHECK_ENABLED,
&matting_audit,
)
.await?;
@@ -2412,47 +2413,73 @@ pub(crate) async fn remove_editor_generated_screen_background_with_bgfilter(
return fallback_editor_screen_background_removal(state, image, screen_color, audit).await;
}
match request_editor_generated_screen_background_with_bgfilter(
state,
image,
screen_color,
seg_model,
cross_check,
)
.await
{
Ok(image) => {
record_editor_bgfilter_success();
Ok(image)
}
Err(error) => {
record_editor_bgfilter_failure(state);
tracing::warn!(
provider = "bgfilter",
screen_color = screen_color.hex,
seg_model,
cross_check,
error = %error,
error_details = ?error.details(),
"editor_bgfilter_fallback_to_aliyun_matting"
);
crate::external_api_audit::record_matting_external_api_failure(
state,
audit,
"bgfilter",
state.config.editor_bgfilter_base_url.clone(),
"editor-screen-background-removal",
"bgfilter_segment",
crate::external_api_audit::matting_failure_audit_status_code(&error),
crate::external_api_audit::matting_failure_audit_timeout(&error),
crate::external_api_audit::matting_failure_audit_latency_ms(&error),
error.message().to_string(),
crate::external_api_audit::matting_failure_audit_raw_excerpt(&error),
)
.await;
fallback_editor_screen_background_removal(state, image, screen_color, audit).await
let max_attempts = EDITOR_BGFILTER_RETRY_COUNT + 1;
let mut final_error = None;
for attempt in 1..=max_attempts {
match request_editor_generated_screen_background_with_bgfilter(
state,
image,
screen_color,
seg_model,
cross_check,
)
.await
{
Ok(image) => {
record_editor_bgfilter_success();
return Ok(image);
}
Err(error) => {
record_editor_bgfilter_failure(state);
let will_retry = attempt < max_attempts;
tracing::warn!(
provider = "bgfilter",
screen_color = screen_color.hex,
seg_model,
cross_check,
attempt,
max_attempts,
will_retry,
error = %error,
error_details = ?error.details(),
"editor_bgfilter_request_attempt_failed"
);
crate::external_api_audit::record_matting_external_api_failure(
state,
audit,
"bgfilter",
state.config.editor_bgfilter_base_url.clone(),
"editor-screen-background-removal",
"bgfilter_segment",
crate::external_api_audit::matting_failure_audit_status_code(&error),
crate::external_api_audit::matting_failure_audit_timeout(&error),
crate::external_api_audit::matting_failure_audit_latency_ms(&error),
error.message().to_string(),
crate::external_api_audit::matting_failure_audit_raw_excerpt(&error),
)
.await;
if will_retry {
continue;
}
final_error = Some(error);
}
}
}
if let Some(error) = final_error {
tracing::warn!(
provider = "bgfilter",
screen_color = screen_color.hex,
seg_model,
cross_check,
error = %error,
error_details = ?error.details(),
"editor_bgfilter_fallback_to_aliyun_matting"
);
return fallback_editor_screen_background_removal(state, image, screen_color, audit).await;
}
fallback_editor_screen_background_removal(state, image, screen_color, audit).await
}
/// BgFilter 不可用(熔断打开或调用失败)时的统一兜底链:先阿里云通用抠图,失败再退本地键色扣除。
@@ -2580,15 +2607,7 @@ async fn request_editor_generated_screen_background_with_bgfilter(
timeout_ms,
"editor_bgfilter_request_start"
);
let http_client = reqwest::Client::builder()
.timeout(std::time::Duration::from_millis(timeout_ms))
.build()
.map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
"provider": "bgfilter",
"message": format!("创建 BgFilter HTTP 客户端失败:{error}"),
}))
})?;
let http_client = state.editor_bgfilter_http_client();
let file_part = reqwest::multipart::Part::bytes(image.bytes.clone())
.file_name(source_file_name)
.mime_str(source_mime_type.as_str())
@@ -7855,7 +7874,7 @@ mod tests {
&[
"persist_editor_green_screen_source_image",
"remove_editor_generated_screen_background_with_bgfilter",
"EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK",
"EDITOR_BGFILTER_CROSS_CHECK_ENABLED",
"if is_character_generation",
],
);
@@ -7866,7 +7885,7 @@ mod tests {
&[
"persist_editor_green_screen_source_image",
"remove_editor_generated_screen_background_with_bgfilter",
"EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK",
"EDITOR_BGFILTER_CROSS_CHECK_ENABLED",
],
);
assert_function_contains(
@@ -7919,7 +7938,10 @@ mod tests {
// BgFilter 调用失败分支同样走该兜底链。
"editor_bgfilter_circuit_open_remaining",
"fallback_editor_screen_background_removal",
"EDITOR_BGFILTER_RETRY_COUNT + 1",
"request_editor_generated_screen_background_with_bgfilter",
"will_retry",
"record_matting_external_api_failure",
"fallback_editor_screen_background_removal",
],
);
@@ -7939,6 +7961,7 @@ mod tests {
&[
"editor_bgfilter_endpoint",
"editor_bgfilter_request_timeout_ms.max(1)",
"state.editor_bgfilter_http_client()",
"\"screen_color\"",
"\"seg_model\"",
"seg_model.to_string()",
@@ -7946,6 +7969,12 @@ mod tests {
"editor_bgfilter_cross_check_form_value(cross_check)",
],
);
assert_function_not_contains(
source,
"async fn request_editor_generated_screen_background_with_bgfilter",
"async fn request_editor_background_removal_image",
&["reqwest::Client::builder"],
);
assert_function_contains(
source,
"async fn request_editor_background_removal_image",
@@ -8017,6 +8046,27 @@ mod tests {
);
}
#[test]
fn editor_bgfilter_retries_once_before_fallback() {
let source = include_str!("editor_project.rs");
assert!(source.contains("const EDITOR_BGFILTER_RETRY_COUNT: usize = 1;"));
assert_function_contains_in_order(
source,
"async fn remove_editor_generated_screen_background_with_bgfilter",
"async fn fallback_editor_screen_background_removal",
&[
"let max_attempts = EDITOR_BGFILTER_RETRY_COUNT + 1",
"for attempt in 1..=max_attempts",
"request_editor_generated_screen_background_with_bgfilter",
"let will_retry = attempt < max_attempts",
"if will_retry",
"continue",
"final_error = Some(error)",
"fallback_editor_screen_background_removal",
],
);
}
#[test]
fn generated_asset_folder_id_normalizes_project_and_legacy_local_folder() {
assert_eq!(
+26
View File
@@ -276,6 +276,7 @@ pub struct AppStateInner {
llm_client: Option<LlmClient>,
creative_agent_gpt5_client: Option<LlmClient>,
matting_client: Option<MattingClient>,
editor_bgfilter_http_client: reqwest::Client,
creative_agent_executor: Arc<MockLangChainRustAgentExecutor>,
// Phase 1 任务 E 的 creative session facade 暂存在 api-server。
// creative_agent_* 表由任务 D 收口后,这里只保留读写 facade。
@@ -510,6 +511,7 @@ impl AppState {
let llm_client = build_llm_client(&config)?;
let creative_agent_gpt5_client = build_creative_agent_gpt5_client(&config)?;
let matting_client = build_matting_client(&config)?;
let editor_bgfilter_http_client = build_editor_bgfilter_http_client(&config)?;
let http_request_permit_pools = HttpRequestPermitPools::from_config(&config);
let (profile_recharge_order_updates, _) = broadcast::channel(128);
@@ -550,6 +552,7 @@ impl AppState {
llm_client,
creative_agent_gpt5_client,
matting_client,
editor_bgfilter_http_client,
creative_agent_executor: Arc::new(MockLangChainRustAgentExecutor),
creative_agent_sessions: Arc::new(Mutex::new(HashMap::new())),
profile_recharge_order_updates,
@@ -1248,6 +1251,10 @@ impl AppState {
self.matting_client.as_ref()
}
pub fn editor_bgfilter_http_client(&self) -> &reqwest::Client {
&self.editor_bgfilter_http_client
}
pub fn creative_agent_executor(&self) -> Arc<MockLangChainRustAgentExecutor> {
self.creative_agent_executor.clone()
}
@@ -1875,6 +1882,25 @@ fn build_matting_client(config: &AppConfig) -> Result<Option<MattingClient>, App
.map_err(|error| AppStateInitError::DependencyUnavailable(error.to_string()))
}
fn build_editor_bgfilter_http_client(
config: &AppConfig,
) -> Result<reqwest::Client, AppStateInitError> {
reqwest::Client::builder()
.timeout(std::time::Duration::from_millis(
config.editor_bgfilter_request_timeout_ms.max(1),
))
.connect_timeout(std::time::Duration::from_secs(30))
.pool_idle_timeout(std::time::Duration::from_secs(300))
.pool_max_idle_per_host(64)
.tcp_keepalive(std::time::Duration::from_secs(60))
.build()
.map_err(|error| {
AppStateInitError::DependencyUnavailable(format!(
"构建共享 BgFilter HTTP 客户端失败:{error}"
))
})
}
fn build_wechat_client(config: &AppConfig) -> WechatClient {
WechatClient::new(WechatConfig {
app_id: config.wechat_mini_program_app_id.clone(),