统一BGFilter抠图降级路线
为角色形象、图标、UI素材和动作帧显式设置cross_check策略 角色动作逐帧复用BGFilter到阿里云再到本地键色的三段式降级 更新BGFilter服务地址、架构文档和项目决策记录
This commit is contained in:
@@ -29,6 +29,7 @@ GENARRATIVE_LLM_PROVIDER="ark"
|
||||
GENARRATIVE_LLM_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
|
||||
GENARRATIVE_LLM_API_KEY="eb750614-e0b5-402a-bfea-4224862d251e"
|
||||
GENARRATIVE_LLM_MODEL="doubao-1-5-pro-32k-character-250715"
|
||||
GENARRATIVE_EDITOR_BGFILTER_BASE_URL=https://u39211-9b1c-e4a7a054.westb.seetacloud.com:8443
|
||||
APIMART_BASE_URL="https://api.apimart.ai/v1"
|
||||
APIMART_API_KEY=""
|
||||
APIMART_IMAGE_REQUEST_TIMEOUT_MS=180000
|
||||
@@ -36,6 +37,7 @@ DASHSCOPE_SCENE_IMAGE_MODEL="wan2.2-t2i-flash"
|
||||
DASHSCOPE_REFERENCE_IMAGE_MODEL="qwen-image-2.0"
|
||||
DASHSCOPE_COVER_IMAGE_MODEL="wan2.2-t2i-flash"
|
||||
ARK_CHARACTER_VIDEO_REQUEST_TIMEOUT_MS=420000
|
||||
|
||||
# 启用服务端大模型调试日志(记录所有输入输出)
|
||||
LLM_DEBUG_LOG="true"
|
||||
|
||||
|
||||
@@ -16,6 +16,14 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-11 BgFilter 交叉模型否决只用于角色形象
|
||||
|
||||
- 背景:新版 BgFilter 的 `cross_check` 默认开启,会额外运行 HR-matting 第二意见模型;角色形象需要保留发丝、镂空等复杂边缘质量,但角色动作序列帧、图标 spritesheet 和 UI 素材提取不需要承担这部分额外推理开销。
|
||||
- 决策:api-server 调用 BgFilter 时必须显式发送 multipart 字段 `cross_check`,不依赖服务端默认值。角色形象生成固定传 `on`;角色动作逐帧去背、图标 spritesheet 生成和 UI 设计图素材提取固定传 `off`。角色动作逐帧去背与三条静态生图路线复用同一条 `BgFilter → 阿里云通用抠图 → 本地键色` 降级链和同一 BgFilter 熔断器。该字段是后端内部供应商策略,不进入前端请求或外部 OpenAPI。
|
||||
- 影响范围:`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/api-server/src/character_animation_assets.rs`、图片画布 BgFilter 调用文档。
|
||||
- 验证方式:运行 `cargo test -p api-server editor_bgfilter_cross_check --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_canvas_screen_background_generation_uses_bgfilter_postprocess --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_character_animation_frames_use_three_stage_matting_fallback --manifest-path server-rs/Cargo.toml`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:encoding` 和 `git diff --check`。
|
||||
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 2026-07-10 BgFilter segModel 保留内部字段,不进入外部 OpenAPI
|
||||
|
||||
- 背景:`api-server` 的图片生成、图标 spritesheet 与 UI 素材提取请求仍可反序列化 `segModel`,并识别 `birefnet` / `anime-seg`,以兼容内部调用和既有任务;但 BgFilter 当前受服务进程内存与并发容量约束,不同分割模型的内存占用并非可由外部调用方自由选择的稳定契约。
|
||||
@@ -35,7 +43,7 @@
|
||||
## 2026-07-09 角色动作视频生成背景色统一为多色自动决策 + 阿里云抠帧
|
||||
|
||||
- 背景:角色动作视频抽帧过去固定 legacy `#00FF00` 绿幕 + 本地 `editor_green_screen`,与生图链路的多色自动决策不一致;实测出现背景色与前景 / 皮肤撞色(蓝撞蓝、桃 / 黄撞肤色)以及图生视频背景变白的问题。
|
||||
- 决策:角色动作视频背景色与生图统一。`screenColor=auto` 时由视觉 LLM(`gpt-5-mini`,Responses 协议、`reasoning_effort=low`,`max_tokens=1024`)读源角色图自动决策,并经硬过滤器(Lab 危险质量 + 皮肤专属三判据:ΔE 距离 / 色调投影 / RGB 分离)剔除与前景及皮肤撞色的候选,手动 hex 仍尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景确定性等于抠图键色。抽帧后逐帧优先走阿里云通用抠图,失败降级本地 `editor_green_screen` 键色兜底(按生成时选定的背景色,而非固定 `#00FF00`)。BgFilter 与阿里云抠图失败均写入 `external_api_call_failure` 失败审计。调色板新增中明度低饱和「灰竹绿 `#A0BBA0`」补齐冷区绿色段。
|
||||
- 决策:角色动作视频背景色与生图统一。`screenColor=auto` 时由视觉 LLM(`gpt-5-mini`,Responses 协议、`reasoning_effort=low`,`max_tokens=1024`)读源角色图自动决策,并经硬过滤器(Lab 危险质量 + 皮肤专属三判据:ΔE 距离 / 色调投影 / RGB 分离)剔除与前景及皮肤撞色的候选,手动 hex 仍尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景确定性等于抠图键色。抽帧后逐帧优先走 BgFilter(固定 `seg_model=birefnet`、`cross_check=off`),失败依次降级阿里云通用抠图和本地 `editor_green_screen` 键色兜底(按生成时选定的背景色,而非固定 `#00FF00`)。BgFilter 与阿里云抠图失败均写入 `external_api_call_failure` 失败审计。调色板新增中明度低饱和「灰竹绿 `#A0BBA0`」补齐冷区绿色段。
|
||||
- 影响范围:`server-rs/crates/api-server/src/character_animation_assets.rs`、`editor_screen_background_decision.rs`、`editor_screen_background_filter.rs`(新增硬过滤模块)、`editor_green_screen.rs`(调色板)、`external_api_audit.rs`、`llm_model_routing.rs`、图片画布 MVP 与后端数据契约文档。
|
||||
- 验证方式:`cargo test -p api-server editor_screen_background character_animation --manifest-path server-rs/Cargo.toml`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、真机对源角色图跑视觉决策与候选危险度表、抽帧后采样序列帧背景色确认落在冷区安全集。
|
||||
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -227,7 +227,7 @@ npm run check:server-rs-ddd
|
||||
|
||||
- LLM:通用 LLM 门面继续使用 `GENARRATIVE_LLM_*`;创意 Agent `gpt-5.4-mini` Chat Completions 文本链路已于 2026-06 从 APIMart 迁移到 VectorEngine,使用 `VECTOR_ENGINE_BASE_URL` / `VECTOR_ENGINE_API_KEY` 构造 OpenAI-compatible client,`api-server` 会把未带 `/v1` 的 VectorEngine base URL 规范化到 `/v1` 后请求 `/chat/completions`。通用 `/api/llm/chat/completions` 代理使用 `GENARRATIVE_LLM_PROVIDER=openai-compatible`、`GENARRATIVE_LLM_BASE_URL=https://api.vectorengine.cn/v1`、`GENARRATIVE_LLM_MODEL=gpt-5.4-mini`;未单独配置 `GENARRATIVE_LLM_API_KEY` 时可复用 `VECTOR_ENGINE_API_KEY`。`APIMART_BASE_URL` / `APIMART_API_KEY` 只作为历史残留,不再作为创意 Agent gpt-5.4-mini 客户端来源;后续排障时优先确认 VectorEngine `/v1/models`、`/v1/chat/completions` 和 `/v1/responses` 可用性。
|
||||
- 图片生成:VectorEngine `gpt-image-2` 图片 provider 归属 `platform-image`,密钥只在后端环境变量中;`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接,不再承载 provider 协议实现。实际外部生成运行记录统一落 `tracking_event`,`event_key = external_generation_run`,metadata 记录开始 / 结束时间、耗时、状态、成功标记、失败原因、provider task id 和结果摘要,不再写回过时的 `ai_task`。DashScope 只按仍在使用的历史能力单独处理,不作为 GPT-image-2 兜底。VectorEngine `/v1/images/generations` 和 `/v1/images/edits` 上游 POST 使用 `libcurl` 发送;`reqwest` 只保留给参考图 URL 下载和响应中图片 URL 下载。`/v1/images/edits` 的 multipart 参考图必须作为 libcurl 文件上传 part 发送,字段名为 `image`,实现上使用 `Form::buffer(file_name, bytes)` 并设置 `Content-Type`;不能只用 `contents(...).filename(...)`,否则上游会把请求转码为缺少图片并返回 `image is required`。`request_send` 阶段的 curl timeout / connect error 按可重试传输错误处理,最多尝试 5 次,并使用指数退避加短抖动;排障时优先看 `attempt`、`max_attempts`、`retry_delay_ms`、`reference_image_bytes_total` 和 `request_params`,不要把 `SendRequest` 当成上游业务错误。
|
||||
- 编辑器抠图服务:手动 `POST /api/editor/images/background-removals` 继续代理独立 BiRefNet 服务,配置为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`、`GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 和 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS`。角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取的生成后纯色背景透明化改走独立 BgFilter 服务,配置为 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL`、`GENARRATIVE_EDITOR_BGFILTER_TOKEN` 和 `GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS`,默认 base URL 为 `http://58.87.105.82/bgfilter`,默认请求超时为 `180000ms`(BgFilter 当前为 CPU 推理,单次抠图较慢,必须留足超时),token 未配置时复用 BiRefNet token。BgFilter 请求必须显式传 `screen_color=<screenColor>` 和 `seg_model=<segModel>`;前端用户路径不展示抠图模型选择并固定提交默认 `birefnet`,后端仍识别内部保留的 `anime-seg`,其中 `birefnet` 只表示 BgFilter 管线内部后端,不等同于手动去背景的独立 BiRefNet 服务。BgFilter 调用失败,或连续失败达到 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_FAILURE_THRESHOLD`(默认 `3`)并在 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_COOLDOWN_SECONDS`(默认 `300`)内打开熔断时,均跳过或结束 BgFilter 调用后复用同一兜底链:先调用阿里云通用抠图,阿里云失败才使用本地 `editor_green_screen` 键色扣除;熔断期不得直接退化到本地兜底。角色动作视频生成的背景色已与生图链路统一:`screenColor=auto` 时由视觉 LLM(`gpt-5-mini`,Responses 协议、low 推理档)读源角色图自动决策,并经硬过滤器剔除与前景 / 皮肤撞色的候选,手动 hex 则尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景等于抠图键色。抽帧后逐帧优先走阿里云通用抠图,失败时降级本地 `editor_green_screen` 键色兜底(按生成时选定的背景色,而非固定 `#00FF00`)。阿里云通用抠图配置为 `GENARRATIVE_ALIYUN_MATTING_ENABLED`、`GENARRATIVE_ALIYUN_MATTING_ENDPOINT`、`GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_ID`、`GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_SECRET` 和 `GENARRATIVE_ALIYUN_MATTING_REQUEST_TIMEOUT_MS`;未配置专用 AK/SK 时可复用 `ALIBABA_CLOUD_ACCESS_KEY_ID` / `ALIBABA_CLOUD_ACCESS_KEY_SECRET`,默认 endpoint 为 `imageseg.cn-shanghai.aliyuncs.com`。BgFilter 与阿里云抠图失败都写入 `external_api_call_failure` 审计。
|
||||
- 编辑器抠图服务:手动 `POST /api/editor/images/background-removals` 继续代理独立 BiRefNet 服务,配置为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`、`GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 和 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS`。角色形象生成、图标 spritesheet 生成、UI 设计图素材提取和角色动作抽帧后的纯色背景透明化统一走独立 BgFilter 服务,配置为 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL`、`GENARRATIVE_EDITOR_BGFILTER_TOKEN` 和 `GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS`,默认 base URL 为 `http://58.87.105.82/bgfilter`,默认请求超时为 `180000ms`(BgFilter 当前为 CPU 推理,单次抠图较慢,必须留足超时),token 未配置时复用 BiRefNet token。BgFilter 请求必须显式传 `screen_color=<screenColor>`、`seg_model=<segModel>` 和 `cross_check=<on|off>`;角色形象生成固定传 `cross_check=on`,角色动作逐帧去背、图标 spritesheet 生成和 UI 设计图素材提取固定传 `cross_check=off`,不依赖服务端默认值。前端用户路径不展示抠图模型选择并固定提交默认 `birefnet`,后端仍识别内部保留的 `anime-seg`,其中 `birefnet` 只表示 BgFilter 管线内部后端,不等同于手动去背景的独立 BiRefNet 服务;`cross_check` 同样只属于后端内部供应商策略,不进入前端或外部 OpenAPI。BgFilter 调用失败,或连续失败达到 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_FAILURE_THRESHOLD`(默认 `3`)并在 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_COOLDOWN_SECONDS`(默认 `300`)内打开熔断时,四条路线均跳过或结束 BgFilter 调用后复用同一兜底链:先调用阿里云通用抠图,阿里云失败才使用本地 `editor_green_screen` 键色扣除;熔断期不得直接退化到本地兜底。角色动作视频生成的背景色已与生图链路统一:`screenColor=auto` 时由视觉 LLM(`gpt-5-mini`,Responses 协议、low 推理档)读源角色图自动决策,并经硬过滤器剔除与前景 / 皮肤撞色的候选,手动 hex 则尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景等于抠图键色;抽帧后逐帧固定使用 `seg_model=birefnet`、`cross_check=off` 进入上述三段式链路。阿里云通用抠图配置为 `GENARRATIVE_ALIYUN_MATTING_ENABLED`、`GENARRATIVE_ALIYUN_MATTING_ENDPOINT`、`GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_ID`、`GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_SECRET` 和 `GENARRATIVE_ALIYUN_MATTING_REQUEST_TIMEOUT_MS`;未配置专用 AK/SK 时可复用 `ALIBABA_CLOUD_ACCESS_KEY_ID` / `ALIBABA_CLOUD_ACCESS_KEY_SECRET`,默认 endpoint 为 `imageseg.cn-shanghai.aliyuncs.com`。BgFilter 与阿里云抠图失败都写入 `external_api_call_failure` 审计。
|
||||
- Match3D 物品 sheet:关卡整图完成后走 VectorEngine `/v1/images/edits` multipart `image`,模型为 `gpt-image-2`,`2K 1:1` 输出 `10*10` spritesheet;物品 sheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG,并把透明整图写入 `itemSpritesheetImageSrc/itemSpritesheetImageObjectKey`。后端优先按透明 alpha 连通域从该 sheet 识别真实素材矩形并持久化 20 个物品、每个 5 个形态;识别数量不足时才回退 `10*10` 固定网格。通用系列素材图集的行列索引按每行 2 个物品计算,必须落在 `1..=10`,难度只决定运行态加载 3 / 9 / 15 / 20 种。
|
||||
- Match3D UI spritesheet 和背景派生图:关卡整图作为参考图并发生成 `1K 1:1` UI spritesheet 与 `1K 9:16` 背景图,模型均为 `gpt-image-2`。UI spritesheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG;背景图必须合成为全画幅不透明 PNG。
|
||||
- Match3D 1:1 容器 UI:VectorEngine `/v1/images/edits` multipart 参考图。该容器参考图是后端生图协议输入,必须通过 `include_bytes!` 随 `api-server` 编译进二进制,避免 API 单独发布或运行目录缺少 `public/` 时生成失败。
|
||||
|
||||
@@ -62,17 +62,18 @@ use crate::{
|
||||
},
|
||||
editor_green_screen::{
|
||||
EditorScreenBackgroundColor, editor_green_screen_character_prompt_clause,
|
||||
remove_editor_generated_green_screen_background,
|
||||
},
|
||||
editor_screen_background_decision::{
|
||||
EditorScreenBackgroundDecisionInput, EditorScreenBackgroundDecisionKind,
|
||||
resolve_editor_screen_background_color,
|
||||
},
|
||||
editor_project::{
|
||||
EDITOR_BGFILTER_CROSS_CHECK_DISABLED, EDITOR_BGFILTER_DEFAULT_SEG_MODEL,
|
||||
EditorCanvasGeneratedLayerInput, PersistEditorGeneratedAssetRequest,
|
||||
apply_editor_screen_background_decision_to_generation_inputs,
|
||||
build_editor_canvas_generated_layer_item, complete_editor_canvas_generation_with_items,
|
||||
persist_editor_generated_media_asset,
|
||||
remove_editor_generated_screen_background_with_bgfilter,
|
||||
},
|
||||
editor_screen_background_decision::{
|
||||
EditorScreenBackgroundDecisionInput, EditorScreenBackgroundDecisionKind,
|
||||
resolve_editor_screen_background_color,
|
||||
},
|
||||
http_error::AppError,
|
||||
openai_image_generation::DownloadedOpenAiImage,
|
||||
@@ -2216,7 +2217,7 @@ async fn persist_editor_character_animation_green_screen_source_frames(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// 逐帧抠图:优先阿里云通用抠图,失败降级本地键色(使用生成时的纯色背景色)。
|
||||
/// 逐帧抠图:优先 BgFilter(关闭 cross-check),失败依次降级阿里云通用抠图和本地键色。
|
||||
/// 小并发保序处理,单帧降级不影响其他帧。
|
||||
async fn remove_editor_character_animation_frame_backgrounds(
|
||||
state: &AppState,
|
||||
@@ -2237,43 +2238,20 @@ async fn remove_editor_character_animation_frame_backgrounds(
|
||||
mime_type: frame.mime_type,
|
||||
extension: frame.extension,
|
||||
};
|
||||
let removed = match crate::aliyun_matting::segment_image_with_aliyun_matting(
|
||||
let removed = remove_editor_generated_screen_background_with_bgfilter(
|
||||
state,
|
||||
&image,
|
||||
"editor-animation-frame",
|
||||
screen_color,
|
||||
EDITOR_BGFILTER_DEFAULT_SEG_MODEL,
|
||||
EDITOR_BGFILTER_CROSS_CHECK_DISABLED,
|
||||
audit,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(removed) => removed,
|
||||
Err(error) => {
|
||||
tracing::warn!(
|
||||
provider = "aliyun-matting",
|
||||
frame_index,
|
||||
screen_color = screen_color.hex,
|
||||
error = %error,
|
||||
error_details = ?error.details(),
|
||||
"editor_animation_frame_aliyun_matting_fallback_to_local"
|
||||
);
|
||||
if crate::external_api_audit::matting_failure_external_call_attempted(&error) {
|
||||
crate::external_api_audit::record_matting_external_api_failure(
|
||||
state,
|
||||
audit,
|
||||
"aliyun-matting",
|
||||
state.config.aliyun_matting_endpoint.clone(),
|
||||
"editor-character-animation-frame-matting",
|
||||
"aliyun_segment",
|
||||
crate::external_api_audit::matting_failure_audit_status_code(&error),
|
||||
crate::external_api_audit::matting_failure_audit_timeout(&error),
|
||||
crate::external_api_audit::matting_failure_audit_latency_ms(&error),
|
||||
error.message().to_string(),
|
||||
crate::external_api_audit::matting_failure_audit_raw_excerpt(&error)
|
||||
.or_else(|| Some(format!("frame_index={frame_index}"))),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
remove_editor_generated_green_screen_background(&image, screen_color)?
|
||||
}
|
||||
};
|
||||
.await?;
|
||||
tracing::debug!(
|
||||
frame_index,
|
||||
screen_color = screen_color.hex,
|
||||
"editor_animation_frame_background_removed"
|
||||
);
|
||||
finalize_animation_frame_payload(
|
||||
removed.bytes.as_slice(),
|
||||
removed.mime_type.as_str(),
|
||||
@@ -5699,7 +5677,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_frames_use_local_green_screen_postprocess() {
|
||||
fn editor_character_animation_frames_use_three_stage_matting_fallback() {
|
||||
let source = include_str!("character_animation_assets.rs");
|
||||
assert_function_contains(
|
||||
source,
|
||||
@@ -5736,7 +5714,9 @@ mod tests {
|
||||
"fn remove_editor_character_animation_frame_backgrounds",
|
||||
"async fn publish_animation_set",
|
||||
&[
|
||||
"remove_editor_generated_green_screen_background",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"EDITOR_BGFILTER_DEFAULT_SEG_MODEL",
|
||||
"EDITOR_BGFILTER_CROSS_CHECK_DISABLED",
|
||||
"finalize_animation_frame_payload",
|
||||
],
|
||||
);
|
||||
|
||||
@@ -113,8 +113,10 @@ const EDITOR_UI_DESIGN_SPRITESHEET_ASSET_KIND: &str = "editor_ui_design_spritesh
|
||||
const EDITOR_UI_DESIGN_ASSET_IMAGE_KIND: &str = "editor_ui_design_asset";
|
||||
const EDITOR_GREEN_SCREEN_SOURCE_ASSET_KIND: &str = "editor_green_screen_source";
|
||||
const EDITOR_GREEN_SCREEN_SOURCE_SLOT: &str = "green_screen_source";
|
||||
const EDITOR_BGFILTER_DEFAULT_SEG_MODEL: &str = "birefnet";
|
||||
pub(crate) const EDITOR_BGFILTER_DEFAULT_SEG_MODEL: &str = "birefnet";
|
||||
const EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG: &str = "anime-seg";
|
||||
const EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK: bool = true;
|
||||
pub(crate) const EDITOR_BGFILTER_CROSS_CHECK_DISABLED: bool = false;
|
||||
const EDITOR_PUBLICATION_MATERIAL_ASSET_KIND: &str = "editor_publication_material";
|
||||
const EDITOR_LEGACY_INLINE_IMAGE_ASSET_KIND: &str = "editor_legacy_inline_image";
|
||||
|
||||
@@ -1585,6 +1587,7 @@ pub(crate) async fn generate_editor_image_for_owner(
|
||||
&image,
|
||||
screen_color.expect("character generation should have screen color"),
|
||||
seg_model.expect("character generation should have BgFilter seg model"),
|
||||
EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK,
|
||||
&matting_audit,
|
||||
)
|
||||
.await?;
|
||||
@@ -2239,11 +2242,12 @@ struct EditorBackgroundRemovalImage {
|
||||
height: u32,
|
||||
}
|
||||
|
||||
async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
pub(crate) async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
state: &AppState,
|
||||
image: &DownloadedOpenAiImage,
|
||||
screen_color: EditorScreenBackgroundColor,
|
||||
seg_model: &str,
|
||||
cross_check: bool,
|
||||
audit: &crate::external_api_audit::ExternalApiAuditContext,
|
||||
) -> Result<DownloadedOpenAiImage, AppError> {
|
||||
if let Some(remaining) = editor_bgfilter_circuit_open_remaining(state) {
|
||||
@@ -2253,6 +2257,7 @@ async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
provider = "bgfilter",
|
||||
screen_color = screen_color.hex,
|
||||
seg_model,
|
||||
cross_check,
|
||||
cooldown_ms = remaining.as_millis() as u64,
|
||||
"editor_bgfilter_circuit_open_fallback_to_aliyun_matting"
|
||||
);
|
||||
@@ -2264,6 +2269,7 @@ async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
image,
|
||||
screen_color,
|
||||
seg_model,
|
||||
cross_check,
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -2277,6 +2283,7 @@ async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
provider = "bgfilter",
|
||||
screen_color = screen_color.hex,
|
||||
seg_model,
|
||||
cross_check,
|
||||
error = %error,
|
||||
error_details = ?error.details(),
|
||||
"editor_bgfilter_fallback_to_aliyun_matting"
|
||||
@@ -2401,6 +2408,7 @@ async fn request_editor_generated_screen_background_with_bgfilter(
|
||||
image: &DownloadedOpenAiImage,
|
||||
screen_color: EditorScreenBackgroundColor,
|
||||
seg_model: &str,
|
||||
cross_check: bool,
|
||||
) -> Result<DownloadedOpenAiImage, AppError> {
|
||||
let url = editor_bgfilter_endpoint(state)?;
|
||||
let call_id = format!("bgfilter-call-{}", current_utc_micros());
|
||||
@@ -2419,6 +2427,7 @@ async fn request_editor_generated_screen_background_with_bgfilter(
|
||||
mime_type = %source_mime_type,
|
||||
screen_color = screen_color.hex,
|
||||
seg_model,
|
||||
cross_check,
|
||||
input_bytes,
|
||||
timeout_ms,
|
||||
"editor_bgfilter_request_start"
|
||||
@@ -2444,7 +2453,11 @@ async fn request_editor_generated_screen_background_with_bgfilter(
|
||||
let form = reqwest::multipart::Form::new()
|
||||
.part("file", file_part)
|
||||
.text("screen_color", screen_color.hex.to_string())
|
||||
.text("seg_model", seg_model.to_string());
|
||||
.text("seg_model", seg_model.to_string())
|
||||
.text(
|
||||
"cross_check",
|
||||
editor_bgfilter_cross_check_form_value(cross_check),
|
||||
);
|
||||
let mut request = http_client.post(url.as_str()).multipart(form);
|
||||
if let Some(token) = state
|
||||
.config
|
||||
@@ -2511,9 +2524,18 @@ async fn request_editor_generated_screen_background_with_bgfilter(
|
||||
.get("x-bgfilter-screen-color")
|
||||
.and_then(|value| value.to_str().ok())
|
||||
.map(ToOwned::to_owned);
|
||||
let bytes =
|
||||
read_editor_image_removal_response_bytes(response, "bgfilter", "BgFilter", request_started_at)
|
||||
.await?;
|
||||
let upstream_cross_check = response
|
||||
.headers()
|
||||
.get("x-bgfilter-cross-check")
|
||||
.and_then(|value| value.to_str().ok())
|
||||
.map(ToOwned::to_owned);
|
||||
let bytes = read_editor_image_removal_response_bytes(
|
||||
response,
|
||||
"bgfilter",
|
||||
"BgFilter",
|
||||
request_started_at,
|
||||
)
|
||||
.await?;
|
||||
if bytes.is_empty() {
|
||||
// HTTP 已成功但 body 为空属于上游内容缺陷(5xx 归类正确),仍补 latencyMs 与 Aliyun 兜底口径对齐。
|
||||
return Err(
|
||||
@@ -2537,8 +2559,10 @@ async fn request_editor_generated_screen_background_with_bgfilter(
|
||||
height = decoded.height(),
|
||||
screen_color = screen_color.hex,
|
||||
seg_model,
|
||||
cross_check,
|
||||
upstream_screen_color = upstream_screen_color.as_deref().unwrap_or(""),
|
||||
upstream_seg_model = upstream_seg_model.as_deref().unwrap_or(""),
|
||||
upstream_cross_check = upstream_cross_check.as_deref().unwrap_or(""),
|
||||
input_bytes,
|
||||
output_bytes = image.bytes.len(),
|
||||
elapsed_ms = request_started_at.elapsed().as_millis() as u64,
|
||||
@@ -2732,6 +2756,10 @@ fn parse_editor_bgfilter_seg_model(value: Option<&str>) -> Result<&'static str,
|
||||
}
|
||||
}
|
||||
|
||||
fn editor_bgfilter_cross_check_form_value(enabled: bool) -> &'static str {
|
||||
if enabled { "on" } else { "off" }
|
||||
}
|
||||
|
||||
fn map_editor_background_removal_error(error: reqwest::Error) -> AppError {
|
||||
let status = if error.is_timeout() {
|
||||
StatusCode::GATEWAY_TIMEOUT
|
||||
@@ -3075,6 +3103,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner(
|
||||
&image,
|
||||
screen_color,
|
||||
seg_model,
|
||||
EDITOR_BGFILTER_CROSS_CHECK_DISABLED,
|
||||
&matting_audit,
|
||||
)
|
||||
.await?;
|
||||
@@ -3376,6 +3405,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
|
||||
&image,
|
||||
screen_color,
|
||||
seg_model,
|
||||
EDITOR_BGFILTER_CROSS_CHECK_DISABLED,
|
||||
&matting_audit,
|
||||
)
|
||||
.await?;
|
||||
@@ -7544,6 +7574,12 @@ mod tests {
|
||||
assert!(parse_editor_bgfilter_seg_model(Some("u2net")).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_bgfilter_cross_check_form_value_matches_service_contract() {
|
||||
assert_eq!(editor_bgfilter_cross_check_form_value(true), "on");
|
||||
assert_eq!(editor_bgfilter_cross_check_form_value(false), "off");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_bgfilter_circuit_opens_after_consecutive_failures_and_resets_on_success() {
|
||||
reset_editor_bgfilter_circuit_for_tests();
|
||||
@@ -7618,6 +7654,7 @@ mod tests {
|
||||
&[
|
||||
"persist_editor_green_screen_source_image",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK",
|
||||
"if is_character_generation",
|
||||
],
|
||||
);
|
||||
@@ -7628,6 +7665,7 @@ mod tests {
|
||||
&[
|
||||
"persist_editor_green_screen_source_image",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"EDITOR_BGFILTER_CHARACTER_IMAGE_CROSS_CHECK",
|
||||
],
|
||||
);
|
||||
assert_function_contains(
|
||||
@@ -7637,6 +7675,7 @@ mod tests {
|
||||
&[
|
||||
"persist_editor_green_screen_source_image",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"EDITOR_BGFILTER_CROSS_CHECK_DISABLED",
|
||||
],
|
||||
);
|
||||
assert_function_contains_in_order(
|
||||
@@ -7646,6 +7685,7 @@ mod tests {
|
||||
&[
|
||||
"persist_editor_green_screen_source_image",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"EDITOR_BGFILTER_CROSS_CHECK_DISABLED",
|
||||
],
|
||||
);
|
||||
assert_function_contains(
|
||||
@@ -7655,6 +7695,7 @@ mod tests {
|
||||
&[
|
||||
"persist_editor_green_screen_source_image",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"EDITOR_BGFILTER_CROSS_CHECK_DISABLED",
|
||||
"slice_generated_icon_spritesheet_all_by_connected_components",
|
||||
],
|
||||
);
|
||||
@@ -7665,6 +7706,7 @@ mod tests {
|
||||
&[
|
||||
"persist_editor_green_screen_source_image",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"EDITOR_BGFILTER_CROSS_CHECK_DISABLED",
|
||||
],
|
||||
);
|
||||
assert_function_contains_in_order(
|
||||
@@ -7699,6 +7741,8 @@ mod tests {
|
||||
"\"screen_color\"",
|
||||
"\"seg_model\"",
|
||||
"seg_model.to_string()",
|
||||
"\"cross_check\"",
|
||||
"editor_bgfilter_cross_check_form_value(cross_check)",
|
||||
],
|
||||
);
|
||||
assert_function_contains(
|
||||
|
||||
Reference in New Issue
Block a user