优化抠图链路原图内存占用
带背景生成图改为上传 OSS 后释放,BgFilter 直接使用短期签名 URL。 阿里云兜底改为按需下载并上传临时桶,本地兜底再次按需读取后及时释放。 透明图集继续沿用内存上传与切分链路,不改变透明结果处理。 补充平台适配器测试与抠图链路文档。
This commit is contained in:
@@ -16,10 +16,17 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-17 生成后抠图原图以 OSS 作为内存生命周期边界
|
||||
|
||||
- 背景:角色形象、图标图集和 UI 素材图集的生成原图虽然已先落私有 OSS,但 api-server 仍把带背景原图字节保留到 BgFilter / 阿里云 / 本地 fallback 结束,造成并发任务下的内存峰值叠加。
|
||||
- 决策:目标链路的带背景原图上传 OSS 时消费 `DownloadedImage` 所有权,不为上传克隆整张字节缓冲;上传完成后不再跨 BgFilter 调用常驻。BgFilter 只读取 600 秒签名 URL;进入阿里云 fallback 时由 `platform-matting` 新 URL 接口下载原图、上传 `AuthorizeFileUpload` 临时对象,并在开始阿里云推理前结束下载缓冲作用域;阿里云继续失败时,api-server 再从私有 OSS 独立下载原图供本地键色,产出后释放本次原图下载缓冲。
|
||||
- 边界:不改变接口 DTO、资源记录、画布原图展示、图集切分行为和降级顺序;“释放”指 Rust 所有权和 `Vec<u8>` 析构,RSS 不保证同步下降。
|
||||
- 验证方式:`platform-matting` 测试覆盖 URL 下载缓冲在临时上传后结束、降尺寸 Alpha 回贴;`api-server` 结构测试覆盖带背景原图 owned 上传、URL 阿里云 fallback、本地重新下载与原图释放;随后运行两个 crate 的测试与编译检查。
|
||||
|
||||
## 2026-07-15 BgFilter 输入改用私有 OSS 短期签名 URL
|
||||
|
||||
- 背景:角色形象、图标图集和 UI 素材图集在调用 BgFilter 前已经把带背景原图持久化到私有 OSS;继续由 api-server 把同一图片作为 multipart `file` 再上传一次,会重复传输图片字节并占用 API 进程网络带宽。
|
||||
- 决策:上述生成后抠图链路统一复用已持久化原图的 object key,签发 600 秒 OSS GET URL,并通过 BgFilter multipart 的 `image_url` 字段提交;请求中不再携带 `file`。签名 URL 只交给 BgFilter,不写日志或持久化。BgFilter 失败或熔断打开后才使用已保留的图片字节进入原有“阿里云通用抠图 → 本地键色”兜底链。
|
||||
- 决策:上述生成后抠图链路统一复用已持久化原图的 object key,签发 600 秒 OSS GET URL,并通过 BgFilter multipart 的 `image_url` 字段提交;请求中不再携带 `file`。签名 URL 只交给 BgFilter,不写日志或持久化。2026-07-17 起,原图上传后不再保留图片字节;进入“阿里云通用抠图 → 本地键色”兜底链时按阶段从私有 OSS 重新下载。
|
||||
- 影响范围:仅 `api-server/editor_project` 的 BgFilter 请求输入与相关文档;不改变 BgFilter endpoint、鉴权、`screen_color`、`seg_model`、输出校验、熔断规则、阿里云上传协议或降级顺序。
|
||||
- 验证方式:定向测试必须断言 BgFilter 请求函数包含 `image_url` 与 600 秒 OSS 换签,不包含 multipart `file` 或源图字节读取;随后运行 `cargo check -p api-server --manifest-path server-rs/Cargo.toml`。
|
||||
|
||||
|
||||
@@ -241,7 +241,7 @@ npm run check:server-rs-ddd
|
||||
- LLM:通用 LLM 门面继续使用 `GENARRATIVE_LLM_*`;创意 Agent `gpt-5.4-mini` Chat Completions 文本链路已于 2026-06 从 APIMart 迁移到 VectorEngine,使用 `VECTOR_ENGINE_BASE_URL` / `VECTOR_ENGINE_API_KEY` 构造 OpenAI-compatible client,`api-server` 会把未带 `/v1` 的 VectorEngine base URL 规范化到 `/v1` 后请求 `/chat/completions`。通用 `/api/llm/chat/completions` 代理使用 `GENARRATIVE_LLM_PROVIDER=openai-compatible`、`GENARRATIVE_LLM_BASE_URL=https://api.vectorengine.cn/v1`、`GENARRATIVE_LLM_MODEL=gpt-5.4-mini`;未单独配置 `GENARRATIVE_LLM_API_KEY` 时可复用 `VECTOR_ENGINE_API_KEY`。`APIMART_BASE_URL` / `APIMART_API_KEY` 只作为历史残留,不再作为创意 Agent gpt-5.4-mini 客户端来源;后续排障时优先确认 VectorEngine `/v1/models`、`/v1/chat/completions` 和 `/v1/responses` 可用性。
|
||||
- 图片生成:VectorEngine `gpt-image-2` 图片 provider 归属 `platform-image`,密钥只在后端环境变量中;`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接,不再承载 provider 协议实现。实际外部生成运行记录统一落 `tracking_event`,`event_key = external_generation_run`,metadata 记录开始 / 结束时间、耗时、状态、成功标记、失败原因、provider task id 和结果摘要,不再写回过时的 `ai_task`。DashScope 只按仍在使用的历史能力单独处理,不作为 GPT-image-2 兜底。VectorEngine `/v1/images/generations` 和 `/v1/images/edits` 上游 POST 使用 `libcurl` 发送;`reqwest` 只保留给参考图 URL 下载和响应中图片 URL 下载。`/v1/images/edits` 的 multipart 参考图必须作为 libcurl 文件上传 part 发送,字段名为 `image`,实现上使用 `Form::buffer(file_name, bytes)` 并设置 `Content-Type`;不能只用 `contents(...).filename(...)`,否则上游会把请求转码为缺少图片并返回 `image is required`。`request_send` 阶段的 curl timeout / connect error 按可重试传输错误处理,最多尝试 5 次,并使用指数退避加短抖动;排障时优先看 `attempt`、`max_attempts`、`retry_delay_ms`、`reference_image_bytes_total` 和 `request_params`,不要把 `SendRequest` 当成上游业务错误。
|
||||
- 编辑器抠图服务:手动 `POST /api/editor/images/background-removals` 继续代理独立 BiRefNet 服务,配置为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`、`GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 和 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_REQUEST_TIMEOUT_MS`。角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取的生成后纯色背景透明化改走独立 BgFilter 服务,配置为 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL`、`GENARRATIVE_EDITOR_BGFILTER_TOKEN` 和 `GENARRATIVE_EDITOR_BGFILTER_REQUEST_TIMEOUT_MS`,默认 base URL 为 `http://58.87.105.82/bgfilter`,默认请求超时为 `180000ms`(BgFilter 当前为 CPU 推理,单次抠图较慢,必须留足超时),token 未配置时复用 BiRefNet token。BgFilter 请求必须显式传 `screen_color=<screenColor>` 和 `seg_model=<segModel>`;前端用户路径不展示抠图模型选择并固定提交默认 `birefnet`,后端仍识别内部保留的 `anime-seg`,其中 `birefnet` 只表示 BgFilter 管线内部后端,不等同于手动去背景的独立 BiRefNet 服务。BgFilter 调用失败,或连续失败达到 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_FAILURE_THRESHOLD`(默认 `3`)并在 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_COOLDOWN_SECONDS`(默认 `300`)内打开熔断时,均跳过或结束 BgFilter 调用后复用同一兜底链:先调用阿里云通用抠图,阿里云失败才使用本地 `editor_green_screen` 键色扣除;熔断期不得直接退化到本地兜底。角色动作视频生成的背景色已与生图链路统一:`screenColor=auto` 时由视觉 LLM(`gpt-5-mini`,Responses 协议、low 推理档)读源角色图自动决策,并经硬过滤器剔除与前景 / 皮肤撞色的候选,手动 hex 则尊重用户选择;透明源角色图在提交 Ark 图生视频前先合成到选定背景色实色,使视频背景等于抠图键色。抽帧后逐帧优先走阿里云通用抠图,失败时降级本地 `editor_green_screen` 键色兜底(按生成时选定的背景色,而非固定 `#00FF00`)。阿里云通用抠图配置为 `GENARRATIVE_ALIYUN_MATTING_ENABLED`、`GENARRATIVE_ALIYUN_MATTING_ENDPOINT`、`GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_ID`、`GENARRATIVE_ALIYUN_MATTING_ACCESS_KEY_SECRET` 和 `GENARRATIVE_ALIYUN_MATTING_REQUEST_TIMEOUT_MS`;未配置专用 AK/SK 时可复用 `ALIBABA_CLOUD_ACCESS_KEY_ID` / `ALIBABA_CLOUD_ACCESS_KEY_SECRET`,默认 endpoint 为 `imageseg.cn-shanghai.aliyuncs.com`。BgFilter 与阿里云抠图失败都写入 `external_api_call_failure` 审计。
|
||||
- BgFilter 处理已经持久化到私有 OSS 的带背景原图时,`api-server` 必须签发 600 秒 GET URL 并通过 multipart `image_url` 提交,不再下载对象或用 `file` 重传图片字节;签名 URL 不得写入日志、审计或持久化。只有 BgFilter 失败或熔断打开进入 fallback 后,才允许使用图片字节继续调用阿里云通用抠图或本地键色。
|
||||
- 生成后抠图以内存中的 `DownloadedImage` → 私有 OSS owned 上传作为带背景原图生命周期边界:上传消费字节所有权,完成后不保留原图缓冲。BgFilter 必须签发 600 秒 GET URL 并通过 multipart `image_url` 提交,不下载对象或用 `file` 重传;进入阿里云 fallback 时由 `platform-matting` URL 接口单独下载并上传 `AuthorizeFileUpload` 临时对象,在推理前释放下载缓冲;继续 fallback 到本地键色时再单独下载一次原图,本地产出后释放本次原图下载缓冲。签名 URL 不得写入日志、审计或持久化。
|
||||
- 阿里云通用抠图的非上海地域输入不得使用 `viapiutils/GetOssStsToken`、固定 `viapi-customer-temp` 或 OSS V1 PUT。`platform-matting` 必须按官方新版 SDK Advance 协议调用 `AuthorizeFileUpload`,使用动态返回的单对象 Policy 执行 multipart POST,再把临时上海 OSS URL 交给 `SegmentCommonImage`;输入归一化、结果下载与原尺寸 Alpha 回贴继续留在同一适配器内。该协议仍上传图片字节,不等同于阿里云服务端直接抓取任意公网 URL,也不改变上层 BgFilter → 阿里云 → 本地降级顺序。
|
||||
- Match3D 物品 sheet:关卡整图完成后走 VectorEngine `/v1/images/edits` multipart `image`,模型为 `gpt-image-2`,`2K 1:1` 输出 `10*10` spritesheet;物品 sheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG,并把透明整图写入 `itemSpritesheetImageSrc/itemSpritesheetImageObjectKey`。后端优先按透明 alpha 连通域从该 sheet 识别真实素材矩形并持久化 20 个物品、每个 5 个形态;识别数量不足时才回退 `10*10` 固定网格。通用系列素材图集的行列索引按每行 2 个物品计算,必须落在 `1..=10`,难度只决定运行态加载 3 / 9 / 15 / 20 种。
|
||||
- Match3D UI spritesheet 和背景派生图:关卡整图作为参考图并发生成 `1K 1:1` UI spritesheet 与 `1K 9:16` 背景图,模型均为 `gpt-image-2`。UI spritesheet prompt 固定要求单一纯绿色 `#00FF00 / RGB(0,255,0)` 绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG;背景图必须合成为全画幅不透明 PNG。
|
||||
|
||||
@@ -67,7 +67,7 @@ lease 过期后不代表任务一定再次执行:claim transaction 只有在 `
|
||||
|
||||
阿里云通用抠图的非上海地域输入使用 `AuthorizeFileUpload → Policy POST → SegmentCommonImage` 正式链路,上传 Bucket / Endpoint / ObjectKey 由阿里云动态返回;不得恢复 `GetOssStsToken`、固定 `viapi-customer-temp`、临时 AK/SK 或 OSS V1 PUT。该切换不新增环境变量;真实链路冒烟可运行 `cargo run -p platform-matting --example segment_smoke --manifest-path server-rs/Cargo.toml -- <图片路径>`,预期日志中的输入 host 为授权响应返回的上海 OSS host,并完成结果下载。图片字节仍经过执行任务的 api-server / worker,排障时不要把 Advance 路径误判为阿里云直接抓取任意公网 URL。
|
||||
|
||||
BgFilter 对已经落入私有 OSS 的生成原图直接使用 600 秒签名 URL:`api-server` 的 multipart 只提交 `image_url`、`screen_color` 和 `seg_model`,不再提交 `file`,也不会在 BgFilter 调用前重新下载 OSS 对象。排障日志只应出现 object key 与签名有效期,不得记录带 `x-oss-*` 查询参数的完整 URL;BgFilter 失败或熔断打开后仍按既有顺序进入阿里云通用抠图和本地键色 fallback。
|
||||
BgFilter 对已经落入私有 OSS 的生成原图直接使用 600 秒签名 URL:`api-server` 的 multipart 只提交 `image_url`、`screen_color` 和 `seg_model`,不再提交 `file`,也不会在 BgFilter 调用前重新下载 OSS 对象。带背景原图上传完成后应已消费并释放字节所有权;BgFilter 失败或熔断打开后,阿里云 fallback 才单独下载源对象并上传动态临时桶,临时上传完成即释放本次下载缓冲;阿里云继续失败时本地 fallback 再独立下载,并在本地处理产出后释放本次原图缓冲。排障日志只应出现 object key 与签名有效期,不得记录带 `x-oss-*` 查询参数的完整 URL。这里的释放是 Rust 缓冲析构,不以操作系统 RSS 立即下降作为判据。
|
||||
|
||||
`我的` 页签或排障面板展示队列等待时,只读取 BFF 队列接口:`GET /api/runtime/external-generation/queue-overview` 查看当前用户可见队列概览,`GET /api/runtime/external-generation/jobs/{jobId}` 查看单 job 状态。生成页 / 进度页不承接队列概览,只展示当前玩法业务进度;队列接口只提供等待 / 运行 / 失败 / 完成状态补充,最终草稿、作品和结果页仍要轮询对应玩法 session/detail 接口收敛到 ready 或 failed;不要直接查询 `external_generation_job` private table,也不要把 worker 内部 payload 暴露到前端。
|
||||
|
||||
|
||||
@@ -43,6 +43,39 @@ pub(crate) async fn segment_image_with_aliyun_matting(
|
||||
})
|
||||
}
|
||||
|
||||
/// 私有 OSS 签名 URL → 延迟下载 → 阿里云临时桶 → 原尺寸透明 PNG。
|
||||
/// 下载和临时上传由 platform-matting 收口,源图缓冲不会跨越整次阿里云推理常驻。
|
||||
pub(crate) async fn segment_image_url_with_aliyun_matting(
|
||||
state: &AppState,
|
||||
image_url: &str,
|
||||
log_label: &str,
|
||||
) -> Result<DownloadedOpenAiImage, AppError> {
|
||||
let matting_client = state
|
||||
.matting_client()
|
||||
.ok_or_else(aliyun_matting_unconfigured_error)?;
|
||||
|
||||
let file_name = format!("{log_label}.png");
|
||||
let started_at = std::time::Instant::now();
|
||||
let output_bytes = matting_client
|
||||
.segment_image_url_to_transparent_png(image_url, &file_name)
|
||||
.await
|
||||
.map_err(|error| {
|
||||
aliyun_matting_failure_to_app_error(&error, started_at.elapsed().as_millis() as u64)
|
||||
})?;
|
||||
tracing::info!(
|
||||
provider = "aliyun-matting",
|
||||
log_label,
|
||||
elapsed_ms = started_at.elapsed().as_millis() as u64,
|
||||
"阿里云通用抠图 URL 输入完成"
|
||||
);
|
||||
|
||||
Ok(DownloadedOpenAiImage {
|
||||
bytes: output_bytes,
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
/// 把 platform-matting 的错误映射成审计友好的 AppError。
|
||||
///
|
||||
/// 分类(是否外部调用、超时、传输层故障、上游 HTTP 状态)由 platform-matting 在错误发生处
|
||||
@@ -99,6 +132,22 @@ mod tests {
|
||||
assert!(!crate::external_api_audit::matting_failure_external_call_attempted(&error));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_input_path_delegates_download_and_temp_upload_to_platform_matting() {
|
||||
let source = include_str!("aliyun_matting.rs");
|
||||
let start = source
|
||||
.find("pub(crate) async fn segment_image_url_with_aliyun_matting")
|
||||
.expect("URL input adapter should exist");
|
||||
let tail = &source[start..];
|
||||
let end = tail
|
||||
.find("/// 把 platform-matting 的错误映射")
|
||||
.expect("URL input adapter should end before error mapper");
|
||||
let body = &tail[..end];
|
||||
|
||||
assert!(body.contains("segment_image_url_to_transparent_png"));
|
||||
assert!(!body.contains("segment_image_to_transparent_png(image.bytes"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn local_preflight_failures_do_not_count_as_external_call() {
|
||||
for error in [
|
||||
|
||||
@@ -126,7 +126,7 @@ const EDITOR_LEGACY_GREEN_SCREEN_SOURCE_ASSET_KIND: &str = "editor_green_screen_
|
||||
const EDITOR_PROVIDER_SOURCE_SLOT: &str = "provider_source";
|
||||
const EDITOR_BGFILTER_DEFAULT_SEG_MODEL: &str = "birefnet";
|
||||
const EDITOR_BGFILTER_SEG_MODEL_ANIME_SEG: &str = "anime-seg";
|
||||
const EDITOR_BGFILTER_IMAGE_URL_EXPIRE_SECONDS: u64 = 600;
|
||||
const EDITOR_MATTING_SOURCE_URL_EXPIRE_SECONDS: u64 = 600;
|
||||
const EDITOR_PUBLICATION_MATERIAL_ASSET_KIND: &str = "editor_publication_material";
|
||||
const EDITOR_LEGACY_INLINE_IMAGE_ASSET_KIND: &str = "editor_legacy_inline_image";
|
||||
|
||||
@@ -1623,7 +1623,7 @@ pub(crate) async fn generate_editor_image_for_owner(
|
||||
state,
|
||||
caller.owner_user_id.as_str(),
|
||||
generated.task_id.as_str(),
|
||||
&image,
|
||||
image,
|
||||
submitted_prompt.as_str(),
|
||||
generated.actual_prompt.as_deref(),
|
||||
storage_profile.asset_kind,
|
||||
@@ -1670,7 +1670,6 @@ pub(crate) async fn generate_editor_image_for_owner(
|
||||
let removal = remove_editor_generated_screen_background_with_bgfilter(
|
||||
state,
|
||||
source_object_key.as_str(),
|
||||
&image,
|
||||
screen_color.expect("character generation should have screen color"),
|
||||
seg_model.expect("character generation should have BgFilter seg model"),
|
||||
&matting_audit,
|
||||
@@ -2759,7 +2758,6 @@ struct EditorScreenBackgroundRemovalOutput {
|
||||
async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
state: &AppState,
|
||||
source_object_key: &str,
|
||||
fallback_image: &DownloadedOpenAiImage,
|
||||
screen_color: EditorScreenBackgroundColor,
|
||||
seg_model: &str,
|
||||
audit: &crate::external_api_audit::ExternalApiAuditContext,
|
||||
@@ -2776,7 +2774,7 @@ async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
);
|
||||
return fallback_editor_screen_background_removal(
|
||||
state,
|
||||
fallback_image,
|
||||
source_object_key,
|
||||
screen_color,
|
||||
audit,
|
||||
)
|
||||
@@ -2825,7 +2823,7 @@ async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
.await;
|
||||
fallback_editor_screen_background_removal(
|
||||
state,
|
||||
fallback_image,
|
||||
source_object_key,
|
||||
screen_color,
|
||||
audit,
|
||||
)
|
||||
@@ -2838,17 +2836,26 @@ async fn remove_editor_generated_screen_background_with_bgfilter(
|
||||
/// 熔断打开路径与 BgFilter 调用失败路径共用此链,确保 cooldown 内仍是「阿里云 → 本地」而非直接本地。
|
||||
async fn fallback_editor_screen_background_removal(
|
||||
state: &AppState,
|
||||
image: &DownloadedOpenAiImage,
|
||||
source_object_key: &str,
|
||||
screen_color: EditorScreenBackgroundColor,
|
||||
audit: &crate::external_api_audit::ExternalApiAuditContext,
|
||||
) -> Result<EditorScreenBackgroundRemovalOutput, AppError> {
|
||||
match crate::aliyun_matting::segment_image_with_aliyun_matting(
|
||||
let aliyun_result = match sign_editor_private_object_read_url(
|
||||
state,
|
||||
image,
|
||||
"editor-screen-background",
|
||||
)
|
||||
.await
|
||||
{
|
||||
source_object_key,
|
||||
EDITOR_MATTING_SOURCE_URL_EXPIRE_SECONDS,
|
||||
) {
|
||||
Ok(source_url) => {
|
||||
crate::aliyun_matting::segment_image_url_with_aliyun_matting(
|
||||
state,
|
||||
source_url.as_str(),
|
||||
"editor-screen-background",
|
||||
)
|
||||
.await
|
||||
}
|
||||
Err(error) => Err(error),
|
||||
};
|
||||
match aliyun_result {
|
||||
Ok(image) => Ok(EditorScreenBackgroundRemovalOutput {
|
||||
image,
|
||||
provider: "Aliyun Matting",
|
||||
@@ -2878,12 +2885,13 @@ async fn fallback_editor_screen_background_removal(
|
||||
)
|
||||
.await;
|
||||
}
|
||||
remove_editor_generated_green_screen_background(image, screen_color).map(|image| {
|
||||
EditorScreenBackgroundRemovalOutput {
|
||||
image,
|
||||
provider: "Genarrative Local",
|
||||
model: "screen-color-keying".to_string(),
|
||||
}
|
||||
let source_image = download_editor_persisted_image_object(state, source_object_key).await?;
|
||||
let output = remove_editor_generated_green_screen_background(&source_image, screen_color);
|
||||
drop(source_image);
|
||||
output.map(|image| EditorScreenBackgroundRemovalOutput {
|
||||
image,
|
||||
provider: "Genarrative Local",
|
||||
model: "screen-color-keying".to_string(),
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -2947,19 +2955,11 @@ async fn request_editor_generated_screen_background_with_bgfilter(
|
||||
seg_model: &str,
|
||||
) -> Result<DownloadedOpenAiImage, AppError> {
|
||||
let url = editor_bgfilter_endpoint(state)?;
|
||||
let oss_client = state.oss_client().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "aliyun-oss",
|
||||
"reason": "OSS 未完成环境变量配置",
|
||||
}))
|
||||
})?;
|
||||
let signed = oss_client
|
||||
.sign_get_object_url(OssSignedGetObjectUrlRequest {
|
||||
object_key: source_object_key.to_string(),
|
||||
expire_seconds: Some(EDITOR_BGFILTER_IMAGE_URL_EXPIRE_SECONDS),
|
||||
})
|
||||
.map_err(|error| map_oss_error(error, "aliyun-oss"))?;
|
||||
let signed_image_url = signed.signed_url;
|
||||
let signed_image_url = sign_editor_private_object_read_url(
|
||||
state,
|
||||
source_object_key,
|
||||
EDITOR_MATTING_SOURCE_URL_EXPIRE_SECONDS,
|
||||
)?;
|
||||
let call_id = format!("bgfilter-call-{}", current_utc_micros());
|
||||
let request_started_at = Instant::now();
|
||||
let timeout_ms = state.config.editor_bgfilter_request_timeout_ms.max(1);
|
||||
@@ -2967,7 +2967,7 @@ async fn request_editor_generated_screen_background_with_bgfilter(
|
||||
%call_id,
|
||||
upstream_url = %url,
|
||||
source_object_key,
|
||||
image_url_expire_seconds = EDITOR_BGFILTER_IMAGE_URL_EXPIRE_SECONDS,
|
||||
image_url_expire_seconds = EDITOR_MATTING_SOURCE_URL_EXPIRE_SECONDS,
|
||||
screen_color = screen_color.hex,
|
||||
seg_model,
|
||||
timeout_ms,
|
||||
@@ -3257,6 +3257,26 @@ fn editor_bgfilter_endpoint(state: &AppState) -> Result<String, AppError> {
|
||||
))
|
||||
}
|
||||
|
||||
fn sign_editor_private_object_read_url(
|
||||
state: &AppState,
|
||||
object_key: &str,
|
||||
expire_seconds: u64,
|
||||
) -> Result<String, AppError> {
|
||||
let oss_client = state.oss_client().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "aliyun-oss",
|
||||
"reason": "OSS 未完成环境变量配置",
|
||||
}))
|
||||
})?;
|
||||
oss_client
|
||||
.sign_get_object_url(OssSignedGetObjectUrlRequest {
|
||||
object_key: object_key.to_string(),
|
||||
expire_seconds: Some(expire_seconds),
|
||||
})
|
||||
.map(|signed| signed.signed_url)
|
||||
.map_err(|error| map_oss_error(error, "aliyun-oss"))
|
||||
}
|
||||
|
||||
fn sanitize_editor_bgfilter_upstream_message(message: &str, signed_image_url: &str) -> String {
|
||||
let sanitized = message.replace(signed_image_url, "[signed OSS URL redacted]");
|
||||
if sanitized.to_ascii_lowercase().contains("x-oss-") {
|
||||
@@ -3714,7 +3734,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner(
|
||||
state,
|
||||
caller.owner_user_id.as_str(),
|
||||
generated.task_id.as_str(),
|
||||
&image,
|
||||
image,
|
||||
prompt.as_str(),
|
||||
generated.actual_prompt.as_deref(),
|
||||
EDITOR_ICON_SPRITESHEET_ASSET_KIND,
|
||||
@@ -3755,7 +3775,6 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner(
|
||||
let removal = remove_editor_generated_screen_background_with_bgfilter(
|
||||
state,
|
||||
source_object_key.as_str(),
|
||||
&image,
|
||||
screen_color,
|
||||
seg_model,
|
||||
&matting_audit,
|
||||
@@ -3891,7 +3910,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner(
|
||||
}),
|
||||
)
|
||||
}
|
||||
};
|
||||
};
|
||||
let (canvas_items, primary_layer_id) =
|
||||
if let Some(completion) = payload.canvas_completion.as_ref() {
|
||||
build_icon_spritesheet_canvas_layer_items(
|
||||
@@ -4325,7 +4344,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
|
||||
state,
|
||||
caller.owner_user_id.as_str(),
|
||||
generated.task_id.as_str(),
|
||||
&image,
|
||||
image,
|
||||
prompt.as_str(),
|
||||
generated.actual_prompt.as_deref(),
|
||||
EDITOR_UI_DESIGN_SPRITESHEET_ASSET_KIND,
|
||||
@@ -4366,7 +4385,6 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
|
||||
let removal = remove_editor_generated_screen_background_with_bgfilter(
|
||||
state,
|
||||
source_object_key.as_str(),
|
||||
&image,
|
||||
screen_color,
|
||||
seg_model,
|
||||
&matting_audit,
|
||||
@@ -4496,7 +4514,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
|
||||
}),
|
||||
)
|
||||
}
|
||||
};
|
||||
};
|
||||
let (canvas_items, primary_layer_id) =
|
||||
if let Some(completion) = payload.canvas_completion.as_ref() {
|
||||
build_icon_spritesheet_canvas_layer_items(
|
||||
@@ -6359,6 +6377,71 @@ pub(crate) async fn persist_editor_generated_image(
|
||||
file_stem: &str,
|
||||
slot: &str,
|
||||
provider: &str,
|
||||
) -> Result<PersistedEditorGeneratedImage, AppError> {
|
||||
persist_editor_generated_image_data(
|
||||
state,
|
||||
owner_user_id,
|
||||
task_id,
|
||||
GeneratedImageAssetDataUrl {
|
||||
format: normalize_generated_image_asset_mime(image.mime_type.as_str()),
|
||||
bytes: image.bytes.clone(),
|
||||
},
|
||||
prompt,
|
||||
actual_prompt,
|
||||
asset_kind,
|
||||
path_kind,
|
||||
file_stem,
|
||||
slot,
|
||||
provider,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn persist_editor_generated_image_owned(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
task_id: &str,
|
||||
image: DownloadedOpenAiImage,
|
||||
prompt: &str,
|
||||
actual_prompt: Option<&str>,
|
||||
asset_kind: &str,
|
||||
path_kind: &str,
|
||||
file_stem: &str,
|
||||
slot: &str,
|
||||
provider: &str,
|
||||
) -> Result<PersistedEditorGeneratedImage, AppError> {
|
||||
let image_data = GeneratedImageAssetDataUrl {
|
||||
format: normalize_generated_image_asset_mime(image.mime_type.as_str()),
|
||||
bytes: image.bytes,
|
||||
};
|
||||
persist_editor_generated_image_data(
|
||||
state,
|
||||
owner_user_id,
|
||||
task_id,
|
||||
image_data,
|
||||
prompt,
|
||||
actual_prompt,
|
||||
asset_kind,
|
||||
path_kind,
|
||||
file_stem,
|
||||
slot,
|
||||
provider,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn persist_editor_generated_image_data(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
task_id: &str,
|
||||
image: GeneratedImageAssetDataUrl,
|
||||
prompt: &str,
|
||||
actual_prompt: Option<&str>,
|
||||
asset_kind: &str,
|
||||
path_kind: &str,
|
||||
file_stem: &str,
|
||||
slot: &str,
|
||||
provider: &str,
|
||||
) -> Result<PersistedEditorGeneratedImage, AppError> {
|
||||
let oss_client = state.oss_client().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
@@ -6375,10 +6458,7 @@ pub(crate) async fn persist_editor_generated_image(
|
||||
sanitize_editor_storage_segment(task_id, "task"),
|
||||
],
|
||||
file_stem: sanitize_editor_storage_segment(file_stem, "image"),
|
||||
image: GeneratedImageAssetDataUrl {
|
||||
format: normalize_generated_image_asset_mime(image.mime_type.as_str()),
|
||||
bytes: image.bytes.clone(),
|
||||
},
|
||||
image,
|
||||
access: OssObjectAccess::Private,
|
||||
metadata: GeneratedImageAssetAdapterMetadata {
|
||||
asset_kind: Some(asset_kind.to_string()),
|
||||
@@ -6454,14 +6534,14 @@ async fn persist_editor_provider_source_image(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
task_id: &str,
|
||||
image: &DownloadedOpenAiImage,
|
||||
image: DownloadedOpenAiImage,
|
||||
prompt: &str,
|
||||
actual_prompt: Option<&str>,
|
||||
asset_kind: &str,
|
||||
path_kind: &str,
|
||||
file_stem: &str,
|
||||
) -> Result<PersistedEditorGeneratedImage, AppError> {
|
||||
persist_editor_generated_image(
|
||||
persist_editor_generated_image_owned(
|
||||
state,
|
||||
owner_user_id,
|
||||
task_id,
|
||||
@@ -6808,7 +6888,7 @@ async fn read_editor_reference_image_object(
|
||||
expire_seconds: Some(EDITOR_REFERENCE_IMAGE_READ_EXPIRE_SECONDS),
|
||||
})
|
||||
.map_err(|error| map_oss_error(error, "aliyun-oss"))?;
|
||||
let response = reqwest::Client::new()
|
||||
let mut response = reqwest::Client::new()
|
||||
.get(signed.signed_url.as_str())
|
||||
.send()
|
||||
.await
|
||||
@@ -6840,12 +6920,20 @@ async fn read_editor_reference_image_object(
|
||||
.and_then(|value| value.to_str().ok())
|
||||
.and_then(normalize_editor_reference_image_mime_type)
|
||||
.map(ToOwned::to_owned);
|
||||
let bytes = response.bytes().await.map_err(|error| {
|
||||
let mut bytes = Vec::new();
|
||||
while let Some(chunk) = response.chunk().await.map_err(|error| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "aliyun-oss",
|
||||
"message": format!("读取图片参考图内容失败:{error}"),
|
||||
}))
|
||||
})?;
|
||||
})? {
|
||||
if bytes.len().saturating_add(chunk.len()) as u64
|
||||
> EDITOR_REFERENCE_IMAGE_MAX_SIZE_BYTES
|
||||
{
|
||||
return Err(editor_reference_image_too_large());
|
||||
}
|
||||
bytes.extend_from_slice(chunk.as_ref());
|
||||
}
|
||||
if bytes.is_empty() {
|
||||
return Err(
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
@@ -6855,11 +6943,8 @@ async fn read_editor_reference_image_object(
|
||||
})),
|
||||
);
|
||||
}
|
||||
if bytes.len() as u64 > EDITOR_REFERENCE_IMAGE_MAX_SIZE_BYTES {
|
||||
return Err(editor_reference_image_too_large());
|
||||
}
|
||||
let mime_type = content_type_mime
|
||||
.or_else(|| infer_editor_reference_image_mime_type(bytes.as_ref()).map(ToOwned::to_owned))
|
||||
.or_else(|| infer_editor_reference_image_mime_type(bytes.as_slice()).map(ToOwned::to_owned))
|
||||
.ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-reference-image",
|
||||
@@ -6871,12 +6956,24 @@ async fn read_editor_reference_image_object(
|
||||
.to_string();
|
||||
let extension = editor_reference_image_extension(mime_type.as_str());
|
||||
Ok(OpenAiReferenceImage {
|
||||
bytes: bytes.to_vec(),
|
||||
bytes,
|
||||
mime_type,
|
||||
file_name: format!("editor-reference.{extension}"),
|
||||
})
|
||||
}
|
||||
|
||||
async fn download_editor_persisted_image_object(
|
||||
state: &AppState,
|
||||
object_key: &str,
|
||||
) -> Result<DownloadedOpenAiImage, AppError> {
|
||||
let image = read_editor_reference_image_object(state, object_key).await?;
|
||||
Ok(DownloadedOpenAiImage {
|
||||
extension: editor_reference_image_extension(image.mime_type.as_str()).to_string(),
|
||||
mime_type: image.mime_type,
|
||||
bytes: image.bytes,
|
||||
})
|
||||
}
|
||||
|
||||
fn normalize_editor_reference_image_mime_type(content_type: &str) -> Option<&str> {
|
||||
let mime_type = content_type.split(';').next()?.trim();
|
||||
mime_type.starts_with("image/").then_some(mime_type)
|
||||
@@ -9445,8 +9542,10 @@ mod tests {
|
||||
"async fn fallback_editor_screen_background_removal",
|
||||
"async fn request_editor_generated_screen_background_with_bgfilter",
|
||||
&[
|
||||
"segment_image_with_aliyun_matting",
|
||||
"segment_image_url_with_aliyun_matting",
|
||||
"download_editor_persisted_image_object",
|
||||
"remove_editor_generated_green_screen_background",
|
||||
"drop(source_image)",
|
||||
],
|
||||
);
|
||||
assert_function_contains(
|
||||
@@ -9455,8 +9554,8 @@ mod tests {
|
||||
"async fn request_editor_background_removal_image",
|
||||
&[
|
||||
"editor_bgfilter_endpoint",
|
||||
"sign_get_object_url",
|
||||
"EDITOR_BGFILTER_IMAGE_URL_EXPIRE_SECONDS",
|
||||
"sign_editor_private_object_read_url",
|
||||
"EDITOR_MATTING_SOURCE_URL_EXPIRE_SECONDS",
|
||||
"\"image_url\"",
|
||||
"editor_bgfilter_request_timeout_ms.max(1)",
|
||||
"\"screen_color\"",
|
||||
@@ -9541,6 +9640,95 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_matting_releases_source_buffers_at_oss_boundaries() {
|
||||
let source = include_str!("editor_project.rs");
|
||||
for (start, end) in [
|
||||
(
|
||||
"pub(crate) async fn generate_editor_image_for_owner",
|
||||
"fn normalize_editor_image_generation_size",
|
||||
),
|
||||
(
|
||||
"pub(crate) async fn generate_editor_icon_spritesheet_for_owner",
|
||||
"pub async fn extract_editor_ui_design_assets",
|
||||
),
|
||||
(
|
||||
"pub(crate) async fn extract_editor_ui_design_assets_for_owner",
|
||||
"pub(crate) fn editor_project_payload_from_record",
|
||||
),
|
||||
] {
|
||||
assert_function_contains_in_order(
|
||||
source,
|
||||
start,
|
||||
end,
|
||||
&[
|
||||
"persist_editor_provider_source_image",
|
||||
"remove_editor_generated_screen_background_with_bgfilter",
|
||||
"persist_editor_generated_image",
|
||||
],
|
||||
);
|
||||
}
|
||||
assert_function_contains(
|
||||
source,
|
||||
"async fn persist_editor_provider_source_image",
|
||||
"async fn persist_editor_provider_source_resource",
|
||||
&["persist_editor_generated_image_owned"],
|
||||
);
|
||||
for (start, end) in [
|
||||
(
|
||||
"pub(crate) async fn generate_editor_icon_spritesheet_for_owner",
|
||||
"pub async fn extract_editor_ui_design_assets",
|
||||
),
|
||||
(
|
||||
"pub(crate) async fn extract_editor_ui_design_assets_for_owner",
|
||||
"pub(crate) fn editor_project_payload_from_record",
|
||||
),
|
||||
] {
|
||||
assert_function_contains(
|
||||
source,
|
||||
start,
|
||||
end,
|
||||
&["bytes: image.bytes.clone()", "slice_source"],
|
||||
);
|
||||
assert_function_not_contains(
|
||||
source,
|
||||
start,
|
||||
end,
|
||||
&["download_editor_persisted_image_object", "drop(slice_source)"],
|
||||
);
|
||||
}
|
||||
assert_function_not_contains(
|
||||
source,
|
||||
"async fn remove_editor_generated_screen_background_with_bgfilter",
|
||||
"async fn fallback_editor_screen_background_removal",
|
||||
&["fallback_image", "DownloadedOpenAiImage"],
|
||||
);
|
||||
assert_function_contains(
|
||||
source,
|
||||
"async fn persist_editor_generated_image_owned",
|
||||
"async fn persist_editor_generated_image_data",
|
||||
&["bytes: image.bytes"],
|
||||
);
|
||||
assert_function_not_contains(
|
||||
source,
|
||||
"async fn persist_editor_generated_image_owned",
|
||||
"async fn persist_editor_generated_image_data",
|
||||
&["image.bytes.clone()"],
|
||||
);
|
||||
assert_function_contains(
|
||||
source,
|
||||
"async fn read_editor_reference_image_object",
|
||||
"async fn download_editor_persisted_image_object",
|
||||
&["response.chunk().await", "bytes.extend_from_slice"],
|
||||
);
|
||||
assert_function_not_contains(
|
||||
source,
|
||||
"async fn read_editor_reference_image_object",
|
||||
"async fn download_editor_persisted_image_object",
|
||||
&["response.bytes().await", "bytes.to_vec()"],
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_paid_image_postprocess_keeps_provider_outputs_recoverable() {
|
||||
let source = include_str!("editor_project.rs");
|
||||
|
||||
@@ -407,16 +407,9 @@ impl MattingClient {
|
||||
let image_url = self
|
||||
.upload_temp_image(upload_bytes, file_name, "image/png")
|
||||
.await?;
|
||||
|
||||
let result = self
|
||||
.segment_common_image(SegmentCommonImageRequest {
|
||||
image_url,
|
||||
// 默认 ReturnForm:原尺寸 + 透明背景,无需本地合成。
|
||||
return_form: None,
|
||||
})
|
||||
let result_image = self
|
||||
.segment_uploaded_input_to_rgba(image_url, (source_width, source_height), upload_dims)
|
||||
.await?;
|
||||
let result_bytes = self.download_result_image(&result.image_url).await?;
|
||||
let result_image = decode_result_image(&result_bytes)?.to_rgba8();
|
||||
|
||||
if !downscaled {
|
||||
if result_image.dimensions() != (source_width, source_height) {
|
||||
@@ -434,18 +427,140 @@ impl MattingClient {
|
||||
return encode_rgba_png(&result_image);
|
||||
}
|
||||
|
||||
// 缩图送抠的场景:只取结果 alpha,上采样回原尺寸后贴回原图 RGB。
|
||||
let alpha_mask = image::DynamicImage::ImageRgba8(result_image).resize_exact(
|
||||
source_width,
|
||||
source_height,
|
||||
image::imageops::FilterType::Triangle,
|
||||
);
|
||||
let alpha_mask = alpha_mask.to_rgba8();
|
||||
let mut output = source_rgba;
|
||||
for (target, mask) in output.pixels_mut().zip(alpha_mask.pixels()) {
|
||||
target.0[3] = target.0[3].min(mask.0[3]);
|
||||
compose_source_with_result_alpha(source_rgba, result_image)
|
||||
}
|
||||
|
||||
/// 私有 OSS 签名 URL → 下载 → AuthorizeFileUpload 临时对象 → 通用抠图 → 透明 PNG。
|
||||
///
|
||||
/// 源图下载缓冲和解码图只活到临时对象上传完成;若输入发生过降尺寸,结果返回后再单独下载
|
||||
/// 一次源图完成 Alpha 回贴,避免在整个阿里云推理期间常驻原图内存。
|
||||
pub async fn segment_image_url_to_transparent_png(
|
||||
&self,
|
||||
source_url: &str,
|
||||
file_name: &str,
|
||||
) -> Result<Vec<u8>, MattingError> {
|
||||
let source_url = source_url.trim();
|
||||
if source_url.is_empty() {
|
||||
return Err(MattingError::InvalidRequest(
|
||||
"待抠图源图 URL 不能为空".to_string(),
|
||||
));
|
||||
}
|
||||
encode_rgba_png(&output)
|
||||
|
||||
let (source_dims, upload_dims, image_url) = {
|
||||
let source_bytes = self.download_source_image(source_url).await?;
|
||||
let source = decode_source_image(&source_bytes)?;
|
||||
let source_dims = (source.width(), source.height());
|
||||
validate_source_dimensions(source_dims)?;
|
||||
let (upload_bytes, upload_dims) = normalize_matting_input_png(&source)?;
|
||||
let image_url = self
|
||||
.upload_temp_image(upload_bytes, file_name, "image/png")
|
||||
.await?;
|
||||
(source_dims, upload_dims, image_url)
|
||||
};
|
||||
|
||||
let result_image = self
|
||||
.segment_uploaded_input_to_rgba(image_url, source_dims, upload_dims)
|
||||
.await?;
|
||||
if upload_dims == source_dims {
|
||||
return encode_rgba_png(&result_image);
|
||||
}
|
||||
|
||||
// 只有降尺寸送抠时才重新下载源图恢复原始 RGB;第一次下载缓冲已在上传临时对象后释放。
|
||||
let source_rgba = {
|
||||
let source_bytes = self.download_source_image(source_url).await?;
|
||||
let source = decode_source_image(&source_bytes)?;
|
||||
if (source.width(), source.height()) != source_dims {
|
||||
return Err(MattingError::upstream_response_error(
|
||||
"待抠图源图在处理期间尺寸发生变化".to_string(),
|
||||
None,
|
||||
));
|
||||
}
|
||||
source.to_rgba8()
|
||||
};
|
||||
compose_source_with_result_alpha(source_rgba, result_image)
|
||||
}
|
||||
|
||||
async fn segment_uploaded_input_to_rgba(
|
||||
&self,
|
||||
image_url: String,
|
||||
source_dims: (u32, u32),
|
||||
upload_dims: (u32, u32),
|
||||
) -> Result<image::RgbaImage, MattingError> {
|
||||
let result = self
|
||||
.segment_common_image(SegmentCommonImageRequest {
|
||||
image_url,
|
||||
// 默认 ReturnForm:原尺寸 + 透明背景,无需本地合成。
|
||||
return_form: None,
|
||||
})
|
||||
.await?;
|
||||
let result_image = {
|
||||
let result_bytes = self.download_result_image(&result.image_url).await?;
|
||||
decode_result_image(&result_bytes)?.to_rgba8()
|
||||
};
|
||||
if upload_dims == source_dims && result_image.dimensions() != source_dims {
|
||||
return Err(MattingError::upstream_response_error(
|
||||
format!(
|
||||
"抠图结果尺寸 {}x{} 与输入 {}x{} 不一致",
|
||||
result_image.width(),
|
||||
result_image.height(),
|
||||
source_dims.0,
|
||||
source_dims.1
|
||||
),
|
||||
None,
|
||||
));
|
||||
}
|
||||
Ok(result_image)
|
||||
}
|
||||
|
||||
async fn download_source_image(&self, url: &str) -> Result<Vec<u8>, MattingError> {
|
||||
let mut response = self.client.get(url).send().await.map_err(|error| {
|
||||
MattingError::upstream_transport_error(
|
||||
format!(
|
||||
"下载待抠图源图失败(transport={}, timeout={}, connect={})",
|
||||
classify_reqwest_error(&error),
|
||||
error.is_timeout(),
|
||||
error.is_connect()
|
||||
),
|
||||
error.is_timeout(),
|
||||
)
|
||||
})?;
|
||||
let status = response.status();
|
||||
if !status.is_success() {
|
||||
return Err(MattingError::upstream_http_error(
|
||||
format!("下载待抠图源图失败(HTTP {})", status.as_u16()),
|
||||
status.as_u16(),
|
||||
));
|
||||
}
|
||||
if response
|
||||
.content_length()
|
||||
.is_some_and(|length| length > MAX_RESULT_RESPONSE_BYTES as u64)
|
||||
{
|
||||
return Err(source_response_too_large_error());
|
||||
}
|
||||
let mut bytes = Vec::new();
|
||||
while let Some(chunk) = response.chunk().await.map_err(|error| {
|
||||
MattingError::upstream_transport_error(
|
||||
format!(
|
||||
"读取待抠图源图失败(transport={}, timeout={}, connect={})",
|
||||
classify_reqwest_error(&error),
|
||||
error.is_timeout(),
|
||||
error.is_connect()
|
||||
),
|
||||
error.is_timeout(),
|
||||
)
|
||||
})? {
|
||||
if bytes.len().saturating_add(chunk.len()) > MAX_RESULT_RESPONSE_BYTES {
|
||||
return Err(source_response_too_large_error());
|
||||
}
|
||||
bytes.extend_from_slice(chunk.as_ref());
|
||||
}
|
||||
if bytes.is_empty() {
|
||||
return Err(MattingError::upstream_response_error(
|
||||
"待抠图源图为空".to_string(),
|
||||
Some(status.as_u16()),
|
||||
));
|
||||
}
|
||||
Ok(bytes)
|
||||
}
|
||||
|
||||
async fn download_result_image(&self, url: &str) -> Result<Vec<u8>, MattingError> {
|
||||
@@ -663,6 +778,33 @@ fn normalize_matting_input_png(
|
||||
normalize_matting_input_png_within(source, MAX_INPUT_BYTES)
|
||||
}
|
||||
|
||||
fn validate_source_dimensions(source_dims: (u32, u32)) -> Result<(), MattingError> {
|
||||
if source_dims.0 <= MIN_INPUT_EDGE || source_dims.1 <= MIN_INPUT_EDGE {
|
||||
return Err(MattingError::InvalidRequest(format!(
|
||||
"待抠图图片尺寸 {}x{} 过小,阿里云通用抠图要求每条边大于 {MIN_INPUT_EDGE} 像素",
|
||||
source_dims.0, source_dims.1
|
||||
)));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn compose_source_with_result_alpha(
|
||||
mut source_rgba: image::RgbaImage,
|
||||
result_image: image::RgbaImage,
|
||||
) -> Result<Vec<u8>, MattingError> {
|
||||
let (source_width, source_height) = source_rgba.dimensions();
|
||||
let alpha_mask = image::DynamicImage::ImageRgba8(result_image).resize_exact(
|
||||
source_width,
|
||||
source_height,
|
||||
image::imageops::FilterType::Triangle,
|
||||
);
|
||||
let alpha_mask = alpha_mask.to_rgba8();
|
||||
for (target, mask) in source_rgba.pixels_mut().zip(alpha_mask.pixels()) {
|
||||
target.0[3] = target.0[3].min(mask.0[3]);
|
||||
}
|
||||
encode_rgba_png(&source_rgba)
|
||||
}
|
||||
|
||||
/// `normalize_matting_input_png` 的可注入体积上限版本,便于单测用小阈值触发降尺寸循环。
|
||||
fn normalize_matting_input_png_within(
|
||||
source: &image::DynamicImage,
|
||||
@@ -720,6 +862,13 @@ fn result_response_too_large_error() -> MattingError {
|
||||
)
|
||||
}
|
||||
|
||||
fn source_response_too_large_error() -> MattingError {
|
||||
MattingError::upstream_response_error(
|
||||
format!("待抠图源图响应过大,超过 {MAX_RESULT_RESPONSE_BYTES} 字节上限"),
|
||||
None,
|
||||
)
|
||||
}
|
||||
|
||||
/// 解码待抠图源图时套上尺寸 / 分配上限。源图直接来自请求体 / 生成产物,缺省
|
||||
/// `image::Limits` 无尺寸上限、仅 512MiB alloc 兜底,12MB 请求体即可构造巨幅压缩图
|
||||
/// (解压炸弹)在缩图前撑爆内存;逐帧动画并发调用时风险叠加。
|
||||
@@ -1311,4 +1460,47 @@ mod tests {
|
||||
"降尺寸后仍是 PNG"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_input_upload_scope_ends_before_aliyun_inference() {
|
||||
let source = include_str!("lib.rs");
|
||||
let start = source
|
||||
.find("pub async fn segment_image_url_to_transparent_png")
|
||||
.expect("URL input method should exist");
|
||||
let tail = &source[start..];
|
||||
let end = tail
|
||||
.find("async fn segment_uploaded_input_to_rgba")
|
||||
.expect("uploaded input helper should follow URL method");
|
||||
let body = &tail[..end];
|
||||
|
||||
let download = body
|
||||
.find("let source_bytes = self.download_source_image(source_url).await?")
|
||||
.expect("source should download lazily");
|
||||
let upload = body
|
||||
.find(".upload_temp_image(upload_bytes, file_name, \"image/png\")")
|
||||
.expect("source should upload to temporary OSS");
|
||||
let scope_end = body
|
||||
.find("let result_image = self")
|
||||
.expect("Aliyun inference should start after upload scope");
|
||||
assert!(download < upload && upload < scope_end);
|
||||
assert!(body.contains("let (source_dims, upload_dims, image_url) = {"));
|
||||
assert_eq!(body.matches("download_source_image(source_url)").count(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn compose_source_with_result_alpha_preserves_rgb() {
|
||||
let source = image::RgbaImage::from_pixel(2, 2, image::Rgba([10, 20, 30, 180]));
|
||||
let mask = image::RgbaImage::from_pixel(1, 1, image::Rgba([200, 210, 220, 90]));
|
||||
|
||||
let encoded = compose_source_with_result_alpha(source, mask)
|
||||
.expect("alpha composition should encode");
|
||||
let output = image::load_from_memory(&encoded)
|
||||
.expect("output should decode")
|
||||
.to_rgba8();
|
||||
|
||||
assert_eq!(output.dimensions(), (2, 2));
|
||||
assert!(output
|
||||
.pixels()
|
||||
.all(|pixel| pixel.0 == [10, 20, 30, 90]));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user