Compare commits
51 Commits
master
...
server-edition
| Author | SHA1 | Date | |
|---|---|---|---|
| f1a083349b | |||
| 78ef411ad2 | |||
| 43797eedeb | |||
| 3a8a1ef38a | |||
| 258e0fcf6e | |||
| 35d8bf9b02 | |||
| 755e5fda44 | |||
| dc4badd0e0 | |||
| 6506bdd643 | |||
| 07ba02fdde | |||
| c748fc22a4 | |||
| a0caac001f | |||
| 6baf6ae541 | |||
| c56e58affe | |||
| 28d3fe8842 | |||
| dbd061e119 | |||
| 9094b5d9c0 | |||
| ece9542978 | |||
| 826ad09098 | |||
| 1861cb0199 | |||
| 4d34c96bed | |||
| ba5d667524 | |||
| ab89121ccb | |||
| 3f85043ccc | |||
| ef11f625f1 | |||
| 8ef713d560 | |||
| a221e69212 | |||
| 9b9d71cf82 | |||
| 6bf648cd64 | |||
| 9931980c72 | |||
| a48d70d9f7 | |||
| 38a0dc34cc | |||
| 3992f65ebf | |||
| ad445611ad | |||
| c19a35de80 | |||
| 291e9faac5 | |||
| c7a2819d33 | |||
| 5265e802da | |||
| 6232887ebd | |||
| bfbeb9fc7f | |||
| 4b5c714fa3 | |||
| d3f19ddcc7 | |||
| edcef6e34d | |||
| 88987f508c | |||
| 8417653598 | |||
| 11c591e5b5 | |||
| 578bd1f4d8 | |||
| 8a7ad60705 | |||
| da4164e95a | |||
| 7a87a00ca1 | |||
| d24c6ed608 |
@@ -1,4 +1,6 @@
|
||||
Samples/
|
||||
Outputs/
|
||||
models/
|
||||
model-cache/
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
|
||||
+468
@@ -0,0 +1,468 @@
|
||||
# 将 BgFilter 部署到 Linux 服务器
|
||||
|
||||
> English version: [DEPLOY.md](DEPLOY.md)
|
||||
|
||||
本指南涵盖在 Linux 服务器上运行 BgFilter HTTP 服务(`POST /remove-background`、
|
||||
`GET /healthz`)所需的全部步骤。默认按 **CPU** 部署(推荐默认);涉及 GPU 的地方会单独标注。
|
||||
|
||||
服务入口是 `app.py`(`uvicorn app:app`)。API 契约和算法细节见 [README.md](README.md)。
|
||||
|
||||
下面第 1–12 节是**手动 CPU 部署**(systemd + nginx,推荐默认)。如果你部署在
|
||||
**AutoDL GPU 盒子**上,走第 0 节的一键脚本更快;两者是各自独立的部署路线。
|
||||
|
||||
---
|
||||
|
||||
## 0. 快速路径:AutoDL 一键部署(GPU)
|
||||
|
||||
在 AutoDL GPU 实例上,用仓库自带的一键脚本部署,无需手动装依赖 / 配 systemd:
|
||||
|
||||
```bash
|
||||
# 假设仓库已 checkout 到实例(脚本不会 git pull),使用 server-edition 分支
|
||||
cd /root/BGfilter-server # 你的实际路径
|
||||
scripts/deploy_autodl.sh --token <TOKEN>
|
||||
```
|
||||
|
||||
脚本**幂等**地完成:装依赖 + 拉模型权重 + 按 CUDA 版本装对应 CuPy(GPU 前景合成)
|
||||
+ 起一个 **supervisord** 实例(崩溃自动重启)+ 往 `~/.bashrc` 写交互登录自启,
|
||||
最后启动服务并校验 `/healthz`。
|
||||
|
||||
参数:
|
||||
|
||||
| 参数 | 默认 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `--token <TOKEN>` | 无 | 鉴权 token,设置/更新后 `/remove-background` 要求 header `X-Genarrative-Image-Token`。**省略则沿用已有 token**(重跑改别的参数不会误关鉴权);只有首次无 token 才开放。 |
|
||||
| `--open` | — | 显式删除 token、以开放模式运行(无鉴权)。 |
|
||||
| `--port <PORT>` | `6006` | 容器内监听端口(对应 AutoDL 的公网映射端口)。 |
|
||||
| `--config <cfg>` | `configs/gpu.yaml` | 流水线配置。 |
|
||||
| `--log-dir <dir>` | `~/autodl-tmp` | 日志目录(默认 AutoDL 持久数据盘——容器重置不丢,`/tmp` 会丢)。含 `bgfilter.log`(散文事件日志)和 `bgfilter-access.jsonl`(结构化每请求日志,应用自轮转)。 |
|
||||
|
||||
**前提**:仓库已 checkout(脚本不 `git pull`)、root 用户、torch/CUDA 由 AutoDL 镜像提供、
|
||||
python 在 `/root/miniconda3`。
|
||||
|
||||
**两个 AutoDL 固有的坑**(脚本管不了):
|
||||
|
||||
- **公网 URL 映射**(如 `6006 → https://...:8443`)要在 **AutoDL 网页控制台**里设,脚本不负责。
|
||||
- **容器重启后**服务不会自动起来——AutoDL 没有 systemd / 开机钩子,自启写在 `~/.bashrc` 里,
|
||||
需要**开一个交互式 shell(SSH / JupyterLab 终端)** 才会触发拉起。
|
||||
|
||||
**服务管理**(supervisorctl 与 python 同目录;socket 在 `~/bgfilter-supervisor.sock`,
|
||||
supervisord 配置在 `~/supervisord.conf`):
|
||||
|
||||
```bash
|
||||
S="/root/miniconda3/bin/supervisorctl -c $HOME/supervisord.conf"
|
||||
$S status bgfilter # 状态 + 运行时长
|
||||
$S restart bgfilter # 重启(如改了 configs/*.yaml 之后)
|
||||
$S stop bgfilter # 停服务(supervisord 本身继续跑)
|
||||
$S start bgfilter # 再启动
|
||||
$S shutdown # 全部停掉,包括 supervisord 自己
|
||||
|
||||
tail -f ~/autodl-tmp/bgfilter.log # 事件日志:启动/告警/堆栈(散文)
|
||||
tail -f ~/autodl-tmp/bgfilter-access.jsonl # 访问日志:每请求一条 JSON 记录
|
||||
tail -f ~/supervisord.log # supervisord 自己的日志(服务起不来时看)
|
||||
```
|
||||
|
||||
若 supervisord 本身没在跑——容器刚重启还没开过 shell,或报 socket 残留错误——照
|
||||
`~/.bashrc` 自启的方式手动拉起:
|
||||
|
||||
```bash
|
||||
rm -f ~/bgfilter-supervisor.sock ~/supervisord.pid
|
||||
/root/miniconda3/bin/supervisord -c ~/supervisord.conf
|
||||
```
|
||||
|
||||
改 `~/supervisord.conf` 本身(端口、日志路径等)的方式是重跑部署脚本——它会重新生成
|
||||
该文件并做 `reread + update + restart`。
|
||||
|
||||
> 改了 `configs/*.yaml`(比如开 `cross_check.reuse_as_seg`)后,用上面的 `restart bgfilter`
|
||||
> 让服务重新加载配置——配置只在启动时读入,没有请求级开关。
|
||||
|
||||
---
|
||||
|
||||
## 1. 前置条件
|
||||
|
||||
- Linux x86-64(在 Ubuntu 上验证过;其它主流发行版同理)。
|
||||
- 推荐 **Python 3.11–3.13**。更新的版本(如 3.14)也可能可用,但部分 ML 包可能还没发布对应
|
||||
wheel——尽量选一个 `torch` / `onnxruntime` / `opencv` / `pymatting` 都有预编译 wheel 的版本。
|
||||
- `git`,以及 venv 模块(Debian/Ubuntu 上是 `python3-venv`)。
|
||||
- 磁盘空间:Python 环境约 3 GB + 模型权重约 1 GB。
|
||||
- **一次性**联网即可(用于装依赖和下模型);服务跑起来后可完全离线。
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y python3-venv python3-pip git
|
||||
```
|
||||
|
||||
## 2. 获取代码
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/genarrative-image-host/bgfilter-service
|
||||
sudo chown "$USER" /opt/genarrative-image-host/bgfilter-service
|
||||
cd /opt/genarrative-image-host/bgfilter-service
|
||||
git clone <repo-url> . # 或把仓库拷到这里;使用 server-edition 分支
|
||||
```
|
||||
|
||||
## 3. Python 环境
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
. .venv/bin/activate
|
||||
python -m pip install -U pip
|
||||
```
|
||||
|
||||
## 4. 安装依赖(CPU)
|
||||
|
||||
**先从 CPU wheel 源装 PyTorch**——否则 `pip` 会拉默认的 CUDA 构建,附带约 2–3 GB 的
|
||||
`nvidia-*` 包,CPU 机器完全用不上:
|
||||
|
||||
```bash
|
||||
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
|
||||
```
|
||||
|
||||
`requirements.txt` 里已经固定用 **`opencv-python-headless`**(GUI 构建依赖 `libGL.so.1`
|
||||
等无头服务器没有的库),所以无需改动。直接装其余依赖——此时 `torch`/`torchvision`
|
||||
已满足,不会再拉 CUDA:
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
验证整套依赖可导入、且 torch 是 CPU 构建(`cuda= None`):
|
||||
|
||||
```bash
|
||||
python -c "import torch,cv2,fastapi,transformers,pymatting,onnxruntime,kornia,timm; \
|
||||
print('torch', torch.__version__, 'cuda=', torch.version.cuda)"
|
||||
```
|
||||
|
||||
> 在墙内,给第二条 `pip install` 加上 `-i https://pypi.tuna.tsinghua.edu.cn/simple` 提速。
|
||||
|
||||
## 5. 准备模型权重
|
||||
|
||||
默认用到三个模型:`hustvl/vitmatte-base-composition-1k`(抠图)、
|
||||
`ZhengPeng7/BiRefNet`(默认分割)、`skytnt/anime-seg`(可选分割)。
|
||||
第四个模型 `ZhengPeng7/BiRefNet_HR-matting`(约 425 MB)只有在启用 **cross-check(交叉校验)
|
||||
否决**时才需要——见本节末尾说明。
|
||||
|
||||
**一键方式(推荐)**:跑仓库自带脚本(无参数),经 hf-mirror 把所有模型下到 `models/`
|
||||
(绕过代理、禁用 Xet、已存在的文件夹自动跳过)。跑完主 app 即离线全功能可用:
|
||||
|
||||
```bash
|
||||
python scripts/fetch_weights.py
|
||||
```
|
||||
|
||||
下面两种手动放法用于更精细的控制(比如共享 HF 缓存、只下部分模型),**二选一**。
|
||||
|
||||
### 方案 A —— HuggingFace 缓存 + 离线模式(推荐)
|
||||
|
||||
先下载到一个缓存目录,之后离线运行:
|
||||
|
||||
```bash
|
||||
export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
|
||||
hf download hustvl/vitmatte-base-composition-1k
|
||||
hf download ZhengPeng7/BiRefNet
|
||||
hf download skytnt/anime-seg
|
||||
hf download ZhengPeng7/BiRefNet_HR-matting # 仅在使用 cross-check 时(见下)
|
||||
```
|
||||
|
||||
运行时把 `HF_HOME` 指向同一路径,并设 `HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1`(见 §6)。
|
||||
也可以在另一台机器下好,把 `hub/` 目录整个拷过来。
|
||||
|
||||
### 方案 B —— 权重作为普通文件夹放进项目
|
||||
|
||||
把每个模型放进 `models/<repo末段名>/`,加载器会直接用它(不走 HF 缓存、不联网)。
|
||||
没放对应文件夹时会回退到 repo id,所以这是可选功能。
|
||||
|
||||
**文件夹名必须和 repo 末段名完全一致**——加载器靠这个找到它
|
||||
(`ZhengPeng7/BiRefNet` → `models/BiRefNet`)。各模型需要的文件:
|
||||
|
||||
```text
|
||||
<项目根目录>/
|
||||
└── models/
|
||||
├── vitmatte-base-composition-1k/ # ViTMatte 抠图模型
|
||||
│ ├── config.json
|
||||
│ ├── preprocessor_config.json
|
||||
│ └── pytorch_model.bin # 约 369 MB
|
||||
├── BiRefNet/ # 默认分割模型(trust_remote_code)
|
||||
│ ├── config.json
|
||||
│ ├── BiRefNet_config.py # 自定义代码——trust_remote_code 必需
|
||||
│ ├── birefnet.py # 自定义代码——trust_remote_code 必需
|
||||
│ └── model.safetensors # 约 424 MB
|
||||
├── anime-seg/ # 可选分割模型(seg_model=anime-seg)
|
||||
│ └── isnetis.onnx # 约 168 MB
|
||||
└── BiRefNet_HR-matting/ # 可选——仅用于 cross-check 否决
|
||||
├── config.json
|
||||
├── BiRefNet_config.py # 自定义代码——trust_remote_code 必需
|
||||
├── birefnet.py # 自定义代码——trust_remote_code 必需
|
||||
└── model.safetensors # 约 425 MB
|
||||
```
|
||||
|
||||
说明:
|
||||
- **ViTMatte**:需要 `config.json`、`preprocessor_config.json` 和权重
|
||||
(`pytorch_model.bin`;若有 `model.safetensors` 也可)。
|
||||
- **BiRefNet**:两个 `.py` 文件是必需的——`trust_remote_code` 会执行它们来构建模型。
|
||||
和 `config.json` + `model.safetensors` 放在一起。(repo 里的 `handler.py` /
|
||||
`requirements.txt` / `README.md` 用不到。)
|
||||
- **anime-seg**:只读 `isnetis.onnx`,其它都不需要。
|
||||
- `anime-seg` 和 `BiRefNet_HR-matting` 都是可选的——不用 `seg_model=anime-seg` /
|
||||
cross-check 否决时,各自的文件夹可以不放。
|
||||
- **`cpu-fast` lane**(§6)只加载 **ViTMatte + BiRefNet**;8 GB 机器用这条 lane 时,
|
||||
`anime-seg` 和 `BiRefNet_HR-matting` 都可以完全不放。
|
||||
(`scripts/fetch_weights.py` 仍会把 4 个都下齐,以保证完整功能。)
|
||||
|
||||
最省事的填充方式是用 `hf download`(它也会顺带拉几个用不到的文件如 `README.md`,无害):
|
||||
|
||||
```bash
|
||||
hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k
|
||||
hf download ZhengPeng7/BiRefNet --local-dir models/BiRefNet
|
||||
hf download skytnt/anime-seg --local-dir models/anime-seg
|
||||
hf download ZhengPeng7/BiRefNet_HR-matting --local-dir models/BiRefNet_HR-matting # 仅 cross-check
|
||||
```
|
||||
|
||||
用 `BGFILTER_WEIGHTS_DIR` 可改基目录。`models/` 已被 gitignore。
|
||||
|
||||
### cross-check 否决(默认开启)
|
||||
|
||||
管线会再跑一个 matting 模型(`ZhengPeng7/BiRefNet_HR-matting`,约 425 MB)作为独立的
|
||||
"第二意见"——它做什么、为什么,见 [README_ZH.md](README_ZH.md) → *管线*。部署侧要点:
|
||||
|
||||
- **CLI 和 HTTP 服务都默认开启**,因此**必须备好这个模型**(上面方案 A 或 B)。
|
||||
⚠️ 离线服务器(`HF_HUB_OFFLINE=1`)必须在第一个请求前就备好它,否则每个默认请求都失败。
|
||||
- 服务级关闭:`BGFILTER_CROSS_CHECK=0`;按请求覆盖:表单字段 `cross_check`。
|
||||
开启时服务启动即预载。
|
||||
- 配置里 `cross_check.reuse_as_seg: true`(默认**关**)让这次前向兼作分割掩码:
|
||||
少载一个模型、CPU 快约 20 秒/张、GPU 省约 1 GB 显存。仅配置文件可设——改动需重启。
|
||||
|
||||
### 背景模式(flat vs complex)
|
||||
|
||||
`background_mode=complex`(非纯色/实景背景)**不需要额外权重**——复用已部署的
|
||||
BiRefNet 分割器 + ViTMatte,跳过色键相关阶段。cross-check 否决在 complex 模式下
|
||||
**默认关闭**(传 `cross_check=on` 启用)。字段语义与响应头见
|
||||
[README_ZH.md](README_ZH.md) → *HTTP 服务*。
|
||||
|
||||
### 墙内下载(国内网络)
|
||||
|
||||
若直连 HuggingFace 被墙,用镜像 **并绕过本地代理、禁用 Xet**——这是稳定可用的组合:
|
||||
|
||||
```bash
|
||||
export HF_ENDPOINT=https://hf-mirror.com # 国内镜像
|
||||
export NO_PROXY='*' # 绕过代理;镜像是国内直连
|
||||
export HF_HUB_DISABLE_XET=1 # 这些 repo 是 Xet 存储;强制走经典 HTTP
|
||||
```
|
||||
|
||||
这样能避开两个坑:(1) 镜像 + 走海外的代理会让镜像把 `resolve` 又重定向回
|
||||
`huggingface.co`,新版 `huggingface_hub` 会因此报 `FileMetadataError`;
|
||||
(2) 装了 `hf-xet` 时 Xet 下载路径在此环境下会瞬间失败。
|
||||
|
||||
## 6. 配置(环境变量)
|
||||
|
||||
| 变量 | 默认 | 作用 |
|
||||
| --- | --- | --- |
|
||||
| `BGFILTER_CONFIG` | `configs/default.yaml` | 管线配置文件 |
|
||||
| `BGFILTER_DEVICE` | `cpu` | 同时覆盖 model 和 segmentation 的 device(GPU 用 `cuda`)|
|
||||
| `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | 超过则返回 `413`(约 4 MP)|
|
||||
| `BGFILTER_ACCESS_LOG` | — | **结构化访问日志**的路径(JSONL,每请求一条记录 + 一条 `startup` 元记录;应用自轮转,50 MB × 5)。字段:`request_id`、参数、尺寸、分阶段耗时(`t_queue/fetch/decode/process/total_ms`)、状态码、失败时截断的 `exc`;该 id 同时以 `X-Request-ID` 响应头返回。不设置 = 记录回退到 stderr(开发用)。散文**事件日志**(启动/告警/堆栈)始终走 stderr 由 supervisord/journald 捕获——每请求数据绝不落在散文流里。 |
|
||||
| `BGFILTER_LOG_LEVEL` | `INFO` | 根日志级别(事件流)。 |
|
||||
| `BGFILTER_FETCH_MAX_BYTES` | `26214400` | `image_url` 模式:下载体上限(约 25 MB)→ `413`。无 SSRF 过滤——务必开鉴权 / 调用方可信。 |
|
||||
| `BGFILTER_FETCH_TIMEOUT` | `15` | `image_url` 模式:fetch 读超时(秒;连接超时固定 5 秒)。出站 fetch 并发固定为 `2 × cpu_workers`(硬顶 64)。 |
|
||||
| `BGFILTER_PRELOAD` | `1` | 启动时预加载默认模型(首个请求不用冷加载)|
|
||||
| `BGFILTER_CROSS_CHECK` | 配置(`on`)| 强制开/关 cross-check 否决(`0` 关闭;需 `BiRefNet_HR-matting`)|
|
||||
| `BGFILTER_CPU_WORKERS` | 配置 `server.cpu_workers`(`1`)| 请求流水线的线程池大小。`1` = 严格串行(一次一个请求)。`>1` 允许这么多请求并发,重叠各自的 CPU 段,同时用一把 GPU 互斥锁串行化模型前向——让 GPU 别闲着。CPU 为瓶颈时(如关 cross-check)收益最大。可写进配置的 `server:` 段,或用此 env 变量按机器覆盖。 |
|
||||
| `BGFILTER_GPU_CONCURRENCY` | 配置 `server.gpu_concurrency`(`1`)| 允许几个请求同时进入 GPU 前向。`1` = 单流(安全)。`2` 让两次前向在 SM 上共同调度(batch-1 喂不满 SM),提升 GPU 瓶颈下的吞吐——但并发激活会成倍占显存,按 `权重 + N × 单次前向峰值 ≤ 显存` 来定(32GB 卡上 `2` 在 2048 cross-check 时峰值约 19GB;≤16GB 卡保持 `1`)。需 `cpu_workers>1` 才有意义。配置键 `server.gpu_concurrency`,此 env 变量覆盖它。 |
|
||||
| `BGFILTER_AUTH_TOKEN` | — | 若设置,`/remove-background` 要求 header `X-Genarrative-Image-Token` 等于它(常量时间比较;否则 `401`)。不设 = **开放**(鉴权交给前置代理)。`/healthz` 始终开放。见 §10。 |
|
||||
| `BGFILTER_WEIGHTS_DIR` | `models` | 本地权重文件夹的查找目录(方案 B)|
|
||||
| `HF_HOME` | — | HF 缓存位置(方案 A)|
|
||||
| `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | 权重缓存好后,生产设为 `1` |
|
||||
| `OMP_NUM_THREADS` / `MKL_NUM_THREADS` | — | 限制 CPU 线程数(如 `4`)|
|
||||
|
||||
### 配置文件:`default.yaml` 与 `cpu-fast` lane
|
||||
|
||||
`BGFILTER_CONFIG` 决定用哪个管线配置。仓库自带两个:
|
||||
|
||||
- **`configs/default.yaml`** —— 完整画质。cross-check 否决**默认开**(额外加载
|
||||
`BiRefNet_HR-matting`@2048 作第二意见),边缘最好。较重:大图时峰值工作集轻松到
|
||||
两位数 GB,需要内存宽裕的机器(或 GPU)。
|
||||
- **`configs/cpu-fast.yaml`** —— 精简 CPU lane,为 **8 GB 机器**调校。fp32、只用 BiRefNet
|
||||
分割、**无 cross-check、无 anime-seg**、分块注意力(`attn_query_chunk: 512`)。只加载
|
||||
**两个**权重(base ViTMatte + BiRefNet)。实测峰值工作集:**2048×1152(2.4 MP)约 5.6 GB**、
|
||||
2048×2048(4.2 MP)约 7.1 GB;约 14–15 秒/张(1.5 MP)。
|
||||
|
||||
> **≤8 GB 的服务器请设 `BGFILTER_CONFIG=configs/cpu-fast.yaml`。** 默认 lane + 大图
|
||||
> 正是把 ViTMatte 注意力瞬时内存推到 OOM 的组合——worker 被 kill(SIGKILL),前面的
|
||||
> nginx 返回 **502**,而应用日志显示 `DefaultCPUAllocator: can't allocate memory`。
|
||||
|
||||
有两个模型旋钮放在**配置文件**里(`model:` 段),不是环境变量:
|
||||
|
||||
| 配置项 | 默认 | 作用 |
|
||||
| --- | --- | --- |
|
||||
| `model.precision` | `fp32` | `fp32` 或 `bf16`。bf16 **仅**在有原生内核的硬件上启用(CUDA,或带 AVX512-BF16/AMX 的 CPU);否则告警并回退 fp32。也可在 `segmentation:` / `cross_check:` 下按模型单独设。 |
|
||||
| `model.attn_query_chunk` | `2048` | ViTMatte 全局注意力的 query 行分块。调小(如 `512`)以几个百分点的额外算力换掉 O(N²) 内存瞬时尖峰;输出逐比特相同。设 `0` 关闭分块(走原始一次性注意力)。 |
|
||||
|
||||
> **despill 默认关闭**(所有 lane,含 cpu-fast)。它沿背景色调方向压前景色度,没有位置/语义护栏,所以主体与背景**同色相**时(如蓝背景前的蓝西装)会被褪色/偏色(约 ΔE 35)。仅当"去边缘溢色"比"同色系保真"更重要时,才在配置里 `despill: {enabled: true}` 单独开启。
|
||||
|
||||
## 7. 运行服务
|
||||
|
||||
```bash
|
||||
BGFILTER_ACCESS_LOG=logs/bgfilter-access.jsonl \
|
||||
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1 --no-access-log
|
||||
```
|
||||
|
||||
- **必须保留 `--workers 1`。** 每个 worker 各加载一整套模型,多 worker 会成倍占内存
|
||||
(8 GB 机器会 OOM)。服务已用全局锁串行化推理。
|
||||
- `--host 127.0.0.1` 只对本机开放(面向同机调用方)。跨机访问见 §10。
|
||||
- `--no-access-log` 关掉 uvicorn 自己的散文版每请求日志——它是应用结构化访问记录
|
||||
(§6 `BGFILTER_ACCESS_LOG`)的子集、纯重复,且每请求数据不该进散文流。
|
||||
|
||||
## 8. 作为 systemd 服务运行
|
||||
|
||||
`/etc/systemd/system/genarrative-bgfilter.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Genarrative BgFilter background removal service
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
|
||||
Environment=HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
|
||||
Environment=HF_HUB_OFFLINE=1
|
||||
Environment=TRANSFORMERS_OFFLINE=1
|
||||
Environment=HF_HUB_DISABLE_XET=1
|
||||
Environment=BGFILTER_CONFIG=configs/default.yaml
|
||||
Environment=BGFILTER_DEVICE=cpu
|
||||
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
|
||||
Environment=BGFILTER_PRELOAD=1
|
||||
Environment=BGFILTER_ACCESS_LOG=/opt/genarrative-image-host/bgfilter-service/logs/bgfilter-access.jsonl
|
||||
Environment=OMP_NUM_THREADS=4
|
||||
Environment=MKL_NUM_THREADS=4
|
||||
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1 --no-access-log
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
MemoryMax=8G
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/opt/genarrative-image-host
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
≤8 GB 机器请把配置那行改成精简 lane:
|
||||
`Environment=BGFILTER_CONFIG=configs/cpu-fast.yaml`(见 §6)——默认 lane 大图会 OOM。
|
||||
|
||||
若用方案 B(本地 `models/` 文件夹)而非 HF 缓存,去掉 `HF_HOME` 和离线那几行——
|
||||
那种权重不需要缓存也不联网。
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now genarrative-bgfilter
|
||||
sudo systemctl status genarrative-bgfilter
|
||||
journalctl -u genarrative-bgfilter -f # 等到 "Application startup complete"
|
||||
```
|
||||
|
||||
## 9. 验证
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:18083/healthz
|
||||
# {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu"}
|
||||
|
||||
curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
|
||||
http://127.0.0.1:18083/remove-background -o out.png
|
||||
|
||||
# 非纯色/实景背景(无色键;此模式 cross-check 默认关)
|
||||
curl -sS -F "file=@scene.jpg" -F "background_mode=complex" \
|
||||
http://127.0.0.1:18083/remove-background -o out.png
|
||||
```
|
||||
|
||||
## 10. 对外暴露
|
||||
|
||||
单元里绑的是 `127.0.0.1`——面向同机调用方(例如 Genarrative Rust BFF,通过
|
||||
`GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL` 调用)。要从其它主机访问:
|
||||
|
||||
- **反向代理(推荐):** 前面放 nginx/caddy,终止 TLS、加限流,转发到
|
||||
`127.0.0.1:18083`。
|
||||
- **直接暴露:** 把 `--host` 改成 `0.0.0.0`,并用防火墙限制来源 IP。
|
||||
切勿把无鉴权的服务裸暴露到公网(见下方鉴权)。
|
||||
|
||||
### 鉴权
|
||||
|
||||
app 层 token 鉴权是**可选开启**,由 `BGFILTER_AUTH_TOKEN` 控制:
|
||||
|
||||
- **设置它** → `/remove-background` 要求 header `X-Genarrative-Image-Token: <token>`
|
||||
(常量时间比较);缺失或错误返回 `401`。`/healthz` 保持开放。**当 app 被直接暴露**
|
||||
(`--host 0.0.0.0`、或云端端口映射无代理)时用这个。
|
||||
- **不设置** → app **开放**(fail-open),并在启动时打一条 WARNING。仅当前置代理
|
||||
已经鉴权时才可这样——例如生产机 nginx 自己校验 `X-Genarrative-Image-Token`,
|
||||
其后的 app 就无需 token。
|
||||
|
||||
header 名和 nginx 层一致,所以调用方只换 base URL + token 值,请求结构不变:
|
||||
|
||||
```bash
|
||||
curl -H "X-Genarrative-Image-Token: <token>" \
|
||||
-F "file=@input.png" \
|
||||
https://<host>/remove-background -o out.png
|
||||
```
|
||||
|
||||
## 11. 容量与性能
|
||||
|
||||
- CPU 吞吐:**`cpu-fast` lane** 约 **14–15 秒/张(1.5 MP)**(base ViTMatte + 一个
|
||||
BiRefNet@1024,分块注意力)。**默认 lane** 更慢——cross-check 会多跑一次 HR-matting@2048
|
||||
前向。请求被全局锁串行,单实例按每分钟几张估算。
|
||||
- **默认 lane,工作站 CPU 参考(Ryzen 9700X,原生 bf16,32 GB;cross-check 开且
|
||||
`reuse_as_seg` 开,分块注意力):**
|
||||
|
||||
| 输入 | 精度 | warm / 每图 | 峰值内存 |
|
||||
|---|---|---|---|
|
||||
| 1024×1536 | fp32 | ~45 s | — |
|
||||
| 1024×1536 | bf16 | ~33 s | — |
|
||||
| 2048×2048 | bf16 | ~53 s | 11.4 GB(批处理)、8.1 GB(单图) |
|
||||
|
||||
不开 `reuse_as_seg` 要加上主 BiRefNet@1024 前向(每图约 +20 s)。内存上限来自
|
||||
cross-check 的 HR 前向(无论图多大都固定 `input_size` 2048)。分块注意力之前,
|
||||
同一个 2048² bf16 run 峰值 25.2 GB;完全未优化的 fp32 估计要 55–60 GB。无原生
|
||||
bf16 的 CPU(如 Zen 2)自动回落 fp32——这类机器上内存或延迟紧张时优先
|
||||
`--no-cross-check`。
|
||||
- 要提吞吐:上 GPU(`BGFILTER_DEVICE=cuda`,装 CUDA 版 torch),大约快一个数量级;
|
||||
和/或多实例 + 负载均衡(每个都 `--workers 1`)。
|
||||
- **GPU 部署(实测,RTX 5070 Ti 16 GB,完整管线——cross-check 开,bf16):**
|
||||
约 1–2 秒/张(4 MP 也只要 2 秒),峰值 VRAM 分配约 10.9 GB(开 `reuse_as_seg`
|
||||
为 10.1 GB)。真正决定「装不装得下」的是分配器的 *reserved* 占用。`bgfilter`
|
||||
在 Linux/WSL2 上会**默认设置** `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`
|
||||
(在 torch 加载前;原生 Windows 不支持该选项;显式设过的环境变量优先)。
|
||||
实测效果:reserved 15.3 GB → **12.0 GB**(开 reuse;默认不开为 13.3 GB),且还
|
||||
**快约 10%**——完整管线在 16 GB 卡上即使旁边挂着桌面会话也有约 2–3.5 GB 余量。
|
||||
没有 expandable segments(如原生 Windows)时 reserved 约 15.3–16.2 GB:在
|
||||
16 GB 卡上从很悬到直接溢出。另外 `bf16` 是 16 GB 可行的前提——fp32 的 reserved
|
||||
约 17 GB,直接溢出。
|
||||
- 内存:分块注意力(`attn_query_chunk`)把 ViTMatte 的 O(N²) 瞬时尖峰削平,于是峰值由
|
||||
稠密计算而非注意力矩阵决定。`cpu-fast` lane 实测峰值 **约 5.6 GB@2.4MP / 7.1 GB@4.2MP**——
|
||||
配合默认 4 MP 输入上限,`MemoryMax=8G` 对单 CPU worker 是稳妥上限。默认 lane(cross-check 开)
|
||||
需要更多余量。
|
||||
- **让 RSS 保持平稳(glibc)。** 每处理完一张图,服务会调用 glibc 的 `malloc_trim(0)`
|
||||
(`bgfilter/memtune.py`)把 BiRefNet@2048 前向释放的大块激活还给 OS——否则 ptmalloc
|
||||
会把它们留在 arena 里,RSS 会随请求逐步抬高。想进一步降内存**并**提升 CPU 吞吐,可以
|
||||
预载 **jemalloc**(PyTorch 对 CPU 推理的推荐),并让它激进归还脏页:
|
||||
```ini
|
||||
# 放进 systemd unit 的 [Service] 段
|
||||
Environment=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
|
||||
Environment=MALLOC_CONF=background_thread:true,dirty_decay_ms:0,muzzy_decay_ms:0
|
||||
```
|
||||
(`apt install libjemalloc2`;tcmalloc 用 `libtcmalloc_minimal.so.4` 同理。)
|
||||
jemalloc/tcmalloc 自己管理归还,此时 `malloc_trim` 会自动空转。若无法预载分配器,
|
||||
在 unit 环境里设 `MALLOC_ARENA_MAX=2`、`MALLOC_TRIM_THRESHOLD_=0` 也能抑制 ptmalloc 滞留。
|
||||
|
||||
## 12. 故障排查
|
||||
|
||||
| 现象 | 原因 / 解决 |
|
||||
| --- | --- |
|
||||
| `pip` 拉了 `nvidia-*`/`cuda-*`(巨大) | §4 里漏了 CPU 源。用 `--index-url https://download.pytorch.org/whl/cpu` 装 torch。 |
|
||||
| `import cv2` 报 `ImportError: libGL.so.1` | 在无 GUI 机器上用了 `opencv-python`。换成 `opencv-python-headless`(§4)。 |
|
||||
| 建 venv 时 `ensurepip is not available` | 装 `python3-venv`(§1)。 |
|
||||
| 下载时 `FileMetadataError: ... not on huggingface.co` | 镜像与代理冲突。绕过代理(`NO_PROXY='*'`)并设 `HF_HUB_DISABLE_XET=1`(§5)。 |
|
||||
| `LocalEntryNotFoundError`、0 字节 | Xet 下载路径失败;设 `HF_HUB_DISABLE_XET=1`。 |
|
||||
| `503 service not ready` | 启动时模型还在加载;等 "Application startup complete"。 |
|
||||
| 刚设 `BGFILTER_CROSS_CHECK` 后(离线)报 `500 inference failed` | `BiRefNet_HR-matting` 没备好;下到缓存/`models/`(§5)。 |
|
||||
| 负载下 OOM / 被 kill | worker 多于一个,或 `MemoryMax` 太低。保持 `--workers 1`。 |
|
||||
| 大图时 worker 被 `kill`(SIGKILL)/ 返回 `502`,日志显示 `DefaultCPUAllocator: can't allocate memory` | 大图触发 ViTMatte OOM。改用 `BGFILTER_CONFIG=configs/cpu-fast.yaml`(§6),和/或调小 `BGFILTER_MAX_IMAGE_PIXELS`。 |
|
||||
| 大文件上传在代理层被 `413` 拒 | nginx `client_max_body_size`(默认 1 MB)——在 server 块里调大(如 `client_max_body_size 12m;`);这和应用的 `BGFILTER_MAX_IMAGE_PIXELS` 是两道独立限制。 |
|
||||
| `400 could not auto-detect background colour` | 输入没有干净的纯色边框;传 `screen_color=#RRGGBB`。 |
|
||||
@@ -1,5 +1,7 @@
|
||||
# BgFilter
|
||||
|
||||
> 中文: [README_ZH.md](README_ZH.md)
|
||||
|
||||
Offline character matting for AI-generated images on a flat-colour background.
|
||||
The background colour is **auto-detected** from the image border (green, pastel, any
|
||||
flat colour); pass `--screen-color` to set it explicitly.
|
||||
@@ -38,15 +40,15 @@ download (~0.9 GB) and one inference pass per image; disable with
|
||||
`--no-cross-check` (see the `cross_check` config section, and
|
||||
`docs/hair_gap_artifacts.md` for the analysis behind it).
|
||||
|
||||
When the cross-check is on, that same HR-matting forward is **reused as the
|
||||
segmentation mask by default**, skipping the primary seg model entirely (one
|
||||
less model to load, ~20 s faster per image on CPU). Pilot-validated
|
||||
(TestImage3 / FixImage1): trimap 99.8% identical, no structural change to
|
||||
fingers, hair wisps or thin lines. Disable with `--no-cross-check-as-seg` to
|
||||
run the dedicated seg model instead. The reuse is a same-family swap, so it
|
||||
When the cross-check is on, that same HR-matting forward can optionally be
|
||||
**reused as the segmentation mask** (`--cross-check-as-seg`), skipping the
|
||||
primary seg model entirely (one less model to load, ~20 s faster per image on
|
||||
CPU, ~1 GB less VRAM on GPU). Pilot-validated (TestImage3 / FixImage1): trimap
|
||||
99.8% identical, no structural change to fingers, hair wisps or thin lines.
|
||||
**Off by default**: the dedicated segmenter keeps its own forward and the veto
|
||||
stays an independent second signal. The reuse is a same-family swap, so it
|
||||
applies to the `birefnet` backend only: with `--seg-backend anime-seg` the
|
||||
anime segmenter keeps its own forward and the cross-check veto still runs
|
||||
independently on top.
|
||||
anime segmenter always keeps its own forward.
|
||||
|
||||
The segmentation trimap defaults to `directional` mode (chroma + seg + a hue-direction
|
||||
split: it keeps a background-coloured garment such as a white shirt while dropping a
|
||||
@@ -63,7 +65,7 @@ frame — detection **fails with an error**; pass `--screen-color` explicitly in
|
||||
|
||||
To set it yourself, give a hex prior:
|
||||
|
||||
```powershell
|
||||
```bash
|
||||
... --screen-color "#CFEFFF"
|
||||
```
|
||||
|
||||
@@ -74,26 +76,24 @@ auto-detected colour is used directly.)
|
||||
|
||||
## Environment
|
||||
|
||||
Use the conda environment `lightML`.
|
||||
Any Python 3.11–3.13 environment (venv or conda) works:
|
||||
|
||||
```powershell
|
||||
conda activate lightML
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
python -m bgfilter.cli --help
|
||||
```
|
||||
|
||||
If the shell is not activated, call the environment Python directly:
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli --help
|
||||
```
|
||||
For a server install — CPU-wheel torch, offline weights, systemd — follow
|
||||
[DEPLOY.md](DEPLOY.md) instead; it covers the same dependencies with the
|
||||
production choices spelled out.
|
||||
|
||||
## Single Image
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
--input Samples\TestImage.png `
|
||||
--output Outputs\TestImage_rgba.png `
|
||||
--debug-dir Outputs\TestImage_debug
|
||||
```bash
|
||||
python -m bgfilter.cli \
|
||||
--input Samples/TestImage.png \
|
||||
--output Outputs/TestImage_rgba.png \
|
||||
--debug-dir Outputs/TestImage_debug
|
||||
```
|
||||
|
||||
No `--config` is needed — the built-in defaults are identical to
|
||||
@@ -110,32 +110,19 @@ auto-detect the flat background), and `--trimap-mode`.
|
||||
|
||||
## Batch
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
--input-dir Samples `
|
||||
--output-dir Outputs `
|
||||
--debug-dir Outputs\debug
|
||||
```bash
|
||||
python -m bgfilter.cli \
|
||||
--input-dir Samples \
|
||||
--output-dir Outputs \
|
||||
--debug-dir Outputs/debug
|
||||
```
|
||||
|
||||
## Performance (CPU reference numbers)
|
||||
## Performance
|
||||
|
||||
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, default settings
|
||||
(cross-check on, reused as seg; chunked global attention on):
|
||||
|
||||
| input | precision | warm / image | peak memory |
|
||||
|---|---|---|---|
|
||||
| 1024x1536 | fp32 | ~45 s | — |
|
||||
| 1024x1536 | bf16 | ~33 s | — |
|
||||
| 2048x2048 | bf16 | ~53 s | 11.4 GB (batch), 8.1 GB (single image) |
|
||||
|
||||
The memory ceiling is the cross-check BiRefNet_HR forward (fixed
|
||||
`input_size` 2048 regardless of the image size) — genuine live activations
|
||||
of a full-resolution dense prediction net. Before chunked attention and the
|
||||
mimalloc fix, the same 2048x2048 bf16 run peaked at 25.2 GB; a fully
|
||||
un-optimized fp32 run would need an estimated 55-60 GB (ViTMatte's un-chunked
|
||||
N^2 attention alone ~39 GB). CPUs without native bf16 (e.g. Zen 2) auto-fall
|
||||
back to fp32 — on such machines prefer `--no-cross-check` if memory or
|
||||
latency is tight.
|
||||
Rough orders of magnitude: **~33–53 s/image** on a fast desktop CPU (bf16,
|
||||
cross-check on), **~1–2 s/image** on a modern GPU. All measured numbers — CPU
|
||||
lanes, GPU VRAM footprints, memory ceilings and the knobs that move them — live
|
||||
in [DEPLOY.md](DEPLOY.md) §11 (*Capacity & performance*).
|
||||
|
||||
## Chroma-alpha debug mode
|
||||
|
||||
@@ -143,12 +130,12 @@ latency is tight.
|
||||
alpha seed. Useful for fast inspection of chroma confidence, trimap, and despill.
|
||||
(Distinct from the chroma-only *pipeline* above, which still runs ViTMatte.)
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
--input-dir Samples `
|
||||
--output-dir Outputs\chroma `
|
||||
--debug-dir Outputs\chroma_debug `
|
||||
--matting-method chroma `
|
||||
```bash
|
||||
python -m bgfilter.cli \
|
||||
--input-dir Samples \
|
||||
--output-dir Outputs/chroma \
|
||||
--debug-dir Outputs/chroma_debug \
|
||||
--matting-method chroma \
|
||||
--device cpu
|
||||
```
|
||||
|
||||
@@ -172,37 +159,140 @@ qa_grid.png
|
||||
metadata.json
|
||||
```
|
||||
|
||||
## HTTP service
|
||||
|
||||
An HTTP wrapper (`app.py` + `bgfilter/service.py`) exposes the pipeline as a
|
||||
long-running FastAPI service. Models load once and are reused across requests.
|
||||
For deploying to a Linux server (dependencies, weights, systemd, offline mode, auth),
|
||||
see [DEPLOY.md](DEPLOY.md).
|
||||
|
||||
```bash
|
||||
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
|
||||
```
|
||||
|
||||
Run a single worker (`--workers 1`) — each worker loads its own copy of the models.
|
||||
|
||||
### Endpoints
|
||||
|
||||
- **`GET /healthz`** — liveness + config JSON, e.g.
|
||||
`{"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu","crossCheck":true}`.
|
||||
Always unauthenticated.
|
||||
- **`POST /remove-background`** — `multipart/form-data` in, `image/png` (RGBA) out.
|
||||
|
||||
### Authentication
|
||||
|
||||
Optional, opt-in via the `BGFILTER_AUTH_TOKEN` env var. When set, `/remove-background`
|
||||
requires header **`X-Genarrative-Image-Token: <token>`** (constant-time compare); a
|
||||
missing or wrong token returns `401`. When unset the endpoint is open (a fronting proxy
|
||||
is expected to authenticate). `/healthz` is always open. See [DEPLOY.md](DEPLOY.md) §10.
|
||||
|
||||
### Request fields (`POST /remove-background`, `multipart/form-data`)
|
||||
|
||||
| Field | Required | Values | Meaning |
|
||||
| --- | --- | --- | --- |
|
||||
| `file` | one of `file`/`image_url` | image file | Source image as an upload (field name fixed as `file`). |
|
||||
| `image_url` | one of `file`/`image_url` | `http(s)://…` | Source image by URL — the service fetches it (bounded by size/time; follows ≤3 redirects). Provide exactly one of `file` or `image_url`. No SSRF filtering: the endpoint trusts authenticated callers, so keep auth on and callers trusted. |
|
||||
| `background_mode` | no | `flat` (default) / `complex` | `flat` = solid-colour background (colour-keyed pipeline). `complex` = non-flat/scene background: no colour key, the segmenter alone drives the matte. `screen_color` is ignored in `complex`; expect flat backgrounds to stay stronger on hair detail. |
|
||||
| `screen_color` | no | `#RRGGBB`, or omit/empty/`auto` | Background-colour prior; omit to auto-detect from the border. Ignored when `background_mode=complex`. |
|
||||
| `seg_model` | no | `birefnet` (default) / `anime-seg` | Segmentation backend. |
|
||||
| `cross_check` | no | `1`/`0` (also `true`/`false`/`yes`/`no`/`on`/`off`) | Per-request override of the cross-model veto; omit/empty = service default in `flat` mode, but **defaults off** in `complex` mode (set `on` to enable its hue-free gate). |
|
||||
|
||||
### Response headers (on `200`)
|
||||
|
||||
| Header | Example | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `X-BGFilter-Elapsed-Ms` | `2372` | Server-side processing time (ms). |
|
||||
| `X-BGFilter-Seg-Model` | `birefnet` | Segmenter actually used. |
|
||||
| `X-BGFilter-Screen-Color` | `#CFEFFF` / `auto` | Background colour used. |
|
||||
| `X-BGFilter-Cross-Check` | `on` / `off` | Effective cross-check state. |
|
||||
| `X-BGFilter-Background-Mode` | `flat` / `complex` | Background mode actually used. |
|
||||
| `X-Request-ID` | `42efe4e0d76f4b83` | Correlation id (echoes an inbound `X-Request-ID` if you send one, else generated). Quote it to correlate with server logs. Returned on every response, including errors. |
|
||||
|
||||
### Status codes
|
||||
|
||||
| Code | When |
|
||||
| --- | --- |
|
||||
| `200` | Success — body is the RGBA PNG. |
|
||||
| `400` | Bad input: unreadable image, not exactly one of `file`/`image_url`, bad `image_url` / fetch failed, invalid `screen_color`/`seg_model`/`cross_check`/`background_mode`, or auto-detect failed (pass `screen_color`). |
|
||||
| `401` | Auth enabled and the token is missing or wrong. |
|
||||
| `413` | Image exceeds `BGFILTER_MAX_IMAGE_PIXELS` (~4 MP by default), or an `image_url` body exceeds `BGFILTER_FETCH_MAX_BYTES` (~25 MB). |
|
||||
| `500` | Inference failed. |
|
||||
| `503` | Service still loading models (not ready). |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# default (birefnet + auto background colour); add the auth header if the service requires it
|
||||
curl -sS -F "file=@input.png" \
|
||||
-H "X-Genarrative-Image-Token: <token>" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
|
||||
# explicit background colour + anime segmenter, cross-check off for this request
|
||||
curl -sS \
|
||||
-F "file=@input.png" \
|
||||
-F "screen_color=#CFEFFF" \
|
||||
-F "seg_model=anime-seg" \
|
||||
-F "cross_check=off" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
|
||||
# non-flat / scene background (no colour key); cross-check defaults off here,
|
||||
# add cross_check=on to enable the hue-free residue veto
|
||||
curl -sS \
|
||||
-F "file=@scene.jpg" \
|
||||
-F "background_mode=complex" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
|
||||
# by URL instead of an upload (service fetches it; use image_url OR file, not both)
|
||||
curl -sS \
|
||||
-F "image_url=https://example.com/input.png" \
|
||||
-F "screen_color=#CFEFFF" \
|
||||
-H "X-Genarrative-Image-Token: <token>" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
```
|
||||
|
||||
Runtime configuration (config file, device, size caps, preload, auth token,
|
||||
pipelining workers) is all via `BGFILTER_*` environment variables — the full table
|
||||
is [DEPLOY.md](DEPLOY.md) §6.
|
||||
|
||||
Like the CLI, the service runs the **cross-model veto by default**; `/healthz`
|
||||
reports the effective default as `crossCheck` and the per-request `cross_check`
|
||||
field overrides it either way (service-wide kill switch: `BGFILTER_CROSS_CHECK=0`).
|
||||
`cross_check.reuse_as_seg` (config-only, default off — see *Pipelines* above) has
|
||||
no per-request switch; changing it means editing the config and restarting.
|
||||
|
||||
## Quality Check
|
||||
|
||||
The quality checker measures alpha validity and edge spill on semi-transparent edge
|
||||
pixels.
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.quality_cli `
|
||||
Outputs\TestImage_rgba.png `
|
||||
```bash
|
||||
python -m bgfilter.quality_cli \
|
||||
Outputs/TestImage_rgba.png \
|
||||
--max-edge-green-excess-p95 0.30
|
||||
```
|
||||
|
||||
Run the bundled sample smoke check:
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe scripts\smoke_samples.py `
|
||||
--samples-dir Samples `
|
||||
--output-dir Outputs\smoke_samples `
|
||||
--config configs\default.yaml `
|
||||
--device cpu `
|
||||
--fallback-to-chroma-alpha `
|
||||
```bash
|
||||
python scripts/smoke_samples.py \
|
||||
--samples-dir Samples \
|
||||
--output-dir Outputs/smoke_samples \
|
||||
--config configs/default.yaml \
|
||||
--device cpu \
|
||||
--fallback-to-chroma-alpha \
|
||||
--max-edge-green-excess-p95 0.30
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- `Samples/` and `Outputs/` are ignored by Git.
|
||||
- ViTMatte and segmentation weights load from Hugging Face on first use. Behind a
|
||||
firewall set `HF_ENDPOINT=https://hf-mirror.com` (and bypass a flaky local proxy).
|
||||
- ViTMatte and segmentation weights load from Hugging Face on first use.
|
||||
`anime-seg` (`skytnt/anime-seg`) is a plain ONNX download; `birefnet`
|
||||
(`ZhengPeng7/BiRefNet`) ships custom modelling code so it needs
|
||||
`trust_remote_code=True` plus `timm` / `einops` / `kornia`.
|
||||
- **Provisioning weights** — offline HF cache, plain `models/` folders
|
||||
(`BGFILTER_WEIGHTS_DIR`), the `scripts/fetch_weights.py` one-shot downloader,
|
||||
and the CN-network mirror workaround — is covered in [DEPLOY.md](DEPLOY.md) §5.
|
||||
- Foreground colour estimation uses pymatting's `estimate_foreground_ml` to propagate
|
||||
clean foreground colour into semi-transparent edges before de-spill. Set
|
||||
`foreground.method: unmix` to fall back to the legacy heuristic.
|
||||
|
||||
+276
@@ -0,0 +1,276 @@
|
||||
# BgFilter
|
||||
|
||||
> English: [README.md](README.md)
|
||||
|
||||
面向"纯色背景 AI 生成图"的离线人物抠图。背景色会**从图像边框自动探测**(绿色、
|
||||
淡彩、任意纯色);也可用 `--screen-color` 显式指定。
|
||||
|
||||
```text
|
||||
RGB 输入
|
||||
-> chroma 背景置信度(按自动探测或指定的颜色打分)
|
||||
-> [可选] 语义分割 mask
|
||||
-> trimap -> ViTMatte -> alpha 清理
|
||||
-> 交叉模型否决(对背景色调残留做第二意见)
|
||||
-> pymatting 前景估计 -> despill 去溢色 -> RGBA PNG -> QA 预览
|
||||
```
|
||||
|
||||
## 管线
|
||||
|
||||
两条管线,由配置里的 `segmentation.enabled` 选择:
|
||||
|
||||
- **仅 Chroma**(`enabled: false`)—— Chroma + ViTMatte,无分割模型。最轻/最快;
|
||||
完全靠颜色键决定拓扑。
|
||||
- **单分割器**(`enabled: true`,默认)—— 一个分割模型定 trimap 拓扑(镂空、发丝),
|
||||
ViTMatte 再细化软边。后端可切换:`birefnet`(默认,通用显著目标——文字、logo、
|
||||
照片;需 `trust_remote_code`)或 `anime-seg`(ONNX,专为动漫角色调优)。运行时用
|
||||
`--seg-backend anime-seg` 切换(会同时选好对应权重)。
|
||||
|
||||
两条管线都共用 pymatting 前景估计和一次颜色 de-spill。**没有**染绿修复 / recolour
|
||||
层——源图干净时它没必要,已移除。
|
||||
|
||||
两条管线**默认都跑交叉模型否决**:第二个、无需 trimap 的抠图模型
|
||||
(`ZhengPeng7/BiRefNet_HR-matting`)只能*降低* alpha,且只作用于主结果自信、
|
||||
但偏背景色调的亮像素——用来清掉卡在发丝间、被 chroma、分割器和 ViTMatte 都误判为
|
||||
前景的偏色背景残留。代价是多下一个模型(~0.9 GB)+ 每图一次额外推理;用
|
||||
`--no-cross-check` 关闭(见 `cross_check` 配置段,分析见 `docs/hair_gap_artifacts.md`)。
|
||||
|
||||
开启交叉否决时,那趟 HR-matting 前向可以选择性地**兼作分割 mask**
|
||||
(`--cross-check-as-seg`),从而完全跳过主分割模型(少加载一个模型,CPU 上每图约快
|
||||
20 s,GPU 上省约 1 GB 显存)。已试点验证(TestImage3 / FixImage1):trimap 99.8% 一致,
|
||||
手指、发丝、细线无结构性变化。**默认关闭**:专用分割器保留自己的前向,否决作为独立
|
||||
的第二信号。该复用是同族替换,故**仅**对 `birefnet` 后端生效:用
|
||||
`--seg-backend anime-seg` 时动漫分割器始终保留自己的前向。
|
||||
|
||||
分割 trimap 默认走 `directional` 模式(chroma + seg + 色调方向切分:保留白衬衫这类
|
||||
背景色系的衣物,同时丢弃卡在发丝间的偏背景色残留如蓝色)。可切换
|
||||
`--trimap-mode seg`(仅拓扑,无色调切分)或 `directional-hard-bg`(激进——硬删偏背景
|
||||
色像素;可能吃掉冷色/阴影下的白布)。
|
||||
|
||||
## 背景色
|
||||
|
||||
默认(`screen_color: null`)背景色**从图像边框自动探测**:边框条带的主导纯色成为键色。
|
||||
若边框不是单一干净纯色——渐变、纹理、或主体占满画面——探测会**报错**;此时请显式传
|
||||
`--screen-color`。
|
||||
|
||||
要自己指定,给一个十六进制先验:
|
||||
|
||||
```bash
|
||||
... --screen-color "#CFEFFF"
|
||||
```
|
||||
|
||||
或在配置里写 `screen_color: "#CFEFFF"`。无论哪种,chroma 键都按到该颜色的感知
|
||||
(Lab/RGB)距离给像素打分,de-spill 沿该颜色方向压掉色度。(显式给的 hex 会用附近边框
|
||||
像素做微调;自动探测的颜色直接使用。)
|
||||
|
||||
## 环境
|
||||
|
||||
任意 Python 3.11–3.13 环境(venv 或 conda)均可:
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
python -m bgfilter.cli --help
|
||||
```
|
||||
|
||||
服务器安装(CPU wheel 版 torch、离线权重、systemd)请直接按 [DEPLOY_ZH.md](DEPLOY_ZH.md)
|
||||
走——同样的依赖,但把生产环境的取舍都写明了。
|
||||
|
||||
## 单张图
|
||||
|
||||
```bash
|
||||
python -m bgfilter.cli \
|
||||
--input Samples/TestImage.png \
|
||||
--output Outputs/TestImage_rgba.png \
|
||||
--debug-dir Outputs/TestImage_debug
|
||||
```
|
||||
|
||||
无需 `--config`——内置默认值与 `configs/default.yaml` 完全一致。只有当你编辑了该文件
|
||||
去调细节参数时,才需要 `--config configs\default.yaml`。运行时选择留在命令行:
|
||||
`--device`(默认 **CPU**;GPU 用 `--device cuda`)、`--precision`(默认 `fp32`;`bf16`
|
||||
给三个模型加速、并把抠图模型的激活显存减半,alpha 视觉上一致——需 bf16 硬件,否则回落
|
||||
fp32;大输入还会默认对 ViTMatte 全局注意力按 query 分块——精确、逐比特相同,把 2048x2048
|
||||
的内存尖峰从 ~19 GB 压到 ~4 GB,见 `model.attn_query_chunk`)、`--seg-backend`
|
||||
(默认 `birefnet`;动漫角色用 `anime-seg`)、`--screen-color`(默认:自动探测纯背景)、
|
||||
以及 `--trimap-mode`。
|
||||
|
||||
## 批量
|
||||
|
||||
```bash
|
||||
python -m bgfilter.cli \
|
||||
--input-dir Samples \
|
||||
--output-dir Outputs \
|
||||
--debug-dir Outputs/debug
|
||||
```
|
||||
|
||||
## 性能
|
||||
|
||||
量级参考:快速桌面 CPU(bf16、cross-check 开)约 **33–53 s/图**,现代 GPU 约
|
||||
**1–2 s/图**。全部实测数据——CPU 各 lane、GPU 显存占用、内存上限及相关旋钮——
|
||||
见 [DEPLOY_ZH.md](DEPLOY_ZH.md) §11(容量与性能)。
|
||||
|
||||
## Chroma-alpha 调试模式
|
||||
|
||||
`--matting-method chroma` 跳过 ViTMatte,直接用 chroma 置信度作为 alpha 种子。适合快速
|
||||
检查 chroma 置信度、trimap 和 despill。(不同于上面的仅 Chroma *管线*,后者仍跑 ViTMatte。)
|
||||
|
||||
```bash
|
||||
python -m bgfilter.cli \
|
||||
--input-dir Samples \
|
||||
--output-dir Outputs/chroma \
|
||||
--debug-dir Outputs/chroma_debug \
|
||||
--matting-method chroma \
|
||||
--device cpu
|
||||
```
|
||||
|
||||
## 输出
|
||||
|
||||
每处理一张图,CLI 写一张 RGBA PNG 和可选的调试文件:
|
||||
|
||||
```text
|
||||
bg_confidence.png
|
||||
trimap.png
|
||||
seg_mask.png # 仅分割管线
|
||||
alpha.png
|
||||
foreground_rgb.png # 去溢色后的前景颜色
|
||||
color_mask.png # 逐像素 despill 权重
|
||||
preview_black.png
|
||||
preview_white.png
|
||||
preview_gray.png
|
||||
preview_red.png
|
||||
preview_blue.png
|
||||
qa_grid.png
|
||||
metadata.json
|
||||
```
|
||||
|
||||
## HTTP 服务
|
||||
|
||||
一个 HTTP 封装(`app.py` + `bgfilter/service.py`)把管线暴露为常驻 FastAPI 服务。模型
|
||||
只加载一次、跨请求复用。部署到 Linux 服务器(依赖、权重、systemd、离线模式、鉴权)见
|
||||
[DEPLOY_ZH.md](DEPLOY_ZH.md)。
|
||||
|
||||
```bash
|
||||
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
|
||||
```
|
||||
|
||||
单 worker 运行(`--workers 1`)—— 每个 worker 各自加载一份模型。
|
||||
|
||||
### 端点
|
||||
|
||||
- **`GET /healthz`** —— 存活 + 配置 JSON,例如
|
||||
`{"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu","crossCheck":true}`。
|
||||
始终免鉴权。
|
||||
- **`POST /remove-background`** —— 入参 `multipart/form-data`,返回 `image/png`(RGBA)。
|
||||
|
||||
### 鉴权
|
||||
|
||||
可选、按需开启,由环境变量 `BGFILTER_AUTH_TOKEN` 控制。设置后,`/remove-background`
|
||||
要求 header **`X-Genarrative-Image-Token: <token>`**(常量时间比较);缺失或错误返回
|
||||
`401`。不设时端点开放(预期由前置代理鉴权)。`/healthz` 始终开放。见
|
||||
[DEPLOY_ZH.md](DEPLOY_ZH.md) §10。
|
||||
|
||||
### 请求字段(`POST /remove-background`,`multipart/form-data`)
|
||||
|
||||
| 字段 | 必填 | 取值 | 含义 |
|
||||
| --- | --- | --- | --- |
|
||||
| `file` | `file`/`image_url` 二选一 | 图片文件 | 以上传方式提供源图(字段名固定为 `file`)。 |
|
||||
| `image_url` | `file`/`image_url` 二选一 | `http(s)://…` | 以 URL 提供源图,服务端自行 fetch(受大小/超时限制,跟随 ≤3 跳重定向)。`file` 与 `image_url` 必须恰好提供一个。无 SSRF 过滤:端点信任已鉴权的调用方,请保持鉴权开启且调用方可信。 |
|
||||
| `background_mode` | 否 | `flat`(默认)/ `complex` | `flat` = 纯色背景(走色键管线)。`complex` = 非纯色/实景背景:无色键,完全由分割器驱动抠图;此模式下 `screen_color` 被忽略,发丝细节通常弱于纯色模式。 |
|
||||
| `screen_color` | 否 | `#RRGGBB`,或留空/`auto` | 背景色先验;留空则从边框自动探测。`background_mode=complex` 时忽略。 |
|
||||
| `seg_model` | 否 | `birefnet`(默认)/ `anime-seg` | 分割后端。 |
|
||||
| `cross_check` | 否 | `1`/`0`(也接受 `true`/`false`/`yes`/`no`/`on`/`off`) | 按请求覆盖交叉否决;留空在 `flat` 模式 = 服务端默认,但在 `complex` 模式**默认关闭**(传 `on` 启用其无色相门控)。 |
|
||||
|
||||
### 响应头(`200` 时)
|
||||
|
||||
| 头 | 示例 | 含义 |
|
||||
| --- | --- | --- |
|
||||
| `X-BGFilter-Elapsed-Ms` | `2372` | 服务端处理耗时(ms)。 |
|
||||
| `X-BGFilter-Seg-Model` | `birefnet` | 实际使用的分割器。 |
|
||||
| `X-BGFilter-Screen-Color` | `#CFEFFF` / `auto` | 实际使用的背景色。 |
|
||||
| `X-BGFilter-Cross-Check` | `on` / `off` | 本次实际的 cross-check 状态。 |
|
||||
| `X-BGFilter-Background-Mode` | `flat` / `complex` | 本次实际的背景模式。 |
|
||||
| `X-Request-ID` | `42efe4e0d76f4b83` | 关联 id(你若传入 `X-Request-ID` 则回显,否则自动生成)。报障时报这个 id 即可对上服务端日志。每个响应都返回,含错误响应。 |
|
||||
|
||||
### 状态码
|
||||
|
||||
| 码 | 何时 |
|
||||
| --- | --- |
|
||||
| `200` | 成功——响应体是 RGBA PNG。 |
|
||||
| `400` | 输入有误:图片无法解码,`file`/`image_url` 不是恰好其一,`image_url` 非法/fetch 失败,`screen_color`/`seg_model`/`cross_check`/`background_mode` 非法,或自动探测失败(请传 `screen_color`)。 |
|
||||
| `401` | 鉴权已开且 token 缺失或错误。 |
|
||||
| `413` | 图片超过 `BGFILTER_MAX_IMAGE_PIXELS`(默认约 4 MP),或 `image_url` 响应体超过 `BGFILTER_FETCH_MAX_BYTES`(默认约 25 MB)。 |
|
||||
| `500` | 推理失败。 |
|
||||
| `503` | 服务仍在加载模型(未就绪)。 |
|
||||
|
||||
### 示例
|
||||
|
||||
```bash
|
||||
# 默认(birefnet + 自动背景色);若服务要求鉴权则加上 auth header
|
||||
curl -sS -F "file=@input.png" \
|
||||
-H "X-Genarrative-Image-Token: <token>" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
|
||||
# 显式背景色 + 动漫分割器,本次关闭 cross-check
|
||||
curl -sS \
|
||||
-F "file=@input.png" \
|
||||
-F "screen_color=#CFEFFF" \
|
||||
-F "seg_model=anime-seg" \
|
||||
-F "cross_check=off" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
|
||||
# 非纯色/实景背景(无色键);此模式 cross-check 默认关闭,
|
||||
# 需要时加 cross_check=on 启用无色相残留否决
|
||||
curl -sS \
|
||||
-F "file=@scene.jpg" \
|
||||
-F "background_mode=complex" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
|
||||
# 用 URL 代替上传(服务端自行 fetch;image_url 与 file 二选一,不可同时给)
|
||||
curl -sS \
|
||||
-F "image_url=https://example.com/input.png" \
|
||||
-F "screen_color=#CFEFFF" \
|
||||
-H "X-Genarrative-Image-Token: <token>" \
|
||||
http://127.0.0.1:18083/remove-background -o output.png
|
||||
```
|
||||
|
||||
运行时配置(配置文件、device、尺寸上限、预加载、鉴权 token、流水线 worker 数)全部
|
||||
通过 `BGFILTER_*` 环境变量——完整表格见 [DEPLOY_ZH.md](DEPLOY_ZH.md) §6。
|
||||
|
||||
与 CLI 一样,服务**默认开启交叉模型否决**;`/healthz` 以 `crossCheck` 报告生效的默认值,
|
||||
按请求的 `cross_check` 表单字段可任意覆盖(全服务关闭用 `BGFILTER_CROSS_CHECK=0`)。
|
||||
`cross_check.reuse_as_seg`(仅配置文件可设,默认关——见上方"管线")没有请求级开关;
|
||||
改它意味着改配置并重启服务。
|
||||
|
||||
## 质量检查
|
||||
|
||||
质量检查器在半透明边缘像素上度量 alpha 有效性和边缘溢色。
|
||||
|
||||
```bash
|
||||
python -m bgfilter.quality_cli \
|
||||
Outputs/TestImage_rgba.png \
|
||||
--max-edge-green-excess-p95 0.30
|
||||
```
|
||||
|
||||
跑内置样本冒烟检查:
|
||||
|
||||
```bash
|
||||
python scripts/smoke_samples.py \
|
||||
--samples-dir Samples \
|
||||
--output-dir Outputs/smoke_samples \
|
||||
--config configs/default.yaml \
|
||||
--device cpu \
|
||||
--fallback-to-chroma-alpha \
|
||||
--max-edge-green-excess-p95 0.30
|
||||
```
|
||||
|
||||
## 备注
|
||||
|
||||
- `Samples/` 和 `Outputs/` 被 Git 忽略。
|
||||
- ViTMatte 和分割权重在首次使用时从 Hugging Face 加载。`anime-seg`
|
||||
(`skytnt/anime-seg`)是普通 ONNX 下载;`birefnet`(`ZhengPeng7/BiRefNet`)自带
|
||||
自定义建模代码,故需 `trust_remote_code=True` 外加 `timm` / `einops` / `kornia`。
|
||||
- **权重准备**——离线 HF 缓存、`models/` 纯文件夹(`BGFILTER_WEIGHTS_DIR`)、一键下载
|
||||
脚本 `scripts/fetch_weights.py`、以及国内网络的镜像 workaround——见
|
||||
[DEPLOY_ZH.md](DEPLOY_ZH.md) §5。
|
||||
- 前景颜色估计用 pymatting 的 `estimate_foreground_ml`,在 de-spill 前把干净前景色
|
||||
传播进半透明边缘。设 `foreground.method: unmix` 回落到旧的启发式方法。
|
||||
- `docs/green_screen_matting_workflow.md` 是最初的第一阶段绿幕规格;本 README 反映当前
|
||||
的、泛化后的架构。
|
||||
@@ -1,12 +1,24 @@
|
||||
"""Flat-background character matting pipeline."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
# Windows PyTorch allocates CPU tensors through a bundled mimalloc whose lazy
|
||||
# purge keeps ~10 GB of already-freed forward activations committed after a
|
||||
# BiRefNet@2048 pass, stacking under ViTMatte's attention peak (measured
|
||||
# 2048x2048 bf16: peak 25.2 -> 21.4 GB with purge 0, and slightly *faster*).
|
||||
# Must be set before the torch DLL loads; harmless where mimalloc is not used.
|
||||
# The Linux/glibc equivalent (ptmalloc arena retention) is handled at runtime by
|
||||
# bgfilter.memtune.release_freed_memory(); see DEPLOY for jemalloc as an alt.
|
||||
os.environ.setdefault("MIMALLOC_PURGE_DELAY", "0")
|
||||
|
||||
# Expandable segments defuse the CUDA caching allocator's fragmentation: the
|
||||
# full bf16 pipeline's reserved VRAM drops 15.3 -> 12.0 GB on a 16 GB card and
|
||||
# runs ~10% faster (measured, RTX 5070 Ti). Read at CUDA-allocator init, so it
|
||||
# must be set before torch loads; a no-op for CPU-only runs. Native Windows
|
||||
# does not support it (torch warns and ignores), so it is only defaulted where
|
||||
# it works (Linux/WSL2); setdefault keeps any explicit user setting in charge.
|
||||
if not sys.platform.startswith("win"):
|
||||
os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
|
||||
|
||||
__version__ = "0.1.0"
|
||||
|
||||
+13
-6
@@ -59,7 +59,7 @@ def suppress_alpha_by_chroma(
|
||||
def cross_check_alpha(
|
||||
alpha: np.ndarray,
|
||||
second_alpha: np.ndarray,
|
||||
proj: np.ndarray,
|
||||
proj: np.ndarray | None,
|
||||
lightness: np.ndarray,
|
||||
trimap: np.ndarray,
|
||||
settings: CrossCheckSettings,
|
||||
@@ -76,6 +76,13 @@ def cross_check_alpha(
|
||||
- the primary alpha is high (``gate_lo -> gate_hi`` ramp): pixels the
|
||||
pipeline already renders soft (outer wisps) are exempt by construction.
|
||||
|
||||
``proj`` is None in complex-background mode (chroma disabled): no key colour
|
||||
means no hue-defined suspect zone, so instead the veto requires the second
|
||||
opinion itself to be confidently near-empty (full strength at/below
|
||||
``second_lo``, none at/above ``second_hi``). A thin strand the downsampled
|
||||
second model merely blurs to mid-alpha is left untouched; only regions it
|
||||
decisively rejects can be cleared.
|
||||
|
||||
Inside the zone this deliberately overrides the trimap-FG clamp -- the
|
||||
residue it exists to clear is mostly trimap-FG. Sure background cannot be
|
||||
disturbed: min-fusion keeps alpha 0 at 0.
|
||||
@@ -83,16 +90,16 @@ def cross_check_alpha(
|
||||
if not settings.enabled:
|
||||
return alpha
|
||||
cv2 = require_cv2()
|
||||
zone = (
|
||||
(proj >= settings.proj_min)
|
||||
& (lightness >= settings.l_min)
|
||||
& (trimap != 0)
|
||||
)
|
||||
zone = (lightness >= settings.l_min) & (trimap != 0)
|
||||
if proj is not None:
|
||||
zone &= proj >= settings.proj_min
|
||||
weight = zone.astype(np.float32)
|
||||
if settings.feather_sigma > 0:
|
||||
blur = cv2.GaussianBlur(weight, (0, 0), settings.feather_sigma)
|
||||
weight = np.where(zone, 1.0, np.clip(blur, 0.0, 1.0)).astype(np.float32)
|
||||
gate = weight * _smoothstep(alpha, settings.gate_lo, settings.gate_hi)
|
||||
if proj is None:
|
||||
gate = gate * (1.0 - _smoothstep(second_alpha, settings.second_lo, settings.second_hi))
|
||||
out = alpha * (1.0 - gate) + np.minimum(alpha, second_alpha) * gate
|
||||
return np.clip(out, 0.0, 1.0).astype(np.float32)
|
||||
|
||||
|
||||
+7
-1
@@ -6,6 +6,7 @@ import typer
|
||||
from rich import print
|
||||
|
||||
from .config import load_settings, override_settings
|
||||
from .memtune import release_freed_memory
|
||||
from .pipeline import MattingPipeline, run_image
|
||||
|
||||
app = typer.Typer(help="Offline flat-background character matting.")
|
||||
@@ -23,6 +24,7 @@ def main(
|
||||
device: str | None = typer.Option(None, "--device"),
|
||||
precision: str | None = typer.Option(None, "--precision", help="Compute precision for all models (ViTMatte cast + BiRefNet autocast): fp32 (default) | bf16 (faster + halves matting activation memory; needs bf16-capable hardware, else falls back to fp32)"),
|
||||
screen_color: str | None = typer.Option(None, "--screen-color", help="Background colour prior as #RRGGBB (default: auto-detect the flat background colour)"),
|
||||
chroma: bool | None = typer.Option(None, "--chroma/--no-chroma", help="Flat-colour background mode (default: on). --no-chroma is non-flat/complex-background mode: no colour key, segmentation drives the trimap; needs a segmentation backend + matting_method vitmatte"),
|
||||
matting_method: str | None = typer.Option(None, "--matting-method"),
|
||||
fallback_to_chroma_alpha: bool | None = typer.Option(None, "--fallback-to-chroma-alpha/--no-fallback-to-chroma-alpha"),
|
||||
sure_bg_threshold: float | None = typer.Option(None, "--sure-bg-threshold", min=0.0, max=1.0),
|
||||
@@ -31,7 +33,7 @@ def main(
|
||||
fg_safe_radius_ratio: float | None = typer.Option(None, "--fg-safe-radius-ratio", min=0.0),
|
||||
despill: bool | None = typer.Option(None, "--despill/--no-despill"),
|
||||
cross_check: bool | None = typer.Option(None, "--cross-check/--no-cross-check", help="Second-opinion veto of background-hued residue between hair strands (default: on; costs one extra model inference)"),
|
||||
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: on; saves ~20s on CPU; needs cross-check on)"),
|
||||
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: off; saves ~20s on CPU; needs cross-check on)"),
|
||||
trimap_mode: str | None = typer.Option(None, "--trimap-mode", help="Trimap mode (segmentation pipeline): directional | seg | directional-hard-bg"),
|
||||
seg_backend: str | None = typer.Option(None, "--seg-backend", help="Segmentation backend: birefnet (default) | anime-seg"),
|
||||
) -> None:
|
||||
@@ -42,6 +44,7 @@ def main(
|
||||
device=device,
|
||||
precision=precision,
|
||||
screen_color=screen_color,
|
||||
chroma=chroma,
|
||||
matting_method=matting_method,
|
||||
fallback_to_chroma_alpha=fallback_to_chroma_alpha,
|
||||
sure_bg_threshold=sure_bg_threshold,
|
||||
@@ -69,6 +72,9 @@ def main(
|
||||
dbg = None if debug_dir is None else debug_dir / image_path.stem
|
||||
result = pipeline.run_image(image_path, out_path, dbg)
|
||||
print(f"[green]wrote[/green] {result['output']}")
|
||||
# Free this image's activations back to the OS before the next one
|
||||
# so batch RSS tracks a single image, not the whole run.
|
||||
release_freed_memory()
|
||||
return
|
||||
|
||||
if input is None or output is None:
|
||||
|
||||
+8
-2
@@ -13,6 +13,7 @@ from .settings import (
|
||||
ModelSettings,
|
||||
PipelineSettings,
|
||||
SegmentationSettings,
|
||||
ServerSettings,
|
||||
TrimapSettings,
|
||||
)
|
||||
|
||||
@@ -49,7 +50,7 @@ def _update_dataclass(instance: T, values: dict[str, Any] | None) -> T:
|
||||
def settings_from_dict(data: dict[str, Any]) -> PipelineSettings:
|
||||
data = dict(data)
|
||||
screen_color = data.pop("screen_color", None)
|
||||
allowed_sections = {"chroma", "trimap", "alpha_post", "cross_check", "foreground", "despill", "model", "segmentation"}
|
||||
allowed_sections = {"chroma", "trimap", "alpha_post", "cross_check", "foreground", "despill", "model", "segmentation", "server"}
|
||||
unknown_sections = sorted(set(data) - allowed_sections)
|
||||
if unknown_sections:
|
||||
raise ValueError(f"Unknown config section(s): {', '.join(unknown_sections)}")
|
||||
@@ -63,6 +64,7 @@ def settings_from_dict(data: dict[str, Any]) -> PipelineSettings:
|
||||
despill=_update_dataclass(DespillSettings(), data.get("despill")),
|
||||
model=_update_dataclass(ModelSettings(), data.get("model")),
|
||||
segmentation=_update_dataclass(SegmentationSettings(), data.get("segmentation")),
|
||||
server=_update_dataclass(ServerSettings(), data.get("server")),
|
||||
screen_color=screen_color,
|
||||
)
|
||||
|
||||
@@ -99,6 +101,9 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
|
||||
}
|
||||
if overrides.get("trimap_mode") is not None:
|
||||
trimap_updates["mode"] = overrides["trimap_mode"]
|
||||
chroma_updates: dict[str, Any] = {}
|
||||
if overrides.get("chroma") is not None:
|
||||
chroma_updates["enabled"] = overrides["chroma"]
|
||||
despill_updates: dict[str, Any] = {}
|
||||
if overrides.get("despill") is not None:
|
||||
despill_updates["enabled"] = overrides["despill"]
|
||||
@@ -130,7 +135,7 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
|
||||
else settings.screen_color
|
||||
)
|
||||
return PipelineSettings(
|
||||
chroma=chroma,
|
||||
chroma=_update_dataclass(chroma, chroma_updates),
|
||||
trimap=_update_dataclass(trimap, trimap_updates),
|
||||
alpha_post=alpha_post,
|
||||
cross_check=_update_dataclass(settings.cross_check, cross_check_updates),
|
||||
@@ -138,5 +143,6 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
|
||||
despill=_update_dataclass(despill, despill_updates),
|
||||
model=_update_dataclass(model, model_updates),
|
||||
segmentation=_update_dataclass(segmentation, seg_updates),
|
||||
server=settings.server,
|
||||
screen_color=screen_color,
|
||||
)
|
||||
|
||||
+3
-2
@@ -10,7 +10,7 @@ from .settings import DespillSettings
|
||||
def despill(
|
||||
rgb: np.ndarray,
|
||||
alpha: np.ndarray,
|
||||
model: BackgroundModel,
|
||||
model: BackgroundModel | None,
|
||||
settings: DespillSettings,
|
||||
) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Remove background-colour spill from the foreground.
|
||||
@@ -24,7 +24,8 @@ def despill(
|
||||
|
||||
Returns the corrected RGB (uint8) and the per-pixel despill weight (float32).
|
||||
"""
|
||||
if not settings.enabled:
|
||||
# model is None with chroma disabled: no key colour means no spill direction.
|
||||
if not settings.enabled or model is None:
|
||||
return rgb.copy(), np.zeros(alpha.shape, dtype=np.float32)
|
||||
|
||||
bg_lab = np.asarray(model.lab_center, dtype=np.float32)
|
||||
|
||||
+47
-2
@@ -1,17 +1,21 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .chroma import BackgroundModel
|
||||
from .deps import require_cv2
|
||||
from .settings import ForegroundSettings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def estimate_foreground_rgb(
|
||||
rgb: np.ndarray,
|
||||
alpha: np.ndarray,
|
||||
bg_confidence: np.ndarray,
|
||||
model: BackgroundModel,
|
||||
model: BackgroundModel | None,
|
||||
settings: ForegroundSettings,
|
||||
) -> np.ndarray:
|
||||
"""Estimated foreground colour F (uint8 HxWx3) for compositing over alpha."""
|
||||
@@ -23,10 +27,15 @@ def estimate_foreground_rgb(
|
||||
try:
|
||||
return _estimate_ml(rgb, alpha, settings)
|
||||
except RuntimeError:
|
||||
if not settings.fallback_to_unmix:
|
||||
if not settings.fallback_to_unmix or model is None:
|
||||
raise
|
||||
method = "unmix"
|
||||
if method == "unmix":
|
||||
if model is None:
|
||||
raise RuntimeError(
|
||||
"Foreground method 'unmix' needs the chroma background model, which "
|
||||
"does not exist with chroma disabled; use method 'ml'."
|
||||
)
|
||||
return _estimate_unmix(rgb, alpha, bg_confidence, model, settings)
|
||||
raise RuntimeError(
|
||||
f"Unsupported foreground method '{settings.method}'. Use 'ml' or 'unmix'."
|
||||
@@ -43,6 +52,10 @@ def _estimate_ml(
|
||||
high-alpha neighbours instead of being divided out pixel-wise. That removes
|
||||
background-colour spill on thin hair where a direct unmix (divide by small
|
||||
alpha) is too noisy to trust.
|
||||
|
||||
Runs on the GPU via pymatting's CuPy backend when ``settings.use_gpu`` is set
|
||||
and cupy + a CUDA device are available (~250x faster; float32, visually
|
||||
identical to the float64 CPU result); otherwise falls back to the CPU solver.
|
||||
"""
|
||||
try:
|
||||
from pymatting import estimate_foreground_ml
|
||||
@@ -53,10 +66,42 @@ def _estimate_ml(
|
||||
|
||||
image = rgb.astype(np.float64) / 255.0
|
||||
a = np.clip(alpha.astype(np.float64), 0.0, 1.0)
|
||||
|
||||
if settings.use_gpu:
|
||||
foreground = _estimate_ml_gpu(image, a, settings.ml_regularization)
|
||||
if foreground is not None:
|
||||
return np.clip(foreground * 255.0, 0, 255).astype(np.uint8)
|
||||
|
||||
foreground = estimate_foreground_ml(image, a, regularization=settings.ml_regularization)
|
||||
return np.clip(foreground * 255.0, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
def _estimate_ml_gpu(
|
||||
image: np.ndarray, alpha: np.ndarray, regularization: float
|
||||
) -> np.ndarray | None:
|
||||
"""Multi-level foreground via pymatting's CuPy backend.
|
||||
|
||||
Returns the foreground as float ``[0, 1]``, or ``None`` when the GPU path is
|
||||
unavailable (cupy not installed, or no CUDA device) so the caller falls back
|
||||
to the CPU solver. pymatting only re-exports ``estimate_foreground_ml_cupy``
|
||||
when cupy is importable, so the import itself is the availability probe.
|
||||
"""
|
||||
try:
|
||||
from pymatting import estimate_foreground_ml_cupy
|
||||
except ImportError:
|
||||
return None
|
||||
try:
|
||||
import cupy
|
||||
|
||||
if cupy.cuda.runtime.getDeviceCount() < 1:
|
||||
return None
|
||||
foreground = estimate_foreground_ml_cupy(image, alpha, regularization=regularization)
|
||||
return np.asarray(foreground)
|
||||
except Exception as exc: # noqa: BLE001 -- any GPU failure falls back to CPU
|
||||
logger.warning("GPU foreground estimation failed (%s); using CPU", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _weighted_blur(values: np.ndarray, weights: np.ndarray, radius: int) -> np.ndarray:
|
||||
if radius <= 0:
|
||||
return values
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
from __future__ import annotations
|
||||
|
||||
"""Cross-platform release of freed allocator memory after a heavy inference.
|
||||
|
||||
Both of the process allocators this pipeline runs under retain large freed
|
||||
blocks instead of returning them to the OS, so a BiRefNet@2048 forward's
|
||||
activations keep inflating RSS long after the tensors are gone:
|
||||
|
||||
- Windows PyTorch bundles mimalloc; its lazy purge is defused at import time by
|
||||
``MIMALLOC_PURGE_DELAY=0`` (set in ``bgfilter/__init__.py`` before torch loads).
|
||||
- Linux/glibc PyTorch uses ptmalloc, which keeps freed arenas committed;
|
||||
``malloc_trim(0)`` hands those pages back and is called at runtime via
|
||||
:func:`release_freed_memory` after each image.
|
||||
|
||||
Neither hook exists on the other's platform, so this stays a no-op there (and on
|
||||
musl, which has no ``malloc_trim``, or under a preloaded jemalloc/tcmalloc that
|
||||
manages its own release). See DEPLOY for the recommended jemalloc LD_PRELOAD.
|
||||
"""
|
||||
|
||||
import ctypes
|
||||
import ctypes.util
|
||||
import sys
|
||||
|
||||
_malloc_trim = None
|
||||
if sys.platform.startswith("linux"):
|
||||
try:
|
||||
_libc = ctypes.CDLL(ctypes.util.find_library("c") or "libc.so.6", use_errno=True)
|
||||
_malloc_trim = _libc.malloc_trim # glibc only; AttributeError on musl
|
||||
_malloc_trim.argtypes = [ctypes.c_size_t]
|
||||
_malloc_trim.restype = ctypes.c_int
|
||||
except (OSError, AttributeError):
|
||||
_malloc_trim = None
|
||||
|
||||
|
||||
def release_freed_memory() -> bool:
|
||||
"""Return freed heap pages to the OS; True if a release was actually issued.
|
||||
|
||||
Cheap (sub-millisecond) and safe to call after every processed image. No-op
|
||||
where no supported hook is available (Windows, musl, non-glibc allocators).
|
||||
"""
|
||||
if _malloc_trim is None:
|
||||
return False
|
||||
_malloc_trim(0)
|
||||
return True
|
||||
+54
-20
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
@@ -60,6 +61,7 @@ class MattingPipeline:
|
||||
runner: ViTMatteRunner | None = None,
|
||||
segmenter=None,
|
||||
cross_checker=None,
|
||||
gpu_lock=None,
|
||||
):
|
||||
self.settings = settings
|
||||
# Pre-loaded models can be injected so a long-lived service shares them
|
||||
@@ -67,6 +69,10 @@ class MattingPipeline:
|
||||
self._runner: ViTMatteRunner | None = runner
|
||||
self._segmenter = segmenter
|
||||
self._cross_checker = cross_checker
|
||||
# Held around every GPU model forward so concurrent requests never drive the
|
||||
# device at once (models are shared, single GPU). None = no contention
|
||||
# (CLI / single-thread), so a no-op context keeps that path lock-free.
|
||||
self._gpu_lock = gpu_lock if gpu_lock is not None else contextlib.nullcontext()
|
||||
|
||||
def _segment(self, rgb: np.ndarray) -> np.ndarray:
|
||||
if self._segmenter is None:
|
||||
@@ -132,10 +138,26 @@ def run_image(
|
||||
|
||||
def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
settings = pipeline.settings
|
||||
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
|
||||
bg_confidence, model = compute_bg_confidence(
|
||||
rgb, settings=settings.chroma, screen_color=screen_color
|
||||
)
|
||||
if settings.chroma.enabled:
|
||||
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
|
||||
bg_confidence, model = compute_bg_confidence(
|
||||
rgb, settings=settings.chroma, screen_color=screen_color
|
||||
)
|
||||
else:
|
||||
if not settings.segmentation.enabled:
|
||||
raise RuntimeError(
|
||||
"chroma.enabled: false (non-flat background mode) needs the segmentation "
|
||||
"pipeline; enable segmentation or re-enable chroma."
|
||||
)
|
||||
if settings.model.matting_method == "chroma":
|
||||
raise RuntimeError(
|
||||
"matting_method 'chroma' needs the chroma key; use 'vitmatte' or re-enable chroma."
|
||||
)
|
||||
# Non-flat background: no colour key exists. A zero background-confidence
|
||||
# map makes every chroma-fused formula degrade to its seg-only form; the
|
||||
# colour-keyed stages (hue split, suppression, veto, despill) are skipped.
|
||||
bg_confidence = np.zeros(rgb.shape[:2], dtype=np.float32)
|
||||
model = None
|
||||
second_alpha = None
|
||||
if settings.segmentation.enabled:
|
||||
reuse = (
|
||||
@@ -150,11 +172,15 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
if reuse:
|
||||
# One HR-matting forward serves both the trimap topology and the
|
||||
# cross-check second opinion; the primary seg model is never loaded.
|
||||
second_alpha = pipeline._second_opinion(rgb)
|
||||
with pipeline._gpu_lock:
|
||||
second_alpha = pipeline._second_opinion(rgb)
|
||||
seg_mask = second_alpha
|
||||
else:
|
||||
seg_mask = pipeline._segment(rgb)
|
||||
mode = settings.trimap.mode
|
||||
with pipeline._gpu_lock:
|
||||
seg_mask = pipeline._segment(rgb)
|
||||
# The directional modes are colour tests against the key colour; without
|
||||
# one, the seg-topology trimap is the only meaningful choice.
|
||||
mode = settings.trimap.mode if settings.chroma.enabled else "seg"
|
||||
if mode == "seg":
|
||||
trimap, trimap_stats = fuse_trimap(seg_mask, bg_confidence, settings.trimap)
|
||||
elif mode in ("directional", "directional-hard-bg"):
|
||||
@@ -170,36 +196,44 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
else:
|
||||
seg_mask = None
|
||||
trimap, trimap_stats = generate_trimap(bg_confidence, settings.trimap)
|
||||
raw_alpha, alpha_source = pipeline._predict_alpha(rgb, trimap, bg_confidence)
|
||||
with pipeline._gpu_lock:
|
||||
raw_alpha, alpha_source = pipeline._predict_alpha(rgb, trimap, bg_confidence)
|
||||
|
||||
alpha = enforce_trimap(raw_alpha, trimap)
|
||||
alpha = suppress_alpha_by_chroma(
|
||||
alpha, bg_confidence, trimap, settings.alpha_post,
|
||||
# The matte-confidence gate only makes sense for a real matting prediction;
|
||||
# a chroma-seeded alpha is itself colour evidence, so no gate there.
|
||||
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
|
||||
)
|
||||
if settings.chroma.enabled:
|
||||
alpha = suppress_alpha_by_chroma(
|
||||
alpha, bg_confidence, trimap, settings.alpha_post,
|
||||
# The matte-confidence gate only makes sense for a real matting prediction;
|
||||
# a chroma-seeded alpha is itself colour evidence, so no gate there.
|
||||
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
|
||||
)
|
||||
alpha = clean_alpha(alpha, trimap, settings.alpha_post)
|
||||
if settings.cross_check.enabled:
|
||||
lab = convert_color_spaces(rgb)[1]
|
||||
if second_alpha is None:
|
||||
second_alpha = pipeline._second_opinion(rgb)
|
||||
with pipeline._gpu_lock:
|
||||
second_alpha = pipeline._second_opinion(rgb)
|
||||
alpha = cross_check_alpha(
|
||||
alpha,
|
||||
second_alpha,
|
||||
bg_hue_projection(lab, model.lab_center),
|
||||
# No key colour in complex-background mode: proj=None switches the
|
||||
# veto to its second-opinion-confidence gate (see cross_check_alpha).
|
||||
bg_hue_projection(lab, model.lab_center) if model is not None else None,
|
||||
lab[..., 0],
|
||||
trimap,
|
||||
settings.cross_check,
|
||||
)
|
||||
foreground = estimate_foreground_rgb(
|
||||
rgb, alpha, bg_confidence, model, settings.foreground
|
||||
)
|
||||
# Foreground colour estimation runs on the GPU (cupy) when enabled, so it shares
|
||||
# the mutex; the CPU fallback path is short and simply runs while holding it.
|
||||
with pipeline._gpu_lock:
|
||||
foreground = estimate_foreground_rgb(
|
||||
rgb, alpha, bg_confidence, model, settings.foreground
|
||||
)
|
||||
corrected_rgb, color_mask = despill(foreground, alpha, model, settings.despill)
|
||||
|
||||
metadata = {
|
||||
"alpha_source": alpha_source,
|
||||
"background_model": model.to_dict(),
|
||||
"background_model": model.to_dict() if model is not None else None,
|
||||
"trimap": trimap_stats,
|
||||
"settings": asdict(settings),
|
||||
}
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from .precision import ensure_deform_conv2d_autocast_cpu, resolve_dtype
|
||||
from .settings import SegmentationSettings
|
||||
from .weights import resolve_model_source
|
||||
|
||||
|
||||
class BiRefNetSegmenter:
|
||||
@@ -36,7 +39,7 @@ class BiRefNetSegmenter:
|
||||
|
||||
self.torch = torch
|
||||
self.model = AutoModelForImageSegmentation.from_pretrained(
|
||||
settings.model_name, trust_remote_code=True
|
||||
resolve_model_source(settings.model_name), trust_remote_code=True
|
||||
)
|
||||
self.model.eval()
|
||||
self.model.float() # checkpoint ships as fp16; force fp32 to match inputs
|
||||
@@ -104,7 +107,11 @@ class AnimeSegSegmenter:
|
||||
"pip install onnxruntime huggingface_hub"
|
||||
) from exc
|
||||
|
||||
model_file = hf_hub_download(settings.model_name, "isnetis.onnx")
|
||||
source = resolve_model_source(settings.model_name)
|
||||
if os.path.isdir(source):
|
||||
model_file = os.path.join(source, "isnetis.onnx")
|
||||
else:
|
||||
model_file = hf_hub_download(source, "isnetis.onnx")
|
||||
providers = ["CPUExecutionProvider"]
|
||||
if settings.device == "cuda" and "CUDAExecutionProvider" in ort.get_available_providers():
|
||||
providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
from dataclasses import replace
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .config import _SEG_MODELS, override_settings
|
||||
from .memtune import release_freed_memory
|
||||
from .pipeline import MattingPipeline, MattingResult
|
||||
from .segmentation import BiRefNetSegmenter, make_segmenter
|
||||
from .settings import PipelineSettings, SegmentationSettings
|
||||
from .vitmatte_infer import ViTMatteRunner
|
||||
|
||||
logger = logging.getLogger("bgfilter.service")
|
||||
|
||||
# HTTP-facing seg_model values, in preference order (first = default).
|
||||
SEG_MODELS: tuple[str, ...] = tuple(_SEG_MODELS)
|
||||
|
||||
|
||||
class PipelineManager:
|
||||
"""Long-lived holder of the matting models.
|
||||
|
||||
ViTMatte is the matting model shared by both segmentation backends, so it is
|
||||
loaded once and reused. Each segmentation backend is lazily loaded once on
|
||||
first use and cached by name. Per-request parameters (``screen_color``,
|
||||
``seg_model``, ``cross_check``, ``chroma``) never trigger a model reload --
|
||||
they only re-parametrise the pipeline over already-cached (or first-time-
|
||||
loaded, then cached) models. ``chroma=False`` selects non-flat/complex
|
||||
background mode (no colour key); ``screen_color`` is then unused.
|
||||
"""
|
||||
|
||||
def __init__(self, base_settings: PipelineSettings, preload: bool = True):
|
||||
self._base = base_settings
|
||||
self._runner: ViTMatteRunner | None = None
|
||||
self._segmenters: dict[str, object] = {}
|
||||
self._cross_checker: BiRefNetSegmenter | None = None
|
||||
self._load_lock = threading.Lock()
|
||||
# Caps how many concurrent requests may be inside a GPU forward at once
|
||||
# (the app may run several process() calls on a thread pool). Shared by every
|
||||
# pipeline this manager builds. =1 (default) makes the device strictly
|
||||
# single-stream; >1 lets that many forwards overlap so batch-1 kernels that
|
||||
# underfill the SMs can co-schedule (higher GPU-bound throughput, but the
|
||||
# concurrent activations multiply GPU memory). Config value
|
||||
# (``server.gpu_concurrency``) is the default; the env var overrides it.
|
||||
gpu_conc = max(1, int(os.environ.get("BGFILTER_GPU_CONCURRENCY", base_settings.server.gpu_concurrency)))
|
||||
self._gpu_lock = threading.Lock() if gpu_conc == 1 else threading.BoundedSemaphore(gpu_conc)
|
||||
if preload:
|
||||
if base_settings.model.matting_method == "vitmatte":
|
||||
self._get_runner()
|
||||
# Cross-check is default behaviour, so its HR-matting model is part of
|
||||
# the warm set. With reuse-as-seg on the birefnet backend that same
|
||||
# forward also serves as the seg mask, so the primary segmenter is
|
||||
# never used on the default path -- only preload it when reuse won't
|
||||
# cover segmentation (reuse off, or a non-birefnet default backend).
|
||||
if base_settings.cross_check.enabled:
|
||||
self._get_cross_checker()
|
||||
reuse_covers_seg = (
|
||||
base_settings.cross_check.enabled
|
||||
and base_settings.cross_check.reuse_as_seg
|
||||
and base_settings.segmentation.backend == "birefnet"
|
||||
)
|
||||
if base_settings.segmentation.enabled and not reuse_covers_seg:
|
||||
self._get_segmenter(base_settings.segmentation.backend)
|
||||
|
||||
@property
|
||||
def default_seg_model(self) -> str:
|
||||
return self._base.segmentation.backend
|
||||
|
||||
@property
|
||||
def device(self) -> str:
|
||||
return self._base.model.device
|
||||
|
||||
@property
|
||||
def cross_check_enabled(self) -> bool:
|
||||
return self._base.cross_check.enabled
|
||||
|
||||
def effective_cross_check(self, requested: bool | None) -> bool:
|
||||
"""Resolve the cross-check state for a request, honoring the lane lock.
|
||||
|
||||
When ``cross_check.lock`` is set (e.g. the cpu-fast lane) the base default
|
||||
is authoritative and a per-request override is ignored -- so a locked
|
||||
off-lane never loads the HR-matting model or risks an OOM, and never errors
|
||||
on a ``cross_check=on`` request. Otherwise the request value wins when
|
||||
given, else the base default applies.
|
||||
"""
|
||||
base = self._base.cross_check
|
||||
if base.lock:
|
||||
return base.enabled
|
||||
return requested if requested is not None else base.enabled
|
||||
|
||||
def _get_runner(self) -> ViTMatteRunner:
|
||||
if self._runner is None:
|
||||
with self._load_lock:
|
||||
if self._runner is None:
|
||||
logger.info(
|
||||
"loading ViTMatte: model=%s device=%s",
|
||||
self._base.model.model_name,
|
||||
self._base.model.device,
|
||||
)
|
||||
self._runner = ViTMatteRunner(self._base.model)
|
||||
return self._runner
|
||||
|
||||
def _get_segmenter(self, backend: str):
|
||||
seg = self._segmenters.get(backend)
|
||||
if seg is None:
|
||||
with self._load_lock:
|
||||
seg = self._segmenters.get(backend)
|
||||
if seg is None:
|
||||
seg_settings = replace(
|
||||
self._base.segmentation,
|
||||
backend=backend,
|
||||
model_name=_SEG_MODELS[backend],
|
||||
)
|
||||
logger.info(
|
||||
"loading segmenter: backend=%s model=%s device=%s",
|
||||
backend,
|
||||
seg_settings.model_name,
|
||||
seg_settings.device,
|
||||
)
|
||||
seg = make_segmenter(seg_settings)
|
||||
self._segmenters[backend] = seg
|
||||
return seg
|
||||
|
||||
def _get_cross_checker(self) -> BiRefNetSegmenter:
|
||||
# The cross-check model depends only on base settings (never on per-request
|
||||
# params), so one shared instance serves all requests.
|
||||
if self._cross_checker is None:
|
||||
with self._load_lock:
|
||||
if self._cross_checker is None:
|
||||
check = self._base.cross_check
|
||||
logger.info(
|
||||
"loading cross-check model: model=%s device=%s",
|
||||
check.model_name,
|
||||
self._base.segmentation.device,
|
||||
)
|
||||
self._cross_checker = BiRefNetSegmenter(
|
||||
SegmentationSettings(
|
||||
model_name=check.model_name,
|
||||
device=self._base.segmentation.device,
|
||||
input_size=check.input_size,
|
||||
precision=check.precision,
|
||||
)
|
||||
)
|
||||
return self._cross_checker
|
||||
|
||||
def process(
|
||||
self,
|
||||
rgb: np.ndarray,
|
||||
*,
|
||||
screen_color: str | None,
|
||||
seg_model: str,
|
||||
cross_check: bool | None = None,
|
||||
chroma: bool | None = None,
|
||||
) -> MattingResult:
|
||||
if seg_model not in _SEG_MODELS:
|
||||
raise ValueError(f"invalid seg_model '{seg_model}'")
|
||||
settings = override_settings(
|
||||
self._base,
|
||||
screen_color=screen_color,
|
||||
seg_backend=seg_model,
|
||||
# None leaves the base default; False selects non-flat/complex mode.
|
||||
chroma=chroma,
|
||||
# Honor the lane lock: a locked lane ignores the per-request override
|
||||
# (a cpu-fast box never runs cross-check, even on cross_check=on).
|
||||
cross_check=self.effective_cross_check(cross_check),
|
||||
)
|
||||
runner = (
|
||||
self._get_runner()
|
||||
if settings.model.matting_method == "vitmatte"
|
||||
else None
|
||||
)
|
||||
segmenter = (
|
||||
self._get_segmenter(seg_model) if settings.segmentation.enabled else None
|
||||
)
|
||||
cross_checker = (
|
||||
self._get_cross_checker() if settings.cross_check.enabled else None
|
||||
)
|
||||
pipeline = MattingPipeline(
|
||||
settings,
|
||||
runner=runner,
|
||||
segmenter=segmenter,
|
||||
cross_checker=cross_checker,
|
||||
gpu_lock=self._gpu_lock,
|
||||
)
|
||||
try:
|
||||
return pipeline.run_rgb(rgb)
|
||||
finally:
|
||||
# Return this request's large freed activations to the OS so RSS does
|
||||
# not ratchet up across requests in the long-lived service process.
|
||||
# Hold the GPU lock: empty_cache must not race a concurrent request's
|
||||
# in-flight allocations on the same device.
|
||||
with self._gpu_lock:
|
||||
release_freed_memory()
|
||||
+63
-7
@@ -5,6 +5,14 @@ from dataclasses import dataclass
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ChromaSettings:
|
||||
# False = non-flat-background mode (--no-chroma): no colour key at all.
|
||||
# Segmentation alone drives the trimap (mode forced to "seg"), and the
|
||||
# colour-keyed stages are bypassed: background auto-detect, the directional
|
||||
# hue split, chroma alpha suppression, despill, and the unmix foreground
|
||||
# fallback. The cross-check veto still runs (if enabled) with a hue-free
|
||||
# gate -- see CrossCheckSettings.second_lo/hi. Requires
|
||||
# segmentation.enabled and matting_method 'vitmatte'.
|
||||
enabled: bool = True
|
||||
border_ratio: float = 0.04
|
||||
min_samples: int = 2048
|
||||
lab_sigma_min: float = 10.0
|
||||
@@ -40,6 +48,14 @@ class TrimapSettings:
|
||||
mode: str = "directional"
|
||||
seg_low: float = 0.15 # seg below this -> background-eligible; at/above -> foreground-eligible
|
||||
bg_hue_proj_min: float = 4.0 # Lab a*/b* projection onto bg direction above which a pixel is background-hued
|
||||
# Semantic override for the directional hue split: a chroma-unknown pixel whose
|
||||
# seg confidence is at/above this is kept sure-FG even when background-hued.
|
||||
# A saturated seg signal outranks colour evidence that only says "same hue
|
||||
# family as the background" (bright skin on a warm background, measured raw
|
||||
# ViTMatte failures at proj ~7-10). Chroma sure-background is never overridden,
|
||||
# and the silhouette band still re-opens the boundary for anti-aliasing.
|
||||
# Set > 1.0 to disable. (seg masks are uint8-quantised: 0.98 = 250/255.)
|
||||
seg_force_fg: float = 0.98
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -72,16 +88,22 @@ class CrossCheckSettings:
|
||||
# that chroma, segmentation and the primary matte all read as foreground;
|
||||
# already-soft wisps and dark hair are exempt by construction.
|
||||
enabled: bool = True
|
||||
# Lock the cross-check state to ``enabled`` and ignore per-request overrides.
|
||||
# A memory-constrained lane (e.g. cpu-fast) sets ``enabled: false, lock: true``
|
||||
# so a request with ``cross_check=on`` is silently honoured as off instead of
|
||||
# loading the HR-matting model and risking an OOM. No error is raised.
|
||||
lock: bool = False
|
||||
model_name: str = "ZhengPeng7/BiRefNet_HR-matting"
|
||||
input_size: int = 2048
|
||||
# Reuse the cross-check forward as the segmentation mask: one HR-matting
|
||||
# inference serves both trimap topology and the second opinion, skipping the
|
||||
# primary seg model entirely (saves its load + forward). Pilot-validated on
|
||||
# TestImage3/FixImage1: trimap 99.8% identical, no structural change. Only
|
||||
# applies to the birefnet segmentation backend (a same-family swap); with
|
||||
# anime-seg the chosen segmenter keeps its own forward and the cross-check
|
||||
# runs independently. Needs cross_check and segmentation both enabled.
|
||||
reuse_as_seg: bool = True
|
||||
# primary seg model entirely (saves its load + forward; measured ~20 s/image
|
||||
# on CPU, ~0.9-1.3 GB VRAM on GPU). Pilot-validated on TestImage3/FixImage1:
|
||||
# trimap 99.8% identical. Default OFF on server-edition: the dedicated
|
||||
# segmenter keeps its own forward and the veto stays an independent signal;
|
||||
# opt in with --cross-check-as-seg. Only applies to the birefnet backend
|
||||
# (a same-family swap); needs cross_check and segmentation both enabled.
|
||||
reuse_as_seg: bool = False
|
||||
# Compute precision of the cross-check forward: "fp32" or "bf16" (autocast;
|
||||
# needs bf16-capable hardware, else falls back to fp32 with a warning).
|
||||
precision: str = "fp32"
|
||||
@@ -90,12 +112,24 @@ class CrossCheckSettings:
|
||||
feather_sigma: float = 2.0 # Gaussian feather of the zone boundary, in px
|
||||
gate_lo: float = 0.70 # primary alpha below this -> fully exempt
|
||||
gate_hi: float = 0.95 # primary alpha above this -> fully vetoable
|
||||
# Complex-background mode only (chroma disabled -> no hue-defined suspect
|
||||
# zone): the veto instead requires the second opinion itself to be
|
||||
# confidently near-empty -- full strength at/below second_lo, none at/above
|
||||
# second_hi. Thin structures the downsampled second model merely blurs to
|
||||
# mid-alpha stay untouched; only decisively-rejected regions can be cleared.
|
||||
second_lo: float = 0.15
|
||||
second_hi: float = 0.40
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ForegroundSettings:
|
||||
enabled: bool = True
|
||||
method: str = "ml" # "ml" (pymatting) or "unmix" (legacy heuristic)
|
||||
# With method "ml", use pymatting's CuPy GPU backend when it is available
|
||||
# (cupy installed + a CUDA device) — ~250x faster, float32 output visually
|
||||
# identical to the float64 CPU solver. Transparently falls back to CPU when
|
||||
# cupy/CUDA are absent, so it is safe to leave on for CPU-only deployments.
|
||||
use_gpu: bool = True
|
||||
fallback_to_unmix: bool = True
|
||||
ml_regularization: float = 1e-5
|
||||
# Legacy "unmix" method parameters (used only when method == "unmix").
|
||||
@@ -113,7 +147,12 @@ class ForegroundSettings:
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DespillSettings:
|
||||
enabled: bool = True
|
||||
# Off by default: despill pulls foreground chroma along the background-hue axis
|
||||
# with no positional or semantic guard, so a subject sharing the background's hue
|
||||
# (e.g. a blue suit on a blue backdrop) is desaturated/hue-shifted (~ΔE 35, blue
|
||||
# -> purple). This is the root default the cpu-fast lane inherits. Re-enable
|
||||
# per-config or with --despill only when edge spill genuinely matters.
|
||||
enabled: bool = False
|
||||
edge_low: float = 0.005
|
||||
strength: float = 0.92
|
||||
edge_expand_radius: int = 2
|
||||
@@ -157,6 +196,22 @@ class SegmentationSettings:
|
||||
precision: str = "fp32"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ServerSettings:
|
||||
"""HTTP-service request concurrency (ignored by the CLI).
|
||||
|
||||
``cpu_workers`` is the thread-pool size for request pipelining: >1 lets that
|
||||
many requests run concurrently, overlapping their CPU stages while the GPU
|
||||
forwards stay serialized. ``gpu_concurrency`` caps how many forwards may
|
||||
co-schedule on the device at once (>1 fills the SMs that batch-1 underuses,
|
||||
but multiplies GPU activation memory -- size it to the VRAM). Both env vars
|
||||
``BGFILTER_CPU_WORKERS`` / ``BGFILTER_GPU_CONCURRENCY`` override these.
|
||||
"""
|
||||
|
||||
cpu_workers: int = 1
|
||||
gpu_concurrency: int = 1
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PipelineSettings:
|
||||
chroma: ChromaSettings = ChromaSettings()
|
||||
@@ -167,6 +222,7 @@ class PipelineSettings:
|
||||
despill: DespillSettings = DespillSettings()
|
||||
model: ModelSettings = ModelSettings()
|
||||
segmentation: SegmentationSettings = SegmentationSettings()
|
||||
server: ServerSettings = ServerSettings()
|
||||
# Background-colour prior. None = auto-detect the flat background colour from the
|
||||
# image border (raises if there is no clean flat background). A hex string like
|
||||
# "#CFEFFF" sets it explicitly.
|
||||
|
||||
+16
-16
@@ -123,16 +123,14 @@ def fuse_trimap_directional(
|
||||
strongly displaced toward it (e.g. blue between hair strands) is left
|
||||
unknown for ViTMatte / chroma-suppress. Lower-seg unknowns stay unknown.
|
||||
|
||||
A thin unknown band at the segmentation silhouette is preserved so ViTMatte can
|
||||
anti-alias the boundary (it is deliberately small so interior holes survive).
|
||||
No silhouette band is forced back to unknown: structures thinner than the band
|
||||
ring (flying hair strands) got wholly confiscated by dilate-minus-erode and then
|
||||
dropped by ViTMatte, which on crisp-edged (anime) images outputs near-binary
|
||||
alpha in the band instead of anti-aliasing (measured: 96% hard values, and 74%
|
||||
of lost strand pixels were rule-FG demoted by the band). The rules' own unknown
|
||||
pixels (background-hued blends, low-seg transitions) remain ViTMatte's domain.
|
||||
"""
|
||||
cv2 = require_cv2()
|
||||
shape = seg_mask.shape
|
||||
fg_safe_radius = radius_from_ratio(
|
||||
shape, settings.fg_safe_radius_ratio, settings.min_fg_safe_radius
|
||||
)
|
||||
band_radius = max(settings.min_fg_safe_radius, fg_safe_radius // 2)
|
||||
|
||||
chroma_bg = bg_confidence >= settings.sure_bg_threshold
|
||||
chroma_fg = bg_confidence <= settings.sure_fg_threshold
|
||||
chroma_unknown = ~chroma_bg & ~chroma_fg
|
||||
@@ -145,26 +143,28 @@ def fuse_trimap_directional(
|
||||
bg = chroma_bg | ((seg_mask < settings.seg_low) & ~chroma_fg)
|
||||
fg = chroma_fg & (seg_mask >= settings.seg_low)
|
||||
rule3_fg = chroma_unknown & core & ~bg_hued
|
||||
# Rule 4 -- semantic override: a chroma-unknown pixel the segmenter is
|
||||
# (near-)certain about stays foreground even when background-hued. Colour
|
||||
# evidence here only says "same hue family as the background" (bright skin
|
||||
# on a warm background), which is exactly where ViTMatte and the suppressor
|
||||
# misfire; a saturated seg signal outranks it. chroma_unknown excludes sure
|
||||
# background, so a real flat-colour backdrop can never be forced foreground.
|
||||
force_fg = chroma_unknown & (seg_mask >= settings.seg_force_fg)
|
||||
if settings.mode == "directional-hard-bg":
|
||||
# Aggressive variant: hard-remove background-hued pixels instead of leaving
|
||||
# them unknown. Can eat cool/shadowed white cloth, so it is not the default.
|
||||
bg = bg | (chroma_unknown & core & bg_hued)
|
||||
bg = bg | (chroma_unknown & core & bg_hued & ~force_fg)
|
||||
|
||||
trimap = np.full(shape, 128, dtype=np.uint8)
|
||||
trimap[bg] = 0
|
||||
trimap[fg | rule3_fg] = 255
|
||||
|
||||
loose = (seg_mask >= settings.seg_loose_threshold).astype(np.uint8)
|
||||
kernel = elliptical_kernel(band_radius)
|
||||
band = cv2.dilate(loose, kernel).astype(bool) & ~cv2.erode(loose, kernel).astype(bool)
|
||||
trimap[band & ~chroma_bg] = 128
|
||||
trimap[fg | rule3_fg | force_fg] = 255
|
||||
|
||||
stats = {
|
||||
"sure_bg_pixels": int((trimap == 0).sum()),
|
||||
"unknown_pixels": int((trimap == 128).sum()),
|
||||
"sure_fg_pixels": int((trimap == 255).sum()),
|
||||
"band_radius": int(band_radius),
|
||||
"rule3_fg_pixels": int(rule3_fg.sum()),
|
||||
"force_fg_pixels": int((force_fg & ~fg & ~rule3_fg).sum()),
|
||||
"bg_hued_pixels": int((bg_hued & chroma_unknown & core).sum()),
|
||||
}
|
||||
return trimap, stats
|
||||
|
||||
@@ -5,6 +5,7 @@ from PIL import Image
|
||||
|
||||
from .precision import resolve_dtype
|
||||
from .settings import ModelSettings
|
||||
from .weights import resolve_model_source
|
||||
|
||||
|
||||
class ViTMatteRunner:
|
||||
@@ -19,10 +20,11 @@ class ViTMatteRunner:
|
||||
) from exc
|
||||
|
||||
self.torch = torch
|
||||
self.processor = VitMatteImageProcessor.from_pretrained(settings.model_name)
|
||||
self.model = VitMatteForImageMatting.from_pretrained(settings.model_name)
|
||||
self.device = self._resolve_device(settings.device)
|
||||
self.dtype = resolve_dtype(self.torch, self.device, settings.precision)
|
||||
source = resolve_model_source(settings.model_name)
|
||||
self.processor = VitMatteImageProcessor.from_pretrained(source)
|
||||
self.model = VitMatteForImageMatting.from_pretrained(source)
|
||||
self.model.to(device=self.device, dtype=self.dtype)
|
||||
self.model.eval()
|
||||
if settings.attn_query_chunk > 0:
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def weights_dir() -> Path:
|
||||
"""Directory scanned for locally-bundled model weights.
|
||||
|
||||
Defaults to ``models`` (relative to the working directory); override with the
|
||||
``BGFILTER_WEIGHTS_DIR`` environment variable.
|
||||
"""
|
||||
return Path(os.environ.get("BGFILTER_WEIGHTS_DIR", "models"))
|
||||
|
||||
|
||||
def resolve_model_source(repo_id: str) -> str:
|
||||
"""Map a HuggingFace repo id to a local weights folder when one is present.
|
||||
|
||||
If ``<weights_dir>/<basename>`` exists it is returned (so models can be shipped
|
||||
as plain folders inside the project, e.g. ``ZhengPeng7/BiRefNet`` ->
|
||||
``models/BiRefNet``); otherwise the repo id is returned unchanged so it loads
|
||||
from the HF hub/cache as before. An absolute/existing path is passed through.
|
||||
"""
|
||||
if os.path.isdir(repo_id):
|
||||
return repo_id
|
||||
local = weights_dir() / repo_id.split("/")[-1]
|
||||
if local.is_dir():
|
||||
return str(local)
|
||||
return repo_id
|
||||
@@ -0,0 +1,33 @@
|
||||
# Urgent CPU-only lane: fp32, birefnet segmentation, NO cross-check, NO anime-seg.
|
||||
#
|
||||
# This is the lightest production path: base ViTMatte + one BiRefNet@1024, ~14-15s
|
||||
# per 1.5 MP image on CPU. It needs ONLY these two weights provisioned --
|
||||
# BiRefNet_HR-matting (cross-check) and anime-seg are never loaded.
|
||||
#
|
||||
# Tuned for an 8 GB box (~7 GB for the service). Measured peak working set:
|
||||
# 2048x1152 (2.4 MP) ~5.6 GB | 2048x2048 (4.2 MP) ~7.1 GB.
|
||||
# The 2048x2048 peak is a floor set by ViTMatte's own dense compute, not the
|
||||
# attention (shrinking the chunk below 512 does not lower it further). Set
|
||||
# MemoryMax=8G with a hard 4 MP input cap (BGFILTER_MAX_IMAGE_PIXELS default),
|
||||
# or cap inputs tighter if you need more headroom.
|
||||
#
|
||||
# Anything omitted here inherits the built-in defaults (identical to default.yaml).
|
||||
|
||||
model:
|
||||
device: cpu
|
||||
precision: fp32 # fp32 only: this lane targets non-bf16 CPUs
|
||||
# Query chunk of 512 (vs the 2048 default) trims ViTMatte's attention transient
|
||||
# ~0.9 GB at 2048x2048 (8.0 -> 7.1 GB) for ~1 s more compute; 256 gives nothing
|
||||
# more. Exact/bitwise-identical output -- purely a memory-vs-speed knob.
|
||||
attn_query_chunk: 512
|
||||
|
||||
segmentation:
|
||||
enabled: true
|
||||
backend: birefnet # the only backend this lane uses
|
||||
device: cpu
|
||||
precision: fp32
|
||||
|
||||
cross_check:
|
||||
enabled: false # no second-opinion HR-matting pass (the big cost/mem sink)
|
||||
lock: true # ignore per-request cross_check=on so a request can't
|
||||
# force the HR-matting load and OOM this low-memory lane
|
||||
+27
-3
@@ -3,6 +3,15 @@
|
||||
# like "#CFEFFF" sets it explicitly. Also overridable via --screen-color.
|
||||
screen_color: null
|
||||
|
||||
chroma:
|
||||
# true = flat-colour background (default). false = non-flat/complex background
|
||||
# mode (--no-chroma): no colour key; segmentation alone drives the trimap (mode
|
||||
# forced to "seg") and the colour-keyed stages are bypassed (auto-detect, hue
|
||||
# split, chroma suppression, despill). The cross-check veto still runs if
|
||||
# enabled, using its hue-free gate (cross_check.second_lo/hi). Needs
|
||||
# segmentation.enabled + matting_method vitmatte.
|
||||
enabled: true
|
||||
|
||||
model:
|
||||
model_name: hustvl/vitmatte-base-composition-1k
|
||||
device: cpu
|
||||
@@ -46,6 +55,12 @@ trimap:
|
||||
mode: directional
|
||||
seg_low: 0.15
|
||||
bg_hue_proj_min: 4.0
|
||||
# Semantic override (directional mode): chroma-unknown pixels with seg confidence
|
||||
# at/above this stay sure-FG even when background-hued — rescues bright skin on a
|
||||
# warm (same-hue-family) background that the hue split would otherwise hand to
|
||||
# ViTMatte/suppression, which misfire exactly there. Never overrides chroma
|
||||
# sure-background. Set > 1.0 to disable.
|
||||
seg_force_fg: 0.98
|
||||
|
||||
alpha_post:
|
||||
chroma_suppress: true
|
||||
@@ -74,8 +89,9 @@ cross_check:
|
||||
# trimap 99.8% identical, no structural change (fingers/wisps/bowstring OK).
|
||||
# Only applies to the birefnet backend (same-family swap); anime-seg keeps
|
||||
# its own forward, with the cross-check running independently on top.
|
||||
# Disable with --no-cross-check-as-seg. No effect when cross_check is off.
|
||||
reuse_as_seg: true
|
||||
# Default OFF (server-edition): the dedicated segmenter keeps its own forward
|
||||
# and the veto stays independent; enable with --cross-check-as-seg.
|
||||
reuse_as_seg: false
|
||||
# Cross-check forward precision, same semantics as segmentation.precision
|
||||
# (bf16 via autocast: 2048 pass ~35s -> ~26s on a native-bf16 CPU).
|
||||
precision: fp32
|
||||
@@ -84,6 +100,11 @@ cross_check:
|
||||
feather_sigma: 2.0 # zone-boundary feather, px
|
||||
gate_lo: 0.70 # primary alpha below this -> fully exempt
|
||||
gate_hi: 0.95 # primary alpha above this -> fully vetoable
|
||||
# Complex-background (no-chroma) mode only: no hue-defined suspect zone, so the
|
||||
# veto requires the second opinion itself to be confidently near-empty -- full
|
||||
# strength at/below second_lo, fading to none at/above second_hi.
|
||||
second_lo: 0.15
|
||||
second_hi: 0.40
|
||||
|
||||
foreground:
|
||||
enabled: true
|
||||
@@ -103,7 +124,10 @@ foreground:
|
||||
bg_confidence_weight: 0.70
|
||||
|
||||
despill:
|
||||
enabled: true
|
||||
# Off: despill has no positional/semantic guard, so a subject sharing the
|
||||
# background's hue (blue suit on a blue backdrop) gets desaturated/hue-shifted
|
||||
# (~ΔE 35, blue -> purple). Re-enable per-job only when edge spill matters.
|
||||
enabled: false
|
||||
edge_low: 0.005
|
||||
strength: 0.92
|
||||
edge_expand_radius: 2
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
# GPU lane: CUDA + bf16 across all three models. Full pipeline (BiRefNet seg +
|
||||
# ViTMatte + HR-matting cross-check). Foreground estimation auto-uses the CuPy GPU
|
||||
# backend when cupy is installed (see foreground.use_gpu, default on). despill is
|
||||
# off (it false-positives on subjects sharing the background hue).
|
||||
screen_color: null
|
||||
|
||||
chroma:
|
||||
# true = flat-colour background (default). false = non-flat/complex background
|
||||
# mode: no colour key; segmentation alone drives the trimap (mode forced to
|
||||
# "seg") and the colour-keyed stages are bypassed (auto-detect, hue split,
|
||||
# chroma suppression, despill). The cross-check veto still runs if enabled,
|
||||
# using its hue-free gate (cross_check.second_lo/hi). The service selects this
|
||||
# per request via background_mode=flat|complex; this is only the base default.
|
||||
enabled: true
|
||||
|
||||
model:
|
||||
model_name: hustvl/vitmatte-base-composition-1k
|
||||
device: cuda
|
||||
matting_method: vitmatte
|
||||
fallback_to_chroma_alpha: false
|
||||
precision: bf16
|
||||
attn_query_chunk: 2048
|
||||
|
||||
segmentation:
|
||||
enabled: true
|
||||
backend: birefnet
|
||||
model_name: ZhengPeng7/BiRefNet
|
||||
device: cuda
|
||||
input_size: 1024
|
||||
precision: bf16
|
||||
|
||||
trimap:
|
||||
sure_bg_threshold: 0.92
|
||||
sure_fg_threshold: 0.12
|
||||
unknown_radius_ratio: 0.012
|
||||
fg_safe_radius_ratio: 0.006
|
||||
seg_core_threshold: 0.60
|
||||
seg_loose_threshold: 0.08
|
||||
mode: directional
|
||||
seg_low: 0.15
|
||||
bg_hue_proj_min: 4.0
|
||||
seg_force_fg: 0.98
|
||||
|
||||
alpha_post:
|
||||
chroma_suppress: true
|
||||
chroma_suppress_bg_low: 0.35
|
||||
chroma_suppress_bg_high: 0.80
|
||||
chroma_suppress_strength: 1.0
|
||||
suppress_raw_lo: 0.85
|
||||
suppress_raw_hi: 0.98
|
||||
|
||||
cross_check:
|
||||
enabled: true
|
||||
model_name: ZhengPeng7/BiRefNet_HR-matting
|
||||
input_size: 2048
|
||||
reuse_as_seg: false
|
||||
precision: bf16
|
||||
proj_min: 3.0
|
||||
l_min: 45.0
|
||||
feather_sigma: 2.0
|
||||
gate_lo: 0.70
|
||||
gate_hi: 0.95
|
||||
# Complex-background (non-flat) mode only: no hue-defined suspect zone, so the
|
||||
# veto requires the second opinion itself to be confidently near-empty -- full
|
||||
# strength at/below second_lo, fading to none at/above second_hi.
|
||||
second_lo: 0.15
|
||||
second_hi: 0.40
|
||||
|
||||
foreground:
|
||||
enabled: true
|
||||
method: ml
|
||||
ml_regularization: 0.00001
|
||||
|
||||
despill:
|
||||
enabled: false
|
||||
|
||||
server:
|
||||
# Request concurrency (HTTP service only). cpu_workers = thread-pool size: >1 runs
|
||||
# several requests at once, overlapping their CPU stages while the GPU forwards
|
||||
# stay serialized -- keeps the GPU fed. gpu_concurrency = how many forwards may
|
||||
# co-schedule on the SMs at once: 2 lifts GPU-bound throughput (batch-1 underfills
|
||||
# the SMs) but roughly DOUBLES GPU activation memory.
|
||||
# * On a >=24 GB card (e.g. 4080 SUPER 32 GB): gpu_concurrency: 2 peaks ~19 GB.
|
||||
# * On a <=16 GB card: keep gpu_concurrency: 1 or it will OOM at 2048 cross-check.
|
||||
# Env vars BGFILTER_CPU_WORKERS / BGFILTER_GPU_CONCURRENCY override these per box.
|
||||
cpu_workers: 8
|
||||
gpu_concurrency: 2
|
||||
@@ -0,0 +1,470 @@
|
||||
# BGfilter 服务版设计与交付需求
|
||||
|
||||
本文档给接手实现服务化的 Agent 使用。目标是在 `BGfilter-server` 仓库内,把现有 BGfilter 抠图算法包装成一个可长期运行的 Python HTTP 服务,接口形态参考当前线上 `birefnet-service`,但算法仍使用本仓库的 BGfilter 管线。
|
||||
|
||||
## 目标
|
||||
|
||||
新增一个服务入口,使调用方可以通过 HTTP 上传图片并得到透明背景 PNG。
|
||||
|
||||
服务首版只暴露两个业务参数:
|
||||
|
||||
- `screen_color`:可选。背景色先验,格式为 `#RRGGBB`。不传或空值时沿用 BGfilter 默认行为,从图片边框自动检测纯色背景。
|
||||
- `seg_model`:可选。分割模型选择,允许值为 `birefnet` 或 `anime-seg`。不传时使用仓库当前默认值 `birefnet`。
|
||||
|
||||
其它算法参数首版不要暴露为 HTTP 参数,直接使用当前默认配置:
|
||||
|
||||
- `trimap.mode = directional`
|
||||
- `despill.enabled = true`
|
||||
- `model.matting_method = vitmatte`
|
||||
- `model.model_name = hustvl/vitmatte-base-composition-1k`
|
||||
- `model.device = cpu`
|
||||
- `segmentation.device = cpu`
|
||||
- `segmentation.input_size = 1024`
|
||||
|
||||
## 非目标
|
||||
|
||||
- 不把 BGfilter 算法移植到 Rust。
|
||||
- 不在 HTTP 接口里暴露 `trimap_mode`、`despill`、阈值、设备、模型权重名等高级参数。
|
||||
- 不复用当前线上 `birefnet-service` 的简单 alpha 直出算法;`seg_model=birefnet` 在本服务中只表示 BGfilter 管线里的语义分割后端。
|
||||
- 不在每次请求里加载并释放模型。
|
||||
- 不要求首版支持批量图片。
|
||||
|
||||
## 现有算法边界
|
||||
|
||||
BGfilter 当前核心入口在:
|
||||
|
||||
- `bgfilter/pipeline.py`
|
||||
- `bgfilter/settings.py`
|
||||
- `bgfilter/config.py`
|
||||
- `bgfilter/segmentation.py`
|
||||
- `bgfilter/vitmatte_infer.py`
|
||||
|
||||
完整管线是:
|
||||
|
||||
```text
|
||||
输入 RGB
|
||||
-> 背景色自动检测或使用 screen_color
|
||||
-> chroma 背景置信度
|
||||
-> 可选语义分割 mask,默认开启
|
||||
-> trimap 生成,默认 directional
|
||||
-> ViTMatte 预测 alpha
|
||||
-> trimap 强约束
|
||||
-> chroma-guided alpha suppression
|
||||
-> alpha 小连通域/孔洞清理
|
||||
-> pymatting foreground estimation
|
||||
-> despill 去背景色溢色
|
||||
-> 输出 RGBA PNG
|
||||
```
|
||||
|
||||
当前默认分割后端已经是:
|
||||
|
||||
```text
|
||||
segmentation.backend = birefnet
|
||||
segmentation.model_name = ZhengPeng7/BiRefNet
|
||||
```
|
||||
|
||||
`anime-seg` 对应权重为:
|
||||
|
||||
```text
|
||||
segmentation.backend = anime-seg
|
||||
segmentation.model_name = skytnt/anime-seg
|
||||
```
|
||||
|
||||
`bgfilter/config.py` 里已有 `_SEG_MODELS` 和 `override_settings(..., seg_backend=...)`,服务层应复用这套映射,不要让 HTTP 调用方传 `model_name`。
|
||||
|
||||
## HTTP 接口契约
|
||||
|
||||
### `GET /healthz`
|
||||
|
||||
返回 JSON,用于进程存活和基础配置检查。
|
||||
|
||||
建议响应:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"service": "bgfilter",
|
||||
"version": "0.1.0",
|
||||
"defaultSegModel": "birefnet",
|
||||
"device": "cpu"
|
||||
}
|
||||
```
|
||||
|
||||
### `POST /remove-background`
|
||||
|
||||
请求类型:
|
||||
|
||||
```text
|
||||
multipart/form-data
|
||||
```
|
||||
|
||||
字段:
|
||||
|
||||
```text
|
||||
file: UploadFile, required
|
||||
screen_color: string, optional, "#RRGGBB"
|
||||
seg_model: string, optional, "birefnet" | "anime-seg"
|
||||
```
|
||||
|
||||
响应:
|
||||
|
||||
```text
|
||||
200 image/png
|
||||
```
|
||||
|
||||
响应体是透明背景 PNG。建议附带诊断 header:
|
||||
|
||||
```text
|
||||
X-BGFilter-Elapsed-Ms: 总耗时毫秒
|
||||
X-BGFilter-Seg-Model: 实际使用的 seg_model
|
||||
X-BGFilter-Screen-Color: 实际请求传入的 screen_color,未传可省略或设为 auto
|
||||
```
|
||||
|
||||
错误码建议:
|
||||
|
||||
```text
|
||||
400 empty image
|
||||
400 invalid image
|
||||
400 invalid screen_color
|
||||
400 invalid seg_model
|
||||
413 image too large
|
||||
500 inference failed
|
||||
```
|
||||
|
||||
为兼容现有 Rust 调用方,`file` 字段名必须保持为 `file`。
|
||||
|
||||
## 参数默认与校验
|
||||
|
||||
`screen_color`:
|
||||
|
||||
- 允许省略、空字符串、`null` 等价于自动检测。
|
||||
- 非空时必须匹配 `#RRGGBB`。
|
||||
- 大小写都可以,内部可以统一为大写或小写。
|
||||
- 校验失败返回 `400 invalid screen_color`。
|
||||
|
||||
`seg_model`:
|
||||
|
||||
- 允许省略,默认 `birefnet`。
|
||||
- 允许值只包括 `birefnet`、`anime-seg`。
|
||||
- 可以兼容内部命名 `seg_backend`,但 HTTP 对外字段用 `seg_model`。
|
||||
- 校验失败返回 `400 invalid seg_model`。
|
||||
|
||||
`MAX_IMAGE_PIXELS`:
|
||||
|
||||
- 建议参考线上 `birefnet-service`,通过环境变量控制,例如 `BGFILTER_MAX_IMAGE_PIXELS`。
|
||||
- 默认建议先设为 `4194304`,即约 4MP。
|
||||
- 超限返回 `413 image too large`。
|
||||
|
||||
## 模型加载与缓存要求
|
||||
|
||||
服务必须做到模型只加载一次或按后端首次懒加载一次,不能每次请求都重新加载模型。
|
||||
|
||||
现有 `MattingPipeline` 已经有实例级懒加载:
|
||||
|
||||
- `self._runner` 缓存 `ViTMatteRunner`
|
||||
- `self._segmenter` 缓存当前 settings 下的 segmenter
|
||||
|
||||
但如果每次请求新建 `MattingPipeline(settings)`,缓存会失效。因此服务版需要持有长期对象。
|
||||
|
||||
推荐实现一个 `PipelineManager` 或等价对象:
|
||||
|
||||
```text
|
||||
长期缓存:
|
||||
ViTMatteRunner: 按 model_name + device + matting_method 缓存
|
||||
Segmenter: 按 backend + model_name + device + input_size 缓存
|
||||
|
||||
请求级参数:
|
||||
screen_color: 只影响本次 chroma/despill,不影响模型缓存
|
||||
seg_model: 选择对应 segmenter;首次使用某后端时加载,之后复用
|
||||
```
|
||||
|
||||
推荐行为:
|
||||
|
||||
```text
|
||||
服务启动:
|
||||
可预加载默认 birefnet + ViTMatte,也可懒加载。
|
||||
|
||||
第一次 seg_model=birefnet:
|
||||
加载 BiRefNet segmenter 和 ViTMatte。
|
||||
|
||||
后续 seg_model=birefnet:
|
||||
复用已有 BiRefNet segmenter 和 ViTMatte。
|
||||
|
||||
第一次 seg_model=anime-seg:
|
||||
加载 AnimeSeg segmenter,ViTMatte 继续复用。
|
||||
|
||||
后续 seg_model=anime-seg:
|
||||
复用已有 AnimeSeg segmenter 和 ViTMatte。
|
||||
```
|
||||
|
||||
可接受的首版简化方案:
|
||||
|
||||
- 长期缓存两个 `MattingPipeline` 实例:一个 `birefnet`,一个 `anime-seg`。
|
||||
- 每个实例内部复用自己的 `_runner` 和 `_segmenter`。
|
||||
- 如果这样做导致 ViTMatte 被加载两次,应优先改为共享 `ViTMatteRunner`,因为 ViTMatte 是两条分割后端共同使用的 matting 模型。
|
||||
|
||||
更推荐的实现是把 `MattingPipeline` 改造成可以注入共享 runner/segmenter,或新增服务专用 pipeline runner,避免重复加载 ViTMatte。
|
||||
|
||||
## 并发控制
|
||||
|
||||
首版建议采用全局 `asyncio.Lock`,与当前线上 `birefnet-service` 类似,保证同一进程内一次只跑一个推理请求。
|
||||
|
||||
原因:
|
||||
|
||||
- BGfilter 管线包含 ViTMatte、segmenter、pymatting、OpenCV/NumPy 后处理,CPU 和内存压力较大。
|
||||
- CPU 默认部署时并发推理容易放大延迟和内存。
|
||||
- 后续确认资源充足后,再考虑按模型或设备拆锁。
|
||||
|
||||
如果使用 Uvicorn 多 worker,每个 worker 都会各自加载一份模型,内存会成倍增长。首版建议 `--workers 1`。
|
||||
|
||||
## 内存入口改造建议
|
||||
|
||||
当前 `MattingPipeline.run_image(...)` 主要面向 CLI 文件输入输出。服务可以先用临时文件桥接,但更推荐新增内存入口,减少磁盘 IO 和临时文件清理风险。
|
||||
|
||||
推荐新增内部函数:
|
||||
|
||||
```text
|
||||
run_rgb(rgb: np.ndarray, settings: PipelineSettings, resources/cache) -> result
|
||||
```
|
||||
|
||||
返回:
|
||||
|
||||
```text
|
||||
corrected_rgb: np.ndarray uint8 HxWx3
|
||||
alpha: np.ndarray float32 HxW, 0..1
|
||||
metadata: dict
|
||||
```
|
||||
|
||||
服务层再把它编码成 PNG:
|
||||
|
||||
```text
|
||||
RGBA = corrected_rgb + alpha
|
||||
PIL.Image.save(..., format="PNG")
|
||||
```
|
||||
|
||||
如果首版为了速度使用临时文件:
|
||||
|
||||
- 临时目录应使用 `tempfile.TemporaryDirectory()`。
|
||||
- 文件名不要使用上传原始文件名。
|
||||
- 请求结束必须清理。
|
||||
- 不要把 debug 输出默认打开。
|
||||
|
||||
## 推荐文件结构
|
||||
|
||||
可以参考线上 `birefnet-service` 的简洁形态,在仓库根目录新增:
|
||||
|
||||
```text
|
||||
app.py
|
||||
requirements.txt
|
||||
```
|
||||
|
||||
也可以新增:
|
||||
|
||||
```text
|
||||
bgfilter/service.py
|
||||
```
|
||||
|
||||
用于放 `PipelineManager` 和内存运行逻辑。`app.py` 保持薄入口。
|
||||
|
||||
推荐分层:
|
||||
|
||||
```text
|
||||
app.py
|
||||
FastAPI 路由、参数校验、图片 decode/encode、错误码、计时、锁
|
||||
|
||||
bgfilter/service.py
|
||||
PipelineManager、模型缓存、按请求组装 settings、调用算法管线
|
||||
|
||||
bgfilter/pipeline.py
|
||||
尽量保留原 CLI 兼容行为;必要时新增 run_rgb
|
||||
```
|
||||
|
||||
## 依赖
|
||||
|
||||
现有算法依赖保留。服务化需要补充:
|
||||
|
||||
```text
|
||||
fastapi
|
||||
uvicorn[standard]
|
||||
python-multipart
|
||||
```
|
||||
|
||||
现有 `requirements.txt` 中已经有:
|
||||
|
||||
```text
|
||||
opencv-python
|
||||
pillow
|
||||
numpy
|
||||
scipy
|
||||
scikit-image
|
||||
pymatting
|
||||
torch
|
||||
torchvision
|
||||
transformers
|
||||
accelerate
|
||||
safetensors
|
||||
huggingface_hub
|
||||
timm
|
||||
einops
|
||||
kornia
|
||||
onnxruntime
|
||||
tqdm
|
||||
typer
|
||||
rich
|
||||
PyYAML
|
||||
```
|
||||
|
||||
如果部署在无 GUI Linux 服务器,`opencv-python-headless` 通常比 `opencv-python` 更合适,但替换前需要确认现有本地脚本是否依赖 GUI 能力。
|
||||
|
||||
## 环境变量
|
||||
|
||||
建议支持:
|
||||
|
||||
```text
|
||||
BGFILTER_CONFIG=configs/default.yaml
|
||||
BGFILTER_DEVICE=cpu
|
||||
BGFILTER_MAX_IMAGE_PIXELS=4194304
|
||||
BGFILTER_PRELOAD=1
|
||||
HF_HOME=/opt/genarrative-image-host/model-cache/hf
|
||||
HF_HUB_OFFLINE=1
|
||||
TRANSFORMERS_OFFLINE=1
|
||||
OMP_NUM_THREADS=4
|
||||
MKL_NUM_THREADS=4
|
||||
```
|
||||
|
||||
`BGFILTER_DEVICE` 应同时覆盖:
|
||||
|
||||
- `settings.model.device`
|
||||
- `settings.segmentation.device`
|
||||
|
||||
如果机器没有 GPU,默认保持 `cpu`。
|
||||
|
||||
## systemd 部署参考
|
||||
|
||||
参考当前线上 `genarrative-birefnet.service`,可以新增:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Genarrative BGfilter background removal service
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
|
||||
Environment=HF_HOME=/opt/genarrative-image-host/model-cache/hf
|
||||
Environment=HF_HUB_OFFLINE=1
|
||||
Environment=TRANSFORMERS_OFFLINE=1
|
||||
Environment=BGFILTER_CONFIG=configs/default.yaml
|
||||
Environment=BGFILTER_DEVICE=cpu
|
||||
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
|
||||
Environment=BGFILTER_PRELOAD=1
|
||||
Environment=OMP_NUM_THREADS=4
|
||||
Environment=MKL_NUM_THREADS=4
|
||||
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
MemoryMax=8G
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/opt/genarrative-image-host
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
端口只是建议值。真正接入时由部署方决定。
|
||||
|
||||
## 调用示例
|
||||
|
||||
默认参数:
|
||||
|
||||
```bash
|
||||
curl -sS \
|
||||
-F "file=@input.png" \
|
||||
http://127.0.0.1:18083/remove-background \
|
||||
-o output.png
|
||||
```
|
||||
|
||||
指定背景色:
|
||||
|
||||
```bash
|
||||
curl -sS \
|
||||
-F "file=@input.png" \
|
||||
-F "screen_color=#CFEFFF" \
|
||||
http://127.0.0.1:18083/remove-background \
|
||||
-o output.png
|
||||
```
|
||||
|
||||
切换 Anime Seg:
|
||||
|
||||
```bash
|
||||
curl -sS \
|
||||
-F "file=@input.png" \
|
||||
-F "seg_model=anime-seg" \
|
||||
http://127.0.0.1:18083/remove-background \
|
||||
-o output.png
|
||||
```
|
||||
|
||||
指定背景色并切换 Anime Seg:
|
||||
|
||||
```bash
|
||||
curl -sS \
|
||||
-F "file=@input.png" \
|
||||
-F "screen_color=#CFEFFF" \
|
||||
-F "seg_model=anime-seg" \
|
||||
http://127.0.0.1:18083/remove-background \
|
||||
-o output.png
|
||||
```
|
||||
|
||||
## 与 Genarrative Rust 服务的接入预期
|
||||
|
||||
当前 Genarrative Rust BFF 已有远端抠图调用边界:
|
||||
|
||||
```text
|
||||
POST {GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL}/remove-background
|
||||
multipart field: file
|
||||
response: image/png
|
||||
```
|
||||
|
||||
BGfilter 服务应保持同样的 `POST /remove-background` 和 `file` 字段名。Rust 侧后续只需要在 multipart 中追加可选文本字段:
|
||||
|
||||
```text
|
||||
screen_color
|
||||
seg_model
|
||||
```
|
||||
|
||||
如果 Rust 侧暂时不传新字段,本服务仍应可用,并按默认 `birefnet + 自动背景色` 执行。
|
||||
|
||||
## 验收要求
|
||||
|
||||
实现完成后至少验证:
|
||||
|
||||
1. `GET /healthz` 返回 `200` 和 JSON。
|
||||
2. `POST /remove-background` 只传 `file` 可返回 `image/png`。
|
||||
3. `screen_color=#CFEFFF` 可正常返回 `image/png`。
|
||||
4. `seg_model=birefnet` 可正常返回 `image/png`。
|
||||
5. `seg_model=anime-seg` 可正常返回 `image/png`。
|
||||
6. 非法 `seg_model` 返回 `400`。
|
||||
7. 非法 `screen_color` 返回 `400`。
|
||||
8. 空文件或非图片返回 `400`。
|
||||
9. 超过 `BGFILTER_MAX_IMAGE_PIXELS` 的图片返回 `413`。
|
||||
10. 同一进程连续请求同一 `seg_model` 时不会重复加载对应模型。
|
||||
11. 从 `birefnet` 切到 `anime-seg` 后,再切回 `birefnet`,已加载模型可复用。
|
||||
12. Uvicorn `--workers 1` 下内存不随请求次数持续增长。
|
||||
|
||||
建议在日志中打印模型加载事件,但不要每个请求打印完整配置或文件内容。日志中不要输出用户上传图片数据。
|
||||
|
||||
## 完成定义
|
||||
|
||||
交付时应包含:
|
||||
|
||||
- 可运行的 FastAPI 服务入口。
|
||||
- 更新后的依赖文件。
|
||||
- 本地启动命令说明。
|
||||
- 至少一组 curl 示例。
|
||||
- 若新增内存入口,应保持原 CLI 文件入口不破坏。
|
||||
- 若为了服务化调整 pipeline,应保证默认 CLI 行为仍按当前配置工作。
|
||||
|
||||
完成后,不看内部代码也应能通过本文档的 HTTP 契约和 curl 示例直接调用服务。
|
||||
+5
-1
@@ -1,4 +1,4 @@
|
||||
opencv-python
|
||||
opencv-python-headless
|
||||
pillow
|
||||
numpy
|
||||
scipy
|
||||
@@ -18,3 +18,7 @@ tqdm
|
||||
typer
|
||||
rich
|
||||
PyYAML
|
||||
fastapi
|
||||
uvicorn[standard]
|
||||
python-multipart
|
||||
httpx
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user