diff --git a/DEPLOY.md b/DEPLOY.md index 108047a..4f5495f 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -161,17 +161,19 @@ Override the base directory with `BGFILTER_WEIGHTS_DIR`. `models/` is gitignored The pipeline runs a second, high-resolution matting model (`ZhengPeng7/BiRefNet_HR-matting`, ~425 MB) as a "second opinion" that clears background residue trapped between hair strands. It is **ON by default** (both CLI -and HTTP service) and, with reuse-as-seg, that same forward doubles as the -segmentation mask on the birefnet backend — so it replaces the primary seg model -rather than adding to it. **You must provision this model** (Option A or B above): +and HTTP service). **You must provision this model** (Option A or B above): - ⚠️ **On an offline server (`HF_HUB_OFFLINE=1`) it must be present before the first request** — otherwise every default request fails, since it cannot download the model at runtime. - Turn it off service-wide with `BGFILTER_CROSS_CHECK=0`, or override per request with the `cross_check` form field (`true`/`false`). -- The service preloads it at startup with the rest of the warm set; on the - birefnet backend the redundant primary segmenter is skipped. +- The service preloads it at startup with the rest of the warm set. +- Optional: `cross_check.reuse_as_seg: true` in the config (default **off**) makes + that same forward double as the segmentation mask on the birefnet backend — + replacing the primary seg model instead of adding to it (~20 s/image faster on + CPU, ~1 GB less VRAM; the redundant primary segmenter is then skipped at + startup). Off by default so the veto stays an independent signal. ### Downloading behind a firewall (CN networks) @@ -289,19 +291,18 @@ service has **no authentication**. To reach it from other hosts: - For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) — roughly an order of magnitude faster — and/or run multiple instances behind a load balancer (each with `--workers 1`). -- **GPU deployment (measured, RTX 5070 Ti 16 GB, full default pipeline — - cross-check on, reused as seg, bf16):** ~1–2 s/image (2 s at 4 MP), peak VRAM - ~10.1 GB allocated. The number that must fit on the card is the allocator's - *reserved* footprint; set this in the service environment on Linux: - ```ini - Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True - ``` - Measured effect: reserved 15.3 GB → **12.0 GB** (fragmentation 5.2 → 1.9 GB) and - ~10 % *faster* — with it, the full pipeline fits a 16 GB card with ~3.5 GB - headroom even next to a desktop session. Without it (or on native Windows, where - the option is silently unsupported), reserved is ~15.3 GB: a headless 16 GB card - still fits, but a desktop-attached one is borderline. Precision `bf16` is what - makes 16 GB viable at all — fp32 reserved ~17 GB overflows. +- **GPU deployment (measured, RTX 5070 Ti 16 GB, full pipeline — cross-check on, + bf16):** ~1–2 s/image (2 s at 4 MP), peak VRAM ~10.9 GB allocated (10.1 GB with + `reuse_as_seg: true`). The number that must fit on the card is the allocator's + *reserved* footprint. `bgfilter` defaults + `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` on Linux/WSL2 (set before + torch loads; native Windows does not support it, and an explicit env value + always wins). Measured effect: reserved 15.3 GB → **12.0 GB** with reuse + (13.3 GB without, the default) and ~10 % *faster* — the full pipeline fits a + 16 GB card with ~2–3.5 GB headroom even next to a desktop session. Without + expandable segments (e.g. native Windows) reserved is ~15.3–16.2 GB: + borderline-to-overflow on a 16 GB card. Precision `bf16` is what makes 16 GB + viable at all — fp32 reserved ~17 GB overflows. - Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G` is a safe ceiling for a single CPU worker. - **Keeping RSS flat (glibc).** After each image the service calls glibc diff --git a/DEPLOY_ZH.md b/DEPLOY_ZH.md index 077fe0a..0a4c7ea 100644 --- a/DEPLOY_ZH.md +++ b/DEPLOY_ZH.md @@ -150,14 +150,17 @@ hf download ZhengPeng7/BiRefNet_HR-matting --local-dir models/BiRefNet_HR- ### cross-check 否决(默认开启) 管线会再跑一个高分辨率 matting 模型(`ZhengPeng7/BiRefNet_HR-matting`,约 425 MB) -作为"第二意见",清理发丝间残留的背景色。**CLI 和 HTTP 服务都默认开启**,而且开了 -reuse-as-seg 之后,在 birefnet 后端这同一次前向还兼作分割掩码——它是**替换**了 primary -分割模型,而不是额外叠加。因此**必须备好这个模型**(上面方案 A 或 B): +作为"第二意见",清理发丝间残留的背景色。**CLI 和 HTTP 服务都默认开启**,因此**必须 +备好这个模型**(上面方案 A 或 B): - ⚠️ **离线服务器(`HF_HUB_OFFLINE=1`)必须在第一个请求前就备好它**,否则每个默认请求都会 失败——运行时它无法联网下载模型。 - 服务级关闭:`BGFILTER_CROSS_CHECK=0`;或按请求覆盖:表单字段 `cross_check`(`true`/`false`)。 -- 服务启动时会随暖机集一起预载它;在 birefnet 后端,冗余的 primary 分割模型会被跳过。 +- 服务启动时会随暖机集一起预载它。 +- 可选:配置里 `cross_check.reuse_as_seg: true`(默认**关**)可让这次前向在 birefnet + 后端兼作分割掩码——**替换**而非叠加 primary 分割模型(CPU 快约 20 秒/张、GPU 省约 + 1 GB 显存;此时冗余的 primary 分割模型在启动时会被跳过)。默认关闭是为了让否决 + 保持为独立信号。 ### 墙内下载(国内网络) @@ -270,16 +273,15 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \ 串行,所以单实例约 1–2 张/分钟。瓶颈是全分辨率的 ViTMatte 和 pymatting 前景估计。 - 要提吞吐:上 GPU(`BGFILTER_DEVICE=cuda`,装 CUDA 版 torch),大约快一个数量级; 和/或多实例 + 负载均衡(每个都 `--workers 1`)。 -- **GPU 部署(实测,RTX 5070 Ti 16 GB,完整默认管线——cross-check 开 + reuse-as-seg, - bf16):** 约 1–2 秒/张(4 MP 也只要 2 秒),峰值 VRAM 分配约 10.1 GB。真正决定 - 「装不装得下」的是分配器的 *reserved* 占用;Linux 上在服务环境里加: - ```ini - Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True - ``` - 实测效果:reserved 15.3 GB → **12.0 GB**(碎片 5.2 → 1.9 GB),而且还**快约 10%**—— - 加上它之后,完整管线在 16 GB 卡上即使旁边挂着桌面会话也有约 3.5 GB 余量。不加 - (或在原生 Windows 上——该选项会被静默忽略)reserved 约 15.3 GB:无头 16 GB 卡勉强 - 能装,带桌面输出的卡就很悬。另外 `bf16` 是 16 GB 可行的前提——fp32 的 reserved +- **GPU 部署(实测,RTX 5070 Ti 16 GB,完整管线——cross-check 开,bf16):** + 约 1–2 秒/张(4 MP 也只要 2 秒),峰值 VRAM 分配约 10.9 GB(开 `reuse_as_seg` + 为 10.1 GB)。真正决定「装不装得下」的是分配器的 *reserved* 占用。`bgfilter` + 在 Linux/WSL2 上会**默认设置** `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` + (在 torch 加载前;原生 Windows 不支持该选项;显式设过的环境变量优先)。 + 实测效果:reserved 15.3 GB → **12.0 GB**(开 reuse;默认不开为 13.3 GB),且还 + **快约 10%**——完整管线在 16 GB 卡上即使旁边挂着桌面会话也有约 2–3.5 GB 余量。 + 没有 expandable segments(如原生 Windows)时 reserved 约 15.3–16.2 GB:在 + 16 GB 卡上从很悬到直接溢出。另外 `bf16` 是 16 GB 可行的前提——fp32 的 reserved 约 17 GB,直接溢出。 - 内存:单个 worker 持有全部三个模型(约 1 GB 权重 + 运行时)。单 CPU worker 用 `MemoryMax=8G` 作上限比较稳妥。 diff --git a/README.md b/README.md index 94efd49..a57e5d9 100644 --- a/README.md +++ b/README.md @@ -38,15 +38,15 @@ download (~0.9 GB) and one inference pass per image; disable with `--no-cross-check` (see the `cross_check` config section, and `docs/hair_gap_artifacts.md` for the analysis behind it). -When the cross-check is on, that same HR-matting forward is **reused as the -segmentation mask by default**, skipping the primary seg model entirely (one -less model to load, ~20 s faster per image on CPU). Pilot-validated -(TestImage3 / FixImage1): trimap 99.8% identical, no structural change to -fingers, hair wisps or thin lines. Disable with `--no-cross-check-as-seg` to -run the dedicated seg model instead. The reuse is a same-family swap, so it +When the cross-check is on, that same HR-matting forward can optionally be +**reused as the segmentation mask** (`--cross-check-as-seg`), skipping the +primary seg model entirely (one less model to load, ~20 s faster per image on +CPU, ~1 GB less VRAM on GPU). Pilot-validated (TestImage3 / FixImage1): trimap +99.8% identical, no structural change to fingers, hair wisps or thin lines. +**Off by default**: the dedicated segmenter keeps its own forward and the veto +stays an independent second signal. The reuse is a same-family swap, so it applies to the `birefnet` backend only: with `--seg-backend anime-seg` the -anime segmenter keeps its own forward and the cross-check veto still runs -independently on top. +anime segmenter always keeps its own forward. The segmentation trimap defaults to `directional` mode (chroma + seg + a hue-direction split: it keeps a background-coloured garment such as a white shirt while dropping a @@ -119,8 +119,10 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli ` ## Performance (CPU reference numbers) -Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, default settings -(cross-check on, reused as seg; chunked global attention on): +Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, with cross-check on, +**`--cross-check-as-seg` enabled** and chunked global attention on. (Reuse is +off by default on this branch; without it, add the primary BiRefNet@1024 +forward — roughly +20 s/image at these sizes.) | input | precision | warm / image | peak memory | |---|---|---|---| @@ -218,12 +220,12 @@ caps input size (default ~4MP → `413`); `BGFILTER_PRELOAD=1` loads the default startup. Run a single worker (`--workers 1`) — each worker loads its own copy of the models. Like the CLI, the service runs the **cross-model veto by default** (config -default: on). With reuse-as-seg it replaces the primary seg forward rather than -adding to it, so it is not the latency hit it once was; its HR-matting model is -preloaded with the rest of the warm set (and, on the birefnet backend, the now- -redundant primary segmenter is skipped at startup). Set `BGFILTER_CROSS_CHECK=0` -to turn it off service-wide, or use the per-request `cross_check` form field to -override either way; `/healthz` reports the default as `crossCheck`. +default: on); its HR-matting model is preloaded with the rest of the warm set. +Set `BGFILTER_CROSS_CHECK=0` to turn it off service-wide, or use the per-request +`cross_check` form field to override either way; `/healthz` reports the default +as `crossCheck`. With `cross_check.reuse_as_seg: true` in the config (off by +default) the HR-matting forward replaces the primary seg forward instead of +adding to it, and the then-redundant primary segmenter is skipped at startup. ## Quality Check diff --git a/bgfilter/__init__.py b/bgfilter/__init__.py index 16a274e..324148f 100644 --- a/bgfilter/__init__.py +++ b/bgfilter/__init__.py @@ -1,6 +1,7 @@ """Flat-background character matting pipeline.""" import os +import sys # Windows PyTorch allocates CPU tensors through a bundled mimalloc whose lazy # purge keeps ~10 GB of already-freed forward activations committed after a @@ -11,4 +12,13 @@ import os # bgfilter.memtune.release_freed_memory(); see DEPLOY for jemalloc as an alt. os.environ.setdefault("MIMALLOC_PURGE_DELAY", "0") +# Expandable segments defuse the CUDA caching allocator's fragmentation: the +# full bf16 pipeline's reserved VRAM drops 15.3 -> 12.0 GB on a 16 GB card and +# runs ~10% faster (measured, RTX 5070 Ti). Read at CUDA-allocator init, so it +# must be set before torch loads; a no-op for CPU-only runs. Native Windows +# does not support it (torch warns and ignores), so it is only defaulted where +# it works (Linux/WSL2); setdefault keeps any explicit user setting in charge. +if not sys.platform.startswith("win"): + os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True") + __version__ = "0.1.0" diff --git a/bgfilter/cli.py b/bgfilter/cli.py index d326f41..8b27865 100644 --- a/bgfilter/cli.py +++ b/bgfilter/cli.py @@ -32,7 +32,7 @@ def main( fg_safe_radius_ratio: float | None = typer.Option(None, "--fg-safe-radius-ratio", min=0.0), despill: bool | None = typer.Option(None, "--despill/--no-despill"), cross_check: bool | None = typer.Option(None, "--cross-check/--no-cross-check", help="Second-opinion veto of background-hued residue between hair strands (default: on; costs one extra model inference)"), - cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: on; saves ~20s on CPU; needs cross-check on)"), + cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: off; saves ~20s on CPU; needs cross-check on)"), trimap_mode: str | None = typer.Option(None, "--trimap-mode", help="Trimap mode (segmentation pipeline): directional | seg | directional-hard-bg"), seg_backend: str | None = typer.Option(None, "--seg-backend", help="Segmentation backend: birefnet (default) | anime-seg"), ) -> None: diff --git a/bgfilter/settings.py b/bgfilter/settings.py index ea72a88..ac818b9 100644 --- a/bgfilter/settings.py +++ b/bgfilter/settings.py @@ -76,12 +76,13 @@ class CrossCheckSettings: input_size: int = 2048 # Reuse the cross-check forward as the segmentation mask: one HR-matting # inference serves both trimap topology and the second opinion, skipping the - # primary seg model entirely (saves its load + forward). Pilot-validated on - # TestImage3/FixImage1: trimap 99.8% identical, no structural change. Only - # applies to the birefnet segmentation backend (a same-family swap); with - # anime-seg the chosen segmenter keeps its own forward and the cross-check - # runs independently. Needs cross_check and segmentation both enabled. - reuse_as_seg: bool = True + # primary seg model entirely (saves its load + forward; measured ~20 s/image + # on CPU, ~0.9-1.3 GB VRAM on GPU). Pilot-validated on TestImage3/FixImage1: + # trimap 99.8% identical. Default OFF on server-edition: the dedicated + # segmenter keeps its own forward and the veto stays an independent signal; + # opt in with --cross-check-as-seg. Only applies to the birefnet backend + # (a same-family swap); needs cross_check and segmentation both enabled. + reuse_as_seg: bool = False # Compute precision of the cross-check forward: "fp32" or "bf16" (autocast; # needs bf16-capable hardware, else falls back to fp32 with a warning). precision: str = "fp32" diff --git a/configs/default.yaml b/configs/default.yaml index 69a71e0..0c11370 100644 --- a/configs/default.yaml +++ b/configs/default.yaml @@ -74,8 +74,9 @@ cross_check: # trimap 99.8% identical, no structural change (fingers/wisps/bowstring OK). # Only applies to the birefnet backend (same-family swap); anime-seg keeps # its own forward, with the cross-check running independently on top. - # Disable with --no-cross-check-as-seg. No effect when cross_check is off. - reuse_as_seg: true + # Default OFF (server-edition): the dedicated segmenter keeps its own forward + # and the veto stays independent; enable with --cross-check-as-seg. + reuse_as_seg: false # Cross-check forward precision, same semantics as segmentation.precision # (bf16 via autocast: 2048 pass ~35s -> ~26s on a native-bf16 CPU). precision: fp32