Merge origin/master: bf16 precision, chunked attention, seg-reuse, mimalloc purge

Adopts master as the standard for all overlapping work. Master's landed
optimizations supersede server-edition's own fp16 experiment:

- Unified --precision fp32|bf16 knob (bgfilter/precision.py) driving all
  three models: ViTMatte weight cast + both BiRefNets via autocast, with a
  hardware gate (falls back to fp32 off native-bf16 CPUs) and an
  AutocastCPU fp32 shim for torchvision deform_conv2d.
- Query-chunked ViTMatte global attention (bgfilter/attn_chunk.py), exact
  and bitwise-identical, caps the N^2 spike (~19 -> ~4 GB at 2048).
- Cross-check HR-matting forward reused as the seg mask (birefnet backend
  only), skipping the primary seg model; MIMALLOC_PURGE_DELAY=0.
- inference_mode and the detect_background_color removal converge with
  server-edition's earlier equivalents.

Conflict resolution (favoring master, preserving server-only features):
- vitmatte_infer/segmentation: dropped server's device-derived fp16 for
  master's precision path, kept resolve_model_source (local weights).
- service.py: cross-check SegmentationSettings now passes precision so the
  HTTP service honors bf16 like the CLI's _second_opinion does.

Verified on CPU: default fp32 pipeline loads 2 models (seg-reuse active)
and bf16 path runs (autocast + deform_conv2d shim) — both exit 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-06 16:02:44 +08:00
14 changed files with 316 additions and 24 deletions
+38 -3
View File
@@ -38,6 +38,16 @@ download (~0.9 GB) and one inference pass per image; disable with
`--no-cross-check` (see the `cross_check` config section, and
`docs/hair_gap_artifacts.md` for the analysis behind it).
When the cross-check is on, that same HR-matting forward is **reused as the
segmentation mask by default**, skipping the primary seg model entirely (one
less model to load, ~20 s faster per image on CPU). Pilot-validated
(TestImage3 / FixImage1): trimap 99.8% identical, no structural change to
fingers, hair wisps or thin lines. Disable with `--no-cross-check-as-seg` to
run the dedicated seg model instead. The reuse is a same-family swap, so it
applies to the `birefnet` backend only: with `--seg-backend anime-seg` the
anime segmenter keeps its own forward and the cross-check veto still runs
independently on top.
The segmentation trimap defaults to `directional` mode (chroma + seg + a hue-direction
split: it keeps a background-coloured garment such as a white shirt while dropping a
background-hued residual such as blue trapped between hair strands). Switch with
@@ -89,9 +99,14 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
No `--config` is needed — the built-in defaults are identical to
`configs/default.yaml`. Pass `--config configs\default.yaml` only after you edit that
file to tune the detailed parameters. Runtime choices stay on the command line:
`--device` (default **CPU**; use `--device cuda` for GPU), `--seg-backend` (default
`birefnet`; `anime-seg` for anime characters), `--screen-color` (default: auto-detect
the flat background), and `--trimap-mode`.
`--device` (default **CPU**; use `--device cuda` for GPU), `--precision` (default
`fp32`; `bf16` speeds up all three models and halves the matting model's activation
memory with visually identical alpha — needs bf16-capable hardware, falls back to
fp32 elsewhere; large inputs additionally get ViTMatte's global attention computed
in query chunks by default — exact, bitwise-identical, caps the memory spike at
~4 GB instead of ~19 GB at 2048x2048, see `model.attn_query_chunk`), `--seg-backend`
(default `birefnet`; `anime-seg` for anime characters), `--screen-color` (default:
auto-detect the flat background), and `--trimap-mode`.
## Batch
@@ -102,6 +117,26 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
--debug-dir Outputs\debug
```
## Performance (CPU reference numbers)
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, default settings
(cross-check on, reused as seg; chunked global attention on):
| input | precision | warm / image | peak memory |
|---|---|---|---|
| 1024x1536 | fp32 | ~45 s | — |
| 1024x1536 | bf16 | ~33 s | — |
| 2048x2048 | bf16 | ~53 s | 11.4 GB (batch), 8.1 GB (single image) |
The memory ceiling is the cross-check BiRefNet_HR forward (fixed
`input_size` 2048 regardless of the image size) — genuine live activations
of a full-resolution dense prediction net. Before chunked attention and the
mimalloc fix, the same 2048x2048 bf16 run peaked at 25.2 GB; a fully
un-optimized fp32 run would need an estimated 55-60 GB (ViTMatte's un-chunked
N^2 attention alone ~39 GB). CPUs without native bf16 (e.g. Zen 2) auto-fall
back to fp32 — on such machines prefer `--no-cross-check` if memory or
latency is tight.
## Chroma-alpha debug mode
`--matting-method chroma` skips ViTMatte and uses chroma confidence directly as the