Merge origin/master: bf16 precision, chunked attention, seg-reuse, mimalloc purge
Adopts master as the standard for all overlapping work. Master's landed optimizations supersede server-edition's own fp16 experiment: - Unified --precision fp32|bf16 knob (bgfilter/precision.py) driving all three models: ViTMatte weight cast + both BiRefNets via autocast, with a hardware gate (falls back to fp32 off native-bf16 CPUs) and an AutocastCPU fp32 shim for torchvision deform_conv2d. - Query-chunked ViTMatte global attention (bgfilter/attn_chunk.py), exact and bitwise-identical, caps the N^2 spike (~19 -> ~4 GB at 2048). - Cross-check HR-matting forward reused as the seg mask (birefnet backend only), skipping the primary seg model; MIMALLOC_PURGE_DELAY=0. - inference_mode and the detect_background_color removal converge with server-edition's earlier equivalents. Conflict resolution (favoring master, preserving server-only features): - vitmatte_infer/segmentation: dropped server's device-derived fp16 for master's precision path, kept resolve_model_source (local weights). - service.py: cross-check SegmentationSettings now passes precision so the HTTP service honors bf16 like the CLI's _second_opinion does. Verified on CPU: default fp32 pipeline loads 2 models (seg-reuse active) and bf16 path runs (autocast + deform_conv2d shim) — both exit 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -38,6 +38,16 @@ download (~0.9 GB) and one inference pass per image; disable with
|
||||
`--no-cross-check` (see the `cross_check` config section, and
|
||||
`docs/hair_gap_artifacts.md` for the analysis behind it).
|
||||
|
||||
When the cross-check is on, that same HR-matting forward is **reused as the
|
||||
segmentation mask by default**, skipping the primary seg model entirely (one
|
||||
less model to load, ~20 s faster per image on CPU). Pilot-validated
|
||||
(TestImage3 / FixImage1): trimap 99.8% identical, no structural change to
|
||||
fingers, hair wisps or thin lines. Disable with `--no-cross-check-as-seg` to
|
||||
run the dedicated seg model instead. The reuse is a same-family swap, so it
|
||||
applies to the `birefnet` backend only: with `--seg-backend anime-seg` the
|
||||
anime segmenter keeps its own forward and the cross-check veto still runs
|
||||
independently on top.
|
||||
|
||||
The segmentation trimap defaults to `directional` mode (chroma + seg + a hue-direction
|
||||
split: it keeps a background-coloured garment such as a white shirt while dropping a
|
||||
background-hued residual such as blue trapped between hair strands). Switch with
|
||||
@@ -89,9 +99,14 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
No `--config` is needed — the built-in defaults are identical to
|
||||
`configs/default.yaml`. Pass `--config configs\default.yaml` only after you edit that
|
||||
file to tune the detailed parameters. Runtime choices stay on the command line:
|
||||
`--device` (default **CPU**; use `--device cuda` for GPU), `--seg-backend` (default
|
||||
`birefnet`; `anime-seg` for anime characters), `--screen-color` (default: auto-detect
|
||||
the flat background), and `--trimap-mode`.
|
||||
`--device` (default **CPU**; use `--device cuda` for GPU), `--precision` (default
|
||||
`fp32`; `bf16` speeds up all three models and halves the matting model's activation
|
||||
memory with visually identical alpha — needs bf16-capable hardware, falls back to
|
||||
fp32 elsewhere; large inputs additionally get ViTMatte's global attention computed
|
||||
in query chunks by default — exact, bitwise-identical, caps the memory spike at
|
||||
~4 GB instead of ~19 GB at 2048x2048, see `model.attn_query_chunk`), `--seg-backend`
|
||||
(default `birefnet`; `anime-seg` for anime characters), `--screen-color` (default:
|
||||
auto-detect the flat background), and `--trimap-mode`.
|
||||
|
||||
## Batch
|
||||
|
||||
@@ -102,6 +117,26 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
--debug-dir Outputs\debug
|
||||
```
|
||||
|
||||
## Performance (CPU reference numbers)
|
||||
|
||||
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, default settings
|
||||
(cross-check on, reused as seg; chunked global attention on):
|
||||
|
||||
| input | precision | warm / image | peak memory |
|
||||
|---|---|---|---|
|
||||
| 1024x1536 | fp32 | ~45 s | — |
|
||||
| 1024x1536 | bf16 | ~33 s | — |
|
||||
| 2048x2048 | bf16 | ~53 s | 11.4 GB (batch), 8.1 GB (single image) |
|
||||
|
||||
The memory ceiling is the cross-check BiRefNet_HR forward (fixed
|
||||
`input_size` 2048 regardless of the image size) — genuine live activations
|
||||
of a full-resolution dense prediction net. Before chunked attention and the
|
||||
mimalloc fix, the same 2048x2048 bf16 run peaked at 25.2 GB; a fully
|
||||
un-optimized fp32 run would need an estimated 55-60 GB (ViTMatte's un-chunked
|
||||
N^2 attention alone ~39 GB). CPUs without native bf16 (e.g. Zen 2) auto-fall
|
||||
back to fp32 — on such machines prefer `--no-cross-check` if memory or
|
||||
latency is tight.
|
||||
|
||||
## Chroma-alpha debug mode
|
||||
|
||||
`--matting-method chroma` skips ViTMatte and uses chroma confidence directly as the
|
||||
|
||||
Reference in New Issue
Block a user