Both sides extended DEPLOY(_ZH): remote documented the cpu-fast lane (weights subset, config comparison, systemd note, OOM troubleshooting), local added the measured GPU profile and the reuse-off default. Conflict was the capacity-section memory bullet: kept the GPU deployment bullet and remote's cpu-fast-aware memory bullet (which supersedes the old three-models line).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two default flips for server-edition:
1. cross_check.reuse_as_seg now defaults to false (settings + default.yaml + CLI help): the dedicated segmenter keeps its own forward and the cross-check veto stays an independent second signal. The reuse remains available via --cross-check-as-seg / config; measured cost of off vs on: ~20 s/image on CPU, +0.1-0.25 s and +0.9-1.3 GB VRAM on GPU (13.3 vs 12.0 GB reserved under expandable segments -- still fits a 16 GB card).
2. bgfilter/__init__.py defaults PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True on non-Windows platforms, before torch loads (setdefault: an explicit env value wins; native Windows is excluded because torch warns and ignores it there). Measured: reserved 15.3 -> 12.0 GB and ~10% faster on the RTX 5070 Ti; no effect on CPU-only runs.
README/DEPLOY(_ZH) synced: reuse documented as opt-in, GPU profile numbers updated for both reuse states, perf table labeled with the config it was measured under.
Verified: defaults resolve off/set as intended on Windows and WSL; explicit PYTORCH_CUDA_ALLOC_CONF override wins.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measured on an RTX 5070 Ti 16 GB, full default pipeline (cross-check on, reused as seg, bf16): ~1-2 s/image, ~10.1 GB allocated. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (Linux/WSL2 only; silently unsupported on native Windows) cuts the allocator's reserved footprint 15.3 -> 12.0 GB and is ~10% faster, leaving ~3.5 GB headroom on a 16 GB card even beside a desktop session. bf16 is the precondition: fp32 reserved ~17 GB overflows. reuse-as-seg also saves ~0.9 GB VRAM / ~0.25 s per image on GPU (reuse off measured 13.3 GB reserved under expandable_segments).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux
server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed
activations in the arena, so RSS ratchets up across requests and the env var is
a no-op there.
A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at
runtime (no-op on Windows/musl/other allocators). Wired into service.process
(after each request) and the CLI batch loop (after each image), the two
long-lived paths where RSS accumulates. Single-image CLI exits, so it is left
alone.
B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also
improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/
MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc
malloc_trim simply no-ops.
Also fixes the stale off-by-default cross-check heading in DEPLOY.md.
Verified: modules import; release_freed_memory() returns False (no-op) on Windows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cross-check + reuse-as-seg are already the config/CLI default; align the HTTP
service with them instead of forcing cross-check off. The earlier off-by-default
was to avoid the extra inference, but reuse-as-seg now makes that forward double
as the seg mask (birefnet backend) -- it replaces the primary seg model rather
than adding to it, so the cost concern is gone.
- app.py: honour the config default (on); BGFILTER_CROSS_CHECK still forces
either way, unset = config.
- service.py preload: warm the cross-check HR-matting model, and skip the now-
redundant primary segmenter when reuse covers segmentation.
- README / DEPLOY(_ZH): document cross-check as default-on, BiRefNet_HR-matting
as required (not optional), BGFILTER_CROSS_CHECK=0 to disable.
Verified: service defaults cross_check on, env forces both ways, preload picks
cross-checker and skips the primary segmenter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The GUI opencv-python build needs libGL.so.1 which headless Linux servers lack,
causing "ImportError: libGL.so.1" at first cv2 use. Pin the headless build so a
plain `pip install -r requirements.txt` works on servers out of the box (and
survives a git reset). BgFilter only uses cv2 for computation, so headless is a
drop-in on desktops too. Drop the now-unnecessary sed step from DEPLOY.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- fetch_weights.py: no-argument script that downloads every model weight into
models/ via the hf-mirror (proxy forced-bypassed, Xet disabled). Model ids are
read from config/code, and folders that already hold a weight file are skipped
(no network, no re-download). After it runs the app has full functionality
offline. Verified end to end (skips the 3 present models, pulls HR-matting).
- DEPLOY.md / DEPLOY_ZH.md: correct BiRefNet_HR-matting size from ~0.9 GB to the
verified ~425 MB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Per-model required-file directory tree for the local models/ layout
(ViTMatte / BiRefNet / anime-seg), noting which files each loader needs.
- Add ZhengPeng7/BiRefNet_HR-matting as the optional 4th model, used only by
the cross-check veto; new "cross-check" subsection covering enablement, the
offline-provisioning requirement, and the ~2x latency.
- Add BGFILTER_CROSS_CHECK to the env-var table and a troubleshooting row for
the offline "500 inference failed" case.
- Mirror all of the above into DEPLOY_ZH.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document deploying the FastAPI service to a Linux server end to end: venv +
CPU-only torch (avoid the CUDA build), opencv-python-headless, the two weight
layouts (HF cache offline vs local models/ folders) with the mirror/proxy/Xet
download workaround, environment variables, uvicorn --workers 1, a systemd unit,
verification, remote exposure, capacity notes, and a troubleshooting table.
Link to it from README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lhk229