The band (dilate-minus-erode ring on the seg silhouette, added in 5f48f7a)
was meant to give ViTMatte room to anti-alias the boundary. Measured on a
warm-background sample (fix_image02, #FFD6C2): ViTMatte outputs near-binary
alpha in the band on crisp-edged anime images (96% hard values, 82% of
silhouette crossings with zero soft pixels), so the band delivered no
anti-aliasing -- it only let ViTMatte relocate the boundary (chewed edges)
and wholly confiscated structures thinner than the ring: 74% of lost hair
strand pixels were rule-FG demoted by the overwrite, then dropped by
ViTMatte. Without the band, 88% of those strands survive, edges follow the
rule-decided contour, and cross-check no longer bites strand edges because
strand cores stay anchored at 255 (re-destroyed: 0 px).
The rules own unknown pixels (background-hued blends, low-seg transitions)
remain ViTMatte territory. fuse_trimap (seg mode) keeps its band untouched --
its rules were designed around it and the removal is only validated for the
directional path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
httpx logs every outbound fetch at INFO with the full presigned image_url,
leaking per-request records (and OSS credential/signature params) into the
event stream that is meant for low-volume prose only. The access JSONL
already records the URL, so cap the httpx logger at WARNING.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rule: bulk per-request data never lands in the prose log; everything with
analysis value goes to a structured file.
- Event stream (stderr, text): startup summary, warnings, tracebacks --
what supervisord/journald already captures. Human-readable, low volume.
- Access stream: one JSONL record per request (plus a "startup" meta
record with the instance config) written to BGFILTER_ACCESS_LOG,
app-rotated 50MB x 5; falls back to stderr when unset (dev).
Failed requests carry a truncated `exc` copy so the access file is
self-contained; the full traceback stays in the event stream, joined
by request_id.
- BGFILTER_LOG_FORMAT retired: format is now a property of the stream,
not a global switch.
- uvicorn runs with --no-access-log everywhere (deploy script + unit
examples): its prose per-request lines duplicated a subset of ours.
- deploy_autodl.sh --log-dir now provisions both files; docs updated
(EN+ZH: env table, AutoDL section, run/systemd examples).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
url_host alone made cross-referencing a failed fetch with the caller's
request awkward. Log the full URL too (truncated to 512 chars). Presigned
URLs do land their query-string token in the log -- acceptable for trusted
callers writing to a private log; drop the query if that assumption changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each fact now has exactly one home, with pointers instead of copies:
README (EN+ZH):
- Drop stale workstation env (conda lightML, D:\MiniConda paths); CLI
examples now use plain `python -m bgfilter.cli` with bash syntax.
- Performance section reduced to orders of magnitude + pointer; the
measured workstation table moved to DEPLOY §11.
- Notes: HF-mirror workaround and models/-folder layout (both duplicated
DEPLOY §5 verbatim) replaced with one-line pointers.
- HTTP service: env-var prose replaced with a pointer to DEPLOY §6.
- ZH parity fixes: add the missing image_url curl example; 400/413 rows
now mention image_url/fetch limits like the EN table.
DEPLOY (EN+ZH):
- §0: supervisord management expanded to a full command set (status/
restart/stop/start/shutdown, both log paths, manual supervisord rescue
after a container restart, how supervisord.conf changes are applied).
- §5: cross-check veto and background-mode subsections trimmed to
deploy-side facts + pointers to README Pipelines/HTTP service.
- §11: absorbs the workstation CPU reference table from README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a section 0 to DEPLOY.md/DEPLOY_ZH.md covering the AutoDL one-click
path (supervisord, not systemd), which was previously only self-documented
in scripts/deploy_autodl.sh.
deploy_autodl.sh:
- --log-dir (default ~/autodl-tmp, the persistent data disk) so the service
log no longer lands in /tmp, which AutoDL wipes on container reset.
- Re-running without --token now REUSES an existing token instead of
deleting it, so tweaking an unrelated knob can't silently disable auth;
add --open to explicitly run without auth.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each request now emits one access line (bgfilter.access) carrying a
request_id, the effective params (source file/url, seg_model, cross_check,
background_mode, screen_color, auth), input/output sizes, and stage
timings (t_queue/fetch/decode/process/total_ms). A middleware binds the id
(reusing a sanitized inbound X-Request-ID, else uuid), a logging filter
injects it into every record, and it is echoed back as the X-Request-ID
response header on every response -- including errors, which also log the
detail. Requests that raise still produce a line (fields accumulate on
request.state; the middleware finalizes status/total/client).
request_id lives in a contextvar (not a thread-local) so a pooled worker
never inherits the previous request's id; propagating it into the executor
for deep pipeline logs is left for later. BGFILTER_LOG_FORMAT=text|json
switches human vs shipper-friendly output. /healthz is not logged.
Verified locally via TestClient (fake manager, no models): 200 path logs
all fields + timings with rid == header; inbound id echoed; over-long id
sanitized; XOR 400s log status + error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
/remove-background now accepts either an uploaded `file` or an `image_url`
the service fetches itself (exactly one required). The fetch runs BEFORE
the compute gate so a slow download never holds a pool slot, and is
bounded by its own semaphore (2 x cpu_workers, hard cap 64) plus a shared
httpx client (connection reuse, matching max_connections) -- this restores
the throttling a direct upload has implicitly, so a burst of tiny URL
requests can't exhaust FDs/ephemeral ports or hammer the upstream.
Bounds: 25 MB body cap (413), 5 s connect / 15 s read timeout, <=3
redirects; MAX_IMAGE_PIXELS still applies post-decode. Deliberately NO
SSRF filtering (no private-IP/host allowlist) -- the endpoint trusts
authenticated callers; auth runs before any fetch. Do not expose to
untrusted clients without adding private-address/redirect filtering to
_fetch_image.
Adds httpx. New env knobs BGFILTER_FETCH_MAX_BYTES / BGFILTER_FETCH_TIMEOUT.
Docs (README/README_ZH/DEPLOY/DEPLOY_ZH) updated. _fetch_image unit-tested
locally (happy path, size cap ->413, bad scheme ->400, 404 ->400);
on-box end-to-end still pending (test box went offline).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CPU_WORKERS / GPU_CONCURRENCY were env-only. Add a `server:` config
section (server.cpu_workers, server.gpu_concurrency) so a lane's
concurrency lives in its yaml alongside device/precision, matching how
model.device already works. Precedence mirrors BGFILTER_DEVICE: config is
the default, the env var overrides per box.
The pool + admission semaphore now build in lifespan (after the config
loads) instead of at import, and PipelineManager reads gpu_concurrency
from the settings. configs/gpu.yaml ships 8 / 2 (its target is the 32 GB
GPU boxes) with an inline VRAM caveat: gpu_concurrency>1 must fit
weights + N × forward, so keep it 1 on ≤16 GB cards.
Verified on a 4080 SUPER: startup logs cpu_workers=8 gpu_concurrency=2
from gpu.yaml with no env set, request returns a valid RGBA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The single asyncio lock serialized the whole request (chroma -> seg ->
trimap -> ViTMatte -> cross-check -> foreground -> despill), so the GPU
sat idle ~70-80% of each request while the CPU stages ran. Replace it
with a thread pool that runs several requests concurrently, and move the
mutex down to just the GPU forwards: when one request steps off the GPU
to do CPU work, another steps on, keeping the device fed.
Two opt-in env knobs, both defaulting to the old strictly-serial
behaviour:
* BGFILTER_CPU_WORKERS (pool size) attacks the CPU-bound regime -- with
cross-check off, throughput went 0.91 -> 1.67 req/s (1.84x) at pool=8
on a 4080S, vs a mere +6% from a second worker process, at half the
VRAM (one model copy).
* BGFILTER_GPU_CONCURRENCY caps how many forwards may co-schedule on the
SMs. batch-1 underfills the GPU, so =2 recovers the SM overlap a lock
forgoes: cross-check on went 0.83 -> 1.01 req/s, beating two worker
processes (0.92) with one model copy (~19 GB vs 24 GB).
The CLI/single-thread path passes no lock and uses a nullcontext, so it
stays allocation- and contention-free. Concurrent output verified
identical (10x same-frame -> one hash). A per-worker admission semaphore
bounds in-flight decoded images to the pool size.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The old one-liner printed `status|restart|stop|tail bgfilter`, which
users pasted verbatim and the shell parsed the `|` as pipes. Split it
into one copy-pasteable command per action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sync the deployment guides with the c56e58a HTTP change: add a
"Background mode (flat vs complex)" subsection and a complex-mode curl
example. Notes that complex reuses existing weights, ignores screen_color,
defaults the cross-check veto off, and returns X-BGFilter-Background-Mode;
points to README for the full field/header reference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Backports master's non-flat matting (chroma.enabled: false + the hue-free
cross-check gate) into server-edition, and exposes it over HTTP without
surfacing the internal "chroma" wording: /remove-background gains a
background_mode form field (flat, default | complex). complex maps to
chroma disabled -- no colour key, segmentation alone drives the trimap and
every colour-keyed stage (auto-detect, hue split, chroma suppression,
despill) is bypassed. The cross-check veto still works in complex mode via
its second-opinion-confidence gate (cross_check.second_lo/hi) but defaults
OFF there (it costs the HR-matting forward); an explicit cross_check=on
re-enables it.
No new model weights: complex mode reuses the already-provisioned BiRefNet
seg + ViTMatte (+ optional HR-matting cross-check). Flat mode is unchanged
(bit-identical), and server-edition's own extras (cross_check.lock,
foreground.use_gpu CuPy path) are preserved -- the port is surgical, not a
copy of master's files.
- settings: ChromaSettings.enabled, CrossCheckSettings.second_lo/hi
- config: override_settings chroma passthrough
- despill/foreground: model=None safe guards (foreground keeps GPU path)
- alpha_post: cross_check_alpha hue-free gate when proj is None
- pipeline: _process_rgb complex branch (seg-only trimap, skip colour stages)
- service/app: process(chroma=), background_mode field, complex-defaults-off
cross-check, X-BGFilter-Background-Mode header
- cli: --chroma/--no-chroma
- configs/docs: gpu.yaml + default.yaml + README/README_ZH
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scripts/deploy_autodl.sh: idempotent setup for an AutoDL box -- installs app
deps (keeping the image's CUDA torch), picks the cupy wheel from torch's CUDA
version for GPU foreground, fetches weights, writes the auth token, renders a
supervisord instance (crash auto-restart) + a ~/.bashrc login autostart, then
starts the service and waits for /healthz. Args: --token, --port (default
6006), --config (default configs/gpu.yaml).
configs/gpu.yaml: the GPU lane (CUDA + bf16, full pipeline with cross-check,
despill off) promoted from a box-local file into the repo so the deploy is
reproducible.
The public port mapping and the "no auto-start without an interactive login"
gap are AutoDL platform limits, documented at the top of the script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cpu-fast lane disables cross-check to fit a low-memory box, but a request
could still send cross_check=on and override it, lazy-loading the HR-matting
model and running the 2048 forward -> the exact OOM (worker SIGKILL -> 502)
cpu-fast exists to avoid. BGFILTER_CROSS_CHECK=0 did not protect against this
because the per-request form field wins.
Add CrossCheckSettings.lock (default False). When set, PipelineManager
resolves cross-check to the base value and ignores per-request overrides, so a
locked off-lane silently honours cross_check=on as off -- no HR-matting load,
no cross-check run, no error (200), and the response header reports "off".
Enable it in cpu-fast.yaml (enabled: false, lock: true). Verified end-to-end:
cross_check=on on the locked lane returns without loading the cross-checker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The multi-level foreground colour estimation (pymatting estimate_foreground_ml)
is the pipeline's CPU bottleneck (~7 s/image). pymatting ships a CuPy backend
(estimate_foreground_ml_cupy) that runs the same multi-level solve on the GPU
~250x faster; verified visually identical to the float64 CPU result (opaque
colours bit-close, the float32 edge delta is alpha-weighted away).
Add ForegroundSettings.use_gpu (default True): with method "ml", try the CuPy
backend when cupy + a CUDA device are present, otherwise transparently fall back
to the CPU solver. pymatting only re-exports the cupy function when cupy is
importable, so CPU-only deployments (no cupy) are unaffected. Requires
cupy-cudaXXx matching the CUDA build on GPU boxes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- fix stale launch example (drop the D:\ lightML path -> generic uvicorn)
- add Endpoints, Authentication (X-Genarrative-Image-Token / BGFILTER_AUTH_TOKEN),
request-field table, response-header table, and status-code table
- update examples to show the auth header and cross_check=off
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DEPLOY.md / DEPLOY_ZH.md:
- add BGFILTER_AUTH_TOKEN to the env-var table
- new "Authentication" subsection under §10: opt-in token via the
X-Genarrative-Image-Token header (401 on missing/wrong), fail-open when
unset (proxy handles it), /healthz always open; drop the stale "no
authentication" line
- note that despill is off by default (same-hue false-positive)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds app-level authentication gated on the BGFILTER_AUTH_TOKEN env var.
When set, /remove-background requires the X-Genarrative-Image-Token header
to match it (constant-time compare via hmac); missing or wrong -> 401. The
header name matches the production nginx layer, so callers only swap the
URL + token value, no interface change.
Fail-open by design: when BGFILTER_AUTH_TOKEN is unset the check is a no-op
(open), preserving current behaviour and leaving auth to an upstream proxy
(nginx in production). A startup WARNING is logged when auth is disabled.
/healthz stays unauthenticated for health checks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
despill pulls foreground chroma along the background-hue axis with no
positional or semantic guard, so a subject sharing the background's hue
(e.g. a blue suit on a blue backdrop) is desaturated and hue-shifted
(measured ~ΔE 35, blue -> purple) — a false positive on correct colours.
Flip the DespillSettings.enabled dataclass default to False so every lane
that omits a despill section — notably cpu-fast — inherits it off, and
mirror it in default.yaml. Re-enable per-config or with --despill when
edge spill genuinely matters.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New rule 4 in fuse_trimap_directional: a chroma-unknown pixel whose seg
confidence is >= trimap.seg_force_fg (default 0.98) stays sure-FG even when
background-hued. Bright skin on a warm same-hue-family background (peach,
pale yellow) was being demoted to unknown by the hue split and then lost --
either ViTMatte itself misfires there (raw 0.07-0.13 measured) or the
chroma suppressor does (raw 0.89 halved to 0.45 via the RGB-proximity
confidence path). The segmenter meanwhile rates those pixels a saturated
1.0; that semantic certainty now outranks same-hue colour suspicion.
Safety: chroma_unknown excludes sure background, so a real flat backdrop
can never be forced foreground; the silhouette band still re-opens the
boundary; the cross-check veto still overrides trimap-FG. Set > 1.0 to
disable.
Verified (GPU bf16): yellow-bg frame face 0.84->1.00, peach-bg thigh
0.87->1.00 (residual softness only where chroma is near-sure-bg, by
design). Pastel-blue TestImage3 regression: 0.047% of pixels differ with
cross-check on, 0.058% without it (hair-gap suppression path), both
visually negligible.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both sides extended DEPLOY(_ZH): remote documented the cpu-fast lane (weights subset, config comparison, systemd note, OOM troubleshooting), local added the measured GPU profile and the reuse-off default. Conflict was the capacity-section memory bullet: kept the GPU deployment bullet and remote's cpu-fast-aware memory bullet (which supersedes the old three-models line).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two default flips for server-edition:
1. cross_check.reuse_as_seg now defaults to false (settings + default.yaml + CLI help): the dedicated segmenter keeps its own forward and the cross-check veto stays an independent second signal. The reuse remains available via --cross-check-as-seg / config; measured cost of off vs on: ~20 s/image on CPU, +0.1-0.25 s and +0.9-1.3 GB VRAM on GPU (13.3 vs 12.0 GB reserved under expandable segments -- still fits a 16 GB card).
2. bgfilter/__init__.py defaults PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True on non-Windows platforms, before torch loads (setdefault: an explicit env value wins; native Windows is excluded because torch warns and ignores it there). Measured: reserved 15.3 -> 12.0 GB and ~10% faster on the RTX 5070 Ti; no effect on CPU-only runs.
README/DEPLOY(_ZH) synced: reuse documented as opt-in, GPU profile numbers updated for both reuse states, perf table labeled with the config it was measured under.
Verified: defaults resolve off/set as intended on Windows and WSL; explicit PYTORCH_CUDA_ALLOC_CONF override wins.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measured on an RTX 5070 Ti 16 GB, full default pipeline (cross-check on, reused as seg, bf16): ~1-2 s/image, ~10.1 GB allocated. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (Linux/WSL2 only; silently unsupported on native Windows) cuts the allocator's reserved footprint 15.3 -> 12.0 GB and is ~10% faster, leaving ~3.5 GB headroom on a 16 GB card even beside a desktop session. bf16 is the precondition: fp32 reserved ~17 GB overflows. reuse-as-seg also saves ~0.9 GB VRAM / ~0.25 s per image on GPU (reuse off measured 13.3 GB reserved under expandable_segments).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adopt the service branch's refactors verbatim so the two branches stop
drifting on these files:
- pipeline.py: split _run_image into _process_rgb + MattingResult (an
in-memory API, exposed as MattingPipeline.run_rgb) and allow injecting
pre-loaded models into MattingPipeline. Disk I/O and debug dumps stay
in _run_image; the algorithm is line-for-line the same.
- foreground.py: estimate_foreground_rgb returns the foreground array
directly; the ForegroundEstimate dataclass carried background/
correction fields nothing consumed.
Verified: default TestImage3 run byte-identical to pre-backport master,
all debug artifacts written, run_rgb output matches run_image bytes
under save_rgba's quantization; both files now diff-clean against
server-edition.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Config-only tuning of the temporary CPU lane (no pipeline/code change). Query chunk 512 trims ViTMatte's attention transient ~0.9 GB at 2048x2048 (8.0 -> 7.1 GB peak working set) with bitwise-identical output; 256 gives nothing more. The remaining ~7.1 GB floor is attributed to ViTMatte's detail-capture decoder (backbone stage peaks 4.6 GB, decoder 7.1 GB, pymatting only 0.08 GB), so it is not further reducible by attention chunking -- noted for a possible future decoder-tiling pass. Accepted as-is for the 8 GB box.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Urgent lightweight production profile for a CPU-only fp32 server: base ViTMatte + one BiRefNet@1024, cross-check disabled. Provisions only those two weights (no BiRefNet_HR-matting, no anime-seg) and returns to the ~14-15s/image baseline. Run with BGFILTER_CONFIG=configs/cpu-fast.yaml.
Verified end-to-end: loads exactly 2 models, cross_check resolves off, service preloads the primary segmenter and not the cross-checker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux
server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed
activations in the arena, so RSS ratchets up across requests and the env var is
a no-op there.
A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at
runtime (no-op on Windows/musl/other allocators). Wired into service.process
(after each request) and the CLI batch loop (after each image), the two
long-lived paths where RSS accumulates. Single-image CLI exits, so it is left
alone.
B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also
improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/
MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc
malloc_trim simply no-ops.
Also fixes the stale off-by-default cross-check heading in DEPLOY.md.
Verified: modules import; release_freed_memory() returns False (no-op) on Windows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cross-check + reuse-as-seg are already the config/CLI default; align the HTTP
service with them instead of forcing cross-check off. The earlier off-by-default
was to avoid the extra inference, but reuse-as-seg now makes that forward double
as the seg mask (birefnet backend) -- it replaces the primary seg model rather
than adding to it, so the cost concern is gone.
- app.py: honour the config default (on); BGFILTER_CROSS_CHECK still forces
either way, unset = config.
- service.py preload: warm the cross-check HR-matting model, and skip the now-
redundant primary segmenter when reuse covers segmentation.
- README / DEPLOY(_ZH): document cross-check as default-on, BiRefNet_HR-matting
as required (not optional), BGFILTER_CROSS_CHECK=0 to disable.
Verified: service defaults cross_check on, env forces both ways, preload picks
cross-checker and skips the primary segmenter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adopts master as the standard for all overlapping work. Master's landed
optimizations supersede server-edition's own fp16 experiment:
- Unified --precision fp32|bf16 knob (bgfilter/precision.py) driving all
three models: ViTMatte weight cast + both BiRefNets via autocast, with a
hardware gate (falls back to fp32 off native-bf16 CPUs) and an
AutocastCPU fp32 shim for torchvision deform_conv2d.
- Query-chunked ViTMatte global attention (bgfilter/attn_chunk.py), exact
and bitwise-identical, caps the N^2 spike (~19 -> ~4 GB at 2048).
- Cross-check HR-matting forward reused as the seg mask (birefnet backend
only), skipping the primary seg model; MIMALLOC_PURGE_DELAY=0.
- inference_mode and the detect_background_color removal converge with
server-edition's earlier equivalents.
Conflict resolution (favoring master, preserving server-only features):
- vitmatte_infer/segmentation: dropped server's device-derived fp16 for
master's precision path, kept resolve_model_source (local weights).
- service.py: cross-check SegmentationSettings now passes precision so the
HTTP service honors bf16 like the CLI's _second_opinion does.
Verified on CPU: default fp32 pipeline loads 2 models (seg-reuse active)
and bf16 path runs (autocast + deform_conv2d shim) — both exit 0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
reuse_as_seg swaps one BiRefNet-family model for another (the validated
equivalence); it must not silently override an explicitly chosen anime-seg
backend. With --seg-backend anime-seg the segmenter now keeps its own
forward and the cross-check veto runs independently on top -- both
combinable, no flag juggling. Default birefnet path is untouched
(verified byte-identical); anime-seg + cross-check verified to run both
models.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- chroma.py: remove detect_background_color — orphaned since auto-detection
goes through estimate_background_model -> _background_border_cluster;
nothing in the repo calls it.
- settings.py: fix stale "Segmentation always stays fp32" comment
(--precision now fans out to the BiRefNet models too).
- README: add a measured CPU performance section (9700X reference numbers,
memory ceiling explanation, Zen 2 fallback guidance).
- docs/hair_gap_artifacts.md: record that the cross-check cost note is
obsolete — reuse_as_seg returns the net model count to 2, bf16 and
chunked attention absorb the rest.
- docs/green_screen_matting_workflow.md: add the 2026-07 additions to the
architecture-evolution note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ViTMatte's VitDet backbone runs 4 global attention blocks that materialize
the full [heads x N x N] map: ~19 GB transient at 2048x2048 (16384 tokens),
the pipeline's memory peak. transformers has no SDPA path for this
architecture (the decomposed rel-pos bias is added to raw scores), so
compute the same attention in query-row chunks instead: the bias
factorizes over query rows, making the chunked form mathematically exact
-- output verified BITWISE-identical (unit: fp32/bf16 x 3 sizes; full
pipeline: TestImage3 fp32 and 2048x2048 bf16, all byte-equal).
model.attn_query_chunk (default 2048, 0 = stock one-shot) engages only on
blocks seeing more tokens than the chunk size, so window blocks keep the
original path. Measured @2048x2048 bf16 (9700X): ViTMatte spike
19.8 -> 4.0 GB for ~15% more ViTMatte time; whole pipeline peak
21.4 -> 11.4 GB, warm 51 -> 53 s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three optimizations from profiling the cross-check-dominated pipeline
(9700X CPU, all pilot-validated on TestImage3/FixImage1):
- Reuse the cross-check HR-matting@2048 forward as the segmentation mask
(cross_check.reuse_as_seg, default ON; --no-cross-check-as-seg to opt
out). Skips the BiRefNet@1024 load+forward entirely: ~66s -> ~45s,
one less 0.9GB model. Trimap 99.8% identical, no structural change.
- --precision bf16 now fans out to all three models: ViTMatte keeps its
weight cast; both BiRefNets run their forward under autocast with a
dispatcher-level AutocastCPU fp32 shim for torchvision::deform_conv2d
(no bf16 CPU kernel, no autocast wrapper upstream). Shared hardware
gate in bgfilter/precision.py falls back to fp32 off native-bf16
hardware. TestImage3: 51.9s -> 33.1s; alpha diff max 0.15, none >0.25.
- MIMALLOC_PURGE_DELAY=0 (bgfilter/__init__.py, before torch loads):
Windows torch's bundled mimalloc lazily retains ~10GB of freed
BiRefNet activations, stacking under ViTMatte's attention peak.
2048x2048 bf16: peak 25.2 -> 21.4GB and slightly faster (73 -> 63s).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bf16 halves the matting model's activation memory -- its full-resolution
attention is the pipeline's memory peak -- with visually identical alpha
(measured: 0 px alpha deviation > 0.25 on samples; cross-check veto
behaviour unchanged, region IoU 0.984).
Guarded by a hardware check so it never lands on a slow emulation path:
CUDA requires torch.cuda.is_bf16_supported(); CPU requires the same
oneDNN native-bf16 gate PyTorch uses for matmul routing (AVX512-BF16/
AMX). Without support it warns and falls back to fp32 -- on a Zen2 EPYC
the fallback kernels measured 17-370x slower than fp32, so silent bf16
there would be a performance landmine.
Segmentation stays fp32: torchvision deform_conv2d (used by BiRefNet)
has no bf16 CPU kernel, and the segmenter is not the memory peak.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both call sites convert outputs to numpy immediately, so the stricter
inference-mode tensors are safe; saves autograd view/version tracking
overhead. Verified end-to-end on CUDA (alpha finite, no NaN).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Benchmarking c7a2819 on CPU showed unconditional fp16 is pathological
there: ~1870 s/image vs ~14.4 s fp32 (~130x) -- x86 has no native fp16
conv/gemm kernels so PyTorch falls back to a slow reference path. fp16
only pays off on CUDA, so ViTMatteRunner and BiRefNetSegmenter now pick
float16 iff the resolved device is cuda, float32 otherwise; inputs cast
to the same dtype. The dtype is intentionally not a setting.
Also replaces the deprecated torch_dtype= kwarg with dtype=.
Verified on CPU: single-image CLI run back to normal speed (~31 s wall
including model load) and output byte-identical (SHA256) to the
fp32-era baseline, confirming inference_mode alone changes nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- ViTMatte and BiRefNet load with torch_dtype=float16 (the BiRefNet
checkpoint ships fp16 anyway; the old .float() upcast is gone) and
floating-point inputs are cast to half to match.
- torch.no_grad() -> torch.inference_mode() in both predict paths.
- Outputs already downcast via .float() before .numpy(), so downstream
stays float32. anime-seg is ONNX and unaffected.
Effect verified by the user beforehand.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The GUI opencv-python build needs libGL.so.1 which headless Linux servers lack,
causing "ImportError: libGL.so.1" at first cv2 use. Pin the headless build so a
plain `pip install -r requirements.txt` works on servers out of the box (and
survives a git reset). BgFilter only uses cv2 for computation, so headless is a
drop-in on desktops too. Drop the now-unnecessary sed step from DEPLOY.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- fetch_weights.py: no-argument script that downloads every model weight into
models/ via the hf-mirror (proxy forced-bypassed, Xet disabled). Model ids are
read from config/code, and folders that already hold a weight file are skipped
(no network, no re-download). After it runs the app has full functionality
offline. Verified end to end (skips the 3 present models, pulls HR-matting).
- DEPLOY.md / DEPLOY_ZH.md: correct BiRefNet_HR-matting size from ~0.9 GB to the
verified ~425 MB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Per-model required-file directory tree for the local models/ layout
(ViTMatte / BiRefNet / anime-seg), noting which files each loader needs.
- Add ZhengPeng7/BiRefNet_HR-matting as the optional 4th model, used only by
the cross-check veto; new "cross-check" subsection covering enablement, the
offline-provisioning requirement, and the ~2x latency.
- Add BGFILTER_CROSS_CHECK to the env-var table and a troubleshooting row for
the offline "500 inference failed" case.
- Mirror all of the above into DEPLOY_ZH.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lhk229