Adopt the service branch's refactors verbatim so the two branches stop
drifting on these files:
- pipeline.py: split _run_image into _process_rgb + MattingResult (an
in-memory API, exposed as MattingPipeline.run_rgb) and allow injecting
pre-loaded models into MattingPipeline. Disk I/O and debug dumps stay
in _run_image; the algorithm is line-for-line the same.
- foreground.py: estimate_foreground_rgb returns the foreground array
directly; the ForegroundEstimate dataclass carried background/
correction fields nothing consumed.
Verified: default TestImage3 run byte-identical to pre-backport master,
all debug artifacts written, run_rgb output matches run_image bytes
under save_rgba's quantization; both files now diff-clean against
server-edition.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Config-only tuning of the temporary CPU lane (no pipeline/code change). Query chunk 512 trims ViTMatte's attention transient ~0.9 GB at 2048x2048 (8.0 -> 7.1 GB peak working set) with bitwise-identical output; 256 gives nothing more. The remaining ~7.1 GB floor is attributed to ViTMatte's detail-capture decoder (backbone stage peaks 4.6 GB, decoder 7.1 GB, pymatting only 0.08 GB), so it is not further reducible by attention chunking -- noted for a possible future decoder-tiling pass. Accepted as-is for the 8 GB box.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Urgent lightweight production profile for a CPU-only fp32 server: base ViTMatte + one BiRefNet@1024, cross-check disabled. Provisions only those two weights (no BiRefNet_HR-matting, no anime-seg) and returns to the ~14-15s/image baseline. Run with BGFILTER_CONFIG=configs/cpu-fast.yaml.
Verified end-to-end: loads exactly 2 models, cross_check resolves off, service preloads the primary segmenter and not the cross-checker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux
server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed
activations in the arena, so RSS ratchets up across requests and the env var is
a no-op there.
A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at
runtime (no-op on Windows/musl/other allocators). Wired into service.process
(after each request) and the CLI batch loop (after each image), the two
long-lived paths where RSS accumulates. Single-image CLI exits, so it is left
alone.
B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also
improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/
MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc
malloc_trim simply no-ops.
Also fixes the stale off-by-default cross-check heading in DEPLOY.md.
Verified: modules import; release_freed_memory() returns False (no-op) on Windows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cross-check + reuse-as-seg are already the config/CLI default; align the HTTP
service with them instead of forcing cross-check off. The earlier off-by-default
was to avoid the extra inference, but reuse-as-seg now makes that forward double
as the seg mask (birefnet backend) -- it replaces the primary seg model rather
than adding to it, so the cost concern is gone.
- app.py: honour the config default (on); BGFILTER_CROSS_CHECK still forces
either way, unset = config.
- service.py preload: warm the cross-check HR-matting model, and skip the now-
redundant primary segmenter when reuse covers segmentation.
- README / DEPLOY(_ZH): document cross-check as default-on, BiRefNet_HR-matting
as required (not optional), BGFILTER_CROSS_CHECK=0 to disable.
Verified: service defaults cross_check on, env forces both ways, preload picks
cross-checker and skips the primary segmenter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adopts master as the standard for all overlapping work. Master's landed
optimizations supersede server-edition's own fp16 experiment:
- Unified --precision fp32|bf16 knob (bgfilter/precision.py) driving all
three models: ViTMatte weight cast + both BiRefNets via autocast, with a
hardware gate (falls back to fp32 off native-bf16 CPUs) and an
AutocastCPU fp32 shim for torchvision deform_conv2d.
- Query-chunked ViTMatte global attention (bgfilter/attn_chunk.py), exact
and bitwise-identical, caps the N^2 spike (~19 -> ~4 GB at 2048).
- Cross-check HR-matting forward reused as the seg mask (birefnet backend
only), skipping the primary seg model; MIMALLOC_PURGE_DELAY=0.
- inference_mode and the detect_background_color removal converge with
server-edition's earlier equivalents.
Conflict resolution (favoring master, preserving server-only features):
- vitmatte_infer/segmentation: dropped server's device-derived fp16 for
master's precision path, kept resolve_model_source (local weights).
- service.py: cross-check SegmentationSettings now passes precision so the
HTTP service honors bf16 like the CLI's _second_opinion does.
Verified on CPU: default fp32 pipeline loads 2 models (seg-reuse active)
and bf16 path runs (autocast + deform_conv2d shim) — both exit 0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
reuse_as_seg swaps one BiRefNet-family model for another (the validated
equivalence); it must not silently override an explicitly chosen anime-seg
backend. With --seg-backend anime-seg the segmenter now keeps its own
forward and the cross-check veto runs independently on top -- both
combinable, no flag juggling. Default birefnet path is untouched
(verified byte-identical); anime-seg + cross-check verified to run both
models.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- chroma.py: remove detect_background_color — orphaned since auto-detection
goes through estimate_background_model -> _background_border_cluster;
nothing in the repo calls it.
- settings.py: fix stale "Segmentation always stays fp32" comment
(--precision now fans out to the BiRefNet models too).
- README: add a measured CPU performance section (9700X reference numbers,
memory ceiling explanation, Zen 2 fallback guidance).
- docs/hair_gap_artifacts.md: record that the cross-check cost note is
obsolete — reuse_as_seg returns the net model count to 2, bf16 and
chunked attention absorb the rest.
- docs/green_screen_matting_workflow.md: add the 2026-07 additions to the
architecture-evolution note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ViTMatte's VitDet backbone runs 4 global attention blocks that materialize
the full [heads x N x N] map: ~19 GB transient at 2048x2048 (16384 tokens),
the pipeline's memory peak. transformers has no SDPA path for this
architecture (the decomposed rel-pos bias is added to raw scores), so
compute the same attention in query-row chunks instead: the bias
factorizes over query rows, making the chunked form mathematically exact
-- output verified BITWISE-identical (unit: fp32/bf16 x 3 sizes; full
pipeline: TestImage3 fp32 and 2048x2048 bf16, all byte-equal).
model.attn_query_chunk (default 2048, 0 = stock one-shot) engages only on
blocks seeing more tokens than the chunk size, so window blocks keep the
original path. Measured @2048x2048 bf16 (9700X): ViTMatte spike
19.8 -> 4.0 GB for ~15% more ViTMatte time; whole pipeline peak
21.4 -> 11.4 GB, warm 51 -> 53 s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three optimizations from profiling the cross-check-dominated pipeline
(9700X CPU, all pilot-validated on TestImage3/FixImage1):
- Reuse the cross-check HR-matting@2048 forward as the segmentation mask
(cross_check.reuse_as_seg, default ON; --no-cross-check-as-seg to opt
out). Skips the BiRefNet@1024 load+forward entirely: ~66s -> ~45s,
one less 0.9GB model. Trimap 99.8% identical, no structural change.
- --precision bf16 now fans out to all three models: ViTMatte keeps its
weight cast; both BiRefNets run their forward under autocast with a
dispatcher-level AutocastCPU fp32 shim for torchvision::deform_conv2d
(no bf16 CPU kernel, no autocast wrapper upstream). Shared hardware
gate in bgfilter/precision.py falls back to fp32 off native-bf16
hardware. TestImage3: 51.9s -> 33.1s; alpha diff max 0.15, none >0.25.
- MIMALLOC_PURGE_DELAY=0 (bgfilter/__init__.py, before torch loads):
Windows torch's bundled mimalloc lazily retains ~10GB of freed
BiRefNet activations, stacking under ViTMatte's attention peak.
2048x2048 bf16: peak 25.2 -> 21.4GB and slightly faster (73 -> 63s).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bf16 halves the matting model's activation memory -- its full-resolution
attention is the pipeline's memory peak -- with visually identical alpha
(measured: 0 px alpha deviation > 0.25 on samples; cross-check veto
behaviour unchanged, region IoU 0.984).
Guarded by a hardware check so it never lands on a slow emulation path:
CUDA requires torch.cuda.is_bf16_supported(); CPU requires the same
oneDNN native-bf16 gate PyTorch uses for matmul routing (AVX512-BF16/
AMX). Without support it warns and falls back to fp32 -- on a Zen2 EPYC
the fallback kernels measured 17-370x slower than fp32, so silent bf16
there would be a performance landmine.
Segmentation stays fp32: torchvision deform_conv2d (used by BiRefNet)
has no bf16 CPU kernel, and the segmenter is not the memory peak.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both call sites convert outputs to numpy immediately, so the stricter
inference-mode tensors are safe; saves autograd view/version tracking
overhead. Verified end-to-end on CUDA (alpha finite, no NaN).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Benchmarking c7a2819 on CPU showed unconditional fp16 is pathological
there: ~1870 s/image vs ~14.4 s fp32 (~130x) -- x86 has no native fp16
conv/gemm kernels so PyTorch falls back to a slow reference path. fp16
only pays off on CUDA, so ViTMatteRunner and BiRefNetSegmenter now pick
float16 iff the resolved device is cuda, float32 otherwise; inputs cast
to the same dtype. The dtype is intentionally not a setting.
Also replaces the deprecated torch_dtype= kwarg with dtype=.
Verified on CPU: single-image CLI run back to normal speed (~31 s wall
including model load) and output byte-identical (SHA256) to the
fp32-era baseline, confirming inference_mode alone changes nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- ViTMatte and BiRefNet load with torch_dtype=float16 (the BiRefNet
checkpoint ships fp16 anyway; the old .float() upcast is gone) and
floating-point inputs are cast to half to match.
- torch.no_grad() -> torch.inference_mode() in both predict paths.
- Outputs already downcast via .float() before .numpy(), so downstream
stays float32. anime-seg is ONNX and unaffected.
Effect verified by the user beforehand.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The GUI opencv-python build needs libGL.so.1 which headless Linux servers lack,
causing "ImportError: libGL.so.1" at first cv2 use. Pin the headless build so a
plain `pip install -r requirements.txt` works on servers out of the box (and
survives a git reset). BgFilter only uses cv2 for computation, so headless is a
drop-in on desktops too. Drop the now-unnecessary sed step from DEPLOY.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- fetch_weights.py: no-argument script that downloads every model weight into
models/ via the hf-mirror (proxy forced-bypassed, Xet disabled). Model ids are
read from config/code, and folders that already hold a weight file are skipped
(no network, no re-download). After it runs the app has full functionality
offline. Verified end to end (skips the 3 present models, pulls HR-matting).
- DEPLOY.md / DEPLOY_ZH.md: correct BiRefNet_HR-matting size from ~0.9 GB to the
verified ~425 MB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Per-model required-file directory tree for the local models/ layout
(ViTMatte / BiRefNet / anime-seg), noting which files each loader needs.
- Add ZhengPeng7/BiRefNet_HR-matting as the optional 4th model, used only by
the cross-check veto; new "cross-check" subsection covering enablement, the
offline-provisioning requirement, and the ~2x latency.
- Add BGFILTER_CROSS_CHECK to the env-var table and a troubleshooting row for
the offline "500 inference failed" case.
- Mirror all of the above into DEPLOY_ZH.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- chroma.detect_background_color had no callers anywhere (auto-detection
goes through estimate_background_model / _background_border_cluster).
- ForegroundEstimate.background and .correction were computed on every
image but never consumed; only .rgb was used. estimate_foreground_rgb
now returns the foreground array directly and the dataclass is gone,
which also drops pymatting return_background work and a per-image
correction-norm pass.
Verified: CLI output on samples/TestImage3.png is byte-identical
(SHA256) to the pre-cleanup baseline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
POST /remove-background gains an optional cross_check form field
(1/0/true/false/yes/no/on/off; empty = service default). The effective
state is reported in the X-BGFilter-Cross-Check response header.
The cross-check model is now never preloaded at startup: the first
request that enables it pays the one-time load (measured ~8 s on top of
the ~50 s cc inference), after which the instance is cached and shared.
Verified with a functional test against PipelineManager: cc-off requests
never load the model, cc=1 lazy-loads then reuses the cached instance,
and omitting the field falls back to the service default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cross-model veto adds one extra large-model inference per request
(measured ~35 s/image on CPU: 14.4 s -> 49.1 s), which is too slow as a
service default. The service now disables it regardless of the config
default; BGFILTER_CROSS_CHECK=1 re-enables it (GPU or quality-first
deployments). CLI behaviour is unchanged (still on by default).
Startup log and /healthz report the effective state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Brings in the two algorithm optimisations from master:
- Gate chroma suppression by the matte's own confidence (639fdf0)
- Add cross-model veto of hair-gap background residue (adbbb9d)
Server-edition integration on top of the plain merge:
- PipelineManager caches the cross-check model (BiRefNet_HR-matting) and
injects it into MattingPipeline, so HTTP requests never reload it.
- MattingResult carries cross_check_alpha; debug output matches master.
- /healthz reports crossCheck; README lists models/BiRefNet_HR-matting
in the local-weights bundle.
Verified: CLI --no-cross-check baseline and full cross-check run both
pass on samples/TestImage3.png; cross_check_alpha.png written to debug.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Colour-drifted background trapped between hair strands defeats every
single-signal defence: the chroma key reads it as foreground (bgc ~0.07),
the segmenter backs it, ViTMatte rates it opaque, and post-hoc removal is
a proven dead end (it shreds the hair volume the same pixels belong to).
A second, trimap-free matting model (BiRefNet_HR-matting) is the only
tested model that separates this residue from the subject, so its opinion
is fused in as a veto: min-fusion that may only LOWER alpha, restricted to
the background-hued bright suspect zone (proj >= 3, L >= 45, feathered)
and gated by primary-alpha confidence (0.70 -> 0.95 ramp) so soft wisps
and dark hair are exempt by construction.
- settings/config/CLI: cross_check block, --cross-check/--no-cross-check
- alpha_post.cross_check_alpha after clean_alpha; second opinion reuses
BiRefNetSegmenter; saved to debug as cross_check_alpha.png
- chroma.bg_hue_projection extracted and shared with the trimap
- docs: methodology.md (new), hair_gap_artifacts.md (investigation log)
Verified: cross-check ON reproduces the visually-reviewed B1gate
prototype byte-for-byte on TestImage3; --no-cross-check reproduces the
previous baseline byte-for-byte; pink-bg FixImage1 face untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
suppress_alpha_by_chroma was the one stage where colour evidence could veto the
matting model outright, and every observed failure had the same shape: ViTMatte
was right (raw ~0.97 on a white shirt over pastel blue, on blush over pink) and
the bgc-keyed suppression crushed it, because bgc magnitude cannot tell a
background-coloured subject from background residue. The matte's raw alpha can:
it rates enclosed background pockets low (~0.28 median) and subject texture high.
Suppression is now scaled by (1 - smoothstep(raw_alpha, suppress_raw_lo,
suppress_raw_hi)) (defaults 0.85/0.98): colour evidence only vetoes the matte
where the matte itself is unsure. The gate applies only when alpha really comes
from ViTMatte; chroma-seeded alpha passes raw_alpha=None and keeps old behaviour.
Set suppress_raw_lo: 1.0 to disable (verified byte-identical to the old output).
Validated on both screen colours: pink #F4D8E8 face restored (skin alpha 0.841 ->
0.993, holes 17.2% -> 0.6%), pastel blue #CFEFFF unchanged-to-better (shirt +0.01,
wisps identical, gap clearing 99.8%). Also fixes the long-deferred chroma-only
pipeline defects (white shirt 0.900, light skin 0.989). CLI output verified
byte-identical to the reviewed prototype on both images.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document deploying the FastAPI service to a Linux server end to end: venv +
CPU-only torch (avoid the CUDA build), opencv-python-headless, the two weight
layouts (HF cache offline vs local models/ folders) with the mirror/proxy/Xet
download workaround, environment variables, uvicorn --workers 1, a systemd unit,
verification, remote exposure, capacity notes, and a troubleshooting table.
Link to it from README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add bgfilter/weights.py: resolve_model_source() maps a HuggingFace repo id
to a local folder under the weights dir (models/ by default, override with
BGFILTER_WEIGHTS_DIR) when one named after the repo basename exists; otherwise
the repo id is returned unchanged. Opt-in and backward compatible.
- vitmatte_infer.py / segmentation.py resolve model_name through it; anime-seg
reads <dir>/isnetis.onnx directly instead of hf_hub_download when local.
- .gitignore: models/, model-cache/
- README: document bundling weights as plain project folders.
Verified: anime-seg loads from a local models/anime-seg/ folder offline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BiRefNet is the default backend and --seg-backend switches segmenters, so the
birefnet.yaml config (a copy of default.yaml with backend: birefnet) no longer had
a purpose. configs/ now holds just default.yaml, the editable parameter template.
Also dropped the now-redundant --config from the chroma-debug README example.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ModelSettings.model_name dataclass default was still vitmatte-small (the
phase-1 "start small" choice) while configs/default.yaml used vitmatte-base, so a
bare run (no --config) silently differed from every documented example. Set the
dataclass default to base too: the built-in defaults now match default.yaml
field-for-field, so --config is only needed after editing that file to tune
parameters. README examples drop the now-redundant --config.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- chroma: drop BackgroundModel.hsv_center/hue_sigma (unused since the green gating
went away), their estimation, _hue_distance, and the now-unused hsv channel from
convert_color_spaces (one fewer cvtColor per image).
- despill: delete the dead green-channel-algebra path (the pipeline always passes a
colour) and fold the remaining Lab-direction despill into a single despill() with
no screen_color arg (127 -> 60 lines). Renamed despill_green -> despill.
- settings: remove the now-dead ChromaSettings.hue_sigma_min and
DespillSettings.green_excess_margin (plus the matching config lines).
- docs/comments: generalise stale "green" wording to "background colour" (module
docstring, alpha_post/trimap/foreground/settings) and rename the trimap variable
green_bg -> screen_bg.
Kept: the unmix foreground fallback, --matting-method chroma, and green/blue as
illustrative examples. Verified: package compiles, both configs parse, and an
auto-detect end-to-end run is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Segmentation backend now defaults to birefnet (ZhengPeng7/BiRefNet); anime-seg is
selected at runtime with --seg-backend anime-seg, which also swaps in the matching
weights (unknown backends raise). Config files stay for detailed tuning, not
backend selection.
- Default device is now cpu everywhere (dataclass defaults, default.yaml,
birefnet.yaml); pass --device cuda for GPU. Avoids a hard CUDA-required failure on
machines without a CUDA-enabled PyTorch build.
- Docs: README + workflow status note updated (backend via --seg-backend, CPU
default, examples no longer force --device cuda).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the hardcoded green-screen auto-detection with a general flat-colour
detector. When screen_color is null the background colour is now detected from the
image border (dominant colour of the border strip) and used directly as the chroma
model, with a failure gate that raises when the border is not one clean flat colour
(gradient / texture / subject filling the frame).
- chroma: add detect_background_color + _background_border_cluster; the null case of
estimate_background_model samples the detected cluster directly (no seed search);
compute_bg_confidence is now always perceptual (Lab/RGB distance). Removed the green
heuristic (initial_green_candidates), the green multiplicative gating, the now-unused
_smoothstep, and five green-only ChromaSettings fields.
- pipeline: reuse the detected colour for de-spill in the auto case.
- settings / default.yaml: add detect_* tuning fields; refresh screen_color docs.
- README: document auto-detection + the failure gate, trimap modes, birefnet config.
A supplied --screen-color hex still uses the seed-search refinement. Validated:
detects green / pastel / text samples correctly, raises on gradient / two-colour, CLI
exits 1 on failure, green and pastel mattes unchanged in quality.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A copy of default.yaml with segmentation.backend switched to birefnet
(ZhengPeng7/BiRefNet). BiRefNet is a general salient-object segmenter, so it
handles non-anime subjects (text, logos, real photos) that anime-seg returns an
empty mask for. Run with: --config configs/birefnet.yaml
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the two experimental bools (directional, bg_hued_to_bg) with a single
TrimapSettings.mode selector for the segmentation pipeline:
- "directional" (new default): chroma magnitude + seg + a Lab hue-direction
split. In the chroma-unknown zone a confidently-segmented pixel stays
foreground unless it is displaced toward the background hue, so a neutral
background-coloured garment (e.g. a white shirt) is kept while a background-
hued residual (blue between hair strands) is left unknown for ViTMatte /
chroma-suppress to clear.
- "seg": original fuse_trimap, unchanged.
- "directional-hard-bg": aggressive variant that hard-removes background-hued
pixels (can eat cool/shadowed white cloth).
Selectable via configs/default.yaml (trimap.mode) or CLI --trimap-mode; unknown
modes raise. Default CLI output verified byte-identical to the reviewed UNK
result on the pastel sample.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
README rewritten for the current design: two pipelines (chroma-only vs single
segmenter, default anime-seg), the screen_color prior (--screen-color / config,
green default), correct debug-file list, and HF/hf-mirror notes for the seg models.
No rescue/recolour layer is documented (it was removed).
green_screen_matting_workflow.md keeps the phase-1 green-screen spec but gains a
status note pointing to the README for the evolved architecture, and its debug-file
list is corrected (foreground_rgb / color_mask / seg_mask, not the old
foreground_background / foreground_correction / despill_mask).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Strategic pivot: the dropout-fill rescue, the anime-seg fill backend, the
BiRefNet+anime-seg intersection, and the recolor pass all existed to repair
source images whose hair was already green-contaminated or broken at generation
time. Fixing the source instead (clean pastel-background generation) makes the
matte high-contrast, so that whole compensation layer is unnecessary. Remove it.
- Two pipelines via segmentation.enabled:
false -> chroma-only (Chroma + ViTMatte)
true -> single segmenter (default anime-seg, switchable birefnet) -> trimap
then ViTMatte refines, pymatting estimates foreground, despill cleans spill.
- Generalise the green-hardcoded colour logic to a screen_color prior (hex, e.g.
"#CFEFFF"; default null = green auto-detect, byte-identical). chroma keys off
Lab/RGB distance to the colour; despill removes chroma along its Lab direction.
Added --screen-color CLI flag.
- Removed dead code: fill_seg_dropouts + seg_fill*/intersection params,
SegmentationSettings.fill_backend/fill_model_name/sharpen, unsharp_mask,
recolor.py + RecolorSettings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add seg_fill_intersect (default off) and seg_fill_gate_min to AlphaPostSettings, plus a gate_mask arg to fill_seg_dropouts. When enabled, the anime-seg dropout-fill is intersected with the reliable BiRefNet mask (>= gate_min): anime-seg proposes wisps at a low seg_fill_seg_min, but only rescues BiRefNet also endorses survive, trimming anime-seg's over-inclusive fringe while keeping its smooth value. Default off, so the shipped Option B (union) output is byte-identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add SegmentationSettings.fill_backend / fill_model_name so the dropout-fill can
use a different (higher-recall) segmenter than the trimap. Default is now the
combination the investigation converged on: BiRefNet builds the trimap (reliable
topology -- finger gap removed, strands kept as sure_fg), anime-seg drives the
fill (more, more-coherent anime wisps). Because the fill only adds (max), a
higher-recall-but-less-reliable mask there is harmless.
On TestImage2 this eliminates the strand breakage a full anime-seg pipeline
caused (lost 699 -> 0) while keeping ~91% of its extra wisp recall (gained
1133 -> 1031). Strands stay continuous; the recovered wisps are anime-seg's
coherent curves rather than BiRefNet's grainy fill.
Also adds an optional unsharp-mask knob (segmentation.sharpen, default off).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add AnimeSegSegmenter (skytnt/anime-seg ISNet ONNX via onnxruntime, no remote
code) and a make_segmenter factory selected by SegmentationSettings.backend
("birefnet" | "anime-seg"). The ONNX output is already 0..1, so it slots into the
same soft-mask interface BiRefNetSegmenter uses.
On the anime samples anime-seg recovers more and more-coherent hair wisps than
BiRefNet (TestImage2 shoulder rescue: added px 1886 -> 3278, largest connected
component 147 -> 454), as expected from an anime-trained model. Default backend
stays birefnet.
Adds onnxruntime to requirements.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The rescue floored alpha with BiRefNet's soft mask verbatim, which sprinkled
~800 isolated <10px specks (burrs) across the matte from the mask's scattered
low-confidence noise. Drop rescued connected components below a min area, keeping
only the connected wisp structures.
On TestImage2 the rescued components fall from 900 (818 isolated specks) to 91
and the added pixels from 5124 to 1886 -- the genuine wisp strands stay, the
burrs go. smoke still passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add bgfilter/recolor.py: after despill, propagate clean neighbour chroma into
pixels whose colour is still off (residual green, or the magenta a green unmix
overshoots into), keeping luminance partway so hair texture survives. It installs
a real observed colour rather than subtracting, so it cannot overshoot. Despill
already neutralises the matte edge (off ~ 0 there), so recolor only touches the
contaminated interior and the wisps the dropout-fill rescued.
Runs as a final colour pass on the despilled foreground; pymatting + despill keep
their stronger edge handling. On the samples the magenta fringe drops sharply
(TestImage vis-magenta 0.065->0.014, TestImage2 0.085->0.031) with edge green
unchanged (vis-green ~0.02). Debug output color_mask.png replaces the despill /
foreground-background / foreground-correction maps; smoke updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add fill_seg_dropouts: where ViTMatte's alpha is ~0 but BiRefNet confidently
asserts foreground, restore a faint alpha from the mask (strength * seg_mask).
This recovers green-contaminated wisps that ViTMatte cuts because they are
nearly the background colour, while leaving ViTMatte's sharper edges elsewhere
untouched (gated to true dropouts) and not refilling the finger gap (low mask).
On the TestImage2 shoulder, faint wisps ViTMatte cut drop from 72% to 33%; the
finger gap stays transparent. The rescued wisps are still green-tinted -- the
recolor pass (next) restores their colour.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route faint hair (BiRefNet M 0.08-0.15) into the unknown band instead of sure
background, giving ViTMatte a chance at it. On its own this does not recover the
green-contaminated shoulder wisps: those are ~pure green (dominance +0.88, like
the screen) so ViTMatte cuts them regardless of trimap -- that is handled by the
upcoming segmentation dropout-fill. The finger gap (M~0.02) stays sure background.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire BiRefNet into the pipeline (lazy-loaded) and build the trimap from the
semantic subject mask refined by the green key, used when segmentation is
enabled. generate_trimap (chroma-only) stays as the no-segmentation fallback.
Authority split: the mask decides subject topology (keeps green-tinted hair as
foreground, drops see-through holes), the bright-green key sharpens the screen
boundary, ViTMatte refines a small protective band. Background follows the mask
directly so interior holes survive; the band is small so it does not swallow
finger gaps.
End-to-end on the samples: green hair preserved, finger-gap green 99%
transparent (BiRefNet marks it background; was 100% foreground before), leg
edges smooth -- topology/colour/edge problems local cues could not solve, with
no hard alpha surgery.
- SegmentationSettings wired into PipelineSettings/config; --device propagates.
- TrimapSettings: seg_core_threshold, seg_loose_threshold.
- New debug output seg_mask.png.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds bgfilter/segmentation.py: BiRefNetSegmenter produces a soft subject mask
that defines topology -- it includes colour-contaminated foreground (green-tinted
hair) and excludes see-through holes (finger gaps), the distinction chroma/colour
cues cannot make. Validated on both samples: green hair -> foreground, finger gap
-> background, both correct.
Foundation of the semantic-guided redesign; the mask will drive trimap
construction next, with ViTMatte still refining soft edges.
- SegmentationSettings (ZhengPeng7/BiRefNet, 1024 input, fp32).
- Deps: torchvision, timm, einops, kornia. Weights load from HuggingFace via
trust_remote_code; behind a firewall set HF_ENDPOINT=https://hf-mirror.com and
bypass the proxy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lhk229