No colour key at all: segmentation alone drives the trimap (mode forced
to 'seg', whose chroma terms degrade to no-ops on a zero background-
confidence map), ViTMatte still refines the unknown band at full
resolution, and pymatting still estimates edge foreground colour. Every
colour-keyed stage is bypassed: background auto-detect, the directional
hue split, chroma alpha suppression, despill, the cross-check veto (its
suspect zone is background-hued by definition; with reuse_as_seg the
second opinion still serves as the segmenter), and the unmix fallback.
Guards: requires segmentation.enabled and matting_method 'vitmatte'
(clear errors otherwise); despill/foreground handle model=None.
Validation: flat-background default path is bit-identical pre/post
(alpha and RGB |D|max = 0 on TestImage3). On a known-alpha subject
composited over a gradient+blotch background, recovered alpha scores
MAE 0.0018 / IoU@0.5 0.993 (0.995 with --cross-check-as-seg); the same
input correctly fails auto-detection in default mode.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adopt server-edition's settings.py and default.yaml verbatim (zero diff on
these files between branches now):
- reuse_as_seg: false. The reuse equivalence holds for solid figures but
breaks on thin pale strokes over a low-contrast background: HR-matting
(trained on P3M-10k portraits + AM-2k animals) reads them as
semi-transparent wisps, and as the trimap topology source it destroys
white calligraphy (TestImage4: stroke alpha 0.92 -> 0.51, 48% of stroke
pixels below 0.5) that the DIS5K-trained segmenter keeps solid (seg 0.97
vs 0.40). As the gated cross-check veto the same signal stays harmless
(hue gate excludes neutral strokes; confidence gate exempts soft edges).
Opt in with --cross-check-as-seg for character batches.
- despill.enabled: false (mirrors server ba5d667): no positional/semantic
guard, desaturates and hue-shifts subjects sharing the background's hue.
README updated accordingly; the performance table notes its numbers were
measured with reuse on.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three optimizations from profiling the cross-check-dominated pipeline
(9700X CPU, all pilot-validated on TestImage3/FixImage1):
- Reuse the cross-check HR-matting@2048 forward as the segmentation mask
(cross_check.reuse_as_seg, default ON; --no-cross-check-as-seg to opt
out). Skips the BiRefNet@1024 load+forward entirely: ~66s -> ~45s,
one less 0.9GB model. Trimap 99.8% identical, no structural change.
- --precision bf16 now fans out to all three models: ViTMatte keeps its
weight cast; both BiRefNets run their forward under autocast with a
dispatcher-level AutocastCPU fp32 shim for torchvision::deform_conv2d
(no bf16 CPU kernel, no autocast wrapper upstream). Shared hardware
gate in bgfilter/precision.py falls back to fp32 off native-bf16
hardware. TestImage3: 51.9s -> 33.1s; alpha diff max 0.15, none >0.25.
- MIMALLOC_PURGE_DELAY=0 (bgfilter/__init__.py, before torch loads):
Windows torch's bundled mimalloc lazily retains ~10GB of freed
BiRefNet activations, stacking under ViTMatte's attention peak.
2048x2048 bf16: peak 25.2 -> 21.4GB and slightly faster (73 -> 63s).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bf16 halves the matting model's activation memory -- its full-resolution
attention is the pipeline's memory peak -- with visually identical alpha
(measured: 0 px alpha deviation > 0.25 on samples; cross-check veto
behaviour unchanged, region IoU 0.984).
Guarded by a hardware check so it never lands on a slow emulation path:
CUDA requires torch.cuda.is_bf16_supported(); CPU requires the same
oneDNN native-bf16 gate PyTorch uses for matmul routing (AVX512-BF16/
AMX). Without support it warns and falls back to fp32 -- on a Zen2 EPYC
the fallback kernels measured 17-370x slower than fp32, so silent bf16
there would be a performance landmine.
Segmentation stays fp32: torchvision deform_conv2d (used by BiRefNet)
has no bf16 CPU kernel, and the segmenter is not the memory peak.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Colour-drifted background trapped between hair strands defeats every
single-signal defence: the chroma key reads it as foreground (bgc ~0.07),
the segmenter backs it, ViTMatte rates it opaque, and post-hoc removal is
a proven dead end (it shreds the hair volume the same pixels belong to).
A second, trimap-free matting model (BiRefNet_HR-matting) is the only
tested model that separates this residue from the subject, so its opinion
is fused in as a veto: min-fusion that may only LOWER alpha, restricted to
the background-hued bright suspect zone (proj >= 3, L >= 45, feathered)
and gated by primary-alpha confidence (0.70 -> 0.95 ramp) so soft wisps
and dark hair are exempt by construction.
- settings/config/CLI: cross_check block, --cross-check/--no-cross-check
- alpha_post.cross_check_alpha after clean_alpha; second opinion reuses
BiRefNetSegmenter; saved to debug as cross_check_alpha.png
- chroma.bg_hue_projection extracted and shared with the trimap
- docs: methodology.md (new), hair_gap_artifacts.md (investigation log)
Verified: cross-check ON reproduces the visually-reviewed B1gate
prototype byte-for-byte on TestImage3; --no-cross-check reproduces the
previous baseline byte-for-byte; pink-bg FixImage1 face untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Segmentation backend now defaults to birefnet (ZhengPeng7/BiRefNet); anime-seg is
selected at runtime with --seg-backend anime-seg, which also swaps in the matching
weights (unknown backends raise). Config files stay for detailed tuning, not
backend selection.
- Default device is now cpu everywhere (dataclass defaults, default.yaml,
birefnet.yaml); pass --device cuda for GPU. Avoids a hard CUDA-required failure on
machines without a CUDA-enabled PyTorch build.
- Docs: README + workflow status note updated (backend via --seg-backend, CPU
default, examples no longer force --device cuda).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the hardcoded green-screen auto-detection with a general flat-colour
detector. When screen_color is null the background colour is now detected from the
image border (dominant colour of the border strip) and used directly as the chroma
model, with a failure gate that raises when the border is not one clean flat colour
(gradient / texture / subject filling the frame).
- chroma: add detect_background_color + _background_border_cluster; the null case of
estimate_background_model samples the detected cluster directly (no seed search);
compute_bg_confidence is now always perceptual (Lab/RGB distance). Removed the green
heuristic (initial_green_candidates), the green multiplicative gating, the now-unused
_smoothstep, and five green-only ChromaSettings fields.
- pipeline: reuse the detected colour for de-spill in the auto case.
- settings / default.yaml: add detect_* tuning fields; refresh screen_color docs.
- README: document auto-detection + the failure gate, trimap modes, birefnet config.
A supplied --screen-color hex still uses the seed-search refinement. Validated:
detects green / pastel / text samples correctly, raises on gradient / two-colour, CLI
exits 1 on failure, green and pastel mattes unchanged in quality.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the two experimental bools (directional, bg_hued_to_bg) with a single
TrimapSettings.mode selector for the segmentation pipeline:
- "directional" (new default): chroma magnitude + seg + a Lab hue-direction
split. In the chroma-unknown zone a confidently-segmented pixel stays
foreground unless it is displaced toward the background hue, so a neutral
background-coloured garment (e.g. a white shirt) is kept while a background-
hued residual (blue between hair strands) is left unknown for ViTMatte /
chroma-suppress to clear.
- "seg": original fuse_trimap, unchanged.
- "directional-hard-bg": aggressive variant that hard-removes background-hued
pixels (can eat cool/shadowed white cloth).
Selectable via configs/default.yaml (trimap.mode) or CLI --trimap-mode; unknown
modes raise. Default CLI output verified byte-identical to the reviewed UNK
result on the pastel sample.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Strategic pivot: the dropout-fill rescue, the anime-seg fill backend, the
BiRefNet+anime-seg intersection, and the recolor pass all existed to repair
source images whose hair was already green-contaminated or broken at generation
time. Fixing the source instead (clean pastel-background generation) makes the
matte high-contrast, so that whole compensation layer is unnecessary. Remove it.
- Two pipelines via segmentation.enabled:
false -> chroma-only (Chroma + ViTMatte)
true -> single segmenter (default anime-seg, switchable birefnet) -> trimap
then ViTMatte refines, pymatting estimates foreground, despill cleans spill.
- Generalise the green-hardcoded colour logic to a screen_color prior (hex, e.g.
"#CFEFFF"; default null = green auto-detect, byte-identical). chroma keys off
Lab/RGB distance to the colour; despill removes chroma along its Lab direction.
Added --screen-color CLI flag.
- Removed dead code: fill_seg_dropouts + seg_fill*/intersection params,
SegmentationSettings.fill_backend/fill_model_name/sharpen, unsharp_mask,
recolor.py + RecolorSettings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lhk229
Codex