Replace the two experimental bools (directional, bg_hued_to_bg) with a single
TrimapSettings.mode selector for the segmentation pipeline:
- "directional" (new default): chroma magnitude + seg + a Lab hue-direction
split. In the chroma-unknown zone a confidently-segmented pixel stays
foreground unless it is displaced toward the background hue, so a neutral
background-coloured garment (e.g. a white shirt) is kept while a background-
hued residual (blue between hair strands) is left unknown for ViTMatte /
chroma-suppress to clear.
- "seg": original fuse_trimap, unchanged.
- "directional-hard-bg": aggressive variant that hard-removes background-hued
pixels (can eat cool/shadowed white cloth).
Selectable via configs/default.yaml (trimap.mode) or CLI --trimap-mode; unknown
modes raise. Default CLI output verified byte-identical to the reviewed UNK
result on the pastel sample.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
README rewritten for the current design: two pipelines (chroma-only vs single
segmenter, default anime-seg), the screen_color prior (--screen-color / config,
green default), correct debug-file list, and HF/hf-mirror notes for the seg models.
No rescue/recolour layer is documented (it was removed).
green_screen_matting_workflow.md keeps the phase-1 green-screen spec but gains a
status note pointing to the README for the evolved architecture, and its debug-file
list is corrected (foreground_rgb / color_mask / seg_mask, not the old
foreground_background / foreground_correction / despill_mask).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Strategic pivot: the dropout-fill rescue, the anime-seg fill backend, the
BiRefNet+anime-seg intersection, and the recolor pass all existed to repair
source images whose hair was already green-contaminated or broken at generation
time. Fixing the source instead (clean pastel-background generation) makes the
matte high-contrast, so that whole compensation layer is unnecessary. Remove it.
- Two pipelines via segmentation.enabled:
false -> chroma-only (Chroma + ViTMatte)
true -> single segmenter (default anime-seg, switchable birefnet) -> trimap
then ViTMatte refines, pymatting estimates foreground, despill cleans spill.
- Generalise the green-hardcoded colour logic to a screen_color prior (hex, e.g.
"#CFEFFF"; default null = green auto-detect, byte-identical). chroma keys off
Lab/RGB distance to the colour; despill removes chroma along its Lab direction.
Added --screen-color CLI flag.
- Removed dead code: fill_seg_dropouts + seg_fill*/intersection params,
SegmentationSettings.fill_backend/fill_model_name/sharpen, unsharp_mask,
recolor.py + RecolorSettings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add seg_fill_intersect (default off) and seg_fill_gate_min to AlphaPostSettings, plus a gate_mask arg to fill_seg_dropouts. When enabled, the anime-seg dropout-fill is intersected with the reliable BiRefNet mask (>= gate_min): anime-seg proposes wisps at a low seg_fill_seg_min, but only rescues BiRefNet also endorses survive, trimming anime-seg's over-inclusive fringe while keeping its smooth value. Default off, so the shipped Option B (union) output is byte-identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add SegmentationSettings.fill_backend / fill_model_name so the dropout-fill can
use a different (higher-recall) segmenter than the trimap. Default is now the
combination the investigation converged on: BiRefNet builds the trimap (reliable
topology -- finger gap removed, strands kept as sure_fg), anime-seg drives the
fill (more, more-coherent anime wisps). Because the fill only adds (max), a
higher-recall-but-less-reliable mask there is harmless.
On TestImage2 this eliminates the strand breakage a full anime-seg pipeline
caused (lost 699 -> 0) while keeping ~91% of its extra wisp recall (gained
1133 -> 1031). Strands stay continuous; the recovered wisps are anime-seg's
coherent curves rather than BiRefNet's grainy fill.
Also adds an optional unsharp-mask knob (segmentation.sharpen, default off).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add AnimeSegSegmenter (skytnt/anime-seg ISNet ONNX via onnxruntime, no remote
code) and a make_segmenter factory selected by SegmentationSettings.backend
("birefnet" | "anime-seg"). The ONNX output is already 0..1, so it slots into the
same soft-mask interface BiRefNetSegmenter uses.
On the anime samples anime-seg recovers more and more-coherent hair wisps than
BiRefNet (TestImage2 shoulder rescue: added px 1886 -> 3278, largest connected
component 147 -> 454), as expected from an anime-trained model. Default backend
stays birefnet.
Adds onnxruntime to requirements.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The rescue floored alpha with BiRefNet's soft mask verbatim, which sprinkled
~800 isolated <10px specks (burrs) across the matte from the mask's scattered
low-confidence noise. Drop rescued connected components below a min area, keeping
only the connected wisp structures.
On TestImage2 the rescued components fall from 900 (818 isolated specks) to 91
and the added pixels from 5124 to 1886 -- the genuine wisp strands stay, the
burrs go. smoke still passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add bgfilter/recolor.py: after despill, propagate clean neighbour chroma into
pixels whose colour is still off (residual green, or the magenta a green unmix
overshoots into), keeping luminance partway so hair texture survives. It installs
a real observed colour rather than subtracting, so it cannot overshoot. Despill
already neutralises the matte edge (off ~ 0 there), so recolor only touches the
contaminated interior and the wisps the dropout-fill rescued.
Runs as a final colour pass on the despilled foreground; pymatting + despill keep
their stronger edge handling. On the samples the magenta fringe drops sharply
(TestImage vis-magenta 0.065->0.014, TestImage2 0.085->0.031) with edge green
unchanged (vis-green ~0.02). Debug output color_mask.png replaces the despill /
foreground-background / foreground-correction maps; smoke updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add fill_seg_dropouts: where ViTMatte's alpha is ~0 but BiRefNet confidently
asserts foreground, restore a faint alpha from the mask (strength * seg_mask).
This recovers green-contaminated wisps that ViTMatte cuts because they are
nearly the background colour, while leaving ViTMatte's sharper edges elsewhere
untouched (gated to true dropouts) and not refilling the finger gap (low mask).
On the TestImage2 shoulder, faint wisps ViTMatte cut drop from 72% to 33%; the
finger gap stays transparent. The rescued wisps are still green-tinted -- the
recolor pass (next) restores their colour.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route faint hair (BiRefNet M 0.08-0.15) into the unknown band instead of sure
background, giving ViTMatte a chance at it. On its own this does not recover the
green-contaminated shoulder wisps: those are ~pure green (dominance +0.88, like
the screen) so ViTMatte cuts them regardless of trimap -- that is handled by the
upcoming segmentation dropout-fill. The finger gap (M~0.02) stays sure background.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire BiRefNet into the pipeline (lazy-loaded) and build the trimap from the
semantic subject mask refined by the green key, used when segmentation is
enabled. generate_trimap (chroma-only) stays as the no-segmentation fallback.
Authority split: the mask decides subject topology (keeps green-tinted hair as
foreground, drops see-through holes), the bright-green key sharpens the screen
boundary, ViTMatte refines a small protective band. Background follows the mask
directly so interior holes survive; the band is small so it does not swallow
finger gaps.
End-to-end on the samples: green hair preserved, finger-gap green 99%
transparent (BiRefNet marks it background; was 100% foreground before), leg
edges smooth -- topology/colour/edge problems local cues could not solve, with
no hard alpha surgery.
- SegmentationSettings wired into PipelineSettings/config; --device propagates.
- TrimapSettings: seg_core_threshold, seg_loose_threshold.
- New debug output seg_mask.png.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds bgfilter/segmentation.py: BiRefNetSegmenter produces a soft subject mask
that defines topology -- it includes colour-contaminated foreground (green-tinted
hair) and excludes see-through holes (finger gaps), the distinction chroma/colour
cues cannot make. Validated on both samples: green hair -> foreground, finger gap
-> background, both correct.
Foundation of the semantic-guided redesign; the mask will drive trimap
construction next, with ViTMatte still refining soft edges.
- SegmentationSettings (ZhengPeng7/BiRefNet, 1024 input, fp32).
- Deps: torchvision, timm, einops, kornia. Weights load from HuggingFace via
trust_remote_code; behind a firewall set HF_ENDPOINT=https://hf-mirror.com and
bypass the proxy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This reverts commit f7e12d3.
The carve set alpha to 0 on a colour-thresholded mask across the whole
foreground, which cannot distinguish background green from green the AI
generator bled onto the subject. It therefore (1) deleted green-tinted hair
strands and (2) hard-cut the green-spill silhouette edge, replacing ViTMatte's
anti-aliased edge with a jagged one. Local cues (surround, opacity, thinness,
connectivity, neighbour colour) could not reliably separate green hair from
green holes, so the mask is not fixable by tuning. Reverting restores intact
hair and smooth edges; the finger-gap green stays a minor residual.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The green in the finger gap survives keying because it is dark and desaturated
(brightness ~0.58, hue-shifted), so chroma confidence reads ~0 and the trimap
leaves it in the unknown band, where ViTMatte fills the pocket as foreground.
It is also connected to the exterior screen (an open notch), so neither
confidence suppression nor classic hole-filling removes it.
Add carve_background_holes (doc section 10): a pixel that is clearly
green-dominant in the input *and* mostly surrounded by genuine (non-green)
foreground is set to alpha 0. The surround cue separates such an intrusion
(finger gaps, inter-strand pockets) from the outer hair edge, which has
background on one side and is preserved. Runs after alpha cleanup.
On TestImage the finger-gap survivors (green & alpha>0.5) drop 170 -> 28 with
no visible loss of hair wisps; inter-strand green pockets clear too. Knobs in
AlphaPostSettings (hole_*, default on, surround_min 0.4).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull alpha toward 0 in the unknown band where bg_confidence is high, so green
that survives in hair gaps and hole pockets goes transparent (doc section 10).
Runs between trimap enforcement and alpha cleanup; sure-foreground pixels are
never touched, keyed by the bg_confidence we already compute.
On the two samples this clears the faint outer green halo (edge pixels
19.3k -> 16.0k) but has little *visible* effect, because their residual is no
longer green: it is magenta from the pymatting unmix on genuine semi-transparent
hair (vis-magenta p95 ~0.07, identical pre/post despill), which this pass
deliberately leaves alone. The magenta needs a separate colour fix.
Knobs live in AlphaPostSettings (chroma_suppress*, default on).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After the full-foreground despill (5ddf0cd) the residual green was confined to
semi-transparent hair (alpha < 0.5): 85% of the worst pixels had alpha < 0.3,
where a pixel-wise unmix (divide by small alpha) is too noisy to trust.
Replace the hand-rolled unmix/local-blur estimator with pymatting's
estimate_foreground_ml (Germer et al. multi-level closed form), which propagates
reliable foreground colour from high-alpha neighbours into the fringe and
estimates the background, so green spill is unmixed rather than clamped. The
despill pass stays as a light cleanup on top.
- foreground.method selects "ml" (default) or "unmix" (legacy heuristic kept as
a fallback when pymatting is unavailable, matching the chroma fallback idiom).
- ForegroundEstimate now exposes rgb (F), background (B) and a correction map;
debug outputs become foreground_rgb / foreground_background / foreground_correction.
Controlled comparison (same ViTMatte alpha, only foreground method changed),
edge_green_excess:
TestImage p95 0.176->0.031, mean 0.061->0.005
TestImage2 p95 0.165->0.004, mean 0.052->0.002
Adds pymatting (pulls in numba) to requirements.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Green spill bleeds into pixels ViTMatte marks as solid foreground (alpha~1),
but despill was gated to the soft edge band (0.005<alpha<0.995) and weakened
exactly there: the alpha tent peaked at 0.5 and bg_confidence (low on the
character edge) further suppressed the correction.
- (1) Apply despill to the whole foreground (alpha > edge_low) instead of the
soft band only; neutral pixels stay untouched via the green-excess term.
- (2) Weight by alpha at full strength across the opaque range (alpha>=0.5),
easing off only where mostly transparent, instead of a tent peaking at 0.5.
- (3) Drop bg_confidence as a despill gate; localise purely by green excess.
Controlled comparison (same alpha, only despill changed), edge_green_excess:
ViTMatte TestImage p95 0.239->0.176, mean 0.131->0.061
ViTMatte TestImage2 p95 0.231->0.165, mean 0.115->0.052
Removes now-unused DespillSettings fields edge_high and bg_confidence_weight.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lhk229
Codex