Compare commits

4 Commits

Author SHA1 Message Date
lhk229 2b3b62ed3f Run the cross-check veto in complex-background mode with a hue-free gate
The veto's suspect zone was defined by background-hue projection, so
--no-chroma silently skipped it. Without a key colour there is no hue
test to lean on, so in complex mode the gate instead requires the second
opinion itself to be confidently near-empty (second_lo -> second_hi ramp,
default 0.15 -> 0.40): regions the HR-matting model decisively rejects
can be cleared, while thin strands it merely blurs to mid-alpha are
untouched -- protecting exactly the crisp-strand advantage the pipeline
has over a raw BiRefNet mask. Flat mode's gate is unchanged.

Validation (BG_IMAGE01-04, cuda fp32): flat default path bit-identical;
complex-mode veto touches 0.007-0.025% of pixels, visibly clearing
blurred residue near strands and milky specks in hair gaps with no
strand erosion. Costs the HR-matting forward in --no-chroma runs
(0.61 -> 1.6 s/image warm GPU); disable with --no-cross-check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 13:09:57 +08:00
lhk229 e61fbe84f0 Add non-flat background mode (--no-chroma / chroma.enabled: false)
No colour key at all: segmentation alone drives the trimap (mode forced
to 'seg', whose chroma terms degrade to no-ops on a zero background-
confidence map), ViTMatte still refines the unknown band at full
resolution, and pymatting still estimates edge foreground colour. Every
colour-keyed stage is bypassed: background auto-detect, the directional
hue split, chroma alpha suppression, despill, the cross-check veto (its
suspect zone is background-hued by definition; with reuse_as_seg the
second opinion still serves as the segmenter), and the unmix fallback.

Guards: requires segmentation.enabled and matting_method 'vitmatte'
(clear errors otherwise); despill/foreground handle model=None.

Validation: flat-background default path is bit-identical pre/post
(alpha and RGB |D|max = 0 on TestImage3). On a known-alpha subject
composited over a gradient+blotch background, recovered alpha scores
MAE 0.0018 / IoU@0.5 0.993 (0.995 with --cross-check-as-seg); the same
input correctly fails auto-detection in default mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:13:08 +08:00
lhk229 f22af16517 Align defaults with server-edition: reuse-as-seg and despill both OFF
Adopt server-edition's settings.py and default.yaml verbatim (zero diff on
these files between branches now):

- reuse_as_seg: false. The reuse equivalence holds for solid figures but
  breaks on thin pale strokes over a low-contrast background: HR-matting
  (trained on P3M-10k portraits + AM-2k animals) reads them as
  semi-transparent wisps, and as the trimap topology source it destroys
  white calligraphy (TestImage4: stroke alpha 0.92 -> 0.51, 48% of stroke
  pixels below 0.5) that the DIS5K-trained segmenter keeps solid (seg 0.97
  vs 0.40). As the gated cross-check veto the same signal stays harmless
  (hue gate excludes neutral strokes; confidence gate exempts soft edges).
  Opt in with --cross-check-as-seg for character batches.

- despill.enabled: false (mirrors server ba5d667): no positional/semantic
  guard, desaturates and hue-shifts subjects sharing the background's hue.

README updated accordingly; the performance table notes its numbers were
measured with reuse on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 12:37:02 +08:00
lhk229 18c6918e45 Add seg_force_fg: saturated seg confidence overrides the hue-split demotion
New rule 4 in fuse_trimap_directional: a chroma-unknown pixel whose seg
confidence is >= trimap.seg_force_fg (default 0.98) stays sure-FG even when
background-hued. Bright skin on a warm same-hue-family background (peach,
pale yellow) was being demoted to unknown by the hue split and then lost --
either ViTMatte itself misfires there (raw 0.07-0.13 measured) or the
chroma suppressor does (raw 0.89 halved to 0.45 via the RGB-proximity
confidence path). The segmenter meanwhile rates those pixels a saturated
1.0; that semantic certainty now outranks same-hue colour suspicion.

Safety: chroma_unknown excludes sure background, so a real flat backdrop
can never be forced foreground; the silhouette band still re-opens the
boundary; the cross-check veto still overrides trimap-FG. Set > 1.0 to
disable.

Verified (GPU bf16): yellow-bg frame face 0.84->1.00, peach-bg thigh
0.87->1.00 (residual softness only where chroma is near-sure-bg, by
design). Pastel-blue TestImage3 regression: 0.047% of pixels differ with
cross-check on, 0.058% without it (hair-gap suppression path), both
visually negligible.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 13:49:31 +08:00
10 changed files with 172 additions and 51 deletions
+35 -14
View File
@@ -25,9 +25,12 @@ Two pipelines, selected by `segmentation.enabled` in the config:
`trust_remote_code`) or `anime-seg` (ONNX, tuned for anime characters). Switch at
runtime with `--seg-backend anime-seg` (it also selects the matching weights).
Both share pymatting foreground estimation and a colour de-spill. There is **no**
green-contamination rescue / recolour layer — with clean source images it is
unnecessary, so it was removed.
Both share pymatting foreground estimation and an optional colour de-spill
(**off by default**: de-spill has no positional/semantic guard, so a subject
sharing the background's hue — a blue suit on a blue backdrop — gets
desaturated and hue-shifted; enable with `--despill` when edge spill genuinely
matters). There is **no** green-contamination rescue / recolour layer — with
clean source images it is unnecessary, so it was removed.
Both also run a **cross-model veto** by default: a second, trimap-free matting
model (`ZhengPeng7/BiRefNet_HR-matting`) may only *lower* alpha, only on
@@ -38,15 +41,17 @@ download (~0.9 GB) and one inference pass per image; disable with
`--no-cross-check` (see the `cross_check` config section, and
`docs/hair_gap_artifacts.md` for the analysis behind it).
When the cross-check is on, that same HR-matting forward is **reused as the
segmentation mask by default**, skipping the primary seg model entirely (one
less model to load, ~20 s faster per image on CPU). Pilot-validated
(TestImage3 / FixImage1): trimap 99.8% identical, no structural change to
fingers, hair wisps or thin lines. Disable with `--no-cross-check-as-seg` to
run the dedicated seg model instead. The reuse is a same-family swap, so it
applies to the `birefnet` backend only: with `--seg-backend anime-seg` the
anime segmenter keeps its own forward and the cross-check veto still runs
independently on top.
When the cross-check is on, `--cross-check-as-seg` reuses that same HR-matting
forward as the segmentation mask, skipping the primary seg model entirely (one
less model to load, ~20 s faster per image on CPU). **Off by default** — use it
only when the subjects are characters or similar solid figures. HR-matting is
trained on portrait/animal matting (P3M-10k, AM-2k) and reads thin pale strokes
on a low-contrast background as semi-transparent wisps: white calligraphy on
the pastel backdrop (TestImage4) loses half its strokes under reuse, while the
dedicated DIS5K-trained segmenter keeps them solid. The reuse is a same-family
swap, so it applies to the `birefnet` backend only: with `--seg-backend
anime-seg` the anime segmenter keeps its own forward and the cross-check veto
still runs independently on top.
The segmentation trimap defaults to `directional` mode (chroma + seg + a hue-direction
split: it keeps a background-coloured garment such as a white shirt while dropping a
@@ -54,6 +59,21 @@ background-hued residual such as blue trapped between hair strands). Switch with
`--trimap-mode seg` (topology only, no hue split) or `directional-hard-bg` (aggressive
— hard-removes background-hued pixels; can eat cool/shadowed white cloth).
## Non-flat backgrounds
Pass `--no-chroma` (or `chroma.enabled: false`) to matte images whose background is
**not** one flat colour — gradients, textures, scenes. There is no colour key in
this mode: the segmentation mask alone drives the trimap (mode forced to `seg`),
ViTMatte still refines the unknown band at full resolution, and pymatting still
estimates edge foreground colour. The colour-keyed stages are bypassed: background
auto-detection, the directional hue split, chroma alpha suppression, and despill.
The cross-model veto still runs (disable with `--no-cross-check`): without a key
colour its suspect zone drops the hue test and instead only vetoes where the
second opinion is itself confidently near-empty (`cross_check.second_lo/hi`), so
thin strands the second model merely blurs are never eroded. Requires the
segmentation pipeline; quality then rests entirely on the segmenter's mask, so
expect flat-background results to stay stronger on hair-level detail.
## Background colour
By default (`screen_color: null`) the background colour is **auto-detected** from the
@@ -119,8 +139,9 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
## Performance (CPU reference numbers)
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, default settings
(cross-check on, reused as seg; chunked global attention on):
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, with cross-check on
and `--cross-check-as-seg` (reuse); chunked global attention on. Without reuse
(the default) add ~9 s fp32 / ~6 s bf16 for the dedicated seg forward:
| input | precision | warm / image | peak memory |
|---|---|---|---|
+13 -6
View File
@@ -59,7 +59,7 @@ def suppress_alpha_by_chroma(
def cross_check_alpha(
alpha: np.ndarray,
second_alpha: np.ndarray,
proj: np.ndarray,
proj: np.ndarray | None,
lightness: np.ndarray,
trimap: np.ndarray,
settings: CrossCheckSettings,
@@ -76,6 +76,13 @@ def cross_check_alpha(
- the primary alpha is high (``gate_lo -> gate_hi`` ramp): pixels the
pipeline already renders soft (outer wisps) are exempt by construction.
``proj`` is None in complex-background mode (chroma disabled): no key colour
means no hue-defined suspect zone, so instead the veto requires the second
opinion itself to be confidently near-empty (full strength at/below
``second_lo``, none at/above ``second_hi``). A thin strand the downsampled
second model merely blurs to mid-alpha is left untouched; only regions it
decisively rejects can be cleared.
Inside the zone this deliberately overrides the trimap-FG clamp -- the
residue it exists to clear is mostly trimap-FG. Sure background cannot be
disturbed: min-fusion keeps alpha 0 at 0.
@@ -83,16 +90,16 @@ def cross_check_alpha(
if not settings.enabled:
return alpha
cv2 = require_cv2()
zone = (
(proj >= settings.proj_min)
& (lightness >= settings.l_min)
& (trimap != 0)
)
zone = (lightness >= settings.l_min) & (trimap != 0)
if proj is not None:
zone &= proj >= settings.proj_min
weight = zone.astype(np.float32)
if settings.feather_sigma > 0:
blur = cv2.GaussianBlur(weight, (0, 0), settings.feather_sigma)
weight = np.where(zone, 1.0, np.clip(blur, 0.0, 1.0)).astype(np.float32)
gate = weight * _smoothstep(alpha, settings.gate_lo, settings.gate_hi)
if proj is None:
gate = gate * (1.0 - _smoothstep(second_alpha, settings.second_lo, settings.second_hi))
out = alpha * (1.0 - gate) + np.minimum(alpha, second_alpha) * gate
return np.clip(out, 0.0, 1.0).astype(np.float32)
+3 -1
View File
@@ -23,6 +23,7 @@ def main(
device: str | None = typer.Option(None, "--device"),
precision: str | None = typer.Option(None, "--precision", help="Compute precision for all models (ViTMatte cast + BiRefNet autocast): fp32 (default) | bf16 (faster + halves matting activation memory; needs bf16-capable hardware, else falls back to fp32)"),
screen_color: str | None = typer.Option(None, "--screen-color", help="Background colour prior as #RRGGBB (default: auto-detect the flat background colour)"),
chroma: bool | None = typer.Option(None, "--chroma/--no-chroma", help="Colour-key the flat background (default: on). --no-chroma = non-flat background mode: segmentation alone drives the trimap; auto-detect, hue split, chroma suppression, despill and the cross-check veto are bypassed"),
matting_method: str | None = typer.Option(None, "--matting-method"),
fallback_to_chroma_alpha: bool | None = typer.Option(None, "--fallback-to-chroma-alpha/--no-fallback-to-chroma-alpha"),
sure_bg_threshold: float | None = typer.Option(None, "--sure-bg-threshold", min=0.0, max=1.0),
@@ -31,7 +32,7 @@ def main(
fg_safe_radius_ratio: float | None = typer.Option(None, "--fg-safe-radius-ratio", min=0.0),
despill: bool | None = typer.Option(None, "--despill/--no-despill"),
cross_check: bool | None = typer.Option(None, "--cross-check/--no-cross-check", help="Second-opinion veto of background-hued residue between hair strands (default: on; costs one extra model inference)"),
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: on; saves ~20s on CPU; needs cross-check on)"),
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: off; saves ~20s on CPU; needs cross-check on)"),
trimap_mode: str | None = typer.Option(None, "--trimap-mode", help="Trimap mode (segmentation pipeline): directional | seg | directional-hard-bg"),
seg_backend: str | None = typer.Option(None, "--seg-backend", help="Segmentation backend: birefnet (default) | anime-seg"),
) -> None:
@@ -42,6 +43,7 @@ def main(
device=device,
precision=precision,
screen_color=screen_color,
chroma=chroma,
matting_method=matting_method,
fallback_to_chroma_alpha=fallback_to_chroma_alpha,
sure_bg_threshold=sure_bg_threshold,
+4 -1
View File
@@ -99,6 +99,9 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
}
if overrides.get("trimap_mode") is not None:
trimap_updates["mode"] = overrides["trimap_mode"]
chroma_updates: dict[str, Any] = {}
if overrides.get("chroma") is not None:
chroma_updates["enabled"] = overrides["chroma"]
despill_updates: dict[str, Any] = {}
if overrides.get("despill") is not None:
despill_updates["enabled"] = overrides["despill"]
@@ -130,7 +133,7 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
else settings.screen_color
)
return PipelineSettings(
chroma=chroma,
chroma=_update_dataclass(chroma, chroma_updates),
trimap=_update_dataclass(trimap, trimap_updates),
alpha_post=alpha_post,
cross_check=_update_dataclass(settings.cross_check, cross_check_updates),
+3 -2
View File
@@ -10,7 +10,7 @@ from .settings import DespillSettings
def despill(
rgb: np.ndarray,
alpha: np.ndarray,
model: BackgroundModel,
model: BackgroundModel | None,
settings: DespillSettings,
) -> tuple[np.ndarray, np.ndarray]:
"""Remove background-colour spill from the foreground.
@@ -24,7 +24,8 @@ def despill(
Returns the corrected RGB (uint8) and the per-pixel despill weight (float32).
"""
if not settings.enabled:
# model is None with chroma disabled: no key colour means no spill direction.
if not settings.enabled or model is None:
return rgb.copy(), np.zeros(alpha.shape, dtype=np.float32)
bg_lab = np.asarray(model.lab_center, dtype=np.float32)
+7 -2
View File
@@ -11,7 +11,7 @@ def estimate_foreground_rgb(
rgb: np.ndarray,
alpha: np.ndarray,
bg_confidence: np.ndarray,
model: BackgroundModel,
model: BackgroundModel | None,
settings: ForegroundSettings,
) -> np.ndarray:
"""Estimated foreground colour F (uint8 HxWx3) for compositing over alpha."""
@@ -23,10 +23,15 @@ def estimate_foreground_rgb(
try:
return _estimate_ml(rgb, alpha, settings)
except RuntimeError:
if not settings.fallback_to_unmix:
if not settings.fallback_to_unmix or model is None:
raise
method = "unmix"
if method == "unmix":
if model is None:
raise RuntimeError(
"Foreground method 'unmix' needs the chroma background model, which "
"does not exist with chroma disabled; use method 'ml'."
)
return _estimate_unmix(rgb, alpha, bg_confidence, model, settings)
raise RuntimeError(
f"Unsupported foreground method '{settings.method}'. Use 'ml' or 'unmix'."
+34 -13
View File
@@ -132,10 +132,26 @@ def run_image(
def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
settings = pipeline.settings
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
bg_confidence, model = compute_bg_confidence(
rgb, settings=settings.chroma, screen_color=screen_color
)
if settings.chroma.enabled:
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
bg_confidence, model = compute_bg_confidence(
rgb, settings=settings.chroma, screen_color=screen_color
)
else:
if not settings.segmentation.enabled:
raise RuntimeError(
"chroma.enabled: false (non-flat background mode) needs the segmentation "
"pipeline; enable segmentation or re-enable chroma."
)
if settings.model.matting_method == "chroma":
raise RuntimeError(
"matting_method 'chroma' needs the chroma key; use 'vitmatte' or re-enable chroma."
)
# Non-flat background: no colour key exists. A zero background-confidence
# map makes every chroma-fused formula degrade to its seg-only form; the
# colour-keyed stages (hue split, suppression, veto, despill) are skipped.
bg_confidence = np.zeros(rgb.shape[:2], dtype=np.float32)
model = None
second_alpha = None
if settings.segmentation.enabled:
reuse = (
@@ -154,7 +170,9 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
seg_mask = second_alpha
else:
seg_mask = pipeline._segment(rgb)
mode = settings.trimap.mode
# The directional modes are colour tests against the key colour; without
# one, the seg-topology trimap is the only meaningful choice.
mode = settings.trimap.mode if settings.chroma.enabled else "seg"
if mode == "seg":
trimap, trimap_stats = fuse_trimap(seg_mask, bg_confidence, settings.trimap)
elif mode in ("directional", "directional-hard-bg"):
@@ -173,12 +191,13 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
raw_alpha, alpha_source = pipeline._predict_alpha(rgb, trimap, bg_confidence)
alpha = enforce_trimap(raw_alpha, trimap)
alpha = suppress_alpha_by_chroma(
alpha, bg_confidence, trimap, settings.alpha_post,
# The matte-confidence gate only makes sense for a real matting prediction;
# a chroma-seeded alpha is itself colour evidence, so no gate there.
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
)
if settings.chroma.enabled:
alpha = suppress_alpha_by_chroma(
alpha, bg_confidence, trimap, settings.alpha_post,
# The matte-confidence gate only makes sense for a real matting prediction;
# a chroma-seeded alpha is itself colour evidence, so no gate there.
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
)
alpha = clean_alpha(alpha, trimap, settings.alpha_post)
if settings.cross_check.enabled:
lab = convert_color_spaces(rgb)[1]
@@ -187,7 +206,9 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
alpha = cross_check_alpha(
alpha,
second_alpha,
bg_hue_projection(lab, model.lab_center),
# No key colour in complex-background mode: proj=None switches the
# veto to its second-opinion-confidence gate (see cross_check_alpha).
bg_hue_projection(lab, model.lab_center) if model is not None else None,
lab[..., 0],
trimap,
settings.cross_check,
@@ -199,7 +220,7 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
metadata = {
"alpha_source": alpha_source,
"background_model": model.to_dict(),
"background_model": model.to_dict() if model is not None else None,
"trimap": trimap_stats,
"settings": asdict(settings),
}
+36 -7
View File
@@ -5,6 +5,14 @@ from dataclasses import dataclass
@dataclass(frozen=True)
class ChromaSettings:
# False = non-flat-background mode (--no-chroma): no colour key at all.
# Segmentation alone drives the trimap (mode forced to "seg"), and the
# colour-keyed stages are bypassed: background auto-detect, the directional
# hue split, chroma alpha suppression, despill, and the unmix foreground
# fallback. The cross-check veto still runs (if enabled) with a hue-free
# gate -- see CrossCheckSettings.second_lo/hi. Requires
# segmentation.enabled and matting_method 'vitmatte'.
enabled: bool = True
border_ratio: float = 0.04
min_samples: int = 2048
lab_sigma_min: float = 10.0
@@ -40,6 +48,14 @@ class TrimapSettings:
mode: str = "directional"
seg_low: float = 0.15 # seg below this -> background-eligible; at/above -> foreground-eligible
bg_hue_proj_min: float = 4.0 # Lab a*/b* projection onto bg direction above which a pixel is background-hued
# Semantic override for the directional hue split: a chroma-unknown pixel whose
# seg confidence is at/above this is kept sure-FG even when background-hued.
# A saturated seg signal outranks colour evidence that only says "same hue
# family as the background" (bright skin on a warm background, measured raw
# ViTMatte failures at proj ~7-10). Chroma sure-background is never overridden,
# and the silhouette band still re-opens the boundary for anti-aliasing.
# Set > 1.0 to disable. (seg masks are uint8-quantised: 0.98 = 250/255.)
seg_force_fg: float = 0.98
@dataclass(frozen=True)
@@ -76,12 +92,13 @@ class CrossCheckSettings:
input_size: int = 2048
# Reuse the cross-check forward as the segmentation mask: one HR-matting
# inference serves both trimap topology and the second opinion, skipping the
# primary seg model entirely (saves its load + forward). Pilot-validated on
# TestImage3/FixImage1: trimap 99.8% identical, no structural change. Only
# applies to the birefnet segmentation backend (a same-family swap); with
# anime-seg the chosen segmenter keeps its own forward and the cross-check
# runs independently. Needs cross_check and segmentation both enabled.
reuse_as_seg: bool = True
# primary seg model entirely (saves its load + forward; measured ~20 s/image
# on CPU, ~0.9-1.3 GB VRAM on GPU). Pilot-validated on TestImage3/FixImage1:
# trimap 99.8% identical. Default OFF on server-edition: the dedicated
# segmenter keeps its own forward and the veto stays an independent signal;
# opt in with --cross-check-as-seg. Only applies to the birefnet backend
# (a same-family swap); needs cross_check and segmentation both enabled.
reuse_as_seg: bool = False
# Compute precision of the cross-check forward: "fp32" or "bf16" (autocast;
# needs bf16-capable hardware, else falls back to fp32 with a warning).
precision: str = "fp32"
@@ -90,6 +107,13 @@ class CrossCheckSettings:
feather_sigma: float = 2.0 # Gaussian feather of the zone boundary, in px
gate_lo: float = 0.70 # primary alpha below this -> fully exempt
gate_hi: float = 0.95 # primary alpha above this -> fully vetoable
# Complex-background mode only (chroma disabled -> no hue-defined suspect
# zone): the veto instead requires the second opinion itself to be
# confidently near-empty -- full strength at/below second_lo, none at/above
# second_hi. Thin structures the downsampled second model merely blurs to
# mid-alpha stay untouched; only decisively-rejected regions can be cleared.
second_lo: float = 0.15
second_hi: float = 0.40
@dataclass(frozen=True)
@@ -113,7 +137,12 @@ class ForegroundSettings:
@dataclass(frozen=True)
class DespillSettings:
enabled: bool = True
# Off by default: despill pulls foreground chroma along the background-hue axis
# with no positional or semantic guard, so a subject sharing the background's hue
# (e.g. a blue suit on a blue backdrop) is desaturated/hue-shifted (~ΔE 35, blue
# -> purple). This is the root default the cpu-fast lane inherits. Re-enable
# per-config or with --despill only when edge spill genuinely matters.
enabled: bool = False
edge_low: float = 0.005
strength: float = 0.92
edge_expand_radius: int = 2
+10 -2
View File
@@ -145,14 +145,21 @@ def fuse_trimap_directional(
bg = chroma_bg | ((seg_mask < settings.seg_low) & ~chroma_fg)
fg = chroma_fg & (seg_mask >= settings.seg_low)
rule3_fg = chroma_unknown & core & ~bg_hued
# Rule 4 -- semantic override: a chroma-unknown pixel the segmenter is
# (near-)certain about stays foreground even when background-hued. Colour
# evidence here only says "same hue family as the background" (bright skin
# on a warm background), which is exactly where ViTMatte and the suppressor
# misfire; a saturated seg signal outranks it. chroma_unknown excludes sure
# background, so a real flat-colour backdrop can never be forced foreground.
force_fg = chroma_unknown & (seg_mask >= settings.seg_force_fg)
if settings.mode == "directional-hard-bg":
# Aggressive variant: hard-remove background-hued pixels instead of leaving
# them unknown. Can eat cool/shadowed white cloth, so it is not the default.
bg = bg | (chroma_unknown & core & bg_hued)
bg = bg | (chroma_unknown & core & bg_hued & ~force_fg)
trimap = np.full(shape, 128, dtype=np.uint8)
trimap[bg] = 0
trimap[fg | rule3_fg] = 255
trimap[fg | rule3_fg | force_fg] = 255
loose = (seg_mask >= settings.seg_loose_threshold).astype(np.uint8)
kernel = elliptical_kernel(band_radius)
@@ -165,6 +172,7 @@ def fuse_trimap_directional(
"sure_fg_pixels": int((trimap == 255).sum()),
"band_radius": int(band_radius),
"rule3_fg_pixels": int(rule3_fg.sum()),
"force_fg_pixels": int((force_fg & ~fg & ~rule3_fg).sum()),
"bg_hued_pixels": int((bg_hued & chroma_unknown & core).sum()),
}
return trimap, stats
+27 -3
View File
@@ -3,6 +3,14 @@
# like "#CFEFFF" sets it explicitly. Also overridable via --screen-color.
screen_color: null
chroma:
# false = non-flat background mode (--no-chroma): no colour key; segmentation
# alone drives the trimap (mode forced to "seg") and the colour-keyed stages
# are bypassed (auto-detect, hue split, chroma suppression, despill). The
# cross-check veto still runs if enabled, using its hue-free gate (see
# cross_check.second_lo/hi). Needs segmentation.enabled + matting vitmatte.
enabled: true
model:
model_name: hustvl/vitmatte-base-composition-1k
device: cpu
@@ -46,6 +54,12 @@ trimap:
mode: directional
seg_low: 0.15
bg_hue_proj_min: 4.0
# Semantic override (directional mode): chroma-unknown pixels with seg confidence
# at/above this stay sure-FG even when background-hued — rescues bright skin on a
# warm (same-hue-family) background that the hue split would otherwise hand to
# ViTMatte/suppression, which misfire exactly there. Never overrides chroma
# sure-background. Set > 1.0 to disable.
seg_force_fg: 0.98
alpha_post:
chroma_suppress: true
@@ -74,8 +88,9 @@ cross_check:
# trimap 99.8% identical, no structural change (fingers/wisps/bowstring OK).
# Only applies to the birefnet backend (same-family swap); anime-seg keeps
# its own forward, with the cross-check running independently on top.
# Disable with --no-cross-check-as-seg. No effect when cross_check is off.
reuse_as_seg: true
# Default OFF (server-edition): the dedicated segmenter keeps its own forward
# and the veto stays independent; enable with --cross-check-as-seg.
reuse_as_seg: false
# Cross-check forward precision, same semantics as segmentation.precision
# (bf16 via autocast: 2048 pass ~35s -> ~26s on a native-bf16 CPU).
precision: fp32
@@ -84,6 +99,12 @@ cross_check:
feather_sigma: 2.0 # zone-boundary feather, px
gate_lo: 0.70 # primary alpha below this -> fully exempt
gate_hi: 0.95 # primary alpha above this -> fully vetoable
# Complex-background (no-chroma) mode only: no hue-defined suspect zone, so
# the veto requires the second opinion itself to be confidently near-empty --
# full strength at/below second_lo, fading to none at/above second_hi. Thin
# strands the 2048-input second model merely blurs stay untouched.
second_lo: 0.15
second_hi: 0.40
foreground:
enabled: true
@@ -103,7 +124,10 @@ foreground:
bg_confidence_weight: 0.70
despill:
enabled: true
# Off: despill has no positional/semantic guard, so a subject sharing the
# background's hue (blue suit on a blue backdrop) gets desaturated/hue-shifted
# (~ΔE 35, blue -> purple). Re-enable per-job only when edge spill matters.
enabled: false
edge_low: 0.005
strength: 0.92
edge_expand_radius: 2