Compare commits
4 Commits
server-edition
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
| 2b3b62ed3f | |||
| e61fbe84f0 | |||
| f22af16517 | |||
| 18c6918e45 |
@@ -25,9 +25,12 @@ Two pipelines, selected by `segmentation.enabled` in the config:
|
||||
`trust_remote_code`) or `anime-seg` (ONNX, tuned for anime characters). Switch at
|
||||
runtime with `--seg-backend anime-seg` (it also selects the matching weights).
|
||||
|
||||
Both share pymatting foreground estimation and a colour de-spill. There is **no**
|
||||
green-contamination rescue / recolour layer — with clean source images it is
|
||||
unnecessary, so it was removed.
|
||||
Both share pymatting foreground estimation and an optional colour de-spill
|
||||
(**off by default**: de-spill has no positional/semantic guard, so a subject
|
||||
sharing the background's hue — a blue suit on a blue backdrop — gets
|
||||
desaturated and hue-shifted; enable with `--despill` when edge spill genuinely
|
||||
matters). There is **no** green-contamination rescue / recolour layer — with
|
||||
clean source images it is unnecessary, so it was removed.
|
||||
|
||||
Both also run a **cross-model veto** by default: a second, trimap-free matting
|
||||
model (`ZhengPeng7/BiRefNet_HR-matting`) may only *lower* alpha, only on
|
||||
@@ -38,15 +41,17 @@ download (~0.9 GB) and one inference pass per image; disable with
|
||||
`--no-cross-check` (see the `cross_check` config section, and
|
||||
`docs/hair_gap_artifacts.md` for the analysis behind it).
|
||||
|
||||
When the cross-check is on, that same HR-matting forward is **reused as the
|
||||
segmentation mask by default**, skipping the primary seg model entirely (one
|
||||
less model to load, ~20 s faster per image on CPU). Pilot-validated
|
||||
(TestImage3 / FixImage1): trimap 99.8% identical, no structural change to
|
||||
fingers, hair wisps or thin lines. Disable with `--no-cross-check-as-seg` to
|
||||
run the dedicated seg model instead. The reuse is a same-family swap, so it
|
||||
applies to the `birefnet` backend only: with `--seg-backend anime-seg` the
|
||||
anime segmenter keeps its own forward and the cross-check veto still runs
|
||||
independently on top.
|
||||
When the cross-check is on, `--cross-check-as-seg` reuses that same HR-matting
|
||||
forward as the segmentation mask, skipping the primary seg model entirely (one
|
||||
less model to load, ~20 s faster per image on CPU). **Off by default** — use it
|
||||
only when the subjects are characters or similar solid figures. HR-matting is
|
||||
trained on portrait/animal matting (P3M-10k, AM-2k) and reads thin pale strokes
|
||||
on a low-contrast background as semi-transparent wisps: white calligraphy on
|
||||
the pastel backdrop (TestImage4) loses half its strokes under reuse, while the
|
||||
dedicated DIS5K-trained segmenter keeps them solid. The reuse is a same-family
|
||||
swap, so it applies to the `birefnet` backend only: with `--seg-backend
|
||||
anime-seg` the anime segmenter keeps its own forward and the cross-check veto
|
||||
still runs independently on top.
|
||||
|
||||
The segmentation trimap defaults to `directional` mode (chroma + seg + a hue-direction
|
||||
split: it keeps a background-coloured garment such as a white shirt while dropping a
|
||||
@@ -54,6 +59,21 @@ background-hued residual such as blue trapped between hair strands). Switch with
|
||||
`--trimap-mode seg` (topology only, no hue split) or `directional-hard-bg` (aggressive
|
||||
— hard-removes background-hued pixels; can eat cool/shadowed white cloth).
|
||||
|
||||
## Non-flat backgrounds
|
||||
|
||||
Pass `--no-chroma` (or `chroma.enabled: false`) to matte images whose background is
|
||||
**not** one flat colour — gradients, textures, scenes. There is no colour key in
|
||||
this mode: the segmentation mask alone drives the trimap (mode forced to `seg`),
|
||||
ViTMatte still refines the unknown band at full resolution, and pymatting still
|
||||
estimates edge foreground colour. The colour-keyed stages are bypassed: background
|
||||
auto-detection, the directional hue split, chroma alpha suppression, and despill.
|
||||
The cross-model veto still runs (disable with `--no-cross-check`): without a key
|
||||
colour its suspect zone drops the hue test and instead only vetoes where the
|
||||
second opinion is itself confidently near-empty (`cross_check.second_lo/hi`), so
|
||||
thin strands the second model merely blurs are never eroded. Requires the
|
||||
segmentation pipeline; quality then rests entirely on the segmenter's mask, so
|
||||
expect flat-background results to stay stronger on hair-level detail.
|
||||
|
||||
## Background colour
|
||||
|
||||
By default (`screen_color: null`) the background colour is **auto-detected** from the
|
||||
@@ -119,8 +139,9 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
|
||||
## Performance (CPU reference numbers)
|
||||
|
||||
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, default settings
|
||||
(cross-check on, reused as seg; chunked global attention on):
|
||||
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, with cross-check on
|
||||
and `--cross-check-as-seg` (reuse); chunked global attention on. Without reuse
|
||||
(the default) add ~9 s fp32 / ~6 s bf16 for the dedicated seg forward:
|
||||
|
||||
| input | precision | warm / image | peak memory |
|
||||
|---|---|---|---|
|
||||
|
||||
+13
-6
@@ -59,7 +59,7 @@ def suppress_alpha_by_chroma(
|
||||
def cross_check_alpha(
|
||||
alpha: np.ndarray,
|
||||
second_alpha: np.ndarray,
|
||||
proj: np.ndarray,
|
||||
proj: np.ndarray | None,
|
||||
lightness: np.ndarray,
|
||||
trimap: np.ndarray,
|
||||
settings: CrossCheckSettings,
|
||||
@@ -76,6 +76,13 @@ def cross_check_alpha(
|
||||
- the primary alpha is high (``gate_lo -> gate_hi`` ramp): pixels the
|
||||
pipeline already renders soft (outer wisps) are exempt by construction.
|
||||
|
||||
``proj`` is None in complex-background mode (chroma disabled): no key colour
|
||||
means no hue-defined suspect zone, so instead the veto requires the second
|
||||
opinion itself to be confidently near-empty (full strength at/below
|
||||
``second_lo``, none at/above ``second_hi``). A thin strand the downsampled
|
||||
second model merely blurs to mid-alpha is left untouched; only regions it
|
||||
decisively rejects can be cleared.
|
||||
|
||||
Inside the zone this deliberately overrides the trimap-FG clamp -- the
|
||||
residue it exists to clear is mostly trimap-FG. Sure background cannot be
|
||||
disturbed: min-fusion keeps alpha 0 at 0.
|
||||
@@ -83,16 +90,16 @@ def cross_check_alpha(
|
||||
if not settings.enabled:
|
||||
return alpha
|
||||
cv2 = require_cv2()
|
||||
zone = (
|
||||
(proj >= settings.proj_min)
|
||||
& (lightness >= settings.l_min)
|
||||
& (trimap != 0)
|
||||
)
|
||||
zone = (lightness >= settings.l_min) & (trimap != 0)
|
||||
if proj is not None:
|
||||
zone &= proj >= settings.proj_min
|
||||
weight = zone.astype(np.float32)
|
||||
if settings.feather_sigma > 0:
|
||||
blur = cv2.GaussianBlur(weight, (0, 0), settings.feather_sigma)
|
||||
weight = np.where(zone, 1.0, np.clip(blur, 0.0, 1.0)).astype(np.float32)
|
||||
gate = weight * _smoothstep(alpha, settings.gate_lo, settings.gate_hi)
|
||||
if proj is None:
|
||||
gate = gate * (1.0 - _smoothstep(second_alpha, settings.second_lo, settings.second_hi))
|
||||
out = alpha * (1.0 - gate) + np.minimum(alpha, second_alpha) * gate
|
||||
return np.clip(out, 0.0, 1.0).astype(np.float32)
|
||||
|
||||
|
||||
+3
-1
@@ -23,6 +23,7 @@ def main(
|
||||
device: str | None = typer.Option(None, "--device"),
|
||||
precision: str | None = typer.Option(None, "--precision", help="Compute precision for all models (ViTMatte cast + BiRefNet autocast): fp32 (default) | bf16 (faster + halves matting activation memory; needs bf16-capable hardware, else falls back to fp32)"),
|
||||
screen_color: str | None = typer.Option(None, "--screen-color", help="Background colour prior as #RRGGBB (default: auto-detect the flat background colour)"),
|
||||
chroma: bool | None = typer.Option(None, "--chroma/--no-chroma", help="Colour-key the flat background (default: on). --no-chroma = non-flat background mode: segmentation alone drives the trimap; auto-detect, hue split, chroma suppression, despill and the cross-check veto are bypassed"),
|
||||
matting_method: str | None = typer.Option(None, "--matting-method"),
|
||||
fallback_to_chroma_alpha: bool | None = typer.Option(None, "--fallback-to-chroma-alpha/--no-fallback-to-chroma-alpha"),
|
||||
sure_bg_threshold: float | None = typer.Option(None, "--sure-bg-threshold", min=0.0, max=1.0),
|
||||
@@ -31,7 +32,7 @@ def main(
|
||||
fg_safe_radius_ratio: float | None = typer.Option(None, "--fg-safe-radius-ratio", min=0.0),
|
||||
despill: bool | None = typer.Option(None, "--despill/--no-despill"),
|
||||
cross_check: bool | None = typer.Option(None, "--cross-check/--no-cross-check", help="Second-opinion veto of background-hued residue between hair strands (default: on; costs one extra model inference)"),
|
||||
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: on; saves ~20s on CPU; needs cross-check on)"),
|
||||
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: off; saves ~20s on CPU; needs cross-check on)"),
|
||||
trimap_mode: str | None = typer.Option(None, "--trimap-mode", help="Trimap mode (segmentation pipeline): directional | seg | directional-hard-bg"),
|
||||
seg_backend: str | None = typer.Option(None, "--seg-backend", help="Segmentation backend: birefnet (default) | anime-seg"),
|
||||
) -> None:
|
||||
@@ -42,6 +43,7 @@ def main(
|
||||
device=device,
|
||||
precision=precision,
|
||||
screen_color=screen_color,
|
||||
chroma=chroma,
|
||||
matting_method=matting_method,
|
||||
fallback_to_chroma_alpha=fallback_to_chroma_alpha,
|
||||
sure_bg_threshold=sure_bg_threshold,
|
||||
|
||||
+4
-1
@@ -99,6 +99,9 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
|
||||
}
|
||||
if overrides.get("trimap_mode") is not None:
|
||||
trimap_updates["mode"] = overrides["trimap_mode"]
|
||||
chroma_updates: dict[str, Any] = {}
|
||||
if overrides.get("chroma") is not None:
|
||||
chroma_updates["enabled"] = overrides["chroma"]
|
||||
despill_updates: dict[str, Any] = {}
|
||||
if overrides.get("despill") is not None:
|
||||
despill_updates["enabled"] = overrides["despill"]
|
||||
@@ -130,7 +133,7 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
|
||||
else settings.screen_color
|
||||
)
|
||||
return PipelineSettings(
|
||||
chroma=chroma,
|
||||
chroma=_update_dataclass(chroma, chroma_updates),
|
||||
trimap=_update_dataclass(trimap, trimap_updates),
|
||||
alpha_post=alpha_post,
|
||||
cross_check=_update_dataclass(settings.cross_check, cross_check_updates),
|
||||
|
||||
+3
-2
@@ -10,7 +10,7 @@ from .settings import DespillSettings
|
||||
def despill(
|
||||
rgb: np.ndarray,
|
||||
alpha: np.ndarray,
|
||||
model: BackgroundModel,
|
||||
model: BackgroundModel | None,
|
||||
settings: DespillSettings,
|
||||
) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Remove background-colour spill from the foreground.
|
||||
@@ -24,7 +24,8 @@ def despill(
|
||||
|
||||
Returns the corrected RGB (uint8) and the per-pixel despill weight (float32).
|
||||
"""
|
||||
if not settings.enabled:
|
||||
# model is None with chroma disabled: no key colour means no spill direction.
|
||||
if not settings.enabled or model is None:
|
||||
return rgb.copy(), np.zeros(alpha.shape, dtype=np.float32)
|
||||
|
||||
bg_lab = np.asarray(model.lab_center, dtype=np.float32)
|
||||
|
||||
@@ -11,7 +11,7 @@ def estimate_foreground_rgb(
|
||||
rgb: np.ndarray,
|
||||
alpha: np.ndarray,
|
||||
bg_confidence: np.ndarray,
|
||||
model: BackgroundModel,
|
||||
model: BackgroundModel | None,
|
||||
settings: ForegroundSettings,
|
||||
) -> np.ndarray:
|
||||
"""Estimated foreground colour F (uint8 HxWx3) for compositing over alpha."""
|
||||
@@ -23,10 +23,15 @@ def estimate_foreground_rgb(
|
||||
try:
|
||||
return _estimate_ml(rgb, alpha, settings)
|
||||
except RuntimeError:
|
||||
if not settings.fallback_to_unmix:
|
||||
if not settings.fallback_to_unmix or model is None:
|
||||
raise
|
||||
method = "unmix"
|
||||
if method == "unmix":
|
||||
if model is None:
|
||||
raise RuntimeError(
|
||||
"Foreground method 'unmix' needs the chroma background model, which "
|
||||
"does not exist with chroma disabled; use method 'ml'."
|
||||
)
|
||||
return _estimate_unmix(rgb, alpha, bg_confidence, model, settings)
|
||||
raise RuntimeError(
|
||||
f"Unsupported foreground method '{settings.method}'. Use 'ml' or 'unmix'."
|
||||
|
||||
+34
-13
@@ -132,10 +132,26 @@ def run_image(
|
||||
|
||||
def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
settings = pipeline.settings
|
||||
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
|
||||
bg_confidence, model = compute_bg_confidence(
|
||||
rgb, settings=settings.chroma, screen_color=screen_color
|
||||
)
|
||||
if settings.chroma.enabled:
|
||||
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
|
||||
bg_confidence, model = compute_bg_confidence(
|
||||
rgb, settings=settings.chroma, screen_color=screen_color
|
||||
)
|
||||
else:
|
||||
if not settings.segmentation.enabled:
|
||||
raise RuntimeError(
|
||||
"chroma.enabled: false (non-flat background mode) needs the segmentation "
|
||||
"pipeline; enable segmentation or re-enable chroma."
|
||||
)
|
||||
if settings.model.matting_method == "chroma":
|
||||
raise RuntimeError(
|
||||
"matting_method 'chroma' needs the chroma key; use 'vitmatte' or re-enable chroma."
|
||||
)
|
||||
# Non-flat background: no colour key exists. A zero background-confidence
|
||||
# map makes every chroma-fused formula degrade to its seg-only form; the
|
||||
# colour-keyed stages (hue split, suppression, veto, despill) are skipped.
|
||||
bg_confidence = np.zeros(rgb.shape[:2], dtype=np.float32)
|
||||
model = None
|
||||
second_alpha = None
|
||||
if settings.segmentation.enabled:
|
||||
reuse = (
|
||||
@@ -154,7 +170,9 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
seg_mask = second_alpha
|
||||
else:
|
||||
seg_mask = pipeline._segment(rgb)
|
||||
mode = settings.trimap.mode
|
||||
# The directional modes are colour tests against the key colour; without
|
||||
# one, the seg-topology trimap is the only meaningful choice.
|
||||
mode = settings.trimap.mode if settings.chroma.enabled else "seg"
|
||||
if mode == "seg":
|
||||
trimap, trimap_stats = fuse_trimap(seg_mask, bg_confidence, settings.trimap)
|
||||
elif mode in ("directional", "directional-hard-bg"):
|
||||
@@ -173,12 +191,13 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
raw_alpha, alpha_source = pipeline._predict_alpha(rgb, trimap, bg_confidence)
|
||||
|
||||
alpha = enforce_trimap(raw_alpha, trimap)
|
||||
alpha = suppress_alpha_by_chroma(
|
||||
alpha, bg_confidence, trimap, settings.alpha_post,
|
||||
# The matte-confidence gate only makes sense for a real matting prediction;
|
||||
# a chroma-seeded alpha is itself colour evidence, so no gate there.
|
||||
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
|
||||
)
|
||||
if settings.chroma.enabled:
|
||||
alpha = suppress_alpha_by_chroma(
|
||||
alpha, bg_confidence, trimap, settings.alpha_post,
|
||||
# The matte-confidence gate only makes sense for a real matting prediction;
|
||||
# a chroma-seeded alpha is itself colour evidence, so no gate there.
|
||||
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
|
||||
)
|
||||
alpha = clean_alpha(alpha, trimap, settings.alpha_post)
|
||||
if settings.cross_check.enabled:
|
||||
lab = convert_color_spaces(rgb)[1]
|
||||
@@ -187,7 +206,9 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
alpha = cross_check_alpha(
|
||||
alpha,
|
||||
second_alpha,
|
||||
bg_hue_projection(lab, model.lab_center),
|
||||
# No key colour in complex-background mode: proj=None switches the
|
||||
# veto to its second-opinion-confidence gate (see cross_check_alpha).
|
||||
bg_hue_projection(lab, model.lab_center) if model is not None else None,
|
||||
lab[..., 0],
|
||||
trimap,
|
||||
settings.cross_check,
|
||||
@@ -199,7 +220,7 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
|
||||
|
||||
metadata = {
|
||||
"alpha_source": alpha_source,
|
||||
"background_model": model.to_dict(),
|
||||
"background_model": model.to_dict() if model is not None else None,
|
||||
"trimap": trimap_stats,
|
||||
"settings": asdict(settings),
|
||||
}
|
||||
|
||||
+36
-7
@@ -5,6 +5,14 @@ from dataclasses import dataclass
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ChromaSettings:
|
||||
# False = non-flat-background mode (--no-chroma): no colour key at all.
|
||||
# Segmentation alone drives the trimap (mode forced to "seg"), and the
|
||||
# colour-keyed stages are bypassed: background auto-detect, the directional
|
||||
# hue split, chroma alpha suppression, despill, and the unmix foreground
|
||||
# fallback. The cross-check veto still runs (if enabled) with a hue-free
|
||||
# gate -- see CrossCheckSettings.second_lo/hi. Requires
|
||||
# segmentation.enabled and matting_method 'vitmatte'.
|
||||
enabled: bool = True
|
||||
border_ratio: float = 0.04
|
||||
min_samples: int = 2048
|
||||
lab_sigma_min: float = 10.0
|
||||
@@ -40,6 +48,14 @@ class TrimapSettings:
|
||||
mode: str = "directional"
|
||||
seg_low: float = 0.15 # seg below this -> background-eligible; at/above -> foreground-eligible
|
||||
bg_hue_proj_min: float = 4.0 # Lab a*/b* projection onto bg direction above which a pixel is background-hued
|
||||
# Semantic override for the directional hue split: a chroma-unknown pixel whose
|
||||
# seg confidence is at/above this is kept sure-FG even when background-hued.
|
||||
# A saturated seg signal outranks colour evidence that only says "same hue
|
||||
# family as the background" (bright skin on a warm background, measured raw
|
||||
# ViTMatte failures at proj ~7-10). Chroma sure-background is never overridden,
|
||||
# and the silhouette band still re-opens the boundary for anti-aliasing.
|
||||
# Set > 1.0 to disable. (seg masks are uint8-quantised: 0.98 = 250/255.)
|
||||
seg_force_fg: float = 0.98
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -76,12 +92,13 @@ class CrossCheckSettings:
|
||||
input_size: int = 2048
|
||||
# Reuse the cross-check forward as the segmentation mask: one HR-matting
|
||||
# inference serves both trimap topology and the second opinion, skipping the
|
||||
# primary seg model entirely (saves its load + forward). Pilot-validated on
|
||||
# TestImage3/FixImage1: trimap 99.8% identical, no structural change. Only
|
||||
# applies to the birefnet segmentation backend (a same-family swap); with
|
||||
# anime-seg the chosen segmenter keeps its own forward and the cross-check
|
||||
# runs independently. Needs cross_check and segmentation both enabled.
|
||||
reuse_as_seg: bool = True
|
||||
# primary seg model entirely (saves its load + forward; measured ~20 s/image
|
||||
# on CPU, ~0.9-1.3 GB VRAM on GPU). Pilot-validated on TestImage3/FixImage1:
|
||||
# trimap 99.8% identical. Default OFF on server-edition: the dedicated
|
||||
# segmenter keeps its own forward and the veto stays an independent signal;
|
||||
# opt in with --cross-check-as-seg. Only applies to the birefnet backend
|
||||
# (a same-family swap); needs cross_check and segmentation both enabled.
|
||||
reuse_as_seg: bool = False
|
||||
# Compute precision of the cross-check forward: "fp32" or "bf16" (autocast;
|
||||
# needs bf16-capable hardware, else falls back to fp32 with a warning).
|
||||
precision: str = "fp32"
|
||||
@@ -90,6 +107,13 @@ class CrossCheckSettings:
|
||||
feather_sigma: float = 2.0 # Gaussian feather of the zone boundary, in px
|
||||
gate_lo: float = 0.70 # primary alpha below this -> fully exempt
|
||||
gate_hi: float = 0.95 # primary alpha above this -> fully vetoable
|
||||
# Complex-background mode only (chroma disabled -> no hue-defined suspect
|
||||
# zone): the veto instead requires the second opinion itself to be
|
||||
# confidently near-empty -- full strength at/below second_lo, none at/above
|
||||
# second_hi. Thin structures the downsampled second model merely blurs to
|
||||
# mid-alpha stay untouched; only decisively-rejected regions can be cleared.
|
||||
second_lo: float = 0.15
|
||||
second_hi: float = 0.40
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -113,7 +137,12 @@ class ForegroundSettings:
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DespillSettings:
|
||||
enabled: bool = True
|
||||
# Off by default: despill pulls foreground chroma along the background-hue axis
|
||||
# with no positional or semantic guard, so a subject sharing the background's hue
|
||||
# (e.g. a blue suit on a blue backdrop) is desaturated/hue-shifted (~ΔE 35, blue
|
||||
# -> purple). This is the root default the cpu-fast lane inherits. Re-enable
|
||||
# per-config or with --despill only when edge spill genuinely matters.
|
||||
enabled: bool = False
|
||||
edge_low: float = 0.005
|
||||
strength: float = 0.92
|
||||
edge_expand_radius: int = 2
|
||||
|
||||
+10
-2
@@ -145,14 +145,21 @@ def fuse_trimap_directional(
|
||||
bg = chroma_bg | ((seg_mask < settings.seg_low) & ~chroma_fg)
|
||||
fg = chroma_fg & (seg_mask >= settings.seg_low)
|
||||
rule3_fg = chroma_unknown & core & ~bg_hued
|
||||
# Rule 4 -- semantic override: a chroma-unknown pixel the segmenter is
|
||||
# (near-)certain about stays foreground even when background-hued. Colour
|
||||
# evidence here only says "same hue family as the background" (bright skin
|
||||
# on a warm background), which is exactly where ViTMatte and the suppressor
|
||||
# misfire; a saturated seg signal outranks it. chroma_unknown excludes sure
|
||||
# background, so a real flat-colour backdrop can never be forced foreground.
|
||||
force_fg = chroma_unknown & (seg_mask >= settings.seg_force_fg)
|
||||
if settings.mode == "directional-hard-bg":
|
||||
# Aggressive variant: hard-remove background-hued pixels instead of leaving
|
||||
# them unknown. Can eat cool/shadowed white cloth, so it is not the default.
|
||||
bg = bg | (chroma_unknown & core & bg_hued)
|
||||
bg = bg | (chroma_unknown & core & bg_hued & ~force_fg)
|
||||
|
||||
trimap = np.full(shape, 128, dtype=np.uint8)
|
||||
trimap[bg] = 0
|
||||
trimap[fg | rule3_fg] = 255
|
||||
trimap[fg | rule3_fg | force_fg] = 255
|
||||
|
||||
loose = (seg_mask >= settings.seg_loose_threshold).astype(np.uint8)
|
||||
kernel = elliptical_kernel(band_radius)
|
||||
@@ -165,6 +172,7 @@ def fuse_trimap_directional(
|
||||
"sure_fg_pixels": int((trimap == 255).sum()),
|
||||
"band_radius": int(band_radius),
|
||||
"rule3_fg_pixels": int(rule3_fg.sum()),
|
||||
"force_fg_pixels": int((force_fg & ~fg & ~rule3_fg).sum()),
|
||||
"bg_hued_pixels": int((bg_hued & chroma_unknown & core).sum()),
|
||||
}
|
||||
return trimap, stats
|
||||
|
||||
+27
-3
@@ -3,6 +3,14 @@
|
||||
# like "#CFEFFF" sets it explicitly. Also overridable via --screen-color.
|
||||
screen_color: null
|
||||
|
||||
chroma:
|
||||
# false = non-flat background mode (--no-chroma): no colour key; segmentation
|
||||
# alone drives the trimap (mode forced to "seg") and the colour-keyed stages
|
||||
# are bypassed (auto-detect, hue split, chroma suppression, despill). The
|
||||
# cross-check veto still runs if enabled, using its hue-free gate (see
|
||||
# cross_check.second_lo/hi). Needs segmentation.enabled + matting vitmatte.
|
||||
enabled: true
|
||||
|
||||
model:
|
||||
model_name: hustvl/vitmatte-base-composition-1k
|
||||
device: cpu
|
||||
@@ -46,6 +54,12 @@ trimap:
|
||||
mode: directional
|
||||
seg_low: 0.15
|
||||
bg_hue_proj_min: 4.0
|
||||
# Semantic override (directional mode): chroma-unknown pixels with seg confidence
|
||||
# at/above this stay sure-FG even when background-hued — rescues bright skin on a
|
||||
# warm (same-hue-family) background that the hue split would otherwise hand to
|
||||
# ViTMatte/suppression, which misfire exactly there. Never overrides chroma
|
||||
# sure-background. Set > 1.0 to disable.
|
||||
seg_force_fg: 0.98
|
||||
|
||||
alpha_post:
|
||||
chroma_suppress: true
|
||||
@@ -74,8 +88,9 @@ cross_check:
|
||||
# trimap 99.8% identical, no structural change (fingers/wisps/bowstring OK).
|
||||
# Only applies to the birefnet backend (same-family swap); anime-seg keeps
|
||||
# its own forward, with the cross-check running independently on top.
|
||||
# Disable with --no-cross-check-as-seg. No effect when cross_check is off.
|
||||
reuse_as_seg: true
|
||||
# Default OFF (server-edition): the dedicated segmenter keeps its own forward
|
||||
# and the veto stays independent; enable with --cross-check-as-seg.
|
||||
reuse_as_seg: false
|
||||
# Cross-check forward precision, same semantics as segmentation.precision
|
||||
# (bf16 via autocast: 2048 pass ~35s -> ~26s on a native-bf16 CPU).
|
||||
precision: fp32
|
||||
@@ -84,6 +99,12 @@ cross_check:
|
||||
feather_sigma: 2.0 # zone-boundary feather, px
|
||||
gate_lo: 0.70 # primary alpha below this -> fully exempt
|
||||
gate_hi: 0.95 # primary alpha above this -> fully vetoable
|
||||
# Complex-background (no-chroma) mode only: no hue-defined suspect zone, so
|
||||
# the veto requires the second opinion itself to be confidently near-empty --
|
||||
# full strength at/below second_lo, fading to none at/above second_hi. Thin
|
||||
# strands the 2048-input second model merely blurs stay untouched.
|
||||
second_lo: 0.15
|
||||
second_hi: 0.40
|
||||
|
||||
foreground:
|
||||
enabled: true
|
||||
@@ -103,7 +124,10 @@ foreground:
|
||||
bg_confidence_weight: 0.70
|
||||
|
||||
despill:
|
||||
enabled: true
|
||||
# Off: despill has no positional/semantic guard, so a subject sharing the
|
||||
# background's hue (blue suit on a blue backdrop) gets desaturated/hue-shifted
|
||||
# (~ΔE 35, blue -> purple). Re-enable per-job only when edge spill matters.
|
||||
enabled: false
|
||||
edge_low: 0.005
|
||||
strength: 0.92
|
||||
edge_expand_radius: 2
|
||||
|
||||
Reference in New Issue
Block a user