BgFilter

中文: README_ZH.md

Offline character matting for AI-generated images on a flat-colour background. The background colour is auto-detected from the image border (green, pastel, any flat colour); pass --screen-color to set it explicitly.

RGB input
  -> chroma bg-confidence (keyed to the auto-detected or given colour)
  -> [optional] semantic segmentation mask
  -> trimap -> ViTMatte -> alpha cleanup
  -> cross-model veto (second matting opinion on bg-hued residue)
  -> pymatting foreground -> despill -> RGBA PNG -> QA previews

Pipelines

Two pipelines, selected by segmentation.enabled in the config:

• Chroma-only (enabled: false) — Chroma + ViTMatte, no segmentation model. Lightest / fastest; leans entirely on the colour key for topology.
• Single segmenter (enabled: true, default) — one segmentation model drives the trimap topology (holes, hair), ViTMatte then refines the soft edges. Backend is switchable: birefnet (default, general salient objects — text, logos, photos; needs trust_remote_code) or anime-seg (ONNX, tuned for anime characters). Switch at runtime with --seg-backend anime-seg (it also selects the matching weights).

Both share pymatting foreground estimation and a colour de-spill. There is no green-contamination rescue / recolour layer — with clean source images it is unnecessary, so it was removed.

Both also run a cross-model veto by default: a second, trimap-free matting model (ZhengPeng7/BiRefNet_HR-matting) may only lower alpha, only on background-hued bright pixels the primary result is confident about — this clears colour-drifted background residue trapped between hair strands that the chroma key, the segmenter and ViTMatte all read as foreground. Costs one extra model download (~0.9 GB) and one inference pass per image; disable with --no-cross-check (see the cross_check config section, and docs/hair_gap_artifacts.md for the analysis behind it).

When the cross-check is on, that same HR-matting forward can optionally be reused as the segmentation mask (--cross-check-as-seg), skipping the primary seg model entirely (one less model to load, ~20 s faster per image on CPU, ~1 GB less VRAM on GPU). Pilot-validated (TestImage3 / FixImage1): trimap 99.8% identical, no structural change to fingers, hair wisps or thin lines. Off by default: the dedicated segmenter keeps its own forward and the veto stays an independent second signal. The reuse is a same-family swap, so it applies to the birefnet backend only: with --seg-backend anime-seg the anime segmenter always keeps its own forward.

The segmentation trimap defaults to directional mode (chroma + seg + a hue-direction split: it keeps a background-coloured garment such as a white shirt while dropping a background-hued residual such as blue trapped between hair strands). Switch with --trimap-mode seg (topology only, no hue split) or directional-hard-bg (aggressive — hard-removes background-hued pixels; can eat cool/shadowed white cloth).

Background colour

By default (screen_color: null) the background colour is auto-detected from the image border: the dominant flat colour of the border strip becomes the key colour. If the border is not one clean flat colour — a gradient, texture, or a subject filling the frame — detection fails with an error; pass --screen-color explicitly in that case.

To set it yourself, give a hex prior:

... --screen-color "#CFEFFF"

or screen_color: "#CFEFFF" in the config. Either way the chroma key scores pixels by perceptual (Lab/RGB) distance to the colour, and de-spill removes chroma along that colour's direction. (A supplied hex is refined against nearby border pixels; an auto-detected colour is used directly.)

Environment

Any Python 3.11–3.13 environment (venv or conda) works:

pip install -r requirements.txt
python -m bgfilter.cli --help

For a server install — CPU-wheel torch, offline weights, systemd — follow DEPLOY.md instead; it covers the same dependencies with the production choices spelled out.

Single Image

python -m bgfilter.cli \
  --input Samples/TestImage.png \
  --output Outputs/TestImage_rgba.png \
  --debug-dir Outputs/TestImage_debug

No --config is needed — the built-in defaults are identical to configs/default.yaml. Pass --config configs\default.yaml only after you edit that file to tune the detailed parameters. Runtime choices stay on the command line: --device (default CPU; use --device cuda for GPU), --precision (default fp32; bf16 speeds up all three models and halves the matting model's activation memory with visually identical alpha — needs bf16-capable hardware, falls back to fp32 elsewhere; large inputs additionally get ViTMatte's global attention computed in query chunks by default — exact, bitwise-identical, caps the memory spike at ~4 GB instead of ~19 GB at 2048x2048, see model.attn_query_chunk), --seg-backend (default birefnet; anime-seg for anime characters), --screen-color (default: auto-detect the flat background), and --trimap-mode.

Batch

python -m bgfilter.cli \
  --input-dir Samples \
  --output-dir Outputs \
  --debug-dir Outputs/debug

Performance

Rough orders of magnitude: ~33–53 s/image on a fast desktop CPU (bf16, cross-check on), ~1–2 s/image on a modern GPU. All measured numbers — CPU lanes, GPU VRAM footprints, memory ceilings and the knobs that move them — live in DEPLOY.md §11 (Capacity & performance).

Chroma-alpha debug mode

--matting-method chroma skips ViTMatte and uses chroma confidence directly as the alpha seed. Useful for fast inspection of chroma confidence, trimap, and despill. (Distinct from the chroma-only pipeline above, which still runs ViTMatte.)

python -m bgfilter.cli \
  --input-dir Samples \
  --output-dir Outputs/chroma \
  --debug-dir Outputs/chroma_debug \
  --matting-method chroma \
  --device cpu

Outputs

For each processed image, the CLI writes an RGBA PNG and optional debug files:

bg_confidence.png
trimap.png
seg_mask.png          # segmentation pipeline only
alpha.png
foreground_rgb.png    # despilled foreground colour
color_mask.png        # per-pixel despill weight
preview_black.png
preview_white.png
preview_gray.png
preview_red.png
preview_blue.png
qa_grid.png
metadata.json

HTTP service

An HTTP wrapper (app.py + bgfilter/service.py) exposes the pipeline as a long-running FastAPI service. Models load once and are reused across requests. For deploying to a Linux server (dependencies, weights, systemd, offline mode, auth), see DEPLOY.md.

python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1

Run a single worker (--workers 1) — each worker loads its own copy of the models.

Endpoints

• GET /healthz — liveness + config JSON, e.g. {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu","crossCheck":true}. Always unauthenticated.
• POST /remove-background — multipart/form-data in, image/png (RGBA) out.

Authentication

Optional, opt-in via the BGFILTER_AUTH_TOKEN env var. When set, /remove-background requires header X-Genarrative-Image-Token: <token> (constant-time compare); a missing or wrong token returns 401. When unset the endpoint is open (a fronting proxy is expected to authenticate). /healthz is always open. See DEPLOY.md §10.

Request fields (POST /remove-background, multipart/form-data)

Field	Required	Values	Meaning
file	one of file/image_url	image file	Source image as an upload (field name fixed as file).
image_url	one of file/image_url	http(s)://…	Source image by URL — the service fetches it (bounded by size/time; follows ≤3 redirects). Provide exactly one of file or image_url. No SSRF filtering: the endpoint trusts authenticated callers, so keep auth on and callers trusted.
background_mode	no	flat (default) / complex	flat = solid-colour background (colour-keyed pipeline). complex = non-flat/scene background: no colour key, the segmenter alone drives the matte. screen_color is ignored in complex; expect flat backgrounds to stay stronger on hair detail.
screen_color	no	#RRGGBB, or omit/empty/auto	Background-colour prior; omit to auto-detect from the border. Ignored when background_mode=complex.
seg_model	no	birefnet (default) / anime-seg	Segmentation backend.
cross_check	no	1/0 (also true/false/yes/no/on/off)	Per-request override of the cross-model veto; omit/empty = service default in flat mode, but defaults off in complex mode (set on to enable its hue-free gate).

Response headers (on 200)

Header	Example	Meaning
X-BGFilter-Elapsed-Ms	2372	Server-side processing time (ms).
X-BGFilter-Seg-Model	birefnet	Segmenter actually used.
X-BGFilter-Screen-Color	#CFEFFF / auto	Background colour used.
X-BGFilter-Cross-Check	on / off	Effective cross-check state.
X-BGFilter-Background-Mode	flat / complex	Background mode actually used.
X-Request-ID	42efe4e0d76f4b83	Correlation id (echoes an inbound X-Request-ID if you send one, else generated). Quote it to correlate with server logs. Returned on every response, including errors.

Status codes

Code	When
200	Success — body is the RGBA PNG.
400	Bad input: unreadable image, not exactly one of file/image_url, bad image_url / fetch failed, invalid screen_color/seg_model/cross_check/background_mode, or auto-detect failed (pass screen_color).
401	Auth enabled and the token is missing or wrong.
413	Image exceeds BGFILTER_MAX_IMAGE_PIXELS (~4 MP by default), or an image_url body exceeds BGFILTER_FETCH_MAX_BYTES (~25 MB).
500	Inference failed.
503	Service still loading models (not ready).

Examples

# default (birefnet + auto background colour); add the auth header if the service requires it
curl -sS -F "file=@input.png" \
  -H "X-Genarrative-Image-Token: <token>" \
  http://127.0.0.1:18083/remove-background -o output.png

# explicit background colour + anime segmenter, cross-check off for this request
curl -sS \
  -F "file=@input.png" \
  -F "screen_color=#CFEFFF" \
  -F "seg_model=anime-seg" \
  -F "cross_check=off" \
  http://127.0.0.1:18083/remove-background -o output.png

# non-flat / scene background (no colour key); cross-check defaults off here,
# add cross_check=on to enable the hue-free residue veto
curl -sS \
  -F "file=@scene.jpg" \
  -F "background_mode=complex" \
  http://127.0.0.1:18083/remove-background -o output.png

# by URL instead of an upload (service fetches it; use image_url OR file, not both)
curl -sS \
  -F "image_url=https://example.com/input.png" \
  -F "screen_color=#CFEFFF" \
  -H "X-Genarrative-Image-Token: <token>" \
  http://127.0.0.1:18083/remove-background -o output.png

Runtime configuration (config file, device, size caps, preload, auth token, pipelining workers) is all via BGFILTER_* environment variables — the full table is DEPLOY.md §6.

Like the CLI, the service runs the cross-model veto by default; /healthz reports the effective default as crossCheck and the per-request cross_check field overrides it either way (service-wide kill switch: BGFILTER_CROSS_CHECK=0). cross_check.reuse_as_seg (config-only, default off — see Pipelines above) has no per-request switch; changing it means editing the config and restarting.

Quality Check

The quality checker measures alpha validity and edge spill on semi-transparent edge pixels.

python -m bgfilter.quality_cli \
  Outputs/TestImage_rgba.png \
  --max-edge-green-excess-p95 0.30

Run the bundled sample smoke check:

python scripts/smoke_samples.py \
  --samples-dir Samples \
  --output-dir Outputs/smoke_samples \
  --config configs/default.yaml \
  --device cpu \
  --fallback-to-chroma-alpha \
  --max-edge-green-excess-p95 0.30

Notes

• Samples/ and Outputs/ are ignored by Git.
• ViTMatte and segmentation weights load from Hugging Face on first use. anime-seg (skytnt/anime-seg) is a plain ONNX download; birefnet (ZhengPeng7/BiRefNet) ships custom modelling code so it needs trust_remote_code=True plus timm / einops / kornia.
• Provisioning weights — offline HF cache, plain models/ folders (BGFILTER_WEIGHTS_DIR), the scripts/fetch_weights.py one-shot downloader, and the CN-network mirror workaround — is covered in DEPLOY.md §5.
• Foreground colour estimation uses pymatting's estimate_foreground_ml to propagate clean foreground colour into semi-transparent edges before de-spill. Set foreground.method: unmix to fall back to the legacy heuristic.
• docs/green_screen_matting_workflow.md is the original phase-1 green-screen spec; this README reflects the current, generalised architecture.