Compare commits

51 Commits

Author SHA1 Message Date
lhk229 f1a083349b Drop the silhouette band overwrite in the directional trimap
The band (dilate-minus-erode ring on the seg silhouette, added in 5f48f7a)
was meant to give ViTMatte room to anti-alias the boundary. Measured on a
warm-background sample (fix_image02, #FFD6C2): ViTMatte outputs near-binary
alpha in the band on crisp-edged anime images (96% hard values, 82% of
silhouette crossings with zero soft pixels), so the band delivered no
anti-aliasing -- it only let ViTMatte relocate the boundary (chewed edges)
and wholly confiscated structures thinner than the ring: 74% of lost hair
strand pixels were rule-FG demoted by the overwrite, then dropped by
ViTMatte. Without the band, 88% of those strands survive, edges follow the
rule-decided contour, and cross-check no longer bites strand edges because
strand cores stay anchored at 255 (re-destroyed: 0 px).

The rules own unknown pixels (background-hued blends, low-seg transitions)
remain ViTMatte territory. fuse_trimap (seg mode) keeps its band untouched --
its rules were designed around it and the removal is only validated for the
directional path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 14:15:53 +00:00
lhk229 78ef411ad2 Silence httpx per-request INFO lines on the prose log stream
httpx logs every outbound fetch at INFO with the full presigned image_url,
leaking per-request records (and OSS credential/signature params) into the
event stream that is meant for low-volume prose only. The access JSONL
already records the URL, so cap the httpx logger at WARNING.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 02:35:54 +00:00
lhk229 43797eedeb Split logging: prose event stream vs structured JSONL access stream
Rule: bulk per-request data never lands in the prose log; everything with
analysis value goes to a structured file.

- Event stream (stderr, text): startup summary, warnings, tracebacks --
  what supervisord/journald already captures. Human-readable, low volume.
- Access stream: one JSONL record per request (plus a "startup" meta
  record with the instance config) written to BGFILTER_ACCESS_LOG,
  app-rotated 50MB x 5; falls back to stderr when unset (dev).
  Failed requests carry a truncated `exc` copy so the access file is
  self-contained; the full traceback stays in the event stream, joined
  by request_id.
- BGFILTER_LOG_FORMAT retired: format is now a property of the stream,
  not a global switch.
- uvicorn runs with --no-access-log everywhere (deploy script + unit
  examples): its prose per-request lines duplicated a subset of ours.
- deploy_autodl.sh --log-dir now provisions both files; docs updated
  (EN+ZH: env table, AutoDL section, run/systemd examples).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 13:40:33 +00:00
lhk229 3a8a1ef38a Log the full image_url in the access line
url_host alone made cross-referencing a failed fetch with the caller's
request awkward. Log the full URL too (truncated to 512 chars). Presigned
URLs do land their query-string token in the log -- acceptable for trusted
callers writing to a private log; drop the query if that assumption changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 06:32:58 +00:00
lhk229 258e0fcf6e default log sets to json 2026-07-17 04:53:50 +00:00
lhk229 35d8bf9b02 Restructure docs: README = what/how-to-call, DEPLOY = how-to-run
Each fact now has exactly one home, with pointers instead of copies:

README (EN+ZH):
- Drop stale workstation env (conda lightML, D:\MiniConda paths); CLI
  examples now use plain `python -m bgfilter.cli` with bash syntax.
- Performance section reduced to orders of magnitude + pointer; the
  measured workstation table moved to DEPLOY §11.
- Notes: HF-mirror workaround and models/-folder layout (both duplicated
  DEPLOY §5 verbatim) replaced with one-line pointers.
- HTTP service: env-var prose replaced with a pointer to DEPLOY §6.
- ZH parity fixes: add the missing image_url curl example; 400/413 rows
  now mention image_url/fetch limits like the EN table.

DEPLOY (EN+ZH):
- §0: supervisord management expanded to a full command set (status/
  restart/stop/start/shutdown, both log paths, manual supervisord rescue
  after a container restart, how supervisord.conf changes are applied).
- §5: cross-check veto and background-mode subsections trimmed to
  deploy-side facts + pointers to README Pipelines/HTTP service.
- §11: absorbs the workstation CPU reference table from README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 04:35:17 +00:00
lhk229 755e5fda44 Document AutoDL deploy path; harden its token/log handling
Add a section 0 to DEPLOY.md/DEPLOY_ZH.md covering the AutoDL one-click
path (supervisord, not systemd), which was previously only self-documented
in scripts/deploy_autodl.sh.

deploy_autodl.sh:
- --log-dir (default ~/autodl-tmp, the persistent data disk) so the service
  log no longer lands in /tmp, which AutoDL wipes on container reset.
- Re-running without --token now REUSES an existing token instead of
  deleting it, so tweaking an unrelated knob can't silently disable auth;
  add --open to explicitly run without auth.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 04:19:10 +00:00
lhk229 dc4badd0e0 Add structured per-request access logging with a request id
Each request now emits one access line (bgfilter.access) carrying a
request_id, the effective params (source file/url, seg_model, cross_check,
background_mode, screen_color, auth), input/output sizes, and stage
timings (t_queue/fetch/decode/process/total_ms). A middleware binds the id
(reusing a sanitized inbound X-Request-ID, else uuid), a logging filter
injects it into every record, and it is echoed back as the X-Request-ID
response header on every response -- including errors, which also log the
detail. Requests that raise still produce a line (fields accumulate on
request.state; the middleware finalizes status/total/client).

request_id lives in a contextvar (not a thread-local) so a pooled worker
never inherits the previous request's id; propagating it into the executor
for deep pipeline logs is left for later. BGFILTER_LOG_FORMAT=text|json
switches human vs shipper-friendly output. /healthz is not logged.

Verified locally via TestClient (fake manager, no models): 200 path logs
all fields + timings with rid == header; inbound id echoed; over-long id
sanitized; XOR 400s log status + error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 05:55:23 +00:00
lhk229 6506bdd643 Add image_url input mode (service fetches the image)
/remove-background now accepts either an uploaded `file` or an `image_url`
the service fetches itself (exactly one required). The fetch runs BEFORE
the compute gate so a slow download never holds a pool slot, and is
bounded by its own semaphore (2 x cpu_workers, hard cap 64) plus a shared
httpx client (connection reuse, matching max_connections) -- this restores
the throttling a direct upload has implicitly, so a burst of tiny URL
requests can't exhaust FDs/ephemeral ports or hammer the upstream.

Bounds: 25 MB body cap (413), 5 s connect / 15 s read timeout, <=3
redirects; MAX_IMAGE_PIXELS still applies post-decode. Deliberately NO
SSRF filtering (no private-IP/host allowlist) -- the endpoint trusts
authenticated callers; auth runs before any fetch. Do not expose to
untrusted clients without adding private-address/redirect filtering to
_fetch_image.

Adds httpx. New env knobs BGFILTER_FETCH_MAX_BYTES / BGFILTER_FETCH_TIMEOUT.
Docs (README/README_ZH/DEPLOY/DEPLOY_ZH) updated. _fetch_image unit-tested
locally (happy path, size cap ->413, bad scheme ->400, 404 ->400);
on-box end-to-end still pending (test box went offline).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 14:33:34 +00:00
lhk229 07ba02fdde Make the pipelining knobs config-file settable (server: section)
CPU_WORKERS / GPU_CONCURRENCY were env-only. Add a `server:` config
section (server.cpu_workers, server.gpu_concurrency) so a lane's
concurrency lives in its yaml alongside device/precision, matching how
model.device already works. Precedence mirrors BGFILTER_DEVICE: config is
the default, the env var overrides per box.

The pool + admission semaphore now build in lifespan (after the config
loads) instead of at import, and PipelineManager reads gpu_concurrency
from the settings. configs/gpu.yaml ships 8 / 2 (its target is the 32 GB
GPU boxes) with an inline VRAM caveat: gpu_concurrency>1 must fit
weights + N × forward, so keep it 1 on ≤16 GB cards.

Verified on a 4080 SUPER: startup logs cpu_workers=8 gpu_concurrency=2
from gpu.yaml with no env set, request returns a valid RGBA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 12:58:15 +00:00
lhk229 c748fc22a4 Pipeline concurrent requests over a shared GPU with a narrowed lock
The single asyncio lock serialized the whole request (chroma -> seg ->
trimap -> ViTMatte -> cross-check -> foreground -> despill), so the GPU
sat idle ~70-80% of each request while the CPU stages ran. Replace it
with a thread pool that runs several requests concurrently, and move the
mutex down to just the GPU forwards: when one request steps off the GPU
to do CPU work, another steps on, keeping the device fed.

Two opt-in env knobs, both defaulting to the old strictly-serial
behaviour:
  * BGFILTER_CPU_WORKERS (pool size) attacks the CPU-bound regime -- with
    cross-check off, throughput went 0.91 -> 1.67 req/s (1.84x) at pool=8
    on a 4080S, vs a mere +6% from a second worker process, at half the
    VRAM (one model copy).
  * BGFILTER_GPU_CONCURRENCY caps how many forwards may co-schedule on the
    SMs. batch-1 underfills the GPU, so =2 recovers the SM overlap a lock
    forgoes: cross-check on went 0.83 -> 1.01 req/s, beating two worker
    processes (0.92) with one model copy (~19 GB vs 24 GB).

The CLI/single-thread path passes no lock and uses a nullcontext, so it
stays allocation- and contention-free. Concurrent output verified
identical (10x same-frame -> one hash). A per-worker admission semaphore
bounds in-flight decoded images to the pool size.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 12:47:50 +00:00
lhk229 a0caac001f Clarify deploy script's final service-management hint
The old one-liner printed `status|restart|stop|tail bgfilter`, which
users pasted verbatim and the shell parsed the `|` as pipes. Split it
into one copy-pasteable command per action.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 09:12:35 +00:00
lhk229 6baf6ae541 docs: document background_mode in DEPLOY.md / DEPLOY_ZH.md
Sync the deployment guides with the c56e58a HTTP change: add a
"Background mode (flat vs complex)" subsection and a complex-mode curl
example. Notes that complex reuses existing weights, ignores screen_color,
defaults the cross-check veto off, and returns X-BGFilter-Background-Mode;
points to README for the full field/header reference.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 14:41:06 +08:00
lhk229 c56e58affe Add complex-background matting mode to the service via background_mode
Backports master's non-flat matting (chroma.enabled: false + the hue-free
cross-check gate) into server-edition, and exposes it over HTTP without
surfacing the internal "chroma" wording: /remove-background gains a
background_mode form field (flat, default | complex). complex maps to
chroma disabled -- no colour key, segmentation alone drives the trimap and
every colour-keyed stage (auto-detect, hue split, chroma suppression,
despill) is bypassed. The cross-check veto still works in complex mode via
its second-opinion-confidence gate (cross_check.second_lo/hi) but defaults
OFF there (it costs the HR-matting forward); an explicit cross_check=on
re-enables it.

No new model weights: complex mode reuses the already-provisioned BiRefNet
seg + ViTMatte (+ optional HR-matting cross-check). Flat mode is unchanged
(bit-identical), and server-edition's own extras (cross_check.lock,
foreground.use_gpu CuPy path) are preserved -- the port is surgical, not a
copy of master's files.

- settings: ChromaSettings.enabled, CrossCheckSettings.second_lo/hi
- config: override_settings chroma passthrough
- despill/foreground: model=None safe guards (foreground keeps GPU path)
- alpha_post: cross_check_alpha hue-free gate when proj is None
- pipeline: _process_rgb complex branch (seg-only trimap, skip colour stages)
- service/app: process(chroma=), background_mode field, complex-defaults-off
  cross-check, X-BGFilter-Background-Mode header
- cli: --chroma/--no-chroma
- configs/docs: gpu.yaml + default.yaml + README/README_ZH

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 14:04:58 +08:00
lhk229 28d3fe8842 Add one-click AutoDL deploy script and GPU lane config
scripts/deploy_autodl.sh: idempotent setup for an AutoDL box -- installs app
deps (keeping the image's CUDA torch), picks the cupy wheel from torch's CUDA
version for GPU foreground, fetches weights, writes the auth token, renders a
supervisord instance (crash auto-restart) + a ~/.bashrc login autostart, then
starts the service and waits for /healthz. Args: --token, --port (default
6006), --config (default configs/gpu.yaml).

configs/gpu.yaml: the GPU lane (CUDA + bf16, full pipeline with cross-check,
despill off) promoted from a box-local file into the repo so the deploy is
reproducible.

The public port mapping and the "no auto-start without an interactive login"
gap are AutoDL platform limits, documented at the top of the script.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 03:42:05 +00:00
lhk229 dbd061e119 Add cross_check lock so a locked lane ignores per-request cross_check=on
The cpu-fast lane disables cross-check to fit a low-memory box, but a request
could still send cross_check=on and override it, lazy-loading the HR-matting
model and running the 2048 forward -> the exact OOM (worker SIGKILL -> 502)
cpu-fast exists to avoid. BGFILTER_CROSS_CHECK=0 did not protect against this
because the per-request form field wins.

Add CrossCheckSettings.lock (default False). When set, PipelineManager
resolves cross-check to the base value and ignores per-request overrides, so a
locked off-lane silently honours cross_check=on as off -- no HR-matting load,
no cross-check run, no error (200), and the response header reports "off".
Enable it in cpu-fast.yaml (enabled: false, lock: true). Verified end-to-end:
cross_check=on on the locked lane returns without loading the cross-checker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 10:26:38 +00:00
lhk229 9094b5d9c0 Run pymatting foreground estimation on GPU via CuPy when available
The multi-level foreground colour estimation (pymatting estimate_foreground_ml)
is the pipeline's CPU bottleneck (~7 s/image). pymatting ships a CuPy backend
(estimate_foreground_ml_cupy) that runs the same multi-level solve on the GPU
~250x faster; verified visually identical to the float64 CPU result (opaque
colours bit-close, the float32 edge delta is alpha-weighted away).

Add ForegroundSettings.use_gpu (default True): with method "ml", try the CuPy
backend when cupy + a CUDA device are present, otherwise transparently fall back
to the CPU solver. pymatting only re-exports the cupy function when cupy is
importable, so CPU-only deployments (no cupy) are unaffected. Requires
cupy-cudaXXx matching the CUDA build on GPU boxes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:32:35 +00:00
lhk229 ece9542978 ADD MANDARIN README 2026-07-11 09:12:35 +00:00
lhk229 826ad09098 docs: rewrite README HTTP service section into a full API reference
- fix stale launch example (drop the D:\ lightML path -> generic uvicorn)
- add Endpoints, Authentication (X-Genarrative-Image-Token / BGFILTER_AUTH_TOKEN),
  request-field table, response-header table, and status-code table
- update examples to show the auth header and cross_check=off

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:08:49 +00:00
lhk229 1861cb0199 docs: document BGFILTER_AUTH_TOKEN auth and despill-off default
DEPLOY.md / DEPLOY_ZH.md:
- add BGFILTER_AUTH_TOKEN to the env-var table
- new "Authentication" subsection under §10: opt-in token via the
  X-Genarrative-Image-Token header (401 on missing/wrong), fail-open when
  unset (proxy handles it), /healthz always open; drop the stale "no
  authentication" line
- note that despill is off by default (same-hue false-positive)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 08:56:39 +00:00
lhk229 4d34c96bed Add optional token auth to /remove-background
Adds app-level authentication gated on the BGFILTER_AUTH_TOKEN env var.
When set, /remove-background requires the X-Genarrative-Image-Token header
to match it (constant-time compare via hmac); missing or wrong -> 401. The
header name matches the production nginx layer, so callers only swap the
URL + token value, no interface change.

Fail-open by design: when BGFILTER_AUTH_TOKEN is unset the check is a no-op
(open), preserving current behaviour and leaving auth to an upstream proxy
(nginx in production). A startup WARNING is logged when auth is disabled.
/healthz stays unauthenticated for health checks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 08:26:04 +00:00
lhk229 ba5d667524 Disable despill by default (all lanes, incl. cpu-fast)
despill pulls foreground chroma along the background-hue axis with no
positional or semantic guard, so a subject sharing the background's hue
(e.g. a blue suit on a blue backdrop) is desaturated and hue-shifted
(measured ~ΔE 35, blue -> purple) — a false positive on correct colours.

Flip the DespillSettings.enabled dataclass default to False so every lane
that omits a despill section — notably cpu-fast — inherits it off, and
mirror it in default.yaml. Re-enable per-config or with --despill when
edge spill genuinely matters.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 14:13:42 +00:00
lhk229 ab89121ccb Merge branch 'server-edition' of https://git.genarrative.world/git/lhk229/BGfilter into server-edition 2026-07-09 12:28:52 +08:00
lhk229 3f85043ccc Merge branch 'server-edition' of https://git.genarrative.world/git/lhk229/BGfilter into server-edition 2026-07-09 02:39:05 +00:00
lhk229 ef11f625f1 Merge branch 'master' of https://git.genarrative.world/git/lhk229/BGfilter into server-edition 2026-07-09 02:34:50 +00:00
lhk229 8ef713d560 Add seg_force_fg: saturated seg confidence overrides the hue-split demotion
New rule 4 in fuse_trimap_directional: a chroma-unknown pixel whose seg
confidence is >= trimap.seg_force_fg (default 0.98) stays sure-FG even when
background-hued. Bright skin on a warm same-hue-family background (peach,
pale yellow) was being demoted to unknown by the hue split and then lost --
either ViTMatte itself misfires there (raw 0.07-0.13 measured) or the
chroma suppressor does (raw 0.89 halved to 0.45 via the RGB-proximity
confidence path). The segmenter meanwhile rates those pixels a saturated
1.0; that semantic certainty now outranks same-hue colour suspicion.

Safety: chroma_unknown excludes sure background, so a real flat backdrop
can never be forced foreground; the silhouette band still re-opens the
boundary; the cross-check veto still overrides trimap-FG. Set > 1.0 to
disable.

Verified (GPU bf16): yellow-bg frame face 0.84->1.00, peach-bg thigh
0.87->1.00 (residual softness only where chroma is near-sure-bg, by
design). Pastel-blue TestImage3 regression: 0.047% of pixels differ with
cross-check on, 0.058% without it (hair-gap suppression path), both
visually negligible.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 18:15:10 +08:00
lhk229 a221e69212 Merge remote server-edition: cpu-fast lane deploy docs
Both sides extended DEPLOY(_ZH): remote documented the cpu-fast lane (weights subset, config comparison, systemd note, OOM troubleshooting), local added the measured GPU profile and the reuse-off default. Conflict was the capacity-section memory bullet: kept the GPU deployment bullet and remote's cpu-fast-aware memory bullet (which supersedes the old three-models line).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 20:52:57 +08:00
lhk229 9b9d71cf82 Default reuse-as-seg OFF; default expandable_segments on Linux/WSL
Two default flips for server-edition:

1. cross_check.reuse_as_seg now defaults to false (settings + default.yaml + CLI help): the dedicated segmenter keeps its own forward and the cross-check veto stays an independent second signal. The reuse remains available via --cross-check-as-seg / config; measured cost of off vs on: ~20 s/image on CPU, +0.1-0.25 s and +0.9-1.3 GB VRAM on GPU (13.3 vs 12.0 GB reserved under expandable segments -- still fits a 16 GB card).

2. bgfilter/__init__.py defaults PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True on non-Windows platforms, before torch loads (setdefault: an explicit env value wins; native Windows is excluded because torch warns and ignores it there). Measured: reserved 15.3 -> 12.0 GB and ~10% faster on the RTX 5070 Ti; no effect on CPU-only runs.

README/DEPLOY(_ZH) synced: reuse documented as opt-in, GPU profile numbers updated for both reuse states, perf table labeled with the config it was measured under.

Verified: defaults resolve off/set as intended on Windows and WSL; explicit PYTORCH_CUDA_ALLOC_CONF override wins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 20:40:19 +08:00
lhk229 6bf648cd64 Document the measured GPU deployment profile (bf16 + expandable_segments)
Measured on an RTX 5070 Ti 16 GB, full default pipeline (cross-check on, reused as seg, bf16): ~1-2 s/image, ~10.1 GB allocated. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (Linux/WSL2 only; silently unsupported on native Windows) cuts the allocator's reserved footprint 15.3 -> 12.0 GB and is ~10% faster, leaving ~3.5 GB headroom on a 16 GB card even beside a desktop session. bf16 is the precondition: fp32 reserved ~17 GB overflows. reuse-as-seg also saves ~0.9 GB VRAM / ~0.25 s per image on GPU (reuse off measured 13.3 GB reserved under expandable_segments).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 20:32:13 +08:00
lhk229 9931980c72 Update Deploy Docs 2026-07-06 10:57:27 +00:00
lhk229 a48d70d9f7 Cap cpu-fast lane at attn_query_chunk 512 for the 8 GB server
Config-only tuning of the temporary CPU lane (no pipeline/code change). Query chunk 512 trims ViTMatte's attention transient ~0.9 GB at 2048x2048 (8.0 -> 7.1 GB peak working set) with bitwise-identical output; 256 gives nothing more. The remaining ~7.1 GB floor is attributed to ViTMatte's detail-capture decoder (backbone stage peaks 4.6 GB, decoder 7.1 GB, pymatting only 0.08 GB), so it is not further reducible by attention chunking -- noted for a possible future decoder-tiling pass. Accepted as-is for the 8 GB box.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:50:36 +08:00
lhk229 38a0dc34cc Add configs/cpu-fast.yaml: fp32 CPU lane, no cross-check, no anime-seg
Urgent lightweight production profile for a CPU-only fp32 server: base ViTMatte + one BiRefNet@1024, cross-check disabled. Provisions only those two weights (no BiRefNet_HR-matting, no anime-seg) and returns to the ~14-15s/image baseline. Run with BGFILTER_CONFIG=configs/cpu-fast.yaml.

Verified end-to-end: loads exactly 2 models, cross_check resolves off, service preloads the primary segmenter and not the cross-checker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:17:25 +08:00
lhk229 3992f65ebf Return freed activations to the OS on Linux; recommend jemalloc (A+B)
MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux
server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed
activations in the arena, so RSS ratchets up across requests and the env var is
a no-op there.

A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at
   runtime (no-op on Windows/musl/other allocators). Wired into service.process
   (after each request) and the CLI batch loop (after each image), the two
   long-lived paths where RSS accumulates. Single-image CLI exits, so it is left
   alone.
B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also
   improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/
   MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc
   malloc_trim simply no-ops.

Also fixes the stale off-by-default cross-check heading in DEPLOY.md.

Verified: modules import; release_freed_memory() returns False (no-op) on Windows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:40:58 +08:00
lhk229 ad445611ad Make cross-check and reuse-as-seg the service default
Cross-check + reuse-as-seg are already the config/CLI default; align the HTTP
service with them instead of forcing cross-check off. The earlier off-by-default
was to avoid the extra inference, but reuse-as-seg now makes that forward double
as the seg mask (birefnet backend) -- it replaces the primary seg model rather
than adding to it, so the cost concern is gone.

- app.py: honour the config default (on); BGFILTER_CROSS_CHECK still forces
  either way, unset = config.
- service.py preload: warm the cross-check HR-matting model, and skip the now-
  redundant primary segmenter when reuse covers segmentation.
- README / DEPLOY(_ZH): document cross-check as default-on, BiRefNet_HR-matting
  as required (not optional), BGFILTER_CROSS_CHECK=0 to disable.

Verified: service defaults cross_check on, env forces both ways, preload picks
cross-checker and skips the primary segmenter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:11:43 +08:00
lhk229 c19a35de80 Merge origin/master: bf16 precision, chunked attention, seg-reuse, mimalloc purge
Adopts master as the standard for all overlapping work. Master's landed
optimizations supersede server-edition's own fp16 experiment:

- Unified --precision fp32|bf16 knob (bgfilter/precision.py) driving all
  three models: ViTMatte weight cast + both BiRefNets via autocast, with a
  hardware gate (falls back to fp32 off native-bf16 CPUs) and an
  AutocastCPU fp32 shim for torchvision deform_conv2d.
- Query-chunked ViTMatte global attention (bgfilter/attn_chunk.py), exact
  and bitwise-identical, caps the N^2 spike (~19 -> ~4 GB at 2048).
- Cross-check HR-matting forward reused as the seg mask (birefnet backend
  only), skipping the primary seg model; MIMALLOC_PURGE_DELAY=0.
- inference_mode and the detect_background_color removal converge with
  server-edition's earlier equivalents.

Conflict resolution (favoring master, preserving server-only features):
- vitmatte_infer/segmentation: dropped server's device-derived fp16 for
  master's precision path, kept resolve_model_source (local weights).
- service.py: cross-check SegmentationSettings now passes precision so the
  HTTP service honors bf16 like the CLI's _second_opinion does.

Verified on CPU: default fp32 pipeline loads 2 models (seg-reuse active)
and bf16 path runs (autocast + deform_conv2d shim) — both exit 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:02:44 +08:00
lhk229 291e9faac5 Derive model dtype from device: fp16 on CUDA only, fp32 on CPU
Benchmarking c7a2819 on CPU showed unconditional fp16 is pathological
there: ~1870 s/image vs ~14.4 s fp32 (~130x) -- x86 has no native fp16
conv/gemm kernels so PyTorch falls back to a slow reference path. fp16
only pays off on CUDA, so ViTMatteRunner and BiRefNetSegmenter now pick
float16 iff the resolved device is cuda, float32 otherwise; inputs cast
to the same dtype. The dtype is intentionally not a setting.

Also replaces the deprecated torch_dtype= kwarg with dtype=.

Verified on CPU: single-image CLI run back to normal speed (~31 s wall
including model load) and output byte-identical (SHA256) to the
fp32-era baseline, confirming inference_mode alone changes nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 11:08:29 +08:00
lhk229 c7a2819d33 Run torch models in fp16 under inference_mode
- ViTMatte and BiRefNet load with torch_dtype=float16 (the BiRefNet
  checkpoint ships fp16 anyway; the old .float() upcast is gone) and
  floating-point inputs are cast to half to match.
- torch.no_grad() -> torch.inference_mode() in both predict paths.
- Outputs already downcast via .float() before .numpy(), so downstream
  stays float32. anime-seg is ONNX and unaffected.

Effect verified by the user beforehand.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 02:50:42 +08:00
lhk229 5265e802da Update Deploy - now use fetch weight 2026-07-05 10:00:20 +00:00
lhk229 6232887ebd Pin opencv-python-headless in requirements
The GUI opencv-python build needs libGL.so.1 which headless Linux servers lack,
causing "ImportError: libGL.so.1" at first cv2 use. Pin the headless build so a
plain `pip install -r requirements.txt` works on servers out of the box (and
survives a git reset). BgFilter only uses cv2 for computation, so headless is a
drop-in on desktops too. Drop the now-unnecessary sed step from DEPLOY.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 09:45:51 +00:00
lhk229 bfbeb9fc7f Add scripts/fetch_weights.py; correct HR-matting size in DEPLOY
- fetch_weights.py: no-argument script that downloads every model weight into
  models/ via the hf-mirror (proxy forced-bypassed, Xet disabled). Model ids are
  read from config/code, and folders that already hold a weight file are skipped
  (no network, no re-download). After it runs the app has full functionality
  offline. Verified end to end (skips the 3 present models, pulls HR-matting).
- DEPLOY.md / DEPLOY_ZH.md: correct BiRefNet_HR-matting size from ~0.9 GB to the
  verified ~425 MB.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 09:35:17 +00:00
lhk229 4b5c714fa3 Document weight file layout and cross-check in DEPLOY
- Per-model required-file directory tree for the local models/ layout
  (ViTMatte / BiRefNet / anime-seg), noting which files each loader needs.
- Add ZhengPeng7/BiRefNet_HR-matting as the optional 4th model, used only by
  the cross-check veto; new "cross-check" subsection covering enablement, the
  offline-provisioning requirement, and the ~2x latency.
- Add BGFILTER_CROSS_CHECK to the env-var table and a troubleshooting row for
  the offline "500 inference failed" case.
- Mirror all of the above into DEPLOY_ZH.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 09:14:57 +00:00
lhk229 d3f19ddcc7 add DEPLOY.md 2026-07-05 09:04:22 +00:00
lhk229 edcef6e34d Merge branch 'server-edition' of ssh://genarrative-station:2222/lhk229/BGfilter into server-edition 2026-07-05 08:47:14 +00:00
lhk229 88987f508c Remove dead code: unused detect_background_color and ForegroundEstimate extras
- chroma.detect_background_color had no callers anywhere (auto-detection
  goes through estimate_background_model / _background_border_cluster).
- ForegroundEstimate.background and .correction were computed on every
  image but never consumed; only .rgb was used. estimate_foreground_rgb
  now returns the foreground array directly and the dataclass is gone,
  which also drops pymatting return_background work and a per-image
  correction-norm pass.

Verified: CLI output on samples/TestImage3.png is byte-identical
(SHA256) to the pre-cleanup baseline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:41:11 +08:00
lhk229 8417653598 Add per-request cross_check toggle to the HTTP service (lazy-loaded)
POST /remove-background gains an optional cross_check form field
(1/0/true/false/yes/no/on/off; empty = service default). The effective
state is reported in the X-BGFilter-Cross-Check response header.

The cross-check model is now never preloaded at startup: the first
request that enables it pays the one-time load (measured ~8 s on top of
the ~50 s cc inference), after which the instance is cached and shared.

Verified with a functional test against PipelineManager: cc-off requests
never load the model, cc=1 lazy-loads then reuses the cached instance,
and omitting the field falls back to the service default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:34:08 +08:00
lhk229 11c591e5b5 Default the HTTP service to cross-check OFF (opt in via BGFILTER_CROSS_CHECK)
The cross-model veto adds one extra large-model inference per request
(measured ~35 s/image on CPU: 14.4 s -> 49.1 s), which is too slow as a
service default. The service now disables it regardless of the config
default; BGFILTER_CROSS_CHECK=1 re-enables it (GPU or quality-first
deployments). CLI behaviour is unchanged (still on by default).

Startup log and /healthz report the effective state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:57:39 +08:00
lhk229 578bd1f4d8 Merge master: matte-confidence gate + cross-model hair-gap veto
Brings in the two algorithm optimisations from master:
- Gate chroma suppression by the matte's own confidence (639fdf0)
- Add cross-model veto of hair-gap background residue (adbbb9d)

Server-edition integration on top of the plain merge:
- PipelineManager caches the cross-check model (BiRefNet_HR-matting) and
  injects it into MattingPipeline, so HTTP requests never reload it.
- MattingResult carries cross_check_alpha; debug output matches master.
- /healthz reports crossCheck; README lists models/BiRefNet_HR-matting
  in the local-weights bundle.

Verified: CLI --no-cross-check baseline and full cross-check run both
pass on samples/TestImage3.png; cross_check_alpha.png written to debug.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:37:07 +08:00
lhk229 8a7ad60705 Add DEPLOY.md: Linux server deployment guide
Document deploying the FastAPI service to a Linux server end to end: venv +
CPU-only torch (avoid the CUDA build), opencv-python-headless, the two weight
layouts (HF cache offline vs local models/ folders) with the mirror/proxy/Xet
download workaround, environment variables, uvicorn --workers 1, a systemd unit,
verification, remote exposure, capacity notes, and a troubleshooting table.
Link to it from README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 03:12:42 +00:00
lhk229 da4164e95a Support loading model weights from local project folders
Add bgfilter/weights.py: resolve_model_source() maps a HuggingFace repo id
to a local folder under the weights dir (models/ by default, override with
BGFILTER_WEIGHTS_DIR) when one named after the repo basename exists; otherwise
the repo id is returned unchanged. Opt-in and backward compatible.

- vitmatte_infer.py / segmentation.py resolve model_name through it; anime-seg
  reads <dir>/isnetis.onnx directly instead of hf_hub_download when local.
- .gitignore: models/, model-cache/
- README: document bundling weights as plain project folders.

Verified: anime-seg loads from a local models/anime-seg/ folder offline.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 02:50:19 +00:00
lhk229 7a87a00ca1 add benchmark script 2026-07-02 12:26:20 +00:00
lhk229 d24c6ed608 Add FastAPI HTTP service for background removal
Wrap the BGfilter matting pipeline in a long-running FastAPI service
(POST /remove-background, GET /healthz) exposing only screen_color and
seg_model, matching the birefnet-service request contract.

- bgfilter/pipeline.py: add in-memory run_rgb entry returning a
  MattingResult; allow injecting a shared runner/segmenter. CLI file
  path (run_image) behaviour is unchanged.
- bgfilter/service.py: PipelineManager caches one shared ViTMatte runner
  and per-backend segmenters, so screen_color/seg_model never reload a
  model.
- app.py: thin FastAPI layer -- param validation, image decode/encode,
  error codes (400/413/500), timing headers, global asyncio lock.
- requirements.txt: fastapi, uvicorn[standard], python-multipart.
- README + design doc: HTTP service usage and HF-mirror/proxy/Xet
  download notes.

Verified end-to-end (base ViTMatte + BiRefNet + anime-seg): 11/11
acceptance checks pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 12:08:25 +00:00
28 changed files with 3473 additions and 127 deletions
+2
View File
@@ -1,4 +1,6 @@
Samples/
Outputs/
models/
model-cache/
__pycache__/
*.py[cod]
+514
View File
File diff suppressed because it is too large Load Diff
+468
View File
@@ -0,0 +1,468 @@
# 将 BgFilter 部署到 Linux 服务器
> English version: [DEPLOY.md](DEPLOY.md)
本指南涵盖在 Linux 服务器上运行 BgFilter HTTP 服务(`POST /remove-background`
`GET /healthz`)所需的全部步骤。默认按 **CPU** 部署(推荐默认);涉及 GPU 的地方会单独标注。
服务入口是 `app.py``uvicorn app:app`)。API 契约和算法细节见 [README.md](README.md)。
下面第 1–12 节是**手动 CPU 部署**systemd + nginx,推荐默认)。如果你部署在
**AutoDL GPU 盒子**上,走第 0 节的一键脚本更快;两者是各自独立的部署路线。
---
## 0. 快速路径:AutoDL 一键部署(GPU)
在 AutoDL GPU 实例上,用仓库自带的一键脚本部署,无需手动装依赖 / 配 systemd:
```bash
# 假设仓库已 checkout 到实例(脚本不会 git pull),使用 server-edition 分支
cd /root/BGfilter-server # 你的实际路径
scripts/deploy_autodl.sh --token <TOKEN>
```
脚本**幂等**地完成:装依赖 + 拉模型权重 + 按 CUDA 版本装对应 CuPyGPU 前景合成)
+ 起一个 **supervisord** 实例(崩溃自动重启)+ 往 `~/.bashrc` 写交互登录自启,
最后启动服务并校验 `/healthz`
参数:
| 参数 | 默认 | 说明 |
| --- | --- | --- |
| `--token <TOKEN>` | 无 | 鉴权 token,设置/更新后 `/remove-background` 要求 header `X-Genarrative-Image-Token`。**省略则沿用已有 token**(重跑改别的参数不会误关鉴权);只有首次无 token 才开放。 |
| `--open` | — | 显式删除 token、以开放模式运行(无鉴权)。 |
| `--port <PORT>` | `6006` | 容器内监听端口(对应 AutoDL 的公网映射端口)。 |
| `--config <cfg>` | `configs/gpu.yaml` | 流水线配置。 |
| `--log-dir <dir>` | `~/autodl-tmp` | 日志目录(默认 AutoDL 持久数据盘——容器重置不丢,`/tmp` 会丢)。含 `bgfilter.log`(散文事件日志)和 `bgfilter-access.jsonl`(结构化每请求日志,应用自轮转)。 |
**前提**:仓库已 checkout(脚本不 `git pull`)、root 用户、torch/CUDA 由 AutoDL 镜像提供、
python 在 `/root/miniconda3`
**两个 AutoDL 固有的坑**(脚本管不了):
- **公网 URL 映射**(如 `6006 → https://...:8443`)要在 **AutoDL 网页控制台**里设,脚本不负责。
- **容器重启后**服务不会自动起来——AutoDL 没有 systemd / 开机钩子,自启写在 `~/.bashrc` 里,
需要**开一个交互式 shellSSH / JupyterLab 终端)** 才会触发拉起。
**服务管理**supervisorctl 与 python 同目录;socket 在 `~/bgfilter-supervisor.sock`
supervisord 配置在 `~/supervisord.conf`):
```bash
S="/root/miniconda3/bin/supervisorctl -c $HOME/supervisord.conf"
$S status bgfilter # 状态 + 运行时长
$S restart bgfilter # 重启(如改了 configs/*.yaml 之后)
$S stop bgfilter # 停服务(supervisord 本身继续跑)
$S start bgfilter # 再启动
$S shutdown # 全部停掉,包括 supervisord 自己
tail -f ~/autodl-tmp/bgfilter.log # 事件日志:启动/告警/堆栈(散文)
tail -f ~/autodl-tmp/bgfilter-access.jsonl # 访问日志:每请求一条 JSON 记录
tail -f ~/supervisord.log # supervisord 自己的日志(服务起不来时看)
```
若 supervisord 本身没在跑——容器刚重启还没开过 shell,或报 socket 残留错误——照
`~/.bashrc` 自启的方式手动拉起:
```bash
rm -f ~/bgfilter-supervisor.sock ~/supervisord.pid
/root/miniconda3/bin/supervisord -c ~/supervisord.conf
```
`~/supervisord.conf` 本身(端口、日志路径等)的方式是重跑部署脚本——它会重新生成
该文件并做 `reread + update + restart`
> 改了 `configs/*.yaml`(比如开 `cross_check.reuse_as_seg`)后,用上面的 `restart bgfilter`
> 让服务重新加载配置——配置只在启动时读入,没有请求级开关。
---
## 1. 前置条件
- Linux x86-64(在 Ubuntu 上验证过;其它主流发行版同理)。
- 推荐 **Python 3.113.13**。更新的版本(如 3.14)也可能可用,但部分 ML 包可能还没发布对应
wheel——尽量选一个 `torch` / `onnxruntime` / `opencv` / `pymatting` 都有预编译 wheel 的版本。
- `git`,以及 venv 模块(Debian/Ubuntu 上是 `python3-venv`)。
- 磁盘空间:Python 环境约 3 GB + 模型权重约 1 GB。
- **一次性**联网即可(用于装依赖和下模型);服务跑起来后可完全离线。
```bash
sudo apt update
sudo apt install -y python3-venv python3-pip git
```
## 2. 获取代码
```bash
sudo mkdir -p /opt/genarrative-image-host/bgfilter-service
sudo chown "$USER" /opt/genarrative-image-host/bgfilter-service
cd /opt/genarrative-image-host/bgfilter-service
git clone <repo-url> . # 或把仓库拷到这里;使用 server-edition 分支
```
## 3. Python 环境
```bash
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip
```
## 4. 安装依赖(CPU
**先从 CPU wheel 源装 PyTorch**——否则 `pip` 会拉默认的 CUDA 构建,附带约 2–3 GB 的
`nvidia-*` 包,CPU 机器完全用不上:
```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
```
`requirements.txt` 里已经固定用 **`opencv-python-headless`**GUI 构建依赖 `libGL.so.1`
等无头服务器没有的库),所以无需改动。直接装其余依赖——此时 `torch`/`torchvision`
已满足,不会再拉 CUDA
```bash
pip install -r requirements.txt
```
验证整套依赖可导入、且 torch 是 CPU 构建(`cuda= None`):
```bash
python -c "import torch,cv2,fastapi,transformers,pymatting,onnxruntime,kornia,timm; \
print('torch', torch.__version__, 'cuda=', torch.version.cuda)"
```
> 在墙内,给第二条 `pip install` 加上 `-i https://pypi.tuna.tsinghua.edu.cn/simple` 提速。
## 5. 准备模型权重
默认用到三个模型:`hustvl/vitmatte-base-composition-1k`(抠图)、
`ZhengPeng7/BiRefNet`(默认分割)、`skytnt/anime-seg`(可选分割)。
第四个模型 `ZhengPeng7/BiRefNet_HR-matting`(约 425 MB)只有在启用 **cross-check(交叉校验)
否决**时才需要——见本节末尾说明。
**一键方式(推荐)**:跑仓库自带脚本(无参数),经 hf-mirror 把所有模型下到 `models/`
(绕过代理、禁用 Xet、已存在的文件夹自动跳过)。跑完主 app 即离线全功能可用:
```bash
python scripts/fetch_weights.py
```
下面两种手动放法用于更精细的控制(比如共享 HF 缓存、只下部分模型),**二选一**。
### 方案 A —— HuggingFace 缓存 + 离线模式(推荐)
先下载到一个缓存目录,之后离线运行:
```bash
export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
hf download hustvl/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet
hf download skytnt/anime-seg
hf download ZhengPeng7/BiRefNet_HR-matting # 仅在使用 cross-check 时(见下)
```
运行时把 `HF_HOME` 指向同一路径,并设 `HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1`(见 §6)。
也可以在另一台机器下好,把 `hub/` 目录整个拷过来。
### 方案 B —— 权重作为普通文件夹放进项目
把每个模型放进 `models/<repo末段名>/`,加载器会直接用它(不走 HF 缓存、不联网)。
没放对应文件夹时会回退到 repo id,所以这是可选功能。
**文件夹名必须和 repo 末段名完全一致**——加载器靠这个找到它
`ZhengPeng7/BiRefNet``models/BiRefNet`)。各模型需要的文件:
```text
<项目根目录>/
└── models/
├── vitmatte-base-composition-1k/ # ViTMatte 抠图模型
│ ├── config.json
│ ├── preprocessor_config.json
│ └── pytorch_model.bin # 约 369 MB
├── BiRefNet/ # 默认分割模型(trust_remote_code
│ ├── config.json
│ ├── BiRefNet_config.py # 自定义代码——trust_remote_code 必需
│ ├── birefnet.py # 自定义代码——trust_remote_code 必需
│ └── model.safetensors # 约 424 MB
├── anime-seg/ # 可选分割模型(seg_model=anime-seg
│ └── isnetis.onnx # 约 168 MB
└── BiRefNet_HR-matting/ # 可选——仅用于 cross-check 否决
├── config.json
├── BiRefNet_config.py # 自定义代码——trust_remote_code 必需
├── birefnet.py # 自定义代码——trust_remote_code 必需
└── model.safetensors # 约 425 MB
```
说明:
- **ViTMatte**:需要 `config.json``preprocessor_config.json` 和权重
`pytorch_model.bin`;若有 `model.safetensors` 也可)。
- **BiRefNet**:两个 `.py` 文件是必需的——`trust_remote_code` 会执行它们来构建模型。
`config.json` + `model.safetensors` 放在一起。(repo 里的 `handler.py` /
`requirements.txt` / `README.md` 用不到。)
- **anime-seg**:只读 `isnetis.onnx`,其它都不需要。
- `anime-seg``BiRefNet_HR-matting` 都是可选的——不用 `seg_model=anime-seg` /
cross-check 否决时,各自的文件夹可以不放。
- **`cpu-fast` lane**(§6)只加载 **ViTMatte + BiRefNet**;8 GB 机器用这条 lane 时,
`anime-seg``BiRefNet_HR-matting` 都可以完全不放。
`scripts/fetch_weights.py` 仍会把 4 个都下齐,以保证完整功能。)
最省事的填充方式是用 `hf download`(它也会顺带拉几个用不到的文件如 `README.md`,无害):
```bash
hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet --local-dir models/BiRefNet
hf download skytnt/anime-seg --local-dir models/anime-seg
hf download ZhengPeng7/BiRefNet_HR-matting --local-dir models/BiRefNet_HR-matting # 仅 cross-check
```
`BGFILTER_WEIGHTS_DIR` 可改基目录。`models/` 已被 gitignore。
### cross-check 否决(默认开启)
管线会再跑一个 matting 模型(`ZhengPeng7/BiRefNet_HR-matting`,约 425 MB)作为独立的
"第二意见"——它做什么、为什么,见 [README_ZH.md](README_ZH.md) → *管线*。部署侧要点:
- **CLI 和 HTTP 服务都默认开启**,因此**必须备好这个模型**(上面方案 A 或 B)。
⚠️ 离线服务器(`HF_HUB_OFFLINE=1`)必须在第一个请求前就备好它,否则每个默认请求都失败。
- 服务级关闭:`BGFILTER_CROSS_CHECK=0`;按请求覆盖:表单字段 `cross_check`
开启时服务启动即预载。
- 配置里 `cross_check.reuse_as_seg: true`(默认**关**)让这次前向兼作分割掩码:
少载一个模型、CPU 快约 20 秒/张、GPU 省约 1 GB 显存。仅配置文件可设——改动需重启。
### 背景模式(flat vs complex
`background_mode=complex`(非纯色/实景背景)**不需要额外权重**——复用已部署的
BiRefNet 分割器 + ViTMatte,跳过色键相关阶段。cross-check 否决在 complex 模式下
**默认关闭**(传 `cross_check=on` 启用)。字段语义与响应头见
[README_ZH.md](README_ZH.md) → *HTTP 服务*
### 墙内下载(国内网络)
若直连 HuggingFace 被墙,用镜像 **并绕过本地代理、禁用 Xet**——这是稳定可用的组合:
```bash
export HF_ENDPOINT=https://hf-mirror.com # 国内镜像
export NO_PROXY='*' # 绕过代理;镜像是国内直连
export HF_HUB_DISABLE_XET=1 # 这些 repo 是 Xet 存储;强制走经典 HTTP
```
这样能避开两个坑:(1) 镜像 + 走海外的代理会让镜像把 `resolve` 又重定向回
`huggingface.co`,新版 `huggingface_hub` 会因此报 `FileMetadataError`
(2) 装了 `hf-xet` 时 Xet 下载路径在此环境下会瞬间失败。
## 6. 配置(环境变量)
| 变量 | 默认 | 作用 |
| --- | --- | --- |
| `BGFILTER_CONFIG` | `configs/default.yaml` | 管线配置文件 |
| `BGFILTER_DEVICE` | `cpu` | 同时覆盖 model 和 segmentation 的 deviceGPU 用 `cuda`|
| `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | 超过则返回 `413`(约 4 MP|
| `BGFILTER_ACCESS_LOG` | — | **结构化访问日志**的路径(JSONL,每请求一条记录 + 一条 `startup` 元记录;应用自轮转,50 MB × 5)。字段:`request_id`、参数、尺寸、分阶段耗时(`t_queue/fetch/decode/process/total_ms`)、状态码、失败时截断的 `exc`;该 id 同时以 `X-Request-ID` 响应头返回。不设置 = 记录回退到 stderr(开发用)。散文**事件日志**(启动/告警/堆栈)始终走 stderr 由 supervisord/journald 捕获——每请求数据绝不落在散文流里。 |
| `BGFILTER_LOG_LEVEL` | `INFO` | 根日志级别(事件流)。 |
| `BGFILTER_FETCH_MAX_BYTES` | `26214400` | `image_url` 模式:下载体上限(约 25 MB)→ `413`。无 SSRF 过滤——务必开鉴权 / 调用方可信。 |
| `BGFILTER_FETCH_TIMEOUT` | `15` | `image_url` 模式:fetch 读超时(秒;连接超时固定 5 秒)。出站 fetch 并发固定为 `2 × cpu_workers`(硬顶 64)。 |
| `BGFILTER_PRELOAD` | `1` | 启动时预加载默认模型(首个请求不用冷加载)|
| `BGFILTER_CROSS_CHECK` | 配置(`on`| 强制开/关 cross-check 否决(`0` 关闭;需 `BiRefNet_HR-matting`|
| `BGFILTER_CPU_WORKERS` | 配置 `server.cpu_workers``1`| 请求流水线的线程池大小。`1` = 严格串行(一次一个请求)。`>1` 允许这么多请求并发,重叠各自的 CPU 段,同时用一把 GPU 互斥锁串行化模型前向——让 GPU 别闲着。CPU 为瓶颈时(如关 cross-check)收益最大。可写进配置的 `server:` 段,或用此 env 变量按机器覆盖。 |
| `BGFILTER_GPU_CONCURRENCY` | 配置 `server.gpu_concurrency``1`)| 允许几个请求同时进入 GPU 前向。`1` = 单流(安全)。`2` 让两次前向在 SM 上共同调度(batch-1 喂不满 SM),提升 GPU 瓶颈下的吞吐——但并发激活会成倍占显存,按 `权重 + N × 单次前向峰值 ≤ 显存` 来定(32GB 卡上 `2` 在 2048 cross-check 时峰值约 19GB;≤16GB 卡保持 `1`)。需 `cpu_workers>1` 才有意义。配置键 `server.gpu_concurrency`,此 env 变量覆盖它。 |
| `BGFILTER_AUTH_TOKEN` | — | 若设置,`/remove-background` 要求 header `X-Genarrative-Image-Token` 等于它(常量时间比较;否则 `401`)。不设 = **开放**(鉴权交给前置代理)。`/healthz` 始终开放。见 §10。 |
| `BGFILTER_WEIGHTS_DIR` | `models` | 本地权重文件夹的查找目录(方案 B)|
| `HF_HOME` | — | HF 缓存位置(方案 A)|
| `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | 权重缓存好后,生产设为 `1` |
| `OMP_NUM_THREADS` / `MKL_NUM_THREADS` | — | 限制 CPU 线程数(如 `4`|
### 配置文件:`default.yaml` 与 `cpu-fast` lane
`BGFILTER_CONFIG` 决定用哪个管线配置。仓库自带两个:
- **`configs/default.yaml`** —— 完整画质。cross-check 否决**默认开**(额外加载
`BiRefNet_HR-matting`@2048 作第二意见),边缘最好。较重:大图时峰值工作集轻松到
两位数 GB,需要内存宽裕的机器(或 GPU)。
- **`configs/cpu-fast.yaml`** —— 精简 CPU lane,为 **8 GB 机器**调校。fp32、只用 BiRefNet
分割、**无 cross-check、无 anime-seg**、分块注意力(`attn_query_chunk: 512`)。只加载
**两个**权重(base ViTMatte + BiRefNet)。实测峰值工作集:**2048×11522.4 MP)约 5.6 GB**
2048×20484.2 MP)约 7.1 GB;约 1415 秒/张(1.5 MP)。
> **≤8 GB 的服务器请设 `BGFILTER_CONFIG=configs/cpu-fast.yaml`。** 默认 lane + 大图
> 正是把 ViTMatte 注意力瞬时内存推到 OOM 的组合——worker 被 killSIGKILL),前面的
> nginx 返回 **502**,而应用日志显示 `DefaultCPUAllocator: can't allocate memory`。
有两个模型旋钮放在**配置文件**里(`model:` 段),不是环境变量:
| 配置项 | 默认 | 作用 |
| --- | --- | --- |
| `model.precision` | `fp32` | `fp32``bf16`。bf16 **仅**在有原生内核的硬件上启用(CUDA,或带 AVX512-BF16/AMX 的 CPU);否则告警并回退 fp32。也可在 `segmentation:` / `cross_check:` 下按模型单独设。 |
| `model.attn_query_chunk` | `2048` | ViTMatte 全局注意力的 query 行分块。调小(如 `512`)以几个百分点的额外算力换掉 O(N²) 内存瞬时尖峰;输出逐比特相同。设 `0` 关闭分块(走原始一次性注意力)。 |
> **despill 默认关闭**(所有 lane,含 cpu-fast)。它沿背景色调方向压前景色度,没有位置/语义护栏,所以主体与背景**同色相**时(如蓝背景前的蓝西装)会被褪色/偏色(约 ΔE 35)。仅当"去边缘溢色"比"同色系保真"更重要时,才在配置里 `despill: {enabled: true}` 单独开启。
## 7. 运行服务
```bash
BGFILTER_ACCESS_LOG=logs/bgfilter-access.jsonl \
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1 --no-access-log
```
- **必须保留 `--workers 1`。** 每个 worker 各加载一整套模型,多 worker 会成倍占内存
(8 GB 机器会 OOM)。服务已用全局锁串行化推理。
- `--host 127.0.0.1` 只对本机开放(面向同机调用方)。跨机访问见 §10。
- `--no-access-log` 关掉 uvicorn 自己的散文版每请求日志——它是应用结构化访问记录
(§6 `BGFILTER_ACCESS_LOG`)的子集、纯重复,且每请求数据不该进散文流。
## 8. 作为 systemd 服务运行
`/etc/systemd/system/genarrative-bgfilter.service`
```ini
[Unit]
Description=Genarrative BgFilter background removal service
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
Environment=HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
Environment=HF_HUB_OFFLINE=1
Environment=TRANSFORMERS_OFFLINE=1
Environment=HF_HUB_DISABLE_XET=1
Environment=BGFILTER_CONFIG=configs/default.yaml
Environment=BGFILTER_DEVICE=cpu
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
Environment=BGFILTER_PRELOAD=1
Environment=BGFILTER_ACCESS_LOG=/opt/genarrative-image-host/bgfilter-service/logs/bgfilter-access.jsonl
Environment=OMP_NUM_THREADS=4
Environment=MKL_NUM_THREADS=4
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1 --no-access-log
Restart=on-failure
RestartSec=5
MemoryMax=8G
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/opt/genarrative-image-host
[Install]
WantedBy=multi-user.target
```
≤8 GB 机器请把配置那行改成精简 lane:
`Environment=BGFILTER_CONFIG=configs/cpu-fast.yaml`(见 §6)——默认 lane 大图会 OOM。
若用方案 B(本地 `models/` 文件夹)而非 HF 缓存,去掉 `HF_HOME` 和离线那几行——
那种权重不需要缓存也不联网。
```bash
sudo systemctl daemon-reload
sudo systemctl enable --now genarrative-bgfilter
sudo systemctl status genarrative-bgfilter
journalctl -u genarrative-bgfilter -f # 等到 "Application startup complete"
```
## 9. 验证
```bash
curl -s http://127.0.0.1:18083/healthz
# {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu"}
curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
http://127.0.0.1:18083/remove-background -o out.png
# 非纯色/实景背景(无色键;此模式 cross-check 默认关)
curl -sS -F "file=@scene.jpg" -F "background_mode=complex" \
http://127.0.0.1:18083/remove-background -o out.png
```
## 10. 对外暴露
单元里绑的是 `127.0.0.1`——面向同机调用方(例如 Genarrative Rust BFF,通过
`GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL` 调用)。要从其它主机访问:
- **反向代理(推荐):** 前面放 nginx/caddy,终止 TLS、加限流,转发到
`127.0.0.1:18083`
- **直接暴露:** 把 `--host` 改成 `0.0.0.0`,并用防火墙限制来源 IP。
切勿把无鉴权的服务裸暴露到公网(见下方鉴权)。
### 鉴权
app 层 token 鉴权是**可选开启**,由 `BGFILTER_AUTH_TOKEN` 控制:
- **设置它** → `/remove-background` 要求 header `X-Genarrative-Image-Token: <token>`
(常量时间比较);缺失或错误返回 `401``/healthz` 保持开放。**当 app 被直接暴露**
`--host 0.0.0.0`、或云端端口映射无代理)时用这个。
- **不设置** → app **开放**(fail-open),并在启动时打一条 WARNING。仅当前置代理
已经鉴权时才可这样——例如生产机 nginx 自己校验 `X-Genarrative-Image-Token`
其后的 app 就无需 token。
header 名和 nginx 层一致,所以调用方只换 base URL + token 值,请求结构不变:
```bash
curl -H "X-Genarrative-Image-Token: <token>" \
-F "file=@input.png" \
https://<host>/remove-background -o out.png
```
## 11. 容量与性能
- CPU 吞吐:**`cpu-fast` lane** 约 **1415 秒/张(1.5 MP**base ViTMatte + 一个
BiRefNet@1024,分块注意力)。**默认 lane** 更慢——cross-check 会多跑一次 HR-matting@2048
前向。请求被全局锁串行,单实例按每分钟几张估算。
- **默认 lane,工作站 CPU 参考(Ryzen 9700X,原生 bf1632 GBcross-check 开且
`reuse_as_seg` 开,分块注意力):**
| 输入 | 精度 | warm / 每图 | 峰值内存 |
|---|---|---|---|
| 1024×1536 | fp32 | ~45 s | — |
| 1024×1536 | bf16 | ~33 s | — |
| 2048×2048 | bf16 | ~53 s | 11.4 GB(批处理)、8.1 GB(单图) |
不开 `reuse_as_seg` 要加上主 BiRefNet@1024 前向(每图约 +20 s)。内存上限来自
cross-check 的 HR 前向(无论图多大都固定 `input_size` 2048)。分块注意力之前,
同一个 2048² bf16 run 峰值 25.2 GB;完全未优化的 fp32 估计要 55–60 GB。无原生
bf16 的 CPU(如 Zen 2)自动回落 fp32——这类机器上内存或延迟紧张时优先
`--no-cross-check`
- 要提吞吐:上 GPU`BGFILTER_DEVICE=cuda`,装 CUDA 版 torch),大约快一个数量级;
和/或多实例 + 负载均衡(每个都 `--workers 1`)。
- **GPU 部署(实测,RTX 5070 Ti 16 GB,完整管线——cross-check 开,bf16):**
约 1–2 秒/张(4 MP 也只要 2 秒),峰值 VRAM 分配约 10.9 GB(开 `reuse_as_seg`
为 10.1 GB)。真正决定「装不装得下」的是分配器的 *reserved* 占用。`bgfilter`
在 Linux/WSL2 上会**默认设置** `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`
(在 torch 加载前;原生 Windows 不支持该选项;显式设过的环境变量优先)。
实测效果:reserved 15.3 GB → **12.0 GB**(开 reuse;默认不开为 13.3 GB),且还
**快约 10%**——完整管线在 16 GB 卡上即使旁边挂着桌面会话也有约 2–3.5 GB 余量。
没有 expandable segments(如原生 Windows)时 reserved 约 15.316.2 GB:在
16 GB 卡上从很悬到直接溢出。另外 `bf16` 是 16 GB 可行的前提——fp32 的 reserved
约 17 GB,直接溢出。
- 内存:分块注意力(`attn_query_chunk`)把 ViTMatte 的 O(N²) 瞬时尖峰削平,于是峰值由
稠密计算而非注意力矩阵决定。`cpu-fast` lane 实测峰值 **约 5.6 GB@2.4MP / 7.1 GB@4.2MP**——
配合默认 4 MP 输入上限,`MemoryMax=8G` 对单 CPU worker 是稳妥上限。默认 lanecross-check 开)
需要更多余量。
- **让 RSS 保持平稳(glibc)。** 每处理完一张图,服务会调用 glibc 的 `malloc_trim(0)`
`bgfilter/memtune.py`)把 BiRefNet@2048 前向释放的大块激活还给 OS——否则 ptmalloc
会把它们留在 arena 里,RSS 会随请求逐步抬高。想进一步降内存**并**提升 CPU 吞吐,可以
预载 **jemalloc**PyTorch 对 CPU 推理的推荐),并让它激进归还脏页:
```ini
# 放进 systemd unit 的 [Service] 段
Environment=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
Environment=MALLOC_CONF=background_thread:true,dirty_decay_ms:0,muzzy_decay_ms:0
```
`apt install libjemalloc2`tcmalloc 用 `libtcmalloc_minimal.so.4` 同理。)
jemalloc/tcmalloc 自己管理归还,此时 `malloc_trim` 会自动空转。若无法预载分配器,
在 unit 环境里设 `MALLOC_ARENA_MAX=2`、`MALLOC_TRIM_THRESHOLD_=0` 也能抑制 ptmalloc 滞留。
## 12. 故障排查
| 现象 | 原因 / 解决 |
| --- | --- |
| `pip` 拉了 `nvidia-*`/`cuda-*`(巨大) | §4 里漏了 CPU 源。用 `--index-url https://download.pytorch.org/whl/cpu` 装 torch。 |
| `import cv2` 报 `ImportError: libGL.so.1` | 在无 GUI 机器上用了 `opencv-python`。换成 `opencv-python-headless`(§4)。 |
| 建 venv 时 `ensurepip is not available` | 装 `python3-venv`(§1)。 |
| 下载时 `FileMetadataError: ... not on huggingface.co` | 镜像与代理冲突。绕过代理(`NO_PROXY='*'`)并设 `HF_HUB_DISABLE_XET=1`(§5)。 |
| `LocalEntryNotFoundError`、0 字节 | Xet 下载路径失败;设 `HF_HUB_DISABLE_XET=1`。 |
| `503 service not ready` | 启动时模型还在加载;等 "Application startup complete"。 |
| 刚设 `BGFILTER_CROSS_CHECK` 后(离线)报 `500 inference failed` | `BiRefNet_HR-matting` 没备好;下到缓存/`models/`(§5)。 |
| 负载下 OOM / 被 kill | worker 多于一个,或 `MemoryMax` 太低。保持 `--workers 1`。 |
| 大图时 worker 被 `kill`SIGKILL/ 返回 `502`,日志显示 `DefaultCPUAllocator: can't allocate memory` | 大图触发 ViTMatte OOM。改用 `BGFILTER_CONFIG=configs/cpu-fast.yaml`(§6),和/或调小 `BGFILTER_MAX_IMAGE_PIXELS`。 |
| 大文件上传在代理层被 `413` 拒 | nginx `client_max_body_size`(默认 1 MB)——在 server 块里调大(如 `client_max_body_size 12m;`);这和应用的 `BGFILTER_MAX_IMAGE_PIXELS` 是两道独立限制。 |
| `400 could not auto-detect background colour` | 输入没有干净的纯色边框;传 `screen_color=#RRGGBB`。 |
+153 -63
View File
@@ -1,5 +1,7 @@
# BgFilter
> 中文: [README_ZH.md](README_ZH.md)
Offline character matting for AI-generated images on a flat-colour background.
The background colour is **auto-detected** from the image border (green, pastel, any
flat colour); pass `--screen-color` to set it explicitly.
@@ -38,15 +40,15 @@ download (~0.9 GB) and one inference pass per image; disable with
`--no-cross-check` (see the `cross_check` config section, and
`docs/hair_gap_artifacts.md` for the analysis behind it).
When the cross-check is on, that same HR-matting forward is **reused as the
segmentation mask by default**, skipping the primary seg model entirely (one
less model to load, ~20 s faster per image on CPU). Pilot-validated
(TestImage3 / FixImage1): trimap 99.8% identical, no structural change to
fingers, hair wisps or thin lines. Disable with `--no-cross-check-as-seg` to
run the dedicated seg model instead. The reuse is a same-family swap, so it
When the cross-check is on, that same HR-matting forward can optionally be
**reused as the segmentation mask** (`--cross-check-as-seg`), skipping the
primary seg model entirely (one less model to load, ~20 s faster per image on
CPU, ~1 GB less VRAM on GPU). Pilot-validated (TestImage3 / FixImage1): trimap
99.8% identical, no structural change to fingers, hair wisps or thin lines.
**Off by default**: the dedicated segmenter keeps its own forward and the veto
stays an independent second signal. The reuse is a same-family swap, so it
applies to the `birefnet` backend only: with `--seg-backend anime-seg` the
anime segmenter keeps its own forward and the cross-check veto still runs
independently on top.
anime segmenter always keeps its own forward.
The segmentation trimap defaults to `directional` mode (chroma + seg + a hue-direction
split: it keeps a background-coloured garment such as a white shirt while dropping a
@@ -63,7 +65,7 @@ frame — detection **fails with an error**; pass `--screen-color` explicitly in
To set it yourself, give a hex prior:
```powershell
```bash
... --screen-color "#CFEFFF"
```
@@ -74,26 +76,24 @@ auto-detected colour is used directly.)
## Environment
Use the conda environment `lightML`.
Any Python 3.113.13 environment (venv or conda) works:
```powershell
conda activate lightML
```bash
pip install -r requirements.txt
python -m bgfilter.cli --help
```
If the shell is not activated, call the environment Python directly:
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli --help
```
For a server install — CPU-wheel torch, offline weights, systemd — follow
[DEPLOY.md](DEPLOY.md) instead; it covers the same dependencies with the
production choices spelled out.
## Single Image
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
--input Samples\TestImage.png `
--output Outputs\TestImage_rgba.png `
--debug-dir Outputs\TestImage_debug
```bash
python -m bgfilter.cli \
--input Samples/TestImage.png \
--output Outputs/TestImage_rgba.png \
--debug-dir Outputs/TestImage_debug
```
No `--config` is needed — the built-in defaults are identical to
@@ -110,32 +110,19 @@ auto-detect the flat background), and `--trimap-mode`.
## Batch
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
--input-dir Samples `
--output-dir Outputs `
--debug-dir Outputs\debug
```bash
python -m bgfilter.cli \
--input-dir Samples \
--output-dir Outputs \
--debug-dir Outputs/debug
```
## Performance (CPU reference numbers)
## Performance
Measured on a Ryzen 9700X (Zen 5, native bf16), 32 GB RAM, default settings
(cross-check on, reused as seg; chunked global attention on):
| input | precision | warm / image | peak memory |
|---|---|---|---|
| 1024x1536 | fp32 | ~45 s | — |
| 1024x1536 | bf16 | ~33 s | — |
| 2048x2048 | bf16 | ~53 s | 11.4 GB (batch), 8.1 GB (single image) |
The memory ceiling is the cross-check BiRefNet_HR forward (fixed
`input_size` 2048 regardless of the image size) — genuine live activations
of a full-resolution dense prediction net. Before chunked attention and the
mimalloc fix, the same 2048x2048 bf16 run peaked at 25.2 GB; a fully
un-optimized fp32 run would need an estimated 55-60 GB (ViTMatte's un-chunked
N^2 attention alone ~39 GB). CPUs without native bf16 (e.g. Zen 2) auto-fall
back to fp32 — on such machines prefer `--no-cross-check` if memory or
latency is tight.
Rough orders of magnitude: **~3353 s/image** on a fast desktop CPU (bf16,
cross-check on), **~12 s/image** on a modern GPU. All measured numbers — CPU
lanes, GPU VRAM footprints, memory ceilings and the knobs that move them — live
in [DEPLOY.md](DEPLOY.md) §11 (*Capacity & performance*).
## Chroma-alpha debug mode
@@ -143,12 +130,12 @@ latency is tight.
alpha seed. Useful for fast inspection of chroma confidence, trimap, and despill.
(Distinct from the chroma-only *pipeline* above, which still runs ViTMatte.)
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
--input-dir Samples `
--output-dir Outputs\chroma `
--debug-dir Outputs\chroma_debug `
--matting-method chroma `
```bash
python -m bgfilter.cli \
--input-dir Samples \
--output-dir Outputs/chroma \
--debug-dir Outputs/chroma_debug \
--matting-method chroma \
--device cpu
```
@@ -172,37 +159,140 @@ qa_grid.png
metadata.json
```
## HTTP service
An HTTP wrapper (`app.py` + `bgfilter/service.py`) exposes the pipeline as a
long-running FastAPI service. Models load once and are reused across requests.
For deploying to a Linux server (dependencies, weights, systemd, offline mode, auth),
see [DEPLOY.md](DEPLOY.md).
```bash
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
```
Run a single worker (`--workers 1`) — each worker loads its own copy of the models.
### Endpoints
- **`GET /healthz`** — liveness + config JSON, e.g.
`{"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu","crossCheck":true}`.
Always unauthenticated.
- **`POST /remove-background`** — `multipart/form-data` in, `image/png` (RGBA) out.
### Authentication
Optional, opt-in via the `BGFILTER_AUTH_TOKEN` env var. When set, `/remove-background`
requires header **`X-Genarrative-Image-Token: <token>`** (constant-time compare); a
missing or wrong token returns `401`. When unset the endpoint is open (a fronting proxy
is expected to authenticate). `/healthz` is always open. See [DEPLOY.md](DEPLOY.md) §10.
### Request fields (`POST /remove-background`, `multipart/form-data`)
| Field | Required | Values | Meaning |
| --- | --- | --- | --- |
| `file` | one of `file`/`image_url` | image file | Source image as an upload (field name fixed as `file`). |
| `image_url` | one of `file`/`image_url` | `http(s)://…` | Source image by URL — the service fetches it (bounded by size/time; follows ≤3 redirects). Provide exactly one of `file` or `image_url`. No SSRF filtering: the endpoint trusts authenticated callers, so keep auth on and callers trusted. |
| `background_mode` | no | `flat` (default) / `complex` | `flat` = solid-colour background (colour-keyed pipeline). `complex` = non-flat/scene background: no colour key, the segmenter alone drives the matte. `screen_color` is ignored in `complex`; expect flat backgrounds to stay stronger on hair detail. |
| `screen_color` | no | `#RRGGBB`, or omit/empty/`auto` | Background-colour prior; omit to auto-detect from the border. Ignored when `background_mode=complex`. |
| `seg_model` | no | `birefnet` (default) / `anime-seg` | Segmentation backend. |
| `cross_check` | no | `1`/`0` (also `true`/`false`/`yes`/`no`/`on`/`off`) | Per-request override of the cross-model veto; omit/empty = service default in `flat` mode, but **defaults off** in `complex` mode (set `on` to enable its hue-free gate). |
### Response headers (on `200`)
| Header | Example | Meaning |
| --- | --- | --- |
| `X-BGFilter-Elapsed-Ms` | `2372` | Server-side processing time (ms). |
| `X-BGFilter-Seg-Model` | `birefnet` | Segmenter actually used. |
| `X-BGFilter-Screen-Color` | `#CFEFFF` / `auto` | Background colour used. |
| `X-BGFilter-Cross-Check` | `on` / `off` | Effective cross-check state. |
| `X-BGFilter-Background-Mode` | `flat` / `complex` | Background mode actually used. |
| `X-Request-ID` | `42efe4e0d76f4b83` | Correlation id (echoes an inbound `X-Request-ID` if you send one, else generated). Quote it to correlate with server logs. Returned on every response, including errors. |
### Status codes
| Code | When |
| --- | --- |
| `200` | Success — body is the RGBA PNG. |
| `400` | Bad input: unreadable image, not exactly one of `file`/`image_url`, bad `image_url` / fetch failed, invalid `screen_color`/`seg_model`/`cross_check`/`background_mode`, or auto-detect failed (pass `screen_color`). |
| `401` | Auth enabled and the token is missing or wrong. |
| `413` | Image exceeds `BGFILTER_MAX_IMAGE_PIXELS` (~4 MP by default), or an `image_url` body exceeds `BGFILTER_FETCH_MAX_BYTES` (~25 MB). |
| `500` | Inference failed. |
| `503` | Service still loading models (not ready). |
### Examples
```bash
# default (birefnet + auto background colour); add the auth header if the service requires it
curl -sS -F "file=@input.png" \
-H "X-Genarrative-Image-Token: <token>" \
http://127.0.0.1:18083/remove-background -o output.png
# explicit background colour + anime segmenter, cross-check off for this request
curl -sS \
-F "file=@input.png" \
-F "screen_color=#CFEFFF" \
-F "seg_model=anime-seg" \
-F "cross_check=off" \
http://127.0.0.1:18083/remove-background -o output.png
# non-flat / scene background (no colour key); cross-check defaults off here,
# add cross_check=on to enable the hue-free residue veto
curl -sS \
-F "file=@scene.jpg" \
-F "background_mode=complex" \
http://127.0.0.1:18083/remove-background -o output.png
# by URL instead of an upload (service fetches it; use image_url OR file, not both)
curl -sS \
-F "image_url=https://example.com/input.png" \
-F "screen_color=#CFEFFF" \
-H "X-Genarrative-Image-Token: <token>" \
http://127.0.0.1:18083/remove-background -o output.png
```
Runtime configuration (config file, device, size caps, preload, auth token,
pipelining workers) is all via `BGFILTER_*` environment variables — the full table
is [DEPLOY.md](DEPLOY.md) §6.
Like the CLI, the service runs the **cross-model veto by default**; `/healthz`
reports the effective default as `crossCheck` and the per-request `cross_check`
field overrides it either way (service-wide kill switch: `BGFILTER_CROSS_CHECK=0`).
`cross_check.reuse_as_seg` (config-only, default off — see *Pipelines* above) has
no per-request switch; changing it means editing the config and restarting.
## Quality Check
The quality checker measures alpha validity and edge spill on semi-transparent edge
pixels.
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.quality_cli `
Outputs\TestImage_rgba.png `
```bash
python -m bgfilter.quality_cli \
Outputs/TestImage_rgba.png \
--max-edge-green-excess-p95 0.30
```
Run the bundled sample smoke check:
```powershell
D:\MiniConda\envs\lightML\python.exe scripts\smoke_samples.py `
--samples-dir Samples `
--output-dir Outputs\smoke_samples `
--config configs\default.yaml `
--device cpu `
--fallback-to-chroma-alpha `
```bash
python scripts/smoke_samples.py \
--samples-dir Samples \
--output-dir Outputs/smoke_samples \
--config configs/default.yaml \
--device cpu \
--fallback-to-chroma-alpha \
--max-edge-green-excess-p95 0.30
```
## Notes
- `Samples/` and `Outputs/` are ignored by Git.
- ViTMatte and segmentation weights load from Hugging Face on first use. Behind a
firewall set `HF_ENDPOINT=https://hf-mirror.com` (and bypass a flaky local proxy).
- ViTMatte and segmentation weights load from Hugging Face on first use.
`anime-seg` (`skytnt/anime-seg`) is a plain ONNX download; `birefnet`
(`ZhengPeng7/BiRefNet`) ships custom modelling code so it needs
`trust_remote_code=True` plus `timm` / `einops` / `kornia`.
- **Provisioning weights** — offline HF cache, plain `models/` folders
(`BGFILTER_WEIGHTS_DIR`), the `scripts/fetch_weights.py` one-shot downloader,
and the CN-network mirror workaround — is covered in [DEPLOY.md](DEPLOY.md) §5.
- Foreground colour estimation uses pymatting's `estimate_foreground_ml` to propagate
clean foreground colour into semi-transparent edges before de-spill. Set
`foreground.method: unmix` to fall back to the legacy heuristic.
+276
View File
@@ -0,0 +1,276 @@
# BgFilter
> English: [README.md](README.md)
面向"纯色背景 AI 生成图"的离线人物抠图。背景色会**从图像边框自动探测**(绿色、
淡彩、任意纯色);也可用 `--screen-color` 显式指定。
```text
RGB 输入
-> chroma 背景置信度(按自动探测或指定的颜色打分)
-> [可选] 语义分割 mask
-> trimap -> ViTMatte -> alpha 清理
-> 交叉模型否决(对背景色调残留做第二意见)
-> pymatting 前景估计 -> despill 去溢色 -> RGBA PNG -> QA 预览
```
## 管线
两条管线,由配置里的 `segmentation.enabled` 选择:
- **仅 Chroma**(`enabled: false`)—— Chroma + ViTMatte,无分割模型。最轻/最快;
完全靠颜色键决定拓扑。
- **单分割器**(`enabled: true`,默认)—— 一个分割模型定 trimap 拓扑(镂空、发丝),
ViTMatte 再细化软边。后端可切换:`birefnet`(默认,通用显著目标——文字、logo、
照片;需 `trust_remote_code`)或 `anime-seg`(ONNX,专为动漫角色调优)。运行时用
`--seg-backend anime-seg` 切换(会同时选好对应权重)。
两条管线都共用 pymatting 前景估计和一次颜色 de-spill。**没有**染绿修复 / recolour
层——源图干净时它没必要,已移除。
两条管线**默认都跑交叉模型否决**:第二个、无需 trimap 的抠图模型
(`ZhengPeng7/BiRefNet_HR-matting`)只能*降低* alpha,且只作用于主结果自信、
但偏背景色调的亮像素——用来清掉卡在发丝间、被 chroma、分割器和 ViTMatte 都误判为
前景的偏色背景残留。代价是多下一个模型(~0.9 GB)+ 每图一次额外推理;用
`--no-cross-check` 关闭(见 `cross_check` 配置段,分析见 `docs/hair_gap_artifacts.md`)。
开启交叉否决时,那趟 HR-matting 前向可以选择性地**兼作分割 mask**
(`--cross-check-as-seg`),从而完全跳过主分割模型(少加载一个模型,CPU 上每图约快
20 s,GPU 上省约 1 GB 显存)。已试点验证(TestImage3 / FixImage1):trimap 99.8% 一致,
手指、发丝、细线无结构性变化。**默认关闭**:专用分割器保留自己的前向,否决作为独立
的第二信号。该复用是同族替换,故**仅**对 `birefnet` 后端生效:用
`--seg-backend anime-seg` 时动漫分割器始终保留自己的前向。
分割 trimap 默认走 `directional` 模式(chroma + seg + 色调方向切分:保留白衬衫这类
背景色系的衣物,同时丢弃卡在发丝间的偏背景色残留如蓝色)。可切换
`--trimap-mode seg`(仅拓扑,无色调切分)或 `directional-hard-bg`(激进——硬删偏背景
色像素;可能吃掉冷色/阴影下的白布)。
## 背景色
默认(`screen_color: null`)背景色**从图像边框自动探测**:边框条带的主导纯色成为键色。
若边框不是单一干净纯色——渐变、纹理、或主体占满画面——探测会**报错**;此时请显式传
`--screen-color`
要自己指定,给一个十六进制先验:
```bash
... --screen-color "#CFEFFF"
```
或在配置里写 `screen_color: "#CFEFFF"`。无论哪种,chroma 键都按到该颜色的感知
(Lab/RGB)距离给像素打分,de-spill 沿该颜色方向压掉色度。(显式给的 hex 会用附近边框
像素做微调;自动探测的颜色直接使用。)
## 环境
任意 Python 3.113.13 环境(venv 或 conda)均可:
```bash
pip install -r requirements.txt
python -m bgfilter.cli --help
```
服务器安装(CPU wheel 版 torch、离线权重、systemd)请直接按 [DEPLOY_ZH.md](DEPLOY_ZH.md)
走——同样的依赖,但把生产环境的取舍都写明了。
## 单张图
```bash
python -m bgfilter.cli \
--input Samples/TestImage.png \
--output Outputs/TestImage_rgba.png \
--debug-dir Outputs/TestImage_debug
```
无需 `--config`——内置默认值与 `configs/default.yaml` 完全一致。只有当你编辑了该文件
去调细节参数时,才需要 `--config configs\default.yaml`。运行时选择留在命令行:
`--device`(默认 **CPU**;GPU 用 `--device cuda`)、`--precision`(默认 `fp32`;`bf16`
给三个模型加速、并把抠图模型的激活显存减半,alpha 视觉上一致——需 bf16 硬件,否则回落
fp32;大输入还会默认对 ViTMatte 全局注意力按 query 分块——精确、逐比特相同,把 2048x2048
的内存尖峰从 ~19 GB 压到 ~4 GB,见 `model.attn_query_chunk`)、`--seg-backend`
(默认 `birefnet`;动漫角色用 `anime-seg`)、`--screen-color`(默认:自动探测纯背景)、
以及 `--trimap-mode`
## 批量
```bash
python -m bgfilter.cli \
--input-dir Samples \
--output-dir Outputs \
--debug-dir Outputs/debug
```
## 性能
量级参考:快速桌面 CPUbf16、cross-check 开)约 **3353 s/图**,现代 GPU 约
**12 s/图**。全部实测数据——CPU 各 lane、GPU 显存占用、内存上限及相关旋钮——
见 [DEPLOY_ZH.md](DEPLOY_ZH.md) §11(容量与性能)。
## Chroma-alpha 调试模式
`--matting-method chroma` 跳过 ViTMatte,直接用 chroma 置信度作为 alpha 种子。适合快速
检查 chroma 置信度、trimap 和 despill。(不同于上面的仅 Chroma *管线*,后者仍跑 ViTMatte。)
```bash
python -m bgfilter.cli \
--input-dir Samples \
--output-dir Outputs/chroma \
--debug-dir Outputs/chroma_debug \
--matting-method chroma \
--device cpu
```
## 输出
每处理一张图,CLI 写一张 RGBA PNG 和可选的调试文件:
```text
bg_confidence.png
trimap.png
seg_mask.png # 仅分割管线
alpha.png
foreground_rgb.png # 去溢色后的前景颜色
color_mask.png # 逐像素 despill 权重
preview_black.png
preview_white.png
preview_gray.png
preview_red.png
preview_blue.png
qa_grid.png
metadata.json
```
## HTTP 服务
一个 HTTP 封装(`app.py` + `bgfilter/service.py`)把管线暴露为常驻 FastAPI 服务。模型
只加载一次、跨请求复用。部署到 Linux 服务器(依赖、权重、systemd、离线模式、鉴权)见
[DEPLOY_ZH.md](DEPLOY_ZH.md)。
```bash
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
```
单 worker 运行(`--workers 1`)—— 每个 worker 各自加载一份模型。
### 端点
- **`GET /healthz`** —— 存活 + 配置 JSON,例如
`{"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu","crossCheck":true}`
始终免鉴权。
- **`POST /remove-background`** —— 入参 `multipart/form-data`,返回 `image/png`(RGBA)。
### 鉴权
可选、按需开启,由环境变量 `BGFILTER_AUTH_TOKEN` 控制。设置后,`/remove-background`
要求 header **`X-Genarrative-Image-Token: <token>`**(常量时间比较);缺失或错误返回
`401`。不设时端点开放(预期由前置代理鉴权)。`/healthz` 始终开放。见
[DEPLOY_ZH.md](DEPLOY_ZH.md) §10。
### 请求字段(`POST /remove-background`,`multipart/form-data`)
| 字段 | 必填 | 取值 | 含义 |
| --- | --- | --- | --- |
| `file` | `file`/`image_url` 二选一 | 图片文件 | 以上传方式提供源图(字段名固定为 `file`)。 |
| `image_url` | `file`/`image_url` 二选一 | `http(s)://…` | 以 URL 提供源图,服务端自行 fetch(受大小/超时限制,跟随 ≤3 跳重定向)。`file``image_url` 必须恰好提供一个。无 SSRF 过滤:端点信任已鉴权的调用方,请保持鉴权开启且调用方可信。 |
| `background_mode` | 否 | `flat`(默认)/ `complex` | `flat` = 纯色背景(走色键管线)。`complex` = 非纯色/实景背景:无色键,完全由分割器驱动抠图;此模式下 `screen_color` 被忽略,发丝细节通常弱于纯色模式。 |
| `screen_color` | 否 | `#RRGGBB`,或留空/`auto` | 背景色先验;留空则从边框自动探测。`background_mode=complex` 时忽略。 |
| `seg_model` | 否 | `birefnet`(默认)/ `anime-seg` | 分割后端。 |
| `cross_check` | 否 | `1`/`0`(也接受 `true`/`false`/`yes`/`no`/`on`/`off`) | 按请求覆盖交叉否决;留空在 `flat` 模式 = 服务端默认,但在 `complex` 模式**默认关闭**(传 `on` 启用其无色相门控)。 |
### 响应头(`200` 时)
| 头 | 示例 | 含义 |
| --- | --- | --- |
| `X-BGFilter-Elapsed-Ms` | `2372` | 服务端处理耗时(ms)。 |
| `X-BGFilter-Seg-Model` | `birefnet` | 实际使用的分割器。 |
| `X-BGFilter-Screen-Color` | `#CFEFFF` / `auto` | 实际使用的背景色。 |
| `X-BGFilter-Cross-Check` | `on` / `off` | 本次实际的 cross-check 状态。 |
| `X-BGFilter-Background-Mode` | `flat` / `complex` | 本次实际的背景模式。 |
| `X-Request-ID` | `42efe4e0d76f4b83` | 关联 id(你若传入 `X-Request-ID` 则回显,否则自动生成)。报障时报这个 id 即可对上服务端日志。每个响应都返回,含错误响应。 |
### 状态码
| 码 | 何时 |
| --- | --- |
| `200` | 成功——响应体是 RGBA PNG。 |
| `400` | 输入有误:图片无法解码,`file`/`image_url` 不是恰好其一,`image_url` 非法/fetch 失败,`screen_color`/`seg_model`/`cross_check`/`background_mode` 非法,或自动探测失败(请传 `screen_color`)。 |
| `401` | 鉴权已开且 token 缺失或错误。 |
| `413` | 图片超过 `BGFILTER_MAX_IMAGE_PIXELS`(默认约 4 MP),或 `image_url` 响应体超过 `BGFILTER_FETCH_MAX_BYTES`(默认约 25 MB)。 |
| `500` | 推理失败。 |
| `503` | 服务仍在加载模型(未就绪)。 |
### 示例
```bash
# 默认(birefnet + 自动背景色);若服务要求鉴权则加上 auth header
curl -sS -F "file=@input.png" \
-H "X-Genarrative-Image-Token: <token>" \
http://127.0.0.1:18083/remove-background -o output.png
# 显式背景色 + 动漫分割器,本次关闭 cross-check
curl -sS \
-F "file=@input.png" \
-F "screen_color=#CFEFFF" \
-F "seg_model=anime-seg" \
-F "cross_check=off" \
http://127.0.0.1:18083/remove-background -o output.png
# 非纯色/实景背景(无色键);此模式 cross-check 默认关闭,
# 需要时加 cross_check=on 启用无色相残留否决
curl -sS \
-F "file=@scene.jpg" \
-F "background_mode=complex" \
http://127.0.0.1:18083/remove-background -o output.png
# 用 URL 代替上传(服务端自行 fetch;image_url 与 file 二选一,不可同时给)
curl -sS \
-F "image_url=https://example.com/input.png" \
-F "screen_color=#CFEFFF" \
-H "X-Genarrative-Image-Token: <token>" \
http://127.0.0.1:18083/remove-background -o output.png
```
运行时配置(配置文件、device、尺寸上限、预加载、鉴权 token、流水线 worker 数)全部
通过 `BGFILTER_*` 环境变量——完整表格见 [DEPLOY_ZH.md](DEPLOY_ZH.md) §6。
与 CLI 一样,服务**默认开启交叉模型否决**;`/healthz``crossCheck` 报告生效的默认值,
按请求的 `cross_check` 表单字段可任意覆盖(全服务关闭用 `BGFILTER_CROSS_CHECK=0`)。
`cross_check.reuse_as_seg`(仅配置文件可设,默认关——见上方"管线")没有请求级开关;
改它意味着改配置并重启服务。
## 质量检查
质量检查器在半透明边缘像素上度量 alpha 有效性和边缘溢色。
```bash
python -m bgfilter.quality_cli \
Outputs/TestImage_rgba.png \
--max-edge-green-excess-p95 0.30
```
跑内置样本冒烟检查:
```bash
python scripts/smoke_samples.py \
--samples-dir Samples \
--output-dir Outputs/smoke_samples \
--config configs/default.yaml \
--device cpu \
--fallback-to-chroma-alpha \
--max-edge-green-excess-p95 0.30
```
## 备注
- `Samples/``Outputs/` 被 Git 忽略。
- ViTMatte 和分割权重在首次使用时从 Hugging Face 加载。`anime-seg`
(`skytnt/anime-seg`)是普通 ONNX 下载;`birefnet`(`ZhengPeng7/BiRefNet`)自带
自定义建模代码,故需 `trust_remote_code=True` 外加 `timm` / `einops` / `kornia`
- **权重准备**——离线 HF 缓存、`models/` 纯文件夹(`BGFILTER_WEIGHTS_DIR`)、一键下载
脚本 `scripts/fetch_weights.py`、以及国内网络的镜像 workaround——见
[DEPLOY_ZH.md](DEPLOY_ZH.md) §5。
- 前景颜色估计用 pymatting 的 `estimate_foreground_ml`,在 de-spill 前把干净前景色
传播进半透明边缘。设 `foreground.method: unmix` 回落到旧的启发式方法。
- `docs/green_screen_matting_workflow.md` 是最初的第一阶段绿幕规格;本 README 反映当前
的、泛化后的架构。
+549
View File
File diff suppressed because it is too large Load Diff
+12
View File
@@ -1,12 +1,24 @@
"""Flat-background character matting pipeline."""
import os
import sys
# Windows PyTorch allocates CPU tensors through a bundled mimalloc whose lazy
# purge keeps ~10 GB of already-freed forward activations committed after a
# BiRefNet@2048 pass, stacking under ViTMatte's attention peak (measured
# 2048x2048 bf16: peak 25.2 -> 21.4 GB with purge 0, and slightly *faster*).
# Must be set before the torch DLL loads; harmless where mimalloc is not used.
# The Linux/glibc equivalent (ptmalloc arena retention) is handled at runtime by
# bgfilter.memtune.release_freed_memory(); see DEPLOY for jemalloc as an alt.
os.environ.setdefault("MIMALLOC_PURGE_DELAY", "0")
# Expandable segments defuse the CUDA caching allocator's fragmentation: the
# full bf16 pipeline's reserved VRAM drops 15.3 -> 12.0 GB on a 16 GB card and
# runs ~10% faster (measured, RTX 5070 Ti). Read at CUDA-allocator init, so it
# must be set before torch loads; a no-op for CPU-only runs. Native Windows
# does not support it (torch warns and ignores), so it is only defaulted where
# it works (Linux/WSL2); setdefault keeps any explicit user setting in charge.
if not sys.platform.startswith("win"):
os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
__version__ = "0.1.0"
+13 -6
View File
@@ -59,7 +59,7 @@ def suppress_alpha_by_chroma(
def cross_check_alpha(
alpha: np.ndarray,
second_alpha: np.ndarray,
proj: np.ndarray,
proj: np.ndarray | None,
lightness: np.ndarray,
trimap: np.ndarray,
settings: CrossCheckSettings,
@@ -76,6 +76,13 @@ def cross_check_alpha(
- the primary alpha is high (``gate_lo -> gate_hi`` ramp): pixels the
pipeline already renders soft (outer wisps) are exempt by construction.
``proj`` is None in complex-background mode (chroma disabled): no key colour
means no hue-defined suspect zone, so instead the veto requires the second
opinion itself to be confidently near-empty (full strength at/below
``second_lo``, none at/above ``second_hi``). A thin strand the downsampled
second model merely blurs to mid-alpha is left untouched; only regions it
decisively rejects can be cleared.
Inside the zone this deliberately overrides the trimap-FG clamp -- the
residue it exists to clear is mostly trimap-FG. Sure background cannot be
disturbed: min-fusion keeps alpha 0 at 0.
@@ -83,16 +90,16 @@ def cross_check_alpha(
if not settings.enabled:
return alpha
cv2 = require_cv2()
zone = (
(proj >= settings.proj_min)
& (lightness >= settings.l_min)
& (trimap != 0)
)
zone = (lightness >= settings.l_min) & (trimap != 0)
if proj is not None:
zone &= proj >= settings.proj_min
weight = zone.astype(np.float32)
if settings.feather_sigma > 0:
blur = cv2.GaussianBlur(weight, (0, 0), settings.feather_sigma)
weight = np.where(zone, 1.0, np.clip(blur, 0.0, 1.0)).astype(np.float32)
gate = weight * _smoothstep(alpha, settings.gate_lo, settings.gate_hi)
if proj is None:
gate = gate * (1.0 - _smoothstep(second_alpha, settings.second_lo, settings.second_hi))
out = alpha * (1.0 - gate) + np.minimum(alpha, second_alpha) * gate
return np.clip(out, 0.0, 1.0).astype(np.float32)
+7 -1
View File
@@ -6,6 +6,7 @@ import typer
from rich import print
from .config import load_settings, override_settings
from .memtune import release_freed_memory
from .pipeline import MattingPipeline, run_image
app = typer.Typer(help="Offline flat-background character matting.")
@@ -23,6 +24,7 @@ def main(
device: str | None = typer.Option(None, "--device"),
precision: str | None = typer.Option(None, "--precision", help="Compute precision for all models (ViTMatte cast + BiRefNet autocast): fp32 (default) | bf16 (faster + halves matting activation memory; needs bf16-capable hardware, else falls back to fp32)"),
screen_color: str | None = typer.Option(None, "--screen-color", help="Background colour prior as #RRGGBB (default: auto-detect the flat background colour)"),
chroma: bool | None = typer.Option(None, "--chroma/--no-chroma", help="Flat-colour background mode (default: on). --no-chroma is non-flat/complex-background mode: no colour key, segmentation drives the trimap; needs a segmentation backend + matting_method vitmatte"),
matting_method: str | None = typer.Option(None, "--matting-method"),
fallback_to_chroma_alpha: bool | None = typer.Option(None, "--fallback-to-chroma-alpha/--no-fallback-to-chroma-alpha"),
sure_bg_threshold: float | None = typer.Option(None, "--sure-bg-threshold", min=0.0, max=1.0),
@@ -31,7 +33,7 @@ def main(
fg_safe_radius_ratio: float | None = typer.Option(None, "--fg-safe-radius-ratio", min=0.0),
despill: bool | None = typer.Option(None, "--despill/--no-despill"),
cross_check: bool | None = typer.Option(None, "--cross-check/--no-cross-check", help="Second-opinion veto of background-hued residue between hair strands (default: on; costs one extra model inference)"),
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: on; saves ~20s on CPU; needs cross-check on)"),
cross_check_as_seg: bool | None = typer.Option(None, "--cross-check-as-seg/--no-cross-check-as-seg", help="Reuse the cross-check forward as the segmentation mask, skipping the primary seg model (default: off; saves ~20s on CPU; needs cross-check on)"),
trimap_mode: str | None = typer.Option(None, "--trimap-mode", help="Trimap mode (segmentation pipeline): directional | seg | directional-hard-bg"),
seg_backend: str | None = typer.Option(None, "--seg-backend", help="Segmentation backend: birefnet (default) | anime-seg"),
) -> None:
@@ -42,6 +44,7 @@ def main(
device=device,
precision=precision,
screen_color=screen_color,
chroma=chroma,
matting_method=matting_method,
fallback_to_chroma_alpha=fallback_to_chroma_alpha,
sure_bg_threshold=sure_bg_threshold,
@@ -69,6 +72,9 @@ def main(
dbg = None if debug_dir is None else debug_dir / image_path.stem
result = pipeline.run_image(image_path, out_path, dbg)
print(f"[green]wrote[/green] {result['output']}")
# Free this image's activations back to the OS before the next one
# so batch RSS tracks a single image, not the whole run.
release_freed_memory()
return
if input is None or output is None:
+8 -2
View File
@@ -13,6 +13,7 @@ from .settings import (
ModelSettings,
PipelineSettings,
SegmentationSettings,
ServerSettings,
TrimapSettings,
)
@@ -49,7 +50,7 @@ def _update_dataclass(instance: T, values: dict[str, Any] | None) -> T:
def settings_from_dict(data: dict[str, Any]) -> PipelineSettings:
data = dict(data)
screen_color = data.pop("screen_color", None)
allowed_sections = {"chroma", "trimap", "alpha_post", "cross_check", "foreground", "despill", "model", "segmentation"}
allowed_sections = {"chroma", "trimap", "alpha_post", "cross_check", "foreground", "despill", "model", "segmentation", "server"}
unknown_sections = sorted(set(data) - allowed_sections)
if unknown_sections:
raise ValueError(f"Unknown config section(s): {', '.join(unknown_sections)}")
@@ -63,6 +64,7 @@ def settings_from_dict(data: dict[str, Any]) -> PipelineSettings:
despill=_update_dataclass(DespillSettings(), data.get("despill")),
model=_update_dataclass(ModelSettings(), data.get("model")),
segmentation=_update_dataclass(SegmentationSettings(), data.get("segmentation")),
server=_update_dataclass(ServerSettings(), data.get("server")),
screen_color=screen_color,
)
@@ -99,6 +101,9 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
}
if overrides.get("trimap_mode") is not None:
trimap_updates["mode"] = overrides["trimap_mode"]
chroma_updates: dict[str, Any] = {}
if overrides.get("chroma") is not None:
chroma_updates["enabled"] = overrides["chroma"]
despill_updates: dict[str, Any] = {}
if overrides.get("despill") is not None:
despill_updates["enabled"] = overrides["despill"]
@@ -130,7 +135,7 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
else settings.screen_color
)
return PipelineSettings(
chroma=chroma,
chroma=_update_dataclass(chroma, chroma_updates),
trimap=_update_dataclass(trimap, trimap_updates),
alpha_post=alpha_post,
cross_check=_update_dataclass(settings.cross_check, cross_check_updates),
@@ -138,5 +143,6 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
despill=_update_dataclass(despill, despill_updates),
model=_update_dataclass(model, model_updates),
segmentation=_update_dataclass(segmentation, seg_updates),
server=settings.server,
screen_color=screen_color,
)
+3 -2
View File
@@ -10,7 +10,7 @@ from .settings import DespillSettings
def despill(
rgb: np.ndarray,
alpha: np.ndarray,
model: BackgroundModel,
model: BackgroundModel | None,
settings: DespillSettings,
) -> tuple[np.ndarray, np.ndarray]:
"""Remove background-colour spill from the foreground.
@@ -24,7 +24,8 @@ def despill(
Returns the corrected RGB (uint8) and the per-pixel despill weight (float32).
"""
if not settings.enabled:
# model is None with chroma disabled: no key colour means no spill direction.
if not settings.enabled or model is None:
return rgb.copy(), np.zeros(alpha.shape, dtype=np.float32)
bg_lab = np.asarray(model.lab_center, dtype=np.float32)
+47 -2
View File
@@ -1,17 +1,21 @@
from __future__ import annotations
import logging
import numpy as np
from .chroma import BackgroundModel
from .deps import require_cv2
from .settings import ForegroundSettings
logger = logging.getLogger(__name__)
def estimate_foreground_rgb(
rgb: np.ndarray,
alpha: np.ndarray,
bg_confidence: np.ndarray,
model: BackgroundModel,
model: BackgroundModel | None,
settings: ForegroundSettings,
) -> np.ndarray:
"""Estimated foreground colour F (uint8 HxWx3) for compositing over alpha."""
@@ -23,10 +27,15 @@ def estimate_foreground_rgb(
try:
return _estimate_ml(rgb, alpha, settings)
except RuntimeError:
if not settings.fallback_to_unmix:
if not settings.fallback_to_unmix or model is None:
raise
method = "unmix"
if method == "unmix":
if model is None:
raise RuntimeError(
"Foreground method 'unmix' needs the chroma background model, which "
"does not exist with chroma disabled; use method 'ml'."
)
return _estimate_unmix(rgb, alpha, bg_confidence, model, settings)
raise RuntimeError(
f"Unsupported foreground method '{settings.method}'. Use 'ml' or 'unmix'."
@@ -43,6 +52,10 @@ def _estimate_ml(
high-alpha neighbours instead of being divided out pixel-wise. That removes
background-colour spill on thin hair where a direct unmix (divide by small
alpha) is too noisy to trust.
Runs on the GPU via pymatting's CuPy backend when ``settings.use_gpu`` is set
and cupy + a CUDA device are available (~250x faster; float32, visually
identical to the float64 CPU result); otherwise falls back to the CPU solver.
"""
try:
from pymatting import estimate_foreground_ml
@@ -53,10 +66,42 @@ def _estimate_ml(
image = rgb.astype(np.float64) / 255.0
a = np.clip(alpha.astype(np.float64), 0.0, 1.0)
if settings.use_gpu:
foreground = _estimate_ml_gpu(image, a, settings.ml_regularization)
if foreground is not None:
return np.clip(foreground * 255.0, 0, 255).astype(np.uint8)
foreground = estimate_foreground_ml(image, a, regularization=settings.ml_regularization)
return np.clip(foreground * 255.0, 0, 255).astype(np.uint8)
def _estimate_ml_gpu(
image: np.ndarray, alpha: np.ndarray, regularization: float
) -> np.ndarray | None:
"""Multi-level foreground via pymatting's CuPy backend.
Returns the foreground as float ``[0, 1]``, or ``None`` when the GPU path is
unavailable (cupy not installed, or no CUDA device) so the caller falls back
to the CPU solver. pymatting only re-exports ``estimate_foreground_ml_cupy``
when cupy is importable, so the import itself is the availability probe.
"""
try:
from pymatting import estimate_foreground_ml_cupy
except ImportError:
return None
try:
import cupy
if cupy.cuda.runtime.getDeviceCount() < 1:
return None
foreground = estimate_foreground_ml_cupy(image, alpha, regularization=regularization)
return np.asarray(foreground)
except Exception as exc: # noqa: BLE001 -- any GPU failure falls back to CPU
logger.warning("GPU foreground estimation failed (%s); using CPU", exc)
return None
def _weighted_blur(values: np.ndarray, weights: np.ndarray, radius: int) -> np.ndarray:
if radius <= 0:
return values
+44
View File
@@ -0,0 +1,44 @@
from __future__ import annotations
"""Cross-platform release of freed allocator memory after a heavy inference.
Both of the process allocators this pipeline runs under retain large freed
blocks instead of returning them to the OS, so a BiRefNet@2048 forward's
activations keep inflating RSS long after the tensors are gone:
- Windows PyTorch bundles mimalloc; its lazy purge is defused at import time by
``MIMALLOC_PURGE_DELAY=0`` (set in ``bgfilter/__init__.py`` before torch loads).
- Linux/glibc PyTorch uses ptmalloc, which keeps freed arenas committed;
``malloc_trim(0)`` hands those pages back and is called at runtime via
:func:`release_freed_memory` after each image.
Neither hook exists on the other's platform, so this stays a no-op there (and on
musl, which has no ``malloc_trim``, or under a preloaded jemalloc/tcmalloc that
manages its own release). See DEPLOY for the recommended jemalloc LD_PRELOAD.
"""
import ctypes
import ctypes.util
import sys
_malloc_trim = None
if sys.platform.startswith("linux"):
try:
_libc = ctypes.CDLL(ctypes.util.find_library("c") or "libc.so.6", use_errno=True)
_malloc_trim = _libc.malloc_trim # glibc only; AttributeError on musl
_malloc_trim.argtypes = [ctypes.c_size_t]
_malloc_trim.restype = ctypes.c_int
except (OSError, AttributeError):
_malloc_trim = None
def release_freed_memory() -> bool:
"""Return freed heap pages to the OS; True if a release was actually issued.
Cheap (sub-millisecond) and safe to call after every processed image. No-op
where no supported hook is available (Windows, musl, non-glibc allocators).
"""
if _malloc_trim is None:
return False
_malloc_trim(0)
return True
+54 -20
View File
@@ -1,5 +1,6 @@
from __future__ import annotations
import contextlib
import json
from dataclasses import asdict, dataclass
from pathlib import Path
@@ -60,6 +61,7 @@ class MattingPipeline:
runner: ViTMatteRunner | None = None,
segmenter=None,
cross_checker=None,
gpu_lock=None,
):
self.settings = settings
# Pre-loaded models can be injected so a long-lived service shares them
@@ -67,6 +69,10 @@ class MattingPipeline:
self._runner: ViTMatteRunner | None = runner
self._segmenter = segmenter
self._cross_checker = cross_checker
# Held around every GPU model forward so concurrent requests never drive the
# device at once (models are shared, single GPU). None = no contention
# (CLI / single-thread), so a no-op context keeps that path lock-free.
self._gpu_lock = gpu_lock if gpu_lock is not None else contextlib.nullcontext()
def _segment(self, rgb: np.ndarray) -> np.ndarray:
if self._segmenter is None:
@@ -132,10 +138,26 @@ def run_image(
def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
settings = pipeline.settings
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
bg_confidence, model = compute_bg_confidence(
rgb, settings=settings.chroma, screen_color=screen_color
)
if settings.chroma.enabled:
screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None
bg_confidence, model = compute_bg_confidence(
rgb, settings=settings.chroma, screen_color=screen_color
)
else:
if not settings.segmentation.enabled:
raise RuntimeError(
"chroma.enabled: false (non-flat background mode) needs the segmentation "
"pipeline; enable segmentation or re-enable chroma."
)
if settings.model.matting_method == "chroma":
raise RuntimeError(
"matting_method 'chroma' needs the chroma key; use 'vitmatte' or re-enable chroma."
)
# Non-flat background: no colour key exists. A zero background-confidence
# map makes every chroma-fused formula degrade to its seg-only form; the
# colour-keyed stages (hue split, suppression, veto, despill) are skipped.
bg_confidence = np.zeros(rgb.shape[:2], dtype=np.float32)
model = None
second_alpha = None
if settings.segmentation.enabled:
reuse = (
@@ -150,11 +172,15 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
if reuse:
# One HR-matting forward serves both the trimap topology and the
# cross-check second opinion; the primary seg model is never loaded.
second_alpha = pipeline._second_opinion(rgb)
with pipeline._gpu_lock:
second_alpha = pipeline._second_opinion(rgb)
seg_mask = second_alpha
else:
seg_mask = pipeline._segment(rgb)
mode = settings.trimap.mode
with pipeline._gpu_lock:
seg_mask = pipeline._segment(rgb)
# The directional modes are colour tests against the key colour; without
# one, the seg-topology trimap is the only meaningful choice.
mode = settings.trimap.mode if settings.chroma.enabled else "seg"
if mode == "seg":
trimap, trimap_stats = fuse_trimap(seg_mask, bg_confidence, settings.trimap)
elif mode in ("directional", "directional-hard-bg"):
@@ -170,36 +196,44 @@ def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult:
else:
seg_mask = None
trimap, trimap_stats = generate_trimap(bg_confidence, settings.trimap)
raw_alpha, alpha_source = pipeline._predict_alpha(rgb, trimap, bg_confidence)
with pipeline._gpu_lock:
raw_alpha, alpha_source = pipeline._predict_alpha(rgb, trimap, bg_confidence)
alpha = enforce_trimap(raw_alpha, trimap)
alpha = suppress_alpha_by_chroma(
alpha, bg_confidence, trimap, settings.alpha_post,
# The matte-confidence gate only makes sense for a real matting prediction;
# a chroma-seeded alpha is itself colour evidence, so no gate there.
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
)
if settings.chroma.enabled:
alpha = suppress_alpha_by_chroma(
alpha, bg_confidence, trimap, settings.alpha_post,
# The matte-confidence gate only makes sense for a real matting prediction;
# a chroma-seeded alpha is itself colour evidence, so no gate there.
raw_alpha=raw_alpha if alpha_source == "vitmatte" else None,
)
alpha = clean_alpha(alpha, trimap, settings.alpha_post)
if settings.cross_check.enabled:
lab = convert_color_spaces(rgb)[1]
if second_alpha is None:
second_alpha = pipeline._second_opinion(rgb)
with pipeline._gpu_lock:
second_alpha = pipeline._second_opinion(rgb)
alpha = cross_check_alpha(
alpha,
second_alpha,
bg_hue_projection(lab, model.lab_center),
# No key colour in complex-background mode: proj=None switches the
# veto to its second-opinion-confidence gate (see cross_check_alpha).
bg_hue_projection(lab, model.lab_center) if model is not None else None,
lab[..., 0],
trimap,
settings.cross_check,
)
foreground = estimate_foreground_rgb(
rgb, alpha, bg_confidence, model, settings.foreground
)
# Foreground colour estimation runs on the GPU (cupy) when enabled, so it shares
# the mutex; the CPU fallback path is short and simply runs while holding it.
with pipeline._gpu_lock:
foreground = estimate_foreground_rgb(
rgb, alpha, bg_confidence, model, settings.foreground
)
corrected_rgb, color_mask = despill(foreground, alpha, model, settings.despill)
metadata = {
"alpha_source": alpha_source,
"background_model": model.to_dict(),
"background_model": model.to_dict() if model is not None else None,
"trimap": trimap_stats,
"settings": asdict(settings),
}
+9 -2
View File
@@ -1,10 +1,13 @@
from __future__ import annotations
import os
import numpy as np
from PIL import Image
from .precision import ensure_deform_conv2d_autocast_cpu, resolve_dtype
from .settings import SegmentationSettings
from .weights import resolve_model_source
class BiRefNetSegmenter:
@@ -36,7 +39,7 @@ class BiRefNetSegmenter:
self.torch = torch
self.model = AutoModelForImageSegmentation.from_pretrained(
settings.model_name, trust_remote_code=True
resolve_model_source(settings.model_name), trust_remote_code=True
)
self.model.eval()
self.model.float() # checkpoint ships as fp16; force fp32 to match inputs
@@ -104,7 +107,11 @@ class AnimeSegSegmenter:
"pip install onnxruntime huggingface_hub"
) from exc
model_file = hf_hub_download(settings.model_name, "isnetis.onnx")
source = resolve_model_source(settings.model_name)
if os.path.isdir(source):
model_file = os.path.join(source, "isnetis.onnx")
else:
model_file = hf_hub_download(source, "isnetis.onnx")
providers = ["CPUExecutionProvider"]
if settings.device == "cuda" and "CUDAExecutionProvider" in ort.get_available_providers():
providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
+196
View File
@@ -0,0 +1,196 @@
from __future__ import annotations
import logging
import os
import threading
from dataclasses import replace
import numpy as np
from .config import _SEG_MODELS, override_settings
from .memtune import release_freed_memory
from .pipeline import MattingPipeline, MattingResult
from .segmentation import BiRefNetSegmenter, make_segmenter
from .settings import PipelineSettings, SegmentationSettings
from .vitmatte_infer import ViTMatteRunner
logger = logging.getLogger("bgfilter.service")
# HTTP-facing seg_model values, in preference order (first = default).
SEG_MODELS: tuple[str, ...] = tuple(_SEG_MODELS)
class PipelineManager:
"""Long-lived holder of the matting models.
ViTMatte is the matting model shared by both segmentation backends, so it is
loaded once and reused. Each segmentation backend is lazily loaded once on
first use and cached by name. Per-request parameters (``screen_color``,
``seg_model``, ``cross_check``, ``chroma``) never trigger a model reload --
they only re-parametrise the pipeline over already-cached (or first-time-
loaded, then cached) models. ``chroma=False`` selects non-flat/complex
background mode (no colour key); ``screen_color`` is then unused.
"""
def __init__(self, base_settings: PipelineSettings, preload: bool = True):
self._base = base_settings
self._runner: ViTMatteRunner | None = None
self._segmenters: dict[str, object] = {}
self._cross_checker: BiRefNetSegmenter | None = None
self._load_lock = threading.Lock()
# Caps how many concurrent requests may be inside a GPU forward at once
# (the app may run several process() calls on a thread pool). Shared by every
# pipeline this manager builds. =1 (default) makes the device strictly
# single-stream; >1 lets that many forwards overlap so batch-1 kernels that
# underfill the SMs can co-schedule (higher GPU-bound throughput, but the
# concurrent activations multiply GPU memory). Config value
# (``server.gpu_concurrency``) is the default; the env var overrides it.
gpu_conc = max(1, int(os.environ.get("BGFILTER_GPU_CONCURRENCY", base_settings.server.gpu_concurrency)))
self._gpu_lock = threading.Lock() if gpu_conc == 1 else threading.BoundedSemaphore(gpu_conc)
if preload:
if base_settings.model.matting_method == "vitmatte":
self._get_runner()
# Cross-check is default behaviour, so its HR-matting model is part of
# the warm set. With reuse-as-seg on the birefnet backend that same
# forward also serves as the seg mask, so the primary segmenter is
# never used on the default path -- only preload it when reuse won't
# cover segmentation (reuse off, or a non-birefnet default backend).
if base_settings.cross_check.enabled:
self._get_cross_checker()
reuse_covers_seg = (
base_settings.cross_check.enabled
and base_settings.cross_check.reuse_as_seg
and base_settings.segmentation.backend == "birefnet"
)
if base_settings.segmentation.enabled and not reuse_covers_seg:
self._get_segmenter(base_settings.segmentation.backend)
@property
def default_seg_model(self) -> str:
return self._base.segmentation.backend
@property
def device(self) -> str:
return self._base.model.device
@property
def cross_check_enabled(self) -> bool:
return self._base.cross_check.enabled
def effective_cross_check(self, requested: bool | None) -> bool:
"""Resolve the cross-check state for a request, honoring the lane lock.
When ``cross_check.lock`` is set (e.g. the cpu-fast lane) the base default
is authoritative and a per-request override is ignored -- so a locked
off-lane never loads the HR-matting model or risks an OOM, and never errors
on a ``cross_check=on`` request. Otherwise the request value wins when
given, else the base default applies.
"""
base = self._base.cross_check
if base.lock:
return base.enabled
return requested if requested is not None else base.enabled
def _get_runner(self) -> ViTMatteRunner:
if self._runner is None:
with self._load_lock:
if self._runner is None:
logger.info(
"loading ViTMatte: model=%s device=%s",
self._base.model.model_name,
self._base.model.device,
)
self._runner = ViTMatteRunner(self._base.model)
return self._runner
def _get_segmenter(self, backend: str):
seg = self._segmenters.get(backend)
if seg is None:
with self._load_lock:
seg = self._segmenters.get(backend)
if seg is None:
seg_settings = replace(
self._base.segmentation,
backend=backend,
model_name=_SEG_MODELS[backend],
)
logger.info(
"loading segmenter: backend=%s model=%s device=%s",
backend,
seg_settings.model_name,
seg_settings.device,
)
seg = make_segmenter(seg_settings)
self._segmenters[backend] = seg
return seg
def _get_cross_checker(self) -> BiRefNetSegmenter:
# The cross-check model depends only on base settings (never on per-request
# params), so one shared instance serves all requests.
if self._cross_checker is None:
with self._load_lock:
if self._cross_checker is None:
check = self._base.cross_check
logger.info(
"loading cross-check model: model=%s device=%s",
check.model_name,
self._base.segmentation.device,
)
self._cross_checker = BiRefNetSegmenter(
SegmentationSettings(
model_name=check.model_name,
device=self._base.segmentation.device,
input_size=check.input_size,
precision=check.precision,
)
)
return self._cross_checker
def process(
self,
rgb: np.ndarray,
*,
screen_color: str | None,
seg_model: str,
cross_check: bool | None = None,
chroma: bool | None = None,
) -> MattingResult:
if seg_model not in _SEG_MODELS:
raise ValueError(f"invalid seg_model '{seg_model}'")
settings = override_settings(
self._base,
screen_color=screen_color,
seg_backend=seg_model,
# None leaves the base default; False selects non-flat/complex mode.
chroma=chroma,
# Honor the lane lock: a locked lane ignores the per-request override
# (a cpu-fast box never runs cross-check, even on cross_check=on).
cross_check=self.effective_cross_check(cross_check),
)
runner = (
self._get_runner()
if settings.model.matting_method == "vitmatte"
else None
)
segmenter = (
self._get_segmenter(seg_model) if settings.segmentation.enabled else None
)
cross_checker = (
self._get_cross_checker() if settings.cross_check.enabled else None
)
pipeline = MattingPipeline(
settings,
runner=runner,
segmenter=segmenter,
cross_checker=cross_checker,
gpu_lock=self._gpu_lock,
)
try:
return pipeline.run_rgb(rgb)
finally:
# Return this request's large freed activations to the OS so RSS does
# not ratchet up across requests in the long-lived service process.
# Hold the GPU lock: empty_cache must not race a concurrent request's
# in-flight allocations on the same device.
with self._gpu_lock:
release_freed_memory()
+63 -7
View File
@@ -5,6 +5,14 @@ from dataclasses import dataclass
@dataclass(frozen=True)
class ChromaSettings:
# False = non-flat-background mode (--no-chroma): no colour key at all.
# Segmentation alone drives the trimap (mode forced to "seg"), and the
# colour-keyed stages are bypassed: background auto-detect, the directional
# hue split, chroma alpha suppression, despill, and the unmix foreground
# fallback. The cross-check veto still runs (if enabled) with a hue-free
# gate -- see CrossCheckSettings.second_lo/hi. Requires
# segmentation.enabled and matting_method 'vitmatte'.
enabled: bool = True
border_ratio: float = 0.04
min_samples: int = 2048
lab_sigma_min: float = 10.0
@@ -40,6 +48,14 @@ class TrimapSettings:
mode: str = "directional"
seg_low: float = 0.15 # seg below this -> background-eligible; at/above -> foreground-eligible
bg_hue_proj_min: float = 4.0 # Lab a*/b* projection onto bg direction above which a pixel is background-hued
# Semantic override for the directional hue split: a chroma-unknown pixel whose
# seg confidence is at/above this is kept sure-FG even when background-hued.
# A saturated seg signal outranks colour evidence that only says "same hue
# family as the background" (bright skin on a warm background, measured raw
# ViTMatte failures at proj ~7-10). Chroma sure-background is never overridden,
# and the silhouette band still re-opens the boundary for anti-aliasing.
# Set > 1.0 to disable. (seg masks are uint8-quantised: 0.98 = 250/255.)
seg_force_fg: float = 0.98
@dataclass(frozen=True)
@@ -72,16 +88,22 @@ class CrossCheckSettings:
# that chroma, segmentation and the primary matte all read as foreground;
# already-soft wisps and dark hair are exempt by construction.
enabled: bool = True
# Lock the cross-check state to ``enabled`` and ignore per-request overrides.
# A memory-constrained lane (e.g. cpu-fast) sets ``enabled: false, lock: true``
# so a request with ``cross_check=on`` is silently honoured as off instead of
# loading the HR-matting model and risking an OOM. No error is raised.
lock: bool = False
model_name: str = "ZhengPeng7/BiRefNet_HR-matting"
input_size: int = 2048
# Reuse the cross-check forward as the segmentation mask: one HR-matting
# inference serves both trimap topology and the second opinion, skipping the
# primary seg model entirely (saves its load + forward). Pilot-validated on
# TestImage3/FixImage1: trimap 99.8% identical, no structural change. Only
# applies to the birefnet segmentation backend (a same-family swap); with
# anime-seg the chosen segmenter keeps its own forward and the cross-check
# runs independently. Needs cross_check and segmentation both enabled.
reuse_as_seg: bool = True
# primary seg model entirely (saves its load + forward; measured ~20 s/image
# on CPU, ~0.9-1.3 GB VRAM on GPU). Pilot-validated on TestImage3/FixImage1:
# trimap 99.8% identical. Default OFF on server-edition: the dedicated
# segmenter keeps its own forward and the veto stays an independent signal;
# opt in with --cross-check-as-seg. Only applies to the birefnet backend
# (a same-family swap); needs cross_check and segmentation both enabled.
reuse_as_seg: bool = False
# Compute precision of the cross-check forward: "fp32" or "bf16" (autocast;
# needs bf16-capable hardware, else falls back to fp32 with a warning).
precision: str = "fp32"
@@ -90,12 +112,24 @@ class CrossCheckSettings:
feather_sigma: float = 2.0 # Gaussian feather of the zone boundary, in px
gate_lo: float = 0.70 # primary alpha below this -> fully exempt
gate_hi: float = 0.95 # primary alpha above this -> fully vetoable
# Complex-background mode only (chroma disabled -> no hue-defined suspect
# zone): the veto instead requires the second opinion itself to be
# confidently near-empty -- full strength at/below second_lo, none at/above
# second_hi. Thin structures the downsampled second model merely blurs to
# mid-alpha stay untouched; only decisively-rejected regions can be cleared.
second_lo: float = 0.15
second_hi: float = 0.40
@dataclass(frozen=True)
class ForegroundSettings:
enabled: bool = True
method: str = "ml" # "ml" (pymatting) or "unmix" (legacy heuristic)
# With method "ml", use pymatting's CuPy GPU backend when it is available
# (cupy installed + a CUDA device) — ~250x faster, float32 output visually
# identical to the float64 CPU solver. Transparently falls back to CPU when
# cupy/CUDA are absent, so it is safe to leave on for CPU-only deployments.
use_gpu: bool = True
fallback_to_unmix: bool = True
ml_regularization: float = 1e-5
# Legacy "unmix" method parameters (used only when method == "unmix").
@@ -113,7 +147,12 @@ class ForegroundSettings:
@dataclass(frozen=True)
class DespillSettings:
enabled: bool = True
# Off by default: despill pulls foreground chroma along the background-hue axis
# with no positional or semantic guard, so a subject sharing the background's hue
# (e.g. a blue suit on a blue backdrop) is desaturated/hue-shifted (~ΔE 35, blue
# -> purple). This is the root default the cpu-fast lane inherits. Re-enable
# per-config or with --despill only when edge spill genuinely matters.
enabled: bool = False
edge_low: float = 0.005
strength: float = 0.92
edge_expand_radius: int = 2
@@ -157,6 +196,22 @@ class SegmentationSettings:
precision: str = "fp32"
@dataclass(frozen=True)
class ServerSettings:
"""HTTP-service request concurrency (ignored by the CLI).
``cpu_workers`` is the thread-pool size for request pipelining: >1 lets that
many requests run concurrently, overlapping their CPU stages while the GPU
forwards stay serialized. ``gpu_concurrency`` caps how many forwards may
co-schedule on the device at once (>1 fills the SMs that batch-1 underuses,
but multiplies GPU activation memory -- size it to the VRAM). Both env vars
``BGFILTER_CPU_WORKERS`` / ``BGFILTER_GPU_CONCURRENCY`` override these.
"""
cpu_workers: int = 1
gpu_concurrency: int = 1
@dataclass(frozen=True)
class PipelineSettings:
chroma: ChromaSettings = ChromaSettings()
@@ -167,6 +222,7 @@ class PipelineSettings:
despill: DespillSettings = DespillSettings()
model: ModelSettings = ModelSettings()
segmentation: SegmentationSettings = SegmentationSettings()
server: ServerSettings = ServerSettings()
# Background-colour prior. None = auto-detect the flat background colour from the
# image border (raises if there is no clean flat background). A hex string like
# "#CFEFFF" sets it explicitly.
+16 -16
View File
@@ -123,16 +123,14 @@ def fuse_trimap_directional(
strongly displaced toward it (e.g. blue between hair strands) is left
unknown for ViTMatte / chroma-suppress. Lower-seg unknowns stay unknown.
A thin unknown band at the segmentation silhouette is preserved so ViTMatte can
anti-alias the boundary (it is deliberately small so interior holes survive).
No silhouette band is forced back to unknown: structures thinner than the band
ring (flying hair strands) got wholly confiscated by dilate-minus-erode and then
dropped by ViTMatte, which on crisp-edged (anime) images outputs near-binary
alpha in the band instead of anti-aliasing (measured: 96% hard values, and 74%
of lost strand pixels were rule-FG demoted by the band). The rules' own unknown
pixels (background-hued blends, low-seg transitions) remain ViTMatte's domain.
"""
cv2 = require_cv2()
shape = seg_mask.shape
fg_safe_radius = radius_from_ratio(
shape, settings.fg_safe_radius_ratio, settings.min_fg_safe_radius
)
band_radius = max(settings.min_fg_safe_radius, fg_safe_radius // 2)
chroma_bg = bg_confidence >= settings.sure_bg_threshold
chroma_fg = bg_confidence <= settings.sure_fg_threshold
chroma_unknown = ~chroma_bg & ~chroma_fg
@@ -145,26 +143,28 @@ def fuse_trimap_directional(
bg = chroma_bg | ((seg_mask < settings.seg_low) & ~chroma_fg)
fg = chroma_fg & (seg_mask >= settings.seg_low)
rule3_fg = chroma_unknown & core & ~bg_hued
# Rule 4 -- semantic override: a chroma-unknown pixel the segmenter is
# (near-)certain about stays foreground even when background-hued. Colour
# evidence here only says "same hue family as the background" (bright skin
# on a warm background), which is exactly where ViTMatte and the suppressor
# misfire; a saturated seg signal outranks it. chroma_unknown excludes sure
# background, so a real flat-colour backdrop can never be forced foreground.
force_fg = chroma_unknown & (seg_mask >= settings.seg_force_fg)
if settings.mode == "directional-hard-bg":
# Aggressive variant: hard-remove background-hued pixels instead of leaving
# them unknown. Can eat cool/shadowed white cloth, so it is not the default.
bg = bg | (chroma_unknown & core & bg_hued)
bg = bg | (chroma_unknown & core & bg_hued & ~force_fg)
trimap = np.full(shape, 128, dtype=np.uint8)
trimap[bg] = 0
trimap[fg | rule3_fg] = 255
loose = (seg_mask >= settings.seg_loose_threshold).astype(np.uint8)
kernel = elliptical_kernel(band_radius)
band = cv2.dilate(loose, kernel).astype(bool) & ~cv2.erode(loose, kernel).astype(bool)
trimap[band & ~chroma_bg] = 128
trimap[fg | rule3_fg | force_fg] = 255
stats = {
"sure_bg_pixels": int((trimap == 0).sum()),
"unknown_pixels": int((trimap == 128).sum()),
"sure_fg_pixels": int((trimap == 255).sum()),
"band_radius": int(band_radius),
"rule3_fg_pixels": int(rule3_fg.sum()),
"force_fg_pixels": int((force_fg & ~fg & ~rule3_fg).sum()),
"bg_hued_pixels": int((bg_hued & chroma_unknown & core).sum()),
}
return trimap, stats
+4 -2
View File
@@ -5,6 +5,7 @@ from PIL import Image
from .precision import resolve_dtype
from .settings import ModelSettings
from .weights import resolve_model_source
class ViTMatteRunner:
@@ -19,10 +20,11 @@ class ViTMatteRunner:
) from exc
self.torch = torch
self.processor = VitMatteImageProcessor.from_pretrained(settings.model_name)
self.model = VitMatteForImageMatting.from_pretrained(settings.model_name)
self.device = self._resolve_device(settings.device)
self.dtype = resolve_dtype(self.torch, self.device, settings.precision)
source = resolve_model_source(settings.model_name)
self.processor = VitMatteImageProcessor.from_pretrained(source)
self.model = VitMatteForImageMatting.from_pretrained(source)
self.model.to(device=self.device, dtype=self.dtype)
self.model.eval()
if settings.attn_query_chunk > 0:
+29
View File
@@ -0,0 +1,29 @@
from __future__ import annotations
import os
from pathlib import Path
def weights_dir() -> Path:
"""Directory scanned for locally-bundled model weights.
Defaults to ``models`` (relative to the working directory); override with the
``BGFILTER_WEIGHTS_DIR`` environment variable.
"""
return Path(os.environ.get("BGFILTER_WEIGHTS_DIR", "models"))
def resolve_model_source(repo_id: str) -> str:
"""Map a HuggingFace repo id to a local weights folder when one is present.
If ``<weights_dir>/<basename>`` exists it is returned (so models can be shipped
as plain folders inside the project, e.g. ``ZhengPeng7/BiRefNet`` ->
``models/BiRefNet``); otherwise the repo id is returned unchanged so it loads
from the HF hub/cache as before. An absolute/existing path is passed through.
"""
if os.path.isdir(repo_id):
return repo_id
local = weights_dir() / repo_id.split("/")[-1]
if local.is_dir():
return str(local)
return repo_id
+33
View File
@@ -0,0 +1,33 @@
# Urgent CPU-only lane: fp32, birefnet segmentation, NO cross-check, NO anime-seg.
#
# This is the lightest production path: base ViTMatte + one BiRefNet@1024, ~14-15s
# per 1.5 MP image on CPU. It needs ONLY these two weights provisioned --
# BiRefNet_HR-matting (cross-check) and anime-seg are never loaded.
#
# Tuned for an 8 GB box (~7 GB for the service). Measured peak working set:
# 2048x1152 (2.4 MP) ~5.6 GB | 2048x2048 (4.2 MP) ~7.1 GB.
# The 2048x2048 peak is a floor set by ViTMatte's own dense compute, not the
# attention (shrinking the chunk below 512 does not lower it further). Set
# MemoryMax=8G with a hard 4 MP input cap (BGFILTER_MAX_IMAGE_PIXELS default),
# or cap inputs tighter if you need more headroom.
#
# Anything omitted here inherits the built-in defaults (identical to default.yaml).
model:
device: cpu
precision: fp32 # fp32 only: this lane targets non-bf16 CPUs
# Query chunk of 512 (vs the 2048 default) trims ViTMatte's attention transient
# ~0.9 GB at 2048x2048 (8.0 -> 7.1 GB) for ~1 s more compute; 256 gives nothing
# more. Exact/bitwise-identical output -- purely a memory-vs-speed knob.
attn_query_chunk: 512
segmentation:
enabled: true
backend: birefnet # the only backend this lane uses
device: cpu
precision: fp32
cross_check:
enabled: false # no second-opinion HR-matting pass (the big cost/mem sink)
lock: true # ignore per-request cross_check=on so a request can't
# force the HR-matting load and OOM this low-memory lane
+27 -3
View File
@@ -3,6 +3,15 @@
# like "#CFEFFF" sets it explicitly. Also overridable via --screen-color.
screen_color: null
chroma:
# true = flat-colour background (default). false = non-flat/complex background
# mode (--no-chroma): no colour key; segmentation alone drives the trimap (mode
# forced to "seg") and the colour-keyed stages are bypassed (auto-detect, hue
# split, chroma suppression, despill). The cross-check veto still runs if
# enabled, using its hue-free gate (cross_check.second_lo/hi). Needs
# segmentation.enabled + matting_method vitmatte.
enabled: true
model:
model_name: hustvl/vitmatte-base-composition-1k
device: cpu
@@ -46,6 +55,12 @@ trimap:
mode: directional
seg_low: 0.15
bg_hue_proj_min: 4.0
# Semantic override (directional mode): chroma-unknown pixels with seg confidence
# at/above this stay sure-FG even when background-hued — rescues bright skin on a
# warm (same-hue-family) background that the hue split would otherwise hand to
# ViTMatte/suppression, which misfire exactly there. Never overrides chroma
# sure-background. Set > 1.0 to disable.
seg_force_fg: 0.98
alpha_post:
chroma_suppress: true
@@ -74,8 +89,9 @@ cross_check:
# trimap 99.8% identical, no structural change (fingers/wisps/bowstring OK).
# Only applies to the birefnet backend (same-family swap); anime-seg keeps
# its own forward, with the cross-check running independently on top.
# Disable with --no-cross-check-as-seg. No effect when cross_check is off.
reuse_as_seg: true
# Default OFF (server-edition): the dedicated segmenter keeps its own forward
# and the veto stays independent; enable with --cross-check-as-seg.
reuse_as_seg: false
# Cross-check forward precision, same semantics as segmentation.precision
# (bf16 via autocast: 2048 pass ~35s -> ~26s on a native-bf16 CPU).
precision: fp32
@@ -84,6 +100,11 @@ cross_check:
feather_sigma: 2.0 # zone-boundary feather, px
gate_lo: 0.70 # primary alpha below this -> fully exempt
gate_hi: 0.95 # primary alpha above this -> fully vetoable
# Complex-background (no-chroma) mode only: no hue-defined suspect zone, so the
# veto requires the second opinion itself to be confidently near-empty -- full
# strength at/below second_lo, fading to none at/above second_hi.
second_lo: 0.15
second_hi: 0.40
foreground:
enabled: true
@@ -103,7 +124,10 @@ foreground:
bg_confidence_weight: 0.70
despill:
enabled: true
# Off: despill has no positional/semantic guard, so a subject sharing the
# background's hue (blue suit on a blue backdrop) gets desaturated/hue-shifted
# (~ΔE 35, blue -> purple). Re-enable per-job only when edge spill matters.
enabled: false
edge_low: 0.005
strength: 0.92
edge_expand_radius: 2
+87
View File
@@ -0,0 +1,87 @@
# GPU lane: CUDA + bf16 across all three models. Full pipeline (BiRefNet seg +
# ViTMatte + HR-matting cross-check). Foreground estimation auto-uses the CuPy GPU
# backend when cupy is installed (see foreground.use_gpu, default on). despill is
# off (it false-positives on subjects sharing the background hue).
screen_color: null
chroma:
# true = flat-colour background (default). false = non-flat/complex background
# mode: no colour key; segmentation alone drives the trimap (mode forced to
# "seg") and the colour-keyed stages are bypassed (auto-detect, hue split,
# chroma suppression, despill). The cross-check veto still runs if enabled,
# using its hue-free gate (cross_check.second_lo/hi). The service selects this
# per request via background_mode=flat|complex; this is only the base default.
enabled: true
model:
model_name: hustvl/vitmatte-base-composition-1k
device: cuda
matting_method: vitmatte
fallback_to_chroma_alpha: false
precision: bf16
attn_query_chunk: 2048
segmentation:
enabled: true
backend: birefnet
model_name: ZhengPeng7/BiRefNet
device: cuda
input_size: 1024
precision: bf16
trimap:
sure_bg_threshold: 0.92
sure_fg_threshold: 0.12
unknown_radius_ratio: 0.012
fg_safe_radius_ratio: 0.006
seg_core_threshold: 0.60
seg_loose_threshold: 0.08
mode: directional
seg_low: 0.15
bg_hue_proj_min: 4.0
seg_force_fg: 0.98
alpha_post:
chroma_suppress: true
chroma_suppress_bg_low: 0.35
chroma_suppress_bg_high: 0.80
chroma_suppress_strength: 1.0
suppress_raw_lo: 0.85
suppress_raw_hi: 0.98
cross_check:
enabled: true
model_name: ZhengPeng7/BiRefNet_HR-matting
input_size: 2048
reuse_as_seg: false
precision: bf16
proj_min: 3.0
l_min: 45.0
feather_sigma: 2.0
gate_lo: 0.70
gate_hi: 0.95
# Complex-background (non-flat) mode only: no hue-defined suspect zone, so the
# veto requires the second opinion itself to be confidently near-empty -- full
# strength at/below second_lo, fading to none at/above second_hi.
second_lo: 0.15
second_hi: 0.40
foreground:
enabled: true
method: ml
ml_regularization: 0.00001
despill:
enabled: false
server:
# Request concurrency (HTTP service only). cpu_workers = thread-pool size: >1 runs
# several requests at once, overlapping their CPU stages while the GPU forwards
# stay serialized -- keeps the GPU fed. gpu_concurrency = how many forwards may
# co-schedule on the SMs at once: 2 lifts GPU-bound throughput (batch-1 underfills
# the SMs) but roughly DOUBLES GPU activation memory.
# * On a >=24 GB card (e.g. 4080 SUPER 32 GB): gpu_concurrency: 2 peaks ~19 GB.
# * On a <=16 GB card: keep gpu_concurrency: 1 or it will OOM at 2048 cross-check.
# Env vars BGFILTER_CPU_WORKERS / BGFILTER_GPU_CONCURRENCY override these per box.
cpu_workers: 8
gpu_concurrency: 2
@@ -0,0 +1,470 @@
# BGfilter 服务版设计与交付需求
本文档给接手实现服务化的 Agent 使用。目标是在 `BGfilter-server` 仓库内,把现有 BGfilter 抠图算法包装成一个可长期运行的 Python HTTP 服务,接口形态参考当前线上 `birefnet-service`,但算法仍使用本仓库的 BGfilter 管线。
## 目标
新增一个服务入口,使调用方可以通过 HTTP 上传图片并得到透明背景 PNG。
服务首版只暴露两个业务参数:
- `screen_color`:可选。背景色先验,格式为 `#RRGGBB`。不传或空值时沿用 BGfilter 默认行为,从图片边框自动检测纯色背景。
- `seg_model`:可选。分割模型选择,允许值为 `birefnet``anime-seg`。不传时使用仓库当前默认值 `birefnet`
其它算法参数首版不要暴露为 HTTP 参数,直接使用当前默认配置:
- `trimap.mode = directional`
- `despill.enabled = true`
- `model.matting_method = vitmatte`
- `model.model_name = hustvl/vitmatte-base-composition-1k`
- `model.device = cpu`
- `segmentation.device = cpu`
- `segmentation.input_size = 1024`
## 非目标
- 不把 BGfilter 算法移植到 Rust。
- 不在 HTTP 接口里暴露 `trimap_mode``despill`、阈值、设备、模型权重名等高级参数。
- 不复用当前线上 `birefnet-service` 的简单 alpha 直出算法;`seg_model=birefnet` 在本服务中只表示 BGfilter 管线里的语义分割后端。
- 不在每次请求里加载并释放模型。
- 不要求首版支持批量图片。
## 现有算法边界
BGfilter 当前核心入口在:
- `bgfilter/pipeline.py`
- `bgfilter/settings.py`
- `bgfilter/config.py`
- `bgfilter/segmentation.py`
- `bgfilter/vitmatte_infer.py`
完整管线是:
```text
输入 RGB
-> 背景色自动检测或使用 screen_color
-> chroma 背景置信度
-> 可选语义分割 mask,默认开启
-> trimap 生成,默认 directional
-> ViTMatte 预测 alpha
-> trimap 强约束
-> chroma-guided alpha suppression
-> alpha 小连通域/孔洞清理
-> pymatting foreground estimation
-> despill 去背景色溢色
-> 输出 RGBA PNG
```
当前默认分割后端已经是:
```text
segmentation.backend = birefnet
segmentation.model_name = ZhengPeng7/BiRefNet
```
`anime-seg` 对应权重为:
```text
segmentation.backend = anime-seg
segmentation.model_name = skytnt/anime-seg
```
`bgfilter/config.py` 里已有 `_SEG_MODELS``override_settings(..., seg_backend=...)`,服务层应复用这套映射,不要让 HTTP 调用方传 `model_name`
## HTTP 接口契约
### `GET /healthz`
返回 JSON,用于进程存活和基础配置检查。
建议响应:
```json
{
"ok": true,
"service": "bgfilter",
"version": "0.1.0",
"defaultSegModel": "birefnet",
"device": "cpu"
}
```
### `POST /remove-background`
请求类型:
```text
multipart/form-data
```
字段:
```text
file: UploadFile, required
screen_color: string, optional, "#RRGGBB"
seg_model: string, optional, "birefnet" | "anime-seg"
```
响应:
```text
200 image/png
```
响应体是透明背景 PNG。建议附带诊断 header:
```text
X-BGFilter-Elapsed-Ms: 总耗时毫秒
X-BGFilter-Seg-Model: 实际使用的 seg_model
X-BGFilter-Screen-Color: 实际请求传入的 screen_color,未传可省略或设为 auto
```
错误码建议:
```text
400 empty image
400 invalid image
400 invalid screen_color
400 invalid seg_model
413 image too large
500 inference failed
```
为兼容现有 Rust 调用方,`file` 字段名必须保持为 `file`
## 参数默认与校验
`screen_color`
- 允许省略、空字符串、`null` 等价于自动检测。
- 非空时必须匹配 `#RRGGBB`
- 大小写都可以,内部可以统一为大写或小写。
- 校验失败返回 `400 invalid screen_color`
`seg_model`
- 允许省略,默认 `birefnet`
- 允许值只包括 `birefnet``anime-seg`
- 可以兼容内部命名 `seg_backend`,但 HTTP 对外字段用 `seg_model`
- 校验失败返回 `400 invalid seg_model`
`MAX_IMAGE_PIXELS`
- 建议参考线上 `birefnet-service`,通过环境变量控制,例如 `BGFILTER_MAX_IMAGE_PIXELS`
- 默认建议先设为 `4194304`,即约 4MP。
- 超限返回 `413 image too large`
## 模型加载与缓存要求
服务必须做到模型只加载一次或按后端首次懒加载一次,不能每次请求都重新加载模型。
现有 `MattingPipeline` 已经有实例级懒加载:
- `self._runner` 缓存 `ViTMatteRunner`
- `self._segmenter` 缓存当前 settings 下的 segmenter
但如果每次请求新建 `MattingPipeline(settings)`,缓存会失效。因此服务版需要持有长期对象。
推荐实现一个 `PipelineManager` 或等价对象:
```text
长期缓存:
ViTMatteRunner: 按 model_name + device + matting_method 缓存
Segmenter: 按 backend + model_name + device + input_size 缓存
请求级参数:
screen_color: 只影响本次 chroma/despill,不影响模型缓存
seg_model: 选择对应 segmenter;首次使用某后端时加载,之后复用
```
推荐行为:
```text
服务启动:
可预加载默认 birefnet + ViTMatte,也可懒加载。
第一次 seg_model=birefnet
加载 BiRefNet segmenter 和 ViTMatte。
后续 seg_model=birefnet
复用已有 BiRefNet segmenter 和 ViTMatte。
第一次 seg_model=anime-seg
加载 AnimeSeg segmenterViTMatte 继续复用。
后续 seg_model=anime-seg
复用已有 AnimeSeg segmenter 和 ViTMatte。
```
可接受的首版简化方案:
- 长期缓存两个 `MattingPipeline` 实例:一个 `birefnet`,一个 `anime-seg`
- 每个实例内部复用自己的 `_runner``_segmenter`
- 如果这样做导致 ViTMatte 被加载两次,应优先改为共享 `ViTMatteRunner`,因为 ViTMatte 是两条分割后端共同使用的 matting 模型。
更推荐的实现是把 `MattingPipeline` 改造成可以注入共享 runner/segmenter,或新增服务专用 pipeline runner,避免重复加载 ViTMatte。
## 并发控制
首版建议采用全局 `asyncio.Lock`,与当前线上 `birefnet-service` 类似,保证同一进程内一次只跑一个推理请求。
原因:
- BGfilter 管线包含 ViTMatte、segmenter、pymatting、OpenCV/NumPy 后处理,CPU 和内存压力较大。
- CPU 默认部署时并发推理容易放大延迟和内存。
- 后续确认资源充足后,再考虑按模型或设备拆锁。
如果使用 Uvicorn 多 worker,每个 worker 都会各自加载一份模型,内存会成倍增长。首版建议 `--workers 1`
## 内存入口改造建议
当前 `MattingPipeline.run_image(...)` 主要面向 CLI 文件输入输出。服务可以先用临时文件桥接,但更推荐新增内存入口,减少磁盘 IO 和临时文件清理风险。
推荐新增内部函数:
```text
run_rgb(rgb: np.ndarray, settings: PipelineSettings, resources/cache) -> result
```
返回:
```text
corrected_rgb: np.ndarray uint8 HxWx3
alpha: np.ndarray float32 HxW, 0..1
metadata: dict
```
服务层再把它编码成 PNG
```text
RGBA = corrected_rgb + alpha
PIL.Image.save(..., format="PNG")
```
如果首版为了速度使用临时文件:
- 临时目录应使用 `tempfile.TemporaryDirectory()`
- 文件名不要使用上传原始文件名。
- 请求结束必须清理。
- 不要把 debug 输出默认打开。
## 推荐文件结构
可以参考线上 `birefnet-service` 的简洁形态,在仓库根目录新增:
```text
app.py
requirements.txt
```
也可以新增:
```text
bgfilter/service.py
```
用于放 `PipelineManager` 和内存运行逻辑。`app.py` 保持薄入口。
推荐分层:
```text
app.py
FastAPI 路由、参数校验、图片 decode/encode、错误码、计时、锁
bgfilter/service.py
PipelineManager、模型缓存、按请求组装 settings、调用算法管线
bgfilter/pipeline.py
尽量保留原 CLI 兼容行为;必要时新增 run_rgb
```
## 依赖
现有算法依赖保留。服务化需要补充:
```text
fastapi
uvicorn[standard]
python-multipart
```
现有 `requirements.txt` 中已经有:
```text
opencv-python
pillow
numpy
scipy
scikit-image
pymatting
torch
torchvision
transformers
accelerate
safetensors
huggingface_hub
timm
einops
kornia
onnxruntime
tqdm
typer
rich
PyYAML
```
如果部署在无 GUI Linux 服务器,`opencv-python-headless` 通常比 `opencv-python` 更合适,但替换前需要确认现有本地脚本是否依赖 GUI 能力。
## 环境变量
建议支持:
```text
BGFILTER_CONFIG=configs/default.yaml
BGFILTER_DEVICE=cpu
BGFILTER_MAX_IMAGE_PIXELS=4194304
BGFILTER_PRELOAD=1
HF_HOME=/opt/genarrative-image-host/model-cache/hf
HF_HUB_OFFLINE=1
TRANSFORMERS_OFFLINE=1
OMP_NUM_THREADS=4
MKL_NUM_THREADS=4
```
`BGFILTER_DEVICE` 应同时覆盖:
- `settings.model.device`
- `settings.segmentation.device`
如果机器没有 GPU,默认保持 `cpu`
## systemd 部署参考
参考当前线上 `genarrative-birefnet.service`,可以新增:
```ini
[Unit]
Description=Genarrative BGfilter background removal service
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
Environment=HF_HOME=/opt/genarrative-image-host/model-cache/hf
Environment=HF_HUB_OFFLINE=1
Environment=TRANSFORMERS_OFFLINE=1
Environment=BGFILTER_CONFIG=configs/default.yaml
Environment=BGFILTER_DEVICE=cpu
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
Environment=BGFILTER_PRELOAD=1
Environment=OMP_NUM_THREADS=4
Environment=MKL_NUM_THREADS=4
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
Restart=on-failure
RestartSec=5
MemoryMax=8G
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/opt/genarrative-image-host
[Install]
WantedBy=multi-user.target
```
端口只是建议值。真正接入时由部署方决定。
## 调用示例
默认参数:
```bash
curl -sS \
-F "file=@input.png" \
http://127.0.0.1:18083/remove-background \
-o output.png
```
指定背景色:
```bash
curl -sS \
-F "file=@input.png" \
-F "screen_color=#CFEFFF" \
http://127.0.0.1:18083/remove-background \
-o output.png
```
切换 Anime Seg
```bash
curl -sS \
-F "file=@input.png" \
-F "seg_model=anime-seg" \
http://127.0.0.1:18083/remove-background \
-o output.png
```
指定背景色并切换 Anime Seg:
```bash
curl -sS \
-F "file=@input.png" \
-F "screen_color=#CFEFFF" \
-F "seg_model=anime-seg" \
http://127.0.0.1:18083/remove-background \
-o output.png
```
## 与 Genarrative Rust 服务的接入预期
当前 Genarrative Rust BFF 已有远端抠图调用边界:
```text
POST {GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL}/remove-background
multipart field: file
response: image/png
```
BGfilter 服务应保持同样的 `POST /remove-background``file` 字段名。Rust 侧后续只需要在 multipart 中追加可选文本字段:
```text
screen_color
seg_model
```
如果 Rust 侧暂时不传新字段,本服务仍应可用,并按默认 `birefnet + 自动背景色` 执行。
## 验收要求
实现完成后至少验证:
1. `GET /healthz` 返回 `200` 和 JSON。
2. `POST /remove-background` 只传 `file` 可返回 `image/png`
3. `screen_color=#CFEFFF` 可正常返回 `image/png`
4. `seg_model=birefnet` 可正常返回 `image/png`
5. `seg_model=anime-seg` 可正常返回 `image/png`
6. 非法 `seg_model` 返回 `400`
7. 非法 `screen_color` 返回 `400`
8. 空文件或非图片返回 `400`
9. 超过 `BGFILTER_MAX_IMAGE_PIXELS` 的图片返回 `413`
10. 同一进程连续请求同一 `seg_model` 时不会重复加载对应模型。
11.`birefnet` 切到 `anime-seg` 后,再切回 `birefnet`,已加载模型可复用。
12. Uvicorn `--workers 1` 下内存不随请求次数持续增长。
建议在日志中打印模型加载事件,但不要每个请求打印完整配置或文件内容。日志中不要输出用户上传图片数据。
## 完成定义
交付时应包含:
- 可运行的 FastAPI 服务入口。
- 更新后的依赖文件。
- 本地启动命令说明。
- 至少一组 curl 示例。
- 若新增内存入口,应保持原 CLI 文件入口不破坏。
- 若为了服务化调整 pipeline,应保证默认 CLI 行为仍按当前配置工作。
完成后,不看内部代码也应能通过本文档的 HTTP 契约和 curl 示例直接调用服务。
+5 -1
View File
@@ -1,4 +1,4 @@
opencv-python
opencv-python-headless
pillow
numpy
scipy
@@ -18,3 +18,7 @@ tqdm
typer
rich
PyYAML
fastapi
uvicorn[standard]
python-multipart
httpx

Some files were not shown because too many files have changed in this diff Show More