BGfilter/DEPLOY.md

Deploying BgFilter to a Linux server

中文版：DEPLOY_ZH.md

This guide covers everything needed to run the BgFilter HTTP service (POST /remove-background, GET /healthz) on a Linux server. It assumes a CPU deployment (the default); GPU notes are called out where relevant.

The service entry point is app.py (uvicorn app:app). See README.md for the API contract and algorithm details.

---

1. Prerequisites

• Linux x86-64 (tested on Ubuntu; any modern distro works).
• Python 3.11–3.13 recommended. Newer (3.14) can work, but some ML wheels may not be published for it yet — stick to a version with prebuilt wheels for torch, onnxruntime, opencv, pymatting.
• git, and the venv module (python3-venv on Debian/Ubuntu).
• ~3 GB free disk for the Python env + ~1 GB for model weights.
• Outbound network once to fetch dependencies and model weights (the running service can then run fully offline).

sudo apt update
sudo apt install -y python3-venv python3-pip git

2. Get the code

sudo mkdir -p /opt/genarrative-image-host/bgfilter-service
sudo chown "$USER" /opt/genarrative-image-host/bgfilter-service
cd /opt/genarrative-image-host/bgfilter-service
git clone <repo-url> .            # or copy the repo here; use the server-edition branch

3. Python environment

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip

4. Install dependencies (CPU)

Install CPU-only PyTorch first from the CPU wheel index — otherwise pip pulls the default CUDA build and ~2–3 GB of nvidia-* packages you don't need on a CPU box:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

requirements.txt already pins opencv-python-headless (the GUI build needs libGL.so.1 etc. that headless servers lack), so no change is needed. Then install the rest — torch/torchvision are already satisfied, so this won't re-pull CUDA:

pip install -r requirements.txt

Verify the stack imports and torch is the CPU build (cuda= None):

python -c "import torch,cv2,fastapi,transformers,pymatting,onnxruntime,kornia,timm; \
print('torch', torch.__version__, 'cuda=', torch.version.cuda)"

Behind the Great Firewall, add -i https://pypi.tuna.tsinghua.edu.cn/simple to the second pip install for speed.

5. Provide the model weights

Three models are used by default: hustvl/vitmatte-base-composition-1k (matting), ZhengPeng7/BiRefNet (default segmenter), skytnt/anime-seg (optional segmenter). A fourth model, ZhengPeng7/BiRefNet_HR-matting (~425 MB), is only needed if you enable the cross-check veto — see the note at the end of this section.

Quick path (recommended): run the bundled script — no arguments — to download every model into models/ via the hf-mirror (proxy bypassed, Xet disabled, already-present folders skipped). Afterwards the app has full functionality offline:

python scripts/fetch_weights.py

The two manual layouts below are for finer control (e.g. a shared HF cache, or picking specific models).

Option A — HuggingFace cache + offline mode (recommended)

Download once into a cache dir, then run offline:

export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
hf download hustvl/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet
hf download skytnt/anime-seg
hf download ZhengPeng7/BiRefNet_HR-matting   # only if using cross-check (see below)

At runtime set HF_HOME to the same path plus HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 (see §6). You can also download on another machine and copy the hub/ folder over.

Option B — plain folders inside the project

Drop each model into models/<repo-basename>/ and the loader uses it directly (no HF cache, no network). Absent folders fall back to the repo id, so this is opt-in.

The folder name must match the repo basename exactly — that is how the loader finds it (ZhengPeng7/BiRefNet → models/BiRefNet). Required files per model:

<project-root>/
└── models/
    ├── vitmatte-base-composition-1k/     # ViTMatte matting model
    │   ├── config.json
    │   ├── preprocessor_config.json
    │   └── pytorch_model.bin             # ~369 MB
    ├── BiRefNet/                         # default segmenter (trust_remote_code)
    │   ├── config.json
    │   ├── BiRefNet_config.py            # custom code — required by trust_remote_code
    │   ├── birefnet.py                   # custom code — required by trust_remote_code
    │   └── model.safetensors             # ~424 MB
    ├── anime-seg/                        # optional segmenter (seg_model=anime-seg)
    │   └── isnetis.onnx                  # ~168 MB
    └── BiRefNet_HR-matting/              # optional — only for the cross-check veto
        ├── config.json
        ├── BiRefNet_config.py            # custom code — required by trust_remote_code
        ├── birefnet.py                   # custom code — required by trust_remote_code
        └── model.safetensors             # ~425 MB

Notes:

• ViTMatte: needs config.json, preprocessor_config.json, and the weights (pytorch_model.bin; model.safetensors also works if present).
• BiRefNet: the two .py files are mandatory — trust_remote_code executes them to build the model. Keep config.json + model.safetensors alongside them. (The repo's handler.py / requirements.txt / README.md are not needed.)
• anime-seg: only isnetis.onnx is read; nothing else is required.
• anime-seg and BiRefNet_HR-matting are optional — omit their folders unless you use seg_model=anime-seg / the cross-check veto respectively.
• The cpu-fast lane (§6) loads only ViTMatte + BiRefNet; on an 8 GB box using that lane you can skip anime-seg and BiRefNet_HR-matting entirely. (scripts/fetch_weights.py still fetches all four for full functionality.)

The easiest way to populate the folders is to let hf download fetch them (it also pulls a couple of unused files like README.md, which is harmless):

hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet                 --local-dir models/BiRefNet
hf download skytnt/anime-seg                     --local-dir models/anime-seg
hf download ZhengPeng7/BiRefNet_HR-matting       --local-dir models/BiRefNet_HR-matting  # cross-check only

Override the base directory with BGFILTER_WEIGHTS_DIR. models/ is gitignored.

Cross-check veto (on by default)

The pipeline runs a second, high-resolution matting model (ZhengPeng7/BiRefNet_HR-matting, ~425 MB) as a "second opinion" that clears background residue trapped between hair strands. It is ON by default (both CLI and HTTP service) and, with reuse-as-seg, that same forward doubles as the segmentation mask on the birefnet backend — so it replaces the primary seg model rather than adding to it. You must provision this model (Option A or B above):

• ⚠️ On an offline server (HF_HUB_OFFLINE=1) it must be present before the first request — otherwise every default request fails, since it cannot download the model at runtime.
• Turn it off service-wide with BGFILTER_CROSS_CHECK=0, or override per request with the cross_check form field (true/false).
• The service preloads it at startup with the rest of the warm set; on the birefnet backend the redundant primary segmenter is skipped.

Downloading behind a firewall (CN networks)

If direct HuggingFace access is blocked, use the mirror with the local proxy bypassed and Xet disabled — the reliable combination:

export HF_ENDPOINT=https://hf-mirror.com   # domestic mirror
export NO_PROXY='*'                        # bypass the proxy; the mirror is direct
export HF_HUB_DISABLE_XET=1                 # repos are Xet-backed; force classic HTTP

This avoids two failure modes: (1) mirror + an overseas proxy makes the mirror redirect back to huggingface.co, which recent huggingface_hub rejects with FileMetadataError; (2) with hf-xet installed the Xet path fails instantly.

6. Configuration (environment variables)

Variable	Default	Purpose
BGFILTER_CONFIG	configs/default.yaml	Pipeline config file
BGFILTER_DEVICE	cpu	Overrides both model and segmentation device (cuda for GPU)
BGFILTER_MAX_IMAGE_PIXELS	4194304	Reject larger inputs with 413 (~4 MP)
BGFILTER_PRELOAD	1	Load default models at startup (first request isn't cold)
BGFILTER_CROSS_CHECK	config (on)	Force the cross-check veto on/off (0 disables; needs BiRefNet_HR-matting)
BGFILTER_WEIGHTS_DIR	models	Where local weight folders are looked up (Option B)
HF_HOME	—	HF cache location (Option A)
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE	—	Set to 1 in production once weights are cached
OMP_NUM_THREADS / MKL_NUM_THREADS	—	Cap CPU threads (e.g. 4)

Config file: default.yaml vs the cpu-fast lane

BGFILTER_CONFIG selects the pipeline config. Two are shipped:

• configs/default.yaml — full quality. Cross-check veto on (loads the extra BiRefNet_HR-matting at 2048 as a second opinion), best edges. Heavier: on large inputs the peak working set runs well into double-digit GB, so it wants a roomy box (or a GPU).
• configs/cpu-fast.yaml — the lean CPU lane, tuned for an 8 GB box. fp32, BiRefNet segmentation only, no cross-check, no anime-seg, chunked attention (attn_query_chunk: 512). Loads only two weights (base ViTMatte + BiRefNet). Measured peak working set: ~5.6 GB @ 2048×1152 (2.4 MP), ~7.1 GB @ 2048×2048 (4.2 MP); ~14–15 s per 1.5 MP image.

On a ≤8 GB server, set BGFILTER_CONFIG=configs/cpu-fast.yaml. The default lane plus a large input is exactly what drives ViTMatte's attention transient into an OOM — the worker is killed (SIGKILL) and a fronting nginx returns 502 while the app log shows DefaultCPUAllocator: can't allocate memory.

Two model knobs live in the config file (model: section), not env vars:

Config key	Default	Purpose
model.precision	fp32	fp32 or bf16. bf16 is used only on hardware with native kernels (CUDA, or a CPU with AVX512-BF16/AMX); otherwise it warns and falls back to fp32. Also settable per model under segmentation: / cross_check:.
model.attn_query_chunk	2048	Query-row chunk for ViTMatte's global attention. Smaller (e.g. 512) caps the O(N²) memory transient for a few % more compute; output is bit-for-bit identical. 0 disables chunking (stock one-shot attention).

7. Run the service

python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1

• Keep --workers 1. Each worker loads its own full copy of the models; more workers multiply memory (an 8 GB box will OOM). The service already serializes inference with a global lock.
• --host 127.0.0.1 exposes it to the local host only (intended for a co-located caller). See §10 for remote access.

8. Run as a systemd service

/etc/systemd/system/genarrative-bgfilter.service:

[Unit]
Description=Genarrative BgFilter background removal service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
Environment=HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
Environment=HF_HUB_OFFLINE=1
Environment=TRANSFORMERS_OFFLINE=1
Environment=HF_HUB_DISABLE_XET=1
Environment=BGFILTER_CONFIG=configs/default.yaml
Environment=BGFILTER_DEVICE=cpu
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
Environment=BGFILTER_PRELOAD=1
Environment=OMP_NUM_THREADS=4
Environment=MKL_NUM_THREADS=4
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
Restart=on-failure
RestartSec=5
MemoryMax=8G
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/opt/genarrative-image-host

[Install]
WantedBy=multi-user.target

On a ≤8 GB box, change the config line to the lean lane: Environment=BGFILTER_CONFIG=configs/cpu-fast.yaml (see §6) — the default lane can OOM on large inputs.

If you use Option B (local models/ folders) instead of the HF cache, drop the HF_HOME/offline lines — those weights need no cache or network.

sudo systemctl daemon-reload
sudo systemctl enable --now genarrative-bgfilter
sudo systemctl status genarrative-bgfilter
journalctl -u genarrative-bgfilter -f          # wait for "Application startup complete"

9. Verify

curl -s http://127.0.0.1:18083/healthz
# {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu"}

curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
  http://127.0.0.1:18083/remove-background -o out.png

10. Exposing the service

The unit binds 127.0.0.1 — intended to be called by a co-located client (e.g. the Genarrative Rust BFF via GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL). The service has no authentication. To reach it from other hosts:

• Reverse proxy (recommended): put nginx/caddy in front, terminate TLS, add auth/rate-limiting, and forward to 127.0.0.1:18083.
• Direct: change --host to 0.0.0.0 and restrict source IPs with a firewall. Do not expose it unauthenticated to the public internet.

11. Capacity & performance

• CPU throughput: the cpu-fast lane does ~14–15 s per 1.5 MP image (base ViTMatte + one BiRefNet@1024, chunked attention). The default lane is slower — the cross-check adds a second HR-matting@2048 forward. Requests are serialized by the global lock, so plan for a handful of images/min per instance.
• For throughput: use a GPU (BGFILTER_DEVICE=cuda, install a CUDA torch build) — roughly an order of magnitude faster — and/or run multiple instances behind a load balancer (each with --workers 1).
• Memory: chunked attention (attn_query_chunk) caps ViTMatte's O(N²) transient, so the peak is set by dense compute, not the attention map. On the cpu-fast lane the measured peak is ~5.6 GB @ 2.4 MP / ~7.1 GB @ 4.2 MP — MemoryMax=8G with the default 4 MP input cap is a safe ceiling for a single CPU worker. The default lane (cross-check on) needs more headroom.
• Keeping RSS flat (glibc). After each image the service calls glibc malloc_trim(0) (bgfilter/memtune.py) to hand a large BiRefNet@2048 forward's freed activations back to the OS — without it, ptmalloc keeps them in the arena and RSS ratchets up across requests. For lower memory and better CPU throughput, preload jemalloc (PyTorch's recommendation for CPU inference) and let it decay dirty pages aggressively:

  # in the [Service] section of the systemd unit
  Environment=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
  Environment=MALLOC_CONF=background_thread:true,dirty_decay_ms:0,muzzy_decay_ms:0
  

  (apt install libjemalloc2; tcmalloc via libtcmalloc_minimal.so.4 works too.) jemalloc/tcmalloc manage their own release, so malloc_trim simply no-ops under them. If you cannot preload an allocator, MALLOC_ARENA_MAX=2 and MALLOC_TRIM_THRESHOLD_=0 in the unit environment curb ptmalloc retention.

12. Troubleshooting

Symptom	Cause / fix
pip pulls nvidia-*/cuda-* (huge)	You skipped the CPU index in §4. Install torch from --index-url https://download.pytorch.org/whl/cpu.
ImportError: libGL.so.1 on import cv2	Using opencv-python on a headless box. Switch to opencv-python-headless (§4).
ensurepip is not available on venv create	Install python3-venv (§1).
FileMetadataError: ... not on huggingface.co while downloading	hf-mirror + a proxy conflict. Bypass the proxy (NO_PROXY='*') and set HF_HUB_DISABLE_XET=1 (§5).
LocalEntryNotFoundError, 0 bytes	Xet download path failing; set HF_HUB_DISABLE_XET=1.
503 service not ready	Models still loading at startup; wait for "Application startup complete".
500 inference failed right after enabling BGFILTER_CROSS_CHECK (offline)	BiRefNet_HR-matting not provisioned; download it into the cache/models/ (§5).
OOM / killed under load	More than one worker, or MemoryMax too low. Keep --workers 1.
Worker killed (SIGKILL) / 502 on a large image; log shows DefaultCPUAllocator: can't allocate memory	ViTMatte OOM on a big input. Use BGFILTER_CONFIG=configs/cpu-fast.yaml (§6), and/or lower BGFILTER_MAX_IMAGE_PIXELS.
413/large upload rejected at the proxy	nginx client_max_body_size (default 1 MB) — raise it in the server block (e.g. client_max_body_size 12m;); this is separate from the app's BGFILTER_MAX_IMAGE_PIXELS.
400 could not auto-detect background colour	Input has no clean flat border; pass screen_color=#RRGGBB.