BGfilter/DEPLOY.md

Deploying BgFilter to a Linux server

This guide covers everything needed to run the BgFilter HTTP service (POST /remove-background, GET /healthz) on a Linux server. It assumes a CPU deployment (the default); GPU notes are called out where relevant.

The service entry point is app.py (uvicorn app:app). See README.md for the API contract and algorithm details.

---

1. Prerequisites

• Linux x86-64 (tested on Ubuntu; any modern distro works).
• Python 3.11–3.13 recommended. Newer (3.14) can work, but some ML wheels may not be published for it yet — stick to a version with prebuilt wheels for torch, onnxruntime, opencv, pymatting.
• git, and the venv module (python3-venv on Debian/Ubuntu).
• ~3 GB free disk for the Python env + ~1 GB for model weights.
• Outbound network once to fetch dependencies and model weights (the running service can then run fully offline).

sudo apt update
sudo apt install -y python3-venv python3-pip git

2. Get the code

sudo mkdir -p /opt/genarrative-image-host/bgfilter-service
sudo chown "$USER" /opt/genarrative-image-host/bgfilter-service
cd /opt/genarrative-image-host/bgfilter-service
git clone <repo-url> .            # or copy the repo here; use the server-edition branch

3. Python environment

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip

4. Install dependencies (CPU)

Install CPU-only PyTorch first from the CPU wheel index — otherwise pip pulls the default CUDA build and ~2–3 GB of nvidia-* packages you don't need on a CPU box:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

On a headless server use opencv-python-headless instead of opencv-python (the GUI build needs libGL.so.1 etc. that servers usually lack):

sed -i 's/^opencv-python$/opencv-python-headless/' requirements.txt

Then install the rest. torch/torchvision are already satisfied, so this won't re-pull CUDA:

pip install -r requirements.txt

Verify the stack imports and torch is the CPU build (cuda= None):

python -c "import torch,cv2,fastapi,transformers,pymatting,onnxruntime,kornia,timm; \
print('torch', torch.__version__, 'cuda=', torch.version.cuda)"

Behind the Great Firewall, add -i https://pypi.tuna.tsinghua.edu.cn/simple to the second pip install for speed.

5. Provide the model weights

Three models are used: hustvl/vitmatte-base-composition-1k (matting), ZhengPeng7/BiRefNet (default segmenter), skytnt/anime-seg (optional segmenter). Pick one of the two layouts below.

Option A — HuggingFace cache + offline mode (recommended)

Download once into a cache dir, then run offline:

export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
hf download hustvl/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet
hf download skytnt/anime-seg

At runtime set HF_HOME to the same path plus HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 (see §6). You can also download on another machine and copy the hub/ folder over.

Option B — plain folders inside the project

Drop each model into models/<repo-basename>/ and the loader uses it directly (no HF cache, no network). Absent folders fall back to the repo id, so this is opt-in:

hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet                 --local-dir models/BiRefNet
hf download skytnt/anime-seg                     --local-dir models/anime-seg

Override the base directory with BGFILTER_WEIGHTS_DIR. models/ is gitignored.

Downloading behind a firewall (CN networks)

If direct HuggingFace access is blocked, use the mirror with the local proxy bypassed and Xet disabled — the reliable combination:

export HF_ENDPOINT=https://hf-mirror.com   # domestic mirror
export NO_PROXY='*'                        # bypass the proxy; the mirror is direct
export HF_HUB_DISABLE_XET=1                 # repos are Xet-backed; force classic HTTP

This avoids two failure modes: (1) mirror + an overseas proxy makes the mirror redirect back to huggingface.co, which recent huggingface_hub rejects with FileMetadataError; (2) with hf-xet installed the Xet path fails instantly.

6. Configuration (environment variables)

Variable	Default	Purpose
BGFILTER_CONFIG	configs/default.yaml	Pipeline config file
BGFILTER_DEVICE	cpu	Overrides both model and segmentation device (cuda for GPU)
BGFILTER_MAX_IMAGE_PIXELS	4194304	Reject larger inputs with 413 (~4 MP)
BGFILTER_PRELOAD	1	Load default models at startup (first request isn't cold)
BGFILTER_WEIGHTS_DIR	models	Where local weight folders are looked up (Option B)
HF_HOME	—	HF cache location (Option A)
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE	—	Set to 1 in production once weights are cached
OMP_NUM_THREADS / MKL_NUM_THREADS	—	Cap CPU threads (e.g. 4)

7. Run the service

python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1

• Keep --workers 1. Each worker loads its own full copy of the models; more workers multiply memory (an 8 GB box will OOM). The service already serializes inference with a global lock.
• --host 127.0.0.1 exposes it to the local host only (intended for a co-located caller). See §10 for remote access.

8. Run as a systemd service

/etc/systemd/system/genarrative-bgfilter.service:

[Unit]
Description=Genarrative BgFilter background removal service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
Environment=HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
Environment=HF_HUB_OFFLINE=1
Environment=TRANSFORMERS_OFFLINE=1
Environment=HF_HUB_DISABLE_XET=1
Environment=BGFILTER_CONFIG=configs/default.yaml
Environment=BGFILTER_DEVICE=cpu
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
Environment=BGFILTER_PRELOAD=1
Environment=OMP_NUM_THREADS=4
Environment=MKL_NUM_THREADS=4
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
Restart=on-failure
RestartSec=5
MemoryMax=8G
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/opt/genarrative-image-host

[Install]
WantedBy=multi-user.target

If you use Option B (local models/ folders) instead of the HF cache, drop the HF_HOME/offline lines — those weights need no cache or network.

sudo systemctl daemon-reload
sudo systemctl enable --now genarrative-bgfilter
sudo systemctl status genarrative-bgfilter
journalctl -u genarrative-bgfilter -f          # wait for "Application startup complete"

9. Verify

curl -s http://127.0.0.1:18083/healthz
# {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu"}

curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
  http://127.0.0.1:18083/remove-background -o out.png

10. Exposing the service

The unit binds 127.0.0.1 — intended to be called by a co-located client (e.g. the Genarrative Rust BFF via GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL). The service has no authentication. To reach it from other hosts:

• Reverse proxy (recommended): put nginx/caddy in front, terminate TLS, add auth/rate-limiting, and forward to 127.0.0.1:18083.
• Direct: change --host to 0.0.0.0 and restrict source IPs with a firewall. Do not expose it unauthenticated to the public internet.

11. Capacity & performance

• CPU throughput is ~45 s per 1.5 MP image with the default (base ViTMatte + BiRefNet); requests are serialized by the global lock, so ~1–2 images/min per instance. Bottleneck is full-resolution ViTMatte + pymatting foreground estimation.
• For throughput: use a GPU (BGFILTER_DEVICE=cuda, install a CUDA torch build) — roughly an order of magnitude faster — and/or run multiple instances behind a load balancer (each with --workers 1).
• Memory: one worker holds all three models (~1 GB weights + runtime). MemoryMax=8G is a safe ceiling for a single CPU worker.

12. Troubleshooting

Symptom	Cause / fix
pip pulls nvidia-*/cuda-* (huge)	You skipped the CPU index in §4. Install torch from --index-url https://download.pytorch.org/whl/cpu.
ImportError: libGL.so.1 on import cv2	Using opencv-python on a headless box. Switch to opencv-python-headless (§4).
ensurepip is not available on venv create	Install python3-venv (§1).
FileMetadataError: ... not on huggingface.co while downloading	hf-mirror + a proxy conflict. Bypass the proxy (NO_PROXY='*') and set HF_HUB_DISABLE_XET=1 (§5).
LocalEntryNotFoundError, 0 bytes	Xet download path failing; set HF_HUB_DISABLE_XET=1.
503 service not ready	Models still loading at startup; wait for "Application startup complete".
OOM / killed under load	More than one worker, or MemoryMax too low. Keep --workers 1.
400 could not auto-detect background colour	Input has no clean flat border; pass screen_color=#RRGGBB.