From 9931980c72f2b9daf4cb13d730dd1e4bb702c062 Mon Sep 17 00:00:00 2001 From: Linghong Date: Mon, 6 Jul 2026 10:57:27 +0000 Subject: [PATCH] Update Deploy Docs --- DEPLOY.md | 49 ++++++++++++++++++++++++++++++++++++++++++++----- DEPLOY_ZH.md | 42 ++++++++++++++++++++++++++++++++++++++---- 2 files changed, 82 insertions(+), 9 deletions(-) diff --git a/DEPLOY.md b/DEPLOY.md index 194cac1..cff4421 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -143,6 +143,9 @@ Notes: - **anime-seg**: only `isnetis.onnx` is read; nothing else is required. - `anime-seg` and `BiRefNet_HR-matting` are optional — omit their folders unless you use `seg_model=anime-seg` / the cross-check veto respectively. +- The **`cpu-fast` lane** (§6) loads only **ViTMatte + BiRefNet**; on an 8 GB box + using that lane you can skip `anime-seg` and `BiRefNet_HR-matting` entirely. + (`scripts/fetch_weights.py` still fetches all four for full functionality.) The easiest way to populate the folders is to let `hf download` fetch them (it also pulls a couple of unused files like `README.md`, which is harmless): @@ -202,6 +205,32 @@ redirect back to `huggingface.co`, which recent `huggingface_hub` rejects with | `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | Set to `1` in production once weights are cached | | `OMP_NUM_THREADS` / `MKL_NUM_THREADS` | — | Cap CPU threads (e.g. `4`) | +### Config file: `default.yaml` vs the `cpu-fast` lane + +`BGFILTER_CONFIG` selects the pipeline config. Two are shipped: + +- **`configs/default.yaml`** — full quality. Cross-check veto **on** (loads the extra + `BiRefNet_HR-matting` at 2048 as a second opinion), best edges. Heavier: on large + inputs the peak working set runs well into double-digit GB, so it wants a roomy box + (or a GPU). +- **`configs/cpu-fast.yaml`** — the lean CPU lane, tuned for an **8 GB box**. fp32, + BiRefNet segmentation only, **no cross-check, no anime-seg**, chunked attention + (`attn_query_chunk: 512`). Loads only **two** weights (base ViTMatte + BiRefNet). + Measured peak working set: **~5.6 GB @ 2048×1152 (2.4 MP)**, ~7.1 GB @ 2048×2048 + (4.2 MP); ~14–15 s per 1.5 MP image. + +> **On a ≤8 GB server, set `BGFILTER_CONFIG=configs/cpu-fast.yaml`.** The default lane +> plus a large input is exactly what drives ViTMatte's attention transient into an +> OOM — the worker is killed (SIGKILL) and a fronting nginx returns **502** while the +> app log shows `DefaultCPUAllocator: can't allocate memory`. + +Two model knobs live in the **config file** (`model:` section), not env vars: + +| Config key | Default | Purpose | +| --- | --- | --- | +| `model.precision` | `fp32` | `fp32` or `bf16`. bf16 is used **only** on hardware with native kernels (CUDA, or a CPU with AVX512-BF16/AMX); otherwise it warns and falls back to fp32. Also settable per model under `segmentation:` / `cross_check:`. | +| `model.attn_query_chunk` | `2048` | Query-row chunk for ViTMatte's global attention. Smaller (e.g. `512`) caps the O(N²) memory transient for a few % more compute; output is bit-for-bit identical. `0` disables chunking (stock one-shot attention). | + ## 7. Run the service ```bash @@ -250,6 +279,10 @@ ReadWritePaths=/opt/genarrative-image-host WantedBy=multi-user.target ``` +On a ≤8 GB box, change the config line to the lean lane: +`Environment=BGFILTER_CONFIG=configs/cpu-fast.yaml` (see §6) — the default lane can +OOM on large inputs. + If you use Option B (local `models/` folders) instead of the HF cache, drop the `HF_HOME`/offline lines — those weights need no cache or network. @@ -283,14 +316,18 @@ service has **no authentication**. To reach it from other hosts: ## 11. Capacity & performance -- CPU throughput is ~**45 s per 1.5 MP image** with the default (base ViTMatte + - BiRefNet); requests are serialized by the global lock, so ~1–2 images/min per - instance. Bottleneck is full-resolution ViTMatte + pymatting foreground estimation. +- CPU throughput: the **`cpu-fast` lane** does ~**14–15 s per 1.5 MP image** (base + ViTMatte + one BiRefNet@1024, chunked attention). The **default lane** is slower — + the cross-check adds a second HR-matting@2048 forward. Requests are serialized by + the global lock, so plan for a handful of images/min per instance. - For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) — roughly an order of magnitude faster — and/or run multiple instances behind a load balancer (each with `--workers 1`). -- Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G` - is a safe ceiling for a single CPU worker. +- Memory: chunked attention (`attn_query_chunk`) caps ViTMatte's O(N²) transient, so + the peak is set by dense compute, not the attention map. On the `cpu-fast` lane the + measured peak is **~5.6 GB @ 2.4 MP / ~7.1 GB @ 4.2 MP** — `MemoryMax=8G` with the + default 4 MP input cap is a safe ceiling for a single CPU worker. The default lane + (cross-check on) needs more headroom. - **Keeping RSS flat (glibc).** After each image the service calls glibc `malloc_trim(0)` (`bgfilter/memtune.py`) to hand a large BiRefNet@2048 forward's freed activations back to the OS — without it, ptmalloc keeps them in the arena @@ -319,4 +356,6 @@ service has **no authentication**. To reach it from other hosts: | `503 service not ready` | Models still loading at startup; wait for "Application startup complete". | | `500 inference failed` right after enabling `BGFILTER_CROSS_CHECK` (offline) | `BiRefNet_HR-matting` not provisioned; download it into the cache/`models/` (§5). | | OOM / killed under load | More than one worker, or `MemoryMax` too low. Keep `--workers 1`. | +| Worker `killed` (SIGKILL) / `502` on a large image; log shows `DefaultCPUAllocator: can't allocate memory` | ViTMatte OOM on a big input. Use `BGFILTER_CONFIG=configs/cpu-fast.yaml` (§6), and/or lower `BGFILTER_MAX_IMAGE_PIXELS`. | +| `413`/large upload rejected at the proxy | nginx `client_max_body_size` (default 1 MB) — raise it in the server block (e.g. `client_max_body_size 12m;`); this is separate from the app's `BGFILTER_MAX_IMAGE_PIXELS`. | | `400 could not auto-detect background colour` | Input has no clean flat border; pass `screen_color=#RRGGBB`. | diff --git a/DEPLOY_ZH.md b/DEPLOY_ZH.md index 7cae7d4..b6b2647 100644 --- a/DEPLOY_ZH.md +++ b/DEPLOY_ZH.md @@ -135,6 +135,9 @@ hf download ZhengPeng7/BiRefNet_HR-matting # 仅在使用 cross-check 时（ - **anime-seg**：只读 `isnetis.onnx`，其它都不需要。 - `anime-seg` 和 `BiRefNet_HR-matting` 都是可选的——不用 `seg_model=anime-seg` / cross-check 否决时，各自的文件夹可以不放。 +- **`cpu-fast` lane**（§6）只加载 **ViTMatte + BiRefNet**；8 GB 机器用这条 lane 时， + `anime-seg` 和 `BiRefNet_HR-matting` 都可以完全不放。 + （`scripts/fetch_weights.py` 仍会把 4 个都下齐，以保证完整功能。） 最省事的填充方式是用 `hf download`（它也会顺带拉几个用不到的文件如 `README.md`，无害）： @@ -187,6 +190,29 @@ export HF_HUB_DISABLE_XET=1 # 这些 repo 是 Xet 存储；强 | `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | 权重缓存好后，生产设为 `1` | | `OMP_NUM_THREADS` / `MKL_NUM_THREADS` | — | 限制 CPU 线程数（如 `4`）| +### 配置文件：`default.yaml` 与 `cpu-fast` lane + +`BGFILTER_CONFIG` 决定用哪个管线配置。仓库自带两个： + +- **`configs/default.yaml`** —— 完整画质。cross-check 否决**默认开**（额外加载 + `BiRefNet_HR-matting`@2048 作第二意见），边缘最好。较重：大图时峰值工作集轻松到 + 两位数 GB，需要内存宽裕的机器（或 GPU）。 +- **`configs/cpu-fast.yaml`** —— 精简 CPU lane，为 **8 GB 机器**调校。fp32、只用 BiRefNet + 分割、**无 cross-check、无 anime-seg**、分块注意力（`attn_query_chunk: 512`）。只加载 + **两个**权重（base ViTMatte + BiRefNet）。实测峰值工作集：**2048×1152（2.4 MP）约 5.6 GB**、 + 2048×2048（4.2 MP）约 7.1 GB；约 14–15 秒/张（1.5 MP）。 + +> **≤8 GB 的服务器请设 `BGFILTER_CONFIG=configs/cpu-fast.yaml`。** 默认 lane + 大图 +> 正是把 ViTMatte 注意力瞬时内存推到 OOM 的组合——worker 被 kill（SIGKILL），前面的 +> nginx 返回 **502**，而应用日志显示 `DefaultCPUAllocator: can't allocate memory`。 + +有两个模型旋钮放在**配置文件**里（`model:` 段），不是环境变量： + +| 配置项 | 默认 | 作用 | +| --- | --- | --- | +| `model.precision` | `fp32` | `fp32` 或 `bf16`。bf16 **仅**在有原生内核的硬件上启用（CUDA，或带 AVX512-BF16/AMX 的 CPU）；否则告警并回退 fp32。也可在 `segmentation:` / `cross_check:` 下按模型单独设。 | +| `model.attn_query_chunk` | `2048` | ViTMatte 全局注意力的 query 行分块。调小（如 `512`）以几个百分点的额外算力换掉 O(N²) 内存瞬时尖峰；输出逐比特相同。设 `0` 关闭分块（走原始一次性注意力）。 | + ## 7. 运行服务 ```bash @@ -233,6 +259,9 @@ ReadWritePaths=/opt/genarrative-image-host WantedBy=multi-user.target ``` +≤8 GB 机器请把配置那行改成精简 lane： +`Environment=BGFILTER_CONFIG=configs/cpu-fast.yaml`（见 §6）——默认 lane 大图会 OOM。 + 若用方案 B（本地 `models/` 文件夹）而非 HF 缓存，去掉 `HF_HOME` 和离线那几行—— 那种权重不需要缓存也不联网。 @@ -266,12 +295,15 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \ ## 11. 容量与性能 -- CPU 吞吐约 **45 秒/张（1.5 MP）**，用默认配置（base ViTMatte + BiRefNet）；请求被全局锁 - 串行，所以单实例约 1–2 张/分钟。瓶颈是全分辨率的 ViTMatte 和 pymatting 前景估计。 +- CPU 吞吐：**`cpu-fast` lane** 约 **14–15 秒/张（1.5 MP）**（base ViTMatte + 一个 + BiRefNet@1024，分块注意力）。**默认 lane** 更慢——cross-check 会多跑一次 HR-matting@2048 + 前向。请求被全局锁串行，单实例按每分钟几张估算。 - 要提吞吐：上 GPU（`BGFILTER_DEVICE=cuda`，装 CUDA 版 torch），大约快一个数量级； 和/或多实例 + 负载均衡（每个都 `--workers 1`）。 -- 内存：单个 worker 持有全部三个模型（约 1 GB 权重 + 运行时）。单 CPU worker 用 - `MemoryMax=8G` 作上限比较稳妥。 +- 内存：分块注意力（`attn_query_chunk`）把 ViTMatte 的 O(N²) 瞬时尖峰削平，于是峰值由 + 稠密计算而非注意力矩阵决定。`cpu-fast` lane 实测峰值 **约 5.6 GB@2.4MP / 7.1 GB@4.2MP**—— + 配合默认 4 MP 输入上限，`MemoryMax=8G` 对单 CPU worker 是稳妥上限。默认 lane（cross-check 开） + 需要更多余量。 - **让 RSS 保持平稳（glibc）。** 每处理完一张图，服务会调用 glibc 的 `malloc_trim(0)` （`bgfilter/memtune.py`）把 BiRefNet@2048 前向释放的大块激活还给 OS——否则 ptmalloc 会把它们留在 arena 里，RSS 会随请求逐步抬高。想进一步降内存**并**提升 CPU 吞吐，可以 @@ -297,4 +329,6 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \ | `503 service not ready` | 启动时模型还在加载；等 "Application startup complete"。 | | 刚设 `BGFILTER_CROSS_CHECK` 后（离线）报 `500 inference failed` | `BiRefNet_HR-matting` 没备好；下到缓存/`models/`（§5）。 | | 负载下 OOM / 被 kill | worker 多于一个，或 `MemoryMax` 太低。保持 `--workers 1`。 | +| 大图时 worker 被 `kill`（SIGKILL）/ 返回 `502`，日志显示 `DefaultCPUAllocator: can't allocate memory` | 大图触发 ViTMatte OOM。改用 `BGFILTER_CONFIG=configs/cpu-fast.yaml`（§6），和/或调小 `BGFILTER_MAX_IMAGE_PIXELS`。 | +| 大文件上传在代理层被 `413` 拒 | nginx `client_max_body_size`（默认 1 MB）——在 server 块里调大（如 `client_max_body_size 12m;`）；这和应用的 `BGFILTER_MAX_IMAGE_PIXELS` 是两道独立限制。 | | `400 could not auto-detect background colour` | 输入没有干净的纯色边框；传 `screen_color=#RRGGBB`。 |