From 4b5c714fa3845f3df6f05202634666e43055559f Mon Sep 17 00:00:00 2001 From: Linghong Date: Sun, 5 Jul 2026 09:14:57 +0000 Subject: [PATCH] Document weight file layout and cross-check in DEPLOY - Per-model required-file directory tree for the local models/ layout (ViTMatte / BiRefNet / anime-seg), noting which files each loader needs. - Add ZhengPeng7/BiRefNet_HR-matting as the optional 4th model, used only by the cross-check veto; new "cross-check" subsection covering enablement, the offline-provisioning requirement, and the ~2x latency. - Add BGFILTER_CROSS_CHECK to the env-var table and a troubleshooting row for the offline "500 inference failed" case. - Mirror all of the above into DEPLOY_ZH.md. Co-Authored-By: Claude Opus 4.8 --- DEPLOY.md | 37 +++++++++++++++++++++++++++++++++---- DEPLOY_ZH.md | 34 +++++++++++++++++++++++++++++----- 2 files changed, 62 insertions(+), 9 deletions(-) diff --git a/DEPLOY.md b/DEPLOY.md index fd45580..2c2ef81 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -79,8 +79,10 @@ print('torch', torch.__version__, 'cuda=', torch.version.cuda)" ## 5. Provide the model weights -Three models are used: `hustvl/vitmatte-base-composition-1k` (matting), +Three models are used by default: `hustvl/vitmatte-base-composition-1k` (matting), `ZhengPeng7/BiRefNet` (default segmenter), `skytnt/anime-seg` (optional segmenter). +A fourth model, `ZhengPeng7/BiRefNet_HR-matting` (~0.9 GB), is only needed if you +enable the **cross-check** veto — see the note at the end of this section. Pick **one** of the two layouts below. ### Option A — HuggingFace cache + offline mode (recommended) @@ -92,6 +94,7 @@ export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf hf download hustvl/vitmatte-base-composition-1k hf download ZhengPeng7/BiRefNet hf download skytnt/anime-seg +hf download ZhengPeng7/BiRefNet_HR-matting # only if using cross-check (see below) ``` At runtime set `HF_HOME` to the same path plus `HF_HUB_OFFLINE=1 @@ -118,8 +121,13 @@ finds it (`ZhengPeng7/BiRefNet` → `models/BiRefNet`). Required files per model │ ├── BiRefNet_config.py # custom code — required by trust_remote_code │ ├── birefnet.py # custom code — required by trust_remote_code │ └── model.safetensors # ~424 MB - └── anime-seg/ # optional segmenter (seg_model=anime-seg) - └── isnetis.onnx # ~168 MB + ├── anime-seg/ # optional segmenter (seg_model=anime-seg) + │ └── isnetis.onnx # ~168 MB + └── BiRefNet_HR-matting/ # optional — only for the cross-check veto + ├── config.json + ├── BiRefNet_config.py # custom code — required by trust_remote_code + ├── birefnet.py # custom code — required by trust_remote_code + └── model.safetensors # ~0.9 GB ``` Notes: @@ -129,7 +137,8 @@ Notes: to build the model. Keep `config.json` + `model.safetensors` alongside them. (The repo's `handler.py` / `requirements.txt` / `README.md` are not needed.) - **anime-seg**: only `isnetis.onnx` is read; nothing else is required. -- `anime-seg` is optional — omit its folder if you never use `seg_model=anime-seg`. +- `anime-seg` and `BiRefNet_HR-matting` are optional — omit their folders unless you + use `seg_model=anime-seg` / the cross-check veto respectively. The easiest way to populate the folders is to let `hf download` fetch them (it also pulls a couple of unused files like `README.md`, which is harmless): @@ -138,10 +147,28 @@ pulls a couple of unused files like `README.md`, which is harmless): hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k hf download ZhengPeng7/BiRefNet --local-dir models/BiRefNet hf download skytnt/anime-seg --local-dir models/anime-seg +hf download ZhengPeng7/BiRefNet_HR-matting --local-dir models/BiRefNet_HR-matting # cross-check only ``` Override the base directory with `BGFILTER_WEIGHTS_DIR`. `models/` is gitignored. +### Cross-check veto (optional, off by default in the service) + +The pipeline can run a second, high-resolution matting model +(`ZhengPeng7/BiRefNet_HR-matting`, ~0.9 GB) as a "second opinion" that only clears +background residue trapped between hair strands. In the **HTTP service it is OFF by +default** and lazy-loaded, so you don't need this model unless you turn it on: + +- Enable service-wide with `BGFILTER_CROSS_CHECK=1`, or per request with the + `cross_check=true` form field. +- ⚠️ **On an offline server (`HF_HUB_OFFLINE=1`) you must provision + `BiRefNet_HR-matting` first** (Option A or B above) — otherwise the first + cross-check request fails, because it cannot download the model at runtime. +- It adds one extra 2048-px inference (~35 s/image on CPU), roughly doubling latency. +- Note the config default (`configs/default.yaml`) has `cross_check.enabled: true`, + which the **CLI** honours; only the HTTP service forces it off unless + `BGFILTER_CROSS_CHECK=1`. + ### Downloading behind a firewall (CN networks) If direct HuggingFace access is blocked, use the mirror **with the local proxy @@ -165,6 +192,7 @@ redirect back to `huggingface.co`, which recent `huggingface_hub` rejects with | `BGFILTER_DEVICE` | `cpu` | Overrides **both** model and segmentation device (`cuda` for GPU) | | `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | Reject larger inputs with `413` (~4 MP) | | `BGFILTER_PRELOAD` | `1` | Load default models at startup (first request isn't cold) | +| `BGFILTER_CROSS_CHECK` | `0` | Enable the cross-check veto (needs `BiRefNet_HR-matting`; ~2× latency) | | `BGFILTER_WEIGHTS_DIR` | `models` | Where local weight folders are looked up (Option B) | | `HF_HOME` | — | HF cache location (Option A) | | `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | Set to `1` in production once weights are cached | @@ -270,5 +298,6 @@ service has **no authentication**. To reach it from other hosts: | `FileMetadataError: ... not on huggingface.co` while downloading | hf-mirror + a proxy conflict. Bypass the proxy (`NO_PROXY='*'`) and set `HF_HUB_DISABLE_XET=1` (§5). | | `LocalEntryNotFoundError`, 0 bytes | Xet download path failing; set `HF_HUB_DISABLE_XET=1`. | | `503 service not ready` | Models still loading at startup; wait for "Application startup complete". | +| `500 inference failed` right after enabling `BGFILTER_CROSS_CHECK` (offline) | `BiRefNet_HR-matting` not provisioned; download it into the cache/`models/` (§5). | | OOM / killed under load | More than one worker, or `MemoryMax` too low. Keep `--workers 1`. | | `400 could not auto-detect background colour` | Input has no clean flat border; pass `screen_color=#RRGGBB`. | diff --git a/DEPLOY_ZH.md b/DEPLOY_ZH.md index 1330544..ec43680 100644 --- a/DEPLOY_ZH.md +++ b/DEPLOY_ZH.md @@ -73,9 +73,10 @@ print('torch', torch.__version__, 'cuda=', torch.version.cuda)" ## 5. 准备模型权重 -共用到三个模型：`hustvl/vitmatte-base-composition-1k`（抠图）、 +默认用到三个模型：`hustvl/vitmatte-base-composition-1k`（抠图）、 `ZhengPeng7/BiRefNet`（默认分割）、`skytnt/anime-seg`（可选分割）。 -下面两种放法**二选一**。 +第四个模型 `ZhengPeng7/BiRefNet_HR-matting`（约 0.9 GB）只有在启用 **cross-check（交叉校验） +否决**时才需要——见本节末尾说明。下面两种放法**二选一**。 ### 方案 A —— HuggingFace 缓存 + 离线模式（推荐） @@ -86,6 +87,7 @@ export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf hf download hustvl/vitmatte-base-composition-1k hf download ZhengPeng7/BiRefNet hf download skytnt/anime-seg +hf download ZhengPeng7/BiRefNet_HR-matting # 仅在使用 cross-check 时（见下） ``` 运行时把 `HF_HOME` 指向同一路径，并设 `HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1`（见 §6）。 @@ -111,8 +113,13 @@ hf download skytnt/anime-seg │ ├── BiRefNet_config.py # 自定义代码——trust_remote_code 必需 │ ├── birefnet.py # 自定义代码——trust_remote_code 必需 │ └── model.safetensors # 约 424 MB - └── anime-seg/ # 可选分割模型（seg_model=anime-seg） - └── isnetis.onnx # 约 168 MB + ├── anime-seg/ # 可选分割模型（seg_model=anime-seg） + │ └── isnetis.onnx # 约 168 MB + └── BiRefNet_HR-matting/ # 可选——仅用于 cross-check 否决 + ├── config.json + ├── BiRefNet_config.py # 自定义代码——trust_remote_code 必需 + ├── birefnet.py # 自定义代码——trust_remote_code 必需 + └── model.safetensors # 约 0.9 GB ``` 说明： @@ -122,7 +129,8 @@ hf download skytnt/anime-seg 和 `config.json` + `model.safetensors` 放在一起。（repo 里的 `handler.py` / `requirements.txt` / `README.md` 用不到。） - **anime-seg**：只读 `isnetis.onnx`，其它都不需要。 -- `anime-seg` 是可选的——若从不使用 `seg_model=anime-seg`，可以不放这个文件夹。 +- `anime-seg` 和 `BiRefNet_HR-matting` 都是可选的——不用 `seg_model=anime-seg` / + cross-check 否决时，各自的文件夹可以不放。 最省事的填充方式是用 `hf download`（它也会顺带拉几个用不到的文件如 `README.md`，无害）： @@ -130,10 +138,24 @@ hf download skytnt/anime-seg hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k hf download ZhengPeng7/BiRefNet --local-dir models/BiRefNet hf download skytnt/anime-seg --local-dir models/anime-seg +hf download ZhengPeng7/BiRefNet_HR-matting --local-dir models/BiRefNet_HR-matting # 仅 cross-check ``` 用 `BGFILTER_WEIGHTS_DIR` 可改基目录。`models/` 已被 gitignore。 +### cross-check 否决（可选，服务里默认关闭） + +管线可以再跑一个高分辨率 matting 模型（`ZhengPeng7/BiRefNet_HR-matting`，约 0.9 GB） +作为"第二意见"，专门清理发丝间残留的背景色。在 **HTTP 服务里默认是关的**、且懒加载， +所以不开启就不需要这个模型： + +- 服务级开启：`BGFILTER_CROSS_CHECK=1`；或按请求：表单字段 `cross_check=true`。 +- ⚠️ **离线服务器（`HF_HUB_OFFLINE=1`）必须先备好 `BiRefNet_HR-matting`**（上面方案 A 或 B）， + 否则第一个 cross-check 请求会失败——运行时它无法联网下载模型。 +- 每张图多一次 2048 分辨率推理（CPU 上约 35s/张），延迟大约翻倍。 +- 注意：配置默认（`configs/default.yaml`）里 `cross_check.enabled: true`，**CLI 会遵循它**； + 只有 HTTP 服务强制关闭，除非设了 `BGFILTER_CROSS_CHECK=1`。 + ### 墙内下载（国内网络） 若直连 HuggingFace 被墙，用镜像 **并绕过本地代理、禁用 Xet**——这是稳定可用的组合： @@ -156,6 +178,7 @@ export HF_HUB_DISABLE_XET=1 # 这些 repo 是 Xet 存储；强 | `BGFILTER_DEVICE` | `cpu` | 同时覆盖 model 和 segmentation 的 device（GPU 用 `cuda`）| | `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | 超过则返回 `413`（约 4 MP）| | `BGFILTER_PRELOAD` | `1` | 启动时预加载默认模型（首个请求不用冷加载）| +| `BGFILTER_CROSS_CHECK` | `0` | 启用 cross-check 否决（需 `BiRefNet_HR-matting`；延迟约 ×2）| | `BGFILTER_WEIGHTS_DIR` | `models` | 本地权重文件夹的查找目录（方案 B）| | `HF_HOME` | — | HF 缓存位置（方案 A）| | `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | 权重缓存好后，生产设为 `1` | @@ -257,5 +280,6 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \ | 下载时 `FileMetadataError: ... not on huggingface.co` | 镜像与代理冲突。绕过代理（`NO_PROXY='*'`）并设 `HF_HUB_DISABLE_XET=1`（§5）。 | | `LocalEntryNotFoundError`、0 字节 | Xet 下载路径失败；设 `HF_HUB_DISABLE_XET=1`。 | | `503 service not ready` | 启动时模型还在加载；等 "Application startup complete"。 | +| 刚设 `BGFILTER_CROSS_CHECK` 后（离线）报 `500 inference failed` | `BiRefNet_HR-matting` 没备好；下到缓存/`models/`（§5）。 | | 负载下 OOM / 被 kill | worker 多于一个，或 `MemoryMax` 太低。保持 `--workers 1`。 | | `400 could not auto-detect background colour` | 输入没有干净的纯色边框；传 `screen_color=#RRGGBB`。 |