Make the pipelining knobs config-file settable (server: section)

CPU_WORKERS / GPU_CONCURRENCY were env-only. Add a `server:` config
section (server.cpu_workers, server.gpu_concurrency) so a lane's
concurrency lives in its yaml alongside device/precision, matching how
model.device already works. Precedence mirrors BGFILTER_DEVICE: config is
the default, the env var overrides per box.

The pool + admission semaphore now build in lifespan (after the config
loads) instead of at import, and PipelineManager reads gpu_concurrency
from the settings. configs/gpu.yaml ships 8 / 2 (its target is the 32 GB
GPU boxes) with an inline VRAM caveat: gpu_concurrency>1 must fit
weights + N × forward, so keep it 1 on ≤16 GB cards.

Verified on a 4080 SUPER: startup logs cpu_workers=8 gpu_concurrency=2
from gpu.yaml with no env set, request returns a valid RGBA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 12:58:15 +00:00
parent c748fc22a4
commit 07ba02fdde
7 changed files with 56 additions and 15 deletions
+2 -2
View File
@@ -202,8 +202,8 @@ export HF_HUB_DISABLE_XET=1 # 这些 repo 是 Xet 存储;强
| `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | 超过则返回 `413`(约 4 MP)|
| `BGFILTER_PRELOAD` | `1` | 启动时预加载默认模型(首个请求不用冷加载)|
| `BGFILTER_CROSS_CHECK` | 配置(`on`)| 强制开/关 cross-check 否决(`0` 关闭;需 `BiRefNet_HR-matting`)|
| `BGFILTER_CPU_WORKERS` | `1` | 请求流水线的线程池大小。`1` = 严格串行(一次一个请求)。`>1` 允许这么多请求并发,重叠各自的 CPU 段,同时用一把 GPU 互斥锁串行化模型前向——让 GPU 别闲着。CPU 为瓶颈时(如关 cross-check)收益最大。 |
| `BGFILTER_GPU_CONCURRENCY` | `1` | 允许几个请求同时进入 GPU 前向。`1` = 单流(安全)。`2` 让两次前向在 SM 上共同调度(batch-1 喂不满 SM),提升 GPU 瓶颈下的吞吐——但并发激活会成倍占显存,按 `权重 + N × 单次前向峰值 ≤ 显存` 来定(32GB 卡上 `2` 在 2048 cross-check 时峰值约 19GB)。需 `BGFILTER_CPU_WORKERS>1` 才有意义。 |
| `BGFILTER_CPU_WORKERS` | 配置 `server.cpu_workers`(`1`)| 请求流水线的线程池大小。`1` = 严格串行(一次一个请求)。`>1` 允许这么多请求并发,重叠各自的 CPU 段,同时用一把 GPU 互斥锁串行化模型前向——让 GPU 别闲着。CPU 为瓶颈时(如关 cross-check)收益最大。可写进配置的 `server:` 段,或用此 env 变量按机器覆盖。 |
| `BGFILTER_GPU_CONCURRENCY` | 配置 `server.gpu_concurrency`(`1`)| 允许几个请求同时进入 GPU 前向。`1` = 单流(安全)。`2` 让两次前向在 SM 上共同调度(batch-1 喂不满 SM),提升 GPU 瓶颈下的吞吐——但并发激活会成倍占显存,按 `权重 + N × 单次前向峰值 ≤ 显存` 来定(32GB 卡上 `2` 在 2048 cross-check 时峰值约 19GB;≤16GB 卡保持 `1`)。需 `cpu_workers>1` 才有意义。配置键 `server.gpu_concurrency`,此 env 变量覆盖它。 |
| `BGFILTER_AUTH_TOKEN` | — | 若设置,`/remove-background` 要求 header `X-Genarrative-Image-Token` 等于它(常量时间比较;否则 `401`)。不设 = **开放**(鉴权交给前置代理)。`/healthz` 始终开放。见 §10。 |
| `BGFILTER_WEIGHTS_DIR` | `models` | 本地权重文件夹的查找目录(方案 B)|
| `HF_HOME` | — | HF 缓存位置(方案 A)|