Pipeline concurrent requests over a shared GPU with a narrowed lock

The single asyncio lock serialized the whole request (chroma -> seg ->
trimap -> ViTMatte -> cross-check -> foreground -> despill), so the GPU
sat idle ~70-80% of each request while the CPU stages ran. Replace it
with a thread pool that runs several requests concurrently, and move the
mutex down to just the GPU forwards: when one request steps off the GPU
to do CPU work, another steps on, keeping the device fed.

Two opt-in env knobs, both defaulting to the old strictly-serial
behaviour:
  * BGFILTER_CPU_WORKERS (pool size) attacks the CPU-bound regime -- with
    cross-check off, throughput went 0.91 -> 1.67 req/s (1.84x) at pool=8
    on a 4080S, vs a mere +6% from a second worker process, at half the
    VRAM (one model copy).
  * BGFILTER_GPU_CONCURRENCY caps how many forwards may co-schedule on the
    SMs. batch-1 underfills the GPU, so =2 recovers the SM overlap a lock
    forgoes: cross-check on went 0.83 -> 1.01 req/s, beating two worker
    processes (0.92) with one model copy (~19 GB vs 24 GB).

The CLI/single-thread path passes no lock and uses a nullcontext, so it
stays allocation- and contention-free. Concurrent output verified
identical (10x same-frame -> one hash). A per-worker admission semaphore
bounds in-flight decoded images to the pool size.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

DEPLOY_ZH.md

@@ -202,6 +202,8 @@ export HF_HUB_DISABLE_XET=1                 # 这些 repo 是 Xet 存储；强
 | `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | 超过则返回 `413`（约 4 MP）|
 | `BGFILTER_PRELOAD` | `1` | 启动时预加载默认模型（首个请求不用冷加载）|
 | `BGFILTER_CROSS_CHECK` | 配置（`on`）| 强制开/关 cross-check 否决（`0` 关闭；需 `BiRefNet_HR-matting`）|
+| `BGFILTER_CPU_WORKERS` | `1` | 请求流水线的线程池大小。`1` = 严格串行（一次一个请求）。`>1` 允许这么多请求并发,重叠各自的 CPU 段,同时用一把 GPU 互斥锁串行化模型前向——让 GPU 别闲着。CPU 为瓶颈时(如关 cross-check)收益最大。 |
+| `BGFILTER_GPU_CONCURRENCY` | `1` | 允许几个请求同时进入 GPU 前向。`1` = 单流(安全)。`2` 让两次前向在 SM 上共同调度(batch-1 喂不满 SM),提升 GPU 瓶颈下的吞吐——但并发激活会成倍占显存,按 `权重 + N × 单次前向峰值 ≤ 显存` 来定(32GB 卡上 `2` 在 2048 cross-check 时峰值约 19GB)。需 `BGFILTER_CPU_WORKERS>1` 才有意义。 |
 | `BGFILTER_AUTH_TOKEN` | — | 若设置，`/remove-background` 要求 header `X-Genarrative-Image-Token` 等于它（常量时间比较；否则 `401`）。不设 = **开放**（鉴权交给前置代理）。`/healthz` 始终开放。见 §10。 |
 | `BGFILTER_WEIGHTS_DIR` | `models` | 本地权重文件夹的查找目录（方案 B）|
 | `HF_HOME` | — | HF 缓存位置（方案 A）|