Make the pipelining knobs config-file settable (server: section)

CPU_WORKERS / GPU_CONCURRENCY were env-only. Add a `server:` config
section (server.cpu_workers, server.gpu_concurrency) so a lane's
concurrency lives in its yaml alongside device/precision, matching how
model.device already works. Precedence mirrors BGFILTER_DEVICE: config is
the default, the env var overrides per box.

The pool + admission semaphore now build in lifespan (after the config
loads) instead of at import, and PipelineManager reads gpu_concurrency
from the settings. configs/gpu.yaml ships 8 / 2 (its target is the 32 GB
GPU boxes) with an inline VRAM caveat: gpu_concurrency>1 must fit
weights + N × forward, so keep it 1 on ≤16 GB cards.

Verified on a 4080 SUPER: startup logs cpu_workers=8 gpu_concurrency=2
from gpu.yaml with no env set, request returns a valid RGBA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 12:58:15 +00:00
parent c748fc22a4
commit 07ba02fdde
7 changed files with 56 additions and 15 deletions
+3 -2
View File
@@ -43,8 +43,9 @@ class PipelineManager:
# pipeline this manager builds. =1 (default) makes the device strictly
# single-stream; >1 lets that many forwards overlap so batch-1 kernels that
# underfill the SMs can co-schedule (higher GPU-bound throughput, but the
# concurrent activations multiply GPU memory). Uncontended when CPU_WORKERS=1.
gpu_conc = max(1, int(os.environ.get("BGFILTER_GPU_CONCURRENCY", "1")))
# concurrent activations multiply GPU memory). Config value
# (``server.gpu_concurrency``) is the default; the env var overrides it.
gpu_conc = max(1, int(os.environ.get("BGFILTER_GPU_CONCURRENCY", base_settings.server.gpu_concurrency)))
self._gpu_lock = threading.Lock() if gpu_conc == 1 else threading.BoundedSemaphore(gpu_conc)
if preload:
if base_settings.model.matting_method == "vitmatte":