Make the pipelining knobs config-file settable (server: section)
CPU_WORKERS / GPU_CONCURRENCY were env-only. Add a `server:` config section (server.cpu_workers, server.gpu_concurrency) so a lane's concurrency lives in its yaml alongside device/precision, matching how model.device already works. Precedence mirrors BGFILTER_DEVICE: config is the default, the env var overrides per box. The pool + admission semaphore now build in lifespan (after the config loads) instead of at import, and PipelineManager reads gpu_concurrency from the settings. configs/gpu.yaml ships 8 / 2 (its target is the 32 GB GPU boxes) with an inline VRAM caveat: gpu_concurrency>1 must fit weights + N × forward, so keep it 1 on ≤16 GB cards. Verified on a 4080 SUPER: startup logs cpu_workers=8 gpu_concurrency=2 from gpu.yaml with no env set, request returns a valid RGBA. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+3
-2
@@ -43,8 +43,9 @@ class PipelineManager:
|
||||
# pipeline this manager builds. =1 (default) makes the device strictly
|
||||
# single-stream; >1 lets that many forwards overlap so batch-1 kernels that
|
||||
# underfill the SMs can co-schedule (higher GPU-bound throughput, but the
|
||||
# concurrent activations multiply GPU memory). Uncontended when CPU_WORKERS=1.
|
||||
gpu_conc = max(1, int(os.environ.get("BGFILTER_GPU_CONCURRENCY", "1")))
|
||||
# concurrent activations multiply GPU memory). Config value
|
||||
# (``server.gpu_concurrency``) is the default; the env var overrides it.
|
||||
gpu_conc = max(1, int(os.environ.get("BGFILTER_GPU_CONCURRENCY", base_settings.server.gpu_concurrency)))
|
||||
self._gpu_lock = threading.Lock() if gpu_conc == 1 else threading.BoundedSemaphore(gpu_conc)
|
||||
if preload:
|
||||
if base_settings.model.matting_method == "vitmatte":
|
||||
|
||||
Reference in New Issue
Block a user