Make the pipelining knobs config-file settable (server: section)

CPU_WORKERS / GPU_CONCURRENCY were env-only. Add a `server:` config
section (server.cpu_workers, server.gpu_concurrency) so a lane's
concurrency lives in its yaml alongside device/precision, matching how
model.device already works. Precedence mirrors BGFILTER_DEVICE: config is
the default, the env var overrides per box.

The pool + admission semaphore now build in lifespan (after the config
loads) instead of at import, and PipelineManager reads gpu_concurrency
from the settings. configs/gpu.yaml ships 8 / 2 (its target is the 32 GB
GPU boxes) with an inline VRAM caveat: gpu_concurrency>1 must fit
weights + N × forward, so keep it 1 on ≤16 GB cards.

Verified on a 4080 SUPER: startup logs cpu_workers=8 gpu_concurrency=2
from gpu.yaml with no env set, request returns a valid RGBA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 12:58:15 +00:00
parent c748fc22a4
commit 07ba02fdde
7 changed files with 56 additions and 15 deletions
+17
View File
@@ -196,6 +196,22 @@ class SegmentationSettings:
precision: str = "fp32"
@dataclass(frozen=True)
class ServerSettings:
"""HTTP-service request concurrency (ignored by the CLI).
``cpu_workers`` is the thread-pool size for request pipelining: >1 lets that
many requests run concurrently, overlapping their CPU stages while the GPU
forwards stay serialized. ``gpu_concurrency`` caps how many forwards may
co-schedule on the device at once (>1 fills the SMs that batch-1 underuses,
but multiplies GPU activation memory -- size it to the VRAM). Both env vars
``BGFILTER_CPU_WORKERS`` / ``BGFILTER_GPU_CONCURRENCY`` override these.
"""
cpu_workers: int = 1
gpu_concurrency: int = 1
@dataclass(frozen=True)
class PipelineSettings:
chroma: ChromaSettings = ChromaSettings()
@@ -206,6 +222,7 @@ class PipelineSettings:
despill: DespillSettings = DespillSettings()
model: ModelSettings = ModelSettings()
segmentation: SegmentationSettings = SegmentationSettings()
server: ServerSettings = ServerSettings()
# Background-colour prior. None = auto-detect the flat background colour from the
# image border (raises if there is no clean flat background). A hex string like
# "#CFEFFF" sets it explicitly.