Make the pipelining knobs config-file settable (server: section)
CPU_WORKERS / GPU_CONCURRENCY were env-only. Add a `server:` config section (server.cpu_workers, server.gpu_concurrency) so a lane's concurrency lives in its yaml alongside device/precision, matching how model.device already works. Precedence mirrors BGFILTER_DEVICE: config is the default, the env var overrides per box. The pool + admission semaphore now build in lifespan (after the config loads) instead of at import, and PipelineManager reads gpu_concurrency from the settings. configs/gpu.yaml ships 8 / 2 (its target is the 32 GB GPU boxes) with an inline VRAM caveat: gpu_concurrency>1 must fit weights + N × forward, so keep it 1 on ≤16 GB cards. Verified on a 4080 SUPER: startup logs cpu_workers=8 gpu_concurrency=2 from gpu.yaml with no env set, request returns a valid RGBA. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -196,6 +196,22 @@ class SegmentationSettings:
|
||||
precision: str = "fp32"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ServerSettings:
|
||||
"""HTTP-service request concurrency (ignored by the CLI).
|
||||
|
||||
``cpu_workers`` is the thread-pool size for request pipelining: >1 lets that
|
||||
many requests run concurrently, overlapping their CPU stages while the GPU
|
||||
forwards stay serialized. ``gpu_concurrency`` caps how many forwards may
|
||||
co-schedule on the device at once (>1 fills the SMs that batch-1 underuses,
|
||||
but multiplies GPU activation memory -- size it to the VRAM). Both env vars
|
||||
``BGFILTER_CPU_WORKERS`` / ``BGFILTER_GPU_CONCURRENCY`` override these.
|
||||
"""
|
||||
|
||||
cpu_workers: int = 1
|
||||
gpu_concurrency: int = 1
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PipelineSettings:
|
||||
chroma: ChromaSettings = ChromaSettings()
|
||||
@@ -206,6 +222,7 @@ class PipelineSettings:
|
||||
despill: DespillSettings = DespillSettings()
|
||||
model: ModelSettings = ModelSettings()
|
||||
segmentation: SegmentationSettings = SegmentationSettings()
|
||||
server: ServerSettings = ServerSettings()
|
||||
# Background-colour prior. None = auto-detect the flat background colour from the
|
||||
# image border (raises if there is no clean flat background). A hex string like
|
||||
# "#CFEFFF" sets it explicitly.
|
||||
|
||||
Reference in New Issue
Block a user