Chunk ViTMatte's global attention to cap its N^2 memory spike

ViTMatte's VitDet backbone runs 4 global attention blocks that materialize
the full [heads x N x N] map: ~19 GB transient at 2048x2048 (16384 tokens),
the pipeline's memory peak. transformers has no SDPA path for this
architecture (the decomposed rel-pos bias is added to raw scores), so
compute the same attention in query-row chunks instead: the bias
factorizes over query rows, making the chunked form mathematically exact
-- output verified BITWISE-identical (unit: fp32/bf16 x 3 sizes; full
pipeline: TestImage3 fp32 and 2048x2048 bf16, all byte-equal).

model.attn_query_chunk (default 2048, 0 = stock one-shot) engages only on
blocks seeing more tokens than the chunk size, so window blocks keep the
original path. Measured @2048x2048 bf16 (9700X): ViTMatte spike
19.8 -> 4.0 GB for ~15% more ViTMatte time; whole pipeline peak
21.4 -> 11.4 GB, warm 51 -> 53 s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bgfilter/settings.py

@@ -134,6 +134,12 @@ class ModelSettings:
     # hardware (any modern GPU, or a CPU with AVX512-BF16/AMX) — otherwise it
     # falls back to fp32 with a warning. Segmentation always stays fp32.
     precision: str = "fp32"
+    # Query-chunked global attention (exact math, bitwise-identical output):
+    # the VitDet backbone's 4 global blocks materialize an N^2 attention map —
+    # ~19 GB at 2048x2048. Blocks seeing more than this many tokens compute it
+    # in query chunks of this size instead, capping the transient at
+    # O(N * chunk): 19.8 -> 4.0 GB for ~15% more ViTMatte time. 0 disables.
+    attn_query_chunk: int = 2048
 
 
 @dataclass(frozen=True)