diff --git a/README.md b/README.md index 0272fb0..8ebbbd5 100644 --- a/README.md +++ b/README.md @@ -127,6 +127,42 @@ qa_grid.png metadata.json ``` +## HTTP service + +An HTTP wrapper (`app.py` + `bgfilter/service.py`) exposes the pipeline as a +long-running FastAPI service. Models load once and are reused across requests. + +```powershell +D:\MiniConda\envs\lightML\python.exe -m uvicorn app:app ` + --host 127.0.0.1 --port 18083 --workers 1 +``` + +Two endpoints: + +- `GET /healthz` — liveness/config JSON. +- `POST /remove-background` — `multipart/form-data`, returns `image/png`. + - `file` (required) — the source image. The field name is fixed as `file`. + - `screen_color` (optional) — `#RRGGBB` prior; omit/empty for auto-detect. + - `seg_model` (optional) — `birefnet` (default) or `anime-seg`. + +```bash +# default (birefnet + auto background colour) +curl -sS -F "file=@input.png" \ + http://127.0.0.1:18083/remove-background -o output.png + +# explicit background colour + anime segmenter +curl -sS \ + -F "file=@input.png" \ + -F "screen_color=#CFEFFF" \ + -F "seg_model=anime-seg" \ + http://127.0.0.1:18083/remove-background -o output.png +``` + +Config comes from `BGFILTER_CONFIG` (defaults to `configs/default.yaml` when present); +`BGFILTER_DEVICE` overrides both model and segmentation device; `BGFILTER_MAX_IMAGE_PIXELS` +caps input size (default ~4MP → `413`); `BGFILTER_PRELOAD=1` loads the default models at +startup. Run a single worker (`--workers 1`) — each worker loads its own copy of the models. + ## Quality Check The quality checker measures alpha validity and edge spill on semi-transparent edge @@ -153,11 +189,22 @@ D:\MiniConda\envs\lightML\python.exe scripts\smoke_samples.py ` ## Notes - `Samples/` and `Outputs/` are ignored by Git. -- ViTMatte and segmentation weights load from Hugging Face on first use. Behind a - firewall set `HF_ENDPOINT=https://hf-mirror.com` (and bypass a flaky local proxy). +- ViTMatte and segmentation weights load from Hugging Face on first use. `anime-seg` (`skytnt/anime-seg`) is a plain ONNX download; `birefnet` (`ZhengPeng7/BiRefNet`) ships custom modelling code so it needs `trust_remote_code=True` plus `timm` / `einops` / `kornia`. +- Downloading the weights behind a firewall — the reliable combination is the + `hf-mirror.com` mirror **with the local proxy bypassed** and **Xet disabled**: + ```powershell + $env:HF_ENDPOINT = "https://hf-mirror.com" # domestic mirror + $env:NO_PROXY = "*" # bypass the proxy; the mirror is direct + $env:HF_HUB_DISABLE_XET = "1" # these repos are Xet-backed; force classic HTTP + ``` + Two gotchas this avoids: (1) mirror + an overseas proxy makes the mirror's `resolve` + bounce back to `huggingface.co`, which recent `huggingface_hub` rejects with + `FileMetadataError`; (2) with `hf-xet` installed the Xet download path fails instantly. + Once the weights are cached, run offline with `HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1` + (as the service does in production). - Foreground colour estimation uses pymatting's `estimate_foreground_ml` to propagate clean foreground colour into semi-transparent edges before de-spill. Set `foreground.method: unmix` to fall back to the legacy heuristic. diff --git a/app.py b/app.py new file mode 100644 index 0000000..875bfca --- /dev/null +++ b/app.py @@ -0,0 +1,196 @@ +"""FastAPI entry for the BGfilter background-removal service. + +Thin HTTP layer: parameter validation, image decode/encode, error codes, timing, +and a global lock. All model handling lives in ``bgfilter.service.PipelineManager``. +""" + +from __future__ import annotations + +import asyncio +import io +import logging +import os +import re +import time +from contextlib import asynccontextmanager +from pathlib import Path + +import numpy as np +from fastapi import FastAPI, File, Form, HTTPException, Response, UploadFile +from PIL import Image, ImageOps, UnidentifiedImageError + +from bgfilter.config import load_settings, override_settings +from bgfilter.service import SEG_MODELS, PipelineManager + +logging.basicConfig( + level=os.environ.get("BGFILTER_LOG_LEVEL", "INFO"), + format="%(asctime)s %(levelname)s %(name)s %(message)s", +) +logger = logging.getLogger("bgfilter.app") + +VERSION = "0.1.0" +_HEX_RE = re.compile(r"^#[0-9a-fA-F]{6}$") + + +def _int_env(name: str, default: int) -> int: + raw = os.environ.get(name) + if not raw: + return default + try: + return int(raw) + except ValueError: + logger.warning("ignoring non-integer %s=%r; using %d", name, raw, default) + return default + + +MAX_IMAGE_PIXELS = _int_env("BGFILTER_MAX_IMAGE_PIXELS", 4_194_304) +# We enforce our own pixel budget from the image header before decoding, so disable +# Pillow's separate decompression-bomb ceiling (which would otherwise raise its own error). +Image.MAX_IMAGE_PIXELS = None + +_manager: PipelineManager | None = None +_lock = asyncio.Lock() + + +def _build_settings(): + config_path = os.environ.get("BGFILTER_CONFIG") + if config_path is None: + default_cfg = Path("configs/default.yaml") + config_path = str(default_cfg) if default_cfg.exists() else None + settings = load_settings(config_path) + device = os.environ.get("BGFILTER_DEVICE") + if device: + # Covers both settings.model.device and settings.segmentation.device. + settings = override_settings(settings, device=device) + return settings + + +@asynccontextmanager +async def lifespan(app: FastAPI): + global _manager + settings = _build_settings() + preload = os.environ.get("BGFILTER_PRELOAD", "1").strip().lower() not in ("0", "false", "no", "") + logger.info( + "starting bgfilter service: device=%s default_seg=%s preload=%s max_pixels=%d", + settings.model.device, + settings.segmentation.backend, + preload, + MAX_IMAGE_PIXELS, + ) + _manager = PipelineManager(settings, preload=preload) + yield + + +app = FastAPI(title="BGfilter", version=VERSION, lifespan=lifespan) + + +@app.get("/healthz") +def healthz(): + if _manager is None: + raise HTTPException(status_code=503, detail="service not ready") + return { + "ok": True, + "service": "bgfilter", + "version": VERSION, + "defaultSegModel": _manager.default_seg_model, + "device": _manager.device, + } + + +def _normalize_screen_color(value: str | None) -> str | None: + """None/empty/"null"/"auto" -> auto-detect; otherwise require #RRGGBB.""" + if value is None: + return None + v = value.strip() + if v == "" or v.lower() in ("null", "auto"): + return None + if not _HEX_RE.match(v): + raise HTTPException(status_code=400, detail="invalid screen_color") + return v.upper() + + +def _normalize_seg_model(value: str | None) -> str | None: + """None/empty -> use the service default; otherwise must be a known backend.""" + if value is None: + return None + v = value.strip().lower() + if v == "": + return None + if v not in SEG_MODELS: + raise HTTPException(status_code=400, detail="invalid seg_model") + return v + + +def _decode_image(data: bytes) -> np.ndarray: + if not data: + raise HTTPException(status_code=400, detail="empty image") + try: + image = Image.open(io.BytesIO(data)) + except (UnidentifiedImageError, OSError, ValueError): + raise HTTPException(status_code=400, detail="invalid image") + width, height = image.size + if width <= 0 or height <= 0: + raise HTTPException(status_code=400, detail="invalid image") + if width * height > MAX_IMAGE_PIXELS: + raise HTTPException(status_code=413, detail="image too large") + try: + image = ImageOps.exif_transpose(image).convert("RGB") + image.load() # force full decode; truncated files raise here + except (OSError, ValueError): + raise HTTPException(status_code=400, detail="invalid image") + return np.asarray(image, dtype=np.uint8) + + +def _encode_png(rgb: np.ndarray, alpha: np.ndarray) -> bytes: + alpha_u8 = np.clip(alpha * 255.0, 0, 255).astype(np.uint8) + rgba = np.dstack([np.clip(rgb, 0, 255).astype(np.uint8), alpha_u8]) + buffer = io.BytesIO() + Image.fromarray(rgba, mode="RGBA").save(buffer, format="PNG") + return buffer.getvalue() + + +@app.post("/remove-background") +async def remove_background( + file: UploadFile = File(...), + screen_color: str | None = Form(None), + seg_model: str | None = Form(None), +): + if _manager is None: + raise HTTPException(status_code=503, detail="service not ready") + + normalized_color = _normalize_screen_color(screen_color) + normalized_seg = _normalize_seg_model(seg_model) or _manager.default_seg_model + + rgb = _decode_image(await file.read()) + + start = time.perf_counter() + async with _lock: # one inference at a time per process + loop = asyncio.get_running_loop() + try: + result = await loop.run_in_executor( + None, + lambda: _manager.process( + rgb, screen_color=normalized_color, seg_model=normalized_seg + ), + ) + except RuntimeError as exc: + # No clean flat background and no screen_color given -> client-actionable. + if "auto-detect" in str(exc).lower(): + raise HTTPException( + status_code=400, + detail="could not auto-detect background colour; pass screen_color", + ) + logger.exception("inference failed") + raise HTTPException(status_code=500, detail="inference failed") + except Exception: + logger.exception("inference failed") + raise HTTPException(status_code=500, detail="inference failed") + elapsed_ms = int((time.perf_counter() - start) * 1000) + + png = _encode_png(result.rgb, result.alpha) + headers = { + "X-BGFilter-Elapsed-Ms": str(elapsed_ms), + "X-BGFilter-Seg-Model": normalized_seg, + "X-BGFilter-Screen-Color": normalized_color if normalized_color else "auto", + } + return Response(content=png, media_type="image/png", headers=headers) diff --git a/bgfilter/pipeline.py b/bgfilter/pipeline.py index 4611dad..209ebf6 100644 --- a/bgfilter/pipeline.py +++ b/bgfilter/pipeline.py @@ -1,7 +1,7 @@ from __future__ import annotations import json -from dataclasses import asdict +from dataclasses import asdict, dataclass from pathlib import Path import numpy as np @@ -22,11 +22,38 @@ from .trimap import ( from .vitmatte_infer import ViTMatteRunner +@dataclass +class MattingResult: + """In-memory result of one matting run. + + ``rgb`` is the despilled foreground (uint8 HxWx3) and ``alpha`` is float32 HxW in + 0..1; together they compose the output RGBA. The remaining arrays are the + intermediate maps used for debug previews. + """ + + rgb: np.ndarray + alpha: np.ndarray + metadata: dict + bg_confidence: np.ndarray + trimap: np.ndarray + seg_mask: np.ndarray | None + color_mask: np.ndarray + input_rgb: np.ndarray + + class MattingPipeline: - def __init__(self, settings: PipelineSettings): + def __init__( + self, + settings: PipelineSettings, + *, + runner: ViTMatteRunner | None = None, + segmenter=None, + ): self.settings = settings - self._runner: ViTMatteRunner | None = None - self._segmenter = None + # Pre-loaded models can be injected so a long-lived service shares them + # across requests instead of reloading per call; None = lazy-load on first use. + self._runner: ViTMatteRunner | None = runner + self._segmenter = segmenter def _segment(self, rgb: np.ndarray) -> np.ndarray: if self._segmenter is None: @@ -53,6 +80,10 @@ class MattingPipeline: raise return trimap_to_alpha_seed(trimap, bg_confidence), "chroma_fallback" + def run_rgb(self, rgb: np.ndarray) -> MattingResult: + """Run the full pipeline on an in-memory RGB array (no disk I/O).""" + return _process_rgb(rgb, self) + def run_image( self, input_path: str | Path, @@ -71,14 +102,8 @@ def run_image( return MattingPipeline(settings).run_image(input_path, output_path, debug_dir) -def _run_image( - input_path: str | Path, - output_path: str | Path, - debug_dir: str | Path | None, - pipeline: MattingPipeline, -) -> dict: +def _process_rgb(rgb: np.ndarray, pipeline: MattingPipeline) -> MattingResult: settings = pipeline.settings - rgb = load_rgb(input_path) screen_color = parse_hex_color(settings.screen_color) if settings.screen_color else None bg_confidence, model = compute_bg_confidence( rgb, settings=settings.chroma, screen_color=screen_color @@ -111,39 +136,56 @@ def _run_image( foreground = estimate_foreground_rgb( rgb, alpha, bg_confidence, model, settings.foreground ) - corrected_rgb, color_mask = despill( - foreground.rgb, alpha, model, settings.despill - ) - save_rgba(output_path, corrected_rgb, alpha) + corrected_rgb, color_mask = despill(foreground.rgb, alpha, model, settings.despill) - result = { - "input": str(input_path), - "output": str(output_path), + metadata = { "alpha_source": alpha_source, "background_model": model.to_dict(), "trimap": trimap_stats, "settings": asdict(settings), } + return MattingResult( + rgb=corrected_rgb, + alpha=alpha, + metadata=metadata, + bg_confidence=bg_confidence, + trimap=trimap, + seg_mask=seg_mask, + color_mask=color_mask, + input_rgb=rgb, + ) + + +def _run_image( + input_path: str | Path, + output_path: str | Path, + debug_dir: str | Path | None, + pipeline: MattingPipeline, +) -> dict: + res = _process_rgb(load_rgb(input_path), pipeline) + save_rgba(output_path, res.rgb, res.alpha) + + result = {"input": str(input_path), "output": str(output_path), **res.metadata} if debug_dir is not None: debug = Path(debug_dir) debug.mkdir(parents=True, exist_ok=True) - save_gray(debug / "bg_confidence.png", bg_confidence) - save_gray(debug / "trimap.png", trimap) - if seg_mask is not None: - save_gray(debug / "seg_mask.png", seg_mask) - save_gray(debug / "alpha.png", alpha) - save_rgb(debug / "foreground_rgb.png", corrected_rgb) - save_gray(debug / "color_mask.png", color_mask) - save_previews(debug, corrected_rgb, alpha) + save_gray(debug / "bg_confidence.png", res.bg_confidence) + save_gray(debug / "trimap.png", res.trimap) + if res.seg_mask is not None: + save_gray(debug / "seg_mask.png", res.seg_mask) + save_gray(debug / "alpha.png", res.alpha) + save_rgb(debug / "foreground_rgb.png", res.rgb) + save_gray(debug / "color_mask.png", res.color_mask) + save_previews(debug, res.rgb, res.alpha) tiles = { - "input": rgb, - "bg confidence": np.repeat((bg_confidence[..., None] * 255).astype(np.uint8), 3, axis=2), - "trimap": np.repeat(trimap[..., None], 3, axis=2), - "alpha": np.repeat((alpha[..., None] * 255).astype(np.uint8), 3, axis=2), + "input": res.input_rgb, + "bg confidence": np.repeat((res.bg_confidence[..., None] * 255).astype(np.uint8), 3, axis=2), + "trimap": np.repeat(res.trimap[..., None], 3, axis=2), + "alpha": np.repeat((res.alpha[..., None] * 255).astype(np.uint8), 3, axis=2), } for name, color in BACKGROUND_COLORS.items(): - tiles[f"preview {name}"] = composite(corrected_rgb, alpha, color) + tiles[f"preview {name}"] = composite(res.rgb, res.alpha, color) make_qa_grid(debug, tiles) write_text(debug / "metadata.json", json.dumps(result, indent=2, ensure_ascii=False)) diff --git a/bgfilter/service.py b/bgfilter/service.py new file mode 100644 index 0000000..7bf35c6 --- /dev/null +++ b/bgfilter/service.py @@ -0,0 +1,101 @@ +from __future__ import annotations + +import logging +import threading +from dataclasses import replace + +import numpy as np + +from .config import _SEG_MODELS, override_settings +from .pipeline import MattingPipeline, MattingResult +from .segmentation import make_segmenter +from .settings import PipelineSettings +from .vitmatte_infer import ViTMatteRunner + +logger = logging.getLogger("bgfilter.service") + +# HTTP-facing seg_model values, in preference order (first = default). +SEG_MODELS: tuple[str, ...] = tuple(_SEG_MODELS) + + +class PipelineManager: + """Long-lived holder of the matting models. + + ViTMatte is the matting model shared by both segmentation backends, so it is + loaded once and reused. Each segmentation backend is lazily loaded once on + first use and cached by name. Per-request parameters (``screen_color``, + ``seg_model``) never trigger a model reload -- ``screen_color`` only affects + this request's chroma/despill, and ``seg_model`` just selects an already-cached + (or first-time-loaded) segmenter. + """ + + def __init__(self, base_settings: PipelineSettings, preload: bool = True): + self._base = base_settings + self._runner: ViTMatteRunner | None = None + self._segmenters: dict[str, object] = {} + self._load_lock = threading.Lock() + if preload: + if base_settings.model.matting_method == "vitmatte": + self._get_runner() + if base_settings.segmentation.enabled: + self._get_segmenter(base_settings.segmentation.backend) + + @property + def default_seg_model(self) -> str: + return self._base.segmentation.backend + + @property + def device(self) -> str: + return self._base.model.device + + def _get_runner(self) -> ViTMatteRunner: + if self._runner is None: + with self._load_lock: + if self._runner is None: + logger.info( + "loading ViTMatte: model=%s device=%s", + self._base.model.model_name, + self._base.model.device, + ) + self._runner = ViTMatteRunner(self._base.model) + return self._runner + + def _get_segmenter(self, backend: str): + seg = self._segmenters.get(backend) + if seg is None: + with self._load_lock: + seg = self._segmenters.get(backend) + if seg is None: + seg_settings = replace( + self._base.segmentation, + backend=backend, + model_name=_SEG_MODELS[backend], + ) + logger.info( + "loading segmenter: backend=%s model=%s device=%s", + backend, + seg_settings.model_name, + seg_settings.device, + ) + seg = make_segmenter(seg_settings) + self._segmenters[backend] = seg + return seg + + def process( + self, rgb: np.ndarray, *, screen_color: str | None, seg_model: str + ) -> MattingResult: + if seg_model not in _SEG_MODELS: + raise ValueError(f"invalid seg_model '{seg_model}'") + settings = override_settings( + self._base, screen_color=screen_color, seg_backend=seg_model + ) + runner = ( + self._get_runner() + if settings.model.matting_method == "vitmatte" + else None + ) + segmenter = ( + self._get_segmenter(seg_model) if settings.segmentation.enabled else None + ) + pipeline = MattingPipeline(settings, runner=runner, segmenter=segmenter) + return pipeline.run_rgb(rgb) diff --git a/docs/【服务化】BGfilter服务版设计与交付需求-2026-07-02.md b/docs/【服务化】BGfilter服务版设计与交付需求-2026-07-02.md new file mode 100644 index 0000000..b954148 --- /dev/null +++ b/docs/【服务化】BGfilter服务版设计与交付需求-2026-07-02.md @@ -0,0 +1,470 @@ +# BGfilter 服务版设计与交付需求 + +本文档给接手实现服务化的 Agent 使用。目标是在 `BGfilter-server` 仓库内，把现有 BGfilter 抠图算法包装成一个可长期运行的 Python HTTP 服务，接口形态参考当前线上 `birefnet-service`，但算法仍使用本仓库的 BGfilter 管线。 + +## 目标 + +新增一个服务入口，使调用方可以通过 HTTP 上传图片并得到透明背景 PNG。 + +服务首版只暴露两个业务参数： + +- `screen_color`：可选。背景色先验，格式为 `#RRGGBB`。不传或空值时沿用 BGfilter 默认行为，从图片边框自动检测纯色背景。 +- `seg_model`：可选。分割模型选择，允许值为 `birefnet` 或 `anime-seg`。不传时使用仓库当前默认值 `birefnet`。 + +其它算法参数首版不要暴露为 HTTP 参数，直接使用当前默认配置： + +- `trimap.mode = directional` +- `despill.enabled = true` +- `model.matting_method = vitmatte` +- `model.model_name = hustvl/vitmatte-base-composition-1k` +- `model.device = cpu` +- `segmentation.device = cpu` +- `segmentation.input_size = 1024` + +## 非目标 + +- 不把 BGfilter 算法移植到 Rust。 +- 不在 HTTP 接口里暴露 `trimap_mode`、`despill`、阈值、设备、模型权重名等高级参数。 +- 不复用当前线上 `birefnet-service` 的简单 alpha 直出算法；`seg_model=birefnet` 在本服务中只表示 BGfilter 管线里的语义分割后端。 +- 不在每次请求里加载并释放模型。 +- 不要求首版支持批量图片。 + +## 现有算法边界 + +BGfilter 当前核心入口在： + +- `bgfilter/pipeline.py` +- `bgfilter/settings.py` +- `bgfilter/config.py` +- `bgfilter/segmentation.py` +- `bgfilter/vitmatte_infer.py` + +完整管线是： + +```text +输入 RGB +-> 背景色自动检测或使用 screen_color +-> chroma 背景置信度 +-> 可选语义分割 mask，默认开启 +-> trimap 生成，默认 directional +-> ViTMatte 预测 alpha +-> trimap 强约束 +-> chroma-guided alpha suppression +-> alpha 小连通域/孔洞清理 +-> pymatting foreground estimation +-> despill 去背景色溢色 +-> 输出 RGBA PNG +``` + +当前默认分割后端已经是： + +```text +segmentation.backend = birefnet +segmentation.model_name = ZhengPeng7/BiRefNet +``` + +`anime-seg` 对应权重为： + +```text +segmentation.backend = anime-seg +segmentation.model_name = skytnt/anime-seg +``` + +`bgfilter/config.py` 里已有 `_SEG_MODELS` 和 `override_settings(..., seg_backend=...)`，服务层应复用这套映射，不要让 HTTP 调用方传 `model_name`。 + +## HTTP 接口契约 + +### `GET /healthz` + +返回 JSON，用于进程存活和基础配置检查。 + +建议响应： + +```json +{ + "ok": true, + "service": "bgfilter", + "version": "0.1.0", + "defaultSegModel": "birefnet", + "device": "cpu" +} +``` + +### `POST /remove-background` + +请求类型： + +```text +multipart/form-data +``` + +字段： + +```text +file: UploadFile, required +screen_color: string, optional, "#RRGGBB" +seg_model: string, optional, "birefnet" | "anime-seg" +``` + +响应： + +```text +200 image/png +``` + +响应体是透明背景 PNG。建议附带诊断 header： + +```text +X-BGFilter-Elapsed-Ms: 总耗时毫秒 +X-BGFilter-Seg-Model: 实际使用的 seg_model +X-BGFilter-Screen-Color: 实际请求传入的 screen_color，未传可省略或设为 auto +``` + +错误码建议： + +```text +400 empty image +400 invalid image +400 invalid screen_color +400 invalid seg_model +413 image too large +500 inference failed +``` + +为兼容现有 Rust 调用方，`file` 字段名必须保持为 `file`。 + +## 参数默认与校验 + +`screen_color`： + +- 允许省略、空字符串、`null` 等价于自动检测。 +- 非空时必须匹配 `#RRGGBB`。 +- 大小写都可以，内部可以统一为大写或小写。 +- 校验失败返回 `400 invalid screen_color`。 + +`seg_model`： + +- 允许省略，默认 `birefnet`。 +- 允许值只包括 `birefnet`、`anime-seg`。 +- 可以兼容内部命名 `seg_backend`，但 HTTP 对外字段用 `seg_model`。 +- 校验失败返回 `400 invalid seg_model`。 + +`MAX_IMAGE_PIXELS`： + +- 建议参考线上 `birefnet-service`，通过环境变量控制，例如 `BGFILTER_MAX_IMAGE_PIXELS`。 +- 默认建议先设为 `4194304`，即约 4MP。 +- 超限返回 `413 image too large`。 + +## 模型加载与缓存要求 + +服务必须做到模型只加载一次或按后端首次懒加载一次，不能每次请求都重新加载模型。 + +现有 `MattingPipeline` 已经有实例级懒加载： + +- `self._runner` 缓存 `ViTMatteRunner` +- `self._segmenter` 缓存当前 settings 下的 segmenter + +但如果每次请求新建 `MattingPipeline(settings)`，缓存会失效。因此服务版需要持有长期对象。 + +推荐实现一个 `PipelineManager` 或等价对象： + +```text +长期缓存： + ViTMatteRunner: 按 model_name + device + matting_method 缓存 + Segmenter: 按 backend + model_name + device + input_size 缓存 + +请求级参数： + screen_color: 只影响本次 chroma/despill，不影响模型缓存 + seg_model: 选择对应 segmenter；首次使用某后端时加载，之后复用 +``` + +推荐行为： + +```text +服务启动： + 可预加载默认 birefnet + ViTMatte，也可懒加载。 + +第一次 seg_model=birefnet： + 加载 BiRefNet segmenter 和 ViTMatte。 + +后续 seg_model=birefnet： + 复用已有 BiRefNet segmenter 和 ViTMatte。 + +第一次 seg_model=anime-seg： + 加载 AnimeSeg segmenter，ViTMatte 继续复用。 + +后续 seg_model=anime-seg： + 复用已有 AnimeSeg segmenter 和 ViTMatte。 +``` + +可接受的首版简化方案： + +- 长期缓存两个 `MattingPipeline` 实例：一个 `birefnet`，一个 `anime-seg`。 +- 每个实例内部复用自己的 `_runner` 和 `_segmenter`。 +- 如果这样做导致 ViTMatte 被加载两次，应优先改为共享 `ViTMatteRunner`，因为 ViTMatte 是两条分割后端共同使用的 matting 模型。 + +更推荐的实现是把 `MattingPipeline` 改造成可以注入共享 runner/segmenter，或新增服务专用 pipeline runner，避免重复加载 ViTMatte。 + +## 并发控制 + +首版建议采用全局 `asyncio.Lock`，与当前线上 `birefnet-service` 类似，保证同一进程内一次只跑一个推理请求。 + +原因： + +- BGfilter 管线包含 ViTMatte、segmenter、pymatting、OpenCV/NumPy 后处理，CPU 和内存压力较大。 +- CPU 默认部署时并发推理容易放大延迟和内存。 +- 后续确认资源充足后，再考虑按模型或设备拆锁。 + +如果使用 Uvicorn 多 worker，每个 worker 都会各自加载一份模型，内存会成倍增长。首版建议 `--workers 1`。 + +## 内存入口改造建议 + +当前 `MattingPipeline.run_image(...)` 主要面向 CLI 文件输入输出。服务可以先用临时文件桥接，但更推荐新增内存入口，减少磁盘 IO 和临时文件清理风险。 + +推荐新增内部函数： + +```text +run_rgb(rgb: np.ndarray, settings: PipelineSettings, resources/cache) -> result +``` + +返回： + +```text +corrected_rgb: np.ndarray uint8 HxWx3 +alpha: np.ndarray float32 HxW, 0..1 +metadata: dict +``` + +服务层再把它编码成 PNG： + +```text +RGBA = corrected_rgb + alpha +PIL.Image.save(..., format="PNG") +``` + +如果首版为了速度使用临时文件： + +- 临时目录应使用 `tempfile.TemporaryDirectory()`。 +- 文件名不要使用上传原始文件名。 +- 请求结束必须清理。 +- 不要把 debug 输出默认打开。 + +## 推荐文件结构 + +可以参考线上 `birefnet-service` 的简洁形态，在仓库根目录新增： + +```text +app.py +requirements.txt +``` + +也可以新增： + +```text +bgfilter/service.py +``` + +用于放 `PipelineManager` 和内存运行逻辑。`app.py` 保持薄入口。 + +推荐分层： + +```text +app.py + FastAPI 路由、参数校验、图片 decode/encode、错误码、计时、锁 + +bgfilter/service.py + PipelineManager、模型缓存、按请求组装 settings、调用算法管线 + +bgfilter/pipeline.py + 尽量保留原 CLI 兼容行为；必要时新增 run_rgb +``` + +## 依赖 + +现有算法依赖保留。服务化需要补充： + +```text +fastapi +uvicorn[standard] +python-multipart +``` + +现有 `requirements.txt` 中已经有： + +```text +opencv-python +pillow +numpy +scipy +scikit-image +pymatting +torch +torchvision +transformers +accelerate +safetensors +huggingface_hub +timm +einops +kornia +onnxruntime +tqdm +typer +rich +PyYAML +``` + +如果部署在无 GUI Linux 服务器，`opencv-python-headless` 通常比 `opencv-python` 更合适，但替换前需要确认现有本地脚本是否依赖 GUI 能力。 + +## 环境变量 + +建议支持： + +```text +BGFILTER_CONFIG=configs/default.yaml +BGFILTER_DEVICE=cpu +BGFILTER_MAX_IMAGE_PIXELS=4194304 +BGFILTER_PRELOAD=1 +HF_HOME=/opt/genarrative-image-host/model-cache/hf +HF_HUB_OFFLINE=1 +TRANSFORMERS_OFFLINE=1 +OMP_NUM_THREADS=4 +MKL_NUM_THREADS=4 +``` + +`BGFILTER_DEVICE` 应同时覆盖： + +- `settings.model.device` +- `settings.segmentation.device` + +如果机器没有 GPU，默认保持 `cpu`。 + +## systemd 部署参考 + +参考当前线上 `genarrative-birefnet.service`，可以新增： + +```ini +[Unit] +Description=Genarrative BGfilter background removal service +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +WorkingDirectory=/opt/genarrative-image-host/bgfilter-service +Environment=HF_HOME=/opt/genarrative-image-host/model-cache/hf +Environment=HF_HUB_OFFLINE=1 +Environment=TRANSFORMERS_OFFLINE=1 +Environment=BGFILTER_CONFIG=configs/default.yaml +Environment=BGFILTER_DEVICE=cpu +Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304 +Environment=BGFILTER_PRELOAD=1 +Environment=OMP_NUM_THREADS=4 +Environment=MKL_NUM_THREADS=4 +ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1 +Restart=on-failure +RestartSec=5 +MemoryMax=8G +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=full +ReadWritePaths=/opt/genarrative-image-host + +[Install] +WantedBy=multi-user.target +``` + +端口只是建议值。真正接入时由部署方决定。 + +## 调用示例 + +默认参数： + +```bash +curl -sS \ + -F "file=@input.png" \ + http://127.0.0.1:18083/remove-background \ + -o output.png +``` + +指定背景色： + +```bash +curl -sS \ + -F "file=@input.png" \ + -F "screen_color=#CFEFFF" \ + http://127.0.0.1:18083/remove-background \ + -o output.png +``` + +切换 Anime Seg： + +```bash +curl -sS \ + -F "file=@input.png" \ + -F "seg_model=anime-seg" \ + http://127.0.0.1:18083/remove-background \ + -o output.png +``` + +指定背景色并切换 Anime Seg： + +```bash +curl -sS \ + -F "file=@input.png" \ + -F "screen_color=#CFEFFF" \ + -F "seg_model=anime-seg" \ + http://127.0.0.1:18083/remove-background \ + -o output.png +``` + +## 与 Genarrative Rust 服务的接入预期 + +当前 Genarrative Rust BFF 已有远端抠图调用边界： + +```text +POST {GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL}/remove-background +multipart field: file +response: image/png +``` + +BGfilter 服务应保持同样的 `POST /remove-background` 和 `file` 字段名。Rust 侧后续只需要在 multipart 中追加可选文本字段： + +```text +screen_color +seg_model +``` + +如果 Rust 侧暂时不传新字段，本服务仍应可用，并按默认 `birefnet + 自动背景色` 执行。 + +## 验收要求 + +实现完成后至少验证： + +1. `GET /healthz` 返回 `200` 和 JSON。 +2. `POST /remove-background` 只传 `file` 可返回 `image/png`。 +3. `screen_color=#CFEFFF` 可正常返回 `image/png`。 +4. `seg_model=birefnet` 可正常返回 `image/png`。 +5. `seg_model=anime-seg` 可正常返回 `image/png`。 +6. 非法 `seg_model` 返回 `400`。 +7. 非法 `screen_color` 返回 `400`。 +8. 空文件或非图片返回 `400`。 +9. 超过 `BGFILTER_MAX_IMAGE_PIXELS` 的图片返回 `413`。 +10. 同一进程连续请求同一 `seg_model` 时不会重复加载对应模型。 +11. 从 `birefnet` 切到 `anime-seg` 后，再切回 `birefnet`，已加载模型可复用。 +12. Uvicorn `--workers 1` 下内存不随请求次数持续增长。 + +建议在日志中打印模型加载事件，但不要每个请求打印完整配置或文件内容。日志中不要输出用户上传图片数据。 + +## 完成定义 + +交付时应包含： + +- 可运行的 FastAPI 服务入口。 +- 更新后的依赖文件。 +- 本地启动命令说明。 +- 至少一组 curl 示例。 +- 若新增内存入口，应保持原 CLI 文件入口不破坏。 +- 若为了服务化调整 pipeline，应保证默认 CLI 行为仍按当前配置工作。 + +完成后，不看内部代码也应能通过本文档的 HTTP 契约和 curl 示例直接调用服务。 diff --git a/requirements.txt b/requirements.txt index b36b1f9..7ed5b5a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -18,3 +18,6 @@ tqdm typer rich PyYAML +fastapi +uvicorn[standard] +python-multipart