diff --git a/README.md b/README.md index 75c8bea..4e1de3b 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,44 @@ # BgFilter -Offline green screen character matting for AI-generated character images. - -The first implementation follows the workflow in -[`docs/green_screen_matting_workflow.md`](docs/green_screen_matting_workflow.md): +Offline character matting for AI-generated images on a flat-colour background. +Green screen is the default; any flat colour (e.g. a pastel `#CFEFFF`) works via a +`screen_color` prior. ```text -RGB input -> chroma confidence -> trimap -> ViTMatte -> alpha cleanup -> despill -> RGBA PNG -> QA previews +RGB input + -> chroma bg-confidence (keyed to the screen colour) + -> [optional] semantic segmentation mask + -> trimap -> ViTMatte -> alpha cleanup + -> pymatting foreground -> despill -> RGBA PNG -> QA previews ``` +## Pipelines + +Two pipelines, selected by `segmentation.enabled` in the config: + +- **Chroma-only** (`enabled: false`) — Chroma + ViTMatte, no segmentation model. + Lightest / fastest; leans entirely on the colour key for topology. +- **Single segmenter** (`enabled: true`, default) — one segmentation model drives the + trimap topology (holes, hair), ViTMatte then refines the soft edges. Backend is + switchable: `anime-seg` (default, ONNX) or `birefnet` (needs `trust_remote_code`). + +Both share pymatting foreground estimation and a colour de-spill. There is **no** +green-contamination rescue / recolour layer — with clean source images it is +unnecessary, so it was removed. + +## Background colour + +Default is green auto-detection. For any other flat background, pass the colour as a +hex prior so both the chroma key and the de-spill work off it: + +```powershell +... --screen-color "#CFEFFF" +``` + +or set `screen_color: "#CFEFFF"` in the config (`null` = green auto-detect). The +chroma key then scores pixels by perceptual (Lab/RGB) distance to that colour, and +de-spill removes chroma along that colour's direction instead of the green channel. + ## Environment Use the conda environment `lightML`. @@ -32,23 +62,13 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli ` --output Outputs\TestImage_rgba.png ` --debug-dir Outputs\TestImage_debug ` --config configs\default.yaml ` + --screen-color "#CFEFFF" ` --device cuda ``` The CLI reads `configs/default.yaml` when `--config` is provided. Command-line -options override config values, so tuning can usually happen in YAML while -runtime choices such as `--device cpu` stay on the command line. - -Use CPU for validation when CUDA is unavailable: - -```powershell -D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli ` - --input Samples\TestImage.png ` - --output Outputs\TestImage_rgba.png ` - --debug-dir Outputs\TestImage_debug ` - --config configs\default.yaml ` - --device cpu -``` +options override config values, so tuning usually happens in YAML while runtime +choices such as `--device cpu` or `--screen-color` stay on the command line. ## Batch @@ -61,10 +81,11 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli ` --device cuda ``` -## Chroma-Only Debug Mode +## Chroma-alpha debug mode -This mode skips ViTMatte and uses chroma confidence as an alpha seed. It is useful -for fast debugging of chroma confidence, trimap, despill, and QA outputs. +`--matting-method chroma` skips ViTMatte and uses chroma confidence directly as the +alpha seed. Useful for fast inspection of chroma confidence, trimap, and despill. +(Distinct from the chroma-only *pipeline* above, which still runs ViTMatte.) ```powershell D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli ` @@ -83,11 +104,10 @@ For each processed image, the CLI writes an RGBA PNG and optional debug files: ```text bg_confidence.png trimap.png +seg_mask.png # segmentation pipeline only alpha.png -foreground_rgb.png -foreground_background.png -foreground_correction.png -despill_mask.png +foreground_rgb.png # despilled foreground colour +color_mask.png # per-pixel despill weight preview_black.png preview_white.png preview_gray.png @@ -99,8 +119,8 @@ metadata.json ## Quality Check -The quality checker measures alpha validity and green spill on semi-transparent -edge pixels. +The quality checker measures alpha validity and edge spill on semi-transparent edge +pixels. ```powershell D:\MiniConda\envs\lightML\python.exe -m bgfilter.quality_cli ` @@ -120,11 +140,16 @@ D:\MiniConda\envs\lightML\python.exe scripts\smoke_samples.py ` --max-edge-green-excess-p95 0.30 ``` -## Current Notes +## Notes - `Samples/` and `Outputs/` are ignored by Git. -- `ViTMatte` model weights are loaded from Hugging Face on first use. -- Foreground color estimation uses pymatting's `estimate_foreground_ml` to - propagate clean foreground colour into semi-transparent edges before final - de-spill, writing the estimated foreground, background, and a correction map. - Set `foreground.method: unmix` to fall back to the legacy heuristic. +- ViTMatte and segmentation weights load from Hugging Face on first use. Behind a + firewall set `HF_ENDPOINT=https://hf-mirror.com` (and bypass a flaky local proxy). + `anime-seg` (`skytnt/anime-seg`) is a plain ONNX download; `birefnet` + (`ZhengPeng7/BiRefNet`) ships custom modelling code so it needs + `trust_remote_code=True` plus `timm` / `einops` / `kornia`. +- Foreground colour estimation uses pymatting's `estimate_foreground_ml` to propagate + clean foreground colour into semi-transparent edges before de-spill. Set + `foreground.method: unmix` to fall back to the legacy heuristic. +- `docs/green_screen_matting_workflow.md` is the original phase-1 green-screen spec; + this README reflects the current, generalised architecture. diff --git a/docs/green_screen_matting_workflow.md b/docs/green_screen_matting_workflow.md index 3fc6fb4..66cd524 100644 --- a/docs/green_screen_matting_workflow.md +++ b/docs/green_screen_matting_workflow.md @@ -4,6 +4,16 @@ 本方案不使用 YOLO / Faster R-CNN 作为主流程。原因是本任务不是目标检测,而是像素级 alpha matting。 +## 0. 现状说明(架构已演进) + +本文档是**第一阶段的绿幕方案原始规格**,仍准确描述 “Chroma + ViTMatte” 这条管线。实现此后已泛化,**当前架构以 [README](../README.md) 为准**,主要变化: + +- **两条管线**(`segmentation.enabled` 切换):无分割模型的 Chroma + ViTMatte;或单个语义分割模型(默认 anime-seg,可切 birefnet)定 trimap 拓扑,ViTMatte 细化。 +- **背景色泛化**:`screen_color` 先验(如 `#CFEFFF`),chroma / despill 不再写死绿色;`null` 时才走本文档的绿幕自动探测。 +- **已移除**:染绿修复(recolor)、发丝补救(dropout-fill)、BiRefNet+anime-seg 结合方案——它们只是给“生图即染色/断裂”的坏素材擦屁股;源头改用干净背景生图后不再需要。 + +下文其余章节描述绿幕基线流程,仍然有效。 + ## 1. 目标 - 输入:RGB / PNG 图像,背景主色为绿色,但不要求严格等于 `#00FF00`。 @@ -371,17 +381,17 @@ edge = alpha 梯度区域 + 外扩 1 - 3 px result_rgba.png debug/bg_confidence.png debug/trimap.png +debug/seg_mask.png # 分割管线才有 debug/alpha.png -debug/foreground_rgb.png -debug/foreground_background.png -debug/foreground_correction.png -debug/despill_mask.png +debug/foreground_rgb.png # 去溢后的前景色 +debug/color_mask.png # 每像素去溢权重 debug/preview_black.png debug/preview_white.png debug/preview_gray.png debug/preview_red.png debug/preview_blue.png debug/qa_grid.png +debug/metadata.json ``` ## 16. QA 质检