Sync docs to current architecture
README rewritten for the current design: two pipelines (chroma-only vs single segmenter, default anime-seg), the screen_color prior (--screen-color / config, green default), correct debug-file list, and HF/hf-mirror notes for the seg models. No rescue/recolour layer is documented (it was removed). green_screen_matting_workflow.md keeps the phase-1 green-screen spec but gains a status note pointing to the README for the evolved architecture, and its debug-file list is corrected (foreground_rgb / color_mask / seg_mask, not the old foreground_background / foreground_correction / despill_mask). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,14 +1,44 @@
|
||||
# BgFilter
|
||||
|
||||
Offline green screen character matting for AI-generated character images.
|
||||
|
||||
The first implementation follows the workflow in
|
||||
[`docs/green_screen_matting_workflow.md`](docs/green_screen_matting_workflow.md):
|
||||
Offline character matting for AI-generated images on a flat-colour background.
|
||||
Green screen is the default; any flat colour (e.g. a pastel `#CFEFFF`) works via a
|
||||
`screen_color` prior.
|
||||
|
||||
```text
|
||||
RGB input -> chroma confidence -> trimap -> ViTMatte -> alpha cleanup -> despill -> RGBA PNG -> QA previews
|
||||
RGB input
|
||||
-> chroma bg-confidence (keyed to the screen colour)
|
||||
-> [optional] semantic segmentation mask
|
||||
-> trimap -> ViTMatte -> alpha cleanup
|
||||
-> pymatting foreground -> despill -> RGBA PNG -> QA previews
|
||||
```
|
||||
|
||||
## Pipelines
|
||||
|
||||
Two pipelines, selected by `segmentation.enabled` in the config:
|
||||
|
||||
- **Chroma-only** (`enabled: false`) — Chroma + ViTMatte, no segmentation model.
|
||||
Lightest / fastest; leans entirely on the colour key for topology.
|
||||
- **Single segmenter** (`enabled: true`, default) — one segmentation model drives the
|
||||
trimap topology (holes, hair), ViTMatte then refines the soft edges. Backend is
|
||||
switchable: `anime-seg` (default, ONNX) or `birefnet` (needs `trust_remote_code`).
|
||||
|
||||
Both share pymatting foreground estimation and a colour de-spill. There is **no**
|
||||
green-contamination rescue / recolour layer — with clean source images it is
|
||||
unnecessary, so it was removed.
|
||||
|
||||
## Background colour
|
||||
|
||||
Default is green auto-detection. For any other flat background, pass the colour as a
|
||||
hex prior so both the chroma key and the de-spill work off it:
|
||||
|
||||
```powershell
|
||||
... --screen-color "#CFEFFF"
|
||||
```
|
||||
|
||||
or set `screen_color: "#CFEFFF"` in the config (`null` = green auto-detect). The
|
||||
chroma key then scores pixels by perceptual (Lab/RGB) distance to that colour, and
|
||||
de-spill removes chroma along that colour's direction instead of the green channel.
|
||||
|
||||
## Environment
|
||||
|
||||
Use the conda environment `lightML`.
|
||||
@@ -32,23 +62,13 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
--output Outputs\TestImage_rgba.png `
|
||||
--debug-dir Outputs\TestImage_debug `
|
||||
--config configs\default.yaml `
|
||||
--screen-color "#CFEFFF" `
|
||||
--device cuda
|
||||
```
|
||||
|
||||
The CLI reads `configs/default.yaml` when `--config` is provided. Command-line
|
||||
options override config values, so tuning can usually happen in YAML while
|
||||
runtime choices such as `--device cpu` stay on the command line.
|
||||
|
||||
Use CPU for validation when CUDA is unavailable:
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
--input Samples\TestImage.png `
|
||||
--output Outputs\TestImage_rgba.png `
|
||||
--debug-dir Outputs\TestImage_debug `
|
||||
--config configs\default.yaml `
|
||||
--device cpu
|
||||
```
|
||||
options override config values, so tuning usually happens in YAML while runtime
|
||||
choices such as `--device cpu` or `--screen-color` stay on the command line.
|
||||
|
||||
## Batch
|
||||
|
||||
@@ -61,10 +81,11 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
--device cuda
|
||||
```
|
||||
|
||||
## Chroma-Only Debug Mode
|
||||
## Chroma-alpha debug mode
|
||||
|
||||
This mode skips ViTMatte and uses chroma confidence as an alpha seed. It is useful
|
||||
for fast debugging of chroma confidence, trimap, despill, and QA outputs.
|
||||
`--matting-method chroma` skips ViTMatte and uses chroma confidence directly as the
|
||||
alpha seed. Useful for fast inspection of chroma confidence, trimap, and despill.
|
||||
(Distinct from the chroma-only *pipeline* above, which still runs ViTMatte.)
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
|
||||
@@ -83,11 +104,10 @@ For each processed image, the CLI writes an RGBA PNG and optional debug files:
|
||||
```text
|
||||
bg_confidence.png
|
||||
trimap.png
|
||||
seg_mask.png # segmentation pipeline only
|
||||
alpha.png
|
||||
foreground_rgb.png
|
||||
foreground_background.png
|
||||
foreground_correction.png
|
||||
despill_mask.png
|
||||
foreground_rgb.png # despilled foreground colour
|
||||
color_mask.png # per-pixel despill weight
|
||||
preview_black.png
|
||||
preview_white.png
|
||||
preview_gray.png
|
||||
@@ -99,8 +119,8 @@ metadata.json
|
||||
|
||||
## Quality Check
|
||||
|
||||
The quality checker measures alpha validity and green spill on semi-transparent
|
||||
edge pixels.
|
||||
The quality checker measures alpha validity and edge spill on semi-transparent edge
|
||||
pixels.
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m bgfilter.quality_cli `
|
||||
@@ -120,11 +140,16 @@ D:\MiniConda\envs\lightML\python.exe scripts\smoke_samples.py `
|
||||
--max-edge-green-excess-p95 0.30
|
||||
```
|
||||
|
||||
## Current Notes
|
||||
## Notes
|
||||
|
||||
- `Samples/` and `Outputs/` are ignored by Git.
|
||||
- `ViTMatte` model weights are loaded from Hugging Face on first use.
|
||||
- Foreground color estimation uses pymatting's `estimate_foreground_ml` to
|
||||
propagate clean foreground colour into semi-transparent edges before final
|
||||
de-spill, writing the estimated foreground, background, and a correction map.
|
||||
Set `foreground.method: unmix` to fall back to the legacy heuristic.
|
||||
- ViTMatte and segmentation weights load from Hugging Face on first use. Behind a
|
||||
firewall set `HF_ENDPOINT=https://hf-mirror.com` (and bypass a flaky local proxy).
|
||||
`anime-seg` (`skytnt/anime-seg`) is a plain ONNX download; `birefnet`
|
||||
(`ZhengPeng7/BiRefNet`) ships custom modelling code so it needs
|
||||
`trust_remote_code=True` plus `timm` / `einops` / `kornia`.
|
||||
- Foreground colour estimation uses pymatting's `estimate_foreground_ml` to propagate
|
||||
clean foreground colour into semi-transparent edges before de-spill. Set
|
||||
`foreground.method: unmix` to fall back to the legacy heuristic.
|
||||
- `docs/green_screen_matting_workflow.md` is the original phase-1 green-screen spec;
|
||||
this README reflects the current, generalised architecture.
|
||||
|
||||
@@ -4,6 +4,16 @@
|
||||
|
||||
本方案不使用 YOLO / Faster R-CNN 作为主流程。原因是本任务不是目标检测,而是像素级 alpha matting。
|
||||
|
||||
## 0. 现状说明(架构已演进)
|
||||
|
||||
本文档是**第一阶段的绿幕方案原始规格**,仍准确描述 “Chroma + ViTMatte” 这条管线。实现此后已泛化,**当前架构以 [README](../README.md) 为准**,主要变化:
|
||||
|
||||
- **两条管线**(`segmentation.enabled` 切换):无分割模型的 Chroma + ViTMatte;或单个语义分割模型(默认 anime-seg,可切 birefnet)定 trimap 拓扑,ViTMatte 细化。
|
||||
- **背景色泛化**:`screen_color` 先验(如 `#CFEFFF`),chroma / despill 不再写死绿色;`null` 时才走本文档的绿幕自动探测。
|
||||
- **已移除**:染绿修复(recolor)、发丝补救(dropout-fill)、BiRefNet+anime-seg 结合方案——它们只是给“生图即染色/断裂”的坏素材擦屁股;源头改用干净背景生图后不再需要。
|
||||
|
||||
下文其余章节描述绿幕基线流程,仍然有效。
|
||||
|
||||
## 1. 目标
|
||||
|
||||
- 输入:RGB / PNG 图像,背景主色为绿色,但不要求严格等于 `#00FF00`。
|
||||
@@ -371,17 +381,17 @@ edge = alpha 梯度区域 + 外扩 1 - 3 px
|
||||
result_rgba.png
|
||||
debug/bg_confidence.png
|
||||
debug/trimap.png
|
||||
debug/seg_mask.png # 分割管线才有
|
||||
debug/alpha.png
|
||||
debug/foreground_rgb.png
|
||||
debug/foreground_background.png
|
||||
debug/foreground_correction.png
|
||||
debug/despill_mask.png
|
||||
debug/foreground_rgb.png # 去溢后的前景色
|
||||
debug/color_mask.png # 每像素去溢权重
|
||||
debug/preview_black.png
|
||||
debug/preview_white.png
|
||||
debug/preview_gray.png
|
||||
debug/preview_red.png
|
||||
debug/preview_blue.png
|
||||
debug/qa_grid.png
|
||||
debug/metadata.json
|
||||
```
|
||||
|
||||
## 16. QA 质检
|
||||
|
||||
Reference in New Issue
Block a user