完善序列帧去背景交互与队列处理
将序列帧去背景选择框改为贴近画布的非模态悬浮窗 增加原始帧取色、实时色样与自适应方向的像素放大预览 区分提交接受状态与 durable 任务处理状态，入队成功后立即关闭悬浮窗 支持纯色键控轻微色差并补充通用模式与后端回归验证 同步更新Spine技术方案与项目决策记录
This commit is contained in:
@@ -24,7 +24,6 @@
|
||||
- 验证方式：覆盖 Codex 分类与敏感诱饵、失败事件公共摘要、最近任务与各正式卡片、game-chat 阶段记录、待核对状态、final-reply fallback 白名单及 malformed 响应完整重试；运行 Rust 定向测试、前端模型/AppSurface 定向测试、Shell typecheck、编码和 diff 门禁。
|
||||
- 关联文档：`docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`。
|
||||
|
||||
|
||||
## 2026-08-10 资源管理评审阻塞项按第二轮正式合同修复
|
||||
|
||||
- 背景：资源管理第一轮实现后，人工验证继续暴露 WebView 默认缩放、预览队列饥饿、过滤后媒体残留播放、外层滚动串 scope、超深依赖坐标越过 Rust 上限和暂时错误无法重试等问题。部分 PRD / 技术方案仍描述第一轮的中央媒体预览、单全局 Overlay 和统一 section scope，已经与第二轮代码及验收结论冲突。
|
||||
@@ -68,6 +67,7 @@
|
||||
- 替代关系：本条替代下方 2026-08-07 阶段二中“缩放只等于可视高度”和“只为完整可见卡片建立端点”的显示口径；其中曾采用的全局 SVG、四 viewport 联合 clip 和单全局 Observer 又由上方“依赖 SVG 改为分区 plane 所有”决定替代。高度模型、四分区、会话隔离、内外滚动和无布局 CAS 等其它决定继续有效。阶段一媒体卡、阶段三确定性聚类、Rust `dependencyDepths`、producer 截断降级、历史手动坐标与 type sidecar 均不变。
|
||||
- 验证方式：纯倍率模型覆盖按钮 / wheel 边界；AppSurface 覆盖项目、mode、section 隔离、普通 wheel、Ctrl wheel、WebKit gesture 与零布局写入；SVG 覆盖同 plane 倍率、橙色 marker、直线 / 小圆角横纵路由、自环间隙、双向边界继续线、分区原生裁剪、每区单 Observer、task-flow 零渲染和 4096 精确关系有界输出。
|
||||
- 关联文档：`docs/prd/【AI游戏创作】项目开发工作台PRD-2026-07-20.md`、`docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`。
|
||||
|
||||
## 2026-08-11 Tauri 无限画布以可证明事务和分类恢复收口
|
||||
|
||||
- 终态与队列：资源编辑账本正式区分可继续阶段、`reconciliation-required`、`remote-failed` 和 `archived`。远端明确失败只保存稳定分类与终态时间，不得再 POST、轮询或重新扣费；只有该终态能由用户显式归档并移出活动恢复队列，归档保留账本且不伪装 `committed`。结果未知和需对账项继续失败关闭。
|
||||
@@ -7192,7 +7192,6 @@
|
||||
- 决策：只有同一可信根 Supervisor 能调用 `agent.acceptance_update`，且该控制面动作必须独占一轮，不能夹带 `plan_update`、legacy plan 或回复。requiredEvidence 采用 `tool:<Runtime 工具名>`，合同冻结前必须命中 Runtime 允许的持久证据工具集合并拒绝拼错、控制面和纯协调工具；动态 MCP catalog 不得冻结为不可变 requiredEvidence。passed 节点必须引用当前根任务树中对应工具的真实成功动作回执；回执同时记录动作执行边界的 `projectRevisionBefore / projectRevisionAfter`，非 mutation evidence 必须满足 before、after 与验收时 current revision 完全相同，mutation evidence 的 after 必须等于 current revision，旧 revision 或执行后延迟落账的回执不可重放。failed、not-observed、缺失节点以及落后当前 project revision 的整图确认状态均阻断普通完成、finalization 与恢复。项目 revision 变化后由 Supervisor 只更新受影响节点并确认当前图，未提交的 passed 节点保持不变；Runtime 不替 Supervisor 推断影响范围、选择具体 Agent 或实现方式。
|
||||
- 决策：不可变 Goal Contract 的根 Run 收到 steer 时，必须按旧 rootRunId 串行化整个转换，并在持锁后重新确认该旧根仍是 Session 当前权威 Run，避免不同 steerId 并发创建多个 replacement。随后为绑定旧 rootRunId 的全部非终态静态、ready 和 isolated 后代写入取消栅栏、打断 Provider 并逐个收束，再终止旧根；所有项目修改入口在看到取消栅栏后立即失败关闭。Runtime 必须确认旧树所有成员都已进入终态或 `needs-reconciliation`，超时则保持等待并拒绝启动 replacement；只有旧树停稳后，才在同一 Session、source 和 Run Profile 启动新根 Run 重新理解完整目标。
|
||||
|
||||
|
||||
## 2026-08-08 External v1 图片编辑来源字段允许原地收紧
|
||||
|
||||
- 背景：图片编辑主来源已经从可由客户端提交 objectKey 和类型提示的 `sourceImageSrc / sourceResourceId / assetKind`，收紧为服务端按项目资源 ID 或素材 ID 解析权威对象与类型的必填 `sourceReferenceId`。这会让严格 External v1 客户端立即失败，属于现役版本策略明确列出的 breaking change；2026-07-31 的历史豁免不能自动覆盖本次变更。
|
||||
@@ -8661,6 +8660,7 @@
|
||||
- 2026-06-19 桌面壳外链打开 helper 共用：Tauri WebView 外域拦截和 HostBridge `app.openExternalUrl` 都必须复用 `open_normalized_desktop_external_url` 执行系统外链打开动作；HostBridge 分支仍先用 `normalize_external_url` 保留 payload 错误语义并把 opener 错误回传给 H5，WebView 拦截保持 best-effort 静默处理。桌面壳配置检查会拒绝 `dispatch.rs` 直接调用 `app.opener().open_url` 绕过该 helper，避免两条离壳路径漂移。
|
||||
|
||||
> 2026-07-18 覆盖说明：本段后续关于微信 `navigation.openNativePage`、生成结果订阅页、`[subscribe-message]` 日志和订阅页路由门禁的 2026-06 决策均已由旧创作模板退役决策废止，只作为历史记录。Expo / Tauri 的同源 H5 受控导航及微信登录、支付、分享能力继续有效。
|
||||
|
||||
- 2026-06-20 H5 原生导航预校验：`navigateHostNativePage()` 在 `native_app` 下发送 `navigation.openNativePage` 前必须先拒绝空值、控制字符、协议相对 URL、外域绝对 URL 和非 `http:` / `https:` 协议目标；同源绝对 URL、`/path` 和保留给桌面壳兼容的相对 route 继续交给 Expo / Tauri 壳二次归一并补写宿主上下文。微信小程序分支仍按小程序页面 URL 语义走 `wx.miniProgram.navigateTo`，不套原生 App 同源 H5 预校验。根级 `npm run check:native-shells` 会反查 H5 facade 仍使用 `normalizeNativeAppPageUrl(...)` 且发送归一后的 URL，避免明显不安全目标触达原生壳。
|
||||
- 2026-06-20 微信受控原生页能力声明：微信小程序壳真实 capability profile 声明 `navigation.openNativePage`，用于承接已经登记并测试的小程序原生页 flow；当前订阅生成结果通知页通过 H5 `requestGenerationResultSubscribePermission()` 调用 `navigateHostNativePage()` 打开 `/pages/subscribe-message/index`，小程序页再调用真实 `wx.requestSubscribeMessage` 并按既有结果协议回灌。根级 `npm run check:native-shells` 必须把该能力反查到共享 profile、微信 `WECHAT_HOST_CAPABILITIES` 镜像、订阅页协议常量、H5 入口、小程序 host-bridge / shell / page 文件和相关测试；该能力不代表开放任意小程序页面跳转。
|
||||
- 2026-06-18 能力声明收紧：`packages/shared/src/contracts/hostBridge.ts` 提供 HostBridge method / capability 白名单，H5 的 `getHostRuntime()` 会解析并过滤 `hostCapabilities`；`openHostShare`、`writeHostClipboardText`、`requestHostHapticsImpact`、`setHostAppTitle`、`exportHostTextFile` 等 native 能力只在宿主声明对应 capability 后调用。发布分享弹窗只有声明 `share.open` 时才显示受控分享动作，并按 `hostShell` 区分 Expo 系统分享面板和 Tauri 剪贴板复制表达，避免旧壳或裁剪壳露出不可用入口。
|
||||
@@ -14075,6 +14075,7 @@
|
||||
- 分类：内部稳定 reason code 固定为 `translation_invalid / translation_upstream_failed / translation_budget_exhausted / elevenlabs_http_failed / invalid_audio / duration_probe_failed / oss_failed / writeback_failed`。MIME、空 body 和大小归 `invalid_audio`；MP3 识别、帧读取和时长门禁归 `duration_probe_failed`。普通用户继续只读稳定短文案，不暴露 endpoint、上游正文或凭据。
|
||||
- 跨入口：画布 Agent `generate-sound-effect` 与站内 / External v1 共用 canonical Prompt、固定模型、nullable `0.5-30` 小数时长和 Loop；省略 duration 为手动 `5s`，显式 null 为自动。SFX 参数解析必须保留该 null，不能被通用 null-default 兼容层改写。最终仍进入相同 `editor_sound_effect_generation` queue payload，不新增 Agent 专属链路。
|
||||
- 发布边界：T6 工程实施和 mock / loopback 门禁不等于真实 provider 或生产验收。发布前关闭 SFX 入队，使用显式 `--server` / `--server-url` 只读查询 `external_generation_job` 中 pending / running 的 `editor_sound_effect_generation`，清零后按 api-server / Worker → Web 顺序部署并灰度；禁止 `--root-dir`、删除任务伪造 drain 或自动回退 Vidu。本次没有 SpacetimeDB schema、migration 或 bindings 变更。
|
||||
|
||||
## 2026-08-06 编辑器生成结果使用 durable receipt 与统一原子提交
|
||||
|
||||
- 背景：图片、改图、去背景、图集 / UI 多产物、角色动作、视频、音效和背景音乐在 OSS 结果可用后，仍分段 confirm object、创建 project resource / account asset、保存 canvas 和 complete job。任一中间失败都会留下部分业务事实；只把 `external_generation_job` 当 operation journal 又无法覆盖无 job 的 inline，也无法独立证明某批 resource/asset/canvas 已作为一笔提交完成。
|
||||
@@ -14189,6 +14190,6 @@
|
||||
## 2026-08-17 Spine 视频转换去背景改为显式双模式
|
||||
|
||||
- 背景：仅凭 `generationInputs.action = character-animation.convert` 无法证明源视频是绿幕；把任意转换结果固定按 `#00FF00` 键控会误处理普通背景或绿色主体。仓库已有 BgFilter `complex + birefnet` 通用分割服务，也已有可传 `screen_color` 的 flat 协议，但纯色场景不需要承担外部服务成本。
|
||||
- 决策：只有视频直接转换序列继续显示“去背景”，点击后必须在独立弹窗显式选择 `solid-color` 或 `general`。纯色模式默认 `#00FF00`，允许颜色输入及浏览器吸管，后端使用参数化本地键色算法；通用模式逐帧调用 BgFilter complex，不携带键色。HTTP 入队和 durable worker 共用白名单校验，后端在结果 `generationInputs` 中权威记录模式、颜色或 Provider/model，不能再把“视频转换”本身当作绿幕证明。两种模式仍生成新序列资源、整批失败关闭并保留源序列。
|
||||
- 决策：只有视频直接转换序列继续显示“去背景”，点击后必须在当前选中序列帧旁的非模态悬浮窗中显式选择 `solid-color` 或 `general`；悬浮窗随画布平移和缩放更新位置，优先置于右侧并在空间不足时翻转，不增加画布遮罩、不覆盖源画面，纯色模式不使用系统吸管；内置逐帧 viewer 从正式 objectKey 读取原始帧字节，按原始比例实际绘制，且仅在可见图像区域内采样。用户悬停时实时显示原始像素色样与 11×11 放大网格，点击才回填键色，以消除容器留白和 CSS 缩放导致的取色位移。纯色模式默认 `#00FF00`，允许颜色输入及逐帧 viewer 原始像素取样；后端使用参数化本地键色算法，对轻微压缩噪声和接近键色的平缓色差有容忍度，但不承诺处理明显渐变、阴影、纹理或与主体相近的背景，后者应选择 `general`。通用模式逐帧调用 BgFilter complex，不携带键色。悬浮窗的“提交中”只表示 HTTP 入队请求，服务端接受任务后立即关闭悬浮窗，由画布生成占位和任务列表承接 durable worker 的后续状态；HTTP 提交失败则保留悬浮窗并恢复可重试状态。HTTP 入队和 durable worker 共用白名单校验，后端在结果 `generationInputs` 中权威记录模式、颜色或 Provider/model，不能再把“视频转换”本身当作绿幕证明。两种模式仍生成新序列资源、整批失败关闭并保留源序列。
|
||||
- 费用边界：本次沿用现有手动 BgFilter 的 0 泥点内部契约；供应商成本不等于已确定的用户泥点售价，在没有正式定价决策时不自行新增价格。若后续收费，必须接入统一后端定价、入队冻结价格和 worker 扣费退款链路。
|
||||
- 验证：覆盖缺失/非法纯色、通用模式禁止颜色、任意键色透传、绿色主体在非绿色键色下保留、双模式 worker 路由、前端模式选择及吸管回填，并运行角色动作 Rust 定向测试、前端 Vitest、类型检查、编码和 diff 门禁。
|
||||
|
||||
@@ -107,21 +107,21 @@ manifest.txt
|
||||
|
||||
主要文件：
|
||||
|
||||
| 责任 | 当前文件 |
|
||||
| --- | --- |
|
||||
| 画布工具类型和状态 | `src/components/image-editor/ImageCanvasEditorTypes.ts` |
|
||||
| 底部工具栏 | `src/components/image-editor/ImageCanvasBottomToolbarView.tsx` |
|
||||
| 生成 Dialog 草稿和恢复 | `src/components/image-editor/ImageCanvasGenerationDialogModel.ts` |
|
||||
| 生成 Composer | `src/components/image-editor/ImageCanvasGenerationComposerView.tsx` |
|
||||
| 角色动作专用旧面板 | `src/components/image-editor/ImageCanvasCharacterAnimationPanelView.tsx` |
|
||||
| 提交计划与前端校验 | `src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts` |
|
||||
| 画布选取与改造恢复 | `src/components/image-editor/useImageCanvasGenerationWorkflow.ts` |
|
||||
| 上传图片/视频 | `src/components/image-editor/useImageCanvasUploadWorkflow.ts` |
|
||||
| 提交、队列和回填 | `src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts` |
|
||||
| API Client 和前端 DTO | `src/services/image-editor/editorProjectClient.ts` |
|
||||
| 序列帧播放 | `src/components/image-editor/ImageCanvasWorldView.tsx` |
|
||||
| 元数据 | `src/components/image-editor/ImageCanvasMetadataModalView.tsx` |
|
||||
| ZIP / Spine JSON | `ImageCanvasExportModel.ts`、`useImageCanvasAssetExportWorkflow.ts` |
|
||||
| 责任 | 当前文件 |
|
||||
| ---------------------- | --------------------------------------------------------------------------- |
|
||||
| 画布工具类型和状态 | `src/components/image-editor/ImageCanvasEditorTypes.ts` |
|
||||
| 底部工具栏 | `src/components/image-editor/ImageCanvasBottomToolbarView.tsx` |
|
||||
| 生成 Dialog 草稿和恢复 | `src/components/image-editor/ImageCanvasGenerationDialogModel.ts` |
|
||||
| 生成 Composer | `src/components/image-editor/ImageCanvasGenerationComposerView.tsx` |
|
||||
| 角色动作专用旧面板 | `src/components/image-editor/ImageCanvasCharacterAnimationPanelView.tsx` |
|
||||
| 提交计划与前端校验 | `src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts` |
|
||||
| 画布选取与改造恢复 | `src/components/image-editor/useImageCanvasGenerationWorkflow.ts` |
|
||||
| 上传图片/视频 | `src/components/image-editor/useImageCanvasUploadWorkflow.ts` |
|
||||
| 提交、队列和回填 | `src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts` |
|
||||
| API Client 和前端 DTO | `src/services/image-editor/editorProjectClient.ts` |
|
||||
| 序列帧播放 | `src/components/image-editor/ImageCanvasWorldView.tsx` |
|
||||
| 元数据 | `src/components/image-editor/ImageCanvasMetadataModalView.tsx` |
|
||||
| ZIP / Spine JSON | `ImageCanvasExportModel.ts`、`useImageCanvasAssetExportWorkflow.ts` |
|
||||
|
||||
通用视频面板已经提供“参考”控件，支持：
|
||||
|
||||
@@ -230,21 +230,21 @@ flowchart LR
|
||||
|
||||
## 5. 与本次需求的差距
|
||||
|
||||
| 需求 | 当前能力 | 结论 |
|
||||
| --- | --- | --- |
|
||||
| 工具栏在视频后新增入口 | 当前已有角色图 `生成动画` 快捷入口 | 新增底部通用入口并复用同一 dialog |
|
||||
| 纯文本生成 | 当前必须有角色源图 | 需要扩展后端 T2V |
|
||||
| 图片 + 文本 | 已实现角色立绘图生视频再抽帧 | 主要复用 |
|
||||
| 已有序列帧改造 | 仅有部分恢复/首帧能力，来源语义不完整 | 需要明确输入形态并补全 |
|
||||
| 视频 + 文本 | 通用视频支持参考视频，角色动作链未接入 | 需要扩展 |
|
||||
| 参考支持画布/上传图片视频 | 通用视频面板已经实现 | 复用并收窄规则 |
|
||||
| 白/绿/蓝/紫背景 | 当前角色动作使用自动安全色，不是这四个纯色 | 需要专用枚举和提示词 |
|
||||
| 动作“更多”逐行展开 | 当前只有固定 7 个预设 | 需要前端扩展 |
|
||||
| 1080p | Fast 前后端均不支持 | 暂不开放 |
|
||||
| 40 泥点/次 | 当前 480p × 4 秒恰好为 40；其他组合不同 | 需确认固定价还是按秒价 |
|
||||
| 去背景 | AI 生成链自动透明化；直接转换保留背景 | 保留零泥点手动派生任务 |
|
||||
| 拆帧生成单 PNG | 帧对象已有，但没有全部登记为独立账号素材 | 可复用对象、补资产绑定 |
|
||||
| ZIP 下载 | 已完成且匹配附件 | 不重做 |
|
||||
| 需求 | 当前能力 | 结论 |
|
||||
| ------------------------- | ------------------------------------------ | --------------------------------- |
|
||||
| 工具栏在视频后新增入口 | 当前已有角色图 `生成动画` 快捷入口 | 新增底部通用入口并复用同一 dialog |
|
||||
| 纯文本生成 | 当前必须有角色源图 | 需要扩展后端 T2V |
|
||||
| 图片 + 文本 | 已实现角色立绘图生视频再抽帧 | 主要复用 |
|
||||
| 已有序列帧改造 | 仅有部分恢复/首帧能力，来源语义不完整 | 需要明确输入形态并补全 |
|
||||
| 视频 + 文本 | 通用视频支持参考视频，角色动作链未接入 | 需要扩展 |
|
||||
| 参考支持画布/上传图片视频 | 通用视频面板已经实现 | 复用并收窄规则 |
|
||||
| 白/绿/蓝/紫背景 | 当前角色动作使用自动安全色，不是这四个纯色 | 需要专用枚举和提示词 |
|
||||
| 动作“更多”逐行展开 | 当前只有固定 7 个预设 | 需要前端扩展 |
|
||||
| 1080p | Fast 前后端均不支持 | 暂不开放 |
|
||||
| 40 泥点/次 | 当前 480p × 4 秒恰好为 40；其他组合不同 | 需确认固定价还是按秒价 |
|
||||
| 去背景 | AI 生成链自动透明化；直接转换保留背景 | 保留零泥点手动派生任务 |
|
||||
| 拆帧生成单 PNG | 帧对象已有，但没有全部登记为独立账号素材 | 可复用对象、补资产绑定 |
|
||||
| ZIP 下载 | 已完成且匹配附件 | 不重做 |
|
||||
|
||||
## 6. 目标业务模型
|
||||
|
||||
@@ -335,20 +335,20 @@ video = 一段视频作为动作/外观参考
|
||||
|
||||
三模态所需事实已经能放入现有字段：
|
||||
|
||||
| 事实 | 存放位置 |
|
||||
| --- | --- |
|
||||
| 输入模式 | `generation_inputs_json.fields[inputMode]` |
|
||||
| 用户动作描述 | `generation_inputs_json.fields[prompt]` |
|
||||
| 动作预设 | `generation_inputs_json.fields[actionPreset]` |
|
||||
| 背景色 | `generation_inputs_json.fields[backgroundColor]` |
|
||||
| 清晰度/比例/时长/帧数 | `generation_inputs_json.fields[...]` |
|
||||
| 参考素材 | `generation_inputs_json.references`（稳定 resource/asset ID） |
|
||||
| 实际模型提示词 | `actual_prompt` |
|
||||
| 模型、provider、task | 现有列 |
|
||||
| 帧顺序和对象 | `image_sequence_frames_json` |
|
||||
| 总时长 | `image_sequence_duration_ms` |
|
||||
| 单帧尺寸 | `width/height` |
|
||||
| 来源链 | `source_resource_id` |
|
||||
| 事实 | 存放位置 |
|
||||
| --------------------- | ------------------------------------------------------------- |
|
||||
| 输入模式 | `generation_inputs_json.fields[inputMode]` |
|
||||
| 用户动作描述 | `generation_inputs_json.fields[prompt]` |
|
||||
| 动作预设 | `generation_inputs_json.fields[actionPreset]` |
|
||||
| 背景色 | `generation_inputs_json.fields[backgroundColor]` |
|
||||
| 清晰度/比例/时长/帧数 | `generation_inputs_json.fields[...]` |
|
||||
| 参考素材 | `generation_inputs_json.references`（稳定 resource/asset ID） |
|
||||
| 实际模型提示词 | `actual_prompt` |
|
||||
| 模型、provider、task | 现有列 |
|
||||
| 帧顺序和对象 | `image_sequence_frames_json` |
|
||||
| 总时长 | `image_sequence_duration_ms` |
|
||||
| 单帧尺寸 | `width/height` |
|
||||
| 来源链 | `source_resource_id` |
|
||||
|
||||
不建议给 `editor_project_resource` 或 `editor_asset` 新增 `input_mode/background_color/fps/frame_count` 列：
|
||||
|
||||
@@ -603,11 +603,11 @@ flowchart TD
|
||||
|
||||
#### 去背景
|
||||
|
||||
AI 生成链默认逐帧去背景，视频直接转换固定保留源背景。只有 `character-animation.convert` 的视频直接转换结果提供 `去背景`；AI 生成的序列帧已自动完成去背景，不展示也不接受重复处理。`character-animation` 浮动工具栏在适用时提供 `去背景`，与 `改造 / 拆帧 / 下载` 并列；点击后弹出独立选择框，用户必须显式选择处理方式，再进入现有 durable 派生任务，不得把序列第一帧当普通图片处理。派生任务保持以下边界：
|
||||
AI 生成链默认逐帧去背景，视频直接转换固定保留源背景。只有 `character-animation.convert` 的视频直接转换结果提供 `去背景`；AI 生成的序列帧已自动完成去背景，不展示也不接受重复处理。`character-animation` 浮动工具栏在适用时提供 `去背景`，与 `改造 / 拆帧 / 下载` 并列；点击后在当前选中序列帧旁打开非模态悬浮窗；悬浮窗随画布平移和缩放更新位置，优先置于画面右侧、空间不足时翻转到其它可用侧，不增加画布遮罩或覆盖源画面；纯色模式内置逐帧 viewer，从正式 objectKey 读取原始帧字节。viewer 以原始比例实际绘制帧，鼠标仅在图像可见区域内采样；悬停实时显示原始像素颜色与放大网格，点击后才回填键色，避免容器留白或 CSS 缩放导致取样偏移。用户必须显式选择处理方式，再进入现有 durable 派生任务，不得把序列第一帧当普通图片处理。派生任务保持以下边界：
|
||||
|
||||
- `solid-color` 为本地纯色键控，默认 `#00FF00`，允许颜色输入和浏览器吸管选择任意 `#RRGGBB`；该模式不调用 BgFilter、阿里云或其它外部模型。
|
||||
- `solid-color` 为本地纯色键控，默认 `#00FF00`，允许颜色输入和逐帧 viewer 原始像素取样选择任意 `#RRGGBB`；本地算法容忍轻微压缩噪声和接近键色的平缓色差，但不承诺处理明显渐变、阴影、纹理或与主体接近的背景，此类输入应选 `general`；该模式不调用 BgFilter、阿里云或其它外部模型。
|
||||
- `general` 为通用智能分割，不携带 `screenColor`，逐帧调用 BgFilter `complex + birefnet`；普通背景不再被隐式当作绿幕处理。当前内部接口沿用现有手动 BgFilter 的 0 泥点契约，不在缺少正式定价决策时自行发明用户收费金额；外部供应商成本与后续用户定价另行配置。
|
||||
- 前端只负责显式选择；HTTP 入队与 durable worker 共用白名单校验：纯色模式必须携带合法颜色，通用模式禁止携带颜色。worker 只读取已持久化任务参数，最终资源的 `generationInputs` 由后端写入模式、键色或 Provider/model，不信任客户端自报配方。
|
||||
- 前端只负责显式选择；HTTP 入队与 durable worker 共用白名单校验：纯色模式必须携带合法颜色，通用模式禁止携带颜色。worker 只读取已持久化任务参数，最终资源的 `generationInputs` 由后端写入模式、键色或 Provider/model，不信任客户端自报配方。悬浮窗的“提交中”只覆盖 HTTP 入队请求；服务端接受任务后立即关闭悬浮窗并交给画布生成占位/任务列表展示后续进度，不能把 durable worker 的完整处理时间继续显示为提交中。HTTP 提交失败时保留悬浮窗并恢复可重试状态。
|
||||
- 原始不透明序列与透明序列是两个正式 resource；
|
||||
- 去背景全帧成功后一次性发布新的 `character-animation` / `image-sequence` 资源、素材和画布图层，保留原始序列图层不被替换；
|
||||
- 任一帧失败不产生残缺结果；
|
||||
@@ -708,6 +708,7 @@ idle
|
||||
### P0：会改变架构或成本
|
||||
|
||||
1. **视频模式到底是哪一种？**
|
||||
|
||||
- A：原视频直接逐帧转 PNG，不调用 Seedance；
|
||||
- B：用视频 + 文本生成新视频（换皮/改造），再抽新视频；
|
||||
- C：两种都要，由用户选择“直接转换/AI 改造”。
|
||||
@@ -729,6 +730,7 @@ idle
|
||||
5. **1080p 是否允许换模型？**
|
||||
|
||||
当前 `seedance2.0-fast` 明确不支持 1080p。可选：
|
||||
|
||||
- 暂不显示（推荐）；
|
||||
- 1080p 自动切换 Seedance 2.0 标准版并重新定价；
|
||||
- 等 provider Fast 真正支持后再开放。
|
||||
@@ -736,6 +738,7 @@ idle
|
||||
### P1：会改变交互或数据语义
|
||||
|
||||
6. **“已有 Spine 序列帧 png 图片”指什么？**
|
||||
|
||||
- 已有 `character-animation` 帧集；
|
||||
- 一张拼好的 Sprite Sheet PNG；
|
||||
- 任意单张角色 PNG。
|
||||
@@ -755,6 +758,7 @@ idle
|
||||
需要确定值，例如纯蓝 `#0000FF`、纯紫 `#8000FF`；抠图必须用确定 RGB，不能只存“蓝色/紫色”。还要确认主体颜色与键色冲突时是阻止提交、警告，还是自动换色。
|
||||
|
||||
10. **拆帧后去哪里？**
|
||||
|
||||
- 只进入素材库（推荐）；
|
||||
- 自动铺到画布；
|
||||
- 弹出独立图片集面板；
|
||||
|
||||
@@ -100,6 +100,82 @@ export function resolveSelectedToolbarStyle({
|
||||
};
|
||||
}
|
||||
|
||||
export function resolveCharacterAnimationBackgroundRemovalPanelStyle({
|
||||
sourceLayer,
|
||||
viewport,
|
||||
canvasSize,
|
||||
}: {
|
||||
sourceLayer: CanvasLayer | null;
|
||||
viewport: CanvasViewport;
|
||||
canvasSize: CanvasSize;
|
||||
}): CanvasOverlayStyle | null {
|
||||
if (!sourceLayer) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const panelWidth = 288;
|
||||
const panelHeight = 440;
|
||||
const gap = 12;
|
||||
const edge = 12;
|
||||
const sourceLeft = viewport.x + sourceLayer.x * viewport.scale;
|
||||
const sourceTop = viewport.y + sourceLayer.y * viewport.scale;
|
||||
const sourceWidth = sourceLayer.width * viewport.scale;
|
||||
const sourceHeight = sourceLayer.height * viewport.scale;
|
||||
const sourceRight = sourceLeft + sourceWidth;
|
||||
const sourceBottom = sourceTop + sourceHeight;
|
||||
const clampPanelLeft = (left: number) =>
|
||||
Math.min(Math.max(edge, left), Math.max(edge, canvasSize.width - panelWidth - edge));
|
||||
const clampPanelTop = (top: number) =>
|
||||
Math.min(Math.max(edge, top), Math.max(edge, canvasSize.height - panelHeight - edge));
|
||||
|
||||
const rightLeft = sourceRight + gap;
|
||||
if (rightLeft + panelWidth <= canvasSize.width - edge) {
|
||||
return { left: rightLeft, top: clampPanelTop(sourceTop) };
|
||||
}
|
||||
|
||||
const leftLeft = sourceLeft - gap - panelWidth;
|
||||
if (leftLeft >= edge) {
|
||||
return { left: leftLeft, top: clampPanelTop(sourceTop) };
|
||||
}
|
||||
|
||||
const bottomTop = sourceBottom + gap;
|
||||
if (bottomTop + panelHeight <= canvasSize.height - edge) {
|
||||
return {
|
||||
left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2),
|
||||
top: bottomTop,
|
||||
};
|
||||
}
|
||||
|
||||
const topTop = sourceTop - gap - panelHeight;
|
||||
if (topTop >= edge) {
|
||||
return {
|
||||
left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2),
|
||||
top: topTop,
|
||||
};
|
||||
}
|
||||
|
||||
// 极端缩放下四侧都放不下完整面板时，仍把面板放在可用空间最大的一侧，
|
||||
// 保证它不覆盖源画面；viewport 会裁掉越界部分，用户可先缩小画布继续操作。
|
||||
const availableSides = [
|
||||
{ space: canvasSize.width - sourceRight, left: rightLeft, top: clampPanelTop(sourceTop) },
|
||||
{ space: sourceLeft, left: leftLeft, top: clampPanelTop(sourceTop) },
|
||||
{
|
||||
space: canvasSize.height - sourceBottom,
|
||||
left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2),
|
||||
top: bottomTop,
|
||||
},
|
||||
{
|
||||
space: sourceTop,
|
||||
left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2),
|
||||
top: topTop,
|
||||
},
|
||||
];
|
||||
const bestSide = availableSides.reduce((best, candidate) =>
|
||||
candidate.space > best.space ? candidate : best,
|
||||
);
|
||||
return { left: bestSide.left, top: bestSide.top };
|
||||
}
|
||||
|
||||
export function resolveQuickEditFocusViewport({
|
||||
sourceLayer,
|
||||
canvasSize,
|
||||
|
||||
@@ -316,6 +316,35 @@ mod tests {
|
||||
assert_eq!(image.get_pixel(1, 1).0, [255, 255, 255, 255]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn character_animation_solid_color_removal_tolerates_small_background_color_variation() {
|
||||
let key = [72, 161, 224];
|
||||
let mut source = RgbaImage::from_pixel(9, 9, Rgba([82, 151, 229, 255]));
|
||||
source.put_pixel(0, 0, Rgba([65, 168, 217, 255]));
|
||||
source.put_pixel(4, 4, Rgba([232, 52, 48, 255]));
|
||||
let mut bytes = Vec::new();
|
||||
DynamicImage::ImageRgba8(source)
|
||||
.write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png)
|
||||
.expect("test PNG should encode");
|
||||
|
||||
let output = remove_editor_character_animation_solid_color_background(
|
||||
&DownloadedOpenAiImage {
|
||||
bytes,
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
},
|
||||
key,
|
||||
)
|
||||
.expect("near-key solid background should be removed");
|
||||
let image = image::load_from_memory(output.bytes.as_slice())
|
||||
.expect("output PNG should decode")
|
||||
.to_rgba8();
|
||||
|
||||
assert_eq!(image.get_pixel(0, 0).0[3], 0);
|
||||
assert_eq!(image.get_pixel(8, 8).0[3], 0);
|
||||
assert_eq!(image.get_pixel(4, 4).0, [232, 52, 48, 255]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn character_animation_custom_key_color_preserves_green_foreground() {
|
||||
let mut source = RgbaImage::from_pixel(3, 3, Rgba([0, 0, 255, 255]));
|
||||
|
||||
+160
@@ -0,0 +1,160 @@
|
||||
export type FrameCanvasPoint = {
|
||||
x: number;
|
||||
y: number;
|
||||
};
|
||||
|
||||
export type FrameCanvasColor = {
|
||||
red: number;
|
||||
green: number;
|
||||
blue: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* 将指针位置映射到实际绘制出的预览 canvas，而非外层容器。
|
||||
* 超出 canvas 可见像素区域时不取样，避免把边缘颜色误当作背景色。
|
||||
*/
|
||||
export function resolveFrameCanvasSamplePoint({
|
||||
clientX,
|
||||
clientY,
|
||||
rect,
|
||||
imageWidth,
|
||||
imageHeight,
|
||||
}: {
|
||||
clientX: number;
|
||||
clientY: number;
|
||||
rect: Pick<DOMRect, 'left' | 'top' | 'width' | 'height'>;
|
||||
imageWidth: number;
|
||||
imageHeight: number;
|
||||
}): FrameCanvasPoint | null {
|
||||
if (
|
||||
rect.width <= 0 ||
|
||||
rect.height <= 0 ||
|
||||
imageWidth <= 0 ||
|
||||
imageHeight <= 0
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const relativeX = clientX - rect.left;
|
||||
const relativeY = clientY - rect.top;
|
||||
if (
|
||||
relativeX < 0 ||
|
||||
relativeY < 0 ||
|
||||
relativeX >= rect.width ||
|
||||
relativeY >= rect.height
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
x: Math.min(
|
||||
imageWidth - 1,
|
||||
Math.floor((relativeX / rect.width) * imageWidth),
|
||||
),
|
||||
y: Math.min(
|
||||
imageHeight - 1,
|
||||
Math.floor((relativeY / rect.height) * imageHeight),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
export function formatFrameCanvasColorHex({
|
||||
red,
|
||||
green,
|
||||
blue,
|
||||
}: FrameCanvasColor) {
|
||||
return `#${[red, green, blue]
|
||||
.map((channel) =>
|
||||
Math.max(0, Math.min(255, channel)).toString(16).padStart(2, '0'),
|
||||
)
|
||||
.join('')}`.toUpperCase();
|
||||
}
|
||||
|
||||
export function resolveFrameColorMagnifierSourcePoint({
|
||||
point,
|
||||
offsetX,
|
||||
offsetY,
|
||||
imageWidth,
|
||||
imageHeight,
|
||||
}: {
|
||||
point: FrameCanvasPoint;
|
||||
offsetX: number;
|
||||
offsetY: number;
|
||||
imageWidth: number;
|
||||
imageHeight: number;
|
||||
}): FrameCanvasPoint | null {
|
||||
if (imageWidth <= 0 || imageHeight <= 0) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
x: Math.min(imageWidth - 1, Math.max(0, point.x + offsetX)),
|
||||
y: Math.min(imageHeight - 1, Math.max(0, point.y + offsetY)),
|
||||
};
|
||||
}
|
||||
|
||||
export type FrameColorMagnifierPlacement = {
|
||||
left: number;
|
||||
top: number;
|
||||
horizontal: 'left' | 'right';
|
||||
vertical: 'top' | 'bottom';
|
||||
};
|
||||
|
||||
/**
|
||||
* 放大镜始终置于指针的对角可用区域：优先沿当前可用边，空间不足即翻转。
|
||||
* 这样即使用户停在预览边缘，放大镜也不会覆盖正在取样的像素。
|
||||
*/
|
||||
export function resolveFrameColorMagnifierPlacement({
|
||||
pointerLeft,
|
||||
pointerTop,
|
||||
previewWidth,
|
||||
previewHeight,
|
||||
magnifierSize,
|
||||
gap = 12,
|
||||
edge = 4,
|
||||
}: {
|
||||
pointerLeft: number;
|
||||
pointerTop: number;
|
||||
previewWidth: number;
|
||||
previewHeight: number;
|
||||
magnifierSize: number;
|
||||
gap?: number;
|
||||
edge?: number;
|
||||
}): FrameColorMagnifierPlacement | null {
|
||||
if (previewWidth <= 0 || previewHeight <= 0 || magnifierSize <= 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const roomRight = previewWidth - pointerLeft - gap - edge;
|
||||
const roomLeft = pointerLeft - gap - edge;
|
||||
const horizontal =
|
||||
roomRight >= magnifierSize || roomRight >= roomLeft ? 'right' : 'left';
|
||||
const roomBottom = previewHeight - pointerTop - gap - edge;
|
||||
const roomTop = pointerTop - gap - edge;
|
||||
const vertical =
|
||||
roomBottom >= magnifierSize || roomBottom >= roomTop ? 'bottom' : 'top';
|
||||
const maxLeft = Math.max(edge, previewWidth - magnifierSize - edge);
|
||||
const maxTop = Math.max(edge, previewHeight - magnifierSize - edge);
|
||||
|
||||
return {
|
||||
horizontal,
|
||||
vertical,
|
||||
left: Math.min(
|
||||
maxLeft,
|
||||
Math.max(
|
||||
edge,
|
||||
horizontal === 'right'
|
||||
? pointerLeft + gap
|
||||
: pointerLeft - gap - magnifierSize,
|
||||
),
|
||||
),
|
||||
top: Math.min(
|
||||
maxTop,
|
||||
Math.max(
|
||||
edge,
|
||||
vertical === 'bottom'
|
||||
? pointerTop + gap
|
||||
: pointerTop - gap - magnifierSize,
|
||||
),
|
||||
),
|
||||
};
|
||||
}
|
||||
-67
@@ -1,67 +0,0 @@
|
||||
/* @vitest-environment jsdom */
|
||||
|
||||
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
import { useState } from 'react';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import type { EditorCharacterAnimationBackgroundRemovalMode } from '../../services/image-editor/editorProjectClient';
|
||||
import { ImageCanvasCharacterAnimationBackgroundRemovalDialog } from './ImageCanvasCharacterAnimationBackgroundRemovalDialog';
|
||||
|
||||
function DialogHarness({ onConfirm = vi.fn() }: { onConfirm?: () => void }) {
|
||||
const [mode, setMode] =
|
||||
useState<EditorCharacterAnimationBackgroundRemovalMode>('solid-color');
|
||||
const [screenColor, setScreenColor] = useState('#00FF00');
|
||||
|
||||
return (
|
||||
<ImageCanvasCharacterAnimationBackgroundRemovalDialog
|
||||
open
|
||||
mode={mode}
|
||||
screenColor={screenColor}
|
||||
onModeChange={setMode}
|
||||
onScreenColorChange={setScreenColor}
|
||||
onClose={vi.fn()}
|
||||
onConfirm={onConfirm}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
delete (window as typeof window & { EyeDropper?: unknown }).EyeDropper;
|
||||
});
|
||||
|
||||
describe('ImageCanvasCharacterAnimationBackgroundRemovalDialog', () => {
|
||||
it('requires an explicit pure-color or general mode selection', () => {
|
||||
const onConfirm = vi.fn();
|
||||
render(<DialogHarness onConfirm={onConfirm} />);
|
||||
|
||||
expect(
|
||||
(screen.getByRole('radio', { name: /纯色背景抠图/ }) as HTMLInputElement)
|
||||
.checked,
|
||||
).toBe(true);
|
||||
expect(
|
||||
(screen.getByLabelText('纯色背景颜色') as HTMLInputElement).value,
|
||||
).toBe('#00ff00');
|
||||
|
||||
fireEvent.click(screen.getByRole('radio', { name: /通用智能抠图/ }));
|
||||
expect(screen.queryByLabelText('纯色背景颜色')).toBeNull();
|
||||
fireEvent.click(screen.getByRole('button', { name: '开始处理' }));
|
||||
expect(onConfirm).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('uses the browser eyedropper result as the solid background color', async () => {
|
||||
const open = vi.fn().mockResolvedValue({ sRGBHex: '#336699' });
|
||||
(window as typeof window & { EyeDropper?: unknown }).EyeDropper = class {
|
||||
open = open;
|
||||
};
|
||||
render(<DialogHarness />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '吸取画布颜色' }));
|
||||
|
||||
await waitFor(() =>
|
||||
expect(
|
||||
(screen.getByLabelText('纯色背景颜色') as HTMLInputElement).value,
|
||||
).toBe('#336699'),
|
||||
);
|
||||
expect(open).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
@@ -1,137 +0,0 @@
|
||||
import { useCallback, useState } from 'react';
|
||||
|
||||
import type { EditorCharacterAnimationBackgroundRemovalMode } from '../../services/image-editor/editorProjectClient';
|
||||
import { PlatformActionButton } from '../common/PlatformActionButton';
|
||||
import { UnifiedConfirmDialog } from '../common/UnifiedConfirmDialog';
|
||||
|
||||
type EyeDropperConstructor = new () => {
|
||||
open: () => Promise<{ sRGBHex: string }>;
|
||||
};
|
||||
|
||||
type ImageCanvasCharacterAnimationBackgroundRemovalDialogProps = {
|
||||
open: boolean;
|
||||
mode: EditorCharacterAnimationBackgroundRemovalMode;
|
||||
screenColor: string;
|
||||
busy?: boolean;
|
||||
onModeChange: (mode: EditorCharacterAnimationBackgroundRemovalMode) => void;
|
||||
onScreenColorChange: (screenColor: string) => void;
|
||||
onClose: () => void;
|
||||
onConfirm: () => void;
|
||||
};
|
||||
|
||||
function resolveEyeDropper() {
|
||||
return (window as typeof window & { EyeDropper?: EyeDropperConstructor })
|
||||
.EyeDropper;
|
||||
}
|
||||
|
||||
export function ImageCanvasCharacterAnimationBackgroundRemovalDialog({
|
||||
open,
|
||||
mode,
|
||||
screenColor,
|
||||
busy = false,
|
||||
onModeChange,
|
||||
onScreenColorChange,
|
||||
onClose,
|
||||
onConfirm,
|
||||
}: ImageCanvasCharacterAnimationBackgroundRemovalDialogProps) {
|
||||
const [isPickingColor, setIsPickingColor] = useState(false);
|
||||
const pickScreenColor = useCallback(async () => {
|
||||
const EyeDropper = resolveEyeDropper();
|
||||
if (!EyeDropper || isPickingColor) {
|
||||
return;
|
||||
}
|
||||
setIsPickingColor(true);
|
||||
try {
|
||||
const result = await new EyeDropper().open();
|
||||
if (/^#[0-9a-f]{6}$/i.test(result.sRGBHex)) {
|
||||
onScreenColorChange(result.sRGBHex.toUpperCase());
|
||||
}
|
||||
} catch {
|
||||
// 用户取消系统吸管不属于错误，保持当前颜色即可。
|
||||
} finally {
|
||||
setIsPickingColor(false);
|
||||
}
|
||||
}, [isPickingColor, onScreenColorChange]);
|
||||
|
||||
return (
|
||||
<UnifiedConfirmDialog
|
||||
open={open}
|
||||
title="序列帧去背景"
|
||||
onClose={onClose}
|
||||
onConfirm={onConfirm}
|
||||
confirmLabel="开始处理"
|
||||
busy={busy}
|
||||
busyConfirmLabel="提交中"
|
||||
confirmDisabled={
|
||||
mode === 'solid-color' && !/^#[0-9A-F]{6}$/i.test(screenColor)
|
||||
}
|
||||
showCancel
|
||||
size="md"
|
||||
panelClassName="platform-remap-surface"
|
||||
>
|
||||
<fieldset className="grid gap-3" aria-label="去背景方式">
|
||||
<label className="flex cursor-pointer gap-3 rounded-xl border border-[var(--platform-border-soft)] p-3">
|
||||
<input
|
||||
type="radio"
|
||||
name="character-animation-background-removal-mode"
|
||||
value="solid-color"
|
||||
checked={mode === 'solid-color'}
|
||||
disabled={busy}
|
||||
onChange={() => onModeChange('solid-color')}
|
||||
/>
|
||||
<span>
|
||||
<span className="block font-medium">纯色背景抠图</span>
|
||||
<span className="block text-xs text-[var(--platform-text-muted)]">
|
||||
使用本地键色算法
|
||||
</span>
|
||||
</span>
|
||||
</label>
|
||||
<label className="flex cursor-pointer gap-3 rounded-xl border border-[var(--platform-border-soft)] p-3">
|
||||
<input
|
||||
type="radio"
|
||||
name="character-animation-background-removal-mode"
|
||||
value="general"
|
||||
checked={mode === 'general'}
|
||||
disabled={busy}
|
||||
onChange={() => onModeChange('general')}
|
||||
/>
|
||||
<span>
|
||||
<span className="block font-medium">通用智能抠图</span>
|
||||
<span className="block text-xs text-[var(--platform-text-muted)]">
|
||||
逐帧调用智能分割服务
|
||||
</span>
|
||||
</span>
|
||||
</label>
|
||||
</fieldset>
|
||||
|
||||
{mode === 'solid-color' ? (
|
||||
<div className="mt-4 flex flex-wrap items-center gap-3">
|
||||
<label className="flex items-center gap-2 text-sm font-medium">
|
||||
背景色
|
||||
<input
|
||||
type="color"
|
||||
aria-label="纯色背景颜色"
|
||||
value={screenColor}
|
||||
disabled={busy}
|
||||
onChange={(event) =>
|
||||
onScreenColorChange(event.currentTarget.value.toUpperCase())
|
||||
}
|
||||
className="h-9 w-12 cursor-pointer rounded border border-[var(--platform-border-soft)] bg-transparent p-0.5"
|
||||
/>
|
||||
</label>
|
||||
<code className="text-xs">{screenColor}</code>
|
||||
<PlatformActionButton
|
||||
type="button"
|
||||
tone="secondary"
|
||||
size="xs"
|
||||
shape="pill"
|
||||
disabled={busy || isPickingColor || !resolveEyeDropper()}
|
||||
onClick={() => void pickScreenColor()}
|
||||
>
|
||||
{isPickingColor ? '取色中' : '吸取画布颜色'}
|
||||
</PlatformActionButton>
|
||||
</div>
|
||||
) : null}
|
||||
</UnifiedConfirmDialog>
|
||||
);
|
||||
}
|
||||
+191
@@ -0,0 +1,191 @@
|
||||
/* @vitest-environment jsdom */
|
||||
|
||||
import { fireEvent, render, screen } from '@testing-library/react';
|
||||
import { useState } from 'react';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import type {
|
||||
EditorCharacterAnimationBackgroundRemovalMode,
|
||||
EditorImageSequenceFrameResult,
|
||||
} from '../../services/image-editor/editorProjectClient';
|
||||
import {
|
||||
formatFrameCanvasColorHex,
|
||||
resolveFrameCanvasSamplePoint,
|
||||
resolveFrameColorMagnifierPlacement,
|
||||
resolveFrameColorMagnifierSourcePoint,
|
||||
} from './ImageCanvasCharacterAnimationBackgroundRemovalColorPickerModel';
|
||||
import { ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel } from './ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel';
|
||||
|
||||
const frames: EditorImageSequenceFrameResult[] = [
|
||||
{
|
||||
imageSrc: '/generated-images/editor/frame-001.png',
|
||||
objectKey: 'generated-images/editor/frame-001.png',
|
||||
width: 320,
|
||||
height: 240,
|
||||
},
|
||||
{
|
||||
imageSrc: '/generated-images/editor/frame-002.png',
|
||||
objectKey: 'generated-images/editor/frame-002.png',
|
||||
width: 320,
|
||||
height: 240,
|
||||
},
|
||||
];
|
||||
|
||||
function FloatingPanelHarness({
|
||||
onConfirm = vi.fn(),
|
||||
}: {
|
||||
onConfirm?: () => void;
|
||||
}) {
|
||||
const [mode, setMode] =
|
||||
useState<EditorCharacterAnimationBackgroundRemovalMode>('solid-color');
|
||||
const [screenColor, setScreenColor] = useState('#00FF00');
|
||||
|
||||
return (
|
||||
<ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel
|
||||
open
|
||||
style={{ left: 352, top: 96 }}
|
||||
frames={frames}
|
||||
mode={mode}
|
||||
screenColor={screenColor}
|
||||
onModeChange={setMode}
|
||||
onScreenColorChange={setScreenColor}
|
||||
onClose={vi.fn()}
|
||||
onConfirm={onConfirm}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
describe('ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel', () => {
|
||||
it('requires an explicit pure-color or general mode selection', () => {
|
||||
const onConfirm = vi.fn();
|
||||
render(<FloatingPanelHarness onConfirm={onConfirm} />);
|
||||
|
||||
const panel = screen.getByRole('dialog', { name: '序列帧去背景' });
|
||||
expect(panel.getAttribute('aria-modal')).toBe('false');
|
||||
expect(panel.className).toContain('absolute');
|
||||
expect(panel.className).toContain('bg-white');
|
||||
expect(panel.style.left).toBe('352px');
|
||||
expect(panel.style.top).toBe('96px');
|
||||
|
||||
expect(
|
||||
(screen.getByRole('radio', { name: /纯色背景抠图/ }) as HTMLInputElement)
|
||||
.checked,
|
||||
).toBe(true);
|
||||
expect(screen.getByLabelText('序列帧像素取色预览')).toBeTruthy();
|
||||
expect(screen.getByText('悬停预览，点击取色')).toBeTruthy();
|
||||
expect(screen.getByLabelText('选择序列帧')).toBeTruthy();
|
||||
|
||||
fireEvent.click(screen.getByRole('radio', { name: /通用智能抠图/ }));
|
||||
expect(screen.queryByLabelText('序列帧像素取色预览')).toBeNull();
|
||||
fireEvent.click(screen.getByRole('button', { name: '开始处理' }));
|
||||
expect(onConfirm).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('maps a rendered preview click back to the original frame pixel', () => {
|
||||
expect(
|
||||
resolveFrameCanvasSamplePoint({
|
||||
clientX: 110,
|
||||
clientY: 70,
|
||||
rect: { left: 10, top: 20, width: 200, height: 100 },
|
||||
imageWidth: 800,
|
||||
imageHeight: 400,
|
||||
}),
|
||||
).toEqual({ x: 400, y: 200 });
|
||||
|
||||
expect(
|
||||
resolveFrameCanvasSamplePoint({
|
||||
clientX: -10,
|
||||
clientY: 300,
|
||||
rect: { left: 10, top: 20, width: 200, height: 100 },
|
||||
imageWidth: 800,
|
||||
imageHeight: 400,
|
||||
}),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects a pointer in preview letterboxing instead of sampling a shifted pixel', () => {
|
||||
expect(
|
||||
resolveFrameCanvasSamplePoint({
|
||||
clientX: 59,
|
||||
clientY: 20,
|
||||
rect: { left: 60, top: 20, width: 120, height: 160 },
|
||||
imageWidth: 1200,
|
||||
imageHeight: 1600,
|
||||
}),
|
||||
).toBeNull();
|
||||
|
||||
expect(
|
||||
resolveFrameCanvasSamplePoint({
|
||||
clientX: 179.9,
|
||||
clientY: 179.9,
|
||||
rect: { left: 60, top: 20, width: 120, height: 160 },
|
||||
imageWidth: 1200,
|
||||
imageHeight: 1600,
|
||||
}),
|
||||
).toEqual({ x: 1199, y: 1599 });
|
||||
});
|
||||
|
||||
it('flips the magnifier away from the pointer when its preferred side is full', () => {
|
||||
expect(
|
||||
resolveFrameColorMagnifierPlacement({
|
||||
pointerLeft: 16,
|
||||
pointerTop: 18,
|
||||
previewWidth: 264,
|
||||
previewHeight: 176,
|
||||
magnifierSize: 88,
|
||||
}),
|
||||
).toMatchObject({
|
||||
horizontal: 'right',
|
||||
vertical: 'bottom',
|
||||
left: 28,
|
||||
top: 30,
|
||||
});
|
||||
expect(
|
||||
resolveFrameColorMagnifierPlacement({
|
||||
pointerLeft: 252,
|
||||
pointerTop: 164,
|
||||
previewWidth: 264,
|
||||
previewHeight: 176,
|
||||
magnifierSize: 88,
|
||||
}),
|
||||
).toMatchObject({
|
||||
horizontal: 'left',
|
||||
vertical: 'top',
|
||||
left: 152,
|
||||
top: 64,
|
||||
});
|
||||
expect(
|
||||
resolveFrameColorMagnifierPlacement({
|
||||
pointerLeft: 250,
|
||||
pointerTop: 18,
|
||||
previewWidth: 264,
|
||||
previewHeight: 176,
|
||||
magnifierSize: 88,
|
||||
}),
|
||||
).toMatchObject({ horizontal: 'left', vertical: 'bottom' });
|
||||
});
|
||||
|
||||
it('keeps the magnifier sample clamped to the original frame and formats hover color', () => {
|
||||
expect(
|
||||
resolveFrameColorMagnifierSourcePoint({
|
||||
point: { x: 0, y: 0 },
|
||||
offsetX: -5,
|
||||
offsetY: -5,
|
||||
imageWidth: 320,
|
||||
imageHeight: 240,
|
||||
}),
|
||||
).toEqual({ x: 0, y: 0 });
|
||||
expect(
|
||||
resolveFrameColorMagnifierSourcePoint({
|
||||
point: { x: 319, y: 239 },
|
||||
offsetX: 5,
|
||||
offsetY: 5,
|
||||
imageWidth: 320,
|
||||
imageHeight: 240,
|
||||
}),
|
||||
).toEqual({ x: 319, y: 239 });
|
||||
expect(formatFrameCanvasColorHex({ red: 4, green: 201, blue: 255 })).toBe(
|
||||
'#04C9FF',
|
||||
);
|
||||
});
|
||||
});
|
||||
+517
File diff suppressed because it is too large
Load Diff
@@ -35,7 +35,7 @@ import {
|
||||
import { usePlatformProfileCenterController } from '../platform-entry/usePlatformProfileCenterController';
|
||||
import type { ImageCanvasActionResult } from './ImageCanvasActionsContext';
|
||||
import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider';
|
||||
import { ImageCanvasCharacterAnimationBackgroundRemovalDialog } from './ImageCanvasCharacterAnimationBackgroundRemovalDialog';
|
||||
import { ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel } from './ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel';
|
||||
import {
|
||||
canvasAssetKindOrNull,
|
||||
DEFAULT_CANVAS_BACKGROUND_COLOR,
|
||||
@@ -79,6 +79,7 @@ import { fitViewportToBounds } from './ImageCanvasInteractionModel';
|
||||
import type { CanvasLayerCopyBlockReason } from './ImageCanvasLayerCommandModel';
|
||||
import {
|
||||
isCanvasGenerationComposerVisible,
|
||||
resolveCharacterAnimationBackgroundRemovalPanelStyle,
|
||||
resolveQuickEditFocusViewport,
|
||||
} from './ImageCanvasOverlayModel';
|
||||
import {
|
||||
@@ -2684,13 +2685,13 @@ export function ImageCanvasEditorView({
|
||||
);
|
||||
},
|
||||
onRemoveCharacterAnimationBackground: (layer: CanvasLayer) => {
|
||||
void flushProjectPersistence().then(() =>
|
||||
void flushProjectPersistence().then(() => {
|
||||
setCharacterAnimationBackgroundRemovalDraft({
|
||||
sourceLayer: layer,
|
||||
mode: 'solid-color',
|
||||
screenColor: '#00FF00',
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
},
|
||||
onExtractUiDesignAssets: extractUiDesignAssets,
|
||||
onUiAssetExtractionToolChange: changeUiAssetExtractionTool,
|
||||
@@ -2738,7 +2739,62 @@ export function ImageCanvasEditorView({
|
||||
onSwitchTool: switchTool,
|
||||
onOpenToolOptions: generationSurface.openToolbarOptionMenu,
|
||||
onCloseToolOptions: generationSurface.closeToolbarOptionMenu,
|
||||
children: generationComposerNode,
|
||||
children: (
|
||||
<>
|
||||
{generationComposerNode}
|
||||
{characterAnimationBackgroundRemovalDraft ? (
|
||||
<ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel
|
||||
open
|
||||
style={
|
||||
resolveCharacterAnimationBackgroundRemovalPanelStyle({
|
||||
sourceLayer:
|
||||
characterAnimationBackgroundRemovalDraft.sourceLayer,
|
||||
viewport,
|
||||
canvasSize,
|
||||
}) ?? {}
|
||||
}
|
||||
frames={
|
||||
characterAnimationBackgroundRemovalDraft.sourceLayer
|
||||
.imageSequenceFrames ?? []
|
||||
}
|
||||
mode={characterAnimationBackgroundRemovalDraft.mode}
|
||||
screenColor={characterAnimationBackgroundRemovalDraft.screenColor}
|
||||
busy={removingBackgroundCharacterAnimationLayerIds.has(
|
||||
characterAnimationBackgroundRemovalDraft.sourceLayer.id,
|
||||
)}
|
||||
onModeChange={(mode) =>
|
||||
setCharacterAnimationBackgroundRemovalDraft((draft) =>
|
||||
draft ? { ...draft, mode } : draft,
|
||||
)
|
||||
}
|
||||
onScreenColorChange={(screenColor) =>
|
||||
setCharacterAnimationBackgroundRemovalDraft((draft) =>
|
||||
draft ? { ...draft, screenColor } : draft,
|
||||
)
|
||||
}
|
||||
onClose={() => setCharacterAnimationBackgroundRemovalDraft(null)}
|
||||
onConfirm={() => {
|
||||
const draft = characterAnimationBackgroundRemovalDraft;
|
||||
void removeSelectedCharacterAnimationBackground(
|
||||
draft.sourceLayer,
|
||||
{
|
||||
mode: draft.mode,
|
||||
...(draft.mode === 'solid-color'
|
||||
? { screenColor: draft.screenColor }
|
||||
: {}),
|
||||
onSubmissionAccepted: () =>
|
||||
setCharacterAnimationBackgroundRemovalDraft((current) =>
|
||||
current?.sourceLayer.id === draft.sourceLayer.id
|
||||
? null
|
||||
: current,
|
||||
),
|
||||
},
|
||||
);
|
||||
}}
|
||||
/>
|
||||
) : null}
|
||||
</>
|
||||
),
|
||||
};
|
||||
|
||||
return (
|
||||
@@ -2760,42 +2816,6 @@ export function ImageCanvasEditorView({
|
||||
onClose: () => setMetadataLayer(null),
|
||||
}}
|
||||
/>
|
||||
<ImageCanvasCharacterAnimationBackgroundRemovalDialog
|
||||
open={Boolean(characterAnimationBackgroundRemovalDraft)}
|
||||
mode={characterAnimationBackgroundRemovalDraft?.mode ?? 'solid-color'}
|
||||
screenColor={
|
||||
characterAnimationBackgroundRemovalDraft?.screenColor ?? '#00FF00'
|
||||
}
|
||||
busy={Boolean(
|
||||
characterAnimationBackgroundRemovalDraft &&
|
||||
removingBackgroundCharacterAnimationLayerIds.has(
|
||||
characterAnimationBackgroundRemovalDraft.sourceLayer.id,
|
||||
),
|
||||
)}
|
||||
onModeChange={(mode) =>
|
||||
setCharacterAnimationBackgroundRemovalDraft((draft) =>
|
||||
draft ? { ...draft, mode } : draft,
|
||||
)
|
||||
}
|
||||
onScreenColorChange={(screenColor) =>
|
||||
setCharacterAnimationBackgroundRemovalDraft((draft) =>
|
||||
draft ? { ...draft, screenColor } : draft,
|
||||
)
|
||||
}
|
||||
onClose={() => setCharacterAnimationBackgroundRemovalDraft(null)}
|
||||
onConfirm={() => {
|
||||
const draft = characterAnimationBackgroundRemovalDraft;
|
||||
if (!draft) {
|
||||
return;
|
||||
}
|
||||
void removeSelectedCharacterAnimationBackground(draft.sourceLayer, {
|
||||
mode: draft.mode,
|
||||
...(draft.mode === 'solid-color'
|
||||
? { screenColor: draft.screenColor }
|
||||
: {}),
|
||||
}).then(() => setCharacterAnimationBackgroundRemovalDraft(null));
|
||||
}}
|
||||
/>
|
||||
<PlatformDangerConfirmDialog
|
||||
open={Boolean(pendingGenerationDeleteDialog)}
|
||||
title="删除生成中的占位图"
|
||||
|
||||
@@ -9,6 +9,7 @@ import type {
|
||||
} from './ImageCanvasEditorTypes';
|
||||
import {
|
||||
isCanvasGenerationComposerVisible,
|
||||
resolveCharacterAnimationBackgroundRemovalPanelStyle,
|
||||
resolveCharacterAnimationPanelStyle,
|
||||
resolveCropExpandPanelStyle,
|
||||
resolveGenerationAnchor,
|
||||
@@ -132,6 +133,34 @@ describe('ImageCanvasOverlayModel', () => {
|
||||
).toEqual({ left: 1320, top: 88 });
|
||||
});
|
||||
|
||||
it('places background removal beside the source and flips when the right side is full', () => {
|
||||
expect(
|
||||
resolveCharacterAnimationBackgroundRemovalPanelStyle({
|
||||
sourceLayer: createLayer({ x: 100, y: 80, width: 240, height: 120 }),
|
||||
viewport: { x: 0, y: 0, scale: 1 },
|
||||
canvasSize: { width: 900, height: 640 },
|
||||
}),
|
||||
).toEqual({ left: 352, top: 80 });
|
||||
|
||||
expect(
|
||||
resolveCharacterAnimationBackgroundRemovalPanelStyle({
|
||||
sourceLayer: createLayer({ x: 620, y: 80, width: 240, height: 120 }),
|
||||
viewport: { x: 0, y: 0, scale: 1 },
|
||||
canvasSize: { width: 900, height: 640 },
|
||||
}),
|
||||
).toEqual({ left: 320, top: 80 });
|
||||
});
|
||||
|
||||
it('keeps background removal adjacent while the canvas pans and zooms', () => {
|
||||
expect(
|
||||
resolveCharacterAnimationBackgroundRemovalPanelStyle({
|
||||
sourceLayer: createLayer({ x: 40, y: 20, width: 120, height: 80 }),
|
||||
viewport: { x: 30, y: 50, scale: 2 },
|
||||
canvasSize: { width: 900, height: 640 },
|
||||
}),
|
||||
).toEqual({ left: 362, top: 90 });
|
||||
});
|
||||
|
||||
it('fits tall quick edit sources above the reserved bottom composer area', () => {
|
||||
const sourceLayer = createLayer({ x: 100, y: 50, width: 360, height: 1600 });
|
||||
const viewport = resolveQuickEditFocusViewport({
|
||||
|
||||
@@ -370,6 +370,7 @@ function GenerationWorkflowHarness({
|
||||
onFlushProjectPersistenceSnapshot,
|
||||
refreshAssetLibrary,
|
||||
upsertGeneratedAsset,
|
||||
onCharacterAnimationBackgroundRemovalSubmissionAccepted,
|
||||
}: {
|
||||
initialLayers?: CanvasLayer[];
|
||||
initialCanvasGenerationDialogs?: CanvasGenerationDialogState[];
|
||||
@@ -395,6 +396,7 @@ function GenerationWorkflowHarness({
|
||||
upsertGeneratedAsset?: Parameters<
|
||||
typeof useImageCanvasGenerationWorkflow
|
||||
>[0]['upsertGeneratedAsset'];
|
||||
onCharacterAnimationBackgroundRemovalSubmissionAccepted?: () => void;
|
||||
}) {
|
||||
const [layers, setLayers] = useState<CanvasLayer[]>(initialLayers);
|
||||
const [viewport, setViewport] = useState(initialViewport);
|
||||
@@ -950,6 +952,8 @@ function GenerationWorkflowHarness({
|
||||
void workflow.removeSelectedCharacterAnimationBackground(layers[0]!, {
|
||||
mode: 'solid-color',
|
||||
screenColor: '#00FF00',
|
||||
onSubmissionAccepted:
|
||||
onCharacterAnimationBackgroundRemovalSubmissionAccepted,
|
||||
})
|
||||
}
|
||||
>
|
||||
@@ -3044,6 +3048,7 @@ describe('useImageCanvasGenerationWorkflow', () => {
|
||||
}>();
|
||||
const applyProjectSnapshot = vi.fn();
|
||||
const refreshAssetLibrary = vi.fn().mockResolvedValue(undefined);
|
||||
const onSubmissionAccepted = vi.fn();
|
||||
const backendProject: EditorProjectSnapshot = {
|
||||
projectId: 'project-1',
|
||||
title: '序列帧去背景项目',
|
||||
@@ -3065,6 +3070,9 @@ describe('useImageCanvasGenerationWorkflow', () => {
|
||||
projectId="project-1"
|
||||
applyProjectSnapshot={applyProjectSnapshot}
|
||||
refreshAssetLibrary={refreshAssetLibrary}
|
||||
onCharacterAnimationBackgroundRemovalSubmissionAccepted={
|
||||
onSubmissionAccepted
|
||||
}
|
||||
initialLayers={[
|
||||
createLayer({
|
||||
title: '勇者奔跑',
|
||||
@@ -3095,6 +3103,7 @@ describe('useImageCanvasGenerationWorkflow', () => {
|
||||
screenColor: '#00FF00',
|
||||
}),
|
||||
);
|
||||
expect(onSubmissionAccepted).not.toHaveBeenCalled();
|
||||
|
||||
deferred.resolve({
|
||||
taskId: 'background-removal-task',
|
||||
@@ -3110,6 +3119,9 @@ describe('useImageCanvasGenerationWorkflow', () => {
|
||||
project: backendProject,
|
||||
});
|
||||
|
||||
await waitFor(() => {
|
||||
expect(onSubmissionAccepted).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
await waitFor(() => {
|
||||
expect(applyProjectSnapshot).toHaveBeenCalledWith(backendProject, {
|
||||
type: 'remove-background',
|
||||
|
||||
@@ -3682,6 +3682,7 @@ export function useImageCanvasGenerationWorkflow({
|
||||
options: {
|
||||
mode: EditorCharacterAnimationBackgroundRemovalMode;
|
||||
screenColor?: string;
|
||||
onSubmissionAccepted?: () => void;
|
||||
},
|
||||
) => {
|
||||
if (
|
||||
@@ -3743,6 +3744,9 @@ export function useImageCanvasGenerationWorkflow({
|
||||
sourceLayer,
|
||||
),
|
||||
});
|
||||
// 选择悬浮窗只负责提交参数；HTTP 接受任务后立即交给画布占位和任务队列，
|
||||
// 不能等待 durable job 终态，否则整个处理周期都会被误显示为“提交中”。
|
||||
options.onSubmissionAccepted?.();
|
||||
const queued = await applyQueuedEditorGenerationProject(
|
||||
result,
|
||||
projectId,
|
||||
|
||||
Reference in New Issue
Block a user