diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index ec6a14157..f28c4b6b3 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -24,7 +24,6 @@ - 验证方式：覆盖 Codex 分类与敏感诱饵、失败事件公共摘要、最近任务与各正式卡片、game-chat 阶段记录、待核对状态、final-reply fallback 白名单及 malformed 响应完整重试；运行 Rust 定向测试、前端模型/AppSurface 定向测试、Shell typecheck、编码和 diff 门禁。 - 关联文档：`docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`。 - ## 2026-08-10 资源管理评审阻塞项按第二轮正式合同修复 - 背景：资源管理第一轮实现后，人工验证继续暴露 WebView 默认缩放、预览队列饥饿、过滤后媒体残留播放、外层滚动串 scope、超深依赖坐标越过 Rust 上限和暂时错误无法重试等问题。部分 PRD / 技术方案仍描述第一轮的中央媒体预览、单全局 Overlay 和统一 section scope，已经与第二轮代码及验收结论冲突。 @@ -68,6 +67,7 @@ - 替代关系：本条替代下方 2026-08-07 阶段二中“缩放只等于可视高度”和“只为完整可见卡片建立端点”的显示口径；其中曾采用的全局 SVG、四 viewport 联合 clip 和单全局 Observer 又由上方“依赖 SVG 改为分区 plane 所有”决定替代。高度模型、四分区、会话隔离、内外滚动和无布局 CAS 等其它决定继续有效。阶段一媒体卡、阶段三确定性聚类、Rust `dependencyDepths`、producer 截断降级、历史手动坐标与 type sidecar 均不变。 - 验证方式：纯倍率模型覆盖按钮 / wheel 边界；AppSurface 覆盖项目、mode、section 隔离、普通 wheel、Ctrl wheel、WebKit gesture 与零布局写入；SVG 覆盖同 plane 倍率、橙色 marker、直线 / 小圆角横纵路由、自环间隙、双向边界继续线、分区原生裁剪、每区单 Observer、task-flow 零渲染和 4096 精确关系有界输出。 - 关联文档：`docs/prd/【AI游戏创作】项目开发工作台PRD-2026-07-20.md`、`docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`。 + ## 2026-08-11 Tauri 无限画布以可证明事务和分类恢复收口 - 终态与队列：资源编辑账本正式区分可继续阶段、`reconciliation-required`、`remote-failed` 和 `archived`。远端明确失败只保存稳定分类与终态时间，不得再 POST、轮询或重新扣费；只有该终态能由用户显式归档并移出活动恢复队列，归档保留账本且不伪装 `committed`。结果未知和需对账项继续失败关闭。 @@ -7192,7 +7192,6 @@ - 决策：只有同一可信根 Supervisor 能调用 `agent.acceptance_update`，且该控制面动作必须独占一轮，不能夹带 `plan_update`、legacy plan 或回复。requiredEvidence 采用 `tool:`，合同冻结前必须命中 Runtime 允许的持久证据工具集合并拒绝拼错、控制面和纯协调工具；动态 MCP catalog 不得冻结为不可变 requiredEvidence。passed 节点必须引用当前根任务树中对应工具的真实成功动作回执；回执同时记录动作执行边界的 `projectRevisionBefore / projectRevisionAfter`，非 mutation evidence 必须满足 before、after 与验收时 current revision 完全相同，mutation evidence 的 after 必须等于 current revision，旧 revision 或执行后延迟落账的回执不可重放。failed、not-observed、缺失节点以及落后当前 project revision 的整图确认状态均阻断普通完成、finalization 与恢复。项目 revision 变化后由 Supervisor 只更新受影响节点并确认当前图，未提交的 passed 节点保持不变；Runtime 不替 Supervisor 推断影响范围、选择具体 Agent 或实现方式。 - 决策：不可变 Goal Contract 的根 Run 收到 steer 时，必须按旧 rootRunId 串行化整个转换，并在持锁后重新确认该旧根仍是 Session 当前权威 Run，避免不同 steerId 并发创建多个 replacement。随后为绑定旧 rootRunId 的全部非终态静态、ready 和 isolated 后代写入取消栅栏、打断 Provider 并逐个收束，再终止旧根；所有项目修改入口在看到取消栅栏后立即失败关闭。Runtime 必须确认旧树所有成员都已进入终态或 `needs-reconciliation`，超时则保持等待并拒绝启动 replacement；只有旧树停稳后，才在同一 Session、source 和 Run Profile 启动新根 Run 重新理解完整目标。 - ## 2026-08-08 External v1 图片编辑来源字段允许原地收紧 - 背景：图片编辑主来源已经从可由客户端提交 objectKey 和类型提示的 `sourceImageSrc / sourceResourceId / assetKind`，收紧为服务端按项目资源 ID 或素材 ID 解析权威对象与类型的必填 `sourceReferenceId`。这会让严格 External v1 客户端立即失败，属于现役版本策略明确列出的 breaking change；2026-07-31 的历史豁免不能自动覆盖本次变更。 @@ -8661,6 +8660,7 @@ - 2026-06-19 桌面壳外链打开 helper 共用：Tauri WebView 外域拦截和 HostBridge `app.openExternalUrl` 都必须复用 `open_normalized_desktop_external_url` 执行系统外链打开动作；HostBridge 分支仍先用 `normalize_external_url` 保留 payload 错误语义并把 opener 错误回传给 H5，WebView 拦截保持 best-effort 静默处理。桌面壳配置检查会拒绝 `dispatch.rs` 直接调用 `app.opener().open_url` 绕过该 helper，避免两条离壳路径漂移。 > 2026-07-18 覆盖说明：本段后续关于微信 `navigation.openNativePage`、生成结果订阅页、`[subscribe-message]` 日志和订阅页路由门禁的 2026-06 决策均已由旧创作模板退役决策废止，只作为历史记录。Expo / Tauri 的同源 H5 受控导航及微信登录、支付、分享能力继续有效。 + - 2026-06-20 H5 原生导航预校验：`navigateHostNativePage()` 在 `native_app` 下发送 `navigation.openNativePage` 前必须先拒绝空值、控制字符、协议相对 URL、外域绝对 URL 和非 `http:` / `https:` 协议目标；同源绝对 URL、`/path` 和保留给桌面壳兼容的相对 route 继续交给 Expo / Tauri 壳二次归一并补写宿主上下文。微信小程序分支仍按小程序页面 URL 语义走 `wx.miniProgram.navigateTo`，不套原生 App 同源 H5 预校验。根级 `npm run check:native-shells` 会反查 H5 facade 仍使用 `normalizeNativeAppPageUrl(...)` 且发送归一后的 URL，避免明显不安全目标触达原生壳。 - 2026-06-20 微信受控原生页能力声明：微信小程序壳真实 capability profile 声明 `navigation.openNativePage`，用于承接已经登记并测试的小程序原生页 flow；当前订阅生成结果通知页通过 H5 `requestGenerationResultSubscribePermission()` 调用 `navigateHostNativePage()` 打开 `/pages/subscribe-message/index`，小程序页再调用真实 `wx.requestSubscribeMessage` 并按既有结果协议回灌。根级 `npm run check:native-shells` 必须把该能力反查到共享 profile、微信 `WECHAT_HOST_CAPABILITIES` 镜像、订阅页协议常量、H5 入口、小程序 host-bridge / shell / page 文件和相关测试；该能力不代表开放任意小程序页面跳转。 - 2026-06-18 能力声明收紧：`packages/shared/src/contracts/hostBridge.ts` 提供 HostBridge method / capability 白名单，H5 的 `getHostRuntime()` 会解析并过滤 `hostCapabilities`；`openHostShare`、`writeHostClipboardText`、`requestHostHapticsImpact`、`setHostAppTitle`、`exportHostTextFile` 等 native 能力只在宿主声明对应 capability 后调用。发布分享弹窗只有声明 `share.open` 时才显示受控分享动作，并按 `hostShell` 区分 Expo 系统分享面板和 Tauri 剪贴板复制表达，避免旧壳或裁剪壳露出不可用入口。 @@ -14075,6 +14075,7 @@ - 分类：内部稳定 reason code 固定为 `translation_invalid / translation_upstream_failed / translation_budget_exhausted / elevenlabs_http_failed / invalid_audio / duration_probe_failed / oss_failed / writeback_failed`。MIME、空 body 和大小归 `invalid_audio`；MP3 识别、帧读取和时长门禁归 `duration_probe_failed`。普通用户继续只读稳定短文案，不暴露 endpoint、上游正文或凭据。 - 跨入口：画布 Agent `generate-sound-effect` 与站内 / External v1 共用 canonical Prompt、固定模型、nullable `0.5-30` 小数时长和 Loop；省略 duration 为手动 `5s`，显式 null 为自动。SFX 参数解析必须保留该 null，不能被通用 null-default 兼容层改写。最终仍进入相同 `editor_sound_effect_generation` queue payload，不新增 Agent 专属链路。 - 发布边界：T6 工程实施和 mock / loopback 门禁不等于真实 provider 或生产验收。发布前关闭 SFX 入队，使用显式 `--server` / `--server-url` 只读查询 `external_generation_job` 中 pending / running 的 `editor_sound_effect_generation`，清零后按 api-server / Worker → Web 顺序部署并灰度；禁止 `--root-dir`、删除任务伪造 drain 或自动回退 Vidu。本次没有 SpacetimeDB schema、migration 或 bindings 变更。 + ## 2026-08-06 编辑器生成结果使用 durable receipt 与统一原子提交 - 背景：图片、改图、去背景、图集 / UI 多产物、角色动作、视频、音效和背景音乐在 OSS 结果可用后，仍分段 confirm object、创建 project resource / account asset、保存 canvas 和 complete job。任一中间失败都会留下部分业务事实；只把 `external_generation_job` 当 operation journal 又无法覆盖无 job 的 inline，也无法独立证明某批 resource/asset/canvas 已作为一笔提交完成。 @@ -14189,6 +14190,6 @@ ## 2026-08-17 Spine 视频转换去背景改为显式双模式 - 背景：仅凭 `generationInputs.action = character-animation.convert` 无法证明源视频是绿幕；把任意转换结果固定按 `#00FF00` 键控会误处理普通背景或绿色主体。仓库已有 BgFilter `complex + birefnet` 通用分割服务，也已有可传 `screen_color` 的 flat 协议，但纯色场景不需要承担外部服务成本。 -- 决策：只有视频直接转换序列继续显示“去背景”，点击后必须在独立弹窗显式选择 `solid-color` 或 `general`。纯色模式默认 `#00FF00`，允许颜色输入及浏览器吸管，后端使用参数化本地键色算法；通用模式逐帧调用 BgFilter complex，不携带键色。HTTP 入队和 durable worker 共用白名单校验，后端在结果 `generationInputs` 中权威记录模式、颜色或 Provider/model，不能再把“视频转换”本身当作绿幕证明。两种模式仍生成新序列资源、整批失败关闭并保留源序列。 +- 决策：只有视频直接转换序列继续显示“去背景”，点击后必须在当前选中序列帧旁的非模态悬浮窗中显式选择 `solid-color` 或 `general`；悬浮窗随画布平移和缩放更新位置，优先置于右侧并在空间不足时翻转，不增加画布遮罩、不覆盖源画面，纯色模式不使用系统吸管；内置逐帧 viewer 从正式 objectKey 读取原始帧字节，按原始比例实际绘制，且仅在可见图像区域内采样。用户悬停时实时显示原始像素色样与 11×11 放大网格，点击才回填键色，以消除容器留白和 CSS 缩放导致的取色位移。纯色模式默认 `#00FF00`，允许颜色输入及逐帧 viewer 原始像素取样；后端使用参数化本地键色算法，对轻微压缩噪声和接近键色的平缓色差有容忍度，但不承诺处理明显渐变、阴影、纹理或与主体相近的背景，后者应选择 `general`。通用模式逐帧调用 BgFilter complex，不携带键色。悬浮窗的“提交中”只表示 HTTP 入队请求，服务端接受任务后立即关闭悬浮窗，由画布生成占位和任务列表承接 durable worker 的后续状态；HTTP 提交失败则保留悬浮窗并恢复可重试状态。HTTP 入队和 durable worker 共用白名单校验，后端在结果 `generationInputs` 中权威记录模式、颜色或 Provider/model，不能再把“视频转换”本身当作绿幕证明。两种模式仍生成新序列资源、整批失败关闭并保留源序列。 - 费用边界：本次沿用现有手动 BgFilter 的 0 泥点内部契约；供应商成本不等于已确定的用户泥点售价，在没有正式定价决策时不自行新增价格。若后续收费，必须接入统一后端定价、入队冻结价格和 worker 扣费退款链路。 - 验证：覆盖缺失/非法纯色、通用模式禁止颜色、任意键色透传、绿色主体在非绿色键色下保留、双模式 worker 路由、前端模式选择及吸管回填，并运行角色动作 Rust 定向测试、前端 Vitest、类型检查、编码和 diff 门禁。 diff --git a/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md b/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md index d42b6f366..b29f290c8 100644 --- a/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md +++ b/docs/technical/【技术方案】ArtAgent多模态Spine序列帧动画生成-2026-08-10.md @@ -107,21 +107,21 @@ manifest.txt 主要文件： -| 责任 | 当前文件 | -| --- | --- | -| 画布工具类型和状态 | `src/components/image-editor/ImageCanvasEditorTypes.ts` | -| 底部工具栏 | `src/components/image-editor/ImageCanvasBottomToolbarView.tsx` | -| 生成 Dialog 草稿和恢复 | `src/components/image-editor/ImageCanvasGenerationDialogModel.ts` | -| 生成 Composer | `src/components/image-editor/ImageCanvasGenerationComposerView.tsx` | -| 角色动作专用旧面板 | `src/components/image-editor/ImageCanvasCharacterAnimationPanelView.tsx` | -| 提交计划与前端校验 | `src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts` | -| 画布选取与改造恢复 | `src/components/image-editor/useImageCanvasGenerationWorkflow.ts` | -| 上传图片/视频 | `src/components/image-editor/useImageCanvasUploadWorkflow.ts` | -| 提交、队列和回填 | `src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts` | -| API Client 和前端 DTO | `src/services/image-editor/editorProjectClient.ts` | -| 序列帧播放 | `src/components/image-editor/ImageCanvasWorldView.tsx` | -| 元数据 | `src/components/image-editor/ImageCanvasMetadataModalView.tsx` | -| ZIP / Spine JSON | `ImageCanvasExportModel.ts`、`useImageCanvasAssetExportWorkflow.ts` | +| 责任 | 当前文件 | +| ---------------------- | --------------------------------------------------------------------------- | +| 画布工具类型和状态 | `src/components/image-editor/ImageCanvasEditorTypes.ts` | +| 底部工具栏 | `src/components/image-editor/ImageCanvasBottomToolbarView.tsx` | +| 生成 Dialog 草稿和恢复 | `src/components/image-editor/ImageCanvasGenerationDialogModel.ts` | +| 生成 Composer | `src/components/image-editor/ImageCanvasGenerationComposerView.tsx` | +| 角色动作专用旧面板 | `src/components/image-editor/ImageCanvasCharacterAnimationPanelView.tsx` | +| 提交计划与前端校验 | `src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts` | +| 画布选取与改造恢复 | `src/components/image-editor/useImageCanvasGenerationWorkflow.ts` | +| 上传图片/视频 | `src/components/image-editor/useImageCanvasUploadWorkflow.ts` | +| 提交、队列和回填 | `src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts` | +| API Client 和前端 DTO | `src/services/image-editor/editorProjectClient.ts` | +| 序列帧播放 | `src/components/image-editor/ImageCanvasWorldView.tsx` | +| 元数据 | `src/components/image-editor/ImageCanvasMetadataModalView.tsx` | +| ZIP / Spine JSON | `ImageCanvasExportModel.ts`、`useImageCanvasAssetExportWorkflow.ts` | 通用视频面板已经提供“参考”控件，支持： @@ -230,21 +230,21 @@ flowchart LR ## 5. 与本次需求的差距 -| 需求 | 当前能力 | 结论 | -| --- | --- | --- | -| 工具栏在视频后新增入口 | 当前已有角色图 `生成动画` 快捷入口 | 新增底部通用入口并复用同一 dialog | -| 纯文本生成 | 当前必须有角色源图 | 需要扩展后端 T2V | -| 图片 + 文本 | 已实现角色立绘图生视频再抽帧 | 主要复用 | -| 已有序列帧改造 | 仅有部分恢复/首帧能力，来源语义不完整 | 需要明确输入形态并补全 | -| 视频 + 文本 | 通用视频支持参考视频，角色动作链未接入 | 需要扩展 | -| 参考支持画布/上传图片视频 | 通用视频面板已经实现 | 复用并收窄规则 | -| 白/绿/蓝/紫背景 | 当前角色动作使用自动安全色，不是这四个纯色 | 需要专用枚举和提示词 | -| 动作“更多”逐行展开 | 当前只有固定 7 个预设 | 需要前端扩展 | -| 1080p | Fast 前后端均不支持 | 暂不开放 | -| 40 泥点/次 | 当前 480p × 4 秒恰好为 40；其他组合不同 | 需确认固定价还是按秒价 | -| 去背景 | AI 生成链自动透明化；直接转换保留背景 | 保留零泥点手动派生任务 | -| 拆帧生成单 PNG | 帧对象已有，但没有全部登记为独立账号素材 | 可复用对象、补资产绑定 | -| ZIP 下载 | 已完成且匹配附件 | 不重做 | +| 需求 | 当前能力 | 结论 | +| ------------------------- | ------------------------------------------ | --------------------------------- | +| 工具栏在视频后新增入口 | 当前已有角色图 `生成动画` 快捷入口 | 新增底部通用入口并复用同一 dialog | +| 纯文本生成 | 当前必须有角色源图 | 需要扩展后端 T2V | +| 图片 + 文本 | 已实现角色立绘图生视频再抽帧 | 主要复用 | +| 已有序列帧改造 | 仅有部分恢复/首帧能力，来源语义不完整 | 需要明确输入形态并补全 | +| 视频 + 文本 | 通用视频支持参考视频，角色动作链未接入 | 需要扩展 | +| 参考支持画布/上传图片视频 | 通用视频面板已经实现 | 复用并收窄规则 | +| 白/绿/蓝/紫背景 | 当前角色动作使用自动安全色，不是这四个纯色 | 需要专用枚举和提示词 | +| 动作“更多”逐行展开 | 当前只有固定 7 个预设 | 需要前端扩展 | +| 1080p | Fast 前后端均不支持 | 暂不开放 | +| 40 泥点/次 | 当前 480p × 4 秒恰好为 40；其他组合不同 | 需确认固定价还是按秒价 | +| 去背景 | AI 生成链自动透明化；直接转换保留背景 | 保留零泥点手动派生任务 | +| 拆帧生成单 PNG | 帧对象已有，但没有全部登记为独立账号素材 | 可复用对象、补资产绑定 | +| ZIP 下载 | 已完成且匹配附件 | 不重做 | ## 6. 目标业务模型 @@ -335,20 +335,20 @@ video = 一段视频作为动作/外观参考 三模态所需事实已经能放入现有字段： -| 事实 | 存放位置 | -| --- | --- | -| 输入模式 | `generation_inputs_json.fields[inputMode]` | -| 用户动作描述 | `generation_inputs_json.fields[prompt]` | -| 动作预设 | `generation_inputs_json.fields[actionPreset]` | -| 背景色 | `generation_inputs_json.fields[backgroundColor]` | -| 清晰度/比例/时长/帧数 | `generation_inputs_json.fields[...]` | -| 参考素材 | `generation_inputs_json.references`（稳定 resource/asset ID） | -| 实际模型提示词 | `actual_prompt` | -| 模型、provider、task | 现有列 | -| 帧顺序和对象 | `image_sequence_frames_json` | -| 总时长 | `image_sequence_duration_ms` | -| 单帧尺寸 | `width/height` | -| 来源链 | `source_resource_id` | +| 事实 | 存放位置 | +| --------------------- | ------------------------------------------------------------- | +| 输入模式 | `generation_inputs_json.fields[inputMode]` | +| 用户动作描述 | `generation_inputs_json.fields[prompt]` | +| 动作预设 | `generation_inputs_json.fields[actionPreset]` | +| 背景色 | `generation_inputs_json.fields[backgroundColor]` | +| 清晰度/比例/时长/帧数 | `generation_inputs_json.fields[...]` | +| 参考素材 | `generation_inputs_json.references`（稳定 resource/asset ID） | +| 实际模型提示词 | `actual_prompt` | +| 模型、provider、task | 现有列 | +| 帧顺序和对象 | `image_sequence_frames_json` | +| 总时长 | `image_sequence_duration_ms` | +| 单帧尺寸 | `width/height` | +| 来源链 | `source_resource_id` | 不建议给 `editor_project_resource` 或 `editor_asset` 新增 `input_mode/background_color/fps/frame_count` 列： @@ -603,11 +603,11 @@ flowchart TD #### 去背景 -AI 生成链默认逐帧去背景，视频直接转换固定保留源背景。只有 `character-animation.convert` 的视频直接转换结果提供 `去背景`；AI 生成的序列帧已自动完成去背景，不展示也不接受重复处理。`character-animation` 浮动工具栏在适用时提供 `去背景`，与 `改造 / 拆帧 / 下载` 并列；点击后弹出独立选择框，用户必须显式选择处理方式，再进入现有 durable 派生任务，不得把序列第一帧当普通图片处理。派生任务保持以下边界： +AI 生成链默认逐帧去背景，视频直接转换固定保留源背景。只有 `character-animation.convert` 的视频直接转换结果提供 `去背景`；AI 生成的序列帧已自动完成去背景，不展示也不接受重复处理。`character-animation` 浮动工具栏在适用时提供 `去背景`，与 `改造 / 拆帧 / 下载` 并列；点击后在当前选中序列帧旁打开非模态悬浮窗；悬浮窗随画布平移和缩放更新位置，优先置于画面右侧、空间不足时翻转到其它可用侧，不增加画布遮罩或覆盖源画面；纯色模式内置逐帧 viewer，从正式 objectKey 读取原始帧字节。viewer 以原始比例实际绘制帧，鼠标仅在图像可见区域内采样；悬停实时显示原始像素颜色与放大网格，点击后才回填键色，避免容器留白或 CSS 缩放导致取样偏移。用户必须显式选择处理方式，再进入现有 durable 派生任务，不得把序列第一帧当普通图片处理。派生任务保持以下边界： -- `solid-color` 为本地纯色键控，默认 `#00FF00`，允许颜色输入和浏览器吸管选择任意 `#RRGGBB`；该模式不调用 BgFilter、阿里云或其它外部模型。 +- `solid-color` 为本地纯色键控，默认 `#00FF00`，允许颜色输入和逐帧 viewer 原始像素取样选择任意 `#RRGGBB`；本地算法容忍轻微压缩噪声和接近键色的平缓色差，但不承诺处理明显渐变、阴影、纹理或与主体接近的背景，此类输入应选 `general`；该模式不调用 BgFilter、阿里云或其它外部模型。 - `general` 为通用智能分割，不携带 `screenColor`，逐帧调用 BgFilter `complex + birefnet`；普通背景不再被隐式当作绿幕处理。当前内部接口沿用现有手动 BgFilter 的 0 泥点契约，不在缺少正式定价决策时自行发明用户收费金额；外部供应商成本与后续用户定价另行配置。 -- 前端只负责显式选择；HTTP 入队与 durable worker 共用白名单校验：纯色模式必须携带合法颜色，通用模式禁止携带颜色。worker 只读取已持久化任务参数，最终资源的 `generationInputs` 由后端写入模式、键色或 Provider/model，不信任客户端自报配方。 +- 前端只负责显式选择；HTTP 入队与 durable worker 共用白名单校验：纯色模式必须携带合法颜色，通用模式禁止携带颜色。worker 只读取已持久化任务参数，最终资源的 `generationInputs` 由后端写入模式、键色或 Provider/model，不信任客户端自报配方。悬浮窗的“提交中”只覆盖 HTTP 入队请求；服务端接受任务后立即关闭悬浮窗并交给画布生成占位/任务列表展示后续进度，不能把 durable worker 的完整处理时间继续显示为提交中。HTTP 提交失败时保留悬浮窗并恢复可重试状态。 - 原始不透明序列与透明序列是两个正式 resource； - 去背景全帧成功后一次性发布新的 `character-animation` / `image-sequence` 资源、素材和画布图层，保留原始序列图层不被替换； - 任一帧失败不产生残缺结果； @@ -708,6 +708,7 @@ idle ### P0：会改变架构或成本 1. **视频模式到底是哪一种？** + - A：原视频直接逐帧转 PNG，不调用 Seedance； - B：用视频 + 文本生成新视频（换皮/改造），再抽新视频； - C：两种都要，由用户选择“直接转换/AI 改造”。 @@ -729,6 +730,7 @@ idle 5. **1080p 是否允许换模型？** 当前 `seedance2.0-fast` 明确不支持 1080p。可选： + - 暂不显示（推荐）； - 1080p 自动切换 Seedance 2.0 标准版并重新定价； - 等 provider Fast 真正支持后再开放。 @@ -736,6 +738,7 @@ idle ### P1：会改变交互或数据语义 6. **“已有 Spine 序列帧 png 图片”指什么？** + - 已有 `character-animation` 帧集； - 一张拼好的 Sprite Sheet PNG； - 任意单张角色 PNG。 @@ -755,6 +758,7 @@ idle 需要确定值，例如纯蓝 `#0000FF`、纯紫 `#8000FF`；抠图必须用确定 RGB，不能只存“蓝色/紫色”。还要确认主体颜色与键色冲突时是阻止提交、警告，还是自动换色。 10. **拆帧后去哪里？** + - 只进入素材库（推荐）； - 自动铺到画布； - 弹出独立图片集面板； diff --git a/packages/image-canvas-core/src/overlays.ts b/packages/image-canvas-core/src/overlays.ts index 65d7276d0..e453d6d60 100644 --- a/packages/image-canvas-core/src/overlays.ts +++ b/packages/image-canvas-core/src/overlays.ts @@ -100,6 +100,82 @@ export function resolveSelectedToolbarStyle({ }; } +export function resolveCharacterAnimationBackgroundRemovalPanelStyle({ + sourceLayer, + viewport, + canvasSize, +}: { + sourceLayer: CanvasLayer | null; + viewport: CanvasViewport; + canvasSize: CanvasSize; +}): CanvasOverlayStyle | null { + if (!sourceLayer) { + return null; + } + + const panelWidth = 288; + const panelHeight = 440; + const gap = 12; + const edge = 12; + const sourceLeft = viewport.x + sourceLayer.x * viewport.scale; + const sourceTop = viewport.y + sourceLayer.y * viewport.scale; + const sourceWidth = sourceLayer.width * viewport.scale; + const sourceHeight = sourceLayer.height * viewport.scale; + const sourceRight = sourceLeft + sourceWidth; + const sourceBottom = sourceTop + sourceHeight; + const clampPanelLeft = (left: number) => + Math.min(Math.max(edge, left), Math.max(edge, canvasSize.width - panelWidth - edge)); + const clampPanelTop = (top: number) => + Math.min(Math.max(edge, top), Math.max(edge, canvasSize.height - panelHeight - edge)); + + const rightLeft = sourceRight + gap; + if (rightLeft + panelWidth <= canvasSize.width - edge) { + return { left: rightLeft, top: clampPanelTop(sourceTop) }; + } + + const leftLeft = sourceLeft - gap - panelWidth; + if (leftLeft >= edge) { + return { left: leftLeft, top: clampPanelTop(sourceTop) }; + } + + const bottomTop = sourceBottom + gap; + if (bottomTop + panelHeight <= canvasSize.height - edge) { + return { + left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2), + top: bottomTop, + }; + } + + const topTop = sourceTop - gap - panelHeight; + if (topTop >= edge) { + return { + left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2), + top: topTop, + }; + } + + // 极端缩放下四侧都放不下完整面板时，仍把面板放在可用空间最大的一侧， + // 保证它不覆盖源画面；viewport 会裁掉越界部分，用户可先缩小画布继续操作。 + const availableSides = [ + { space: canvasSize.width - sourceRight, left: rightLeft, top: clampPanelTop(sourceTop) }, + { space: sourceLeft, left: leftLeft, top: clampPanelTop(sourceTop) }, + { + space: canvasSize.height - sourceBottom, + left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2), + top: bottomTop, + }, + { + space: sourceTop, + left: clampPanelLeft(sourceLeft + sourceWidth / 2 - panelWidth / 2), + top: topTop, + }, + ]; + const bestSide = availableSides.reduce((best, candidate) => + candidate.space > best.space ? candidate : best, + ); + return { left: bestSide.left, top: bestSide.top }; +} + export function resolveQuickEditFocusViewport({ sourceLayer, canvasSize, diff --git a/server-rs/crates/api-server/src/editor_green_screen.rs b/server-rs/crates/api-server/src/editor_green_screen.rs index c7ab30e53..9bdfdf87c 100644 --- a/server-rs/crates/api-server/src/editor_green_screen.rs +++ b/server-rs/crates/api-server/src/editor_green_screen.rs @@ -316,6 +316,35 @@ mod tests { assert_eq!(image.get_pixel(1, 1).0, [255, 255, 255, 255]); } + #[test] + fn character_animation_solid_color_removal_tolerates_small_background_color_variation() { + let key = [72, 161, 224]; + let mut source = RgbaImage::from_pixel(9, 9, Rgba([82, 151, 229, 255])); + source.put_pixel(0, 0, Rgba([65, 168, 217, 255])); + source.put_pixel(4, 4, Rgba([232, 52, 48, 255])); + let mut bytes = Vec::new(); + DynamicImage::ImageRgba8(source) + .write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png) + .expect("test PNG should encode"); + + let output = remove_editor_character_animation_solid_color_background( + &DownloadedOpenAiImage { + bytes, + mime_type: "image/png".to_string(), + extension: "png".to_string(), + }, + key, + ) + .expect("near-key solid background should be removed"); + let image = image::load_from_memory(output.bytes.as_slice()) + .expect("output PNG should decode") + .to_rgba8(); + + assert_eq!(image.get_pixel(0, 0).0[3], 0); + assert_eq!(image.get_pixel(8, 8).0[3], 0); + assert_eq!(image.get_pixel(4, 4).0, [232, 52, 48, 255]); + } + #[test] fn character_animation_custom_key_color_preserves_green_foreground() { let mut source = RgbaImage::from_pixel(3, 3, Rgba([0, 0, 255, 255])); diff --git a/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalColorPickerModel.ts b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalColorPickerModel.ts new file mode 100644 index 000000000..61804c8dd --- /dev/null +++ b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalColorPickerModel.ts @@ -0,0 +1,160 @@ +export type FrameCanvasPoint = { + x: number; + y: number; +}; + +export type FrameCanvasColor = { + red: number; + green: number; + blue: number; +}; + +/** + * 将指针位置映射到实际绘制出的预览 canvas，而非外层容器。 + * 超出 canvas 可见像素区域时不取样，避免把边缘颜色误当作背景色。 + */ +export function resolveFrameCanvasSamplePoint({ + clientX, + clientY, + rect, + imageWidth, + imageHeight, +}: { + clientX: number; + clientY: number; + rect: Pick; + imageWidth: number; + imageHeight: number; +}): FrameCanvasPoint | null { + if ( + rect.width <= 0 || + rect.height <= 0 || + imageWidth <= 0 || + imageHeight <= 0 + ) { + return null; + } + + const relativeX = clientX - rect.left; + const relativeY = clientY - rect.top; + if ( + relativeX < 0 || + relativeY < 0 || + relativeX >= rect.width || + relativeY >= rect.height + ) { + return null; + } + + return { + x: Math.min( + imageWidth - 1, + Math.floor((relativeX / rect.width) * imageWidth), + ), + y: Math.min( + imageHeight - 1, + Math.floor((relativeY / rect.height) * imageHeight), + ), + }; +} + +export function formatFrameCanvasColorHex({ + red, + green, + blue, +}: FrameCanvasColor) { + return `#${[red, green, blue] + .map((channel) => + Math.max(0, Math.min(255, channel)).toString(16).padStart(2, '0'), + ) + .join('')}`.toUpperCase(); +} + +export function resolveFrameColorMagnifierSourcePoint({ + point, + offsetX, + offsetY, + imageWidth, + imageHeight, +}: { + point: FrameCanvasPoint; + offsetX: number; + offsetY: number; + imageWidth: number; + imageHeight: number; +}): FrameCanvasPoint | null { + if (imageWidth <= 0 || imageHeight <= 0) { + return null; + } + return { + x: Math.min(imageWidth - 1, Math.max(0, point.x + offsetX)), + y: Math.min(imageHeight - 1, Math.max(0, point.y + offsetY)), + }; +} + +export type FrameColorMagnifierPlacement = { + left: number; + top: number; + horizontal: 'left' | 'right'; + vertical: 'top' | 'bottom'; +}; + +/** + * 放大镜始终置于指针的对角可用区域：优先沿当前可用边，空间不足即翻转。 + * 这样即使用户停在预览边缘，放大镜也不会覆盖正在取样的像素。 + */ +export function resolveFrameColorMagnifierPlacement({ + pointerLeft, + pointerTop, + previewWidth, + previewHeight, + magnifierSize, + gap = 12, + edge = 4, +}: { + pointerLeft: number; + pointerTop: number; + previewWidth: number; + previewHeight: number; + magnifierSize: number; + gap?: number; + edge?: number; +}): FrameColorMagnifierPlacement | null { + if (previewWidth <= 0 || previewHeight <= 0 || magnifierSize <= 0) { + return null; + } + + const roomRight = previewWidth - pointerLeft - gap - edge; + const roomLeft = pointerLeft - gap - edge; + const horizontal = + roomRight >= magnifierSize || roomRight >= roomLeft ? 'right' : 'left'; + const roomBottom = previewHeight - pointerTop - gap - edge; + const roomTop = pointerTop - gap - edge; + const vertical = + roomBottom >= magnifierSize || roomBottom >= roomTop ? 'bottom' : 'top'; + const maxLeft = Math.max(edge, previewWidth - magnifierSize - edge); + const maxTop = Math.max(edge, previewHeight - magnifierSize - edge); + + return { + horizontal, + vertical, + left: Math.min( + maxLeft, + Math.max( + edge, + horizontal === 'right' + ? pointerLeft + gap + : pointerLeft - gap - magnifierSize, + ), + ), + top: Math.min( + maxTop, + Math.max( + edge, + vertical === 'bottom' + ? pointerTop + gap + : pointerTop - gap - magnifierSize, + ), + ), + }; +} diff --git a/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalDialog.test.tsx b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalDialog.test.tsx deleted file mode 100644 index 82346227a..000000000 --- a/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalDialog.test.tsx +++ /dev/null @@ -1,67 +0,0 @@ -/* @vitest-environment jsdom */ - -import { fireEvent, render, screen, waitFor } from '@testing-library/react'; -import { useState } from 'react'; -import { afterEach, describe, expect, it, vi } from 'vitest'; - -import type { EditorCharacterAnimationBackgroundRemovalMode } from '../../services/image-editor/editorProjectClient'; -import { ImageCanvasCharacterAnimationBackgroundRemovalDialog } from './ImageCanvasCharacterAnimationBackgroundRemovalDialog'; - -function DialogHarness({ onConfirm = vi.fn() }: { onConfirm?: () => void }) { - const [mode, setMode] = - useState('solid-color'); - const [screenColor, setScreenColor] = useState('#00FF00'); - - return ( - - ); -} - -afterEach(() => { - delete (window as typeof window & { EyeDropper?: unknown }).EyeDropper; -}); - -describe('ImageCanvasCharacterAnimationBackgroundRemovalDialog', () => { - it('requires an explicit pure-color or general mode selection', () => { - const onConfirm = vi.fn(); - render(); - - expect( - (screen.getByRole('radio', { name: /纯色背景抠图/ }) as HTMLInputElement) - .checked, - ).toBe(true); - expect( - (screen.getByLabelText('纯色背景颜色') as HTMLInputElement).value, - ).toBe('#00ff00'); - - fireEvent.click(screen.getByRole('radio', { name: /通用智能抠图/ })); - expect(screen.queryByLabelText('纯色背景颜色')).toBeNull(); - fireEvent.click(screen.getByRole('button', { name: '开始处理' })); - expect(onConfirm).toHaveBeenCalledTimes(1); - }); - - it('uses the browser eyedropper result as the solid background color', async () => { - const open = vi.fn().mockResolvedValue({ sRGBHex: '#336699' }); - (window as typeof window & { EyeDropper?: unknown }).EyeDropper = class { - open = open; - }; - render(); - - fireEvent.click(screen.getByRole('button', { name: '吸取画布颜色' })); - - await waitFor(() => - expect( - (screen.getByLabelText('纯色背景颜色') as HTMLInputElement).value, - ).toBe('#336699'), - ); - expect(open).toHaveBeenCalledTimes(1); - }); -}); diff --git a/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalDialog.tsx b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalDialog.tsx deleted file mode 100644 index beee18158..000000000 --- a/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalDialog.tsx +++ /dev/null @@ -1,137 +0,0 @@ -import { useCallback, useState } from 'react'; - -import type { EditorCharacterAnimationBackgroundRemovalMode } from '../../services/image-editor/editorProjectClient'; -import { PlatformActionButton } from '../common/PlatformActionButton'; -import { UnifiedConfirmDialog } from '../common/UnifiedConfirmDialog'; - -type EyeDropperConstructor = new () => { - open: () => Promise<{ sRGBHex: string }>; -}; - -type ImageCanvasCharacterAnimationBackgroundRemovalDialogProps = { - open: boolean; - mode: EditorCharacterAnimationBackgroundRemovalMode; - screenColor: string; - busy?: boolean; - onModeChange: (mode: EditorCharacterAnimationBackgroundRemovalMode) => void; - onScreenColorChange: (screenColor: string) => void; - onClose: () => void; - onConfirm: () => void; -}; - -function resolveEyeDropper() { - return (window as typeof window & { EyeDropper?: EyeDropperConstructor }) - .EyeDropper; -} - -export function ImageCanvasCharacterAnimationBackgroundRemovalDialog({ - open, - mode, - screenColor, - busy = false, - onModeChange, - onScreenColorChange, - onClose, - onConfirm, -}: ImageCanvasCharacterAnimationBackgroundRemovalDialogProps) { - const [isPickingColor, setIsPickingColor] = useState(false); - const pickScreenColor = useCallback(async () => { - const EyeDropper = resolveEyeDropper(); - if (!EyeDropper || isPickingColor) { - return; - } - setIsPickingColor(true); - try { - const result = await new EyeDropper().open(); - if (/^#[0-9a-f]{6}$/i.test(result.sRGBHex)) { - onScreenColorChange(result.sRGBHex.toUpperCase()); - } - } catch { - // 用户取消系统吸管不属于错误，保持当前颜色即可。 - } finally { - setIsPickingColor(false); - } - }, [isPickingColor, onScreenColorChange]); - - return ( - -
- - -
- - {mode === 'solid-color' ? ( -
- - {screenColor} - void pickScreenColor()} - > - {isPickingColor ? '取色中' : '吸取画布颜色'} - -
- ) : null} - - ); -} diff --git a/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel.test.tsx b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel.test.tsx new file mode 100644 index 000000000..562d61211 --- /dev/null +++ b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel.test.tsx @@ -0,0 +1,191 @@ +/* @vitest-environment jsdom */ + +import { fireEvent, render, screen } from '@testing-library/react'; +import { useState } from 'react'; +import { describe, expect, it, vi } from 'vitest'; + +import type { + EditorCharacterAnimationBackgroundRemovalMode, + EditorImageSequenceFrameResult, +} from '../../services/image-editor/editorProjectClient'; +import { + formatFrameCanvasColorHex, + resolveFrameCanvasSamplePoint, + resolveFrameColorMagnifierPlacement, + resolveFrameColorMagnifierSourcePoint, +} from './ImageCanvasCharacterAnimationBackgroundRemovalColorPickerModel'; +import { ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel } from './ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel'; + +const frames: EditorImageSequenceFrameResult[] = [ + { + imageSrc: '/generated-images/editor/frame-001.png', + objectKey: 'generated-images/editor/frame-001.png', + width: 320, + height: 240, + }, + { + imageSrc: '/generated-images/editor/frame-002.png', + objectKey: 'generated-images/editor/frame-002.png', + width: 320, + height: 240, + }, +]; + +function FloatingPanelHarness({ + onConfirm = vi.fn(), +}: { + onConfirm?: () => void; +}) { + const [mode, setMode] = + useState('solid-color'); + const [screenColor, setScreenColor] = useState('#00FF00'); + + return ( + + ); +} + +describe('ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel', () => { + it('requires an explicit pure-color or general mode selection', () => { + const onConfirm = vi.fn(); + render(); + + const panel = screen.getByRole('dialog', { name: '序列帧去背景' }); + expect(panel.getAttribute('aria-modal')).toBe('false'); + expect(panel.className).toContain('absolute'); + expect(panel.className).toContain('bg-white'); + expect(panel.style.left).toBe('352px'); + expect(panel.style.top).toBe('96px'); + + expect( + (screen.getByRole('radio', { name: /纯色背景抠图/ }) as HTMLInputElement) + .checked, + ).toBe(true); + expect(screen.getByLabelText('序列帧像素取色预览')).toBeTruthy(); + expect(screen.getByText('悬停预览，点击取色')).toBeTruthy(); + expect(screen.getByLabelText('选择序列帧')).toBeTruthy(); + + fireEvent.click(screen.getByRole('radio', { name: /通用智能抠图/ })); + expect(screen.queryByLabelText('序列帧像素取色预览')).toBeNull(); + fireEvent.click(screen.getByRole('button', { name: '开始处理' })); + expect(onConfirm).toHaveBeenCalledTimes(1); + }); + + it('maps a rendered preview click back to the original frame pixel', () => { + expect( + resolveFrameCanvasSamplePoint({ + clientX: 110, + clientY: 70, + rect: { left: 10, top: 20, width: 200, height: 100 }, + imageWidth: 800, + imageHeight: 400, + }), + ).toEqual({ x: 400, y: 200 }); + + expect( + resolveFrameCanvasSamplePoint({ + clientX: -10, + clientY: 300, + rect: { left: 10, top: 20, width: 200, height: 100 }, + imageWidth: 800, + imageHeight: 400, + }), + ).toBeNull(); + }); + + it('rejects a pointer in preview letterboxing instead of sampling a shifted pixel', () => { + expect( + resolveFrameCanvasSamplePoint({ + clientX: 59, + clientY: 20, + rect: { left: 60, top: 20, width: 120, height: 160 }, + imageWidth: 1200, + imageHeight: 1600, + }), + ).toBeNull(); + + expect( + resolveFrameCanvasSamplePoint({ + clientX: 179.9, + clientY: 179.9, + rect: { left: 60, top: 20, width: 120, height: 160 }, + imageWidth: 1200, + imageHeight: 1600, + }), + ).toEqual({ x: 1199, y: 1599 }); + }); + + it('flips the magnifier away from the pointer when its preferred side is full', () => { + expect( + resolveFrameColorMagnifierPlacement({ + pointerLeft: 16, + pointerTop: 18, + previewWidth: 264, + previewHeight: 176, + magnifierSize: 88, + }), + ).toMatchObject({ + horizontal: 'right', + vertical: 'bottom', + left: 28, + top: 30, + }); + expect( + resolveFrameColorMagnifierPlacement({ + pointerLeft: 252, + pointerTop: 164, + previewWidth: 264, + previewHeight: 176, + magnifierSize: 88, + }), + ).toMatchObject({ + horizontal: 'left', + vertical: 'top', + left: 152, + top: 64, + }); + expect( + resolveFrameColorMagnifierPlacement({ + pointerLeft: 250, + pointerTop: 18, + previewWidth: 264, + previewHeight: 176, + magnifierSize: 88, + }), + ).toMatchObject({ horizontal: 'left', vertical: 'bottom' }); + }); + + it('keeps the magnifier sample clamped to the original frame and formats hover color', () => { + expect( + resolveFrameColorMagnifierSourcePoint({ + point: { x: 0, y: 0 }, + offsetX: -5, + offsetY: -5, + imageWidth: 320, + imageHeight: 240, + }), + ).toEqual({ x: 0, y: 0 }); + expect( + resolveFrameColorMagnifierSourcePoint({ + point: { x: 319, y: 239 }, + offsetX: 5, + offsetY: 5, + imageWidth: 320, + imageHeight: 240, + }), + ).toEqual({ x: 319, y: 239 }); + expect(formatFrameCanvasColorHex({ red: 4, green: 201, blue: 255 })).toBe( + '#04C9FF', + ); + }); +}); diff --git a/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel.tsx b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel.tsx new file mode 100644 index 000000000..925d3c6ad --- /dev/null +++ b/src/components/image-editor/ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel.tsx @@ -0,0 +1,517 @@ +import type { CSSProperties, PointerEvent as ReactPointerEvent } from 'react'; +import { useCallback, useEffect, useRef, useState } from 'react'; + +import { readAssetBytes } from '../../services/assetReadUrlService'; +import type { EditorImageSequenceFrameResult } from '../../services/image-editor/editorProjectClient'; +import type { EditorCharacterAnimationBackgroundRemovalMode } from '../../services/image-editor/editorProjectClient'; +import { PlatformActionButton } from '../common/PlatformActionButton'; +import { + formatFrameCanvasColorHex, + resolveFrameCanvasSamplePoint, + resolveFrameColorMagnifierPlacement, + resolveFrameColorMagnifierSourcePoint, + type FrameCanvasColor, + type FrameCanvasPoint, +} from './ImageCanvasCharacterAnimationBackgroundRemovalColorPickerModel'; + +type DecodedFrameImage = { + image: CanvasImageSource; + width: number; + height: number; + dispose: () => void; +}; + +type FrameHoverSample = { + point: FrameCanvasPoint; + color: FrameCanvasColor; + left: number; + top: number; + previewWidth: number; + previewHeight: number; +}; + +const magnifierRadius = 5; +const magnifierCellSize = 8; +const magnifierSize = (magnifierRadius * 2 + 1) * magnifierCellSize; + +type ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanelProps = { + open: boolean; + style: CSSProperties; + frames: readonly EditorImageSequenceFrameResult[]; + mode: EditorCharacterAnimationBackgroundRemovalMode; + screenColor: string; + busy?: boolean; + onModeChange: (mode: EditorCharacterAnimationBackgroundRemovalMode) => void; + onScreenColorChange: (screenColor: string) => void; + onClose: () => void; + onConfirm: () => void; +}; + +async function decodeFrameImage(blob: Blob): Promise { + if (typeof createImageBitmap === 'function') { + const bitmap = await createImageBitmap(blob); + return { + image: bitmap, + width: bitmap.width, + height: bitmap.height, + dispose: () => bitmap.close(), + }; + } + + const url = URL.createObjectURL(blob); + const image = new Image(); + await new Promise((resolve, reject) => { + image.onload = () => resolve(); + image.onerror = () => reject(new Error('帧图片解码失败')); + image.src = url; + }); + return { + image, + width: image.naturalWidth || image.width, + height: image.naturalHeight || image.height, + dispose: () => URL.revokeObjectURL(url), + }; +} + +export function ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel({ + open, + style, + frames, + mode, + screenColor, + busy = false, + onModeChange, + onScreenColorChange, + onClose, + onConfirm, +}: ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanelProps) { + const [frameIndex, setFrameIndex] = useState(0); + const [isLoadingFrame, setIsLoadingFrame] = useState(false); + const [frameLoadError, setFrameLoadError] = useState(null); + const [hoverSample, setHoverSample] = useState(null); + const frameCanvasRef = useRef(null); + const framePreviewContainerRef = useRef(null); + const magnifierCanvasRef = useRef(null); + const decodedFrameRef = useRef(null); + const activeFrame = + frames[Math.min(frameIndex, Math.max(0, frames.length - 1))]; + + useEffect(() => { + setFrameIndex((currentIndex) => + Math.min(currentIndex, Math.max(0, frames.length - 1)), + ); + }, [frames.length]); + + useEffect(() => { + setHoverSample(null); + }, [activeFrame, mode]); + + useEffect(() => { + if (!open || mode !== 'solid-color' || !activeFrame) { + return; + } + let cancelled = false; + let decodedFrame: DecodedFrameImage | null = null; + setIsLoadingFrame(true); + setFrameLoadError(null); + + void readAssetBytes(activeFrame.imageSrc, { + objectKey: activeFrame.objectKey, + }) + .then((response) => response.blob()) + .then(decodeFrameImage) + .then((decoded) => { + decodedFrame = decoded; + if (cancelled) { + decoded.dispose(); + return; + } + decodedFrameRef.current?.dispose(); + decodedFrameRef.current = decoded; + const canvas = frameCanvasRef.current; + const context = canvas?.getContext('2d', { willReadFrequently: true }); + if (!canvas || !context) { + throw new Error('当前浏览器不支持帧像素取色'); + } + canvas.width = decoded.width; + canvas.height = decoded.height; + context.clearRect(0, 0, decoded.width, decoded.height); + context.drawImage(decoded.image, 0, 0, decoded.width, decoded.height); + }) + .catch((error) => { + if (!cancelled) { + setFrameLoadError( + error instanceof Error && error.message.trim() + ? error.message + : '读取序列帧失败', + ); + } + }) + .finally(() => { + if (!cancelled) { + setIsLoadingFrame(false); + } + }); + + return () => { + cancelled = true; + if (decodedFrame && decodedFrameRef.current !== decodedFrame) { + decodedFrame.dispose(); + } + }; + }, [activeFrame, mode, open]); + + useEffect( + () => () => { + decodedFrameRef.current?.dispose(); + decodedFrameRef.current = null; + }, + [], + ); + + const resolvePointerSample = useCallback( + (event: ReactPointerEvent): FrameHoverSample | null => { + const canvas = frameCanvasRef.current; + const context = canvas?.getContext('2d', { willReadFrequently: true }); + if (!canvas || !context) { + return null; + } + const rect = event.currentTarget.getBoundingClientRect(); + const previewRect = + framePreviewContainerRef.current?.getBoundingClientRect() ?? rect; + const point = resolveFrameCanvasSamplePoint({ + clientX: event.clientX, + clientY: event.clientY, + rect, + imageWidth: canvas.width, + imageHeight: canvas.height, + }); + if (!point) { + return null; + } + const pixel = context.getImageData(point.x, point.y, 1, 1).data; + return { + point, + color: { + red: pixel[0] ?? 0, + green: pixel[1] ?? 0, + blue: pixel[2] ?? 0, + }, + left: event.clientX - previewRect.left, + top: event.clientY - previewRect.top, + previewWidth: previewRect.width, + previewHeight: previewRect.height, + }; + }, + [], + ); + + const drawMagnifier = useCallback((point: FrameCanvasPoint) => { + const sourceCanvas = frameCanvasRef.current; + const sourceContext = sourceCanvas?.getContext('2d', { + willReadFrequently: true, + }); + const magnifierCanvas = magnifierCanvasRef.current; + const magnifierContext = magnifierCanvas?.getContext('2d'); + if ( + !sourceCanvas || + !sourceContext || + !magnifierCanvas || + !magnifierContext + ) { + return; + } + + magnifierCanvas.width = magnifierSize; + magnifierCanvas.height = magnifierSize; + magnifierContext.imageSmoothingEnabled = false; + for (let row = -magnifierRadius; row <= magnifierRadius; row += 1) { + for ( + let column = -magnifierRadius; + column <= magnifierRadius; + column += 1 + ) { + const sourcePoint = resolveFrameColorMagnifierSourcePoint({ + point, + offsetX: column, + offsetY: row, + imageWidth: sourceCanvas.width, + imageHeight: sourceCanvas.height, + }); + if (!sourcePoint) { + continue; + } + const pixel = sourceContext.getImageData( + sourcePoint.x, + sourcePoint.y, + 1, + 1, + ).data; + magnifierContext.fillStyle = `rgb(${pixel[0] ?? 0}, ${pixel[1] ?? 0}, ${pixel[2] ?? 0})`; + magnifierContext.fillRect( + (column + magnifierRadius) * magnifierCellSize, + (row + magnifierRadius) * magnifierCellSize, + magnifierCellSize, + magnifierCellSize, + ); + } + } + magnifierContext.strokeStyle = 'rgba(15, 23, 42, 0.35)'; + magnifierContext.lineWidth = 1; + for (let index = 0; index <= magnifierRadius * 2 + 1; index += 1) { + const offset = index * magnifierCellSize + 0.5; + magnifierContext.beginPath(); + magnifierContext.moveTo(offset, 0); + magnifierContext.lineTo(offset, magnifierSize); + magnifierContext.moveTo(0, offset); + magnifierContext.lineTo(magnifierSize, offset); + magnifierContext.stroke(); + } + magnifierContext.strokeStyle = '#FFFFFF'; + magnifierContext.lineWidth = 2; + magnifierContext.strokeRect( + magnifierRadius * magnifierCellSize + 1, + magnifierRadius * magnifierCellSize + 1, + magnifierCellSize - 2, + magnifierCellSize - 2, + ); + }, []); + + useEffect(() => { + if (hoverSample) { + drawMagnifier(hoverSample.point); + } + }, [drawMagnifier, hoverSample]); + + const handleFramePointerMove = useCallback( + (event: ReactPointerEvent) => { + const sample = resolvePointerSample(event); + if (!sample) { + setHoverSample(null); + return; + } + setHoverSample(sample); + }, + [resolvePointerSample], + ); + + const sampleFrameColor = useCallback( + (event: ReactPointerEvent) => { + const sample = resolvePointerSample(event); + if (sample) { + onScreenColorChange(formatFrameCanvasColorHex(sample.color)); + } + }, + [onScreenColorChange, resolvePointerSample], + ); + + if (!open) { + return null; + } + + const confirmDisabled = + busy || (mode === 'solid-color' && !/^#[0-9A-F]{6}$/i.test(screenColor)); + const magnifierPlacement = hoverSample + ? resolveFrameColorMagnifierPlacement({ + pointerLeft: hoverSample.left, + pointerTop: hoverSample.top, + previewWidth: hoverSample.previewWidth, + previewHeight: hoverSample.previewHeight, + magnifierSize, + }) + : null; + + return ( +
event.stopPropagation()} + > +
+

序列帧去背景

+ +
+ +
+
+ + +
+ + {mode === 'solid-color' ? ( +
+
+ + {screenColor} +
+ + {activeFrame ? ( +
+
+ 逐帧取色 + {`${frameIndex + 1}/${frames.length}`} +
+
+ setHoverSample(null)} + onPointerDown={sampleFrameColor} + /> + {magnifierPlacement ? ( +
+
+ ) : null} +
+
+ 悬停预览，点击取色 + {hoverSample ? ( + + + + {formatFrameCanvasColorHex(hoverSample.color)} + + + ) : null} +
+ {isLoadingFrame ? ( + + 正在读取原始帧… + + ) : null} + {frameLoadError ? ( + {frameLoadError} + ) : null} + {frames.length > 1 ? ( + + setFrameIndex( + Number.parseInt(event.currentTarget.value, 10) || 0, + ) + } + /> + ) : null} +
+ ) : ( + + 序列帧列表为空，无法取色。 + + )} +
+ ) : null} +
+ +
+ + 取消 + + + {busy ? '提交中' : '开始处理'} + +
+
+ ); +} diff --git a/src/components/image-editor/ImageCanvasEditorView.tsx b/src/components/image-editor/ImageCanvasEditorView.tsx index f4833ce5e..a3129c193 100644 --- a/src/components/image-editor/ImageCanvasEditorView.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.tsx @@ -35,7 +35,7 @@ import { import { usePlatformProfileCenterController } from '../platform-entry/usePlatformProfileCenterController'; import type { ImageCanvasActionResult } from './ImageCanvasActionsContext'; import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider'; -import { ImageCanvasCharacterAnimationBackgroundRemovalDialog } from './ImageCanvasCharacterAnimationBackgroundRemovalDialog'; +import { ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel } from './ImageCanvasCharacterAnimationBackgroundRemovalFloatingPanel'; import { canvasAssetKindOrNull, DEFAULT_CANVAS_BACKGROUND_COLOR, @@ -79,6 +79,7 @@ import { fitViewportToBounds } from './ImageCanvasInteractionModel'; import type { CanvasLayerCopyBlockReason } from './ImageCanvasLayerCommandModel'; import { isCanvasGenerationComposerVisible, + resolveCharacterAnimationBackgroundRemovalPanelStyle, resolveQuickEditFocusViewport, } from './ImageCanvasOverlayModel'; import { @@ -2684,13 +2685,13 @@ export function ImageCanvasEditorView({ ); }, onRemoveCharacterAnimationBackground: (layer: CanvasLayer) => { - void flushProjectPersistence().then(() => + void flushProjectPersistence().then(() => { setCharacterAnimationBackgroundRemovalDraft({ sourceLayer: layer, mode: 'solid-color', screenColor: '#00FF00', - }), - ); + }); + }); }, onExtractUiDesignAssets: extractUiDesignAssets, onUiAssetExtractionToolChange: changeUiAssetExtractionTool, @@ -2738,7 +2739,62 @@ export function ImageCanvasEditorView({ onSwitchTool: switchTool, onOpenToolOptions: generationSurface.openToolbarOptionMenu, onCloseToolOptions: generationSurface.closeToolbarOptionMenu, - children: generationComposerNode, + children: ( + <> + {generationComposerNode} + {characterAnimationBackgroundRemovalDraft ? ( + + setCharacterAnimationBackgroundRemovalDraft((draft) => + draft ? { ...draft, mode } : draft, + ) + } + onScreenColorChange={(screenColor) => + setCharacterAnimationBackgroundRemovalDraft((draft) => + draft ? { ...draft, screenColor } : draft, + ) + } + onClose={() => setCharacterAnimationBackgroundRemovalDraft(null)} + onConfirm={() => { + const draft = characterAnimationBackgroundRemovalDraft; + void removeSelectedCharacterAnimationBackground( + draft.sourceLayer, + { + mode: draft.mode, + ...(draft.mode === 'solid-color' + ? { screenColor: draft.screenColor } + : {}), + onSubmissionAccepted: () => + setCharacterAnimationBackgroundRemovalDraft((current) => + current?.sourceLayer.id === draft.sourceLayer.id + ? null + : current, + ), + }, + ); + }} + /> + ) : null} + + ), }; return ( @@ -2760,42 +2816,6 @@ export function ImageCanvasEditorView({ onClose: () => setMetadataLayer(null), }} /> - - setCharacterAnimationBackgroundRemovalDraft((draft) => - draft ? { ...draft, mode } : draft, - ) - } - onScreenColorChange={(screenColor) => - setCharacterAnimationBackgroundRemovalDraft((draft) => - draft ? { ...draft, screenColor } : draft, - ) - } - onClose={() => setCharacterAnimationBackgroundRemovalDraft(null)} - onConfirm={() => { - const draft = characterAnimationBackgroundRemovalDraft; - if (!draft) { - return; - } - void removeSelectedCharacterAnimationBackground(draft.sourceLayer, { - mode: draft.mode, - ...(draft.mode === 'solid-color' - ? { screenColor: draft.screenColor } - : {}), - }).then(() => setCharacterAnimationBackgroundRemovalDraft(null)); - }} - /> { ).toEqual({ left: 1320, top: 88 }); }); + it('places background removal beside the source and flips when the right side is full', () => { + expect( + resolveCharacterAnimationBackgroundRemovalPanelStyle({ + sourceLayer: createLayer({ x: 100, y: 80, width: 240, height: 120 }), + viewport: { x: 0, y: 0, scale: 1 }, + canvasSize: { width: 900, height: 640 }, + }), + ).toEqual({ left: 352, top: 80 }); + + expect( + resolveCharacterAnimationBackgroundRemovalPanelStyle({ + sourceLayer: createLayer({ x: 620, y: 80, width: 240, height: 120 }), + viewport: { x: 0, y: 0, scale: 1 }, + canvasSize: { width: 900, height: 640 }, + }), + ).toEqual({ left: 320, top: 80 }); + }); + + it('keeps background removal adjacent while the canvas pans and zooms', () => { + expect( + resolveCharacterAnimationBackgroundRemovalPanelStyle({ + sourceLayer: createLayer({ x: 40, y: 20, width: 120, height: 80 }), + viewport: { x: 30, y: 50, scale: 2 }, + canvasSize: { width: 900, height: 640 }, + }), + ).toEqual({ left: 362, top: 90 }); + }); + it('fits tall quick edit sources above the reserved bottom composer area', () => { const sourceLayer = createLayer({ x: 100, y: 50, width: 360, height: 1600 }); const viewport = resolveQuickEditFocusViewport({ diff --git a/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx b/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx index 898101275..810f40920 100644 --- a/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx +++ b/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx @@ -370,6 +370,7 @@ function GenerationWorkflowHarness({ onFlushProjectPersistenceSnapshot, refreshAssetLibrary, upsertGeneratedAsset, + onCharacterAnimationBackgroundRemovalSubmissionAccepted, }: { initialLayers?: CanvasLayer[]; initialCanvasGenerationDialogs?: CanvasGenerationDialogState[]; @@ -395,6 +396,7 @@ function GenerationWorkflowHarness({ upsertGeneratedAsset?: Parameters< typeof useImageCanvasGenerationWorkflow >[0]['upsertGeneratedAsset']; + onCharacterAnimationBackgroundRemovalSubmissionAccepted?: () => void; }) { const [layers, setLayers] = useState(initialLayers); const [viewport, setViewport] = useState(initialViewport); @@ -950,6 +952,8 @@ function GenerationWorkflowHarness({ void workflow.removeSelectedCharacterAnimationBackground(layers[0]!, { mode: 'solid-color', screenColor: '#00FF00', + onSubmissionAccepted: + onCharacterAnimationBackgroundRemovalSubmissionAccepted, }) } > @@ -3044,6 +3048,7 @@ describe('useImageCanvasGenerationWorkflow', () => { }>(); const applyProjectSnapshot = vi.fn(); const refreshAssetLibrary = vi.fn().mockResolvedValue(undefined); + const onSubmissionAccepted = vi.fn(); const backendProject: EditorProjectSnapshot = { projectId: 'project-1', title: '序列帧去背景项目', @@ -3065,6 +3070,9 @@ describe('useImageCanvasGenerationWorkflow', () => { projectId="project-1" applyProjectSnapshot={applyProjectSnapshot} refreshAssetLibrary={refreshAssetLibrary} + onCharacterAnimationBackgroundRemovalSubmissionAccepted={ + onSubmissionAccepted + } initialLayers={[ createLayer({ title: '勇者奔跑', @@ -3095,6 +3103,7 @@ describe('useImageCanvasGenerationWorkflow', () => { screenColor: '#00FF00', }), ); + expect(onSubmissionAccepted).not.toHaveBeenCalled(); deferred.resolve({ taskId: 'background-removal-task', @@ -3110,6 +3119,9 @@ describe('useImageCanvasGenerationWorkflow', () => { project: backendProject, }); + await waitFor(() => { + expect(onSubmissionAccepted).toHaveBeenCalledTimes(1); + }); await waitFor(() => { expect(applyProjectSnapshot).toHaveBeenCalledWith(backendProject, { type: 'remove-background', diff --git a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts index f781ee7de..37732828c 100644 --- a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts +++ b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts @@ -3682,6 +3682,7 @@ export function useImageCanvasGenerationWorkflow({ options: { mode: EditorCharacterAnimationBackgroundRemovalMode; screenColor?: string; + onSubmissionAccepted?: () => void; }, ) => { if ( @@ -3743,6 +3744,9 @@ export function useImageCanvasGenerationWorkflow({ sourceLayer, ), }); + // 选择悬浮窗只负责提交参数；HTTP 接受任务后立即交给画布占位和任务队列， + // 不能等待 durable job 终态，否则整个处理周期都会被误显示为“提交中”。 + options.onSubmissionAccepted?.(); const queued = await applyQueuedEditorGenerationProject( result, projectId,