实现生成媒体画布定位与平滑聚焦
为生成图片、视频和音频的右键菜单新增画布定位动作，普通点击与键盘保持被动 根据资源标识查找对应画布图层并执行平滑视口聚焦，不改变当前选择与面板状态 同步视口状态引用、动画取消与减少动态效果逻辑，保证连续操作结果一致 补齐媒体定位和视口控制测试，并更新相关技术文档与决策记录
This commit is contained in:
@@ -16,6 +16,16 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-30 画布 Agent 结果通过实例级 Action Context 刷新并从右键菜单定位资源
|
||||
|
||||
- 背景：画布 Agent 图片、视频和音频结果需要提供画布定位和结果完成后的工程刷新；若继续从舞台向面板、消息和工具结果逐层传 callback，会扩大现有 prop drilling，而把 callback 或瞬时命令放入全局 Zustand 又会引入多实例和卸载残留问题。直接把点击和定位语义附到生成媒体上还会让视频 / 音频的播放、暂停、拖动与音量操作误触发画布聚焦，并给原生媒体控件附加错误的定位标签。
|
||||
- 决策：`ImageCanvasEditorView` 提供实例级 `ImageCanvasActionsContext`，暴露 `focusResource(resourceId)` 与 `refreshCanvas()`；Agent 工具结果刷新、生成结果右键菜单和任务侧栏直接消费对应动作，不新增中间 props，也不扩展现有只保存 `projectId` 的 Zustand store。`refreshCanvas()` 统一重新读取当前工程快照并刷新素材库，任务列表入队后的立即失效继续保持独立。带有效 `resourceId` 的图片、视频和音频生成结果只在素材右键菜单显示“在画布中定位”，普通媒体卡片不声明按钮语义或 `tabIndex`，点击、Enter 和 Space 均不触发定位；视频和音频原生播放器只使用描述媒体自身的标签，不承载定位标签或点击处理。`focusResource(resourceId)` 命中当前图层后只按完整画布 viewport 播放固定 `420ms` ease-out fit 动画，不改变图层选择、工具、侧栏或 Agent 面板；普通 viewport 写入和用户交互可取消动画，reduced-motion 直接完成，缺失 ID 或图层时无动作。
|
||||
- 影响范围：图片画布 Action Context、viewport controls、Agent 结果媒体交互、前端测试和编辑器专题文档；不修改共享 DTO、后端 API 或 SpacetimeDB。
|
||||
- 验证方式：覆盖 Context 作用域、三类媒体普通点击无动作与右键菜单分发、原生媒体控件无定位标签、动画中间帧与终态、手动取消、reduced-motion，以及编辑器集成中选择态和面板保持不变；运行前端定向测试、`npm run typecheck`、`npm run lint:eslint`、`npm run check:encoding` 和 `git diff --check`。
|
||||
- 关联文档：`docs/【编辑器】画布Agent对话面板-2026-07-03.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/technical/【前端架构】图片画布编辑器前端拆分计划-2026-06-17.md`。
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-29 图集切片必须受前置容量和有界 CPU 保护
|
||||
|
||||
- 背景：图标与 UI 图集的 alpha 连通域识别会在 async handler 上同步执行；原始连通域合并采用全量两两比较，`64` 个输出限制又晚于排序、裁剪和 PNG 编码。碎块或噪点图会放大 CPU 与内存成本，手动拆分、图标自动拆分和 UI 提取都受影响。另一方面，图标与 UI 的 Alpha 尺寸恢复、provider 原图回读或透明图解码失败此前只记日志，仍会把不可信透明图持久化并拆分。
|
||||
|
||||
@@ -153,7 +153,7 @@
|
||||
- Agent 面板能读取当前工程会话列表；无历史会话时发送第一条消息会先创建“新对话”。支持新建会话、切换会话和删除当前会话；删除必须通过独立确认弹窗完成，不能在面板下方追加确认内容。
|
||||
- Agent 输入支持文本消息、附件消息和纯附件消息；附件选择弹窗可在“画布 / 素材库”之间切换，只展示图片类资源，最多选择 9 张。
|
||||
- 发送消息后，面板先展示本地用户消息和请求等待态，再应用普通 JSON 响应中的 `deltaMessages`；客户端取消等待只终止本次 transport 等待，不把已经确认入队的外部生成任务改成停止态。
|
||||
- Agent 工具任务完成并懒回填后，消息内缩略图只作纯预览，不显示名称也不点击聚焦图层；前端同时重新读取工程快照和素材库。对话入口触发生成时不创建“即将生成”画布占位，生成完成后由后端 `canvasCompletion` 落新图层。规划或工具失败时消息内必须保留可回读的失败状态和错误气泡，不能只弹一次性 toast 或返回瞬时 `errorMessage`。
|
||||
- Agent 工具任务完成并懒回填后，消息内缩略图不显示名称；前端通过编辑器作用域 Action Context 的 `refreshCanvas()` 直接重新读取工程快照和素材库，不从 Editor 经 Stage、Panel 和 MessageBubble 透传刷新 callback。图片、视频和音频结果携带有效 `resourceId` 时，在素材右键菜单显示“在画布中定位”，并通过同一 Context 的 `focusResource(resourceId)` 请求画布在 `420ms` 内平滑 fit 到对应图层；普通媒体卡片不声明按钮语义或 `tabIndex`，点击、Enter 和 Space 均不得触发定位，视频和音频的原生播放器交互保持独立。定位只改变 viewport，不选择图层、不切换工具或侧栏、不收起 Agent 面板，也不避让面板覆盖区。缺少 `resourceId` 时不显示定位菜单项，目标图层已删除时保持无动作。对话入口触发生成时不创建“即将生成”画布占位，生成完成后由后端 `canvasCompletion` 落新图层。规划或工具失败时消息内必须保留可回读的失败状态和错误气泡，不能只弹一次性 toast 或返回瞬时 `errorMessage`。
|
||||
- 画布 Agent 会话刷新后能从后端恢复会话标题、消息、附件和生成记录；前端不得根据本地临时状态伪造会话持久化结果。
|
||||
- 图片选中后的浮动工具栏按钮顺序固定为：快速编辑、分割线、裁扩按钮、去除背景按钮、UI设计图专属提取素材、角色图专属生成动画、分割线、重绘、下载按钮。裁扩通过画布边界拖拉完成，不再展示四边数值输入；默认自由比例，选择固定比例后拖拉边界保持对应比例，完成后在原素材旁边新增裁扩结果图层，扩展区域透明填充。去除背景调用同源 BFF `POST /api/editor/images/background-removals`；父流程解析并校验私有 OSS object key 后只调用一次唯一内部 `bgfilter-worker` 的 complex 链路，子 worker 负责签发 600 秒 URL、`N / Q` 限流和最多两次顺序 provider attempt，complex 失败不接入 fallback，成功二进制返回后仍由父流程完成最终持久化。有项目上下文时先在画布创建关闭面板的去背景生成占位，完成后由后端通过 `canvasCompletion` 把新 project resource 写入该占位并返回快照，无占位上下文时才用新的 project resource 引用替换当前图层。画布任务侧栏按“排队/生成中”和“已完成”分页，生成中排在排队前，生成中耗时从任务开始时间戳实时计算，排队中不计时；进行中任务只显示阶段文本和已用时，不显示百分比；完成态生成任务副标题显示用户提示词并单行截断；点击任务只聚焦对应画布内容，不激活生成面板或改变任务顺序，聚焦时必须预留图片上方工具栏、底部工具栏和可见生成对话框空间。UI设计图的提取素材必须先进入红框素材框选状态，默认启用矩形框选，右侧框选工具与快速编辑统一且可再次点击取消启用态，当前启用工具按钮必须保持高亮。素材提取面板必须在素材下方，使用与生成新素材一致的面板宽度和底部模型 / 按钮样式，提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选区域截图预览、固定模型 `gpt-image-2`、左下角计划规格 `1:1·1K/2K` 和 `提取 · N泥点` 按钮，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。至少框选一个区域后才可提交，前端把红色轮廓绘入原图后固定走 `gpt-image-2` 和自动决策纯色背景素材提取提示词。透明处理及拆分正常完成时，透明 spritesheet 和拆分素材都按后端快照保留为画布图层；透明处理失败时仅原图作为主结果，既不要求透明图也不要求切片；透明图成功但拆分失败时保留整张透明图并展示拆分告警。三种完成结果都以后端项目快照为准。
|
||||
- 重绘生成资源后，右侧出现新生成结果图层，并自动 fit 原图 + 新图，且重绘面板保持打开。
|
||||
|
||||
@@ -51,6 +51,10 @@
|
||||
|
||||
第三阶段以后，主视图仍是画布编排入口。继续拆分前应优先选择能形成稳定边界的深模块，避免把上传链路、DataTransfer、画布坐标和历史快照拆成互相回调的小碎片。
|
||||
|
||||
- `ImageCanvasActionsContext.ts` / `ImageCanvasActionsProvider.tsx`
|
||||
- 承载编辑器实例内深层组件向画布编排入口发起的窄动作协议；当前暴露按 `resourceId` 请求视口聚焦的 `focusResource(resourceId)`，以及重新读取当前工程快照并刷新素材库的 `refreshCanvas()`，不保存图层、视口或正式业务状态。
|
||||
- Provider 由 `ImageCanvasEditorView` 持有，Agent 工具结果刷新、生成结果素材右键菜单和任务侧栏直接消费 Context；不得为这些画布动作新增 `Stage → Panel → MessageBubble` props 链，也不得把 callback 或一次性命令注册进全局 Zustand store。生成结果普通点击以及视频 / 音频播放器操作不消费 `focusResource`。任务列表自身的入队后立即失效不属于画布刷新，继续保持独立。
|
||||
|
||||
## 第四阶段模块
|
||||
|
||||
- `ImageCanvasGenerationComposerView.tsx`
|
||||
@@ -151,9 +155,9 @@
|
||||
## 第十七阶段模块
|
||||
|
||||
- `useImageCanvasViewportControls.ts`
|
||||
- 承载画布视口控制：`viewport`、`canvasSize`、小地图投影、适合视图、中心缩放、普通滚轮按原始 `deltaX / deltaY` 二维平移、Shift 且 `deltaX = 0` 时的横向位移适配、Ctrl / Cmd 滚轮缩放、屏幕点到画布 / 世界坐标换算和小地图点击 / 拖拽移动视图。
|
||||
- 承载画布视口控制：`viewport`、`canvasSize`、小地图投影、适合视图、中心缩放、普通滚轮按原始 `deltaX / deltaY` 二维平移、Shift 且 `deltaX = 0` 时的横向位移适配、Ctrl / Cmd 滚轮缩放、屏幕点到画布 / 世界坐标换算、小地图点击 / 拖拽移动视图，以及可由普通视口写入或用户交互取消的程序化平滑聚焦动画。
|
||||
- 主视图继续负责图层拖拽、生成占位框拖拽、框选、多选、历史触发时机、上传 drop 分流和小地图 pointer down 事件；该 hook 只作为视口控制协调器，不接管画布完整 pointer 状态机。
|
||||
- 该 hook 用独立单测覆盖尺寸同步、适合视图、中心缩放、坐标换算、滚轮语义和小地图移动，为后续抽 `useImageCanvasStageInteractions` 预留更清晰的视口接口。
|
||||
- 该 hook 用独立单测覆盖尺寸同步、适合视图、中心缩放、坐标换算、滚轮语义、小地图移动、动画插值、重定向、手动取消和 reduced-motion，为后续抽 `useImageCanvasStageInteractions` 预留更清晰的视口接口。
|
||||
|
||||
## 第十八阶段模块
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 画布Agent对话面板
|
||||
|
||||
日期：`2026-07-23`
|
||||
日期：`2026-07-29`
|
||||
|
||||
## 定位与边界
|
||||
|
||||
@@ -50,7 +50,7 @@
|
||||
## 生成结果落画板（对现有占位规则的例外）
|
||||
|
||||
- 对话入口触发的生成**不创建"即将生成"画布占位**（区别于其余生成面板）；生成中状态由工具消息和外部任务状态承载。
|
||||
- 生成完成后：结果图按统一 placement 避让模型（视口中心就近、避开现有图层、32px 间距）落画板为新图层，同时登记到默认项目素材库；前端轮询到任务终态并重新读取会话后，以回填的轻量媒体引用显示纯缩略图并刷新工程快照与素材库，缩略图本身不显示名称也不承担图层跳转。
|
||||
- 生成完成后：结果图按统一 placement 避让模型（视口中心就近、避开现有图层、32px 间距）落画板为新图层，同时登记到默认项目素材库；前端轮询到任务终态或收到已经包含结果媒体的会话增量时，直接通过编辑器作用域 `ImageCanvasActionsContext.refreshCanvas()` 重新读取工程快照并刷新素材库，不经过 Stage、Agent 面板或消息组件逐层透传 callback。重新读取会话后，以回填的轻量媒体引用显示缩略图。图片、视频或音频结果携带有效 `resourceId` 时，右键素材后通过“在画布中定位”调用同一 Context 的 `focusResource(resourceId)`，请求画布视口在 `420ms` 内平滑 fit 到对应图层；普通媒体卡片不承载按钮语义、不进入键盘 Tab 顺序，普通点击以及 Enter / Space 均不触发画布聚焦，视频和音频的播放、暂停、拖动、音量等原生控制只操作播放器。定位不选中图层、不切换工具或侧栏、不收起 Agent 面板，也不为 Agent 面板预留可见区域。
|
||||
- 消息内生成结果缩略图必须携带并优先使用 `objectKey` / `assetObjectId`，前端通过 `ResolvedAssetImage` / `/api/assets/read-url` 换签后渲染，不能把裸 `/generated-*` 私有路径直接交给 `<img>`。
|
||||
- 既有编辑器 worker 通过 `canvasCompletion` 写回工程快照；刷新后由 external generation task 状态和会话懒回填恢复结果。
|
||||
- 该例外已同步登记在《生成类面板Lovart统一改造方案-2026-06-17》「画布占位落点」节。
|
||||
@@ -126,11 +126,11 @@
|
||||
1. 助手文本随普通 JSON 消息响应一次性返回；
|
||||
2. 消息请求等待态，以及工具任务的待确认、生成中、完成 / 失败状态；
|
||||
3. 工具/模型标注行（生成时显示模型名 + 图标）；
|
||||
4. 消息内生成结果缩略图（纯预览，不显示名称，不点击聚焦图层）；
|
||||
4. 消息内生成结果缩略图不显示名称；携带有效 `resourceId` 时可从素材右键菜单选择“在画布中定位”，以平滑视口动画定位对应画布图层；普通点击只保留媒体自身交互；
|
||||
5. 生成中的进行中动画；
|
||||
6. 错误气泡（失败/余额不足，带原因）；
|
||||
7. 普通消息请求等待期间禁用发送按钮，不提供客户端停止操作；前端持续等待后端响应，超过 120 秒但 POST 仍 pending 时在思考气泡中显示“仍在处理中，请耐心等待”，最终成功或失败后自动移除，避免后端已持久化消息但前端中断请求后产生会话状态错位。
|
||||
8. 桌面端右键消息正文可复制该条可见文本；右键消息附件或生成结果可下载素材，图片额外支持复制图片本体和“引用”到当前输入区。引用复用附件去重、9 张上限和发送链路；
|
||||
8. 桌面端右键消息正文可复制该条可见文本；右键消息附件或生成结果可下载素材，图片额外支持复制图片本体和“引用”到当前输入区，带有效 `resourceId` 的图片、视频和音频生成结果额外支持“在画布中定位”。引用复用附件去重、9 张上限和发送链路；
|
||||
9. 消息右键菜单遵循 Canva 式单实例交互：任一菜单已打开时，下一次右键必须先关闭旧菜单；新落点是消息正文或素材时再在新位置打开对应菜单，新落点没有右键动作时仅收起旧菜单，不允许多个消息菜单并存。复制、引用或下载成功后自动关闭菜单；失败时保留菜单和失败状态，避免错误无提示消失。
|
||||
|
||||
不做（明确排除，防止后人补齐）：
|
||||
@@ -172,6 +172,6 @@
|
||||
|
||||
- 打开画布 Agent 后任务侧栏应关闭，再次打开任务侧栏时 Agent 面板应关闭；素材 / 图层面板与 Agent 可同时展开，互不改写开关状态。
|
||||
- 发送消息时先本地追加用户消息，再应用 JSON 响应中的 `deltaMessages`；请求等待期间发送按钮保持禁用，前端不主动中断当前回合。
|
||||
- Agent 消息内生成结果缩略图只用于预览，不显示名称，也不点击跳转图层；轮询到任务终态并完成会话懒回填后统一刷新工程快照和素材库。
|
||||
- Agent 消息内生成结果缩略图不显示名称；轮询到任务终态并完成会话懒回填后，通过实例级 `refreshCanvas()` 统一刷新工程快照和素材库，禁止重新引入中间刷新 props。只允许从有效结果的素材右键菜单触发画布定位，普通点击以及视频 / 音频原生播放器操作不得触发；定位时必须保持 Agent 面板、当前图层选择、工具和侧栏不变，只平滑调整 viewport 到对应图层；旧消息缺少 `resourceId` 或目标图层已删除时不显示定位菜单项。
|
||||
- 对话内容可被用户选中复制；用户从输入框或对话内容点击回画布图层 / 生成器时，焦点应回到画布对象，Backspace / Delete 等画布快捷键继续生效。
|
||||
- 对话正文右键菜单只复制当前气泡展示的完整文本，隐藏的内部 system 文本不得进入菜单；素材右键菜单优先于正文菜单，私有素材继续通过既有读取链路换签或代理下载，不复制会过期的临时链接。
|
||||
|
||||
+6
-1
@@ -26,12 +26,16 @@ import { EditorAgentConversationPanelView } from './EditorAgentConversationPanel
|
||||
const createEditorProjectResourceMock = vi.hoisted(() => vi.fn());
|
||||
const uploadEditorMediaAssetFileMock = vi.hoisted(() => vi.fn());
|
||||
const probeImageFileDimensionsMock = vi.hoisted(() => vi.fn());
|
||||
const focusResourceMock = vi.fn();
|
||||
const refreshCanvasMock = vi.fn();
|
||||
|
||||
function render(ui: ReactElement) {
|
||||
return testingLibraryRender(ui, {
|
||||
wrapper: ({ children }: { children: ReactNode }) => (
|
||||
<ImageCanvasActionsProvider refreshCanvas={refreshCanvasMock}>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusResourceMock}
|
||||
refreshCanvas={refreshCanvasMock}
|
||||
>
|
||||
{children}
|
||||
</ImageCanvasActionsProvider>
|
||||
),
|
||||
@@ -197,6 +201,7 @@ function createPendingToolCallMessage(): EditorAgentMessage {
|
||||
|
||||
describe('EditorAgentConversationPanelView', () => {
|
||||
beforeEach(() => {
|
||||
focusResourceMock.mockReset();
|
||||
useImageCanvasContextStore.getState().setProjectId('project-1');
|
||||
uploadEditorMediaAssetFileMock.mockReset();
|
||||
uploadEditorMediaAssetFileMock.mockResolvedValue({
|
||||
|
||||
@@ -4,6 +4,7 @@ import { fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import type { EditorAgentMessage } from '@/packages/shared/src/contracts';
|
||||
import { ImageCanvasActionsProvider } from '@/src/components/image-editor/ImageCanvasActionsProvider.tsx';
|
||||
import { readAssetBytes } from '@/src/services/assetReadUrlService.ts';
|
||||
import { copyTextToClipboard } from '@/src/services/clipboard.ts';
|
||||
import {
|
||||
@@ -25,18 +26,25 @@ vi.mock('@/src/services/clipboard.ts', () => ({
|
||||
copyTextToClipboard: vi.fn(),
|
||||
}));
|
||||
|
||||
const focusResourceMock = vi.fn();
|
||||
|
||||
function renderMessage(
|
||||
message: EditorAgentMessage,
|
||||
onReferenceImage?: Parameters<typeof MessageBubble>[0]['onReferenceImage'],
|
||||
) {
|
||||
return render(
|
||||
<MessageBubble
|
||||
message={message}
|
||||
busyAction={null}
|
||||
onConfirmToolCall={vi.fn()}
|
||||
onCancelToolCall={vi.fn()}
|
||||
onReferenceImage={onReferenceImage}
|
||||
/>,
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusResourceMock}
|
||||
refreshCanvas={vi.fn()}
|
||||
>
|
||||
<MessageBubble
|
||||
message={message}
|
||||
busyAction={null}
|
||||
onConfirmToolCall={vi.fn()}
|
||||
onCancelToolCall={vi.fn()}
|
||||
onReferenceImage={onReferenceImage}
|
||||
/>
|
||||
</ImageCanvasActionsProvider>,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -124,7 +132,10 @@ describe('MessageBubble', () => {
|
||||
|
||||
it('closes the previous menu before another right click opens a new one', () => {
|
||||
render(
|
||||
<>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusResourceMock}
|
||||
refreshCanvas={vi.fn()}
|
||||
>
|
||||
<MessageBubble
|
||||
message={{
|
||||
id: 40,
|
||||
@@ -151,7 +162,7 @@ describe('MessageBubble', () => {
|
||||
onConfirmToolCall={vi.fn()}
|
||||
onCancelToolCall={vi.fn()}
|
||||
/>
|
||||
</>,
|
||||
</ImageCanvasActionsProvider>,
|
||||
);
|
||||
|
||||
const messages = screen.getAllByLabelText('Agent消息');
|
||||
@@ -369,11 +380,76 @@ describe('MessageBubble', () => {
|
||||
);
|
||||
|
||||
fireEvent.contextMenu(imageCards[1]!, { clientX: 50, clientY: 60 });
|
||||
expect(screen.queryByRole('menuitem', { name: '在画布中定位' })).toBeNull();
|
||||
expect(screen.getByRole('menuitem', { name: '引用' })).toBeTruthy();
|
||||
expect(screen.getByRole('menuitem', { name: '复制图片' })).toBeTruthy();
|
||||
expect(screen.getByRole('menuitem', { name: '下载图片' })).toBeTruthy();
|
||||
});
|
||||
|
||||
it('focuses generated image, video and audio only from their right-click menus', async () => {
|
||||
const { container } = renderMessage({
|
||||
id: 9,
|
||||
role: 'system',
|
||||
text: 'internal tool result',
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'generate_media',
|
||||
status: 'completed',
|
||||
args: {},
|
||||
displayArgs: {
|
||||
stringArgs: [],
|
||||
imageArgs: [],
|
||||
extras: { priceMudPoints: 1 },
|
||||
},
|
||||
images: [
|
||||
{
|
||||
resourceId: ' resource-image ',
|
||||
imageSrc: '/generated-editor-images/image.png',
|
||||
},
|
||||
],
|
||||
videos: [
|
||||
{
|
||||
resourceId: 'resource-video',
|
||||
videoSrc: '/generated-editor-videos/video.mp4',
|
||||
},
|
||||
],
|
||||
audios: [
|
||||
{
|
||||
resourceId: 'resource-audio',
|
||||
audioSrc: '/generated-editor-audios/audio.mp3',
|
||||
},
|
||||
],
|
||||
},
|
||||
createdAt: '2026-07-20T00:00:00Z',
|
||||
});
|
||||
await waitFor(() => expect(container.querySelector('video')).toBeTruthy());
|
||||
|
||||
const imageCard = container.querySelector('.grid.grid-cols-3 > div');
|
||||
const video = container.querySelector('video');
|
||||
const audio = container.querySelector('audio');
|
||||
fireEvent.click(imageCard!);
|
||||
fireEvent.click(video!);
|
||||
fireEvent.click(audio!);
|
||||
expect(focusResourceMock).not.toHaveBeenCalled();
|
||||
|
||||
const targets = [
|
||||
[imageCard, 'resource-image'],
|
||||
[video!.parentElement, 'resource-video'],
|
||||
[audio!.parentElement, 'resource-audio'],
|
||||
] as const;
|
||||
for (const [target, resourceId] of targets) {
|
||||
fireEvent.contextMenu(target!, { clientX: 30, clientY: 40 });
|
||||
fireEvent.click(screen.getByRole('menuitem', { name: '在画布中定位' }));
|
||||
expect(focusResourceMock).toHaveBeenLastCalledWith(resourceId);
|
||||
await waitFor(() =>
|
||||
expect(
|
||||
screen.queryByRole('menu', { name: '消息素材右键菜单' }),
|
||||
).toBeNull(),
|
||||
);
|
||||
}
|
||||
expect(focusResourceMock).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it('exports right-click images through the native HostBridge', async () => {
|
||||
const invoke = vi.fn(
|
||||
async (_command: string, args?: Record<string, unknown>) => {
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
} from '@/src/components/image-editor/EditorAgentConversation/common.ts';
|
||||
import { PendingToolCall } from '@/src/components/image-editor/EditorAgentConversation/PendingToolCall.tsx';
|
||||
import ToolCallView from '@/src/components/image-editor/EditorAgentConversation/ToolCallView.tsx';
|
||||
import { useImageCanvasActions } from '@/src/components/image-editor/ImageCanvasActionsContext.ts';
|
||||
|
||||
import { MessageBubbleRightClickMenu } from './MessageBubbleRightClickMenu.tsx';
|
||||
import { useRightClickMenu } from './useRightClickMenu.ts';
|
||||
@@ -73,12 +74,16 @@ export function MessageBubble({
|
||||
onJobCompleted,
|
||||
onReferenceImage,
|
||||
}: MessageBubbleProps) {
|
||||
const { focusResource } = useImageCanvasActions();
|
||||
const {
|
||||
rightClickMenu,
|
||||
openRightClickMenu,
|
||||
closeRightClickMenu,
|
||||
runRightClickAction,
|
||||
} = useRightClickMenu({ onReferenceImage });
|
||||
} = useRightClickMenu({
|
||||
onReferenceImage,
|
||||
onFocusResource: focusResource,
|
||||
});
|
||||
const systemErrorText =
|
||||
message.role === 'system' &&
|
||||
!message.toolCall &&
|
||||
|
||||
+42
-20
@@ -34,31 +34,36 @@ function actionLabel({
|
||||
resultAction: EditorAgentRightClickAction | null;
|
||||
result: 'success' | 'error' | null;
|
||||
}) {
|
||||
let actionState: 'pending' | 'success' | 'error' | null = null;
|
||||
if (pendingAction === action) {
|
||||
return action === EditorAgentRightClickAction.DownloadAsset
|
||||
? '下载中'
|
||||
: action === EditorAgentRightClickAction.ReferenceImage
|
||||
? '引用中'
|
||||
: '复制中';
|
||||
actionState = 'pending';
|
||||
} else if (resultAction === action) {
|
||||
actionState = result;
|
||||
}
|
||||
if (resultAction !== action) {
|
||||
|
||||
if (!actionState) {
|
||||
return idleLabel;
|
||||
}
|
||||
if (result === 'success') {
|
||||
return action === EditorAgentRightClickAction.DownloadAsset
|
||||
? '已下载'
|
||||
: action === EditorAgentRightClickAction.ReferenceImage
|
||||
? '已引用'
|
||||
: '已复制';
|
||||
|
||||
switch (action) {
|
||||
case EditorAgentRightClickAction.FocusCanvas:
|
||||
if (actionState === 'pending') return '定位中';
|
||||
if (actionState === 'success') return '已定位';
|
||||
return '定位失败';
|
||||
case EditorAgentRightClickAction.DownloadAsset:
|
||||
if (actionState === 'pending') return '下载中';
|
||||
if (actionState === 'success') return '已下载';
|
||||
return '下载失败';
|
||||
case EditorAgentRightClickAction.ReferenceImage:
|
||||
if (actionState === 'pending') return '引用中';
|
||||
if (actionState === 'success') return '已引用';
|
||||
return '引用失败';
|
||||
case EditorAgentRightClickAction.CopyText:
|
||||
case EditorAgentRightClickAction.CopyImage:
|
||||
if (actionState === 'pending') return '复制中';
|
||||
if (actionState === 'success') return '已复制';
|
||||
return '复制失败';
|
||||
}
|
||||
if (result === 'error') {
|
||||
return action === EditorAgentRightClickAction.DownloadAsset
|
||||
? '下载失败'
|
||||
: action === EditorAgentRightClickAction.ReferenceImage
|
||||
? '引用失败'
|
||||
: '复制失败';
|
||||
}
|
||||
return idleLabel;
|
||||
}
|
||||
|
||||
function assetDownloadLabel(asset: EditorAgentContextAsset) {
|
||||
@@ -175,6 +180,23 @@ export function MessageBubbleRightClickMenu({
|
||||
</button>
|
||||
) : (
|
||||
<>
|
||||
{target.asset.kind === 'generated_media' &&
|
||||
target.asset.resourceId?.trim() ? (
|
||||
<button
|
||||
type="button"
|
||||
role="menuitem"
|
||||
disabled={pendingAction !== null}
|
||||
onClick={() => onAction(EditorAgentRightClickAction.FocusCanvas)}
|
||||
>
|
||||
{actionLabel({
|
||||
action: EditorAgentRightClickAction.FocusCanvas,
|
||||
idleLabel: '在画布中定位',
|
||||
pendingAction,
|
||||
resultAction,
|
||||
result,
|
||||
})}
|
||||
</button>
|
||||
) : null}
|
||||
{target.asset.mediaType === 'image' ? (
|
||||
<>
|
||||
{contextAssetMediaSrc(target.asset).trim() ? (
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
/* @vitest-environment jsdom */
|
||||
|
||||
import { act, render, screen, waitFor } from '@testing-library/react';
|
||||
import {
|
||||
act,
|
||||
fireEvent,
|
||||
render,
|
||||
screen,
|
||||
waitFor,
|
||||
} from '@testing-library/react';
|
||||
import type { ReactElement, ReactNode } from 'react';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
@@ -10,6 +16,7 @@ import { ImageCanvasActionsProvider } from '@/src/components/image-editor/ImageC
|
||||
import ToolCallView from './ToolCallView.tsx';
|
||||
|
||||
const getExternalGenerationJobStatusMock = vi.hoisted(() => vi.fn());
|
||||
const focusResourceMock = vi.fn();
|
||||
const refreshCanvasMock = vi.fn();
|
||||
|
||||
vi.mock('@/src/services/external-generation', () => ({
|
||||
@@ -60,7 +67,10 @@ function createCompletedJobResponse(jobId: string) {
|
||||
function renderWithCanvasActions(ui: ReactElement) {
|
||||
return render(ui, {
|
||||
wrapper: ({ children }: { children: ReactNode }) => (
|
||||
<ImageCanvasActionsProvider refreshCanvas={refreshCanvasMock}>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusResourceMock}
|
||||
refreshCanvas={refreshCanvasMock}
|
||||
>
|
||||
{children}
|
||||
</ImageCanvasActionsProvider>
|
||||
),
|
||||
@@ -70,6 +80,7 @@ function renderWithCanvasActions(ui: ReactElement) {
|
||||
describe('ToolCallView', () => {
|
||||
beforeEach(() => {
|
||||
getExternalGenerationJobStatusMock.mockReset();
|
||||
focusResourceMock.mockReset();
|
||||
refreshCanvasMock.mockReset();
|
||||
});
|
||||
|
||||
@@ -165,4 +176,88 @@ describe('ToolCallView', () => {
|
||||
expect(refreshCanvasMock).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('keeps media clicks passive and exposes canvas resources to the right-click menu', () => {
|
||||
const onRightClickMenu = vi.fn();
|
||||
const { container } = renderWithCanvasActions(
|
||||
<ToolCallView
|
||||
toolCall={createToolCall({
|
||||
status: 'completed',
|
||||
externalJobId: null,
|
||||
images: [
|
||||
{
|
||||
resourceId: ' resource-image ',
|
||||
imageSrc: 'data:image/png;base64,aW1hZ2U=',
|
||||
},
|
||||
{
|
||||
imageSrc: 'data:image/png;base64,bGVnYWN5',
|
||||
},
|
||||
],
|
||||
videos: [
|
||||
{
|
||||
resourceId: 'resource-video',
|
||||
videoSrc: 'data:video/mp4;base64,dmlkZW8=',
|
||||
},
|
||||
],
|
||||
audios: [
|
||||
{
|
||||
resourceId: 'resource-audio',
|
||||
audioSrc: 'data:audio/wav;base64,YXVkaW8=',
|
||||
},
|
||||
],
|
||||
})}
|
||||
onRightClickMenu={onRightClickMenu}
|
||||
/>,
|
||||
);
|
||||
|
||||
const imageCards = container.querySelectorAll('.grid.grid-cols-3 > div');
|
||||
const video = container.querySelector('video');
|
||||
const audio = container.querySelector('audio');
|
||||
expect(imageCards).toHaveLength(2);
|
||||
expect(video).toBeTruthy();
|
||||
expect(audio).toBeTruthy();
|
||||
expect(imageCards[0]!.getAttribute('role')).toBeNull();
|
||||
expect(imageCards[0]!.getAttribute('tabindex')).toBeNull();
|
||||
|
||||
fireEvent.click(imageCards[0]!);
|
||||
fireEvent.keyDown(imageCards[0]!, { key: 'Enter' });
|
||||
fireEvent.keyDown(imageCards[0]!, { key: ' ', code: 'Space' });
|
||||
fireEvent.click(video!);
|
||||
fireEvent.click(audio!);
|
||||
expect(focusResourceMock).not.toHaveBeenCalled();
|
||||
expect(
|
||||
screen.queryByRole('button', {
|
||||
name: '在画布中定位Agent生成图片-1',
|
||||
}),
|
||||
).toBeNull();
|
||||
expect(video!.getAttribute('aria-label')).toBe('Agent生成视频-1');
|
||||
expect(audio!.getAttribute('aria-label')).toBe('Agent生成音频-1');
|
||||
|
||||
fireEvent.contextMenu(imageCards[0]!);
|
||||
fireEvent.contextMenu(video!.parentElement!);
|
||||
fireEvent.contextMenu(audio!.parentElement!);
|
||||
fireEvent.contextMenu(imageCards[1]!);
|
||||
|
||||
expect(onRightClickMenu.mock.calls.map((call) => call[1])).toEqual([
|
||||
expect.objectContaining({
|
||||
kind: 'generated_media',
|
||||
mediaType: 'image',
|
||||
resourceId: 'resource-image',
|
||||
}),
|
||||
expect.objectContaining({
|
||||
kind: 'generated_media',
|
||||
mediaType: 'video',
|
||||
resourceId: 'resource-video',
|
||||
}),
|
||||
expect.objectContaining({
|
||||
kind: 'generated_media',
|
||||
mediaType: 'audio',
|
||||
resourceId: 'resource-audio',
|
||||
}),
|
||||
expect.objectContaining({
|
||||
kind: 'generated_media',
|
||||
mediaType: 'image',
|
||||
resourceId: null,
|
||||
}),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -116,104 +116,118 @@ function ToolCallView({
|
||||
) : null}
|
||||
{toolCall.images.length ? (
|
||||
<div className="mt-2 grid grid-cols-3 gap-2">
|
||||
{toolCall.images.map((image, index) => (
|
||||
<div
|
||||
key={`${toolCall.toolName}-${image.resourceId ?? index}`}
|
||||
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
|
||||
onContextMenu={
|
||||
onRightClickMenu
|
||||
? (event) =>
|
||||
onRightClickMenu(event, {
|
||||
kind: 'generated_media',
|
||||
mediaType: 'image',
|
||||
mediaSrc: image.imageSrc,
|
||||
objectKey: image.objectKey,
|
||||
suggestedFileName: `Agent生成图片-${index + 1}`,
|
||||
})
|
||||
: undefined
|
||||
}
|
||||
>
|
||||
<ResolvedAssetImage
|
||||
src={image.thumbnailSrc ?? image.imageSrc}
|
||||
objectKey={image.objectKey}
|
||||
refreshKey={image.resourceId ?? toolCall.toolName}
|
||||
alt=""
|
||||
className="h-20 w-full object-cover"
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
{toolCall.images.map((image, index) => {
|
||||
const resourceId = image.resourceId?.trim() || null;
|
||||
return (
|
||||
<div
|
||||
key={`${toolCall.toolName}-${image.resourceId ?? index}`}
|
||||
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
|
||||
onContextMenu={
|
||||
onRightClickMenu
|
||||
? (event) =>
|
||||
onRightClickMenu(event, {
|
||||
kind: 'generated_media',
|
||||
mediaType: 'image',
|
||||
mediaSrc: image.imageSrc,
|
||||
objectKey: image.objectKey,
|
||||
resourceId,
|
||||
suggestedFileName: `Agent生成图片-${index + 1}`,
|
||||
})
|
||||
: undefined
|
||||
}
|
||||
>
|
||||
<ResolvedAssetImage
|
||||
src={image.thumbnailSrc ?? image.imageSrc}
|
||||
objectKey={image.objectKey}
|
||||
refreshKey={image.resourceId ?? toolCall.toolName}
|
||||
alt=""
|
||||
className="h-20 w-full object-cover"
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
) : null}
|
||||
{videos.length ? (
|
||||
<div className="mt-2 grid grid-cols-1 gap-2">
|
||||
{videos.map((video, index) => (
|
||||
<div
|
||||
key={`${toolCall.toolName}-${video.resourceId ?? video.objectKey ?? index}`}
|
||||
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
|
||||
onContextMenu={
|
||||
onRightClickMenu
|
||||
? (event) =>
|
||||
onRightClickMenu(event, {
|
||||
kind: 'generated_media',
|
||||
mediaType: 'video',
|
||||
mediaSrc: video.videoSrc,
|
||||
objectKey: video.objectKey,
|
||||
suggestedFileName: `Agent生成视频-${index + 1}`,
|
||||
})
|
||||
: undefined
|
||||
}
|
||||
>
|
||||
<ResolvedAssetVideo
|
||||
src={video.videoSrc}
|
||||
objectKey={video.objectKey}
|
||||
refreshKey={
|
||||
video.resourceId ?? video.objectKey ?? toolCall.toolName
|
||||
{videos.map((video, index) => {
|
||||
const resourceId = video.resourceId?.trim() || null;
|
||||
return (
|
||||
<div
|
||||
key={`${toolCall.toolName}-${video.resourceId ?? video.objectKey ?? index}`}
|
||||
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
|
||||
onContextMenu={
|
||||
onRightClickMenu
|
||||
? (event) =>
|
||||
onRightClickMenu(event, {
|
||||
kind: 'generated_media',
|
||||
mediaType: 'video',
|
||||
mediaSrc: video.videoSrc,
|
||||
objectKey: video.objectKey,
|
||||
resourceId,
|
||||
suggestedFileName: `Agent生成视频-${index + 1}`,
|
||||
})
|
||||
: undefined
|
||||
}
|
||||
poster={video.thumbnailSrc ?? undefined}
|
||||
controls
|
||||
playsInline
|
||||
preload="metadata"
|
||||
className="max-h-56 w-full bg-black object-contain"
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
>
|
||||
<ResolvedAssetVideo
|
||||
src={video.videoSrc}
|
||||
objectKey={video.objectKey}
|
||||
refreshKey={
|
||||
video.resourceId ?? video.objectKey ?? toolCall.toolName
|
||||
}
|
||||
poster={video.thumbnailSrc ?? undefined}
|
||||
controls
|
||||
playsInline
|
||||
preload="metadata"
|
||||
className="max-h-56 w-full bg-black object-contain"
|
||||
aria-label={`Agent生成视频-${index + 1}`}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
) : null}
|
||||
{audios.length ? (
|
||||
<div className="mt-2 space-y-2">
|
||||
{audios.map((audio, index) => (
|
||||
<div
|
||||
key={`${toolCall.toolName}-${audio.resourceId ?? audio.objectKey ?? index}`}
|
||||
className="flex items-center gap-2 rounded-xl border border-slate-200 bg-slate-50 p-2"
|
||||
onContextMenu={
|
||||
onRightClickMenu
|
||||
? (event) =>
|
||||
onRightClickMenu(event, {
|
||||
kind: 'generated_media',
|
||||
mediaType: 'audio',
|
||||
mediaSrc: audio.audioSrc,
|
||||
objectKey: audio.objectKey,
|
||||
suggestedFileName: `Agent生成音频-${index + 1}`,
|
||||
})
|
||||
: undefined
|
||||
}
|
||||
>
|
||||
<Volume2
|
||||
className="h-4 w-4 shrink-0 text-slate-500"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<ResolvedAssetAudio
|
||||
src={audio.audioSrc}
|
||||
objectKey={audio.objectKey}
|
||||
refreshKey={
|
||||
audio.resourceId ?? audio.objectKey ?? toolCall.toolName
|
||||
{audios.map((audio, index) => {
|
||||
const resourceId = audio.resourceId?.trim() || null;
|
||||
return (
|
||||
<div
|
||||
key={`${toolCall.toolName}-${audio.resourceId ?? audio.objectKey ?? index}`}
|
||||
className="flex items-center gap-2 rounded-xl border border-slate-200 bg-slate-50 p-2"
|
||||
onContextMenu={
|
||||
onRightClickMenu
|
||||
? (event) =>
|
||||
onRightClickMenu(event, {
|
||||
kind: 'generated_media',
|
||||
mediaType: 'audio',
|
||||
mediaSrc: audio.audioSrc,
|
||||
objectKey: audio.objectKey,
|
||||
resourceId,
|
||||
suggestedFileName: `Agent生成音频-${index + 1}`,
|
||||
})
|
||||
: undefined
|
||||
}
|
||||
controls
|
||||
preload="metadata"
|
||||
className="min-w-0 flex-1"
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
>
|
||||
<Volume2
|
||||
className="h-4 w-4 shrink-0 text-slate-500"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<ResolvedAssetAudio
|
||||
src={audio.audioSrc}
|
||||
objectKey={audio.objectKey}
|
||||
refreshKey={
|
||||
audio.resourceId ?? audio.objectKey ?? toolCall.toolName
|
||||
}
|
||||
controls
|
||||
preload="metadata"
|
||||
className="min-w-0 flex-1"
|
||||
aria-label={`Agent生成音频-${index + 1}`}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
@@ -6,6 +6,7 @@ export enum EditorAgentRightClickAction {
|
||||
CopyText = 'copy_text',
|
||||
CopyImage = 'copy_image',
|
||||
ReferenceImage = 'reference_image',
|
||||
FocusCanvas = 'focus_canvas',
|
||||
DownloadAsset = 'download_asset',
|
||||
}
|
||||
|
||||
@@ -20,6 +21,7 @@ export type EditorAgentContextAsset =
|
||||
mediaType: 'image' | 'video' | 'audio';
|
||||
mediaSrc: string;
|
||||
objectKey?: string | null;
|
||||
resourceId?: string | null;
|
||||
suggestedFileName: string;
|
||||
};
|
||||
|
||||
|
||||
+5
-1
@@ -23,6 +23,7 @@ import {
|
||||
useEditorAgentConversation,
|
||||
} from './useEditorAgentConversation.ts';
|
||||
|
||||
const focusResourceMock = vi.fn();
|
||||
const refreshCanvasMock = vi.fn();
|
||||
|
||||
function ImageCanvasActionsTestWrapper({
|
||||
@@ -31,7 +32,10 @@ function ImageCanvasActionsTestWrapper({
|
||||
children: ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<ImageCanvasActionsProvider refreshCanvas={refreshCanvasMock}>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusResourceMock}
|
||||
refreshCanvas={refreshCanvasMock}
|
||||
>
|
||||
{children}
|
||||
</ImageCanvasActionsProvider>
|
||||
);
|
||||
|
||||
@@ -232,10 +232,27 @@ async function downloadAsset(asset: EditorAgentContextAsset) {
|
||||
}
|
||||
}
|
||||
|
||||
function focusAssetOnCanvas(
|
||||
asset: EditorAgentContextAsset,
|
||||
onFocusResource?: (resourceId: string) => void,
|
||||
) {
|
||||
if (asset.kind !== 'generated_media' || !onFocusResource) {
|
||||
return false;
|
||||
}
|
||||
const resourceId = asset.resourceId?.trim();
|
||||
if (!resourceId) {
|
||||
return false;
|
||||
}
|
||||
onFocusResource(resourceId);
|
||||
return true;
|
||||
}
|
||||
|
||||
export function useRightClickMenu({
|
||||
onReferenceImage,
|
||||
onFocusResource,
|
||||
}: {
|
||||
onReferenceImage?: (asset: EditorAgentContextAsset) => boolean;
|
||||
onFocusResource?: (resourceId: string) => void;
|
||||
} = {}) {
|
||||
const [rightClickMenu, setRightClickMenu] =
|
||||
useState<RightClickMenuState | null>(null);
|
||||
@@ -289,10 +306,13 @@ export function useRightClickMenu({
|
||||
target.asset.mediaType === 'image' &&
|
||||
contextAssetMediaSrc(target.asset).trim()
|
||||
? (onReferenceImage?.(target.asset) ?? false)
|
||||
: action === EditorAgentRightClickAction.DownloadAsset &&
|
||||
: action === EditorAgentRightClickAction.FocusCanvas &&
|
||||
target.kind === 'asset'
|
||||
? await downloadAsset(target.asset)
|
||||
: false;
|
||||
? focusAssetOnCanvas(target.asset, onFocusResource)
|
||||
: action === EditorAgentRightClickAction.DownloadAsset &&
|
||||
target.kind === 'asset'
|
||||
? await downloadAsset(target.asset)
|
||||
: false;
|
||||
setRightClickMenu((current) =>
|
||||
current?.target === target && current.pendingAction === action
|
||||
? succeeded
|
||||
@@ -306,7 +326,7 @@ export function useRightClickMenu({
|
||||
: current,
|
||||
);
|
||||
},
|
||||
[onReferenceImage, rightClickMenu],
|
||||
[onFocusResource, onReferenceImage, rightClickMenu],
|
||||
);
|
||||
|
||||
return {
|
||||
|
||||
@@ -11,19 +11,25 @@ import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider';
|
||||
|
||||
describe('ImageCanvasActionsContext', () => {
|
||||
it('exposes the editor-scoped canvas actions', () => {
|
||||
const focusResource = vi.fn();
|
||||
const refreshCanvas = vi.fn();
|
||||
const { result } = renderHook(useImageCanvasActions, {
|
||||
wrapper: ({ children }: { children: ReactNode }) => (
|
||||
<ImageCanvasActionsProvider refreshCanvas={refreshCanvas}>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusResource}
|
||||
refreshCanvas={refreshCanvas}
|
||||
>
|
||||
{children}
|
||||
</ImageCanvasActionsProvider>
|
||||
),
|
||||
});
|
||||
|
||||
act(() => {
|
||||
result.current.focusResource('resource-a');
|
||||
result.current.refreshCanvas();
|
||||
});
|
||||
|
||||
expect(focusResource).toHaveBeenCalledWith('resource-a');
|
||||
expect(refreshCanvas).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { createContext, useContext } from 'react';
|
||||
|
||||
export type ImageCanvasActions = {
|
||||
focusResource: (resourceId: string) => void;
|
||||
refreshCanvas: () => void;
|
||||
};
|
||||
|
||||
|
||||
@@ -7,14 +7,16 @@ import {
|
||||
|
||||
export function ImageCanvasActionsProvider({
|
||||
children,
|
||||
focusResource,
|
||||
refreshCanvas,
|
||||
}: {
|
||||
children: ReactNode;
|
||||
focusResource: ImageCanvasActions['focusResource'];
|
||||
refreshCanvas: ImageCanvasActions['refreshCanvas'];
|
||||
}) {
|
||||
const actions = useMemo<ImageCanvasActions>(
|
||||
() => ({ refreshCanvas }),
|
||||
[refreshCanvas],
|
||||
() => ({ focusResource, refreshCanvas }),
|
||||
[focusResource, refreshCanvas],
|
||||
);
|
||||
|
||||
return (
|
||||
|
||||
@@ -20,7 +20,10 @@ import type { ImageCanvasTopbarViewProps } from './ImageCanvasTopbarView';
|
||||
function render(ui: ReactElement) {
|
||||
return testingLibraryRender(ui, {
|
||||
wrapper: ({ children }: { children: ReactNode }) => (
|
||||
<ImageCanvasActionsProvider refreshCanvas={vi.fn()}>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={vi.fn()}
|
||||
refreshCanvas={vi.fn()}
|
||||
>
|
||||
{children}
|
||||
</ImageCanvasActionsProvider>
|
||||
),
|
||||
|
||||
@@ -2545,9 +2545,15 @@ describe('ImageCanvasEditorView', () => {
|
||||
expect(screen.queryByRole('button', { name: '画布小地图' })).toBeNull();
|
||||
});
|
||||
|
||||
it('renders Agent generation thumbnails as passive previews', async () => {
|
||||
it('smoothly focuses an Agent generation resource without changing canvas selection or panels', async () => {
|
||||
const rafCallbacks: FrameRequestCallback[] = [];
|
||||
vi.spyOn(window, 'requestAnimationFrame').mockImplementation((callback) => {
|
||||
rafCallbacks.push(callback);
|
||||
return rafCallbacks.length;
|
||||
});
|
||||
vi.spyOn(window, 'cancelAnimationFrame').mockImplementation(() => {});
|
||||
enableEditorAgentSidebarForTest();
|
||||
const detail = createEditorAgentDetailWithGeneration('resource-puzzle');
|
||||
const detail = createEditorAgentDetailWithGeneration('resource-big-fish');
|
||||
listEditorAgentConversationsMock.mockResolvedValueOnce([
|
||||
createEditorAgentConversationSummary(),
|
||||
]);
|
||||
@@ -2559,10 +2565,44 @@ describe('ImageCanvasEditorView', () => {
|
||||
const messageLog = await screen.findByRole('log', {
|
||||
name: '画布 Agent 消息流',
|
||||
});
|
||||
const world = screen.getByTestId('image-canvas-world') as HTMLElement;
|
||||
const puzzleLayer = screen
|
||||
.getByAltText('画布图片：拼图素材')
|
||||
.closest('button')!;
|
||||
const bigFishLayer = screen
|
||||
.getByAltText('画布图片：大鱼素材')
|
||||
.closest('button')!;
|
||||
const initialTransform = world.style.transform;
|
||||
|
||||
expect(
|
||||
within(messageLog).queryByRole('button', { name: '生成结果' }),
|
||||
).toBeNull();
|
||||
puzzleLayer.classList.contains('image-canvas-editor__layer--selected'),
|
||||
).toBe(true);
|
||||
expect(
|
||||
bigFishLayer.classList.contains('image-canvas-editor__layer--selected'),
|
||||
).toBe(false);
|
||||
fireEvent.click(
|
||||
within(messageLog).getByRole('button', {
|
||||
name: '在画布中定位Agent生成图片-1',
|
||||
}),
|
||||
);
|
||||
expect(world.style.transform).toBe(initialTransform);
|
||||
|
||||
act(() => rafCallbacks.shift()?.(0));
|
||||
act(() => rafCallbacks.shift()?.(210));
|
||||
expect(world.style.transform).not.toBe(initialTransform);
|
||||
expect(world.style.transform).not.toBe(
|
||||
'translate(-840px, -246.5px) scale(1)',
|
||||
);
|
||||
|
||||
act(() => rafCallbacks.shift()?.(420));
|
||||
expect(world.style.transform).toBe('translate(-840px, -246.5px) scale(1)');
|
||||
expect(screen.getByLabelText('发送给画布 Agent')).toBeTruthy();
|
||||
expect(
|
||||
puzzleLayer.classList.contains('image-canvas-editor__layer--selected'),
|
||||
).toBe(true);
|
||||
expect(
|
||||
bigFishLayer.classList.contains('image-canvas-editor__layer--selected'),
|
||||
).toBe(false);
|
||||
expect(within(messageLog).queryByText('gpt-image-2')).toBeNull();
|
||||
expect(loadEditorProjectMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -454,6 +454,7 @@ export function ImageCanvasEditorView({
|
||||
const {
|
||||
viewport,
|
||||
setViewport,
|
||||
animateViewportTo,
|
||||
canvasSize,
|
||||
minimapModel,
|
||||
updateScaleFromCenter,
|
||||
@@ -876,7 +877,7 @@ export function ImageCanvasEditorView({
|
||||
const focusCanvasBounds = useCallback(
|
||||
(
|
||||
bounds: { x: number; y: number; width: number; height: number },
|
||||
options: { reserveDialogSpace?: boolean } = {},
|
||||
options: { reserveDialogSpace?: boolean; animate?: boolean } = {},
|
||||
) => {
|
||||
const nextViewport = fitViewportToBounds({
|
||||
bounds,
|
||||
@@ -891,9 +892,29 @@ export function ImageCanvasEditorView({
|
||||
},
|
||||
padding: TASK_FOCUS_PADDING,
|
||||
});
|
||||
if (options.animate) {
|
||||
animateViewportTo(nextViewport);
|
||||
return;
|
||||
}
|
||||
setViewport(nextViewport);
|
||||
},
|
||||
[canvasSize, setViewport],
|
||||
[animateViewportTo, canvasSize, setViewport],
|
||||
);
|
||||
const focusCanvasResource = useCallback(
|
||||
(resourceId: string) => {
|
||||
const normalizedResourceId = resourceId.trim();
|
||||
if (!normalizedResourceId) {
|
||||
return;
|
||||
}
|
||||
const targetLayer = layersRef.current.find(
|
||||
(layer) => layer.resourceId === normalizedResourceId,
|
||||
);
|
||||
if (!targetLayer) {
|
||||
return;
|
||||
}
|
||||
focusCanvasBounds(targetLayer, { animate: true });
|
||||
},
|
||||
[focusCanvasBounds],
|
||||
);
|
||||
const focusCanvasLayerById = useCallback(
|
||||
(layerId: string) => {
|
||||
@@ -2457,7 +2478,10 @@ export function ImageCanvasEditorView({
|
||||
};
|
||||
|
||||
return (
|
||||
<ImageCanvasActionsProvider refreshCanvas={refreshCanvas}>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusCanvasResource}
|
||||
refreshCanvas={refreshCanvas}
|
||||
>
|
||||
<ImageCanvasEditorShellView
|
||||
editorRootRef={editorRootRef}
|
||||
uploadInputRef={uploadInputRef}
|
||||
|
||||
@@ -16,12 +16,16 @@ import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider.tsx';
|
||||
import { ImageCanvasTaskSidebarView } from './ImageCanvasTaskSidebarView';
|
||||
import { useImageCanvasContextStore } from './useImageCanvasContextStore.ts';
|
||||
|
||||
const focusResourceMock = vi.fn();
|
||||
const refreshCanvasMock = vi.fn();
|
||||
|
||||
function render(ui: ReactElement) {
|
||||
return testingLibraryRender(ui, {
|
||||
wrapper: ({ children }: { children: ReactNode }) => (
|
||||
<ImageCanvasActionsProvider refreshCanvas={refreshCanvasMock}>
|
||||
<ImageCanvasActionsProvider
|
||||
focusResource={focusResourceMock}
|
||||
refreshCanvas={refreshCanvasMock}
|
||||
>
|
||||
{children}
|
||||
</ImageCanvasActionsProvider>
|
||||
),
|
||||
@@ -44,6 +48,7 @@ const listExternalGenerationTasksMock = vi.mocked(listExternalGenerationTasks);
|
||||
|
||||
beforeEach(() => {
|
||||
useImageCanvasContextStore.getState().setProjectId('project-1');
|
||||
focusResourceMock.mockReset();
|
||||
refreshCanvasMock.mockReset();
|
||||
listExternalGenerationTasksMock.mockClear();
|
||||
listExternalGenerationTasksMock.mockResolvedValue({
|
||||
|
||||
@@ -116,6 +116,7 @@ function expectViewport(viewport: CanvasViewport, expected: CanvasViewport) {
|
||||
describe('useImageCanvasViewportControls', () => {
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
vi.unstubAllGlobals();
|
||||
document.body.replaceChildren();
|
||||
});
|
||||
|
||||
@@ -158,6 +159,91 @@ describe('useImageCanvasViewportControls', () => {
|
||||
expect(captureCanvasHistory).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('animates, retargets from the current frame and lets manual input cancel the transition', () => {
|
||||
const rafCallbacks = new Map<number, FrameRequestCallback>();
|
||||
let nextFrameId = 0;
|
||||
vi.spyOn(window, 'requestAnimationFrame').mockImplementation((callback) => {
|
||||
nextFrameId += 1;
|
||||
rafCallbacks.set(nextFrameId, callback);
|
||||
return nextFrameId;
|
||||
});
|
||||
const cancelAnimationFrameSpy = vi
|
||||
.spyOn(window, 'cancelAnimationFrame')
|
||||
.mockImplementation((frameId) => {
|
||||
rafCallbacks.delete(frameId);
|
||||
});
|
||||
vi.stubGlobal(
|
||||
'matchMedia',
|
||||
vi.fn(() => ({ matches: false })) as unknown as typeof window.matchMedia,
|
||||
);
|
||||
const { result, viewportElement, captureCanvasHistory } =
|
||||
renderViewportControls();
|
||||
const targetViewport = { x: 120, y: -80, scale: 1.25 };
|
||||
const runNextFrame = (timestamp: number) => {
|
||||
const entry = rafCallbacks.entries().next().value as
|
||||
[number, FrameRequestCallback] | undefined;
|
||||
expect(entry).toBeTruthy();
|
||||
if (!entry) {
|
||||
return;
|
||||
}
|
||||
rafCallbacks.delete(entry[0]);
|
||||
entry[1](timestamp);
|
||||
};
|
||||
|
||||
act(() => {
|
||||
result.current.animateViewportTo(targetViewport);
|
||||
});
|
||||
expectViewport(result.current.viewport, {
|
||||
x: -260,
|
||||
y: 70,
|
||||
scale: 0.5,
|
||||
});
|
||||
|
||||
act(() => runNextFrame(0));
|
||||
act(() => runNextFrame(210));
|
||||
expect(result.current.viewport.x).toBeGreaterThan(-260);
|
||||
expect(result.current.viewport.x).toBeLessThan(120);
|
||||
expect(result.current.viewport.scale).toBeGreaterThan(0.5);
|
||||
expect(result.current.viewport.scale).toBeLessThan(1.25);
|
||||
|
||||
const retargetedViewport = { x: 420, y: 260, scale: 1.6 };
|
||||
act(() => {
|
||||
result.current.animateViewportTo(retargetedViewport);
|
||||
});
|
||||
expect(cancelAnimationFrameSpy).toHaveBeenCalled();
|
||||
act(() => runNextFrame(300));
|
||||
act(() => runNextFrame(720));
|
||||
expectViewport(result.current.viewport, retargetedViewport);
|
||||
expect(captureCanvasHistory).not.toHaveBeenCalled();
|
||||
|
||||
act(() => {
|
||||
result.current.animateViewportTo({ x: 600, y: 400, scale: 2 });
|
||||
});
|
||||
act(() => runNextFrame(800));
|
||||
act(() => {
|
||||
fireEvent.wheel(viewportElement, { deltaY: 40 });
|
||||
});
|
||||
expect(cancelAnimationFrameSpy).toHaveBeenCalled();
|
||||
expect(rafCallbacks.size).toBe(0);
|
||||
});
|
||||
|
||||
it('finishes viewport animation immediately when reduced motion is requested', () => {
|
||||
const requestAnimationFrameSpy = vi.spyOn(window, 'requestAnimationFrame');
|
||||
vi.stubGlobal(
|
||||
'matchMedia',
|
||||
vi.fn(() => ({ matches: true })) as unknown as typeof window.matchMedia,
|
||||
);
|
||||
const { result } = renderViewportControls();
|
||||
const targetViewport = { x: 20, y: 30, scale: 0.8 };
|
||||
|
||||
act(() => {
|
||||
result.current.animateViewportTo(targetViewport);
|
||||
});
|
||||
|
||||
expectViewport(result.current.viewport, targetViewport);
|
||||
expect(requestAnimationFrameSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('keeps ordinary mouse wheel vertical and maps Shift wheel to horizontal', () => {
|
||||
const { result, viewportElement } = renderViewportControls();
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import {
|
||||
type Dispatch,
|
||||
type RefObject,
|
||||
type SetStateAction,
|
||||
useCallback,
|
||||
useEffect,
|
||||
useMemo,
|
||||
@@ -34,6 +36,31 @@ export const DEFAULT_IMAGE_CANVAS_VIEWPORT: CanvasViewport = {
|
||||
scale: 0.5,
|
||||
};
|
||||
|
||||
export const IMAGE_CANVAS_VIEWPORT_ANIMATION_DURATION_MS = 420;
|
||||
|
||||
function easeOutCubic(progress: number) {
|
||||
return 1 - (1 - progress) ** 3;
|
||||
}
|
||||
|
||||
function interpolateViewport(
|
||||
start: CanvasViewport,
|
||||
target: CanvasViewport,
|
||||
progress: number,
|
||||
): CanvasViewport {
|
||||
return {
|
||||
x: start.x + (target.x - start.x) * progress,
|
||||
y: start.y + (target.y - start.y) * progress,
|
||||
scale: start.scale + (target.scale - start.scale) * progress,
|
||||
};
|
||||
}
|
||||
|
||||
function prefersReducedCanvasMotion() {
|
||||
return (
|
||||
typeof window.matchMedia === 'function' &&
|
||||
window.matchMedia('(prefers-reduced-motion: reduce)').matches
|
||||
);
|
||||
}
|
||||
|
||||
function isCanvasWheelInteractionTarget(target: EventTarget | null) {
|
||||
return (
|
||||
target instanceof Element &&
|
||||
@@ -54,9 +81,11 @@ export function useImageCanvasViewportControls({
|
||||
layers,
|
||||
captureCanvasHistory,
|
||||
}: UseImageCanvasViewportControlsOptions) {
|
||||
const [viewport, setViewport] = useState<CanvasViewport>(
|
||||
const [viewport, setViewportState] = useState<CanvasViewport>(
|
||||
DEFAULT_IMAGE_CANVAS_VIEWPORT,
|
||||
);
|
||||
const viewportRef = useRef(viewport);
|
||||
const viewportAnimationFrameRef = useRef<number | null>(null);
|
||||
const [canvasSize, setCanvasSize] = useState(DEFAULT_CANVAS_SIZE);
|
||||
const pendingMinimapDragRef = useRef<{
|
||||
dragState: Extract<DragState, { kind: 'minimap' }>;
|
||||
@@ -65,6 +94,66 @@ export function useImageCanvasViewportControls({
|
||||
} | null>(null);
|
||||
const minimapDragFrameRef = useRef<number | null>(null);
|
||||
|
||||
viewportRef.current = viewport;
|
||||
|
||||
const cancelViewportAnimation = useCallback(() => {
|
||||
if (viewportAnimationFrameRef.current === null) {
|
||||
return;
|
||||
}
|
||||
window.cancelAnimationFrame(viewportAnimationFrameRef.current);
|
||||
viewportAnimationFrameRef.current = null;
|
||||
}, []);
|
||||
|
||||
const setViewport = useCallback<Dispatch<SetStateAction<CanvasViewport>>>(
|
||||
(nextViewport) => {
|
||||
cancelViewportAnimation();
|
||||
const resolvedViewport =
|
||||
typeof nextViewport === 'function'
|
||||
? nextViewport(viewportRef.current)
|
||||
: nextViewport;
|
||||
viewportRef.current = resolvedViewport;
|
||||
setViewportState(resolvedViewport);
|
||||
},
|
||||
[cancelViewportAnimation],
|
||||
);
|
||||
|
||||
const animateViewportTo = useCallback(
|
||||
(targetViewport: CanvasViewport) => {
|
||||
cancelViewportAnimation();
|
||||
if (prefersReducedCanvasMotion()) {
|
||||
viewportRef.current = targetViewport;
|
||||
setViewportState(targetViewport);
|
||||
return;
|
||||
}
|
||||
|
||||
const startViewport = viewportRef.current;
|
||||
let startedAt: number | null = null;
|
||||
const step = (timestamp: number) => {
|
||||
startedAt ??= timestamp;
|
||||
const elapsedMs = Math.max(0, timestamp - startedAt);
|
||||
const progress = Math.min(
|
||||
1,
|
||||
elapsedMs / IMAGE_CANVAS_VIEWPORT_ANIMATION_DURATION_MS,
|
||||
);
|
||||
const nextViewport = interpolateViewport(
|
||||
startViewport,
|
||||
targetViewport,
|
||||
easeOutCubic(progress),
|
||||
);
|
||||
viewportRef.current = nextViewport;
|
||||
setViewportState(nextViewport);
|
||||
if (progress >= 1) {
|
||||
viewportAnimationFrameRef.current = null;
|
||||
return;
|
||||
}
|
||||
viewportAnimationFrameRef.current = window.requestAnimationFrame(step);
|
||||
};
|
||||
|
||||
viewportAnimationFrameRef.current = window.requestAnimationFrame(step);
|
||||
},
|
||||
[cancelViewportAnimation],
|
||||
);
|
||||
|
||||
const minimapModel = useMemo(
|
||||
() => createMinimapModel({ layers, viewport, canvasSize }),
|
||||
[canvasSize, layers, viewport],
|
||||
@@ -122,7 +211,13 @@ export function useImageCanvasViewportControls({
|
||||
}),
|
||||
);
|
||||
},
|
||||
[canvasSize.height, canvasSize.width, canvasViewportRef, captureCanvasHistory],
|
||||
[
|
||||
canvasSize.height,
|
||||
canvasSize.width,
|
||||
canvasViewportRef,
|
||||
captureCanvasHistory,
|
||||
setViewport,
|
||||
],
|
||||
);
|
||||
|
||||
const fitLayers = useCallback(
|
||||
@@ -143,7 +238,7 @@ export function useImageCanvasViewportControls({
|
||||
}
|
||||
setViewport(nextViewport);
|
||||
},
|
||||
[captureCanvasHistory, canvasSize, layers],
|
||||
[captureCanvasHistory, canvasSize, layers, setViewport],
|
||||
);
|
||||
|
||||
const resolveCanvasPoint = useCallback(
|
||||
@@ -200,7 +295,7 @@ export function useImageCanvasViewportControls({
|
||||
}),
|
||||
);
|
||||
},
|
||||
[canvasSize, minimapModel],
|
||||
[canvasSize, minimapModel, setViewport],
|
||||
);
|
||||
|
||||
const flushMinimapViewportDrag = useCallback(() => {
|
||||
@@ -219,7 +314,7 @@ export function useImageCanvasViewportControls({
|
||||
y: pendingDrag.clientY,
|
||||
}),
|
||||
);
|
||||
}, []);
|
||||
}, [setViewport]);
|
||||
|
||||
const updateViewportFromMinimapDrag = useCallback(
|
||||
(
|
||||
@@ -246,18 +341,19 @@ export function useImageCanvasViewportControls({
|
||||
);
|
||||
});
|
||||
},
|
||||
[],
|
||||
[setViewport],
|
||||
);
|
||||
|
||||
useEffect(
|
||||
() => () => {
|
||||
cancelViewportAnimation();
|
||||
if (minimapDragFrameRef.current !== null) {
|
||||
window.cancelAnimationFrame(minimapDragFrameRef.current);
|
||||
minimapDragFrameRef.current = null;
|
||||
}
|
||||
pendingMinimapDragRef.current = null;
|
||||
},
|
||||
[],
|
||||
[cancelViewportAnimation],
|
||||
);
|
||||
|
||||
const handleNativeWheel = useCallback(
|
||||
@@ -302,7 +398,7 @@ export function useImageCanvasViewportControls({
|
||||
}),
|
||||
);
|
||||
},
|
||||
[canvasViewportRef],
|
||||
[canvasViewportRef, setViewport],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
@@ -321,6 +417,7 @@ export function useImageCanvasViewportControls({
|
||||
return {
|
||||
viewport,
|
||||
setViewport,
|
||||
animateViewportTo,
|
||||
canvasSize,
|
||||
minimapModel,
|
||||
updateScaleFromCenter,
|
||||
|
||||
Reference in New Issue
Block a user