Feat/artagent 点击/右键功能 聚焦到画布上的素材 (#123)
Project CI / Frontend tests (push) Failing after 22s
Project CI / Repository checks (push) Successful in 1m10s
Project CI / Backend tests (push) Successful in 3m40s
Project CI / Native shell tests (push) Successful in 10m56s

- 为避免prop drilling把原有的刷新画布集中到一个useContext中,
   在此基础上实现画布聚焦, 并在画布agent中使用

![shotmd-1785482938-compressed.webp](/attachments/59d71873-79fc-40e8-a9dc-7830a5ad7f3c)

Reviewed-on: http://192.168.35.82/git/GenarrativeAI/Genarrative/pulls/123
Co-authored-by: 王德宇 <kvtodev@outlook.com>
Co-committed-by: 王德宇 <kvtodev@outlook.com>
This commit was merged in pull request #123.
This commit is contained in:
2026-07-31 17:25:35 +08:00
committed by 段舒康
parent 72acd0ec05
commit 386534a415
26 changed files with 1064 additions and 269 deletions
@@ -87,6 +87,26 @@
- 关联文档:`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
---
## 2026-07-31 画布 Agent 图片结果单击直接定位
- 背景:生成图片在对话中是静态缩略图,用户需要用更直接的方式回到对应画布图层;视频和音频仍有播放、拖动和音量等原生点击交互,不能共用该行为。
- 决策:携带有效 `resourceId` 的 Agent 生成图片在普通单击时直接调用实例级 `ImageCanvasActionsContext.focusResource(resourceId)`;无 `resourceId` 的旧图片保持无动作。视频和音频的普通点击仍只操作播放器,三类媒体均保留右键菜单的“在画布中定位”。图片卡片本次不新增按钮语义或键盘 Tab 停靠,Enter / Space 不触发定位。
- 影响范围:`ToolCallView`、消息气泡交互测试和画布 Agent 前端专题文档;不修改共享 DTO、后端 API、SpacetimeDB 或 viewport 动画语义。
- 验证方式:覆盖有效图片单击、旧图片无动作、视频 / 音频单击无定位及三类媒体右键定位;运行前端定向测试、`npm run typecheck``npm run lint:eslint``npm run check:encoding``git diff --check`
- 关联文档:`docs/【编辑器】画布Agent对话面板-2026-07-03.md``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
---
## 2026-07-30 画布 Agent 结果通过实例级 Action Context 刷新并从右键菜单定位资源
- 背景:画布 Agent 图片、视频和音频结果需要提供画布定位和结果完成后的工程刷新;若继续从舞台向面板、消息和工具结果逐层传 callback,会扩大现有 prop drilling,而把 callback 或瞬时命令放入全局 Zustand 又会引入多实例和卸载残留问题。直接把点击和定位语义附到生成媒体上还会让视频 / 音频的播放、暂停、拖动与音量操作误触发画布聚焦,并给原生媒体控件附加错误的定位标签。
- 决策:`ImageCanvasEditorView` 提供实例级 `ImageCanvasActionsContext`,暴露 `focusResource(resourceId)``refreshCanvas()`;Agent 工具结果刷新、生成结果右键菜单和任务侧栏直接消费对应动作,不新增中间 props,也不扩展现有只保存 `projectId` 的 Zustand store。`refreshCanvas()` 统一重新读取当前工程快照并刷新素材库,任务列表入队后的立即失效继续保持独立。带有效 `resourceId` 的图片、视频和音频生成结果只在素材右键菜单显示“在画布中定位”,普通媒体卡片不声明按钮语义或 `tabIndex`,点击、Enter 和 Space 均不触发定位;视频和音频原生播放器只使用描述媒体自身的标签,不承载定位标签或点击处理。`focusResource(resourceId)` 命中当前图层后只按完整画布 viewport 播放固定 `420ms` ease-out fit 动画,不改变图层选择、工具、侧栏或 Agent 面板;普通 viewport 写入和用户交互可取消动画,reduced-motion 直接完成,缺失 ID 或图层时无动作。
- 影响范围:图片画布 Action Context、viewport controls、Agent 结果媒体交互、前端测试和编辑器专题文档;不修改共享 DTO、后端 API 或 SpacetimeDB。
- 验证方式:覆盖 Context 作用域、三类媒体普通点击无动作与右键菜单分发、原生媒体控件无定位标签、动画中间帧与终态、手动取消、reduced-motion,以及编辑器集成中选择态和面板保持不变;运行前端定向测试、`npm run typecheck``npm run lint:eslint``npm run check:encoding``git diff --check`
- 关联文档:`docs/【编辑器】画布Agent对话面板-2026-07-03.md``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md``docs/technical/【前端架构】图片画布编辑器前端拆分计划-2026-06-17.md`
---
## 2026-07-29 图集切片必须受前置容量和有界 CPU 保护
- 背景:图标与 UI 图集的 alpha 连通域识别会在 async handler 上同步执行;原始连通域合并采用全量两两比较,`64` 个输出限制又晚于排序、裁剪和 PNG 编码。碎块或噪点图会放大 CPU 与内存成本,手动拆分、图标自动拆分和 UI 提取都受影响。另一方面,图标与 UI 的 Alpha 尺寸恢复、provider 原图回读或透明图解码失败此前只记日志,仍会把不可信透明图持久化并拆分。
@@ -157,7 +157,7 @@
- Agent 面板能读取当前工程会话列表;无历史会话时发送第一条消息会先创建“新对话”。支持新建会话、切换会话和删除当前会话;删除必须通过独立确认弹窗完成,不能在面板下方追加确认内容。
- Agent 输入支持文本消息和带附件的文本消息;不支持纯附件消息。附件选择弹窗可在“画布 / 素材库”之间切换,只展示图片类资源,最多选择 9 张。
- 发送消息后,面板先展示本地用户消息和请求等待态,再应用普通 JSON 响应中的 `deltaMessages`;客户端取消等待只终止本次 transport 等待,不把已经确认入队的外部生成任务改成停止态。
- Agent 工具任务完成并懒回填后,消息内缩略图只作纯预览,不显示名称也不点击聚焦图层;前端同时重新读取工程快照和素材库。对话入口触发生成时不创建“即将生成”画布占位,生成完成后由后端 `canvasCompletion` 落新图层。规划或工具失败时消息内必须保留可回读的失败状态和错误气泡,不能只弹一次性 toast 或返回瞬时 `errorMessage`
- Agent 工具任务完成并懒回填后,消息内缩略图不显示名称;前端通过编辑器作用域 Action Context 的 `refreshCanvas()` 直接重新读取工程快照和素材库,不从 Editor 经 Stage、Panel 和 MessageBubble 透传刷新 callback。图片、视频和音频结果携带有效 `resourceId` 时,在素材右键菜单显示“在画布中定位”;有效图片结果的普通单击也直接通过同一 Context 的 `focusResource(resourceId)` 请求画布在 `420ms` 内平滑 fit 到对应图层。结果卡片不声明按钮语义或 `tabIndex`Enter 和 Space 不得触发定位;视频和音频的普通点击及原生播放器交互保持独立。定位只改变 viewport,不选择图层、不切换工具或侧栏、不收起 Agent 面板,也不避让面板覆盖区。缺少 `resourceId` 时单击无动作且不显示定位菜单项,目标图层已删除时保持无动作。对话入口触发生成时不创建“即将生成”画布占位,生成完成后由后端 `canvasCompletion` 落新图层。规划或工具失败时消息内必须保留可回读的失败状态和错误气泡,不能只弹一次性 toast 或返回瞬时 `errorMessage`
- 画布 Agent 会话刷新后能从后端恢复会话标题、消息、附件和生成记录;前端不得根据本地临时状态伪造会话持久化结果。
- 图片选中后的浮动工具栏按钮顺序固定为:快速编辑、分割线、裁扩按钮、去除背景按钮、UI设计图专属提取素材、角色图专属生成动画、分割线、重绘、下载按钮。裁扩通过画布边界拖拉完成,不再展示四边数值输入;默认自由比例,选择固定比例后拖拉边界保持对应比例,完成后在原素材旁边新增裁扩结果图层,扩展区域透明填充。去除背景调用同源 BFF `POST /api/editor/images/background-removals`;父流程解析并校验私有 OSS object key 后只调用一次唯一内部 `bgfilter-worker` 的 complex 链路,子 worker 负责签发 600 秒 URL、`N / Q` 限流和最多两次顺序 provider attemptcomplex 失败不接入 fallback,成功二进制返回后仍由父流程完成最终持久化。有项目上下文时先在画布创建关闭面板的去背景生成占位,完成后由后端通过 `canvasCompletion` 把新 project resource 写入该占位并返回快照,无占位上下文时才用新的 project resource 引用替换当前图层。画布任务侧栏按“排队/生成中”和“已完成”分页,生成中排在排队前,生成中耗时从任务开始时间戳实时计算,排队中不计时;进行中任务只显示阶段文本和已用时,不显示百分比;完成态生成任务副标题显示用户提示词并单行截断;点击任务只聚焦对应画布内容,不激活生成面板或改变任务顺序,聚焦时必须预留图片上方工具栏、底部工具栏和可见生成对话框空间。UI设计图的提取素材必须先进入红框素材框选状态,默认启用矩形框选,右侧框选工具与快速编辑统一且可再次点击取消启用态,当前启用工具按钮必须保持高亮。素材提取面板必须在素材下方,使用与生成新素材一致的面板宽度和底部模型 / 按钮样式,提示语显示 `使用框选工具框选你希望从画面中提取的素材`,并展示按原图坐标准确裁剪的框选区域截图预览、固定模型 `gpt-image-2`、左下角计划规格 `1:1·1K/2K``提取 · N泥点` 按钮,不显示额外取消按钮;点击素材和面板以外的画布区域即退出 UI 素材提取。至少框选一个区域后才可提交,前端把红色轮廓绘入原图后固定走 `gpt-image-2` 和自动决策纯色背景素材提取提示词。透明处理及拆分正常完成时,透明 spritesheet 和拆分素材都按后端快照保留为画布图层;透明处理失败时仅原图作为主结果,既不要求透明图也不要求切片;透明图成功但拆分失败时保留整张透明图并展示拆分告警。三种完成结果都以后端项目快照为准。
- 重绘生成资源后,右侧出现新生成结果图层,并自动 fit 原图 + 新图,且重绘面板保持打开。
@@ -51,6 +51,10 @@
第三阶段以后,主视图仍是画布编排入口。继续拆分前应优先选择能形成稳定边界的深模块,避免把上传链路、DataTransfer、画布坐标和历史快照拆成互相回调的小碎片。
- `ImageCanvasActionsContext.ts` / `ImageCanvasActionsProvider.tsx`
- 承载编辑器实例内深层组件向画布编排入口发起的窄动作协议;当前暴露按 `resourceId` 请求视口聚焦的 `focusResource(resourceId)`,以及重新读取当前工程快照并刷新素材库的 `refreshCanvas()`,不保存图层、视口或正式业务状态。
- Provider 由 `ImageCanvasEditorView` 持有,Agent 工具结果刷新、生成结果素材右键菜单和任务侧栏直接消费 Context;不得为这些画布动作新增 `Stage → Panel → MessageBubble` props 链,也不得把 callback 或一次性命令注册进全局 Zustand store。生成结果普通点击以及视频 / 音频播放器操作不消费 `focusResource`。任务列表自身的入队后立即失效不属于画布刷新,继续保持独立。
## 第四阶段模块
- `ImageCanvasGenerationComposerView.tsx`
@@ -151,9 +155,9 @@
## 第十七阶段模块
- `useImageCanvasViewportControls.ts`
- 承载画布视口控制:`viewport``canvasSize`、小地图投影、适合视图、中心缩放、普通滚轮按原始 `deltaX / deltaY` 二维平移、Shift 且 `deltaX = 0` 时的横向位移适配、Ctrl / Cmd 滚轮缩放、屏幕点到画布 / 世界坐标换算小地图点击 / 拖拽移动视图。
- 承载画布视口控制:`viewport``canvasSize`、小地图投影、适合视图、中心缩放、普通滚轮按原始 `deltaX / deltaY` 二维平移、Shift 且 `deltaX = 0` 时的横向位移适配、Ctrl / Cmd 滚轮缩放、屏幕点到画布 / 世界坐标换算小地图点击 / 拖拽移动视图,以及可由普通视口写入或用户交互取消的程序化平滑聚焦动画
- 主视图继续负责图层拖拽、生成占位框拖拽、框选、多选、历史触发时机、上传 drop 分流和小地图 pointer down 事件;该 hook 只作为视口控制协调器,不接管画布完整 pointer 状态机。
- 该 hook 用独立单测覆盖尺寸同步、适合视图、中心缩放、坐标换算、滚轮语义小地图移动,为后续抽 `useImageCanvasStageInteractions` 预留更清晰的视口接口。
- 该 hook 用独立单测覆盖尺寸同步、适合视图、中心缩放、坐标换算、滚轮语义小地图移动、动画插值、重定向、手动取消和 reduced-motion,为后续抽 `useImageCanvasStageInteractions` 预留更清晰的视口接口。
## 第十八阶段模块
@@ -1,6 +1,6 @@
# 画布Agent对话面板
日期:`2026-07-23`
日期:`2026-07-29`
## 定位与边界
@@ -50,7 +50,7 @@
## 生成结果落画板(对现有占位规则的例外)
- 对话入口触发的生成**不创建"即将生成"画布占位**(区别于其余生成面板);生成中状态由工具消息和外部任务状态承载。
- 生成完成后:结果图按统一 placement 避让模型(视口中心就近、避开现有图层、32px 间距)落画板为新图层,同时登记到默认项目素材库;前端轮询到任务终态重新读取会话后,以回填的轻量媒体引用显示缩略图并刷新工程快照与素材库,缩略图本身不显示名称也不承担图层跳转
- 生成完成后:结果图按统一 placement 避让模型(视口中心就近、避开现有图层、32px 间距)落画板为新图层,同时登记到默认项目素材库;前端轮询到任务终态或收到已经包含结果媒体的会话增量时,直接通过编辑器作用域 `ImageCanvasActionsContext.refreshCanvas()` 重新读取工程快照并刷新素材库,不经过 Stage、Agent 面板或消息组件逐层透传 callback。重新读取会话后,以回填的轻量媒体引用显示缩略图。携带有效 `resourceId` 的图片结果在普通单击时直接调用同一 Context 的 `focusResource(resourceId)`;图片、视频或音频结果仍可从右键素材菜单选择“在画布中定位”。两种入口都请求画布视口在 `420ms` 内平滑 fit 到对应图层;同一 `resourceId` 对应多个画布图层时只聚焦当前图层顺序中的首个,不同时聚焦或轮询多个图层。结果卡片不进入键盘 Tab 顺序,Enter / Space 不触发画布聚焦,视频和音频的点击、播放、暂停、拖动、音量等原生交互只操作播放器。定位不选中图层、不切换工具或侧栏、不收起 Agent 面板,也不为 Agent 面板预留可见区域。定位返回 `{ successed, reason? }`:画布中无对应图层时为 `reason="not-found-on-canva"`,右键菜单显示「画布上不存在」;其余失败使用 `reason="other"`,仅显示「失败」
- 消息内生成结果缩略图必须携带并优先使用 `objectKey` / `assetObjectId`,前端通过 `ResolvedAssetImage` / `/api/assets/read-url` 换签后渲染,不能把裸 `/generated-*` 私有路径直接交给 `<img>`
- 既有编辑器 worker 通过 `canvasCompletion` 写回工程快照;刷新后由 external generation task 状态和会话懒回填恢复结果。
- 该例外已同步登记在《生成类面板Lovart统一改造方案-2026-06-17》「画布占位落点」节。
@@ -126,11 +126,11 @@
1. 助手文本随普通 JSON 消息响应一次性返回;
2. 消息请求等待态,以及工具任务的待确认、生成中、完成 / 失败状态;
3. 工具/模型标注行(生成时显示模型名 + 图标);
4. 消息内生成结果缩略图(纯预览,不显示名称,不点击聚焦图层)
4. 消息内生成结果缩略图不显示名称;携带有效 `resourceId` 的图片可单击定位,图片、视频和音频都可从素材右键菜单选择“在画布中定位”,以平滑视口动画定位对应画布图层;视频和音频的普通点击只保留媒体自身交互
5. 生成中的进行中动画;
6. 错误气泡(失败/余额不足,带原因);
7. 普通消息请求等待期间禁用发送按钮,不提供客户端停止操作;前端持续等待后端响应,超过 120 秒但 POST 仍 pending 时在思考气泡中显示“仍在处理中,请耐心等待”,最终成功或失败后自动移除,避免后端已持久化消息但前端中断请求后产生会话状态错位。
8. 桌面端右键消息正文可复制该条可见文本;右键消息附件或生成结果可下载素材,图片额外支持复制图片本体和“引用”到当前输入区。引用复用附件去重、9 张上限和发送链路;
8. 桌面端右键消息正文可复制该条可见文本;右键消息附件或生成结果可下载素材,图片额外支持复制图片本体和“引用”到当前输入区,带有效 `resourceId` 的图片、视频和音频生成结果额外支持“在画布中定位”。引用复用附件去重、9 张上限和发送链路;
9. 消息右键菜单遵循 Canva 式单实例交互:任一菜单已打开时,下一次右键必须先关闭旧菜单;新落点是消息正文或素材时再在新位置打开对应菜单,新落点没有右键动作时仅收起旧菜单,不允许多个消息菜单并存。复制、引用或下载成功后自动关闭菜单;失败时保留菜单和失败状态,避免错误无提示消失。
不做(明确排除,防止后人补齐):
@@ -172,6 +172,6 @@
- 打开画布 Agent 后任务侧栏应关闭,再次打开任务侧栏时 Agent 面板应关闭;素材 / 图层面板与 Agent 可同时展开,互不改写开关状态。
- 发送消息时先本地追加用户消息,再应用 JSON 响应中的 `deltaMessages`;请求等待期间发送按钮保持禁用,前端不主动中断当前回合。
- Agent 消息内生成结果缩略图只用于预览,不显示名称,也不点击跳转图层;轮询到任务终态并完成会话懒回填后统一刷新工程快照和素材库
- Agent 消息内生成结果缩略图不显示名称;轮询到任务终态并完成会话懒回填后,通过实例级 `refreshCanvas()` 统一刷新工程快照和素材库,禁止重新引入中间刷新 props。有效图片结果的普通单击和三类媒体的“在画布中定位”右键菜单项都可触发画布定位;视频 / 音频普通点击与原生播放器操作不得触发。定位时必须保持 Agent 面板、当前图层选择、工具和侧栏不变,只平滑调整 viewport 到对应图层;旧消息缺少 `resourceId` 时单击无动作且不显示定位菜单项,目标图层已删除时保持无动作
- 对话内容可被用户选中复制;用户从输入框或对话内容点击回画布图层 / 生成器时,焦点应回到画布对象,Backspace / Delete 等画布快捷键继续生效。
- 对话正文右键菜单只复制当前气泡展示的完整文本,隐藏的内部 system 文本不得进入菜单;素材右键菜单优先于正文菜单,私有素材继续通过既有读取链路换签或代理下载,不复制会过期的临时链接。
@@ -3,11 +3,12 @@
import {
act,
fireEvent,
render,
render as testingLibraryRender,
screen,
waitFor,
within,
} from '@testing-library/react';
import type { ReactElement, ReactNode } from 'react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type {
@@ -17,6 +18,7 @@ import type {
} from '@/packages/shared/src/contracts';
import type { EditorAgentConversationClient } from '@/src/components/image-editor/EditorAgentConversation/useEditorAgentConversation.ts';
import { EDITOR_AGENT_PATIENCE_NOTICE_DELAY_MS } from '@/src/components/image-editor/EditorAgentConversation/useEditorAgentConversation.ts';
import { ImageCanvasActionsProvider } from '@/src/components/image-editor/ImageCanvasActionsProvider.tsx';
import { useImageCanvasContextStore } from '@/src/components/image-editor/useImageCanvasContextStore.ts';
import { EditorAgentConversationPanelView } from './EditorAgentConversationPanelView.tsx';
@@ -24,6 +26,21 @@ import { EditorAgentConversationPanelView } from './EditorAgentConversationPanel
const createEditorProjectResourceMock = vi.hoisted(() => vi.fn());
const uploadEditorMediaAssetFileMock = vi.hoisted(() => vi.fn());
const probeImageFileDimensionsMock = vi.hoisted(() => vi.fn());
const focusResourceMock = vi.fn();
const refreshCanvasMock = vi.fn();
function render(ui: ReactElement) {
return testingLibraryRender(ui, {
wrapper: ({ children }: { children: ReactNode }) => (
<ImageCanvasActionsProvider
focusResource={focusResourceMock}
refreshCanvas={refreshCanvasMock}
>
{children}
</ImageCanvasActionsProvider>
),
});
}
const ATTACHMENT_PROMPT = '请参考附件';
vi.mock('@/src/services/image-editor/editorProjectClient.ts', async () => {
@@ -191,6 +208,7 @@ function createPendingToolCallMessage(): EditorAgentMessage {
describe('EditorAgentConversationPanelView', () => {
beforeEach(() => {
focusResourceMock.mockReset();
useImageCanvasContextStore.getState().setProjectId('project-1');
uploadEditorMediaAssetFileMock.mockReset();
uploadEditorMediaAssetFileMock.mockResolvedValue({
@@ -1809,14 +1827,11 @@ describe('EditorAgentConversationPanelView', () => {
resolveConfirmation = resolve;
}),
);
const onCanvasRefreshRequested = vi.fn();
render(
<EditorAgentConversationPanelView
open
onToggleOpen={vi.fn()}
client={client}
onCanvasRefreshRequested={onCanvasRefreshRequested}
/>,
);
@@ -1858,7 +1873,7 @@ describe('EditorAgentConversationPanelView', () => {
'internal completed tool output that must stay hidden',
),
).toBeNull();
expect(onCanvasRefreshRequested).not.toHaveBeenCalled();
expect(refreshCanvasMock).not.toHaveBeenCalled();
expect(screen.queryByRole('button', { name: '确认' })).toBeNull();
});
@@ -42,7 +42,6 @@ type EditorAgentConversationPanelViewProps = {
onToggleOpen: () => void;
layers?: CanvasLayer[];
assets?: EditorAsset[];
onCanvasRefreshRequested?: () => void;
// TODO refactor: move the task list update seperate
onConfirmSent?: () => void;
client?: EditorAgentConversationClient;
@@ -57,7 +56,6 @@ export function EditorAgentConversationPanelView({
onToggleOpen,
layers = [],
assets = [],
onCanvasRefreshRequested,
onConfirmSent,
client,
}: EditorAgentConversationPanelViewProps) {
@@ -93,7 +91,6 @@ export function EditorAgentConversationPanelView({
} = useEditorAgentConversation({
projectId: effectiveProjectId,
client,
onCanvasRefreshRequested,
onConfirmSent,
});
const [draftText, setDraftText] = useState('');
@@ -308,7 +305,6 @@ export function EditorAgentConversationPanelView({
onReferenceImage={referenceContextAsset}
onJobCompleted={() => {
void refreshActiveConversation();
onCanvasRefreshRequested?.();
}}
/>
))}
@@ -4,6 +4,7 @@ import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { EditorAgentMessage } from '@/packages/shared/src/contracts';
import { ImageCanvasActionsProvider } from '@/src/components/image-editor/ImageCanvasActionsProvider.tsx';
import { readAssetBytes } from '@/src/services/assetReadUrlService.ts';
import { copyTextToClipboard } from '@/src/services/clipboard.ts';
import {
@@ -25,24 +26,32 @@ vi.mock('@/src/services/clipboard.ts', () => ({
copyTextToClipboard: vi.fn(),
}));
const focusResourceMock = vi.fn();
function renderMessage(
message: EditorAgentMessage,
onReferenceImage?: Parameters<typeof MessageBubble>[0]['onReferenceImage'],
) {
return render(
<MessageBubble
message={message}
busyAction={null}
onConfirmToolCall={vi.fn()}
onCancelToolCall={vi.fn()}
onReferenceImage={onReferenceImage}
/>,
<ImageCanvasActionsProvider
focusResource={focusResourceMock}
refreshCanvas={vi.fn()}
>
<MessageBubble
message={message}
busyAction={null}
onConfirmToolCall={vi.fn()}
onCancelToolCall={vi.fn()}
onReferenceImage={onReferenceImage}
/>
</ImageCanvasActionsProvider>,
);
}
describe('MessageBubble', () => {
beforeEach(() => {
vi.clearAllMocks();
focusResourceMock.mockReturnValue({ successed: true });
vi.mocked(copyTextToClipboard).mockResolvedValue(true);
});
@@ -124,7 +133,10 @@ describe('MessageBubble', () => {
it('closes the previous menu before another right click opens a new one', () => {
render(
<>
<ImageCanvasActionsProvider
focusResource={focusResourceMock}
refreshCanvas={vi.fn()}
>
<MessageBubble
message={{
id: 40,
@@ -151,7 +163,7 @@ describe('MessageBubble', () => {
onConfirmToolCall={vi.fn()}
onCancelToolCall={vi.fn()}
/>
</>,
</ImageCanvasActionsProvider>,
);
const messages = screen.getAllByLabelText('Agent消息');
@@ -369,11 +381,99 @@ describe('MessageBubble', () => {
);
fireEvent.contextMenu(imageCards[1]!, { clientX: 50, clientY: 60 });
expect(screen.queryByRole('menuitem', { name: '在画布中定位' })).toBeNull();
expect(screen.getByRole('menuitem', { name: '引用' })).toBeTruthy();
expect(screen.getByRole('menuitem', { name: '复制图片' })).toBeTruthy();
expect(screen.getByRole('menuitem', { name: '下载图片' })).toBeTruthy();
});
it('focuses generated images on click and all generated media from their right-click menus', async () => {
const { container } = renderMessage({
id: 9,
role: 'system',
text: 'internal tool result',
attachments: [],
toolCall: {
toolName: 'generate_media',
status: 'completed',
args: {},
displayArgs: {
stringArgs: [],
imageArgs: [],
extras: { priceMudPoints: 1 },
},
images: [
{
resourceId: ' resource-image ',
imageSrc: '/generated-editor-images/image.png',
},
],
videos: [
{
resourceId: 'resource-video',
videoSrc: '/generated-editor-videos/video.mp4',
},
],
audios: [
{
resourceId: 'resource-audio',
audioSrc: '/generated-editor-audios/audio.mp3',
},
],
},
createdAt: '2026-07-20T00:00:00Z',
});
await waitFor(() => expect(container.querySelector('video')).toBeTruthy());
const imageCard = container.querySelector('.grid.grid-cols-3 > div');
const video = container.querySelector('video');
const audio = container.querySelector('audio');
fireEvent.click(imageCard!);
fireEvent.click(video!);
fireEvent.click(audio!);
expect(focusResourceMock).toHaveBeenCalledTimes(1);
expect(focusResourceMock).toHaveBeenCalledWith('resource-image');
const targets = [
[imageCard, 'resource-image'],
[video!.parentElement, 'resource-video'],
[audio!.parentElement, 'resource-audio'],
] as const;
for (const [target, resourceId] of targets) {
fireEvent.contextMenu(target!, { clientX: 30, clientY: 40 });
fireEvent.click(screen.getByRole('menuitem', { name: '在画布中定位' }));
expect(focusResourceMock).toHaveBeenLastCalledWith(resourceId);
await waitFor(() =>
expect(
screen.queryByRole('menu', { name: '消息素材右键菜单' }),
).toBeNull(),
);
}
expect(focusResourceMock).toHaveBeenCalledTimes(4);
focusResourceMock.mockReturnValueOnce({
successed: false,
reason: 'not-found-on-canva',
});
fireEvent.contextMenu(imageCard!, { clientX: 30, clientY: 40 });
fireEvent.click(screen.getByRole('menuitem', { name: '在画布中定位' }));
expect(focusResourceMock).toHaveBeenLastCalledWith('resource-image');
expect(
screen.getByRole('menuitem', { name: '画布上不存在' }),
).toBeTruthy();
fireEvent.keyDown(window, { key: 'Escape' });
focusResourceMock.mockReturnValueOnce({
successed: false,
reason: 'other',
});
fireEvent.contextMenu(imageCard!, { clientX: 30, clientY: 40 });
fireEvent.click(screen.getByRole('menuitem', { name: '在画布中定位' }));
expect(screen.getByRole('menuitem', { name: '失败' })).toBeTruthy();
});
it('exports right-click images through the native HostBridge', async () => {
const invoke = vi.fn(
async (_command: string, args?: Record<string, unknown>) => {
@@ -9,6 +9,7 @@ import {
} from '@/src/components/image-editor/EditorAgentConversation/common.ts';
import { PendingToolCall } from '@/src/components/image-editor/EditorAgentConversation/PendingToolCall.tsx';
import ToolCallView from '@/src/components/image-editor/EditorAgentConversation/ToolCallView.tsx';
import { useImageCanvasActions } from '@/src/components/image-editor/ImageCanvasActionsContext.ts';
import { MessageBubbleRightClickMenu } from './MessageBubbleRightClickMenu.tsx';
import { useRightClickMenu } from './useRightClickMenu.ts';
@@ -73,12 +74,16 @@ export function MessageBubble({
onJobCompleted,
onReferenceImage,
}: MessageBubbleProps) {
const { focusResource } = useImageCanvasActions();
const {
rightClickMenu,
openRightClickMenu,
closeRightClickMenu,
runRightClickAction,
} = useRightClickMenu({ onReferenceImage });
} = useRightClickMenu({
onReferenceImage,
onFocusResource: focusResource,
});
const systemErrorText =
message.role === 'system' &&
!message.toolCall &&
@@ -1,6 +1,8 @@
import { useEffect, useLayoutEffect, useRef, useState } from 'react';
import { createPortal } from 'react-dom';
import type { ImageCanvasActionResult } from '@/src/components/image-editor/ImageCanvasActionsContext.ts';
import {
contextAssetMediaSrc,
type EditorAgentContextAsset,
@@ -14,7 +16,7 @@ type MessageBubbleRightClickMenuProps = {
target: RightClickMenuTarget;
pendingAction: EditorAgentRightClickAction | null;
resultAction: EditorAgentRightClickAction | null;
result: 'success' | 'error' | null;
result: ImageCanvasActionResult | null;
onAction: (action: EditorAgentRightClickAction) => void;
onClose: () => void;
};
@@ -32,33 +34,48 @@ function actionLabel({
idleLabel: string;
pendingAction: EditorAgentRightClickAction | null;
resultAction: EditorAgentRightClickAction | null;
result: 'success' | 'error' | null;
result: ImageCanvasActionResult | null;
}) {
if (pendingAction === action) {
return action === EditorAgentRightClickAction.DownloadAsset
? '下载中'
: action === EditorAgentRightClickAction.ReferenceImage
? '引用中'
: '复制中';
switch (action) {
case EditorAgentRightClickAction.FocusCanvas:
return '定位中';
case EditorAgentRightClickAction.DownloadAsset:
return '下载中';
case EditorAgentRightClickAction.ReferenceImage:
return '引用中';
case EditorAgentRightClickAction.CopyText:
case EditorAgentRightClickAction.CopyImage:
return '复制中';
}
}
if (resultAction !== action) {
const actionResult = resultAction === action ? result : null;
if (!actionResult) {
return idleLabel;
}
if (result === 'success') {
return action === EditorAgentRightClickAction.DownloadAsset
? '已下载'
: action === EditorAgentRightClickAction.ReferenceImage
? '已引用'
: '已复制';
switch (action) {
case EditorAgentRightClickAction.FocusCanvas:
if (actionResult.successed) return '已定位';
switch (actionResult.reason) {
case 'not-found-on-canva':
return '画布上不存在';
case 'other':
default:
return '失败';
}
case EditorAgentRightClickAction.DownloadAsset:
if (actionResult.successed) return '已下载';
return '下载失败';
case EditorAgentRightClickAction.ReferenceImage:
if (actionResult.successed) return '已引用';
return '引用失败';
case EditorAgentRightClickAction.CopyText:
case EditorAgentRightClickAction.CopyImage:
if (actionResult.successed) return '已复制';
return '复制失败';
}
if (result === 'error') {
return action === EditorAgentRightClickAction.DownloadAsset
? '下载失败'
: action === EditorAgentRightClickAction.ReferenceImage
? '引用失败'
: '复制失败';
}
return idleLabel;
}
function assetDownloadLabel(asset: EditorAgentContextAsset) {
@@ -175,6 +192,23 @@ export function MessageBubbleRightClickMenu({
</button>
) : (
<>
{target.asset.kind === 'generated_media' &&
target.asset.resourceId?.trim() ? (
<button
type="button"
role="menuitem"
disabled={pendingAction !== null}
onClick={() => onAction(EditorAgentRightClickAction.FocusCanvas)}
>
{actionLabel({
action: EditorAgentRightClickAction.FocusCanvas,
idleLabel: '在画布中定位',
pendingAction,
resultAction,
result,
})}
</button>
) : null}
{target.asset.mediaType === 'image' ? (
<>
{contextAssetMediaSrc(target.asset).trim() ? (
@@ -1,13 +1,23 @@
/* @vitest-environment jsdom */
import { act, render, screen, waitFor } from '@testing-library/react';
import {
act,
fireEvent,
render,
screen,
waitFor,
} from '@testing-library/react';
import type { ReactElement, ReactNode } from 'react';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import type { EditorAgentToolCall } from '@/packages/shared/src/contracts';
import { ImageCanvasActionsProvider } from '@/src/components/image-editor/ImageCanvasActionsProvider.tsx';
import ToolCallView from './ToolCallView.tsx';
const getExternalGenerationJobStatusMock = vi.hoisted(() => vi.fn());
const focusResourceMock = vi.fn();
const refreshCanvasMock = vi.fn();
vi.mock('@/src/services/external-generation', () => ({
getExternalGenerationJobStatus: getExternalGenerationJobStatusMock,
@@ -54,9 +64,24 @@ function createCompletedJobResponse(jobId: string) {
};
}
function renderWithCanvasActions(ui: ReactElement) {
return render(ui, {
wrapper: ({ children }: { children: ReactNode }) => (
<ImageCanvasActionsProvider
focusResource={focusResourceMock}
refreshCanvas={refreshCanvasMock}
>
{children}
</ImageCanvasActionsProvider>
),
});
}
describe('ToolCallView', () => {
beforeEach(() => {
getExternalGenerationJobStatusMock.mockReset();
focusResourceMock.mockReset();
refreshCanvasMock.mockReset();
});
it.each([
@@ -65,14 +90,13 @@ describe('ToolCallView', () => {
] as const)(
'keeps the server %s state when the pending response arrives late',
async (status, statusLabel, error) => {
const pendingResponse = createDeferred<
ReturnType<typeof createCompletedJobResponse>
>();
const pendingResponse =
createDeferred<ReturnType<typeof createCompletedJobResponse>>();
getExternalGenerationJobStatusMock.mockReturnValueOnce(
pendingResponse.promise,
);
const onJobCompleted = vi.fn();
const { rerender } = render(
const { rerender } = renderWithCanvasActions(
<ToolCallView
toolCall={createToolCall()}
onJobCompleted={onJobCompleted}
@@ -101,21 +125,20 @@ describe('ToolCallView', () => {
expect(screen.getByText(statusLabel)).toBeTruthy();
expect(screen.queryByText('已完成')).toBeNull();
expect(onJobCompleted).not.toHaveBeenCalled();
expect(refreshCanvasMock).not.toHaveBeenCalled();
},
);
it('ignores job A after switching to job B and completes job B once', async () => {
const jobAResponse = createDeferred<
ReturnType<typeof createCompletedJobResponse>
>();
const jobBResponse = createDeferred<
ReturnType<typeof createCompletedJobResponse>
>();
const jobAResponse =
createDeferred<ReturnType<typeof createCompletedJobResponse>>();
const jobBResponse =
createDeferred<ReturnType<typeof createCompletedJobResponse>>();
getExternalGenerationJobStatusMock.mockImplementation((jobId: string) =>
jobId === 'job-a' ? jobAResponse.promise : jobBResponse.promise,
);
const onJobCompleted = vi.fn();
const { rerender } = render(
const { rerender } = renderWithCanvasActions(
<ToolCallView
toolCall={createToolCall()}
onJobCompleted={onJobCompleted}
@@ -150,5 +173,93 @@ describe('ToolCallView', () => {
expect(await screen.findByText('已完成')).toBeTruthy();
expect(onJobCompleted).toHaveBeenCalledTimes(1);
expect(refreshCanvasMock).toHaveBeenCalledTimes(1);
});
it('focuses image clicks only and exposes canvas resources to the right-click menu', () => {
const onRightClickMenu = vi.fn();
const { container } = renderWithCanvasActions(
<ToolCallView
toolCall={createToolCall({
status: 'completed',
externalJobId: null,
images: [
{
resourceId: ' resource-image ',
imageSrc: 'data:image/png;base64,aW1hZ2U=',
},
{
imageSrc: 'data:image/png;base64,bGVnYWN5',
},
],
videos: [
{
resourceId: 'resource-video',
videoSrc: 'data:video/mp4;base64,dmlkZW8=',
},
],
audios: [
{
resourceId: 'resource-audio',
audioSrc: 'data:audio/wav;base64,YXVkaW8=',
},
],
})}
onRightClickMenu={onRightClickMenu}
/>,
);
const imageCards = container.querySelectorAll('.grid.grid-cols-3 > div');
const video = container.querySelector('video');
const audio = container.querySelector('audio');
expect(imageCards).toHaveLength(2);
expect(video).toBeTruthy();
expect(audio).toBeTruthy();
expect(imageCards[0]!.getAttribute('role')).toBeNull();
expect(imageCards[0]!.getAttribute('tabindex')).toBeNull();
fireEvent.click(imageCards[0]!);
fireEvent.keyDown(imageCards[0]!, { key: 'Enter' });
fireEvent.keyDown(imageCards[0]!, { key: ' ', code: 'Space' });
fireEvent.click(imageCards[1]!);
fireEvent.click(video!);
fireEvent.click(audio!);
expect(focusResourceMock).toHaveBeenCalledTimes(1);
expect(focusResourceMock).toHaveBeenCalledWith('resource-image');
expect(
screen.queryByRole('button', {
name: '在画布中定位Agent生成图片-1',
}),
).toBeNull();
expect(video!.getAttribute('aria-label')).toBe('Agent生成视频-1');
expect(audio!.getAttribute('aria-label')).toBe('Agent生成音频-1');
fireEvent.contextMenu(imageCards[0]!);
fireEvent.contextMenu(video!.parentElement!);
fireEvent.contextMenu(audio!.parentElement!);
fireEvent.contextMenu(imageCards[1]!);
expect(onRightClickMenu.mock.calls.map((call) => call[1])).toEqual([
expect.objectContaining({
kind: 'generated_media',
mediaType: 'image',
resourceId: 'resource-image',
}),
expect.objectContaining({
kind: 'generated_media',
mediaType: 'video',
resourceId: 'resource-video',
}),
expect.objectContaining({
kind: 'generated_media',
mediaType: 'audio',
resourceId: 'resource-audio',
}),
expect.objectContaining({
kind: 'generated_media',
mediaType: 'image',
resourceId: null,
}),
]);
});
});
@@ -3,6 +3,7 @@ import { useEffect, useRef, useState } from 'react';
import type { EditorAgentToolCall } from '@/packages/shared/src/contracts';
import { editorAgentToolLabel } from '@/src/components/image-editor/EditorAgentConversation/toolCallPresentation.ts';
import { useImageCanvasActions } from '@/src/components/image-editor/ImageCanvasActionsContext.ts';
import { ResolvedAssetAudio } from '@/src/components/ResolvedAssetAudio.tsx';
import { ResolvedAssetImage } from '@/src/components/ResolvedAssetImage.tsx';
import { ResolvedAssetVideo } from '@/src/components/ResolvedAssetVideo.tsx';
@@ -10,6 +11,57 @@ import { getExternalGenerationJobStatus } from '@/src/services/external-generati
import type { RightClickMenuHandler } from './common.ts';
type ToolCallDisplayStatus = 'pending' | 'completed' | 'failed' | 'cancelled';
function getInitialDisplayStatus(
status: EditorAgentToolCall['status'],
): ToolCallDisplayStatus {
switch (status) {
case 'completed':
case 'failed':
case 'cancelled':
return status;
case 'not_completed':
return 'pending';
}
}
function getStatusPresentation(
displayStatus: ToolCallDisplayStatus,
hasJobId: boolean,
) {
switch (displayStatus) {
case 'completed':
return {
label: '已完成',
icon: <Check className="h-3.5 w-3.5" aria-hidden="true" />,
};
case 'failed':
return {
label: '失败',
icon: <X className="h-3.5 w-3.5" aria-hidden="true" />,
};
case 'cancelled':
return {
label: '已取消',
icon: <X className="h-3.5 w-3.5" aria-hidden="true" />,
};
case 'pending':
if (hasJobId) {
return {
label: '执行中',
icon: (
<Loader2 className="h-3.5 w-3.5 animate-spin" aria-hidden="true" />
),
};
}
return {
label: '待确认',
icon: <ImageIcon className="h-3.5 w-3.5" aria-hidden="true" />,
};
}
}
function ToolCallView({
toolCall,
onJobCompleted,
@@ -19,27 +71,26 @@ function ToolCallView({
onJobCompleted?: () => void;
onRightClickMenu?: RightClickMenuHandler;
}) {
const { focusResource, refreshCanvas } = useImageCanvasActions();
const videos = toolCall.videos ?? [];
const audios = toolCall.audios ?? [];
const jobId = toolCall.externalJobId?.trim() || null;
const initialDisplayStatus =
toolCall.status === 'completed'
? 'completed'
: toolCall.status === 'failed'
? 'failed'
: toolCall.status === 'cancelled'
? 'cancelled'
: 'pending';
const initialDisplayStatus = getInitialDisplayStatus(toolCall.status);
const initialDisplayError = toolCall.error ?? null;
const shouldPoll = toolCall.status === 'not_completed' && Boolean(jobId);
const [displayStatus, setDisplayStatus] = useState(initialDisplayStatus);
const [displayError, setDisplayError] =
useState<string | null>(initialDisplayError);
const [displayError, setDisplayError] = useState<string | null>(
initialDisplayError,
);
const terminalNotifiedRef = useRef(false);
const onJobCompletedRef = useRef(onJobCompleted);
const refreshCanvasRef = useRef(refreshCanvas);
useEffect(() => {
onJobCompletedRef.current = onJobCompleted;
}, [onJobCompleted]);
useEffect(() => {
refreshCanvasRef.current = refreshCanvas;
}, [refreshCanvas]);
useEffect(() => {
setDisplayStatus(initialDisplayStatus);
setDisplayError(initialDisplayError);
@@ -59,6 +110,7 @@ function ToolCallView({
setDisplayError(response.job.error ?? null);
if (!terminalNotifiedRef.current) {
terminalNotifiedRef.current = true;
refreshCanvasRef.current();
onJobCompletedRef.current?.();
}
return;
@@ -76,137 +128,139 @@ function ToolCallView({
// 轮询只允许新的 job/status source 重置;其余值通过当前 source 对应的闭包读取。
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [initialDisplayError, initialDisplayStatus, jobId, shouldPoll]);
const isCancelled = displayStatus === 'cancelled';
const isCompleted = displayStatus === 'completed';
const isFailed = displayStatus === 'failed';
const isExecuting = displayStatus === 'pending' && Boolean(jobId);
const statusLabel = isCompleted
? '已完成'
: isFailed
? '失败'
: isCancelled
? '已取消'
: isExecuting
? '执行中'
: '待确认';
const statusPresentation = getStatusPresentation(
displayStatus,
Boolean(jobId),
);
return (
<div className="rounded-lg border border-slate-200 bg-white/80 p-2 text-xs text-slate-600">
<div className="flex items-center gap-2">
{isExecuting ? (
<Loader2 className="h-3.5 w-3.5 animate-spin" aria-hidden="true" />
) : isCompleted ? (
<Check className="h-3.5 w-3.5" aria-hidden="true" />
) : isCancelled || isFailed ? (
<X className="h-3.5 w-3.5" aria-hidden="true" />
) : (
<ImageIcon className="h-3.5 w-3.5" aria-hidden="true" />
)}
{statusPresentation.icon}
<span>{editorAgentToolLabel(toolCall.toolName)}</span>
<span className="ml-auto text-slate-400">{statusLabel}</span>
<span className="ml-auto text-slate-400">
{statusPresentation.label}
</span>
</div>
{displayError ? (
<div className="mt-1 text-red-600">{displayError}</div>
) : null}
{toolCall.images.length ? (
<div className="mt-2 grid grid-cols-3 gap-2">
{toolCall.images.map((image, index) => (
<div
key={`${toolCall.toolName}-${image.resourceId ?? index}`}
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'image',
mediaSrc: image.imageSrc,
objectKey: image.objectKey,
suggestedFileName: `Agent生成图片-${index + 1}`,
})
: undefined
}
>
<ResolvedAssetImage
src={image.thumbnailSrc ?? image.imageSrc}
objectKey={image.objectKey}
refreshKey={image.resourceId ?? toolCall.toolName}
alt=""
className="h-20 w-full object-cover"
/>
</div>
))}
{toolCall.images.map((image, index) => {
const resourceId = image.resourceId?.trim() || null;
return (
<div
key={`${toolCall.toolName}-${image.resourceId ?? index}`}
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
onClick={
resourceId ? () => focusResource(resourceId) : undefined
}
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'image',
mediaSrc: image.imageSrc,
objectKey: image.objectKey,
resourceId,
suggestedFileName: `Agent生成图片-${index + 1}`,
})
: undefined
}
>
<ResolvedAssetImage
src={image.thumbnailSrc ?? image.imageSrc}
objectKey={image.objectKey}
refreshKey={image.resourceId ?? toolCall.toolName}
alt=""
className="h-20 w-full object-cover"
/>
</div>
);
})}
</div>
) : null}
{videos.length ? (
<div className="mt-2 grid grid-cols-1 gap-2">
{videos.map((video, index) => (
<div
key={`${toolCall.toolName}-${video.resourceId ?? video.objectKey ?? index}`}
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'video',
mediaSrc: video.videoSrc,
objectKey: video.objectKey,
suggestedFileName: `Agent生成视频-${index + 1}`,
})
: undefined
}
>
<ResolvedAssetVideo
src={video.videoSrc}
objectKey={video.objectKey}
refreshKey={
video.resourceId ?? video.objectKey ?? toolCall.toolName
{videos.map((video, index) => {
const resourceId = video.resourceId?.trim() || null;
return (
<div
key={`${toolCall.toolName}-${video.resourceId ?? video.objectKey ?? index}`}
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'video',
mediaSrc: video.videoSrc,
objectKey: video.objectKey,
resourceId,
suggestedFileName: `Agent生成视频-${index + 1}`,
})
: undefined
}
poster={video.thumbnailSrc ?? undefined}
controls
playsInline
preload="metadata"
className="max-h-56 w-full bg-black object-contain"
/>
</div>
))}
>
<ResolvedAssetVideo
src={video.videoSrc}
objectKey={video.objectKey}
refreshKey={
video.resourceId ?? video.objectKey ?? toolCall.toolName
}
poster={video.thumbnailSrc ?? undefined}
controls
playsInline
preload="metadata"
className="max-h-56 w-full bg-black object-contain"
aria-label={`Agent生成视频-${index + 1}`}
/>
</div>
);
})}
</div>
) : null}
{audios.length ? (
<div className="mt-2 space-y-2">
{audios.map((audio, index) => (
<div
key={`${toolCall.toolName}-${audio.resourceId ?? audio.objectKey ?? index}`}
className="flex items-center gap-2 rounded-xl border border-slate-200 bg-slate-50 p-2"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'audio',
mediaSrc: audio.audioSrc,
objectKey: audio.objectKey,
suggestedFileName: `Agent生成音频-${index + 1}`,
})
: undefined
}
>
<Volume2
className="h-4 w-4 shrink-0 text-slate-500"
aria-hidden="true"
/>
<ResolvedAssetAudio
src={audio.audioSrc}
objectKey={audio.objectKey}
refreshKey={
audio.resourceId ?? audio.objectKey ?? toolCall.toolName
{audios.map((audio, index) => {
const resourceId = audio.resourceId?.trim() || null;
return (
<div
key={`${toolCall.toolName}-${audio.resourceId ?? audio.objectKey ?? index}`}
className="flex items-center gap-2 rounded-xl border border-slate-200 bg-slate-50 p-2"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'audio',
mediaSrc: audio.audioSrc,
objectKey: audio.objectKey,
resourceId,
suggestedFileName: `Agent生成音频-${index + 1}`,
})
: undefined
}
controls
preload="metadata"
className="min-w-0 flex-1"
/>
</div>
))}
>
<Volume2
className="h-4 w-4 shrink-0 text-slate-500"
aria-hidden="true"
/>
<ResolvedAssetAudio
src={audio.audioSrc}
objectKey={audio.objectKey}
refreshKey={
audio.resourceId ?? audio.objectKey ?? toolCall.toolName
}
controls
preload="metadata"
className="min-w-0 flex-1"
aria-label={`Agent生成音频-${index + 1}`}
/>
</div>
);
})}
</div>
) : null}
</div>
@@ -6,6 +6,7 @@ export enum EditorAgentRightClickAction {
CopyText = 'copy_text',
CopyImage = 'copy_image',
ReferenceImage = 'reference_image',
FocusCanvas = 'focus_canvas',
DownloadAsset = 'download_asset',
}
@@ -20,6 +21,7 @@ export type EditorAgentContextAsset =
mediaType: 'image' | 'video' | 'audio';
mediaSrc: string;
objectKey?: string | null;
resourceId?: string | null;
suggestedFileName: string;
};
@@ -1,6 +1,13 @@
/* @vitest-environment jsdom */
import { act, renderHook, waitFor } from '@testing-library/react';
import {
act,
renderHook as testingLibraryRenderHook,
type RenderHookOptions,
type RenderHookResult,
waitFor,
} from '@testing-library/react';
import type { ReactNode } from 'react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
@@ -9,12 +16,41 @@ import {
type EditorAgentMessage,
type EditorAgentMessageResponse,
} from '../../../../packages/shared/src/contracts/editorAgent.ts';
import { ImageCanvasActionsProvider } from '../ImageCanvasActionsProvider.tsx';
import {
EDITOR_AGENT_PATIENCE_NOTICE_DELAY_MS,
type EditorAgentConversationClient,
useEditorAgentConversation,
} from './useEditorAgentConversation.ts';
const focusResourceMock = vi.fn();
const refreshCanvasMock = vi.fn();
function ImageCanvasActionsTestWrapper({
children,
}: {
children: ReactNode;
}) {
return (
<ImageCanvasActionsProvider
focusResource={focusResourceMock}
refreshCanvas={refreshCanvasMock}
>
{children}
</ImageCanvasActionsProvider>
);
}
function renderHook<Result, Props>(
callback: (initialProps: Props) => Result,
options?: RenderHookOptions<Props>,
): RenderHookResult<Result, Props> {
return testingLibraryRenderHook(callback, {
...options,
wrapper: ImageCanvasActionsTestWrapper,
});
}
function createEditImageDisplayArgs(prompt: string) {
return {
stringArgs: [{ name: 'prompt', label: '修改要求', value: prompt }],
@@ -124,12 +160,10 @@ describe('useEditorAgentConversation', () => {
it('loads conversations and applies delta messages', async () => {
const client = createClient();
const onCanvasRefreshRequested = vi.fn();
const { result } = renderHook(() =>
useEditorAgentConversation({
projectId: 'project-1',
client,
onCanvasRefreshRequested,
}),
);
@@ -156,7 +190,7 @@ describe('useEditorAgentConversation', () => {
expect(result.current.activeConversation?.title).toBe(
'把这个角色改成像素风',
);
expect(onCanvasRefreshRequested).toHaveBeenCalledTimes(1);
expect(refreshCanvasMock).toHaveBeenCalledTimes(1);
expect(result.current.messages.map((message) => message.text)).toEqual([
'把这个角色改成像素风',
'我来处理',
@@ -595,7 +629,6 @@ describe('useEditorAgentConversation', () => {
it('does not apply a completed message response after switching conversations', async () => {
const client = createClient();
const onCanvasRefreshRequested = vi.fn();
let resolveSend!: (response: EditorAgentMessageResponse) => void;
vi.mocked(client.sendMessage).mockImplementation(
() =>
@@ -607,7 +640,6 @@ describe('useEditorAgentConversation', () => {
useEditorAgentConversation({
projectId: 'project-1',
client,
onCanvasRefreshRequested,
}),
);
@@ -682,7 +714,7 @@ describe('useEditorAgentConversation', () => {
expect(result.current.activeConversationId).toBe('conversation-2');
expect(result.current.messages).toHaveLength(1);
expect(result.current.messages[0]?.text).toBe('第二个会话原有消息');
expect(onCanvasRefreshRequested).not.toHaveBeenCalled();
expect(refreshCanvasMock).not.toHaveBeenCalled();
expect(
result.current.conversations.find(
(conversation) => conversation.conversationId === 'conversation-1',
@@ -918,13 +950,11 @@ describe('useEditorAgentConversation', () => {
createdAt: '2026-07-03T00:00:00.000Z',
};
vi.mocked(client.confirmToolCall).mockResolvedValue(undefined);
const onCanvasRefreshRequested = vi.fn();
const onConfirmSent = vi.fn();
const { result } = renderHook(() =>
useEditorAgentConversation({
projectId: 'project-1',
client,
onCanvasRefreshRequested,
onConfirmSent,
}),
);
@@ -968,7 +998,7 @@ describe('useEditorAgentConversation', () => {
expect(result.current.messages[1]?.toolCall?.externalJobId).toBe(
'task-edit-1',
);
expect(onCanvasRefreshRequested).not.toHaveBeenCalled();
expect(refreshCanvasMock).not.toHaveBeenCalled();
expect(onConfirmSent).toHaveBeenCalledTimes(1);
});
@@ -1096,12 +1126,10 @@ describe('useEditorAgentConversation', () => {
createdAt: '2026-07-03T00:00:00.000Z',
updatedAt: '2026-07-03T00:00:00.000Z',
});
const onCanvasRefreshRequested = vi.fn();
const { result } = renderHook(() =>
useEditorAgentConversation({
projectId: 'project-1',
client,
onCanvasRefreshRequested,
}),
);
@@ -1122,7 +1150,7 @@ describe('useEditorAgentConversation', () => {
);
expect(result.current.messages).toHaveLength(1);
expect(result.current.messages[0]?.toolCall?.status).toBe('cancelled');
expect(onCanvasRefreshRequested).not.toHaveBeenCalled();
expect(refreshCanvasMock).not.toHaveBeenCalled();
});
it('rethrows fetch errors and rolls back the optimistic message', async () => {
@@ -9,6 +9,7 @@ import type {
EditorAgentMessageRequest,
EditorAgentMessageResponse,
} from '@/packages/shared/src/contracts';
import { useImageCanvasActions } from '@/src/components/image-editor/ImageCanvasActionsContext.ts';
import {
cancelEditorAgentToolCall,
@@ -47,7 +48,6 @@ export type EditorAgentConversationClient = {
type UseEditorAgentConversationOptions = {
projectId?: string | null;
client?: EditorAgentConversationClient;
onCanvasRefreshRequested?: () => void;
onConfirmSent?: () => void;
};
@@ -135,9 +135,9 @@ function upsertConversationSummary(
export function useEditorAgentConversation({
projectId,
client = defaultEditorAgentConversationClient,
onCanvasRefreshRequested,
onConfirmSent,
}: UseEditorAgentConversationOptions) {
const { refreshCanvas } = useImageCanvasActions();
const normalizedProjectId = projectId?.trim() ?? '';
const [conversations, setConversations] = useState<
EditorAgentConversationSummary[]
@@ -424,10 +424,10 @@ export function useEditorAgentConversation({
);
})
) {
onCanvasRefreshRequested?.();
refreshCanvas();
}
},
[onCanvasRefreshRequested],
[refreshCanvas],
);
const applyDeltaMessages = useCallback(
@@ -4,6 +4,10 @@ import {
useState,
} from 'react';
import type {
ImageCanvasActionResult,
ImageCanvasActions,
} from '@/src/components/image-editor/ImageCanvasActionsContext.ts';
import { readAssetBytes } from '@/src/services/assetReadUrlService.ts';
import { copyTextToClipboard } from '@/src/services/clipboard.ts';
import {
@@ -29,7 +33,7 @@ type RightClickMenuState = {
target: RightClickMenuTarget;
pendingAction: EditorAgentRightClickAction | null;
resultAction: EditorAgentRightClickAction | null;
result: 'success' | 'error' | null;
result: ImageCanvasActionResult | null;
};
function sanitizeDownloadName(value: string, extension: string) {
@@ -232,10 +236,34 @@ async function downloadAsset(asset: EditorAgentContextAsset) {
}
}
function focusAssetOnCanvas(
asset: EditorAgentContextAsset,
onFocusResource?: ImageCanvasActions['focusResource'],
): ReturnType<ImageCanvasActions['focusResource']> {
if (asset.kind !== 'generated_media' || !onFocusResource) {
return { successed: false, reason: 'other' };
}
const resourceId = asset.resourceId?.trim();
if (!resourceId) {
return { successed: false, reason: 'other' };
}
try {
return onFocusResource(resourceId);
} catch {
return { successed: false, reason: 'other' };
}
}
function actionResultFromSuccess(successed: boolean): ImageCanvasActionResult {
return successed ? { successed: true } : { successed: false, reason: 'other' };
}
export function useRightClickMenu({
onReferenceImage,
onFocusResource,
}: {
onReferenceImage?: (asset: EditorAgentContextAsset) => boolean;
onFocusResource?: ImageCanvasActions['focusResource'];
} = {}) {
const [rightClickMenu, setRightClickMenu] =
useState<RightClickMenuState | null>(null);
@@ -276,37 +304,65 @@ export function useRightClickMenu({
}
: current,
);
const succeeded =
action === EditorAgentRightClickAction.CopyText &&
target.kind === 'text'
? await copyTextToClipboard(target.text)
: action === EditorAgentRightClickAction.CopyImage &&
target.kind === 'asset' &&
target.asset.mediaType === 'image'
? await copyAssetImage(target.asset)
: action === EditorAgentRightClickAction.ReferenceImage &&
target.kind === 'asset' &&
target.asset.mediaType === 'image' &&
contextAssetMediaSrc(target.asset).trim()
? (onReferenceImage?.(target.asset) ?? false)
: action === EditorAgentRightClickAction.DownloadAsset &&
target.kind === 'asset'
? await downloadAsset(target.asset)
: false;
setRightClickMenu((current) =>
current?.target === target && current.pendingAction === action
? succeeded
? null
: {
...current,
pendingAction: null,
resultAction: action,
result: 'error',
}
: current,
);
let actionResult: ImageCanvasActionResult = {
successed: false,
reason: 'other',
};
switch (action) {
case EditorAgentRightClickAction.CopyText:
if (target.kind === 'text') {
actionResult = actionResultFromSuccess(
await copyTextToClipboard(target.text),
);
}
break;
case EditorAgentRightClickAction.CopyImage:
if (target.kind === 'asset' && target.asset.mediaType === 'image') {
actionResult = actionResultFromSuccess(
await copyAssetImage(target.asset),
);
}
break;
case EditorAgentRightClickAction.ReferenceImage:
if (
target.kind === 'asset' &&
target.asset.mediaType === 'image' &&
contextAssetMediaSrc(target.asset).trim()
) {
actionResult = actionResultFromSuccess(
onReferenceImage?.(target.asset) ?? false,
);
}
break;
case EditorAgentRightClickAction.FocusCanvas:
if (target.kind === 'asset') {
actionResult = focusAssetOnCanvas(target.asset, onFocusResource);
}
break;
case EditorAgentRightClickAction.DownloadAsset:
if (target.kind === 'asset') {
actionResult = actionResultFromSuccess(
await downloadAsset(target.asset),
);
}
break;
}
setRightClickMenu((current) => {
if (current?.target !== target || current.pendingAction !== action) {
return current;
}
if (actionResult.successed) {
return null;
}
return {
...current,
pendingAction: null,
resultAction: action,
result: actionResult,
};
});
},
[onReferenceImage, rightClickMenu],
[onFocusResource, onReferenceImage, rightClickMenu],
);
return {
@@ -0,0 +1,43 @@
/* @vitest-environment jsdom */
import { act, renderHook } from '@testing-library/react';
import type { ReactNode } from 'react';
import { describe, expect, it, vi } from 'vitest';
import {
useImageCanvasActions,
} from './ImageCanvasActionsContext';
import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider';
describe('ImageCanvasActionsContext', () => {
it('exposes the editor-scoped canvas actions', () => {
const focusResource = vi.fn(() => ({ successed: true }));
const refreshCanvas = vi.fn();
const { result } = renderHook(useImageCanvasActions, {
wrapper: ({ children }: { children: ReactNode }) => (
<ImageCanvasActionsProvider
focusResource={focusResource}
refreshCanvas={refreshCanvas}
>
{children}
</ImageCanvasActionsProvider>
),
});
let focusResult: { successed: boolean } | undefined;
act(() => {
focusResult = result.current.focusResource('resource-a');
result.current.refreshCanvas();
});
expect(focusResult).toEqual({ successed: true });
expect(focusResource).toHaveBeenCalledWith('resource-a');
expect(refreshCanvas).toHaveBeenCalledTimes(1);
});
it('fails clearly when consumed outside the editor provider', () => {
expect(() => renderHook(useImageCanvasActions)).toThrow(
'useImageCanvasActions must be used within ImageCanvasActionsProvider',
);
});
});
@@ -0,0 +1,24 @@
import { createContext, useContext } from 'react';
export interface ImageCanvasActionResult {
successed: boolean;
reason?: 'not-found-on-canva' | 'other';
}
export type ImageCanvasActions = {
focusResource: (resourceId: string) => ImageCanvasActionResult;
refreshCanvas: () => void;
};
export const ImageCanvasActionsContext =
createContext<ImageCanvasActions | null>(null);
export function useImageCanvasActions() {
const actions = useContext(ImageCanvasActionsContext);
if (!actions) {
throw new Error(
'useImageCanvasActions must be used within ImageCanvasActionsProvider',
);
}
return actions;
}
@@ -0,0 +1,27 @@
import { type ReactNode, useMemo } from 'react';
import {
type ImageCanvasActions,
ImageCanvasActionsContext,
} from './ImageCanvasActionsContext';
export function ImageCanvasActionsProvider({
children,
focusResource,
refreshCanvas,
}: {
children: ReactNode;
focusResource: ImageCanvasActions['focusResource'];
refreshCanvas: ImageCanvasActions['refreshCanvas'];
}) {
const actions = useMemo<ImageCanvasActions>(
() => ({ focusResource, refreshCanvas }),
[focusResource, refreshCanvas],
);
return (
<ImageCanvasActionsContext.Provider value={actions}>
{children}
</ImageCanvasActionsContext.Provider>
);
}
@@ -1,9 +1,15 @@
/* @vitest-environment jsdom */
import { fireEvent, render, screen, within } from '@testing-library/react';
import { createRef } from 'react';
import {
fireEvent,
render as testingLibraryRender,
screen,
within,
} from '@testing-library/react';
import { createRef, type ReactElement, type ReactNode } from 'react';
import { describe, expect, it, vi } from 'vitest';
import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider';
import { ImageCanvasEditorShellView } from './ImageCanvasEditorShellView';
import type { CanvasLayer } from './ImageCanvasEditorTypes';
import type { ImageCanvasMetadataModalViewProps } from './ImageCanvasMetadataModalView';
@@ -11,6 +17,19 @@ import type { ImageCanvasSidebarViewProps } from './ImageCanvasSidebarView';
import type { ImageCanvasStageViewProps } from './ImageCanvasStageView';
import type { ImageCanvasTopbarViewProps } from './ImageCanvasTopbarView';
function render(ui: ReactElement) {
return testingLibraryRender(ui, {
wrapper: ({ children }: { children: ReactNode }) => (
<ImageCanvasActionsProvider
focusResource={vi.fn()}
refreshCanvas={vi.fn()}
>
{children}
</ImageCanvasActionsProvider>
),
});
}
function createLayer(overrides: Partial<CanvasLayer> = {}): CanvasLayer {
return {
id: 'layer-1',
@@ -2545,9 +2545,15 @@ describe('ImageCanvasEditorView', () => {
expect(screen.queryByRole('button', { name: '画布小地图' })).toBeNull();
});
it('renders Agent generation thumbnails as passive previews', async () => {
it('smoothly focuses an Agent generation resource without changing canvas selection or panels', async () => {
const rafCallbacks: FrameRequestCallback[] = [];
vi.spyOn(window, 'requestAnimationFrame').mockImplementation((callback) => {
rafCallbacks.push(callback);
return rafCallbacks.length;
});
vi.spyOn(window, 'cancelAnimationFrame').mockImplementation(() => {});
enableEditorAgentSidebarForTest();
const detail = createEditorAgentDetailWithGeneration('resource-puzzle');
const detail = createEditorAgentDetailWithGeneration('resource-big-fish');
listEditorAgentConversationsMock.mockResolvedValueOnce([
createEditorAgentConversationSummary(),
]);
@@ -2559,10 +2565,44 @@ describe('ImageCanvasEditorView', () => {
const messageLog = await screen.findByRole('log', {
name: '画布 Agent 消息流',
});
const world = screen.getByTestId('image-canvas-world') as HTMLElement;
const puzzleLayer = screen
.getByAltText('画布图片:拼图素材')
.closest('button')!;
const bigFishLayer = screen
.getByAltText('画布图片:大鱼素材')
.closest('button')!;
const initialTransform = world.style.transform;
expect(
within(messageLog).queryByRole('button', { name: '生成结果' }),
).toBeNull();
puzzleLayer.classList.contains('image-canvas-editor__layer--selected'),
).toBe(true);
expect(
bigFishLayer.classList.contains('image-canvas-editor__layer--selected'),
).toBe(false);
fireEvent.contextMenu(
within(messageLog).getByRole('presentation'),
{ clientX: 30, clientY: 40 },
);
fireEvent.click(screen.getByRole('menuitem', { name: '在画布中定位' }));
expect(world.style.transform).toBe(initialTransform);
act(() => rafCallbacks.shift()?.(0));
act(() => rafCallbacks.shift()?.(210));
expect(world.style.transform).not.toBe(initialTransform);
expect(world.style.transform).not.toBe(
'translate(-840px, -246.5px) scale(1)',
);
act(() => rafCallbacks.shift()?.(420));
expect(world.style.transform).toBe('translate(-840px, -246.5px) scale(1)');
expect(screen.getByLabelText('发送给画布 Agent')).toBeTruthy();
expect(
puzzleLayer.classList.contains('image-canvas-editor__layer--selected'),
).toBe(true);
expect(
bigFishLayer.classList.contains('image-canvas-editor__layer--selected'),
).toBe(false);
expect(within(messageLog).queryByText('gpt-image-2')).toBeNull();
expect(loadEditorProjectMock).not.toHaveBeenCalled();
});
@@ -32,6 +32,8 @@ import {
PlatformRechargePaymentResultDialog,
} from '../platform-entry/PlatformRechargePaymentStatusDialogs';
import { usePlatformProfileCenterController } from '../platform-entry/usePlatformProfileCenterController';
import type { ImageCanvasActionResult } from './ImageCanvasActionsContext';
import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider';
import {
canvasAssetKindOrNull,
DEFAULT_CANVAS_BACKGROUND_COLOR,
@@ -453,6 +455,7 @@ export function ImageCanvasEditorView({
const {
viewport,
setViewport,
animateViewportTo,
canvasSize,
minimapModel,
updateScaleFromCenter,
@@ -875,7 +878,7 @@ export function ImageCanvasEditorView({
const focusCanvasBounds = useCallback(
(
bounds: { x: number; y: number; width: number; height: number },
options: { reserveDialogSpace?: boolean } = {},
options: { reserveDialogSpace?: boolean; animate?: boolean } = {},
) => {
const nextViewport = fitViewportToBounds({
bounds,
@@ -890,9 +893,31 @@ export function ImageCanvasEditorView({
},
padding: TASK_FOCUS_PADDING,
});
if (options.animate) {
animateViewportTo(nextViewport);
return;
}
setViewport(nextViewport);
},
[canvasSize, setViewport],
[animateViewportTo, canvasSize, setViewport],
);
const focusCanvasResource = useCallback(
(resourceId: string): ImageCanvasActionResult => {
const normalizedResourceId = resourceId.trim();
if (!normalizedResourceId) {
return { successed: false, reason: 'other' };
}
// 同一 resourceId 对应多个画布图层时,定位顺序中的首个图层,保持单目标聚焦行为。
const targetLayer = layersRef.current.find(
(layer) => layer.resourceId === normalizedResourceId,
);
if (!targetLayer) {
return { successed: false, reason: 'not-found-on-canva' };
}
focusCanvasBounds(targetLayer, { animate: true });
return { successed: true };
},
[focusCanvasBounds],
);
const focusCanvasLayerById = useCallback(
(layerId: string) => {
@@ -1195,7 +1220,7 @@ export function ImageCanvasEditorView({
},
[applyProjectSnapshot, captureCanvasHistory, refreshAssetLibrary],
);
const handleEditorAgentCanvasRefreshRequested = useCallback(() => {
const refreshCanvas = useCallback(() => {
if (!projectId) {
return;
}
@@ -1469,10 +1494,6 @@ export function ImageCanvasEditorView({
const handleEditorAgentConfirmSent = useCallback(() => {
generationSurface.refreshTaskList();
}, [generationSurface]);
const handleEditorAgentJobCompleted = useCallback(() => {
generationSurface.refreshTaskList();
handleEditorAgentCanvasRefreshRequested();
}, [generationSurface, handleEditorAgentCanvasRefreshRequested]);
const showGenerationWarning = generationSurface.showGenerationWarning;
const handleExternalGenerationTasksCompleted = useCallback(
(tasks: ExternalGenerationTaskRecord[]) => {
@@ -1486,12 +1507,8 @@ export function ImageCanvasEditorView({
showGenerationWarning(warning);
}
refreshEditorWalletState();
void loadEditorProject(projectId)
.then(applyGeneratedProjectSnapshot)
.catch(() => undefined);
},
[
applyGeneratedProjectSnapshot,
projectId,
refreshEditorWalletState,
showGenerationWarning,
@@ -2400,7 +2417,6 @@ export function ImageCanvasEditorView({
onActivateGenerationDialog: activateCanvasGenerationDialog,
onFocusExternalTask: focusExternalGenerationTask,
onExternalTasksCompleted: handleExternalGenerationTasksCompleted,
onEditorAgentCanvasRefreshRequested: handleEditorAgentJobCompleted,
onEditorAgentConfirmSent: handleEditorAgentConfirmSent,
onToggleTaskSidebar: toggleTaskSidebar,
onToggleAgentConversation: toggleAgentConversation,
@@ -2465,7 +2481,10 @@ export function ImageCanvasEditorView({
};
return (
<>
<ImageCanvasActionsProvider
focusResource={focusCanvasResource}
refreshCanvas={refreshCanvas}
>
<ImageCanvasEditorShellView
editorRootRef={editorRootRef}
uploadInputRef={uploadInputRef}
@@ -2539,7 +2558,7 @@ export function ImageCanvasEditorView({
orderId={wechatRechargeOrderConfirmationState.orderId}
/>
) : null}
</>
</ImageCanvasActionsProvider>
);
}
@@ -134,7 +134,6 @@ export type ImageCanvasStageViewProps = {
onActivateGenerationDialog: (dialog: CanvasGenerationDialogState) => void;
onFocusExternalTask: (task: ExternalGenerationTaskRecord) => void;
onExternalTasksCompleted?: (tasks: ExternalGenerationTaskRecord[]) => void;
onEditorAgentCanvasRefreshRequested?: () => void;
onEditorAgentConfirmSent?: () => void;
onToggleTaskSidebar: () => void;
onToggleAgentConversation: () => void;
@@ -271,7 +270,6 @@ export function ImageCanvasStageView({
onActivateGenerationDialog,
onFocusExternalTask,
onExternalTasksCompleted,
onEditorAgentCanvasRefreshRequested,
onEditorAgentConfirmSent,
onToggleTaskSidebar,
onToggleAgentConversation,
@@ -512,7 +510,6 @@ export function ImageCanvasStageView({
onToggleOpen={onToggleAgentConversation}
layers={layers}
assets={editorAgentAssets}
onCanvasRefreshRequested={onEditorAgentCanvasRefreshRequested}
onConfirmSent={onEditorAgentConfirmSent}
/>
) : null}
@@ -3,17 +3,35 @@
import {
act,
fireEvent,
render,
render as testingLibraryRender,
screen,
waitFor,
} from '@testing-library/react';
import type { ReactElement, ReactNode } from 'react';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import type { ExternalGenerationTaskRecord } from '../../../packages/shared/src/contracts/externalGeneration';
import { listExternalGenerationTasks } from '../../services/external-generation';
import { ImageCanvasActionsProvider } from './ImageCanvasActionsProvider.tsx';
import { ImageCanvasTaskSidebarView } from './ImageCanvasTaskSidebarView';
import { useImageCanvasContextStore } from './useImageCanvasContextStore.ts';
const focusResourceMock = vi.fn();
const refreshCanvasMock = vi.fn();
function render(ui: ReactElement) {
return testingLibraryRender(ui, {
wrapper: ({ children }: { children: ReactNode }) => (
<ImageCanvasActionsProvider
focusResource={focusResourceMock}
refreshCanvas={refreshCanvasMock}
>
{children}
</ImageCanvasActionsProvider>
),
});
}
vi.mock('../../services/external-generation', () => ({
listExternalGenerationTasks: vi.fn().mockResolvedValue({
overview: {
@@ -30,6 +48,8 @@ const listExternalGenerationTasksMock = vi.mocked(listExternalGenerationTasks);
beforeEach(() => {
useImageCanvasContextStore.getState().setProjectId('project-1');
focusResourceMock.mockReset();
refreshCanvasMock.mockReset();
listExternalGenerationTasksMock.mockClear();
listExternalGenerationTasksMock.mockResolvedValue({
overview: {
@@ -448,6 +468,7 @@ describe('ImageCanvasTaskSidebarView', () => {
}),
]);
expect(onExternalTasksCompleted).toHaveBeenCalledTimes(1);
expect(refreshCanvasMock).toHaveBeenCalledTimes(1);
} finally {
vi.useRealTimers();
}
@@ -17,6 +17,7 @@ import {
import type { ExternalGenerationTaskRecord } from '../../../packages/shared/src/contracts/externalGeneration';
import { listExternalGenerationTasks } from '../../services/external-generation';
import { useImageCanvasActions } from './ImageCanvasActionsContext.ts';
import { EditorIconButton } from './ImageCanvasEditorPrimitives';
import type { CanvasTaskStatus } from './ImageCanvasEditorTypes';
import { useImageCanvasContextStore } from './useImageCanvasContextStore.ts';
@@ -280,6 +281,7 @@ export function ImageCanvasTaskSidebarView({
onFocusExternalTask,
onExternalTasksCompleted,
}: ImageCanvasTaskSidebarViewProps) {
const { refreshCanvas } = useImageCanvasActions();
const projectId = useImageCanvasContextStore((state) => state.projectId);
const normalizedProjectId = projectId?.trim() ?? '';
const [activeTab, setActiveTab] = useState<TaskSidebarTab>('active');
@@ -334,9 +336,10 @@ export function ImageCanvasTaskSidebarView({
for (const task of newlyCompletedTasks) {
completedRefreshNotifiedTaskIdsRef.current.add(task.jobId);
}
refreshCanvas();
onExternalTasksCompleted?.(newlyCompletedTasks);
},
[onExternalTasksCompleted],
[onExternalTasksCompleted, refreshCanvas],
);
useEffect(() => {
@@ -116,6 +116,7 @@ function expectViewport(viewport: CanvasViewport, expected: CanvasViewport) {
describe('useImageCanvasViewportControls', () => {
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllGlobals();
document.body.replaceChildren();
});
@@ -158,6 +159,91 @@ describe('useImageCanvasViewportControls', () => {
expect(captureCanvasHistory).toHaveBeenCalledTimes(2);
});
it('animates, retargets from the current frame and lets manual input cancel the transition', () => {
const rafCallbacks = new Map<number, FrameRequestCallback>();
let nextFrameId = 0;
vi.spyOn(window, 'requestAnimationFrame').mockImplementation((callback) => {
nextFrameId += 1;
rafCallbacks.set(nextFrameId, callback);
return nextFrameId;
});
const cancelAnimationFrameSpy = vi
.spyOn(window, 'cancelAnimationFrame')
.mockImplementation((frameId) => {
rafCallbacks.delete(frameId);
});
vi.stubGlobal(
'matchMedia',
vi.fn(() => ({ matches: false })) as unknown as typeof window.matchMedia,
);
const { result, viewportElement, captureCanvasHistory } =
renderViewportControls();
const targetViewport = { x: 120, y: -80, scale: 1.25 };
const runNextFrame = (timestamp: number) => {
const entry = rafCallbacks.entries().next().value as
[number, FrameRequestCallback] | undefined;
expect(entry).toBeTruthy();
if (!entry) {
return;
}
rafCallbacks.delete(entry[0]);
entry[1](timestamp);
};
act(() => {
result.current.animateViewportTo(targetViewport);
});
expectViewport(result.current.viewport, {
x: -260,
y: 70,
scale: 0.5,
});
act(() => runNextFrame(0));
act(() => runNextFrame(210));
expect(result.current.viewport.x).toBeGreaterThan(-260);
expect(result.current.viewport.x).toBeLessThan(120);
expect(result.current.viewport.scale).toBeGreaterThan(0.5);
expect(result.current.viewport.scale).toBeLessThan(1.25);
const retargetedViewport = { x: 420, y: 260, scale: 1.6 };
act(() => {
result.current.animateViewportTo(retargetedViewport);
});
expect(cancelAnimationFrameSpy).toHaveBeenCalled();
act(() => runNextFrame(300));
act(() => runNextFrame(720));
expectViewport(result.current.viewport, retargetedViewport);
expect(captureCanvasHistory).not.toHaveBeenCalled();
act(() => {
result.current.animateViewportTo({ x: 600, y: 400, scale: 2 });
});
act(() => runNextFrame(800));
act(() => {
fireEvent.wheel(viewportElement, { deltaY: 40 });
});
expect(cancelAnimationFrameSpy).toHaveBeenCalled();
expect(rafCallbacks.size).toBe(0);
});
it('finishes viewport animation immediately when reduced motion is requested', () => {
const requestAnimationFrameSpy = vi.spyOn(window, 'requestAnimationFrame');
vi.stubGlobal(
'matchMedia',
vi.fn(() => ({ matches: true })) as unknown as typeof window.matchMedia,
);
const { result } = renderViewportControls();
const targetViewport = { x: 20, y: 30, scale: 0.8 };
act(() => {
result.current.animateViewportTo(targetViewport);
});
expectViewport(result.current.viewport, targetViewport);
expect(requestAnimationFrameSpy).not.toHaveBeenCalled();
});
it('keeps ordinary mouse wheel vertical and maps Shift wheel to horizontal', () => {
const { result, viewportElement } = renderViewportControls();

Some files were not shown because too many files have changed in this diff Show More