画布Agent关联外部生成任务
工具消息改为保存外部任务 ID 与取消事实，不再复制执行状态。 确认请求以稳定身份提交外部任务，并由前端查询任务状态和结果。 同步更新画布 Agent 测试与执行契约文档。
This commit is contained in:
@@ -3104,6 +3104,12 @@
|
||||
- 影响范围：`server-rs/crates/api-server/src/editor_generation_queue.rs`、`server-rs/crates/api-server/src/external_generation_worker.rs`、`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`src/services/image-editor/editorProjectClient.ts`。
|
||||
- 验证方式：`cargo test -p api-server external_generation_worker --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_canvas_generation --manifest-path server-rs/Cargo.toml`、`cargo test -p shared-contracts --manifest-path server-rs/Cargo.toml`、`npm run test -- src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
|
||||
## 2026-07-13 画布 Agent 工具执行状态复用外部生成任务
|
||||
|
||||
- 决策：画布 Agent 的 OSS 工具消息删除独立 `status`，只保存可选 `externalJobId`、确认前取消事实 `cancelledAt` 和消息展示结果；`external_generation_job` 是排队、执行、完成、失败、lease 与计费结算的唯一真相。确认接口按 `conversationId + messageId + toolName` 稳定去重并复用既有编辑器 worker job kind，不新增 `editor_agent_tool_execution` 或其它关联表。
|
||||
- 影响范围：画布 Agent 共享契约、确认/取消接口、编辑器生成入队 helper、对话状态展示与恢复。
|
||||
- 验证方式：`cargo check -p api-server -p shared-contracts --manifest-path server-rs/Cargo.toml`、画布 Agent 定向前端测试、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
|
||||
## 2026-06-18 原生壳 WebView 刷新能力只保留受控当前页刷新
|
||||
|
||||
- 背景：Expo 移动壳和 Tauri 桌面壳都需要一个真实的宿主级刷新入口，供 H5 在检测到资源、登录态或运行态需要重新载入时请求宿主刷新当前容器；该能力不能演变成任意 URL 导航或原生 WebView ref 透传。
|
||||
|
||||
@@ -73,7 +73,11 @@
|
||||
|
||||
## 工具调用确认展示契约
|
||||
|
||||
- Agent 规划出生成或编辑工具后，先把工具消息持久化为 `pending_confirmation`；确认卡必须在真正调用生成 provider 前展示本次提示词、规格参数、目标图和参考图缩略图，用户确认后才执行，取消后保留同一条已取消记录。内部 system 文本和图片哈希 ID 不直接展示给用户。
|
||||
- Agent 规划出生成或编辑工具后，先把没有 `externalJobId` / `cancelledAt` 的工具消息持久化为待确认记录；确认卡必须在真正调用生成 provider 前展示本次提示词、规格参数、目标图和参考图缩略图，用户确认后才执行，取消后保留同一条已取消记录。内部 system 文本和图片哈希 ID 不直接展示给用户。
|
||||
- 工具消息不再保存独立 `status`。OSS 文档只保存可选 `externalJobId` 与确认前取消时间 `cancelledAt`：两者都为空表示待确认，只有 `cancelledAt` 表示已取消；存在 `externalJobId` 时，排队、执行、完成和失败状态统一读取 SpacetimeDB `external_generation_job`，不得在 OSS 中复制第二套执行状态。
|
||||
- 确认接口必须先把工具参数转换为既有编辑器 worker payload，再使用 `editor-agent:{conversationId}:{messageId}:{toolName}` 稳定 dedupe key 入队；同一确认的请求重试只能得到同一个 external job。入队成功后把返回的 job id 写回同一条 OSS 工具消息，不新增 Agent 工具执行关联表。
|
||||
- 前端根据 `externalJobId` 查询通用 external-generation job 状态；worker 继续通过 `canvasCompletion` 把生成结果写回工程与素材库。浏览器断线、刷新或 api-server 重启不得导致确认接口重新扣费或重新提交 provider。
|
||||
- worker 成功时把既有编辑器生成回包写入该 job 的 `result_payload_json`；通用 job 状态接口只读返回其中的 `result` 投影，Agent 卡据此恢复图片、视频或音频预览。该投影属于 SpacetimeDB job，不回写 OSS 对话文档。
|
||||
- `EditorAgentToolCall.args` 保留为工具返回的原始 JSON，是确认接口重新反序列化并执行工具的唯一参数真相。图片参数继续只保存由真实 data key 计算出的 opaque SHA-256 `imageId`；不得为了前端预览把 `args` 中的图片 ID 改写成 `objectKey`、URL 或展示对象，也不得由前端重组或回传一份新的执行参数。
|
||||
- `EditorAgentToolCall.displayArgs` 是必填、只读的用户确认展示投影，与 `args` 分离：
|
||||
- `stringArgs` 保存提示词、比例、清晰度、模型、时长等可展示参数的稳定名称、用户可见标题和值；
|
||||
|
||||
@@ -7,13 +7,6 @@ export const EDITOR_AGENT_MESSAGES_DOCUMENT_VERSION = 1;
|
||||
|
||||
export type EditorAgentMessageRole = 'user' | 'assistant' | 'system';
|
||||
|
||||
export type EditorAgentToolCallStatus =
|
||||
| 'pending_confirmation'
|
||||
| 'executing'
|
||||
| 'completed'
|
||||
| 'cancelled'
|
||||
| 'failed';
|
||||
|
||||
export type EditorAgentAttachmentSource = 'canvas_resource' | 'library_asset';
|
||||
|
||||
export interface EditorAgentAttachmentRef {
|
||||
@@ -88,9 +81,10 @@ export interface EditorAgentToolCallDisplayArgs {
|
||||
|
||||
export interface EditorAgentToolCall {
|
||||
toolName: string;
|
||||
status: EditorAgentToolCallStatus;
|
||||
args: unknown;
|
||||
displayArgs: EditorAgentToolCallDisplayArgs;
|
||||
externalJobId?: string | null;
|
||||
cancelledAt?: string | null;
|
||||
images: EditorAgentGeneratedImage[];
|
||||
// Older persisted conversation documents do not contain these media fields.
|
||||
videos?: EditorAgentGeneratedVideo[];
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -23,16 +23,6 @@ pub enum EditorAgentAttachmentSource {
|
||||
LibraryAsset,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum EditorAgentToolCallStatus {
|
||||
PendingConfirmation,
|
||||
Executing,
|
||||
Completed,
|
||||
Failed,
|
||||
Cancelled,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorAgentAttachmentRef {
|
||||
@@ -147,13 +137,16 @@ pub struct EditorAgentToolCallImageRef {
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase", deny_unknown_fields)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorAgentToolCall {
|
||||
pub tool_name: String,
|
||||
pub status: EditorAgentToolCallStatus,
|
||||
pub args: serde_json::Value,
|
||||
pub display_args: EditorAgentToolCallDisplayArgs,
|
||||
#[serde(default)]
|
||||
pub external_job_id: Option<String>,
|
||||
#[serde(default)]
|
||||
pub cancelled_at: Option<String>,
|
||||
#[serde(default)]
|
||||
pub images: Vec<EditorAgentGeneratedImage>,
|
||||
#[serde(default)]
|
||||
pub videos: Vec<EditorAgentGeneratedVideo>,
|
||||
@@ -248,7 +241,6 @@ mod tests {
|
||||
fn tool_call_display_args_and_media_use_camel_case() {
|
||||
let missing_display_args = serde_json::from_value::<EditorAgentToolCall>(json!({
|
||||
"toolName": "generate-video",
|
||||
"status": "completed",
|
||||
"args": {},
|
||||
"images": []
|
||||
}));
|
||||
@@ -256,7 +248,6 @@ mod tests {
|
||||
|
||||
let tool_call = EditorAgentToolCall {
|
||||
tool_name: "generate-video".to_string(),
|
||||
status: EditorAgentToolCallStatus::Completed,
|
||||
args: json!({}),
|
||||
display_args: EditorAgentToolCallDisplayArgs {
|
||||
string_args: vec![EditorAgentToolCallStringArg {
|
||||
@@ -281,6 +272,8 @@ mod tests {
|
||||
price_mud_points: 5,
|
||||
},
|
||||
},
|
||||
external_job_id: Some("task-editor-agent-1".to_string()),
|
||||
cancelled_at: None,
|
||||
images: Vec::new(),
|
||||
videos: vec![EditorAgentGeneratedVideo {
|
||||
resource_id: Some("resource-video-1".to_string()),
|
||||
@@ -301,6 +294,8 @@ mod tests {
|
||||
};
|
||||
let payload = serde_json::to_value(tool_call).expect("tool call should serialize");
|
||||
assert_eq!(payload["videos"][0]["videoSrc"], "/generated-video.mp4");
|
||||
assert_eq!(payload["externalJobId"], "task-editor-agent-1");
|
||||
assert!(payload.get("status").is_none());
|
||||
assert_eq!(payload["audios"][0]["audioSrc"], "/generated-audio.mp3");
|
||||
assert_eq!(
|
||||
payload["displayArgs"]["stringArgs"][0]["value"],
|
||||
@@ -316,4 +311,23 @@ mod tests {
|
||||
);
|
||||
assert_eq!(payload["displayArgs"]["extras"]["priceMudPoints"], 5);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn legacy_tool_call_status_is_ignored_on_read() {
|
||||
let tool_call = serde_json::from_value::<EditorAgentToolCall>(json!({
|
||||
"toolName": "generate-image",
|
||||
"status": "executing",
|
||||
"args": {},
|
||||
"displayArgs": {
|
||||
"stringArgs": [],
|
||||
"imageArgs": [],
|
||||
"extras": { "priceMudPoints": 3 }
|
||||
},
|
||||
"images": []
|
||||
}))
|
||||
.expect("legacy status should not break persisted OSS documents");
|
||||
|
||||
let payload = serde_json::to_value(tool_call).expect("tool call should serialize");
|
||||
assert!(payload.get("status").is_none());
|
||||
}
|
||||
}
|
||||
|
||||
+2
-3
@@ -122,7 +122,6 @@ function createPendingToolCallMessage(): EditorAgentMessage {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'edit-image',
|
||||
status: 'pending_confirmation',
|
||||
args: {
|
||||
object_image_id: 'source-image-1',
|
||||
reference_image_ids: ['reference-image-1', 'reference-image-2'],
|
||||
@@ -570,7 +569,7 @@ describe('EditorAgentConversationPanelView', () => {
|
||||
text: 'internal completed tool output that must stay hidden',
|
||||
toolCall: {
|
||||
...pendingMessage.toolCall!,
|
||||
status: 'completed',
|
||||
externalJobId: 'task-edit-panel-1',
|
||||
images: [
|
||||
{
|
||||
resourceId: null,
|
||||
@@ -610,7 +609,7 @@ describe('EditorAgentConversationPanelView', () => {
|
||||
text: 'internal cancelled tool output that must stay hidden',
|
||||
toolCall: {
|
||||
...pendingMessage.toolCall!,
|
||||
status: 'cancelled',
|
||||
cancelledAt: '2026-07-03T00:00:11.000Z',
|
||||
},
|
||||
});
|
||||
|
||||
|
||||
+1
@@ -604,6 +604,7 @@ export function EditorAgentConversationPanelView({
|
||||
onCancelToolCall={(messageId) => {
|
||||
void cancelToolCall(messageId);
|
||||
}}
|
||||
onJobCompleted={onCanvasRefreshRequested}
|
||||
/>
|
||||
))}
|
||||
{isWaiting ? <ThinkingBubble /> : null}
|
||||
|
||||
@@ -1,16 +1,20 @@
|
||||
import { Check, Image as ImageIcon, Loader2, Volume2, X } from 'lucide-react';
|
||||
import { useEffect, useRef, useState } from 'react';
|
||||
|
||||
import type {
|
||||
EditorAgentAttachmentRef,
|
||||
EditorAgentMessage,
|
||||
EditorAgentToolCall,
|
||||
} from '@/packages/shared/src/contracts';
|
||||
import type { ExternalGenerationJobStatus } from '@/packages/shared/src/contracts/externalGeneration.ts';
|
||||
import type { ExternalGenerationJobStatusRecord } from '@/packages/shared/src/contracts/externalGeneration.ts';
|
||||
import { attachmentKey } from '@/src/components/image-editor/EditorAgentConversation/common.ts';
|
||||
import { PendingToolCall } from '@/src/components/image-editor/EditorAgentConversation/PendingToolCall.tsx';
|
||||
import { editorAgentToolLabel } from '@/src/components/image-editor/EditorAgentConversation/toolCallPresentation.ts';
|
||||
import { ResolvedAssetAudio } from '@/src/components/ResolvedAssetAudio.tsx';
|
||||
import { ResolvedAssetImage } from '@/src/components/ResolvedAssetImage.tsx';
|
||||
import { ResolvedAssetVideo } from '@/src/components/ResolvedAssetVideo.tsx';
|
||||
import { getExternalGenerationJobStatus } from '@/src/services/external-generation';
|
||||
|
||||
function messageRoleLabel(role: EditorAgentMessage['role']) {
|
||||
if (role === 'user') {
|
||||
@@ -56,18 +60,88 @@ export function AttachmentChip({
|
||||
);
|
||||
}
|
||||
|
||||
function ToolCallView({ toolCall }: { toolCall: EditorAgentToolCall }) {
|
||||
function ToolCallView({
|
||||
toolCall,
|
||||
jobStatus,
|
||||
onJobCompleted,
|
||||
}: {
|
||||
toolCall: EditorAgentToolCall;
|
||||
jobStatus?: ExternalGenerationJobStatus | null;
|
||||
onJobCompleted?: () => void;
|
||||
}) {
|
||||
const videos = toolCall.videos ?? [];
|
||||
const audios = toolCall.audios ?? [];
|
||||
const isExecuting = toolCall.status === 'executing';
|
||||
const [resolvedJob, setResolvedJob] =
|
||||
useState<ExternalGenerationJobStatusRecord | null>(null);
|
||||
const completionNotifiedRef = useRef(false);
|
||||
useEffect(() => {
|
||||
const jobId = toolCall.externalJobId?.trim();
|
||||
if (!jobId) {
|
||||
setResolvedJob(null);
|
||||
return;
|
||||
}
|
||||
let disposed = false;
|
||||
let timeoutId: ReturnType<typeof setTimeout> | undefined;
|
||||
const refresh = async () => {
|
||||
try {
|
||||
const response = await getExternalGenerationJobStatus(jobId);
|
||||
if (disposed) return;
|
||||
setResolvedJob(response.job);
|
||||
if (
|
||||
response.job.status === 'completed' &&
|
||||
!completionNotifiedRef.current
|
||||
) {
|
||||
completionNotifiedRef.current = true;
|
||||
onJobCompleted?.();
|
||||
}
|
||||
if (
|
||||
response.job.status === 'queued' ||
|
||||
response.job.status === 'running'
|
||||
) {
|
||||
timeoutId = setTimeout(refresh, 1500);
|
||||
}
|
||||
} catch {
|
||||
if (!disposed) timeoutId = setTimeout(refresh, 3000);
|
||||
}
|
||||
};
|
||||
void refresh();
|
||||
return () => {
|
||||
disposed = true;
|
||||
if (timeoutId) clearTimeout(timeoutId);
|
||||
};
|
||||
}, [onJobCompleted, toolCall.externalJobId]);
|
||||
const effectiveJobStatus = jobStatus ?? resolvedJob?.status ?? null;
|
||||
const generationResult =
|
||||
resolvedJob?.result && typeof resolvedJob.result === 'object'
|
||||
? (resolvedJob.result as Record<string, unknown>)
|
||||
: null;
|
||||
const resultImageSrc =
|
||||
typeof generationResult?.imageSrc === 'string'
|
||||
? generationResult.imageSrc
|
||||
: null;
|
||||
const resultVideoSrc =
|
||||
typeof generationResult?.videoSrc === 'string'
|
||||
? generationResult.videoSrc
|
||||
: null;
|
||||
const resultAudioSrc =
|
||||
typeof generationResult?.audioSrc === 'string'
|
||||
? generationResult.audioSrc
|
||||
: null;
|
||||
const resultObjectKey =
|
||||
typeof generationResult?.objectKey === 'string'
|
||||
? generationResult.objectKey
|
||||
: null;
|
||||
const isCancelled = Boolean(toolCall.cancelledAt);
|
||||
const isExecuting =
|
||||
effectiveJobStatus === 'queued' || effectiveJobStatus === 'running';
|
||||
const statusLabel =
|
||||
toolCall.status === 'completed'
|
||||
effectiveJobStatus === 'completed'
|
||||
? '已完成'
|
||||
: toolCall.status === 'cancelled'
|
||||
: isCancelled
|
||||
? '已取消'
|
||||
: toolCall.status === 'failed'
|
||||
: effectiveJobStatus === 'failed'
|
||||
? '失败'
|
||||
: toolCall.status === 'pending_confirmation'
|
||||
: !toolCall.externalJobId
|
||||
? '待确认'
|
||||
: '执行中';
|
||||
return (
|
||||
@@ -75,9 +149,9 @@ function ToolCallView({ toolCall }: { toolCall: EditorAgentToolCall }) {
|
||||
<div className="flex items-center gap-2">
|
||||
{isExecuting ? (
|
||||
<Loader2 className="h-3.5 w-3.5 animate-spin" aria-hidden="true" />
|
||||
) : toolCall.status === 'completed' ? (
|
||||
) : effectiveJobStatus === 'completed' ? (
|
||||
<Check className="h-3.5 w-3.5" aria-hidden="true" />
|
||||
) : toolCall.status === 'cancelled' ? (
|
||||
) : isCancelled ? (
|
||||
<X className="h-3.5 w-3.5" aria-hidden="true" />
|
||||
) : (
|
||||
<ImageIcon className="h-3.5 w-3.5" aria-hidden="true" />
|
||||
@@ -85,8 +159,10 @@ function ToolCallView({ toolCall }: { toolCall: EditorAgentToolCall }) {
|
||||
<span>{editorAgentToolLabel(toolCall.toolName)}</span>
|
||||
<span className="ml-auto text-slate-400">{statusLabel}</span>
|
||||
</div>
|
||||
{toolCall.error ? (
|
||||
<div className="mt-1 text-red-600">{toolCall.error}</div>
|
||||
{toolCall.error || resolvedJob?.error ? (
|
||||
<div className="mt-1 text-red-600">
|
||||
{toolCall.error ?? resolvedJob?.error}
|
||||
</div>
|
||||
) : null}
|
||||
{toolCall.images.length ? (
|
||||
<div className="mt-2 grid grid-cols-3 gap-2">
|
||||
@@ -105,6 +181,16 @@ function ToolCallView({ toolCall }: { toolCall: EditorAgentToolCall }) {
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
) : resultImageSrc ? (
|
||||
<div className="mt-2 overflow-hidden rounded-xl border border-slate-200 bg-slate-100">
|
||||
<ResolvedAssetImage
|
||||
src={resultImageSrc}
|
||||
objectKey={resultObjectKey}
|
||||
refreshKey={toolCall.externalJobId ?? toolCall.toolName}
|
||||
alt=""
|
||||
className="h-20 w-full object-cover"
|
||||
/>
|
||||
</div>
|
||||
) : null}
|
||||
{videos.length ? (
|
||||
<div className="mt-2 grid grid-cols-1 gap-2">
|
||||
@@ -128,6 +214,16 @@ function ToolCallView({ toolCall }: { toolCall: EditorAgentToolCall }) {
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
) : resultVideoSrc ? (
|
||||
<ResolvedAssetVideo
|
||||
src={resultVideoSrc}
|
||||
objectKey={resultObjectKey}
|
||||
refreshKey={toolCall.externalJobId ?? toolCall.toolName}
|
||||
controls
|
||||
playsInline
|
||||
preload="metadata"
|
||||
className="mt-2 max-h-56 w-full rounded-xl bg-black object-contain"
|
||||
/>
|
||||
) : null}
|
||||
{audios.length ? (
|
||||
<div className="mt-2 space-y-2">
|
||||
@@ -153,6 +249,15 @@ function ToolCallView({ toolCall }: { toolCall: EditorAgentToolCall }) {
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
) : resultAudioSrc ? (
|
||||
<ResolvedAssetAudio
|
||||
src={resultAudioSrc}
|
||||
objectKey={resultObjectKey}
|
||||
refreshKey={toolCall.externalJobId ?? toolCall.toolName}
|
||||
controls
|
||||
preload="metadata"
|
||||
className="mt-2 w-full"
|
||||
/>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
@@ -185,23 +290,29 @@ export function ThinkingBubble() {
|
||||
|
||||
type MessageBubbleProps = {
|
||||
message: EditorAgentMessage;
|
||||
jobStatus?: ExternalGenerationJobStatus | null;
|
||||
busyAction: 'confirm' | 'cancel' | null;
|
||||
onConfirmToolCall: (messageId: number) => void;
|
||||
onCancelToolCall: (messageId: number) => void;
|
||||
onJobCompleted?: () => void;
|
||||
};
|
||||
|
||||
export function MessageBubble({
|
||||
message,
|
||||
jobStatus,
|
||||
busyAction,
|
||||
onConfirmToolCall,
|
||||
onCancelToolCall,
|
||||
onJobCompleted,
|
||||
}: MessageBubbleProps) {
|
||||
if (message.role === 'system' && !message.toolCall) {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
message.role === 'system' &&
|
||||
message.toolCall?.status === 'pending_confirmation'
|
||||
message.toolCall &&
|
||||
!message.toolCall.externalJobId &&
|
||||
!message.toolCall.cancelledAt
|
||||
) {
|
||||
return (
|
||||
<PendingToolCall
|
||||
@@ -230,7 +341,7 @@ export function MessageBubble({
|
||||
: `max-w-[86%] rounded-3xl px-3.5 py-3 text-sm leading-6 shadow-sm ${
|
||||
isUser
|
||||
? 'bg-slate-900 text-white'
|
||||
: message.toolCall?.status === 'failed'
|
||||
: jobStatus === 'failed'
|
||||
? 'border border-red-200 bg-red-50 text-red-700'
|
||||
: 'border border-slate-200 bg-white text-slate-700'
|
||||
}`
|
||||
@@ -249,7 +360,13 @@ export function MessageBubble({
|
||||
))}
|
||||
</div>
|
||||
) : null}
|
||||
{message.toolCall ? <ToolCallView toolCall={message.toolCall} /> : null}
|
||||
{message.toolCall ? (
|
||||
<ToolCallView
|
||||
toolCall={message.toolCall}
|
||||
jobStatus={jobStatus}
|
||||
onJobCompleted={onJobCompleted}
|
||||
/>
|
||||
) : null}
|
||||
</div>
|
||||
</article>
|
||||
);
|
||||
|
||||
+27
-26
@@ -81,7 +81,7 @@ function createClient(): EditorAgentConversationClient {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'generate_image',
|
||||
status: 'completed',
|
||||
externalJobId: 'task-generated-1',
|
||||
args: {},
|
||||
displayArgs: {
|
||||
stringArgs: [],
|
||||
@@ -157,7 +157,7 @@ describe('useEditorAgentConversation', () => {
|
||||
expect(result.current.messages[1]?.toolCall).toEqual(
|
||||
expect.objectContaining({
|
||||
toolName: 'generate_image',
|
||||
status: 'completed',
|
||||
externalJobId: 'task-generated-1',
|
||||
images: [
|
||||
expect.objectContaining({
|
||||
resourceId: 'resource-result-1',
|
||||
@@ -276,7 +276,6 @@ describe('useEditorAgentConversation', () => {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'edit-image',
|
||||
status: 'pending_confirmation',
|
||||
args: {
|
||||
object_image_id: 'source-image',
|
||||
prompt: '换成像素风',
|
||||
@@ -297,7 +296,7 @@ describe('useEditorAgentConversation', () => {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'edit-image',
|
||||
status: 'completed',
|
||||
externalJobId: 'task-edit-1',
|
||||
args: {
|
||||
object_image_id: 'source-image',
|
||||
prompt: '换成像素风',
|
||||
@@ -338,7 +337,7 @@ describe('useEditorAgentConversation', () => {
|
||||
|
||||
const systemMsg = result.current.messages[1];
|
||||
expect(systemMsg?.role).toBe('system');
|
||||
expect(systemMsg?.toolCall?.status).toBe('pending_confirmation');
|
||||
expect(systemMsg?.toolCall?.externalJobId).toBeUndefined();
|
||||
|
||||
const messageCount = result.current.messages.length;
|
||||
await act(async () => {
|
||||
@@ -347,14 +346,16 @@ describe('useEditorAgentConversation', () => {
|
||||
|
||||
expect(client.confirmToolCall).toHaveBeenCalledWith('conversation-1', 1);
|
||||
expect(result.current.messages).toHaveLength(messageCount);
|
||||
expect(result.current.messages[1]?.toolCall?.status).toBe('completed');
|
||||
expect(result.current.messages[1]?.toolCall?.externalJobId).toBe(
|
||||
'task-edit-1',
|
||||
);
|
||||
expect(result.current.messages[1]?.toolCall?.images[0]?.objectKey).toBe(
|
||||
'generated/result.png',
|
||||
);
|
||||
expect(onCanvasRefreshRequested).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('marks a confirmation as executing before the request finishes', async () => {
|
||||
it('keeps the action busy until confirmation returns an external job id', async () => {
|
||||
const client = createClient();
|
||||
const pendingMessage: EditorAgentMessage = {
|
||||
id: 0,
|
||||
@@ -363,7 +364,6 @@ describe('useEditorAgentConversation', () => {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'edit-image',
|
||||
status: 'pending_confirmation',
|
||||
args: { object_image_id: 'source-image', prompt: '换成像素风' },
|
||||
displayArgs: createEditImageDisplayArgs('换成像素风'),
|
||||
images: [],
|
||||
@@ -375,11 +375,12 @@ describe('useEditorAgentConversation', () => {
|
||||
...pendingMessage,
|
||||
toolCall: {
|
||||
...pendingMessage.toolCall!,
|
||||
status: 'completed',
|
||||
externalJobId: 'task-edit-2',
|
||||
images: [],
|
||||
},
|
||||
};
|
||||
let resolveConfirmation: ((message: EditorAgentMessage) => void) | undefined;
|
||||
let resolveConfirmation:
|
||||
((message: EditorAgentMessage) => void) | undefined;
|
||||
vi.mocked(client.getConversation).mockResolvedValue({
|
||||
conversationId: 'conversation-1',
|
||||
projectId: 'project-1',
|
||||
@@ -399,26 +400,27 @@ describe('useEditorAgentConversation', () => {
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(result.current.messages[0]?.toolCall?.status).toBe(
|
||||
'pending_confirmation',
|
||||
);
|
||||
expect(result.current.messages[0]?.role).toBe('system');
|
||||
expect(
|
||||
result.current.messages[0]?.toolCall?.externalJobId,
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
let confirmationPromise: Promise<EditorAgentMessage | null> | undefined;
|
||||
act(() => {
|
||||
void result.current.confirmToolCall(0);
|
||||
confirmationPromise = result.current.confirmToolCall(0);
|
||||
});
|
||||
|
||||
await waitFor(() => {
|
||||
expect(result.current.messages[0]?.toolCall?.status).toBe('executing');
|
||||
});
|
||||
await result.current.confirmToolCall(0);
|
||||
await waitFor(() => expect(resolveConfirmation).toBeTypeOf('function'));
|
||||
expect(client.confirmToolCall).toHaveBeenCalledTimes(1);
|
||||
|
||||
await act(async () => {
|
||||
resolveConfirmation?.(completedMessage);
|
||||
await confirmationPromise;
|
||||
});
|
||||
await waitFor(() => {
|
||||
expect(result.current.messages[0]?.toolCall?.status).toBe('completed');
|
||||
expect(result.current.messages[0]?.toolCall?.externalJobId).toBe(
|
||||
'task-edit-2',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -431,7 +433,6 @@ describe('useEditorAgentConversation', () => {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'edit-image',
|
||||
status: 'pending_confirmation' as const,
|
||||
args: { object_image_id: 'source-image', prompt: '换成像素风' },
|
||||
displayArgs: createEditImageDisplayArgs('换成像素风'),
|
||||
images: [],
|
||||
@@ -452,7 +453,7 @@ describe('useEditorAgentConversation', () => {
|
||||
text: 'internal cancelled tool output',
|
||||
toolCall: {
|
||||
...pendingMessage.toolCall,
|
||||
status: 'cancelled',
|
||||
cancelledAt: '2026-07-03T00:00:01.000Z',
|
||||
},
|
||||
});
|
||||
const onCanvasRefreshRequested = vi.fn();
|
||||
@@ -465,9 +466,7 @@ describe('useEditorAgentConversation', () => {
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(result.current.messages[0]?.toolCall?.status).toBe(
|
||||
'pending_confirmation',
|
||||
);
|
||||
expect(result.current.messages[0]?.toolCall?.cancelledAt).toBeUndefined();
|
||||
});
|
||||
await act(async () => {
|
||||
await result.current.cancelToolCall(0);
|
||||
@@ -475,7 +474,9 @@ describe('useEditorAgentConversation', () => {
|
||||
|
||||
expect(client.cancelToolCall).toHaveBeenCalledWith('conversation-1', 0);
|
||||
expect(result.current.messages).toHaveLength(1);
|
||||
expect(result.current.messages[0]?.toolCall?.status).toBe('cancelled');
|
||||
expect(result.current.messages[0]?.toolCall?.cancelledAt).toBe(
|
||||
'2026-07-03T00:00:01.000Z',
|
||||
);
|
||||
expect(onCanvasRefreshRequested).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
|
||||
@@ -295,13 +295,15 @@ export function useEditorAgentConversation({
|
||||
const requestCanvasRefreshForMessages = useCallback(
|
||||
(nextMessages: EditorAgentMessage[]) => {
|
||||
if (
|
||||
nextMessages.some(
|
||||
(message) =>
|
||||
message.toolCall?.status === 'completed' &&
|
||||
(message.toolCall.images.length > 0 ||
|
||||
(message.toolCall.videos?.length ?? 0) > 0 ||
|
||||
(message.toolCall.audios?.length ?? 0) > 0),
|
||||
)
|
||||
nextMessages.some((message) => {
|
||||
const toolCall = message.toolCall;
|
||||
return Boolean(
|
||||
toolCall?.externalJobId &&
|
||||
(toolCall.images.length > 0 ||
|
||||
(toolCall.videos?.length ?? 0) > 0 ||
|
||||
(toolCall.audios?.length ?? 0) > 0),
|
||||
);
|
||||
})
|
||||
) {
|
||||
onCanvasRefreshRequested?.();
|
||||
}
|
||||
@@ -418,23 +420,6 @@ export function useEditorAgentConversation({
|
||||
setToolCallAction(nextAction);
|
||||
setErrorMessage(null);
|
||||
|
||||
if (action === 'confirm') {
|
||||
setMessages((currentMessages) =>
|
||||
currentMessages.map((message) =>
|
||||
message.id === messageId &&
|
||||
message.toolCall?.status === 'pending_confirmation'
|
||||
? {
|
||||
...message,
|
||||
toolCall: {
|
||||
...message.toolCall,
|
||||
status: 'executing',
|
||||
},
|
||||
}
|
||||
: message,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const updatedMessage = await (action === 'confirm'
|
||||
? client.confirmToolCall(conversationId, messageId)
|
||||
|
||||
@@ -213,7 +213,7 @@ function createEditorAgentDetailWithGeneration(
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'generate_image',
|
||||
status: 'completed',
|
||||
externalJobId: 'task-generation-1',
|
||||
args: {},
|
||||
displayArgs: {
|
||||
stringArgs: [],
|
||||
@@ -2069,7 +2069,6 @@ describe('ImageCanvasEditorView', () => {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'edit-image',
|
||||
status: 'pending_confirmation',
|
||||
args: {
|
||||
object_image_id: 'source-image',
|
||||
prompt: '把图片换成像素风',
|
||||
@@ -2092,10 +2091,9 @@ describe('ImageCanvasEditorView', () => {
|
||||
attachments: [],
|
||||
toolCall: {
|
||||
toolName: 'edit-image',
|
||||
status: 'completed',
|
||||
externalJobId: 'task-edit-canvas-1',
|
||||
args: {},
|
||||
displayArgs:
|
||||
createEditImageToolCallDisplayArgs('把图片换成像素风'),
|
||||
displayArgs: createEditImageToolCallDisplayArgs('把图片换成像素风'),
|
||||
images: [
|
||||
{
|
||||
resourceId: null,
|
||||
|
||||
Reference in New Issue
Block a user