修复画布Agent聊天生成体验
让画布 Agent 消息统一走后端规划与 LLM 回复 修复生成图片在聊天、画布和素材库中的展示与恢复 补齐右键删除、画布焦点、聊天滚轮和面板避让交互 同步 Agent 事件契约、文档和定向回归测试
This commit is contained in:
@@ -26,7 +26,7 @@
|
||||
|
||||
## 当前分支落地状态
|
||||
|
||||
- 已落地:会话元数据、OSS 消息文档、会话 CRUD、SSE 消息流、后端 LLM / 启发式工具规划、右侧对话面板、会话历史、新建 / 软删会话、停止当前 SSE 回合、附件从画布资源 / 账号素材库选择,以及五类图片工具对既有生成入口的复用。
|
||||
- 已落地:会话元数据、OSS 消息文档、会话 CRUD、SSE 消息流、后端 LLM 工具规划、右侧对话面板、会话历史、新建 / 软删会话、停止当前 SSE 回合、附件从画布资源 / 账号素材库选择,以及五类图片工具对既有生成入口的复用。
|
||||
- 已落地:`tool_started` / `tool_completed` 事件携带 `status`;工具失败时也会写入失败 generation record,并随后发送 `stage=failed` 与 `error`,前端应保留消息内失败条目。
|
||||
- 未落地:附件弹窗末尾上传格、跨刷新异步生成恢复、external generation task 轮询回填。未落地前,对话消息状态只表示本次 SSE 回合记录,不作为后台任务队列真相。
|
||||
|
||||
@@ -42,7 +42,8 @@
|
||||
## 生成结果落画板(对现有占位规则的例外)
|
||||
|
||||
- 对话入口触发的生成**不创建"即将生成"画布占位**(区别于其余生成面板);生成中状态由对话消息流承载。
|
||||
- 生成完成后:结果图按统一 placement 避让模型(视口中心就近、避开现有图层、32px 间距)落画板为新图层,同时在对话消息内显示缩略图(点击可让画布聚焦到对应图层;如果当前画布尚无该资源图层,前端先刷新工程快照再聚焦)。
|
||||
- 生成完成后:结果图按统一 placement 避让模型(视口中心就近、避开现有图层、32px 间距)落画板为新图层,同时登记到默认项目素材库,并在对话消息内显示纯缩略图;前端收到 `generation_result` 后立即刷新工程快照与素材库,缩略图本身不显示名称也不承担图层跳转。
|
||||
- 消息内生成结果缩略图必须携带并优先使用 `objectKey` / `assetObjectId`,前端通过 `ResolvedAssetImage` / `/api/assets/read-url` 换签后渲染,不能把裸 `/generated-*` 私有路径直接交给 `<img>`。
|
||||
- 当前第一阶段通过既有编辑器生成 BFF 的 `canvasCompletion` 写回工程快照;刷新后异步任务恢复和轮询回填属于后续能力,不在本阶段声明为已完成。
|
||||
- 该例外已同步登记在《生成类面板Lovart统一改造方案-2026-06-17》「画布占位落点」节。
|
||||
|
||||
@@ -52,6 +53,7 @@
|
||||
- 对话框与左侧素材 / 图层侧栏也互斥:打开画布 Agent 时收起左侧栏;再次打开素材、图层或任务侧栏时收起 Agent 面板。
|
||||
- 桌面端对话框固定宽约 360–400px;移动端抽屉式全宽覆盖;收起态为胶囊/圆形入口按钮。
|
||||
- 会话管理入口在对话框头部:当前会话标题 + 历史会话下拉(按更新时间倒序)+ 新建对话按钮,全部包在对话框内。
|
||||
- 收起对话框只是隐藏面板,不卸载当前会话 hook;流式回复、`生成中` 阶段和停止按钮状态必须在收起 / 重新打开之间保持一致。
|
||||
|
||||
## 附件
|
||||
|
||||
@@ -65,7 +67,9 @@
|
||||
|
||||
## LLM 与计费
|
||||
|
||||
- 编排复用 `creative_agent_gpt5_client` 的 LLM 接入配置(同 provider/env,独立用途标识),function-calling 注册五类工具。
|
||||
- 编排复用 `creative_agent_gpt5_client` 的 LLM 接入配置(同 provider/env,独立用途标识),但画布 Agent 的轻量规划请求显式使用 `gpt-4o`,避免把简单 JSON 意图识别交给容易长 reasoning 的 gpt-5;function-calling 注册五类工具。
|
||||
- 每个用户回合必须由 LLM 返回结构化计划;LLM 未配置、请求失败或返回格式不可解析时,后端写入明确错误消息,不使用本地关键词或“收到:...”回显兜底。
|
||||
- 画布 Agent 规划请求使用 Responses API、1024 `max_output_tokens` 和 60 秒 Agent 专用请求超时;生成图片/编辑图片仍走对应生成工具和模型计费。
|
||||
- **对话回合免费**(聊天、分析回复不扣泥点),仅 Agent 实际触发生成工具时按对应模型定价扣泥点。
|
||||
- 工具调用前后端校验泥点余额;不足时该次生成失败并在对话中以明确错误气泡告知,对话本身可继续。
|
||||
|
||||
@@ -76,7 +80,7 @@
|
||||
1. 助手文本 SSE 流式输出;
|
||||
2. 阶段提示行(思考中 → 思考完成 → 生成中 → 完成/失败);
|
||||
3. 工具/模型标注行(生成时显示模型名 + 图标);
|
||||
4. 消息内生成结果缩略图(点击 → 画布聚焦对应图层);
|
||||
4. 消息内生成结果缩略图(纯预览,不显示名称,不点击聚焦图层);
|
||||
5. 生成中的进行中动画;
|
||||
6. 错误气泡(失败/余额不足,带原因);
|
||||
7. 发送中断:进行中时发送按钮变「停止」,可中断当前回合(已提交的生成任务不追回,照常落画板)。
|
||||
@@ -120,4 +124,5 @@
|
||||
|
||||
- 打开画布 Agent 后,任务侧栏和左侧素材 / 图层面板应关闭;再次打开任务侧栏或素材 / 图层面板时,Agent 面板应关闭。
|
||||
- 发送消息时先本地追加用户消息,再消费 SSE 增量;停止按钮只中断当前 SSE 回合,不追回已经提交的生成工具调用。
|
||||
- 点击 Agent 消息内生成结果缩略图时,若当前画布已有该资源图层则直接聚焦;若尚无该资源图层,应先刷新工程快照,再聚焦新图层。
|
||||
- Agent 消息内生成结果缩略图只用于预览,不显示名称,也不点击跳转图层;收到 `generation_result` 时统一刷新工程快照和素材库。
|
||||
- 对话内容可被用户选中复制;用户从输入框或对话内容点击回画布图层 / 生成器时,焦点应回到画布对象,Backspace / Delete 等画布快捷键继续生效。
|
||||
|
||||
@@ -46,6 +46,8 @@ export interface EditorAgentAttachmentRef {
|
||||
|
||||
export interface EditorAgentGeneratedImage {
|
||||
resourceId: string | null;
|
||||
objectKey?: string | null;
|
||||
assetObjectId?: string | null;
|
||||
imageSrc: string;
|
||||
thumbnailSrc: string | null;
|
||||
width: number | null;
|
||||
|
||||
@@ -18,7 +18,7 @@ use module_editor_agent::{
|
||||
EDITOR_AGENT_MESSAGE_ID_PREFIX, derive_conversation_title, editor_agent_messages_object_key,
|
||||
validate_user_message,
|
||||
};
|
||||
use platform_llm::{LlmMessage, LlmMessageContentPart, LlmTextRequest};
|
||||
use platform_llm::{LlmError, LlmErrorKind, LlmMessage, LlmMessageContentPart, LlmTextRequest};
|
||||
use platform_oss::{
|
||||
LegacyAssetPrefix, OssObjectAccess, OssPutObjectRequest, OssSignedGetObjectUrlRequest,
|
||||
};
|
||||
@@ -65,6 +65,9 @@ const EDITOR_AGENT_MESSAGES_DOCUMENT_MAX_BYTES: usize = 2 * 1024 * 1024;
|
||||
const EDITOR_AGENT_MESSAGES_READ_EXPIRE_SECONDS: u64 = 60;
|
||||
const EDITOR_AGENT_LLM_ATTACHMENT_URL_EXPIRE_SECONDS: u64 = 300;
|
||||
const EDITOR_AGENT_LLM_MAX_HISTORY_MESSAGES: usize = 12;
|
||||
const EDITOR_AGENT_LLM_PLANNING_MODEL: &str = "gpt-4o";
|
||||
const EDITOR_AGENT_LLM_MAX_OUTPUT_TOKENS: u32 = 1024;
|
||||
const EDITOR_AGENT_LLM_REQUEST_TIMEOUT_MS: u64 = 60_000;
|
||||
const EDITOR_AGENT_TOOL_CALL_ID_PREFIX: &str = "editor-agent-tool";
|
||||
const EDITOR_AGENT_CANVAS_RESULT_GAP: f64 = 32.0;
|
||||
type EditorAgentConversationLockMap = Mutex<BTreeMap<String, Arc<tokio::sync::Mutex<()>>>>;
|
||||
@@ -279,23 +282,35 @@ pub async fn stream_editor_agent_message(
|
||||
},
|
||||
));
|
||||
|
||||
let turn_plan = plan_editor_agent_turn(
|
||||
let mut assistant_status = EditorAgentMessageStatus::Completed;
|
||||
let mut generation_records = Vec::new();
|
||||
let mut planning_failed = false;
|
||||
|
||||
let turn_plan = match plan_editor_agent_turn(
|
||||
&stream_state,
|
||||
&stream_conversation,
|
||||
&document,
|
||||
&stream_user_message,
|
||||
)
|
||||
.await;
|
||||
let mut assistant_text = turn_plan.reply_text;
|
||||
if assistant_text.trim().is_empty() {
|
||||
assistant_text = heuristic_editor_agent_turn_plan(
|
||||
stream_user_message.text.as_str(),
|
||||
stream_user_message.attachments.as_slice(),
|
||||
)
|
||||
.reply_text;
|
||||
}
|
||||
let mut assistant_status = EditorAgentMessageStatus::Completed;
|
||||
let mut generation_records = Vec::new();
|
||||
.await {
|
||||
Ok(turn_plan) => turn_plan,
|
||||
Err(error) => {
|
||||
let app_error = error.into_app_error();
|
||||
let error_message = app_error.body_text();
|
||||
assistant_status = EditorAgentMessageStatus::Failed;
|
||||
planning_failed = true;
|
||||
EditorAgentTurnPlan {
|
||||
reply_text: error_message,
|
||||
tool_call: None,
|
||||
}
|
||||
}
|
||||
};
|
||||
let assistant_text = turn_plan.reply_text;
|
||||
let assistant_kind = if assistant_status == EditorAgentMessageStatus::Failed {
|
||||
EditorAgentMessageKind::Error
|
||||
} else {
|
||||
EditorAgentMessageKind::Chat
|
||||
};
|
||||
|
||||
yield Ok::<Event, Infallible>(editor_agent_sse_json_event_or_error(
|
||||
"stage",
|
||||
@@ -310,10 +325,19 @@ pub async fn stream_editor_agent_message(
|
||||
conversation_id: stream_conversation.conversation_id.clone(),
|
||||
message_id: stream_assistant_message_id.clone(),
|
||||
role: EditorAgentMessageRole::Assistant,
|
||||
kind: EditorAgentMessageKind::Chat,
|
||||
kind: assistant_kind,
|
||||
text_delta: assistant_text.clone(),
|
||||
},
|
||||
));
|
||||
if planning_failed {
|
||||
yield Ok::<Event, Infallible>(editor_agent_sse_json_event_or_error(
|
||||
"stage",
|
||||
EditorAgentStageEvent {
|
||||
conversation_id: stream_conversation.conversation_id.clone(),
|
||||
stage: EditorAgentStage::Failed,
|
||||
},
|
||||
));
|
||||
}
|
||||
|
||||
if let Some(tool_call) = turn_plan.tool_call {
|
||||
let tool_call_id = build_prefixed_uuid_id(EDITOR_AGENT_TOOL_CALL_ID_PREFIX);
|
||||
@@ -951,30 +975,65 @@ struct EditorAgentToolExecutionError {
|
||||
error: AppError,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum EditorAgentPlanningError {
|
||||
LlmUnavailable,
|
||||
LlmRequestFailed(LlmError),
|
||||
InvalidLlmResponse,
|
||||
}
|
||||
|
||||
impl EditorAgentPlanningError {
|
||||
fn into_app_error(self) -> AppError {
|
||||
match self {
|
||||
Self::LlmUnavailable => AppError::from_status(StatusCode::SERVICE_UNAVAILABLE)
|
||||
.with_details(json!({
|
||||
"provider": "editor-agent-llm",
|
||||
"message": "画布 Agent 的 LLM 未配置,无法处理这句话。",
|
||||
})),
|
||||
Self::LlmRequestFailed(error) => {
|
||||
let status = match error.kind() {
|
||||
LlmErrorKind::Timeout => StatusCode::GATEWAY_TIMEOUT,
|
||||
LlmErrorKind::InvalidConfig | LlmErrorKind::InvalidRequest => {
|
||||
StatusCode::SERVICE_UNAVAILABLE
|
||||
}
|
||||
LlmErrorKind::Connectivity
|
||||
| LlmErrorKind::Upstream
|
||||
| LlmErrorKind::StreamUnavailable
|
||||
| LlmErrorKind::EmptyResponse
|
||||
| LlmErrorKind::Transport
|
||||
| LlmErrorKind::Deserialize => StatusCode::BAD_GATEWAY,
|
||||
};
|
||||
AppError::from_status(status).with_details(json!({
|
||||
"provider": "editor-agent-llm",
|
||||
"message": format!("画布 Agent 调用 LLM 失败:{error}"),
|
||||
}))
|
||||
}
|
||||
Self::InvalidLlmResponse => AppError::from_status(StatusCode::BAD_GATEWAY)
|
||||
.with_details(json!({
|
||||
"provider": "editor-agent-llm",
|
||||
"message": "画布 Agent 的 LLM 返回格式错误,未能解析回复。",
|
||||
})),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn plan_editor_agent_turn(
|
||||
state: &AppState,
|
||||
conversation: &EditorAgentConversationRecord,
|
||||
document: &EditorAgentConversationMessagesDocument,
|
||||
user_message: &EditorAgentMessage,
|
||||
) -> EditorAgentTurnPlan {
|
||||
let fallback = heuristic_editor_agent_turn_plan(
|
||||
user_message.text.as_str(),
|
||||
user_message.attachments.as_slice(),
|
||||
);
|
||||
) -> Result<EditorAgentTurnPlan, EditorAgentPlanningError> {
|
||||
let Some(llm_client) = state.creative_agent_gpt5_client() else {
|
||||
return fallback;
|
||||
return Err(EditorAgentPlanningError::LlmUnavailable);
|
||||
};
|
||||
|
||||
let request = build_editor_agent_llm_request(state, conversation, document, user_message).await;
|
||||
match llm_client.request_text(request).await {
|
||||
Ok(response) => parse_editor_agent_turn_plan(
|
||||
response.content.as_str(),
|
||||
fallback.reply_text.as_str(),
|
||||
user_message.text.as_str(),
|
||||
)
|
||||
.unwrap_or(fallback),
|
||||
Err(_) => fallback,
|
||||
}
|
||||
let response = llm_client
|
||||
.request_text(request)
|
||||
.await
|
||||
.map_err(EditorAgentPlanningError::LlmRequestFailed)?;
|
||||
parse_editor_agent_turn_plan(response.content.as_str())
|
||||
.ok_or(EditorAgentPlanningError::InvalidLlmResponse)
|
||||
}
|
||||
|
||||
async fn build_editor_agent_llm_request(
|
||||
@@ -997,7 +1056,9 @@ async fn build_editor_agent_llm_request(
|
||||
LlmMessage::user_multimodal(user_parts),
|
||||
])
|
||||
.with_responses_api()
|
||||
.with_max_tokens(900)
|
||||
.with_model(EDITOR_AGENT_LLM_PLANNING_MODEL)
|
||||
.with_max_tokens(EDITOR_AGENT_LLM_MAX_OUTPUT_TOKENS)
|
||||
.with_request_timeout_ms(EDITOR_AGENT_LLM_REQUEST_TIMEOUT_MS)
|
||||
}
|
||||
|
||||
fn editor_agent_llm_system_prompt() -> String {
|
||||
@@ -1080,34 +1141,30 @@ fn sign_editor_agent_attachment_image_url(
|
||||
.map(|signed| signed.signed_url)
|
||||
}
|
||||
|
||||
fn parse_editor_agent_turn_plan(
|
||||
raw_text: &str,
|
||||
fallback_reply_text: &str,
|
||||
fallback_prompt: &str,
|
||||
) -> Option<EditorAgentTurnPlan> {
|
||||
fn parse_editor_agent_turn_plan(raw_text: &str) -> Option<EditorAgentTurnPlan> {
|
||||
let json_text = extract_editor_agent_json_object(raw_text)?;
|
||||
let raw: EditorAgentRawTurnPlan = serde_json::from_str(json_text).ok()?;
|
||||
let reply_text = normalize_optional_string(raw.reply_text)
|
||||
.or_else(|| normalize_optional_string(Some(fallback_reply_text.to_string())))
|
||||
.unwrap_or_else(|| build_echo_assistant_text(fallback_prompt, 0));
|
||||
let tool_call = raw.tool_call.and_then(|tool| {
|
||||
let prompt = normalize_optional_string(tool.prompt)
|
||||
.or_else(|| normalize_optional_string(Some(fallback_prompt.to_string())))?;
|
||||
Some(EditorAgentToolCallPlan {
|
||||
tool_name: tool.tool_name,
|
||||
prompt,
|
||||
summary: normalize_optional_string(tool.summary),
|
||||
model: normalize_optional_string(tool.model),
|
||||
aspect_ratio: normalize_optional_string(tool.aspect_ratio),
|
||||
image_size: normalize_optional_string(tool.image_size),
|
||||
icon_descriptions: tool
|
||||
.icon_descriptions
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.filter_map(|description| normalize_optional_string(Some(description)))
|
||||
.collect(),
|
||||
})
|
||||
});
|
||||
let reply_text = normalize_optional_string(raw.reply_text)?;
|
||||
let tool_call = match raw.tool_call {
|
||||
Some(tool) => {
|
||||
let prompt = normalize_optional_string(tool.prompt)?;
|
||||
Some(EditorAgentToolCallPlan {
|
||||
tool_name: tool.tool_name,
|
||||
prompt,
|
||||
summary: normalize_optional_string(tool.summary),
|
||||
model: normalize_optional_string(tool.model),
|
||||
aspect_ratio: normalize_optional_string(tool.aspect_ratio),
|
||||
image_size: normalize_optional_string(tool.image_size),
|
||||
icon_descriptions: tool
|
||||
.icon_descriptions
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.filter_map(|description| normalize_optional_string(Some(description)))
|
||||
.collect(),
|
||||
})
|
||||
}
|
||||
None => None,
|
||||
};
|
||||
|
||||
Some(EditorAgentTurnPlan {
|
||||
reply_text,
|
||||
@@ -1129,6 +1186,7 @@ fn extract_editor_agent_json_object(raw_text: &str) -> Option<&str> {
|
||||
(start <= end).then_some(&without_fence[start..=end])
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn heuristic_editor_agent_turn_plan(
|
||||
text: &str,
|
||||
attachments: &[EditorAgentAttachmentRef],
|
||||
@@ -1202,6 +1260,7 @@ fn heuristic_editor_agent_turn_plan(
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn contains_any(text: &str, needles: &[&str]) -> bool {
|
||||
needles.iter().any(|needle| text.contains(needle))
|
||||
}
|
||||
@@ -1316,7 +1375,7 @@ async fn execute_editor_agent_tool_call(
|
||||
project_id: Some(conversation.project_id.clone()),
|
||||
asset_kind: Some(editor_agent_tool_asset_kind(tool_name).to_string()),
|
||||
generation_inputs,
|
||||
asset_folder_id: None,
|
||||
asset_folder_id: Some(editor_agent_default_asset_folder_id()),
|
||||
asset_label: tool_call.summary.clone(),
|
||||
source_resource_id: user_message
|
||||
.attachments
|
||||
@@ -1351,7 +1410,7 @@ async fn execute_editor_agent_tool_call(
|
||||
project_id: Some(conversation.project_id.clone()),
|
||||
asset_kind: Some(editor_agent_tool_asset_kind(tool_name).to_string()),
|
||||
generation_inputs,
|
||||
asset_folder_id: None,
|
||||
asset_folder_id: Some(editor_agent_default_asset_folder_id()),
|
||||
asset_label: tool_call.summary.clone(),
|
||||
source_resource_id: user_message
|
||||
.attachments
|
||||
@@ -1390,7 +1449,7 @@ async fn execute_editor_agent_tool_call(
|
||||
image_size: tool_call.image_size.clone(),
|
||||
project_id: Some(conversation.project_id.clone()),
|
||||
generation_inputs,
|
||||
asset_folder_id: None,
|
||||
asset_folder_id: Some(editor_agent_default_asset_folder_id()),
|
||||
canvas_completion: Some(completion),
|
||||
},
|
||||
)
|
||||
@@ -1508,6 +1567,8 @@ fn editor_agent_generated_images_from_response(
|
||||
.map(|image_src| {
|
||||
vec![EditorAgentGeneratedImage {
|
||||
resource_id: None,
|
||||
object_key: None,
|
||||
asset_object_id: None,
|
||||
image_src: image_src.to_string(),
|
||||
thumbnail_src: None,
|
||||
width: data
|
||||
@@ -1535,6 +1596,14 @@ fn editor_agent_generated_image_from_resource(
|
||||
.get("resourceId")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_string),
|
||||
object_key: resource
|
||||
.get("objectKey")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_string),
|
||||
asset_object_id: resource
|
||||
.get("assetObjectId")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_string),
|
||||
image_src: resource
|
||||
.get("imageSrc")
|
||||
.and_then(Value::as_str)
|
||||
@@ -1589,6 +1658,10 @@ fn editor_agent_tool_asset_kind(tool_name: EditorAgentToolName) -> &'static str
|
||||
}
|
||||
}
|
||||
|
||||
fn editor_agent_default_asset_folder_id() -> String {
|
||||
"project".to_string()
|
||||
}
|
||||
|
||||
fn editor_agent_tool_default_summary(tool_name: EditorAgentToolName) -> String {
|
||||
match tool_name {
|
||||
EditorAgentToolName::GenerateImage => "生成图片",
|
||||
@@ -1658,6 +1731,7 @@ fn next_editor_agent_canvas_position(layers: Value, _width: f64, _height: f64) -
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn build_echo_assistant_text(text: &str, attachment_count: usize) -> String {
|
||||
let trimmed = text.trim();
|
||||
if trimmed.is_empty() {
|
||||
@@ -1838,6 +1912,41 @@ mod tests {
|
||||
assert_eq!(normalized.height, Some(128));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generated_image_from_resource_keeps_private_asset_keys() {
|
||||
let image = editor_agent_generated_image_from_resource(
|
||||
&json!({
|
||||
"resourceId": "resource-1",
|
||||
"imageSrc": "/generated-editor-assets/result.png",
|
||||
"thumbnailSrc": "/generated-editor-assets/result-thumb.png",
|
||||
"objectKey": "generated-editor-assets/result.png",
|
||||
"assetObjectId": "asset-object-result",
|
||||
"width": 1024,
|
||||
"height": 768
|
||||
}),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
);
|
||||
|
||||
assert_eq!(image.resource_id.as_deref(), Some("resource-1"));
|
||||
assert_eq!(
|
||||
image.object_key.as_deref(),
|
||||
Some("generated-editor-assets/result.png")
|
||||
);
|
||||
assert_eq!(
|
||||
image.asset_object_id.as_deref(),
|
||||
Some("asset-object-result")
|
||||
);
|
||||
assert_eq!(image.width, Some(1024));
|
||||
assert_eq!(image.height, Some(768));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_agent_generations_target_default_asset_library_folder() {
|
||||
assert_eq!(editor_agent_default_asset_folder_id(), "project");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_agent_turn_plan_parses_fenced_json_tool_call() {
|
||||
let plan = parse_editor_agent_turn_plan(
|
||||
@@ -1852,8 +1961,6 @@ mod tests {
|
||||
}
|
||||
}
|
||||
```"#,
|
||||
"fallback",
|
||||
"像素风银发游侠",
|
||||
)
|
||||
.expect("fenced JSON should parse");
|
||||
|
||||
@@ -1865,6 +1972,23 @@ mod tests {
|
||||
assert_eq!(tool_call.image_size.as_deref(), Some("1K"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_agent_turn_plan_rejects_invalid_llm_response() {
|
||||
assert!(parse_editor_agent_turn_plan("收到:画一棵树").is_none());
|
||||
assert!(
|
||||
parse_editor_agent_turn_plan(
|
||||
r#"{"toolCall":{"toolName":"generate_image","prompt":"画一棵树"}}"#,
|
||||
)
|
||||
.is_none()
|
||||
);
|
||||
assert!(
|
||||
parse_editor_agent_turn_plan(
|
||||
r#"{"replyText":"我来生成图片。","toolCall":{"toolName":"generate_image"}}"#,
|
||||
)
|
||||
.is_none()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_agent_heuristic_routes_tools_from_user_text() {
|
||||
let character_plan =
|
||||
|
||||
@@ -44,6 +44,8 @@ pub async fn proxy_llm_chat_completions(
|
||||
max_tokens: None,
|
||||
enable_web_search: false,
|
||||
request_timeout_ms: None,
|
||||
response_reasoning_effort: None,
|
||||
response_text_verbosity: None,
|
||||
};
|
||||
|
||||
if payload.stream {
|
||||
|
||||
@@ -55,5 +55,7 @@ pub fn build_gpt5_multimodal_request(
|
||||
request_timeout_ms: None,
|
||||
enable_web_search: false,
|
||||
protocol: LlmTextProtocol::Responses,
|
||||
response_reasoning_effort: None,
|
||||
response_text_verbosity: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -81,6 +81,8 @@ pub struct LlmTextRequest {
|
||||
pub enable_web_search: bool,
|
||||
pub protocol: LlmTextProtocol,
|
||||
pub request_timeout_ms: Option<u64>,
|
||||
pub response_reasoning_effort: Option<LlmResponseReasoningEffort>,
|
||||
pub response_text_verbosity: Option<LlmResponseTextVerbosity>,
|
||||
}
|
||||
|
||||
// 文本协议必须由业务请求显式选择,避免全局默认模型把不同场景混到同一上游形态。
|
||||
@@ -90,6 +92,40 @@ pub enum LlmTextProtocol {
|
||||
Responses,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum LlmResponseReasoningEffort {
|
||||
Low,
|
||||
Medium,
|
||||
High,
|
||||
}
|
||||
|
||||
impl LlmResponseReasoningEffort {
|
||||
fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Low => "low",
|
||||
Self::Medium => "medium",
|
||||
Self::High => "high",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum LlmResponseTextVerbosity {
|
||||
Low,
|
||||
Medium,
|
||||
High,
|
||||
}
|
||||
|
||||
impl LlmResponseTextVerbosity {
|
||||
fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Low => "low",
|
||||
Self::Medium => "medium",
|
||||
Self::High => "high",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 上层在流式消费时拿到的是“累计文本 + 当前增量”,避免每层重新自己拼接。
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct LlmStreamDelta {
|
||||
@@ -213,6 +249,10 @@ struct ResponsesRequestBody {
|
||||
max_output_tokens: Option<u32>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
tools: Option<Vec<ResponsesWebSearchTool>>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
reasoning: Option<ResponsesReasoningOptions>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
text: Option<ResponsesTextOptions>,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
@@ -235,6 +275,16 @@ struct ResponsesWebSearchTool {
|
||||
max_keyword: u8,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct ResponsesReasoningOptions {
|
||||
effort: &'static str,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct ResponsesTextOptions {
|
||||
verbosity: &'static str,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
struct LlmRawFailureInputLog<'a> {
|
||||
@@ -519,6 +569,8 @@ impl LlmTextRequest {
|
||||
enable_web_search: false,
|
||||
protocol: LlmTextProtocol::ChatCompletions,
|
||||
request_timeout_ms: None,
|
||||
response_reasoning_effort: None,
|
||||
response_text_verbosity: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -549,6 +601,16 @@ impl LlmTextRequest {
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_response_reasoning_effort(mut self, effort: LlmResponseReasoningEffort) -> Self {
|
||||
self.response_reasoning_effort = Some(effort);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_response_text_verbosity(mut self, verbosity: LlmResponseTextVerbosity) -> Self {
|
||||
self.response_text_verbosity = Some(verbosity);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_request_timeout_ms(mut self, request_timeout_ms: u64) -> Self {
|
||||
self.request_timeout_ms = Some(request_timeout_ms);
|
||||
self
|
||||
@@ -1178,6 +1240,16 @@ fn build_request_body(
|
||||
max_keyword: 3,
|
||||
}]
|
||||
}),
|
||||
reasoning: request
|
||||
.response_reasoning_effort
|
||||
.map(|effort| ResponsesReasoningOptions {
|
||||
effort: effort.as_str(),
|
||||
}),
|
||||
text: request
|
||||
.response_text_verbosity
|
||||
.map(|verbosity| ResponsesTextOptions {
|
||||
verbosity: verbosity.as_str(),
|
||||
}),
|
||||
}),
|
||||
}
|
||||
}
|
||||
@@ -2040,6 +2112,8 @@ mod tests {
|
||||
.with_model("deepseek-v3-2-251201")
|
||||
.with_responses_api()
|
||||
.with_web_search(true)
|
||||
.with_response_reasoning_effort(LlmResponseReasoningEffort::Low)
|
||||
.with_response_text_verbosity(LlmResponseTextVerbosity::Low)
|
||||
.with_max_tokens(128),
|
||||
)
|
||||
.await
|
||||
@@ -2074,6 +2148,14 @@ mod tests {
|
||||
request_json["tools"],
|
||||
serde_json::json!([{ "type": "web_search", "max_keyword": 3 }])
|
||||
);
|
||||
assert_eq!(
|
||||
request_json["reasoning"],
|
||||
serde_json::json!({ "effort": "low" })
|
||||
);
|
||||
assert_eq!(
|
||||
request_json["text"],
|
||||
serde_json::json!({ "verbosity": "low" })
|
||||
);
|
||||
assert!(request_json.get("official_fallback").is_none());
|
||||
assert_eq!(
|
||||
request_json["input"][0]["content"][0],
|
||||
|
||||
@@ -92,6 +92,10 @@ pub struct EditorAgentAttachmentRef {
|
||||
pub struct EditorAgentGeneratedImage {
|
||||
#[serde(default)]
|
||||
pub resource_id: Option<String>,
|
||||
#[serde(default)]
|
||||
pub object_key: Option<String>,
|
||||
#[serde(default)]
|
||||
pub asset_object_id: Option<String>,
|
||||
pub image_src: String,
|
||||
#[serde(default)]
|
||||
pub thumbnail_src: Option<String>,
|
||||
@@ -338,6 +342,8 @@ mod tests {
|
||||
model: Some("z-image".to_string()),
|
||||
images: vec![EditorAgentGeneratedImage {
|
||||
resource_id: Some("resource-2".to_string()),
|
||||
object_key: Some("generated/result.png".to_string()),
|
||||
asset_object_id: Some("asset-object-2".to_string()),
|
||||
image_src: "https://example.test/generated.png".to_string(),
|
||||
thumbnail_src: None,
|
||||
width: Some(1024),
|
||||
@@ -354,6 +360,10 @@ mod tests {
|
||||
payload["data"]["images"][0]["resourceId"],
|
||||
json!("resource-2")
|
||||
);
|
||||
assert_eq!(
|
||||
payload["data"]["images"][0]["objectKey"],
|
||||
json!("generated/result.png")
|
||||
);
|
||||
|
||||
let error_envelope = EditorAgentSseEnvelope {
|
||||
event: EditorAgentSseEventType::Error,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
/* @vitest-environment jsdom */
|
||||
|
||||
import {
|
||||
act,
|
||||
fireEvent,
|
||||
render,
|
||||
screen,
|
||||
@@ -253,6 +254,135 @@ describe('EditorAgentConversationPanelView', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps streamed reply and generating state when the panel is collapsed and reopened', async () => {
|
||||
const client = createClient();
|
||||
let finishStream = () => {};
|
||||
vi.mocked(client.streamMessage).mockImplementation(
|
||||
async (conversationId, _payload, options) => {
|
||||
options.onEvent?.({
|
||||
event: 'message_delta',
|
||||
data: {
|
||||
conversationId,
|
||||
messageId: 'assistant-stream',
|
||||
role: 'assistant',
|
||||
kind: 'chat',
|
||||
textDelta: '我来生成图片。',
|
||||
},
|
||||
});
|
||||
options.onEvent?.({
|
||||
event: 'stage',
|
||||
data: { conversationId, stage: 'generating' },
|
||||
});
|
||||
options.onEvent?.({
|
||||
event: 'tool_started',
|
||||
data: {
|
||||
conversationId,
|
||||
messageId: 'assistant-stream',
|
||||
toolCallId: 'tool-call-generating',
|
||||
toolName: 'generate_image',
|
||||
taskId: 'task-generating',
|
||||
model: 'gpt-image-2',
|
||||
},
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
finishStream = resolve;
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
const { rerender } = render(
|
||||
<EditorAgentConversationPanelView
|
||||
projectId="project-1"
|
||||
open
|
||||
onToggleOpen={vi.fn()}
|
||||
client={client}
|
||||
/>,
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('已经看到画布内容')).toBeTruthy();
|
||||
});
|
||||
|
||||
fireEvent.change(screen.getByLabelText('发送给画布 Agent'), {
|
||||
target: { value: '生成一张图片' },
|
||||
});
|
||||
fireEvent.click(screen.getByRole('button', { name: '发送' }));
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('我来生成图片。')).toBeTruthy();
|
||||
});
|
||||
expect(screen.getByText('生成中')).toBeTruthy();
|
||||
expect(screen.getByRole('button', { name: '停止' })).toBeTruthy();
|
||||
|
||||
rerender(
|
||||
<EditorAgentConversationPanelView
|
||||
projectId="project-1"
|
||||
open={false}
|
||||
onToggleOpen={vi.fn()}
|
||||
client={client}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByRole('button', { name: '打开画布 Agent' })).toBeTruthy();
|
||||
|
||||
rerender(
|
||||
<EditorAgentConversationPanelView
|
||||
projectId="project-1"
|
||||
open
|
||||
onToggleOpen={vi.fn()}
|
||||
client={client}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(screen.getByText('我来生成图片。')).toBeTruthy();
|
||||
expect(screen.getByText('生成中')).toBeTruthy();
|
||||
expect(screen.getByRole('button', { name: '停止' })).toBeTruthy();
|
||||
|
||||
await act(async () => {
|
||||
finishStream();
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps wheel scrolling inside the message history and input', async () => {
|
||||
const client = createClient();
|
||||
const parentWheel = vi.fn();
|
||||
|
||||
render(
|
||||
<div onWheel={parentWheel}>
|
||||
<EditorAgentConversationPanelView
|
||||
projectId="project-1"
|
||||
open
|
||||
onToggleOpen={vi.fn()}
|
||||
client={client}
|
||||
/>
|
||||
</div>,
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('已经看到画布内容')).toBeTruthy();
|
||||
});
|
||||
|
||||
const messageWheel = new WheelEvent('wheel', {
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
deltaY: 120,
|
||||
});
|
||||
fireEvent(
|
||||
screen.getByRole('log', { name: '画布 Agent 消息流' }),
|
||||
messageWheel,
|
||||
);
|
||||
expect(parentWheel).not.toHaveBeenCalled();
|
||||
expect(messageWheel.defaultPrevented).toBe(false);
|
||||
|
||||
const inputWheel = new WheelEvent('wheel', {
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
deltaY: 120,
|
||||
});
|
||||
fireEvent(screen.getByLabelText('发送给画布 Agent'), inputWheel);
|
||||
expect(parentWheel).not.toHaveBeenCalled();
|
||||
expect(inputWheel.defaultPrevented).toBe(false);
|
||||
});
|
||||
|
||||
it('confirms conversation deletion from an independent dialog', async () => {
|
||||
const client = createClient();
|
||||
render(
|
||||
|
||||
@@ -10,11 +10,18 @@ import {
|
||||
Trash2,
|
||||
X,
|
||||
} from 'lucide-react';
|
||||
import { type FormEvent, useMemo, useState } from 'react';
|
||||
import {
|
||||
type FormEvent,
|
||||
type WheelEvent as ReactWheelEvent,
|
||||
useEffect,
|
||||
useMemo,
|
||||
useState,
|
||||
} from 'react';
|
||||
|
||||
import {
|
||||
EDITOR_AGENT_MAX_ATTACHMENTS,
|
||||
type EditorAgentAttachmentRef,
|
||||
type EditorAgentGenerationResultEvent,
|
||||
type EditorAgentGenerationRecord,
|
||||
type EditorAgentMessage,
|
||||
type EditorAgentStage,
|
||||
@@ -22,6 +29,7 @@ import {
|
||||
import { PlatformActionButton } from '../common/PlatformActionButton';
|
||||
import { PlatformDangerConfirmDialog } from '../common/PlatformDangerConfirmDialog';
|
||||
import { UnifiedModal } from '../common/UnifiedModal';
|
||||
import { ResolvedAssetImage } from '../ResolvedAssetImage';
|
||||
import type { CanvasLayer, EditorAsset } from './ImageCanvasEditorTypes';
|
||||
import {
|
||||
type EditorAgentConversationClient,
|
||||
@@ -42,10 +50,14 @@ type EditorAgentConversationPanelViewProps = {
|
||||
onToggleOpen: () => void;
|
||||
layers?: CanvasLayer[];
|
||||
assets?: EditorAsset[];
|
||||
onFocusResource?: (resourceId: string) => void;
|
||||
onGenerationResult?: (event: EditorAgentGenerationResultEvent) => void;
|
||||
client?: EditorAgentConversationClient;
|
||||
};
|
||||
|
||||
function stopAgentPanelWheel(event: ReactWheelEvent<HTMLElement>) {
|
||||
event.stopPropagation();
|
||||
}
|
||||
|
||||
function attachmentKey(attachment: EditorAgentAttachmentRef) {
|
||||
return `${attachment.source}:${attachment.referenceId}`;
|
||||
}
|
||||
@@ -169,8 +181,10 @@ function AttachmentChip({
|
||||
) : null}
|
||||
{attachment.thumbnailSrc || attachment.imageSrc ? (
|
||||
<span className="pointer-events-none absolute bottom-[calc(100%+0.4rem)] left-0 hidden rounded-2xl border border-white bg-white p-1 shadow-xl group-hover:block">
|
||||
<img
|
||||
<ResolvedAssetImage
|
||||
src={attachment.thumbnailSrc ?? attachment.imageSrc}
|
||||
objectKey={attachment.objectKey}
|
||||
refreshKey={attachment.referenceId}
|
||||
alt=""
|
||||
className="h-24 w-24 rounded-xl object-cover"
|
||||
/>
|
||||
@@ -182,10 +196,8 @@ function AttachmentChip({
|
||||
|
||||
function GenerationRecordsView({
|
||||
generations,
|
||||
onFocusResource,
|
||||
}: {
|
||||
generations: EditorAgentGenerationRecord[];
|
||||
onFocusResource?: (resourceId: string) => void;
|
||||
}) {
|
||||
if (!generations.length) {
|
||||
return null;
|
||||
@@ -197,42 +209,42 @@ function GenerationRecordsView({
|
||||
key={generation.toolCallId}
|
||||
className="rounded-2xl border border-slate-200 bg-white/80 p-2 text-xs text-slate-600"
|
||||
>
|
||||
<div className="flex items-center gap-2">
|
||||
{generation.status === 'generating' ? (
|
||||
<Loader2
|
||||
className="h-3.5 w-3.5 animate-spin"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
) : (
|
||||
<ImageIcon className="h-3.5 w-3.5" aria-hidden="true" />
|
||||
)}
|
||||
<span>{toolLabel(generation.toolName)}</span>
|
||||
{generation.model ? <span>{generation.model}</span> : null}
|
||||
</div>
|
||||
{generation.status === 'generating' || generation.error ? (
|
||||
<div className="flex items-center gap-2">
|
||||
{generation.status === 'generating' ? (
|
||||
<Loader2
|
||||
className="h-3.5 w-3.5 animate-spin"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
) : (
|
||||
<ImageIcon className="h-3.5 w-3.5" aria-hidden="true" />
|
||||
)}
|
||||
<span>{toolLabel(generation.toolName)}</span>
|
||||
{generation.model ? <span>{generation.model}</span> : null}
|
||||
</div>
|
||||
) : null}
|
||||
{generation.error ? (
|
||||
<div className="mt-1 text-red-600">{generation.error}</div>
|
||||
) : null}
|
||||
{generation.images.length ? (
|
||||
<div className="mt-2 grid grid-cols-3 gap-2">
|
||||
{generation.images.map((image, index) => (
|
||||
<button
|
||||
<div
|
||||
key={`${generation.toolCallId}-${image.resourceId ?? index}`}
|
||||
type="button"
|
||||
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
|
||||
title="生成结果"
|
||||
disabled={!image.resourceId || !onFocusResource}
|
||||
onClick={() => {
|
||||
if (image.resourceId) {
|
||||
onFocusResource?.(image.resourceId);
|
||||
}
|
||||
}}
|
||||
>
|
||||
<img
|
||||
<ResolvedAssetImage
|
||||
src={image.thumbnailSrc ?? image.imageSrc}
|
||||
objectKey={image.objectKey}
|
||||
refreshKey={
|
||||
image.resourceId ??
|
||||
generation.taskId ??
|
||||
generation.toolCallId
|
||||
}
|
||||
alt=""
|
||||
className="h-20 w-full object-cover"
|
||||
/>
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
) : null}
|
||||
@@ -244,10 +256,8 @@ function GenerationRecordsView({
|
||||
|
||||
function MessageBubble({
|
||||
message,
|
||||
onFocusResource,
|
||||
}: {
|
||||
message: EditorAgentMessage;
|
||||
onFocusResource?: (resourceId: string) => void;
|
||||
}) {
|
||||
const isUser = message.role === 'user';
|
||||
return (
|
||||
@@ -279,7 +289,6 @@ function MessageBubble({
|
||||
) : null}
|
||||
<GenerationRecordsView
|
||||
generations={message.generations}
|
||||
onFocusResource={onFocusResource}
|
||||
/>
|
||||
</div>
|
||||
</article>
|
||||
@@ -367,10 +376,12 @@ function AttachmentPickerModal({
|
||||
key={option.key}
|
||||
className="flex cursor-pointer flex-col gap-2 rounded-2xl border border-slate-200 bg-white p-2 text-sm text-slate-700 shadow-sm"
|
||||
>
|
||||
<img
|
||||
<ResolvedAssetImage
|
||||
src={
|
||||
option.attachment.thumbnailSrc ?? option.attachment.imageSrc
|
||||
}
|
||||
objectKey={option.attachment.objectKey}
|
||||
refreshKey={option.attachment.referenceId}
|
||||
alt=""
|
||||
className="aspect-square rounded-xl bg-slate-100 object-cover"
|
||||
/>
|
||||
@@ -398,10 +409,16 @@ export function EditorAgentConversationPanelView({
|
||||
onToggleOpen,
|
||||
layers = [],
|
||||
assets = [],
|
||||
onFocusResource,
|
||||
onGenerationResult,
|
||||
client,
|
||||
}: EditorAgentConversationPanelViewProps) {
|
||||
const effectiveProjectId = open ? projectId : null;
|
||||
const [hasConversationMounted, setHasConversationMounted] = useState(open);
|
||||
useEffect(() => {
|
||||
if (open) {
|
||||
setHasConversationMounted(true);
|
||||
}
|
||||
}, [open]);
|
||||
const effectiveProjectId = hasConversationMounted ? projectId : null;
|
||||
const {
|
||||
conversations,
|
||||
activeConversationId,
|
||||
@@ -418,7 +435,11 @@ export function EditorAgentConversationPanelView({
|
||||
sendMessage,
|
||||
stopCurrentTurn,
|
||||
deleteActiveConversation,
|
||||
} = useEditorAgentConversation({ projectId: effectiveProjectId, client });
|
||||
} = useEditorAgentConversation({
|
||||
projectId: effectiveProjectId,
|
||||
client,
|
||||
onGenerationResult,
|
||||
});
|
||||
const [draftText, setDraftText] = useState('');
|
||||
const [attachments, setAttachments] = useState<EditorAgentAttachmentRef[]>(
|
||||
[],
|
||||
@@ -517,7 +538,7 @@ export function EditorAgentConversationPanelView({
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
className="absolute right-3 top-20 z-40 inline-flex items-center gap-2 rounded-full border border-white/70 bg-white/95 px-3 py-2 text-sm font-semibold text-slate-700 shadow-lg backdrop-blur hover:bg-white"
|
||||
className="absolute bottom-24 right-4 z-40 inline-flex items-center gap-2 rounded-full border border-white/70 bg-white/95 px-3 py-2 text-sm font-semibold text-slate-700 shadow-lg backdrop-blur hover:bg-white"
|
||||
aria-label="打开画布 Agent"
|
||||
onClick={onToggleOpen}
|
||||
>
|
||||
@@ -533,6 +554,8 @@ export function EditorAgentConversationPanelView({
|
||||
className="absolute inset-y-0 right-0 z-50 flex w-full flex-col border-l border-slate-200 bg-slate-50/95 shadow-2xl backdrop-blur sm:inset-y-3 sm:right-3 sm:w-[390px] sm:overflow-hidden sm:rounded-3xl sm:border"
|
||||
aria-label="画布 Agent 对话"
|
||||
onPointerDown={(event) => event.stopPropagation()}
|
||||
onWheel={stopAgentPanelWheel}
|
||||
onWheelCapture={stopAgentPanelWheel}
|
||||
>
|
||||
<header className="flex items-center gap-2 border-b border-slate-200 bg-white/90 px-3 py-3">
|
||||
<Bot className="h-5 w-5 text-slate-700" aria-hidden="true" />
|
||||
@@ -604,7 +627,7 @@ export function EditorAgentConversationPanelView({
|
||||
</div>
|
||||
) : null}
|
||||
<div
|
||||
className="min-h-0 flex-1 space-y-3 overflow-y-auto px-3 py-4"
|
||||
className="min-h-0 flex-1 space-y-3 overflow-y-auto overscroll-contain px-3 py-4"
|
||||
role="log"
|
||||
aria-label="画布 Agent 消息流"
|
||||
>
|
||||
@@ -618,7 +641,6 @@ export function EditorAgentConversationPanelView({
|
||||
<MessageBubble
|
||||
key={message.id}
|
||||
message={message}
|
||||
onFocusResource={onFocusResource}
|
||||
/>
|
||||
))
|
||||
) : (
|
||||
@@ -657,7 +679,7 @@ export function EditorAgentConversationPanelView({
|
||||
<Paperclip className="h-4 w-4" aria-hidden="true" />
|
||||
</button>
|
||||
<textarea
|
||||
className="max-h-32 min-h-10 flex-1 resize-none rounded-3xl border border-slate-200 bg-slate-50 px-3 py-2 text-sm text-slate-800 outline-none focus:border-slate-400"
|
||||
className="max-h-32 min-h-10 flex-1 resize-none overflow-y-auto overscroll-contain rounded-3xl border border-slate-200 bg-slate-50 px-3 py-2 text-sm text-slate-800 outline-none focus:border-slate-400"
|
||||
aria-label="发送给画布 Agent"
|
||||
value={draftText}
|
||||
rows={1}
|
||||
|
||||
@@ -100,7 +100,11 @@ export function ImageCanvasContextMenusView({
|
||||
? [
|
||||
'context',
|
||||
contextMenu.kind,
|
||||
contextMenu.kind === 'layer' ? contextMenu.layerId : 'blank',
|
||||
contextMenu.kind === 'layer'
|
||||
? contextMenu.layerId
|
||||
: contextMenu.kind === 'generation-dialog'
|
||||
? contextMenu.dialogId
|
||||
: 'blank',
|
||||
contextMenu.x,
|
||||
contextMenu.y,
|
||||
isImageSequenceLayer ? 'sequence' : 'single',
|
||||
@@ -259,7 +263,11 @@ export function ImageCanvasContextMenusView({
|
||||
className={contextMenuClassName}
|
||||
role="menu"
|
||||
aria-label={
|
||||
contextMenu.kind === 'blank' ? '画布右键菜单' : '图片功能面板'
|
||||
contextMenu.kind === 'blank'
|
||||
? '画布右键菜单'
|
||||
: contextMenu.kind === 'generation-dialog'
|
||||
? '生成器右键菜单'
|
||||
: '图片功能面板'
|
||||
}
|
||||
style={{
|
||||
left: currentMeasuredMenuLayout?.x ?? contextMenu.x,
|
||||
@@ -308,6 +316,15 @@ export function ImageCanvasContextMenusView({
|
||||
显示画布所有元素
|
||||
</button>
|
||||
</>
|
||||
) : contextMenu.kind === 'generation-dialog' ? (
|
||||
<button
|
||||
type="button"
|
||||
role="menuitem"
|
||||
className="image-canvas-editor__context-menu-danger"
|
||||
onClick={onDeleteContextLayers}
|
||||
>
|
||||
删除
|
||||
</button>
|
||||
) : (
|
||||
<>
|
||||
<button
|
||||
|
||||
@@ -48,6 +48,7 @@ export const CONTEXT_MENU_VIEWPORT_MARGIN = 8;
|
||||
export const CONTEXT_MENU_SIZE = {
|
||||
blank: { width: 188, height: 176 },
|
||||
layer: { width: 188, height: 492 },
|
||||
'generation-dialog': { width: 188, height: 64 },
|
||||
} as const;
|
||||
export const CANVAS_BACKGROUND_OPTIONS = [
|
||||
{ label: '白色', value: '#ffffff' },
|
||||
|
||||
@@ -180,6 +180,7 @@ function createStageProps(): ImageCanvasStageViewProps {
|
||||
onOpenLayerMetadata: vi.fn(),
|
||||
onUpdateLayerAssetKind: vi.fn(),
|
||||
onGenerationFramePointerDown: vi.fn(),
|
||||
onGenerationFrameContextMenu: vi.fn(),
|
||||
onActivateGenerationDialog: vi.fn(),
|
||||
onFocusExternalTask: vi.fn(),
|
||||
onToggleTaskSidebar: vi.fn(),
|
||||
|
||||
@@ -285,6 +285,13 @@ export type CanvasContextMenuState =
|
||||
y: number;
|
||||
layerId: string;
|
||||
canvasPoint: { x: number; y: number };
|
||||
}
|
||||
| {
|
||||
kind: 'generation-dialog';
|
||||
x: number;
|
||||
y: number;
|
||||
dialogId: string;
|
||||
canvasPoint: { x: number; y: number };
|
||||
};
|
||||
|
||||
export type QuickEditPanelState = {
|
||||
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
ApiClientError,
|
||||
AuthUiContext,
|
||||
createAuthValue,
|
||||
defaultEditorAssetLibraryAssets,
|
||||
defaultEditorProjectLayers,
|
||||
defaultEditorProjectResources,
|
||||
dispatchPointerEvent,
|
||||
@@ -937,6 +938,33 @@ describe('ImageCanvasEditorView', () => {
|
||||
expect(screen.getByAltText('画布图片:大鱼素材')).toBeTruthy();
|
||||
});
|
||||
|
||||
it('moves focus from the Agent input back to the canvas layer before handling shortcuts', async () => {
|
||||
render(<ImageCanvasEditorView />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '画布 Agent' }));
|
||||
const agentInput = await screen.findByLabelText('发送给画布 Agent');
|
||||
agentInput.focus();
|
||||
expect(document.activeElement).toBe(agentInput);
|
||||
|
||||
const layerButton = screen
|
||||
.getByAltText('画布图片:拼图素材')
|
||||
.closest('button')!;
|
||||
fireEvent.pointerDown(layerButton, {
|
||||
button: 0,
|
||||
pointerId: 53,
|
||||
clientX: 120,
|
||||
clientY: 120,
|
||||
});
|
||||
|
||||
expect(document.activeElement).toBe(layerButton);
|
||||
await act(async () => {
|
||||
fireEvent.keyDown(window, { key: 'Backspace', code: 'Backspace' });
|
||||
});
|
||||
|
||||
expect(screen.queryByAltText('画布图片:拼图素材')).toBeNull();
|
||||
expect(screen.getByAltText('画布图片:大鱼素材')).toBeTruthy();
|
||||
});
|
||||
|
||||
it('opens the matching audio generation panel when remodeling an existing sound effect layer', async () => {
|
||||
loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({
|
||||
projectId: 'editor-project-audio-remodel',
|
||||
@@ -1123,6 +1151,23 @@ describe('ImageCanvasEditorView', () => {
|
||||
expect(screen.getByRole('menuitem', { name: '创建副本' })).toBeTruthy();
|
||||
});
|
||||
|
||||
it('shows delete when right-clicking a generation placeholder', async () => {
|
||||
render(<ImageCanvasEditorView />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '生成图片' }));
|
||||
|
||||
const frame = await screen.findByLabelText('图像生成占位图');
|
||||
fireEvent.contextMenu(frame, {
|
||||
clientX: 510,
|
||||
clientY: 330,
|
||||
});
|
||||
|
||||
const menu = screen.getByRole('menu', { name: '生成器右键菜单' });
|
||||
fireEvent.click(within(menu).getByRole('menuitem', { name: '删除' }));
|
||||
|
||||
expect(screen.queryByLabelText('图像生成占位图')).toBeNull();
|
||||
});
|
||||
|
||||
it('copies, cuts, and pastes layers from the context menus', () => {
|
||||
render(<ImageCanvasEditorView />);
|
||||
|
||||
@@ -1511,7 +1556,7 @@ describe('ImageCanvasEditorView', () => {
|
||||
expect(screen.queryByRole('button', { name: '画布小地图' })).toBeNull();
|
||||
});
|
||||
|
||||
it('focuses an existing canvas layer from an Agent generation thumbnail', async () => {
|
||||
it('renders Agent generation thumbnails as passive previews', async () => {
|
||||
const detail = createEditorAgentDetailWithGeneration('resource-puzzle');
|
||||
listEditorAgentConversationsMock.mockResolvedValueOnce([
|
||||
createEditorAgentConversationSummary(),
|
||||
@@ -1521,22 +1566,90 @@ describe('ImageCanvasEditorView', () => {
|
||||
render(<ImageCanvasEditorView />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '画布 Agent' }));
|
||||
fireEvent.click(await screen.findByRole('button', { name: '生成结果' }));
|
||||
|
||||
await waitFor(() => {
|
||||
expect(
|
||||
screen.getByAltText('画布图片:拼图素材').closest('button')?.className,
|
||||
).toContain('image-canvas-editor__layer--selected');
|
||||
const messageLog = await screen.findByRole('log', {
|
||||
name: '画布 Agent 消息流',
|
||||
});
|
||||
|
||||
expect(
|
||||
within(messageLog).queryByRole('button', { name: '生成结果' }),
|
||||
).toBeNull();
|
||||
expect(within(messageLog).queryByText('gpt-image-2')).toBeNull();
|
||||
expect(loadEditorProjectMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('refreshes the project and focuses a newly added Agent result layer', async () => {
|
||||
const detail = createEditorAgentDetailWithGeneration('resource-agent-new');
|
||||
it('refreshes canvas and asset library when Agent generation finishes', async () => {
|
||||
let showGeneratedAsset = false;
|
||||
loadEditorAssetLibraryMock.mockImplementation(async () => ({
|
||||
folders: [
|
||||
{
|
||||
folderId: 'project',
|
||||
label: '项目素材',
|
||||
sortOrder: 0,
|
||||
collapsed: false,
|
||||
systemDefault: true,
|
||||
},
|
||||
],
|
||||
assets: showGeneratedAsset
|
||||
? [
|
||||
...defaultEditorAssetLibraryAssets,
|
||||
{
|
||||
assetId: 'asset-agent-stream',
|
||||
folderId: 'project',
|
||||
label: 'Agent结果素材',
|
||||
imageSrc: '/agent-stream.png',
|
||||
width: 512,
|
||||
height: 512,
|
||||
sourceType: 'generated',
|
||||
assetKind: 'editor_agent_generated_image',
|
||||
},
|
||||
]
|
||||
: defaultEditorAssetLibraryAssets,
|
||||
}));
|
||||
listEditorAgentConversationsMock.mockResolvedValueOnce([
|
||||
createEditorAgentConversationSummary(),
|
||||
]);
|
||||
getEditorAgentConversationMock.mockResolvedValueOnce(detail);
|
||||
getEditorAgentConversationMock.mockResolvedValueOnce({
|
||||
...createEditorAgentConversationSummary(),
|
||||
messages: [],
|
||||
});
|
||||
streamEditorAgentMessageMock.mockImplementation(
|
||||
async (conversationId, _payload, options) => {
|
||||
options.onEvent?.({
|
||||
event: 'message_delta',
|
||||
data: {
|
||||
conversationId,
|
||||
messageId: 'assistant-agent-stream',
|
||||
role: 'assistant',
|
||||
kind: 'chat',
|
||||
textDelta: '我来生成图片。',
|
||||
},
|
||||
});
|
||||
showGeneratedAsset = true;
|
||||
options.onEvent?.({
|
||||
event: 'generation_result',
|
||||
data: {
|
||||
conversationId,
|
||||
messageId: 'assistant-agent-stream',
|
||||
toolCallId: 'tool-call-agent-stream',
|
||||
toolName: 'generate_image',
|
||||
model: 'gpt-image-2',
|
||||
images: [
|
||||
{
|
||||
resourceId: 'resource-agent-stream',
|
||||
imageSrc: '/agent-stream.png',
|
||||
thumbnailSrc: null,
|
||||
width: 512,
|
||||
height: 512,
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
options.onEvent?.({
|
||||
event: 'done',
|
||||
data: { conversationId, title: null },
|
||||
});
|
||||
},
|
||||
);
|
||||
loadEditorProjectMock.mockResolvedValueOnce({
|
||||
projectId: 'editor-project-default',
|
||||
title: '默认项目',
|
||||
@@ -1544,9 +1657,10 @@ describe('ImageCanvasEditorView', () => {
|
||||
layers: [
|
||||
...defaultEditorProjectLayers,
|
||||
{
|
||||
layerId: 'layer-agent-new',
|
||||
resourceId: 'resource-agent-new',
|
||||
layerId: 'layer-agent-stream',
|
||||
resourceId: 'resource-agent-stream',
|
||||
title: 'Agent结果图',
|
||||
src: '/agent-stream.png',
|
||||
x: 1500,
|
||||
y: 300,
|
||||
width: 512,
|
||||
@@ -1560,38 +1674,44 @@ describe('ImageCanvasEditorView', () => {
|
||||
resources: [
|
||||
...defaultEditorProjectResources,
|
||||
{
|
||||
resourceId: 'resource-agent-new',
|
||||
resourceId: 'resource-agent-stream',
|
||||
projectId: 'editor-project-default',
|
||||
imageSrc: '/agent-result.png',
|
||||
imageSrc: '/agent-stream.png',
|
||||
width: 512,
|
||||
height: 512,
|
||||
sourceType: 'generated',
|
||||
prompt: 'Agent生成结果',
|
||||
model: 'gpt-image-2',
|
||||
provider: 'VectorEngine',
|
||||
taskId: 'task-agent-1',
|
||||
taskId: 'task-agent-stream',
|
||||
assetKind: 'editor_agent_generated_image',
|
||||
},
|
||||
],
|
||||
updatedAt: '2026-07-03T00:00:00.000Z',
|
||||
updatedAt: '2026-07-03T00:02:00.000Z',
|
||||
});
|
||||
|
||||
render(<ImageCanvasEditorView />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '画布 Agent' }));
|
||||
fireEvent.click(await screen.findByRole('button', { name: '生成结果' }));
|
||||
await waitFor(() => {
|
||||
expect(getEditorAgentConversationMock).toHaveBeenCalledWith(
|
||||
'editor-agent-conv-test',
|
||||
);
|
||||
});
|
||||
fireEvent.change(await screen.findByLabelText('发送给画布 Agent'), {
|
||||
target: { value: '生成一张图' },
|
||||
});
|
||||
fireEvent.click(screen.getByRole('button', { name: '发送' }));
|
||||
|
||||
await waitFor(() => {
|
||||
expect(loadEditorProjectMock).toHaveBeenCalledWith(
|
||||
'editor-project-default',
|
||||
);
|
||||
});
|
||||
await waitFor(() => {
|
||||
expect(
|
||||
screen.getByAltText('画布图片:Agent结果图').closest('button')
|
||||
?.className,
|
||||
).toContain('image-canvas-editor__layer--selected');
|
||||
});
|
||||
expect(await screen.findByAltText('画布图片:Agent结果图')).toBeTruthy();
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '打开素材' }));
|
||||
expect(await screen.findByText('Agent结果素材')).toBeTruthy();
|
||||
});
|
||||
|
||||
it('resets the canvas view without forwarding the click event to fit layers', () => {
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
} from 'react';
|
||||
|
||||
import type { ExternalGenerationTaskRecord } from '../../../packages/shared/src/contracts/externalGeneration';
|
||||
import type { EditorAgentGenerationResultEvent } from '../../../packages/shared/src/contracts/editorAgent';
|
||||
import {
|
||||
createEditorAsset,
|
||||
createEditorProjectResource,
|
||||
@@ -1008,40 +1009,19 @@ export function ImageCanvasEditorView() {
|
||||
},
|
||||
[applyProjectSnapshot, refreshAssetLibrary],
|
||||
);
|
||||
const focusCanvasLayerByResourceId = useCallback(
|
||||
(resourceId: string) => {
|
||||
const targetLayer = layersRef.current.find(
|
||||
(layer) => layer.resourceId === resourceId || layer.id === resourceId,
|
||||
const handleEditorAgentGenerationResult = useCallback(
|
||||
(event: EditorAgentGenerationResultEvent) => {
|
||||
const hasGeneratedResource = event.images.some((image) =>
|
||||
image.resourceId?.trim(),
|
||||
);
|
||||
if (targetLayer) {
|
||||
focusCanvasLayerById(targetLayer.id);
|
||||
return;
|
||||
}
|
||||
if (!projectId) {
|
||||
if (!projectId || !hasGeneratedResource) {
|
||||
return;
|
||||
}
|
||||
void loadEditorProject(projectId)
|
||||
.then((project) => {
|
||||
applyGeneratedProjectSnapshot(project);
|
||||
const targetLayerItem = project.layers.find(
|
||||
(layer) =>
|
||||
layer.resourceId === resourceId || layer.layerId === resourceId,
|
||||
);
|
||||
const targetLayerId = targetLayerItem?.layerId;
|
||||
if (!targetLayerId) {
|
||||
return;
|
||||
}
|
||||
window.setTimeout(() => {
|
||||
focusCanvasLayerById(targetLayerId);
|
||||
}, 0);
|
||||
})
|
||||
.then(applyGeneratedProjectSnapshot)
|
||||
.catch(() => undefined);
|
||||
},
|
||||
[
|
||||
applyGeneratedProjectSnapshot,
|
||||
focusCanvasLayerById,
|
||||
projectId,
|
||||
],
|
||||
[applyGeneratedProjectSnapshot, projectId],
|
||||
);
|
||||
const persistUpdatedLayerResource = useCallback(
|
||||
(layer: CanvasLayer) => {
|
||||
@@ -1318,11 +1298,14 @@ export function ImageCanvasEditorView() {
|
||||
clearCanvasFocus,
|
||||
handleCanvasContextMenu,
|
||||
handleLayerContextMenu,
|
||||
handleGenerationFrameContextMenu,
|
||||
} = useImageCanvasStageController({
|
||||
layers,
|
||||
canvasGenerationDialogs,
|
||||
selectedLayerId,
|
||||
selectedLayerIds,
|
||||
setSelectedLayerId,
|
||||
setSelectedLayerIds,
|
||||
imageContextMenu,
|
||||
setImageContextMenu,
|
||||
contextMenu,
|
||||
@@ -1979,9 +1962,10 @@ export function ImageCanvasEditorView() {
|
||||
},
|
||||
onUpdateLayerAssetKind: updateLayerAssetKind,
|
||||
onGenerationFramePointerDown: handleGenerationFramePointerDown,
|
||||
onGenerationFrameContextMenu: handleGenerationFrameContextMenu,
|
||||
onActivateGenerationDialog: activateCanvasGenerationDialog,
|
||||
onFocusExternalTask: focusExternalGenerationTask,
|
||||
onFocusEditorAgentResource: focusCanvasLayerByResourceId,
|
||||
onEditorAgentGenerationResult: handleEditorAgentGenerationResult,
|
||||
onToggleTaskSidebar: toggleTaskSidebar,
|
||||
onToggleAgentConversation: toggleAgentConversation,
|
||||
onCropExpandHandlePointerDown: generationSurface.startCropExpandFrameResize,
|
||||
|
||||
@@ -3,6 +3,7 @@ import type {
|
||||
CanvasContextMenuState,
|
||||
CanvasLayer,
|
||||
} from './ImageCanvasEditorTypes';
|
||||
import { getCanvasGenerationSelectionId } from './ImageCanvasSelectionModel';
|
||||
|
||||
export type CanvasLayerMoveMode = 'up' | 'down' | 'top' | 'bottom';
|
||||
|
||||
@@ -31,12 +32,14 @@ export function resolveContextTargetLayerIds(
|
||||
menu: CanvasContextMenuState | null,
|
||||
selectedLayerIds: string[],
|
||||
) {
|
||||
if (menu?.kind !== 'layer') {
|
||||
if (!menu || menu.kind === 'blank') {
|
||||
return [];
|
||||
}
|
||||
return selectedLayerIds.includes(menu.layerId)
|
||||
? [...selectedLayerIds]
|
||||
: [menu.layerId];
|
||||
const targetId =
|
||||
menu.kind === 'generation-dialog'
|
||||
? getCanvasGenerationSelectionId(menu.dialogId)
|
||||
: menu.layerId;
|
||||
return selectedLayerIds.includes(targetId) ? [...selectedLayerIds] : [targetId];
|
||||
}
|
||||
|
||||
export function getCanvasLayersByIds(
|
||||
|
||||
@@ -57,6 +57,9 @@ describe('ImageCanvasPanelDockView', () => {
|
||||
|
||||
const toolbar = screen.getByRole('toolbar', { name: '画布面板入口' });
|
||||
|
||||
expect(toolbar.className).toContain(
|
||||
'image-canvas-editor__panel-dock--sidebar-open',
|
||||
);
|
||||
expect(
|
||||
within(toolbar)
|
||||
.getByRole('button', { name: '打开素材' })
|
||||
|
||||
@@ -90,7 +90,14 @@ export function ImageCanvasPanelDockView({
|
||||
/>
|
||||
|
||||
<div
|
||||
className="image-canvas-editor__panel-dock"
|
||||
className={[
|
||||
'image-canvas-editor__panel-dock',
|
||||
activeSidebarPanel
|
||||
? 'image-canvas-editor__panel-dock--sidebar-open'
|
||||
: '',
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join(' ')}
|
||||
role="toolbar"
|
||||
aria-label="画布面板入口"
|
||||
onPointerDown={(event) => event.stopPropagation()}
|
||||
|
||||
@@ -135,3 +135,22 @@ export function createLayerCanvasContextMenus({
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function createGenerationDialogCanvasContextMenu({
|
||||
clientX,
|
||||
clientY,
|
||||
dialogId,
|
||||
canvasPoint,
|
||||
}: {
|
||||
clientX: number;
|
||||
clientY: number;
|
||||
dialogId: string;
|
||||
canvasPoint: { x: number; y: number };
|
||||
}): CanvasContextMenuState {
|
||||
return {
|
||||
kind: 'generation-dialog',
|
||||
dialogId,
|
||||
...resolveContextMenuPosition(clientX, clientY, 'generation-dialog'),
|
||||
canvasPoint,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -8,6 +8,7 @@ import type {
|
||||
} from 'react';
|
||||
|
||||
import type { ExternalGenerationTaskRecord } from '../../../packages/shared/src/contracts/externalGeneration';
|
||||
import type { EditorAgentGenerationResultEvent } from '../../../packages/shared/src/contracts/editorAgent';
|
||||
import { EditorAgentConversationPanelView } from './EditorAgentConversationPanelView';
|
||||
import { ImageCanvasBottomToolbarView } from './ImageCanvasBottomToolbarView';
|
||||
import { ImageCanvasContextMenusView } from './ImageCanvasContextMenusView';
|
||||
@@ -118,9 +119,15 @@ export type ImageCanvasStageViewProps = {
|
||||
event: ReactPointerEvent<HTMLDivElement>,
|
||||
dialog: CanvasGenerationDialogState,
|
||||
) => void;
|
||||
onGenerationFrameContextMenu: (
|
||||
event: ReactMouseEvent<HTMLDivElement>,
|
||||
dialog: CanvasGenerationDialogState,
|
||||
) => void;
|
||||
onActivateGenerationDialog: (dialog: CanvasGenerationDialogState) => void;
|
||||
onFocusExternalTask: (task: ExternalGenerationTaskRecord) => void;
|
||||
onFocusEditorAgentResource?: (resourceId: string) => void;
|
||||
onEditorAgentGenerationResult?: (
|
||||
event: EditorAgentGenerationResultEvent,
|
||||
) => void;
|
||||
onToggleTaskSidebar: () => void;
|
||||
onToggleAgentConversation: () => void;
|
||||
onCropExpandHandlePointerDown: (
|
||||
@@ -247,9 +254,10 @@ export function ImageCanvasStageView({
|
||||
onOpenLayerMetadata,
|
||||
onUpdateLayerAssetKind,
|
||||
onGenerationFramePointerDown,
|
||||
onGenerationFrameContextMenu,
|
||||
onActivateGenerationDialog,
|
||||
onFocusExternalTask,
|
||||
onFocusEditorAgentResource,
|
||||
onEditorAgentGenerationResult,
|
||||
onToggleTaskSidebar,
|
||||
onToggleAgentConversation,
|
||||
onCropExpandHandlePointerDown,
|
||||
@@ -347,6 +355,7 @@ export function ImageCanvasStageView({
|
||||
onOpenLayerMetadata={onOpenLayerMetadata}
|
||||
onUpdateLayerAssetKind={onUpdateLayerAssetKind}
|
||||
onGenerationFramePointerDown={onGenerationFramePointerDown}
|
||||
onGenerationFrameContextMenu={onGenerationFrameContextMenu}
|
||||
onActivateGenerationDialog={onActivateGenerationDialog}
|
||||
onCropExpandHandlePointerDown={onCropExpandHandlePointerDown}
|
||||
/>
|
||||
@@ -464,7 +473,7 @@ export function ImageCanvasStageView({
|
||||
onToggleOpen={onToggleAgentConversation}
|
||||
layers={layers}
|
||||
assets={editorAgentAssets}
|
||||
onFocusResource={onFocusEditorAgentResource}
|
||||
onGenerationResult={onEditorAgentGenerationResult}
|
||||
/>
|
||||
|
||||
{isToolbarGuideVisible ? (
|
||||
|
||||
@@ -85,6 +85,7 @@ function renderWorldView(
|
||||
onOpenLayerMetadata: vi.fn(),
|
||||
onUpdateLayerAssetKind: vi.fn(),
|
||||
onGenerationFramePointerDown: vi.fn(),
|
||||
onGenerationFrameContextMenu: vi.fn(),
|
||||
onActivateGenerationDialog: vi.fn(),
|
||||
onCropExpandHandlePointerDown: vi.fn(),
|
||||
...overrides,
|
||||
|
||||
@@ -274,6 +274,10 @@ export type ImageCanvasWorldViewProps = {
|
||||
event: ReactPointerEvent<HTMLDivElement>,
|
||||
dialog: CanvasGenerationDialogState,
|
||||
) => void;
|
||||
onGenerationFrameContextMenu: (
|
||||
event: ReactMouseEvent<HTMLDivElement>,
|
||||
dialog: CanvasGenerationDialogState,
|
||||
) => void;
|
||||
onActivateGenerationDialog: (dialog: CanvasGenerationDialogState) => void;
|
||||
onCropExpandHandlePointerDown: (
|
||||
event: ReactPointerEvent<HTMLButtonElement>,
|
||||
@@ -921,6 +925,7 @@ export function ImageCanvasWorldView({
|
||||
onOpenLayerMetadata,
|
||||
onUpdateLayerAssetKind,
|
||||
onGenerationFramePointerDown,
|
||||
onGenerationFrameContextMenu,
|
||||
onActivateGenerationDialog,
|
||||
onCropExpandHandlePointerDown,
|
||||
}: ImageCanvasWorldViewProps) {
|
||||
@@ -1259,6 +1264,9 @@ export function ImageCanvasWorldView({
|
||||
onPointerDown={(event) =>
|
||||
onGenerationFramePointerDown(event, dialog)
|
||||
}
|
||||
onContextMenu={(event) =>
|
||||
onGenerationFrameContextMenu(event, dialog)
|
||||
}
|
||||
onDoubleClick={() => onActivateGenerationDialog(dialog)}
|
||||
>
|
||||
{showFocusedChrome ? (
|
||||
|
||||
@@ -111,8 +111,13 @@ describe('useEditorAgentConversation', () => {
|
||||
|
||||
it('loads conversations and applies message stream events', async () => {
|
||||
const client = createClient();
|
||||
const onGenerationResult = vi.fn();
|
||||
const { result } = renderHook(() =>
|
||||
useEditorAgentConversation({ projectId: 'project-1', client }),
|
||||
useEditorAgentConversation({
|
||||
projectId: 'project-1',
|
||||
client,
|
||||
onGenerationResult,
|
||||
}),
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
@@ -138,6 +143,16 @@ describe('useEditorAgentConversation', () => {
|
||||
);
|
||||
expect(result.current.stage).toBe('completed');
|
||||
expect(result.current.isStreaming).toBe(false);
|
||||
expect(onGenerationResult).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
toolCallId: 'tool-call-1',
|
||||
images: [
|
||||
expect.objectContaining({
|
||||
resourceId: 'resource-result-1',
|
||||
}),
|
||||
],
|
||||
}),
|
||||
);
|
||||
expect(result.current.messages.map((message) => message.text)).toEqual([
|
||||
'把这个角色改成像素风',
|
||||
'我来处理',
|
||||
@@ -221,6 +236,65 @@ describe('useEditorAgentConversation', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps LLM planning failures as one failed assistant message', async () => {
|
||||
const client = createClient();
|
||||
vi.mocked(client.streamMessage).mockImplementation(
|
||||
async (_conversationId, _payload, options) => {
|
||||
options.onEvent?.({
|
||||
event: 'message_delta',
|
||||
data: {
|
||||
conversationId: 'conversation-1',
|
||||
messageId: 'assistant-planning-error',
|
||||
role: 'assistant',
|
||||
kind: 'error',
|
||||
textDelta: '画布 Agent 的 LLM 未配置,无法处理这句话。',
|
||||
},
|
||||
});
|
||||
options.onEvent?.({
|
||||
event: 'stage',
|
||||
data: {
|
||||
conversationId: 'conversation-1',
|
||||
stage: 'failed',
|
||||
},
|
||||
});
|
||||
options.onEvent?.({
|
||||
event: 'done',
|
||||
data: {
|
||||
conversationId: 'conversation-1',
|
||||
title: null,
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
const { result } = renderHook(() =>
|
||||
useEditorAgentConversation({ projectId: 'project-1', client }),
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(result.current.activeConversation?.conversationId).toBe(
|
||||
'conversation-1',
|
||||
);
|
||||
});
|
||||
|
||||
await act(async () => {
|
||||
await result.current.sendMessage('这是美术素材');
|
||||
});
|
||||
|
||||
expect(result.current.stage).toBe('failed');
|
||||
expect(result.current.errorMessage).toBe(
|
||||
'画布 Agent 的 LLM 未配置,无法处理这句话。',
|
||||
);
|
||||
expect(
|
||||
result.current.messages.filter((message) => message.kind === 'error'),
|
||||
).toEqual([
|
||||
expect.objectContaining({
|
||||
id: 'assistant-planning-error',
|
||||
text: '画布 Agent 的 LLM 未配置,无法处理这句话。',
|
||||
status: 'failed',
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it('shows a failed tool completion as a generation record', async () => {
|
||||
const client = createClient();
|
||||
vi.mocked(client.streamMessage).mockImplementation(
|
||||
|
||||
@@ -5,6 +5,7 @@ import type {
|
||||
EditorAgentAttachmentRef,
|
||||
EditorAgentConversationDetail,
|
||||
EditorAgentConversationSummary,
|
||||
EditorAgentGenerationResultEvent,
|
||||
EditorAgentGenerationRecord,
|
||||
EditorAgentMessage,
|
||||
EditorAgentSseEvent,
|
||||
@@ -44,6 +45,7 @@ export type EditorAgentConversationClient = {
|
||||
type UseEditorAgentConversationOptions = {
|
||||
projectId?: string | null;
|
||||
client?: EditorAgentConversationClient;
|
||||
onGenerationResult?: (event: EditorAgentGenerationResultEvent) => void;
|
||||
};
|
||||
|
||||
const defaultEditorAgentConversationClient: EditorAgentConversationClient = {
|
||||
@@ -208,6 +210,7 @@ function upsertConversationSummary(
|
||||
export function useEditorAgentConversation({
|
||||
projectId,
|
||||
client = defaultEditorAgentConversationClient,
|
||||
onGenerationResult,
|
||||
}: UseEditorAgentConversationOptions) {
|
||||
const normalizedProjectId = projectId?.trim() ?? '';
|
||||
const [conversations, setConversations] = useState<
|
||||
@@ -377,6 +380,12 @@ export function useEditorAgentConversation({
|
||||
}
|
||||
|
||||
if (event.event === 'message_delta') {
|
||||
if (event.data.kind === 'error') {
|
||||
const nextErrorMessage = event.data.textDelta.trim();
|
||||
if (nextErrorMessage) {
|
||||
setErrorMessage(nextErrorMessage);
|
||||
}
|
||||
}
|
||||
setMessages((currentMessages) =>
|
||||
upsertMessage(
|
||||
currentMessages,
|
||||
@@ -437,6 +446,7 @@ export function useEditorAgentConversation({
|
||||
}
|
||||
|
||||
if (event.event === 'generation_result') {
|
||||
onGenerationResult?.(event.data);
|
||||
const nextRecord: EditorAgentGenerationRecord = {
|
||||
toolCallId: event.data.toolCallId,
|
||||
toolName: event.data.toolName,
|
||||
@@ -485,7 +495,9 @@ export function useEditorAgentConversation({
|
||||
}
|
||||
|
||||
if (event.event === 'done') {
|
||||
setStage('completed');
|
||||
setStage((currentStage) =>
|
||||
currentStage === 'failed' ? 'failed' : 'completed',
|
||||
);
|
||||
setMessages((currentMessages) =>
|
||||
markStreamingMessages(currentMessages, 'completed'),
|
||||
);
|
||||
@@ -500,7 +512,7 @@ export function useEditorAgentConversation({
|
||||
);
|
||||
}
|
||||
}
|
||||
}, []);
|
||||
}, [onGenerationResult]);
|
||||
|
||||
const ensureConversationForSend = useCallback(async () => {
|
||||
if (activeConversationId) {
|
||||
|
||||
@@ -496,21 +496,41 @@ export function useImageCanvasLayerCommands({
|
||||
|
||||
const deleteContextLayers = useCallback(() => {
|
||||
const targetIds = getContextTargetLayerIds();
|
||||
if (!targetIds.length) {
|
||||
const targetLayerIds = getSelectedLayerIds(targetIds);
|
||||
const targetDialogIds = targetIds
|
||||
.map(getCanvasGenerationDialogIdFromSelectionId)
|
||||
.filter((dialogId): dialogId is string => Boolean(dialogId))
|
||||
.filter((dialogId) =>
|
||||
canvasGenerationDialogs.some((dialog) => dialog.id === dialogId),
|
||||
);
|
||||
if (!targetLayerIds.length && !targetDialogIds.length) {
|
||||
return;
|
||||
}
|
||||
captureCanvasHistory();
|
||||
setLayers((currentLayers) => removeCanvasLayers(currentLayers, targetIds));
|
||||
if (targetLayerIds.length) {
|
||||
setLayers((currentLayers) =>
|
||||
removeCanvasLayers(currentLayers, targetLayerIds),
|
||||
);
|
||||
}
|
||||
selectSingleLayer(null);
|
||||
setHoveredLayerId(null);
|
||||
setMetadataLayer((currentLayer) =>
|
||||
currentLayer && targetIds.includes(currentLayer.id) ? null : currentLayer,
|
||||
currentLayer && targetLayerIds.includes(currentLayer.id)
|
||||
? null
|
||||
: currentLayer,
|
||||
);
|
||||
targetLayerIds.forEach((targetId) => onDeleteLayerSideEffects(targetId));
|
||||
targetDialogIds.forEach((targetId) =>
|
||||
onDeleteGenerationDialogSideEffects?.(targetId),
|
||||
);
|
||||
closeContextMenus();
|
||||
}, [
|
||||
captureCanvasHistory,
|
||||
canvasGenerationDialogs,
|
||||
closeContextMenus,
|
||||
getContextTargetLayerIds,
|
||||
onDeleteGenerationDialogSideEffects,
|
||||
onDeleteLayerSideEffects,
|
||||
selectSingleLayer,
|
||||
setHoveredLayerId,
|
||||
setLayers,
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user