修正图标规范主参考权威校验
Project CI / Repository checks (pull_request) Has been cancelled
Project CI / Frontend tests (pull_request) Has been cancelled
Project CI / Backend tests (pull_request) Has been cancelled
Project CI / Native shell tests (pull_request) Has been cancelled

规划与确认时从 SpacetimeDB 重建工具上下文
仅允许 icon-spec 作为精灵图主规范并完善提示
补充普通图片拒绝与图标规范通过测试
同步后端数据契约文档
This commit is contained in:
2026-08-08 16:02:47 +08:00
parent d1562ab65f
commit 3451c17142
7 changed files with 236 additions and 30 deletions
@@ -83,6 +83,7 @@ npm run check:server-rs-ddd
- 完整消息文档存 OSS `editor-agent/{conversationId}.json`,由 `api-server` 负责 2 MiB 上限、会话内串行锁、读改写、消息与工具结果持久化和 `touch` 元数据更新时间;该 JSON 不进入 `editor_canvas.layers_json`,也不作为画布布局真相。LLM 未配置、连接已经断开、请求明确失败、达到最终安全上限或规划不可解析时,必须写入 `role=system`、正文以 `ERROR ` 开头的消息,并通过 `deltaMessages` 返回,`errorMessage` 保持为空;前端隐藏前缀并显示红色错误气泡,面向用户的错误正文使用中文语义,不暴露 `completion error` 等 framework 内部前缀或原始配置/定价错误;原始诊断只写后端结构化日志。后端仍把该 system 消息注入后续 LLM memory,使 Agent 能读取失败上下文。普通 JSON POST 尚未结束不形成持久化消息;工具失败同样必须形成可回读记录,不能只返回瞬时错误。
- 画布 Agent 的 `gpt-5.4-mini` Chat Completions 规划使用 1024 生成 token 预算;VectorEngine 专用 client 显式发送当前字段 `max_completion_tokens`,其预算包含可见输出和隐藏 reasoning token。通用 OpenAI-compatible client 默认保留旧 `max_tokens`,只有确认 endpoint 能力后才 opt-in,禁止按模型名猜测或在 `400` 后自动重放。前端在 POST pending 120 秒后显示不入库的耐心等待提示;provider request future 明确返回 connect/timeout/HTTP/transport 错误时立即进入正式失败,尚未返回则继续等待。专用 provider 单 attempt hard timeout 为 8 分钟;请求发起阶段的 timeout、连接失败、`408``429``5xx` 读取 `GENARRATIVE_LLM_MAX_RETRIES`,但画布 Agent 最多重试 1 次,显式配置 0 仍可关闭,专用重试退避最多 60 秒。消息规划生命周期从 handler 入口开始计入 18 分钟总 deadline,进入 `agent.prompt(...)` 时只使用剩余预算;该 deadline 覆盖会话锁/上下文准备与最多 3 轮规划,并为错误持久化/HTTP 返回预留约 2 分钟,不允许多轮规划绕过前端 20 分钟 timeout。已收到成功响应头后的响应体读取或解析失败直接按明确失败收口,并使用该成功响应所属的真实 attempt 记录错误。重试只包围 LLM 规划请求并发生在任何待确认工具执行之前,因此不会重复提交生成任务或扣费。
- 对话附件只允许引用当前工程 `editor_project_resource` 或当前账号 `editor_asset` 的图片;前端可提交展示用 `imageSrc` / `thumbnailSrc`,后端必须按 `resourceId` / `assetId` 重新归一、校验 owner / project 和 `objectKey`,再给 LLM 或生成工具使用。
- 工具上下文不得把 OSS 消息附件当作 `assetKind` 真相;每次规划和每次确认都按附件 `source + referenceId``spacetime-client` 重新读取当前工程资源或账号素材库,只把权威 `asset_kind` 放入服务端内存 `ImageMetadata`。图标 spritesheet 的主参考必须精确为 `icon-spec`,普通图片或风格参考图只能作为额外参考;主参考类型缺失、已删除或不是 `icon-spec` 时必须在生成任务入队和用户确认生效前失败关闭,并提示重新选择图标规范。
- `edit-image` 只接受当前图片上下文中的 `object_image_id``source_image_id` 不是现役 schema 字段,prompt、tool args、确认执行和测试中都不得生成或兼容该字段。
- 画布 Agent 工具复用既有编辑器图片生成 / 修改 / 图标 spritesheet BFF,并继续使用后端模型定价和 `execute_billable_asset_operation_with_cost`;前端不提交 `priceMudPoints`
- api-server 对 `PromptRunError` 的持久化顺序固定为:先按 `partial_outputs` 原顺序映射已成功工具,将其保存为 `status=not_completed` 且无 `externalJobId` 的待确认消息;再在同一会话增量末尾追加 `ERROR ` terminal system 消息并整体写入 OSS。后续规划失败不得吞掉失败前已执行的成功工具结果;结构化 `ToolFailed` 可用于调用方诊断与流程决策,但画布确认面不得把它伪装成成功待确认卡。
@@ -197,8 +197,9 @@ pub async fn editor_agent_message(
// Tool calls and attachment bookkeeping are separate system messages.
let previous_messages = build_prompt_memory(&document, history_end);
// Build tool context from document
let tool_context = context::build_tool_context(&document);
// Re-read authoritative resource and asset metadata for every planning turn. The message
// document only identifies attachments; it is not the source of truth for asset kind.
let tool_context = context::build_tool_context(&state, &conversation, &document).await?;
// Build and run agent
let Some(llm_client) = state.vector_engine_llm_client() else {
@@ -1046,7 +1047,7 @@ pub async fn confirm_editor_agent_tool_call(
}))
})?;
let project = load_editor_agent_project(&state, &conversation).await?;
let context = context::build_tool_context(&document);
let context = context::build_tool_context(&state, &conversation, &document).await?;
let tool = editor_agent_tool(tool_name.as_str(), &context)
.ok_or_else(|| editor_agent_bad_request(format!("unsupported tool: {tool_name}")))?;
let normalized_args = tool
@@ -1,10 +1,82 @@
use crate::editor_agent::utils::IntoDataKey;
use crate::editor_project::{current_utc_micros, map_editor_project_error};
use crate::http_error::AppError;
use crate::state::AppState;
use platform_editor_agent::agent::asset::{ImageId, ImageMetadata};
use platform_editor_agent::agent::tools::context::EditorToolContext;
use shared_contracts::editor_agent::EditorAgentConversationMessagesDocument;
use shared_contracts::editor_agent::{
EditorAgentAttachmentSource, EditorAgentConversationMessagesDocument,
};
use spacetime_client::{
EditorAgentConversationRecord, EditorAssetLibraryRecord, EditorProjectGetRecordInput,
EditorProjectRecord,
};
use std::collections::HashMap;
pub fn build_tool_context(document: &EditorAgentConversationMessagesDocument) -> EditorToolContext {
pub async fn build_tool_context(
state: &AppState,
conversation: &EditorAgentConversationRecord,
document: &EditorAgentConversationMessagesDocument,
) -> Result<EditorToolContext, AppError> {
let needs_project = document.messages.iter().any(|message| {
message
.attachments
.iter()
.any(|attachment| attachment.source == EditorAgentAttachmentSource::CanvasResource)
|| message.tool_call.as_ref().is_some_and(|tool_call| {
tool_call.images.iter().any(|image| {
image
.resource_id
.as_deref()
.is_some_and(|id| !id.trim().is_empty())
})
})
});
let needs_library = document.messages.iter().any(|message| {
message
.attachments
.iter()
.any(|attachment| attachment.source == EditorAgentAttachmentSource::LibraryAsset)
});
let project = if needs_project {
Some(
state
.spacetime_client()
.get_editor_project(EditorProjectGetRecordInput {
project_id: conversation.project_id.clone(),
owner_user_id: conversation.owner_user_id.clone(),
})
.await
.map_err(map_editor_project_error)?,
)
} else {
None
};
let library = if needs_library {
Some(
state
.spacetime_client()
.get_editor_asset_library(conversation.owner_user_id.clone(), current_utc_micros())
.await
.map_err(map_editor_project_error)?,
)
} else {
None
};
Ok(build_tool_context_from_authoritative_records(
document,
project.as_ref(),
library.as_ref(),
))
}
fn build_tool_context_from_authoritative_records(
document: &EditorAgentConversationMessagesDocument,
project: Option<&EditorProjectRecord>,
library: Option<&EditorAssetLibraryRecord>,
) -> EditorToolContext {
let mut images: HashMap<ImageId, ImageMetadata> = HashMap::new();
for msg in document.messages.iter().rev() {
@@ -14,6 +86,24 @@ pub fn build_tool_context(document: &EditorAgentConversationMessagesDocument) ->
let metadata = ImageMetadata {
data_key,
reference_id: Some(a.reference_id.clone()),
asset_kind: match a.source {
EditorAgentAttachmentSource::CanvasResource => project
.and_then(|project| {
project
.resources
.iter()
.find(|resource| resource.resource_id == a.reference_id)
})
.and_then(|resource| resource.asset_kind.clone()),
EditorAgentAttachmentSource::LibraryAsset => library
.and_then(|library| {
library
.assets
.iter()
.find(|asset| asset.asset_id == a.reference_id)
})
.and_then(|asset| asset.asset_kind.clone()),
},
image_src: a.image_src.clone(),
object_key: a.object_key.clone(),
thumbnail_src: a.thumbnail_src.clone(),
@@ -31,6 +121,16 @@ pub fn build_tool_context(document: &EditorAgentConversationMessagesDocument) ->
let metadata = ImageMetadata {
data_key,
reference_id: img.resource_id.clone(),
asset_kind: img.resource_id.as_deref().and_then(|resource_id| {
project
.and_then(|project| {
project
.resources
.iter()
.find(|resource| resource.resource_id == resource_id)
})
.and_then(|resource| resource.asset_kind.clone())
}),
image_src: img.image_src.clone(),
object_key: img.object_key.clone(),
thumbnail_src: img.thumbnail_src.clone(),
@@ -49,10 +149,17 @@ pub fn build_tool_context(document: &EditorAgentConversationMessagesDocument) ->
#[cfg(test)]
mod tests {
use super::*;
use crate::editor_agent::tool::editor_agent_tool;
use platform_editor_agent::agent::tools::generate_icon_spritesheet::GenerateIconSpritesheetTool;
use platform_editor_agent::framework::tool::Tool;
use serde_json::json;
use shared_contracts::editor_agent::{
EditorAgentAttachmentRef, EditorAgentAttachmentSource, EditorAgentMessage,
EditorAgentMessageRole,
};
use spacetime_client::{
EditorCanvasRecord, EditorCanvasViewportRecord, EditorProjectResourceRecord,
};
fn message(id: usize, label: &str, image_src: &str) -> EditorAgentMessage {
EditorAgentMessage {
@@ -75,6 +182,59 @@ mod tests {
}
}
fn project_with_resource_asset_kind(asset_kind: &str) -> EditorProjectRecord {
let viewport = EditorCanvasViewportRecord {
x: 0.0,
y: 0.0,
scale: 1.0,
};
EditorProjectRecord {
project_id: "project-1".to_string(),
owner_user_id: "user-1".to_string(),
title: "测试工程".to_string(),
canvas: EditorCanvasRecord {
canvas_id: "canvas-1".to_string(),
project_id: "project-1".to_string(),
title: "测试画布".to_string(),
viewport: viewport.clone(),
layers: json!([]),
revision: 0,
layout_storage_version: 2,
background_color: None,
created_at: "2026-08-08T00:00:00Z".to_string(),
updated_at: "2026-08-08T00:00:00Z".to_string(),
},
viewport,
layers: json!([]),
resources: vec![EditorProjectResourceRecord {
resource_id: "resource-1".to_string(),
project_id: "project-1".to_string(),
owner_user_id: "user-1".to_string(),
asset_object_id: Some("object-1".to_string()),
image_src: "/api/assets/read/current.png".to_string(),
object_key: Some("generated/reference.png".to_string()),
width: 640,
height: 480,
source_type: "generated".to_string(),
prompt: None,
actual_prompt: None,
model: None,
provider: None,
task_id: None,
source_resource_id: None,
asset_kind: Some(asset_kind.to_string()),
generation_inputs: None,
public_showcase_enabled: false,
created_at: "2026-08-08T00:00:00Z".to_string(),
updated_at: "2026-08-08T00:00:00Z".to_string(),
image_sequence_frames: None,
image_sequence_duration_ms: None,
}],
created_at: "2026-08-08T00:00:00Z".to_string(),
updated_at: "2026-08-08T00:00:00Z".to_string(),
}
}
#[test]
fn tool_context_keeps_complete_metadata_from_the_latest_image_reference() {
let document = EditorAgentConversationMessagesDocument {
@@ -86,7 +246,7 @@ mod tests {
],
};
let context = build_tool_context(&document);
let context = build_tool_context_from_authoritative_records(&document, None, None);
let image_id = ImageId::from_data_key("generated/reference.png");
let metadata = context
.image_metadata(&image_id)
@@ -98,4 +258,38 @@ mod tests {
assert_eq!(metadata.width, Some(640));
assert_eq!(metadata.height, Some(480));
}
#[test]
fn icon_spritesheet_validation_uses_authoritative_project_asset_kind() {
let document = EditorAgentConversationMessagesDocument {
version: 2,
conversation_id: "conversation-1".to_string(),
messages: vec![message(0, "主规范", "/api/assets/read/current.png")],
};
let args = json!({
"reference_image_id": ImageId::from_data_key("generated/reference.png").id,
"icon_descriptions": ["背包"]
});
let ordinary_project = project_with_resource_asset_kind("image");
let ordinary_context =
build_tool_context_from_authoritative_records(&document, Some(&ordinary_project), None);
let ordinary_tool = editor_agent_tool(GenerateIconSpritesheetTool::NAME, &ordinary_context)
.expect("icon spritesheet tool should resolve");
let error = ordinary_tool
.validate_args(&args)
.expect_err("ordinary registered image must be rejected before confirmation");
assert!(error.to_string().contains("icon-spec"));
let icon_spec_project = project_with_resource_asset_kind("icon-spec");
let icon_spec_context = build_tool_context_from_authoritative_records(
&document,
Some(&icon_spec_project),
None,
);
let icon_spec_tool =
editor_agent_tool(GenerateIconSpritesheetTool::NAME, &icon_spec_context)
.expect("icon spritesheet tool should resolve");
assert!(icon_spec_tool.validate_args(&args).is_ok());
}
}
@@ -719,21 +719,37 @@ impl EditorAgentTool for EditImageTool {
}
}
// TODO(editor-agent, 延后开发): 将主规范图的权威 assetKind 带入工具上下文,并在规划与确认阶段仅接受 icon-spec。
fn resolve_icon_spec_reference_id<'a>(
context: &'a EditorToolContext,
image_id: &ImageId,
) -> Result<&'a str, EditorAgentToolError> {
let metadata = context.image_metadata(image_id).ok_or_else(|| {
EditorAgentToolError::invalid_args("reference image is not available in tool context")
})?;
let reference_id = metadata
.reference_id
.as_deref()
.map(str::trim)
.filter(|reference_id| !reference_id.is_empty())
.ok_or_else(|| {
EditorAgentToolError::invalid_args(
"reference image is not a registered resource or asset",
)
})?;
if metadata.asset_kind.as_deref().map(str::trim) != Some("icon-spec") {
return Err(EditorAgentToolError::invalid_args(
"reference image must be a registered icon-spec; select an icon specification image and try again",
));
}
Ok(reference_id)
}
impl EditorAgentTool for GenerateIconSpritesheetTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: GenerateIconSpritesheetToolArgs = parse_invalid_args(Self::NAME, args)?;
let args = GenerateIconSpritesheetTool::normalize_args(self, args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
self.context
.image_metadata(&args.reference_image_id)
.and_then(|metadata| metadata.reference_id.as_deref())
.filter(|reference_id| !reference_id.trim().is_empty())
.ok_or_else(|| {
EditorAgentToolError::invalid_args(
"reference image is not a registered resource or asset",
)
})?;
resolve_icon_spec_reference_id(&self.context, &args.reference_image_id)?;
serialize_normalized_args(Self::NAME, &args)
}
@@ -801,18 +817,8 @@ impl EditorAgentTool for GenerateIconSpritesheetTool {
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: GenerateIconSpritesheetToolArgs = parse_invalid_args(Self::NAME, args)?;
let reference_id = self
.context
.image_metadata(&args.reference_image_id)
.and_then(|metadata| metadata.reference_id.as_deref())
.filter(|reference_id| !reference_id.trim().is_empty())
.ok_or_else(|| {
EditorAgentToolError::invalid_args(
"reference image is not a registered resource or asset",
)
})?
.trim()
.to_string();
let reference_id =
resolve_icon_spec_reference_id(&self.context, &args.reference_image_id)?.to_string();
let reference_image_srcs = resolve_image_ids(&args.reference_image_ids, &self.context)?;
let payload = EditorIconSpritesheetGenerationRequest {
reference_id,
@@ -1201,6 +1207,7 @@ mod tests {
ImageMetadata {
data_key: format!("generated/{image_id}.png"),
reference_id: Some(format!("resource-{image_id}")),
asset_kind: Some("icon-spec".to_string()),
image_src: format!("/api/assets/read/generated/{image_id}.png"),
object_key: Some(format!("generated/{image_id}.png")),
thumbnail_src: Some(format!("/api/assets/read/generated/{image_id}-thumb.png")),
@@ -40,6 +40,8 @@ impl Display for ImageId {
pub struct ImageMetadata {
pub data_key: String,
pub reference_id: Option<String>,
#[serde(default)]
pub asset_kind: Option<String>,
pub image_src: String,
pub object_key: Option<String>,
pub thumbnail_src: Option<String>,
@@ -72,7 +72,7 @@ pub fn generate_character_tool_description() -> String {
pub fn generate_icon_spritesheet_tool_description() -> String {
format!(
"仅用于生成多个图标成品、图标素材图集或 spritesheet。必须提供图标规范或风格参考图,并填写多个 icon_descriptions;没有参考图时不要调用,应提示用户先选择参考图。图标规范图或图标视觉规范展板属于规范展板,{SPEC_BOARD_ROUTE_POLICY}。可以一次调用生成多张图"
"仅用于生成多个图标成品、图标素材图集或 spritesheet。主参考图必须是系统中已登记为 icon-spec 的图标规范图,并填写多个 icon_descriptions;普通风格参考图只能放入额外参考图,不能充当主规范。没有图标规范图时不要调用,应提示用户先选择或创建图标规范。图标规范图或图标视觉规范展板属于规范展板,{SPEC_BOARD_ROUTE_POLICY}。可以一次调用生成多张图"
)
}
@@ -149,7 +149,7 @@ impl Tool for GenerateIconSpritesheetTool {
json!({
"type": "object",
"properties": {
"reference_image_id": { "type": "string", "description": "必填的图标规范或风格参考图 ID" },
"reference_image_id": { "type": "string", "description": "必填、且系统已登记为 icon-spec 的图标规范图 ID。普通风格参考图不能作为主规范" },
"model": image_model_parameter_schema(),
"reference_image_ids": { "type": "array", "items": { "type": "string" }, "description": "image_id(s) for desc" },
"icon_descriptions": {
@@ -344,6 +344,7 @@ mod tests {
ImageMetadata {
data_key: "asset://reference-image".to_string(),
reference_id: Some("resource-reference-image".to_string()),
asset_kind: Some("icon-spec".to_string()),
image_src: "asset://reference-image".to_string(),
object_key: None,
thumbnail_src: None,