Files
Genarrative/server-rs/crates/api-server/src/editor_agent/tool.rs
T
kdletters fc14190f58
Project CI / AI game creator shell Rust shard 1/4 (push) Has been cancelled
Project CI / AI game creator shell Rust shard 2/4 (push) Has been cancelled
Project CI / AI game creator shell Rust shard 3/4 (push) Has been cancelled
Project CI / AI game creator shell Rust shard 4/4 (push) Has been cancelled
Project CI / AI game creator shell Rust smoke (push) Has been cancelled
Project CI / AI game creator shell Rust crates (push) Has been cancelled
Project CI / Backend tests (push) Has been cancelled
Project CI / Native shell tests (push) Has been cancelled
Project CI / Frontend tests (push) Has been cancelled
Project CI / Repository checks (push) Has been cancelled
Project CI / AI game creator shell web tests (push) Has been cancelled
图集切片模式改为必须显式声明并补齐决策要求 (#408)
## 背景

切图新增基于连通域的切分后,LLM 仍倾向显式传 `sliceMode=grid`:参数只存在于部分 LLM 可见面、带默认值、没有任何决策规则,生成结果也不回显生效模式。

## 变更

- 平台:`/api/editor/icon-spritesheets/generations` 与 `/api/external/v1/editor/icon-spritesheets/generations` 把 `sliceMode` 改为必填并移除默认值;缺失、空白或未知取值在引用解析、定价与任何 provider / OSS 副作用之前返回 `400`,错误统一带 `field` 与决策要求。
- 契约:`grid` 必须同时提供 `gridX`/`gridY`,`connected-components` 不接受网格尺寸;`sliceCount` 只约束连通域切分,请求与响应的公开上限统一为 `256`;OpenAPI 去掉默认值并补必填与失败语义。
- AGC:MCP 工具说明去掉默认值并补决策要求,桥接层新增可测试的切分声明校验;原生工具 `canvas.asset_generate` 暴露 `sliceMode/gridX/gridY/sliceCount` 并要求图集显式声明;生成结果回显 `sliceMode/gridX/gridY` 与 `slicePaths`;严格图集在本地提交前校验平台回显与请求声明一致。
- 标准美术包:显式声明 `connected-components` 加 `sliceCount=4`,并在四张 canonical 切片用途映射前校验数量,禁止截断或错位。
- 前端与画板:画板 Agent 工具装配与画板提交计划显式声明连通域切分;前端类型要求显式 `sliceMode` 并在本地校验声明自洽。
- 文档与 Skill:主规范、OpenAPI、AGC Skill、外部编辑器 Skill、里程碑与实施计划、共享决策记录同步更新。
- 测试环境:测试构建对提权 Windows 主机上系统临时目录的所有者偏差做一次性所有者初始化重试,临时目录之外的越权所有者继续失败关闭。

## 兼容性影响

省略 `sliceMode` 的旧调用方(含已发布但未更新的 AGC 客户端与第三方外部 API 调用方)会在图集生成上收到 `400`;这是本次"不允许默认值"的预期结果,仓库内自有调用方已全部改为显式声明。

## 验证

- 平台:`slice_mode_must_be_declared_*` 与 OpenAPI 契约测试通过;全量 `cargo test -p api-server` 1043 通过 / 11 失败(`wallet_refund_outbox` 临时文件 `拒绝访问`,已在改动前基线复现,属本机环境)。
- AGC:`slice` 30、`spritesheet` 21、`direct_tools_mcp` 23、`agent_native_tools` 16、`canvas_generation_tests` 83、提示词上限与桥接门禁各 1 条、`cargo check --tests` 全部通过。
- 前端:182 条定向测试与 `typecheck` 通过。
- 门禁:`cargo fmt --check`(两个 workspace)、`check:encoding`、`check:doc-index`、`git diff --check` 通过。
- 未验证:真实 Provider 与浏览器试玩、确定性 e2e 车道;整机全量 AGC 单进程运行在本机受提权 shell 的所有者与时序问题影响,不作为门禁信号。

---------

Co-authored-by: kdletters <61648117+kdletters@users.noreply.github.com>
Reviewed-on: http://192.168.35.82/git/GenarrativeAI/Genarrative/pulls/408
2026-09-17 18:10:34 +08:00

1774 lines
67 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
use std::fmt::{Display, Formatter};
use platform_editor_agent::agent::asset::ImageId;
use platform_editor_agent::agent::tools::context::EditorToolContext;
use platform_editor_agent::agent::tools::edit_image::{
EditImageTool, EditImageToolArgs, EditorImageEditResult,
};
use platform_editor_agent::agent::tools::generate_background_music::{
GenerateBackgroundMusicTool, GenerateBackgroundMusicToolArgs,
};
use platform_editor_agent::agent::tools::generate_character::{
GenerateCharacterTool, GenerateCharacterToolArgs,
};
use platform_editor_agent::agent::tools::generate_icon_spritesheet::{
EditorIconSpritesheetResult, GenerateIconSpritesheetTool, GenerateIconSpritesheetToolArgs,
};
use platform_editor_agent::agent::tools::generate_image::{
EditorImageGenerationResult, GenerateImageTool, GenerateImageToolArgs,
};
use platform_editor_agent::agent::tools::generate_sound_effect::{
GenerateSoundEffectTool, GenerateSoundEffectToolArgs,
};
use platform_editor_agent::agent::tools::generate_ui_design::{
GenerateUiDesignTool, GenerateUiDesignToolArgs,
};
use platform_editor_agent::agent::tools::generate_video::{
GenerateVideoTool, GenerateVideoToolArgs,
};
use platform_editor_agent::framework::error::PromptError;
use platform_editor_agent::framework::tool::{Tool, ToolDyn, null_tool_args_as_missing};
use platform_image::GPT_IMAGE_2_MODEL;
use serde::Serialize;
use serde::de::DeserializeOwned;
use serde_json::{Value, json};
use shared_contracts::assets::{
EditorAudioGenerateResponse, EditorBackgroundMusicGenerateRequest,
EditorSoundEffectGenerateRequest, EditorVideoGenerateRequest, EditorVideoGenerateResponse,
};
use shared_contracts::editor_agent::{
EditorAgentGeneratedAudio, EditorAgentGeneratedImage, EditorAgentGeneratedVideo,
EditorAgentToolCallDisplayArgs, EditorAgentToolCallDisplayExtras,
};
use spacetime_client::{EditorAgentConversationRecord, EditorProjectRecord};
use crate::editor_agent::display_args::{
push_image_display_arg, push_image_generation_display_args, push_string_display_arg,
};
use crate::editor_agent::resp_to_asset;
use crate::editor_agent::utils::build_editor_agent_canvas_completion;
use crate::editor_generation_config::EditorGenerationPricingConfig;
use crate::editor_generation_queue::{
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND,
EDITOR_IMAGE_EDIT_JOB_KIND, EDITOR_IMAGE_GENERATION_JOB_KIND,
EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND,
};
use crate::editor_project::{EditorImageEditRequest, EditorImageGenerationRequest};
use crate::editor_project_icon::EditorIconSpritesheetGenerationRequest;
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) enum EditorAgentToolError {
InvalidArgs(String),
Internal(String),
}
impl EditorAgentToolError {
fn invalid_args(message: impl Into<String>) -> Self {
Self::InvalidArgs(message.into())
}
fn internal(message: impl Into<String>) -> Self {
Self::Internal(message.into())
}
pub(crate) fn into_prompt_error(self, tool_name: &str) -> PromptError {
match self {
Self::InvalidArgs(message) => {
PromptError::ToolError(format!("invalid args for tool {tool_name}: {message}"))
}
Self::Internal(message) => PromptError::InternalError(message),
}
}
pub(crate) fn is_invalid_args(&self) -> bool {
matches!(self, Self::InvalidArgs(_))
}
}
/// `get_external_generation_job_generated_artifacts` 的精简读取结果不带 `job_kind`;
/// Agent 会话回填只能从已持久化的 tool name 恢复音频任务类型,以复用普通任务读取侧的
/// 用户可见错误投影。
pub(crate) fn editor_agent_audio_generation_job_kind(tool_name: &str) -> Option<&'static str> {
match tool_name {
GenerateSoundEffectTool::NAME => Some(EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND),
GenerateBackgroundMusicTool::NAME => Some(EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND),
_ => None,
}
}
impl Display for EditorAgentToolError {
fn fmt(&self, formatter: &mut Formatter<'_>) -> std::fmt::Result {
match self {
Self::InvalidArgs(message) | Self::Internal(message) => formatter.write_str(message),
}
}
}
impl std::error::Error for EditorAgentToolError {}
pub(crate) struct EditorAgentPrepareJobContext<'a> {
pub(crate) conversation: &'a EditorAgentConversationRecord,
pub(crate) project: &'a EditorProjectRecord,
pub(crate) message_id: usize,
pub(crate) pricing: &'a EditorGenerationPricingConfig,
}
impl EditorAgentPrepareJobContext<'_> {
fn generation_inputs(&self, title: &str, value: &str) -> Option<Value> {
Some(json!({
"source": "editor-agent",
"conversationId": self.conversation.conversation_id,
"toolCallMessageId": self.message_id,
"fields": [{ "title": title, "value": value }],
}))
}
}
pub(crate) struct PreparedEditorAgentJob {
pub(crate) job_kind: &'static str,
pub(crate) request_label: &'static str,
pub(crate) price_mud_points: u32,
pub(crate) payload: Value,
}
#[derive(Default)]
pub(crate) struct EditorAgentResultAssets {
pub(crate) images: Option<Vec<EditorAgentGeneratedImage>>,
pub(crate) videos: Option<Vec<EditorAgentGeneratedVideo>>,
pub(crate) audios: Option<Vec<EditorAgentGeneratedAudio>>,
}
/// api-server 侧统一的画布 Agent 工具能力。
///
/// 所有方法都使用统一 JSON 边界以保持 trait object-safe;每个具体实现负责把 JSON
/// 反序列化为真实 Tool Args / 执行结果,并转发到具体工具已有的强类型行为。
pub(crate) trait EditorAgentTool: ToolDyn {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError>;
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError>;
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError>;
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError>;
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError>;
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError>;
}
pub(crate) fn editor_agent_tool(
tool_name: &str,
context: &EditorToolContext,
) -> Option<Box<dyn EditorAgentTool>> {
match tool_name {
GenerateImageTool::NAME => Some(Box::new(GenerateImageTool {
context: context.clone(),
})),
GenerateCharacterTool::NAME => Some(Box::new(GenerateCharacterTool {
context: context.clone(),
})),
GenerateUiDesignTool::NAME => Some(Box::new(GenerateUiDesignTool {
context: context.clone(),
})),
EditImageTool::NAME => Some(Box::new(EditImageTool {
context: context.clone(),
})),
GenerateIconSpritesheetTool::NAME => Some(Box::new(GenerateIconSpritesheetTool {
context: context.clone(),
})),
GenerateVideoTool::NAME => Some(Box::new(GenerateVideoTool {
context: context.clone(),
})),
GenerateSoundEffectTool::NAME => Some(Box::new(GenerateSoundEffectTool)),
GenerateBackgroundMusicTool::NAME => Some(Box::new(GenerateBackgroundMusicTool)),
_ => None,
}
}
fn parse_invalid_args<T: DeserializeOwned>(
tool_name: &str,
value: &Value,
) -> Result<T, EditorAgentToolError> {
serde_json::from_value(null_tool_args_as_missing(value.clone())).map_err(|error| {
EditorAgentToolError::invalid_args(format!(
"failed to deserialize {tool_name} args: {error}"
))
})
}
fn parse_internal<T: DeserializeOwned>(
label: &str,
value: &Value,
) -> Result<T, EditorAgentToolError> {
serde_json::from_value(null_tool_args_as_missing(value.clone())).map_err(|error| {
EditorAgentToolError::internal(format!("failed to deserialize {label}: {error}"))
})
}
/// SFX V2 的显式 `duration: null` 表示自动时长,必须原样保留,不能被通用的
/// “顶层 null 当缺省”兼容层恢复成手动 5 秒。但其余字段仍要走该兼容层:
/// `model` 是无 Option 的 String,LLM 传 `model: null` 时若不剥离会直接反序列化失败。
fn parse_sound_effect_args(
label: &str,
value: &Value,
invalid_args: bool,
) -> Result<GenerateSoundEffectToolArgs, EditorAgentToolError> {
let mut normalized = null_tool_args_as_missing(value.clone());
let has_explicit_null_duration = value
.get("duration")
.is_some_and(serde_json::Value::is_null);
if has_explicit_null_duration && let Some(fields) = normalized.as_object_mut() {
fields.insert("duration".to_string(), Value::Null);
}
serde_json::from_value(normalized).map_err(|error| {
let message = format!("failed to deserialize {label}: {error}");
if invalid_args {
EditorAgentToolError::invalid_args(message)
} else {
EditorAgentToolError::internal(message)
}
})
}
fn serialize_normalized_args<T: Serialize>(
tool_name: &str,
args: &T,
) -> Result<Value, EditorAgentToolError> {
serde_json::to_value(args).map_err(|error| {
EditorAgentToolError::internal(format!(
"failed to normalize args for tool {tool_name}: {error}"
))
})
}
fn serialize_job_payload<T: Serialize>(
tool_name: &str,
payload: T,
) -> Result<Value, EditorAgentToolError> {
serde_json::to_value(payload).map_err(|error| {
EditorAgentToolError::internal(format!(
"failed to serialize tool payload for {tool_name}: {error}"
))
})
}
fn finish_display_args(
mut display_args: EditorAgentToolCallDisplayArgs,
price_mud_points: u32,
) -> EditorAgentToolCallDisplayArgs {
display_args.extras = EditorAgentToolCallDisplayExtras { price_mud_points };
display_args
}
fn map_display_error(error: PromptError) -> EditorAgentToolError {
EditorAgentToolError::internal(error.to_string())
}
fn resolve_image_ids(
image_ids: &[ImageId],
context: &EditorToolContext,
) -> Result<Vec<String>, EditorAgentToolError> {
image_ids
.iter()
.map(|image_id| {
context
.image_data_key(image_id)
.map(ToOwned::to_owned)
.ok_or_else(|| {
EditorAgentToolError::invalid_args(format!("image {image_id} not found"))
})
})
.collect()
}
fn editor_agent_image_mud_points(
pricing: &EditorGenerationPricingConfig,
kind: Option<&str>,
model: &str,
image_size: Option<&str>,
) -> u32 {
pricing.image_generation_mud_points(kind, Some(model), image_size)
}
impl EditorAgentTool for GenerateImageTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: GenerateImageToolArgs = parse_invalid_args(Self::NAME, args)?;
GenerateImageTool::validate_args(self, &args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let args: GenerateImageToolArgs = parse_internal("generate image args", args)?;
Ok(editor_agent_image_mud_points(
pricing,
None,
args.model.as_str(),
Some(args.image_size.as_str()),
))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args: GenerateImageToolArgs = parse_internal("generate image display args", args)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_image_generation_display_args(
&mut display_args,
&self.context,
args.prompt,
args.model,
args.aspect_ratio,
args.image_size,
args.reference_image_ids,
)
.map_err(map_display_error)?;
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: GenerateImageToolArgs = parse_invalid_args(Self::NAME, args)?;
let title = args.prompt.clone();
let reference_image_srcs = resolve_image_ids(&args.reference_image_ids, &self.context)?;
let payload = EditorImageGenerationRequest {
prompt: args.prompt,
size: None,
kind: None,
style: None,
model: Some(args.model),
screen_color: None,
seg_model: None,
aspect_ratio: Some(args.aspect_ratio),
image_size: Some(args.image_size),
reference_image_srcs: Some(reference_image_srcs),
project_id: Some(context.conversation.project_id.clone()),
asset_kind: Some("editor_agent_generated_image".to_string()),
generation_inputs: context.generation_inputs("用户指令", &title),
asset_folder_id: Some("project".to_string()),
asset_label: Some(title.clone()),
source_resource_id: None,
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
&title,
)),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_IMAGE_GENERATION_JOB_KIND,
request_label: "画布 Agent 生成图片",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args: GenerateImageToolArgs = parse_internal("generate image args", args)?;
let result: EditorImageGenerationResult = parse_internal("generate image result", result)?;
Ok(GenerateImageTool::format_execute_message(
self, &args, &result,
))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorImageGenerationResult = parse_internal("generate image result", result)?;
Ok(EditorAgentResultAssets {
images: Some(vec![resp_to_asset::editor_agent_generated_image(&result)]),
..Default::default()
})
}
}
impl EditorAgentTool for GenerateCharacterTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: GenerateCharacterToolArgs = parse_invalid_args(Self::NAME, args)?;
GenerateCharacterTool::validate_args(self, &args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let args: GenerateCharacterToolArgs = parse_internal("generate character args", args)?;
Ok(editor_agent_image_mud_points(
pricing,
Some("character"),
args.model.as_str(),
Some(args.image_size.as_str()),
))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args: GenerateCharacterToolArgs =
parse_internal("generate character display args", args)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_image_generation_display_args(
&mut display_args,
&self.context,
args.prompt,
args.model,
args.aspect_ratio,
args.image_size,
args.reference_image_ids,
)
.map_err(map_display_error)?;
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: GenerateCharacterToolArgs = parse_invalid_args(Self::NAME, args)?;
let title = args.prompt.clone();
let reference_image_srcs = resolve_image_ids(&args.reference_image_ids, &self.context)?;
let payload = EditorImageGenerationRequest {
prompt: args.prompt,
size: None,
kind: Some("character".to_string()),
style: None,
model: Some(args.model),
screen_color: Some("auto".to_string()),
seg_model: Some("birefnet".to_string()),
aspect_ratio: Some(args.aspect_ratio),
image_size: Some(args.image_size),
reference_image_srcs: Some(reference_image_srcs),
project_id: Some(context.conversation.project_id.clone()),
asset_kind: Some("character".to_string()),
generation_inputs: context.generation_inputs("用户指令", &title),
asset_folder_id: Some("project".to_string()),
asset_label: Some(title.clone()),
source_resource_id: None,
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
&title,
)),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_IMAGE_GENERATION_JOB_KIND,
request_label: "画布 Agent 生成图片",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args: GenerateCharacterToolArgs = parse_internal("generate character args", args)?;
let result: EditorImageGenerationResult =
parse_internal("generate character result", result)?;
Ok(GenerateCharacterTool::format_execute_message(
self, &args, &result,
))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorImageGenerationResult =
parse_internal("generate character result", result)?;
Ok(EditorAgentResultAssets {
images: Some(vec![resp_to_asset::editor_agent_generated_image(&result)]),
..Default::default()
})
}
}
impl EditorAgentTool for GenerateUiDesignTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: GenerateUiDesignToolArgs = parse_invalid_args(Self::NAME, args)?;
GenerateUiDesignTool::validate_args(self, &args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let args: GenerateUiDesignToolArgs = parse_internal("generate UI design args", args)?;
Ok(editor_agent_image_mud_points(
pricing,
Some("ui-design"),
args.model.as_str(),
Some(args.image_size.as_str()),
))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args: GenerateUiDesignToolArgs =
parse_internal("generate UI design display args", args)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_image_generation_display_args(
&mut display_args,
&self.context,
args.prompt,
args.model,
args.aspect_ratio,
args.image_size,
args.reference_image_ids,
)
.map_err(map_display_error)?;
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: GenerateUiDesignToolArgs = parse_invalid_args(Self::NAME, args)?;
let title = args.prompt.clone();
let reference_image_srcs = resolve_image_ids(&args.reference_image_ids, &self.context)?;
let payload = EditorImageGenerationRequest {
prompt: args.prompt,
size: None,
kind: Some("ui-design".to_string()),
style: None,
model: Some(args.model),
screen_color: None,
seg_model: None,
aspect_ratio: Some(args.aspect_ratio),
image_size: Some(args.image_size),
reference_image_srcs: Some(reference_image_srcs),
project_id: Some(context.conversation.project_id.clone()),
asset_kind: Some("ui-design".to_string()),
generation_inputs: context.generation_inputs("用户指令", &title),
asset_folder_id: Some("project".to_string()),
asset_label: Some(title.clone()),
source_resource_id: None,
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
&title,
)),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_IMAGE_GENERATION_JOB_KIND,
request_label: "画布 Agent 生成图片",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args: GenerateUiDesignToolArgs = parse_internal("generate UI design args", args)?;
let result: EditorImageGenerationResult =
parse_internal("generate UI design result", result)?;
Ok(GenerateUiDesignTool::format_execute_message(
self, &args, &result,
))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorImageGenerationResult =
parse_internal("generate UI design result", result)?;
Ok(EditorAgentResultAssets {
images: Some(vec![resp_to_asset::editor_agent_generated_image(&result)]),
..Default::default()
})
}
}
impl EditorAgentTool for EditImageTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: EditImageToolArgs = parse_invalid_args(Self::NAME, args)?;
if let Some(error) = EditImageTool::validate_args(self, &args) {
return Err(EditorAgentToolError::invalid_args(error.to_string()));
}
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let _: EditImageToolArgs = parse_internal("edit image args", args)?;
Ok(editor_agent_image_mud_points(
pricing,
Some("quick-edit"),
GPT_IMAGE_2_MODEL,
Some("1K"),
))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args: EditImageToolArgs = parse_internal("edit image display args", args)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_string_display_arg(&mut display_args, "prompt", "修改要求", args.prompt);
push_string_display_arg(&mut display_args, "model", "模型", args.model);
push_image_display_arg(
&mut display_args,
&self.context,
"object_image_id",
"目标图片",
std::slice::from_ref(&args.object_image_id),
)
.map_err(map_display_error)?;
push_image_display_arg(
&mut display_args,
&self.context,
"reference_image_ids",
"参考图片",
args.reference_image_ids.as_slice(),
)
.map_err(map_display_error)?;
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: EditImageToolArgs = parse_invalid_args(Self::NAME, args)?;
let source_reference_id = self
.context
.image_metadata(&args.object_image_id)
.and_then(|metadata| metadata.reference_id.as_deref())
.ok_or_else(|| {
EditorAgentToolError::invalid_args(
"object image is not a registered resource or asset",
)
})?
.to_string();
let reference_image_srcs = resolve_image_ids(&args.reference_image_ids, &self.context)?;
let title = args.prompt.clone();
let payload = EditorImageEditRequest {
prompt: args.prompt,
source_reference_id,
size: None,
model: Some(args.model),
aspect_ratio: None,
image_size: None,
reference_image_srcs: Some(reference_image_srcs),
project_id: Some(context.conversation.project_id.clone()),
generation_inputs: context.generation_inputs("用户指令", &title),
asset_folder_id: Some("project".to_string()),
asset_label: Some(title.clone()),
target_layer_id: None,
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
&title,
)),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_IMAGE_EDIT_JOB_KIND,
request_label: "画布 Agent 修改图片",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args: EditImageToolArgs = parse_internal("edit image args", args)?;
let result: EditorImageEditResult = parse_internal("edit image result", result)?;
Ok(EditImageTool::format_execute_message(self, args, result))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorImageEditResult = parse_internal("edit image result", result)?;
Ok(EditorAgentResultAssets {
images: Some(vec![resp_to_asset::editor_agent_generated_edited_image(
&result,
)]),
..Default::default()
})
}
}
fn resolve_icon_spec_reference_id<'a>(
context: &'a EditorToolContext,
image_id: &ImageId,
) -> Result<&'a str, EditorAgentToolError> {
let metadata = context.image_metadata(image_id).ok_or_else(|| {
EditorAgentToolError::invalid_args("reference image is not available in tool context")
})?;
let reference_id = metadata
.reference_id
.as_deref()
.map(str::trim)
.filter(|reference_id| !reference_id.is_empty())
.ok_or_else(|| {
EditorAgentToolError::invalid_args(
"reference image is not a registered resource or asset",
)
})?;
if metadata.asset_kind.as_deref().map(str::trim) != Some("icon-spec") {
return Err(EditorAgentToolError::invalid_args(
"reference image must be a registered icon-spec; select an icon specification image and try again",
));
}
Ok(reference_id)
}
impl EditorAgentTool for GenerateIconSpritesheetTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: GenerateIconSpritesheetToolArgs = parse_invalid_args(Self::NAME, args)?;
let args = GenerateIconSpritesheetTool::normalize_args(self, args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
resolve_icon_spec_reference_id(&self.context, &args.reference_image_id)?;
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let args: GenerateIconSpritesheetToolArgs =
parse_internal("generate icon spritesheet args", args)?;
Ok(editor_agent_image_mud_points(
pricing,
Some("icon"),
args.model.as_str(),
Some(args.image_size.as_str()),
))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args: GenerateIconSpritesheetToolArgs =
parse_internal("generate icon spritesheet display args", args)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_string_display_arg(
&mut display_args,
"icon_descriptions",
"图标描述",
args.icon_descriptions.join("\n"),
);
push_string_display_arg(&mut display_args, "model", "模型", args.model);
push_string_display_arg(
&mut display_args,
"aspect_ratio",
"画面比例",
args.aspect_ratio,
);
push_string_display_arg(&mut display_args, "image_size", "图片尺寸", args.image_size);
push_image_display_arg(
&mut display_args,
&self.context,
"reference_image_id",
"规范参考图",
std::slice::from_ref(&args.reference_image_id),
)
.map_err(map_display_error)?;
push_image_display_arg(
&mut display_args,
&self.context,
"reference_image_ids",
"额外参考图",
args.reference_image_ids.as_slice(),
)
.map_err(map_display_error)?;
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: GenerateIconSpritesheetToolArgs = parse_invalid_args(Self::NAME, args)?;
let reference_id =
resolve_icon_spec_reference_id(&self.context, &args.reference_image_id)?.to_string();
let reference_image_srcs = resolve_image_ids(&args.reference_image_ids, &self.context)?;
let payload = EditorIconSpritesheetGenerationRequest {
reference_id,
reference_image_srcs: Some(reference_image_srcs),
icon_descriptions: args.icon_descriptions,
slice_count: None,
// 画板 Agent 只生成自由排布的图标表,因此显式声明连通域切分;等分网格或固定
// 槽位需求必须由调用方在外部 API 显式传 grid + gridX/gridY,不能依赖任何默认值。
slice_mode: Some("connected-components".to_string()),
grid_x: None,
grid_y: None,
style: None,
model: Some(args.model),
screen_color: Some("auto".to_string()),
seg_model: Some("birefnet".to_string()),
aspect_ratio: Some(args.aspect_ratio),
image_size: Some(args.image_size),
project_id: Some(context.conversation.project_id.clone()),
generation_inputs: Some(json!({
"source": "editor-agent",
"conversationId": context.conversation.conversation_id,
"toolCallMessageId": context.message_id,
})),
asset_folder_id: Some("project".to_string()),
asset_label: Some("图标素材图集".to_string()),
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
"图标素材图集",
)),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND,
request_label: "画布 Agent 生成图标素材",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args: GenerateIconSpritesheetToolArgs =
parse_internal("generate icon spritesheet args", args)?;
let result: EditorIconSpritesheetResult =
parse_internal("generate icon spritesheet result", result)?;
Ok(GenerateIconSpritesheetTool::format_execute_message(
self, &args, &result,
))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorIconSpritesheetResult =
parse_internal("generate icon spritesheet result", result)?;
Ok(EditorAgentResultAssets {
images: Some(resp_to_asset::editor_agent_generated_spritesheet_images(
&result,
)),
..Default::default()
})
}
}
impl EditorAgentTool for GenerateVideoTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: GenerateVideoToolArgs = parse_invalid_args(Self::NAME, args)?;
GenerateVideoTool::validate_args(self, &args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let args: GenerateVideoToolArgs = parse_internal("generate video args", args)?;
Ok(pricing.video_model_mud_points(
Some(args.model.as_str()),
args.resolution.as_str(),
args.duration_seconds,
))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args: GenerateVideoToolArgs = parse_internal("generate video display args", args)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_string_display_arg(&mut display_args, "prompt", "视频描述", args.prompt);
push_string_display_arg(
&mut display_args,
"aspect_ratio",
"画面比例",
args.aspect_ratio,
);
push_string_display_arg(
&mut display_args,
"duration_seconds",
"时长(秒)",
args.duration_seconds.to_string(),
);
push_string_display_arg(&mut display_args, "model", "模型", args.model);
push_string_display_arg(&mut display_args, "resolution", "分辨率", args.resolution);
push_string_display_arg(&mut display_args, "sound", "声音", args.sound);
push_image_display_arg(
&mut display_args,
&self.context,
"reference_image_ids",
"参考图片",
args.reference_image_ids.as_slice(),
)
.map_err(map_display_error)?;
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: GenerateVideoToolArgs = parse_invalid_args(Self::NAME, args)?;
let reference_image_srcs = resolve_image_ids(&args.reference_image_ids, &self.context)?;
let title = args.prompt.clone();
let payload = EditorVideoGenerateRequest {
prompt: args.prompt,
model: args.model,
aspect_ratio: args.aspect_ratio,
duration_seconds: args.duration_seconds,
resolution: args.resolution,
mode: "std".to_string(),
sound: args.sound,
web_search_enabled: false,
reference_image_srcs,
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
project_id: Some(context.conversation.project_id.clone()),
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
&title,
)),
generation_inputs: context.generation_inputs("用户指令", &title),
source_resource_id: None,
asset_kind: Some("video".to_string()),
asset_folder_id: Some("project".to_string()),
asset_label: Some(title),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_VIDEO_GENERATION_JOB_KIND,
request_label: "画布 Agent 生成视频",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args: GenerateVideoToolArgs = parse_internal("generate video args", args)?;
let result: EditorVideoGenerateResponse = parse_internal("generate video result", result)?;
Ok(GenerateVideoTool::format_execute_message(
self, &args, &result,
))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorVideoGenerateResponse = parse_internal("generate video result", result)?;
Ok(EditorAgentResultAssets {
videos: Some(vec![resp_to_asset::editor_agent_generated_video(&result)]),
..Default::default()
})
}
}
impl EditorAgentTool for GenerateSoundEffectTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args = parse_sound_effect_args(Self::NAME, args, true)?;
let args = GenerateSoundEffectTool::normalize_args(self, args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let args = parse_sound_effect_args("generate sound effect args", args, false)?;
Ok(pricing.sound_effect_model_mud_points(Some(args.model.as_str())))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args = parse_sound_effect_args("generate sound effect display args", args, false)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_string_display_arg(&mut display_args, "prompt", "音效描述", args.prompt);
push_string_display_arg(&mut display_args, "model", "模型", args.model);
push_string_display_arg(
&mut display_args,
"duration",
"时长(秒)",
args.duration
.map(|duration| format!("{duration}秒"))
.unwrap_or_else(|| "自动".to_string()),
);
push_string_display_arg(
&mut display_args,
"loop",
"Loop",
if args.loop_enabled {
"开启"
} else {
"关闭"
}
.to_string(),
);
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args = parse_sound_effect_args(Self::NAME, args, true)?;
let title = args.prompt.clone();
let payload = EditorSoundEffectGenerateRequest {
prompt: args.prompt,
model: Some(args.model),
duration: args.duration,
loop_enabled: args.loop_enabled,
project_id: Some(context.conversation.project_id.clone()),
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
&title,
)),
generation_inputs: context.generation_inputs("用户指令", &title),
asset_folder_id: Some("project".to_string()),
asset_label: Some(title),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
request_label: "画布 Agent 生成音效",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args = parse_sound_effect_args("generate sound effect args", args, false)?;
let result: EditorAudioGenerateResponse =
parse_internal("generate sound effect result", result)?;
Ok(GenerateSoundEffectTool::format_execute_message(
self, &args, &result,
))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorAudioGenerateResponse =
parse_internal("generate sound effect result", result)?;
Ok(EditorAgentResultAssets {
audios: Some(vec![resp_to_asset::editor_agent_generated_audio(&result)]),
..Default::default()
})
}
}
impl EditorAgentTool for GenerateBackgroundMusicTool {
fn validate_args(&self, args: &Value) -> Result<Value, EditorAgentToolError> {
let args: GenerateBackgroundMusicToolArgs = parse_invalid_args(Self::NAME, args)?;
GenerateBackgroundMusicTool::validate_args(self, &args)
.map_err(|error| EditorAgentToolError::invalid_args(error.to_string()))?;
serialize_normalized_args(Self::NAME, &args)
}
fn pricing(
&self,
pricing: &EditorGenerationPricingConfig,
args: &Value,
) -> Result<u32, EditorAgentToolError> {
let args: GenerateBackgroundMusicToolArgs =
parse_internal("generate background music args", args)?;
Ok(pricing.background_music_model_mud_points(Some(args.model.as_str())))
}
fn build_display_args(
&self,
args: &Value,
pricing: &EditorGenerationPricingConfig,
) -> Result<EditorAgentToolCallDisplayArgs, EditorAgentToolError> {
let price_mud_points = self.pricing(pricing, args)?;
let args: GenerateBackgroundMusicToolArgs =
parse_internal("generate background music display args", args)?;
let mut display_args = EditorAgentToolCallDisplayArgs::default();
push_string_display_arg(&mut display_args, "prompt", "音乐描述", args.prompt);
push_string_display_arg(&mut display_args, "model", "模型", args.model);
Ok(finish_display_args(display_args, price_mud_points))
}
fn prepare_job(
&self,
args: &Value,
context: &EditorAgentPrepareJobContext<'_>,
) -> Result<PreparedEditorAgentJob, EditorAgentToolError> {
let price_mud_points = self.pricing(context.pricing, args)?;
let args: GenerateBackgroundMusicToolArgs = parse_invalid_args(Self::NAME, args)?;
let title = args.prompt.clone();
let payload = EditorBackgroundMusicGenerateRequest {
gpt_description_prompt: args.prompt,
make_instrumental: true,
project_id: Some(context.conversation.project_id.clone()),
canvas_completion: Some(build_editor_agent_canvas_completion(
context.project,
Self::NAME,
"生成背景音乐",
)),
generation_inputs: context.generation_inputs("用户指令", &title),
asset_folder_id: Some("project".to_string()),
asset_label: Some("生成背景音乐".to_string()),
};
Ok(PreparedEditorAgentJob {
job_kind: EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
request_label: "画布 Agent 生成背景音乐",
price_mud_points,
payload: serialize_job_payload(Self::NAME, payload)?,
})
}
fn format_execute_message(
&self,
args: &Value,
result: &Value,
) -> Result<String, EditorAgentToolError> {
let args: GenerateBackgroundMusicToolArgs =
parse_internal("generate background music args", args)?;
let result: EditorAudioGenerateResponse =
parse_internal("generate background music result", result)?;
Ok(GenerateBackgroundMusicTool::format_execute_message(
self, &args, &result,
))
}
fn result_assets(
&self,
result: &Value,
) -> Result<EditorAgentResultAssets, EditorAgentToolError> {
let result: EditorAudioGenerateResponse =
parse_internal("generate background music result", result)?;
Ok(EditorAgentResultAssets {
audios: Some(vec![resp_to_asset::editor_agent_generated_audio(&result)]),
..Default::default()
})
}
}
#[cfg(test)]
mod tests {
use std::collections::HashMap;
use platform_editor_agent::agent::asset::ImageMetadata;
use serde_json::json;
use spacetime_client::{EditorCanvasRecord, EditorCanvasViewportRecord};
use super::*;
use crate::editor_generation_config::load_editor_generation_pricing_from_paths;
fn pricing() -> EditorGenerationPricingConfig {
load_editor_generation_pricing_from_paths(None).expect("default editor pricing should load")
}
fn context_with_image(image_id: &str) -> EditorToolContext {
EditorToolContext {
images: HashMap::from([(
ImageId {
id: image_id.to_string(),
},
ImageMetadata {
data_key: format!("generated/{image_id}.png"),
reference_id: Some(format!("resource-{image_id}")),
asset_kind: Some("icon-spec".to_string()),
image_src: format!("/api/assets/read/generated/{image_id}.png"),
object_key: Some(format!("generated/{image_id}.png")),
thumbnail_src: Some(format!("/api/assets/read/generated/{image_id}-thumb.png")),
label: Some("测试图片".to_string()),
width: Some(640),
height: Some(480),
},
)]),
}
}
fn conversation() -> EditorAgentConversationRecord {
EditorAgentConversationRecord {
conversation_id: "conversation-1".to_string(),
project_id: "project-1".to_string(),
owner_user_id: "user-1".to_string(),
title: "测试会话".to_string(),
messages_object_key: "editor-agent/conversation-1.json".to_string(),
deleted: false,
created_at: "2026-07-23T00:00:00Z".to_string(),
updated_at: "2026-07-23T00:00:00Z".to_string(),
updated_at_micros: 0,
}
}
fn project() -> EditorProjectRecord {
let viewport = EditorCanvasViewportRecord {
x: 0.0,
y: 0.0,
scale: 1.0,
};
EditorProjectRecord {
project_id: "project-1".to_string(),
owner_user_id: "user-1".to_string(),
title: "测试工程".to_string(),
canvas: EditorCanvasRecord {
canvas_id: "canvas-1".to_string(),
project_id: "project-1".to_string(),
title: "测试画布".to_string(),
viewport: viewport.clone(),
layers: json!([]),
revision: 0,
layout_storage_version: 2,
background_color: None,
created_at: "2026-07-23T00:00:00Z".to_string(),
updated_at: "2026-07-23T00:00:00Z".to_string(),
},
viewport,
layers: json!([]),
resources: Vec::new(),
created_at: "2026-07-23T00:00:00Z".to_string(),
updated_at: "2026-07-23T00:00:00Z".to_string(),
}
}
#[test]
fn factory_resolves_every_editor_agent_tool_as_one_dyn_contract() {
let context = EditorToolContext::default();
for name in [
GenerateImageTool::NAME,
GenerateCharacterTool::NAME,
GenerateUiDesignTool::NAME,
EditImageTool::NAME,
GenerateIconSpritesheetTool::NAME,
GenerateVideoTool::NAME,
GenerateSoundEffectTool::NAME,
GenerateBackgroundMusicTool::NAME,
] {
let tool = editor_agent_tool(name, &context).expect("registered tool should resolve");
assert_eq!(tool.tool_name(), name);
}
assert!(editor_agent_tool("unknown-tool", &context).is_none());
}
#[test]
fn dyn_validation_deserializes_forwards_and_reserializes_typed_args() {
let ui = editor_agent_tool(GenerateUiDesignTool::NAME, &EditorToolContext::default())
.expect("UI tool should resolve");
let normalized = ui
.validate_args(&json!({
"prompt": "生成游戏主界面",
"unknown": "drop-me"
}))
.expect("typed UI validation should pass");
assert_eq!(normalized["model"], GPT_IMAGE_2_MODEL);
assert_eq!(normalized["aspect_ratio"], "1:1");
assert_eq!(normalized["image_size"], "1K");
assert!(normalized.get("unknown").is_none());
let music = editor_agent_tool(
GenerateBackgroundMusicTool::NAME,
&EditorToolContext::default(),
)
.expect("music tool should resolve");
let normalized = music
.validate_args(&json!({
"prompt": "温暖舒缓的钢琴背景音乐",
"make_instrumental": false
}))
.expect("legacy music args should normalize");
assert_eq!(
normalized["model"],
GenerateBackgroundMusicTool::DEFAULT_MODEL
);
assert!(normalized.get("make_instrumental").is_none());
let sound = editor_agent_tool(GenerateSoundEffectTool::NAME, &EditorToolContext::default())
.expect("sound tool should resolve");
// SFX V2 手动时长边界是 0.5-30 秒(旧 Vidu 契约是 10 秒上限),越界值随之上移。
assert!(
sound
.validate_args(&json!({ "prompt": "按钮点击声", "duration": 30.01 }))
.is_err()
);
assert!(
sound
.validate_args(&json!({ "prompt": "按钮点击声", "duration": 11 }))
.is_ok()
);
}
#[test]
fn dyn_validation_treats_explicit_null_as_missing_before_canonical_persistence() {
let image = editor_agent_tool(GenerateImageTool::NAME, &EditorToolContext::default())
.expect("image tool should resolve");
let image_args = image
.validate_args(&json!({
"prompt": "生成森林场景",
"model": null,
"reference_image_ids": null,
"aspect_ratio": null,
"image_size": null
}))
.expect("null image defaults should normalize");
assert_eq!(image_args["model"], platform_image::NANOBANANA_2_MODEL);
assert_eq!(image_args["reference_image_ids"], json!([]));
assert_eq!(image_args["aspect_ratio"], "1:1");
assert_eq!(image_args["image_size"], "1K");
let video = editor_agent_tool(GenerateVideoTool::NAME, &EditorToolContext::default())
.expect("video tool should resolve");
let video_args = video
.validate_args(&json!({
"prompt": "镜头向前推进",
"model": null,
"aspect_ratio": null,
"duration_seconds": null,
"resolution": null,
"sound": null
}))
.expect("null video defaults should normalize");
assert_eq!(video_args["model"], GenerateVideoTool::DEFAULT_VIDEO_MODEL);
assert_eq!(video_args["aspect_ratio"], "16:9");
assert_eq!(video_args["duration_seconds"], 4);
assert_eq!(video_args["resolution"], "720p");
assert_eq!(video_args["sound"], "on");
let sound = editor_agent_tool(GenerateSoundEffectTool::NAME, &EditorToolContext::default())
.expect("sound tool should resolve");
let sound_args = sound
.validate_args(&json!({
"prompt": "按钮点击声",
"model": null,
"duration": null
}))
.expect("null sound defaults should normalize");
assert_eq!(sound_args["model"], GenerateSoundEffectTool::DEFAULT_MODEL);
// SFX V2 与图片 / 视频不同:`duration: null` 是「自动时长」这一有效取值,
// 不能被 null-as-missing 兼容层恢复成手动默认 5 秒。
assert!(sound_args["duration"].is_null());
assert!(
sound
.validate_args(&json!({ "prompt": null, "duration": null }))
.is_err()
);
}
#[test]
fn dyn_validation_uses_the_context_bound_to_the_concrete_tool() {
let tool = editor_agent_tool(EditImageTool::NAME, &context_with_image("image-1"))
.expect("edit tool should resolve");
let normalized = tool
.validate_args(&json!({
"object_image_id": "image-1",
"prompt": "改成蓝色"
}))
.expect("bound image should pass typed edit validation");
assert_eq!(normalized["object_image_id"], "image-1");
let missing = editor_agent_tool(EditImageTool::NAME, &EditorToolContext::default())
.expect("edit tool should resolve");
assert!(missing.validate_args(&normalized).is_err());
let registered_icon = editor_agent_tool(
GenerateIconSpritesheetTool::NAME,
&context_with_image("image-1"),
)
.expect("icon spritesheet tool should resolve");
assert!(
registered_icon
.validate_args(&json!({
"reference_image_id": "image-1",
"icon_descriptions": ["背包"]
}))
.is_ok()
);
let mut unregistered_context = context_with_image("image-1");
unregistered_context
.images
.get_mut(&ImageId {
id: "image-1".to_string(),
})
.expect("test image should exist")
.reference_id = None;
let unregistered_icon =
editor_agent_tool(GenerateIconSpritesheetTool::NAME, &unregistered_context)
.expect("icon spritesheet tool should resolve");
assert!(
unregistered_icon
.validate_args(&json!({
"reference_image_id": "image-1",
"icon_descriptions": ["背包"]
}))
.is_err()
);
}
#[test]
fn dyn_pricing_keeps_all_existing_tool_formulas() {
let pricing = pricing();
let context = context_with_image("image-1");
let cases = [
(
GenerateImageTool::NAME,
json!({ "prompt": "图片", "model": platform_image::NANOBANANA_2_MODEL, "image_size": "2K" }),
24,
),
(
GenerateCharacterTool::NAME,
json!({ "prompt": "角色", "model": platform_image::NANOBANANA_2_MODEL }),
12,
),
(
GenerateUiDesignTool::NAME,
json!({ "prompt": "界面", "model": platform_image::GPT_IMAGE_2_MODEL, "image_size": "2K" }),
5,
),
(
EditImageTool::NAME,
json!({ "object_image_id": "image-1", "prompt": "编辑" }),
3,
),
(
GenerateIconSpritesheetTool::NAME,
json!({ "reference_image_id": "image-1", "icon_descriptions": ["背包"], "model": platform_image::NANOBANANA_2_MODEL, "image_size": "2K" }),
24,
),
(
GenerateVideoTool::NAME,
json!({ "prompt": "视频", "model": "seedance2.0", "duration_seconds": 6, "resolution": "720p" }),
144,
),
(
GenerateSoundEffectTool::NAME,
json!({ "prompt": "音效" }),
5,
),
(
GenerateBackgroundMusicTool::NAME,
json!({ "prompt": "音乐" }),
12,
),
];
for (name, raw_args, expected) in cases {
let tool = editor_agent_tool(name, &context).expect("tool should resolve");
let args = tool.validate_args(&raw_args).expect("args should validate");
assert_eq!(
tool.pricing(&pricing, &args).expect("pricing should work"),
expected
);
}
}
#[test]
fn dyn_pricing_uses_the_supplied_runtime_snapshot() {
let mut pricing = pricing();
pricing
.models
.get_mut(GPT_IMAGE_2_MODEL)
.expect("gpt image pricing should exist")
.prices
.insert("2K".to_string(), 37);
let tool = editor_agent_tool(GenerateImageTool::NAME, &EditorToolContext::default())
.expect("image tool should resolve");
let args = tool
.validate_args(&json!({
"prompt": "生成图片",
"model": GPT_IMAGE_2_MODEL,
"image_size": "2K"
}))
.expect("image args should validate");
assert_eq!(
tool.pricing(&pricing, &args).expect("pricing should work"),
37
);
}
#[test]
fn dyn_display_projection_uses_the_same_validated_json_and_pricing() {
let tool = editor_agent_tool(GenerateSoundEffectTool::NAME, &EditorToolContext::default())
.expect("sound tool should resolve");
let args = tool
.validate_args(&json!({ "prompt": "按钮点击声" }))
.expect("sound args should validate");
let display = tool
.build_display_args(&args, &pricing())
.expect("display projection should build");
assert_eq!(display.extras.price_mud_points, 5);
assert_eq!(display.string_args[0].label, "音效描述");
assert_eq!(display.string_args[0].value, "按钮点击声");
assert_eq!(display.string_args[2].value, "5秒");
assert_eq!(display.string_args[3].value, "关闭");
}
#[test]
fn dyn_display_projection_resolves_images_from_bound_context() {
let tool = editor_agent_tool(GenerateImageTool::NAME, &context_with_image("image-1"))
.expect("image tool should resolve");
let args = tool
.validate_args(&json!({
"prompt": "生成图片",
"reference_image_ids": ["image-1"]
}))
.expect("image args should validate");
let display = tool
.build_display_args(&args, &pricing())
.expect("display projection should resolve context image");
let image_ref = &display.image_args[0].refs[0];
assert_eq!(image_ref.image_id, "image-1");
assert_eq!(
image_ref.object_key.as_deref(),
Some("generated/image-1.png")
);
assert_eq!(image_ref.label.as_deref(), Some("测试图片"));
assert_eq!(image_ref.width, Some(640));
assert_eq!(image_ref.height, Some(480));
}
#[test]
fn dyn_formatter_and_asset_projection_forward_to_typed_audio_behavior() {
let tool = editor_agent_tool(GenerateSoundEffectTool::NAME, &EditorToolContext::default())
.expect("sound tool should resolve");
let args = tool
.validate_args(&json!({
"prompt": "\u{feff}按钮点击声\u{2003}",
"duration": null,
"loop": true
}))
.expect("sound args should validate");
let result = json!({
"ok": true,
"audioSrc": "/generated/click.mp3",
"objectKey": "generated/click.mp3",
"assetObjectId": "asset-1",
"width": 0,
"height": 0,
"sourceType": "generated",
"prompt": "按钮点击声",
"actualPrompt": "A short button click",
"model": "eleven_text_to_sound_v2",
"taskId": "task-1",
"priceMudPoints": 5,
"audioKind": "sound-effect"
});
let message = tool
.format_execute_message(&args, &result)
.expect("typed formatter should run");
assert!(message.contains("generated audio saved as: /generated/click.mp3"));
let assets = tool
.result_assets(&result)
.expect("audio asset should project");
assert_eq!(
assets.audios.expect("audio projection")[0].audio_src,
"/generated/click.mp3"
);
}
#[test]
fn edit_image_job_uses_registered_reference_id_only() {
let pricing = pricing();
let conversation = conversation();
let project = project();
let tool = editor_agent_tool(EditImageTool::NAME, &context_with_image("image-1"))
.expect("edit tool should resolve");
let args = tool
.validate_args(&json!({
"object_image_id": "image-1",
"prompt": "改成蓝色"
}))
.expect("edit args should validate");
let job = tool
.prepare_job(
&args,
&EditorAgentPrepareJobContext {
conversation: &conversation,
project: &project,
message_id: 7,
pricing: &pricing,
},
)
.expect("edit job should prepare");
assert_eq!(job.job_kind, EDITOR_IMAGE_EDIT_JOB_KIND);
assert_eq!(job.payload["sourceReferenceId"], "resource-image-1");
assert!(job.payload.get("sourceImageSrc").is_none());
assert!(job.payload.get("sourceResourceId").is_none());
assert!(job.payload.get("assetKind").is_none());
assert_eq!(job.payload["projectId"], "project-1");
assert!(job.payload["targetLayerId"].is_null());
}
#[test]
fn edit_image_job_rejects_image_without_registered_reference_id() {
let pricing = pricing();
let conversation = conversation();
let project = project();
let mut context = context_with_image("image-1");
context
.images
.values_mut()
.next()
.expect("test image should exist")
.reference_id = None;
let tool =
editor_agent_tool(EditImageTool::NAME, &context).expect("edit tool should resolve");
let args = tool
.validate_args(&json!({
"object_image_id": "image-1",
"prompt": "改成蓝色"
}))
.expect("edit args should validate");
let error = match tool.prepare_job(
&args,
&EditorAgentPrepareJobContext {
conversation: &conversation,
project: &project,
message_id: 7,
pricing: &pricing,
},
) {
Ok(_) => panic!("unregistered image should fail closed"),
Err(error) => error,
};
assert!(error.is_invalid_args());
assert_eq!(
error.to_string(),
"object image is not a registered resource or asset"
);
}
#[test]
fn dyn_job_preparation_keeps_queue_contract_and_defaults() {
let pricing = pricing();
let conversation = conversation();
let project = project();
let tool = editor_agent_tool(GenerateSoundEffectTool::NAME, &EditorToolContext::default())
.expect("sound tool should resolve");
let args = tool
.validate_args(&json!({
"prompt": "\u{feff}按钮点击声\u{2003}",
"duration": null,
"loop": true
}))
.expect("sound args should validate");
let job = tool
.prepare_job(
&args,
&EditorAgentPrepareJobContext {
conversation: &conversation,
project: &project,
message_id: 7,
pricing: &pricing,
},
)
.expect("sound job should prepare");
assert_eq!(job.job_kind, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND);
assert_eq!(job.request_label, "画布 Agent 生成音效");
assert_eq!(job.price_mud_points, 5);
assert_eq!(job.payload["duration"], Value::Null);
assert_eq!(job.payload["prompt"], "按钮点击声");
assert_eq!(
job.payload["model"],
shared_contracts::assets::EDITOR_SOUND_EFFECT_MODEL
);
assert_eq!(job.payload["loop"], json!(true));
assert_eq!(job.payload["projectId"], "project-1");
assert_eq!(job.payload["generationInputs"]["toolCallMessageId"], 7);
}
}