diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md index 55800447c..e22a0c104 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md @@ -1,6 +1,6 @@ --- name: agc-client-projection -description: Preserve AGC client resource and version projection semantics after real project changes. Use when adding, replacing, or removing game files or registered art, when reasoning about project revisions and versions, or when client-visible resources appear out of sync. +description: Preserve AGC client resource and version projection semantics after real project changes. Use when querying, creating, deriving, adding, replacing, or removing registered resources, when reasoning about project revisions and versions, or when client-visible resources appear out of sync. --- # AGC Client Projection @@ -10,16 +10,19 @@ Let the client derive projections from real disk changes and trusted tool result ## Workflow 1. Write executable source to the current `game/` files and media to the relative paths returned by approved tools. -2. Preserve existing relative paths when a small edit is sufficient so client resource identities remain stable. -3. Do not edit `.agent/manifest.json`, revision counters, version records, resource IDs, canvas identities, source provenance, generation ledgers, or browser receipts by hand. -4. Do not create a version when no game file changed. The client compares content fingerprints and advances revision only after an actual source change. -5. Do not claim a resource or version is visible before the client projects it. If projection is missing, report the changed relative files and let the client re-read durable state. -6. Never move HTML, CSS, or JavaScript into documentation folders. They belong to the game-code projection; prose, design notes, and instructions remain documents. +2. Before using or deriving an existing asset, call `agc_list_registered_assets` and select its `localAssetId`; never infer a source from a filename or submit a local path, platform ID, object key, operation ID, or idempotency key as a generation argument. +3. When the user explicitly asks to create or derive video, character animation, sound effect, or background music, call `agc_create_or_derive_resource`. Use `create` only for video/audio without a source and `derive` with a registered `sourceLocalAssetId`; character animation is always derived from an image. +4. Preserve existing relative paths when a small edit is sufficient so client resource identities remain stable. +5. Do not edit `.agent/manifest.json`, revision counters, version records, resource IDs, canvas identities, source provenance, generation ledgers, or browser receipts by hand. +6. Do not create a version when no game file changed. The client compares content fingerprints and advances revision only after an actual source change. +7. Do not claim a resource or version is visible before the client projects it. If projection is missing, report the changed relative files and let the client re-read durable state. +8. Never move HTML, CSS, or JavaScript into documentation folders. They belong to the game-code projection; prose, design notes, and instructions remain documents. Call `agc_read_skill_resource` with `skillName="agc-client-projection"` and `relativePath="references/projection-contract.md"` when a request touches asset identity, revision behavior, or version history. ## Boundaries -- Platform provenance comes only from the approved art tool. +- Platform provenance comes only from approved client tools. +- The client owns permission checks, project identity, revision, project locks, paid submission, idempotency, operation recovery, download validation, warning projection, and manifest transactions. - Browser evidence proves runtime behavior, not resource ownership. - Source changes, resource registration, and version projection are distinct facts; report each accurately. diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md index 3aa48e8c6..b7f754400 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md @@ -3,7 +3,11 @@ The client projects three distinct facts: 1. Game-code resources come from actual source files such as `game/index.html`, `game/style.css`, and `game/game.js`. -2. Art resources come from approved tool registration with durable platform provenance. +2. Art and media resources come from approved tool registration with durable platform provenance. 3. A project version is created only after a real game-source fingerprint change and a monotonic project revision update. Do not collapse these facts. A playable file can exist before projection refresh, a registered image can exist without being used by the game, and browser success does not create platform provenance. + +`agc_list_registered_assets` is the only Direct read path for manifest resource identity. Its relative path and stable identifiers are evidence; omitted prompt, model, provider route, signed URL, host path, and credentials are intentionally not available to Codex. + +`agc_create_or_derive_resource` accepts only semantic intent. The client resolves `sourceLocalAssetId`, creates stable request identities, recovers matching pending operations, serializes paid submissions, writes supported media into the current canvas and same-name asset folder, validates downloaded bytes, commits the local manifest transaction, and returns redacted warnings. A tool error or timeout is not permission to generate again with a new identity. diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json index eee1533b4..d04fdb6e3 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json @@ -1,6 +1,6 @@ { "schemaVersion": "agc-skill-pack.v1", - "version": "2026-08-23.7", + "version": "2026-08-24.1", "skills": [ { "name": "agc-project-structure", @@ -78,16 +78,21 @@ "purpose": "保持游戏代码、美术资源、revision 与正式版本的客户端投影一致", "triggers": [ "新增或替换游戏文件", + "查询、创建或派生已登记媒体资源", "素材或版本未显示", "推理 revision 与版本关系" ], - "requiredTools": ["agc_tools.agc_read_skill_resource"], + "requiredTools": [ + "agc_tools.agc_read_skill_resource", + "agc_tools.agc_list_registered_assets", + "agc_tools.agc_create_or_derive_resource" + ], "files": [ "SKILL.md", "agents/openai.yaml", "references/projection-contract.md" ], - "sha256": "e9da95c2f371620e045a3ab9f4721078c805e108263f479b4e01b1af96d462d3" + "sha256": "4b49d54f430028839362a0accf1a5c83878fb1e4eb9b929e0c4912c17470f112" } ] } diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs index c2cbacad9..e1d4f12f7 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs @@ -5,6 +5,7 @@ use axum::{Json, Router}; use base64::{engine::general_purpose::STANDARD as BASE64_STANDARD, Engine as _}; use serde::Deserialize; use serde_json::{json, Value}; +use std::collections::BTreeMap; use std::path::{Path, PathBuf}; use std::sync::{Arc, Mutex as StdMutex}; use unicode_normalization::UnicodeNormalization; @@ -17,12 +18,18 @@ const DIRECT_TOOL_BRIDGE_MAX_ART_BRIEF_CHARS: usize = 4_000; const DIRECT_TOOL_BRIDGE_MAX_IMAGE_BYTES: u64 = 6 * 1024 * 1024; const DIRECT_TOOL_BRIDGE_MAX_SEARCH_QUERY_CHARS: usize = 400; const DIRECT_TOOL_BRIDGE_MAX_SEARCH_RESULTS: usize = 5; +const DIRECT_TOOL_BRIDGE_MAX_RESOURCE_PROMPT_CHARS: usize = 4_000; +const DIRECT_TOOL_BRIDGE_MAX_RESOURCE_NAME_CHARS: usize = 120; +const DIRECT_TOOL_BRIDGE_MAX_RESOURCE_KIND_CHARS: usize = 80; +const DIRECT_TOOL_BRIDGE_MAX_RESOURCE_PAGE_SIZE: usize = 100; +const DIRECT_TOOL_BRIDGE_MAX_RESOURCE_CALLS_PER_TURN: usize = 4; const DIRECT_TOOL_BRIDGE_SEARCH_URL: &str = "https://www.bing.com/search?format=rss"; struct DirectToolBridgeState { root: PathBuf, turn_authorization: StdMutex, regeneration_gate: tokio::sync::Mutex<()>, + resource_generation_gate: tokio::sync::Mutex<()>, } #[derive(Default)] @@ -35,6 +42,7 @@ struct DirectToolBridgeActiveTurnAuthorization { allows_regeneration: bool, brief_sha256: Option, completed_result: Option, + resource_request_ids: BTreeMap, } enum DirectToolBridgeRegenerationCall { @@ -53,6 +61,82 @@ struct DirectToolBridgeRequest { arguments: Value, } +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum DirectResourceGenerationKind { + Video, + CharacterAnimation, + SoundEffect, + BackgroundMusic, +} + +impl DirectResourceGenerationKind { + fn parse(value: &str) -> Result { + match value { + "video" => Ok(Self::Video), + "character-animation" => Ok(Self::CharacterAnimation), + "sound-effect" => Ok(Self::SoundEffect), + "background-music" => Ok(Self::BackgroundMusic), + _ => Err("工具参数 kind 不是受支持的媒体资源类型".to_string()), + } + } + + fn as_str(self) -> &'static str { + match self { + Self::Video => "video", + Self::CharacterAnimation => "character-animation", + Self::SoundEffect => "sound-effect", + Self::BackgroundMusic => "background-music", + } + } + + fn edit_kind(self) -> LocalProjectResourceEditKind { + match self { + Self::Video => LocalProjectResourceEditKind::Video, + Self::CharacterAnimation => LocalProjectResourceEditKind::CharacterAnimation, + Self::SoundEffect => LocalProjectResourceEditKind::SoundEffect, + Self::BackgroundMusic => LocalProjectResourceEditKind::BackgroundMusic, + } + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum DirectResourceGenerationMode { + Create, + Derive, +} + +impl DirectResourceGenerationMode { + fn parse(value: &str) -> Result { + match value { + "create" => Ok(Self::Create), + "derive" => Ok(Self::Derive), + _ => Err("工具参数 mode 必须是 create 或 derive".to_string()), + } + } + + fn as_str(self) -> &'static str { + match self { + Self::Create => "create", + Self::Derive => "derive", + } + } + + fn project_mode(self) -> LocalProjectResourceGenerationMode { + match self { + Self::Create => LocalProjectResourceGenerationMode::Create, + Self::Derive => LocalProjectResourceGenerationMode::Derive, + } + } +} + +struct DirectResourceGenerationInput { + kind: DirectResourceGenerationKind, + mode: DirectResourceGenerationMode, + source_local_asset_id: Option, + prompt: String, + asset_name: String, +} + pub(crate) struct DirectToolBridge { url: String, state: Arc, @@ -95,6 +179,7 @@ impl DirectToolBridgeState { allows_regeneration, brief_sha256: None, completed_result: None, + resource_request_ids: BTreeMap::new(), }); Ok(DirectToolBridgeTurnGuard { state: Arc::clone(self), @@ -526,6 +611,48 @@ impl DirectToolBridgeState { active.completed_result = Some(result.clone()); Ok(()) } + + fn resource_request_ids(&self, request_fingerprint: &str) -> Result<(String, String), String> { + let mut authorization = self + .turn_authorization + .lock() + .map_err(|_| "AGC 工具桥回合授权状态不可用".to_string())?; + let active = authorization + .active + .as_mut() + .ok_or_else(|| "当前没有客户端签发的资源生成回合身份".to_string())?; + if let Some(ids) = active.resource_request_ids.get(request_fingerprint) { + return Ok(ids.clone()); + } + if active.resource_request_ids.len() >= DIRECT_TOOL_BRIDGE_MAX_RESOURCE_CALLS_PER_TURN { + return Err("单个用户回合最多只能创建四项媒体资源请求".to_string()); + } + let operation_id = + direct_resource_request_uuid(&active.turn_id, "operation", request_fingerprint); + let idempotency_key = + direct_resource_request_uuid(&active.turn_id, "idempotency", request_fingerprint); + active.resource_request_ids.insert( + request_fingerprint.to_string(), + (operation_id.clone(), idempotency_key.clone()), + ); + Ok((operation_id, idempotency_key)) + } +} + +fn direct_resource_request_uuid(turn_id: &str, domain: &str, request_fingerprint: &str) -> String { + let mut digest = Sha256::new(); + digest.update(b"genarrative-direct-resource-request-v1\0"); + digest.update(turn_id.as_bytes()); + digest.update([0]); + digest.update(domain.as_bytes()); + digest.update([0]); + digest.update(request_fingerprint.as_bytes()); + let digest = digest.finalize(); + let mut bytes = [0_u8; 16]; + bytes.copy_from_slice(&digest[..16]); + bytes[6] = (bytes[6] & 0x0f) | 0x40; + bytes[8] = (bytes[8] & 0x3f) | 0x80; + uuid::Uuid::from_bytes(bytes).hyphenated().to_string() } fn direct_tool_bridge_state(root: PathBuf) -> Arc { @@ -533,6 +660,7 @@ fn direct_tool_bridge_state(root: PathBuf) -> Arc { root, turn_authorization: StdMutex::new(DirectToolBridgeTurnAuthorization::default()), regeneration_gate: tokio::sync::Mutex::new(()), + resource_generation_gate: tokio::sync::Mutex::new(()), }) } @@ -565,6 +693,130 @@ fn bridge_bounded_string( Ok(value.to_string()) } +fn bridge_optional_bounded_string( + arguments: &Value, + field: &str, + max_chars: usize, +) -> Result, String> { + let Some(value) = arguments.get(field) else { + return Ok(None); + }; + let value = value + .as_str() + .map(str::trim) + .filter(|value| !value.is_empty()) + .ok_or_else(|| format!("工具参数 {field} 不能为空"))?; + if value.chars().count() > max_chars || value.chars().any(char::is_control) { + return Err(format!("工具参数 {field} 超出安全边界")); + } + Ok(Some(value.to_string())) +} + +fn bridge_reject_unknown_fields(arguments: &Value, allowed: &[&str]) -> Result<(), String> { + let object = arguments + .as_object() + .ok_or_else(|| "工具参数必须是对象".to_string())?; + if let Some(field) = object + .keys() + .find(|field| !allowed.contains(&field.as_str())) + { + return Err(format!("工具参数包含未审核字段：{field}")); + } + Ok(()) +} + +fn bridge_registered_asset_page(arguments: &Value) -> Result<(usize, usize), String> { + let offset = arguments + .get("offset") + .map(|value| { + value + .as_u64() + .and_then(|value| usize::try_from(value).ok()) + .ok_or_else(|| "工具参数 offset 必须是非负整数".to_string()) + }) + .transpose()? + .unwrap_or(0); + let limit = arguments + .get("limit") + .map(|value| { + value + .as_u64() + .and_then(|value| usize::try_from(value).ok()) + .ok_or_else(|| "工具参数 limit 必须是 1 到 100 的整数".to_string()) + }) + .transpose()? + .unwrap_or(50); + if limit == 0 || limit > DIRECT_TOOL_BRIDGE_MAX_RESOURCE_PAGE_SIZE { + return Err("工具参数 limit 必须是 1 到 100 的整数".to_string()); + } + Ok((offset, limit)) +} + +fn bridge_resource_generation_input( + arguments: &Value, +) -> Result { + bridge_reject_unknown_fields( + arguments, + &["kind", "mode", "sourceLocalAssetId", "prompt", "assetName"], + )?; + let kind = DirectResourceGenerationKind::parse(&bridge_bounded_string( + arguments, + "kind", + DIRECT_TOOL_BRIDGE_MAX_RESOURCE_KIND_CHARS, + )?)?; + let mode = DirectResourceGenerationMode::parse(&bridge_bounded_string(arguments, "mode", 16)?)?; + let source_local_asset_id = bridge_optional_bounded_string( + arguments, + "sourceLocalAssetId", + DIRECT_TOOL_BRIDGE_MAX_RESOURCE_KIND_CHARS, + )?; + let prompt = bridge_bounded_string( + arguments, + "prompt", + DIRECT_TOOL_BRIDGE_MAX_RESOURCE_PROMPT_CHARS, + )?; + let asset_name = bridge_bounded_string( + arguments, + "assetName", + DIRECT_TOOL_BRIDGE_MAX_RESOURCE_NAME_CHARS, + )?; + if kind == DirectResourceGenerationKind::BackgroundMusic && prompt.chars().count() > 140 { + return Err("背景音乐提示词必须在 1..=140 字符内".to_string()); + } + match (kind, mode, source_local_asset_id.as_ref()) { + ( + DirectResourceGenerationKind::CharacterAnimation, + DirectResourceGenerationMode::Create, + _, + ) => return Err("角色动画必须基于已登记图片资源派生".to_string()), + (_, DirectResourceGenerationMode::Create, Some(_)) => { + return Err("create 模式不能携带源资源".to_string()) + } + (_, DirectResourceGenerationMode::Derive, None) => { + return Err("derive 模式必须携带 sourceLocalAssetId".to_string()) + } + _ => {} + } + Ok(DirectResourceGenerationInput { + kind, + mode, + source_local_asset_id, + prompt, + asset_name, + }) +} + +fn bridge_resource_request_fingerprint(input: &DirectResourceGenerationInput) -> String { + let value = json!({ + "kind": input.kind.as_str(), + "mode": input.mode.as_str(), + "sourceLocalAssetId": input.source_local_asset_id, + "prompt": input.prompt, + "assetName": input.asset_name, + }); + format!("{:x}", Sha256::digest(value.to_string().as_bytes())) +} + fn bridge_attempt(arguments: &Value) -> Result { let attempt = arguments .get("attempt") @@ -770,6 +1022,283 @@ fn bridge_art_resource(asset: &GameCreationAppAssetManifestEntry) -> Option Value { + let sequence_frames = include_sequence_frames.then(|| { + asset + .image_sequence_frames + .as_deref() + .unwrap_or_default() + .iter() + .map(|frame| { + json!({ + "objectKey": frame.object_key, + "assetObjectId": frame.asset_object_id, + "width": frame.width, + "height": frame.height, + }) + }) + .collect::>() + }); + json!({ + "localAssetId": asset.id, + "localPath": asset.local_path, + "kind": asset.kind, + "mediaType": asset.media_type, + "canvasProjectId": asset.source.canvas_project_id, + "resourceId": asset.source.resource_id, + "assetObjectId": asset.source.asset_object_id, + "taskId": asset.source.task_id, + "referenceResourceIds": asset.source.reference_resource_ids, + "imageSequenceDurationMs": asset.image_sequence_duration_ms, + "imageSequenceFrameCount": asset.image_sequence_frames.as_ref().map(Vec::len).unwrap_or(0), + "imageSequenceFrames": sequence_frames, + }) +} + +fn bridge_asset_canonical_resource_id(asset: &GameCreationAppAssetManifestEntry) -> String { + asset + .source + .resource_id + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(str::to_string) + .unwrap_or_else(|| format!("local-asset:{}", asset.id)) +} + +fn bridge_list_registered_assets(root: &Path, arguments: &Value) -> Value { + let result = (|| { + bridge_reject_unknown_fields( + arguments, + &[ + "kind", + "assetId", + "includeSequenceFrames", + "offset", + "limit", + ], + )?; + enforce_project_permission_policy(root, "file.list")?; + let kind = bridge_optional_bounded_string( + arguments, + "kind", + DIRECT_TOOL_BRIDGE_MAX_RESOURCE_KIND_CHARS, + )?; + let asset_id = bridge_optional_bounded_string( + arguments, + "assetId", + DIRECT_TOOL_BRIDGE_MAX_RESOURCE_KIND_CHARS, + )?; + let include_sequence_frames = arguments + .get("includeSequenceFrames") + .map(|value| { + value + .as_bool() + .ok_or_else(|| "工具参数 includeSequenceFrames 必须是布尔值".to_string()) + }) + .transpose()? + .unwrap_or(false); + let (offset, limit) = bridge_registered_asset_page(arguments)?; + let manifest = read_existing_manifest_for_project(root)?; + let mut assets = manifest + .assets + .iter() + .filter(|asset| kind.as_ref().is_none_or(|kind| asset.kind == *kind)) + .filter(|asset| { + asset_id + .as_ref() + .is_none_or(|asset_id| asset.id == *asset_id) + }) + .collect::>(); + assets.sort_by(|left, right| { + (&left.local_path, &left.id).cmp(&(&right.local_path, &right.id)) + }); + let total = assets.len(); + let resources = assets + .into_iter() + .skip(offset) + .take(limit) + .map(|asset| bridge_registered_resource(asset, include_sequence_frames)) + .collect::>(); + let next_offset = (offset + resources.len() < total).then_some(offset + resources.len()); + let pending = list_pending_local_project_resource_edits_at( + ListPendingLocalProjectResourceEditsInput { + project_path: root + .to_str() + .ok_or_else(|| "当前项目路径不能安全投影到资源查询接口".to_string())? + .to_string(), + expected_project_id: manifest.project_id, + }, + )? + .into_iter() + .map(|edit| { + json!({ + "operationId": edit.operation_id, + "kind": edit.edit_kind, + "mode": edit.generation_mode, + "sourceResourceId": edit.source_resource_id, + "assetName": edit.asset_name, + "phase": edit.phase, + "createdAt": edit.created_at, + }) + }) + .collect::>(); + Ok::<_, String>(json!({ + "status": "completed", + "total": total, + "offset": offset, + "limit": limit, + "nextOffset": next_offset, + "resources": resources, + "pendingOperations": pending, + })) + })(); + match result { + Ok(result) => bridge_tool_result(result.to_string(), Vec::new(), false), + Err(error) => bridge_tool_result( + redact_agent_runtime_error(root, &error, 480), + Vec::new(), + true, + ), + } +} + +fn bridge_completed_resource_result( + root: &Path, + kind: DirectResourceGenerationKind, + mode: DirectResourceGenerationMode, + result: DeriveLocalProjectResourceResult, +) -> Result { + let asset = result + .asset + .as_ref() + .ok_or_else(|| "媒体资源生成完成但没有登记 asset".to_string())?; + let (warnings, slice_warnings) = + local_project_resource_edit_warnings_at(root, &result.operation_id)?; + Ok(json!({ + "status": "completed", + "operationId": result.operation_id, + "kind": kind.as_str(), + "mode": mode.as_str(), + "sourceResourceId": result.source_resource_id, + "committedProjectRevision": result.committed_project_revision, + "resource": bridge_registered_resource(asset, true), + "warnings": bridge_safe_warning_messages(root, warnings), + "sliceWarnings": bridge_safe_warning_messages(root, slice_warnings), + })) +} + +async fn bridge_create_or_derive_resource( + state: &DirectToolBridgeState, + arguments: &Value, +) -> Value { + let input = match bridge_resource_generation_input(arguments) { + Ok(input) => input, + Err(error) => return bridge_tool_result(error, Vec::new(), true), + }; + let request_fingerprint = bridge_resource_request_fingerprint(&input); + let _generation_guard = state.resource_generation_gate.lock().await; + let result = async { + enforce_project_permission_policy(&state.root, "canvas.asset_generate")?; + enforce_project_permission_policy(&state.root, "asset.register")?; + let manifest = read_existing_manifest_for_project(&state.root)?; + let source_asset = input + .source_local_asset_id + .as_deref() + .map(|asset_id| { + manifest + .assets + .iter() + .find(|asset| asset.id == asset_id) + .cloned() + .ok_or_else(|| "sourceLocalAssetId 不属于当前项目已登记资源".to_string()) + }) + .transpose()?; + let prompt_sha256 = format!("{:x}", Sha256::digest(input.prompt.as_bytes())); + let pending = list_pending_local_project_resource_edits_at( + ListPendingLocalProjectResourceEditsInput { + project_path: state + .root + .to_str() + .ok_or_else(|| "当前项目路径不能安全投影到资源生成接口".to_string())? + .to_string(), + expected_project_id: manifest.project_id.clone(), + }, + )?; + let matching_pending = pending + .into_iter() + .filter(|pending| { + pending.edit_kind == input.kind.edit_kind() + && pending.generation_mode == input.mode.project_mode() + && pending.source_asset_id.as_deref() == input.source_local_asset_id.as_deref() + && pending.asset_name == input.asset_name + && pending.prompt_sha256 == prompt_sha256 + }) + .collect::>(); + if matching_pending.len() > 1 { + return Err("存在多个相同资源生成 operation，必须先在客户端完成对账".to_string()); + } + let credentials = ensure_private_external_editor_api_credentials().await?; + let completed = if let Some(pending) = matching_pending.into_iter().next() { + with_external_editor_api_credentials( + credentials, + resume_local_project_resource_edit_at(ResumeLocalProjectResourceEditInput { + project_path: state.root.to_string_lossy().into_owned(), + expected_project_id: manifest.project_id, + operation_id: pending.operation_id, + }), + ) + .await? + } else { + let (operation_id, idempotency_key) = + state.resource_request_ids(&request_fingerprint)?; + let revision = read_game_creator_agent_runtime_project_revision(&state.root)?.revision; + let source_resource_id = source_asset + .as_ref() + .map(bridge_asset_canonical_resource_id) + .unwrap_or_else(|| format!("create:{operation_id}")); + let request = DeriveLocalProjectResourceInput { + project_path: state.root.to_string_lossy().into_owned(), + expected_project_id: manifest.project_id, + expected_project_revision: revision, + operation_id, + idempotency_key, + edit_kind: input.kind.edit_kind(), + generation_mode: input.mode.project_mode(), + source_resource_id, + source_asset_id: source_asset.as_ref().map(|asset| asset.id.clone()), + source_path: source_asset.as_ref().map(|asset| asset.local_path.clone()), + source_media_type: source_asset.as_ref().map(|asset| asset.media_type.clone()), + source_subtype: source_asset.as_ref().map(|asset| asset.kind.clone()), + producer_task_id: source_asset + .as_ref() + .and_then(|asset| asset.source.task_id.clone()), + source_version_id: None, + prompt: input.prompt.clone(), + asset_name: input.asset_name.clone(), + }; + with_external_editor_api_credentials( + credentials, + derive_local_project_resource_at(request), + ) + .await? + }; + bridge_completed_resource_result(&state.root, input.kind, input.mode, completed) + } + .await; + match result { + Ok(result) => bridge_tool_result(result.to_string(), Vec::new(), false), + Err(error) => bridge_tool_result( + redact_agent_runtime_error(&state.root, &error, 480), + Vec::new(), + true, + ), + } +} + fn bridge_art_resources( root: &Path, asset_paths: &[String], @@ -1013,6 +1542,12 @@ async fn handle_direct_tool_bridge( ) -> Json { let result = match request.tool.as_str() { "taonier_prepare_game_art" => bridge_prepare_game_art(&state, &request.arguments).await, + "agc_list_registered_assets" => { + bridge_list_registered_assets(&state.root, &request.arguments) + } + "agc_create_or_derive_resource" => { + bridge_create_or_derive_resource(&state, &request.arguments).await + } "agc_browser_playtest" => bridge_browser_playtest(&state.root, &request.arguments).await, "agc_web_search" => bridge_web_search(&state.root, &request.arguments).await, _ => bridge_tool_result("未知或未审核的客户端工具".to_string(), Vec::new(), true), @@ -1082,6 +1617,40 @@ mod tests { assert!(bridge_search_max_results(&json!({ "maxResults": 0 })).is_err()); assert!(bridge_search_max_results(&json!({ "maxResults": 6 })).is_err()); assert!(bridge_search_max_results(&json!({ "maxResults": "3" })).is_err()); + assert!(bridge_resource_generation_input(&json!({ + "kind": "video", + "mode": "create", + "prompt": "生成过场", + "assetName": "开场" + })) + .is_ok()); + assert!(bridge_resource_generation_input(&json!({ + "kind": "character-animation", + "mode": "create", + "prompt": "待机", + "assetName": "待机" + })) + .is_err()); + assert!(bridge_resource_generation_input(&json!({ + "kind": "video", + "mode": "derive", + "prompt": "调整", + "assetName": "调整版" + })) + .is_err()); + } + + #[test] + fn resource_request_uuid_is_stable_v4_and_domain_separated() { + let operation = direct_resource_request_uuid("turn-1", "operation", "abc"); + let replay = direct_resource_request_uuid("turn-1", "operation", "abc"); + let idempotency = direct_resource_request_uuid("turn-1", "idempotency", "abc"); + assert_eq!(operation, replay); + assert_ne!(operation, idempotency); + assert_eq!( + uuid::Uuid::parse_str(&operation).unwrap().get_version_num(), + 4 + ); } #[test] @@ -1343,6 +1912,49 @@ mod tests { assert!(!serialized.contains("secret")); } + #[test] + fn bridge_registered_resource_exposes_formal_sequence_without_private_generation_fields() { + let asset = GameCreationAppAssetManifestEntry { + id: "animation-1".to_string(), + kind: "character-animation".to_string(), + media_type: "video/mp4".to_string(), + local_path: "assets/edits/animation.mp4".to_string(), + image_sequence_frames: Some(vec![ + shared_contracts::game_creation_app::GameCreationAppImageSequenceFrame { + image_src: "https://signed.invalid/private".to_string(), + object_key: Some("animations/frame-01.png".to_string()), + asset_object_id: Some("frame-object-1".to_string()), + width: 720, + height: 1280, + }, + ]), + image_sequence_duration_ms: Some(4_000), + source: GameCreationAppAssetSource { + kind: GameCreationAppAssetSourceKind::Canvas, + canvas_project_id: Some("canvas-1".to_string()), + resource_id: Some("resource-animation-1".to_string()), + asset_object_id: Some("object-animation-1".to_string()), + task_id: Some("task-animation-1".to_string()), + prompt: Some("private prompt".to_string()), + model: Some("private model".to_string()), + generation_route: Some("https://private.invalid/generate".to_string()), + generation_kind: Some("character-animation".to_string()), + reference_resource_ids: vec!["source-character-1".to_string()], + }, + }; + let projection = bridge_registered_resource(&asset, true); + assert_eq!(projection["imageSequenceFrameCount"], 1); + assert_eq!( + projection["imageSequenceFrames"][0]["objectKey"], + "animations/frame-01.png" + ); + let serialized = projection.to_string(); + assert!(!serialized.contains("signed.invalid")); + assert!(!serialized.contains("private prompt")); + assert!(!serialized.contains("private model")); + assert!(!serialized.contains("private.invalid")); + } + #[test] fn bridge_success_warnings_are_redacted_before_serialization() { let root = tempfile::tempdir().expect("warning redaction root"); diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs index 1f3ec8560..c825cc377 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs @@ -7,6 +7,8 @@ pub(crate) const DIRECT_TOOLS_MCP_MODE_FLAG: &str = "--agc-direct-tools-mcp"; const DIRECT_TOOLS_MCP_MAX_REQUEST_BYTES: usize = 1024 * 1024; const DIRECT_TOOLS_MCP_MAX_ART_BRIEF_CHARS: usize = 4_000; const DIRECT_TOOLS_MCP_MAX_SEARCH_QUERY_CHARS: usize = 400; +const DIRECT_TOOLS_MCP_MAX_RESOURCE_PROMPT_CHARS: usize = 4_000; +const DIRECT_TOOLS_MCP_MAX_RESOURCE_NAME_CHARS: usize = 120; const DIRECT_TOOLS_MCP_MAX_BRIDGE_RESPONSE_BYTES: usize = 32 * 1024 * 1024; pub(crate) const DIRECT_TOOLS_MCP_CONTROLLED_WEB_SEARCH_ENV: &str = "AGC_CONTROLLED_WEB_SEARCH_ENABLED"; @@ -82,6 +84,80 @@ fn direct_tools_mcp_specs_for(controlled_web_search: bool) -> Value { "additionalProperties": false } }), + json!({ + "name": "agc_list_registered_assets", + "description": "查询当前项目由客户端权威 manifest 登记的资源与未完成资源 operation。结果有界且只包含项目相对路径、稳定资源身份、序列帧身份和恢复状态，不返回 prompt、模型、签名 URL、宿主路径或凭据。", + "inputSchema": { + "type": "object", + "properties": { + "kind": { + "type": "string", + "minLength": 1, + "maxLength": 80, + "description": "可选的 manifest 资源 kind 精确过滤，例如 video、character-animation、sound-effect、background-music 或 art-spritesheet-slice" + }, + "assetId": { + "type": "string", + "minLength": 1, + "maxLength": 80, + "description": "可选的本地 manifest asset ID 精确过滤" + }, + "includeSequenceFrames": { + "type": "boolean", + "default": false, + "description": "是否返回角色动画各帧的稳定 objectKey/assetObjectId 与尺寸；不返回签名 URL" + }, + "offset": { "type": "integer", "minimum": 0, "default": 0 }, + "limit": { "type": "integer", "minimum": 1, "maximum": 100, "default": 50 } + }, + "additionalProperties": false + } + }), + json!({ + "name": "agc_create_or_derive_resource", + "description": "按用户当前意图创建或派生视频、角色动画、音效或背景音乐。模型只表达资源语义；客户端掌管项目路径、来源解析、权限、revision、项目锁、幂等键、operation 恢复、付费提交、下载校验和 manifest 事务。相同未完成请求会优先恢复，不能用它绕过账本重发付费请求。", + "inputSchema": { + "type": "object", + "properties": { + "kind": { + "type": "string", + "enum": ["video", "character-animation", "sound-effect", "background-music"] + }, + "mode": { "type": "string", "enum": ["create", "derive"] }, + "sourceLocalAssetId": { + "type": "string", + "minLength": 1, + "maxLength": 80, + "description": "derive 时必须使用 agc_list_registered_assets 返回的当前项目 localAssetId；禁止传路径、URL、objectKey 或平台凭据" + }, + "prompt": { + "type": "string", + "minLength": 1, + "maxLength": DIRECT_TOOLS_MCP_MAX_RESOURCE_PROMPT_CHARS + }, + "assetName": { + "type": "string", + "minLength": 1, + "maxLength": DIRECT_TOOLS_MCP_MAX_RESOURCE_NAME_CHARS + } + }, + "required": ["kind", "mode", "prompt", "assetName"], + "oneOf": [ + { + "properties": { + "mode": { "const": "create" }, + "kind": { "enum": ["video", "sound-effect", "background-music"] } + }, + "not": { "required": ["sourceLocalAssetId"] } + }, + { + "properties": { "mode": { "const": "derive" } }, + "required": ["sourceLocalAssetId"] + } + ], + "additionalProperties": false + } + }), json!({ "name": "agc_browser_playtest", "description": "使用当前客户端的受限 Chromium 对当前游戏执行真实 desktop/mobile 双视口运行、截图、控制台、网络、Canvas/WebGL 和有限交互探针。", @@ -205,6 +281,105 @@ fn bounded_tool_string(arguments: &Value, field: &str, max_chars: usize) -> Resu Ok(value.to_string()) } +fn validate_tool_object_fields(arguments: &Value, allowed: &[&str]) -> Result<(), String> { + let object = arguments + .as_object() + .ok_or_else(|| "工具参数必须是对象".to_string())?; + if let Some(field) = object + .keys() + .find(|field| !allowed.contains(&field.as_str())) + { + return Err(format!("工具参数包含未审核字段：{field}")); + } + Ok(()) +} + +fn validate_registered_assets_arguments(arguments: &Value) -> Result<(), String> { + validate_tool_object_fields( + arguments, + &[ + "kind", + "assetId", + "includeSequenceFrames", + "offset", + "limit", + ], + )?; + for field in ["kind", "assetId"] { + if arguments.get(field).is_some() { + bounded_tool_string(arguments, field, 80)?; + } + } + if arguments + .get("includeSequenceFrames") + .is_some_and(|value| !value.is_boolean()) + { + return Err("工具参数 includeSequenceFrames 必须是布尔值".to_string()); + } + if arguments + .get("offset") + .is_some_and(|value| value.as_u64().is_none()) + { + return Err("工具参数 offset 必须是非负整数".to_string()); + } + if let Some(limit) = arguments.get("limit") { + let limit = limit + .as_u64() + .ok_or_else(|| "工具参数 limit 必须是 1 到 100 的整数".to_string())?; + if !(1..=100).contains(&limit) { + return Err("工具参数 limit 必须是 1 到 100 的整数".to_string()); + } + } + Ok(()) +} + +fn validate_resource_generation_arguments(arguments: &Value) -> Result<(), String> { + validate_tool_object_fields( + arguments, + &["kind", "mode", "sourceLocalAssetId", "prompt", "assetName"], + )?; + let kind = bounded_tool_string(arguments, "kind", 80)?; + if ![ + "video", + "character-animation", + "sound-effect", + "background-music", + ] + .contains(&kind.as_str()) + { + return Err("工具参数 kind 不是受支持的媒体资源类型".to_string()); + } + let mode = bounded_tool_string(arguments, "mode", 16)?; + if !["create", "derive"].contains(&mode.as_str()) { + return Err("工具参数 mode 必须是 create 或 derive".to_string()); + } + let prompt = bounded_tool_string( + arguments, + "prompt", + DIRECT_TOOLS_MCP_MAX_RESOURCE_PROMPT_CHARS, + )?; + bounded_tool_string( + arguments, + "assetName", + DIRECT_TOOLS_MCP_MAX_RESOURCE_NAME_CHARS, + )?; + let source = arguments.get("sourceLocalAssetId"); + if source.is_some() { + bounded_tool_string(arguments, "sourceLocalAssetId", 80)?; + } + if kind == "background-music" && prompt.chars().count() > 140 { + return Err("背景音乐提示词必须在 1..=140 字符内".to_string()); + } + if kind == "character-animation" && mode == "create" { + return Err("角色动画必须基于已登记图片资源派生".to_string()); + } + match (mode.as_str(), source.is_some()) { + ("create", true) => Err("create 模式不能携带源资源".to_string()), + ("derive", false) => Err("derive 模式必须携带 sourceLocalAssetId".to_string()), + _ => Ok(()), + } +} + fn tool_attempt(arguments: &Value) -> Result { let attempt = arguments .get("attempt") @@ -329,6 +504,20 @@ async fn call_taonier_prepare_game_art(arguments: &Value) -> Value { call_client_tool_bridge("taonier_prepare_game_art", arguments).await } +async fn call_agc_list_registered_assets(arguments: &Value) -> Value { + if let Err(error) = validate_registered_assets_arguments(arguments) { + return mcp_tool_result(error, Vec::new(), true); + } + call_client_tool_bridge("agc_list_registered_assets", arguments).await +} + +async fn call_agc_create_or_derive_resource(arguments: &Value) -> Value { + if let Err(error) = validate_resource_generation_arguments(arguments) { + return mcp_tool_result(error, Vec::new(), true); + } + call_client_tool_bridge("agc_create_or_derive_resource", arguments).await +} + async fn call_agc_browser_playtest(arguments: &Value) -> Value { if let Err(error) = tool_attempt(arguments) { return mcp_tool_result(error, Vec::new(), true); @@ -396,6 +585,10 @@ async fn handle_direct_tools_mcp_request(_root: &Path, request: Value) -> Option let result = match tool { "agc_read_skill_resource" => call_agc_read_skill_resource(&arguments), "taonier_prepare_game_art" => call_taonier_prepare_game_art(&arguments).await, + "agc_list_registered_assets" => call_agc_list_registered_assets(&arguments).await, + "agc_create_or_derive_resource" => { + call_agc_create_or_derive_resource(&arguments).await + } "agc_browser_playtest" => call_agc_browser_playtest(&arguments).await, "agc_web_search" => call_agc_web_search(&arguments).await, _ => mcp_tool_result("未知或未审核的 AGC 工具".to_string(), Vec::new(), true), @@ -518,6 +711,8 @@ mod tests { vec![ "agc_read_skill_resource", "taonier_prepare_game_art", + "agc_list_registered_assets", + "agc_create_or_derive_resource", "agc_browser_playtest" ] ); @@ -570,6 +765,8 @@ mod tests { vec![ "agc_read_skill_resource", "taonier_prepare_game_art", + "agc_list_registered_assets", + "agc_create_or_derive_resource", "agc_browser_playtest", "agc_web_search" ] @@ -577,6 +774,67 @@ mod tests { assert!(!specs.to_string().contains("apiKey")); } + #[test] + fn semantic_resource_tools_reject_unreviewed_or_inconsistent_arguments() { + assert!(validate_registered_assets_arguments(&json!({ + "kind": "character-animation", + "includeSequenceFrames": true, + "offset": 0, + "limit": 100 + })) + .is_ok()); + assert!(validate_registered_assets_arguments(&json!({ "limit": 101 })).is_err()); + assert!( + validate_registered_assets_arguments(&json!({ "projectPath": "/private" })).is_err() + ); + + assert!(validate_resource_generation_arguments(&json!({ + "kind": "video", + "mode": "create", + "prompt": "生成森林过场", + "assetName": "森林过场" + })) + .is_ok()); + assert!(validate_resource_generation_arguments(&json!({ + "kind": "character-animation", + "mode": "derive", + "sourceLocalAssetId": "hero", + "prompt": "待机呼吸", + "assetName": "角色待机" + })) + .is_ok()); + for malformed in [ + json!({ + "kind": "character-animation", + "mode": "create", + "prompt": "待机呼吸", + "assetName": "角色待机" + }), + json!({ + "kind": "video", + "mode": "derive", + "prompt": "调整节奏", + "assetName": "新视频" + }), + json!({ + "kind": "video", + "mode": "create", + "sourceLocalAssetId": "old-video", + "prompt": "生成视频", + "assetName": "新视频" + }), + json!({ + "kind": "video", + "mode": "create", + "prompt": "生成视频", + "assetName": "新视频", + "operationId": "model-owned" + }), + ] { + assert!(validate_resource_generation_arguments(&malformed).is_err()); + } + } + #[test] fn controlled_search_tool_rejects_malformed_result_bounds() { assert_eq!(tool_search_max_results(&json!({})).expect("default"), 3); diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent/skill_pack.rs b/apps/ai-game-creator-shell/src-tauri/src/agent/skill_pack.rs index a7a9f5b80..395718231 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent/skill_pack.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent/skill_pack.rs @@ -362,6 +362,8 @@ mod tests { let index = render_agc_skill_pack_index().expect("render index"); assert!(index.contains("taonier-art-assets")); assert!(index.contains("agc_tools.taonier_prepare_game_art")); + assert!(index.contains("agc_tools.agc_list_registered_assets")); + assert!(index.contains("agc_tools.agc_create_or_derive_resource")); assert!(!index.contains("Use real platform assets only")); assert!(!index.contains("postprocess-failed-source-preserved")); } diff --git a/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs b/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs index 6b5fe501c..35cbc0294 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs @@ -182,8 +182,11 @@ pub(crate) struct ListPendingLocalProjectResourceEditsInput { pub(crate) struct PendingLocalProjectResourceEdit { pub(crate) operation_id: String, pub(crate) edit_kind: LocalProjectResourceEditKind, + pub(crate) generation_mode: LocalProjectResourceGenerationMode, pub(crate) source_resource_id: String, + pub(crate) source_asset_id: Option, pub(crate) asset_name: String, + pub(crate) prompt_sha256: String, pub(crate) phase: String, pub(crate) created_at: u64, } @@ -366,6 +369,12 @@ struct ResourceEditLedger { remote_sequence_frames_json: Option, #[serde(default)] remote_sequence_duration_ms: Option, + #[serde(default)] + remote_canvas_project_id: Option, + #[serde(default)] + remote_warnings: Vec, + #[serde(default)] + remote_slice_warnings: Vec, remote_asset_object_id: Option, remote_model: Option, #[serde(default)] @@ -1949,9 +1958,8 @@ fn resource_edit_remote_request( "generationInputs": generation_inputs, }), )), - LocalProjectResourceEditKind::CharacterAnimation => Ok(( - "/api/external/v1/editor/character-animations/generations", - serde_json::json!({ + LocalProjectResourceEditKind::CharacterAnimation => { + let mut body = serde_json::json!({ "sourceLayerId": format!("resource-{}", input.operation_id), "sourceImageSrc": source_reference.ok_or_else(|| "角色动画缺少稳定源图片引用".to_string())?, "sourceWidth": source.source_width.ok_or_else(|| "角色动画缺少源图片宽度".to_string())?, @@ -1964,8 +1972,33 @@ fn resource_edit_remote_request( "model": "seedance2.0-fast", "assetLabel": asset_name, "generationInputs": generation_inputs, - }), - )), + }); + if let Some(context) = canvas_context { + let width = source + .source_width + .ok_or_else(|| "角色动画缺少源图片宽度".to_string())?; + let height = source + .source_height + .ok_or_else(|| "角色动画缺少源图片高度".to_string())?; + body["projectId"] = serde_json::json!(context.project_id); + body["assetFolderId"] = serde_json::json!(context.asset_folder_id); + body["canvasCompletion"] = serde_json::json!({ + "title": asset_name, + "placeholder": { + "x": 0, + "y": 0, + "width": width, + "height": height, + "originalWidth": width, + "originalHeight": height, + }, + }); + } + Ok(( + "/api/external/v1/editor/character-animations/generations", + body, + )) + } LocalProjectResourceEditKind::Video => { let mut body = serde_json::json!({ "prompt": prompt, @@ -2216,11 +2249,17 @@ async fn wait_for_resource_edit_remote( let job = platform_generation_status_data(&payload); match json_string_field(job, "status").as_deref() { Some("completed") => { - let result = job + let mut result = job .get("result") .filter(|value| !value.is_null()) .cloned() .ok_or_else(|| "result-unknown: 资源编辑任务完成但缺少 result".to_string())?; + if let (Some(result), Some(warning)) = ( + result.as_object_mut(), + job.get("warning").filter(|value| !value.is_null()), + ) { + result.insert("_queryWarning".to_string(), warning.clone()); + } if !resource_edit_result_has_download(&result) { return Err("result-unknown: 资源编辑结果缺少可下载媒体".to_string()); } @@ -2247,10 +2286,33 @@ struct ResourceEditRemoteIdentity { sequence_frames: Option>, sequence_duration_ms: Option, + canvas_project_id: Option, + warnings: Vec, + slice_warnings: Vec, asset_object_id: Option, model: Option, } +fn resource_edit_warning_text(value: &serde_json::Value) -> Option { + if let Some(value) = value + .as_str() + .map(str::trim) + .filter(|value| !value.is_empty()) + { + return Some(value.to_string()); + } + let code = json_string_field(value, "code"); + let message = json_string_field(value, "message") + .or_else(|| json_string_field(value, "reason")) + .or_else(|| json_string_field(value, "detail")); + match (code, message) { + (Some(code), Some(message)) => Some(format!("{code}: {message}")), + (Some(code), None) => Some(code), + (None, Some(message)) => Some(message), + (None, None) => None, + } +} + fn extract_resource_edit_remote_identity( generated: &serde_json::Value, edit_kind: LocalProjectResourceEditKind, @@ -2314,6 +2376,18 @@ fn extract_resource_edit_remote_identity( } let resource_id = json_string_field(resource, "resourceId").or_else(|| json_string_field(data, "resourceId")); + let canvas_project_id = + json_string_field(resource, "projectId").or_else(|| json_string_field(data, "projectId")); + let warnings = [generated.get("_queryWarning"), data.get("warning")] + .into_iter() + .flatten() + .filter_map(resource_edit_warning_text) + .collect(); + let slice_warnings = data + .get("sliceWarning") + .and_then(resource_edit_warning_text) + .into_iter() + .collect(); let asset_object_id = json_string_field(resource, "assetObjectId") .or_else(|| json_string_field(asset, "assetObjectId")) .or_else(|| json_string_field(data, "assetObjectId")); @@ -2324,6 +2398,9 @@ fn extract_resource_edit_remote_identity( legacy_public_path, sequence_frames, sequence_duration_ms, + canvas_project_id, + warnings, + slice_warnings, asset_object_id, model, }) @@ -2650,20 +2727,25 @@ async fn prepare_remote_resource_edit( write_resource_edit_ledger(root, ledger)?; } if ledger.endpoint.is_none() || ledger.request_body_json.is_none() { - let canvas_context = - if input.generation_mode == LocalProjectResourceGenerationMode::Create { - Some( - prepare_external_canvas_generation_context( - root, - &client, - &api_base_url, - &api_key, - ) - .await?, + let canvas_context = if matches!( + input.edit_kind, + LocalProjectResourceEditKind::CharacterAnimation + | LocalProjectResourceEditKind::Video + | LocalProjectResourceEditKind::SoundEffect + | LocalProjectResourceEditKind::BackgroundMusic + ) { + Some( + prepare_external_canvas_generation_context( + root, + &client, + &api_base_url, + &api_key, ) - } else { - None - }; + .await?, + ) + } else { + None + }; let (endpoint, body) = resource_edit_remote_request( input, source, @@ -2706,6 +2788,15 @@ async fn prepare_remote_resource_edit( }) .transpose()?; ledger.remote_sequence_duration_ms = identity.sequence_duration_ms; + ledger.remote_canvas_project_id = identity.canvas_project_id.or_else(|| { + ledger + .request_body_json + .as_deref() + .and_then(|body| serde_json::from_str::(body).ok()) + .and_then(|body| json_string_field(&body, "projectId")) + }); + ledger.remote_warnings = identity.warnings; + ledger.remote_slice_warnings = identity.slice_warnings; ledger.remote_asset_object_id = identity.asset_object_id; ledger.remote_model = identity.model; update_resource_edit_phase(root, ledger, ResourceEditLedgerPhase::RemoteCompleted)?; @@ -3427,7 +3518,7 @@ fn commit_resource_edit_asset_internal( } else { GameCreationAppAssetSourceKind::Generated }, - canvas_project_id: None, + canvas_project_id: ledger.remote_canvas_project_id.clone(), resource_id: ledger .remote_resource_id .clone() @@ -4077,8 +4168,11 @@ pub(crate) fn list_pending_local_project_resource_edits_at( pending.push(PendingLocalProjectResourceEdit { operation_id: ledger.operation_id, edit_kind: ledger.edit_kind, + generation_mode: ledger.generation_mode, source_resource_id: ledger.source_resource_id, + source_asset_id: ledger.source_asset_id, asset_name: ledger.asset_name, + prompt_sha256: sha256_hex(ledger.prompt.as_bytes()), phase: ledger.phase.as_str().to_string(), created_at: ledger.created_at, }); @@ -4088,6 +4182,17 @@ pub(crate) fn list_pending_local_project_resource_edits_at( Ok(pending) } +pub(crate) fn local_project_resource_edit_warnings_at( + root: &Path, + operation_id: &str, +) -> Result<(Vec, Vec), String> { + validate_project_root(root)?; + validate_resource_edit_uuid(operation_id, "operationId")?; + let ledger = read_resource_edit_ledger(root, operation_id)? + .ok_or_else(|| "资源编辑告警对应的 operation 不存在".to_string())?; + Ok((ledger.remote_warnings, ledger.remote_slice_warnings)) +} + pub(crate) async fn request_resource_edit_service_identity_confirmation_at( input: RequestResourceEditServiceIdentityConfirmationInput, ) -> Result { @@ -4410,6 +4515,9 @@ pub(crate) async fn derive_local_project_resource_at( remote_legacy_public_path: None, remote_sequence_frames_json: None, remote_sequence_duration_ms: None, + remote_canvas_project_id: None, + remote_warnings: Vec::new(), + remote_slice_warnings: Vec::new(), remote_asset_object_id: None, remote_model: None, terminal_failure_code: None, @@ -4698,6 +4806,9 @@ mod tests { remote_legacy_public_path: None, remote_sequence_frames_json: None, remote_sequence_duration_ms: None, + remote_canvas_project_id: None, + remote_warnings: Vec::new(), + remote_slice_warnings: Vec::new(), remote_asset_object_id: None, remote_model: None, terminal_failure_code: None, @@ -5564,7 +5675,7 @@ mod tests { } #[tokio::test] - async fn submission_transport_failure_requires_reconciliation_before_resume() { + async fn canvas_context_transport_failure_stays_prepared_before_resume() { let directory = tempfile::tempdir().expect("create transport failure fixture"); let root = directory.path(); init_local_game_project_at(root, PROJECT_ID, "远端提交传输失败测试") @@ -5605,14 +5716,12 @@ mod tests { }) .await .expect_err("transport failure must fail closed"); - assert!(error.contains("result-unknown"), "{error}"); - assert_eq!( - read_resource_edit_ledger(root, &request.operation_id) - .expect("read transport failure ledger") - .expect("persisted transport failure ledger") - .phase, - ResourceEditLedgerPhase::ReconciliationRequired - ); + assert!(error.contains("读取外部画布项目"), "{error}"); + let persisted = read_resource_edit_ledger(root, &request.operation_id) + .expect("read transport failure ledger") + .expect("persisted transport failure ledger"); + assert_eq!(persisted.phase, ResourceEditLedgerPhase::Prepared); + assert_eq!(persisted.remote_operation_id, None); let resume_error = resume_local_project_resource_edit_at(ResumeLocalProjectResourceEditInput { @@ -5623,7 +5732,7 @@ mod tests { .await .expect_err("transport failure must stop resume before network access"); assert!( - resume_error.contains("reconciliation-required"), + resume_error.contains("authentication-required"), "{resume_error}" ); } @@ -5965,7 +6074,7 @@ mod tests { "让角色自然呼吸", "角色动画", Some("stable-image-object-key"), - None, + Some(&canvas_context), ) .expect("build character animation request"); assert_eq!( @@ -5978,6 +6087,23 @@ mod tests { ); assert_eq!(animation_body["sourceWidth"], serde_json::json!(720)); assert_eq!(animation_body["sourceHeight"], serde_json::json!(1280)); + assert_eq!(animation_body["projectId"], serde_json::json!("project-1")); + assert_eq!( + animation_body["assetFolderId"], + serde_json::json!("folder-1") + ); + assert_eq!( + animation_body["canvasCompletion"]["title"], + serde_json::json!("角色动画") + ); + assert_eq!( + animation_body["canvasCompletion"]["placeholder"]["width"], + serde_json::json!(720) + ); + assert_eq!( + animation_body["canvasCompletion"]["placeholder"]["height"], + serde_json::json!(1280) + ); let generated = serde_json::json!({ "data": { @@ -5995,7 +6121,16 @@ mod tests { ], "resource": { "resourceId": "editor-resource-animation", + "projectId": "project-1", "objectKey": "generated-character-drafts/layer/animation/frame-01.png" + }, + "warning": { + "code": "animation-normalized", + "message": "序列帧尺寸已归一化" + }, + "sliceWarning": { + "code": "partial-preview", + "reason": "预览仅展示首段" } } }); @@ -6013,6 +6148,15 @@ mod tests { Some("generated-character-drafts/layer/animation/frame-01.png") ); assert_eq!(identity.sequence_duration_ms, Some(4_000)); + assert_eq!(identity.canvas_project_id.as_deref(), Some("project-1")); + assert_eq!( + identity.warnings, + vec!["animation-normalized: 序列帧尺寸已归一化"] + ); + assert_eq!( + identity.slice_warnings, + vec!["partial-preview: 预览仅展示首段"] + ); assert_eq!(identity.sequence_frames.as_ref().map(Vec::len), Some(1)); assert_eq!( identity @@ -6058,7 +6202,7 @@ mod tests { let server_generated_video = generated_video.clone(); let (sender, receiver) = mpsc::channel(); let server = std::thread::spawn(move || { - for request_index in 0..7 { + for request_index in 0..9 { let mut stream = accept_resource_editor_fixture_connection( &listener, "External video fixture", @@ -6070,7 +6214,27 @@ mod tests { .expect("capture External video request"); let request_line = request.lines().next().unwrap_or_default(); let request_lower = request.to_ascii_lowercase(); - if request_line.starts_with("POST /api/external/v1/assets/direct-upload-tickets ") { + if request_line == "GET /api/external/v1/editor/projects HTTP/1.1" { + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": {"projects": [{ + "projectId": "canvas-project-video", + "title": "External 视频编辑测试" + }]}}), + ); + } else if request_line == "GET /api/external/v1/editor/assets/library HTTP/1.1" { + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": {"library": {"folders": [{ + "folderId": "canvas-folder-video", + "label": "External 视频编辑测试" + }]}}}), + ); + } else if request_line + .starts_with("POST /api/external/v1/assets/direct-upload-tickets ") + { assert!(request_lower .contains("authorization: bearer resource-editor-external-key")); write_json( @@ -6109,6 +6273,9 @@ mod tests { .contains("authorization: bearer resource-editor-external-key")); assert!(request_lower.contains("idempotency-key:")); assert!(request.contains(&server_source_key)); + assert!(request.contains("canvas-project-video")); + assert!(request.contains("canvas-folder-video")); + assert!(request.contains("canvasCompletion")); write_json( &mut stream, "202 Accepted", @@ -6130,12 +6297,14 @@ mod tests { "operationId": "external-video-operation", "status": "completed", "result": { - "resource": { + "resource": { "resourceId": "external-video-resource", "objectKey": "generated/result-video.mp4", "assetObjectId": "generated-video-object" - }, - "model": "seedance2.0-fast" + }, + "warning": "视频已登记到画布", + "sliceWarning": "视频切片将在后台完成", + "model": "seedance2.0-fast" } }}), ); @@ -6175,13 +6344,27 @@ mod tests { .expect("derive External video"); server.join().expect("join External video server"); let requests = std::iter::from_fn(|| receiver.try_recv().ok()).collect::>(); - assert_eq!(requests.len(), 7); + assert_eq!(requests.len(), 9); assert!(requests.iter().all(|request| { !request.starts_with("POST /api/editor/") && !request.starts_with("POST /api/assets/") && !request.starts_with("GET /api/runtime/external-generation/") })); let derivative = result.asset.expect("derived video asset"); + let (warnings, slice_warnings) = + local_project_resource_edit_warnings_at(root, &request.operation_id) + .expect("read persisted canvas warnings"); + assert!(warnings.iter().any(|warning| warning == "视频已登记到画布")); + assert!(slice_warnings + .iter() + .any(|warning| warning == "视频切片将在后台完成")); + let persisted = read_resource_edit_ledger(root, &request.operation_id) + .expect("read completed video ledger") + .expect("completed video ledger"); + assert_eq!( + persisted.remote_canvas_project_id.as_deref(), + Some("canvas-project-video") + ); assert_ne!(derivative.id, source_asset.id); assert_eq!( fs::read(root.join(&source_asset.local_path)).expect("read preserved source video"), diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index c3f1c69c2..69740348a 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -1,5 +1,12 @@ # 决策记录 +## 2026-08-24 AGC Direct 媒体能力只通过客户端语义工具开放 + +- 背景：资源页已经补齐视频、角色动画、音效和背景音乐的 create/derive 能力，但 Direct Codex 只能准备标准美术包，无法查询已登记源资源或表达新增媒体意图。直接开放 Tauri invoke 会把项目路径、revision、operation、幂等键、登录态和事务权力交给模型。 +- 决策：只新增 `agc_list_registered_assets` 与 `agc_create_or_derive_resource` 两个语义工具。Codex 只能提交资源过滤条件或 kind/mode/localAssetId/prompt/name；客户端权威解析 manifest 来源，生成并恢复稳定 operation/idempotency，串行付费调用，执行权限、项目锁、画布/素材目录准备、下载校验和 manifest CAS。完全匹配的 pending 请求自动恢复，不创建替代付费请求。 +- 输出边界：资源查询和生成结果只投影相对路径、稳定 Canvas/resource/asset/task 身份、序列帧身份、pending 状态及脱敏告警；不返回完整 manifest、prompt、model、provider route、绝对路径、URL、Token、Cookie 或 API Key。角色动画及视频/音频新请求统一携带同名画布与素材目录上下文；已有冻结请求不迁移、不重写。 +- 关联文档：`docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`、`apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md`。 + ## 2026-08-20 UI Editor LLM 递归输出与参考图单文件限制 - 背景：结构识别、界面语义建议和多图合并直接把 LLM 工具 arguments 反序列化为递归树；结构识别与语义建议还在 async command 中同步读取并 base64 编码参考图。模型异常输出或过大图片可能造成不受控内存、栈和 async worker 占用。 diff --git a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md index 4878c451c..e0093ad6b 100644 --- a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md +++ b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md @@ -1,5 +1,12 @@ # AI 游戏创作智能体 App 实施计划 +## 2026-08-24 Direct Codex 已登记资源查询与媒体生成语义工具 + +- `agc_tools` 新增 `agc_list_registered_assets` 与 `agc_create_or_derive_resource`。前者按 `kind / assetId / offset / limit` 有界查询客户端权威 manifest，并可显式返回角色动画正式序列帧的稳定 objectKey、assetObjectId 和尺寸；结果不包含完整 manifest、prompt、model、provider route、签名 URL、宿主路径或凭据。后者只接受 `kind / mode / sourceLocalAssetId / prompt / assetName`，`create` 仅允许无源视频、音效和背景音乐，`derive` 必须引用当前项目已登记的 localAssetId，角色动画固定为 derive。 +- 项目路径、projectId、当前 revision、源文件路径与媒体类型、operationId、Idempotency-Key、登录态、项目锁、付费提交、轮询恢复、下载校验与 manifest 事务全部由客户端持有。模型不能提交或覆盖这些字段。同一 Direct `clientTurnId + 规范语义参数` 生成稳定 UUID v4 身份；单回合同参重试复用原 operation，不同请求串行且最多四项。跨回合存在完全匹配的 pending 账本时优先恢复原 operation，不能换键重发。 +- 资源查询同时投影未完成 operation 的安全状态。媒体工具成功只返回 operation、本地相对路径、资源类型、Canvas/resource/asset/task 身份、正式序列帧以及脱敏后的 `warnings / sliceWarnings`；错误继续使用统一脱敏边界。客户端资源账本持久化 completed 结果的两类告警，committed replay 不能把历史告警伪装成空集合。 +- 角色动画、视频、音效和背景音乐在构造新的远端请求前统一准备当前项目同名画布与素材目录上下文，并在端点支持时携带 `projectId / assetFolderId / canvasCompletion`。角色动画 placeholder 使用源图片真实宽高，避免非方形角色进入画布时失真；正式 resource/asset 与序列帧继续直接复用 External 返回身份，不从首帧伪造重复资源。已有冻结 request body 或已受理 operation 保持不变，不因本次升级重建请求或重复扣费。 + ## 2026-08-23 AGC 资源生成补齐（视频 / 动画 / 音效 / 背景音乐） - 统一资源编辑命令新增 `generationMode=create|derive`：`create` 用于无源生成视频、音效和背景音乐，`derive` 保持现有“基于已有资源派生”。视频新建请求不再携带 `referenceVideoSrcs`；音效 / 背景音乐新建使用同款文本生成端点，但提示词前缀改为“生成新音频”。 @@ -1176,7 +1183,7 @@ game-project/ - 普通项目对话只由一个 project-bound Codex app-server thread 执行。客户端系统提示词只放最小工程合同、当前游戏源码有界快照、项目 prompts 和审核 Skill 索引；不再批量读取项目 `.codex/.agents/.hermes` Skill 正文，也不恢复 Supervisor、专业 Agent 或 harness。 - 首页恢复“做游戏 / 做素材 / 做方案”三个创作类型，默认“做游戏”。该选择与设置页的 Agent Runtime 模式无关；每次首页提交仍只自动创建一个新项目并进入项目工作台。用户正文原样进入项目对话，`game|art|doc` 仅作为受限结构化首轮上下文传给同一 Codex thread，不拼接“初始意图”文案、不产生首页对话、不切换 Provider 或恢复旧 Runtime 编排。 - `agc-skill-pack.v1` 只包含项目结构、陶泥儿美术、Web 游戏实现、真实浏览器试玩、客户端资源投影五项 Skill。清单记录用途、触发条件、所需工具、版本和内容 SHA-256；审核文本按 UTF-8 读取并将 CRLF 规范为 LF 后计算指纹和安装，避免混合换行造成 Windows / Linux 构建结果漂移，语义内容变化时必须同步重算对应清单指纹并提升版本。客户端把审核文件安装到隔离目录后通过 app-server `skills/extraRoots/set + skills/list` 注册并复核，完整正文由 Codex 原生 Skill 机制按意图加载，一层引用只能经 `agc_read_skill_resource` 读取清单内 Markdown。引用路径按平台无关规则拒绝反斜杠、盘符、UNC、绝对路径和 `..`，不能依赖当前宿主的 `std::path` 语义判断其它平台路径。 -- DirectProject 只连接客户端内置的 `agc_tools` STDIO MCP，基础工具固定为审核引用读取、标准陶泥儿美术准备和 desktop/mobile 浏览器试玩；`webSearchEnabled=true` 时才追加受控联网搜索。MCP 进程只做协议；真实浏览器、付费 External v1 调用和受控搜索通过随机 loopback 地址回到客户端主进程，因此不复制 GUI 登录态、开发者 Key 或项目路径到模型上下文。已登记工具固定自动批准，通用 shell、Codex 原生 webSearch、任意网络、多 Agent、插件和外部 MCP 继续关闭。 +- DirectProject 只连接客户端内置的 `agc_tools` STDIO MCP，基础工具固定为审核引用读取、标准陶泥儿美术准备、已登记资源有界查询、视频/角色动画/音效/BGM 的 create-or-derive 语义生成和 desktop/mobile 浏览器试玩；`webSearchEnabled=true` 时才追加受控联网搜索。MCP 进程只做协议；真实浏览器、付费 External v1 调用和受控搜索通过随机 loopback 地址回到客户端主进程，因此不复制 GUI 登录态、开发者 Key、项目路径、revision、operation 或幂等键到模型上下文。已登记工具固定自动批准，但付费资源工具仍由客户端绑定稳定回合身份、限制单回合请求数、串行执行并优先恢复匹配账本；通用 shell、Codex 原生 webSearch、任意网络、多 Agent、插件和外部 MCP 继续关闭。 - `llm.webSearchEnabled=true` 在 `codex_app_server` 模式下只把 `agc_web_search` 加入 DirectProject 的 `agc_tools` 目录，并作为 app-server 连接池隔离键；关闭时目录与 MCP 环境白名单均不含该能力。客户端主进程只允许固定 Bing RSS 出站请求，禁用代理和重定向，设置 20 秒超时、400 字符查询上限、512 KiB 响应上限和最多 5 条结果；解析后仅向模型返回去 HTML 的有界标题、摘要和公网 HTTPS 链接，拒绝 loopback、私网、带凭据 URL 和非 HTTPS 结果。网页结果始终标记为不可信资料，只能引用，不能作为用户或系统指令执行。 - 陶泥儿生成继续复用既有私有 Key、持久幂等账本、operation 恢复、来源/下载/PNG 解码和 manifest 登记。完整可信图集缺切片可以继续，固定四切片只是推荐路径；凭据失效、来源不明或结果未知时失败关闭，不能自动换 Key 或重新扣费。 - 自定义 LLM API Key 路由只在 DirectHome/DirectProject 经 loopback `/responses` 流式代理转发。代理不注入 Key，只要求请求自带 Bearer，并剥离开发网关错误携带的 `X-Codex-*` ChatGPT 账户额度头，防止隔离 app-server 把 API Provider 误判为余额 0；旧 ToolHost 保持原 Provider 行为。