diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md index e22a0c104..f496d14b7 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md @@ -12,11 +12,12 @@ Let the client derive projections from real disk changes and trusted tool result 1. Write executable source to the current `game/` files and media to the relative paths returned by approved tools. 2. Before using or deriving an existing asset, call `agc_list_registered_assets` and select its `localAssetId`; never infer a source from a filename or submit a local path, platform ID, object key, operation ID, or idempotency key as a generation argument. 3. When the user explicitly asks to create or derive video, character animation, sound effect, or background music, call `agc_create_or_derive_resource`. Use `create` only for video/audio without a source and `derive` with a registered `sourceLocalAssetId`; character animation is always derived from an image. -4. Preserve existing relative paths when a small edit is sufficient so client resource identities remain stable. -5. Do not edit `.agent/manifest.json`, revision counters, version records, resource IDs, canvas identities, source provenance, generation ledgers, or browser receipts by hand. -6. Do not create a version when no game file changed. The client compares content fingerprints and advances revision only after an actual source change. -7. Do not claim a resource or version is visible before the client projects it. If projection is missing, report the changed relative files and let the client re-read durable state. -8. Never move HTML, CSS, or JavaScript into documentation folders. They belong to the game-code projection; prose, design notes, and instructions remain documents. +4. When the user explicitly asks to remove an image background, call `agc_remove_background` with a registered image `sourceLocalAssetId` and an output name. The client requires the signed-in account, owns canvas/folder context and task identity, and returns only bounded queue state. +5. Preserve existing relative paths when a small edit is sufficient so client resource identities remain stable. +6. Do not edit `.agent/manifest.json`, revision counters, version records, resource IDs, canvas identities, source provenance, generation ledgers, or browser receipts by hand. +7. Do not create a version when no game file changed. The client compares content fingerprints and advances revision only after an actual source change. +8. Do not claim a resource or version is visible before the client projects it. If projection is missing, report the changed relative files and let the client re-read durable state. +9. Never move HTML, CSS, or JavaScript into documentation folders. They belong to the game-code projection; prose, design notes, and instructions remain documents. Call `agc_read_skill_resource` with `skillName="agc-client-projection"` and `relativePath="references/projection-contract.md"` when a request touches asset identity, revision behavior, or version history. diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md index b7f754400..811e9c507 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md @@ -11,3 +11,5 @@ Do not collapse these facts. A playable file can exist before projection refresh `agc_list_registered_assets` is the only Direct read path for manifest resource identity. Its relative path and stable identifiers are evidence; omitted prompt, model, provider route, signed URL, host path, and credentials are intentionally not available to Codex. `agc_create_or_derive_resource` accepts only semantic intent. The client resolves `sourceLocalAssetId`, creates stable request identities, recovers matching pending operations, serializes paid submissions, writes supported media into the current canvas and same-name asset folder, validates downloaded bytes, commits the local manifest transaction, and returns redacted warnings. A tool error or timeout is not permission to generate again with a new identity. + +`agc_remove_background` is the account-scoped image post-processing path. It accepts only a registered image `sourceLocalAssetId` and output name; the client resolves the formal source resource, canvas/folder context, task identity, and authenticated `/api/editor/images/background-removals` call. Its result is bounded queue state; Codex must not poll internal workers, construct source URLs, or retry with a new task identity after an uncertain response. diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json index d04fdb6e3..464662c30 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json @@ -1,6 +1,6 @@ { "schemaVersion": "agc-skill-pack.v1", - "version": "2026-08-24.1", + "version": "2026-08-24.2", "skills": [ { "name": "agc-project-structure", @@ -79,20 +79,22 @@ "triggers": [ "新增或替换游戏文件", "查询、创建或派生已登记媒体资源", + "去除已登记图片背景", "素材或版本未显示", "推理 revision 与版本关系" ], "requiredTools": [ "agc_tools.agc_read_skill_resource", "agc_tools.agc_list_registered_assets", - "agc_tools.agc_create_or_derive_resource" + "agc_tools.agc_create_or_derive_resource", + "agc_tools.agc_remove_background" ], "files": [ "SKILL.md", "agents/openai.yaml", "references/projection-contract.md" ], - "sha256": "4b49d54f430028839362a0accf1a5c83878fb1e4eb9b929e0c4912c17470f112" + "sha256": "9b59936000a4b1d69a66bfa132dccac0d972dfcbba4ad130aae6e9f8cc7d35be" } ] } diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs index e1d4f12f7..95355a0ed 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs @@ -1299,6 +1299,106 @@ async fn bridge_create_or_derive_resource( } } +async fn bridge_remove_background(state: &DirectToolBridgeState, arguments: &Value) -> Value { + let result = async { + bridge_reject_unknown_fields(arguments, &["sourceLocalAssetId", "assetName"])?; + enforce_project_permission_policy(&state.root, "canvas.asset_generate")?; + enforce_project_permission_policy(&state.root, "asset.register")?; + let source_asset_id = bridge_bounded_string(arguments, "sourceLocalAssetId", 80)?; + let asset_name = bridge_bounded_string( + arguments, + "assetName", + DIRECT_TOOL_BRIDGE_MAX_RESOURCE_NAME_CHARS, + )?; + let manifest = read_existing_manifest_for_project(&state.root)?; + let source_asset = manifest + .assets + .iter() + .find(|asset| asset.id == source_asset_id) + .ok_or_else(|| "sourceLocalAssetId 不属于当前项目已登记资源".to_string())?; + if !source_asset.media_type.starts_with("image/") { + return Err("抠图工具只接受当前项目已登记的图片资源".to_string()); + } + let source_resource_id = source_asset + .source + .resource_id + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty() && !value.starts_with("local-asset:")) + .ok_or_else(|| "图片资源缺少可供抠图服务使用的正式 resourceId".to_string())? + .to_string(); + let (api_base_url, api_key, session) = resolve_canvas_sync_api_credentials(None, None)?; + if session.is_none() { + return Err("authentication-required: 抠图工具需要当前登录账号".to_string()); + } + let context = prepare_external_canvas_generation_context( + &state.root, + &reqwest::Client::new(), + &ExternalEditorBindingAccess::new(&api_base_url, &api_key, session.as_ref())?, + ) + .await?; + let fingerprint = format!("{}\0{}", source_asset_id, asset_name); + let (task_id, _) = state.resource_request_ids(&fingerprint)?; + let route = + resolve_platform_editor_api_route("/api/external/v1/editor/images/background-removals"); + let response = reqwest::Client::new() + .post(format!("{}{}", api_base_url, route)) + .bearer_auth(api_key) + .json(&json!({ + "sourceImageSrc": source_resource_id, + "projectId": manifest.project_id, + "assetKind": "background-removal", + "assetFolderId": context.asset_folder_id, + "assetLabel": asset_name, + "sourceResourceId": source_resource_id, + "taskId": task_id, + })) + .send() + .await + .map_err(|error| format!("抠图服务提交失败：{error}"))?; + let status = response.status(); + let payload = response + .json::() + .await + .map_err(|error| format!("抠图服务响应无法解析：{error}"))?; + if !status.is_success() { + return Err(format!("抠图服务提交失败：HTTP {}", status.as_u16())); + } + let queue_state = external_editor_response_data(&payload) + .get("queueState") + .cloned() + .ok_or_else(|| "抠图服务响应缺少 queueState".to_string())?; + Ok::<_, String>(json!({ + "status": "queued", + "sourceLocalAssetId": source_asset_id, + "assetName": asset_name, + "projectId": manifest.project_id, + "assetFolderId": context.asset_folder_id, + "queueState": bridge_safe_queue_state(queue_state), + })) + } + .await; + match result { + Ok(value) => bridge_tool_result(value.to_string(), Vec::new(), false), + Err(error) => bridge_tool_result( + redact_agent_runtime_error(&state.root, &error, 480), + Vec::new(), + true, + ), + } +} + +fn bridge_safe_queue_state(value: Value) -> Value { + let object = value.as_object(); + json!({ + "operationId": object.and_then(|value| value.get("operationId")).and_then(Value::as_str), + "status": object.and_then(|value| value.get("status")).and_then(Value::as_str), + "phaseLabel": object.and_then(|value| value.get("phaseLabel")).and_then(Value::as_str), + "progress": object.and_then(|value| value.get("progress")).and_then(Value::as_u64), + "updatedAtMicros": object.and_then(|value| value.get("updatedAtMicros")).and_then(Value::as_u64), + }) +} + fn bridge_art_resources( root: &Path, asset_paths: &[String], @@ -1548,6 +1648,7 @@ async fn handle_direct_tool_bridge( "agc_create_or_derive_resource" => { bridge_create_or_derive_resource(&state, &request.arguments).await } + "agc_remove_background" => bridge_remove_background(&state, &request.arguments).await, "agc_browser_playtest" => bridge_browser_playtest(&state.root, &request.arguments).await, "agc_web_search" => bridge_web_search(&state.root, &request.arguments).await, _ => bridge_tool_result("未知或未审核的客户端工具".to_string(), Vec::new(), true), @@ -1992,4 +2093,20 @@ mod tests { ); } } + + #[test] + fn bridge_background_removal_queue_projection_is_bounded() { + let projection = bridge_safe_queue_state(json!({ + "operationId": "background-removal-1", + "status": "queued", + "phaseLabel": "排队中", + "progress": 0, + "updatedAtMicros": 1, + "error": "private provider detail", + "signedUrl": "https://private.invalid/result" + })); + assert_eq!(projection["operationId"], "background-removal-1"); + assert!(projection.get("error").is_none()); + assert!(projection.get("signedUrl").is_none()); + } } diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs index c825cc377..605dad098 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tools_mcp.rs @@ -158,6 +158,28 @@ fn direct_tools_mcp_specs_for(controlled_web_search: bool) -> Value { "additionalProperties": false } }), + json!({ + "name": "agc_remove_background", + "description": "为当前项目已登记的图片资源去除背景。客户端使用当前登录账号的抠图服务、项目画布和素材目录，模型只能提供已登记资源身份与结果名称；不会返回 Token、内部路由、宿主路径或临时签名 URL。", + "inputSchema": { + "type": "object", + "properties": { + "sourceLocalAssetId": { + "type": "string", + "minLength": 1, + "maxLength": 80, + "description": "必须来自 agc_list_registered_assets 返回的当前项目图片资源 localAssetId" + }, + "assetName": { + "type": "string", + "minLength": 1, + "maxLength": DIRECT_TOOLS_MCP_MAX_RESOURCE_NAME_CHARS + } + }, + "required": ["sourceLocalAssetId", "assetName"], + "additionalProperties": false + } + }), json!({ "name": "agc_browser_playtest", "description": "使用当前客户端的受限 Chromium 对当前游戏执行真实 desktop/mobile 双视口运行、截图、控制台、网络、Canvas/WebGL 和有限交互探针。", @@ -380,6 +402,17 @@ fn validate_resource_generation_arguments(arguments: &Value) -> Result<(), Strin } } +fn validate_remove_background_arguments(arguments: &Value) -> Result<(), String> { + validate_tool_object_fields(arguments, &["sourceLocalAssetId", "assetName"])?; + bounded_tool_string(arguments, "sourceLocalAssetId", 80)?; + bounded_tool_string( + arguments, + "assetName", + DIRECT_TOOLS_MCP_MAX_RESOURCE_NAME_CHARS, + )?; + Ok(()) +} + fn tool_attempt(arguments: &Value) -> Result { let attempt = arguments .get("attempt") @@ -518,6 +551,13 @@ async fn call_agc_create_or_derive_resource(arguments: &Value) -> Value { call_client_tool_bridge("agc_create_or_derive_resource", arguments).await } +async fn call_agc_remove_background(arguments: &Value) -> Value { + if let Err(error) = validate_remove_background_arguments(arguments) { + return mcp_tool_result(error, Vec::new(), true); + } + call_client_tool_bridge("agc_remove_background", arguments).await +} + async fn call_agc_browser_playtest(arguments: &Value) -> Value { if let Err(error) = tool_attempt(arguments) { return mcp_tool_result(error, Vec::new(), true); @@ -589,6 +629,7 @@ async fn handle_direct_tools_mcp_request(_root: &Path, request: Value) -> Option "agc_create_or_derive_resource" => { call_agc_create_or_derive_resource(&arguments).await } + "agc_remove_background" => call_agc_remove_background(&arguments).await, "agc_browser_playtest" => call_agc_browser_playtest(&arguments).await, "agc_web_search" => call_agc_web_search(&arguments).await, _ => mcp_tool_result("未知或未审核的 AGC 工具".to_string(), Vec::new(), true), @@ -713,6 +754,7 @@ mod tests { "taonier_prepare_game_art", "agc_list_registered_assets", "agc_create_or_derive_resource", + "agc_remove_background", "agc_browser_playtest" ] ); @@ -767,6 +809,7 @@ mod tests { "taonier_prepare_game_art", "agc_list_registered_assets", "agc_create_or_derive_resource", + "agc_remove_background", "agc_browser_playtest", "agc_web_search" ] @@ -795,6 +838,17 @@ mod tests { "assetName": "森林过场" })) .is_ok()); + assert!(validate_remove_background_arguments(&json!({ + "sourceLocalAssetId": "asset-image-1", + "assetName": "角色透明图" + })) + .is_ok()); + assert!(validate_remove_background_arguments(&json!({ + "sourceLocalAssetId": "asset-image-1", + "assetName": "角色透明图", + "projectPath": "/private" + })) + .is_err()); assert!(validate_resource_generation_arguments(&json!({ "kind": "character-animation", "mode": "derive", diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index 89368955c..4fd310038 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -14401,3 +14401,7 @@ - Direct 恢复 claim：同一 App 实例重复水合相同 stable turn 并收到“仍在运行”时，必须释放该 `projectPath + clientTurnId` 的恢复 claim，且不得写稳定 assistant 终态。后续显式刷新对话可按原身份重新读取或续跑；不新增无界自动重试。 - 严格图集崩溃收口：workflow 在严格图集调用前先持久化 `strictSpritesheetPending` 并冻结底层严格事务覆盖的九项旧合同身份；旧路径可精确冻结为缺失。Provider 完成结果先绑定原 retained stage ledger。恢复在同一项目锁内对账严格事务；只有新九项合同、规范图/背景图替换锚点与 retained spritesheet result 三者一致才补写 `completed`，旧九项合同才允许补偿。旧合同判定、写 `compensating`、恢复两项素材与登记、回读和清锚点必须在同一项目锁内，重启已有 `compensating` 也重新判定；第三种混合、漂移或 foreign result 状态进入 reconciliation。不能在主图集与四切片已整体提交后仍按两文件 rollback 制造混合包；若中断前阶段告警尚未进入 durable completed result，恢复结果追加“原阶段告警无法完整重放”的明确 warning，不静默清空。 - Direct 对话恢复从新到旧扫描全部合法 User 回合，遇到较新已回答回合继续向前，不得丢失更早未回答回合。成功返回时 Rust 已先持久化 assistant，前端冗余 append 失败也不得重跑 Provider；普通错误终态的显式 append 失败后，恢复 claim 必须保持到 React fallback writer 对同一稳定 assistant messageId 的写入明确成功或失败，不能在 writer 尚在途时按旧 `/history` 快照重跑。fallback 成功后释放 claim；fallback 失败时跳过该 writer 的无界迟到重试并释放 claim，后续显式 `/history` 才可复用原稳定 `clientTurnId`。终态收敛后删除 claim，避免长会话无界增长。 +# 2026-08-24 AGC Direct 抠图语义工具 + +- 决策：将账号态 `/api/editor/images/background-removals` 通过 `agc_remove_background` 加入受控 `agc_tools`。工具只接受当前 manifest 的图片 `sourceLocalAssetId` 与结果名称；客户端负责正式 resourceId、账号态、画布/素材目录、taskId、权限和错误脱敏，不向 Codex 暴露内部 BgFilter worker、凭据或任意 API。 +- 约束：该能力不是 External v1 契约，不扩展 External v1 OpenAPI；异步结果只投影有界 `queueState`，不允许模型自行构造源 URL 或在不确定提交后更换 taskId 重试。 diff --git a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md index 3fb40db9e..f06ff72e1 100644 --- a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md +++ b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md @@ -6,6 +6,7 @@ - 项目路径、projectId、当前 revision、源文件路径与媒体类型、operationId、Idempotency-Key、登录态、项目锁、付费提交、轮询恢复、下载校验与 manifest 事务全部由客户端持有。模型不能提交或覆盖这些字段。同一 Direct `clientTurnId + 规范语义参数` 生成稳定 UUID v4 身份；单回合同参重试复用原 operation，不同请求串行且最多四项。跨回合存在完全匹配的 pending 账本时优先恢复原 operation，不能换键重发。 - 资源查询同时投影未完成 operation 的安全状态。媒体工具成功只返回 operation、本地相对路径、资源类型、Canvas/resource/asset/task 身份、正式序列帧以及脱敏后的 `warnings / sliceWarnings`；错误继续使用统一脱敏边界。客户端资源账本持久化 completed 结果的两类告警，committed replay 不能把历史告警伪装成空集合。 - 角色动画、视频、音效和背景音乐在构造新的远端请求前统一准备当前项目同名画布与素材目录上下文，并在端点支持时携带 `projectId / assetFolderId / canvasCompletion`。角色动画 placeholder 使用源图片真实宽高，避免非方形角色进入画布时失真；正式 resource/asset 与序列帧继续直接复用 External 返回身份，不从首帧伪造重复资源。已有冻结 request body 或已受理 operation 保持不变，不因本次升级重建请求或重复扣费。 +- 抠图通过新增 `agc_remove_background` 语义工具开放：模型只提交当前 manifest 的图片 `sourceLocalAssetId` 与结果名称；客户端要求账号态，解析稳定 `resourceId`，准备同名画布/素材目录并生成受控 `taskId`，调用 `/api/editor/images/background-removals` 后只返回有界 `queueState`。抠图服务仍由客户端和服务端负责源校验、BgFilter、素材登记与画布事务，Codex 不获得内部 worker、凭据或任意 API 调用权。 ## 2026-08-23 AGC 资源生成补齐（视频 / 动画 / 音效 / 背景音乐） @@ -16,7 +17,7 @@ - 新建视频 / 音效 / 背景音乐时，先复用 `prepare_external_canvas_generation_context` 创建或匹配同名画板项目与素材库目录，并在生成请求中携带 `projectId / assetFolderId / canvasCompletion`，结果同时进入平台画布 / 素材库与本地 manifest。 - durable 账本新增 `remoteLegacyPublicPath` 字段（旧账本默认缺省），用于角色动画预览视频的换签下载与恢复；资源提交、轮询、下载、manifest 提交和恢复队列继续复用既有资源编辑事务链。 - 资源编辑请求指纹纳入 `generationMode`，同一 `operationId` 换生成模式重试时不再按旧账本模式恢复；旧账本继续通过 legacy 指纹兼容。 -- 去背景（`/api/editor/images/background-removals`）仍缺 External v1 契约，不在本次补齐范围；本次只补视频 / 角色动画 / 音效 / 背景音乐。 +- 去背景使用账号态客户端语义工具，不伪装成 External v1 生成契约；External v1 仍不直接开放该内部服务。 ## 2026-08-19 UI Editor 节点右键菜单