diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md index f25ab993d..5614ef9d4 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/SKILL.md @@ -14,7 +14,7 @@ Let the client derive projections from real disk changes and trusted tool result 3. Keep read scopes separate: `asset.list` is the current project manifest, `asset.library.list` is the signed-in account library, and the web project's canvas resource read model is the authoritative canvas list. The account library is not the complete canvas list. 4. Use `canvas.asset_import` for safe account/canvas asset IDs or project-relative local paths. The client rechecks ownership and validates bytes; host absolute paths require native UI file-picker authorization. 5. When the user explicitly asks to create or derive video, character animation, sound effect, or background music, call `agc_create_or_derive_resource`. Use `create` only for video/audio without a source and `derive` with a registered `sourceLocalAssetId`; character animation is always derived from an image. Keep `prompt` inside the per-kind limit that the client really enforces: background music at most 140 characters, sound effect at most 1900, video and character animation at most 4000. A longer prompt is rejected before submission, so write the short version first instead of retrying the same text. -6. When the user explicitly asks to remove an image background, call `agc_remove_background` with a registered image `sourceLocalAssetId` and `assetName`. Optional `backgroundMode` is `complex` (semantic foreground segmentation; default) or `flat` (solid-colour background removal). Prefer `flat` when the background is known to be solid. Only `flat` accepts optional `screenColor`: `auto`, `#RRGGBB`, or omitted for automatic detection by the service. Do not select a colour on behalf of `auto`. The client requires the signed-in account, owns canvas/folder context and task identity, and returns only bounded queue state. +6. When the user explicitly asks to remove an image background, call `agc_remove_background` with a registered image `sourceLocalAssetId` and `assetName`. Optional `backgroundMode` is `complex` (semantic foreground segmentation; default) or `flat` (solid-colour background removal). Prefer `flat` when the background is known to be solid. Only `flat` accepts optional `screenColor`: `auto`, `#RRGGBB`, or omitted for automatic detection by the service. Do not select a colour on behalf of `auto`. The client requires the signed-in account, owns canvas/folder context and task identity, waits for the accepted operation, downloads and registers the completed local asset, and preserves the operation for recovery when the remote result is not yet known. 7. Preserve existing relative paths when a small edit is sufficient so client resource identities remain stable. 8. Do not edit `.agent/manifest.json`, revision counters, version records, resource IDs, canvas identities, source provenance, generation ledgers, or browser receipts by hand. 9. Do not create a version when no game file changed. The client compares content fingerprints and advances revision only after an actual source change. diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md index 9129efc1a..7374d1149 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/agc-client-projection/references/projection-contract.md @@ -16,4 +16,6 @@ Read scopes remain separate: `asset.list` is the current project's local manifes `prompt` limits are per kind and are enforced before any paid submission: background music accepts 1-140 characters, sound effect 1-1900, video and character animation 1-4000, and image editing (`agc_edit_image`) 1-32000. The client composes the submitted request from a fixed prefix plus your prompt, so an over-limit prompt fails locally with the exact limit; shorten the text rather than resubmitting the same value. `agc_edit_image` remains the image path; this tool never generates or edits still images. -`agc_remove_background` accepts a registered image `sourceLocalAssetId`, `assetName`, and optional `backgroundMode` and `screenColor`. `complex` uses semantic segmentation to identify the foreground; `flat` removes a solid-colour background. Prefer `flat` when the background is known to be solid; omitting the mode selects `complex`. Only `flat` accepts a colour: `auto`, `#RRGGBB`, or omitted for automatic service detection. Never infer a concrete colour for `auto`. Empty or invalid values and colour without `flat` are rejected. The client resolves the formal source resource, canvas/folder context, stable operation identity, idempotency key, and authenticated External v1 `/api/external/v1/editor/images/background-removals` call. Mode and colour are part of request identity. Its result is bounded queue state; Codex must not poll internal workers, construct source URLs, or retry with a new identity after an uncertain response. +`agc_remove_background` accepts a registered image `sourceLocalAssetId`, `assetName`, and optional `backgroundMode` and `screenColor`. `complex` uses semantic segmentation to identify the foreground; `flat` removes a solid-colour background. Prefer `flat` when the background is known to be solid; omitting the mode selects `complex`. Only `flat` accepts a colour: `auto`, `#RRGGBB`, or omitted for automatic service detection. Never infer a concrete colour for `auto`. Empty or invalid values and colour without `flat` are rejected. The client resolves the formal source resource, canvas/folder context, stable operation identity, idempotency key, and authenticated request. Ordinary account mode maps the External v1 shaped route to `/api/editor/images/background-removals`; ExternalDeveloper mode uses `/api/external/v1/editor/images/background-removals`. Mode and colour are part of request identity. After acceptance, the client polls the authenticated generation status route, downloads the completed media, and commits it to the local manifest. If completion is unknown, it retains the same local operation for recovery; it never retries with a new identity or exposes internal worker details. + +After an interrupted call, inspect `agc_list_registered_assets.pendingOperations`. Calling `agc_remove_background` again with the same source, name, mode, and colour resumes the matching pending operation. A submission marked `reconciliation-required` needs client-side reconciliation and cannot be automatically resumed. Do not change parameters to bypass a pending task. A queued receipt, fixed progress value, or absent local file does not establish that the background-removal provider is waiting in a queue; report only the observed state. diff --git a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json index 6debfefb3..26790f360 100644 --- a/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json +++ b/apps/ai-game-creator-shell/src-tauri/resources/agc-skills/manifest.json @@ -1,6 +1,6 @@ { "schemaVersion": "agc-skill-pack.v1", - "version": "2026-08-26.20", + "version": "2026-08-26.23", "skills": [ { "name": "agc-game-production-workflow", @@ -123,7 +123,7 @@ "agents/openai.yaml", "references/projection-contract.md" ], - "sha256": "93210c0eeb73b279d35aa85c201c226139b0bdf041f3300ac2c6e2c1bdd63afe" + "sha256": "247787975944ce8b21d7c879c39c60ec13608056cff9426ac374c9299937d475" } ] } diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs index f31a3cdd4..9984ff4d6 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent/direct_tool_bridge.rs @@ -1294,7 +1294,10 @@ fn bridge_list_registered_assets(root: &Path, arguments: &Value) -> Value { .map(|asset| bridge_registered_resource(asset, include_sequence_frames)) .collect::>(); let next_offset = (offset + resources.len() < total).then_some(offset + resources.len()); - let pending = list_pending_local_project_resource_edits_at( + let platform_session = (editor_api_mode() == EditorApiMode::PlatformAccount) + .then(current_platform_session) + .flatten(); + let pending = list_pending_local_project_resource_edits_for_session_at( ListPendingLocalProjectResourceEditsInput { project_path: root .to_str() @@ -1302,6 +1305,7 @@ fn bridge_list_registered_assets(root: &Path, arguments: &Value) -> Value { .to_string(), expected_project_id: manifest.project_id, }, + platform_session.as_ref(), )? .into_iter() .map(|edit| { @@ -1311,6 +1315,8 @@ fn bridge_list_registered_assets(root: &Path, arguments: &Value) -> Value { "mode": edit.generation_mode, "sourceResourceId": edit.source_resource_id, "assetName": edit.asset_name, + "backgroundMode": edit.background_mode, + "screenColor": edit.screen_color, "phase": edit.phase, "createdAt": edit.created_at, }) @@ -1801,8 +1807,8 @@ async fn bridge_import_account_assets(state: &DirectToolBridgeState, arguments: fn bridge_completed_resource_result( root: &Path, - kind: DirectResourceGenerationKind, - mode: DirectResourceGenerationMode, + kind: &str, + mode: &str, result: DeriveLocalProjectResourceResult, ) -> Result { let asset = result @@ -1814,8 +1820,8 @@ fn bridge_completed_resource_result( Ok(json!({ "status": "completed", "operationId": result.operation_id, - "kind": kind.as_str(), - "mode": mode.as_str(), + "kind": kind, + "mode": mode, "sourceResourceId": result.source_resource_id, "committedProjectRevision": result.committed_project_revision, "resource": bridge_registered_resource(asset, true), @@ -1910,10 +1916,17 @@ async fn bridge_create_or_derive_resource( source_version_id: None, prompt: input.prompt.clone(), asset_name: input.asset_name.clone(), + background_mode: None, + screen_color: None, }; with_direct_editor_api_credentials(derive_local_project_resource_at(request)).await? }; - bridge_completed_resource_result(&state.root, input.kind, input.mode, completed) + bridge_completed_resource_result( + &state.root, + input.kind.as_str(), + input.mode.as_str(), + completed, + ) } .await; match result { @@ -1927,7 +1940,8 @@ async fn bridge_create_or_derive_resource( } async fn bridge_remove_background(state: &DirectToolBridgeState, arguments: &Value) -> Value { - let result = async { + let _generation_guard = state.resource_generation_gate.lock().await; + let result = with_direct_editor_api_credentials(async { super::direct_tools_mcp::validate_remove_background_arguments(arguments)?; enforce_project_permission_policy(&state.root, "canvas.asset_generate")?; enforce_project_permission_policy(&state.root, "asset.register")?; @@ -1948,74 +1962,72 @@ async fn bridge_remove_background(state: &DirectToolBridgeState, arguments: &Val if !source_asset.media_type.starts_with("image/") { return Err("抠图工具只接受当前项目已登记的图片资源".to_string()); } - let source_resource_id = source_asset - .source - .resource_id - .as_deref() - .map(str::trim) - .filter(|value| !value.is_empty() && !value.starts_with("local-asset:")) - .ok_or_else(|| "图片资源缺少可供抠图服务使用的正式 resourceId".to_string())? - .to_string(); - let (api_base_url, api_key, session) = resolve_canvas_sync_api_credentials(None, None)?; - let access = ExternalEditorBindingAccess::new(&api_base_url, &api_key, session.as_ref())?; - let client = crate::http_client::agc_main_site_client_builder() - .build() - .map_err(|_| "创建抠图服务连接失败".to_string())?; - let context = - prepare_external_canvas_generation_context(&state.root, &client, &access).await?; + let background_mode = background_mode.unwrap_or("complex").to_string(); + let source_resource_id = bridge_asset_canonical_resource_id(source_asset); let fingerprint = background_removal_request_fingerprint( &source_asset_id, &asset_name, - background_mode, + Some(background_mode.as_str()), screen_color, ); - let (_operation_id, idempotency_key) = state.resource_request_ids(&fingerprint)?; - let route = "/api/external/v1/editor/images/background-removals"; - let mut request_body = json!({ - "sourceImageSrc": source_resource_id, - "projectId": manifest.project_id, - "assetKind": source_asset.kind, - "assetFolderId": context.asset_folder_id, - "assetLabel": asset_name, - "sourceResourceId": source_resource_id, - }); - if background_mode == Some("flat") { - request_body["backgroundMode"] = json!("flat"); + let (_, _, platform_session) = resolve_canvas_sync_api_credentials(None, None)?; + let pending = list_pending_local_project_resource_edits_for_session_at( + ListPendingLocalProjectResourceEditsInput { + project_path: state.root.to_string_lossy().into_owned(), + expected_project_id: manifest.project_id.clone(), + }, + platform_session.as_ref(), + )?; + let matching_pending = pending + .into_iter() + .filter(|pending| { + pending.edit_kind == LocalProjectResourceEditKind::BackgroundRemoval + && (pending.source_asset_id.as_deref() == Some(source_asset_id.as_str()) + || pending.source_resource_id == format!("local-asset:{source_asset_id}")) + && pending.asset_name == asset_name + && pending.background_mode.as_deref().unwrap_or("complex") + == background_mode.as_str() + && pending.screen_color.as_deref() == screen_color + }) + .collect::>(); + if matching_pending.len() > 1 { + return Err("存在多个相同抠图 operation，必须先在客户端完成对账".to_string()); } - if let Some(color) = screen_color { - request_body["screenColor"] = json!(color); - } - let response = crate::http_client::with_agc_main_site_marker( - client - .post(format!("{}{}", api_base_url, route)) - .bearer_auth(api_key) - .header("Idempotency-Key", idempotency_key) - .json(&request_body), - ) - .send() - .await - .map_err(|error| format!("抠图服务提交失败：{error}"))?; - let status = response.status(); - let payload = response - .json::() - .await - .map_err(|error| format!("抠图服务响应无法解析：{error}"))?; - if !status.is_success() { - if status == reqwest::StatusCode::UNAUTHORIZED { - return Err("authentication-required: 抠图服务提交失败：HTTP 401".to_string()); - } - return Err(format!("抠图服务提交失败：HTTP {}", status.as_u16())); - } - let queue_state = external_editor_response_data(&payload).clone(); - Ok::<_, String>(json!({ - "status": "queued", - "sourceLocalAssetId": source_asset_id, - "assetName": asset_name, - "projectId": manifest.project_id, - "assetFolderId": context.asset_folder_id, - "queueState": bridge_safe_queue_state(queue_state), - })) - } + let completed = if let Some(pending) = matching_pending.into_iter().next() { + resume_local_project_resource_edit_at(ResumeLocalProjectResourceEditInput { + project_path: state.root.to_string_lossy().into_owned(), + expected_project_id: manifest.project_id.clone(), + operation_id: pending.operation_id, + }) + .await? + } else { + let (operation_id, idempotency_key) = state.resource_request_ids(&fingerprint)?; + let revision = read_game_creator_agent_runtime_project_revision(&state.root)?.revision; + let request = DeriveLocalProjectResourceInput { + project_path: state.root.to_string_lossy().into_owned(), + expected_project_id: manifest.project_id.clone(), + expected_project_revision: revision, + operation_id, + idempotency_key, + edit_kind: LocalProjectResourceEditKind::BackgroundRemoval, + generation_mode: LocalProjectResourceGenerationMode::Derive, + source_resource_id, + source_asset_id: Some(source_asset_id.clone()), + source_path: Some(source_asset.local_path.clone()), + source_media_type: Some(source_asset.media_type.clone()), + source_subtype: Some(source_asset.kind.clone()), + producer_task_id: source_asset.source.task_id.clone(), + source_version_id: None, + prompt: "去除背景".to_string(), + asset_name: asset_name.clone(), + background_mode: Some(background_mode), + screen_color: screen_color.map(str::to_string), + }; + derive_local_project_resource_at(request).await? + }; + emit_game_creator_manifest_invalidated(&state.root, "direct-background-removal"); + bridge_completed_resource_result(&state.root, "background-removal", "derive", completed) + }) .await; match result { Ok(value) => bridge_tool_result(value.to_string(), Vec::new(), false), @@ -2041,17 +2053,6 @@ fn background_removal_request_fingerprint( } } -fn bridge_safe_queue_state(value: Value) -> Value { - let object = value.as_object(); - json!({ - "operationId": object.and_then(|value| value.get("operationId")).and_then(Value::as_str), - "status": object.and_then(|value| value.get("status")).and_then(Value::as_str), - "phaseLabel": object.and_then(|value| value.get("phaseLabel")).and_then(Value::as_str), - "progress": object.and_then(|value| value.get("progress")).and_then(Value::as_u64), - "updatedAtMicros": object.and_then(|value| value.get("updatedAtMicros")).and_then(Value::as_u64), - }) -} - fn bridge_art_resources( root: &Path, asset_paths: &[String], @@ -3844,20 +3845,4 @@ mod tests { ); } } - - #[test] - fn bridge_background_removal_queue_projection_is_bounded() { - let projection = bridge_safe_queue_state(json!({ - "operationId": "background-removal-1", - "status": "queued", - "phaseLabel": "排队中", - "progress": 0, - "updatedAtMicros": 1, - "error": "private provider detail", - "signedUrl": "https://private.invalid/result" - })); - assert_eq!(projection["operationId"], "background-removal-1"); - assert!(projection.get("error").is_none()); - assert!(projection.get("signedUrl").is_none()); - } } diff --git a/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs b/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs index 46bb81c24..145330f56 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs @@ -113,6 +113,7 @@ fn resource_edit_project_mutation_lock(root: &Path) -> Result &'static str { match self { Self::ImageReference => "image-reference", + Self::BackgroundRemoval => "background-removal", Self::Svg => "svg", Self::CharacterAnimation => "character-animation", Self::Video => "video", @@ -188,6 +191,10 @@ pub(crate) struct DeriveLocalProjectResourceInput { pub(crate) source_version_id: Option, pub(crate) prompt: String, pub(crate) asset_name: String, + #[serde(default)] + pub(crate) background_mode: Option, + #[serde(default)] + pub(crate) screen_color: Option, } #[derive(Clone, Debug, PartialEq, Serialize)] @@ -220,6 +227,10 @@ pub(crate) struct PendingLocalProjectResourceEdit { pub(crate) source_resource_id: String, pub(crate) source_asset_id: Option, pub(crate) asset_name: String, + #[serde(default)] + pub(crate) background_mode: Option, + #[serde(default)] + pub(crate) screen_color: Option, pub(crate) prompt_sha256: String, pub(crate) phase: String, pub(crate) created_at: u64, @@ -385,6 +396,10 @@ struct ResourceEditLedger { prompt: String, asset_name: String, #[serde(default)] + background_mode: Option, + #[serde(default)] + screen_color: Option, + #[serde(default)] provider_request_issued_at: Option, #[serde(default)] access_scheme: Option, @@ -829,7 +844,7 @@ fn resource_edit_request_fingerprint( prompt: &str, asset_name: &str, ) -> Result { - let payload = serde_json::to_vec(&serde_json::json!({ + let mut identity = serde_json::json!({ "schemaVersion": RESOURCE_EDIT_SCHEMA_VERSION, "projectId": input.expected_project_id, "operationId": input.operation_id, @@ -840,8 +855,13 @@ fn resource_edit_request_fingerprint( "sourceSha256": source.source_sha256, "prompt": prompt, "assetName": asset_name, - })) - .map_err(|error| format!("序列化资源编辑请求失败：{error}"))?; + }); + if input.edit_kind == LocalProjectResourceEditKind::BackgroundRemoval { + identity["backgroundMode"] = serde_json::json!(input.background_mode); + identity["screenColor"] = serde_json::json!(input.screen_color); + } + let payload = serde_json::to_vec(&identity) + .map_err(|error| format!("序列化资源编辑请求失败：{error}"))?; Ok(sha256_hex(&payload)) } @@ -851,7 +871,7 @@ fn legacy_resource_edit_request_fingerprint( prompt: &str, asset_name: &str, ) -> Result { - let payload = serde_json::to_vec(&serde_json::json!({ + let mut identity = serde_json::json!({ "schemaVersion": RESOURCE_EDIT_SCHEMA_VERSION, "projectId": input.expected_project_id, "expectedProjectRevision": input.expected_project_revision, @@ -862,8 +882,13 @@ fn legacy_resource_edit_request_fingerprint( "sourceSha256": source.source_sha256, "prompt": prompt, "assetName": asset_name, - })) - .map_err(|error| format!("序列化旧资源编辑请求失败：{error}"))?; + }); + if input.edit_kind == LocalProjectResourceEditKind::BackgroundRemoval { + identity["backgroundMode"] = serde_json::json!(input.background_mode); + identity["screenColor"] = serde_json::json!(input.screen_color); + } + let payload = serde_json::to_vec(&identity) + .map_err(|error| format!("序列化旧资源编辑请求失败：{error}"))?; Ok(sha256_hex(&payload)) } @@ -1106,9 +1131,21 @@ fn infer_resource_edit_source_media_type( .and_then(|value| value.to_str()) .map(str::to_ascii_lowercase); let media_type = match (edit_kind, extension.as_deref()) { - (LocalProjectResourceEditKind::ImageReference, Some("png")) => "image/png", - (LocalProjectResourceEditKind::ImageReference, Some("jpg" | "jpeg")) => "image/jpeg", - (LocalProjectResourceEditKind::ImageReference, Some("webp")) => "image/webp", + ( + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval, + Some("png"), + ) => "image/png", + ( + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval, + Some("jpg" | "jpeg"), + ) => "image/jpeg", + ( + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval, + Some("webp"), + ) => "image/webp", (LocalProjectResourceEditKind::Svg, Some("svg")) => "image/svg+xml", (LocalProjectResourceEditKind::Video, Some("mp4")) => "video/mp4", (LocalProjectResourceEditKind::Video, Some("webm")) => "video/webm", @@ -1145,7 +1182,8 @@ fn infer_resource_edit_source_media_type( fn infer_resource_edit_source_asset_kind(edit_kind: &LocalProjectResourceEditKind) -> String { match edit_kind { - LocalProjectResourceEditKind::ImageReference => "art-image", + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval => "art-image", LocalProjectResourceEditKind::Svg => "svg", LocalProjectResourceEditKind::CharacterAnimation => "character-animation", LocalProjectResourceEditKind::Video => "video", @@ -1373,8 +1411,11 @@ fn resolve_resource_edit_source( { return Err("音频编辑只能用于音频资源".to_string()); } - LocalProjectResourceEditKind::ImageReference if !lower_media_type.starts_with("image/") => { - return Err("图片参考编辑只能用于图片资源".to_string()); + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval + if !lower_media_type.starts_with("image/") => + { + return Err("此操作只能用于图片资源".to_string()); } _ => {} } @@ -2023,8 +2064,11 @@ fn validate_resource_edit_source_recovery_state( || (ledger.source_remote_asset_object_id.is_some() && !ledger.source_upload_completed) || (ledger.source_remote_resource_id.is_some() && ledger.source_remote_asset_object_id.is_none()) - || (input.edit_kind != LocalProjectResourceEditKind::ImageReference - && ledger.source_remote_resource_id.is_some()) + || (!matches!( + input.edit_kind, + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval + ) && ledger.source_remote_resource_id.is_some()) { return Err("result-unknown: 源资源远端恢复阶段不完整，禁止自动重放".to_string()); } @@ -2081,7 +2125,8 @@ async fn ensure_resource_edit_source_reference( &source_identity, )? { let stable_reference = match input.edit_kind { - LocalProjectResourceEditKind::ImageReference => binding + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval => binding .remote_resource_id .filter(|reference| is_registered_editor_reference_id(reference)), LocalProjectResourceEditKind::Video @@ -2188,45 +2233,48 @@ async fn ensure_resource_edit_source_reference( confirmed.value }; - let (stable_reference, remote_resource_id, width, height) = - if input.edit_kind == LocalProjectResourceEditKind::ImageReference { - let bytes = source - .bytes - .as_deref() - .ok_or_else(|| "登记源图片缺少文件内容".to_string())?; - let decoded = image::load_from_memory(bytes) - .map_err(|_| "登记源图片前无法解析图片尺寸".to_string())?; - let resource_id = if let Some(resource_id) = ledger.source_remote_resource_id.clone() { - resource_id - } else { - let registered = register_resource_edit_source_image( - client, - access, - source, - local_asset_id, - &binding_key, - &canvas_context.project_id, - &object_key, - &asset_object_id, - ) - .await?; - let resource_id = registered.value.0; - ledger.source_remote_resource_id = Some(resource_id.clone()); - write_resource_edit_source_stage(root, ledger, "源图片项目资源登记")?; - if let Some(error) = registered.post_response_session_error { - return Err(error); - } - resource_id - }; - ( - resource_id.clone(), - Some(resource_id), - Some(decoded.width()), - Some(decoded.height()), - ) + let (stable_reference, remote_resource_id, width, height) = if matches!( + input.edit_kind, + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval + ) { + let bytes = source + .bytes + .as_deref() + .ok_or_else(|| "登记源图片缺少文件内容".to_string())?; + let decoded = image::load_from_memory(bytes) + .map_err(|_| "登记源图片前无法解析图片尺寸".to_string())?; + let resource_id = if let Some(resource_id) = ledger.source_remote_resource_id.clone() { + resource_id } else { - (object_key.clone(), None, None, None) + let registered = register_resource_edit_source_image( + client, + access, + source, + local_asset_id, + &binding_key, + &canvas_context.project_id, + &object_key, + &asset_object_id, + ) + .await?; + let resource_id = registered.value.0; + ledger.source_remote_resource_id = Some(resource_id.clone()); + write_resource_edit_source_stage(root, ledger, "源图片项目资源登记")?; + if let Some(error) = registered.post_response_session_error { + return Err(error); + } + resource_id }; + ( + resource_id.clone(), + Some(resource_id), + Some(decoded.width()), + Some(decoded.height()), + ) + } else { + (object_key.clone(), None, None, None) + }; access.validate_frozen_session()?; let binding = new_external_editor_resource_binding( &input.expected_project_id, @@ -2330,6 +2378,28 @@ fn resource_edit_remote_request( "generationInputs": generation_inputs, }), )), + LocalProjectResourceEditKind::BackgroundRemoval => { + let background_mode = input.background_mode.as_deref().unwrap_or("complex"); + validate_background_removal_options(background_mode, input.screen_color.as_deref())?; + let mut body = serde_json::json!({ + "sourceImageSrc": source_reference + .ok_or_else(|| "抠图缺少正式源资源 ID".to_string())?, + "sourceResourceId": source_reference + .ok_or_else(|| "抠图缺少正式源资源 ID".to_string())?, + "assetKind": source.asset_kind, + "assetLabel": asset_name, + "backgroundMode": background_mode, + "generationInputs": generation_inputs, + }); + if let Some(screen_color) = input.screen_color.as_deref() { + body["screenColor"] = serde_json::json!(screen_color); + } + if let Some(context) = canvas_context { + body["projectId"] = serde_json::json!(context.project_id); + body["assetFolderId"] = serde_json::json!(context.asset_folder_id); + } + Ok(("/api/external/v1/editor/images/background-removals", body)) + } LocalProjectResourceEditKind::CharacterAnimation => { let mut body = serde_json::json!({ "sourceLayerId": format!("resource-{}", input.operation_id), @@ -2436,6 +2506,31 @@ fn resource_edit_remote_request( } } +fn validate_background_removal_options( + background_mode: &str, + screen_color: Option<&str>, +) -> Result<(), String> { + if !matches!(background_mode, "complex" | "flat") { + return Err("抠图 backgroundMode 必须是 complex 或 flat".to_string()); + } + if background_mode == "complex" && screen_color.is_some() { + return Err("complex 抠图不能携带 screenColor".to_string()); + } + if let Some(screen_color) = screen_color { + let valid_hex = screen_color.len() == 7 + && screen_color.starts_with('#') + && screen_color.as_bytes()[1..].iter().all(|byte| { + byte.is_ascii_digit() + || (b'a'..=b'f').contains(byte) + || (b'A'..=b'F').contains(byte) + }); + if screen_color != "auto" && !valid_hex { + return Err("flat 抠图 screenColor 必须是 auto 或 #RRGGBB".to_string()); + } + } + Ok(()) +} + fn resource_edit_operation_id(payload: &serde_json::Value) -> Option { let data = external_editor_response_data(payload); json_string_field(data, "operationId").or_else(|| { @@ -2464,11 +2559,13 @@ fn is_external_resource_edit_endpoint(endpoint: &str) -> bool { matches!( endpoint, "/api/editor/images/edits" + | "/api/editor/images/background-removals" | "/api/editor/character-animations/generations" | "/api/editor/videos/generations" | "/api/editor/audios/sound-effects/generations" | "/api/editor/audios/background-music/generations" | "/api/external/v1/editor/images/edits" + | "/api/external/v1/editor/images/background-removals" | "/api/external/v1/editor/character-animations/generations" | "/api/external/v1/editor/videos/generations" | "/api/external/v1/editor/audios/sound-effects/generations" @@ -2954,6 +3051,17 @@ fn validate_downloaded_media( let is_jpeg = starts(&[0xff, 0xd8, 0xff]); let is_webp = bytes.len() >= 12 && starts(b"RIFF") && &bytes[8..12] == b"WEBP"; match edit_kind { + LocalProjectResourceEditKind::BackgroundRemoval => { + if !is_png { + return Err("抠图结果必须是带透明通道的 PNG".to_string()); + } + let decoded = image::load_from_memory_with_format(bytes, image::ImageFormat::Png) + .map_err(|_| "抠图结果不是有效的 PNG".to_string())?; + if !decoded.color().has_alpha() { + return Err("抠图结果缺少透明通道".to_string()); + } + Ok(("image/png".to_string(), "png".to_string())) + } LocalProjectResourceEditKind::ImageReference => { if is_png { Ok(("image/png".to_string(), "png".to_string())) @@ -3246,6 +3354,7 @@ async fn prepare_remote_resource_edit( let prepared_source = if matches!( input.edit_kind, LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval | LocalProjectResourceEditKind::CharacterAnimation ) || (input.edit_kind == LocalProjectResourceEditKind::Video && input.generation_mode == LocalProjectResourceGenerationMode::Derive) @@ -3275,6 +3384,7 @@ async fn prepare_remote_resource_edit( } else if matches!( input.edit_kind, LocalProjectResourceEditKind::CharacterAnimation + | LocalProjectResourceEditKind::BackgroundRemoval | LocalProjectResourceEditKind::Video | LocalProjectResourceEditKind::SoundEffect | LocalProjectResourceEditKind::BackgroundMusic @@ -4444,8 +4554,11 @@ fn write_resource_edit_result_binding( let Some(asset_object_id) = ledger.remote_asset_object_id.as_deref() else { return Ok(()); }; - if input.edit_kind == LocalProjectResourceEditKind::ImageReference - && ledger.remote_resource_id.is_none() + if matches!( + input.edit_kind, + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval + ) && ledger.remote_resource_id.is_none() { return Ok(()); } @@ -4455,7 +4568,11 @@ fn write_resource_edit_result_binding( media_read_limit(&input.edit_kind), "派生资源 binding 文件", )?; - let (width, height) = if input.edit_kind == LocalProjectResourceEditKind::ImageReference { + let (width, height) = if matches!( + input.edit_kind, + LocalProjectResourceEditKind::ImageReference + | LocalProjectResourceEditKind::BackgroundRemoval + ) { let decoded = image::load_from_memory(&bytes) .map_err(|_| "派生图片 binding 无法解析尺寸".to_string())?; (Some(decoded.width()), Some(decoded.height())) @@ -4775,6 +4892,17 @@ fn commit_resource_edit_version( pub(crate) fn list_pending_local_project_resource_edits_at( input: ListPendingLocalProjectResourceEditsInput, +) -> Result, String> { + let current_platform_session = current_platform_session(); + list_pending_local_project_resource_edits_for_session_at( + input, + current_platform_session.as_ref(), + ) +} + +pub(crate) fn list_pending_local_project_resource_edits_for_session_at( + input: ListPendingLocalProjectResourceEditsInput, + platform_session: Option<&PlatformSessionSnapshot>, ) -> Result, String> { let root = Path::new(input.project_path.trim()); validate_project_root(root)?; @@ -4782,8 +4910,7 @@ pub(crate) fn list_pending_local_project_resource_edits_at( if manifest.project_id != input.expected_project_id { return Err("project-identity-conflict".to_string()); } - let current_platform_session = current_platform_session(); - let _platform_session_lease = current_platform_session + let _platform_session_lease = platform_session .as_ref() .map(|session| acquire_platform_session_identity_lease(&session.identity())) .transpose()?; @@ -4825,10 +4952,8 @@ pub(crate) fn list_pending_local_project_resource_edits_at( if !matches!( ledger.phase, ResourceEditLedgerPhase::Committed | ResourceEditLedgerPhase::Archived - ) && resource_edit_pending_is_visible_to_current_principal( - &ledger, - current_platform_session.as_ref(), - ) { + ) && resource_edit_pending_is_visible_to_current_principal(&ledger, platform_session) + { pending.push(PendingLocalProjectResourceEdit { operation_id: ledger.operation_id, edit_kind: ledger.edit_kind, @@ -4836,6 +4961,8 @@ pub(crate) fn list_pending_local_project_resource_edits_at( source_resource_id: ledger.source_resource_id, source_asset_id: ledger.source_asset_id, asset_name: ledger.asset_name, + background_mode: ledger.background_mode, + screen_color: ledger.screen_color, prompt_sha256: sha256_hex(ledger.prompt.as_bytes()), phase: ledger.phase.as_str().to_string(), created_at: ledger.created_at, @@ -5186,6 +5313,8 @@ pub(crate) async fn resume_local_project_resource_edit_at( source_version_id, prompt: ledger.prompt, asset_name: ledger.asset_name, + background_mode: ledger.background_mode, + screen_color: ledger.screen_color, }) .await } @@ -5195,6 +5324,12 @@ pub(crate) async fn derive_local_project_resource_at( ) -> Result { validate_resource_edit_uuid(&input.operation_id, "operationId")?; validate_resource_edit_uuid(&input.idempotency_key, "idempotencyKey")?; + if input.edit_kind == LocalProjectResourceEditKind::BackgroundRemoval { + validate_background_removal_options( + input.background_mode.as_deref().unwrap_or("complex"), + input.screen_color.as_deref(), + )?; + } if input.expected_project_revision > 9_007_199_254_740_991 { return Err("expectedProjectRevision 超出 JavaScript 安全整数范围".to_string()); } @@ -5281,6 +5416,8 @@ pub(crate) async fn derive_local_project_resource_at( source_sha256: source.source_sha256.clone(), prompt: prompt.clone(), asset_name: asset_name.clone(), + background_mode: input.background_mode.clone(), + screen_color: input.screen_color.clone(), provider_request_issued_at: None, access_scheme: None, api_identity_scheme: None, @@ -5484,6 +5621,9 @@ pub(crate) async fn derive_local_project_resource_at( Ok(result) } +#[cfg(test)] +mod background_removal_tests; + #[cfg(test)] mod tests { use super::*; @@ -5963,6 +6103,8 @@ mod tests { source_version_id: None, prompt: "保留原意并补充红发角色设定".to_string(), asset_name: "规则编辑版".to_string(), + background_mode: None, + screen_color: None, } } @@ -6000,6 +6142,8 @@ mod tests { source_sha256: source.source_sha256.clone(), prompt: input.prompt.clone(), asset_name: input.asset_name.clone(), + background_mode: input.background_mode.clone(), + screen_color: input.screen_color.clone(), provider_request_issued_at: None, access_scheme: input .edit_kind @@ -8539,6 +8683,37 @@ mod tests { serde_json::json!("stable-image-reference") ); assert!(image_body.get("sourceImageSrc").is_none()); + let mut background = image; + background.edit_kind = LocalProjectResourceEditKind::BackgroundRemoval; + background.background_mode = Some("flat".to_string()); + background.screen_color = Some("auto".to_string()); + let (background_endpoint, background_body) = resource_edit_remote_request( + &background, + &ResourceEditSourceSnapshot { + media_type: "image/png".to_string(), + asset_kind: "art-image".to_string(), + source_path: Some("assets/source.png".to_string()), + bytes: Some(resource_editor_test_png()), + ..source.clone() + }, + "去除背景", + "透明图", + Some("stable-image-reference"), + Some(&ExternalCanvasGenerationContext { + project_id: "project-1".to_string(), + asset_folder_id: "folder-1".to_string(), + canvas_name: "测试画板".to_string(), + }), + ) + .expect("build background removal request"); + assert_eq!( + background_endpoint, + "/api/external/v1/editor/images/background-removals" + ); + assert_eq!(background_body["backgroundMode"], "flat"); + assert_eq!(background_body["screenColor"], "auto"); + assert_eq!(background_body["projectId"], "project-1"); + assert_eq!(background_body["assetFolderId"], "folder-1"); assert!(is_external_resource_edit_endpoint( "/api/editor/videos/generations" )); diff --git a/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor/background_removal_tests.rs b/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor/background_removal_tests.rs new file mode 100644 index 000000000..aa731aeef --- /dev/null +++ b/apps/ai-game-creator-shell/src-tauri/src/project/resource_editor/background_removal_tests.rs @@ -0,0 +1,725 @@ +use super::*; + +use std::io::{Cursor, Read, Write}; +use std::net::{TcpListener, TcpStream}; +use std::sync::{Arc, Mutex}; + +const TEST_PROJECT_ID: &str = "background-removal-project"; +const TEST_API_KEY: &str = "background-removal-key"; +const REMOTE_PROJECT_ID: &str = "background-removal-remote-project"; +const REMOTE_FOLDER_ID: &str = "background-removal-folder"; +const SOURCE_RESOURCE_ID: &str = "editor-resource-background-removal-source"; + +struct BackgroundRemovalFixture { + directory: tempfile::TempDir, + request: DeriveLocalProjectResourceInput, + source_asset_id: String, + source_bytes: Vec, +} + +#[tokio::test] +async fn background_removal_platform_account_uses_runtime_job_and_platform_read_url_routes() { + let listener = TcpListener::bind("127.0.0.1:0").expect("bind platform account server"); + let base_url = format!( + "http://{}", + listener.local_addr().expect("platform address") + ); + let _session_guard = install_test_platform_session( + "background-removal-owner", + "background-removal-platform-token", + &base_url, + ); + let session = current_platform_session().expect("platform account session"); + let fixture = background_removal_fixture_for_session(&base_url, Some(&session)); + let generated_png = test_png(); + let signed_url = format!("{base_url}/platform-result.png"); + let captured = Arc::new(Mutex::new(Vec::new())); + let server_captured = Arc::clone(&captured); + let server_png = generated_png.clone(); + let server = std::thread::spawn(move || { + for index in 0..6 { + let mut stream = accept_request(&listener, index); + let request = read_request(&mut stream); + server_captured + .lock() + .expect("capture requests") + .push(request.clone()); + if respond_canvas_context_request(&mut stream, &request) { + continue; + } + let line = request.lines().next().unwrap_or_default(); + if index != 5 { + assert!(request + .to_ascii_lowercase() + .contains("authorization: bearer background-removal-platform-token")); + } + match index { + 2 => { + assert!(line.starts_with("POST /api/editor/images/background-removals ")); + assert_submission(&request); + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": {"queueState": { + "operationId": "platform-background-removal", "status": "queued", "pollAfterMs": 0 + }}}), + ); + } + 3 => { + assert!(line.starts_with( + "GET /api/runtime/external-generation/jobs/platform-background-removal " + )); + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": {"job": { + "operationId": "platform-background-removal", "status": "completed", + "result": {"resource": { + "resourceId": "editor-resource-platform-background-removal", + "projectId": REMOTE_PROJECT_ID, + "objectKey": "generated/platform-background-removal.png", + "assetObjectId": "platform-background-removal-object" + }} + }}}), + ); + } + 4 => { + assert!(line.starts_with("GET /api/assets/read-url?")); + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": {"read": {"signedUrl": signed_url}}}), + ); + } + 5 => { + assert!(line.starts_with("GET /platform-result.png ")); + write_bytes(&mut stream, "image/png", &server_png); + } + _ => unreachable!(), + } + } + }); + + let result = derive_local_project_resource_at(fixture.request.clone()) + .await + .expect("complete platform account background removal"); + server.join().expect("join platform account server"); + let requests = captured.lock().expect("read platform requests"); + assert_eq!( + requests + .iter() + .filter(|request| request.starts_with("POST /api/editor/images/background-removals ")) + .count(), + 1 + ); + assert_eq!( + fs::read( + fixture + .root() + .join(result.asset.expect("platform asset").local_path) + ) + .expect("read platform result"), + generated_png + ); +} + +impl BackgroundRemovalFixture { + fn root(&self) -> &Path { + self.directory.path() + } +} + +fn test_png() -> Vec { + let mut output = Cursor::new(Vec::new()); + image::DynamicImage::new_rgba8(2, 2) + .write_to(&mut output, image::ImageFormat::Png) + .expect("encode test PNG"); + output.into_inner() +} + +#[test] +fn background_removal_download_rejects_non_png_and_png_without_alpha() { + let malformed = validate_downloaded_media( + &LocalProjectResourceEditKind::BackgroundRemoval, + "image/png", + b"not-a-png", + ) + .expect_err("background removal must reject non-PNG bytes"); + assert!(malformed.contains("PNG")); + + let mut rgb_png = Cursor::new(Vec::new()); + image::DynamicImage::ImageRgb8(image::RgbImage::new(2, 2)) + .write_to(&mut rgb_png, image::ImageFormat::Png) + .expect("encode RGB PNG"); + let missing_alpha = validate_downloaded_media( + &LocalProjectResourceEditKind::BackgroundRemoval, + "image/png", + rgb_png.get_ref(), + ) + .expect_err("background removal must reject PNG without alpha channel"); + assert!(missing_alpha.contains("透明通道")); +} + +fn background_removal_fixture(base_url: &str) -> BackgroundRemovalFixture { + background_removal_fixture_for_session(base_url, None) +} + +fn background_removal_fixture_for_session( + base_url: &str, + platform_session: Option<&PlatformSessionSnapshot>, +) -> BackgroundRemovalFixture { + let directory = tempfile::tempdir().expect("create background removal fixture"); + let root = directory.path(); + init_local_game_project_at(root, TEST_PROJECT_ID, "抠图账本闭环测试") + .expect("initialize local project"); + let source_bytes = test_png(); + let uploaded = upload_local_asset_at(root, "source.png", "image/png", &source_bytes) + .expect("register source image"); + let manifest = read_existing_manifest_for_project(root).expect("read source manifest"); + let source_asset = manifest + .assets + .iter() + .find(|asset| asset.id == uploaded.id) + .expect("find source asset") + .clone(); + let revision = read_game_creator_agent_runtime_project_revision(root) + .expect("read source project revision") + .revision; + let request = DeriveLocalProjectResourceInput { + project_path: root.to_string_lossy().into_owned(), + expected_project_id: TEST_PROJECT_ID.to_string(), + expected_project_revision: revision, + operation_id: Uuid::new_v4().to_string(), + idempotency_key: Uuid::new_v4().to_string(), + edit_kind: LocalProjectResourceEditKind::BackgroundRemoval, + generation_mode: LocalProjectResourceGenerationMode::Derive, + source_resource_id: format!("local-asset:{}", source_asset.id), + source_asset_id: Some(source_asset.id.clone()), + source_path: Some(source_asset.local_path.clone()), + source_media_type: Some(source_asset.media_type.clone()), + source_subtype: Some(source_asset.kind.clone()), + producer_task_id: source_asset.source.task_id.clone(), + source_version_id: None, + prompt: "去除背景".to_string(), + asset_name: "透明角色".to_string(), + background_mode: Some("flat".to_string()), + screen_color: Some("auto".to_string()), + }; + let source = resolve_resource_edit_source(root, &manifest, &request) + .expect("resolve background removal source"); + let bearer_token = platform_session + .map(|session| session.access_token.as_str()) + .unwrap_or(TEST_API_KEY); + let access = ExternalEditorBindingAccess::new(base_url, bearer_token, platform_session) + .expect("create developer access"); + let principal = + external_editor_binding_principal(&access).expect("resolve developer principal"); + write_external_editor_project_binding_at( + root, + &new_external_editor_project_binding( + TEST_PROJECT_ID, + &principal, + REMOTE_PROJECT_ID, + REMOTE_FOLDER_ID, + unix_timestamp(), + ) + .expect("build project binding"), + ) + .expect("write project binding"); + let source_identity = new_external_editor_source_identity( + &source_asset.id, + &source.source_sha256, + &source.media_type, + &source.asset_kind, + ) + .expect("build source identity"); + write_external_editor_resource_binding_at( + root, + &new_external_editor_resource_binding( + TEST_PROJECT_ID, + &principal, + REMOTE_PROJECT_ID, + &source_identity, + Some(SOURCE_RESOURCE_ID), + "registered/source.png", + "registered-source-object", + Some(2), + Some(2), + unix_timestamp(), + ) + .expect("build source binding"), + ) + .expect("write source binding"); + BackgroundRemovalFixture { + directory, + request, + source_asset_id: source_asset.id, + source_bytes, + } +} + +async fn with_test_credentials( + base_url: &str, + operation: impl std::future::Future, +) -> T { + let _platform_session_guard = clear_test_platform_session(); + crate::assets::with_external_editor_api_credentials( + crate::assets::external_editor_api_credentials_for_test( + base_url.to_string(), + TEST_API_KEY.to_string(), + ), + operation, + ) + .await +} + +fn accept_request(listener: &TcpListener, index: usize) -> TcpStream { + listener + .set_nonblocking(true) + .expect("set background removal listener nonblocking"); + let deadline = std::time::Instant::now() + Duration::from_secs(5); + loop { + match listener.accept() { + Ok((stream, _)) => { + stream + .set_nonblocking(false) + .expect("restore background removal stream blocking mode"); + return stream; + } + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + assert!( + std::time::Instant::now() < deadline, + "timed out waiting for background removal request {index}" + ); + std::thread::sleep(Duration::from_millis(5)); + } + Err(error) => panic!("accept background removal request {index}: {error}"), + } + } +} + +fn read_request(stream: &mut TcpStream) -> String { + stream + .set_read_timeout(Some(Duration::from_secs(2))) + .expect("set request read timeout"); + let mut bytes = Vec::new(); + let mut buffer = [0_u8; 4096]; + let (header_end, content_length) = loop { + let read = stream.read(&mut buffer).expect("read HTTP request"); + assert!(read > 0, "request closed before headers"); + bytes.extend_from_slice(&buffer[..read]); + let Some(header_end) = bytes.windows(4).position(|part| part == b"\r\n\r\n") else { + continue; + }; + let headers = String::from_utf8_lossy(&bytes[..header_end]); + let content_length = headers + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::().ok()) + .flatten() + }) + .unwrap_or(0); + break (header_end + 4, content_length); + }; + while bytes.len() < header_end + content_length { + let read = stream.read(&mut buffer).expect("read HTTP request body"); + assert!(read > 0, "request closed before body"); + bytes.extend_from_slice(&buffer[..read]); + } + String::from_utf8_lossy(&bytes).into_owned() +} + +fn write_json(stream: &mut TcpStream, status: &str, value: serde_json::Value) { + let body = value.to_string(); + write!( + stream, + "HTTP/1.1 {status}\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ) + .expect("write JSON response"); +} + +fn write_bytes(stream: &mut TcpStream, media_type: &str, bytes: &[u8]) { + write!( + stream, + "HTTP/1.1 200 OK\r\nContent-Type: {media_type}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", + bytes.len() + ) + .expect("write media headers"); + stream.write_all(bytes).expect("write media body"); +} + +fn completed_status(operation_id: &str) -> serde_json::Value { + serde_json::json!({"data": { + "operationId": operation_id, + "status": "completed", + "result": {"resource": { + "resourceId": "editor-resource-background-removal-result", + "projectId": REMOTE_PROJECT_ID, + "objectKey": "generated/background-removal.png", + "assetObjectId": "background-removal-result-object" + }} + }}) +} + +fn assert_submission(request: &str) { + assert!( + request.starts_with("POST /api/editor/images/background-removals ") + || request.starts_with("POST /api/external/v1/editor/images/background-removals ") + ); + let lower = request.to_ascii_lowercase(); + assert!(lower.contains("idempotency-key:")); + let body = request.split("\r\n\r\n").nth(1).expect("submission body"); + let payload: serde_json::Value = serde_json::from_str(body).expect("parse submission body"); + assert_eq!(payload["sourceImageSrc"], SOURCE_RESOURCE_ID); + assert_eq!(payload["sourceResourceId"], SOURCE_RESOURCE_ID); + assert_eq!(payload["projectId"], REMOTE_PROJECT_ID); + assert_eq!(payload["assetFolderId"], REMOTE_FOLDER_ID); + assert_eq!(payload["backgroundMode"], "flat"); + assert_eq!(payload["screenColor"], "auto"); +} + +fn assert_developer_submission(request: &str) { + assert!(request.starts_with("POST /api/external/v1/editor/images/background-removals ")); + assert!(request + .to_ascii_lowercase() + .contains("authorization: bearer background-removal-key")); + assert_submission(request); +} + +fn respond_canvas_context_request(stream: &mut TcpStream, request: &str) -> bool { + let line = request.lines().next().unwrap_or_default(); + if line.starts_with("GET /api/external/v1/editor/projects ") + || line.starts_with("GET /api/editor/projects ") + { + write_json( + stream, + "200 OK", + serde_json::json!({"data": {"projects": [{"projectId": REMOTE_PROJECT_ID}]}}), + ); + true + } else if line.starts_with("GET /api/external/v1/editor/assets/library ") + || line.starts_with("GET /api/editor/assets/library ") + { + write_json( + stream, + "200 OK", + serde_json::json!({"data": {"library": {"folders": [{"folderId": REMOTE_FOLDER_ID}]}}}), + ); + true + } else { + false + } +} + +#[tokio::test] +async fn background_removal_derives_through_poll_download_and_manifest_commit() { + let listener = TcpListener::bind("127.0.0.1:0").expect("bind background removal server"); + let base_url = format!("http://{}", listener.local_addr().expect("server address")); + let fixture = background_removal_fixture(&base_url); + let generated_png = test_png(); + let signed_url = format!("{base_url}/generated.png"); + let server_png = generated_png.clone(); + let server_signed_url = signed_url.clone(); + let requests = Arc::new(Mutex::new(Vec::new())); + let server_requests = Arc::clone(&requests); + let server = std::thread::spawn(move || { + for index in 0..8 { + let mut stream = accept_request(&listener, index); + let request = read_request(&mut stream); + server_requests + .lock() + .expect("capture requests") + .push(request.clone()); + let line = request.lines().next().unwrap_or_default(); + if respond_canvas_context_request(&mut stream, &request) { + continue; + } + match index { + 2 => { + assert_developer_submission(&request); + write_json( + &mut stream, + "202 Accepted", + serde_json::json!({"data": { + "operationId": "background-removal-operation", + "status": "queued", + "pollAfterMs": 0 + }}), + ); + } + 3 => write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": { + "operationId": "background-removal-operation", "status": "queued", "pollAfterMs": 0 + }}), + ), + 4 => write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": { + "operationId": "background-removal-operation", "status": "running", "pollAfterMs": 0 + }}), + ), + 5 => write_json( + &mut stream, + "200 OK", + completed_status("background-removal-operation"), + ), + 6 => { + assert!(line.starts_with("GET /api/external/v1/assets/read-url?")); + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": {"read": { + "signedUrl": server_signed_url + }}}), + ); + } + 7 => { + assert!(line.starts_with("GET /generated.png ")); + write_bytes(&mut stream, "image/png", &server_png); + } + _ => unreachable!(), + } + } + }); + + let result = with_test_credentials( + &base_url, + derive_local_project_resource_at(fixture.request.clone()), + ) + .await + .expect("complete background removal"); + server + .join() + .expect("join complete background removal server"); + let captured = requests.lock().expect("read captured requests"); + assert_eq!(captured.len(), 8); + assert_eq!( + captured + .iter() + .filter(|request| request.starts_with("POST ")) + .count(), + 1 + ); + let derived = result.asset.expect("derived background removal asset"); + assert_ne!(derived.id, fixture.source_asset_id); + assert_eq!( + fs::read(fixture.root().join(&derived.local_path)).expect("read result"), + generated_png + ); + assert_eq!( + fs::read( + fixture.root().join( + result + .manifest + .assets + .iter() + .find(|asset| asset.id == fixture.source_asset_id) + .expect("source preserved") + .local_path + .clone() + ) + ) + .expect("read source"), + fixture.source_bytes + ); + let ledger = read_resource_edit_ledger(fixture.root(), &fixture.request.operation_id) + .expect("read ledger") + .expect("ledger exists"); + assert_eq!(ledger.phase, ResourceEditLedgerPhase::Committed); + assert_eq!( + ledger.remote_canvas_project_id.as_deref(), + Some(REMOTE_PROJECT_ID) + ); +} + +#[tokio::test] +async fn background_removal_resumes_accepted_operation_without_reposting() { + let listener = TcpListener::bind("127.0.0.1:0").expect("bind resume server"); + let base_url = format!("http://{}", listener.local_addr().expect("server address")); + let fixture = background_removal_fixture(&base_url); + let generated_png = test_png(); + let signed_url = format!("{base_url}/resumed.png"); + let captured = Arc::new(Mutex::new(Vec::new())); + let server_captured = Arc::clone(&captured); + let server_png = generated_png.clone(); + let server = std::thread::spawn(move || { + for index in 0..7 { + let mut stream = accept_request(&listener, index); + let request = read_request(&mut stream); + server_captured + .lock() + .expect("capture resume requests") + .push(request.clone()); + let line = request.lines().next().unwrap_or_default(); + if respond_canvas_context_request(&mut stream, &request) { + continue; + } + match index { + 2 => { + assert_developer_submission(&request); + write_json( + &mut stream, + "202 Accepted", + serde_json::json!({"data": { + "operationId": "resume-background-removal", "status": "queued", "pollAfterMs": 0 + }}), + ); + } + 3 => write_json( + &mut stream, + "500 Internal Server Error", + serde_json::json!({"error": "interrupted"}), + ), + 4 => write_json( + &mut stream, + "200 OK", + completed_status("resume-background-removal"), + ), + 5 => { + assert!(line.starts_with("GET /api/external/v1/assets/read-url?")); + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": {"read": {"signedUrl": signed_url}}}), + ); + } + 6 => { + assert!(line.starts_with("GET /resumed.png ")); + write_bytes(&mut stream, "image/png", &server_png); + } + _ => unreachable!(), + } + } + }); + + let first_error = with_test_credentials( + &base_url, + derive_local_project_resource_at(fixture.request.clone()), + ) + .await + .expect_err("first polling attempt is interrupted"); + assert!(first_error.contains("result-unknown")); + let accepted = read_resource_edit_ledger(fixture.root(), &fixture.request.operation_id) + .expect("read accepted ledger") + .expect("accepted ledger exists"); + assert_eq!(accepted.phase, ResourceEditLedgerPhase::Accepted); + assert_eq!( + accepted.remote_operation_id.as_deref(), + Some("resume-background-removal") + ); + let pending = list_pending_local_project_resource_edits_for_session_at( + ListPendingLocalProjectResourceEditsInput { + project_path: fixture.root().to_string_lossy().into_owned(), + expected_project_id: TEST_PROJECT_ID.to_string(), + }, + None, + ) + .expect("list developer pending operations"); + assert_eq!(pending.len(), 1); + assert_eq!(pending[0].operation_id, fixture.request.operation_id); + assert_eq!(pending[0].background_mode.as_deref(), Some("flat")); + assert_eq!(pending[0].screen_color.as_deref(), Some("auto")); + let result = with_test_credentials( + &base_url, + resume_local_project_resource_edit_at(ResumeLocalProjectResourceEditInput { + project_path: fixture.root().to_string_lossy().into_owned(), + expected_project_id: TEST_PROJECT_ID.to_string(), + operation_id: fixture.request.operation_id.clone(), + }), + ) + .await + .expect("resume accepted background removal"); + server.join().expect("join resume server"); + let captured = captured.lock().expect("read resume requests"); + assert_eq!( + captured + .iter() + .filter(|request| request.starts_with("POST ")) + .count(), + 1 + ); + assert_eq!(result.manifest.assets.len(), 2); + assert!(list_pending_local_project_resource_edits_for_session_at( + ListPendingLocalProjectResourceEditsInput { + project_path: fixture.root().to_string_lossy().into_owned(), + expected_project_id: TEST_PROJECT_ID.to_string(), + }, + None, + ) + .expect("list pending after resume") + .is_empty()); + assert_eq!( + fs::read( + fixture + .root() + .join(result.asset.expect("resumed asset").local_path) + ) + .expect("read resumed result"), + generated_png + ); +} + +#[tokio::test] +async fn background_removal_remote_failure_keeps_manifest_without_result() { + let listener = TcpListener::bind("127.0.0.1:0").expect("bind failure server"); + let base_url = format!("http://{}", listener.local_addr().expect("server address")); + let fixture = background_removal_fixture(&base_url); + let server = std::thread::spawn(move || { + for index in 0..4 { + let mut stream = accept_request(&listener, index); + let request = read_request(&mut stream); + if respond_canvas_context_request(&mut stream, &request) { + continue; + } + if index == 2 { + assert_developer_submission(&request); + write_json( + &mut stream, + "202 Accepted", + serde_json::json!({"data": { + "operationId": "failed-background-removal", "status": "queued", "pollAfterMs": 0 + }}), + ); + } else { + assert!(request + .starts_with("GET /api/external/v1/generations/failed-background-removal ")); + write_json( + &mut stream, + "200 OK", + serde_json::json!({"data": { + "operationId": "failed-background-removal", "status": "failed" + }}), + ); + } + } + }); + + let error = with_test_credentials( + &base_url, + derive_local_project_resource_at(fixture.request.clone()), + ) + .await + .expect_err("remote failure must be returned"); + server.join().expect("join failure server"); + assert!(error.contains("remote-terminal-failed")); + let manifest = + read_existing_manifest_for_project(fixture.root()).expect("read unchanged manifest"); + assert_eq!(manifest.assets.len(), 1); + assert_eq!(manifest.assets[0].id, fixture.source_asset_id); + let ledger = read_resource_edit_ledger(fixture.root(), &fixture.request.operation_id) + .expect("read failed ledger") + .expect("failed ledger exists"); + assert_eq!(ledger.phase, ResourceEditLedgerPhase::RemoteFailed); + assert!(ledger.result_asset_id.is_none()); + assert!( + read_optional_resource_edit_staging(fixture.root(), &fixture.request.operation_id) + .expect("read absent staging") + .is_none() + ); +} diff --git a/apps/ai-game-creator-shell/src/view/project-development/index.tsx b/apps/ai-game-creator-shell/src/view/project-development/index.tsx index 1bd650976..184007760 100644 --- a/apps/ai-game-creator-shell/src/view/project-development/index.tsx +++ b/apps/ai-game-creator-shell/src/view/project-development/index.tsx @@ -44,6 +44,7 @@ import { Replace, RotateCcw, Search, + Shapes, SlidersHorizontal, Sparkles, Trash2, @@ -427,6 +428,8 @@ type PendingLocalProjectResourceEdit = { editKind: string; sourceResourceId: string; assetName: string; + backgroundMode?: string | null; + screenColor?: string | null; phase: string; createdAt: number; }; @@ -467,8 +470,20 @@ type ResourceEditServiceIdentityConfirmation = { expiresAt: number; }; -function pendingResourceEditKindLabel(editKind: string) { +function pendingResourceEditKindLabel({ + editKind, + backgroundMode, + screenColor, +}: PendingLocalProjectResourceEdit) { if (editKind === 'image') return '图片编辑'; + if (editKind === 'background-removal') { + if (backgroundMode === 'complex') return '图片抠图 · 复杂背景'; + if (backgroundMode === 'flat') { + const color = screenColor === 'auto' ? '自动背景色' : screenColor; + return `图片抠图 · 平面背景${color ? ` · ${color}` : ''}`; + } + return '图片抠图'; + } if (editKind === 'text') return '文本编辑'; if (editKind === 'agent-result') return '智能体结果编辑'; return '资源编辑'; @@ -9268,7 +9283,7 @@ export default function ProjectDevelopmentView({
{pending.assetName} - {pendingResourceEditKindLabel(pending.editKind)} ·{' '} + {pendingResourceEditKindLabel(pending)} ·{' '} {pendingResourceEditCreatedAtLabel(pending.createdAt)} diff --git a/apps/ai-game-creator-shell/tests/appSurface/home.suite.ts b/apps/ai-game-creator-shell/tests/appSurface/home.suite.ts index a31d69f89..475242de0 100644 --- a/apps/ai-game-creator-shell/tests/appSurface/home.suite.ts +++ b/apps/ai-game-creator-shell/tests/appSurface/home.suite.ts @@ -411,9 +411,7 @@ export function registerClientHomeTests() { await openResourceBookCategory('UI 交互'); expect(await findResourceSelectButton('live-hero.png')).not.toBeNull(); await openResourceBookCategory('项目版本'); - expect( - await findResourceSelectButton('版本 1'), - ).not.toBeNull(); + expect(await findResourceSelectButton('版本 1')).not.toBeNull(); expect(runButton.getAttribute('data-unavailable')).toBeNull(); await waitFor(() => { expect(invoke).toHaveBeenCalledWith( @@ -582,9 +580,7 @@ export function registerClientHomeTests() { }), ).not.toBeNull(); await openResourceBookCategory('项目版本'); - expect( - await findResourceSelectButton('版本 1'), - ).not.toBeNull(); + expect(await findResourceSelectButton('版本 1')).not.toBeNull(); expect(runButton.getAttribute('data-unavailable')).toBeNull(); await waitFor(() => { expect( diff --git a/apps/ai-game-creator-shell/tests/appSurface/project-development.suite.ts b/apps/ai-game-creator-shell/tests/appSurface/project-development.suite.ts index ae09f6798..18bda4813 100644 --- a/apps/ai-game-creator-shell/tests/appSurface/project-development.suite.ts +++ b/apps/ai-game-creator-shell/tests/appSurface/project-development.suite.ts @@ -3836,7 +3836,9 @@ export function registerProjectWorkbenchFoundationTests() { ]; await openResourceBookCategory('角色与对象'); - const infoButton = await screen.findByRole('button', { name: '查看hero.png资源信息' }); + const infoButton = await screen.findByRole('button', { + name: '查看hero.png资源信息', + }); expect(infoButton.getAttribute('aria-pressed')).toBe('false'); // 未选中的卡片直接打开信息，不被选中变化 effect 立即关闭。 @@ -4096,13 +4098,7 @@ export function registerProjectWorkbenchFoundationTests() { within(audioToolbar) .getAllByRole('button') .map((button) => button.getAttribute('aria-label')), - ).toEqual([ - '引用资源 bgm.mp3', - '编辑标签', - '重命名', - '导出', - '删除素材', - ]); + ).toEqual(['引用资源 bgm.mp3', '编辑标签', '重命名', '导出', '删除素材']); // 工具条的「导出」必须真的走通落盘链路：原生保存对话框 + Rust 分块复制， // 而不是只渲染一个按钮。原生对话框由入口文件 mock 成"用户选了 diff --git a/apps/ai-game-creator-shell/tests/directActiveTurns.test.tsx b/apps/ai-game-creator-shell/tests/directActiveTurns.test.tsx index f7acdba4e..043918975 100644 --- a/apps/ai-game-creator-shell/tests/directActiveTurns.test.tsx +++ b/apps/ai-game-creator-shell/tests/directActiveTurns.test.tsx @@ -51,9 +51,11 @@ describe('useDirectActiveTurns', () => { }); it('clears to a stable empty snapshot when the hook is disabled', async () => { - const invoke = vi.fn( - async () => [] as GameCreatorDirectActiveTurn[], - ) as never; + let resolveSnapshot!: (turns: GameCreatorDirectActiveTurn[]) => void; + const snapshot = new Promise((resolve) => { + resolveSnapshot = resolve; + }); + const invoke = vi.fn(() => snapshot) as never; const { result, rerender } = renderHook( ({ enabled }: { enabled: boolean }) => useDirectActiveTurns({ invoke, enabled, pollIntervalMs: 60_000 }), @@ -62,7 +64,8 @@ describe('useDirectActiveTurns', () => { const emptySnapshot = result.current.activeTurns; await act(async () => { - await result.current.refreshActiveTurns(); + resolveSnapshot([]); + await snapshot; }); expect(result.current.activeTurns).toBe(emptySnapshot); rerender({ enabled: false }); diff --git a/apps/ai-game-creator-shell/tests/projectResourceLiveIntegration.test.tsx b/apps/ai-game-creator-shell/tests/projectResourceLiveIntegration.test.tsx index 4234d82b8..ef129cd12 100644 --- a/apps/ai-game-creator-shell/tests/projectResourceLiveIntegration.test.tsx +++ b/apps/ai-game-creator-shell/tests/projectResourceLiveIntegration.test.tsx @@ -109,6 +109,8 @@ type PendingResourceEditFixture = { editKind: string; sourceResourceId: string; assetName: string; + backgroundMode?: string | null; + screenColor?: string | null; phase: string; createdAt: number; }; @@ -1294,6 +1296,42 @@ describe('project resource live canvas integration', () => { }); }); + it('恢复面板显示同名抠图的模式和颜色，缺失字段不推断默认值', async () => { + installTauri({ + pendingResourceEdits: [ + { backgroundMode: 'complex' }, + { backgroundMode: 'flat', screenColor: 'auto' }, + { backgroundMode: 'flat', screenColor: '#AABBCC' }, + { backgroundMode: 'flat' }, + {}, + ].map((options, index) => ({ + operationId: `background-${index}`, + editKind: 'background-removal', + sourceResourceId: 'source-art', + assetName: '透明底', + phase: 'accepted', + createdAt: 1, + ...options, + })), + }); + render(); + fireEvent.click( + await screen.findByRole('button', { name: '管理未完成编辑 (5)' }), + ); + const labels = screen + .getAllByText('透明底') + .map((name) => name.nextElementSibling?.textContent); + expect( + labels.map((label) => label?.split(' · ').slice(0, -1).join(' · ')), + ).toEqual([ + '图片抠图 · 复杂背景', + '图片抠图 · 平面背景 · 自动背景色', + '图片抠图 · 平面背景 · #AABBCC', + '图片抠图 · 平面背景', + '图片抠图', + ]); + }); + it('恢复面板可跳过首条失败任务继续任意 operation，且对账项不会被重放', async () => { const failedOperationId = '11111111-1111-4111-8111-111111111111'; const resumableOperationId = '22222222-2222-4222-8222-222222222222'; @@ -2072,7 +2110,9 @@ describe('project resource live canvas integration', () => { await openResourceBookCategory('角色与对象'); fireEvent.click(await findResourceSelectButton('hero.png')); const toolbar = await screen.findByRole('toolbar', { name: '图片工具栏' }); - fireEvent.click(screen.getByRole('button', { name: '查看hero.png资源信息' })); + fireEvent.click( + screen.getByRole('button', { name: '查看hero.png资源信息' }), + ); const infoPanel = await screen.findByRole('dialog', { name: '资源信息' }); // 分类值本身仍是只读文本（`dd` 里只有值，入口按钮在它外面）。 diff --git a/apps/ai-game-creator-shell/tests/resourceVersionReplacement.test.tsx b/apps/ai-game-creator-shell/tests/resourceVersionReplacement.test.tsx index cfbc324f9..e8f2b5e4b 100644 --- a/apps/ai-game-creator-shell/tests/resourceVersionReplacement.test.tsx +++ b/apps/ai-game-creator-shell/tests/resourceVersionReplacement.test.tsx @@ -379,7 +379,10 @@ function toolbarAction(toolbar: HTMLElement, name: string) { const visible = within(toolbar).queryByRole('button', { name }); if (visible) return visible; fireEvent.mouseEnter(within(toolbar).getByRole('button', { name: '更多' })); - return within(screen.getByRole('group', { name: '更多操作' })).getByRole('button', { name }); + return within(screen.getByRole('group', { name: '更多操作' })).getByRole( + 'button', + { name }, + ); } async function selectCardAndOpenToolbar(label: string) { @@ -550,12 +553,20 @@ describe('版本级资源替换', () => { const toolbar = await selectCardAndOpenToolbar('legacy.png'); fireEvent.mouseEnter(within(toolbar).getByRole('button', { name: '更多' })); const menu = screen.getByRole('group', { name: '更多操作' }); - const viewport = () => document.querySelector('[data-resource-viewport]') - ?.getAttribute('data-resource-viewport'); + const viewport = () => + document + .querySelector('[data-resource-viewport]') + ?.getAttribute('data-resource-viewport'); const before = viewport(); expect(before).toBeTruthy(); - const wheel = new WheelEvent('wheel', { bubbles: true, cancelable: true, deltaY: 120 }); - act(() => { menu.dispatchEvent(wheel); }); + const wheel = new WheelEvent('wheel', { + bubbles: true, + cancelable: true, + deltaY: 120, + }); + act(() => { + menu.dispatchEvent(wheel); + }); expect(wheel.defaultPrevented).toBe(false); expect(viewport()).toBe(before); @@ -568,9 +579,15 @@ describe('版本级资源替换', () => { const scene = document.querySelector('.game-resource-book-scene')!; act(() => { - scene.dispatchEvent(new WheelEvent('wheel', { - bubbles: true, cancelable: true, deltaY: 120, clientX: 90, clientY: 70, - })); + scene.dispatchEvent( + new WheelEvent('wheel', { + bubbles: true, + cancelable: true, + deltaY: 120, + clientX: 90, + clientY: 70, + }), + ); }); expect(viewport()).not.toBe(before); }); @@ -578,13 +595,17 @@ describe('版本级资源替换', () => { it('信息从未选中卡打开并跟随资源身份，普通换选会关闭', async () => { renderReplacementWorkbench(); await selectCardAndOpenToolbar('legacy.png'); - const lateInfo = screen.getByRole('button', { name: '查看late.png资源信息' }); + const lateInfo = screen.getByRole('button', { + name: '查看late.png资源信息', + }); fireEvent.pointerDown(lateInfo, { button: 0 }); fireEvent.click(lateInfo); const panel = screen.getByRole('dialog', { name: '资源信息' }); expect(within(panel).getByText('late.png')).toBeTruthy(); expect(lateInfo.getAttribute('aria-pressed')).toBe('true'); - fireEvent.click(screen.getByRole('button', { name: '查看legacy.png资源信息' })); + fireEvent.click( + screen.getByRole('button', { name: '查看legacy.png资源信息' }), + ); const switched = screen.getByRole('dialog', { name: '资源信息' }); expect(within(switched).getByText('legacy.png')).toBeTruthy(); expect(within(switched).queryByText('late.png')).toBeNull(); @@ -597,10 +618,10 @@ describe('版本级资源替换', () => { // 未被初始版本绑定的素材（版本创建之后才登记）：工具条照常出现，但没有「替换素材」。 const lateToolbar = await selectCardAndOpenToolbar('late.png'); - fireEvent.mouseEnter(within(lateToolbar).getByRole('button', { name: '更多' })); - expect( - screen.queryByRole('button', { name: '替换素材' }), - ).toBeNull(); + fireEvent.mouseEnter( + within(lateToolbar).getByRole('button', { name: '更多' }), + ); + expect(screen.queryByRole('button', { name: '替换素材' })).toBeNull(); expect( within(lateToolbar).getByRole('button', { name: '快速编辑' }), ).not.toBeNull(); @@ -614,9 +635,7 @@ describe('版本级资源替换', () => { // 被当前版本绑定的素材：入口出现。 const sourceToolbar = await selectCardAndOpenToolbar('legacy.png'); - expect( - toolbarAction(sourceToolbar, '替换素材'), - ).not.toBeNull(); + expect(toolbarAction(sourceToolbar, '替换素材')).not.toBeNull(); }); it('从入口一路走到写入：候选弹窗禁用硬门禁项、给出格式提示、直接替换且不产生新版本', async () => { @@ -895,9 +914,10 @@ describe('版本级资源替换', () => { const deleteButton = toolbarAction(toolbar, '删除素材'); const toolbarLabels = [ ...within(toolbar).getAllByRole('button'), - ...within(screen.getByRole('group', { name: '更多操作' })).getAllByRole('button'), - ] - .map((button) => button.getAttribute('aria-label') ?? ''); + ...within(screen.getByRole('group', { name: '更多操作' })).getAllByRole( + 'button', + ), + ].map((button) => button.getAttribute('aria-label') ?? ''); // 末位：在最后一个非破坏性动作（替换素材）之后、共享导出按钮之前。 expect(toolbarLabels.indexOf('删除素材')).toBeGreaterThan( toolbarLabels.indexOf('替换素材'), @@ -1411,9 +1431,7 @@ describe('版本级资源替换', () => { // 第一次：legacy → final。 const legacyToolbar = await selectCardAndOpenToolbar('legacy.png'); - fireEvent.click( - toolbarAction(legacyToolbar, '替换素材'), - ); + fireEvent.click(toolbarAction(legacyToolbar, '替换素材')); let dialog = await screen.findByRole('dialog', { name: '选择替换素材', }); @@ -1431,9 +1449,7 @@ describe('版本级资源替换', () => { // 第二次：final → final.webp（同一个工作台会话内）。 const finalToolbar = await selectCardAndOpenToolbar('final.png'); - fireEvent.click( - toolbarAction(finalToolbar, '替换素材'), - ); + fireEvent.click(toolbarAction(finalToolbar, '替换素材')); dialog = await screen.findByRole('dialog', { name: '选择替换素材' }); fireEvent.click( within(dialog).getByRole('option', { name: '选择替换素材final.webp' }), diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index 1b57435a0..0be34b555 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -2,6 +2,13 @@ > 用途：记录已经确认、会影响后续开发的长期技术/产品/协作决策。短期讨论不要写在这里。 > 当前口径：历史条目的旧路径、旧版本和已退役对象只用于追溯，不构成现行实现依据；如与当前代码或 `docs/README.md` 冲突，以当前代码和最新专题文档为准。 + +## 2026-09-17 AGC 抠图提交使用远端画布项目身份 + +- 背景：AGC 已通过本地项目 ID 建立并持久化本地项目到主站远端画布项目的绑定，但 `agc_remove_background` 提交请求仍把本地 `manifest.project_id` 放入 `projectId`；`assetFolderId` 已使用远端素材目录 ID。主站因此按项目不存在或不属于当前账号返回 404，主站抠图和 BgFilter 本身均正常。 +- 决策：抠图请求及工具回执统一使用 `prepare_external_canvas_generation_context` 返回的远端 `context.project_id`；本地 manifest 项目 ID 只用于绑定键和本地状态，不得作为主站业务请求的 `projectId`。 +- 验证：客户端定向 Rust 测试、格式、编码和 diff 检查通过；未修改主站路由或 BgFilter。 + ## 2026-09-17 图集切分模式改为显式声明 - 决策：`sliceMode` 在图标图集生成入口成为必填字段且不保留任何默认值。省略、`null` 或空字符串必须在引用解析、定价、入队和 provider / OSS 副作用之前返回 `400`（`field=sliceMode`）；`grid` 必须同时提供 `gridX`/`gridY`，`connected-components` 不得携带网格尺寸，二者矛盾同样在副作用前失败关闭。 @@ -22,6 +29,7 @@ - 影响范围：`apps/ai-game-creator-shell/src-tauri/src/project/resource_editor.rs`（上限与文案的唯一口径）、`agent/direct_tool_bridge.rs`（按 kind 判定与未登记源资源提示）、`agent/direct_tools_mcp.rs`（schema 与校验）、`resources/agc-skills/agc-client-projection/**` 与清单指纹（version `2026-08-26.18`）。**未改** `/api/external/v1` 契约与 OpenAPI、SpacetimeDB schema、前端 TS 侧 `resourceEditPromptMaxLength` 数字、客户端 UI 行为。 - 验证方式：新增 `tool_prompt_limits_agree_with_the_client_authority`（四个 kind 的 schema 上限、MCP 校验与客户端权威口径同数字，超限文案带真实上限）、`bridge_resource_prompt_limits_follow_the_client_authority`（工具桥侧同类门禁，含图片编辑的 32000 边界）、`edit_image_tool_reaches_the_platform_image_edit_route` 与 `background_music_tool_reaches_the_platform_audio_route`（MCP 工具层 → 真实工具桥 → 假平台，断言 `/api/editor/images/edits` 与 `/api/editor/audios/background-music/generations` 的路径、Bearer、Idempotency-Key、正文与派生资源落盘，图片编辑正文不得回填 assetKind）、`background_music_prompt_over_the_limit_is_rejected_before_any_bridge_call`（超限在桥请求之前失败）、`unregistered_source_reports_the_registration_follow_up_tools`；`agent::direct_tools_mcp` 22 passed、`agent::skill_pack` 4 passed、`agent::direct_tool_bridge` 17 passed（7 条本机既有失败见下）、`npm run agc:skill-pack:check` 与 `skill-pack:test` 通过。本机 `tempfile::tempdir()` 归属校验失败导致的既有用例（`project::resource_editor` 45 条、`agent::direct_tool_bridge` 7 条）在本轮改动前后**同为失败**（stash 基线复跑确认），与本次无关。 - 关联文档：[AI游戏创作智能体App实施计划](../../technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md)、[踩坑记录](pitfalls.md)。 + ## 2026-09-17 资源画布支持引擎资源只读预览 - 背景：Cocos Creator 工程里已有的引擎资源（模型、动画、预制体、材质、图集、压缩纹理…）此前在发现层就止步：`.glb` / `.prefab` / `.anim` / `.texture` 等扩展名既不可登记，也不进资源画布，工程导入后画布上只看得到位图、音频与脚本。 @@ -8211,8 +8219,8 @@ CI 上 `background_agent_runtime_recovers_stale_running_before_pending_task` 在 ## 2026-08-24 AGC Direct 抠图语义工具 -- 决策：将 External v1 `/api/external/v1/editor/images/background-removals` 通过 `agc_remove_background` 加入受控 `agc_tools`。工具只接受当前 manifest 的图片 `sourceLocalAssetId` 与结果名称；客户端负责正式 resourceId、画布/素材目录、稳定 operation/idempotency 身份、权限和错误脱敏，不向 Codex 暴露内部 BgFilter worker、凭据或任意 API。 -- 约束：异步结果只投影有界队列状态，不允许模型自行构造源 URL 或在不确定提交后更换请求身份；External v1 负责 API Key、幂等接收与统一 operation 查询，客户端不得绕过该契约。 +- 决策：将抠图能力通过 `agc_remove_background` 加入受控 `agc_tools`。工具只接受当前 manifest 的图片 `sourceLocalAssetId` 与结果名称；普通登录态使用账号鉴权的 `/api/editor/images/background-removals`，ExternalDeveloper 模式使用 External v1 `/api/external/v1/editor/images/background-removals`。客户端负责正式 resourceId、画布/素材目录、稳定 operation/idempotency 身份、权限和错误脱敏，不向 Codex 暴露内部 BgFilter worker、凭据或任意 API。 +- 约束：异步结果与恢复语义以本文「2026-09-17 AGC 抠图接入本地资源编辑恢复闭环」决策为准。不允许模型自行构造源 URL 或在不确定提交后更换请求身份；两种路由都接收客户端稳定幂等身份，External v1 继续负责 API Key、幂等接收与统一 operation 查询，客户端不得绕过该契约。 ## 2026-08-24 资源详情动作、空态滚动与最终图多步恢复 @@ -8865,6 +8873,14 @@ CI 上 `background_agent_runtime_recovers_stale_running_before_pending_task` 在 - 验证：限速后 `Genarrative-Full-Build-And-Deploy` #289 / #290 SUCCESS；采样期 Jenkins 峰值 10.2~10.5 核、限流不足 2s（可忽略），runner 峰值 12.07 核且持续出现 throttling，整机回落到 2.6%~19.8%。 - 关联文档：[开发运维](../../【开发运维】本地开发验证与生产运维-2026-05-15.md)。 +## 2026-09-17 AGC 抠图接入本地资源编辑恢复闭环 + +- 背景：`agc_remove_background` 原先只提交 `/api/editor/images/background-removals` 并返回 `queued`，没有轮询远端任务、下载完成媒体或写入本地 manifest；BgFilter 已成功处理但 Agent 因此永远只能看到受理回执。 +- 决策：抠图作为 `LocalProjectResourceEditKind::BackgroundRemoval` 接入现有资源编辑账本，模式和背景色写入 operation 身份；提交后复用同一套轮询、结果下载、staging、manifest 提交和恢复逻辑。已有账本优先恢复，禁止在未知结果时换 operation/idempotency 重发。 +- 边界：主站异步队列、BgFilter 和 SpacetimeDB schema 不变；Agent 只获得本地完成资源和安全身份投影，不接触内部 worker 或凭据。 +- 恢复：已受理任务中断后按原 operation 续查；提交结果不确定时保留账本并人工对账。升级前无账本的 queued 回执不自动迁移或重发，已有远端成果通过正式素材导入恢复。 +- 展示：恢复面板按账本显示抠图模式，平面背景模式同时显示已记录的自动背景色或颜色值，缺失字段不推断默认值，帮助区分同名待处理任务。 + ## 2026-09-17 Jenkins 公网入口 jenkins.genarrative.world 复用 router 反向隧道口径 - 背景：Jenkins controller 实际与 Gitea 同机运行在 `genarrative-station`（`jenkins.service`，`--httpPort=8080 --prefix=/jenkins`，`JENKINS_HOME=/var/lib/jenkins`），此前只有内网入口 `http://192.168.35.82:8080/jenkins/`；`router.genarrative.world` 已有「dev Nginx → dev loopback → station 反向隧道」的成熟口径。 diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index d59e89037..2a440a3b8 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -1,5 +1,9 @@ # 踩坑与排障记录 +## AGC 空快照测试必须等待请求完成 + +`waitFor(() => expect(activeTurns).toEqual([]))` 在 Hook 初始状态就能成功，不能证明首次异步读取已经完成。引用稳定性回归应显式控制 Promise 完成，并同时检查首次空响应与禁用后的引用；快照签名初值必须与初始空数组一致。窗口同步测试应验证未变化状态不重复发布，不能依赖一次多余的空态更新。 + ## AGC Windows 开发态首次页面加载缓慢 Vite 默认监听应用根下的 Rust `src-tauri/target`，构建产物较多时会创建大量 Windows 文件监听器。AGC 配置通过 `server.watch.ignored: ['**/src-tauri/target/**']` 排除此目录，不关闭业务源码、CSS、共享组件监听或 HMR。排查时区分后端就绪、Vite 扫描和原生窗口首绘；监听目录回归不能代替实机首绘测量，验证入口见本地开发运维文档。 diff --git a/docs/technical/【技术方案】AGC抠图模式与背景色透传-2026-09-16.md b/docs/technical/【技术方案】AGC抠图模式与背景色透传-2026-09-16.md index a7e9525fd..f32fcffa6 100644 --- a/docs/technical/【技术方案】AGC抠图模式与背景色透传-2026-09-16.md +++ b/docs/technical/【技术方案】AGC抠图模式与背景色透传-2026-09-16.md @@ -61,10 +61,25 @@ OpenAPI 与客户端工具的对外说明只描述模式用途、参数约束和 } ``` -客户端保留旧参数调用；新字段不填时不改变旧调用语义。客户端不读取图片、不自动选色、不把 `auto` 改写为具体颜色，使用原有 Bearer 认证、幂等键和队列返回模型。 +客户端保留旧参数调用；新字段不填时不改变模式语义。客户端不自动选色、不把 `auto` 改写为具体颜色；源资源校验、认证、异步任务查询、结果下载和本地登记由客户端负责。 工具 schema、桥接参数校验和随包 `agc-client-projection` Skill/契约说明必须保持一致。模式与颜色属于请求意图，必须参与客户端幂等指纹；同一图片与名称的不同模式不能复用同一次请求。缺省 complex 且没有颜色时保留既有指纹。主站在默认值归一化之前计算 External 请求指纹，缺失的新字段不序列化，避免旧请求重放发生冲突。 +客户端提交前建立的本地项目绑定会返回主站远端 `projectId` 与 `assetFolderId`，抠图请求必须使用这两个远端身份；本地 manifest `projectId` 仅用于绑定和本地状态，不能直接提交给主站。 + +### 本地结果与恢复合同 + +`agc_remove_background` 复用本地资源编辑账本，类型为 `background-removal`。源图片保持不变，抠图结果作为新资源写入项目。模式和背景色随账本持久化并参与请求指纹，其它编辑类型的历史指纹保持不变。 + +1. 客户端建立源图片的正式资源绑定，在提交前保存 operation、幂等键和请求意图。普通登录态提交 `/api/editor/images/background-removals`，从 `data.queueState.operationId` 读取受理身份；开发者模式提交 External v1 对应路由，从 `data.operationId` 读取身份。 +2. 受理后持续查询账号路由 `/api/runtime/external-generation/jobs/{operationId}` 或 External v1 对应状态路由。`queued`、`running` 只描述远端返回状态；固定进度值、本地文件缺失或 pending 清单为空均不能证明 BgFilter 排队。 +3. 远端 completed 后按稳定资源身份换取有效下载 URL，校验结果为带 alpha 通道的有效 PNG，随后复用 staging、manifest 和 revision 提交。只在本地登记完成后向 Agent 返回 `completed`、`operationId`、`resource.localAssetId`、相对路径和安全告警，不暴露临时 URL 或凭据。 +4. 轮询中断、超时或下载失败保留已受理 operation；`agc_list_registered_assets.pendingOperations` 与客户端恢复面板可见。相同源资源、结果名称、模式和颜色的后续调用优先恢复同一任务，不再次提交。不同账号不能恢复原账号任务；切回原账号后按既有恢复规则续接。 +5. 远端 failed 明确失败；提交响应不确定且无法确认 operation 时进入人工对账状态，不自动换键重发。失败/未知均不得伪造透明图或自动切换本地抠图方式。 +6. 升级前仅返回 queued、没有本地账本的任务不自动迁移；已有远端结果须通过正式资源查询和导入恢复，不据旧回执重新发起付费请求。 + +本修复只扩展 AGC 客户端现有工作流，不修改主站队列、BgFilter 或 SpacetimeDB schema。验收覆盖账号与开发者两种响应封装、queued/running/completed、已受理中断恢复不重复 POST、远端失败不登记结果，以及既有资源编辑回归。 + ## 实施任务 ### 任务一：冻结 BgFilter 契约 @@ -93,6 +108,13 @@ OpenAPI 与客户端工具的对外说明只描述模式用途、参数约束和 ## 验收证据 +2026-09-17 客户端闭环验证： + +- `cargo test --locked --manifest-path apps/ai-game-creator-shell/src-tauri/Cargo.toml --bin genarrative-ai-game-creator-shell project::resource_editor:: -- --test-threads=1` 通过。新增 HTTP fixture 覆盖开发者 queued/running/completed、账号 HTTP 200 queueState 与 `data.job`、换签下载、PNG alpha 校验、原图保留和新资源提交。 +- 已受理任务的首次轮询失败后，pending 保留模式与颜色；恢复只查询原 operation，整个流程只 POST 一次，成功后清除 pending。远端 failed 不新增结果资源。既有账号隔离、提交原子性和崩溃恢复用例通过。 +- `cargo test --locked --manifest-path apps/ai-game-creator-shell/src-tauri/Cargo.toml --bin genarrative-ai-game-creator-shell agent::direct_tool_bridge::tests -- --test-threads=1`、AGC 类型检查、技能包校验、Rust 格式、编码、文档索引与 diff 检查通过。 +- 本轮未运行真实登录客户端 → 本地主站 → BgFilter 的端到端 smoke；当前本地后端已停止，自动化证据使用模拟 HTTP 服务。此前 BgFilter 成功日志只证明上游处理完成，不证明主站结果持久化或客户端导入成功。 + 2026-09-16 实测： - 主站 `cargo test -p api-server background_removal`：36 项通过，覆盖非法请求入队前拒绝、缺省 complex、队列参数保留、旧请求指纹、父侧内部 RPC 和 provider multipart。 diff --git a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md index 77c5d8108..3c864437b 100644 --- a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md +++ b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md @@ -12,6 +12,9 @@ ## 资源画布交互与工作台状态同步 +- 未完成抠图的恢复项在现有类型行显示账本中的复杂/平面背景模式；平面模式显示已记录的自动背景色或颜色值，缺失模式/颜色不补默认值，恢复仍按原 operation 身份执行。 +- 活动回合初始空快照、首次成功读取的空结果及禁用后的空态保持同一数组引用；快照签名初值与空态重置值均为 `[]`。无原生 invoke 的窗口测试只期待首次状态发布，异步空结果测试显式控制请求完成，不以初始空数组作为请求已完成的证据。 + - 活动回合轮询的初始空快照与后续空结果保持同一引用；停用或切换读取器使旧请求失效，晚到快照不得恢复已停用的活动回合或覆盖新轮询结果。无原生读取器时窗口只发布一次空状态，不通过额外空数组触发重复发布。 - 工作台向窗口标题栏发布正在运行的项目时，输入未变化不得形成重复发布与清理的渲染循环；打开项目动作始终使用当前工作台处理逻辑，退出工作台后清除其标题栏状态。 - 资源子画布（含「所有资源」）保留空白处左键框选、资源卡左键选中/拖动、触摸板双指平移及捏合缩放；右键按住空白处或资源卡拖动时平移画布，不改变资源选择与布局。中键和空格抓手继续可用。总览保留既有左键平移，并支持右键平移。 @@ -167,7 +170,7 @@ npm 游戏的可预览产物固定为对应 package 目录下的 `dist/index.htm - 项目路径、projectId、当前 revision、源文件路径与媒体类型、operationId、Idempotency-Key、登录态、项目锁、付费提交、轮询恢复、下载校验与 manifest 事务全部由客户端持有。模型不能提交或覆盖这些字段。同一 Direct `clientTurnId + 规范语义参数` 生成稳定 UUID v4 身份；单回合同参重试复用原 operation，不同请求串行且最多四项。跨回合存在完全匹配的 pending 账本时优先恢复原 operation，不能换键重发。 - 资源查询同时投影未完成 operation 的安全状态。媒体工具成功只返回 operation、本地相对路径、资源类型、Canvas/resource/asset/task 身份、正式序列帧以及脱敏后的 `warnings / sliceWarnings`；错误继续使用统一脱敏边界。客户端资源账本持久化 completed 结果的两类告警，committed replay 不能把历史告警伪装成空集合。 - 角色动画、视频、音效和背景音乐在构造新的远端请求前统一准备当前项目同名画布与素材目录上下文，并在端点支持时携带 `projectId / assetFolderId / canvasCompletion`。角色动画 placeholder 使用源图片真实宽高，避免非方形角色进入画布时失真；正式 resource/asset 与序列帧继续直接复用 External 返回身份，不从首帧伪造重复资源。已有冻结 request body 或已受理 operation 保持不变，不因本次升级重建请求或重复扣费。 -- 抠图通过新增 `agc_remove_background` 语义工具开放：模型只提交当前 manifest 的图片 `sourceLocalAssetId` 与结果名称；客户端解析稳定 `resourceId`，准备同名画布/素材目录并生成稳定 operation/idempotency 身份，调用 External v1 `/api/external/v1/editor/images/background-removals` 后只返回有界队列状态。抠图服务仍由客户端和服务端负责源校验、BgFilter、素材登记与画布事务，Codex 不获得内部 worker、凭据或任意 API 调用权。 +- 抠图通过新增 `agc_remove_background` 语义工具开放：模型只提交当前 manifest 的图片 `sourceLocalAssetId` 与结果名称；客户端解析稳定 `resourceId`，准备同名画布/素材目录并生成稳定 operation/idempotency 身份。普通登录态使用账号鉴权的 `/api/editor/images/background-removals`，ExternalDeveloper 模式使用 External v1 `/api/external/v1/editor/images/background-removals`；客户端接收异步受理后轮询任务状态，下载完成媒体并登记到本地 manifest，未知结果保留同一 operation 供恢复。抠图服务仍由客户端和服务端负责源校验、BgFilter、素材登记与画布事务，Codex 不获得内部 worker、凭据或任意 API 调用权。 ## 2026-08-23 AGC 资源生成补齐（视频 / 动画 / 音效 / 背景音乐） diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index 5f92b841a..bc3f69744 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -6266,17 +6266,19 @@ pub(crate) async fn edit_editor_image_for_owner_with_source_snapshot( pub async fn remove_editor_image_background( State(state): State, + headers: HeaderMap, Extension(request_context): Extension, Extension(authenticated): Extension, Json(payload): Json, ) -> Result, AppError> { let caller = EditorGenerationCaller::from_authenticated(&authenticated); + let idempotency_key = optional_editor_idempotency_key(&headers)?; let queue_job = enqueue_editor_background_removal_for_owner( &state, &request_context, &caller, payload, - None, + idempotency_key, ) .await?; Ok(json_success_body( diff --git a/src/components/image-editor/ImageCanvasWorldView.test.tsx b/src/components/image-editor/ImageCanvasWorldView.test.tsx index bd03cd7ab..d4ffc48d5 100644 --- a/src/components/image-editor/ImageCanvasWorldView.test.tsx +++ b/src/components/image-editor/ImageCanvasWorldView.test.tsx @@ -825,8 +825,9 @@ describe('ImageCanvasWorldView', () => { expect( within(layerButton) .getByRole('button', { name: '查看角色主图图片信息' }) - .parentElement! - .style.getPropertyValue('--image-canvas-editor-inverse-scale'), + .parentElement!.style.getPropertyValue( + '--image-canvas-editor-inverse-scale', + ), ).toBe(inverseScale); expect( (