diff --git a/.codex/skills/genarrative-external-editor-api/SKILL.md b/.codex/skills/genarrative-external-editor-api/SKILL.md index be69ad12b..6a99b11c1 100644 --- a/.codex/skills/genarrative-external-editor-api/SKILL.md +++ b/.codex/skills/genarrative-external-editor-api/SKILL.md @@ -1,93 +1,70 @@ --- name: genarrative-external-editor-api -description: Guide use of Genarrative's external editor/canvas OpenAPI. Use when a user describes a canvas/editor integration need and Codex must infer the right `/api/external/v1` API automatically, prepare canvas and asset-library context, abstract reusable art specs before generating assets, upload references, draft curl/HTTP/SDK requests, or set up and safely handle a Genarrative developer API Key. +description: Guide use of Genarrative's hosted external editor/canvas MCP or asynchronous `/api/external/v1` OpenAPI. Use when an Agent needs to discover the hosted integration, choose a canvas or asset operation, upload local reference media, create or update projects and asset-library records, submit and poll image/video/audio generation, interpret generated artifacts and warnings, draft HTTP/Python calls, or securely handle a Genarrative developer API Key. --- # Genarrative External Editor API -Use the live OpenAPI contract as the source of truth: `GET https://www.genarrative.world/api/external/v1/openapi.json`. In this repository, the same contract is `docs/openapi/genarrative-external-v1.openapi.json`. If exact fields or enums matter, read the contract before emitting final code. +Discover the live integration through `GET https://www.genarrative.world/api/external/v1/agent-integration.json`. Treat `GET https://www.genarrative.world/api/external/v1/openapi.json` as the field-level source of truth. In this repository, the same contract is `docs/openapi/genarrative-external-v1.openapi.json`. -Prefer the bundled Python helper for runnable examples: `scripts/genarrative_external_api.py`. It uses only Python stdlib, reads the local JSON API Key file, fixes the production base URL, and wraps upload/confirm/generation routes. +Prefer the hosted Streamable HTTP MCP at `https://www.genarrative.world/api/external/v1/mcp` when the Agent supports remote MCP with a custom Bearer token. It exposes the External v1 operations as tools and the Skill documentation as resources; it does not require a local MCP server. Use this complete Skill package when remote MCP is unavailable or local-file upload needs client-side orchestration. + +Prefer `scripts/genarrative_external_api.py` for runnable REST calls. It uses only Python stdlib, reads the local private API Key file, keeps the production base URL fixed, uploads local references, and wraps asynchronous submission, polling, and result retrieval. ## Workflow -1. At the start of a new conversation, ask the user for the canvas name before the first generation call unless an existing session is already provided. Create or use a project with that name and an asset-library folder with the same name. Keep `canvasName`, `projectId`, `assetFolderId`, and the current art spec in conversation state. -2. Before any art asset generation, abstract the user's request into a reusable art spec. Ask only for missing spec fields required by the selected asset type. If a current spec already exists and the user does not request a new style/spec, reuse it automatically. -3. Classify the user's natural-language intent. Do not ask the user to choose an API: - - "生成/生图/做一张图" -> image generation - - "重绘/修改这张图" -> image edit - - "用这张参考图/基于本地图生成" -> upload local reference image, then generation or edit - - "上传本地素材" -> upload ticket, OSS form upload, object confirm - - "保存画板/更新布局" -> canvas save - - "读取私有素材" -> signed read URL -4. Ask only for missing inputs that affect the request body or an actually ambiguous route: - - credentials JSON path only if the user cannot use the default local path - - canvas name when no current canvas session exists; existing `projectId`, folder/resource IDs only when resuming a known project - - media type, prompt, references, dimensions, model, ratio, duration, and resolution - - whether referenced media is already uploaded as `objectKey` or still local -5. Every external generation must write to both the canvas and the asset library. Include `projectId`, `assetFolderId`, a display label, and `canvasCompletion` whenever the target endpoint supports them. For character animation, use the helper's two-step fallback: generate with `projectId` + `canvasCompletion`, then create a library asset from the first returned frame in the session folder. -6. If the user lacks an API Key, guide setup before request design. -7. Read `references/api-selection.md` before finalizing any request. Use the core table below for fast routing, then verify details in the reference. -8. Use `scripts/genarrative_external_api.py` when the user wants runnable Python, reference image upload, canvas/folder session setup, art-spec carrying, or a chain that should execute with fewer hand-written curl steps. -9. Keep to `/api/external/v1` unless the user explicitly asks for internal profile/admin APIs. +1. Discover the integration manifest. Choose hosted MCP when supported; otherwise use the helper or direct REST. +2. Before the first generation in a new conversation, obtain a canvas name unless an existing `projectId` and `assetFolderId` were supplied. Create or reuse a project and a same-name asset-library folder. Retain `canvasName`, `projectId`, `assetFolderId`, and the current art spec. +3. Normalize art requests into a reusable spec. Ask only for missing values that block the selected operation. Reuse the spec until the user changes its style, subject family, palette, format, or constraints. +4. Infer the operation from the user's intent. Do not ask the user to select an API unless two operations would produce materially different artifacts. +5. If a reference exists only as a local file, upload and confirm it first. Pass the stable returned `objectKey` to generation; never substitute a temporary signed URL. +6. For generation endpoints that support the fields, include `projectId`, `assetFolderId`, an asset label, and `canvasCompletion` so the result enters both the canvas and its same-name library folder. +7. Treat every generation POST as asynchronous. Send one stable `Idempotency-Key` per logical request, retain the returned `operationId`, and poll the returned `statusUrl` or `GET /api/external/v1/generations/{operationId}` according to `pollAfterMs`. +8. Consume `result` only after `status=completed`. On `failed`, surface the safe error. On a client timeout or lost response, retain the operation/key; do not create a replacement request. +9. Reload the normal project or asset-library read endpoint when the caller needs complete authoritative state. Generation results are intentionally compact. +10. Stay within `/api/external/v1`. Never call internal workers, queues, admin/profile APIs, or SpacetimeDB endpoints unless the user explicitly changes scope. -## Core Routes +## Essential Invariants -| Intent | Method and path | Required fields | -| --- | --- | --- | -| List/create projects | `GET/POST /api/external/v1/editor/projects` | create: optional `title` | -| Save canvas | `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | `viewport`, `layers`, `expectedRevision` | -| Upload local media | `POST /api/external/v1/assets/direct-upload-tickets` -> OSS form -> `POST /api/external/v1/assets/objects/confirm` | ticket: `legacyPrefix`, `fileName`; confirm: `objectKey`, `assetKind` | -| Read private media | `GET /api/external/v1/assets/read-url` | `objectKey` or `legacyPublicPath` | -| Image generation | `POST /api/external/v1/editor/images/generations` | `prompt` | -| Image edit/redraw | `POST /api/external/v1/editor/images/edits` | `prompt`, `sourceImageSrc` | -| Icon spritesheet | `POST /api/external/v1/editor/icon-spritesheets/generations` | `referenceImageSrc`, `iconDescriptions` | -| UI asset extraction | `POST /api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize`; use `assetFolderId` for library folder | -| Character animation | `POST /api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | -| Video generation | `POST /api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | -| Sound effect | `POST /api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` | -| Background music | `POST /api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` | +- Authenticate MCP and business API calls with `Authorization: Bearer `. Never ask the user to paste a key into chat or place one in repository files. +- All eight generation POST routes require `Idempotency-Key` and return HTTP `202`; `202` is durable acceptance, not a media result. +- Retry an uncertain submission only with the exact same body and the same idempotency key. A polling timeout is not permission to generate again. +- Use stable references such as `objectKey`, project resource ID, or asset ID in generation requests. Use `/assets/read-url` only for temporary preview/download access. +- Preserve both warning channels after completion. A general `warning` can coexist with `sliceWarning`; do not discard either. +- Do not invent missing derivatives. A source-preserved warning means the main source remains usable but requested post-processing failed. A slice warning means the complete transparent sheet is usable but individual slices are absent. +- Keep generated artifacts in the canvas and asset library together. Character animation may need a post-completion library fallback from the first returned frame when no direct asset is present; the helper implements it. -## Art Spec Interface +## Documentation Navigation -Maintain one current art spec per conversation. A compact spec is enough: +Read only the references needed for the task, but always verify exact schemas and enums against live OpenAPI: -```json -{ - "assetType": "character | background | prop | ui | icon | animation | video | audio", - "subject": "要生成的主体", - "style": "画风/材质/时代/参考风格", - "palette": "主色与禁用色", - "composition": "构图、镜头、姿态或布局", - "format": "比例、尺寸、分辨率、帧数、时长", - "constraints": "必须保留/禁止出现/透明或绿幕要求", - "references": ["objectKey 或本地路径说明"] -} -``` +- `references/capability-routing.md`: read before selecting an MCP tool or REST operation, creating a canvas session, or working in the AI game creator visual DAG. +- `references/api-operations.md`: read when constructing project, canvas, asset-library, upload, generation, or generation-status calls. +- `references/authentication-and-safety.md`: read before handling credentials, local files, OSS form upload, retries, private media, or logs. +- `references/requests-and-outputs.md`: read before building generation payloads, polling, interpreting compact results, applying canvas completion, or handling post-processing warnings. -For a first spec, infer fields from the user's words and ask only for missing fields that block the selected API. Examples: character animation needs source image/layer, dimensions, motion, ratio, frame count, and duration; UI extraction needs source design image plus target density; icon spritesheet needs reference image and icon descriptions. After a spec exists, reuse it for later assets unless the user changes style, subject family, palette, format, or constraints. +The hosted MCP exposes the same documents through: -## API Key +- `genarrative://external-editor/skill` +- `genarrative://external-editor/skill/references/capability-routing.md` +- `genarrative://external-editor/skill/references/api-operations.md` +- `genarrative://external-editor/skill/references/authentication-and-safety.md` +- `genarrative://external-editor/skill/references/requests-and-outputs.md` +- `genarrative://external-editor/openapi` -The external OpenAPI uses: +## Hosted Integration Discovery -```text -Authorization: Bearer -``` +- Manifest: `GET /api/external/v1/agent-integration.json`. +- Hosted MCP: `POST /api/external/v1/mcp`, Streamable HTTP, same Bearer API Key. +- OpenAPI: `GET /api/external/v1/openapi.json`. +- Raw Skill entry: `GET /api/external/v1/skill/SKILL.md`. +- Complete Skill archive: `GET /api/external/v1/skill.zip`. -The OpenAPI JSON endpoint is public; every other external endpoint requires the Bearer API Key. +The archive contains this main file, four one-level references, the Python helper, and `agents/openai.yaml`. Verify its SHA-256 against `agent-integration.json` before installing. Discovery, OpenAPI, and Skill downloads are public; MCP and business operations require authentication. -Use this fixed production base URL: +## Python Helper -```text -https://www.genarrative.world/ -``` - -Guide the user to create a key from the logged-in product UI under `开发者 API Key`. The raw key is shown only once; never ask the user to paste it into chat. Tell them to store it in this local private JSON file, outside the repository: - -```text -~/.config/genarrative/external-editor-api.json -``` +Store the API Key outside the repository at `~/.config/genarrative/external-editor-api.json`: ```json { @@ -95,274 +72,45 @@ Guide the user to create a key from the logged-in product UI under `开发者 AP } ``` -Set the file readable only by the current user where possible: `chmod 600 ~/.config/genarrative/external-editor-api.json`. Do not use environment variables for this API. - -Smoke test by reading the JSON file, without printing the key: - -```bash -api_key="$(node -e 'const fs=require("fs"); const p=process.argv[1]; const c=JSON.parse(fs.readFileSync(p,"utf8")); process.stdout.write(c.apiKey || "");' "$HOME/.config/genarrative/external-editor-api.json")" -curl -fsS "https://www.genarrative.world/api/external/v1/editor/projects" \ - -H "Authorization: Bearer $api_key" -``` - -For generated client code, read `apiKey` from the JSON file, fail with a clear missing-config error, and redact keys in logs. - -Python smoke without printing the key: +Set restrictive permissions where possible, then smoke-test without printing the key: ```bash +chmod 600 ~/.config/genarrative/external-editor-api.json python3 .codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py list-projects ``` -## Request Patterns - -For image and icon generation, the request-body top-level `style` field both appends a server-side pixel-art clause to the prompt sent to the provider and enables deterministic post-processing. It is distinct from `generationInputs.artSpec.style`, which is stored metadata describing the requested visual language. Pass `style="pixelArt"` in Python or `"style": "pixelArt"` in JSON to enable both on supported generation types; use `"none"` or omit the field otherwise, in which case the submitted prompt is identical to what the same request would send with the field omitted. That is not the same as passing your text through untouched — see `references/api-selection.md`, which also covers compatibility and fallback semantics. - -For Python callers, prefer: +For a canvas-backed generation: ```python from genarrative_external_api import GenarrativeExternalClient client = GenarrativeExternalClient() session = client.prepare_canvas_session("新画板") -art_spec = { - "assetType": "background", - "subject": "幻想森林主视觉", - "style": "手绘游戏概念图", - "palette": "翡翠绿、金色光斑，避免低饱和灰", - "composition": "16:9 横版，中心留出角色站位", - "format": "16:9, 1K", - "constraints": "无文字、无 UI 按钮", - "references": [], -} client.generate_image( - "生成幻想森林背景", + "生成一张 16:9 幻想森林游戏背景", canvasSession=session, assetLabel="森林背景", aspectRatio="16:9", imageSize="1K", - artSpec=art_spec, + artSpec={ + "assetType": "background", + "subject": "幻想森林主视觉", + "style": "手绘游戏概念图", + "palette": "翡翠绿与金色光斑", + "composition": "横版，中心留出角色站位", + "format": "16:9, 1K", + "constraints": "无文字、无 UI 按钮", + "references": [], + }, ) ``` -For a transparent game/UI atlas, call the dedicated helper instead of ordinary image generation: - -```python -client.generate_icon_spritesheet( - "editor-resource-current-art-spec", - ["蛇头四方向", "直身与四种转角", "尾部四方向", "四类可区分食物"], - canvasSession=session, - assetLabel="贪吃蛇透明图集", - screenColor="auto", -) -``` - -Pass the registered visual-spec resource ID as `reference_image_src`; do not pass the UI prototype or a local path. - -Use the helper directly from this skill path, or copy it into the caller's project. Do not change the fixed base URL or move the API Key into environment variables. - -Use this shared base: - -```bash -api="https://www.genarrative.world" -credentials_file="$HOME/.config/genarrative/external-editor-api.json" -api_key="$(node -e 'const fs=require("fs"); const p=process.argv[1]; const c=JSON.parse(fs.readFileSync(p,"utf8")); process.stdout.write(c.apiKey || "");' "$credentials_file")" -auth=(-H "Authorization: Bearer $api_key") -json=(-H "Content-Type: application/json") -``` - -Create a project: - -```bash -curl -fsS "$api/api/external/v1/editor/projects" \ - "${auth[@]}" "${json[@]}" \ - -d '{"title":"新画板"}' -``` - -Generate an image and save it into both the canvas and the asset-library folder: - -```json -{ - "prompt": "一张横版幻想森林背景，适合游戏主视觉", - "kind": "spec", - "aspectRatio": "16:9", - "imageSize": "1K", - "projectId": "", - "assetFolderId": "", - "assetLabel": "森林背景", - "generationInputs": { - "artSpec": { - "assetType": "background", - "style": "手绘游戏概念图" - } - }, - "canvasCompletion": { - "title": "森林背景", - "placeholder": { - "x": 0, - "y": 0, - "width": 1024, - "height": 576, - "originalWidth": 1024, - "originalHeight": 576 - } - } -} -``` - -Then call `POST /api/external/v1/editor/images/generations`. - -For direct HTTP/curl, create or find the folder first with `GET /api/external/v1/editor/assets/library` and `POST /api/external/v1/editor/assets/folders`. The folder label should match the canvas name. - -## Reference Images - -When the user provides a local reference image path/file, upload it first; do not ask the user to convert it to base64. - -Python helper path: - -```python -from genarrative_external_api import GenarrativeExternalClient - -client = GenarrativeExternalClient() -session = client.prepare_canvas_session("新画板") -ref = client.upload_reference_image("/path/to/reference.png") -client.generate_image( - "基于参考图生成一张 16:9 游戏背景", - canvasSession=session, - assetLabel="参考图背景", - aspectRatio="16:9", - imageSize="1K", - referenceImageSrcs=[ref["objectKey"]], -) -``` - -Use the normal upload flow with: - -```json -{ - "legacyPrefix": "generated-character-drafts", - "pathSegments": ["editor", "external-editor-references"], - "fileName": "", - "contentType": "image/png", - "access": "private" -} -``` - -After OSS form upload, confirm the object with `assetKind: "editor_reference_image"`. Put the returned `objectKey` into the generation request: - -- image generation: `referenceImageSrcs` -- image edit/redraw: `sourceImageSrc`; extra references go in `referenceImageSrcs` -- icon spritesheet: `referenceImageSrc` -- UI asset extraction: `sourceImageSrc`; extra references go in `referenceImageSrcs` -- character animation: `sourceImageSrc` -- video generation image references: `referenceImageSrcs` - -Use `signedUrl` only for display/download. For generation requests, use `objectKey`, project resource ID, asset ID, public URL, or Data URL as the endpoint allows; prefer uploaded `objectKey` for local/private reference images. - -OSS form upload shape, using the ticket response saved as `ticket.json`. The default response has `upload`; if the caller explicitly requested the API response envelope, use `data.upload`: - -```bash -node - <<'NODE' ticket.json /path/to/reference.png -const fs = require('fs'); -const path = require('path'); - -(async () => { - const body = JSON.parse(fs.readFileSync(process.argv[2], 'utf8')); - const ticket = body.upload || body.data?.upload; - if (!ticket) throw new Error('Upload ticket response missing upload payload'); - const filePath = process.argv[3]; - const form = new FormData(); - for (const [key, value] of Object.entries(ticket.formFields)) { - if (value != null) form.append(key, value); - } - const bytes = fs.readFileSync(filePath); - form.append( - 'file', - new Blob([bytes], { type: ticket.contentType || 'application/octet-stream' }), - path.basename(filePath), - ); - const response = await fetch(ticket.host, { method: 'POST', body: form }); - if (!response.ok) { - throw new Error(`OSS upload failed: ${response.status} ${await response.text()}`); - } -})().catch((error) => { - console.error(error.message); - process.exit(1); -}); -NODE -``` - -Then confirm with `contentLength`: - -```json -{ - "objectKey": "", - "contentType": "image/png", - "contentLength": 12345, - "assetKind": "editor_reference_image", - "accessPolicy": "private" -} -``` - -`contentLength` is a JSON number from the local file byte size, not a quoted string. - -For character animation from an uploaded local image, set: - -```json -{ - "sourceLayerId": "external-reference-hero", - "sourceImageSrc": "", - "sourceWidth": 720, - "sourceHeight": 1280, - "promptText": "让角色自然呼吸并轻微转身", - "resolution": "720p", - "ratio": "9:16", - "frameCount": 40, - "durationSeconds": 5, - "model": "seedance2.0-fast" -} -``` - -Use an existing canvas layer ID when the image came from a project layer. If it came only from a local upload, derive a stable synthetic `sourceLayerId` from the file name, for example `external-reference-hero`. Read `sourceWidth` and `sourceHeight` from the actual image before upload; ask the user only if the dimensions cannot be determined. - -The helper uses a 420 second timeout for generation calls, including character animation and video. Direct HTTP clients should not use a 70 second request timeout for animation. - -Character animation currently returns canvas completion data but not a direct `asset` payload. To keep the "canvas + asset library" invariant, call `client.animate_character(..., canvasSession=session, canvasTitle="...")`; the helper creates a library asset from the first returned frame in the session folder after the animation call succeeds. - -For video generation, always include `mode: "std"`. When using image/video/audio references, default to `model: "seedance2.0-fast"` unless the user asks for another listed model, because reference media support is limited to the Seedance 2.0 family. - -For image edit/redraw that should replace an existing canvas layer, pass `projectId` and `targetLayerId`. If the user instead gives an explicit `canvasCompletion`, let that placement win. - -For sound effects and BGM, `assetFolderId` and `assetLabel` can write the generated audio to the account asset library, same as image/video generation. - -## Successful Responses with Warnings - -Character image generation (including character redraw through `kind: "character"`), icon spritesheet generation, and UI asset extraction can return HTTP 2xx with an optional structured `warning`. A 2xx response means the task completed, but it does not guarantee that every requested post-processed derivative exists. - -- Apply the returned `project` and media snapshots before interpreting optional derivatives: character responses use `resource` / `asset`, while icon spritesheet and UI extraction responses use `spritesheetResource` / `spritesheetAsset`. When `warning.code` is `postprocess-failed-source-preserved`, the saved provider source image is the authoritative main result. Character output has no transparent derivative; icon spritesheet and UI extraction output have neither a transparent spritesheet nor slices. Display `warning.reason` directly, and do not synthesize missing derivatives or restart generation. -- `sliceWarning` is a separate condition used only when transparent spritesheet post-processing succeeded but automatic slicing failed. Keep `sliceWarning.reason` as the original diagnostic and continue using the complete transparent spritesheet; a UI may add context when displaying it, but must not rewrite the stored reason. -- `warning` and `sliceWarning` are mutually exclusive only for `postprocess-failed-source-preserved`, because a failed transparent post-process never reaches slicing. Since 2026-07-29 a general `warning` may also come from image-style normalization (`unsupported-image-style`) or pixel-art snapping, and those can coexist with `sliceWarning` in the same response. Display both reasons; do not drop either one and do not misclassify a source-preserved result as a slicing-only warning. - -For reusable transparent game/UI sheets, do not substitute ordinary image generation merely because it can draw several objects in one image. Use icon spritesheet generation when a stable visual-spec reference and `iconDescriptions` exist; use UI extraction only for an existing annotated UI design. Pass `screenColor: "auto"` unless the art direction requires one of the supported solid chroma colors. A client must verify the returned full sheet really contains transparency before treating it as a transparent spritesheet. If a source-preserved `warning` is present, do not register the opaque provider source as the requested transparent deliverable. When only `sliceWarning` is present, the full transparent sheet remains usable, but no individual slices may be claimed. - -## AI Game Creator Canonical Visual DAG - -The AI game creator reuses its existing 16-task manifest; do not add a parallel task system or collapse the following artifacts into one ordinary generation request: - -1. `art-director` generates `assets/art-spec.png` with `POST /api/external/v1/editor/images/generations`, `kind: "spec"`, and registers it as `assetKind: "icon-spec"`. This is the real visual-spec image. The JSON value in `generationInputs.artSpec` is supporting structured context and does not replace this image. -2. `design-foundation` uses the registered External Editor resource ID for `assets/art-spec.png` in `referenceImageSrcs`, then generates the complete `assets/ui-prototype.png` through `POST /api/external/v1/editor/images/generations` with `kind: "ui-design"`. -3. `art-asset-plan` uses the same registered `assets/art-spec.png` resource ID as the required `referenceImageSrc` for `POST /api/external/v1/editor/icon-spritesheets/generations`, supplies concrete `iconDescriptions`, and registers the transparent full result as `assets/art-spritesheet.png`. - -Never use `assets/ui-prototype.png` as the icon spritesheet's visual-spec reference. `POST /api/external/v1/editor/ui-designs/assets/extractions` requires an existing UI design image with red-box annotations; it is not UI generation and is not part of this canonical DAG. +Helper convenience methods wait locally, but the server still uses short asynchronous submit/status requests. For durable caller-controlled orchestration, call `submit_generation`, persist its `operationId` and idempotency key, then call `get_generation` or `wait_for_generation`. ## Guardrails -- Do not invent endpoints outside the OpenAPI, especially internal worker or runtime task-list routes. -- Do not omit canvas/library context for generation. New generated assets should enter both the canvas and the asset-library folder named after the canvas. -- Do not put API Keys in repository files, generated project files, command history snippets with literal secrets, logs, docs, commits, or screenshots. The only default storage is the user's local private JSON credentials file. -- Do not use account JWT endpoints as the default external integration path. The profile API can create/revoke keys for logged-in product users, but it is not part of the external editor OpenAPI. -- When an endpoint returns `project`, `resource`, or `asset`, treat those as the authoritative updated project/resource/asset snapshots. - -## Resources - -- `references/api-selection.md`: intent routing and required-field cheat sheet. -- `scripts/genarrative_external_api.py`: stdlib Python helper for OpenAPI fetch, API Key loading, local reference upload, object confirm, project/canvas calls, and generation requests. +- Do not change the fixed production base URL in generated examples. +- Do not move the API Key into environment variables, source files, generated projects, logs, docs, screenshots, or shell snippets containing literal secrets. +- Do not treat a Data URL, Blob URL, expiring signed URL, worker lease, or provider diagnostic as a durable result. +- Do not reconstruct authoritative canvas, resource, or library snapshots from a compact generation response. +- Do not replace icon-spritesheet generation with ordinary image generation when the deliverable requires a reusable transparent atlas. diff --git a/.codex/skills/genarrative-external-editor-api/agents/openai.yaml b/.codex/skills/genarrative-external-editor-api/agents/openai.yaml index a9b66dccb..446170018 100644 --- a/.codex/skills/genarrative-external-editor-api/agents/openai.yaml +++ b/.codex/skills/genarrative-external-editor-api/agents/openai.yaml @@ -1,6 +1,6 @@ interface: display_name: "Genarrative External Editor API" - short_description: "Auto-route canvas API generation" - default_prompt: "Use $genarrative-external-editor-api to prepare a canvas session, infer the right API, and generate assets into the canvas and library." + short_description: "Route async canvas generation safely" + default_prompt: "Use $genarrative-external-editor-api to discover the hosted integration, prepare a canvas session, and submit and poll asset generation into the canvas and library." policy: allow_implicit_invocation: true diff --git a/.codex/skills/genarrative-external-editor-api/references/api-operations.md b/.codex/skills/genarrative-external-editor-api/references/api-operations.md new file mode 100644 index 000000000..06fbf30a1 --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/references/api-operations.md @@ -0,0 +1,99 @@ +# API Operations + +Use this reference after selecting a capability. Treat `GET /api/external/v1/openapi.json` as authoritative for exact request/response schemas, required fields, constraints, and operation IDs. + +All paths below are relative to `https://www.genarrative.world`. Discovery and Skill download routes are public. Project, asset, upload, generation, and generation-query operations require the Bearer API Key. + +## Project and Canvas Operations + +| Operation | Method and path | Minimum input | +| --- | --- | --- | +| List projects | `GET /api/external/v1/editor/projects` | Authentication | +| Create project | `POST /api/external/v1/editor/projects` | Optional `title` | +| Load recent project | `GET /api/external/v1/editor/projects/recent` | Authentication | +| Get project | `GET /api/external/v1/editor/projects/{projectId}` | `projectId` | +| Delete project | `DELETE /api/external/v1/editor/projects/{projectId}` | `projectId` | +| Rename project | `PATCH /api/external/v1/editor/projects/{projectId}/metadata` | `title` | +| Save canvas | `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | `viewport`, `layers`, `expectedRevision` | +| Add project resource | `POST /api/external/v1/editor/projects/{projectId}/resources` | `imageSrc`, `width`, `height`, `sourceType` | + +Canvas save uses optimistic revision control. Pass the last authoritative `expectedRevision`; on conflict, reload instead of replaying a stale full layout. + +## Asset and Upload Operations + +| Operation | Method and path | Minimum input | +| --- | --- | --- | +| Create direct-upload ticket | `POST /api/external/v1/assets/direct-upload-tickets` | `legacyPrefix`, `fileName` | +| Confirm uploaded object | `POST /api/external/v1/assets/objects/confirm` | `objectKey`, `assetKind` | +| Get signed read URL | `GET /api/external/v1/assets/read-url` | `objectKey` or `legacyPublicPath` | +| Read asset library | `GET /api/external/v1/editor/assets/library` | Authentication | +| Create folder | `POST /api/external/v1/editor/assets/folders` | `label` | +| Update folder | `PATCH /api/external/v1/editor/assets/folders/{folderId}` | `label` or `collapsed` | +| Delete folder | `DELETE /api/external/v1/editor/assets/folders/{folderId}` | `folderId` | +| Create asset record | `POST /api/external/v1/editor/assets` | `folderId`, `label`, `imageSrc`, `width`, `height`, `sourceType` | +| Update asset record | `PATCH /api/external/v1/editor/assets/{assetId}` | `label` or `folderId` | +| Delete asset record | `DELETE /api/external/v1/editor/assets/{assetId}` | `assetId` | + +Upload is a three-step client flow: create a ticket, POST the file and returned fields directly to the OSS form endpoint, then confirm the returned `objectKey`. See `authentication-and-safety.md` before implementing this flow. + +## Generation Operations + +Every generation row requires a stable `Idempotency-Key` header and returns HTTP `202` with an asynchronous submission, not the generated media. + +| Capability | POST path | Required body fields | Common optional body fields | +| --- | --- | --- | --- | +| Image generation | `/api/external/v1/editor/images/generations` | `prompt` | `kind`, `style`, `model`, `aspectRatio`, `imageSize`, `size`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | +| Image edit/redraw | `/api/external/v1/editor/images/edits` | `prompt`, `sourceImageSrc` | `referenceImageSrcs`, `model`, `size`, `projectId`, `assetFolderId`, `assetLabel`, `sourceResourceId`, `targetLayerId`, `canvasCompletion` | +| Icon spritesheet | `/api/external/v1/editor/icon-spritesheets/generations` | `referenceImageSrc`, `iconDescriptions` | `style`, `referenceImageSrcs`, `screenColor`, `model`, `aspectRatio`, `imageSize`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | +| UI asset extraction | `/api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize` | `screenColor`, `model`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `spritesheetLabel`, `canvasCompletion` | +| Character animation | `/api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `canvasCompletion` | +| Video generation | `/api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | `referenceImageSrcs`, `referenceVideoSrcs`, `referenceAudioSrcs`, `webSearchEnabled`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | +| Sound effect | `/api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` | `model`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | +| Background music | `/api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` | `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | + +Poll all eight through: + +```text +GET /api/external/v1/generations/{operationId} +``` + +Supply the `operationId` returned by submission. Poll no faster than `pollAfterMs` and retain the ID after a caller-side timeout. + +## Canvas and Library Field Rules + +- Pass `projectId` and `canvasCompletion` to write generated output into the canvas. +- Pass `assetFolderId` plus `assetLabel` for image, edit, icon spritesheet, video, sound effect, and BGM operations when supported. +- UI extraction uses `assetFolderId` and `spritesheetLabel`. +- Character animation does not accept the same library fields. If its completed compact result lacks a direct `asset`, create a library record from the first returned frame; do not duplicate one when an asset already exists. +- Reload project/library state after completion when full current state is required. + +## Reference Field Mapping + +After confirming a local upload, pass its stable `objectKey` into: + +| Target capability | Field | +| --- | --- | +| Image generation | `referenceImageSrcs` | +| Image edit/redraw | `sourceImageSrc`; additional references in `referenceImageSrcs` | +| Icon spritesheet | `referenceImageSrc`; additional style references in `referenceImageSrcs` | +| UI design extraction | `sourceImageSrc`; additional references in `referenceImageSrcs` | +| Character animation | `sourceImageSrc` | +| Video with image references | `referenceImageSrcs` | + +Use video/audio reference arrays only with models that support them. Do not pass an expiring signed read URL as a generation reference. + +## Common Values + +Use OpenAPI as the final authority; these common values are a routing aid: + +- Image `kind`: `spec`, `character`, `quick-edit`, `ui-design`, `publication-material`; ordinary image generation may omit it. +- Image `model`: `gpt-image-2`, `gemini-3.1-flash-image-preview`, `nanobanana2`, `nano-banana`. +- Image `aspectRatio`: `1:1`, `2:3`, `3:2`, `9:16`, `16:9`. +- Image `imageSize`: `0.5K`, `1K`, `2K`. +- Video `model`: `seedance2.0`, `seedance2.0-fast`, `kling3.0`, `kling3.0-omni`, `veo3.1`, `veo3.1-fast`. +- Video `aspectRatio`: `16:9`, `9:16`, `1:1`, `4:3`, `3:4`, `21:9`. +- Video `resolution`: `480p`, `720p`, `1080p`; `mode`: `std`; `sound`: `on` or `off`. +- Character animation uses `model: "seedance2.0-fast"`; `resolution`: `480p` or `720p`; `frameCount`: `32`, `40`, or `48`; `durationSeconds`: `4`, `5`, or `6`; `ratio`: `same`, `1:1`, `4:3`, `16:9`, `9:16`, or `3:4`. +- UI extraction uses `aspectRatio: "1:1"`; use `imageSize: "1K"` for normal/small extraction and `2K` for dense designs. + +Do not hard-code this list as a replacement client schema. In particular, the top-level image `style` field is intentionally extensible; see `requests-and-outputs.md` for its fallback behavior. diff --git a/.codex/skills/genarrative-external-editor-api/references/api-selection.md b/.codex/skills/genarrative-external-editor-api/references/api-selection.md deleted file mode 100644 index a895da28e..000000000 --- a/.codex/skills/genarrative-external-editor-api/references/api-selection.md +++ /dev/null @@ -1,240 +0,0 @@ -# External Editor API Routing - -Source of truth: `docs/openapi/genarrative-external-v1.openapi.json`. - -## Base - -- Fixed base URL: `https://www.genarrative.world/`. -- Public contract: `GET /api/external/v1/openapi.json`. -- Authenticated calls: `Authorization: Bearer `. -- Default credentials file: `~/.config/genarrative/external-editor-api.json` with an `apiKey` string. -- Generation clients should allow long-running responses. Use at least 420 seconds for character animation and video; 70 seconds is too short for animation. - -## Canvas Session and Art Spec - -At the start of a new conversation, ask for a canvas name before the first generation call unless the user already supplied `projectId` and `assetFolderId`. Create or reuse: - -1. `POST /api/external/v1/editor/projects` with `title` = canvas name. -2. `GET /api/external/v1/editor/assets/library`; if no folder has the same label, `POST /api/external/v1/editor/assets/folders` with `label` = canvas name. -3. Keep `canvasName`, `projectId`, `assetFolderId`, and the current art spec in conversation state. - -Before generating art assets, normalize the user's request into a current art spec with `assetType`, `subject`, `style`, `palette`, `composition`, `format`, `constraints`, and `references`. Ask follow-up questions only for missing fields that block the selected endpoint. Reuse the current spec automatically when the user asks for another asset without changing style/spec requirements. Put the spec in `generationInputs.artSpec` and summarize it in the prompt when useful. - -For the AI game creator's existing 16-task autonomous build, distinguish that JSON art spec from the required visual-spec image and keep this dependency chain: - -1. `art-director` -> `assets/art-spec.png` via `POST /api/external/v1/editor/images/generations`, with `kind=spec` and registered `assetKind=icon-spec`. -2. `design-foundation` -> `assets/ui-prototype.png` via the same image generation endpoint with `kind=ui-design`, using the registered art-spec resource ID in `referenceImageSrcs`. -3. `art-asset-plan` -> transparent `assets/art-spritesheet.png` via `POST /api/external/v1/editor/icon-spritesheets/generations`, using the registered art-spec resource ID as `referenceImageSrc` and providing `iconDescriptions`. - -Do not use the UI prototype as the spritesheet specification. UI extraction requires a stable source image with red-box annotations and is outside this canonical DAG. - -## Intent Routing - -Infer the endpoint from the user's description. Do not present this as a menu unless the request is genuinely ambiguous. - -| User says | Route | -| --- | --- | -| "生成图片", "生图", "做一张背景/角色/宣发图" | `POST /api/external/v1/editor/images/generations` | -| "重绘", "调整这张图", "基于这张图修改" | `POST /api/external/v1/editor/images/edits` | -| "用这张参考图", "参考本地图片生成", "基于本地图做图" | Upload local image first, then pass returned `objectKey` into the generation/edit reference field | -| "按规范图生成图标", "拆图标" | `POST /api/external/v1/editor/icon-spritesheets/generations` | -| "从 UI 设计图提取素材" | `POST /api/external/v1/editor/ui-designs/assets/extractions` | -| "让角色动起来", "生成角色动画帧" | `POST /api/external/v1/editor/character-animations/generations` | -| "生成视频" | `POST /api/external/v1/editor/videos/generations` | -| "生成音效" | `POST /api/external/v1/editor/audios/sound-effects/generations` | -| "生成背景音乐/BGM" | `POST /api/external/v1/editor/audios/background-music/generations` | -| "上传本地素材/图片/音频/视频" | Upload flow: direct upload ticket -> OSS form upload -> object confirm | -| "保存画板布局" | `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | -| "创建/读取/删除画板项目" | Project endpoints | -| "素材库/文件夹/素材记录" | Asset library endpoints | -| "读取私有素材/拿可访问链接" | `GET /api/external/v1/assets/read-url` | - -Ask a follow-up only when two routes could both be correct and produce different artifacts, for example "处理这张图" without saying edit, extract UI assets, or use it as a reference for new generation. - -## Endpoint Map - -| User intent | Endpoint | Minimum request | -| --- | --- | --- | -| Read contract | `GET /api/external/v1/openapi.json` | No auth required | -| List projects | `GET /api/external/v1/editor/projects` | API Key | -| Create project | `POST /api/external/v1/editor/projects` | Optional `title` | -| Load recent project | `GET /api/external/v1/editor/projects/recent` | API Key | -| Get/delete project | `GET` or `DELETE /api/external/v1/editor/projects/{projectId}` | `projectId` | -| Rename project | `PATCH /api/external/v1/editor/projects/{projectId}/metadata` | `title` | -| Save canvas layout | `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | `viewport`, `layers`, `expectedRevision` | -| Add project resource | `POST /api/external/v1/editor/projects/{projectId}/resources` | `imageSrc`, `width`, `height`, `sourceType` | -| Create upload ticket | `POST /api/external/v1/assets/direct-upload-tickets` | `legacyPrefix`, `fileName` | -| Confirm uploaded object | `POST /api/external/v1/assets/objects/confirm` | `objectKey`, `assetKind` | -| Get signed read URL | `GET /api/external/v1/assets/read-url` | `objectKey` or `legacyPublicPath` | -| Read asset library | `GET /api/external/v1/editor/assets/library` | API Key | -| Create/update/delete folder | `POST /api/external/v1/editor/assets/folders`, `PATCH`/`DELETE /api/external/v1/editor/assets/folders/{folderId}` | create: `label`; update: `label` or `collapsed` | -| Create asset record | `POST /api/external/v1/editor/assets` | `folderId`, `label`, `imageSrc`, `width`, `height`, `sourceType` | -| Update/delete asset | `PATCH`/`DELETE /api/external/v1/editor/assets/{assetId}` | update: `label` or `folderId` | - -## Generation Endpoints - -| User intent | Endpoint | Required fields | Common optional fields | -| --- | --- | --- | --- | -| Generate image/spec/character/UI/publication material | `POST /api/external/v1/editor/images/generations` | `prompt` | `kind`, `style`, `model`, `aspectRatio`, `imageSize`, `size`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | -| Edit/redraw image | `POST /api/external/v1/editor/images/edits` | `prompt`, `sourceImageSrc` | `referenceImageSrcs`, `model`, `size`, `projectId`, `assetFolderId`, `assetLabel`, `sourceResourceId`, `targetLayerId`, `canvasCompletion` | -| Generate icon spritesheet | `POST /api/external/v1/editor/icon-spritesheets/generations` | `referenceImageSrc`, `iconDescriptions` | `style`, `referenceImageSrcs`, `screenColor`, `model`, `aspectRatio`, `imageSize`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | -| Extract assets from UI design | `POST /api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize` | `screenColor`, `model`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `spritesheetLabel`, `canvasCompletion` | -| Generate character animation | `POST /api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `canvasCompletion`; then create a library asset from the first returned frame | -| Generate video | `POST /api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | `referenceImageSrcs`, `referenceVideoSrcs`, `referenceAudioSrcs`, `webSearchEnabled`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | -| Generate sound effect | `POST /api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` | `model`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | -| Generate background music | `POST /api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` | `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | - -## Image Post-processing Style - -The request-body top-level `style` field controls both the prompt actually submitted to the provider and deterministic image post-processing. It is separate from `generationInputs.artSpec.style`, which is stored metadata describing the requested visual language. - -- Omitted, `null`, an empty string, and `"none"` all disable post-processing without a warning, and leave the submitted prompt identical to what the same request would send with the field omitted. -- That guarantee is about this field only, not about passthrough of your text. The server always builds the submitted prompt: ordinary image prompts are trimmed, `kind: "character"` prompts are wrapped in a server-side template that pins the flat key-colour background matting depends on, and icon spritesheet generation has no `prompt` field at all — its prompt is assembled from the trimmed `iconDescriptions`. `style` does not participate in any of that. -- `"pixelArt"` appends one server-side clause to the end of the submitted prompt and enables deterministic pixel-art snapping, for ordinary image generation (omit `kind`), `kind: "character"`, and icon spritesheet generation. The clause is scoped per path — `画面为像素风格` for ordinary images, `角色主体为像素风格` for characters, and `每个图标素材均为像素风格` for icon spritesheets. Character and icon spritesheet generation are matted against a flat key-colour background afterwards, so their clauses deliberately never demand a whole-canvas pixelation that would fight the flat-background requirement already in those prompts. -- Unknown strings, or `"pixelArt"` on unsupported image kinds such as `spec`, `quick-edit`, `ui-design`, or `publication-material`, continue without style processing and return `warning.code: "unsupported-image-style"`. -- A non-string JSON value is malformed and returns HTTP `400`. Keep the field extensible; do not treat the current examples as a closed client-side enum. - -Image or character generation with pixel-art snapping: - -```json -{ - "prompt": "生成一个正面站立的像素风冒险者角色", - "kind": "character", - "style": "pixelArt" -} -``` - -Icon spritesheet generation with pixel-art snapping: - -```json -{ - "referenceImageSrc": "generated-character-drafts/editor/external-editor-references/icon-spec.png", - "iconDescriptions": ["木剑", "圆盾", "红色药水"], - "style": "pixelArt" -} -``` - -All generation requests should be placed into both the current canvas and its same-name asset-library folder. For endpoints that support `assetLabel`, pass it. For UI extraction, use `spritesheetLabel`. For icon spritesheet, the folder is enough. For character animation, the endpoint does not return `asset`; after success call `POST /api/external/v1/editor/assets` using the first returned frame as `imageSrc`, the session `assetFolderId`, and `assetKind: "character-animation"`. - -## HTTP 2xx Warning Handling - -Character image generation (including character redraw through `kind: "character"`), icon spritesheet generation, and UI asset extraction may return HTTP 2xx while carrying a structured `warning`; completion does not imply that all post-processed derivatives exist. - -- Consume the returned `project` and media snapshots as authoritative: character responses use `resource` / `asset`, while icon spritesheet and UI extraction responses use `spritesheetResource` / `spritesheetAsset`. `warning.code: "postprocess-failed-source-preserved"` means the saved provider source is the main result. Character output has no transparent derivative, while icon spritesheet and UI extraction have no transparent spritesheet and no slices. Display `warning.reason` directly; do not construct missing assets or retry the provider generation from scratch. -- `sliceWarning` is only for a transparent spritesheet that was created successfully but could not be split automatically. Use the complete transparent spritesheet and preserve `sliceWarning.reason` as the original diagnostic; it is not a post-processing/source-preserved warning. -- `warning` and `sliceWarning` are mutually exclusive only for `postprocess-failed-source-preserved`, because that failure never reaches slicing. A general `warning` produced by image-style normalization (`unsupported-image-style`) or pixel-art snapping can coexist with `sliceWarning`; render both reasons instead of picking one. - -## Reference Image Upload - -If the user provides a local file as a reference image, run upload before the generation request: - -1. `POST /api/external/v1/assets/direct-upload-tickets`. - Use `legacyPrefix: "generated-character-drafts"`, `pathSegments: ["editor", "external-editor-references"]`, original `fileName`, detected image `contentType`, and `access: "private"`. -2. Upload the file to the returned OSS form endpoint with all returned `formFields`. -3. `POST /api/external/v1/assets/objects/confirm` with returned `objectKey`, detected `contentType`, `contentLength` if known, `assetKind: "editor_reference_image"`, and `accessPolicy: "private"`. -4. Use the returned `objectKey` in the actual editor request. - -OSS form upload uses `upload.host` and every non-null `upload.formFields` entry, then the file part named `file`. Default responses expose `upload`; envelope responses expose `data.upload`. Save the upload ticket response as `ticket.json`: - -```bash -node - <<'NODE' ticket.json /path/to/reference.png -const fs = require('fs'); -const path = require('path'); - -(async () => { - const body = JSON.parse(fs.readFileSync(process.argv[2], 'utf8')); - const ticket = body.upload || body.data?.upload; - if (!ticket) throw new Error('Upload ticket response missing upload payload'); - const filePath = process.argv[3]; - const form = new FormData(); - for (const [key, value] of Object.entries(ticket.formFields)) { - if (value != null) form.append(key, value); - } - const bytes = fs.readFileSync(filePath); - form.append( - 'file', - new Blob([bytes], { type: ticket.contentType || 'application/octet-stream' }), - path.basename(filePath), - ); - const response = await fetch(ticket.host, { method: 'POST', body: form }); - if (!response.ok) { - throw new Error(`OSS upload failed: ${response.status} ${await response.text()}`); - } -})().catch((error) => { - console.error(error.message); - process.exit(1); -}); -NODE -``` - -Field mapping after upload: - -| Target API | Put uploaded `objectKey` in | -| --- | --- | -| Image generation | `referenceImageSrcs` | -| Image edit/redraw | `sourceImageSrc`; additional references in `referenceImageSrcs` | -| Icon spritesheet | `referenceImageSrc`; additional style refs in `referenceImageSrcs` | -| UI design extraction | `sourceImageSrc`; additional refs in `referenceImageSrcs` | -| Character animation | `sourceImageSrc` | -| Video generation with image references | `referenceImageSrcs` | - -Do not put the signed read URL into generation fields. Signed URLs are for user-visible preview/download; generation fields should use the stable `objectKey` for uploaded private references. - -## Common Enums - -- Image `kind`: `spec`, `character`, `quick-edit`, `ui-design`, `publication-material`. -- Image `model`: `gpt-image-2`, `gemini-3.1-flash-image-preview`, `nanobanana2`, `nano-banana`. -- Image `aspectRatio`: `1:1`, `2:3`, `3:2`, `9:16`, `16:9`. -- Image `imageSize`: `0.5K`, `1K`, `2K`. -- Video `model`: `seedance2.0`, `seedance2.0-fast`, `kling3.0`, `kling3.0-omni`, `veo3.1`, `veo3.1-fast`. -- Video `aspectRatio`: `16:9`, `9:16`, `1:1`, `4:3`, `3:4`, `21:9`. -- Video `resolution`: `480p`, `720p`, `1080p`. -- Video `mode`: always `std`. -- Video `sound`: `on`, `off`. -- Character animation `model`: always `seedance2.0-fast`. -- Character animation `resolution`: `480p`, `720p`; `frameCount`: `32`, `40`, `48`; `durationSeconds`: `4`, `5`, `6`; `ratio`: `same`, `1:1`, `4:3`, `16:9`, `9:16`, `3:4`. - -## Local Reference Media Details - -- `contentLength` in object confirm is a JSON number from local byte size, not a string. -- For character animation, use an existing project layer ID as `sourceLayerId` when available. -- If the source is only an uploaded local image, derive `sourceLayerId` from the file name, such as `external-reference-hero`, and keep it stable across retries. -- Read `sourceWidth` and `sourceHeight` from the local image. If dimensions cannot be read, ask instead of inventing dimensions. -- UI design extraction uses fixed `aspectRatio: "1:1"`; choose `imageSize: "1K"` for normal/small extractions and `2K` for dense designs. -- Video image/video/audio references are supported only by the Seedance 2.0 family; default referenced-media video requests to `model: "seedance2.0-fast"`, `mode: "std"`, and explicit `sound`. -- Image edit/redraw can pass `targetLayerId` with `projectId` to replace an existing canvas layer when no explicit `canvasCompletion` is supplied. -- Image, edit, video, sound effect, and BGM generation can pass `assetFolderId` and `assetLabel`; response `asset` is the created/updated library record. -- Icon spritesheet and UI extraction can pass `assetFolderId`; UI extraction can also pass `spritesheetLabel`. - -## Canvas Completion - -Use `canvasCompletion` for generation in this skill so the generated result is written back into the project canvas by the backend. - -Required: - -```json -{ - "title": "素材名称", - "placeholder": { - "x": 0, - "y": 0, - "width": 512, - "height": 512, - "originalWidth": 512, - "originalHeight": 512 - } -} -``` - -`dialogId` is optional. If the response includes `project`, `resource`, or `asset`, use those snapshots instead of reconstructing canvas/resource/library state locally. - -## Upload Flow - -For a local file that should become a project resource or library asset: - -1. `POST /api/external/v1/assets/direct-upload-tickets` with `legacyPrefix`, `fileName`, and optional `contentType`, `access`, `maxSizeBytes`. -2. Submit the file to the returned OSS form endpoint with returned `formFields`. -3. `POST /api/external/v1/assets/objects/confirm` with returned `objectKey` and an `assetKind`. -4. Create a project resource or library asset with the confirmed `assetObjectId`/`objectKey`. - -For reading private/generated assets, call `GET /api/external/v1/assets/read-url?objectKey=...` and use the returned `signedUrl`. diff --git a/.codex/skills/genarrative-external-editor-api/references/authentication-and-safety.md b/.codex/skills/genarrative-external-editor-api/references/authentication-and-safety.md new file mode 100644 index 000000000..ec51c8675 --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/references/authentication-and-safety.md @@ -0,0 +1,146 @@ +# Authentication and Safety + +Read this reference before handling credentials, local files, private objects, uploads, retries, or logs. + +## Contents + +- [API Key Setup](#api-key-setup) +- [Idempotency and Unknown Outcomes](#idempotency-and-unknown-outcomes) +- [Local Reference Upload](#local-reference-upload) +- [Stable and Temporary Media References](#stable-and-temporary-media-references) +- [Logging and Command Safety](#logging-and-command-safety) +- [Scope and Retry Guardrails](#scope-and-retry-guardrails) + +## API Key Setup + +Authenticated calls use: + +```text +Authorization: Bearer +``` + +Guide a logged-in user to create a key in the product UI under `开发者 API Key`. The raw key is shown only once. Never ask the user to paste it into chat. + +Store it outside repositories in the user's private JSON file: + +```text +~/.config/genarrative/external-editor-api.json +``` + +```json +{ + "apiKey": "tnr_sk_..." +} +``` + +Set the file readable only by the current user where supported: + +```bash +chmod 600 ~/.config/genarrative/external-editor-api.json +``` + +Use this fixed production base URL: + +```text +https://www.genarrative.world/ +``` + +Do not use environment variables as the default API Key storage for this integration. Generated clients must load the JSON file, fail clearly when it is absent or malformed, and redact credentials from errors and logs. + +Smoke-test without printing the key: + +```bash +api_key="$(node -e 'const fs=require("fs"); const p=process.argv[1]; const c=JSON.parse(fs.readFileSync(p,"utf8")); process.stdout.write(c.apiKey || "");' "$HOME/.config/genarrative/external-editor-api.json")" +curl -fsS "https://www.genarrative.world/api/external/v1/editor/projects" \ + -H "Authorization: Bearer $api_key" +``` + +The OpenAPI document, integration manifest, raw Skill entry, and Skill archive are public. Hosted MCP and all project, asset, upload, generation, and generation-query operations require the Bearer API Key. + +## Idempotency and Unknown Outcomes + +For each logical generation: + +1. Create one printable ASCII `Idempotency-Key` of 1-128 bytes. +2. Persist the key with the exact request body and returned `operationId`. +3. If submission transport fails or the response is lost, resend only the exact same body with the same key. +4. Never allocate a new key merely because the outcome is unknown. +5. On a polling timeout, retain `operationId` and query later. Do not submit another generation. + +Treat a different body under the same key as invalid. Do not automatically replay a failed terminal generation unless the user intentionally requests a new logical generation. + +## Local Reference Upload + +Do not ask the user to convert local files to base64. Upload from the Agent/client machine: + +1. Detect the original filename, MIME type, byte length, and image dimensions when relevant. +2. Create a ticket with `POST /api/external/v1/assets/direct-upload-tickets`. +3. POST all returned non-null `formFields` and the file part named `file` directly to `upload.host`. +4. Confirm the object with `POST /api/external/v1/assets/objects/confirm`. +5. Pass the confirmed stable `objectKey` to the selected editor operation. + +For a private reference image, use a ticket body shaped like: + +```json +{ + "legacyPrefix": "generated-character-drafts", + "pathSegments": ["editor", "external-editor-references"], + "fileName": "", + "contentType": "image/png", + "access": "private" +} +``` + +The default response exposes `upload`; an explicitly enveloped response exposes `data.upload`. Treat the returned host and form fields as opaque. Do not log the entire ticket or persist it longer than needed. + +Confirm with the actual file metadata: + +```json +{ + "objectKey": "", + "contentType": "image/png", + "contentLength": 12345, + "assetKind": "editor_reference_image", + "accessPolicy": "private" +} +``` + +`contentLength` is a JSON number in bytes, not a quoted string. Never invent `sourceWidth` or `sourceHeight`; read them from the local image or ask the user if they cannot be determined. + +For character animation, reuse a real canvas layer ID when available. For a local-only source, derive a stable synthetic `sourceLayerId`, such as `external-reference-hero`, from the filename and keep it unchanged across retries. + +The bundled helper implements ticket creation, a stdlib multipart upload, confirmation, dimension detection for common formats, and stable source-layer IDs: + +```python +from genarrative_external_api import GenarrativeExternalClient + +client = GenarrativeExternalClient() +reference = client.upload_reference_image("/path/to/reference.png") +print(reference["objectKey"]) +``` + +Do not print the complete confirmation response if it may contain temporary access data. Prefer passing the returned `objectKey` directly to the next call. + +## Stable and Temporary Media References + +- Use `objectKey`, project resource ID, asset ID, or an allowed durable public URL for generation input. +- Use a Data URL only when the endpoint explicitly allows it and the caller has a deliberate reason; do not persist it as a durable output. +- Never use a Blob URL outside the browser process that created it. +- Use `GET /api/external/v1/assets/read-url` to obtain a short-lived `signedUrl` for display/download. +- Never store or feed an expiring signed URL back into generation when a stable `objectKey` exists. + +## Logging and Command Safety + +- Never place an API Key in repository files, generated projects, command arguments containing a literal key, docs, commits, screenshots, stack traces, test fixtures, or telemetry. +- Redact `Authorization`, API Key values, upload signatures, cookies, signed URL query strings, and private absolute paths from logs and user-visible errors. +- Do not print credentials while diagnosing JSON configuration. Report only presence/absence and safe validation errors. +- Do not commit the credentials file or copy it into the Skill archive. +- Do not expose provider diagnostics, worker leases, queue internals, or server filesystem paths returned by an unexpected error. + +## Scope and Retry Guardrails + +- Do not use account JWT/profile endpoints as the default external integration. Logged-in profile APIs may create/revoke developer keys, but they are outside this external editor contract. +- Do not call internal workers, queues, SpacetimeDB, or admin endpoints. +- Do not bypass upload confirmation or invent an object key. +- Do not retry post-processing locally by fabricating assets. Respect completed warning semantics from `requests-and-outputs.md`. +- Use bounded polling. A local wait budget ending does not cancel or fail the server operation. diff --git a/.codex/skills/genarrative-external-editor-api/references/capability-routing.md b/.codex/skills/genarrative-external-editor-api/references/capability-routing.md new file mode 100644 index 000000000..b89050080 --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/references/capability-routing.md @@ -0,0 +1,87 @@ +# Capability Routing + +Use this reference to translate user intent into a hosted MCP tool or its corresponding External v1 REST operation. Use `genarrative://external-editor/openapi` or `GET /api/external/v1/openapi.json` for exact schemas. + +## Integration Surface + +- Fixed production base URL: `https://www.genarrative.world/`. +- Discovery manifest: `GET /api/external/v1/agent-integration.json`. +- Hosted MCP: `/api/external/v1/mcp`, Streamable HTTP, authenticated with the same Bearer API Key as REST. +- Public contract: `GET /api/external/v1/openapi.json`. +- Skill fallback: `GET /api/external/v1/skill/SKILL.md` or `GET /api/external/v1/skill.zip`. + +Prefer MCP when the Agent supports a remote endpoint plus a custom Bearer token. Prefer the complete Skill and Python helper when MCP is unavailable or a client-side local-file upload must be orchestrated. The MCP tool names are derived from OpenAPI `operationId` values in snake case; select by capability instead of memorizing the name. + +## Canvas Session + +Before the first generation in a new conversation, obtain a canvas name unless the user already supplied an existing `projectId` and `assetFolderId`. + +1. List or create a project. When creating one, use the canvas name as `title`. +2. Read the asset library. Reuse a folder with the same label or create one with the canvas name. +3. Retain `canvasName`, `projectId`, `assetFolderId`, and the current art spec in conversation state. + +Generated artifacts must enter both the current canvas and its same-name library folder whenever the endpoint supports that invariant. Pass `projectId`, `assetFolderId`, the endpoint's label field, and `canvasCompletion`. Character animation may return no direct library asset; after completion, create one from the first returned frame only when the compact result still lacks an asset. + +## Art Spec Routing + +Before art generation, normalize the user's request into: + +```json +{ + "assetType": "character | background | prop | ui | icon | animation | video | audio", + "subject": "要生成的主体", + "style": "画风、材质、时代或参考风格", + "palette": "主色与禁用色", + "composition": "构图、镜头、姿态或布局", + "format": "比例、尺寸、分辨率、帧数或时长", + "constraints": "必须保留、禁止出现、透明或绿幕要求", + "references": ["objectKey、资源 ID 或本地文件说明"] +} +``` + +Infer what is already clear and ask only for missing fields that block the selected endpoint. Reuse the current spec unless the user changes style, subject family, palette, format, or constraints. Store structured context under `generationInputs.artSpec` where supported and summarize it in the prompt when useful. + +## Intent Map + +| User intent | MCP/REST capability | +| --- | --- | +| Generate a background, character, spec, UI mockup, or publication image | Image generation | +| Redraw, retouch, or replace an existing image | Image edit | +| Generate from a local reference | Upload and confirm the local file, then image generation or edit | +| Build a reusable transparent icon/game atlas from a visual spec | Icon spritesheet generation | +| Extract marked assets from an existing UI design | UI design asset extraction | +| Animate a character into frames | Character animation generation | +| Generate video | Video generation | +| Generate a sound effect | Sound-effect generation | +| Generate background music/BGM | Background-music generation | +| Upload a local image/audio/video asset | Upload ticket -> OSS form upload -> object confirm | +| Save viewport/layers | Canvas save | +| Create, load, rename, or delete a canvas | Project operations | +| Organize folders and asset records | Asset-library operations | +| Obtain temporary access to private media | Signed read URL | +| Check generation progress or retrieve its result | Generation query | + +Do not present an API menu unless the request is genuinely ambiguous. Ask a follow-up when two routes create different artifacts, for example “处理这张图” could mean edit, extract marked UI assets, or use it as a reference for a new generation. + +## Route-Specific Decisions + +- Use image edit when the requested output replaces or modifies a source image. With `projectId`, pass `targetLayerId` to replace an existing layer when no explicit `canvasCompletion` is supplied. +- Use icon spritesheet generation for a transparent reusable atlas when a stable visual-spec reference and concrete `iconDescriptions` exist. Do not use ordinary image generation just because it can draw several objects. +- Use UI extraction only for an existing UI design image with red-box annotations. It is not UI generation. +- Use a project layer ID as character animation `sourceLayerId` when one exists. For a local-only source, derive a stable synthetic ID from the filename. +- For video with image/video/audio references, use a Seedance 2.0-family model; default to `seedance2.0-fast`, `mode: "std"`, and explicit `sound`. +- Use `signedUrl` only for preview/download. Feed stable `objectKey` or registered resource/asset identifiers into generation. + +## AI Game Creator Canonical Visual DAG + +Keep the existing autonomous-build task graph. Do not add a parallel task system or collapse these artifacts into one ordinary generation request: + +1. `art-director` generates `assets/art-spec.png` with image generation, `kind: "spec"`, then registers it as `assetKind: "icon-spec"`. This image is the authoritative visual spec; `generationInputs.artSpec` is supporting structured context. +2. `design-foundation` generates `assets/ui-prototype.png` with `kind: "ui-design"`, using the registered art-spec resource ID in `referenceImageSrcs`. +3. `art-asset-plan` generates transparent `assets/art-spritesheet.png` through icon spritesheet generation, using the same registered art-spec resource ID as `referenceImageSrc` plus concrete `iconDescriptions`. + +Never use `assets/ui-prototype.png` as the spritesheet visual-spec reference. UI extraction is outside this canonical DAG. + +## Scope Boundary + +Stay within `/api/external/v1`. Do not invent worker, queue, runtime task-list, admin, profile, or SpacetimeDB calls. The only external generation query is `GET /api/external/v1/generations/{operationId}`. diff --git a/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md b/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md new file mode 100644 index 000000000..cbd26dfa7 --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md @@ -0,0 +1,236 @@ +# Requests and Outputs + +Use this reference to build generation payloads, carry canvas/library context, poll asynchronous jobs, and interpret compact completed results. Verify exact schemas against `GET /api/external/v1/openapi.json`. + +## Contents + +- [Asynchronous Submission](#asynchronous-submission) +- [Polling State Machine](#polling-state-machine) +- [Canvas and Asset-Library Completion](#canvas-and-asset-library-completion) +- [Art Spec and Image Request](#art-spec-and-image-request) +- [Local Reference Requests](#local-reference-requests) +- [Compact Completed Result](#compact-completed-result) +- [Warning Semantics](#warning-semantics) +- [Output Handling Checklist](#output-handling-checklist) + +## Asynchronous Submission + +All eight generation POST routes require `Idempotency-Key` and return HTTP `202` with an `ExternalEditorGenerationSubmissionResponse` shaped like: + +```json +{ + "operationId": "task-...", + "kind": "editor_image_generation", + "status": "queued", + "statusUrl": "/api/external/v1/generations/task-...", + "pollAfterMs": 1500, + "updatedAtMicros": 1785456000000000 +} +``` + +The response acknowledges durable submission only. It is never the completed media response. + +Submit with one stable key per logical request: + +```bash +api="https://www.genarrative.world" +credentials_file="$HOME/.config/genarrative/external-editor-api.json" +api_key="$(node -e 'const fs=require("fs"); const p=process.argv[1]; const c=JSON.parse(fs.readFileSync(p,"utf8")); process.stdout.write(c.apiKey || "");' "$credentials_file")" +idempotency_key="$(node -e 'process.stdout.write(require("node:crypto").randomUUID())')" + +submission="$(curl -fsS "$api/api/external/v1/editor/images/generations" \ + -H "Authorization: Bearer $api_key" \ + -H "Content-Type: application/json" \ + -H "Idempotency-Key: $idempotency_key" \ + -d @request.json)" +operation_id="$(node -e 'const v=JSON.parse(process.argv[1]); process.stdout.write(v.operationId || v.data?.operationId || "")' "$submission")" +``` + +Persist the key, exact request body, and `operationId`. If submission outcome is uncertain, reuse the same body and key; do not submit a replacement key. + +## Polling State Machine + +Poll `statusUrl`, or `GET /api/external/v1/generations/{operationId}`, no faster than `pollAfterMs`: + +- `queued` / `running`: retain `operationId`; show `phaseLabel`, `phaseDetail`, and `progress` when present; wait before querying again. +- `completed`: consume the compact `result` and all warning fields, then stop polling. +- `failed`: surface the safe `error`, stop polling, and do not infer provider or worker internals. + +A caller-side timeout leaves the operation pending. Persist the ID for later query. Do not keep the original POST connection open and do not infer failure from a local wait budget. + +The helper's convenience generation methods block only in the local process while sending short submit and status requests. Its default overall wait budget is 1800 seconds. For explicit orchestration: + +```python +submission = client.submit_generation( + "/api/external/v1/editor/images/generations", + request_body, + idempotency_key=stable_key, +) +operation_id = submission["operationId"] +status = client.get_generation(operation_id) +completed = client.wait_for_generation(operation_id) +``` + +## Canvas and Asset-Library Completion + +For endpoints that support these fields, include: + +- `projectId`: target canvas project. +- `assetFolderId`: folder whose label matches the canvas name. +- `assetLabel` or UI extraction's `spritesheetLabel`: user-visible library label. +- `canvasCompletion`: backend canvas placement instructions. + +A minimal `canvasCompletion` is: + +```json +{ + "title": "素材名称", + "placeholder": { + "x": 0, + "y": 0, + "width": 1024, + "height": 576, + "originalWidth": 1024, + "originalHeight": 576 + } +} +``` + +`dialogId` is optional. Do not reconstruct canvas state from completion results. Reload the project and asset library when complete authoritative snapshots are needed. + +Character animation may complete without a direct `asset` field. To preserve the canvas/library invariant, create a library asset from the first returned frame only if the compact result lacks one. Prefer `client.animate_character(..., canvasSession=session, canvasTitle="...")`, which implements this fallback. + +## Art Spec and Image Request + +Carry the current art spec in `generationInputs.artSpec` and reflect important constraints in the prompt: + +```json +{ + "prompt": "一张横版幻想森林背景，适合游戏主视觉，无文字", + "aspectRatio": "16:9", + "imageSize": "1K", + "projectId": "", + "assetFolderId": "", + "assetLabel": "森林背景", + "generationInputs": { + "artSpec": { + "assetType": "background", + "subject": "幻想森林主视觉", + "style": "手绘游戏概念图", + "palette": "翡翠绿与金色光斑", + "composition": "横版，中心留出角色站位", + "format": "16:9, 1K", + "constraints": "无文字、无 UI 按钮", + "references": [] + } + }, + "canvasCompletion": { + "title": "森林背景", + "placeholder": { + "x": 0, + "y": 0, + "width": 1024, + "height": 576, + "originalWidth": 1024, + "originalHeight": 576 + } + } +} +``` + +The top-level `style` field is not the art spec's visual-style prose. It controls deterministic post-processing: + +- Omitted, `null`, empty string, or `"none"`: disable post-processing without warning. +- `"pixelArt"`: enable pixel-art snapping for ordinary image generation, `kind: "character"`, and icon spritesheet generation. +- Unknown strings, or `"pixelArt"` on unsupported kinds such as `spec`, `quick-edit`, `ui-design`, or `publication-material`: continue without style processing and return `warning.code: "unsupported-image-style"`. +- Non-string JSON values: malformed request, HTTP `400`. + +Keep this field extensible. Do not impose a closed client enum beyond the server contract. + +## Local Reference Requests + +Upload and confirm a local file before generation, then use the stable `objectKey`: + +```python +client = GenarrativeExternalClient() +session = client.prepare_canvas_session("新画板") +reference = client.upload_reference_image("/path/to/reference.png") +client.generate_image( + "基于参考图生成一张 16:9 游戏背景", + canvasSession=session, + assetLabel="参考图背景", + aspectRatio="16:9", + imageSize="1K", + referenceImageSrcs=[reference["objectKey"]], +) +``` + +For character animation from a local-only source, use actual dimensions and a stable synthetic layer ID: + +```json +{ + "sourceLayerId": "external-reference-hero", + "sourceImageSrc": "", + "sourceWidth": 720, + "sourceHeight": 1280, + "promptText": "让角色自然呼吸并轻微转身", + "resolution": "720p", + "ratio": "9:16", + "frameCount": 40, + "durationSeconds": 5, + "model": "seedance2.0-fast" +} +``` + +Do not guess dimensions or pass a temporary signed read URL. See `authentication-and-safety.md` for upload and credential rules. + +## Compact Completed Result + +The completed `result` may contain stable artifact fields such as: + +- `objectKey`, media type, dimensions, or task ID. +- `resource`, `resourceId`, or equivalent canvas reference. +- `asset`, `assetId`, or equivalent library reference. +- `spritesheetResource`, `spritesheetAsset`, and stable spritesheet metadata. +- `warning` and `sliceWarning` structures. + +It deliberately excludes a complete project/canvas/library snapshot, Data URL, Blob URL, expiring signed URL, worker lease, queue state, and internal provider diagnostics. Use `/assets/read-url` for temporary access to a stable `objectKey`. + +## Warning Semantics + +Interpret warnings only after the query reaches `status=completed`. The query-level `warning` is display-ready text. Compact `result.warning` and `result.sliceWarning` preserve structured artifact semantics. + +### Source-preserved post-processing failure + +When `result.warning.code` is `postprocess-failed-source-preserved`: + +- Treat the saved provider source as the authoritative main result. +- For character output, do not claim a transparent derivative. +- For icon spritesheet or UI extraction, do not claim a transparent spritesheet or individual slices. +- Display the safe reason. +- Do not fabricate derivatives or restart generation automatically. + +Use `resource` / `asset` for character results and `spritesheetResource` / `spritesheetAsset` for icon/UI results, then reload authoritative project/library state. + +### Slice failure after transparent-sheet success + +`result.sliceWarning` means transparent spritesheet post-processing succeeded but automatic splitting failed: + +- Continue using the complete transparent spritesheet. +- Do not claim individual slices. +- Display the slice reason. + +### Coexisting warnings + +`warning` and `sliceWarning` are mutually exclusive only for `postprocess-failed-source-preserved`, because that path never reaches slicing. A general warning from unsupported style normalization or pixel-art snapping can coexist with `sliceWarning`. Render both reasons. + +Before registering a requested transparent deliverable, verify the full sheet actually contains transparency. If source-preserved warning is present, do not register the opaque provider source as the requested transparent atlas. If only `sliceWarning` is present, the transparent full sheet remains valid. + +## Output Handling Checklist + +1. Require terminal `completed` before consuming artifacts. +2. Preserve stable IDs and `objectKey` values. +3. Surface all warning channels without downgrading completion to failure. +4. Avoid claiming absent transparent derivatives or slices. +5. Obtain temporary preview/download URLs only through `/assets/read-url`. +6. Reload authoritative project and library state when downstream logic needs complete records. diff --git a/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py b/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py index a099fd8c9..0e45ccf6d 100644 --- a/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py +++ b/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py @@ -11,6 +11,7 @@ import re import struct import sys import tempfile +import time import urllib.error import urllib.parse import urllib.request @@ -22,7 +23,7 @@ from typing import Any BASE_URL = "https://www.genarrative.world/" DEFAULT_CREDENTIALS_FILE = Path.home() / ".config/genarrative/external-editor-api.json" DEFAULT_REQUEST_TIMEOUT_SECONDS = 60 -GENERATION_REQUEST_TIMEOUT_SECONDS = 420 +GENERATION_WAIT_TIMEOUT_SECONDS = 1800 class GenarrativeApiError(RuntimeError): @@ -125,17 +126,18 @@ class GenarrativeExternalClient: query: dict[str, Any] | None = None, auth: bool = True, timeout: int = DEFAULT_REQUEST_TIMEOUT_SECONDS, + headers: dict[str, str] | None = None, ) -> Any: url = f"{self.base_url}{path}" if query: url = f"{url}?{urllib.parse.urlencode({k: v for k, v in query.items() if v is not None})}" data = None if body is None else json.dumps(body).encode("utf-8") - headers = {"Accept": "application/json"} + request_headers = {"Accept": "application/json", **(headers or {})} if data is not None: - headers["Content-Type"] = "application/json" + request_headers["Content-Type"] = "application/json" if auth: - headers["Authorization"] = f"Bearer {self.api_key}" - request = urllib.request.Request(url, data=data, headers=headers, method=method.upper()) + request_headers["Authorization"] = f"Bearer {self.api_key}" + request = urllib.request.Request(url, data=data, headers=request_headers, method=method.upper()) try: with urllib.request.urlopen(request, timeout=timeout) as response: payload = response.read() @@ -440,24 +442,120 @@ class GenarrativeExternalClient: def read_url(self, object_key: str) -> Any: return self.request_json("GET", "/api/external/v1/assets/read-url", query={"objectKey": object_key}) + def submit_generation( + self, + path: str, + body: dict[str, Any], + idempotency_key: str | None = None, + ) -> dict[str, Any]: + key = normalize_optional_text(idempotency_key) or str(uuid.uuid4()) + submission = None + for attempt in range(2): + try: + submission = self.request_json( + "POST", + path, + body, + headers={"Idempotency-Key": key}, + ) + break + except (urllib.error.URLError, TimeoutError) as error: + if attempt == 0: + time.sleep(0.5) + continue + raise GenarrativeApiError( + "Generation submission transport outcome is unknown. " + f"Retry the same body with Idempotency-Key {key}; do not create a new key." + ) from error + if not isinstance(submission, dict) or not normalize_optional_text(submission.get("operationId")): + raise GenarrativeApiError("Generation submission response missing operationId.") + submission.setdefault("idempotencyKey", key) + return submission + + def get_generation(self, operation_id: str) -> dict[str, Any]: + result = self.request_json( + "GET", + f"/api/external/v1/generations/{urllib.parse.quote(operation_id, safe='')}", + ) + if not isinstance(result, dict): + raise GenarrativeApiError("Generation status response must be an object.") + return result + + def wait_for_generation( + self, + submission_or_operation_id: dict[str, Any] | str, + timeout_seconds: int = GENERATION_WAIT_TIMEOUT_SECONDS, + ) -> dict[str, Any]: + operation_id = ( + submission_or_operation_id.get("operationId") + if isinstance(submission_or_operation_id, dict) + else submission_or_operation_id + ) + operation_id = normalize_optional_text(operation_id) + if not operation_id: + raise GenarrativeApiError("Generation operationId is required.") + deadline = time.monotonic() + max(1, timeout_seconds) + while True: + try: + job = self.get_generation(operation_id) + except GenarrativeApiError as error: + if any(f"HTTP {status}" in str(error) for status in (429, 502, 503, 504)): + if time.monotonic() >= deadline: + raise GenarrativeApiError( + f"Generation {operation_id} is still running; keep this operationId and continue polling." + ) from error + time.sleep(1.5) + continue + raise + status = job.get("status") + if status == "completed": + result = job.get("result") + if not isinstance(result, dict): + raise GenarrativeApiError( + f"Generation {operation_id} completed without a result payload." + ) + return result + if status == "failed": + raise GenarrativeApiError( + f"Generation {operation_id} failed: {job.get('error') or 'unknown error'}" + ) + if time.monotonic() >= deadline: + raise GenarrativeApiError( + f"Generation {operation_id} is still running; keep this operationId and continue polling." + ) + poll_after_ms = job.get("pollAfterMs", 1500) + if not isinstance(poll_after_ms, (int, float)): + poll_after_ms = 1500 + time.sleep(max(0.25, min(float(poll_after_ms) / 1000.0, 5.0))) + + def submit_and_wait_generation( + self, + path: str, + body: dict[str, Any], + idempotency_key: str | None = None, + timeout_seconds: int = GENERATION_WAIT_TIMEOUT_SECONDS, + ) -> dict[str, Any]: + submission = self.submit_generation(path, body, idempotency_key=idempotency_key) + return self.wait_for_generation(submission, timeout_seconds=timeout_seconds) + def generate_image(self, prompt: str, **fields: Any) -> Any: self._apply_canvas_session_fields(fields, prompt, 1024, 1024) prompt = self._apply_art_spec(fields, prompt) - return self.request_json( - "POST", + idempotency_key = fields.pop("idempotencyKey", None) + return self.submit_and_wait_generation( "/api/external/v1/editor/images/generations", {"prompt": prompt, **fields}, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) def edit_image(self, prompt: str, source_image_src: str, **fields: Any) -> Any: self._apply_canvas_session_fields(fields, prompt, 1024, 1024) prompt = self._apply_art_spec(fields, prompt) - return self.request_json( - "POST", + idempotency_key = fields.pop("idempotencyKey", None) + return self.submit_and_wait_generation( "/api/external/v1/editor/images/edits", {"prompt": prompt, "sourceImageSrc": source_image_src, **fields}, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) def generate_icon_spritesheet( @@ -472,25 +570,25 @@ class GenarrativeExternalClient: label = fields.get("assetLabel", "图标图集") self._apply_canvas_session_fields(fields, label, 1024, 1024) fields.setdefault("screenColor", "auto") - return self.request_json( - "POST", + idempotency_key = fields.pop("idempotencyKey", None) + return self.submit_and_wait_generation( "/api/external/v1/editor/icon-spritesheets/generations", { "referenceImageSrc": reference_image_src, "iconDescriptions": descriptions, **fields, }, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) def extract_ui_assets(self, source_image_src: str, image_size: str = "1K", **fields: Any) -> Any: fields.pop("aspectRatio", None) self._apply_canvas_session_fields(fields, fields.get("spritesheetLabel", "UI 素材拆分"), 1024, 1024, "spritesheetLabel") - return self.request_json( - "POST", + idempotency_key = fields.pop("idempotencyKey", None) + return self.submit_and_wait_generation( "/api/external/v1/editor/ui-designs/assets/extractions", {"sourceImageSrc": source_image_src, "imageSize": image_size, **fields, "aspectRatio": "1:1"}, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) def animate_character( @@ -512,6 +610,7 @@ class GenarrativeExternalClient: prompt_text = self._apply_art_spec(fields, prompt_text) fields.pop("assetFolderId", None) fields.pop("assetLabel", None) + idempotency_key = fields.pop("idempotencyKey", None) body = { "sourceLayerId": source_layer_id, "sourceImageSrc": source_image_src, @@ -525,11 +624,10 @@ class GenarrativeExternalClient: **fields, "model": "seedance2.0-fast", } - result = self.request_json( - "POST", + result = self.submit_and_wait_generation( "/api/external/v1/editor/character-animations/generations", body, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) if isinstance(session, dict) and isinstance(result, dict) and not result.get("asset"): frames = result.get("frames") @@ -559,6 +657,7 @@ class GenarrativeExternalClient: fields.pop("mode", None) self._apply_canvas_session_fields(fields, prompt, 1280, 720) prompt = self._apply_art_spec(fields, prompt) + idempotency_key = fields.pop("idempotencyKey", None) body = { "prompt": prompt, "model": fields.pop("model", "seedance2.0-fast"), @@ -569,31 +668,30 @@ class GenarrativeExternalClient: **fields, "mode": "std", } - return self.request_json( - "POST", + return self.submit_and_wait_generation( "/api/external/v1/editor/videos/generations", body, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) def generate_sound_effect(self, prompt: str, duration: int, **fields: Any) -> Any: self._apply_canvas_session_fields(fields, prompt, 360, 120) prompt = self._apply_art_spec(fields, prompt) - return self.request_json( - "POST", + idempotency_key = fields.pop("idempotencyKey", None) + return self.submit_and_wait_generation( "/api/external/v1/editor/audios/sound-effects/generations", {"prompt": prompt, "duration": duration, **fields}, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) def generate_background_music(self, description: str, **fields: Any) -> Any: self._apply_canvas_session_fields(fields, description, 360, 120) description = self._apply_art_spec(fields, description) - return self.request_json( - "POST", + idempotency_key = fields.pop("idempotencyKey", None) + return self.submit_and_wait_generation( "/api/external/v1/editor/audios/background-music/generations", {"gptDescriptionPrompt": description, **fields, "makeInstrumental": True}, - timeout=GENERATION_REQUEST_TIMEOUT_SECONDS, + idempotency_key=idempotency_key, ) @@ -625,17 +723,29 @@ def _self_test() -> None: query: dict[str, Any] | None = None, auth: bool = True, timeout: int = DEFAULT_REQUEST_TIMEOUT_SECONDS, + headers: dict[str, str] | None = None, ) -> Any: - calls.append({"method": method, "path": path, "body": body, "timeout": timeout}) + calls.append({ + "method": method, + "path": path, + "body": body, + "timeout": timeout, + "headers": headers, + }) if path == "/api/external/v1/editor/assets": return {"asset": {"assetId": "editor-asset-demo"}} - return { + generated = { "taskId": "task-demo", "model": "seedance2.0-fast", "prompt": "角色呼吸", "previewVideoPath": "/generated/preview.mp4", "frames": [{"frameIndex": 1, "imageSrc": "/generated/frame01.png", "width": 512, "height": 768}], } + if method == "POST": + return {"operationId": "task-operation-demo", "status": "queued", "pollAfterMs": 1} + if path == "/api/external/v1/generations/task-operation-demo": + return {"operationId": "task-operation-demo", "status": "completed", "result": generated} + return generated client.request_json = fake_request_json # type: ignore[method-assign] result = client.animate_character( @@ -647,10 +757,12 @@ def _self_test() -> None: canvasSession=session, canvasTitle="角色呼吸动画", ) - assert calls[0]["timeout"] == GENERATION_REQUEST_TIMEOUT_SECONDS + assert calls[0]["timeout"] == DEFAULT_REQUEST_TIMEOUT_SECONDS + assert calls[0]["headers"]["Idempotency-Key"] assert calls[0]["body"]["projectId"] == "proj-demo" assert calls[0]["body"]["canvasCompletion"]["title"] == "角色呼吸动画" - assert calls[1]["path"] == "/api/external/v1/editor/assets" + assert calls[1]["path"] == "/api/external/v1/generations/task-operation-demo" + assert calls[2]["path"] == "/api/external/v1/editor/assets" assert result["asset"]["assetId"] == "editor-asset-demo" calls.clear() client.generate_icon_spritesheet( @@ -663,6 +775,7 @@ def _self_test() -> None: assert calls[0]["body"]["referenceImageSrc"] == "editor-resource-spec" assert calls[0]["body"]["screenColor"] == "auto" assert calls[0]["body"]["iconDescriptions"][0] == "蛇头向上" + assert calls[1]["path"] == "/api/external/v1/generations/task-operation-demo" print("self-test ok") diff --git a/AGENTS.md b/AGENTS.md index 21884a813..694ec26fe 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -45,6 +45,7 @@ - DDD 分层边界按总纲执行：领域规则沉到 `module-*`，SpacetimeDB 表和事务编排留在 `spacetime-module`，后端访问 SpacetimeDB 统一经 `spacetime-client` facade，HTTP/SSE/BFF 留在 `api-server`，外部副作用留在 `platform-*`，前后端 DTO 留在 `shared-contracts`。 - 前端只做表现、交互和临时 UI 状态，不承接正式业务真相，不绕过后端投影或后端 API 直接实现业务规则。 - 契约、路由、DTO 去留和 breaking change 以当前后端架构文档、`server-rs/crates/api-server/src/app.rs`、`shared-contracts` 和 `packages/shared` 为准；不得在前端、`api-server` 或临时兼容层中重新发明旧接口。 +- 凡修改 `/api/external/v1` 的路由、HTTP 方法、请求 / 响应 DTO、请求头、状态码、鉴权或异步语义，必须在同一次变更中同步更新权威契约 [`docs/openapi/genarrative-external-v1.openapi.json`](docs/openapi/genarrative-external-v1.openapi.json) 及对应契约测试；Rust 实现与 OpenAPI 未保持一致时任务不得视为完成。 - SpacetimeDB 已有表新增字段时，字段必须放在 Rust 表结构体最后，并设置明确默认值；需要删除、改名、重排或改类型时，必须先询问用户并确认迁移计划。 - 修改 SpacetimeDB schema 后必须同步 `migration.rs`、表目录和生成绑定，并运行 `npm run check:spacetime-schema`。 - 除 CI/CD 脚本内部受控用法外，人工命令、本地联调、排障步骤和文档示例禁止继续使用 `spacetime --root-dir`。 diff --git a/apps/ai-game-creator-shell/scripts/check-config.mjs b/apps/ai-game-creator-shell/scripts/check-config.mjs index df99a183c..91e8f5e2d 100644 --- a/apps/ai-game-creator-shell/scripts/check-config.mjs +++ b/apps/ai-game-creator-shell/scripts/check-config.mjs @@ -1485,8 +1485,15 @@ for (const snippet of [ 'assertSafeGameCreatorConfigDestination', 'readGameCreatorWizardConfigState', '$security.SetAccessRuleProtection($true, $false)', + '$targetItem = Get-Item -LiteralPath $target -Force', + '$targetItem.SetAccessControl($security)', + '$verified = $targetItem.GetAccessControl()', '$rules.Count -ne 1', '[System.Security.AccessControl.FileSystemRights]::FullControl', + "runChildCapture('powershell.exe'", + "'-NoProfile'", + "'-Command'", + 'windowsPrivateAclScript', 'await secureWindowsPath(temporaryPath, { isDirectory: false })', 'await temporaryFile.writeFile', ]) { @@ -1496,6 +1503,11 @@ for (const snippet of [ ); } } +if (/\bGet-Acl\b/u.test(configWizardSource)) { + throw new Error( + 'AI game creator config wizard must not rely on Get-Acl module auto-loading', + ); +} await runConfigWizardRegressionChecks(); await runHiddenInputRegressionChecks(); diff --git a/apps/ai-game-creator-shell/scripts/deterministic-lane-defense-provider.mjs b/apps/ai-game-creator-shell/scripts/deterministic-lane-defense-provider.mjs index 98bf22ef3..7cd45583a 100644 --- a/apps/ai-game-creator-shell/scripts/deterministic-lane-defense-provider.mjs +++ b/apps/ai-game-creator-shell/scripts/deterministic-lane-defense-provider.mjs @@ -2370,6 +2370,7 @@ function createDeterministicCanvasFixture(apiKey) { const projectId = 'deterministic-canvas-project'; const folderId = 'deterministic-canvas-folder'; const images = new Map(); + const generationOperations = new Map(); const imageCache = new Map(); const stats = { canvasApiRequestCount: 0, @@ -2476,6 +2477,16 @@ function createDeterministicCanvasFixture(apiKey) { request.method === 'POST' && parsed.pathname === '/api/external/v1/editor/images/generations' ) { + const idempotencyKey = request.headers['idempotency-key']; + if ( + typeof idempotencyKey !== 'string' || + !/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test( + idempotencyKey, + ) + ) { + request.resume(); + return json(400, { error: { message: 'invalid idempotency key' } }); + } const body = await readJsonBody(request); const image = imageForAspectRatio(body?.aspectRatio); generationSequence += 1; @@ -2489,8 +2500,22 @@ function createDeterministicCanvasFixture(apiKey) { const assetKind = typeof body?.assetKind === 'string' ? body.assetKind : 'game-art'; images.set(imageId, { ...image, objectKey }); - return json(200, { - data: { + const operationId = `task-${imageId}`; + generationOperations.set(operationId, { + imageSrc: `/${objectKey}`, + objectKey, + assetObjectId, + width: image.width, + height: image.height, + sourceType: 'generated', + prompt: body?.prompt ?? 'deterministic canvas fixture', + actualPrompt: body?.prompt ?? 'deterministic canvas fixture', + model: 'deterministic-canvas-v1', + provider: 'deterministic-loopback', + taskId: `task-${imageId}`, + resource: { + resourceId, + projectId, imageSrc: `/${objectKey}`, objectKey, assetObjectId, @@ -2502,22 +2527,45 @@ function createDeterministicCanvasFixture(apiKey) { model: 'deterministic-canvas-v1', provider: 'deterministic-loopback', taskId: `task-${imageId}`, - resource: { - resourceId, - projectId, - imageSrc: `/${objectKey}`, - objectKey, - assetObjectId, - width: image.width, - height: image.height, - sourceType: 'generated', - assetKind, - }, - asset: { - assetId: `asset-${imageId}`, - assetObjectId, - assetKind, - }, + assetKind, + }, + asset: { + assetId: `asset-${imageId}`, + assetObjectId, + assetKind, + }, + }); + return json(202, { + data: { + operationId, + kind: 'editor_image_generation', + status: 'queued', + statusUrl: `/api/external/v1/generations/${operationId}`, + pollAfterMs: 1, + updatedAtMicros: generationSequence, + }, + }); + } + if ( + request.method === 'GET' && + parsed.pathname.startsWith('/api/external/v1/generations/') + ) { + request.resume(); + const operationId = parsed.pathname.slice( + '/api/external/v1/generations/'.length, + ); + const result = generationOperations.get(operationId); + if (!result) return json(404, { error: { message: 'operation not found' } }); + return json(200, { + data: { + operationId, + kind: 'editor_image_generation', + status: 'completed', + phaseLabel: '图片画布生成图片', + phaseDetail: '生成已完成。', + progress: 100, + result, + updatedAtMicros: generationSequence, }, }); } diff --git a/apps/ai-game-creator-shell/scripts/game-creator-config-wizard.mjs b/apps/ai-game-creator-shell/scripts/game-creator-config-wizard.mjs index 0d0522f22..dee615c1e 100644 --- a/apps/ai-game-creator-shell/scripts/game-creator-config-wizard.mjs +++ b/apps/ai-game-creator-shell/scripts/game-creator-config-wizard.mjs @@ -51,9 +51,10 @@ $rule = [System.Security.AccessControl.FileSystemAccessRule]::new( [System.Security.AccessControl.AccessControlType]::Allow ) $security.AddAccessRule($rule) | Out-Null -(Get-Item -LiteralPath $target -Force).SetAccessControl($security) +$targetItem = Get-Item -LiteralPath $target -Force +$targetItem.SetAccessControl($security) -$verified = Get-Acl -LiteralPath $target +$verified = $targetItem.GetAccessControl() $owner = $verified.GetOwner([System.Security.Principal.SecurityIdentifier]) $rules = @($verified.GetAccessRules($true, $true, [System.Security.Principal.SecurityIdentifier])) if (-not $owner.Equals($currentSid) -or -not $verified.AreAccessRulesProtected -or $rules.Count -ne 1) { diff --git a/apps/ai-game-creator-shell/scripts/start-dev-stack.mjs b/apps/ai-game-creator-shell/scripts/start-dev-stack.mjs index 81fae0bee..562e9baaf 100644 --- a/apps/ai-game-creator-shell/scripts/start-dev-stack.mjs +++ b/apps/ai-game-creator-shell/scripts/start-dev-stack.mjs @@ -1,5 +1,5 @@ import { spawn } from 'node:child_process'; -import { existsSync, readFileSync } from 'node:fs'; +import { existsSync, readdirSync, readFileSync } from 'node:fs'; import http from 'node:http'; import net from 'node:net'; import { resolve } from 'node:path'; @@ -300,16 +300,69 @@ function stopChild(child, signal = 'SIGTERM') { } } -function isProcessGroupAlive(processGroupId, killImpl = process.kill) { +function readLinuxProcessGroupAlive( + processGroupId, + { readdirImpl = readdirSync, readFileImpl = readFileSync } = {}, +) { + let processIds; + try { + processIds = readdirImpl('/proc'); + } catch { + return null; + } + + for (const processId of processIds) { + if (!/^\d+$/.test(processId)) { + continue; + } + let stat; + try { + stat = readFileImpl(`/proc/${processId}/stat`, 'utf8'); + } catch { + continue; + } + const commandEnd = stat.lastIndexOf(') '); + if (commandEnd < 0) { + continue; + } + const [state, , processGroup] = stat + .slice(commandEnd + 2) + .trim() + .split(/\s+/); + if ( + Number(processGroup) === processGroupId && + state !== 'Z' && + state !== 'X' + ) { + return true; + } + } + return false; +} + +function isProcessGroupAlive( + processGroupId, + { + platform = process.platform, + killImpl = process.kill, + readLinuxGroupAlive = readLinuxProcessGroupAlive, + } = {}, +) { if (!Number.isInteger(processGroupId)) { return false; } try { killImpl(-processGroupId, 0); - return true; } catch (error) { return error?.code !== 'ESRCH'; } + if (platform === 'linux') { + const linuxGroupAlive = readLinuxGroupAlive(processGroupId); + if (typeof linuxGroupAlive === 'boolean') { + return linuxGroupAlive; + } + } + return true; } async function waitUntil(check, timeoutMs, pollIntervalMs = 25) { @@ -415,7 +468,7 @@ async function terminateChildTree( stopChild(child, 'SIGTERM'); if ( await waitUntil( - () => !isProcessGroupAlive(processGroupId, killImpl), + () => !isProcessGroupAlive(processGroupId, { platform, killImpl }), gracefulTimeoutMs, ) ) { @@ -430,7 +483,7 @@ async function terminateChildTree( } } const stopped = await waitUntil( - () => !isProcessGroupAlive(processGroupId, killImpl), + () => !isProcessGroupAlive(processGroupId, { platform, killImpl }), forceTimeoutMs, ); return { stopped, forced: true }; @@ -600,8 +653,10 @@ export { ensureBackend, formatChildFailure, isDirectModuleExecution, + isProcessGroupAlive, preflightExistingVite, readChildFailure, + readLinuxProcessGroupAlive, resolveBackendTargetsFromState, runWindowsTaskkill, spawnChild, diff --git a/apps/ai-game-creator-shell/src-tauri/Cargo.lock b/apps/ai-game-creator-shell/src-tauri/Cargo.lock index dc4f2819d..b5e4b8579 100644 --- a/apps/ai-game-creator-shell/src-tauri/Cargo.lock +++ b/apps/ai-game-creator-shell/src-tauri/Cargo.lock @@ -1507,6 +1507,7 @@ dependencies = [ "tokio", "unicode-normalization", "url", + "uuid", "windows-sys 0.61.2", "zip", ] diff --git a/apps/ai-game-creator-shell/src-tauri/Cargo.toml b/apps/ai-game-creator-shell/src-tauri/Cargo.toml index 5ef59f69a..990ddd864 100644 --- a/apps/ai-game-creator-shell/src-tauri/Cargo.toml +++ b/apps/ai-game-creator-shell/src-tauri/Cargo.toml @@ -36,6 +36,7 @@ tempfile = "3" tokio = { version = "1", features = ["io-util", "macros", "process", "rt-multi-thread", "signal", "sync", "time"] } url = "2" unicode-normalization = "0.1" +uuid = { version = "1", features = ["v4"] } zip = { version = "2", default-features = false, features = ["deflate"] } tauri-plugin-clipboard-manager = "2.3.2" diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent/generation/canvas_generation.rs b/apps/ai-game-creator-shell/src-tauri/src/agent/generation/canvas_generation.rs index d2dc1f727..74d514b2e 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent/generation/canvas_generation.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent/generation/canvas_generation.rs @@ -1,5 +1,11 @@ use super::*; +const EXTERNAL_GENERATION_POLL_TIMEOUT: Duration = Duration::from_secs(35 * 60); +const EXTERNAL_GENERATION_DEFAULT_POLL_AFTER_MS: u64 = 2_000; +const EXTERNAL_GENERATION_MAX_POLL_AFTER_MS: u64 = 5_000; +const EXTERNAL_GENERATION_SUBMIT_MAX_ATTEMPTS: usize = 3; +const EXTERNAL_GENERATION_SUBMIT_RETRY_BACKOFF_MS: u64 = 250; + pub(crate) fn project_canvas_asset_media_types(root: &Path) -> Vec { read_manifest_for_project(root) .map(|manifest| { @@ -270,6 +276,132 @@ async fn external_editor_json_request( .map_err(|error| format!("解析{action}响应失败：{error}")) } +fn external_generation_poll_after_ms(payload: &serde_json::Value) -> u64 { + external_editor_response_data(payload) + .get("pollAfterMs") + .and_then(serde_json::Value::as_u64) + .unwrap_or(EXTERNAL_GENERATION_DEFAULT_POLL_AFTER_MS) + .min(EXTERNAL_GENERATION_MAX_POLL_AFTER_MS) +} + +async fn wait_for_external_generation_result( + client: &reqwest::Client, + api_base_url: &str, + api_key: &str, + submission_payload: &serde_json::Value, +) -> Result { + let submission = external_editor_response_data(submission_payload); + let operation_id = json_string_field(submission, "operationId") + .ok_or_else(|| "外部图片生成提交响应缺少 operationId".to_string())?; + let operation_id_path = + url::form_urlencoded::byte_serialize(operation_id.as_bytes()).collect::(); + let status_url = format!("{api_base_url}/api/external/v1/generations/{operation_id_path}"); + let started_at = tokio::time::Instant::now(); + let mut poll_after_ms = external_generation_poll_after_ms(submission_payload); + + loop { + if started_at.elapsed() >= EXTERNAL_GENERATION_POLL_TIMEOUT { + return Err(format!( + "平台图片生成任务仍在执行，已停止本地等待；operationId={operation_id}" + )); + } + if poll_after_ms > 0 { + tokio::time::sleep(Duration::from_millis(poll_after_ms)).await; + } + let payload = match external_editor_json_request( + client.get(&status_url).bearer_auth(api_key), + "查询平台图片生成任务", + ) + .await + { + Ok(payload) => payload, + Err(error) + if !error.contains("HTTP ") + || [429, 502, 503, 504] + .iter() + .any(|status| error.contains(&format!("HTTP {status}"))) => + { + poll_after_ms = EXTERNAL_GENERATION_DEFAULT_POLL_AFTER_MS; + continue; + } + Err(error) => return Err(format!("{error}；operationId={operation_id}")), + }; + let generation = external_editor_response_data(&payload); + match json_string_field(generation, "status").as_deref() { + Some("completed") => { + return generation + .get("result") + .filter(|result| !result.is_null()) + .cloned() + .ok_or_else(|| { + format!( + "平台图片生成任务已完成但响应缺少 result；operationId={operation_id}" + ) + }); + } + Some("failed") => { + let error = json_string_field(generation, "error") + .or_else(|| json_string_field(generation, "phaseDetail")) + .unwrap_or_else(|| "生成任务失败".to_string()); + return Err(format!( + "平台图片生成任务失败：{error}；operationId={operation_id}" + )); + } + Some("queued" | "running") => { + poll_after_ms = external_generation_poll_after_ms(&payload); + } + Some(status) => { + return Err(format!( + "平台图片生成任务返回未知状态 {status}；operationId={operation_id}" + )); + } + None => { + return Err(format!( + "平台图片生成任务状态响应缺少 status；operationId={operation_id}" + )); + } + } + } +} + +async fn submit_external_generation_request( + client: &reqwest::Client, + api_base_url: &str, + endpoint: &str, + api_key: &str, + idempotency_key: &str, + request_body: &serde_json::Value, +) -> Result { + let mut last_error = None; + for attempt in 1..=EXTERNAL_GENERATION_SUBMIT_MAX_ATTEMPTS { + match client + .post(format!("{api_base_url}{endpoint}")) + .bearer_auth(api_key) + .header("Idempotency-Key", idempotency_key) + .json(request_body) + .send() + .await + { + Ok(response) => return Ok(response), + Err(error) => { + last_error = Some(error); + if attempt < EXTERNAL_GENERATION_SUBMIT_MAX_ATTEMPTS { + tokio::time::sleep(Duration::from_millis( + EXTERNAL_GENERATION_SUBMIT_RETRY_BACKOFF_MS * attempt as u64, + )) + .await; + } + } + } + } + Err(format!( + "请求平台图片生成失败：{}", + last_error + .map(|error| error.to_string()) + .unwrap_or_else(|| "未知传输错误".to_string()) + )) +} + async fn prepare_external_canvas_generation_context( root: &Path, client: &reqwest::Client, @@ -519,7 +651,10 @@ pub(in crate::agent) async fn request_platform_art_asset_with_options_at( prepared_output_path.and_then(|(_, _, replacement_fingerprint)| replacement_fingerprint); let api_base_url = resolve_canvas_sync_api_base_url(None)?; let api_key = resolve_canvas_sync_api_key(None)?; - let client = reqwest::Client::new(); + let client = reqwest::Client::builder() + .timeout(Duration::from_secs(60)) + .build() + .map_err(|error| format!("创建 External Editor HTTP 客户端失败：{error}"))?; let canvas_context = prepare_external_canvas_generation_context(root, &client, &api_base_url, &api_key).await?; let generation_prompt = build_platform_art_asset_prompt(prompt, briefs, options); @@ -582,22 +717,28 @@ pub(in crate::agent) async fn request_platform_art_asset_with_options_at( }), ) }; - let response = client - .post(format!("{api_base_url}{endpoint}")) - .bearer_auth(&api_key) - .json(&request_body) - .send() - .await - .map_err(|error| format!("请求平台图片生成失败：{error}"))?; + let idempotency_key = uuid::Uuid::new_v4().to_string(); + let response = submit_external_generation_request( + &client, + &api_base_url, + endpoint, + &api_key, + &idempotency_key, + &request_body, + ) + .await?; let status = response.status(); if !status.is_success() { return Err(format!("请求平台图片生成失败：HTTP {}", status.as_u16())); } - let payload = response + let submission_payload = response .json::() .await - .map_err(|error| format!("解析平台图片生成响应失败：{error}"))?; - let generated = payload.get("data").unwrap_or(&payload); + .map_err(|error| format!("解析平台图片生成提交响应失败：{error}"))?; + let generated = + wait_for_external_generation_result(&client, &api_base_url, &api_key, &submission_payload) + .await?; + let generated = &generated; if let Some(error) = platform_art_generation_postprocess_failure(generated) { return Err(error); } @@ -643,7 +784,8 @@ pub(in crate::agent) async fn request_platform_art_asset_with_options_at( let generated_prompt = json_string_field(generated, "actualPrompt") .or_else(|| json_string_field(generated, "prompt")) .or_else(|| json_string_field(resource, "actualPrompt")) - .or_else(|| json_string_field(resource, "prompt")); + .or_else(|| json_string_field(resource, "prompt")) + .or_else(|| Some(generation_prompt.clone())); let model = json_string_field(generated, "model").or_else(|| json_string_field(resource, "model")); let provider = json_string_field(generated, "provider") @@ -1023,6 +1165,49 @@ mod canvas_generation_tests { use super::*; use image::{codecs::png::PngEncoder, ColorType, ImageEncoder}; + fn read_test_http_request(stream: &mut std::net::TcpStream) -> String { + stream + .set_read_timeout(Some(Duration::from_secs(2))) + .expect("set request read timeout"); + let mut bytes = Vec::new(); + let mut buffer = [0_u8; 4096]; + loop { + let read = stream.read(&mut buffer).expect("read request bytes"); + if read == 0 { + break; + } + bytes.extend_from_slice(&buffer[..read]); + let Some(header_end) = bytes.windows(4).position(|window| window == b"\r\n\r\n") else { + continue; + }; + let headers = String::from_utf8_lossy(&bytes[..header_end]); + let content_length = headers + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::().ok()) + .flatten() + }) + .unwrap_or(0); + if bytes.len() >= header_end + 4 + content_length { + break; + } + } + String::from_utf8(bytes).expect("request must be UTF-8") + } + + fn test_request_header<'a>(request: &'a str, expected_name: &str) -> &'a str { + request + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case(expected_name) + .then_some(value.trim()) + }) + .expect("expected request header") + } + fn rgba_test_png(alpha: u8) -> CanvasResourceDownload { let mut bytes = Vec::new(); PngEncoder::new(&mut bytes) @@ -1034,6 +1219,76 @@ mod canvas_generation_tests { } } + #[tokio::test] + async fn generation_submit_transport_retry_reuses_body_and_idempotency_key() { + let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("bind retry fixture"); + let base_url = format!("http://{}", listener.local_addr().expect("fixture address")); + let (sender, receiver) = std::sync::mpsc::channel(); + std::thread::spawn(move || { + for attempt in 0..2 { + let (mut stream, _) = listener.accept().expect("accept submit request"); + let request = read_test_http_request(&mut stream); + sender.send(request).expect("capture submit request"); + if attempt == 0 { + continue; + } + let body = serde_json::json!({ + "data": { + "operationId": "task-retry-1", + "status": "queued", + "pollAfterMs": 1 + } + }) + .to_string(); + let response = format!( + "HTTP/1.1 202 Accepted\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}", + body.len(), + body, + ); + stream + .write_all(response.as_bytes()) + .expect("write accepted response"); + } + }); + + let client = reqwest::Client::builder() + .timeout(Duration::from_secs(2)) + .build() + .expect("build retry client"); + let idempotency_key = uuid::Uuid::new_v4().to_string(); + let body = serde_json::json!({ "prompt": "stable retry" }); + let response = submit_external_generation_request( + &client, + &base_url, + "/generation", + "test-key", + &idempotency_key, + &body, + ) + .await + .expect("transport retry should succeed"); + assert_eq!(response.status(), reqwest::StatusCode::ACCEPTED); + + let first = receiver + .recv_timeout(Duration::from_secs(2)) + .expect("first request"); + let second = receiver + .recv_timeout(Duration::from_secs(2)) + .expect("retried request"); + assert_eq!( + test_request_header(&first, "idempotency-key"), + &idempotency_key + ); + assert_eq!( + test_request_header(&second, "idempotency-key"), + &idempotency_key + ); + assert_eq!( + first.split_once("\r\n\r\n").map(|(_, body)| body), + second.split_once("\r\n\r\n").map(|(_, body)| body), + ); + } + #[test] fn canonical_art_spritesheet_requires_real_transparent_pixels() { assert!(platform_art_spritesheet_has_transparent_pixels( diff --git a/apps/ai-game-creator-shell/src-tauri/src/agent_native_tools.rs b/apps/ai-game-creator-shell/src-tauri/src/agent_native_tools.rs index 95e7ab68a..4be2f67d0 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/agent_native_tools.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/agent_native_tools.rs @@ -811,7 +811,101 @@ fn plan_update_schema() -> Value { }) } -fn action_function_parameters(input_schema: Value) -> Value { +fn rebase_action_input_schema_refs_in_scope(value: &mut Value, has_local_resource_id: bool) { + let Value::Object(object) = value else { + return; + }; + + // `$id` 会建立独立 schema resource；其内部 fragment 应继续相对该 resource + // 解析，不能按外层 function parameters 根重定位。 + let has_local_resource_id = has_local_resource_id || object.contains_key("$id"); + let reference = object + .get("$ref") + .and_then(Value::as_str) + .map(ToString::to_string); + if let Some(reference) = reference { + // 只有空 fragment 和 JSON Pointer fragment 相对当前 document 根。 + // `#Mode` 是命名 anchor，外部 URI 也有自己的解析范围，必须保持原样。 + if !has_local_resource_id && (reference == "#" || reference.starts_with("#/")) { + let rebased = if reference == "#" { + "#/properties/input".to_string() + } else { + format!("#/properties/input{}", &reference[1..]) + }; + object.insert("$ref".to_string(), Value::String(rebased)); + } + } + + // 只进入 JSON Schema 明确定义为 subschema 的位置。default、const、examples、 + // enum 等关键词承载普通 JSON 数据，其中即使出现 `$ref` 也不能改写。 + for keyword in [ + "additionalProperties", + "unevaluatedProperties", + "propertyNames", + "additionalItems", + "unevaluatedItems", + "contains", + "not", + "if", + "then", + "else", + "contentSchema", + ] { + if let Some(child) = object.get_mut(keyword) { + rebase_action_input_schema_refs_in_scope(child, has_local_resource_id); + } + } + + for keyword in ["allOf", "anyOf", "oneOf", "prefixItems"] { + if let Some(Value::Array(children)) = object.get_mut(keyword) { + for child in children { + rebase_action_input_schema_refs_in_scope(child, has_local_resource_id); + } + } + } + + // draft-07 的 tuple validation 允许 items 为 schema 数组；新版本则为单 schema。 + if let Some(items) = object.get_mut("items") { + match items { + Value::Array(children) => { + for child in children { + rebase_action_input_schema_refs_in_scope(child, has_local_resource_id); + } + } + child => rebase_action_input_schema_refs_in_scope(child, has_local_resource_id), + } + } + + for keyword in [ + "$defs", + "definitions", + "properties", + "patternProperties", + "dependentSchemas", + ] { + if let Some(Value::Object(children)) = object.get_mut(keyword) { + for child in children.values_mut() { + rebase_action_input_schema_refs_in_scope(child, has_local_resource_id); + } + } + } + + // draft-07 dependencies 的 value 可能是 subschema，也可能是属性名数组。 + if let Some(Value::Object(dependencies)) = object.get_mut("dependencies") { + for dependency in dependencies.values_mut().filter(|value| value.is_object()) { + rebase_action_input_schema_refs_in_scope(dependency, has_local_resource_id); + } + } +} + +fn rebase_action_input_schema_refs(value: &mut Value) { + rebase_action_input_schema_refs_in_scope(value, false); +} + +fn action_function_parameters(mut input_schema: Value) -> Value { + // MCP 的 input schema 会被包进 action.input。局部 JSON Pointer 仍从整个 + // function parameters 根解析，因此必须同步重定位；否则 #/$defs/... 会悬空。 + rebase_action_input_schema_refs(&mut input_schema); json!({ "type": "object", "required": ["reason", "input"], @@ -1369,6 +1463,108 @@ mod tests { assert!(issues.is_empty(), "{}", issues.join("\n")); } + #[test] + fn action_function_parameters_rebases_local_schema_refs_after_wrapping() { + let parameters = action_function_parameters(json!({ + "type": "object", + "$defs": { + "Mode": {"type": "string", "enum": ["fast", "safe"]}, + "Options": { + "type": "object", + "properties": {"mode": {"$ref": "#/$defs/Mode"}}, + "required": ["mode"], + "additionalProperties": false + } + }, + "properties": { + "options": {"$ref": "#/$defs/Options"}, + "recursive": {"$ref": "#"}, + "anchor": {"$ref": "#Mode"}, + "scoped": { + "$id": "nested.json", + "$defs": {"Value": {"type": "string"}}, + "properties": {"value": {"$ref": "#/$defs/Value"}} + }, + "external": {"$ref": "https://schemas.example/tool.json"} + }, + "required": ["options"], + "additionalProperties": false + })); + + let input = ¶meters["properties"]["input"]; + assert_eq!( + input["properties"]["options"]["$ref"], + "#/properties/input/$defs/Options" + ); + assert_eq!( + input["$defs"]["Options"]["properties"]["mode"]["$ref"], + "#/properties/input/$defs/Mode" + ); + assert_eq!( + input["properties"]["recursive"]["$ref"], + "#/properties/input" + ); + assert_eq!(input["properties"]["anchor"]["$ref"], "#Mode"); + assert_eq!( + input["properties"]["scoped"]["properties"]["value"]["$ref"], + "#/$defs/Value" + ); + assert_eq!( + input["properties"]["external"]["$ref"], + "https://schemas.example/tool.json" + ); + for reference in [ + input["properties"]["options"]["$ref"] + .as_str() + .expect("options ref"), + input["$defs"]["Options"]["properties"]["mode"]["$ref"] + .as_str() + .expect("mode ref"), + input["properties"]["recursive"]["$ref"] + .as_str() + .expect("recursive ref"), + ] { + assert!( + parameters + .pointer(reference.trim_start_matches('#')) + .is_some(), + "rebased ref must resolve: {reference}" + ); + } + } + + #[test] + fn action_function_parameters_preserves_refs_inside_schema_data_keywords() { + let parameters = action_function_parameters(json!({ + "type": "object", + "$defs": { + "Value": {"type": "string"} + }, + "properties": { + "value": { + "$ref": "#/$defs/Value", + "default": {"$ref": "#/literal-default"}, + "const": { + "nested": [{"$ref": "#/literal-const"}] + }, + "examples": [ + {"$ref": "#/literal-example"}, + [{"$ref": "#/nested-literal-example"}] + ] + } + }, + "required": ["value"], + "additionalProperties": false + })); + + let value = ¶meters["properties"]["input"]["properties"]["value"]; + assert_eq!(value["$ref"], "#/properties/input/$defs/Value"); + assert_eq!(value["default"]["$ref"], "#/literal-default"); + assert_eq!(value["const"]["nested"][0]["$ref"], "#/literal-const"); + assert_eq!(value["examples"][0]["$ref"], "#/literal-example"); + assert_eq!(value["examples"][1][0]["$ref"], "#/nested-literal-example"); + } + #[test] fn native_project_patchset_normalizes_nullable_strict_shape() { let arguments = json!({ diff --git a/apps/ai-game-creator-shell/src-tauri/src/config.rs b/apps/ai-game-creator-shell/src-tauri/src/config.rs index 7561cb15b..36ce07533 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/config.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/config.rs @@ -53,7 +53,7 @@ fn build_game_creator_platform_llm_config( llm: &GameCreatorLlmConfig, config_path: &str, ) -> Result { - validate_game_creator_llm_web_search_config(llm, config_path)?; + let api_kind = validate_game_creator_llm_web_search_config(llm, config_path)?; let api_key = trim_config_string(&llm.api_key).ok_or_else(|| llm_api_key_config_error(config_path))?; let base_url = @@ -61,6 +61,8 @@ fn build_game_creator_platform_llm_config( let model = trim_config_string(&llm.model).ok_or_else(|| llm_model_config_error(config_path))?; validate_game_creator_llm_timing_config(llm, config_path)?; + let anthropic_strict_tool_support = + game_creator_supports_anthropic_strict_tools(api_kind, &base_url, &model); LlmConfig::new( LlmProvider::OpenAiCompatible, base_url, @@ -70,9 +72,54 @@ fn build_game_creator_platform_llm_config( llm.max_retries, llm.retry_backoff_ms, ) + .map(|config| config.with_anthropic_strict_tool_support(anthropic_strict_tool_support)) .map_err(|error| format!("LLM 配置无效：{error}")) } +fn game_creator_supports_anthropic_strict_tools( + api_kind: LlmApiKind, + base_url: &str, + model: &str, +) -> bool { + if api_kind != LlmApiKind::Anthropic { + return false; + } + + // 兼容网关即使复用了 Anthropic messages 协议，也不能据此推断 structured + // outputs 能力。只对无凭据、无自定义端口/路径的官方 HTTPS endpoint 开启。 + let Ok(endpoint) = url::Url::parse(base_url) else { + return false; + }; + if endpoint.scheme() != "https" + || endpoint.host_str() != Some("api.anthropic.com") + || endpoint.port().is_some() + || !endpoint.username().is_empty() + || endpoint.password().is_some() + || endpoint.path() != "/" + || endpoint.query().is_some() + || endpoint.fragment().is_some() + { + return false; + } + + // Claude API 的 structured outputs 从 Claude 4.5 起可用。仅识别官方 Claude + // family 的版本化 model id；不凭 `latest`、第三方别名或未知产品名猜能力。 + let normalized = model.trim().to_ascii_lowercase(); + let mut parts = normalized.split('-'); + if parts.next() != Some("claude") || !matches!(parts.next(), Some("opus" | "sonnet" | "haiku")) + { + return false; + } + let Some(major) = parts.next().and_then(|value| value.parse::().ok()) else { + return false; + }; + let minor = parts + .next() + .and_then(|value| value.parse::().ok()) + .unwrap_or(0); + major > 4 || (major == 4 && minor >= 5) +} + pub(crate) fn build_game_creator_llm_client_from_config() -> Result { let app_config = load_game_creator_app_config()?; build_game_creator_llm_client_from_llm_config(&app_config.llm, "llm") @@ -174,7 +221,7 @@ pub(crate) fn check_game_creator_llm_config_from_config() -> GameCreatorLlmConfi retry_backoff_ms: DEFAULT_RETRY_BACKOFF_MS, error: Some(error), agents: Vec::new(), - } + }; } }; let global_route_shape_error = @@ -1586,3 +1633,54 @@ pub(crate) fn game_creator_config_file_label(file_name: &str) -> String { .map(|directory| directory.join(file_name).display().to_string()) .unwrap_or_else(|| file_name.to_string()) } + +#[cfg(test)] +mod anthropic_strict_capability_tests { + use super::*; + + fn anthropic_config(base_url: &str, model: &str) -> GameCreatorLlmConfig { + GameCreatorLlmConfig { + api_key: "test-key".to_string(), + base_url: base_url.to_string(), + model: model.to_string(), + api_kind: "anthropic".to_string(), + web_search_enabled: false, + ..GameCreatorLlmConfig::default() + } + } + + #[test] + fn official_supported_claude_model_opts_in_to_anthropic_strict_tools() { + for model in [ + "claude-sonnet-4-5-20250929", + "claude-opus-4-6", + "claude-haiku-5", + ] { + let config = build_game_creator_platform_llm_config( + &anthropic_config("https://api.anthropic.com", model), + "llm", + ) + .expect("supported official Anthropic config"); + assert!(config.anthropic_strict_tool_support(), "model={model}"); + } + } + + #[test] + fn old_models_and_compatible_or_lookalike_endpoints_keep_strict_disabled() { + for (base_url, model) in [ + ("https://api.anthropic.com", "claude-3-5-sonnet-latest"), + ("https://api.anthropic.com", "claude-sonnet-latest"), + ("https://minimax.example.com", "claude-sonnet-4-5"), + ("https://api.anthropic.com.example.com", "claude-sonnet-4-5"), + ("http://api.anthropic.com", "claude-sonnet-4-5"), + ] { + let config = + build_game_creator_platform_llm_config(&anthropic_config(base_url, model), "llm") + .expect("non-capable Anthropic config remains usable without strict"); + assert!( + !config.anthropic_strict_tool_support(), + "base_url={base_url}, model={model}" + ); + } + } +} diff --git a/apps/ai-game-creator-shell/src-tauri/src/mcp.rs b/apps/ai-game-creator-shell/src-tauri/src/mcp.rs index 066a6a37a..2c5f26c40 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/mcp.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/mcp.rs @@ -899,13 +899,47 @@ fn normalize_game_creator_mcp_catalog_tool( }) } +fn normalize_game_creator_mcp_server_tools( + server_id: &str, + config: &GameCreatorMcpServerConfig, + listed_tools: Vec, +) -> Result, String> { + if listed_tools.len() > GAME_CREATOR_MCP_MAX_TOOLS_PER_SERVER { + return Err(format!( + "MCP server {server_id} 返回 {} 个工具，超过单 server 上限 {GAME_CREATOR_MCP_MAX_TOOLS_PER_SERVER}", + listed_tools.len() + )); + } + let mut server_tools = Vec::new(); + let mut server_tool_names = BTreeSet::new(); + for tool in listed_tools { + if !game_creator_mcp_tool_is_enabled(config, tool.name.as_ref()) { + continue; + } + if tool.task_support() == TaskSupport::Required { + return Err(format!( + "MCP tool {server_id}/{} 要求 task-mode，当前切片未支持", + tool.name + )); + } + if !server_tool_names.insert(tool.name.to_string()) { + return Err(format!( + "MCP server {server_id} 返回重复 tool identity：{}", + tool.name + )); + } + server_tools.push(normalize_game_creator_mcp_catalog_tool( + server_id, config, tool, + )?); + } + server_tools.sort_by(|left, right| left.name.cmp(&right.name)); + Ok(server_tools) +} + pub(crate) async fn read_game_creator_mcp_catalog_at( root: &Path, ) -> Result { let config = load_game_creator_app_config()?; - let mut servers = Vec::new(); - let mut tools = Vec::new(); - let mut catalog_identity = Vec::new(); let server_reads = config .mcp_servers .into_iter() @@ -999,37 +1033,33 @@ pub(crate) async fn read_game_creator_mcp_catalog_at( )); } }; - if listed_tools.len() > GAME_CREATOR_MCP_MAX_TOOLS_PER_SERVER { - return Err(format!( - "MCP server {server_id} 返回 {} 个工具，超过单 server 上限 {GAME_CREATOR_MCP_MAX_TOOLS_PER_SERVER}", - listed_tools.len() - )); - } - let mut server_tools = Vec::new(); - let mut server_tool_names = BTreeSet::new(); - for tool in listed_tools { - if !game_creator_mcp_tool_is_enabled(&server_config, tool.name.as_ref()) { - continue; - } - if tool.task_support() == TaskSupport::Required { - return Err(format!( - "MCP tool {server_id}/{} 要求 task-mode，当前切片未支持", - tool.name + let server_tools = match normalize_game_creator_mcp_server_tools( + &server_id, + &server_config, + listed_tools, + ) { + Ok(server_tools) => server_tools, + Err(error) if server_config.required => return Err(error), + Err(error) => { + return Ok(( + GameCreatorMcpServerStatus { + server_id, + enabled: true, + required: false, + transport: server_config.transport, + connected: false, + server_name, + server_version, + instructions: instructions.clone(), + instructions_chars: instructions.chars().count(), + tool_count: 0, + error: Some(sanitize_game_creator_mcp_error(root, &error, 240)), + }, + Vec::new(), + None, )); } - if !server_tool_names.insert(tool.name.to_string()) { - return Err(format!( - "MCP server {server_id} 返回重复 tool identity：{}", - tool.name - )); - } - server_tools.push(normalize_game_creator_mcp_catalog_tool( - &server_id, - &server_config, - tool, - )?); - } - server_tools.sort_by(|left, right| left.name.cmp(&right.name)); + }; let catalog_identity = serde_json::json!({ "serverId": server_id, "configFingerprint": entry.config_fingerprint, @@ -1054,25 +1084,117 @@ pub(crate) async fn read_game_creator_mcp_catalog_at( drop(entry); Ok((status, server_tools, Some(catalog_identity))) }); + let mut server_results = Vec::new(); for server_read in futures::future::join_all(server_reads).await { - let (server, server_tools, identity) = server_read?; - servers.push(server); - tools.extend(server_tools); - if let Some(identity) = identity { - catalog_identity.push(identity); + server_results.push(server_read?); + } + + let collect_catalog_parts = + |results: &[( + GameCreatorMcpServerStatus, + Vec, + Option, + )], + included_optional_servers: &BTreeSet| { + let mut candidate_servers = Vec::with_capacity(results.len()); + let mut candidate_tools = Vec::new(); + let mut candidate_identity = Vec::new(); + for (status, server_tools, identity) in results { + let included = status.required + || included_optional_servers.contains(status.server_id.as_str()); + let mut candidate_status = status.clone(); + if !included && candidate_status.connected { + candidate_status.connected = false; + candidate_status.tool_count = 0; + } + candidate_servers.push(candidate_status); + if included && status.connected { + candidate_tools.extend(server_tools.iter().cloned()); + if let Some(identity) = identity { + candidate_identity.push(identity.clone()); + } + } + } + (candidate_servers, candidate_tools, candidate_identity) + }; + + let catalog_prompt_bytes = |candidate_servers: Vec, + candidate_tools: Vec, + candidate_identity: &[serde_json::Value]| + -> Result { + let candidate = GameCreatorMcpCatalog { + fingerprint: game_creator_mcp_sha256(candidate_identity)?, + servers: candidate_servers, + tools: candidate_tools, + }; + Ok(render_game_creator_mcp_catalog_for_prompt(&candidate)? + .into_bytes() + .len()) + }; + + let mut included_optional_servers = BTreeSet::new(); + let (required_servers, required_tools, required_identity) = + collect_catalog_parts(&server_results, &included_optional_servers); + if required_tools.len() > GAME_CREATOR_MCP_MAX_TOOLS { + return Err(format!( + "required MCP catalog 共 {} 个工具，超过上限 {GAME_CREATOR_MCP_MAX_TOOLS}", + required_tools.len() + )); + } + let required_catalog_bytes = + catalog_prompt_bytes(required_servers, required_tools, &required_identity)?; + if required_catalog_bytes > GAME_CREATOR_MCP_MAX_CATALOG_BYTES { + return Err(format!( + "required MCP catalog 为 {required_catalog_bytes} bytes，超过上限 {GAME_CREATOR_MCP_MAX_CATALOG_BYTES}" + )); + } + + for index in 0..server_results.len() { + let status = &server_results[index].0; + if status.required || !status.connected { + continue; + } + let server_id = status.server_id.clone(); + included_optional_servers.insert(server_id.clone()); + let (candidate_servers, candidate_tools, candidate_identity) = + collect_catalog_parts(&server_results, &included_optional_servers); + let candidate_tool_count = candidate_tools.len(); + let candidate_bytes = if candidate_tool_count <= GAME_CREATOR_MCP_MAX_TOOLS { + Some(catalog_prompt_bytes( + candidate_servers, + candidate_tools, + &candidate_identity, + )?) + } else { + None + }; + let capacity_error = if candidate_tool_count > GAME_CREATOR_MCP_MAX_TOOLS { + Some(format!( + "MCP server {server_id} 使目录工具总数达到 {candidate_tool_count}，超过上限 {GAME_CREATOR_MCP_MAX_TOOLS}" + )) + } else if candidate_bytes.is_some_and(|bytes| bytes > GAME_CREATOR_MCP_MAX_CATALOG_BYTES) { + Some(format!( + "MCP server {server_id} 使目录超过 {GAME_CREATOR_MCP_MAX_CATALOG_BYTES} bytes 上限" + )) + } else { + None + }; + if let Some(error) = capacity_error { + included_optional_servers.remove(server_id.as_str()); + let status = &mut server_results[index].0; + status.connected = false; + status.tool_count = 0; + status.error = Some(sanitize_game_creator_mcp_error(root, &error, 240)); } } + + let (servers, mut tools, catalog_identity) = + collect_catalog_parts(&server_results, &included_optional_servers); tools.sort_by(|left, right| { left.server_id .cmp(&right.server_id) .then_with(|| left.name.cmp(&right.name)) }); - if tools.len() > GAME_CREATOR_MCP_MAX_TOOLS { - return Err(format!( - "MCP catalog 共 {} 个工具，超过上限 {GAME_CREATOR_MCP_MAX_TOOLS}", - tools.len() - )); - } let fingerprint = game_creator_mcp_sha256(&catalog_identity)?; let catalog = GameCreatorMcpCatalog { fingerprint, @@ -1080,12 +1202,7 @@ pub(crate) async fn read_game_creator_mcp_catalog_at( tools, }; let catalog_bytes = render_game_creator_mcp_catalog_for_prompt(&catalog)?.into_bytes(); - if catalog_bytes.len() > GAME_CREATOR_MCP_MAX_CATALOG_BYTES { - return Err(format!( - "MCP catalog 为 {} bytes，超过上限 {GAME_CREATOR_MCP_MAX_CATALOG_BYTES}", - catalog_bytes.len() - )); - } + debug_assert!(catalog_bytes.len() <= GAME_CREATOR_MCP_MAX_CATALOG_BYTES); Ok(catalog) } diff --git a/apps/ai-game-creator-shell/src-tauri/src/tests/mod.rs b/apps/ai-game-creator-shell/src-tauri/src/tests/mod.rs index 32e1692c9..e85374e95 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/tests/mod.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/tests/mod.rs @@ -2651,6 +2651,17 @@ fn spawn_mock_external_canvas_api_server_with_capture_and_generation_gate( } }) .to_string(); + let generation_accepted_body = serde_json::json!({ + "data": { + "operationId": "task-external-fixture-1", + "kind": "editor_image_generation", + "status": "queued", + "statusUrl": "/api/external/v1/generations/task-external-fixture-1", + "pollAfterMs": 1, + "updatedAtMicros": 1 + } + }) + .to_string(); let read_body = serde_json::json!({ "read": { "provider": "aliyun-oss", @@ -2677,6 +2688,8 @@ fn spawn_mock_external_canvas_api_server_with_capture_and_generation_gate( .to_string(); std::thread::spawn(move || { let mut generation_response_gate = generation_response_gate; + let mut pending_generation_result: Option = None; + let mut generation_poll_index = 0_u8; for _ in 0..expected_requests { let (mut stream, _) = listener.accept().expect("mock canvas api accept"); let mut request_buffer = [0_u8; 8192]; @@ -2686,61 +2699,119 @@ fn spawn_mock_external_canvas_api_server_with_capture_and_generation_gate( let _ = sender.send(request.to_string()); } let normalized_request = request.to_ascii_lowercase(); - let (content_type, body) = if request + let (status, content_type, body) = if request .starts_with("GET /api/external/v1/editor/projects ") { assert!(normalized_request.contains("authorization: bearer ")); - ("application/json", projects_body.as_bytes().to_vec()) + ("200 OK", "application/json", projects_body.as_bytes().to_vec()) } else if request.starts_with("GET /api/external/v1/editor/assets/library ") { assert!(normalized_request.contains("authorization: bearer ")); - ("application/json", library_body.as_bytes().to_vec()) + ("200 OK", "application/json", library_body.as_bytes().to_vec()) } else if request.starts_with("GET /api/external/v1/editor/projects/canvas-project-1 ") { assert!(normalized_request.contains("authorization: bearer ")); - ("application/json", project_body.as_bytes().to_vec()) + ("200 OK", "application/json", project_body.as_bytes().to_vec()) } else if request.starts_with("POST /api/external/v1/editor/images/generations ") { assert!(normalized_request.contains("authorization: bearer ")); - if let Some(gate) = generation_response_gate.take() { - gate.recv_timeout(Duration::from_secs(5)) - .expect("release mock canvas generation response"); - } - ("application/json", generation_body.as_bytes().to_vec()) + let idempotency_key = request + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("idempotency-key") + .then_some(value.trim()) + }) + .expect("generation request idempotency key"); + uuid::Uuid::parse_str(idempotency_key.trim()) + .expect("generation idempotency key must be UUID"); + pending_generation_result = Some(generation_body.clone()); + generation_poll_index = 0; + ( + "202 Accepted", + "application/json", + generation_accepted_body.as_bytes().to_vec(), + ) } else if request .starts_with("POST /api/external/v1/editor/icon-spritesheets/generations ") { + assert!(normalized_request.contains("authorization: bearer ")); + let idempotency_key = request + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("idempotency-key") + .then_some(value.trim()) + }) + .expect("generation request idempotency key"); + uuid::Uuid::parse_str(idempotency_key.trim()) + .expect("generation idempotency key must be UUID"); + pending_generation_result = Some(icon_spritesheet_body.clone()); + generation_poll_index = 0; + ( + "202 Accepted", + "application/json", + generation_accepted_body.as_bytes().to_vec(), + ) + } else if request.starts_with( + "GET /api/external/v1/generations/task-external-fixture-1 ", + ) { assert!(normalized_request.contains("authorization: bearer ")); if let Some(gate) = generation_response_gate.take() { gate.recv_timeout(Duration::from_secs(5)) .expect("release mock canvas generation response"); } + let status = match generation_poll_index { + 0 => "queued", + 1 => "running", + _ => "completed", + }; + generation_poll_index = generation_poll_index.saturating_add(1); + let result = (status == "completed").then(|| { + serde_json::from_str::( + pending_generation_result + .as_deref() + .expect("generation query follows one submission"), + ) + .expect("fixture generation result JSON") + }); ( + "200 OK", "application/json", - icon_spritesheet_body.as_bytes().to_vec(), + serde_json::json!({ + "data": { + "operationId": "task-external-fixture-1", + "kind": "editor_image_generation", + "status": status, + "phaseLabel": "图片画布生成图片", + "phaseDetail": if status == "completed" { "生成已完成。" } else { "正在生成。" }, + "progress": if status == "completed" { 100 } else { 35 }, + "result": result, + "pollAfterMs": 1, + "updatedAtMicros": 2 + } + }) + .to_string() + .into_bytes(), ) } else if request.starts_with( "GET /api/external/v1/assets/read-url?objectKey=generated%2Fcanvas%2Fhero.png ", ) { assert!(normalized_request.contains("authorization: bearer ")); - ("application/json", read_body.as_bytes().to_vec()) + ("200 OK", "application/json", read_body.as_bytes().to_vec()) } else if request.starts_with( "GET /api/external/v1/assets/read-url?objectKey=generated%2Fcanvas%2Fspritesheet.png ", ) { assert!(normalized_request.contains("authorization: bearer ")); ( + "200 OK", "application/json", spritesheet_read_body.as_bytes().to_vec(), ) } else if request.starts_with("GET /signed/hero.png ") { - ("image/png", valid_test_png_bytes()) + ("200 OK", "image/png", valid_test_png_bytes()) } else if request.starts_with("GET /signed/spritesheet.png ") { - ("image/png", transparent_test_png_bytes()) + ("200 OK", "image/png", transparent_test_png_bytes()) } else { - ("text/plain", b"not found".to_vec()) - }; - let status = if content_type == "text/plain" { - "404 Not Found" - } else { - "200 OK" + ("404 Not Found", "text/plain", b"not found".to_vec()) }; let response = format!( "HTTP/1.1 {status}\r\nContent-Type: {content_type}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", @@ -2762,7 +2833,7 @@ fn spawn_mock_external_canvas_api_server() -> String { fn spawn_mock_external_canvas_generation_api_server( request_sender: Option>, ) -> String { - spawn_mock_external_canvas_api_server_with_capture(5, request_sender) + spawn_mock_external_canvas_api_server_with_capture(8, request_sender) } fn spawn_mock_external_canvas_generation_api_server_with_gate( @@ -2770,7 +2841,7 @@ fn spawn_mock_external_canvas_generation_api_server_with_gate( generation_response_gate: mpsc::Receiver<()>, ) -> String { spawn_mock_external_canvas_api_server_with_capture_and_generation_gate( - 5, + 8, Some(request_sender), Some(generation_response_gate), ) diff --git a/apps/ai-game-creator-shell/src-tauri/src/tests/project.rs b/apps/ai-game-creator-shell/src-tauri/src/tests/project.rs index 344461628..072b763de 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/tests/project.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/tests/project.rs @@ -791,7 +791,7 @@ async fn background_agent_runtime_can_generate_platform_art_asset() { assert!(agent_db.contains("\"agentId\":\"art-asset-plan\"")); assert!(agent_db.contains("测试图集保持整图，未生成独立切片。")); assert!(!agent_db.contains("editor-runtime-key")); - let canvas_requests = (0..5) + let canvas_requests = (0..8) .map(|_| { canvas_receiver .recv_timeout(Duration::from_secs(2)) @@ -804,6 +804,22 @@ async fn background_agent_runtime_can_generate_platform_art_asset() { request.starts_with("POST /api/external/v1/editor/icon-spritesheets/generations ") }) .expect("canvas generation request"); + assert_eq!( + canvas_requests + .iter() + .filter(|request| request.starts_with("POST /api/external/v1/editor/")) + .count(), + 1, + "queued/running polling must not submit generation again" + ); + assert_eq!( + canvas_requests + .iter() + .filter(|request| request.starts_with("GET /api/external/v1/generations/")) + .count(), + 3, + "fixture should exercise queued, running, and completed states" + ); for expected in [ r#""referenceImageSrc":"resource-icon-spec""#, r#""iconDescriptions":"#, @@ -850,7 +866,7 @@ async fn canonical_art_spec_and_ui_requests_use_the_shared_reference_chain() { request_platform_art_asset_with_options_for_test(root, "原创贪吃蛇视觉", &options) .await .expect("prepare canonical visual request"); - (0..5) + (0..8) .map(|_| { request_receiver .recv_timeout(Duration::from_secs(2)) diff --git a/apps/ai-game-creator-shell/src-tauri/src/tests/provider.rs b/apps/ai-game-creator-shell/src-tauri/src/tests/provider.rs index 4f098396f..47ffc7c36 100644 --- a/apps/ai-game-creator-shell/src-tauri/src/tests/provider.rs +++ b/apps/ai-game-creator-shell/src-tauri/src/tests/provider.rs @@ -222,6 +222,120 @@ async fn mcp_optional_tools_list_failure_is_bounded_but_required_fails() { fs::remove_dir_all(config_dir).ok(); } +#[tokio::test] +async fn mcp_optional_invalid_tool_catalog_is_isolated_but_required_fails() { + let root = unique_project_path(); + init_local_game_project_at(&root, "mcp-invalid-catalog", "MCP 非法目录项目") + .expect("initialize invalid MCP catalog project"); + let config_dir = unique_project_path(); + fs::create_dir_all(&config_dir).expect("create invalid MCP catalog config dir"); + let config_guard = use_test_runtime_config_dir(config_dir.clone()); + + for (fixture_arg, expected_error) in [ + ("--oversized-input-schema", "input schema 超过上限"), + ("--duplicate-tool", "重复 tool identity"), + ("--require-task-mode", "要求 task-mode"), + ] { + let fixture = serde_json::json!({ + "required": false, + "transport": "stdio", + "command": "node", + "args": [mcp_fixture_script_path(), "stdio", fixture_arg] + }); + write_mcp_transport_test_config(&config_dir, "optional-invalid-fixture", fixture.clone()); + + let catalog = read_game_creator_mcp_catalog_at(&root) + .await + .expect("optional invalid tool catalog must stay in server status"); + assert!(catalog.tools.is_empty()); + let server = catalog.servers.first().expect("optional invalid status"); + assert!(!server.connected); + assert!( + server + .error + .as_deref() + .is_some_and(|error| { error.contains(expected_error) }), + "fixture={fixture_arg} status={server:?}" + ); + + shutdown_game_creator_mcp_clients_for_tests().await; + let mut required_fixture = fixture; + required_fixture["required"] = serde_json::Value::Bool(true); + write_mcp_transport_test_config(&config_dir, "required-invalid-fixture", required_fixture); + let error = read_game_creator_mcp_catalog_at(&root) + .await + .expect_err("required invalid tool catalog must fail the catalog"); + assert!( + error.contains(expected_error), + "fixture={fixture_arg} error={error}" + ); + shutdown_game_creator_mcp_clients_for_tests().await; + } + + drop(config_guard); + fs::remove_dir_all(root).ok(); + fs::remove_dir_all(config_dir).ok(); +} + +#[tokio::test] +async fn mcp_optional_server_is_isolated_when_aggregate_catalog_exceeds_tool_limit() { + let root = unique_project_path(); + init_local_game_project_at(&root, "mcp-aggregate-limit", "MCP 聚合上限项目") + .expect("initialize MCP aggregate limit project"); + let config_dir = unique_project_path(); + fs::create_dir_all(&config_dir).expect("create MCP aggregate limit config dir"); + let config_guard = use_test_runtime_config_dir(config_dir.clone()); + let fixture = mcp_fixture_script_path(); + let config = serde_json::json!({ + "mcpServers": { + "required-alpha": { + "required": true, + "transport": "stdio", + "command": "node", + "args": [fixture, "stdio", "--tool-count=64"] + }, + "required-beta": { + "required": true, + "transport": "stdio", + "command": "node", + "args": [mcp_fixture_script_path(), "stdio", "--tool-count=64"] + }, + "optional-gamma": { + "required": false, + "transport": "stdio", + "command": "node", + "args": [mcp_fixture_script_path(), "stdio", "--tool-count=64"] + } + } + }); + fs::write( + config_dir.join(GAME_CREATOR_CONFIG_FILE_NAME), + serde_json::to_vec_pretty(&config).expect("serialize MCP aggregate limit config"), + ) + .expect("write MCP aggregate limit config"); + + let catalog = read_game_creator_mcp_catalog_at(&root) + .await + .expect("optional aggregate overflow must stay in server status"); + assert_eq!(catalog.tools.len(), 128); + let optional = catalog + .servers + .iter() + .find(|server| server.server_id == "optional-gamma") + .expect("optional aggregate overflow status"); + assert!(!optional.connected); + assert_eq!(optional.tool_count, 0); + assert!(optional + .error + .as_deref() + .is_some_and(|error| error.contains("工具总数") && error.contains("超过上限"))); + + shutdown_game_creator_mcp_clients_for_tests().await; + drop(config_guard); + fs::remove_dir_all(root).ok(); + fs::remove_dir_all(config_dir).ok(); +} + #[tokio::test] async fn mcp_catalog_refreshes_independent_servers_in_parallel() { let root = unique_project_path(); diff --git a/apps/ai-game-creator-shell/src-tauri/test-fixtures/mcp-server.mjs b/apps/ai-game-creator-shell/src-tauri/test-fixtures/mcp-server.mjs index aaa811adf..0dd384e63 100644 --- a/apps/ai-game-creator-shell/src-tauri/test-fixtures/mcp-server.mjs +++ b/apps/ai-game-creator-shell/src-tauri/test-fixtures/mcp-server.mjs @@ -6,6 +6,13 @@ const args = process.argv.slice(2); const mode = args[0] ?? 'stdio'; const failList = args.includes('--fail-list'); const includeUnannotated = args.includes('--include-unannotated'); +const duplicateTool = args.includes('--duplicate-tool'); +const oversizedInputSchema = args.includes('--oversized-input-schema'); +const requireTaskMode = args.includes('--require-task-mode'); +const toolCountArgument = args.find((value) => value.startsWith('--tool-count=')); +const toolCount = toolCountArgument + ? Number(toolCountArgument.slice('--tool-count='.length)) + : null; const listDelayArgument = args.find((value) => value.startsWith('--list-delay-ms='), ); @@ -89,6 +96,28 @@ const tools = [ }, ]; +if (duplicateTool) { + tools.push({ ...tools[0] }); +} + +if (oversizedInputSchema) { + tools[0].inputSchema.properties.query.description = 'x'.repeat(70 * 1024); +} + +if (requireTaskMode) { + tools[0].execution = { taskSupport: 'required' }; +} + +if (Number.isInteger(toolCount) && toolCount > tools.length) { + for (let index = tools.length; index < toolCount; index += 1) { + tools.push({ + ...tools[0], + name: `lookup-${index}`, + title: `Lookup ${index}`, + }); + } +} + if (includeUnannotated) { tools.push({ name: 'mutate-unannotated', diff --git a/apps/ai-game-creator-shell/tests/start-dev-stack.test.ts b/apps/ai-game-creator-shell/tests/start-dev-stack.test.ts index 50ea09925..07e5d6a4f 100644 --- a/apps/ai-game-creator-shell/tests/start-dev-stack.test.ts +++ b/apps/ai-game-creator-shell/tests/start-dev-stack.test.ts @@ -7,7 +7,9 @@ import { describe, expect, test, vi } from 'vitest'; import { ensureBackend, + isProcessGroupAlive, preflightExistingVite, + readLinuxProcessGroupAlive, resolveBackendTargetsFromState, runWindowsTaskkill, spawnChild, @@ -91,6 +93,36 @@ describe('AI 游戏创作配套后端复用门禁', () => { describe('AI 游戏创作启动子进程生命周期', () => { const posixTest = process.platform === 'win32' ? test.skip : test; + test('Linux 进程组只剩僵尸进程时视为已经停止', () => { + const procStats = new Map([ + ['/proc/101/stat', '101 (node worker) Z 1 700 700 0'], + ['/proc/102/stat', '102 (other worker) S 1 701 701 0'], + ]); + const readLinuxGroupAlive = (processGroupId: number) => + readLinuxProcessGroupAlive(processGroupId, { + readdirImpl: () => ['101', '102', 'not-a-pid'], + readFileImpl: (path: string) => { + const stat = procStats.get(path); + if (!stat) { + throw new Error('missing proc stat fixture'); + } + return stat; + }, + }); + const killImpl = vi.fn(); + + expect(readLinuxGroupAlive(700)).toBe(false); + expect(readLinuxGroupAlive(701)).toBe(true); + expect( + isProcessGroupAlive(700, { + platform: 'linux', + killImpl, + readLinuxGroupAlive, + }), + ).toBe(false); + expect(killImpl).toHaveBeenCalledWith(-700, 0); + }); + posixTest('npm 不可解析时进入受控 error 结果而不是未处理事件', async () => { const child = spawnChild('genarrative-command-that-does-not-exist', [], { cwd: process.cwd(), diff --git a/deploy/container/api-server.Dockerfile b/deploy/container/api-server.Dockerfile index ef72a30a9..37ed259e6 100644 --- a/deploy/container/api-server.Dockerfile +++ b/deploy/container/api-server.Dockerfile @@ -2,6 +2,8 @@ FROM rust:1.93-bookworm AS rust-builder WORKDIR /workspace COPY server-rs ./server-rs +COPY docs/openapi ./docs/openapi +COPY .codex/skills/genarrative-external-editor-api ./.codex/skills/genarrative-external-editor-api COPY public ./public RUN cargo build --release -p api-server --manifest-path server-rs/Cargo.toml && \ cp server-rs/target/release/api-server /tmp/api-server diff --git a/docs/openapi/genarrative-external-v1.openapi.json b/docs/openapi/genarrative-external-v1.openapi.json index 67c0698c7..6c65f059f 100644 --- a/docs/openapi/genarrative-external-v1.openapi.json +++ b/docs/openapi/genarrative-external-v1.openapi.json @@ -3,7 +3,7 @@ "info": { "title": "陶泥儿外部编辑器 OpenAPI", "version": "1.0.0", - "description": "外部系统调用陶泥儿图片画布项目、画布布局、素材库，以及图片、视频、音效、音乐等编辑器素材生成/编辑能力的 v1 契约。新建 projectId 使用 proj- 前缀，新建 taskId / operationId 使用 task- 前缀；历史 editor-project-*、aitask_*、extgen-* ID 仍可作为既有资源标识传入。\n\n兼容性说明：v1 当前处于无外部存量调用方阶段，正式对外发放 API Key 之前，契约可能在不升 info.version、不设弃用期的情况下发生包含字段移除在内的破坏性变更。生成客户端时请勿假定本文档已冻结。" + "description": "外部系统调用陶泥儿图片画布项目、画布布局、素材库，以及图片、视频、音效、音乐等编辑器素材生成/编辑能力的 v1 契约。全部生成 POST 都是异步提交：必须携带 Idempotency-Key，收到 202 后使用 operationId 查询统一生成状态。支持远程 MCP 的 Agent 可连接 /api/external/v1/mcp；不支持 MCP 的 Agent 可从 /api/external/v1/skill.zip 下载完整 Skill 包。新建 projectId 使用 proj- 前缀，新建 taskId / operationId 使用 task- 前缀；历史 editor-project-*、aitask_*、extgen-* ID 仍可作为既有资源标识传入。\n\n兼容性说明：v1 当前处于无外部存量调用方阶段，正式对外发放 API Key 之前，契约可能在不升 info.version、不设弃用期的情况下发生包含字段移除在内的破坏性变更。生成客户端时请勿假定本文档已冻结。" }, "servers": [ { @@ -39,6 +39,10 @@ { "name": "Editor Audio", "description": "编辑器音效与音乐生成" + }, + { + "name": "Agent Integration", + "description": "远程 MCP、OpenAPI 和完整 Skill 包发现" } ], "paths": { @@ -64,6 +68,137 @@ } } }, + "/api/external/v1/agent-integration.json": { + "get": { + "tags": [ + "Agent Integration" + ], + "operationId": "getExternalAgentIntegrationManifest", + "summary": "读取 Agent 集成清单", + "description": "返回远程 MCP、OpenAPI、Skill 入口、完整 Skill ZIP、包内文件列表和归档 SHA-256。", + "security": [], + "x-mcp-excluded": true, + "responses": { + "200": { + "description": "Agent 集成清单", + "content": { + "application/json": { + "schema": { + "type": "object" + } + } + } + } + } + } + }, + "/api/external/v1/skill/SKILL.md": { + "get": { + "tags": [ + "Agent Integration" + ], + "operationId": "getExternalEditorSkillEntry", + "summary": "读取外部编辑器 Skill 入口", + "security": [], + "x-mcp-excluded": true, + "responses": { + "200": { + "description": "SKILL.md", + "content": { + "text/markdown": { + "schema": { + "type": "string" + } + } + } + } + } + } + }, + "/api/external/v1/skill.zip": { + "get": { + "tags": [ + "Agent Integration" + ], + "operationId": "downloadExternalEditorSkillArchive", + "summary": "下载完整外部编辑器 Skill 包", + "security": [], + "x-mcp-excluded": true, + "responses": { + "200": { + "description": "包含 SKILL.md、references、scripts 和 agents metadata 的 ZIP", + "content": { + "application/zip": { + "schema": { + "type": "string", + "format": "binary" + } + } + } + } + } + } + }, + "/api/external/v1/mcp": { + "post": { + "tags": [ + "Agent Integration" + ], + "operationId": "callExternalEditorMcp", + "summary": "调用托管式远程 MCP", + "description": "MCP 2025-11-25 Streamable HTTP JSON 端点。使用与 REST API 相同的 Bearer API Key；生成工具立即返回异步 operation。resources/list 和 resources/read 提供 usage、OpenAPI、Skill 主入口以及 capability routing、API operations、authentication and safety、requests and outputs 四篇渐进式 reference；本地脚本仍只通过完整 Skill ZIP 提供。", + "security": [ + { + "ExternalApiKey": [] + } + ], + "x-mcp-excluded": true, + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "type": "object" + } + } + } + }, + "responses": { + "200": { + "description": "MCP JSON-RPC 响应", + "content": { + "application/json": { + "schema": { + "type": "object" + } + } + } + }, + "202": { + "description": "MCP notification 已接受" + }, + "401": { + "description": "缺少、格式错误或无法验证 Bearer API Key。返回 WWW-Authenticate 以及机器可读的 MCP 鉴权引导，说明 Header 格式、开发者 API Key 创建位置、凭据安全要求和公开 discovery/Skill/OpenAPI 地址；不暴露 tools、resources、owner 或 Key 是否存在。", + "headers": { + "WWW-Authenticate": { + "description": "Bearer 鉴权挑战。", + "schema": { + "type": "string", + "const": "Bearer realm=\"genarrative-external-editor\"" + } + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/McpAuthenticationGuideResponse" + } + } + } + } + } + } + }, "/api/external/v1/assets/direct-upload-tickets": { "post": { "tags": [ @@ -881,6 +1016,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -892,12 +1032,12 @@ } }, "responses": { - "200": { - "description": "生成结果与落库资源", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorImageGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -930,6 +1070,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -941,12 +1086,12 @@ } }, "responses": { - "200": { - "description": "重绘结果与落库资源", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorImageGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -978,6 +1123,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -989,12 +1139,12 @@ } }, "responses": { - "200": { - "description": "图标 spritesheet、实际切片结果、可选非阻断告警与落库资源", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorIconSpritesheetGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -1026,6 +1176,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -1037,12 +1192,12 @@ } }, "responses": { - "200": { - "description": "UI 设计图素材 spritesheet、实际切片结果、可选非阻断告警与落库资源", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorIconSpritesheetGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -1074,6 +1229,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -1085,12 +1245,12 @@ } }, "responses": { - "200": { - "description": "角色动画视频预览与抽帧结果", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorCharacterAnimationGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -1122,6 +1282,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -1133,12 +1298,12 @@ } }, "responses": { - "200": { - "description": "视频生成结果", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorVideoGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -1170,6 +1335,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -1181,12 +1351,12 @@ } }, "responses": { - "200": { - "description": "音效生成结果", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorAudioGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -1218,6 +1388,11 @@ "ExternalApiKey": [] } ], + "parameters": [ + { + "$ref": "#/components/parameters/IdempotencyKey" + } + ], "requestBody": { "required": true, "content": { @@ -1229,12 +1404,12 @@ } }, "responses": { - "200": { - "description": "背景音乐生成结果", + "202": { + "description": "生成任务已持久化入队", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/EditorAudioGenerationResponse" + "$ref": "#/components/schemas/ExternalEditorGenerationSubmissionResponse" } } } @@ -1253,6 +1428,55 @@ } } } + }, + "/api/external/v1/generations/{operationId}": { + "get": { + "tags": [ + "Editor Generations" + ], + "operationId": "getExternalEditorGenerationJob", + "summary": "查询异步生成任务", + "description": "queued/running 时返回进度，completed 时返回 compact 稳定结果引用，failed 时返回脱敏错误。跨账号任务按不存在处理。", + "security": [ + { + "ExternalApiKey": [] + } + ], + "parameters": [ + { + "name": "operationId", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "生成任务状态及可选结果", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ExternalEditorGenerationJobResponse" + } + } + } + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "502": { + "$ref": "#/components/responses/UpstreamError" + } + } + } } }, "components": { @@ -1287,6 +1511,18 @@ "schema": { "type": "string" } + }, + "IdempotencyKey": { + "name": "Idempotency-Key", + "in": "header", + "required": true, + "description": "本次逻辑生成请求的稳定幂等键；网络结果不确定时必须复用原值，不得换键重提。", + "schema": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "pattern": "^[!-~]+$" + } } }, "responses": { @@ -2516,6 +2752,198 @@ "JsonValue": { "description": "任意 JSON 值。" }, + "McpAuthenticationGuideResponse": { + "type": "object", + "required": [ + "error", + "meta" + ], + "additionalProperties": false, + "properties": { + "error": { + "type": "object", + "required": [ + "code", + "message", + "details" + ], + "additionalProperties": false, + "properties": { + "code": { + "const": "UNAUTHORIZED" + }, + "message": { + "const": "连接陶泥儿托管 MCP 需要开发者 API Key" + }, + "details": { + "type": "object", + "required": [ + "guide" + ], + "additionalProperties": false, + "properties": { + "guide": { + "type": "object", + "required": [ + "reason", + "action", + "authentication", + "keyManagement", + "retry", + "steps", + "credentialSafety", + "publicDiscovery" + ], + "additionalProperties": false, + "properties": { + "reason": { + "const": "MCP_AUTHENTICATION_REQUIRED" + }, + "action": { + "const": "CONFIGURE_BEARER_API_KEY" + }, + "authentication": { + "type": "object", + "required": [ + "scheme", + "header", + "valueFormat" + ], + "additionalProperties": false, + "properties": { + "scheme": { + "const": "Bearer" + }, + "header": { + "const": "Authorization" + }, + "valueFormat": { + "const": "Bearer " + } + } + }, + "keyManagement": { + "type": "object", + "required": [ + "navigationLabel", + "rawKeyShownOnce" + ], + "additionalProperties": false, + "properties": { + "navigationLabel": { + "const": "开发者 API Key" + }, + "rawKeyShownOnce": { + "const": true + } + } + }, + "retry": { + "type": "object", + "required": [ + "method", + "path", + "rpcMethod" + ], + "additionalProperties": false, + "properties": { + "method": { + "const": "POST" + }, + "path": { + "const": "/api/external/v1/mcp" + }, + "rpcMethod": { + "const": "initialize" + } + } + }, + "steps": { + "type": "array", + "items": { + "type": "string" + } + }, + "credentialSafety": { + "type": "object", + "required": [ + "rawKeyShownOnce", + "neverPasteIntoChat", + "neverStoreInRepository" + ], + "additionalProperties": false, + "properties": { + "rawKeyShownOnce": { + "const": true + }, + "neverPasteIntoChat": { + "const": true + }, + "neverStoreInRepository": { + "const": true + } + } + }, + "publicDiscovery": { + "type": "object", + "required": [ + "manifest", + "skill", + "openapi" + ], + "additionalProperties": false, + "properties": { + "manifest": { + "const": "/api/external/v1/agent-integration.json" + }, + "skill": { + "const": "/api/external/v1/skill/SKILL.md" + }, + "openapi": { + "const": "/api/external/v1/openapi.json" + } + } + } + } + } + } + } + } + }, + "meta": { + "type": "object", + "required": [ + "apiVersion", + "routeVersion", + "latencyMs", + "timestamp" + ], + "additionalProperties": false, + "properties": { + "apiVersion": { + "type": "string" + }, + "requestId": { + "type": "string" + }, + "routeVersion": { + "type": "string" + }, + "operation": { + "type": "string" + }, + "latencyMs": { + "type": "integer", + "minimum": 0 + }, + "timestamp": { + "type": "string", + "format": "date-time" + } + } + } + } + }, "ErrorResponse": { "type": "object", "properties": { @@ -4037,6 +4465,104 @@ } } }, + "ExternalEditorGenerationSubmissionResponse": { + "type": "object", + "required": [ + "operationId", + "kind", + "status", + "statusUrl", + "pollAfterMs", + "updatedAtMicros" + ], + "properties": { + "operationId": { + "type": "string" + }, + "kind": { + "type": "string" + }, + "status": { + "type": "string", + "enum": [ + "queued", + "running", + "completed", + "failed" + ] + }, + "statusUrl": { + "type": "string" + }, + "pollAfterMs": { + "type": "integer", + "minimum": 250 + }, + "updatedAtMicros": { + "type": "integer" + } + }, + "additionalProperties": false + }, + "ExternalEditorGenerationJobResponse": { + "type": "object", + "required": [ + "operationId", + "kind", + "status", + "phaseLabel", + "phaseDetail", + "progress", + "updatedAtMicros" + ], + "properties": { + "operationId": { + "type": "string" + }, + "kind": { + "type": "string" + }, + "status": { + "type": "string", + "enum": [ + "queued", + "running", + "completed", + "failed" + ] + }, + "phaseLabel": { + "type": "string" + }, + "phaseDetail": { + "type": "string" + }, + "progress": { + "type": "integer", + "minimum": 0, + "maximum": 100 + }, + "error": { + "type": "string" + }, + "warning": { + "type": "string" + }, + "result": { + "type": "object", + "description": "completed 时返回的 compact 稳定结果引用；不包含完整 project/canvas、Data URL、Blob URL 或临时签名 URL。", + "additionalProperties": true + }, + "pollAfterMs": { + "type": "integer", + "minimum": 250 + }, + "updatedAtMicros": { + "type": "integer" + } + }, + "additionalProperties": false + }, "ExternalGenerationJobStatusRecord": { "type": "object", "required": [ diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index ff8a4f7b1..b54244014 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -205,6 +205,15 @@ - 验证方式：以当前 run 成功 `preview.validate` revision N 后断言 iframe 自动出现且 server 归 Tauri registry；再完成 revision N+1，断言 server 进程和 loopback origin 不变、iframe 重新加载新内容且所有响应为 `no-store`。Runner registry 单独 running 不得让页面显示预览；相同 / 更低 revision 不得刷新；停止预览后顶部必须显示“预览未启动”；构建产物和安装信息必须为 `0.1.1`。 - 关联文档：`docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`、`docs/project-memory/shared-memory/development-workflow.md`。 +## 2026-08-03 开放 Issue 115、118、127、128 的修复边界 + +- AGC 的 MCP 目录以 server 为隔离单元：可选 server 的连接、tools/list、工具归一化或聚合容量失败只关闭该 server，required server 仍失败关闭；MCP schema 包入原生 action 后只沿 subschema 关键词重定位当前 document 根的 JSON Pointer fragment，`default / const / examples / enum` 等数据值、命名 anchor 与 `$id` resource 内 fragment 保持不变。Anthropic strict 不由 `apiKind` 单独推断：AGC 只对官方 HTTPS endpoint 与 Claude 4.5+ 版本化 model id 显式开启，旧模型、未知别名和兼容网关默认关闭。开启后使用官方支持关键词白名单生成专用传输 schema，剔除不受支持的约束但不修改调用方原 schema；未知关键词、不可解析 / 递归 `$ref` 或请求复杂度超限时保持 non-strict。最后一个工具设置 ephemeral prompt-cache breakpoint；usage 统计把 cache creation / read token 一并计入 prompt 和 total。 +- Windows 私有 ACL 检查复用 `Get-Item` 对象的 `GetAccessControl()`，避免从 PowerShell 7 启动时继承的模块路径让 Windows PowerShell 5.1 的 `Get-Acl` 加载不兼容模块；静态配置门禁禁止重新引入该命令。 +- 编辑器持久化的 `prompt` 统一表示规范化用户意图；provider `actual_prompt` 只保留在 resource / asset 审计字段，系统 prompt 不进入跨资源检索字段。角色和图标的透明图、切片继承源用户 prompt；本次只修新写入，不迁移历史记录，不修改 SpacetimeDB schema。 +- 画布收到生成完成等较新权威快照时，必须把同项目待保存或在途的本地布局重放到新 revision：后端资源与生成终态优先，本地布局编辑优先；后端新增项合入，后端删除项和用户本地删除项均不得复活，合并后立即进入既有串行 CAS 保存队列。 +- 生成器合并必须把 `status / composerOpen / generatedLayerId / errorMessage / generation timestamps / characterAnimationResult` 视为后端生命周期事实；生成完成快照继续保持 `composerOpen=false`，不得被本地在途快照重新展开。提示词、参数和占位位置等本地布局编辑继续保留。 +- 同项目权威快照刷新不得无条件选择第一张图层：当前仍有效的单选、多选和生成占位选择保持，已删除的选择过滤，本来未选择时保持空选；只有首次载入或切换到另一项目时才默认选择第一张可用图层。生成完成结果需要用户显式点击后才进入选中态，后台完成回包不能偷走用户当前焦点。 + ## 2026-07-30 Provider 503 等待与耗尽状态使用严格字段派生的安全摘要 - 背景：game-chat 的进度卡只显示“等待 Provider upstream-5xx 瞬态故障退避到期”，没有 HTTP 状态、重试次数或等待时间；重试耗尽后，Runtime 和持久 conversation 又可能直接展示 `fingerprint/chars` 或 ` [redacted sensitive context]`，用户既无法判断是否在恢复，也看不到可操作的失败原因。 @@ -5721,7 +5730,7 @@ - 背景：旧创作模板退役时误把新版 `/creation`、桌面公共侧边栏和“我的”完整资料页一起缩减；只恢复视觉后，现役 profile client 又经 `rpg-entry` barrel 把旧作品库、旧 runtime request 和展示模型重新带入 Vite 与 TypeScript 图。 - 决策：桌面端继续使用原平台公共结构，一级导航固定为 `创作 / 项目 / 我的`；顶栏保留编辑器项目 / 素材搜索、泥点入口和账号胶囊；“我的”全宽保留资料编辑、陶泥号、三项统计、充值、兑换码、社区、反馈、通用设置、API Key 和法律信息。搜索只面向编辑器项目与公开编辑器素材，不恢复旧公开作品搜索。 - 依赖边界：公共 dashboard、钱包、充值、兑换码、邀请码、API Key 和设置请求迁入 `services/platform-entry`，公共账单展示迁入现役 profile model。Vite 新增退役模块 graph 门禁，ESLint 对现役源码禁止导入旧目录；目录 watch ignore、Tailwind source、tsconfig include 和 tree-shaking 都不能作为依赖隔离证明。 -- 路由与响应式边界：`/creation`、`/project`、`/profile` 都是可刷新、可前进 / 后退的稳定路由；桌面端使用侧边栏，移动端必须提供同样 `创作 / 项目 / 我的` 的三项底部 dock，不得因隐藏桌面侧边栏而丢失移动导航。 +- 路由与响应式边界（2026-08-03 纠正）：`/creation`、`/project`、`/profile` 都是稳定路由，但旧模板退役不授权扩大移动端创作范围。桌面端使用 `创作 / 项目 / 我的` 侧边栏；移动端底部 dock 只保留“我的”，直达 `/creation`、`/project`、`/editor/canvas` 或从首页触发项目 / 画布动作时统一显示桌面端提示，不挂载创作主页、项目列表或图片画布。2026-07-18 同批加入的移动端三入口口径无效，不作为产品决策依据。 - 公共设置边界：`runtime_setting` 保持原表结构与历史数据，但它是音乐音量和平台主题的现役账号级公共能力，不归入旧玩法数据壳。鉴权后的 `GET/PUT /api/runtime/settings` 必须经 `spacetime-client` 调用 `get_runtime_setting_or_default` / `upsert_runtime_setting_and_return`；保留该路由不构成恢复旧 runtime API 的先例。 - 编译门禁：除旧业务目录外，`src/uiAssets.ts`、`src/types.ts`、`src/types/**`、`src/services/runtimeAudioFeedback.ts` 和 `src/services/publicWorkCode.ts` 也是顶层退役 module，必须同时退出 Vite module graph、TypeScript、ESLint 和 Vitest；`/audio/**`、`/chat.png`、`/fusion-pixel.ttf` 及旧 pixel / story-tab / 玩法 CSS 不得进入 dev 服务或生产产物。验收时必须同时检查 `tsc --listFilesOnly`、Vite 依赖图 / 产物和退役资产路径，不能只依赖 tree-shaking。 - Rust 产物边界：`module-runtime` 继续承载账号、钱包、公共设置、追踪和 feature gate，但 `CreationEntry*`、旧公开作品、存档、浏览历史与游玩统计 DTO / command / mapper / 规则必须退出实际 rlib；只保留历史表需要的 `RuntimeBrowseHistoryThemeMode`、完整保序的钱包流水来源枚举等持久化 ABI。`check:server-rs-ddd` 必须执行 `check:module-runtime-artifact`，同时验证旧符号和字面量为零、必要 ABI 仍存在，不能以源码存在 `#[cfg(any())]` 或路由未挂载代替产物证明。 @@ -6061,3 +6070,17 @@ - 分档的双向后果写进了断言注释：把用户上传的坏图（`Decode`）报成 500 会让客户端当服务端故障去重试；把服务端自身失败（`Encode` / `Processing`）报成 400 又会让用户以为是自己的输入有问题。另断言底层文案原样带上，否则「识别不到网格」与「解码失败」在用户侧无法区分。 - 验证：删掉 `retry-after` 或把 `Decode` 改判 500，两条新用例分别精确变红。api-server 679 通过 / 3 失败（`wallet_refund_outbox` 本机环境失败，与基线一致），`cargo fmt --check` 通过。 - 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 + +## 2026-08-03 托管 MCP 未鉴权响应提供安全接入引导 + +- 决策：`/api/external/v1/mcp` 缺少、格式错误或无法验证 Bearer API Key 时继续返回相同 HTTP `401`，并增加 `WWW-Authenticate: Bearer realm="genarrative-external-editor"` 与机器可读 `details.guide`。引导只说明 Bearer Header 格式、登录后在「开发者 API Key」创建密钥、原始密钥只显示一次、凭据不得进入聊天或仓库、配置后重试 `initialize`，以及公开 manifest、Skill 与 OpenAPI 地址。 +- 安全边界：三种鉴权失败不得通过 code、message、details 结构差异暴露 Key 是否存在；未鉴权响应不得包含 MCP tools、resources、owner 或内部鉴权诊断。其它 External v1 业务路由继续使用原通用 401，不继承 MCP 专用引导。 +- 关联：`server-rs/crates/api-server/src/external_api_auth.rs`、`server-rs/crates/api-server/src/modules/external_api.rs`、`docs/openapi/genarrative-external-v1.openapi.json`、`docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md`。 +## 2026-07-31 External v1 生成统一异步并提供托管 MCP 与完整 Skill 包 + +- 异步契约：External v1 的图片生成、图片编辑、图标图集、UI 素材提取、角色动画、视频、音效和背景音乐八类 POST 固定持久化入 `external_generation_job` 并返回 HTTP `202 + operationId/statusUrl/pollAfterMs`；不受站内 `GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 影响。每次逻辑生成必须携带稳定 `Idempotency-Key`，网络结果未知或调用方轮询超时时复用原键和原 operationId，不得换键重提。 +- 查询与结果：新增 owner-safe `GET /api/external/v1/generations/{operationId}`。`queued/running` 返回 phase/progress，`completed` 返回 compact 稳定 artifact 引用，`failed` 返回脱敏错误，跨 owner 按不存在处理。compact result 允许 objectKey、resource/asset ID、assetObjectId、尺寸、媒体类型、taskId 和告警；禁止完整 project/canvas、Data URL、Blob URL、临时 signed URL、内部 provider 原文和 lease/fencing 控制字段。 +- MCP：新增托管 `/api/external/v1/mcp`，使用现有 External API Key Bearer 鉴权和无协议 session 的 Streamable HTTP JSON direct 模式。MCP tools 从同一 OpenAPI operation 形成并复用 External REST router；生成 tool 显式要求 `idempotencyKey`，另有统一任务查询 tool。MCP resources 提供使用说明、OpenAPI、Skill 入口 `SKILL.md` 和 `references/capability-routing.md`、`references/api-operations.md`、`references/authentication-and-safety.md`、`references/requests-and-outputs.md` 四篇稳定 reference；日后新增 reference 时必须同步新增独立 resource。MCP Agent 直接调用托管 tools，不安装 CLI，也不将脚本、测试或 workflow 暴露为 MCP resources。禁止开放内部 SpacetimeDB MCP、worker procedure、controller 或队列控制面。 +- Agent 发现：新增公开 `agent-integration.json`、`skill/SKILL.md` 和 `skill.zip`。manifest 同时声明 MCP、OpenAPI、完整 Skill archive、SHA-256 和包内清单；archive 必须包含 `SKILL.md`、上述四篇 references、stdlib Python helper 和 `agents/openai.yaml` 七个声明文件，不能只提供 OpenAPI JSON，也不能包含 API Key、本机路径或个人配置。完整 `skill.zip` 只供不支持 MCP 或需要本地文件上传编排的 Agent 使用，不作为 MCP resource。 +- 兼容边界：这是基于「截至 2026-07-31 尚无外部第三方存量调用方」接受的 v1 原地 breaking change；一旦出现外部活跃 Key、公开契约或联调方，后续破坏性变更必须保留兼容、经过弃用期或升级 `/api/external/v2`。 +- 关联文档：`docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md`、`docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md`、`.codex/skills/genarrative-external-editor-api/SKILL.md`。 diff --git a/docs/project-memory/shared-memory/development-workflow.md b/docs/project-memory/shared-memory/development-workflow.md index 8f26bcc0c..acc57dd6f 100644 --- a/docs/project-memory/shared-memory/development-workflow.md +++ b/docs/project-memory/shared-memory/development-workflow.md @@ -595,7 +595,7 @@ npm run check:server-rs-ddd - 移动端优先，再兼容网页端。 - 页面只展示后端返回的状态，不自行计算结论型业务状态。 -- 现役一级入口为 `/creation`、`/project`、`/profile`，桌面侧边栏和移动端底部 dock 都固定显示“创作 / 项目 / 我的”。`/creation` 只读取图片编辑器项目与 `GET /api/editor/showcase/resources`，不得重新接入旧模板入口配置、旧作品架或专属运行态。 +- 现役稳定路由为 `/creation`、`/project`、`/profile`。桌面侧边栏固定显示“创作 / 项目 / 我的”；移动端底部 dock 只显示“我的”，并对 `/creation`、`/project`、`/editor/canvas` 及页面内项目 / 画布动作统一显示桌面端提示，不挂载创作工具、项目列表或图片画布。桌面端 `/creation` 只读取图片编辑器项目与 `GET /api/editor/showcase/resources`，不得重新接入旧模板入口配置、旧作品架或专属运行态。 - 旧创作模板目录和顶层旧业务模块必须持续退出 Vite、TypeScript、ESLint 与 Vitest；旧 `/api/creation-entry/config`、模板 API、公开作品详情和运行态 API 必须保持未挂载。SpacetimeDB 历史表、迁移白名单与必要兼容类型只作为数据壳保留，不得据此恢复业务逻辑。 - 优先复用现有面板、抽屉、弹窗，不新建独立大系统。 - 不在 UI 中默认写功能说明类文本。 diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index 7e86c9b17..8770963a3 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -4020,6 +4020,7 @@ - 现象：旧 worktree 的 AGC Vite 长期占用 `127.0.0.1:3080`，marker 仍指向旧 API；新 worktree 启动 game-chat 后，配套后端在新端口 ready，随后 `beforeDevCommand` 因代理 target 不匹配返回非零，终端已经回到提示符，但原生客户端和它启动的 Runner 仍存活。客户端 WebView 实际加载旧 Vite，因此当前 master 的界面优化看起来全部缺失。 - 原因：Tauri 的字符串 `beforeDevCommand` 默认 `wait=false`。只要固定 `devUrl` 上已有可访问页面，Tauri CLI 可以在配套启动脚本完成前创建原生窗口；旧实现又直接从 npm 启动 Tauri CLI，没有在 CLI leader 退出后继续持有其 PGID / Windows 进程树。`start-dev-stack.mjs` 虽会在后端 ready 后识别 marker/API 错配，但检查时机已经晚于窗口创建，且只清理自己登记的后端和 Vite。 - 处理：`dev` 与 `game-chat` 统一先进入 `start-tauri-dev.mjs`，在启动 Tauri CLI 前无副作用检查 3080。现有 marker 只有 API target，不能证明监听器属于当前 worktree，因此任何已存在的 3080 都失败关闭，不主动杀不能证明归属的旧服务，也不因 target 看似匹配而复用。Tauri CLI 使用独立 POSIX 进程组，任意退出后按负 PGID 先 TERM、有界等待、再 KILL；Windows 固定调用 `taskkill /PID /T /F`。`start-dev-stack.mjs` 自己的后端 / Vite 独立组也在返回前有界收束。 +- Linux 容器边界：最小化 CI 容器的 PID 1 可能不回收孤儿后代，进程组在所有可执行成员退出后仍只剩 `Z` 僵尸；此时 `kill(-pgid, 0)` 仍成功，不能据此把已经完成的收束误报为失败。Linux 等待逻辑在 signal 探活后必须核对 `/proc//stat`，只把同 PGID 的非 `Z / X` 成员视为存活；`/proc` 不可读时继续使用原保守判断，macOS 等其它 POSIX 平台仍只走 signal 探活。 - 验证：定向测试必须覆盖旧 marker target 在 CLI spawn 前被拒绝、target 看似匹配仍拒绝无归属 Vite、非 HTTP 3080 失败、预检调用顺序、CLI leader 先退出后同 PGID 客户端仍收到 TERM、忽略 TERM 时升级 KILL，以及 Windows taskkill 的 `/PID /T /F` 参数。人工复验旧 worktree 占用 3080 时，新命令不得启动后端或弹出新窗口；正常启动后退出，确认 Tauri 客户端、Runner 和本轮自有后端 / Vite 均按生命周期收束。 - 关联：`apps/ai-game-creator-shell/scripts/start-tauri-dev.mjs`、`apps/ai-game-creator-shell/scripts/start-dev-stack.mjs`、`apps/ai-game-creator-shell/tests/start-tauri-dev.test.ts`、`apps/ai-game-creator-shell/tests/start-dev-stack.test.ts`。 @@ -4030,3 +4031,36 @@ - 处理：从当前 root source 的 seed lane 动态解析全部零依赖首波任务，只对这些 child 容忍 hydration `Pending`，后续 code prototype / preview 仍严格要求 Running/Completed。`streaming / ready` 仍要求当前 revision，`committed` 回复改为依据 finalization 的稳定身份查询，不随后续项目 revision 失效。 - 验证：覆盖 `design-director / art-director / code-director` 三个 Pending 首波 child 均可投影 Completed、`code-prototype` Pending 仍被拒绝；非流式专业 Agent 在 finalization 前无 stream，提交后形成 committed stream，再推进项目 revision 后仍可查询且正文不变。 - 关联：`apps/ai-game-creator-shell/src-tauri/src/agent/runtime_protocol/autonomous_completion.rs`、`apps/ai-game-creator-shell/src-tauri/src/agent/runtime_protocol/response_stream.rs`。 +## 异步生成结果未知时不能换幂等键重提（2026-07-31） + +- 现象：生成提交发生客户端超时、连接中断或响应丢失后，调用方创建新的 `Idempotency-Key` 再提交一次；原任务其实已经入队，最终造成重复生成、重复扣费和重复画布 / 素材库写入。 +- 原因：把“客户端没有收到结果”误判为“服务端没有受理”，又没有持久保留逻辑请求的幂等键和服务端返回的 `operationId`。托管 MCP 若绕过 External REST router 直接调用 worker 或 SpacetimeDB，也会形成第二套去重与状态语义。 +- 处理：一次逻辑生成只分配一个稳定幂等键；传输重试必须使用完全相同的请求体和原键。收到 `operationId` 后只查询 `/api/external/v1/generations/{operationId}`，调用方轮询超时不改变服务端任务状态。结果未知且尚未拿到 operationId 时也只用原键重试提交。MCP 生成工具必须把 `idempotencyKey` 映射到同一 REST header，并复用同一 External router、owner 和任务账本。 +- 验证：覆盖“服务端已入队但提交响应丢失”后原键重试仍返回同一 operation、换 owner 不可见、查询最终只出现一份 completed result 和一次计费 / 写回；MCP 与 REST 对同一 owner、同一请求和同一键必须命中同一 operation。 +- 关联：`server-rs/crates/api-server/src/external_generation.rs`、`server-rs/crates/api-server/src/external_mcp.rs`、`docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md`。 + +## api-server 嵌入仓库外资源时必须同步容器构建上下文（2026-07-31） + +- 现象：本地 `cargo test` 可以编译 MCP 与 Skill 下载模块，但 api-server 镜像在 Rust 编译阶段报 `include_str!` 找不到 OpenAPI 或 Skill 文件。 +- 原因：本地工作树包含完整仓库，而容器 Rust builder 原先只复制 `server-rs/` 和 `public/`；crate 中向上引用的 `docs/openapi/`、`.codex/skills/` 不会自动进入镜像构建文件系统。 +- 处理：凡 api-server 通过 `include_str!` 使用仓库根目录资源，都要在 `deploy/container/api-server.Dockerfile` 的 builder 阶段显式复制对应权威目录；不要再复制一份内容到 crate 内形成平行事实源。 +- 验证：除本地 Cargo 测试外，检查 Dockerfile 构建上下文覆盖所有 `include_str!` 相对路径；新增或移动嵌入资源时同步更新容器 COPY 和接入文档。 +- 关联：`deploy/container/api-server.Dockerfile`、`server-rs/crates/api-server/src/external_mcp.rs`、`server-rs/crates/api-server/src/external_skill_api.rs`、`docs/openapi/genarrative-external-v1.openapi.json`。 + +## 权威画布快照不能清掉本地待保存或在途布局（2026-08-03） + +- 现象：用户拖动、缩放、改层序、背景色或 viewport 后，生成完成回包立即覆盖画布；450ms 防抖尚未触发或布局保存仍在途时，编辑静默丢失，undo 也可能被生成保护项阻断。 +- 原因：服务端 revision 只能排序已提交事实，本地未落库布局没有 revision；直接清空 pending save 并整体应用权威快照等同于把“服务端更新更晚”误判成“服务端知道本地编辑”。 +- 处理：保留同项目最新本地 dirty snapshot，权威回包先更新资源和生成终态，再按稳定 item ID 合并本地布局字段并基于新 revision 保存。旧权威项在新快照缺失表示后端删除，不能从 pending 或在途旧输入复活；新权威项必须合入，本地删除的旧项不能从权威回包复活。 +- 生成器边界：`composerOpen` 与 `status / generatedLayerId / errorMessage` 一样属于后端生命周期事实；生成完成快照要求保持面板关闭时，不得被本地在途快照重新展开。提示词、参数和占位位置等本地布局编辑继续保留。集成测试夹具必须模拟后端真实完成快照：既有布局保持原位，完成结果层追加到末尾。同项目权威刷新还必须保留仍有效的单选、多选、生成占位选择或空选，只过滤已删除目标，不得无条件降成第一张图层的单选；首次载入 / 项目切换才设置默认选择。不要只跑 persistence Hook 单测，必须同时运行图片画布生成集成测试，覆盖完成后面板关闭、显式选择结果、背景清选和合并后 CAS 保存。 +- 验证：分别覆盖防抖 pending、真实在途成功与 409、后端新增、后端删除、本地删除、viewport、背景色和生成面板完成态；运行 `npm run test -- src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx`。 + +## Provider schema 能力不能从统一工具标记直接推断（2026-08-03） + +- 现象：把 OpenAI 风格 `strict` 原样透传给完整 Anthropic 工具目录，单个 schema 不支持的约束或全请求工具 / optional / union 上限会让整次 planning 返回 400。 +- 处理：能力不能从 `apiKind=anthropic` 推断；只对已验证 endpoint/model 显式开启，AGC 当前仅自动识别官方 HTTPS endpoint 与 Claude 4.5+ 版本化 model id，旧模型、未知别名和第三方兼容网关默认关闭。协议适配层用官方支持关键词白名单生成 Anthropic 专用传输 schema，对已知不支持约束仅从传输副本剔除，未知关键词、不可解析 / 递归 `$ref` 和复杂度超限均失败关闭为 non-strict，不删工具或修改调用方原 schema。真实 live 样例应包含 `$defs/$ref` 嵌套 schema，并使用官方 Anthropic endpoint，第三方兼容网关不能替代官方能力证据。 + +## 可选 MCP server 的坏目录不能拖垮全部工具（2026-08-03） + +- 现象：可选 server 已成功连接，但返回超限 schema、重复 tool identity 或要求未支持 task-mode 时，整个 MCP catalog 和本轮 Agent planning 一起失败。 +- 处理：连接、tools/list、工具归一化与聚合容量都使用同一 required / optional 边界。optional 将该 server 投影为 `connected=false + error + tool_count=0`，required 保持失败关闭；被包入 `action.input` 的 `$ref` 只重定位当前 document 根的 `#` / `#/...` JSON Pointer，命名 anchor、外部 URI 与带 `$id` 的 schema resource 内 fragment 不得改写。 diff --git a/docs/project-memory/shared-memory/project-overview.md b/docs/project-memory/shared-memory/project-overview.md index 75bc6edb5..2c846bb1a 100644 --- a/docs/project-memory/shared-memory/project-overview.md +++ b/docs/project-memory/shared-memory/project-overview.md @@ -21,7 +21,7 @@ Genarrative / 陶泥儿是一个 AI 原生互动内容与小游戏平台，把 A - 小程序 WebView 外壳：`miniprogram/`。 - 法律文本：`media/files/user_agreement.md`、`media/files/privacy_policy.md`、`media/files/disclaimer.md`。 -桌面端侧边栏和移动端底部 dock 的一级入口统一为 `创作 / 项目 / 我的`。`/creation` 是独立创作工具主页，`/project` 是画布项目入口，`/profile` 是“我的”稳定路由，继续承载账号、钱包、统计和通用设置等平台公共能力；刷新及浏览器前进 / 后退必须保持当前入口与选中态一致。 +桌面端侧边栏的一级入口为 `创作 / 项目 / 我的`；移动端底部 dock 只保留 `我的`。`/creation` 是桌面端独立创作工具主页，`/project` 是桌面端画布项目入口，`/profile` 是桌面端和移动端共用的“我的”稳定路由，继续承载账号、钱包、统计和通用设置等平台公共能力。移动端直达 `/creation`、`/project` 或 `/editor/canvas`，以及从首页触发项目 / 画布动作时，只显示桌面端创作提示，不挂载对应工具页面。 ## 当前后端路线 diff --git a/docs/project-memory/shared-memory/team-conventions.md b/docs/project-memory/shared-memory/team-conventions.md index d2fe77662..aa8fe5978 100644 --- a/docs/project-memory/shared-memory/team-conventions.md +++ b/docs/project-memory/shared-memory/team-conventions.md @@ -48,6 +48,7 @@ - 涉及中文文本时注意 UTF-8 编码和乱码排查。 - 涉及后端时遵循 DDD 分层，不把业务真相下沉到前端或临时兼容层。 - `packages/shared` 用于前后端 DTO、公开契约及跨页面复用的无业务真相 UI 组件和纯工具；不得把领域规则、后端副作用或正式状态放入其中。 +- 修改 `/api/external/v1` 的路由、HTTP 方法、请求 / 响应 DTO、请求头、状态码、鉴权或异步语义时，必须同批更新 `docs/openapi/genarrative-external-v1.openapi.json` 和对应契约测试；Rust 实现与 OpenAPI 未对齐时不得完成、提交或发布。 - `maincloud` / `Maincloud` / `MAINCLOUD` 相关代码、脚本、测试、环境变量、命令和文档要求均视为历史残留，禁止新增、运行或引用；API smoke 统一使用 `npm run dev:api-server` 与 `/healthz`。 - 涉及 SpacetimeDB 表结构、发布或迁移时，先看 `SPACETIMEDB_SCHEMA_CHANGE_CONSTRAINTS.md` 和 `SPACETIMEDB_TABLE_CATALOG.md`。 - 涉及生产发布、服务器配置、Jenkins Job 重建或回滚时，先看 `PRODUCTION_DEPLOYMENT_PLAN_2026-05-02.md`。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 5c0fd7a84..2f8d2bad6 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -69,7 +69,7 @@ - 吸附阈值以屏幕像素为准，换算到世界坐标后参与拖拽计算；边缘 / 中心线和等距吸附共用同一阈值。拖拽结束后只保存最终图层或生成占位布局，不保存临时参考线。 - 项目页封面和画布图片图层必须先渲染项目卡、图层外框、标题、尺寸和操作 chrome；图片换签或解码未完成时，只在图片区域显示轻量加载态，不阻塞外框和文字等低成本信息先出现。 - 素材量增大时，拖拽吸附热路径不得对所有素材做全量两两配对。边缘 / 中心线吸附保持线性扫描；等距吸附只在跨轴相交且轴向邻近的候选图层之间计算，避免大量远处素材拖慢 pointermove。 -- 画布自动保存使用防抖 + 串行队列：图层拖拽、缩放、资源新增和修改结果创建后延迟保存工程快照；如果上一次 `PATCH /api/editor/projects/{projectId}` 尚未完成，只保留最新待保存快照，待当前请求结束后再发送下一次保存，避免慢保存请求并发堆积触发发布入口连接限流。手型平移和小地图拖动属于临时 viewport 交互，拖动中只更新画布显示，不触发 `serializeCanvasLayout`、sessionStorage 项目缓存写入或封面快照上传，`pointerup` / `pointercancel` 后再保存最终 viewport。每次 `PATCH /api/editor/projects/{projectId}` 都必须携带最近一次服务端权威快照或保存 ack 给出的 `expectedRevision`；缺少版本号的请求在 HTTP 写入口直接拒绝，不允许回退到无版本覆盖。接口只返回 `{ projectId, canvasId, revision, updatedAt }` 轻量 ack，不返回完整 project；前端用 ack 更新后续保存版本，仍必须以后续显式读取或生成完成返回的后端快照作为项目真相。 +- 画布自动保存使用防抖 + 串行队列：图层拖拽、缩放、资源新增和修改结果创建后延迟保存工程快照；如果上一次 `PATCH /api/editor/projects/{projectId}` 尚未完成，只保留最新待保存快照，待当前请求结束后再发送下一次保存，避免慢保存请求并发堆积触发发布入口连接限流。手型平移和小地图拖动属于临时 viewport 交互，拖动中只更新画布显示，不触发 `serializeCanvasLayout`、sessionStorage 项目缓存写入或封面快照上传，`pointerup` / `pointercancel` 后再保存最终 viewport。每次 `PATCH /api/editor/projects/{projectId}` 都必须携带最近一次服务端权威快照或保存 ack 给出的 `expectedRevision`；缺少版本号的请求在 HTTP 写入口直接拒绝，不允许回退到无版本覆盖。接口只返回 `{ projectId, canvasId, revision, updatedAt }` 轻量 ack，不返回完整 project；前端用 ack 更新后续保存版本。生成完成或显式读取返回较新权威快照时，若同项目仍有防抖待保存或在途保存的本地布局，前端必须以新快照的资源和生成终态为权威，只重放本地几何、层序、分组、隐藏、锁定、翻转、viewport、背景色和生成面板编辑，并立即基于新 revision 入队保存；后端新增项必须合入，后端已删除的旧项不得被本地旧快照复活，本地在请求期间删除的旧项也不得复活。 - 移动端保留同一套状态模型，底部工具栏可横向滚动，侧边栏默认可收起。 - 项目页卡片默认点击打开工程；hover 项目卡片右下角显示 `...` 菜单，菜单承载重命名和删除。选择模式下项目卡片只切换选中态，不进入画布；底部批量工具栏提供全选 / 取消全选、已选数量、批量删除和退出选择模式。 diff --git a/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md b/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md index d5839a983..d81274588 100644 --- a/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md +++ b/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md @@ -4,18 +4,18 @@ > 2026-07-21 已实施、待生产压测专题：BgFilter 作为受限内部资源，仍遵守“单用户动作一个外部生成 job”；用户可见层与调度层都只有父 `external_generation_job`。父 future 保持原 lease 和 attempt，在当前调用栈内同步请求唯一 `bgfilter-worker` 的内部 HTTP，成功图片字节直接返回父流程。首版不新增 SpacetimeDB 子任务表、父 checkpoint / continuation 或 raw 中间结果 OSS。完整边界见 [`BgFilter 受限资源调度方案（同步内部 HTTP 原地等待版）`](./【后端架构】BgFilter受限资源调度方案-2026-07-21.md)。 -更新时间：`2026-07-21` +更新时间：`2026-07-31` ## 背景 -当前 VectorEngine `gpt-image-2`、音频、LLM 等外部生成链路多数由 `api-server` 的 HTTP handler 直接等待上游、OSS 持久化和 SpacetimeDB 回写完成。前端虽然有生成页和会话轮询，但 HTTP 进程仍承担长耗时副作用，导致接入更多玩法或大图生成时只能放大 API 进程，而不能单独扩展外部生成吞吐。 +VectorEngine `gpt-image-2`、音频、LLM 等外部生成不能由面向外部调用方的 HTTP 请求长期等待上游、OSS 持久化和 SpacetimeDB 回写。站内保留受控 `inline` 排障模式；External v1 的八类生成则固定使用持久队列和统一查询接口，避免调用方超时后重复提交、重复扣费或丢失已完成结果。 ## 目标 - 默认 `queue` 模式下，`api-server` 的 HTTP 角色只负责鉴权、入参校验、扣费前置/状态初始化、任务入队和返回 `queued` 操作结果。 - 外部生成副作用由独立 `external-generation-worker` 角色执行。 - 多个 worker 进程通过 SpacetimeDB 任务表抢占任务，依赖 lease 超时恢复，支持按进程数和单进程并发动态缩扩容。 -- 本地或小流量同步排查可显式启用 `inline` 模式，由 HTTP handler 复用同一 worker executor 同步执行并返回 `completed`；该模式不创建队列任务，也不具备 worker 横向扩容能力。 +- 本地或小流量站内同步排查可显式启用 `inline` 模式，由站内 HTTP handler 复用同一 worker executor 同步执行并返回 `completed`；该模式不创建队列任务，也不具备 worker 横向扩容能力。External v1 不继承此例外，始终异步入队。 - SpacetimeDB reducer / procedure 只做任务状态流转，不做网络、文件系统或外部 provider I/O。 - 已接入拼图 `compile_puzzle_draft`、结果页 `generate_puzzle_images` 与结果页 `generate_puzzle_ui_background`，跳一跳、拼消消和敲木鱼的外部图片生成动作，以及图片画布编辑器的图片、改图、手动去背景、图标 spritesheet、UI 素材提取、角色动作、视频、音效和背景音乐生成。后续玩法和编辑器生成入口继续复用同一队列 Module，不再为每个入口发明独立队列。 - 第一版外部生成队列粒度固定为“单个用户动作对应单个 job”。例如草稿编译、结果页单槽重生、图集重生都各自入一个 job；job 内部可以串行或并行调用 provider、OSS、SpacetimeDB 写回，但不再拆成“提示词 / 生图 / 切图 / 去背景 / 持久化 / 回写”等阶段 job。用户可见执行阶段通过现有任务行及摘要投影的轻量 `phase` 保存，不作为队列调度单位，也不写回大 payload。 @@ -37,6 +37,8 @@ - `get_external_generation_job_summary_and_return`：按 `job_id` 从轻量摘要投影读取单个任务状态，给 BFF 和生成页展示使用；必须只返回调用者有权读取的任务，不能暴露其它用户的 payload、错误详情或 worker 内部字段。 - `get_external_generation_job_result_and_return`：仅供后端内部回填异步编辑器 Agent 工具调用；按 `job_id + owner_user_id` 返回 `status`、`last_error_message` 和已持久化的 `result_payload_json`，不返回请求 payload、lease 或其它 worker 字段。该 procedure 不替代摘要状态读取接口，也不经 BFF 暴露给前端。 +External API job 复用同一个 `result_payload_json` 列，但只额外保存 `result` compact 引用：允许 objectKey、resource/asset ID、assetObjectId、尺寸、媒体类型、taskId 和告警；禁止完整 project/canvas、大布局、Data URL、Blob URL、临时 signed URL、provider 原始响应和 lease/fencing 控制字段。普通站内 job 继续保持原 payload 语义，不能为了 External 查询把所有队列结果扩成第二套资产 read model。 + 不带 `summary / summaries` 的旧 `get / list / acknowledge_external_generation_job*` procedure 只保留给受控内部兼容，不是 BFF 正式读取入口。 这个 Module 的 **Seam** 在 SpacetimeDB procedure + `spacetime-client` facade；`api-server` HTTP role 和 worker role 都只依赖这个 Interface。外部 provider、OSS、计费补偿、玩法草稿回写仍留在 `api-server` worker implementation 内，不进入 SpacetimeDB reducer。 @@ -112,6 +114,8 @@ pending/running -> cancelled (预留) - `queue`：默认值，HTTP handler 入队 `external_generation_job`，由 `external-generation-worker` 角色 claim lease 后执行；生产、预发和压测默认使用该模式。 - `inline`：HTTP handler 直接调用同一个 worker executor，同步等待 provider、OSS 和 SpacetimeDB 写回完成后返回 `operation.status = completed`；只用于本地或低并发排查，不提供队列持久化、lease 重领和 worker 横向扩容。 +External v1 八类生成不读取上述模式分支：即使进程配置为 `inline`，External handler 仍只做校验、幂等入队并返回 HTTP `202`。调用方按 `/api/external/v1/generations/{operationId}` 查询；这条外部契约不能因部署环境不同而从异步退化为同步响应。 + 同一个 Rust binary 通过 `GENARRATIVE_PROCESS_ROLE` 切换： - `api`：只启动 HTTP server。 @@ -207,7 +211,19 @@ controller 配置： 透明背景处理正常成功时，角色形象、图标 spritesheet 和 UI 素材提取的画布都同时放透明主结果与 provider 原图：透明主结果保持生成器 `generatedLayerId` 主锚点，provider 原图作为第二个图层放在其右侧；图标和 UI 实际拆分出的业务素材从 provider 原图右侧继续排列。 -inline 与 external v1 成功响应继续使用结构化 `warning.code/reason`；图标 / UI 的透明图已经成功、只有自动拆分失败时，继续返回结构化 `sliceWarning.code/reason`，其中 `sliceWarning.reason` 保留原始诊断。queue worker 把两类告警归一为有界的 `result_payload_json.warning`：只有一条时原样保留完整 `reason`；两条并存时按“通用在前、拆分在后”拼接，`code` 收敛为 `multiple-generation-warnings`（两条 `code` 相同则沿用原 `code`），任何一条都不得被丢弃。`sliceWarning.reason` 无论是否与通用告警并存都由 worker 添加“图集已生成，但自动拆分未完成：”前缀，拼接结果最后统一做长度上界收敛。任务摘要将该展示就绪的 `reason` 原样提取到 `warning_message`，单 job 状态和刷新后的任务列表 BFF 再以 `warning: string` 返回；Web 必须直接展示，不再补前缀或按 code 推断类型。历史任务保留写入时的 `reason` 快照，摘要 backfill 不按当前格式重新解释或补写前缀。该字符串语义是 worker / BFF / Web 的内部同版本契约，三者必须协调发布，不承诺滚动混部或旧 Web 缓存下的跨版本字符串兼容。 +inline 完成结果与 External v1 completed compact result 继续使用结构化 `warning.code/reason`；图标 / UI 的透明图已经成功、只有自动拆分失败时，继续返回结构化 `sliceWarning.code/reason`，其中 `sliceWarning.reason` 保留原始诊断。queue worker 把两类告警归一为有界的 `result_payload_json.warning`：只有一条时原样保留完整 `reason`；两条并存时按“通用在前、拆分在后”拼接，`code` 收敛为 `multiple-generation-warnings`（两条 `code` 相同则沿用原 `code`），任何一条都不得被丢弃。`sliceWarning.reason` 无论是否与通用告警并存都由 worker 添加“图集已生成，但自动拆分未完成：”前缀，拼接结果最后统一做长度上界收敛。任务摘要将该展示就绪的 `reason` 原样提取到 `warning_message`，单 job 状态和刷新后的任务列表 BFF 再以 `warning: string` 返回；Web 必须直接展示，不再补前缀或按 code 推断类型。历史任务保留写入时的 `reason` 快照，摘要 backfill 不按当前格式重新解释或补写前缀。该字符串语义是 worker / BFF / Web 的内部同版本契约，三者必须协调发布，不承诺滚动混部或旧 Web 缓存下的跨版本字符串兼容。 + +### External v1 异步提交与查询 + +External v1 复用上述九类 editor job kind 中除手动去背景外的八类生成 kind。外部 POST handler 只负责 API Key scope、owner、请求校验和入队，不调用 `*_for_owner` 同步执行函数： + +1. 每个生成 POST 必须携带 `Idempotency-Key`。服务端把 owner、job kind、稳定键和规范请求纳入 dedupe；未知结果重试必须复用原键。 +2. 成功入队返回 HTTP `202`、`operationId`、`statusUrl`、`pollAfterMs`，并设置 `Location` / `Retry-After`；不返回 project、asset 或媒体结果。 +3. `GET /api/external/v1/generations/{operationId}` 通过 owner-safe facade 读取摘要。`queued/running` 返回 phase/progress；`failed` 返回脱敏错误；`completed` 再读取同一 owner 的生成 artifacts 并返回 `result_payload_json.result`。 +4. 跨 owner operationId 按不存在处理。查询路径不开放 claim、renew、complete、fail、retry、acknowledge 或 controller 控制面。 +5. completed 查询返回 compact artifact 引用。调用方需要完整画布时重新读取项目，需要媒体临时 URL 时再对稳定 objectKey 换签。 + +托管 MCP 的生成 tools 也走同一 External REST router：MCP 参数中的 `idempotencyKey` 映射到 HTTP `Idempotency-Key`，`get_external_editor_generation_job` 映射统一查询。MCP 不直接调用 SpacetimeDB procedure，不形成平行队列或结果账本。 ## 验收 @@ -226,6 +242,8 @@ cargo check -p api-server --manifest-path server-rs/Cargo.toml cargo test -p spacetime-module external_generation --manifest-path server-rs/Cargo.toml cargo test -p spacetime-module level_generation_failure --manifest-path server-rs/Cargo.toml cargo test -p api-server external_generation_worker --manifest-path server-rs/Cargo.toml +cargo test -p api-server external_editor_generation --manifest-path server-rs/Cargo.toml +cargo test -p api-server external_mcp --manifest-path server-rs/Cargo.toml npm run test -- src/components/puzzle-result/PuzzleResultView.test.tsx -t "keeps generation progress visible" npm run test -- src/components/rpg-entry/RpgEntryFlowShell.agent.interaction.test.tsx -t "compile_puzzle_draft" ``` @@ -239,7 +257,7 @@ curl -f http://127.0.0.1:/healthz 本地 `npm run dev` 与 `npm run dev:api-server` 默认注入 `GENARRATIVE_PROCESS_ROLE=all`，同一 Rust 进程同时监听 HTTP 并消费外部生成队列；显式设置 `GENARRATIVE_PROCESS_ROLE` 时保留显式值。需要验证生产式拆分角色、lease 重领或扩缩容时，再分别启动 `api`、`external-generation-worker` 和 `external-generation-controller`，也可以使用隔离容器 smoke。 -生产 smoke 需要保持 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue`，并至少启动一个 `api` 角色、一个 `external-generation-worker` 角色和一个 `external-generation-controller` 角色；发布脚本会在默认 worker pattern 下自动启用并启动 `genarrative-external-generation-worker@1.service`，重启并验活 `genarrative-external-generation-controller.service`。`genarrative-api.service` 还通过 systemd `Wants=genarrative-external-generation-controller.service` 弱依赖覆盖只启动 API 的现场兜底；controller 仍是独立进程，不由 HTTP 进程内执行 `systemctl`。若 worker 数量归零，生成任务会保持 `queued/running`，不会由 HTTP 进程偷偷执行。部署验证除 `/healthz` / `/readyz` 外，还要确认任务列表 BFF 可读、未确认终态任务会弹出提示、提示展示后后台 acknowledge 且刷新后不再弹出，单 job 状态能从 `queued/running` 收敛到业务 session/detail 的 ready 或 failed。 +生产 smoke 需要保持 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue`，并至少启动一个 `api` 角色、一个 `external-generation-worker` 角色和一个 `external-generation-controller` 角色；发布脚本会在默认 worker pattern 下自动启用并启动 `genarrative-external-generation-worker@1.service`，重启并验活 `genarrative-external-generation-controller.service`。`genarrative-api.service` 还通过 systemd `Wants=genarrative-external-generation-controller.service` 弱依赖覆盖只启动 API 的现场兜底；controller 仍是独立进程，不由 HTTP 进程内执行 `systemctl`。若 worker 数量归零，生成任务会保持 `queued/running`，不会由 HTTP 进程偷偷执行。部署验证除 `/healthz` / `/readyz` 外，还要确认任务列表 BFF 可读、未确认终态任务会弹出提示、提示展示后后台 acknowledge 且刷新后不再弹出，单 job 状态能从 `queued/running` 收敛到业务 session/detail 的 ready 或 failed。External smoke 还必须证明生成 POST 返回 `202`、同幂等键不重复创建任务、统一查询能读到 compact completed result、跨 owner 返回 `404`，并通过托管 MCP 调用同一提交/查询工具链。 systemd 生产 controller 与手动兜底示例： diff --git a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md index 3fd742e30..1fa52c0f1 100644 --- a/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md +++ b/docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md @@ -248,7 +248,7 @@ Agent Runtime 负责： - 2026-07-11 调整：后台任务的可执行正文上限统一为 4,000 字符。入队 JSONL、启动后的 `currentTask/currentGoal`、planning prompt、待确认动作 task context、确认续跑和重启恢复都保留同一份正文；对话仍保存用户原始消息。状态事件、列表卡片和 `agent.db` 摘要可继续使用较短安全预览，但不能再反向作为后续 LLM 执行输入。这样长任务末尾的验收标记和输出格式要求不会在队列边界被 180 字符截断。 - 2026-07-11 调整，2026-07-12 由 Runtime V1.2 更新：后台 planning 使用 4,000 输出 token，最终回复使用 2,400，并继续叠加最多 3 次 EmptyResponse 重试。推理档位不再硬编码为 `low`：planning、普通单 Agent 聊天和最终回复统一使用解析后的 `llm.reasoningEffort`，`agentLlm..reasoningEffort` 有值时覆盖全局、缺省时继承全局；取值只允许 `default / low / medium / high`，发布默认 `high`，`default` 表示不向 Provider 发送推理档位。 - 2026-07-11 补充，2026-07-15 由 V1.17 更新：后台单 Agent 的工具 planning 响应必须提供可反序列化为 `thinkingSummary / planUpdate / plan / actions / response` schema 的 JSON object。Runtime 从模型输出中解析首个完整对象，因此对象后的尾随说明可以忽略；只有普通文本、没有完整对象，或对象无法反序列化时都不构成有效工具计划。对于这两类无效输出，Runtime 最多追加 2 次自动格式修复请求；同一次 planning 的私有 repair 请求可携带限长且经过统一敏感信息过滤的上一条模型输出或 function call 预览与协议错误，以便 Provider 真正修正格式。`.agent/agent.db` 的 `agent.runtime.tool_plan.repair` 公共审计只写 attempt/maxAttempts、protocol，以及错误、输出/调用体预览、callId 和 functionName 的 SHA-256、字符数或计数，不保存原始模型正文、错误或 function arguments。修复预算耗尽后进入既有工具规划失败路径，不得把普通文本折算为空 actions + response，也不得因此进入 completed；最终回复阶段仍按其独立的普通文本契约处理。旧文本协议可省略 `planUpdate`，但只能继续走 legacy `plan` fallback。 -- 2026-07-12 补充，2026-07-15 由 V1.17 更新，2026-07-27 由「Anthropic 与流式统一使用 Provider 原生工具」更新：OpenAI Chat / Responses 的后台工具 planning 优先注册唯一的 `submit_agent_tool_plan` function tool，并使用字符串形式 `tool_choice=required` 和 strict schema；Runtime 只接受恰好一次同名 function call，并把 arguments 复用现有 `AgentRuntimeToolPlan` 校验与两次格式修复循环。strict arguments 中 `planUpdate` 必须出现但可为 `null`，使用结构化更新时 legacy `plan` 必须为空。错误函数名、多次调用和非法 arguments 都不得执行工具。Anthropic 自 2026-07-27 起与另外两种协议一致发送原生工具目录：请求体顶层携带 `tools`（schema 字段名为 `input_schema`，无 `strict`）与对象形态 `tool_choice`（`Auto → {"type":"auto"}`、`Required → {"type":"any"}`，裸字符串会被上游拒绝），响应解析 `tool_use` block 并把 `input` 序列化为 `arguments`。planning 不再因协议强制非流式，最终普通回复继续按 Agent 配置决定是否流式。`platform-llm` 仍在本地拒绝无 function tools 的 tool choice，但不再拒绝 Anthropic function tools；协议类型继续写入 `agent.runtime.tool_plan.protocol` 审计，Anthropic 正常路径的取值为 `native_runtime_tools` 而不是 `text_json`。 +- 2026-07-12 补充，2026-07-15 由 V1.17 更新，2026-07-27 由「Anthropic 与流式统一使用 Provider 原生工具」更新，2026-08-03 收紧 strict 边界：OpenAI Chat / Responses 的后台工具 planning 优先注册唯一的 `submit_agent_tool_plan` function tool，并使用字符串形式 `tool_choice=required` 和 strict schema；Runtime 只接受恰好一次同名 function call，并把 arguments 复用现有 `AgentRuntimeToolPlan` 校验与两次格式修复循环。strict arguments 中 `planUpdate` 必须出现但可为 `null`，使用结构化更新时 legacy `plan` 必须为空。错误函数名、多次调用和非法 arguments 都不得执行工具。Anthropic 自 2026-07-27 起与另外两种协议一致发送原生工具目录：请求体顶层携带 `tools`（schema 字段名为 `input_schema`）。`strict` 能力不从 `apiKind` 推断：AGC 只对无凭据 / 自定义端口 / 路径的官方 HTTPS endpoint 和 Claude 4.5+ 版本化 model id 显式开启，旧模型、未知别名和兼容网关默认关闭。开启后使用官方支持关键词白名单生成 Anthropic 专用传输 schema，已知不支持约束只从传输副本剔除，调用方原 schema 保持不变；未知关键词、不可解析 / 递归 `$ref` 和 strict 工具 / optional / union 请求级复杂度超限时该工具保持 non-strict，不能因完整 AGC 工具集超限让整次请求被上游拒绝。工具数组最后一项携带 `cache_control: {"type":"ephemeral"}` 作为 prompt cache breakpoint；非流式和流式 usage 都将 `input_tokens + cache_creation_input_tokens + cache_read_input_tokens` 合并为 prompt tokens。`tool_choice` 使用对象形态（`Auto → {"type":"auto"}`、`Required → {"type":"any"}`，裸字符串会被上游拒绝），响应解析 `tool_use` block 并把 `input` 序列化为 `arguments`。planning 不再因协议强制非流式，最终普通回复继续按 Agent 配置决定是否流式。`platform-llm` 仍在本地拒绝无 function tools 的 tool choice，但不再拒绝 Anthropic function tools；协议类型继续写入 `agent.runtime.tool_plan.protocol` 审计，Anthropic 正常路径的取值为 `native_runtime_tools` 而不是 `text_json`。 - 2026-07-11 调整，2026-07-15 由 V1.17 更新：工具计划五个顶层字段均为必填并拒绝未知顶层字段；`thinkingSummary`、结构化计划的 `explanation / step` 与 `action.tool` 必须非空。`planUpdate` 只接受 `null` 或最多 8 个唯一步骤，状态限于 `pending / in_progress / completed` 且至多一个 `in_progress`。这样 `{}`、前置无关 JSON 或结构不完整对象会触发格式修复，不会成为假完成信号。空 actions 只有在 verification、process/join/delivery 和结构化计划完成门禁都通过后才表示 planning 收束；response 非空时直接采用，response 为空时进入独立最终回复生成。`agent.runtime.project.verify` 记录补充 `runId / actionId / actionFingerprint`，用于在多 Agent 并行验证时把命令终态与具体 Runtime 动作关联。 - 2026-07-15 V1.17 公共审计收紧：`thinking_summary` event 只保存固定摘要、正文 SHA-256 与字符数，legacy `plan` event 只保存步骤数；结构化计划审计只保存 explanation 的哈希与字符数，以及 step 标题哈希、状态和数量。模型 thinking、legacy plan 标题、repair 错误和调用体只允许出现在对应私有 Runtime 上下文或有界 repair 请求中，不得复制到公共 event、task 或 Agent DB 正文字段。 - 2026-07-10 补充：Agent Runtime state / result 新增 `taskQueue`，从 `.agent/runtime/tasks/.jsonl` 中每个 `runId` 的最新记录汇总 `total / pending / running / completed / failed / latestRunId`；开发窗口 Runtime 面板、主窗口 Agent 状态列表、`agent.run_status` observation 和下一轮 planning prompt 都读取该摘要，用于判断同一 Agent 是否仍有排队任务。该字段是运行观测摘要，不新增调度器、SQLite 或独立 worker。 @@ -259,7 +259,7 @@ Agent Runtime 负责： - 2026-07-10 补充：后台任务工具箱已加入 `project.restore`。Agent 可在 diff 或自检发现本轮修改走偏后请求恢复到指定 checkpoint；Runtime 复用 `project.restore` 权限策略和项目写锁，observation 只返回 checkpoint id、恢复文件数和删除文件数，不返回本机绝对路径。默认确认策略下不会静默回滚用户项目。 - 2026-07-10 补充：单 Agent 聊天和后台 planning prompt 会读取同一个 Agent 的 Runtime 连续上下文，把本 Agent 最近 status / phase / runId / 当前任务 / 下一步、最近回复、计划、观察、最近 3 条工具动作、最近事件、最近 3 条任务记录和工具策略摘要带入下一轮推理；上下文按规范 taskId 隔离，不读取其他 Agent 的 runtime 文件，并在进入 prompt 前过滤密钥和本机绝对路径。新后台 run 启动时会继承本 Agent 上次 `recentToolCalls` 和 `lastResponse`，让多轮任务不丢失结构化行动证据。 - 2026-07-10 补充：后台任务工具箱已加入 `preview.start`。Agent 可在 loop 中自行请求启动当前项目的本地 HTTP 预览；Runtime 会复用 `preview.start` 策略、项目写锁、共享 `PreviewRegistry`、manifest 预览状态、`.agent/logs/preview.log` 和 run trace 追加逻辑，并把 `agent.runtime.preview.start` 写入 `.agent/agent.db`。该 observation 只向 LLM 返回 localhost URL 与端口，不返回用户项目绝对路径。 -- 2026-07-10 补充，2026-07-28 收紧：后台任务工具箱提供 `canvas.asset_generate`。Agent 在 loop 中给出素材 prompt、`outputPath`、比例、尺寸、kind 与展示名；Runtime 通过 AppData / Tauri 配置里的 `editorApi` 调用 External Editor API。生成前按本地项目名称创建或复用同名画布项目和同名素材库目录，请求必须携带 `projectId + assetFolderId + canvasCompletion`，生成结果同时进入平台画布、平台素材库和本地项目。canonical 视觉 DAG 固定为：`art-director` 通过 `POST /api/external/v1/editor/images/generations` + `kind=spec` 生成 `assets/art-spec.png`；`design-foundation` 精确引用该 resourceId，通过同一路由 + `kind=ui-design` 生成 `assets/ui-prototype.png`；`art-asset-plan` 使用同一 resourceId 和具体 `iconDescriptions`，通过 `POST /api/external/v1/editor/icon-spritesheets/generations` 生成真实透明的 `assets/art-spritesheet.png`。UI extraction 只处理已有带标注 UI 图，不属于这条 DAG；图集不得回退到普通生图。UI 原型 prompt、`generationInputs.artSpec` 和 `ui-prototype.v2` 验收必须从当前项目玩法合同提取 HUD、可玩区域、关键实体、操作、失败/重开与移动布局，禁止预设塔防或补入合同中不存在的卡牌、波次、敌人入口。canonical UI 原型固定请求 `2K + 16:9`。旧正式图不合格时，普通原合同只能返回 `needs-repair`；Supervisor 认领后仅可签发一次完整继承原合同的 repair，由原 owner 使用 `replaceExisting=true` 原位替换，禁止先删除正式图。图集响应含 `warning.code=postprocess-failed-source-preserved` 或不含任意 `alpha < 255` 时不得登记为正式透明图集；仅有 `sliceWarning` 时可保留完整透明图，但不宣称已有独立切片。本地 manifest 持久生成 route、kind 与精确参考 resourceId；登记失败时删除本轮刚写入的新文件。API Key 不进入 observation、manifest、agent.db 或日志。 +- 2026-07-10 补充，2026-07-31 收紧：后台任务工具箱提供 `canvas.asset_generate`。Agent 在 loop 中给出素材 prompt、`outputPath`、比例、尺寸、kind 与展示名；Runtime 通过 AppData / Tauri 配置里的 `editorApi` 调用 External Editor API。生成前按本地项目名称创建或复用同名画布项目和同名素材库目录，请求必须携带 `projectId + assetFolderId + canvasCompletion`，生成结果同时进入平台画布、平台素材库和本地项目。canonical 视觉 DAG 固定为：`art-director` 通过 `POST /api/external/v1/editor/images/generations` + `kind=spec` 生成 `assets/art-spec.png`；`design-foundation` 精确引用该 resourceId，通过同一路由 + `kind=ui-design` 生成 `assets/ui-prototype.png`；`art-asset-plan` 使用同一 resourceId 和具体 `iconDescriptions`，通过 `POST /api/external/v1/editor/icon-spritesheets/generations` 生成真实透明的 `assets/art-spritesheet.png`。每次生成提交必须携带稳定 `Idempotency-Key`，持久保留返回的 `operationId`，按 `pollAfterMs` 查询 `/api/external/v1/generations/{operationId}`；只有 `completed` 才消费 compact result 并换签下载，客户端超时或结果未知时不得换键重提。UI extraction 只处理已有带标注 UI 图，不属于这条 DAG；图集不得回退到普通生图。UI 原型 prompt、`generationInputs.artSpec` 和 `ui-prototype.v2` 验收必须从当前项目玩法合同提取 HUD、可玩区域、关键实体、操作、失败/重开与移动布局，禁止预设塔防或补入合同中不存在的卡牌、波次、敌人入口。canonical UI 原型固定请求 `2K + 16:9`。旧正式图不合格时，普通原合同只能返回 `needs-repair`；Supervisor 认领后仅可签发一次完整继承原合同的 repair，由原 owner 使用 `replaceExisting=true` 原位替换，禁止先删除正式图。completed result 含 `warning.code=postprocess-failed-source-preserved` 或对应媒体不含任意 `alpha < 255` 时不得登记为正式透明图集；仅有 `sliceWarning` 时可保留完整透明图，但不宣称已有独立切片。本地 manifest 持久生成 route、kind、operationId 与精确参考 resourceId；登记失败时删除本轮刚写入的新文件。API Key 和幂等键不进入 observation、manifest、agent.db 或日志。 - 2026-07-10 补充：后台任务工具箱已加入 `task.list`。Agent 可在 loop 中读取 manifest 任务图、每个 seed task 的状态 / 依赖 / 产物交接，以及按依赖计算的 `readyTaskIds`；Runtime 复用 `task.list` 项目权限策略，策略要求确认或拒绝时只返回策略 observation，不向 LLM 暴露任务图细节。 - 2026-07-10 补充：后台任务工具箱已加入 `task.update`。Agent 可在 loop 中把 manifest 种子任务状态更新为 `pending / running / waiting-for-confirmation / completed / failed`，用于表达长期后台任务的当前进度；Runtime 复用 `task.update` 策略和项目写锁，实际只修改 `.agent/manifest.json` 中已有 taskId 的 `status`，并写入 `agent.runtime.task.update` 审计记录。策略要求确认或拒绝时不会修改 manifest，也不会创建新任务。 - 2026-07-10 补充：后台任务工具箱已加入 `file.list`。Agent 可在 loop 中自行列出项目文件摘要或某个相对目录下的文件摘要，再决定是否继续读取具体文件；Runtime 复用 `file.list` 项目权限策略，observation 只包含项目相对路径、类型和大小，不读取文件内容、不返回项目绝对路径。 @@ -346,7 +346,7 @@ game-project/ - `canvas.project_open` 只打开本机 Genarrative 编辑器的 `/editor/canvas?projectid=...`，默认地址为 `http://127.0.0.1:3000`，开发者可在开发窗口改成本机端口；不允许打开远程站点或任意 URL。 - 画板资源回流到本地项目 `assets/`，并在 manifest 中记录画板项目、资源 ID、assetObjectId、prompt、model、taskId 和 assetKind；当前最小落地提供 `asset.register` 登记项目内已有资产，并提供 `canvas.export_import` 读取现有画板素材导出 ZIP。 - `canvas.project_sync` 复用 Genarrative External Editor API，读取用户平台 API Key 可访问的画板项目快照，通过 `/api/external/v1/assets/read-url` 换签并把资源下载到本地项目 `assets/canvas-sync/`；默认 API base URL 为 `http://127.0.0.1:8082`，可用 Tauri 应用配置目录中的 `game-creator.config.json` 的 `editorApi.baseUrl` 覆盖，API Key 从同一配置的 `editorApi.apiKey` 读取，不写入项目文件、trace、manifest 或日志。 -- Agent loop 中美术组 `Asset` 和音乐组 `SFX` 会读取 `.agent/manifest.json`；图片生成先通过 External Editor API 项目与素材库接口准备同名画布会话，再调用 `/api/external/v1/editor/images/generations`，携带 `projectId`、`assetFolderId`、`assetLabel`、`generationInputs.artSpec` 和 `canvasCompletion`，随后通过 `/api/external/v1/assets/read-url` 换签下载到受控本地 `assets/` 路径，登记为 `canvas` 来源资产并追加 `canvas.asset_generate` 本地索引记录。API Key 不写入项目文件、agent.db、trace、manifest 或日志；未配置 Key 或生成失败时，图片产物型任务保持阻塞/失败，不能以文字计划完成。音乐组仍只建议同步已有音频资源，不调用图片生成接口。 +- Agent loop 中美术组 `Asset` 和音乐组 `SFX` 会读取 `.agent/manifest.json`；图片生成先通过 External Editor API 项目与素材库接口准备同名画布会话，再带稳定 `Idempotency-Key` 调用 `/api/external/v1/editor/images/generations`，请求携带 `projectId`、`assetFolderId`、`assetLabel`、`generationInputs.artSpec` 和 `canvasCompletion`。Runtime 必须保存返回的 `operationId` 并按 `pollAfterMs` 查询统一状态端点；completed 后从 compact result 取得稳定 objectKey/resourceId，再通过 `/api/external/v1/assets/read-url` 换签下载到受控本地 `assets/` 路径，登记为 `canvas` 来源资产并追加 `canvas.asset_generate` 本地索引记录。API Key 和幂等键不写入项目文件、agent.db、trace、manifest 或日志；operationId 只作为该生成动作的可恢复身份保存。未配置 Key、查询 failed 或 compact result 缺少稳定媒体引用时，图片产物型任务保持阻塞/失败，不能以文字计划完成。音乐组仍只建议同步已有音频资源，不调用图片生成接口。 - `canvas.asset_import` 当前作为最小真实链路：导入项目目录内已有文件为 `canvas` 来源资产，并要求记录画板项目 ID 以及 resourceId 或 assetObjectId。 - 项目工作台点击已登记图片时必须在客户端资源浮层中直接渲染图片，而不是只展示路径与 MIME。图片通过受控 Tauri 命令从项目 `assets/` / `game/` 读取，只允许 manifest 已登记资产或已完成任务产物，并复用 `file.read` auto 权限、图片魔数、文件大小、像素尺寸、普通文件、路径漂移和符号链接校验后以 data URL 返回；首版只支持 PNG、JPEG、WEBP，不向 WebView 暴露任意本机文件协议或绝对路径。 - `canvas.export_import` 复用 `/editor/canvas` 已有素材导出 ZIP 格式，读取根 `metadata.json`、复制 `images/` / `media/` / `sequences/` 到本地项目 `assets/canvas-imports/`，再按导出层登记为 `canvas` 来源资产；导出包不保存真实 resourceId 时，使用 `canvas-export:` 作为可追踪 assetObjectId，不伪造后端资源行。 @@ -717,7 +717,7 @@ game-project/ - 2026-07-15 当前正式 `openai_chat / gpt-5.5` 路由的三轮真实联网专项均 FAIL：上游接受搜索开启请求并完成 lifecycle，但模型没有获得原生搜索能力，无法命中动态 GitHub release baseline。客户端能力已落地但该路由不可启用；最终复验使用正式 AppData 同级的 `0600` 私有配置副本，源配置 inode/nlink/timestamps/hash 前后完全一致，隔离 Runner/AppData/项目和全部泄漏门禁均安全收束。 - 2026-07-15 起，同一 Runtime 文档的“V1.21 单 Agent token-aware 持久上下文压缩”作为长会话预算事实源。全局/per-Agent LLM 配置提供 context window、自动压缩阈值和工具输出 token 限额；后台 planning 超阈值时只压缩旧 conversation/observation prefix，Goal、任务、计划、steer、pending、verification 与副作用身份逐字段保留。开发 Agent 窗口与 `agc:chat` 提供同一安全 `/compact`，正式用户首页不新增控制项。私有 sidecar、context bundle 绑定、Provider orphan barrier、公共零正文和 30 轮真实长链路按 Runtime V1.1 的 V1.21 章节验收。 - 2026-07-15 V1.21 已落地并完成真实验收：`context-compaction` suite 在正式 `openai_chat / gpt-5.5` 路由上完成 30/30 轮、两次压缩 revision、一次 Runner pidfd 强杀恢复和早期显式约束召回；最大估算输入 29134/64000，32 组 Provider lifecycle 唯一闭合，重复 assistant/audit、工具重放以及公共正文、summary、API Key、诱饵、项目路径和正式配置路径泄漏均为 0。首轮第 22 轮 Provider transport 失败按单次请求终态停止且零重放，新 disposable 项目完整重跑后 PASS。 -- 2026-07-15 起，同一 Runtime 文档的“V1.22 Runner-owned MCP 动态工具”作为外部工具扩展事实源。AppData 配置管理 STDIO / Streamable HTTP server、Bearer/static header、工具 allow/deny 与 `auto / confirm / writes / deny` 审批；独立 Runner 持有连接并把过滤后的真实 tool schema 和 server instructions 送入 planning。模型通过现有 `submit_agent_tool_plan` 请求 `mcp.call`，调用继续复用 durable pending action、确认、steer、Goal、reconciliation 和 V1.21 token 预算；完整结果只落私有 sidecar，正式用户首页不新增 MCP 调试配置。本切片不宣称 OAuth、resources/prompts、sampling、elicitation 或 MCP task-mode 已实现。 +- 2026-07-15 起，同一 Runtime 文档的“V1.22 Runner-owned MCP 动态工具”作为外部工具扩展事实源。AppData 配置管理 STDIO / Streamable HTTP server、Bearer/static header、工具 allow/deny 与 `auto / confirm / writes / deny` 审批；独立 Runner 持有连接并把过滤后的真实 tool schema 和 server instructions 送入 planning。模型通过现有 `submit_agent_tool_plan` 请求 `mcp.call`，调用继续复用 durable pending action、确认、steer、Goal、reconciliation 和 V1.21 token 预算；完整结果只落私有 sidecar，正式用户首页不新增 MCP 调试配置。可选 server 的 tools/list、schema 归一化、重复 tool identity、单 server 或聚合目录容量、未支持 task-mode 错误只隔离该 server，目录状态记录 `connected=false + error` 且不暴露其工具；required server 对相同错误继续失败关闭。MCP input schema 包入原生 action 的 `input` 属性时，只把当前 schema document 根的 `#` 与 `#/...` JSON Pointer 重定位到 `#/properties/input...`；命名 anchor、外部引用和带 `$id` 的独立 schema resource 内 fragment 保持不变。本切片不宣称 OAuth、resources/prompts、sampling、elicitation 或 MCP task-mode 已实现。 - 2026-07-15 V1.22 已落地并完成真实验收：开发配置窗可管理 server、敏感凭据、工具过滤和审批并通过 Runner 查看有界目录，`agc:chat` / `agc:swarm` 可用 `/mcp` 查询状态。正式 `openai_chat / gpt-5.5` 路由真实调用 STDIO/Streamable HTTP lookup 和确认后的 mutate，正常 run 的 action/sidecar/receipt 各 3 且最终 assistant 唯一；第二 run 在 HTTP mutate 副作用后强杀 Runner，只进入 1 次 reconciliation，调用、sidecar、receipt 和 assistant 均未重放。公共 arguments、结果正文、instructions、凭据和项目/配置路径泄漏为 0，一次性现场已清理。 - 2026-07-16 起，同一 Runtime 文档的“V1.23 单 Agent 持久用户输入请求”作为 Needs input 事实源。Agent 可在计划未完成时通过 `user.input_request` 提出 1-3 个结构化问题，Runtime 保持同一 run 并暂停；Project Supervisor、开发 Agent 窗口和 `agc:chat` 从私有 sidecar 展示并提交答案。普通 steer、工具确认和最终回复不再承担问题回答语义，问题/答案正文不进入公共审计。 - 2026-07-16 V1.23 已完成真实验收：正式 `openai_chat / gpt-5.5` 路由在 Project Supervisor 上产生 1 个含 2 选项的 Needs input，等待期 Runner pidfd 强杀恢复未增加 Provider 请求，回答后同 Session/run 完成唯一最终回复。问题/回答各一条，重复消息、公共正文、密钥、路径和报告泄漏均为 0，隔离现场已清理。 @@ -802,7 +802,7 @@ game-project/ - Project Supervisor 只有在本轮必需 manifest tasks 全部 `completed`、当前配置对应的正式路径齐全且通过类型 / 可解析性检查、最新 project revision 的 `game.static_smoke` 与 `preview.validate` 都通过后，才能写入唯一最终回复。delivery 的 `completed / evidence-ready`、历史 revision 成功或单个文件存在都不能替代最终集成验收。已 `ready / claimed-by-parent` 的相同终态 delivery 在恢复扫描中按幂等重放，保留首次冻结结果，不再制造重复 `agent.delegate.result_failed`；真实终态冲突仍失败关闭。 - 验证：`npm run agc:test` 已通过确定性 loopback Provider、真实 Runtime、项目写入和浏览器链路验收：同一父 Run 下 16 个 manifest task 均只有一个 logical run、一次 start、一次 completed 和一次 manifest projection，且无 failed / cancelled；父 run 与全部子 run 完成，最终 revision 为 `11`，基础正式产物、静态 smoke、桌面 / 移动 `37/37` 试玩通过，pending、reconciliation、Provider 失败、重复和泄漏计数均为 `0`。该结果不替代独立外部 Provider 验收。 - 2026-07-26 本轮已验证 `npm run agc:config` 的终端配置链路。向导与 GUI 使用同一 Tauri identifier 对应的系统 AppData 和同名 `game-creator.config.json` / 可选 local overlay；读取已有配置时只更新有效 LLM 层，保留 `agentLlm`、`editorApi`、`mcpServers` 等其它配置。API Key 只从隐藏输入读取，拒绝 `--api-key`、仓库内目录、Git 已跟踪配置、符号链接，以及不是以 `world.genarrative.ai-game-creator` 为独立叶目录的 `--config-dir`，防止把任意父目录整体改成私有权限。保存使用同目录 `0600` 临时文件原子替换，POSIX AppData 目录保持 `0700`，Windows 使用当前用户独占 DACL，写后复用真实 `--llm-status` 检查；隐藏输入收到 `SIGINT / SIGTERM / SIGHUP` 时先恢复 raw mode 和 pause 状态再重发原信号，向导启动的 Cargo / npm 使用独立进程组并在信号路径有界收束整棵子进程树。 -- 2026-07-27 Windows DACL 启动回归修正：`powershell.exe -Command` 后追加的位置参数会被 PowerShell 5.1 拼接进命令文本，不能用 `$args` 安全接收包含空格的 AppData / 临时目录。DACL 脚本改为从仅传给该子进程的环境变量读取目标绝对路径和目录标记；`npm run agc:typecheck` 必须在真实 Windows 上执行配置回归，保证 `npm run agc` 的 `beforeDevCommand` 不因路径解析失败退出。 +- 2026-07-27 Windows DACL 启动回归修正，2026-08-03 补充 pwsh 模块隔离：`powershell.exe -Command` 后追加的位置参数会被 PowerShell 5.1 拼接进命令文本，不能用 `$args` 安全接收包含空格的 AppData / 临时目录。DACL 脚本改为从仅传给该子进程的环境变量读取目标绝对路径和目录标记，并复用 `Get-Item` 返回的 `FileSystemInfo.GetAccessControl()` 读取 ACL，不调用会因父 PowerShell 7 `PSModulePath` 污染而自动加载不兼容模块的 `Get-Acl`；`npm run agc:typecheck` 必须在真实 Windows 上执行配置回归，保证 `npm run agc` 的 `beforeDevCommand` 不因路径解析或模块加载失败退出。 - 2026-07-31 macOS 临时路径回归修正：配置目的地安全检查返回解析过现存父目录的真实路径，回归 fixture 的期望值也必须先使用平台原生 `realpath` 规范化临时根目录。macOS 下 `/var/folders/...` 与 `/private/var/folders/...` 是同一目录身份，不得用未规范化字符串阻断 `agc:typecheck`。CI 还必须使用“真实目录 + 符号链接父目录 + 不存在叶目录”确定性复现该语义；Windows 使用 junction 覆盖驱动器号、大小写与链接路径差异。fixture 的规范化与断言必须位于同一 `try/finally` 清理边界内。 - 2026-07-27 项目总控右栏空态与持久状态水合修正：项目尚未产生 Runtime 时仍显示“尚未开始”状态块和创作入口，不把消息列表的弹性剩余空间裸露为空白；若 active Session 索引缺失但项目内已有 `project-supervisor` Runtime，工作台必须从 `read_game_creator_agent_runtimes` 的权威项目列表恢复总控 Session 与状态。`needs-reconciliation` 统一显示为“失败 / 待核对”，不能因对话索引缺失隐藏已落盘的失败事实。 - 2026-07-28 Windows `tool-plan` 成功响应交接修正：相对目录句柄下安装 handoff 账本改用 `NtSetInformationFile(FileRenameInformation)`；`SetFileInformationByHandle(FileRenameInfo)` 不接受当前实现所需的非空 `RootDirectory`，会稳定返回 `ERROR_INVALID_PARAMETER (87)` 并让总控首轮进入 `needs-reconciliation`。实现继续绑定已验证的父目录句柄和相对 hash 文件名，不退化为绝对路径 rename；“按句柄安装”归入 `tool-plan-storage`。总控对 reconciliation 提供“已核对，结束旧任务”，取消后有 pending task 时只等待 Runner 续跑，队列为空时才允许显式 retry；自主构建 Supervisor 的 retry source 从原 Run Profile 绑定恢复并重新验证为可信 GUI / CLI 根入口，不降级成普通后台任务来源。 diff --git a/docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md b/docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md index 804bcd7a9..6f52dcf98 100644 --- a/docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md +++ b/docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md @@ -26,7 +26,7 @@ - 为历史审计、迁移、资产归属核对所必需的最小只读表定义；不得借兼容读取重新暴露旧创作、发布、公开详情或运行接口。 - 编辑器、项目、账号、钱包、资产、HostBridge、运维和安全等平台公共能力。 - 通用 `feature_gate_config`、`GET/PUT /admin/api/feature-gates` 与后台 `#gray-release` 控制页。灰度页只读取通用 gate，不再请求旧 `/admin/api/creation-entry/config`，固定目标只登记现役功能；不得恢复 `creation-entry:*` 动态目标。 -- 新版 `/creation` 创作工具主页、`/project` 项目入口、稳定的 `/profile` 个人页路由、`creation-home` 展示组件与现役静态资产。桌面端保留“创作 / 项目 / 我的”公共侧边栏，移动端保留同样三项的底部 dock；“我的”保留头像 / 昵称编辑、陶泥号复制、钱包与账单、统计、充值、兑换码、玩家社区、反馈、通用设置、开发者 API Key 和法律信息，不恢复旧模板入口、旧作品架或生成队列。 +- 新版 `/creation` 创作工具主页、`/project` 项目入口、稳定的 `/profile` 个人页路由、`creation-home` 展示组件与现役静态资产。桌面端保留“创作 / 项目 / 我的”公共侧边栏；移动端底部 dock 只保留“我的”，不得因退役旧模板而扩大移动端创作范围。“我的”保留头像 / 昵称编辑、陶泥号复制、钱包与账单、统计、充值、兑换码、玩家社区、反馈、通用设置、开发者 API Key 和法律信息，不恢复旧模板入口、旧作品架或生成队列。 - `runtime_setting` 是账号级公共设置事实，不属于旧模板运行态。原表结构和数据不变，继续由鉴权后的 `GET/PUT /api/runtime/settings`、`get_runtime_setting_or_default` 与 `upsert_runtime_setting_and_return` procedure 支撑音乐音量和平台主题读写。 - 旧页面、测试、素材、handler、service、worker、生成 bindings 和纯业务 crate 的源码目录；它们仅用于历史追溯，不属于任何正式入口或编译目标。 @@ -65,7 +65,7 @@ ## 验收 - 旧 URL 不再命中旧页面或后端路由。 -- `/creation`、`/project` 与 `/profile` 在桌面端显示“创作 / 项目 / 我的”公共侧边栏，在 `390x844` 等移动视口显示同样三项的底部 dock；点击、刷新及浏览器前进 / 后退均保持路由与选中态一致。新创作主页只调用编辑器项目和公开编辑器素材接口；顶栏保持现役搜索、公共泥点入口与账号胶囊，不重新拼装平行账号按钮组。 +- `/creation`、`/project` 与 `/profile` 在桌面端显示“创作 / 项目 / 我的”公共侧边栏；在 `390x844` 等移动视口，底部 dock 只显示“我的”。移动端直达 `/creation`、`/project`、`/editor/canvas` 时显示桌面端创作提示，且不得挂载创作主页、项目列表或图片画布；移动端首页触发项目或画布动作时使用同一门禁。桌面端新创作主页只调用编辑器项目和公开编辑器素材接口；顶栏保持现役搜索、公共泥点入口与账号胶囊，不重新拼装平行账号按钮组。 - “我的”桌面布局按原平台公共资料页全宽展示四个常用入口、两行设置和法律栏；头像、昵称、复制、充值、兑换码、社区、反馈、API Key 等入口可用，但不发起旧模板、旧公开作品或旧运行态请求。 - 鉴权访问 `GET/PUT /api/runtime/settings` 不得返回 404，读写必须经 `spacetime-client` 调用现役 settings procedure；未鉴权请求返回 401，不恢复任何旧运行态设置路由。 - `tsc --listFilesOnly` 与 Vite 干净加载均不得出现旧业务目录、上述顶层退役 module 或小程序旧订阅授权实现。 diff --git a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md index e44d5e67a..4be842ace 100644 --- a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md +++ b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md @@ -640,7 +640,7 @@ Responses 的终态载荷既是工具调用的恢复源，也是正文的恢复 - Rust 结构体：`EditorAsset` - 源码：`server-rs/crates/spacetime-module/src/editor_project_storage.rs` -- 说明：图片画布账号级素材表，保存用户上传 / 生成素材的名称、文件夹、图片读取地址、可选封面 `thumbnail_src`、OSS 引用、尺寸、来源类型、prompt、provider、真实操作 `task_id`、可选后台归组 `group_task_id`、拆分批次预期数量 `group_task_expected_asset_count`、`asset_kind`、`generation_inputs_json`、可选 `source_resource_id` 和 `generation_cost_mud_points`。归组字段追加在表尾并默认 `None`，只用于稳定派生任务的后台分组，不替代 `task_id`；批次是否初始完整以独立完成事实为准，不按当前剩余素材数反推。素材在同一账号的所有项目中可见；图片 / 图标 / UI 提取等生成 BFF 在请求携带 `asset_folder_id` 时负责创建账号级生成素材并返回 asset 快照，若同次生成也创建了 `editor_project_resource`，则把该 `resource_id` 写入 `source_resource_id`。角色动作生成保留原始绿幕视频中间素材，同时把最终帧序列作为一条 `asset_kind = character-animation` 素材入库：首帧写入 `image_src` / `thumbnail_src`，完整帧列表、FPS、时长和预览视频写入 `generation_inputs_json.characterAnimation`，不把每帧拆成独立素材。生成视频会抽取首帧封面写入 `thumbnail_src`，素材库和再次放入画布时用它作为 video poster。素材库快照通过 `asset_id` 回查对应 `editor_showcase_asset`，供左侧素材菜单展示 `pending` / `approved` / `rejected` 审核状态；公开事实不落在账号素材表，素材库只发起提交审核。素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId，画布从 resource / asset 级元数据恢复素材类别和用户可见生成输入快照。 +- 说明：图片画布账号级素材表，保存用户上传 / 生成素材的名称、文件夹、图片读取地址、可选封面 `thumbnail_src`、OSS 引用、尺寸、来源类型、prompt、provider、真实操作 `task_id`、可选后台归组 `group_task_id`、拆分批次预期数量 `group_task_expected_asset_count`、`asset_kind`、`generation_inputs_json`、可选 `source_resource_id` 和 `generation_cost_mud_points`。`prompt` 固定表示规范化后的用户原始意图，供跨资源搜索和用户侧元数据使用；provider 实际返回的改写只写 `actual_prompt`，提交给 provider 的系统 / 工程化 prompt 不得写入 `prompt`。角色透明图、图标透明图和自动切片等派生产物继承源用户 prompt，并用 `source_resource_id`、`generation_inputs_json`、provider / asset kind 表达处理来源。归组字段追加在表尾并默认 `None`，只用于稳定派生任务的后台分组，不替代 `task_id`；批次是否初始完整以独立完成事实为准，不按当前剩余素材数反推。素材在同一账号的所有项目中可见；图片 / 图标 / UI 提取等生成 BFF 在请求携带 `asset_folder_id` 时负责创建账号级生成素材并返回 asset 快照，若同次生成也创建了 `editor_project_resource`，则把该 `resource_id` 写入 `source_resource_id`。角色动作生成保留原始绿幕视频中间素材，同时把最终帧序列作为一条 `asset_kind = character-animation` 素材入库：首帧写入 `image_src` / `thumbnail_src`，完整帧列表、FPS、时长和预览视频写入 `generation_inputs_json.characterAnimation`，不把每帧拆成独立素材。生成视频会抽取首帧封面写入 `thumbnail_src`，素材库和再次放入画布时用它作为 video poster。素材库快照通过 `asset_id` 回查对应 `editor_showcase_asset`，供左侧素材菜单展示 `pending` / `approved` / `rejected` 审核状态；公开事实不落在账号素材表，素材库只发起提交审核。素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId，画布从 resource / asset 级元数据恢复素材类别和用户可见生成输入快照。 - 索引：`by_editor_asset_owner_user_id`、`by_editor_asset_folder_id`。 ### `editor_asset_group_source_provenance` diff --git a/docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md b/docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md index 3c41aae27..fbac43ea3 100644 --- a/docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md +++ b/docs/【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md @@ -2,7 +2,7 @@ ## 背景 -外部调用方需要通过稳定 HTTP 契约使用图片画布编辑器内的素材生成、编辑和管理能力，并能创建项目、保存画板布局和管理账号级素材库。该能力必须走 `server-rs + Axum + SpacetimeDB` 正式链路，不能把 API Key、画板状态、素材状态或生成结果放到前端临时状态中。 +外部调用方需要通过稳定 HTTP 契约或托管式远程 MCP 使用图片画布编辑器内的素材生成、编辑和管理能力，并能创建项目、保存画板布局和管理账号级素材库。不支持 MCP 的 Agent 还需要可发现、可校验、可完整下载的 Skill 包，而不是只有一份 OpenAPI JSON。全部入口必须走 `server-rs + Axum + SpacetimeDB` 正式链路，不能把 API Key、画板状态、素材状态、生成任务或生成结果放到前端临时状态中。 ## v1 范围 @@ -32,24 +32,86 @@ v1 只开放以下能力： - `POST /api/external/v1/editor/assets`：创建素材记录。 - `PATCH /api/external/v1/editor/assets/{assetId}`：更新素材名称或所在文件夹。 - `DELETE /api/external/v1/editor/assets/{assetId}`：删除素材记录。 -- `POST /api/external/v1/editor/images/generations`：调用编辑器图片素材生成能力；通过 `kind` 支持普通图、规范图 `spec`、角色图 `character`、快速编辑参考图 `quick-edit`、UI 设计图 `ui-design` 和宣发素材 `publication-material`。可选传入 `projectId` 和 `assetFolderId`，生成后按站内编辑器规则写入 `editor_project_resource` 和账号级 `editor_asset`。 -- `POST /api/external/v1/editor/images/edits`：重绘 / 调整已有图片，结果可写入项目资源和素材库。 -- `POST /api/external/v1/editor/icon-spritesheets/generations`：按规范图生成图标 spritesheet，并拆分为独立图标素材。 -- `POST /api/external/v1/editor/ui-designs/assets/extractions`：从 UI 设计图中提取 / 拆分素材。 -- `POST /api/external/v1/editor/character-animations/generations`：基于角色图片生成角色动画预览和帧序列。 -- `POST /api/external/v1/editor/videos/generations`：生成编辑器视频素材，支持现有 Seedance / Kling / Veo 模型参数和参考媒体限制。 -- `POST /api/external/v1/editor/audios/sound-effects/generations`：生成编辑器音效素材。 -- `POST /api/external/v1/editor/audios/background-music/generations`：生成编辑器背景音乐素材。 +- `POST /api/external/v1/editor/images/generations`：异步提交编辑器图片素材生成；通过 `kind` 支持普通图、规范图 `spec`、角色图 `character`、快速编辑参考图 `quick-edit`、UI 设计图 `ui-design` 和宣发素材 `publication-material`。 +- `POST /api/external/v1/editor/images/edits`：异步提交已有图片重绘 / 调整。 +- `POST /api/external/v1/editor/icon-spritesheets/generations`：异步提交规范图驱动的图标 spritesheet 生成和拆分。 +- `POST /api/external/v1/editor/ui-designs/assets/extractions`：异步提交 UI 设计图素材提取 / 拆分。 +- `POST /api/external/v1/editor/character-animations/generations`：异步提交角色动画预览和帧序列生成。 +- `POST /api/external/v1/editor/videos/generations`：异步提交编辑器视频生成，支持现有 Seedance / Kling / Veo 模型参数和参考媒体限制。 +- `POST /api/external/v1/editor/audios/sound-effects/generations`：异步提交编辑器音效生成。 +- `POST /api/external/v1/editor/audios/background-music/generations`：异步提交编辑器背景音乐生成。 +- `GET /api/external/v1/generations/{operationId}`：按 API Key owner 查询异步生成状态；`completed` 时返回 compact 稳定结果引用，跨 owner 按不存在处理。 - `GET /api/external/v1/openapi.json`：导出本版本 OpenAPI 3.1 JSON。 +- `GET /api/external/v1/agent-integration.json`：公开导出 Agent 集成发现 manifest，声明 MCP、OpenAPI、Skill 入口、完整 Skill archive、archive SHA-256 和包内文件清单。 +- `GET /api/external/v1/skill/SKILL.md`：公开读取 Skill 原始入口。 +- `GET /api/external/v1/skill.zip`：公开下载完整 Skill 包。 +- `POST /api/external/v1/mcp`：使用相同 Bearer API Key 的托管式 Streamable HTTP MCP；对外暴露本节 OpenAPI operation tools 以及使用说明、OpenAPI、Skill 入口 `SKILL.md` 和逐个 Skill reference 文档，不开放内部 SpacetimeDB MCP。 -图片生成、图标 spritesheet 和 UI 素材提取的 2xx 成功响应可携带可选结构化 `warning { code, reason }`。外部 OpenAPI 当前公开四个稳定 `code`： +八类生成 POST 全部要求 `Idempotency-Key`，成功只返回 HTTP `202 Accepted`、`operationId`、`kind`、`status`、`statusUrl`、`pollAfterMs` 和 `updatedAtMicros`。调用方不得把 `202` 当作媒体生成完成，也不得在网络结果不确定时换一个幂等键重新提交。 + +图片生成、图标 spritesheet 和 UI 素材提取的 completed compact `result` 可携带可选结构化 `warning { code, reason }`；任务查询顶层 `warning` 是可直接展示的有界摘要。外部 OpenAPI 当前公开四个稳定 `code`： - `postprocess-failed-source-preserved`：生成成功，但透明处理、像素规整等后处理未完成，接口保留仍可使用的原图或进入该步骤前的结果。 - `dimension-restore-fallback`：provider 回图无法安全收口到目标交付尺寸，接口保留实际回图尺寸。 - `unsupported-image-style`：请求的图片后处理风格未知或不适用于当前生成类型，接口按无风格继续生成。 - `multiple-generation-warnings`：同一成功响应合并了不同 `code` 的多条非阻断告警，具体原因按顺序拼接在 `reason`。 -provider 原图已保存但透明背景处理最终失败时，接口返回原图，不返回不存在的透明处理图，图标和 UI 也不继续拆分；有 `projectId + canvasCompletion` 时由原图完成画布写回，无画布上下文时只返回原图及实际存在的资源 / 素材快照。调用方应展示 `warning.reason`，但不得把任务改判为失败。该降级只覆盖透明背景处理的最终失败，phase 上报、原图或透明处理图持久化、画布写回失败仍返回错误。图标 / UI 已成功生成透明图、只有自动拆分失败时继续使用既有 `sliceWarning`。通用 `warning` 与 `sliceWarning` 只在「透明背景最终失败」这一条上互斥（该情况不会进入拆分）；风格归一化或像素规整产生的通用 `warning` 可以与 `sliceWarning` 并存，调用方必须同时展示两者，不得只取其一。 +provider 原图已保存但透明背景处理最终失败时，worker 保留原图稳定引用，不返回不存在的透明处理图，图标和 UI 也不继续拆分；有 `projectId + canvasCompletion` 时由原图完成画布写回。调用方应展示告警，但不得把 completed 任务改判为失败。该降级只覆盖透明背景处理的最终失败，phase 上报、原图或透明处理图持久化、画布写回失败仍使任务失败。图标 / UI 已成功生成透明图、只有自动拆分失败时继续使用既有 `sliceWarning`。通用 `warning` 与 `sliceWarning` 只在「透明背景最终失败」这一条上互斥；风格归一化或像素规整产生的通用 `warning` 可以与 `sliceWarning` 并存，compact result 不得丢弃任一条。 + +## 异步提交、查询与幂等 + +外部生成不受 `GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 影响：无论站内本地排障模式如何配置，External v1 都只持久化入队并返回 `202`，不在 API 请求中同步执行 provider。正式状态源是既有 `external_generation_job`；生成核心、计费、OSS、画布写回、lease 续租和 fencing 继续由现役 worker 链路负责。 + +调用规则： + +1. 调用方为一次逻辑生成分配 `1-128` 字节、无空格的可打印 ASCII `Idempotency-Key`。 +2. 服务端以 owner、job kind、幂等键和规范请求建立稳定去重身份；同一请求的传输重试必须复用原键。 +3. `202` 响应通过 `Location` / `statusUrl` 指向 `/api/external/v1/generations/{operationId}`，并提供 `Retry-After` / `pollAfterMs`。 +4. `queued/running` 返回 phase、进度与下一次建议轮询间隔；`completed` 返回 `result`；`failed` 返回脱敏 `error`。调用方自己的轮询超时不改变任务状态。 +5. `result` 只保留稳定 `objectKey`、`resourceId`、`assetId`、`assetObjectId`、尺寸、媒体类型、taskId、warning 等轻量引用；禁止持久化完整 project/canvas、大型布局快照、Data URL、Blob URL、过期 signed URL、worker lease/fencing 字段和内部 provider 诊断。 +6. 需要完整项目或素材库状态时，调用方在 completed 后重新读取项目或素材库；需要下载媒体时，用稳定 `objectKey` 调 `/assets/read-url` 获取短期签名 URL。 + +结果查询使用 API Key owner 过滤。任务不存在、已删除或属于其他 owner 时统一返回 `404`，不能通过差异错误枚举他人 operationId。 + +## 托管远程 MCP + +`/api/external/v1/mcp` 是 Genarrative 托管的远程端点，Agent 只需配置 URL 和现有 API Key，不安装本地 MCP server。首版兼容 MCP `2025-11-25` initialize 生命周期，使用 JSON-RPC 2.0 和 Streamable HTTP，支持 `initialize`、`notifications/initialized`、`ping`、`tools/list`、`tools/call`、`resources/list`、`resources/read`。服务端使用无协议 session 的 JSON direct 模式，不依赖 sticky session，也不把 `Mcp-Session-Id` 作为业务身份。 + +MCP tools 从同一份 OpenAPI operation 自动形成 snake_case 名称，并在进程内复用 External REST router，因此鉴权、scope、owner、入参、幂等、计费和结果查询契约只有一份。生成 tools 把 `idempotencyKey` 显式放进参数，因为 MCP transport 的 Authorization 头不能代替逐次业务幂等键。工具结果使用 `structuredContent`；业务失败使用 `isError=true` 的结构化安全错误，协议不可路由时才返回 JSON-RPC error。 + +MCP 暴露下列稳定文本资源： + +- `genarrative://external-editor/usage`：关键工作流和异步轮询规则。 +- `genarrative://external-editor/openapi`：完整 External v1 OpenAPI。 +- `genarrative://external-editor/skill`：Skill 入口原文；保留首版已声明的稳定 URI。 +- `genarrative://external-editor/skill/references/capability-routing.md`：能力选路与场景边界。 +- `genarrative://external-editor/skill/references/api-operations.md`：公开 API 操作、必填字段与调用顺序。 +- `genarrative://external-editor/skill/references/authentication-and-safety.md`：API Key 鉴权、幂等与安全边界。 +- `genarrative://external-editor/skill/references/requests-and-outputs.md`：异步提交、状态轮询与 compact 结果语义。 + +Skill 日后新增 `references/` 文档时，MCP 必须按包内相对路径逐个增加 `genarrative://external-editor/skill/references/` resource，不得只暴露 `SKILL.md` 而让 Agent 无法读取其引用。当前稳定 reference 精确为上述四篇，不得声明不存在的 reference。MCP Agent 直接调用托管 tools，不下载或安装 Python CLI；`scripts/`、`tests/` 和 `.github/workflows/` 不作为 MCP resources。 + +MCP 必须始终复用 `require_external_api_key`，owner 从 `ExternalApiPrincipal` 获取，不接受请求参数伪造 owner。禁止透传内部 `external_generation_job` procedure、worker controller、SpacetimeDB MCP 或 lease/fencing 控制面。 + +MCP 缺少、格式错误或无法验证 Bearer API Key 时仍返回 HTTP `401`，但不能只返回通用“未授权访问”。响应必须附带 `WWW-Authenticate: Bearer realm="genarrative-external-editor"`，并在安全 JSON `details.guide` 中给出稳定 `reason=MCP_AUTHENTICATION_REQUIRED`、`action=CONFIGURE_BEARER_API_KEY`、`Authorization: Bearer ` 格式、登录后前往「开发者 API Key」创建密钥、原始密钥只显示一次、不得粘贴到聊天或写入仓库、配置后重试 `initialize` 的结构化信息，以及公开 manifest、Skill 入口和 OpenAPI 地址。三种失败使用同一响应，不得通过文案或结构差异枚举 Key 是否存在；引导不得匿名暴露 tools、resources 或 owner 信息。`agent-integration.json` 的 `mcp.credentialSetup` 同步提供 action、Header 值格式、导航标签和公开 Skill 引导地址，不编造未纳入公开契约的账户页面 URL。 + +## Agent 集成发现与完整 Skill 包 + +`agent-integration.json` 是机器可读的统一发现入口。支持远程 MCP 的 Agent 读取其中 `mcp.transport/url/authentication`，通过 MCP resources 读取 Skill 入口和所需 references，直接调用 MCP tools，不安装 CLI。仅不支持 MCP，或需要在 Agent 所在机器上编排本地文件上传的调用方下载 `skill.archive`，核对 `archiveSha256`，解压后从 `genarrative-external-editor-api/SKILL.md` 进入。 + +Skill archive 必须至少包含： + +- `SKILL.md` +- `references/capability-routing.md` +- `references/api-operations.md` +- `references/authentication-and-safety.md` +- `references/requests-and-outputs.md` +- `scripts/genarrative_external_api.py` +- `agents/openai.yaml` + +包由 api-server 直接从仓库同源文件构建，不能只返回光秃秃的 OpenAPI JSON，也不能把个人 API Key、环境配置或本机路径写入包。完整 `skill.zip` 只服务不支持 MCP 或需要本地文件编排的 Agent，不是 MCP resource catalog 的压缩包镜像。Python helper 对上层保持便利的同步函数外观，但内部必须执行“异步提交 → 保存 operationId → 按 pollAfterMs 查询 → completed 返回 result”，查询超时应保留 operationId 供后续继续，不得换键重提。 + +api-server 使用 `include_str!` 嵌入 OpenAPI 与 Skill 源文件；容器构建阶段必须同时复制 `docs/openapi/` 和 `.codex/skills/genarrative-external-editor-api/`，不能只复制 `server-rs/`，否则本地 Cargo 验证虽可通过，隔离镜像构建会在编译期找不到同源资源。 管理 API Key 的登录态接口保留在站内个人中心链路，但不写入外部 OpenAPI JSON： @@ -78,6 +140,8 @@ DELETE /api/profile/api-keys/{keyId} 这是 breaking change，不是文档同步：严格反序列化的调用方（OpenAPI Generator 生成的 Java / Kotlin / C#、pydantic、serde 非 `Option` 字段）在 `required` 字段缺失时直接失败，且失败发生在服务端上线瞬间，不需要调用方做任何动作。脱敏目标本身成立，接受不升版本、不设弃用期的唯一依据是当前无存量调用方。 +2026-07-31 同一豁免还覆盖了「八类生成从同步成功响应切换为 `202 + operationId`，新增统一查询接口」这一 breaking change。旧调用方若仍把生成 POST 响应当作媒体结果会立即失败；接受原地修改 v1 的唯一依据同样是上线前已确认没有外部第三方存量调用方。托管 MCP、集成 manifest 与 Skill archive 均为新增入口，不产生既有客户端兼容债务。 + ### 豁免的失效条件 API Key 由用户在个人中心自助发放，因此「无外部调用方」不是受控状态，可能在无人决策的情况下变为假。本节豁免在下列任一条件出现后立即失效： @@ -113,7 +177,7 @@ Authorization: Bearer tnr_sk_xxx - 明文 Key 只在创建接口返回一次，后端只保存 `key_hash` 与 `key_prefix`。 - API Key 被撤销后立即不可再用于外部接口。 - 外部 API 鉴权不复用登录态 JWT，不检查 refresh session；它是独立开发者凭据。 -- OpenAPI JSON 公共可读，不需要鉴权。 +- OpenAPI JSON、Agent 集成 manifest、原始 Skill 入口和完整 Skill archive 公共可读，不需要鉴权；MCP 与全部业务操作需要鉴权。 ## 数据模型 @@ -142,19 +206,19 @@ SpacetimeDB procedure： ## 素材生成与落库 -外部生成接口复用站内编辑器已有 handler 和 DTO，不维护第二套生成语义： +外部生成接口复用站内编辑器已有 DTO、入队器和 worker executor，不维护第二套生成语义： -- 图片生成 / 重绘 / 规范图 / 宣发图 / UI 设计图复用 `/api/editor/images/generations` 与 `/api/editor/images/edits` 的校验、模型归一、计费和持久化规则。 +- 图片生成 / 重绘 / 规范图 / 宣发图 / UI 设计图复用 `/api/editor/images/generations` 与 `/api/editor/images/edits` 的校验、模型归一、计费和持久化规则，但 External handler 固定只入队。 - 图标 spritesheet 和 UI 设计图素材提取复用站内拆分逻辑，生成图集后按连通域切片，并把图集与切片都按请求写入项目资源和素材库。 - 角色动画、视频、音效和背景音乐复用站内编辑器生成链路；请求携带 `assetFolderId` 时按站内规则写入素材库，音频类外部调用使用 API Key 所属账号作为 asset owner。 - API Key 管理接口仍只属于登录态个人中心，不进入外部 OpenAPI JSON。 -素材外部生成成功后，后端拿到素材后： +素材外部生成由 worker 成功后： 1. 通过 OSS / asset object adapter 持久化媒体文件。 2. 写入 `editor_asset`，让生成素材进入账号级素材库。 3. 如果请求带 `projectId`，写入 `editor_project_resource`。 -4. 返回图片读取地址、素材 ID、资源 ID、尺寸、prompt、model 和 taskId；普通 External API 响应、项目资源与素材 read model 不返回生成 provider 或内部抠图审计字段。同源画布前端自动提交默认 `segModel` 属于站内 BFF 请求契约，不因此向 External OpenAPI 开放该字段。 +4. 在 `result_payload_json` 保存 compact 稳定引用，由查询接口返回素材 ID、资源 ID、objectKey、尺寸、媒体类型、model、taskId 和 warning；普通 External API 响应、项目资源与素材 read model 不返回生成 provider、原始 prompt 或内部抠图审计字段。同源画布前端自动提交默认 `segModel` 属于站内 BFF 请求契约，不因此向 External OpenAPI 开放该字段。 如果请求未带 `projectId`，只生成并写入素材库；调用方可随后创建项目或自行保存画板布局。 @@ -177,17 +241,21 @@ OpenAPI 3.1 JSON 固定落在： docs/openapi/genarrative-external-v1.openapi.json ``` -服务端 `GET /api/external/v1/openapi.json` 使用同一份 JSON，通过 `include_str!` 导出，避免运行时生成结果与仓库文档漂移。 +服务端 `GET /api/external/v1/openapi.json` 和 MCP OpenAPI resource 使用同一份 JSON，通过 `include_str!` 导出，避免运行时生成结果与仓库文档漂移。MCP tool catalog 同样以这份 OpenAPI 的 path、method、operationId、参数和 request body 是否存在为来源；字段级精确约束继续以 OpenAPI resource 为准。 ## 验收 - API Key 创建只返回一次明文，列表不返回明文。 - 撤销后的 API Key 调用外部接口返回 `401`。 -- 外部图片生成、重绘、图标拆分、UI 素材拆分、视频、音效和音乐生成成功后，生成结果按请求同时出现在画布资源和账号级素材库。 -- 角色图、图标 spritesheet 和 UI 素材提取的 2xx 成功响应允许携带 `EditorGenerationWarning`；provider 原图保留降级与自动拆分降级必须保持成功状态，并分别使用通用 `warning` 与兼容 `sliceWarning` 表达。 +- 八类外部生成 POST 缺少或携带非法 `Idempotency-Key` 时返回 `400`；同一 owner、请求和 key 重试只得到同一 operation。 +- 八类外部生成 POST 固定返回 `202`，查询能从 `queued/running` 收敛到 `completed/failed`；调用方超时后使用原 operationId 继续查询。 +- 外部图片生成、重绘、图标拆分、UI 素材拆分、角色动画、视频、音效和音乐 completed 后，生成结果按请求同时出现在画布资源和账号级素材库。 +- 角色图、图标 spritesheet 和 UI 素材提取的 completed result 允许携带 `EditorGenerationWarning`；provider 原图保留降级与自动拆分降级必须保持成功状态，并分别使用通用 `warning` 与兼容 `sliceWarning` 表达。 - 外部视频、角色动画、音效和音乐接口使用站内编辑器相同的请求校验、模型限制和价格校验。 - OpenAPI JSON 能被 `serde_json` 解析，且 security scheme 为 Bearer API Key。 - OpenAPI JSON 不包含 `/api/profile/api-keys`、`UserAccessToken` 或 API Key 管理 schema。 +- `agent-integration.json` 能发现 MCP、OpenAPI、Skill entry/archive；下载 archive 的 SHA-256 与 manifest 一致，ZIP 包含 `SKILL.md`、四篇 references、Python helper 和 `agents/openai.yaml` 七个声明文件且不含凭据。 +- MCP 在无 Bearer、Bearer 格式错误或 Key 无效时返回相同的 `401 + WWW-Authenticate + details.guide` 鉴权引导，且不暴露 tools/resources/owner；合法 Key 可完成 initialize、tools/list、resources/list/read 和生成提交/查询；resource catalog 必须包含 usage、OpenAPI、`skill` 主入口和当前全部 Skill references，当前精确为 `skill/references/capability-routing.md`、`skill/references/api-operations.md`、`skill/references/authentication-and-safety.md` 与 `skill/references/requests-and-outputs.md`，且不包含 CLI 脚本、测试或 workflow；多实例不依赖 sticky session，不暴露内部 SpacetimeDB MCP 或 worker 控制面。 - 外部素材库接口覆盖当前已有素材操作：直传凭证、素材对象确认、签名读取、读取素材库、创建 / 更新 / 删除文件夹、创建 / 更新 / 删除素材、创建项目画布资源。 - 外部项目接口覆盖当前已有项目管理操作：项目列表、最近项目、创建、读取、重命名、删除和默认画布保存。 - 外部素材生成接口覆盖当前已有编辑器素材操作：生图、重绘 / 调整、规范图生成、宣发素材生成、图标素材生成与拆分、UI 设计图生成与拆分、角色动画、视频、音效和背景音乐。 diff --git a/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md b/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md index 053e616b7..0a3214591 100644 --- a/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md +++ b/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md @@ -1,6 +1,6 @@ # 创作主页与项目入口改版计划 -> 2026-07-18 退役覆盖：本文关于旧模板入口、`/creation/`、移动端隐藏“创作 / 项目”和 `/api/creation-entry/config` 的内容均已被后续实现替代，只保留为阶段设计记录。现役口径是桌面侧边栏与移动端底部 dock 都显示“创作 / 项目 / 我的”，稳定路由为 `/creation`、`/project`、`/profile`；旧模板业务只保留历史数据壳。当前实现与验收以 `docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md` 为准。 +> 2026-08-03 纠正：2026-07-18 的旧模板退役只替代本文关于旧模板入口、`/creation/` 和 `/api/creation-entry/config` 的内容，不替代“移动端隐藏创作 / 项目并阻止进入画布”的既有边界。现役稳定路由仍为 `/creation`、`/project`、`/profile`，但创作主页、项目管理和图片画布只在桌面端挂载；移动端底部 dock 只保留“我的”，触发创作工具时显示桌面端提示。当前实现与验收以 `docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md` 为准。 日期：2026-06-18 diff --git a/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md b/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md index 013d80523..875a7a0ea 100644 --- a/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md +++ b/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md @@ -12,7 +12,7 @@ - `/project` 展示当前账号的图片编辑器项目，项目卡继续进入 `/editor/canvas`。 - `/profile` 是“我的”稳定路由，保留头像与昵称编辑、陶泥号复制、泥点余额与账单、累计统计、泥点充值、兑换码、玩家社区、反馈与建议、通用设置、开发者 API Key 和法律信息等平台公共能力。 - 桌面顶栏保留现役项目 / 素材搜索、泥点入口和账号胶囊。搜索只筛选当前编辑器项目与已读取的公开编辑器素材，不恢复旧公开作品号搜索、旧广场、旧作品详情或旧运行态。 -- 桌面端使用公共侧边栏，移动端使用同样包含“创作 / 项目 / 我的”的三项底部 dock；点击、刷新及浏览器前进 / 后退都必须保持 URL、标题和选中态一致。 +- 桌面端公共侧边栏固定显示“创作 / 项目 / 我的”；移动端底部 dock 只保留“我的”，不暴露“创作 / 项目”。移动端直达 `/creation`、`/project` 或 `/editor/canvas` 时显示桌面端创作提示，不挂载创作主页、项目列表或图片画布；从移动端首页触发项目或画布动作时也只显示同一提示。 现役入口和公共资料能力只能依赖 `creation-home`、`project`、`image-editor`、公共组件及 `services/platform-entry` 等现役模块。Vite 模块门禁会拒绝 `components/rpg-entry`、`services/rpg-entry`、旧玩法目录和旧平台业务模块进入依赖图；Tailwind `@source`、TypeScript `include`、ESLint ignore 或 Vite watch ignore 都不能替代这条运行时依赖门禁。 diff --git a/server-rs/Cargo.lock b/server-rs/Cargo.lock index 7c01f7be6..d3424f54c 100644 --- a/server-rs/Cargo.lock +++ b/server-rs/Cargo.lock @@ -231,6 +231,7 @@ dependencies = [ "platform-wechat", "reqwest", "ring", + "rmcp", "serde", "serde_json", "sha1", @@ -827,6 +828,17 @@ dependencies = [ "libc", ] +[[package]] +name = "chacha20" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d524456ba66e72eb8b115ff89e01e497f8e6d11d78b70b1aa13c0fbd97540a81" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.0", + "rand_core 0.10.1", +] + [[package]] name = "chrono" version = "0.4.45" @@ -1932,6 +1944,7 @@ dependencies = [ "cfg-if", "libc", "r-efi 6.0.0", + "rand_core 0.10.1", "wasip2", "wasip3", ] @@ -3679,6 +3692,12 @@ version = "1.0.15" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a" +[[package]] +name = "pastey" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4" + [[package]] name = "pem" version = "3.0.6" @@ -4516,6 +4535,17 @@ dependencies = [ "rand_core 0.9.5", ] +[[package]] +name = "rand" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80" +dependencies = [ + "chacha20", + "getrandom 0.4.2", + "rand_core 0.10.1", +] + [[package]] name = "rand_chacha" version = "0.3.1" @@ -4554,6 +4584,12 @@ dependencies = [ "getrandom 0.3.4", ] +[[package]] +name = "rand_core" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" + [[package]] name = "raw-window-handle" version = "0.6.2" @@ -4720,6 +4756,35 @@ dependencies = [ "windows-sys 0.52.0", ] +[[package]] +name = "rmcp" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "14db48ee17a9ba61810ab1a9c1beb7d06d8136ae39ac25a1137f10d357af01af" +dependencies = [ + "async-trait", + "bytes", + "chrono", + "futures", + "http", + "http-body", + "http-body-util", + "pastey", + "pin-project-lite", + "rand 0.10.2", + "schemars 1.2.1", + "serde", + "serde_json", + "sse-stream", + "thiserror 2.0.18", + "tokio", + "tokio-stream", + "tokio-util", + "tower-service", + "tracing", + "uuid", +] + [[package]] name = "rmp" version = "0.8.15" @@ -4931,12 +4996,26 @@ version = "1.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a2b42f36aa1cd011945615b92222f6bf73c599a102a300334cd7f8dbeec726cc" dependencies = [ + "chrono", "dyn-clone", "ref-cast", + "schemars_derive", "serde", "serde_json", ] +[[package]] +name = "schemars_derive" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7d115b50f4aaeea07e79c1912f645c7513d81715d0420f8bc77a18c6260b307f" +dependencies = [ + "proc-macro2", + "quote", + "serde_derive_internals", + "syn 2.0.118", +] + [[package]] name = "scoped-tls" version = "1.0.1" @@ -5027,6 +5106,17 @@ dependencies = [ "syn 2.0.118", ] +[[package]] +name = "serde_derive_internals" +version = "0.29.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "18d26a20a969b9e3fdf2fc2d9f21eda6c40e2de84c9408bb5d3b05d499aae711" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "serde_json" version = "1.0.150" @@ -5670,6 +5760,19 @@ dependencies = [ "log", ] +[[package]] +name = "sse-stream" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c123f296ade4ec4b8b0f6162116e6629f5146922ca5ab40ca9d3c2e73ab4761e" +dependencies = [ + "bytes", + "futures-util", + "http-body", + "http-body-util", + "pin-project-lite", +] + [[package]] name = "stable_deref_trait" version = "1.2.1" diff --git a/server-rs/Cargo.toml b/server-rs/Cargo.toml index 44de73be6..dbf37e43e 100644 --- a/server-rs/Cargo.toml +++ b/server-rs/Cargo.toml @@ -112,6 +112,7 @@ pingora-http = { version = "0.8.1", default-features = false } pingora-proxy = { version = "0.8.1", default-features = false } rand_core = "0.6" reqwest = { version = "0.12", default-features = false } +rmcp = { version = "=2.2.0", default-features = false } ring = "0.17" serde = { version = "1", features = ["derive"] } serde_json = "1" diff --git a/server-rs/crates/api-server/Cargo.toml b/server-rs/crates/api-server/Cargo.toml index 27b3bc76c..ff0357094 100644 --- a/server-rs/crates/api-server/Cargo.toml +++ b/server-rs/crates/api-server/Cargo.toml @@ -16,6 +16,7 @@ hex = { workspace = true } image = { workspace = true, features = ["jpeg", "png", "webp"] } http-body-util = { workspace = true } reqwest = { workspace = true, features = ["json", "multipart", "rustls-tls"] } +rmcp = { workspace = true, features = ["server", "transport-streamable-http-server"] } webp = { workspace = true } module-ai = { workspace = true } module-assets = { workspace = true, features = ["server-service"] } @@ -48,6 +49,7 @@ tokio-stream = { workspace = true } futures-util = { workspace = true } time = { workspace = true, features = ["formatting"] } tower-http = { workspace = true, features = ["trace"] } +tower = { workspace = true, features = ["util"] } tracing = { workspace = true } opentelemetry = { workspace = true } url = { workspace = true } @@ -62,4 +64,3 @@ windows-sys = { workspace = true, features = ["Win32_Foundation", "Win32_System_ base64 = { workspace = true } http-body-util = { workspace = true } reqwest = { workspace = true, features = ["json", "multipart", "rustls-tls"] } -tower = { workspace = true, features = ["util"] } diff --git a/server-rs/crates/api-server/src/character_animation_assets.rs b/server-rs/crates/api-server/src/character_animation_assets.rs index b44a66496..5a47c01a9 100644 --- a/server-rs/crates/api-server/src/character_animation_assets.rs +++ b/server-rs/crates/api-server/src/character_animation_assets.rs @@ -48,7 +48,7 @@ use shared_contracts::assets::{ use shared_contracts::assets::{ CharacterRoleAssetWorkflowResolveRequest, CharacterRoleAssetWorkflowResponse, }; -use spacetime_client::SpacetimeClientError; +use spacetime_client::{ExternalGenerationJobRecord, SpacetimeClientError}; use crate::{ api_response::json_success_body, @@ -61,7 +61,7 @@ use crate::{ editor_generation_queue::{ EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND, EditorGenerationQueuedResponse, editor_generation_queue_state, - editor_generation_source_entity_id, enqueue_editor_generation_job, + editor_generation_source_entity_id, enqueue_editor_generation_job_for_caller, }, editor_green_screen::{ EditorScreenBackgroundColor, editor_green_screen_character_prompt_clause, @@ -578,38 +578,14 @@ pub async fn generate_editor_character_animation( )); } if !state.config.external_generation_mode.is_inline() { - let pricing = state.editor_generation_pricing().await.map_err(|error| { - character_animation_error_response( - &request_context, - AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ - "provider": "editor-generation-pricing", - "message": error.to_string(), - })), - ) - })?; - // 队列路径只取定价，背景色决策留到实际执行时再做，这里用默认色占位。 - let normalized = normalize_editor_character_animation_request_with_pricing( - payload.clone(), - &pricing, - crate::editor_green_screen::default_editor_screen_background_color(), - ) - .map_err(|error| character_animation_error_response(&request_context, error))?; - let source_entity_id = editor_generation_source_entity_id( - payload.project_id.as_deref(), - payload.source_layer_id.as_str(), - ); - let queue_job = enqueue_editor_generation_job( + let queue_job = enqueue_editor_character_animation_for_owner( &state, &request_context, owner_user_id.as_str(), - EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, - source_entity_id, - "图片画布生成角色动作", - u64::from(normalized.price_mud_points), - &payload, + payload, + None, ) - .await - .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + .await?; return Ok(json_success_body( Some(&request_context), EditorGenerationQueuedResponse { @@ -627,6 +603,56 @@ pub async fn generate_editor_character_animation( .await } +pub(crate) async fn enqueue_editor_character_animation_for_owner( + state: &AppState, + request_context: &RequestContext, + owner_user_id: &str, + payload: EditorCharacterAnimationGenerateRequest, + external_idempotency_key: Option<&str>, +) -> Result { + if matches_inline_media_source(payload.source_image_src.as_str()) { + return Err(character_animation_error_response( + request_context, + editor_character_animation_bad_request( + "sourceImageSrc 必须先上传 OSS，并使用 objectKey 或画板资源引用。", + ), + )); + } + let pricing = state.editor_generation_pricing().await.map_err(|error| { + character_animation_error_response( + request_context, + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "editor-generation-pricing", + "message": error.to_string(), + })), + ) + })?; + // 队列路径只取定价，背景色决策留到实际执行时再做，这里用默认色占位。 + let normalized = normalize_editor_character_animation_request_with_pricing( + payload.clone(), + &pricing, + crate::editor_green_screen::default_editor_screen_background_color(), + ) + .map_err(|error| character_animation_error_response(request_context, error))?; + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + payload.source_layer_id.as_str(), + ); + enqueue_editor_generation_job_for_caller( + state, + request_context, + owner_user_id, + EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成角色动作", + u64::from(normalized.price_mud_points), + &payload, + external_idempotency_key, + ) + .await + .map_err(|error| error.into_response_with_context(Some(request_context))) +} + pub(crate) async fn generate_editor_character_animation_for_owner( state: AppState, request_context: RequestContext, @@ -970,31 +996,14 @@ pub async fn generate_editor_video( })?; let owner_user_id = authenticated.claims().user_id().to_string(); if !state.config.external_generation_mode.is_inline() { - let pricing = state.editor_generation_pricing().await.map_err(|error| { - editor_video_error_response( - &request_context, - AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ - "provider": "editor-generation-pricing", - "message": error.to_string(), - })), - ) - })?; - let normalized = normalize_editor_video_request_with_pricing(payload.clone(), &pricing) - .map_err(|error| editor_video_error_response(&request_context, error))?; - let source_entity_id = - editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-video"); - let queue_job = enqueue_editor_generation_job( + let queue_job = enqueue_editor_video_generation_for_owner( &state, &request_context, owner_user_id.as_str(), - EDITOR_VIDEO_GENERATION_JOB_KIND, - source_entity_id, - "图片画布生成视频", - u64::from(normalized.price_mud_points), - &payload, + payload, + None, ) - .await - .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + .await?; return Ok(json_success_body( Some(&request_context), EditorGenerationQueuedResponse { @@ -1005,6 +1014,41 @@ pub async fn generate_editor_video( generate_editor_video_for_owner(state, request_context, owner_user_id, Ok(Json(payload))).await } +pub(crate) async fn enqueue_editor_video_generation_for_owner( + state: &AppState, + request_context: &RequestContext, + owner_user_id: &str, + payload: EditorVideoGenerateRequest, + external_idempotency_key: Option<&str>, +) -> Result { + let pricing = state.editor_generation_pricing().await.map_err(|error| { + editor_video_error_response( + request_context, + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "editor-generation-pricing", + "message": error.to_string(), + })), + ) + })?; + let normalized = normalize_editor_video_request_with_pricing(payload.clone(), &pricing) + .map_err(|error| editor_video_error_response(request_context, error))?; + let source_entity_id = + editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-video"); + enqueue_editor_generation_job_for_caller( + state, + request_context, + owner_user_id, + EDITOR_VIDEO_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成视频", + u64::from(normalized.price_mud_points), + &payload, + external_idempotency_key, + ) + .await + .map_err(|error| error.into_response_with_context(Some(request_context))) +} + pub(crate) async fn generate_editor_video_for_owner( state: AppState, request_context: RequestContext, diff --git a/server-rs/crates/api-server/src/editor_generation_queue.rs b/server-rs/crates/api-server/src/editor_generation_queue.rs index 640aa6dc9..248adf54a 100644 --- a/server-rs/crates/api-server/src/editor_generation_queue.rs +++ b/server-rs/crates/api-server/src/editor_generation_queue.rs @@ -1,6 +1,7 @@ use axum::http::StatusCode; use serde::Serialize; use serde_json::{Value, json}; +use sha2::{Digest, Sha256}; use shared_contracts::external_generation::{ ExternalGenerationJobStatus, ExternalGenerationJobStatusRecord, }; @@ -26,6 +27,7 @@ pub(crate) const EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND: &str = pub(crate) const EDITOR_GENERATION_QUEUE_SOURCE_MODULE: &str = "editor-canvas"; const EDITOR_GENERATION_QUEUE_PROVIDER: &str = "editor-generation-worker"; const MAX_EDITOR_GENERATION_JOB_PAYLOAD_BYTES: usize = 512 * 1024; +const EXTERNAL_API_GENERATION_DEDUPE_PREFIX: &str = "external-api-generation"; #[derive(Debug, Serialize)] #[serde(rename_all = "camelCase")] @@ -77,6 +79,132 @@ where T: Serialize, { let request_payload_json = serialize_editor_generation_job_payload(payload)?; + enqueue_serialized_editor_generation_job_with_identity( + state, + owner_user_id, + job_kind, + source_entity_id, + request_label, + price_mud_points, + request_payload_json, + job_id, + dedupe_key, + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn enqueue_external_api_editor_generation_job( + state: &AppState, + owner_user_id: &str, + job_kind: &str, + source_entity_id: impl Into, + request_label: impl Into, + price_mud_points: u64, + payload: &T, + idempotency_key: &str, +) -> Result +where + T: Serialize, +{ + let request_payload_json = serialize_editor_generation_job_payload(payload)?; + let mut hasher = Sha256::new(); + hasher.update(owner_user_id.trim().as_bytes()); + hasher.update(b"\0"); + hasher.update(job_kind.trim().as_bytes()); + hasher.update(b"\0"); + hasher.update(idempotency_key.as_bytes()); + let dedupe_key = format!( + "{EXTERNAL_API_GENERATION_DEDUPE_PREFIX}:{job_kind}:{:x}", + hasher.finalize() + ); + let requested_job_id = build_prefixed_uuid_id("task-"); + let job = enqueue_serialized_editor_generation_job_with_identity( + state, + owner_user_id, + job_kind, + source_entity_id, + request_label, + price_mud_points, + request_payload_json.clone(), + requested_job_id.clone(), + dedupe_key, + ) + .await?; + + if job.job_kind != job_kind + || job.owner_user_id != owner_user_id + || job.request_payload_json != request_payload_json + { + return Err( + AppError::from_status(StatusCode::CONFLICT).with_details(json!({ + "provider": EDITOR_GENERATION_QUEUE_PROVIDER, + "message": "Idempotency-Key 已用于不同的生成请求，请复用原请求参数或更换幂等键。", + })), + ); + } + Ok(job) +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn enqueue_editor_generation_job_for_caller( + state: &AppState, + request_context: &RequestContext, + owner_user_id: &str, + job_kind: &str, + source_entity_id: impl Into, + request_label: impl Into, + price_mud_points: u64, + payload: &T, + external_idempotency_key: Option<&str>, +) -> Result +where + T: Serialize, +{ + let source_entity_id = source_entity_id.into(); + let request_label = request_label.into(); + match external_idempotency_key { + Some(idempotency_key) => { + enqueue_external_api_editor_generation_job( + state, + owner_user_id, + job_kind, + source_entity_id, + request_label, + price_mud_points, + payload, + idempotency_key, + ) + .await + } + None => { + enqueue_editor_generation_job( + state, + request_context, + owner_user_id, + job_kind, + source_entity_id, + request_label, + price_mud_points, + payload, + ) + .await + } + } +} + +#[allow(clippy::too_many_arguments)] +async fn enqueue_serialized_editor_generation_job_with_identity( + state: &AppState, + owner_user_id: &str, + job_kind: &str, + source_entity_id: impl Into, + request_label: impl Into, + price_mud_points: u64, + request_payload_json: String, + job_id: String, + dedupe_key: String, +) -> Result { let now_micros = current_utc_micros(); state .spacetime_client() @@ -164,12 +292,21 @@ fn is_inline_media_reference(value: &str) -> bool { pub(crate) fn editor_generation_queue_state( job: ExternalGenerationJobRecord, ) -> ExternalGenerationJobStatusRecord { + let (status, phase_detail, progress) = match job.status.as_str() { + "completed" => (ExternalGenerationJobStatus::Completed, "生成已完成。", 100), + "running" if job.phase.as_deref() == Some("processing") => { + (ExternalGenerationJobStatus::Running, "正在处理。", 70) + } + "running" => (ExternalGenerationJobStatus::Running, "正在生成。", 35), + "failed" | "cancelled" => (ExternalGenerationJobStatus::Failed, "生成失败。", 0), + _ => (ExternalGenerationJobStatus::Queued, "排队中。", 8), + }; ExternalGenerationJobStatusRecord { operation_id: job.job_id, - status: ExternalGenerationJobStatus::Queued, + status, phase_label: job.request_label, - phase_detail: "排队中。".to_string(), - progress: 8, + phase_detail: phase_detail.to_string(), + progress, error: job.last_error_message, updated_at_micros: job.updated_at_micros, } @@ -194,6 +331,38 @@ fn current_utc_micros() -> i64 { mod tests { use super::*; + fn queue_job_fixture(status: &str, phase: Option<&str>) -> ExternalGenerationJobRecord { + ExternalGenerationJobRecord { + job_id: "task-queue-test".to_string(), + dedupe_key: "editor-canvas:test:task-queue-test".to_string(), + job_kind: EDITOR_IMAGE_GENERATION_JOB_KIND.to_string(), + owner_user_id: "user-1".to_string(), + source_module: EDITOR_GENERATION_QUEUE_SOURCE_MODULE.to_string(), + source_entity_id: "project-1".to_string(), + request_label: "图片画布生成图片".to_string(), + request_payload_json: "{}".to_string(), + status: status.to_string(), + attempt: 0, + max_attempts: 1, + last_error_message: None, + worker_id: None, + lease_expires_at: None, + available_at: "2026-07-31T00:00:00Z".to_string(), + result_payload_json: None, + created_at: "2026-07-31T00:00:00Z".to_string(), + started_at: None, + completed_at: None, + updated_at: "2026-07-31T00:00:00Z".to_string(), + updated_at_micros: 1_785_456_000_000_000, + lease_token: None, + price_mud_points: 2, + refund_ledger_id: None, + notification_acknowledged_at: None, + notification_acknowledged_at_micros: None, + phase: phase.map(ToOwned::to_owned), + } + } + #[test] fn serialize_payload_accepts_persistable_media_references() { let payload = json!({ @@ -253,4 +422,72 @@ mod tests { assert_eq!(error.status_code().as_u16(), 413); assert!(error.body_text().contains("超过持久化上限")); } + + #[test] + fn queue_state_maps_idempotent_replays_to_the_persisted_status() { + let cases = [ + ( + "pending", + None, + ExternalGenerationJobStatus::Queued, + "排队中。", + 8, + ), + ( + "running", + Some("generating"), + ExternalGenerationJobStatus::Running, + "正在生成。", + 35, + ), + ( + "running", + Some("processing"), + ExternalGenerationJobStatus::Running, + "正在处理。", + 70, + ), + ( + "completed", + None, + ExternalGenerationJobStatus::Completed, + "生成已完成。", + 100, + ), + ( + "failed", + None, + ExternalGenerationJobStatus::Failed, + "生成失败。", + 0, + ), + ]; + + for (persisted_status, phase, expected_status, expected_detail, expected_progress) in cases + { + let state = editor_generation_queue_state(queue_job_fixture(persisted_status, phase)); + + assert_eq!(state.operation_id, "task-queue-test"); + assert_eq!(state.status, expected_status, "status={persisted_status}"); + assert_eq!( + state.phase_detail, expected_detail, + "status={persisted_status}" + ); + assert_eq!( + state.progress, expected_progress, + "status={persisted_status}" + ); + } + } + + #[test] + fn queue_state_keeps_failed_error_for_idempotent_replay() { + let mut job = queue_job_fixture("failed", None); + job.last_error_message = Some("生成失败摘要".to_string()); + + let state = editor_generation_queue_state(job); + + assert_eq!(state.status, ExternalGenerationJobStatus::Failed); + assert_eq!(state.error.as_deref(), Some("生成失败摘要")); + } } diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index 2e3655989..dcfbeb626 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -54,7 +54,7 @@ use spacetime_client::{ EditorShowcaseAssetRecord, EditorShowcaseAssetSubmitRecordInput, EditorShowcaseCampaignConfigGetRecordInput, EditorShowcaseCampaignConfigRecord, ExternalGenerationJobPhaseUpdateError, ExternalGenerationJobPhaseUpdateRecordInput, - SpacetimeClientError, + ExternalGenerationJobRecord, SpacetimeClientError, }; use crate::{ @@ -66,7 +66,7 @@ use crate::{ EDITOR_IMAGE_EDIT_JOB_KIND, EDITOR_IMAGE_GENERATION_JOB_KIND, EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, EditorGenerationQueuedResponse, editor_generation_queue_state, editor_generation_source_entity_id, - enqueue_editor_generation_job, + enqueue_editor_generation_job, enqueue_editor_generation_job_for_caller, }, editor_green_screen::{ EditorScreenBackgroundColor, editor_green_screen_asset_prompt_clause, @@ -1663,59 +1663,15 @@ pub async fn generate_editor_image( Extension(authenticated): Extension, payload: Result, JsonRejection>, ) -> Result, AppError> { - let Json(mut payload) = parse_editor_generation_json_payload(payload)?; - payload.generation_inputs = - sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); + let Json(payload) = parse_editor_generation_json_payload(payload)?; let caller = EditorGenerationCaller::from_authenticated(&authenticated); if !state.config.external_generation_mode.is_inline() { - ensure_editor_reference_image_sources_are_stable( - payload.reference_image_srcs.as_deref(), - "editor-image-generation", - "referenceImageSrcs", - "生成参考图", - )?; - let normalized_kind = payload.kind.as_deref().map(str::trim); - let is_ui_design_generation = matches!(normalized_kind, Some("ui-design")); - let is_publication_material_generation = - matches!(normalized_kind, Some("publication-material")); - let generation_options = normalize_editor_generation_options( - if is_ui_design_generation || is_publication_material_generation { - Some(GPT_IMAGE_2_MODEL) - } else { - payload.model.as_deref() - }, - payload.aspect_ratio.as_deref(), - payload.image_size.as_deref(), - ); - let price_mud_points = u64::from( - state - .editor_generation_pricing() - .await - .map_err(|error| { - AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ - "provider": "editor-generation-pricing", - "message": error.to_string(), - })) - })? - .image_generation_mud_points( - normalized_kind, - Some(generation_options.model), - Some(generation_options.image_size), - ), - ); - let source_entity_id = editor_generation_source_entity_id( - payload.project_id.as_deref(), - "editor-image-generation", - ); - let queue_job = enqueue_editor_generation_job( + let queue_job = enqueue_editor_image_generation_for_owner( &state, &request_context, - caller.owner_user_id.as_str(), - EDITOR_IMAGE_GENERATION_JOB_KIND, - source_entity_id, - "图片画布生成图片", - price_mud_points, - &payload, + &caller, + payload, + None, ) .await?; return Ok(json_success_body( @@ -1728,6 +1684,68 @@ pub async fn generate_editor_image( generate_editor_image_for_owner(&state, &request_context, caller, payload).await } +pub(crate) async fn enqueue_editor_image_generation_for_owner( + state: &AppState, + request_context: &RequestContext, + caller: &EditorGenerationCaller, + mut payload: EditorImageGenerationRequest, + external_idempotency_key: Option<&str>, +) -> Result { + payload.generation_inputs = + sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); + ensure_editor_reference_image_sources_are_stable( + payload.reference_image_srcs.as_deref(), + "editor-image-generation", + "referenceImageSrcs", + "生成参考图", + )?; + let normalized_kind = payload.kind.as_deref().map(str::trim); + let is_ui_design_generation = matches!(normalized_kind, Some("ui-design")); + let is_publication_material_generation = + matches!(normalized_kind, Some("publication-material")); + let generation_options = normalize_editor_generation_options( + if is_ui_design_generation || is_publication_material_generation { + Some(GPT_IMAGE_2_MODEL) + } else { + payload.model.as_deref() + }, + payload.aspect_ratio.as_deref(), + payload.image_size.as_deref(), + ); + let price_mud_points = u64::from( + state + .editor_generation_pricing() + .await + .map_err(|error| { + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "editor-generation-pricing", + "message": error.to_string(), + })) + })? + .image_generation_mud_points( + normalized_kind, + Some(generation_options.model), + Some(generation_options.image_size), + ), + ); + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-image-generation", + ); + enqueue_editor_generation_job_for_caller( + state, + request_context, + caller.owner_user_id.as_str(), + EDITOR_IMAGE_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成图片", + price_mud_points, + &payload, + external_idempotency_key, + ) + .await +} + pub(crate) async fn generate_editor_image_for_owner( state: &AppState, request_context: &RequestContext, @@ -2040,7 +2058,7 @@ pub(crate) async fn generate_editor_image_for_owner( caller.owner_user_id.as_str(), generated.task_id.as_str(), image, - submitted_prompt.as_str(), + role_setting.as_str(), generated.actual_prompt.as_deref(), storage_profile.asset_kind, "character-images", @@ -2222,7 +2240,7 @@ pub(crate) async fn generate_editor_image_for_owner( ); } image = restored_removal_image; - output_prompt = "去除纯色背景".to_string(); + output_prompt = role_setting.clone(); output_actual_prompt = None; output_provider = removal_provider.to_string(); output_generation_inputs = apply_editor_matting_metadata_to_generation_inputs( @@ -3995,55 +4013,13 @@ pub async fn edit_editor_image( State(state): State, Extension(request_context): Extension, Extension(authenticated): Extension, - Json(mut payload): Json, + Json(payload): Json, ) -> Result, AppError> { - payload.generation_inputs = - sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); let caller = EditorGenerationCaller::from_authenticated(&authenticated); if !state.config.external_generation_mode.is_inline() { - ensure_editor_reference_image_source_is_stable( - payload.source_image_src.as_str(), - "editor-image-edit", - "sourceImageSrc", - "待修改图片", - )?; - ensure_editor_reference_image_sources_are_stable( - payload.reference_image_srcs.as_deref(), - "editor-image-edit", - "referenceImageSrcs", - "修改参考图", - )?; - ensure_editor_image_edit_source_allowed(&state, caller.owner_user_id.as_str(), &payload) - .await?; - let generation_options = normalize_editor_image_edit_generation_options( - payload.model.as_deref(), - payload.aspect_ratio.as_deref(), - payload.image_size.as_deref(), - payload.size.as_deref(), - ); - let image_size = normalize_editor_image_generation_size(payload.size.as_deref()); - let price_mud_points = u64::from( - resolve_editor_image_edit_price( - &state, - generation_options.model, - image_size.as_ref(), - Some(generation_options.image_size), - ) - .await?, - ); - let source_entity_id = - editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-image-edit"); - let queue_job = enqueue_editor_generation_job( - &state, - &request_context, - caller.owner_user_id.as_str(), - EDITOR_IMAGE_EDIT_JOB_KIND, - source_entity_id, - "图片画布修改图片", - price_mud_points, - &payload, - ) - .await?; + let queue_job = + enqueue_editor_image_edit_for_owner(&state, &request_context, &caller, payload, None) + .await?; return Ok(json_success_body( Some(&request_context), EditorGenerationQueuedResponse { @@ -4054,6 +4030,60 @@ pub async fn edit_editor_image( edit_editor_image_for_owner(&state, &request_context, caller, payload).await } +pub(crate) async fn enqueue_editor_image_edit_for_owner( + state: &AppState, + request_context: &RequestContext, + caller: &EditorGenerationCaller, + mut payload: EditorImageEditRequest, + external_idempotency_key: Option<&str>, +) -> Result { + payload.generation_inputs = + sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); + ensure_editor_reference_image_source_is_stable( + payload.source_image_src.as_str(), + "editor-image-edit", + "sourceImageSrc", + "待修改图片", + )?; + ensure_editor_reference_image_sources_are_stable( + payload.reference_image_srcs.as_deref(), + "editor-image-edit", + "referenceImageSrcs", + "修改参考图", + )?; + ensure_editor_image_edit_source_allowed(state, caller.owner_user_id.as_str(), &payload).await?; + let generation_options = normalize_editor_image_edit_generation_options( + payload.model.as_deref(), + payload.aspect_ratio.as_deref(), + payload.image_size.as_deref(), + payload.size.as_deref(), + ); + let image_size = normalize_editor_image_generation_size(payload.size.as_deref()); + let price_mud_points = u64::from( + resolve_editor_image_edit_price( + state, + generation_options.model, + image_size.as_ref(), + Some(generation_options.image_size), + ) + .await?, + ); + let source_entity_id = + editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-image-edit"); + enqueue_editor_generation_job_for_caller( + state, + request_context, + caller.owner_user_id.as_str(), + EDITOR_IMAGE_EDIT_JOB_KIND, + source_entity_id, + "图片画布修改图片", + price_mud_points, + &payload, + external_idempotency_key, + ) + .await +} + pub(crate) async fn edit_editor_image_for_owner( state: &AppState, request_context: &RequestContext, @@ -5804,49 +5834,15 @@ pub async fn generate_editor_icon_spritesheet( Extension(authenticated): Extension, payload: Result, JsonRejection>, ) -> Result, AppError> { - let Json(mut payload) = parse_editor_generation_json_payload(payload)?; - payload.generation_inputs = - sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); + let Json(payload) = parse_editor_generation_json_payload(payload)?; let caller = EditorGenerationCaller::from_authenticated(&authenticated); if !state.config.external_generation_mode.is_inline() { - ensure_editor_reference_image_source_is_stable( - payload.reference_image_src.as_str(), - "editor-icon-spritesheet", - "referenceImageSrc", - "图标素材规范", - )?; - ensure_editor_reference_image_sources_are_stable( - payload.reference_image_srcs.as_deref(), - "editor-icon-spritesheet", - "referenceImageSrcs", - "图标素材参考图", - )?; - let generation_options = normalize_editor_generation_options( - payload.model.as_deref(), - payload.aspect_ratio.as_deref(), - payload.image_size.as_deref(), - ); - let price_mud_points = u64::from( - resolve_editor_icon_spritesheet_price( - &state, - Some(generation_options.model), - Some(generation_options.image_size), - ) - .await?, - ); - let source_entity_id = editor_generation_source_entity_id( - payload.project_id.as_deref(), - "editor-icon-spritesheet-generation", - ); - let queue_job = enqueue_editor_generation_job( + let queue_job = enqueue_editor_icon_spritesheet_generation_for_owner( &state, &request_context, - caller.owner_user_id.as_str(), - EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, - source_entity_id, - "图片画布生成图标素材", - price_mud_points, - &payload, + &caller, + payload, + None, ) .await?; return Ok(json_success_body( @@ -5859,6 +5855,58 @@ pub async fn generate_editor_icon_spritesheet( generate_editor_icon_spritesheet_for_owner(&state, &request_context, caller, payload).await } +pub(crate) async fn enqueue_editor_icon_spritesheet_generation_for_owner( + state: &AppState, + request_context: &RequestContext, + caller: &EditorGenerationCaller, + mut payload: EditorIconSpritesheetGenerationRequest, + external_idempotency_key: Option<&str>, +) -> Result { + payload.generation_inputs = + sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); + ensure_editor_reference_image_source_is_stable( + payload.reference_image_src.as_str(), + "editor-icon-spritesheet", + "referenceImageSrc", + "图标素材规范", + )?; + ensure_editor_reference_image_sources_are_stable( + payload.reference_image_srcs.as_deref(), + "editor-icon-spritesheet", + "referenceImageSrcs", + "图标素材参考图", + )?; + let generation_options = normalize_editor_generation_options( + payload.model.as_deref(), + payload.aspect_ratio.as_deref(), + payload.image_size.as_deref(), + ); + let price_mud_points = u64::from( + resolve_editor_icon_spritesheet_price( + state, + Some(generation_options.model), + Some(generation_options.image_size), + ) + .await?, + ); + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-icon-spritesheet-generation", + ); + enqueue_editor_generation_job_for_caller( + state, + request_context, + caller.owner_user_id.as_str(), + EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成图标素材", + price_mud_points, + &payload, + external_idempotency_key, + ) + .await +} + pub(crate) async fn generate_editor_icon_spritesheet_for_owner( state: &AppState, request_context: &RequestContext, @@ -5880,6 +5928,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( "图标素材参考图", )?; let icon_descriptions = normalize_icon_descriptions(payload.icon_descriptions)?; + let user_prompt = icon_descriptions.join("\n"); let (image_style, mut generation_warning) = normalize_editor_image_generation_style(payload.style.as_deref(), true); // 背景色决策挪到预扣泥点之后（见下方 execute_billable 闭包），避免余额不足 / 生成注定失败时 @@ -5956,7 +6005,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( EditorScreenBackgroundDecisionInput { kind: EditorScreenBackgroundDecisionKind::IconSpritesheet, screen_color: requested_screen_color.clone(), - prompt: icon_descriptions.join("\n"), + prompt: user_prompt.clone(), icon_descriptions: icon_descriptions.clone(), reference_count, source_image_data_url: None, @@ -6039,7 +6088,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( caller.owner_user_id.as_str(), generated.task_id.as_str(), image, - prompt.as_str(), + user_prompt.as_str(), generated.actual_prompt.as_deref(), EDITOR_ICON_SPRITESHEET_ASSET_KIND, "icon-spritesheets", @@ -6058,7 +6107,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( label: editor_generated_asset_variant_label(spritesheet_label.as_str(), "原图"), width: source_width, height: source_height, - prompt: prompt.clone(), + prompt: user_prompt.clone(), actual_prompt: generated.actual_prompt.clone(), model: generation_options.model.to_string(), task_id: generated.task_id.clone(), @@ -6224,7 +6273,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( owner_user_id.as_str(), generated.task_id.as_str(), &image, - "去除纯色背景", + user_prompt.as_str(), None, EDITOR_ICON_SPRITESHEET_ASSET_KIND, "icon-spritesheets", @@ -6247,7 +6296,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( asset_object_id: Some(spritesheet_persisted.asset_object_id.clone()), width: spritesheet_width, height: spritesheet_height, - prompt: "去除纯色背景".to_string(), + prompt: user_prompt.clone(), actual_prompt: None, model: generation_options.model.to_string(), provider: removal_provider.to_string(), @@ -6290,7 +6339,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( .map(|resource| resource.resource_id.clone()), task_id: generated.task_id.clone(), group_task_id: None, - prompt: "自动拆分图集".to_string(), + prompt: user_prompt.clone(), actual_prompt: None, model: generation_options.model.to_string(), provider: "Genarrative".to_string(), @@ -6872,50 +6921,16 @@ pub async fn extract_editor_ui_design_assets( State(state): State, Extension(request_context): Extension, Extension(authenticated): Extension, - Json(mut payload): Json, + Json(payload): Json, ) -> Result, AppError> { - payload.generation_inputs = - sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); let caller = EditorGenerationCaller::from_authenticated(&authenticated); if !state.config.external_generation_mode.is_inline() { - ensure_editor_reference_image_source_is_stable( - payload.source_image_src.as_str(), - "editor-ui-design-asset-extraction", - "sourceImageSrc", - "UI设计图", - )?; - ensure_editor_reference_image_sources_are_stable( - payload.reference_image_srcs.as_deref(), - "editor-ui-design-asset-extraction", - "referenceImageSrcs", - "UI素材参考图", - )?; - let generation_options = normalize_editor_ui_design_asset_extraction_options( - payload.model.as_deref(), - payload.aspect_ratio.as_str(), - payload.image_size.as_str(), - )?; - let price_mud_points = u64::from( - resolve_editor_ui_design_asset_extraction_price( - &state, - Some(generation_options.model), - Some(generation_options.image_size), - ) - .await?, - ); - let source_entity_id = editor_generation_source_entity_id( - payload.project_id.as_deref(), - "editor-ui-design-asset-extraction", - ); - let queue_job = enqueue_editor_generation_job( + let queue_job = enqueue_editor_ui_design_asset_extraction_for_owner( &state, &request_context, - caller.owner_user_id.as_str(), - EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, - source_entity_id, - "图片画布提取UI设计图素材", - price_mud_points, - &payload, + &caller, + payload, + None, ) .await?; return Ok(json_success_body( @@ -6928,6 +6943,58 @@ pub async fn extract_editor_ui_design_assets( extract_editor_ui_design_assets_for_owner(&state, &request_context, caller, payload).await } +pub(crate) async fn enqueue_editor_ui_design_asset_extraction_for_owner( + state: &AppState, + request_context: &RequestContext, + caller: &EditorGenerationCaller, + mut payload: EditorUiDesignAssetExtractionRequest, + external_idempotency_key: Option<&str>, +) -> Result { + payload.generation_inputs = + sanitize_editor_client_generation_inputs(payload.generation_inputs.take()); + ensure_editor_reference_image_source_is_stable( + payload.source_image_src.as_str(), + "editor-ui-design-asset-extraction", + "sourceImageSrc", + "UI设计图", + )?; + ensure_editor_reference_image_sources_are_stable( + payload.reference_image_srcs.as_deref(), + "editor-ui-design-asset-extraction", + "referenceImageSrcs", + "UI素材参考图", + )?; + let generation_options = normalize_editor_ui_design_asset_extraction_options( + payload.model.as_deref(), + payload.aspect_ratio.as_str(), + payload.image_size.as_str(), + )?; + let price_mud_points = u64::from( + resolve_editor_ui_design_asset_extraction_price( + state, + Some(generation_options.model), + Some(generation_options.image_size), + ) + .await?, + ); + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-ui-design-asset-extraction", + ); + enqueue_editor_generation_job_for_caller( + state, + request_context, + caller.owner_user_id.as_str(), + EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, + source_entity_id, + "图片画布提取UI设计图素材", + price_mud_points, + &payload, + external_idempotency_key, + ) + .await +} + pub(crate) async fn extract_editor_ui_design_assets_for_owner( state: &AppState, request_context: &RequestContext, @@ -9478,7 +9545,7 @@ async fn persist_editor_generated_image_data( task_id: &str, image: GeneratedImageAssetDataUrl, prompt: &str, - actual_prompt: Option<&str>, + _actual_prompt: Option<&str>, asset_kind: &str, path_kind: &str, file_stem: &str, @@ -9562,7 +9629,9 @@ async fn persist_editor_generated_image_data( AssetObjectAccessPolicy::Private, head.content_type.or(Some(persisted_mime_type)), head.content_length, - Some(actual_prompt.unwrap_or(prompt).to_string()), + // asset_object.prompt 是跨资源检索用的用户意图，不承载 provider + // actual/system prompt；后者仍只保存在 resource/asset 审计字段。 + Some(prompt.to_string()), asset_kind.to_string(), Some(task_id.to_string()), Some(owner_user_id.to_string()), @@ -13388,6 +13457,43 @@ mod tests { assert!(prompt.contains("角色设定：菜市场卖菜大妈")); } + #[test] + fn editor_generated_asset_persistence_keeps_user_prompt_separate_from_system_prompt() { + let source = include_str!("editor_project.rs"); + assert_function_contains( + source, + "pub(crate) async fn generate_editor_image_for_owner", + "fn normalize_editor_image_generation_size", + &[ + "image,\n role_setting.as_str(),\n generated.actual_prompt.as_deref(),", + "output_prompt = role_setting.clone();", + ], + ); + assert_function_contains( + source, + "pub(crate) async fn generate_editor_icon_spritesheet_for_owner", + "pub async fn extract_editor_ui_design_assets", + &[ + "let user_prompt = icon_descriptions.join(\"\\n\");", + "image,\n user_prompt.as_str(),\n generated.actual_prompt.as_deref(),", + "prompt: user_prompt.clone(),", + "&image,\n user_prompt.as_str(),\n None,", + ], + ); + assert_function_contains( + source, + "async fn persist_editor_generated_image_data", + "async fn persist_editor_provider_source_image", + &["Some(prompt.to_string())"], + ); + assert_function_not_contains( + source, + "async fn persist_editor_generated_image_data", + "async fn persist_editor_provider_source_image", + &["actual_prompt.unwrap_or(prompt)"], + ); + } + #[test] fn editor_canvas_generation_completion_inserts_result_layer_and_keeps_composer_closed() { let layers = json!([ diff --git a/server-rs/crates/api-server/src/external_api_auth.rs b/server-rs/crates/api-server/src/external_api_auth.rs index e7ee85b29..5f6f5044d 100644 --- a/server-rs/crates/api-server/src/external_api_auth.rs +++ b/server-rs/crates/api-server/src/external_api_auth.rs @@ -1,9 +1,13 @@ use axum::{ extract::{Request, State}, - http::{HeaderMap, StatusCode, header::AUTHORIZATION}, + http::{ + HeaderMap, HeaderValue, StatusCode, + header::{AUTHORIZATION, WWW_AUTHENTICATE}, + }, middleware::Next, response::Response, }; +use serde_json::json; use spacetime_client::ExternalApiKeyAuthenticateRecordInput; use tracing::warn; @@ -74,6 +78,71 @@ pub async fn require_external_api_key( Ok(response) } +pub async fn require_external_mcp_api_key( + State(state): State, + request: Request, + next: Next, +) -> Result { + let request_context = request.extensions().get::().cloned(); + match require_external_api_key(State(state), request, next).await { + Ok(response) => Ok(response), + Err(error) if error.status_code() == StatusCode::UNAUTHORIZED => { + Ok(map_external_mcp_authentication_error(error) + .into_response_with_context(request_context.as_ref())) + } + Err(error) => Err(error), + } +} + +fn map_external_mcp_authentication_error(error: AppError) -> AppError { + debug_assert_eq!(error.status_code(), StatusCode::UNAUTHORIZED); + external_mcp_authentication_guide_error() +} + +fn external_mcp_authentication_guide_error() -> AppError { + AppError::from_status(StatusCode::UNAUTHORIZED) + .with_message("连接陶泥儿托管 MCP 需要开发者 API Key") + .with_details(json!({ + "guide": { + "reason": "MCP_AUTHENTICATION_REQUIRED", + "action": "CONFIGURE_BEARER_API_KEY", + "authentication": { + "scheme": "Bearer", + "header": "Authorization", + "valueFormat": "Bearer " + }, + "keyManagement": { + "navigationLabel": "开发者 API Key", + "rawKeyShownOnce": true + }, + "retry": { + "method": "POST", + "path": "/api/external/v1/mcp", + "rpcMethod": "initialize" + }, + "steps": [ + "登录陶泥儿，在「开发者 API Key」中创建密钥；原始密钥只显示一次", + "把密钥配置为 MCP 连接的 Bearer token；不要粘贴到聊天或写入仓库", + "使用相同 MCP URL 重新发送 initialize" + ], + "credentialSafety": { + "rawKeyShownOnce": true, + "neverPasteIntoChat": true, + "neverStoreInRepository": true + }, + "publicDiscovery": { + "manifest": "/api/external/v1/agent-integration.json", + "skill": "/api/external/v1/skill/SKILL.md", + "openapi": "/api/external/v1/openapi.json" + } + } + })) + .with_header( + WWW_AUTHENTICATE.as_str(), + HeaderValue::from_static("Bearer realm=\"genarrative-external-editor\""), + ) +} + fn extract_external_api_bearer(headers: &HeaderMap) -> Result { let authorization = headers .get(AUTHORIZATION) @@ -89,3 +158,32 @@ fn extract_external_api_bearer(headers: &HeaderMap) -> Result .map(ToOwned::to_owned) .ok_or_else(|| AppError::from_status(StatusCode::UNAUTHORIZED)) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn mcp_authentication_guide_replaces_sensitive_key_diagnostics() { + let error = AppError::from_status(StatusCode::UNAUTHORIZED).with_details(json!({ + "provider": "external-api-key", + "message": "SENSITIVE_KEY_LURE 不存在或已失效" + })); + + let mapped = map_external_mcp_authentication_error(error); + let serialized = serde_json::to_string( + mapped + .details() + .expect("mapped authentication error should contain guide details"), + ) + .expect("guide should serialize"); + assert_eq!(mapped.status_code(), StatusCode::UNAUTHORIZED); + assert_eq!(mapped.message(), "连接陶泥儿托管 MCP 需要开发者 API Key"); + assert!(serialized.contains("MCP_AUTHENTICATION_REQUIRED")); + assert!(serialized.contains("CONFIGURE_BEARER_API_KEY")); + assert!(!serialized.contains("SENSITIVE_KEY_LURE")); + assert!(!serialized.contains("provider")); + assert!(!serialized.contains("不存在")); + assert!(!serialized.contains("已失效")); + } +} diff --git a/server-rs/crates/api-server/src/external_editor_api.rs b/server-rs/crates/api-server/src/external_editor_api.rs index 583203d07..1da75d7b5 100644 --- a/server-rs/crates/api-server/src/external_editor_api.rs +++ b/server-rs/crates/api-server/src/external_editor_api.rs @@ -1,46 +1,57 @@ use axum::{ Json, extract::{Extension, Path, State, rejection::JsonRejection}, - http::{StatusCode, header::CONTENT_TYPE}, + http::{HeaderMap, HeaderValue, StatusCode, header::CONTENT_TYPE}, response::{IntoResponse, Response}, }; +use serde::de::DeserializeOwned; use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; +use shared_contracts::external_generation::{ + ExternalEditorGenerationJobResponse, ExternalEditorGenerationSubmissionResponse, + ExternalGenerationJobStatus, +}; use shared_kernel::build_prefixed_uuid_id; use spacetime_client::{ EditorAssetCreateRecordInput, EditorAssetDeleteRecordInput, EditorAssetFolderCreateRecordInput, EditorAssetFolderDeleteRecordInput, EditorAssetFolderUpdateRecordInput, EditorAssetUpdateRecordInput, EditorProjectCreateRecordInput, EditorProjectDeleteRecordInput, EditorProjectGetRecordInput, EditorProjectRenameRecordInput, - EditorProjectResourceCreateRecordInput, + EditorProjectResourceCreateRecordInput, ExternalGenerationJobGetRecordInput, + ExternalGenerationJobRecord, SpacetimeClientError, }; use crate::{ api_response::json_success_body, character_animation_assets::{ - generate_editor_character_animation_for_owner, generate_editor_video_for_owner, + enqueue_editor_character_animation_for_owner, enqueue_editor_video_generation_for_owner, }, + editor_generation_queue::editor_generation_queue_state, editor_project::{ EDITOR_ASSET_FOLDER_ID_PREFIX, EDITOR_ASSET_ID_PREFIX, EDITOR_PROJECT_DEFAULT_TITLE, EDITOR_PROJECT_ID_PREFIX, EDITOR_RESOURCE_ID_PREFIX, EditorAssetFolderPayload, EditorAssetLibraryPayload, EditorAssetPayload, EditorCanvasViewportPayload, EditorGenerationCaller, EditorIconSpritesheetGenerationRequest, EditorImageEditRequest, EditorImageGenerationRequest, EditorProjectPayload, EditorProjectResourcePayload, - EditorUiDesignAssetExtractionRequest, current_utc_micros, edit_editor_image_for_owner, + EditorUiDesignAssetExtractionRequest, current_utc_micros, editor_asset_folder_payload_from_record, editor_asset_library_payload_from_record, editor_asset_payload_from_record, editor_project_payload_from_record, - editor_project_resource_payload_from_record, extract_editor_ui_design_assets_for_owner, - generate_editor_icon_spritesheet_for_owner, generate_editor_image_for_owner, - map_editor_project_error, normalize_editor_persisted_media_src, normalize_optional_string, + editor_project_resource_payload_from_record, + enqueue_editor_icon_spritesheet_generation_for_owner, enqueue_editor_image_edit_for_owner, + enqueue_editor_image_generation_for_owner, + enqueue_editor_ui_design_asset_extraction_for_owner, map_editor_project_error, + normalize_editor_persisted_media_src, normalize_optional_string, parse_editor_generation_json_payload, sanitize_editor_client_generation_inputs, save_editor_project_layout_with_revision_and_get, serialize_editor_asset_metadata, }, external_api_auth::ExternalApiPrincipal, + external_generation::map_external_generation_job_status_detail, http_error::AppError, request_context::RequestContext, state::AppState, vector_engine_audio_generation::{ - generate_editor_background_music_for_owner, generate_editor_sound_effect_for_owner, + enqueue_editor_background_music_generation_for_owner, + enqueue_editor_sound_effect_generation_for_owner, }, }; @@ -51,6 +62,8 @@ const SCOPE_EDITOR_IMAGE_GENERATE: &str = "editor:image-generate"; const SCOPE_EDITOR_ASSET: &str = "editor:asset"; const OPENAPI_JSON: &str = include_str!("../../../../docs/openapi/genarrative-external-v1.openapi.json"); +const EXTERNAL_GENERATION_POLL_AFTER_MS: u64 = 1_500; +const IDEMPOTENCY_KEY_HEADER: &str = "idempotency-key"; #[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] @@ -609,140 +622,328 @@ pub async fn generate_external_editor_image( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, payload: Result, JsonRejection>, -) -> Result, AppError> { +) -> Result { let Json(payload) = parse_editor_generation_json_payload(payload)?; require_scope(&principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - generate_editor_image_for_owner( + let idempotency_key = require_idempotency_key(&headers)?; + let project_id = payload.project_id.clone(); + let job = enqueue_editor_image_generation_for_owner( &state, &request_context, - editor_generation_caller(&principal, payload.project_id.clone()), + &editor_generation_caller(&principal, project_id), payload, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) } pub async fn edit_external_editor_image( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, Json(payload): Json, -) -> Result, AppError> { +) -> Result { require_scope(&principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - edit_editor_image_for_owner( + let idempotency_key = require_idempotency_key(&headers)?; + let project_id = payload.project_id.clone(); + let job = enqueue_editor_image_edit_for_owner( &state, &request_context, - editor_generation_caller(&principal, payload.project_id.clone()), + &editor_generation_caller(&principal, project_id), payload, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) } pub async fn generate_external_editor_icon_spritesheet( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, payload: Result, JsonRejection>, -) -> Result, AppError> { +) -> Result { let Json(payload) = parse_editor_generation_json_payload(payload)?; require_scope(&principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - generate_editor_icon_spritesheet_for_owner( + let idempotency_key = require_idempotency_key(&headers)?; + let project_id = payload.project_id.clone(); + let job = enqueue_editor_icon_spritesheet_generation_for_owner( &state, &request_context, - editor_generation_caller(&principal, payload.project_id.clone()), + &editor_generation_caller(&principal, project_id), payload, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) } pub async fn extract_external_editor_ui_design_assets( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, Json(payload): Json, -) -> Result, AppError> { +) -> Result { require_scope(&principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - extract_editor_ui_design_assets_for_owner( + let idempotency_key = require_idempotency_key(&headers)?; + let project_id = payload.project_id.clone(); + let job = enqueue_editor_ui_design_asset_extraction_for_owner( &state, &request_context, - editor_generation_caller(&principal, payload.project_id.clone()), + &editor_generation_caller(&principal, project_id), payload, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) } pub async fn generate_external_editor_character_animation( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, payload: Result< Json, JsonRejection, >, -) -> Result, Response> { +) -> Result { require_scope_response(&request_context, &principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - generate_editor_character_animation_for_owner( - state, - request_context, - principal.owner_user_id().to_string(), + let idempotency_key = require_idempotency_key(&headers) + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + let Json(payload) = parse_external_generation_json_payload(&request_context, payload)?; + let job = enqueue_editor_character_animation_for_owner( + &state, + &request_context, + principal.owner_user_id(), payload, - None, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) } pub async fn generate_external_editor_video( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, payload: Result, JsonRejection>, -) -> Result, Response> { +) -> Result { require_scope_response(&request_context, &principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - generate_editor_video_for_owner( - state, - request_context, - principal.owner_user_id().to_string(), + let idempotency_key = require_idempotency_key(&headers) + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + let Json(payload) = parse_external_generation_json_payload(&request_context, payload)?; + let job = enqueue_editor_video_generation_for_owner( + &state, + &request_context, + principal.owner_user_id(), payload, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) } pub async fn generate_external_editor_sound_effect( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, payload: Result< Json, JsonRejection, >, -) -> Result, Response> { +) -> Result { require_scope_response(&request_context, &principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - generate_editor_sound_effect_for_owner( - state, - request_context, - principal.owner_user_id().to_string(), + let idempotency_key = require_idempotency_key(&headers) + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + let Json(payload) = parse_external_generation_json_payload(&request_context, payload)?; + let job = enqueue_editor_sound_effect_generation_for_owner( + &state, + &request_context, + principal.owner_user_id(), payload, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) } pub async fn generate_external_editor_background_music( State(state): State, Extension(request_context): Extension, Extension(principal): Extension, + headers: HeaderMap, payload: Result< Json, JsonRejection, >, -) -> Result, Response> { +) -> Result { require_scope_response(&request_context, &principal, SCOPE_EDITOR_IMAGE_GENERATE)?; - generate_editor_background_music_for_owner( - state, - request_context, - principal.owner_user_id().to_string(), + let idempotency_key = require_idempotency_key(&headers) + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + let Json(payload) = parse_external_generation_json_payload(&request_context, payload)?; + let job = enqueue_editor_background_music_generation_for_owner( + &state, + &request_context, + principal.owner_user_id(), payload, + Some(idempotency_key), ) - .await + .await?; + Ok(external_generation_accepted_response(&request_context, job)) +} + +pub async fn get_external_editor_generation_job( + State(state): State, + Path(operation_id): Path, + Extension(request_context): Extension, + Extension(principal): Extension, +) -> Result, AppError> { + require_scope(&principal, SCOPE_EDITOR_IMAGE_GENERATE)?; + let input = ExternalGenerationJobGetRecordInput { + job_id: operation_id, + owner_user_id: principal.owner_user_id().to_string(), + }; + let summary = state + .spacetime_client() + .get_external_generation_job_summary(input.clone()) + .await + .map_err(map_external_generation_lookup_error)?; + let detail = map_external_generation_job_status_detail(summary.clone()); + let result = if detail.status.status == ExternalGenerationJobStatus::Completed { + let artifacts = state + .spacetime_client() + .get_external_generation_job_generated_artifacts(input) + .await + .map_err(map_external_generation_lookup_error)?; + let payload = artifacts + .result_payload_json + .as_deref() + .and_then(|payload| serde_json::from_str::(payload).ok()) + .ok_or_else(|| { + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": EXTERNAL_EDITOR_PROVIDER, + "message": "生成任务已完成，但结果暂时不可读取。", + })) + })?; + Some(payload.get("result").cloned().ok_or_else(|| { + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": EXTERNAL_EDITOR_PROVIDER, + "message": "生成任务已完成，但稳定结果引用缺失。", + })) + })?) + } else { + None + }; + let poll_after_ms = matches!( + detail.status.status, + ExternalGenerationJobStatus::Queued | ExternalGenerationJobStatus::Running + ) + .then_some(EXTERNAL_GENERATION_POLL_AFTER_MS); + + Ok(json_success_body( + Some(&request_context), + ExternalEditorGenerationJobResponse { + operation_id: detail.status.operation_id, + kind: summary.job_kind, + status: detail.status.status, + phase_label: detail.status.phase_label, + phase_detail: detail.status.phase_detail, + progress: detail.status.progress, + error: detail.status.error, + warning: detail.warning, + result, + poll_after_ms, + updated_at_micros: detail.status.updated_at_micros, + }, + )) +} + +fn external_generation_accepted_response( + request_context: &RequestContext, + job: ExternalGenerationJobRecord, +) -> Response { + let kind = job.job_kind.clone(); + let status = editor_generation_queue_state(job); + let status_url = format!("/api/external/v1/generations/{}", status.operation_id); + let mut response = ( + StatusCode::ACCEPTED, + json_success_body( + Some(request_context), + ExternalEditorGenerationSubmissionResponse { + operation_id: status.operation_id, + kind, + status: status.status, + status_url: status_url.clone(), + poll_after_ms: EXTERNAL_GENERATION_POLL_AFTER_MS, + updated_at_micros: status.updated_at_micros, + }, + ), + ) + .into_response(); + if let Ok(location) = HeaderValue::from_str(&status_url) { + response.headers_mut().insert("location", location); + } + response + .headers_mut() + .insert("retry-after", HeaderValue::from_static("2")); + response +} + +fn require_idempotency_key(headers: &HeaderMap) -> Result<&str, AppError> { + let value = headers + .get(IDEMPOTENCY_KEY_HEADER) + .and_then(|value| value.to_str().ok()) + .map(str::trim) + .filter(|value| !value.is_empty()) + .ok_or_else(|| { + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": EXTERNAL_EDITOR_PROVIDER, + "message": "生成请求必须携带 Idempotency-Key 请求头。", + })) + })?; + if value.len() > 128 || !value.bytes().all(|byte| (0x21..=0x7e).contains(&byte)) { + return Err( + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": EXTERNAL_EDITOR_PROVIDER, + "message": "Idempotency-Key 必须是 1-128 个可打印 ASCII 字符，且不能包含空格。", + })), + ); + } + Ok(value) +} + +fn parse_external_generation_json_payload( + request_context: &RequestContext, + payload: Result, JsonRejection>, +) -> Result, Response> { + payload.map_err(|error| { + AppError::from_status(StatusCode::BAD_REQUEST) + .with_details(json!({ + "provider": EXTERNAL_EDITOR_PROVIDER, + "message": error.body_text(), + })) + .into_response_with_context(Some(request_context)) + }) +} + +fn map_external_generation_lookup_error(error: SpacetimeClientError) -> AppError { + if error.to_string().contains("不存在") { + AppError::from_status(StatusCode::NOT_FOUND) + } else { + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": EXTERNAL_EDITOR_PROVIDER, + "message": "生成任务状态暂时不可用。", + })) + } } fn editor_generation_caller( @@ -797,6 +998,47 @@ fn serialize_external_editor_generation_inputs( mod tests { use super::*; + fn external_generation_job_fixture(status: &str) -> ExternalGenerationJobRecord { + ExternalGenerationJobRecord { + job_id: "task-external-test".to_string(), + dedupe_key: "external-api-generation:editor_image_generation:fingerprint".to_string(), + job_kind: "editor_image_generation".to_string(), + owner_user_id: "user-1".to_string(), + source_module: "editor-canvas".to_string(), + source_entity_id: "project-1".to_string(), + request_label: "图片画布生成图片".to_string(), + request_payload_json: "{}".to_string(), + status: status.to_string(), + attempt: 0, + max_attempts: 1, + last_error_message: None, + worker_id: None, + lease_expires_at: None, + available_at: "2026-07-31T00:00:00Z".to_string(), + result_payload_json: None, + created_at: "2026-07-31T00:00:00Z".to_string(), + started_at: None, + completed_at: None, + updated_at: "2026-07-31T00:00:00Z".to_string(), + updated_at_micros: 1_785_456_000_000_000, + lease_token: None, + price_mud_points: 2, + refund_ledger_id: None, + notification_acknowledged_at: None, + notification_acknowledged_at_micros: None, + phase: None, + } + } + + fn request_context(wants_envelope: bool) -> RequestContext { + RequestContext::new( + "req-external-generation-test".to_string(), + "POST /api/external/v1/editor/images/generations".to_string(), + std::time::Duration::ZERO, + wants_envelope, + ) + } + #[test] fn external_editor_canvas_save_request_requires_expected_revision() { let missing_revision = serde_json::from_value::(json!({ @@ -835,6 +1077,117 @@ mod tests { assert!(parsed.get("mattingModel").is_none()); } + #[test] + fn external_generation_requires_bounded_printable_idempotency_key() { + let missing = HeaderMap::new(); + let error = require_idempotency_key(&missing).expect_err("外部生成必须显式提供幂等键"); + assert_eq!(error.status_code(), StatusCode::BAD_REQUEST); + assert!(error.body_text().contains("Idempotency-Key")); + + let mut whitespace = HeaderMap::new(); + whitespace.insert(IDEMPOTENCY_KEY_HEADER, HeaderValue::from_static(" ")); + assert!(require_idempotency_key(&whitespace).is_err()); + + let mut valid = HeaderMap::new(); + let longest_valid = "x".repeat(128); + valid.insert( + IDEMPOTENCY_KEY_HEADER, + HeaderValue::from_str(&longest_valid).expect("128 字节可打印 ASCII 应是合法 header"), + ); + assert_eq!( + require_idempotency_key(&valid).expect("边界长度幂等键应通过"), + longest_valid + ); + + for invalid in [ + "x".repeat(129), + "contains space".to_string(), + "中文".to_string(), + ] { + let mut headers = HeaderMap::new(); + headers.insert( + IDEMPOTENCY_KEY_HEADER, + HeaderValue::from_str(&invalid).expect("测试值应可构造为 HTTP header"), + ); + let error = require_idempotency_key(&headers) + .expect_err("超长、含空格或非 ASCII 的幂等键必须拒绝"); + assert_eq!(error.status_code(), StatusCode::BAD_REQUEST); + } + } + + #[tokio::test] + async fn external_generation_submission_is_accepted_with_poll_contract() { + let response = external_generation_accepted_response( + &request_context(false), + external_generation_job_fixture("pending"), + ); + + assert_eq!(response.status(), StatusCode::ACCEPTED); + assert_eq!( + response + .headers() + .get("location") + .and_then(|value| value.to_str().ok()), + Some("/api/external/v1/generations/task-external-test") + ); + assert_eq!( + response + .headers() + .get("retry-after") + .and_then(|value| value.to_str().ok()), + Some("2") + ); + let body = axum::body::to_bytes(response.into_body(), 64 * 1024) + .await + .expect("submission body 应可读取"); + let payload: Value = serde_json::from_slice(&body).expect("submission body 应为 JSON"); + + assert_eq!(payload["operationId"], json!("task-external-test")); + assert_eq!(payload["kind"], json!("editor_image_generation")); + assert_eq!(payload["status"], json!("queued")); + assert_eq!( + payload["statusUrl"], + json!("/api/external/v1/generations/task-external-test") + ); + assert_eq!( + payload["pollAfterMs"], + json!(EXTERNAL_GENERATION_POLL_AFTER_MS) + ); + } + + #[tokio::test] + async fn idempotent_completed_submission_reports_completed_in_envelope() { + let response = external_generation_accepted_response( + &request_context(true), + external_generation_job_fixture("completed"), + ); + let body = axum::body::to_bytes(response.into_body(), 64 * 1024) + .await + .expect("submission envelope 应可读取"); + let payload: Value = serde_json::from_slice(&body).expect("submission envelope 应为 JSON"); + + assert_eq!(payload["ok"], json!(true)); + assert_eq!(payload["data"]["operationId"], json!("task-external-test")); + assert_eq!(payload["data"]["status"], json!("completed")); + assert_eq!( + payload["data"]["pollAfterMs"], + json!(EXTERNAL_GENERATION_POLL_AFTER_MS) + ); + } + + #[test] + fn generation_lookup_hides_cross_owner_jobs_as_not_found() { + let not_found = map_external_generation_lookup_error(SpacetimeClientError::Procedure( + "external_generation_job 不存在".to_string(), + )); + assert_eq!(not_found.status_code(), StatusCode::NOT_FOUND); + + let unavailable = + map_external_generation_lookup_error(SpacetimeClientError::ConnectDropped); + assert_eq!(unavailable.status_code(), StatusCode::BAD_GATEWAY); + assert!(!unavailable.body_text().contains("ConnectDropped")); + } + #[test] fn exported_openapi_json_contains_external_editor_routes_and_security() { let parsed: Value = serde_json::from_str(OPENAPI_JSON).expect("openapi json should parse"); @@ -890,6 +1243,43 @@ mod tests { .get("/api/external/v1/editor/images/generations") .is_some() ); + for path in [ + "/api/external/v1/editor/images/generations", + "/api/external/v1/editor/images/edits", + "/api/external/v1/editor/icon-spritesheets/generations", + "/api/external/v1/editor/ui-designs/assets/extractions", + "/api/external/v1/editor/character-animations/generations", + "/api/external/v1/editor/videos/generations", + "/api/external/v1/editor/audios/sound-effects/generations", + "/api/external/v1/editor/audios/background-music/generations", + ] { + let operation = &parsed["paths"][path]["post"]; + assert!(operation["responses"].get("202").is_some(), "{path}"); + assert!(operation["responses"].get("200").is_none(), "{path}"); + assert!( + operation["parameters"] + .as_array() + .is_some_and(|parameters| { + parameters.iter().any(|parameter| { + parameter.get("$ref").and_then(Value::as_str) + == Some("#/components/parameters/IdempotencyKey") + }) + }) + ); + } + assert!( + parsed["paths"] + .get("/api/external/v1/generations/{operationId}") + .is_some() + ); + for path in [ + "/api/external/v1/agent-integration.json", + "/api/external/v1/skill/SKILL.md", + "/api/external/v1/skill.zip", + "/api/external/v1/mcp", + ] { + assert!(parsed["paths"].get(path).is_some(), "{path}"); + } assert!( parsed["components"]["schemas"]["EditorImageGenerationRequest"]["required"] .as_array() diff --git a/server-rs/crates/api-server/src/external_generation.rs b/server-rs/crates/api-server/src/external_generation.rs index b3ad859d3..1fba1ce65 100644 --- a/server-rs/crates/api-server/src/external_generation.rs +++ b/server-rs/crates/api-server/src/external_generation.rs @@ -217,7 +217,7 @@ fn user_visible_external_generation_error(job_kind: &str, error: Option) error } -fn map_external_generation_job_status_detail( +pub(crate) fn map_external_generation_job_status_detail( job: ExternalGenerationJobSummaryRecord, ) -> ExternalGenerationJobStatusDetailRecord { let warning = job.warning_message.clone(); diff --git a/server-rs/crates/api-server/src/external_generation_worker.rs b/server-rs/crates/api-server/src/external_generation_worker.rs index 2c3db1624..b2b3bdcc9 100644 --- a/server-rs/crates/api-server/src/external_generation_worker.rs +++ b/server-rs/crates/api-server/src/external_generation_worker.rs @@ -1213,6 +1213,14 @@ fn editor_generation_result_payload_json( compact_editor_generation_result(response.clone()), ); } + if is_external_api_generation_job(job) + && let Some(object) = payload.as_object_mut() + { + object.insert( + "result".to_string(), + compact_external_api_generation_result(response.clone()), + ); + } if let Some(warning) = extract_editor_generation_warning(response) && let Some(object) = payload.as_object_mut() { @@ -1221,6 +1229,12 @@ fn editor_generation_result_payload_json( payload.to_string() } +fn is_external_api_generation_job(job: &ExternalGenerationJobRecord) -> bool { + job.dedupe_key + .trim() + .starts_with("external-api-generation:") +} + fn is_editor_agent_generation_job(job: &ExternalGenerationJobRecord) -> bool { serde_json::from_str::(job.request_payload_json.as_str()) .ok() @@ -1289,6 +1303,177 @@ fn compact_editor_generation_result(mut result: Value) -> Value { result } +fn compact_external_api_generation_result(result: Value) -> Value { + let mut result = result.get("data").cloned().unwrap_or(result); + let Some(object) = result.as_object_mut() else { + return Value::Null; + }; + object.retain(|key, _| { + matches!( + key.as_str(), + "ok" | "imageSrc" + | "videoSrc" + | "audioSrc" + | "previewVideoPath" + | "thumbnailSrc" + | "objectKey" + | "assetObjectId" + | "width" + | "height" + | "sourceType" + | "model" + | "taskId" + | "durationSeconds" + | "resolution" + | "priceMudPoints" + | "audioKind" + | "spritesheetImageSrc" + | "spritesheetWidth" + | "spritesheetHeight" + | "iconImageSrcs" + | "frames" + | "frameCount" + | "frameWidth" + | "frameHeight" + | "fps" + | "resource" + | "asset" + | "spritesheetResource" + | "spritesheetAsset" + | "warning" + | "sliceWarning" + ) + }); + for field in ["resource", "spritesheetResource"] { + if let Some(resource) = object.get_mut(field).and_then(Value::as_object_mut) { + compact_external_generation_resource(resource); + } + } + for field in ["asset", "spritesheetAsset"] { + if let Some(asset) = object.get_mut(field).and_then(Value::as_object_mut) { + compact_external_generation_asset(asset); + } + } + if let Some(icons) = object + .get_mut("iconImageSrcs") + .and_then(Value::as_array_mut) + { + for icon in icons { + let Some(icon) = icon.as_object_mut() else { + continue; + }; + icon.retain(|key, _| { + matches!( + key.as_str(), + "name" | "imageSrc" | "objectKey" | "width" | "height" | "resource" | "asset" + ) + }); + if let Some(resource) = icon.get_mut("resource").and_then(Value::as_object_mut) { + compact_external_generation_resource(resource); + } + if let Some(asset) = icon.get_mut("asset").and_then(Value::as_object_mut) { + compact_external_generation_asset(asset); + } + remove_unstable_external_generation_media_fields(icon); + } + } + if let Some(frames) = object.get_mut("frames").and_then(Value::as_array_mut) { + for frame in frames { + let Some(frame) = frame.as_object_mut() else { + continue; + }; + frame.retain(|key, _| { + matches!( + key.as_str(), + "frameIndex" | "imageSrc" | "objectKey" | "width" | "height" + ) + }); + remove_unstable_external_generation_media_fields(frame); + } + } + for field in ["warning", "sliceWarning"] { + if let Some(warning) = object.get_mut(field).and_then(Value::as_object_mut) + && let Some(reason) = warning.get_mut("reason") + && let Some(value) = reason.as_str() + { + *reason = Value::String(normalize_editor_generation_warning_reason(value)); + } + } + remove_unstable_external_generation_media_fields(object); + result +} + +fn compact_external_generation_resource(resource: &mut serde_json::Map) { + resource.retain(|key, _| { + matches!( + key.as_str(), + "resourceId" + | "projectId" + | "objectKey" + | "assetObjectId" + | "imageSrc" + | "width" + | "height" + | "sourceType" + | "assetKind" + | "taskId" + ) + }); + remove_unstable_external_generation_media_fields(resource); +} + +fn compact_external_generation_asset(asset: &mut serde_json::Map) { + asset.retain(|key, _| { + matches!( + key.as_str(), + "assetId" + | "folderId" + | "objectKey" + | "assetObjectId" + | "imageSrc" + | "thumbnailSrc" + | "width" + | "height" + | "sourceType" + | "assetKind" + | "taskId" + ) + }); + remove_unstable_external_generation_media_fields(asset); +} + +fn remove_unstable_external_generation_media_fields(object: &mut serde_json::Map) { + object.retain(|key, value| { + if !matches!( + key.as_str(), + "imageSrc" + | "videoSrc" + | "audioSrc" + | "previewVideoPath" + | "thumbnailSrc" + | "spritesheetImageSrc" + ) { + return true; + } + value + .as_str() + .is_some_and(is_stable_external_generation_media_reference) + }); +} + +fn is_stable_external_generation_media_reference(value: &str) -> bool { + let value = value.trim(); + !value.is_empty() + && value.starts_with('/') + && !value.starts_with("//") + && !value.contains('?') + && !value.contains('#') + && !value.to_ascii_lowercase().starts_with("data:") + && !value.to_ascii_lowercase().starts_with("blob:") + && !value.to_ascii_lowercase().starts_with("http://") + && !value.to_ascii_lowercase().starts_with("https://") +} + fn is_editor_internal_processing_model(model: &str) -> bool { matches!( model.trim().to_ascii_lowercase().as_str(), @@ -1348,7 +1533,13 @@ fn extract_editor_generation_warning(response: &Value) -> Option { fn normalize_editor_generation_warning_reason(reason: &str) -> String { let normalized = reason.to_ascii_lowercase(); - if normalized.contains("data:") || normalized.contains("blob:") { + if normalized.contains("data:") + || normalized.contains("blob:") + || normalized.contains("http://") + || normalized.contains("https://") + || normalized.contains("x-amz-") + || normalized.contains("signature=") + { return EDITOR_GENERATION_WARNING_REDACTED_MESSAGE.to_string(); } let mut chars = reason.chars(); @@ -2051,6 +2242,135 @@ mod tests { assert!(!payload.to_string().contains("data:image")); } + #[test] + fn external_api_result_keeps_stable_artifacts_and_removes_unstable_media() { + let mut job = external_generation_job_record_fixture(Some("lease-1")); + job.dedupe_key = "external-api-generation:editor_image_generation:fingerprint".to_string(); + let response = json!({ + "data": { + "imageSrc": "data:image/png;base64,SHOULD_NOT_PERSIST", + "videoSrc": "blob:https://example.test/video", + "audioSrc": "https://cdn.example.test/audio.mp3?X-Amz-Signature=secret", + "previewVideoPath": "https://cdn.example.test/stable-looking-but-external.mp4", + "thumbnailSrc": "/api/assets/object/thumbnail.png?expires=1&signature=secret", + "objectKey": "users/user-1/generated/main.png", + "assetObjectId": "asset-object-main", + "width": 1024, + "height": 1024, + "provider": "internal-provider-must-not-persist", + "resource": { + "resourceId": "resource-main", + "projectId": "project-1", + "objectKey": "users/user-1/generated/main.png", + "assetObjectId": "asset-object-main", + "imageSrc": "https://cdn.example.test/main.png?signature=secret", + "width": 1024, + "height": 1024, + "prompt": "不应复制完整资源元数据" + }, + "asset": { + "assetId": "asset-main", + "folderId": "folder-1", + "objectKey": "users/user-1/generated/main.png", + "assetObjectId": "asset-object-main", + "imageSrc": "/api/assets/object/main.png", + "thumbnailSrc": "https://cdn.example.test/thumb.png?signature=secret", + "width": 1024, + "height": 1024, + "generationInputs": {"private": true} + }, + "project": { + "projectId": "project-1", + "canvas": {"layers": ["large-layout-must-not-persist"]} + }, + "warning": { + "code": "dimension-restore-fallback", + "reason": "已保留 provider 实际输出尺寸。" + } + }, + "meta": { + "requestId": "worker-envelope-must-not-persist" + } + }); + + let payload: Value = + serde_json::from_str(&editor_generation_result_payload_json(&job, &response)) + .expect("外部生成结果应是合法 JSON"); + let result = &payload["result"]; + + assert!(result.get("project").is_none()); + assert!(result.get("provider").is_none()); + for unstable_field in [ + "imageSrc", + "videoSrc", + "audioSrc", + "previewVideoPath", + "thumbnailSrc", + ] { + assert!( + result.get(unstable_field).is_none(), + "不稳定媒体字段 {unstable_field} 不得持久化" + ); + } + assert_eq!( + result["objectKey"], + json!("users/user-1/generated/main.png") + ); + assert_eq!(result["assetObjectId"], json!("asset-object-main")); + assert_eq!(result["resource"]["resourceId"], json!("resource-main")); + assert_eq!( + result["resource"]["objectKey"], + json!("users/user-1/generated/main.png") + ); + assert!(result["resource"].get("imageSrc").is_none()); + assert!(result["resource"].get("prompt").is_none()); + assert_eq!(result["asset"]["assetId"], json!("asset-main")); + assert_eq!( + result["asset"]["imageSrc"], + json!("/api/assets/object/main.png") + ); + assert!(result["asset"].get("thumbnailSrc").is_none()); + assert!(result["asset"].get("generationInputs").is_none()); + assert_eq!( + result["warning"], + json!({ + "code": "dimension-restore-fallback", + "reason": "已保留 provider 实际输出尺寸。" + }) + ); + assert_eq!(payload["warning"], result["warning"]); + assert!(result.get("prompt").is_none()); + assert!(result.get("actualPrompt").is_none()); + let serialized = payload.to_string().to_ascii_lowercase(); + for forbidden in [ + "data:", + "blob:", + "x-amz-signature", + "?signature=", + "large-layout", + ] { + assert!( + !serialized.contains(forbidden), + "compact result 不应包含 {forbidden}" + ); + } + } + + #[test] + fn non_external_job_does_not_publish_query_result() { + let job = external_generation_job_record_fixture(Some("lease-1")); + let payload: Value = serde_json::from_str(&editor_generation_result_payload_json( + &job, + &json!({ + "objectKey": "users/user-1/generated/main.png", + "resource": {"resourceId": "resource-main"} + }), + )) + .expect("普通编辑器任务结果应为合法 JSON"); + + assert!(payload.get("result").is_none()); + } + #[test] fn worker_job_timeout_uses_long_budget_for_image_and_video_jobs() { let config = AppConfig { diff --git a/server-rs/crates/api-server/src/external_mcp.rs b/server-rs/crates/api-server/src/external_mcp.rs new file mode 100644 index 000000000..a7f9d60fe --- /dev/null +++ b/server-rs/crates/api-server/src/external_mcp.rs @@ -0,0 +1,985 @@ +use std::sync::{Arc, LazyLock}; + +use axum::{ + body::Body, + http::{ + Method, Request, + header::{AUTHORIZATION, CONTENT_TYPE}, + }, +}; +use http_body_util::BodyExt; +use rmcp::{ + RoleServer, ServerHandler, + model::{ + CallToolRequestParams, CallToolResult, ErrorData, Implementation, ListResourcesResult, + ListToolsResult, PaginatedRequestParams, ReadResourceRequestParams, ReadResourceResult, + Resource, ResourceContents, ServerCapabilities, ServerInfo, Tool, ToolAnnotations, + }, + service::RequestContext as McpRequestContext, + transport::streamable_http_server::{ + StreamableHttpServerConfig, StreamableHttpService, session::local::LocalSessionManager, + }, +}; +use serde_json::{Map, Value, json}; +use tower::ServiceExt; + +use crate::{modules, request_context::RequestContext, state::AppState}; + +const OPENAPI_JSON: &str = + include_str!("../../../../docs/openapi/genarrative-external-v1.openapi.json"); +const SKILL_MD: &str = + include_str!("../../../../.codex/skills/genarrative-external-editor-api/SKILL.md"); +const SKILL_CAPABILITY_ROUTING_MD: &str = include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/capability-routing.md" +); +const SKILL_API_OPERATIONS_MD: &str = include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/api-operations.md" +); +const SKILL_AUTHENTICATION_AND_SAFETY_MD: &str = include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/authentication-and-safety.md" +); +const SKILL_REQUESTS_AND_OUTPUTS_MD: &str = include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md" +); +const USAGE_URI: &str = "genarrative://external-editor/usage"; +const OPENAPI_URI: &str = "genarrative://external-editor/openapi"; +const SKILL_URI: &str = "genarrative://external-editor/skill"; +const SKILL_CAPABILITY_ROUTING_URI: &str = + "genarrative://external-editor/skill/references/capability-routing.md"; +const SKILL_API_OPERATIONS_URI: &str = + "genarrative://external-editor/skill/references/api-operations.md"; +const SKILL_AUTHENTICATION_AND_SAFETY_URI: &str = + "genarrative://external-editor/skill/references/authentication-and-safety.md"; +const SKILL_REQUESTS_AND_OUTPUTS_URI: &str = + "genarrative://external-editor/skill/references/requests-and-outputs.md"; +const MAX_MCP_REST_RESPONSE_BYTES: usize = 4 * 1024 * 1024; + +const MCP_INSTRUCTIONS: &str = r#"陶泥儿外部编辑器工具。先创建或复用画布项目，并创建与画布同名的素材文件夹；生成结果应同时写入画布和素材库。参考本地文件时先走上传票据和对象确认，不要把 Data URL、Blob URL 或临时签名 URL写入生成参数。所有生成工具都是异步提交：必须提供 idempotencyKey，提交后按 pollAfterMs 调用 get_external_editor_generation_job，只有 status=completed 时消费 result；查询超时不能重新提交。warning 表示主结果可用但存在降级，sliceWarning 表示完整透明图集可用但切片未完成。详细说明、OpenAPI、Skill 主入口和分主题 references 见 resources/list；需要本地文件编排或不支持 MCP 时再下载 skill.zip。"#; + +#[derive(Clone, Debug)] +struct McpOperation { + tool_name: String, + operation_id: String, + method: Method, + path_template: String, + description: String, + input_schema: Arc>, + requires_idempotency_key: bool, +} + +static MCP_OPERATIONS: LazyLock> = LazyLock::new(build_mcp_operations); + +#[derive(Clone, Debug, Default)] +pub(crate) struct GenarrativeExternalMcp; + +pub(crate) type GenarrativeExternalMcpService = + StreamableHttpService; + +pub(crate) fn service() -> GenarrativeExternalMcpService { + let config = StreamableHttpServerConfig::default() + .with_stateful_mode(false) + .with_json_response(true) + .with_sse_keep_alive(None) + .with_allowed_hosts([ + "www.genarrative.world", + "genarrative.world", + "localhost", + "127.0.0.1", + "::1", + ]) + .with_allowed_origins([ + "https://www.genarrative.world", + "https://genarrative.world", + "http://localhost:3000", + "http://127.0.0.1:3000", + ]); + StreamableHttpService::new( + || Ok(GenarrativeExternalMcp), + Arc::new(LocalSessionManager::default()), + config, + ) +} + +impl ServerHandler for GenarrativeExternalMcp { + fn get_info(&self) -> ServerInfo { + ServerInfo::new( + ServerCapabilities::builder() + .enable_tools() + .enable_resources() + .build(), + ) + .with_server_info( + Implementation::new("genarrative-external-editor", env!("CARGO_PKG_VERSION")) + .with_title("陶泥儿外部编辑器") + .with_description("通过托管式 MCP 使用陶泥儿画布、素材库和异步生成 API") + .with_website_url("https://www.genarrative.world"), + ) + .with_instructions(MCP_INSTRUCTIONS) + } + + async fn list_tools( + &self, + _request: Option, + _context: McpRequestContext, + ) -> Result { + Ok(ListToolsResult::with_all_items( + MCP_OPERATIONS.iter().map(mcp_operation_tool).collect(), + )) + } + + fn get_tool(&self, name: &str) -> Option { + MCP_OPERATIONS + .iter() + .find(|operation| operation.tool_name == name) + .map(mcp_operation_tool) + } + + async fn call_tool( + &self, + request: CallToolRequestParams, + context: McpRequestContext, + ) -> Result { + let operation = MCP_OPERATIONS + .iter() + .find(|operation| operation.tool_name == request.name.as_ref()) + .ok_or_else(|| ErrorData::invalid_params("未知的陶泥儿外部 API 工具", None))?; + let arguments = request.arguments.unwrap_or_default(); + match dispatch_operation(operation, arguments, &context).await { + Ok(value) => Ok(CallToolResult::structured(value)), + Err(value) => Ok(CallToolResult::structured_error(value)), + } + } + + async fn list_resources( + &self, + _request: Option, + _context: McpRequestContext, + ) -> Result { + Ok(ListResourcesResult::with_all_items(mcp_resources())) + } + + async fn read_resource( + &self, + request: ReadResourceRequestParams, + _context: McpRequestContext, + ) -> Result { + let (text, mime_type) = mcp_resource_contents(request.uri.as_str()) + .ok_or_else(|| ErrorData::resource_not_found("资源不存在", None))?; + Ok(ReadResourceResult::new(vec![ + ResourceContents::text(text, request.uri).with_mime_type(mime_type), + ])) + } +} + +fn mcp_resources() -> Vec { + vec![ + Resource::new(USAGE_URI, "usage") + .with_title("陶泥儿外部编辑器使用说明") + .with_description("画布、素材、上传、异步生成和告警处理工作流") + .with_mime_type("text/markdown"), + Resource::new(OPENAPI_URI, "openapi") + .with_title("陶泥儿外部编辑器 OpenAPI") + .with_description("MCP 工具所映射的完整 REST 契约") + .with_mime_type("application/json"), + Resource::new(SKILL_URI, "skill") + .with_title("陶泥儿外部编辑器 Skill") + .with_description("外部编辑器 Skill 主入口；细节按 references 渐进读取") + .with_mime_type("text/markdown"), + Resource::new(SKILL_CAPABILITY_ROUTING_URI, "skill-capability-routing") + .with_title("陶泥儿外部编辑器能力路由") + .with_description("按用户意图选择 MCP tool 或 External v1 API") + .with_mime_type("text/markdown"), + Resource::new(SKILL_API_OPERATIONS_URI, "skill-api-operations") + .with_title("陶泥儿外部编辑器 API 操作") + .with_description("项目、素材、上传、异步生成和任务查询操作表") + .with_mime_type("text/markdown"), + Resource::new( + SKILL_AUTHENTICATION_AND_SAFETY_URI, + "skill-authentication-and-safety", + ) + .with_title("陶泥儿外部编辑器认证与安全") + .with_description("API Key、幂等、重试、本地文件和安全边界") + .with_mime_type("text/markdown"), + Resource::new(SKILL_REQUESTS_AND_OUTPUTS_URI, "skill-requests-and-outputs") + .with_title("陶泥儿外部编辑器请求与输出") + .with_description("请求构造、异步轮询、完成结果和告警处理") + .with_mime_type("text/markdown"), + ] +} + +fn mcp_resource_contents(uri: &str) -> Option<(&'static str, &'static str)> { + match uri { + USAGE_URI => Some((MCP_INSTRUCTIONS, "text/markdown")), + OPENAPI_URI => Some((OPENAPI_JSON, "application/json")), + SKILL_URI => Some((SKILL_MD, "text/markdown")), + SKILL_CAPABILITY_ROUTING_URI => Some((SKILL_CAPABILITY_ROUTING_MD, "text/markdown")), + SKILL_API_OPERATIONS_URI => Some((SKILL_API_OPERATIONS_MD, "text/markdown")), + SKILL_AUTHENTICATION_AND_SAFETY_URI => { + Some((SKILL_AUTHENTICATION_AND_SAFETY_MD, "text/markdown")) + } + SKILL_REQUESTS_AND_OUTPUTS_URI => Some((SKILL_REQUESTS_AND_OUTPUTS_MD, "text/markdown")), + _ => None, + } +} + +fn build_mcp_operations() -> Vec { + let openapi: Value = serde_json::from_str(OPENAPI_JSON).expect("embedded OpenAPI must parse"); + let mut operations = Vec::new(); + let Some(paths) = openapi.get("paths").and_then(Value::as_object) else { + return operations; + }; + for (path, path_item) in paths { + let Some(path_item) = path_item.as_object() else { + continue; + }; + for method_name in ["get", "post", "patch", "put", "delete"] { + let Some(operation) = path_item.get(method_name).and_then(Value::as_object) else { + continue; + }; + if operation.get("x-mcp-excluded").and_then(Value::as_bool) == Some(true) { + continue; + } + let Some(operation_id) = operation.get("operationId").and_then(Value::as_str) else { + continue; + }; + let method = Method::from_bytes(method_name.to_ascii_uppercase().as_bytes()) + .expect("known HTTP method"); + let requires_idempotency_key = matches!( + operation_id, + "generateExternalEditorImage" + | "editExternalEditorImage" + | "generateExternalEditorIconSpritesheet" + | "extractExternalEditorUiDesignAssets" + | "generateExternalEditorCharacterAnimation" + | "generateExternalEditorVideo" + | "generateExternalEditorSoundEffect" + | "generateExternalEditorBackgroundMusic" + ); + let description = operation + .get("description") + .or_else(|| operation.get("summary")) + .and_then(Value::as_str) + .unwrap_or("调用陶泥儿外部编辑器 API"); + operations.push(McpOperation { + tool_name: camel_to_snake(operation_id), + operation_id: operation_id.to_string(), + method, + path_template: path.clone(), + description: format!("{description}（{} {path}）", method_name.to_uppercase()), + input_schema: Arc::new(build_operation_input_schema( + &openapi, + path_item, + operation, + requires_idempotency_key, + )), + requires_idempotency_key, + }); + } + } + operations.sort_by(|left, right| left.tool_name.cmp(&right.tool_name)); + operations +} + +fn build_operation_input_schema( + openapi: &Value, + path_item: &Map, + operation: &Map, + requires_idempotency_key: bool, +) -> Map { + let mut properties = Map::new(); + let parameters = path_item + .get("parameters") + .and_then(Value::as_array) + .into_iter() + .flatten() + .chain( + operation + .get("parameters") + .and_then(Value::as_array) + .into_iter() + .flatten(), + ) + .filter_map(|parameter| resolve_openapi_reference(openapi, parameter)) + .collect::>(); + let mut top_level_required = Vec::new(); + for location in ["path", "query"] { + let mut parameter_properties = Map::new(); + let mut required = Vec::new(); + for parameter in ¶meters { + if parameter.get("in").and_then(Value::as_str) != Some(location) { + continue; + } + let Some(name) = parameter.get("name").and_then(Value::as_str) else { + continue; + }; + parameter_properties.insert( + name.to_string(), + parameter + .get("schema") + .cloned() + .unwrap_or_else(|| json!({})), + ); + if parameter.get("required").and_then(Value::as_bool) == Some(true) { + required.push(Value::String(name.to_string())); + } + } + if !parameter_properties.is_empty() { + let mut schema = json!({ + "type": "object", + "properties": parameter_properties, + "additionalProperties": false, + }); + if !required.is_empty() { + schema["required"] = Value::Array(required); + top_level_required.push(Value::String(format!("{location}Parameters"))); + } + properties.insert(format!("{location}Parameters"), schema); + } + } + if let Some(request_body) = operation + .get("requestBody") + .and_then(|value| resolve_openapi_reference(openapi, value)) + { + let body_schema = request_body + .get("content") + .and_then(|content| content.get("application/json")) + .and_then(|media_type| media_type.get("schema")) + .map(|schema| inline_openapi_schema(openapi, schema, 0)) + .unwrap_or_else(|| { + json!({ + "type": "object", + "description": "请求体。精确字段、枚举和约束见 genarrative://external-editor/openapi。", + "additionalProperties": true, + }) + }); + properties.insert("body".to_string(), body_schema); + if request_body.get("required").and_then(Value::as_bool) == Some(true) { + top_level_required.push(json!("body")); + } + } + if requires_idempotency_key { + properties.insert( + "idempotencyKey".to_string(), + json!({ + "type": "string", + "minLength": 1, + "maxLength": 128, + "description": "本次逻辑生成请求的稳定幂等键；结果不确定时必须复用原值。" + }), + ); + top_level_required.push(json!("idempotencyKey")); + } + let mut schema = Map::from_iter([ + ("type".to_string(), json!("object")), + ("properties".to_string(), Value::Object(properties)), + ("additionalProperties".to_string(), json!(false)), + ]); + if !top_level_required.is_empty() { + top_level_required.sort_by(|left, right| left.as_str().cmp(&right.as_str())); + top_level_required.dedup(); + schema.insert("required".to_string(), Value::Array(top_level_required)); + } + schema +} + +fn resolve_openapi_reference<'a>(openapi: &'a Value, value: &'a Value) -> Option<&'a Value> { + let Some(reference) = value.get("$ref").and_then(Value::as_str) else { + return Some(value); + }; + let pointer = reference.strip_prefix('#')?; + openapi.pointer(pointer) +} + +fn inline_openapi_schema(openapi: &Value, schema: &Value, depth: usize) -> Value { + if depth >= 32 { + return json!({"type": "object"}); + } + if let Some(reference) = schema.get("$ref").and_then(Value::as_str) + && let Some(pointer) = reference.strip_prefix('#') + && let Some(resolved) = openapi.pointer(pointer) + { + return inline_openapi_schema(openapi, resolved, depth + 1); + } + match schema { + Value::Array(values) => Value::Array( + values + .iter() + .map(|value| inline_openapi_schema(openapi, value, depth + 1)) + .collect(), + ), + Value::Object(values) => Value::Object( + values + .iter() + .map(|(key, value)| { + ( + key.clone(), + inline_openapi_schema(openapi, value, depth + 1), + ) + }) + .collect(), + ), + value => value.clone(), + } +} + +fn mcp_operation_tool(operation: &McpOperation) -> Tool { + let read_only = operation.method == Method::GET; + let destructive = operation.method == Method::DELETE || operation.requires_idempotency_key; + let annotations = ToolAnnotations::new() + .read_only(read_only) + .destructive(destructive) + .idempotent(read_only || operation.requires_idempotency_key) + .open_world(operation.requires_idempotency_key); + let mut tool = Tool::new( + operation.tool_name.clone(), + operation.description.clone(), + operation.input_schema.clone(), + ); + tool.title = Some(operation.operation_id.clone()); + tool.annotations = Some(annotations); + tool +} + +async fn dispatch_operation( + operation: &McpOperation, + arguments: Map, + context: &McpRequestContext, +) -> Result { + let parts = context + .extensions + .get::() + .ok_or_else(|| json!({"error": "MCP HTTP 请求上下文缺失"}))?; + let state = parts + .extensions + .get::() + .cloned() + .ok_or_else(|| json!({"error": "MCP 应用状态缺失"}))?; + let request_context = parts + .extensions + .get::() + .cloned() + .ok_or_else(|| json!({"error": "MCP request_id 上下文缺失"}))?; + let authorization = parts + .headers + .get(AUTHORIZATION) + .cloned() + .ok_or_else(|| json!({"error": "Authorization 请求头缺失"}))?; + + let mut path = operation.path_template.clone(); + if let Some(path_parameters) = arguments.get("pathParameters").and_then(Value::as_object) { + for (name, value) in path_parameters { + let value = json_scalar_string(value) + .ok_or_else(|| json!({"error": format!("路径参数 {name} 必须是标量")}))?; + path = path.replace(&format!("{{{name}}}"), urlencoding::encode(&value).as_ref()); + } + } + if path.contains('{') { + return Err(json!({"error": "缺少必填路径参数"})); + } + if let Some(query) = arguments.get("queryParameters").and_then(Value::as_object) { + let mut serializer = url::form_urlencoded::Serializer::new(String::new()); + for (name, value) in query { + match value { + Value::Array(values) => { + for value in values { + if let Some(value) = json_scalar_string(value) { + serializer.append_pair(name, &value); + } + } + } + value => { + if let Some(value) = json_scalar_string(value) { + serializer.append_pair(name, &value); + } + } + } + } + let query = serializer.finish(); + if !query.is_empty() { + path.push('?'); + path.push_str(&query); + } + } + + let body = arguments.get("body").cloned().unwrap_or(Value::Null); + let body = if body.is_null() { + Body::empty() + } else { + Body::from(body.to_string()) + }; + let mut request = Request::builder() + .method(operation.method.clone()) + .uri(path) + .header(AUTHORIZATION, authorization) + .body(body) + .map_err(|_| json!({"error": "无法构造内部 API 请求"}))?; + request.extensions_mut().insert(request_context); + if arguments.get("body").is_some() { + request.headers_mut().insert( + CONTENT_TYPE, + "application/json".parse().expect("valid content type"), + ); + } + if operation.requires_idempotency_key { + let idempotency_key = arguments + .get("idempotencyKey") + .and_then(Value::as_str) + .ok_or_else(|| json!({"error": "生成工具必须提供 idempotencyKey"}))?; + request.headers_mut().insert( + "idempotency-key", + idempotency_key + .parse() + .map_err(|_| json!({"error": "idempotencyKey 不是合法 HTTP 头值"}))?, + ); + } + + let response = modules::external_api::router(state.clone()) + .with_state(state) + .oneshot(request) + .await + .unwrap_or_else(|never| match never {}); + let status = response.status(); + let bytes = response + .into_body() + .collect() + .await + .map_err(|_| json!({"error": "读取外部 API 响应失败"}))? + .to_bytes(); + if bytes.len() > MAX_MCP_REST_RESPONSE_BYTES { + return Err(json!({"error": "外部 API 响应超过 MCP 返回上限"})); + } + let payload = serde_json::from_slice::(&bytes).unwrap_or_else(|_| { + json!({ + "status": status.as_u16(), + "message": "外部 API 返回了非 JSON 响应" + }) + }); + if status.is_success() { + Ok(unwrap_external_api_success_payload(payload)) + } else { + Err(json!({ + "status": status.as_u16(), + "response": payload, + })) + } +} + +fn unwrap_external_api_success_payload(payload: Value) -> Value { + payload + .get("data") + .filter(|_| payload.get("ok").and_then(Value::as_bool) == Some(true)) + .cloned() + .unwrap_or(payload) +} + +fn json_scalar_string(value: &Value) -> Option { + match value { + Value::String(value) => Some(value.clone()), + Value::Number(value) => Some(value.to_string()), + Value::Bool(value) => Some(value.to_string()), + Value::Null | Value::Array(_) | Value::Object(_) => None, + } +} + +fn camel_to_snake(value: &str) -> String { + let mut output = String::with_capacity(value.len() + 8); + for (index, character) in value.chars().enumerate() { + if character.is_ascii_uppercase() { + if index > 0 { + output.push('_'); + } + output.push(character.to_ascii_lowercase()); + } else { + output.push(character); + } + } + output +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::{config::AppConfig, request_context::attach_request_context}; + use axum::{ + http::{ + StatusCode, + header::{ACCEPT, HOST}, + }, + middleware, + }; + + #[test] + fn mcp_resources_expose_complete_progressive_skill_documents() { + let resources = + serde_json::to_string(&mcp_resources()).expect("resources should serialize"); + let expected = [ + (USAGE_URI, MCP_INSTRUCTIONS, "text/markdown"), + (OPENAPI_URI, OPENAPI_JSON, "application/json"), + (SKILL_URI, SKILL_MD, "text/markdown"), + ( + SKILL_CAPABILITY_ROUTING_URI, + SKILL_CAPABILITY_ROUTING_MD, + "text/markdown", + ), + ( + SKILL_API_OPERATIONS_URI, + SKILL_API_OPERATIONS_MD, + "text/markdown", + ), + ( + SKILL_AUTHENTICATION_AND_SAFETY_URI, + SKILL_AUTHENTICATION_AND_SAFETY_MD, + "text/markdown", + ), + ( + SKILL_REQUESTS_AND_OUTPUTS_URI, + SKILL_REQUESTS_AND_OUTPUTS_MD, + "text/markdown", + ), + ]; + assert_eq!(mcp_resources().len(), expected.len()); + for (uri, contents, mime_type) in expected { + assert!(resources.contains(uri), "missing MCP resource {uri}"); + assert_eq!(mcp_resource_contents(uri), Some((contents, mime_type))); + } + } + + #[test] + fn openapi_operations_become_unique_mcp_tools() { + let names: std::collections::BTreeMap<_, _> = MCP_OPERATIONS + .iter() + .map(|operation| { + ( + operation.tool_name.as_str(), + operation.path_template.as_str(), + ) + }) + .collect(); + assert_eq!(names.len(), MCP_OPERATIONS.len()); + assert!(names.contains_key("generate_external_editor_image")); + assert!(names.contains_key("get_external_editor_generation_job")); + + let list_projects = MCP_OPERATIONS + .iter() + .find(|operation| operation.tool_name == "list_editor_projects") + .expect("project list tool should exist"); + assert_eq!(list_projects.method, Method::GET); + + let create_project = MCP_OPERATIONS + .iter() + .find(|operation| operation.tool_name == "create_editor_project") + .expect("project create tool should exist"); + assert_eq!(create_project.method, Method::POST); + } + + #[test] + fn generation_tools_require_idempotency_key() { + let operation = MCP_OPERATIONS + .iter() + .find(|operation| operation.tool_name == "generate_external_editor_image") + .expect("image generation tool should exist"); + assert!(operation.requires_idempotency_key); + assert_eq!( + operation.input_schema.get("required"), + Some(&json!(["body", "idempotencyKey"])) + ); + assert_eq!( + operation.input_schema["properties"]["body"]["properties"]["projectId"]["type"], + json!(["string", "null"]) + ); + } + + #[test] + fn referenced_path_parameters_are_exposed_to_agents() { + let operation = MCP_OPERATIONS + .iter() + .find(|operation| operation.tool_name == "get_editor_project") + .expect("project lookup tool should exist"); + assert_eq!( + operation.input_schema["required"], + json!(["pathParameters"]) + ); + assert_eq!( + operation.input_schema["properties"]["pathParameters"]["required"], + json!(["projectId"]) + ); + } + + #[test] + fn tool_catalog_has_self_contained_bounded_schemas() { + let tools = MCP_OPERATIONS + .iter() + .map(mcp_operation_tool) + .collect::>(); + let serialized = serde_json::to_vec(&tools).expect("tool catalog should serialize"); + assert!(serialized.len() < 512 * 1024); + for operation in MCP_OPERATIONS.iter() { + let serialized = serde_json::to_string(&operation.input_schema) + .expect("tool input schema should serialize"); + assert!(serialized.len() < 64 * 1024, "{}", operation.tool_name); + assert!(!serialized.contains("\"$ref\""), "{}", operation.tool_name); + assert_eq!(operation.input_schema.get("type"), Some(&json!("object"))); + } + } + + #[test] + fn openapi_documents_mcp_authentication_guide() { + let openapi: Value = + serde_json::from_str(OPENAPI_JSON).expect("external OpenAPI should parse"); + let unauthorized = &openapi["paths"]["/api/external/v1/mcp"]["post"]["responses"]["401"]; + assert_eq!( + unauthorized["headers"]["WWW-Authenticate"]["schema"]["const"], + json!("Bearer realm=\"genarrative-external-editor\"") + ); + assert_eq!( + unauthorized["content"]["application/json"]["schema"]["$ref"], + json!("#/components/schemas/McpAuthenticationGuideResponse") + ); + let guide = &openapi["components"]["schemas"]["McpAuthenticationGuideResponse"]["properties"] + ["error"]["properties"]["details"]["properties"]["guide"]; + assert_eq!( + guide["properties"]["reason"]["const"], + json!("MCP_AUTHENTICATION_REQUIRED") + ); + assert_eq!( + guide["properties"]["action"]["const"], + json!("CONFIGURE_BEARER_API_KEY") + ); + } + + #[test] + fn mcp_tools_return_business_data_without_rest_envelope() { + assert_eq!( + unwrap_external_api_success_payload(json!({ + "ok": true, + "data": {"operationId": "task-1", "status": "queued"}, + "meta": {"requestId": "request-1"} + })), + json!({"operationId": "task-1", "status": "queued"}) + ); + } + + #[tokio::test] + async fn streamable_http_initialize_is_stateless_json() { + let request = Request::builder() + .method(Method::POST) + .uri("/api/external/v1/mcp") + .header(HOST, "localhost") + .header(CONTENT_TYPE, "application/json") + .header(ACCEPT, "application/json, text/event-stream") + .body(Body::from( + r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-11-25","capabilities":{},"clientInfo":{"name":"test-agent","version":"1.0"}}}"#, + )) + .expect("initialize request should build"); + + let response = service() + .oneshot(request) + .await + .expect("MCP service should be infallible"); + assert_eq!(response.status(), StatusCode::OK); + assert!(response.headers().get("mcp-session-id").is_none()); + assert_eq!( + response + .headers() + .get(CONTENT_TYPE) + .and_then(|value| value.to_str().ok()), + Some("application/json") + ); + let payload: Value = serde_json::from_slice( + &response + .into_body() + .collect() + .await + .expect("initialize response body should read") + .to_bytes(), + ) + .expect("initialize response should be JSON"); + assert_eq!(payload["id"], json!(1)); + assert_eq!( + payload["result"]["serverInfo"]["name"], + json!("genarrative-external-editor") + ); + assert!(payload["result"]["capabilities"]["tools"].is_object()); + assert!(payload["result"]["capabilities"]["resources"].is_object()); + assert!( + payload["result"]["instructions"] + .as_str() + .is_some_and(|value| value.contains("异步提交")) + ); + + for (method, assertion) in [ + ("tools/list", "generate_external_editor_image"), + ("resources/list", USAGE_URI), + ] { + let request = Request::builder() + .method(Method::POST) + .uri("/api/external/v1/mcp") + .header(HOST, "localhost") + .header(CONTENT_TYPE, "application/json") + .header(ACCEPT, "application/json, text/event-stream") + .header("mcp-protocol-version", "2025-11-25") + .body(Body::from( + json!({"jsonrpc": "2.0", "id": 2, "method": method}).to_string(), + )) + .expect("catalog request should build"); + let response = service() + .oneshot(request) + .await + .expect("MCP service should be infallible"); + assert_eq!(response.status(), StatusCode::OK, "{method}"); + let body = response + .into_body() + .collect() + .await + .expect("catalog response body should read") + .to_bytes(); + let payload: Value = + serde_json::from_slice(&body).expect("catalog response should be JSON"); + assert!( + payload["result"].to_string().contains(assertion), + "{method}" + ); + } + + for (id, uri, expected_text) in [ + (3, OPENAPI_URI, "陶泥儿外部编辑器 OpenAPI"), + ( + 4, + SKILL_REQUESTS_AND_OUTPUTS_URI, + "All eight generation POST routes require", + ), + ] { + let request = Request::builder() + .method(Method::POST) + .uri("/api/external/v1/mcp") + .header(HOST, "localhost") + .header(CONTENT_TYPE, "application/json") + .header(ACCEPT, "application/json, text/event-stream") + .header("mcp-protocol-version", "2025-11-25") + .body(Body::from( + json!({ + "jsonrpc": "2.0", + "id": id, + "method": "resources/read", + "params": {"uri": uri} + }) + .to_string(), + )) + .expect("resource read request should build"); + let response = service() + .oneshot(request) + .await + .expect("MCP service should be infallible"); + assert_eq!(response.status(), StatusCode::OK); + let payload: Value = serde_json::from_slice( + &response + .into_body() + .collect() + .await + .expect("resource response body should read") + .to_bytes(), + ) + .expect("resource response should be JSON"); + assert!( + payload["result"]["contents"][0]["text"] + .as_str() + .is_some_and(|value| value.contains(expected_text)), + "{uri}" + ); + } + } + + #[tokio::test] + async fn mounted_mcp_route_requires_external_api_key() { + let state = AppState::new(AppConfig::default()).expect("test state should build"); + let mut errors = Vec::new(); + for authorization in [None, Some("Basic not-a-bearer-token")] { + let mut request = Request::builder() + .method(Method::POST) + .uri("/api/external/v1/mcp") + .header(HOST, "localhost") + .header("x-request-id", "mcp-auth-guide-test") + .header(CONTENT_TYPE, "application/json") + .header(ACCEPT, "application/json, text/event-stream"); + if let Some(authorization) = authorization { + request = request.header(AUTHORIZATION, authorization); + } + let request = request + .body(Body::from( + r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-11-25","capabilities":{},"clientInfo":{"name":"test-agent","version":"1.0"}}}"#, + )) + .expect("initialize request should build"); + let response = modules::external_api::router(state.clone()) + .with_state(state.clone()) + .layer(middleware::from_fn(attach_request_context)) + .oneshot(request) + .await + .expect("external router should be infallible"); + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); + assert!(response.headers().get("mcp-session-id").is_none()); + assert!( + response + .headers() + .get(CONTENT_TYPE) + .and_then(|value| value.to_str().ok()) + .is_some_and(|value| value.starts_with("application/json")) + ); + assert_eq!( + response + .headers() + .get("www-authenticate") + .and_then(|value| value.to_str().ok()), + Some("Bearer realm=\"genarrative-external-editor\"") + ); + let payload: Value = serde_json::from_slice( + &response + .into_body() + .collect() + .await + .expect("authentication guide should read") + .to_bytes(), + ) + .expect("authentication guide should be JSON"); + assert_eq!(payload["error"]["code"], json!("UNAUTHORIZED")); + assert_eq!( + payload["error"]["message"], + json!("连接陶泥儿托管 MCP 需要开发者 API Key") + ); + assert_eq!( + payload["error"]["details"]["guide"]["reason"], + json!("MCP_AUTHENTICATION_REQUIRED") + ); + assert_eq!( + payload["error"]["details"]["guide"]["action"], + json!("CONFIGURE_BEARER_API_KEY") + ); + assert_eq!( + payload["error"]["details"]["guide"]["authentication"]["valueFormat"], + json!("Bearer ") + ); + assert_eq!( + payload["error"]["details"]["guide"]["publicDiscovery"]["manifest"], + json!("/api/external/v1/agent-integration.json") + ); + assert_eq!( + payload["error"]["details"]["guide"]["credentialSafety"]["neverPasteIntoChat"], + json!(true) + ); + assert_eq!(payload["meta"]["requestId"], json!("mcp-auth-guide-test")); + assert_eq!( + payload["meta"]["operation"], + json!("POST /api/external/v1/mcp") + ); + let error = payload["error"].clone(); + let serialized = payload.to_string(); + assert!(!serialized.contains("tools")); + assert!(!serialized.contains("resources")); + assert!(!serialized.contains("provider")); + assert!(!serialized.contains("procedure")); + assert!(!serialized.contains("owner")); + assert!(!serialized.contains("SENSITIVE_KEY_LURE")); + errors.push(error); + } + assert_eq!(errors[0], errors[1]); + } +} diff --git a/server-rs/crates/api-server/src/external_skill_api.rs b/server-rs/crates/api-server/src/external_skill_api.rs new file mode 100644 index 000000000..2472b90ea --- /dev/null +++ b/server-rs/crates/api-server/src/external_skill_api.rs @@ -0,0 +1,180 @@ +use std::io::{Cursor, Write}; + +use axum::{ + Json, + body::Body, + http::{ + HeaderValue, StatusCode, + header::{CONTENT_DISPOSITION, CONTENT_TYPE}, + }, + response::{IntoResponse, Response}, +}; +use serde_json::{Value, json}; +use sha2::{Digest, Sha256}; +use zip::{ZipWriter, write::SimpleFileOptions}; + +use crate::http_error::AppError; + +const SKILL_ROOT: &str = "genarrative-external-editor-api"; +const SKILL_FILES: [(&str, &str); 7] = [ + ( + "SKILL.md", + include_str!("../../../../.codex/skills/genarrative-external-editor-api/SKILL.md"), + ), + ( + "references/capability-routing.md", + include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/capability-routing.md" + ), + ), + ( + "references/api-operations.md", + include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/api-operations.md" + ), + ), + ( + "references/authentication-and-safety.md", + include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/authentication-and-safety.md" + ), + ), + ( + "references/requests-and-outputs.md", + include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md" + ), + ), + ( + "scripts/genarrative_external_api.py", + include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py" + ), + ), + ( + "agents/openai.yaml", + include_str!( + "../../../../.codex/skills/genarrative-external-editor-api/agents/openai.yaml" + ), + ), +]; + +pub async fn get_external_skill_entry() -> Response { + let mut response = Body::from(SKILL_FILES[0].1).into_response(); + response.headers_mut().insert( + CONTENT_TYPE, + HeaderValue::from_static("text/markdown; charset=utf-8"), + ); + response +} + +pub async fn download_external_skill_archive() -> Result { + let bytes = build_external_skill_archive()?; + let mut response = Body::from(bytes).into_response(); + response + .headers_mut() + .insert(CONTENT_TYPE, HeaderValue::from_static("application/zip")); + response.headers_mut().insert( + CONTENT_DISPOSITION, + HeaderValue::from_static( + "attachment; filename=\"genarrative-external-editor-api.skill.zip\"", + ), + ); + Ok(response) +} + +pub async fn get_external_agent_integration_manifest() -> Result, AppError> { + let archive = build_external_skill_archive()?; + let sha256 = format!("{:x}", Sha256::digest(&archive)); + Ok(Json(json!({ + "name": SKILL_ROOT, + "version": env!("CARGO_PKG_VERSION"), + "mcp": { + "transport": "streamable-http", + "url": "/api/external/v1/mcp", + "authentication": "bearer-api-key", + "credentialSetup": { + "action": "CONFIGURE_BEARER_API_KEY", + "authorizationValueFormat": "Bearer ", + "navigationLabel": "开发者 API Key", + "guide": "/api/external/v1/skill/SKILL.md" + } + }, + "openapi": "/api/external/v1/openapi.json", + "skill": { + "entry": "/api/external/v1/skill/SKILL.md", + "archive": "/api/external/v1/skill.zip", + "archiveSha256": sha256, + "files": SKILL_FILES.map(|(path, _)| format!("{SKILL_ROOT}/{path}")), + } + }))) +} + +fn build_external_skill_archive() -> Result, AppError> { + let cursor = Cursor::new(Vec::new()); + let mut archive = ZipWriter::new(cursor); + let options = SimpleFileOptions::default().unix_permissions(0o644); + for (path, contents) in SKILL_FILES { + archive + .start_file(format!("{SKILL_ROOT}/{path}"), options) + .map_err(skill_archive_error)?; + archive + .write_all(contents.as_bytes()) + .map_err(skill_archive_error)?; + } + archive + .finish() + .map(Cursor::into_inner) + .map_err(skill_archive_error) +} + +fn skill_archive_error(error: impl std::fmt::Display) -> AppError { + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "external-skill-archive", + "message": format!("构建外部 Skill 包失败：{error}"), + })) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn archive_contains_complete_skill_bundle() { + let bytes = build_external_skill_archive().expect("skill archive should build"); + let mut archive = zip::ZipArchive::new(Cursor::new(bytes)).expect("archive should parse"); + assert_eq!(archive.len(), SKILL_FILES.len()); + for (path, contents) in SKILL_FILES { + let name = format!("{SKILL_ROOT}/{path}"); + let mut file = archive.by_name(&name).expect("skill file should exist"); + let mut actual = String::new(); + std::io::Read::read_to_string(&mut file, &mut actual).expect("skill file should read"); + assert_eq!(actual, contents); + } + } + + #[tokio::test] + async fn integration_manifest_matches_complete_skill_archive() { + let bytes = build_external_skill_archive().expect("skill archive should build"); + let Json(manifest) = get_external_agent_integration_manifest() + .await + .expect("integration manifest should build"); + let expected_files = SKILL_FILES + .map(|(path, _)| format!("{SKILL_ROOT}/{path}")) + .to_vec(); + assert_eq!(manifest["skill"]["files"], json!(expected_files)); + assert_eq!( + manifest["mcp"]["credentialSetup"], + json!({ + "action": "CONFIGURE_BEARER_API_KEY", + "authorizationValueFormat": "Bearer ", + "navigationLabel": "开发者 API Key", + "guide": "/api/external/v1/skill/SKILL.md" + }) + ); + assert_eq!( + manifest["skill"]["archiveSha256"], + json!(format!("{:x}", Sha256::digest(&bytes))) + ); + } +} diff --git a/server-rs/crates/api-server/src/main.rs b/server-rs/crates/api-server/src/main.rs index 4f2b485dc..80872cf47 100644 --- a/server-rs/crates/api-server/src/main.rs +++ b/server-rs/crates/api-server/src/main.rs @@ -37,6 +37,8 @@ mod external_editor_api; mod external_generation; mod external_generation_worker; mod external_generation_worker_controller; +mod external_mcp; +mod external_skill_api; mod frontend_runtime_config; mod generated_image_assets; mod health; diff --git a/server-rs/crates/api-server/src/modules/external_api.rs b/server-rs/crates/api-server/src/modules/external_api.rs index a3341c2ee..ac2fbf362 100644 --- a/server-rs/crates/api-server/src/modules/external_api.rs +++ b/server-rs/crates/api-server/src/modules/external_api.rs @@ -1,5 +1,5 @@ use axum::{ - Router, + Extension, Router, extract::DefaultBodyLimit, middleware, routing::{get, patch, post}, @@ -7,7 +7,7 @@ use axum::{ use crate::{ editor_project::EDITOR_LAYOUT_REQUEST_BODY_MAX_BYTES, - external_api_auth::require_external_api_key, + external_api_auth::{require_external_api_key, require_external_mcp_api_key}, external_assets_api::{ confirm_external_asset_object, create_external_direct_upload_ticket, get_external_asset_read_url, @@ -21,17 +21,43 @@ use crate::{ generate_external_editor_character_animation, generate_external_editor_icon_spritesheet, generate_external_editor_image, generate_external_editor_sound_effect, generate_external_editor_video, get_external_editor_asset_library, - get_external_editor_project, list_external_editor_projects, - load_recent_external_editor_project, openapi_json, rename_external_editor_project, - save_external_editor_canvas, update_external_editor_asset, + get_external_editor_generation_job, get_external_editor_project, + list_external_editor_projects, load_recent_external_editor_project, openapi_json, + rename_external_editor_project, save_external_editor_canvas, update_external_editor_asset, update_external_editor_asset_folder, }, + external_mcp, + external_skill_api::{ + download_external_skill_archive, get_external_agent_integration_manifest, + get_external_skill_entry, + }, state::AppState, }; pub fn router(state: AppState) -> Router { + let mcp_router = Router::new() + .nest_service("/api/external/v1/mcp", external_mcp::service()) + .layer(Extension(state.clone())) + .route_layer(middleware::from_fn_with_state( + state.clone(), + require_external_mcp_api_key, + )); + Router::new() + .merge(mcp_router) .route("/api/external/v1/openapi.json", get(openapi_json)) + .route( + "/api/external/v1/agent-integration.json", + get(get_external_agent_integration_manifest), + ) + .route( + "/api/external/v1/skill/SKILL.md", + get(get_external_skill_entry), + ) + .route( + "/api/external/v1/skill.zip", + get(download_external_skill_archive), + ) .route( "/api/external/v1/assets/direct-upload-tickets", post(create_external_direct_upload_ticket).route_layer(middleware::from_fn_with_state( @@ -139,6 +165,13 @@ pub fn router(state: AppState) -> Router { require_external_api_key, )), ) + .route( + "/api/external/v1/generations/{operation_id}", + get(get_external_editor_generation_job).route_layer(middleware::from_fn_with_state( + state.clone(), + require_external_api_key, + )), + ) .route( "/api/external/v1/editor/images/generations", post(generate_external_editor_image).route_layer(middleware::from_fn_with_state( diff --git a/server-rs/crates/api-server/src/vector_engine_audio_generation.rs b/server-rs/crates/api-server/src/vector_engine_audio_generation.rs index 1e6a698b8..9021cb656 100644 --- a/server-rs/crates/api-server/src/vector_engine_audio_generation.rs +++ b/server-rs/crates/api-server/src/vector_engine_audio_generation.rs @@ -6,7 +6,9 @@ mod publish; mod settings; mod types; -pub use generation::{generate_editor_background_music, generate_editor_sound_effect}; pub(crate) use generation::{ - generate_editor_background_music_for_owner, generate_editor_sound_effect_for_owner, + enqueue_editor_background_music_generation_for_owner, + enqueue_editor_sound_effect_generation_for_owner, generate_editor_background_music_for_owner, + generate_editor_sound_effect_for_owner, }; +pub use generation::{generate_editor_background_music, generate_editor_sound_effect}; diff --git a/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs b/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs index d02ffca32..6b1f77cb9 100644 --- a/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs +++ b/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs @@ -9,6 +9,7 @@ use platform_oss::LegacyAssetPrefix; use serde_json::Value; use serde_json::json; use shared_contracts::assets; +use spacetime_client::ExternalGenerationJobRecord; use crate::{ api_response::json_success_body, @@ -17,7 +18,7 @@ use crate::{ editor_generation_queue::{ EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, EditorGenerationQueuedResponse, editor_generation_queue_state, - editor_generation_source_entity_id, enqueue_editor_generation_job, + editor_generation_source_entity_id, enqueue_editor_generation_job_for_caller, }, editor_project::{ EditorCanvasGeneratedLayerInput, PersistEditorGeneratedAssetRequest, @@ -142,33 +143,14 @@ pub async fn generate_editor_sound_effect( let Json(payload) = parse_json_payload(&request_context, payload)?; let owner_user_id = authenticated.claims().user_id().to_string(); if !state.config.external_generation_mode.is_inline() { - let pricing = state.editor_generation_pricing().await.map_err(|error| { - AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR) - .with_details(json!({ - "provider": "editor-generation-pricing", - "message": error.to_string(), - })) - .into_response_with_context(Some(&request_context)) - })?; - let normalized = - normalize_editor_sound_effect_request_with_pricing(payload.clone(), &pricing) - .map_err(|error| error.into_response_with_context(Some(&request_context)))?; - let source_entity_id = editor_generation_source_entity_id( - payload.project_id.as_deref(), - "editor-sound-effect", - ); - let queue_job = enqueue_editor_generation_job( + let queue_job = enqueue_editor_sound_effect_generation_for_owner( &state, &request_context, owner_user_id.as_str(), - EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, - source_entity_id, - "图片画布生成音效", - u64::from(normalized.price_mud_points), - &payload, + payload, + None, ) - .await - .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + .await?; return Ok(json_success_body( Some(&request_context), EditorGenerationQueuedResponse { @@ -180,6 +162,40 @@ pub async fn generate_editor_sound_effect( .await } +pub(crate) async fn enqueue_editor_sound_effect_generation_for_owner( + state: &AppState, + request_context: &RequestContext, + owner_user_id: &str, + payload: assets::EditorSoundEffectGenerateRequest, + external_idempotency_key: Option<&str>, +) -> Result { + let pricing = state.editor_generation_pricing().await.map_err(|error| { + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR) + .with_details(json!({ + "provider": "editor-generation-pricing", + "message": error.to_string(), + })) + .into_response_with_context(Some(request_context)) + })?; + let normalized = normalize_editor_sound_effect_request_with_pricing(payload.clone(), &pricing) + .map_err(|error| error.into_response_with_context(Some(request_context)))?; + let source_entity_id = + editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-sound-effect"); + enqueue_editor_generation_job_for_caller( + state, + request_context, + owner_user_id, + EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成音效", + u64::from(normalized.price_mud_points), + &payload, + external_idempotency_key, + ) + .await + .map_err(|error| error.into_response_with_context(Some(request_context))) +} + pub(crate) async fn generate_editor_sound_effect_for_owner( state: AppState, request_context: RequestContext, @@ -358,33 +374,14 @@ pub async fn generate_editor_background_music( let Json(payload) = parse_json_payload(&request_context, payload)?; let owner_user_id = authenticated.claims().user_id().to_string(); if !state.config.external_generation_mode.is_inline() { - let pricing = state.editor_generation_pricing().await.map_err(|error| { - AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR) - .with_details(json!({ - "provider": "editor-generation-pricing", - "message": error.to_string(), - })) - .into_response_with_context(Some(&request_context)) - })?; - let normalized = - normalize_editor_background_music_request_with_pricing(payload.clone(), &pricing) - .map_err(|error| error.into_response_with_context(Some(&request_context)))?; - let source_entity_id = editor_generation_source_entity_id( - payload.project_id.as_deref(), - "editor-background-music", - ); - let queue_job = enqueue_editor_generation_job( + let queue_job = enqueue_editor_background_music_generation_for_owner( &state, &request_context, owner_user_id.as_str(), - EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, - source_entity_id, - "图片画布生成背景音乐", - u64::from(normalized.price_mud_points), - &payload, + payload, + None, ) - .await - .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + .await?; return Ok(json_success_body( Some(&request_context), EditorGenerationQueuedResponse { @@ -401,6 +398,43 @@ pub async fn generate_editor_background_music( .await } +pub(crate) async fn enqueue_editor_background_music_generation_for_owner( + state: &AppState, + request_context: &RequestContext, + owner_user_id: &str, + payload: assets::EditorBackgroundMusicGenerateRequest, + external_idempotency_key: Option<&str>, +) -> Result { + let pricing = state.editor_generation_pricing().await.map_err(|error| { + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR) + .with_details(json!({ + "provider": "editor-generation-pricing", + "message": error.to_string(), + })) + .into_response_with_context(Some(request_context)) + })?; + let normalized = + normalize_editor_background_music_request_with_pricing(payload.clone(), &pricing) + .map_err(|error| error.into_response_with_context(Some(request_context)))?; + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-background-music", + ); + enqueue_editor_generation_job_for_caller( + state, + request_context, + owner_user_id, + EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成背景音乐", + u64::from(normalized.price_mud_points), + &payload, + external_idempotency_key, + ) + .await + .map_err(|error| error.into_response_with_context(Some(request_context))) +} + pub(crate) async fn generate_editor_background_music_for_owner( state: AppState, request_context: RequestContext, diff --git a/server-rs/crates/platform-llm/README.md b/server-rs/crates/platform-llm/README.md index e379a8588..353b55661 100644 --- a/server-rs/crates/platform-llm/README.md +++ b/server-rs/crates/platform-llm/README.md @@ -30,7 +30,7 @@ | --- | --- | --- | --- | --- | | `OpenAiChat` | `tools[].type=function`，函数内为 `name` / `description` / `parameters` / `strict` | `"auto"` / `"required"` | `choices[0].message.tool_calls` | `delta.tool_calls[].index`；首片提供 id/name，后续拼接 arguments | | `OpenAiResponses` | `tools[].type=function`，函数内为 `name` / `description` / `parameters` / `strict` | `"auto"` / `"required"` | `output[].type=function_call` | `output_index`；`output_item.added` 提供身份，`function_call_arguments.delta` 拼接，`.done` 覆盖完整参数 | -| `Anthropic` | 顶层 `tools[]` 为 `name` / `description` / `input_schema`，没有 `function` 包装层和 `strict` | `{ "type": "auto" }` / `{ "type": "any" }`；`Required` 映射为 `any` | `content[].type=tool_use`，`input` 序列化为 `arguments` | content block `index`；`content_block_start` 提供身份，`input_json_delta` 拼接参数 | +| `Anthropic` | 顶层 `tools[]` 为 `name` / `description` / `input_schema`，没有 `function` 包装层；只有 endpoint/model 配置显式声明支持且 schema / 请求复杂度满足 Anthropic 当前边界时才发送 `strict: true`。strict 传输 schema 会剥离不支持的约束，调用方原 schema 保持不变；最后一项带 ephemeral cache breakpoint | `{ "type": "auto" }` / `{ "type": "any" }`；`Required` 映射为 `any` | `content[].type=tool_use`，`input` 序列化为 `arguments` | content block `index`；`content_block_start` 提供身份，`input_json_delta` 拼接参数；`message_start/message_delta` 合并 cache/input/output usage | Responses 如果只发送 `response.completed` 或 `response.incomplete`，解析器会从其中的 `response.output[]` 恢复 `function_call`；恢复时使用 output 数组下标作为 slot。`response.incomplete` 表示上游没有完成本轮生成：其中的工具调用即使参数是完整 JSON 也返回 `Deserialize`，纯正文则保留为可用的降级结果。三种协议的并行工具调用只在平台层做 slot 聚合，不代表工具会在平台层并发执行。 diff --git a/server-rs/crates/platform-llm/src/lib.rs b/server-rs/crates/platform-llm/src/lib.rs index 207614051..51857995b 100644 --- a/server-rs/crates/platform-llm/src/lib.rs +++ b/server-rs/crates/platform-llm/src/lib.rs @@ -59,6 +59,7 @@ pub struct LlmConfig { max_retries: u32, retry_backoff_ms: u64, official_fallback: bool, + anthropic_strict_tool_support: bool, } // 首版只冻结当前项目已稳定使用的 system/user/assistant 三种消息角色。 @@ -417,12 +418,22 @@ struct AnthropicInputMessage { content: String, } -// Anthropic 工具与 OpenAI 的差异：schema 字段名为 input_schema，且没有 function 包装层与 strict。 +// Anthropic 工具与 OpenAI 的差异：schema 字段名为 input_schema，且没有 function 包装层。 #[derive(Serialize)] struct AnthropicTool { name: String, description: String, input_schema: serde_json::Value, + #[serde(skip_serializing_if = "Option::is_none")] + strict: Option, + #[serde(skip_serializing_if = "Option::is_none")] + cache_control: Option, +} + +#[derive(Serialize)] +struct AnthropicCacheControl { + #[serde(rename = "type")] + cache_type: &'static str, } // Anthropic 的 tool_choice 必须是对象，发送裸字符串会被上游拒绝。 @@ -611,9 +622,25 @@ struct AnthropicUsage { #[serde(default)] input_tokens: u64, #[serde(default)] + cache_creation_input_tokens: u64, + #[serde(default)] + cache_read_input_tokens: u64, + #[serde(default)] output_tokens: u64, } +fn map_anthropic_usage(usage: AnthropicUsage) -> LlmTokenUsage { + let prompt_tokens = usage + .input_tokens + .saturating_add(usage.cache_creation_input_tokens) + .saturating_add(usage.cache_read_input_tokens); + LlmTokenUsage { + prompt_tokens, + completion_tokens: usage.output_tokens, + total_tokens: prompt_tokens.saturating_add(usage.output_tokens), + } +} + struct OpenAiCompatibleSseParser { buffer: String, raw_text: String, @@ -994,6 +1021,7 @@ impl LlmConfig { max_retries, retry_backoff_ms, official_fallback: false, + anthropic_strict_tool_support: false, }) } @@ -1002,6 +1030,15 @@ impl LlmConfig { self } + /// 显式声明当前 Anthropic endpoint 与 model 组合支持 strict tool use。 + /// + /// 该能力不能由 `api_kind` 推断：旧 Claude 模型和 Anthropic-compatible + /// 网关未必接受 `strict: true`。因此默认关闭，仅允许已验证的配置启用。 + pub fn with_anthropic_strict_tool_support(mut self, supported: bool) -> Self { + self.anthropic_strict_tool_support = supported; + self + } + pub fn with_raw_log_dir(mut self, raw_log_dir: impl Into) -> Self { self.raw_log_dir = raw_log_dir.into(); self @@ -1055,6 +1092,10 @@ impl LlmConfig { self.official_fallback } + pub fn anthropic_strict_tool_support(&self) -> bool { + self.anthropic_strict_tool_support + } + pub fn chat_completions_url(&self) -> String { format!( "{}/{}", @@ -2107,7 +2148,23 @@ where } if let Some(event_usage) = event_usage { - accumulation.usage = Some(event_usage); + accumulation.usage = Some(match accumulation.usage.take() { + Some(previous) => { + let prompt_tokens = previous.prompt_tokens.max(event_usage.prompt_tokens); + let completion_tokens = previous + .completion_tokens + .max(event_usage.completion_tokens); + LlmTokenUsage { + prompt_tokens, + completion_tokens, + total_tokens: previous + .total_tokens + .max(event_usage.total_tokens) + .max(prompt_tokens.saturating_add(completion_tokens)), + } + } + None => event_usage, + }); } // 工具调用只累加，不进 on_delta：调用方的流式通道仍然只承载文本。 @@ -2242,6 +2299,7 @@ fn build_request_body(request: &LlmRunRequest, config: &LlmConfig, stream: bool) request, fallback_model, stream, + config.anthropic_strict_tool_support(), )), } } @@ -2268,7 +2326,12 @@ fn build_anthropic_messages_request_body( request: &LlmRunRequest, fallback_model: &str, stream: bool, + strict_tool_support: bool, ) -> AnthropicMessagesRequestBody { + // capability 绑定 LlmConfig 的 endpoint/model 组合；请求级 model override 没有经过 + // 同一轮能力确认，即使协议仍是 Anthropic 也必须 fail closed。 + let strict_tool_support = + strict_tool_support && request.resolved_model(fallback_model) == fallback_model; let system = request .messages .iter() @@ -2290,13 +2353,26 @@ fn build_anthropic_messages_request_body( .collect(); let tools = (!request.function_tools.is_empty()).then(|| { + let strict_schemas = + anthropic_strict_transport_schemas(&request.function_tools, strict_tool_support); + let last_index = request.function_tools.len().saturating_sub(1); request .function_tools .iter() - .map(|function| AnthropicTool { - name: function.name.clone(), - description: function.description.clone(), - input_schema: function.parameters.clone(), + .enumerate() + .map(|(index, function)| { + let strict_schema = strict_schemas[index].as_ref(); + AnthropicTool { + name: function.name.clone(), + description: function.description.clone(), + input_schema: strict_schema + .cloned() + .unwrap_or_else(|| function.parameters.clone()), + strict: strict_schema.is_some().then_some(true), + cache_control: (index == last_index).then_some(AnthropicCacheControl { + cache_type: "ephemeral", + }), + } }) .collect() }); @@ -2316,6 +2392,388 @@ fn build_anthropic_messages_request_body( } } +#[derive(Default)] +struct AnthropicStrictSchemaComplexity { + optional_parameters: usize, + union_parameters: usize, +} + +// Anthropic 官方 SDK 同样会先把调用方 schema 转成服务端可编译的传输 schema，再用 +// 原 schema 做本地校验。这里绝不修改 LlmFunctionTool.parameters，只剥离 strict grammar +// 不支持的约束。未显式列入支持或可安全剥离集合的 keyword 一律拒绝 strict，避免新 keyword +// 穿透有限黑名单后把整轮请求变成 400。 +fn anthropic_strict_transport_schema(schema: &serde_json::Value) -> Option { + const BASIC_TYPES: &[&str] = &[ + "object", "array", "string", "integer", "number", "boolean", "null", + ]; + const SUPPORTED_FORMATS: &[&str] = &[ + "date-time", + "time", + "date", + "duration", + "email", + "hostname", + "uri", + "ipv4", + "ipv6", + "uuid", + ]; + + fn transform_schema_map( + object: &serde_json::Map, + ) -> Option> { + let mut transformed = serde_json::Map::new(); + for (keyword, value) in object { + match keyword.as_str() { + "type" => { + let supported = match value { + serde_json::Value::String(value) => { + BASIC_TYPES.contains(&value.as_str()) + } + serde_json::Value::Array(values) => { + !values.is_empty() + && values.iter().all(|value| { + value.as_str().is_some_and(|value| BASIC_TYPES.contains(&value)) + }) + && values + .iter() + .filter_map(serde_json::Value::as_str) + .collect::>() + .len() + == values.len() + } + _ => false, + }; + if !supported { + return None; + } + transformed.insert(keyword.clone(), value.clone()); + } + "properties" | "$defs" | "definitions" => { + let children = value.as_object()?; + let mut transformed_children = serde_json::Map::new(); + for (name, child) in children { + transformed_children + .insert(name.clone(), anthropic_strict_transport_schema(child)?); + } + transformed.insert( + keyword.clone(), + serde_json::Value::Object(transformed_children), + ); + } + "items" => { + transformed.insert( + keyword.clone(), + anthropic_strict_transport_schema(value)?, + ); + } + "anyOf" | "allOf" => { + let children = value.as_array().filter(|children| !children.is_empty())?; + let transformed_children = children + .iter() + .map(anthropic_strict_transport_schema) + .collect::>>()?; + // Anthropic 明确不支持 allOf 中的 $ref；该组合不能靠删除 $ref + // 降级，否则会丢失整个结构定义。 + if keyword == "allOf" + && transformed_children.iter().any(anthropic_schema_uses_ref) + { + return None; + } + transformed.insert( + keyword.clone(), + serde_json::Value::Array(transformed_children), + ); + } + "$ref" => { + if !value + .as_str() + .is_some_and(|reference| reference.starts_with("#/")) + { + return None; + } + transformed.insert(keyword.clone(), value.clone()); + } + "required" => { + let values = value.as_array()?; + let names = values + .iter() + .map(serde_json::Value::as_str) + .collect::>>()?; + if names.iter().collect::>().len() + != names.len() + { + return None; + } + transformed.insert(keyword.clone(), value.clone()); + } + "additionalProperties" => { + if value != &serde_json::Value::Bool(false) { + return None; + } + transformed.insert(keyword.clone(), value.clone()); + } + "enum" => { + let values = value.as_array().filter(|values| !values.is_empty())?; + if values.len() > 100 + || values + .iter() + .any(|value| value.is_array() || value.is_object()) + { + return None; + } + transformed.insert(keyword.clone(), value.clone()); + } + "const" => { + if value.is_array() || value.is_object() { + return None; + } + transformed.insert(keyword.clone(), value.clone()); + } + "title" | "description" => { + value.as_str()?; + transformed.insert(keyword.clone(), value.clone()); + } + "default" => { + // default 是普通 JSON 数据，不递归解释其中可能出现的 `$ref`。 + transformed.insert(keyword.clone(), value.clone()); + } + "format" => { + if value + .as_str() + .is_some_and(|format| SUPPORTED_FORMATS.contains(&format)) + { + transformed.insert(keyword.clone(), value.clone()); + } + } + "minItems" => { + if value.as_u64().is_some_and(|minimum| minimum <= 1) { + transformed.insert(keyword.clone(), value.clone()); + } + } + // 这些是 Anthropic strict grammar 不支持的约束，或（如 pattern）带有 + // 本适配器未完整校验的编译器子集。传输时剥离；调用方持有的原 schema + // 不变，仍可用于 ToolHost 入参校验。 + "minimum" + | "maximum" + | "exclusiveMinimum" + | "exclusiveMaximum" + | "multipleOf" + | "minLength" + | "maxLength" + | "maxItems" + | "uniqueItems" + | "contains" + | "minContains" + | "maxContains" + | "minProperties" + | "maxProperties" + | "pattern" + // Anthropic 未列这些 annotation 为传输 schema 支持项；安全删除不会 + // 改变结构，且不会递归误读其中的普通 JSON 数据。 + | "examples" + | "$comment" + | "deprecated" + | "readOnly" + | "writeOnly" => {} + // `$id`、`$anchor`、dependentRequired 等所有未声明 keyword 都在这里 + // fail closed，不能靠 apiKind 或“看起来像 JSON Schema”发送 strict。 + _ => return None, + } + } + Some(transformed) + } + + Some(serde_json::Value::Object(transform_schema_map( + schema.as_object()?, + )?)) +} + +fn anthropic_schema_uses_ref(schema: &serde_json::Value) -> bool { + let Some(object) = schema.as_object() else { + return false; + }; + if object.contains_key("$ref") { + return true; + } + ["items"] + .into_iter() + .filter_map(|keyword| object.get(keyword)) + .any(anthropic_schema_uses_ref) + || ["properties", "$defs", "definitions"] + .into_iter() + .filter_map(|keyword| object.get(keyword).and_then(serde_json::Value::as_object)) + .flat_map(|children| children.values()) + .any(anthropic_schema_uses_ref) + || ["anyOf", "allOf"] + .into_iter() + .filter_map(|keyword| object.get(keyword).and_then(serde_json::Value::as_array)) + .flat_map(|children| children.iter()) + .any(anthropic_schema_uses_ref) +} + +fn collect_anthropic_strict_schema_complexity( + schema: &serde_json::Value, + complexity: &mut AnthropicStrictSchemaComplexity, +) -> bool { + let Some(object) = schema.as_object() else { + return false; + }; + if object + .get("type") + .and_then(serde_json::Value::as_array) + .is_some_and(|types| types.len() > 1) + || object.contains_key("anyOf") + { + complexity.union_parameters = complexity.union_parameters.saturating_add(1); + } + let is_object_schema = object.get("type").and_then(serde_json::Value::as_str) == Some("object") + || object + .get("type") + .and_then(serde_json::Value::as_array) + .is_some_and(|types| types.iter().any(|value| value.as_str() == Some("object"))) + || object.contains_key("properties"); + if is_object_schema + && object.get("additionalProperties") != Some(&serde_json::Value::Bool(false)) + { + return false; + } + if let Some(properties) = object + .get("properties") + .and_then(serde_json::Value::as_object) + { + let required = object + .get("required") + .and_then(serde_json::Value::as_array) + .map(|values| { + values + .iter() + .filter_map(serde_json::Value::as_str) + .collect::>() + }) + .unwrap_or_default(); + if required.len() + != object + .get("required") + .and_then(serde_json::Value::as_array) + .map(Vec::len) + .unwrap_or_default() + || required.iter().any(|name| !properties.contains_key(*name)) + { + return false; + } + complexity.optional_parameters = complexity.optional_parameters.saturating_add( + properties + .keys() + .filter(|name| !required.contains(name.as_str())) + .count(), + ); + } + ["items"] + .into_iter() + .filter_map(|keyword| object.get(keyword)) + .all(|child| collect_anthropic_strict_schema_complexity(child, complexity)) + && ["properties", "$defs", "definitions"] + .into_iter() + .filter_map(|keyword| object.get(keyword).and_then(serde_json::Value::as_object)) + .flat_map(|children| children.values()) + .all(|child| collect_anthropic_strict_schema_complexity(child, complexity)) + && ["anyOf", "allOf"] + .into_iter() + .filter_map(|keyword| object.get(keyword).and_then(serde_json::Value::as_array)) + .flat_map(|children| children.iter()) + .all(|child| collect_anthropic_strict_schema_complexity(child, complexity)) +} + +fn anthropic_strict_schema_refs_are_supported(schema: &serde_json::Value) -> bool { + fn visit( + value: &serde_json::Value, + root: &serde_json::Value, + active_refs: &mut std::collections::BTreeSet, + ) -> bool { + match value { + serde_json::Value::Object(object) => { + if let Some(reference) = object.get("$ref").and_then(serde_json::Value::as_str) { + // Anthropic strict 不支持递归 schema；为避免把命名 anchor 或外部 + // resource 误判为可编译，只接受能在当前 document 内解析的 Pointer。 + if !reference.starts_with("#/") { + return false; + } + let Some(target) = root.pointer(reference.trim_start_matches('#')) else { + return false; + }; + if !active_refs.insert(reference.to_string()) { + return false; + } + let target_is_supported = visit(target, root, active_refs); + active_refs.remove(reference); + if !target_is_supported { + return false; + } + } + ["items"] + .into_iter() + .filter_map(|keyword| object.get(keyword)) + .all(|child| visit(child, root, active_refs)) + && ["properties", "$defs", "definitions"] + .into_iter() + .filter_map(|keyword| { + object.get(keyword).and_then(serde_json::Value::as_object) + }) + .flat_map(|children| children.values()) + .all(|child| visit(child, root, active_refs)) + && ["anyOf", "allOf"] + .into_iter() + .filter_map(|keyword| { + object.get(keyword).and_then(serde_json::Value::as_array) + }) + .flat_map(|children| children.iter()) + .all(|child| visit(child, root, active_refs)) + } + _ => false, + } + } + + visit(schema, schema, &mut std::collections::BTreeSet::new()) +} + +fn anthropic_strict_transport_schemas( + functions: &[LlmFunctionTool], + strict_tool_support: bool, +) -> Vec> { + const MAX_STRICT_TOOLS: usize = 20; + const MAX_OPTIONAL_PARAMETERS: usize = 24; + const MAX_UNION_PARAMETERS: usize = 16; + + let mut strict_count = 0usize; + let mut optional_parameters = 0usize; + let mut union_parameters = 0usize; + functions + .iter() + .map(|function| { + if !strict_tool_support || !function.strict || strict_count >= MAX_STRICT_TOOLS { + return None; + } + let transport_schema = anthropic_strict_transport_schema(&function.parameters)?; + let mut complexity = AnthropicStrictSchemaComplexity::default(); + if !anthropic_strict_schema_refs_are_supported(&transport_schema) + || !collect_anthropic_strict_schema_complexity(&transport_schema, &mut complexity) + || optional_parameters.saturating_add(complexity.optional_parameters) + > MAX_OPTIONAL_PARAMETERS + || union_parameters.saturating_add(complexity.union_parameters) + > MAX_UNION_PARAMETERS + { + return None; + } + strict_count = strict_count.saturating_add(1); + optional_parameters = + optional_parameters.saturating_add(complexity.optional_parameters); + union_parameters = union_parameters.saturating_add(complexity.union_parameters); + Some(transport_schema) + }) + .collect() +} + fn map_chat_completions_input_messages( messages: &[LlmMessage], ) -> Vec { @@ -2692,11 +3150,7 @@ fn parse_anthropic_response( text: content, finish_reason: parsed.stop_reason, response_id: parsed.id, - usage: parsed.usage.map(|usage| LlmTokenUsage { - prompt_tokens: usage.input_tokens, - completion_tokens: usage.output_tokens, - total_tokens: usage.input_tokens.saturating_add(usage.output_tokens), - }), + usage: parsed.usage.map(map_anthropic_usage), tool_calls, }) } @@ -3219,6 +3673,21 @@ fn parse_anthropic_sse_event(data: &str) -> Result, Ll .unwrap_or_default(); match event_type { + "message_start" => Ok(parsed + .get("message") + .and_then(|message| message.get("usage")) + .cloned() + .map(serde_json::from_value::) + .transpose() + .map_err(|error| { + LlmError::Deserialize(format!( + "解析 LLM Anthropic message_start usage 失败：{error}" + )) + })? + .map(|usage| ParsedStreamEvent { + usage: Some(map_anthropic_usage(usage)), + ..Default::default() + })), // tool_use block 的 id 与 name 只在 content_block_start 出现；此时 input 恒为空对象， // 不能拿它初始化参数，否则会和后续 input_json_delta 拼出非法 JSON。 "content_block_start" => { @@ -3292,11 +3761,23 @@ fn parse_anthropic_sse_event(data: &str) -> Result, Ll .and_then(|value| value.get("stop_reason")) .and_then(serde_json::Value::as_str) .map(str::to_string); + let usage = parsed + .get("usage") + .cloned() + .map(serde_json::from_value::) + .transpose() + .map_err(|error| { + LlmError::Deserialize(format!( + "解析 LLM Anthropic message_delta usage 失败：{error}" + )) + })? + .map(map_anthropic_usage); Ok(Some(ParsedStreamEvent { is_completion: stop_reason .as_deref() .is_some_and(|reason| !reason.trim().is_empty()), finish_reason: stop_reason, + usage, ..Default::default() })) } @@ -3468,6 +3949,27 @@ mod tests { assert!(config.with_official_fallback(true).official_fallback()); } + #[test] + fn llm_config_anthropic_strict_tool_support_is_opt_in() { + let config = LlmConfig::new( + LlmProvider::OpenAiCompatible, + "https://api.anthropic.com".to_string(), + "secret".to_string(), + "claude-sonnet-4-5".to_string(), + DEFAULT_REQUEST_TIMEOUT_MS, + DEFAULT_MAX_RETRIES, + DEFAULT_RETRY_BACKOFF_MS, + ) + .expect("config should be valid"); + + assert!(!config.anthropic_strict_tool_support()); + assert!( + config + .with_anthropic_strict_tool_support(true) + .anthropic_strict_tool_support() + ); + } + #[test] fn run_request_defaults_to_openai_responses_api_kind() { let request = LlmRunRequest::single_turn("系统", "用户"); @@ -3507,7 +4009,12 @@ mod tests { LlmFunctionTool::new( "get_weather", "查询天气", - serde_json::json!({ "type": "object", "properties": { "city": { "type": "string" } } }), + serde_json::json!({ + "type": "object", + "properties": { "city": { "type": "string" } }, + "required": ["city"], + "additionalProperties": false + }), ) .with_strict(true), ]) @@ -3520,13 +4027,240 @@ mod tests { assert_eq!(json["tools"][0]["name"], "get_weather"); assert_eq!(json["tools"][0]["description"], "查询天气"); assert_eq!(json["tools"][0]["input_schema"]["type"], "object"); - // Anthropic 没有 parameters / strict 字段，映射时必须丢弃。 + // apiKind 不能证明 endpoint/model 支持 strict，未显式声明 capability 时必须关闭。 assert!(json["tools"][0].get("parameters").is_none()); assert!(json["tools"][0].get("strict").is_none()); + assert_eq!( + json["tools"][0]["cache_control"], + serde_json::json!({"type": "ephemeral"}) + ); // tool_choice 必须是对象；Required 对应 Anthropic 的 any。 assert_eq!(json["tool_choice"], serde_json::json!({ "type": "any" })); } + #[test] + fn anthropic_strict_uses_transformed_schema_without_mutating_the_original() { + let config = LlmConfig::new( + LlmProvider::OpenAiCompatible, + "https://example.com/anthropic".to_string(), + "secret".to_string(), + "model-a".to_string(), + DEFAULT_REQUEST_TIMEOUT_MS, + DEFAULT_MAX_RETRIES, + DEFAULT_RETRY_BACKOFF_MS, + ) + .expect("config should be valid") + .with_anthropic_strict_tool_support(true); + let request = LlmRunRequest::single_turn("系统", "用户") + .with_anthropic() + .with_function_tools(vec![ + LlmFunctionTool::new( + "bounded_text", + "包含 Anthropic strict 暂不支持的长度约束", + serde_json::json!({ + "type": "object", + "properties": { + "value": {"type": "string", "minLength": 1}, + "steps": { + "type": "array", + "minItems": 1, + "maxItems": 8, + "items": {"type": "string", "maxLength": 240} + } + }, + "required": ["value", "steps"], + "additionalProperties": false + }), + ) + .with_strict(true), + LlmFunctionTool::new( + "plain_text", + "支持严格模式的简单 schema", + serde_json::json!({ + "type": "object", + "properties": {"value": {"type": "string"}}, + "required": ["value"], + "additionalProperties": false + }), + ) + .with_strict(true), + ]); + + let json = serde_json::to_value(build_request_body(&request, &config, false)) + .expect("body should serialize"); + assert_eq!(json["tools"][0]["strict"], true); + assert!( + json["tools"][0]["input_schema"]["properties"]["value"] + .get("minLength") + .is_none() + ); + assert_eq!( + request.function_tools[0].parameters["properties"]["value"]["minLength"], + 1 + ); + assert_eq!( + json["tools"][0]["input_schema"]["properties"]["steps"]["minItems"], + 1 + ); + assert!( + json["tools"][0]["input_schema"]["properties"]["steps"] + .get("maxItems") + .is_none() + ); + assert_eq!( + request.function_tools[0].parameters["properties"]["steps"]["maxItems"], + 8 + ); + assert!(json["tools"][0].get("cache_control").is_none()); + assert_eq!(json["tools"][1]["strict"], true); + assert_eq!( + json["tools"][1]["cache_control"], + serde_json::json!({"type": "ephemeral"}) + ); + } + + #[test] + fn anthropic_request_model_override_does_not_reuse_config_scoped_strict_capability() { + let config = LlmConfig::new( + LlmProvider::OpenAiCompatible, + "https://api.anthropic.com".to_string(), + "secret".to_string(), + "claude-sonnet-4-5".to_string(), + DEFAULT_REQUEST_TIMEOUT_MS, + DEFAULT_MAX_RETRIES, + DEFAULT_RETRY_BACKOFF_MS, + ) + .expect("config should be valid") + .with_anthropic_strict_tool_support(true); + let request = LlmRunRequest::single_turn("系统", "用户") + .with_anthropic() + .with_model("claude-3-5-sonnet-latest") + .with_function_tools(vec![ + LlmFunctionTool::new( + "plain_text", + "simple schema", + serde_json::json!({ + "type": "object", + "properties": {"value": {"type": "string"}}, + "required": ["value"], + "additionalProperties": false + }), + ) + .with_strict(true), + ]); + + let json = serde_json::to_value(build_request_body(&request, &config, false)) + .expect("body should serialize"); + assert!(json["tools"][0].get("strict").is_none()); + } + + #[test] + fn anthropic_strict_rejects_unclosed_objects_recursive_or_missing_refs_and_complex_enums() { + let schemas = [ + serde_json::json!({"type": "object"}), + serde_json::json!({ + "type": "object", + "$defs": { + "Node": { + "type": "object", + "properties": {"next": {"$ref": "#/$defs/Node"}}, + "additionalProperties": false + } + }, + "properties": {"node": {"$ref": "#/$defs/Node"}}, + "required": ["node"], + "additionalProperties": false + }), + serde_json::json!({ + "type": "object", + "properties": {"value": {"$ref": "#/$defs/Missing"}}, + "required": ["value"], + "additionalProperties": false + }), + serde_json::json!({ + "type": "object", + "properties": {"value": {"enum": [{"nested": true}]}}, + "required": ["value"], + "additionalProperties": false + }), + ]; + for schema in schemas { + let tools = vec![LlmFunctionTool::new("unsafe", "unsafe", schema).with_strict(true)]; + assert_eq!(anthropic_strict_transport_schemas(&tools, true), vec![None]); + } + + let valid_ref = LlmFunctionTool::new( + "valid_ref", + "valid ref", + serde_json::json!({ + "type": "object", + "$defs": {"Value": {"type": "string"}}, + "properties": {"value": {"$ref": "#/$defs/Value"}}, + "required": ["value"], + "additionalProperties": false + }), + ) + .with_strict(true); + assert!(anthropic_strict_transport_schemas(&[valid_ref], true)[0].is_some()); + } + + #[test] + fn anthropic_strict_rejects_unknown_or_scope_changing_keywords() { + for (keyword, value) in [ + ("$id", serde_json::json!("nested.json")), + ("$anchor", serde_json::json!("node")), + ("dependentRequired", serde_json::json!({"value": ["other"]})), + ] { + let mut schema = serde_json::json!({ + "type": "object", + "properties": {"value": {"type": "string"}}, + "required": ["value"], + "additionalProperties": false + }); + schema + .as_object_mut() + .expect("schema should be an object") + .insert(keyword.to_string(), value); + let tools = vec![LlmFunctionTool::new("unsafe", "unsafe", schema).with_strict(true)]; + assert_eq!( + anthropic_strict_transport_schemas(&tools, true), + vec![None], + "{keyword} must fail closed" + ); + } + } + + #[test] + fn anthropic_strict_does_not_interpret_refs_inside_default_data() { + let tool = LlmFunctionTool::new( + "default_payload", + "default payload", + serde_json::json!({ + "type": "object", + "properties": { + "value": { + "type": "object", + "properties": {}, + "required": [], + "additionalProperties": false, + "default": {"$ref": "#/literal-data"} + } + }, + "required": ["value"], + "additionalProperties": false + }), + ) + .with_strict(true); + + let transformed = anthropic_strict_transport_schemas(&[tool], true)[0] + .clone() + .expect("data-valued ref must not disable strict"); + assert_eq!( + transformed["properties"]["value"]["default"]["$ref"], + "#/literal-data" + ); + } + #[test] fn anthropic_request_body_omits_tool_fields_without_tools() { let config = LlmConfig::new( @@ -4937,7 +5671,7 @@ mod tests { MockResponse { status_line: "200 OK", content_type: "application/json; charset=utf-8", - body: r#"{"id":"msg_01","model":"claude-test","content":[{"type":"text","text":"Anthropic 成功"}],"stop_reason":"end_turn","usage":{"input_tokens":5,"output_tokens":3}}"#.to_string(), + body: r#"{"id":"msg_01","model":"claude-test","content":[{"type":"text","text":"Anthropic 成功"}],"stop_reason":"end_turn","usage":{"input_tokens":5,"cache_creation_input_tokens":4,"cache_read_input_tokens":3,"output_tokens":3}}"#.to_string(), extra_headers: Vec::new(), }, ); @@ -4966,9 +5700,9 @@ mod tests { assert_eq!( response.usage, Some(LlmTokenUsage { - prompt_tokens: 5, + prompt_tokens: 12, completion_tokens: 3, - total_tokens: 8, + total_tokens: 15, }) ); assert_eq!(request_json["model"], serde_json::json!("test-model")); @@ -4985,9 +5719,10 @@ mod tests { status_line: "200 OK", content_type: "text/event-stream; charset=utf-8", body: concat!( + "data: {\"type\":\"message_start\",\"message\":{\"usage\":{\"input_tokens\":5,\"cache_creation_input_tokens\":4,\"cache_read_input_tokens\":3,\"output_tokens\":0}}}\n\n", "data: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"text_delta\",\"text\":\"你\"}}\n\n", "data: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"text_delta\",\"text\":\"好\"}}\n\n", - "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"}}\n\n", + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"},\"usage\":{\"output_tokens\":3}}\n\n", "data: {\"type\":\"message_stop\"}\n\n" ) .to_string(), @@ -5013,6 +5748,14 @@ mod tests { response.response_id.as_deref(), Some("req_anthropic_stream_01") ); + assert_eq!( + response.usage, + Some(LlmTokenUsage { + prompt_tokens: 12, + completion_tokens: 3, + total_tokens: 15, + }) + ); } // 以下三个流式工具用例使用取自真实端点的 checked-in SSE fixture：Anthropic 与 diff --git a/server-rs/crates/platform-llm/tests/live_stream_tool_calls.rs b/server-rs/crates/platform-llm/tests/live_stream_tool_calls.rs index d40ee8def..fe35e7891 100644 --- a/server-rs/crates/platform-llm/tests/live_stream_tool_calls.rs +++ b/server-rs/crates/platform-llm/tests/live_stream_tool_calls.rs @@ -3,10 +3,10 @@ //! 仓库根目录没有 Cargo.toml，必须显式指定 workspace manifest： //! //! ```powershell -//! $env:PLATFORM_LLM_LIVE_BASE_URL = 'https://api.minimaxi.com/anthropic' +//! $env:PLATFORM_LLM_LIVE_BASE_URL = 'https://api.anthropic.com' //! $env:PLATFORM_LLM_LIVE_API_KEY = '...' -//! $env:PLATFORM_LLM_LIVE_MODEL = 'MiniMax-M3' -//! $env:PLATFORM_LLM_LIVE_API_KIND = 'anthropic' # 或 openai_chat / openai_responses +//! $env:PLATFORM_LLM_LIVE_MODEL = '<当前支持 strict tool use 的 Claude 模型>' +//! $env:PLATFORM_LLM_LIVE_API_KIND = 'anthropic' //! cargo test -p platform-llm --manifest-path server-rs/Cargo.toml --test live_stream_tool_calls -- --ignored --nocapture //! ``` //! @@ -107,7 +107,8 @@ async fn live_stream_run_returns_native_tool_calls() { 0, 1_000, ) - .expect("live config should be valid"); + .expect("live config should be valid") + .with_anthropic_strict_tool_support(api_kind == LlmApiKind::Anthropic); let client = LlmClient::new(config).expect("live client should be created"); let request = LlmRunRequest::new(vec![ @@ -116,15 +117,27 @@ async fn live_stream_run_returns_native_tool_calls() { ]) .with_api_kind(api_kind) .with_max_output_tokens(512) - .with_function_tools(vec![LlmFunctionTool::new( - "get_weather", - "查询指定城市的当前天气。", - serde_json::json!({ - "type": "object", - "properties": { "city": { "type": "string" } }, - "required": ["city"] - }), - )]) + .with_function_tools(vec![ + LlmFunctionTool::new( + "get_weather", + "查询指定城市的当前天气。", + serde_json::json!({ + "type": "object", + "$defs": { + "WeatherRequest": { + "type": "object", + "properties": { "city": { "type": "string" } }, + "required": ["city"], + "additionalProperties": false + } + }, + "properties": { "request": { "$ref": "#/$defs/WeatherRequest" } }, + "required": ["request"], + "additionalProperties": false + }), + ) + .with_strict(true), + ]) .with_tool_choice(LlmToolChoice::Required); let mut streamed_chars = 0usize; @@ -157,7 +170,10 @@ async fn live_stream_run_returns_native_tool_calls() { let arguments: serde_json::Value = serde_json::from_str(&call.arguments).expect("参数必须是完整 JSON"); assert!( - arguments.get("city").is_some(), - "参数应包含 city，实际为 {arguments}" + arguments + .get("request") + .and_then(|request| request.get("city")) + .is_some(), + "参数应包含 request.city，实际为 {arguments}" ); } diff --git a/server-rs/crates/shared-contracts/src/external_generation.rs b/server-rs/crates/shared-contracts/src/external_generation.rs index a93705986..945c8adf7 100644 --- a/server-rs/crates/shared-contracts/src/external_generation.rs +++ b/server-rs/crates/shared-contracts/src/external_generation.rs @@ -1,4 +1,5 @@ use serde::{Deserialize, Serialize}; +use serde_json::Value; #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "kebab-case")] @@ -51,6 +52,37 @@ pub struct ExternalGenerationJobStatusResponse { pub job: ExternalGenerationJobStatusDetailRecord, } +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ExternalEditorGenerationSubmissionResponse { + pub operation_id: String, + pub kind: String, + pub status: ExternalGenerationJobStatus, + pub status_url: String, + pub poll_after_ms: u64, + pub updated_at_micros: i64, +} + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ExternalEditorGenerationJobResponse { + pub operation_id: String, + pub kind: String, + pub status: ExternalGenerationJobStatus, + pub phase_label: String, + pub phase_detail: String, + pub progress: u8, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub error: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub warning: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub result: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub poll_after_ms: Option, + pub updated_at_micros: i64, +} + #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub struct ExternalGenerationTaskRecord { diff --git a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx index c307a36ee..ffe305bff 100644 --- a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx @@ -428,6 +428,24 @@ describe('ImageCanvasEditorView generation integration', () => { }), ], layers: [ + ...projectLayers, + { + itemType: 'generation-dialog', + layerId: `generation-dialog:${canvasCompletion.dialogId}`, + resourceId: `generation-dialog:${canvasCompletion.dialogId}`, + dialog: { + id: canvasCompletion.dialogId, + mode: dialogMode, + prompt, + status: 'idle', + composerOpen: false, + generatedLayerId: layerId, + placeholder, + imageModel: model, + aspectRatio: input.aspectRatio, + imageSize: input.imageSize, + }, + }, { layerId, resourceId, @@ -450,24 +468,6 @@ describe('ImageCanvasEditorView generation integration', () => { assetKind: input.assetKind ?? undefined, generationInputs: input.generationInputs, }, - { - itemType: 'generation-dialog', - layerId: `generation-dialog:${canvasCompletion.dialogId}`, - resourceId: `generation-dialog:${canvasCompletion.dialogId}`, - dialog: { - id: canvasCompletion.dialogId, - mode: dialogMode, - prompt, - status: 'idle', - composerOpen: true, - generatedLayerId: layerId, - placeholder, - imageModel: model, - aspectRatio: input.aspectRatio, - imageSize: input.imageSize, - }, - }, - ...projectLayers, ], updatedAt: '2026-06-19T00:00:00.000Z', }, @@ -887,10 +887,7 @@ describe('ImageCanvasEditorView generation integration', () => { .getByAltText(/画布图片：生成图片/) .closest('button')!; expect(generatedLayer).toBeTruthy(); - const anchoredGenerateDialog = screen.getByRole('dialog', { - name: '生成图片', - }); - expect(anchoredGenerateDialog).toBeTruthy(); + expect(screen.queryByRole('dialog', { name: '生成图片' })).toBeNull(); expect( Number.isFinite( Number.parseFloat((generatedLayer as HTMLElement).style.top), @@ -978,10 +975,7 @@ describe('ImageCanvasEditorView generation integration', () => { const generatedLayer = screen .getByAltText(/画布图片：生成图片/) .closest('button')!; - const anchoredGenerateDialog = screen.getByRole('dialog', { - name: '生成图片', - }); - expect(anchoredGenerateDialog).toBeTruthy(); + expect(screen.queryByRole('dialog', { name: '生成图片' })).toBeNull(); expect(screen.queryByLabelText('图像生成占位图')).toBeNull(); expect( Number.parseFloat((generatedLayer as HTMLElement).style.left) + @@ -1522,7 +1516,28 @@ describe('ImageCanvasEditorView generation integration', () => { expect(screen.getByAltText(/画布图片：角色规范/)).toBeTruthy(); }); expect(screen.getByText('规范')).toBeTruthy(); - expect(saveEditorProjectLayoutMock).not.toHaveBeenCalled(); + expect(saveEditorProjectLayoutMock).toHaveBeenCalledWith( + 'editor-project-default', + expect.objectContaining({ + expectedRevision: 1, + layers: expect.arrayContaining([ + expect.objectContaining({ + itemType: 'generation-dialog', + dialog: expect.objectContaining({ + mode: 'spec', + status: 'idle', + generatedLayerId: 'layer-editor-spec-role-1', + specValues: expect.objectContaining({ + playSetting: '平台跳跃玩法', + artStyle: '低多边形卡通', + bodyRatio: '4', + characterView: '左向三分之二侧身站姿', + }), + }), + }), + ]), + }), + ); }); it('shows visible titles for character spec, icon spec, and icon spritesheet generation fields', async () => { @@ -2094,10 +2109,11 @@ describe('ImageCanvasEditorView generation integration', () => { const generatedImage = await screen.findByAltText(/画布图片：生成图片/u); const generatedLayerButton = generatedImage.closest('button')!; + expect(screen.queryByRole('dialog', { name: '生成图片' })).toBeNull(); + fireEvent.click(generatedLayerButton); expect(generatedLayerButton.className).toContain( 'image-canvas-editor__layer--selected', ); - expect(screen.getByRole('dialog', { name: '生成图片' })).toBeTruthy(); fireEvent.pointerDown(screen.getByLabelText('画布工作区'), { button: 0, @@ -3652,7 +3668,7 @@ describe('ImageCanvasEditorView generation integration', () => { .closest('button') as HTMLElement; expect(Number.parseFloat(generatedLayer.style.width)).toBe(1024); expect(Number.parseFloat(generatedLayer.style.height)).toBe(1024); - expect(screen.getByRole('dialog', { name: '生成图片' })).toBeTruthy(); + expect(screen.queryByRole('dialog', { name: '生成图片' })).toBeNull(); const metadataCornerButton = screen.getAllByRole('button', { name: /查看生成图片 .*图片信息/, diff --git a/src/components/image-editor/ImageCanvasEditorView.tsx b/src/components/image-editor/ImageCanvasEditorView.tsx index 37eb6c7b8..45d31c45f 100644 --- a/src/components/image-editor/ImageCanvasEditorView.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.tsx @@ -1152,6 +1152,8 @@ export function ImageCanvasEditorView({ viewportRef, canvasGenerationDialogsRef, canvasBackgroundColorRef, + selectedLayerIdRef, + selectedLayerIdsRef, }), [], ); @@ -1161,6 +1163,8 @@ export function ImageCanvasEditorView({ setProjectRenameValue, setViewport, setLayers, + setSelectedLayerId, + setSelectedLayerIds, selectSingleLayer, setLayerCounter: (value: number) => { layerCounterRef.current = value; @@ -1170,6 +1174,8 @@ export function ImageCanvasEditorView({ }), [ applyCanvasBackgroundColor, + setSelectedLayerId, + setSelectedLayerIds, restoreCanvasGenerationDialogs, selectSingleLayer, setLayers, diff --git a/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx b/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx index 03f7d4439..3dae18b95 100644 --- a/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx +++ b/src/components/image-editor/useImageCanvasProjectPersistence.test.tsx @@ -5,7 +5,10 @@ import { useCallback, useMemo, useRef, useState } from 'react'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { ApiClientError } from '../../services/apiClient'; -import type { EditorProjectSnapshot } from '../../services/image-editor/editorProjectClient'; +import type { + EditorProjectLayerSnapshot, + EditorProjectSnapshot, +} from '../../services/image-editor/editorProjectClient'; import { DEFAULT_CANVAS_BACKGROUND_COLOR, normalizeCanvasBackgroundHex, @@ -15,7 +18,10 @@ import type { CanvasLayer, CanvasViewport, } from './ImageCanvasEditorTypes'; -import { useImageCanvasProjectPersistence } from './useImageCanvasProjectPersistence'; +import { + mergeAuthoritativeCanvasLayoutWithPendingLocalLayout, + useImageCanvasProjectPersistence, +} from './useImageCanvasProjectPersistence'; const createEditorProjectResourceMock = vi.hoisted(() => vi.fn()); const createProjectCoverSnapshotBlobMock = vi.hoisted(() => vi.fn()); @@ -84,6 +90,245 @@ function createDeferred() { return { promise, resolve, reject }; } +function createCompletedEditorProjectSnapshot( + revision = 1, +): EditorProjectSnapshot { + return { + projectId: 'editor-project-default', + title: '空画布项目', + canvas: { + canvasId: 'editor-project-default:canvas:default', + projectId: 'editor-project-default', + title: '默认画布', + viewport: { x: 0, y: 0, scale: 1 }, + layers: [], + revision, + layoutStorageVersion: 0, + updatedAt: '2026-06-12T00:00:01.000Z', + }, + viewport: { x: 0, y: 0, scale: 1 }, + layers: [ + { + itemType: 'generation-dialog', + layerId: 'generation-dialog:generation-dialog-1', + resourceId: 'generation-dialog:generation-dialog-1', + dialog: { + id: 'generation-dialog-1', + mode: 'generate', + prompt: '后端完成生成器', + status: 'idle', + composerOpen: false, + generatedLayerId: 'layer-generated', + placeholder: { + x: 42, + y: 56, + width: 420, + height: 420, + originalWidth: 420, + originalHeight: 420, + }, + }, + }, + { + layerId: 'layer-generated', + resourceId: 'resource-generated', + title: '生成结果', + x: 42, + y: 56, + width: 420, + height: 420, + originalWidth: 420, + originalHeight: 420, + zIndex: 1, + sourceType: 'generated', + }, + ], + resources: [ + { + resourceId: 'resource-generated', + projectId: 'editor-project-default', + imageSrc: '/generated/result.png', + objectKey: 'generated/result.png', + assetObjectId: 'asset-object-result', + width: 420, + height: 420, + sourceType: 'generated', + }, + ], + updatedAt: '2026-06-12T00:00:01.000Z', + }; +} + +it('merges pending geometry and dialog edits while retaining backend additions and deletions', () => { + const authoritativeItems: EditorProjectLayerSnapshot[] = [ + { + itemType: 'canvas-settings', + layerId: 'canvas-settings:default', + resourceId: 'canvas-settings:default', + canvasBackgroundColor: '#FFFFFF', + }, + { + layerId: 'layer-existing', + resourceId: 'resource-existing-v2', + title: '后端标题', + x: 10, + y: 20, + zIndex: 1, + sourceType: 'generated', + objectKey: 'generated/new.png', + }, + { + layerId: 'layer-deleted-locally', + resourceId: 'resource-deleted-locally', + title: '本地已删除', + x: 0, + y: 0, + zIndex: 2, + sourceType: 'generated', + }, + { + layerId: 'layer-generated-by-backend', + resourceId: 'resource-generated-by-backend', + title: '后端新增结果', + x: 40, + y: 50, + zIndex: 3, + sourceType: 'generated', + }, + { + itemType: 'generation-dialog', + layerId: 'generation-dialog:dialog-1', + resourceId: 'generation-dialog:dialog-1', + dialog: { + id: 'dialog-1', + mode: 'generate', + prompt: '后端原提示词', + status: 'idle', + composerOpen: false, + generatedLayerId: 'layer-generated-by-backend', + placeholder: { + x: 40, + y: 50, + width: 320, + height: 320, + originalWidth: 1024, + originalHeight: 1024, + }, + }, + }, + ]; + const pendingItems: EditorProjectLayerSnapshot[] = [ + { + itemType: 'canvas-settings', + layerId: 'canvas-settings:default', + resourceId: 'canvas-settings:default', + canvasBackgroundColor: '#AABBCC', + }, + { + layerId: 'layer-existing', + resourceId: 'resource-existing-v1', + title: '本地标题', + x: 88, + y: 99, + zIndex: 7, + sourceType: 'generated', + objectKey: 'generated/old.png', + }, + { + layerId: 'layer-deleted-by-backend', + resourceId: 'resource-deleted-by-backend', + title: '后端已删除', + x: 1, + y: 2, + zIndex: 8, + sourceType: 'generated', + }, + { + layerId: 'layer-added-locally', + resourceId: 'resource-added-locally', + title: '本地新增', + x: 3, + y: 4, + zIndex: 9, + sourceType: 'generated', + }, + { + itemType: 'generation-dialog', + layerId: 'generation-dialog:dialog-1', + resourceId: 'generation-dialog:dialog-1', + dialog: { + id: 'dialog-1', + mode: 'generate', + prompt: '请求在途期间的新提示词', + status: 'generating', + composerOpen: true, + placeholder: { + x: 88, + y: 99, + width: 320, + height: 320, + originalWidth: 1024, + originalHeight: 1024, + }, + }, + }, + ]; + + const merged = mergeAuthoritativeCanvasLayoutWithPendingLocalLayout({ + authoritativeItems, + pendingItems, + previousAuthoritativeItemIds: new Set([ + 'canvas-settings:default', + 'layer-existing', + 'layer-deleted-locally', + 'layer-deleted-by-backend', + 'generation-dialog:dialog-1', + ]), + }); + + expect(merged).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + layerId: 'canvas-settings:default', + canvasBackgroundColor: '#AABBCC', + }), + expect.objectContaining({ + layerId: 'layer-existing', + resourceId: 'resource-existing-v2', + title: '本地标题', + x: 88, + y: 99, + zIndex: 7, + objectKey: 'generated/new.png', + }), + expect.objectContaining({ + layerId: 'layer-generated-by-backend', + resourceId: 'resource-generated-by-backend', + }), + expect.objectContaining({ + layerId: 'layer-added-locally', + resourceId: 'resource-added-locally', + }), + expect.objectContaining({ + layerId: 'generation-dialog:dialog-1', + dialog: expect.objectContaining({ + prompt: '请求在途期间的新提示词', + status: 'idle', + composerOpen: false, + generatedLayerId: 'layer-generated-by-backend', + placeholder: expect.objectContaining({ x: 88, y: 99 }), + }), + }), + ]), + ); + expect(merged.some((item) => item.layerId === 'layer-deleted-locally')).toBe( + false, + ); + expect( + merged.some((item) => item.layerId === 'layer-deleted-by-backend'), + ).toBe(false); +}); + function ProjectPersistenceHarness({ canAccessProtectedData = true, currentUserId = 'user-test', @@ -114,11 +359,14 @@ function ProjectPersistenceHarness({ const [projectTitle, setProjectTitle] = useState(''); const [projectRenameValue, setProjectRenameValue] = useState(''); const [flushCompleted, setFlushCompleted] = useState(false); + const [selectedLayerId, setSelectedLayerId] = useState(null); + const [selectedLayerIds, setSelectedLayerIds] = useState([]); const layersRef = useRef(layers); const viewportRef = useRef(viewport); const canvasGenerationDialogsRef = useRef(generationDialogs); const canvasBackgroundColorRef = useRef(canvasBackgroundColor); - const selectedLayerRef = useRef(null); + const selectedLayerRef = useRef(selectedLayerId); + const selectedLayerIdsRef = useRef(selectedLayerIds); const layerCounterRef = useRef(0); const openEditorLoginModalRef = useRef(vi.fn()); @@ -126,8 +374,11 @@ function ProjectPersistenceHarness({ viewportRef.current = viewport; canvasGenerationDialogsRef.current = generationDialogs; canvasBackgroundColorRef.current = canvasBackgroundColor; + selectedLayerRef.current = selectedLayerId; + selectedLayerIdsRef.current = selectedLayerIds; const selectSingleLayer = useCallback((layerId: string | null) => { - selectedLayerRef.current = layerId; + setSelectedLayerId(layerId); + setSelectedLayerIds(layerId ? [layerId] : []); }, []); const setLayerCounter = useCallback((value: number) => { layerCounterRef.current = value; @@ -146,6 +397,8 @@ function ProjectPersistenceHarness({ viewportRef, canvasGenerationDialogsRef, canvasBackgroundColorRef, + selectedLayerIdRef: selectedLayerRef, + selectedLayerIdsRef, }), [], ); @@ -155,6 +408,8 @@ function ProjectPersistenceHarness({ setProjectRenameValue, setViewport, setLayers, + setSelectedLayerId, + setSelectedLayerIds, selectSingleLayer, setLayerCounter, restoreCanvasGenerationDialogs: setGenerationDialogs, @@ -194,6 +449,9 @@ function ProjectPersistenceHarness({ .join(',')} {selectedLayerRef.current ?? '-'} + + {selectedLayerIdsRef.current.join(',') || '-'} + {layerCounterRef.current} {viewport.x},{viewport.y},{viewport.scale} @@ -330,6 +588,16 @@ function ProjectPersistenceHarness({ > append generated + + +