Merge branch 'master' into enhance/default-model
Project CI / AI game creator shell Rust crates (pull_request) Successful in 1m35s
Project CI / AI game creator shell Rust smoke (pull_request) Successful in 2m5s
Project CI / Backend tests (pull_request) Successful in 4m40s
Project CI / Native shell tests (pull_request) Successful in 6m13s
Project CI / Frontend tests (pull_request) Successful in 2m7s
Project CI / AI game creator shell Rust lane 2/2 (pull_request) Successful in 9m13s
Project CI / AI game creator shell Rust lane 1/2 (pull_request) Successful in 9m55s
Project CI / AI game creator shell web tests (pull_request) Successful in 1m34s
Project CI / Repository checks (pull_request) Successful in 1m58s
Project CI / AI game creator shell Rust crates (pull_request) Successful in 1m35s
Project CI / AI game creator shell Rust smoke (pull_request) Successful in 2m5s
Project CI / Backend tests (pull_request) Successful in 4m40s
Project CI / Native shell tests (pull_request) Successful in 6m13s
Project CI / Frontend tests (pull_request) Successful in 2m7s
Project CI / AI game creator shell Rust lane 2/2 (pull_request) Successful in 9m13s
Project CI / AI game creator shell Rust lane 1/2 (pull_request) Successful in 9m55s
Project CI / AI game creator shell web tests (pull_request) Successful in 1m34s
Project CI / Repository checks (pull_request) Successful in 1m58s
This commit is contained in:
@@ -5,131 +5,42 @@ description: Guide use of Genarrative's hosted external editor/canvas MCP or asy
|
||||
|
||||
# Genarrative External Editor API
|
||||
|
||||
Discover the live integration through `GET https://www.genarrative.world/api/external/v1/agent-integration.json`. Treat `GET https://www.genarrative.world/api/external/v1/openapi.json` as the field-level source of truth. In this repository, the same contract is `docs/openapi/genarrative-external-v1.openapi.json`.
|
||||
Use Genarrative to manage canvas projects and asset-library records, generate images, character animations, videos, and audio, and edit images. Discover the hosted integration at `https://www.genarrative.world/api/external/v1/agent-integration.json`.
|
||||
|
||||
Prefer the hosted Streamable HTTP MCP at `https://www.genarrative.world/api/external/v1/mcp` when the Agent supports remote MCP with a custom Bearer token. It exposes the External v1 operations as tools and the Skill documentation as resources; it does not require a local MCP server. Use this complete Skill package when remote MCP is unavailable or local-file upload needs client-side orchestration.
|
||||
Connect to `https://www.genarrative.world/api/external/v1/mcp` using Streamable HTTP and a Bearer API Key. Read tool descriptions and input schemas from `tools/list`; read the documents below through `resources/read` when needed. The hosted service needs no local MCP server. For direct REST, use the live `/api/external/v1/openapi.json` contract (in this repository: `docs/openapi/genarrative-external-v1.openapi.json`).
|
||||
|
||||
Prefer `scripts/genarrative_external_api.py` for runnable REST calls. It uses only Python stdlib, reads the local private API Key file, keeps the production base URL fixed, uploads local references, and wraps asynchronous submission, polling, and result retrieval.
|
||||
## Working with the Service
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Discover the integration manifest. Choose hosted MCP when supported; otherwise use the helper or direct REST.
|
||||
2. Before the first generation in a new conversation, obtain a canvas name unless an existing `projectId` and `assetFolderId` were supplied. Create or reuse a project and a same-name asset-library folder. Retain `canvasName`, `projectId`, `assetFolderId`, and the current art spec.
|
||||
3. Normalize art requests into a reusable spec. Ask only for missing values that block the selected operation. Reuse the spec until the user changes its style, subject family, palette, format, or constraints.
|
||||
4. Infer the operation from the user's intent. Do not ask the user to select an API unless two operations would produce materially different artifacts.
|
||||
5. If a reference exists only as a local file, upload and confirm it first. Pass the stable returned `objectKey` to operations that accept object references; never substitute a temporary signed URL. For an icon-spritesheet primary spec, additionally create a project resource or asset record with `assetKind="icon-spec"`, then pass the returned resource or asset ID as `referenceId`.
|
||||
6. For generation endpoints that support the fields, include `projectId`, `assetFolderId`, an asset label, and `canvasCompletion` so the result enters both the canvas and its same-name library folder.
|
||||
7. Treat every generation POST as asynchronous. Send one stable `Idempotency-Key` per logical request, retain the returned `operationId`, and poll the returned `statusUrl` or `GET /api/external/v1/generations/{operationId}` according to `pollAfterMs`.
|
||||
8. Consume `result` only after `status=completed`. On `failed`, surface the safe error. On a client timeout or lost response, retain the operation/key; do not create a replacement request.
|
||||
9. Reload the normal project or asset-library read endpoint when the caller needs complete authoritative state. Generation results are intentionally compact.
|
||||
10. Stay within `/api/external/v1`. Never call internal workers, queues, admin/profile APIs, or SpacetimeDB endpoints unless the user explicitly changes scope.
|
||||
|
||||
## Essential Invariants
|
||||
|
||||
- Authenticate MCP and business API calls with `Authorization: Bearer <tnr_sk_...>`. Never ask the user to paste a key into chat or place one in repository files.
|
||||
- All nine generation POST routes require `Idempotency-Key` and return HTTP `202`; `202` is durable acceptance, not a media result.
|
||||
- Retry an uncertain submission only with the exact same body and the same idempotency key. A polling timeout is not permission to generate again.
|
||||
- Use stable references such as `objectKey`, project resource ID, or asset ID where each operation permits them. Image edit/redraw is stricter: `sourceReferenceId` accepts only a registered project resource ID or asset ID; upload confirmation alone is not enough. Use `/assets/read-url` only for temporary preview/download access.
|
||||
- Preserve both warning channels after completion. A general `warning` can coexist with `sliceWarning`; do not discard either.
|
||||
- Do not invent missing derivatives. A source-preserved warning means the main source remains usable but requested post-processing failed. A slice warning means the complete transparent sheet is usable but individual slices are absent.
|
||||
- Icon spritesheet generation requires an explicit `sliceMode` and has no default. Use `sliceMode="grid"` with the `gridX` and `gridY` the requirement actually names (1-32 each) only for equal grid cells or fixed slots; use `sliceMode="connected-components"` for free-form sheets or an open number of subjects, and constrain the count with `sliceCount` instead of inventing grid dimensions. `connected-components` must not carry `gridX`/`gridY`; an omitted, contradictory, or misapplied declaration returns 400 before billing.
|
||||
- For successful `style="pixelArt"`, treat completed-result and nested resource/asset dimensions as the final logical-grid PNG dimensions. They may differ from `size`, `imageSize`, the provider image, and `canvasCompletion.placeholder`; do not rescale or reject the artifact to match those inputs.
|
||||
- Keep generated artifacts in the canvas and asset library together. Character animation accepts `assetFolderId` and `assetLabel`; its completed result directly returns the final `assetKind="character-animation"` resource and asset with formal sequence fields. Do not create a duplicate first-frame record.
|
||||
- Select tools by the requested outcome. Use `find_canvas_projects` and `find_assets` to locate existing context; create projects or folders only when the task needs them. A folder need not have the same name as the project.
|
||||
- For generation, specify project, library, and `canvasCompletion` fields only as supported by the selected tool and needed for the requested destination. Do not duplicate records already created by generation.
|
||||
- Upload local references using `prepare_asset_upload`: request a ticket, transfer the file from the client, then confirm the object. Confirmation does not create a canvas layer or a project/library record. Use the reference type accepted by the target tool; some operations require a registered resource or asset ID rather than an object key.
|
||||
- Generation is paid and asynchronous. Keep one stable `idempotencyKey` per logical generation and retain the returned `operationId`. Call `check_generation` according to `pollAfterMs`; consume `result` only after `completed`, and report the safe error on `failed`. A polling timeout does not justify another generation.
|
||||
- Read actual artifacts and warnings before claiming the requested deliverable is complete. Use project/library reads for complete persisted records, and `find_assets` with `action=get_download_url` for temporary media access.
|
||||
- Keep API Keys and temporary upload/download credentials out of chat, repository files, and logs. Business calls operate within the API Key's owner and scopes.
|
||||
|
||||
## Documentation Navigation
|
||||
|
||||
Read only the references needed for the task, but always verify exact schemas and enums against live OpenAPI:
|
||||
Read the reference relevant to the current operation; exact input fields and enums come from the tool schema or OpenAPI.
|
||||
|
||||
- `references/capability-routing.md`: read before selecting an MCP tool or REST operation, creating a canvas session, or working in the AI game creator visual DAG.
|
||||
- `references/api-operations.md`: read when constructing project, canvas, asset-library, upload, generation, or generation-status calls.
|
||||
- `references/authentication-and-safety.md`: read before handling credentials, local files, OSS form upload, retries, private media, or logs.
|
||||
- `references/requests-and-outputs.md`: read before building generation payloads, polling, interpreting compact results, applying canvas completion, or handling post-processing warnings.
|
||||
| Need | Reference | MCP resource URI |
|
||||
| --- | --- | --- |
|
||||
| Choose tools and actions by user intent | [Capability routing](references/capability-routing.md) | `genarrative://external-editor/skill/references/capability-routing.md` |
|
||||
| Map tool calls to REST operations | [API operations](references/api-operations.md) | `genarrative://external-editor/skill/references/api-operations.md` |
|
||||
| Configure credentials, upload files, handle retries and deletion | [Authentication and safety](references/authentication-and-safety.md) | `genarrative://external-editor/skill/references/authentication-and-safety.md` |
|
||||
| Construct requests, poll results, place media, handle warnings | [Requests and outputs](references/requests-and-outputs.md) | `genarrative://external-editor/skill/references/requests-and-outputs.md` |
|
||||
|
||||
The hosted MCP exposes the same documents through:
|
||||
`genarrative://external-editor/usage` contains the short service instructions; `genarrative://external-editor/openapi` contains the REST contract. This entry is available at `genarrative://external-editor/skill`. Reading a resource does not install the downloadable Skill or its Python helper.
|
||||
|
||||
- `genarrative://external-editor/skill`
|
||||
- `genarrative://external-editor/skill/references/capability-routing.md`
|
||||
- `genarrative://external-editor/skill/references/api-operations.md`
|
||||
- `genarrative://external-editor/skill/references/authentication-and-safety.md`
|
||||
- `genarrative://external-editor/skill/references/requests-and-outputs.md`
|
||||
- `genarrative://external-editor/openapi`
|
||||
## Direct REST and Local Helpers
|
||||
|
||||
## Hosted Integration Discovery
|
||||
When remote MCP is unavailable or local-file orchestration needs a helper, the complete package is available at `GET /api/external/v1/skill.zip`; the raw entry is at `GET /api/external/v1/skill/SKILL.md`. Verify the archive SHA-256 against the integration manifest before installing. The archive includes this entry, four references, `scripts/genarrative_external_api.py`, and `agents/openai.yaml`. Discovery and documentation downloads are public; MCP and business calls require authentication.
|
||||
|
||||
- Manifest: `GET /api/external/v1/agent-integration.json`.
|
||||
- Hosted MCP: `POST /api/external/v1/mcp`, Streamable HTTP, same Bearer API Key.
|
||||
- OpenAPI: `GET /api/external/v1/openapi.json`.
|
||||
- Raw Skill entry: `GET /api/external/v1/skill/SKILL.md`.
|
||||
- Complete Skill archive: `GET /api/external/v1/skill.zip`.
|
||||
|
||||
The archive contains this main file, four one-level references, the Python helper, and `agents/openai.yaml`. Verify its SHA-256 against `agent-integration.json` before installing. Discovery, OpenAPI, and Skill downloads are public; MCP and business operations require authentication.
|
||||
|
||||
## Python Helper
|
||||
|
||||
Store the API Key outside the repository at `~/.config/genarrative/external-editor-api.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"apiKey": "tnr_sk_..."
|
||||
}
|
||||
```
|
||||
|
||||
Set restrictive permissions where possible, then smoke-test without printing the key:
|
||||
The Python stdlib helper reads the private API Key file described in [authentication and safety](references/authentication-and-safety.md) and uses the production base URL. For a read-only smoke test:
|
||||
|
||||
```bash
|
||||
chmod 600 ~/.config/genarrative/external-editor-api.json
|
||||
python3 .codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py list-projects
|
||||
```
|
||||
|
||||
For a canvas-backed generation:
|
||||
Its `prepare_canvas_session` convenience method creates or reuses a project and a same-name folder. Use it only when that organization matches the task; it is not a prerequisite for MCP or REST calls. Convenience generation methods wait locally while the server uses short asynchronous submit/status requests. Use `submit_generation`, `get_generation`, and `wait_for_generation` for caller-controlled orchestration; see [requests and outputs](references/requests-and-outputs.md).
|
||||
|
||||
```python
|
||||
from genarrative_external_api import GenarrativeExternalClient
|
||||
|
||||
client = GenarrativeExternalClient()
|
||||
session = client.prepare_canvas_session("新画板")
|
||||
client.generate_image(
|
||||
"生成一张 16:9 幻想森林游戏背景",
|
||||
canvasSession=session,
|
||||
assetLabel="森林背景",
|
||||
aspectRatio="16:9",
|
||||
imageSize="1K",
|
||||
artSpec={
|
||||
"assetType": "background",
|
||||
"subject": "幻想森林主视觉",
|
||||
"style": "手绘游戏概念图",
|
||||
"palette": "翡翠绿与金色光斑",
|
||||
"composition": "横版，中心留出角色站位",
|
||||
"format": "16:9, 1K",
|
||||
"constraints": "无文字、无 UI 按钮",
|
||||
"references": [],
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
For background removal, pass a stable owner-scoped object key, project resource ID, or asset ID; the helper keeps the same asynchronous submission and polling contract:
|
||||
|
||||
```python
|
||||
session = client.prepare_canvas_session("去背景画布")
|
||||
client.remove_background(
|
||||
"editor-upload/object.png",
|
||||
source_width=720,
|
||||
source_height=1280,
|
||||
canvasSession=session,
|
||||
assetLabel="去背景结果",
|
||||
)
|
||||
```
|
||||
|
||||
Background removal preserves the source pixel size. For normal canvas placement with `canvasSession`, pass the real `source_width` and `source_height`, or provide both `canvasWidth` and `canvasHeight`; the helper rejects missing dimensions instead of guessing a square placeholder. `assetKind` may only describe a static image and must match the authoritative source record. Prefer a project resource ID or asset ID when the same object key has multiple semantic registrations; for a raw object key outside in-place replacement, pass `sourceResourceId` to disambiguate. Passing `targetLayerId` selects in-place replacement: the helper retains the session's project/library context but does not inject `canvasCompletion`, and it rejects an explicit `canvasCompletion` combined with `targetLayerId`. The target layer must point to the same authoritative object as the source, and the server durably binds a raw object key to that target resource for Worker revalidation.
|
||||
|
||||
Helper convenience methods wait locally, but the server still uses short asynchronous submit/status requests. For durable caller-controlled orchestration, call `submit_generation`, persist its `operationId` and idempotency key, then call `get_generation` or `wait_for_generation`.
|
||||
|
||||
For character animation, pass the canvas session and asset label to `animate_character`. The helper submits asynchronously and returns the completed compact result containing the authoritative formal `resource` and `asset`; do not synthesize a library asset from the first frame.
|
||||
|
||||
## Guardrails
|
||||
|
||||
- Do not change the fixed production base URL in generated examples.
|
||||
- Do not move the API Key into environment variables, source files, generated projects, logs, docs, screenshots, or shell snippets containing literal secrets.
|
||||
- Do not treat a Data URL, Blob URL, expiring signed URL, worker lease, or provider diagnostic as a durable result.
|
||||
- Do not reconstruct authoritative canvas, resource, or library snapshots from a compact generation response.
|
||||
- Do not replace icon-spritesheet generation with ordinary image generation when the deliverable requires a reusable transparent atlas.
|
||||
Stay within `/api/external/v1` for this integration. Internal workers, queues, admin/profile APIs, and SpacetimeDB endpoints are outside this contract.
|
||||
|
||||
@@ -4,6 +4,44 @@ Use this reference after selecting a capability. Treat `GET /api/external/v1/ope
|
||||
|
||||
All paths below are relative to `https://www.genarrative.world`. Discovery and Skill download routes are public. Project, asset, upload, generation, and generation-query operations require the Bearer API Key.
|
||||
|
||||
## MCP Tool to API Map
|
||||
|
||||
The hosted MCP offers the following tools. Choose the task tool when its action matches the request; the operation tool calls the indicated REST operation directly. Task tools with actions take `{ "action": "...", "input": { ... } }`; tools without actions take the operation fields directly. `idempotencyKey` is top-level in task tools. Operation tools use `body`, `pathParameters`, and `queryParameters` wrappers from their live input schemas. Read the live tool schema and OpenAPI for exact required fields.
|
||||
|
||||
| REST operation | Task tool (action) | Operation tool |
|
||||
| --- | --- | --- |
|
||||
| `GET /api/external/v1/openapi.json` | — | `get_external_open_api_json` |
|
||||
| `GET /api/external/v1/editor/projects` | `find_canvas_projects` (`list`) | `list_editor_projects` |
|
||||
| `GET /api/external/v1/editor/projects/recent` | `find_canvas_projects` (`recent`) | `load_recent_editor_project` |
|
||||
| `GET /api/external/v1/editor/projects/{projectId}` | `find_canvas_projects` (`get`), `find_assets` (`get_project_resources`), `edit_canvas` (`get`) | `get_editor_project` |
|
||||
| `POST /api/external/v1/editor/projects` | `manage_canvas_projects` (`create`) | `create_editor_project` |
|
||||
| `PATCH /api/external/v1/editor/projects/{projectId}/metadata` | `manage_canvas_projects` (`rename`) | `rename_editor_project` |
|
||||
| `DELETE /api/external/v1/editor/projects/{projectId}` | `delete_resources` (`delete_project`) | `delete_editor_project` |
|
||||
| `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | `edit_canvas` (`save_layout`) | `save_editor_project_canvas` |
|
||||
| `POST /api/external/v1/editor/projects/{projectId}/resources` | `edit_canvas` (`register_resource`) | `create_editor_project_resource` |
|
||||
| `POST /api/external/v1/assets/direct-upload-tickets` | `prepare_asset_upload` (`create_upload_ticket`) | `create_external_direct_upload_ticket` |
|
||||
| `POST /api/external/v1/assets/objects/confirm` | `prepare_asset_upload` (`confirm_upload`) | `confirm_external_asset_object` |
|
||||
| `GET /api/external/v1/assets/read-url` | `find_assets` (`get_download_url`) | `get_external_asset_read_url` |
|
||||
| `GET /api/external/v1/editor/assets/library` | `find_assets` (`list_library`) | `get_editor_asset_library` |
|
||||
| `POST /api/external/v1/editor/assets/folders` | `organize_asset_library` (`create_folder`) | `create_editor_asset_folder` |
|
||||
| `PATCH /api/external/v1/editor/assets/folders/{folderId}` | `organize_asset_library` (`update_folder`) | `update_editor_asset_folder` |
|
||||
| `DELETE /api/external/v1/editor/assets/folders/{folderId}` | `delete_resources` (`delete_folder`) | `delete_editor_asset_folder` |
|
||||
| `POST /api/external/v1/editor/assets` | `organize_asset_library` (`create_asset`) | `create_editor_asset` |
|
||||
| `PATCH /api/external/v1/editor/assets/{assetId}` | `organize_asset_library` (`update_asset`) | `update_editor_asset` |
|
||||
| `DELETE /api/external/v1/editor/assets/{assetId}` | `delete_resources` (`delete_asset`) | `delete_editor_asset` |
|
||||
| `POST /api/external/v1/editor/images/generations` | `generate_image`, `modify_image` (`variation`, fixed `kind="quick-edit"`) | `generate_external_editor_image` |
|
||||
| `POST /api/external/v1/editor/images/edits` | `modify_image` (`edit`) | `edit_external_editor_image` |
|
||||
| `POST /api/external/v1/editor/images/background-removals` | `modify_image` (`remove_background`) | `remove_external_editor_image_background` |
|
||||
| `POST /api/external/v1/editor/icon-spritesheets/generations` | `generate_icon_spritesheet` | `generate_external_editor_icon_spritesheet` |
|
||||
| `POST /api/external/v1/editor/ui-designs/assets/extractions` | `extract_ui_assets` | `extract_external_editor_ui_design_assets` |
|
||||
| `POST /api/external/v1/editor/character-animations/generations` | `generate_character_animation` | `generate_external_editor_character_animation` |
|
||||
| `POST /api/external/v1/editor/videos/generations` | `generate_video` | `generate_external_editor_video` |
|
||||
| `POST /api/external/v1/editor/audios/sound-effects/generations` | `generate_audio` (`sound_effect`) | `generate_external_editor_sound_effect` |
|
||||
| `POST /api/external/v1/editor/audios/background-music/generations` | `generate_audio` (`background_music`) | `generate_external_editor_background_music` |
|
||||
| `GET /api/external/v1/generations/{operationId}` | `check_generation` | `get_external_editor_generation_job` |
|
||||
|
||||
The public `agent-integration.json`, `skill/SKILL.md`, and `skill.zip` routes and the MCP transport route are HTTP entry points, not callable MCP tools. The hosted resource URIs remain `genarrative://external-editor/skill`, `genarrative://external-editor/skill/references/capability-routing.md`, `genarrative://external-editor/skill/references/api-operations.md`, `genarrative://external-editor/skill/references/authentication-and-safety.md`, `genarrative://external-editor/skill/references/requests-and-outputs.md`, and `genarrative://external-editor/openapi`.
|
||||
|
||||
## Project and Canvas Operations
|
||||
|
||||
| Operation | Method and path | Minimum input |
|
||||
@@ -23,7 +61,7 @@ Project listing supports two views:
|
||||
|
||||
- `view=full` is the REST default and returns the complete project, canvas, layers, and resources.
|
||||
- `view=summary` returns only `projectId`, `title`, `updatedAt`, and nullable `cover`, so callers can display, search, disambiguate same-name projects, and select a safe target without loading every canvas snapshot.
|
||||
- Hosted MCP `list_editor_projects` always uses `summary`; call `get_editor_project` after selecting a `projectId` when complete authoritative state is required.
|
||||
- Hosted MCP `list_editor_projects` and `find_canvas_projects` (`list`) use `summary`; call `get_editor_project` or `find_canvas_projects` (`get`) after selecting a `projectId` when complete authoritative state is required.
|
||||
- `cover` contains only `resourceId`, stable `objectKey`, dimensions, and `updatedAt`. It never embeds image bytes, a Data URL, or a signed URL. To display it, pass `cover.objectKey` to `get_external_asset_read_url`; signed URLs are temporary and must not be persisted or reused as generation references.
|
||||
|
||||
## Asset and Upload Operations
|
||||
@@ -42,6 +80,7 @@ Project listing supports two views:
|
||||
| Delete asset record | `DELETE /api/external/v1/editor/assets/{assetId}` | `assetId` |
|
||||
|
||||
Upload is a three-step client flow: create a ticket, POST the file and returned fields directly to the OSS form endpoint, then confirm the returned `objectKey`. See `authentication-and-safety.md` before implementing this flow.
|
||||
`prepare_asset_upload` handles the ticket and confirmation as separate calls; it does not send local bytes to OSS or automatically register a project resource, asset record, or canvas layer. `manage_canvas_projects` (`create`) likewise does not create a same-name asset folder. Register or organize records only when the task needs them.
|
||||
|
||||
## Generation Operations
|
||||
|
||||
@@ -52,7 +91,7 @@ Every generation row requires a stable `Idempotency-Key` header and returns HTTP
|
||||
| Image generation | `/api/external/v1/editor/images/generations` | `prompt` | `kind`, `style`, `model`, `aspectRatio`, `imageSize`, `size`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` |
|
||||
| Image edit/redraw | `/api/external/v1/editor/images/edits` | `prompt`, `sourceReferenceId` | `referenceImageSrcs`, `model`, `size`, `projectId`, `assetFolderId`, `assetLabel`, `targetLayerId`, `canvasCompletion` |
|
||||
| Background removal | `/api/external/v1/editor/images/background-removals` | `sourceImageSrc` | `projectId`, `sourceResourceId`, `targetLayerId`, static-image `assetKind`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` |
|
||||
| Icon spritesheet | `/api/external/v1/editor/icon-spritesheets/generations` | `referenceId`, `iconDescriptions` | `sliceMode`, `gridX`, `gridY`, `sliceCount`, `style`, `referenceImageSrcs`, `screenColor`, `model`, `aspectRatio`, `imageSize`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` |
|
||||
| Icon spritesheet | `/api/external/v1/editor/icon-spritesheets/generations` | `referenceId`, `iconDescriptions`, `sliceMode` | `gridX`, `gridY`, `sliceCount`, `style`, `referenceImageSrcs`, `screenColor`, `model`, `aspectRatio`, `imageSize`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` |
|
||||
| UI asset extraction | `/api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize` | `screenColor`, `model`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `spritesheetLabel`, `canvasCompletion` |
|
||||
| Character animation | `/api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `assetFolderId`, `assetLabel`, `canvasCompletion` |
|
||||
| Video generation | `/api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | `referenceImageSrcs`, `referenceVideoSrcs`, `referenceAudioSrcs`, `webSearchEnabled`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` |
|
||||
@@ -69,10 +108,10 @@ Supply the `operationId` returned by submission. Poll no faster than `pollAfterM
|
||||
|
||||
## Canvas and Library Field Rules
|
||||
|
||||
- Pass `projectId` and `canvasCompletion` to write generated output into the canvas.
|
||||
- Pass `assetFolderId` plus `assetLabel` for image, edit, icon spritesheet, video, sound effect, and BGM operations when supported.
|
||||
- Pass `projectId` and `canvasCompletion` when the task calls for generated output in a canvas.
|
||||
- Pass `assetFolderId` plus the relevant label field when the task calls for a library record. Neither destination requires the other, and their names need not match.
|
||||
- UI extraction uses `assetFolderId` and `spritesheetLabel`.
|
||||
- Character animation accepts `assetFolderId` and `assetLabel`. Its completed compact result directly returns the final `assetKind="character-animation"` resource and asset with `imageSequenceFrames` and `imageSequenceDurationMs`; never create a duplicate first-frame resource or asset.
|
||||
- Character animation accepts `assetFolderId` and `assetLabel` and persists the generated sequence. Consume returned artifacts and persisted identities; never create a duplicate first-frame resource or asset.
|
||||
- Background removal derives the final static-image `assetKind` from the authoritative source record. A conflicting request kind or any video, audio, animation, or image-sequence kind returns `400` before queueing. Without `canvasCompletion`, `targetLayerId` must point to the same authoritative object as `sourceImageSrc` (prefer `assetObjectId`, otherwise canonical bucket/object key).
|
||||
- If a caller must manually create a `character-animation` resource or asset, put the authoritative frames and total sequence duration in `imageSequenceFrames` and `imageSequenceDurationMs`. Keep `generationInputs` replayable: it must not contain legacy runtime fields such as `characterAnimation`, `frames`, `previewVideoPath`, `frameCount`, `fps`, or `durationSeconds`.
|
||||
- Reload project/library state after completion when full current state is required.
|
||||
|
||||
+15
-7
@@ -21,7 +21,7 @@ Authorization: Bearer <tnr_sk_...>
|
||||
|
||||
Guide a logged-in user to create a key in the product UI under `开发者 API Key`. The raw key is shown only once. Never ask the user to paste it into chat.
|
||||
|
||||
Store it outside repositories in the user's private JSON file:
|
||||
For hosted MCP, configure the Bearer token in the client's private connection settings. For the bundled REST helper, store it outside repositories in the user's private JSON file:
|
||||
|
||||
```text
|
||||
~/.config/genarrative/external-editor-api.json
|
||||
@@ -61,7 +61,7 @@ The OpenAPI document, integration manifest, raw Skill entry, and Skill archive a
|
||||
|
||||
For each logical generation:
|
||||
|
||||
1. Create one printable ASCII `Idempotency-Key` of 1-128 bytes.
|
||||
1. Create one printable ASCII key of 1-128 bytes. MCP takes `idempotencyKey` at the top level of the tool arguments (outside `input`); REST takes the `Idempotency-Key` header.
|
||||
2. Persist the key with the exact request body and returned `operationId`.
|
||||
3. If submission transport fails or the response is lost, resend only the exact same body with the same key.
|
||||
4. Never allocate a new key merely because the outcome is unknown.
|
||||
@@ -69,15 +69,21 @@ For each logical generation:
|
||||
|
||||
Treat a different body under the same key as invalid. Do not automatically replay a failed terminal generation unless the user intentionally requests a new logical generation.
|
||||
|
||||
Keep the API operation as well as the request and key unchanged across a submission retry. A different tool name does not create a separate idempotency namespace. When an operation ID is known, query `check_generation` directly. A rejected submission is not permission to switch keys and generate again.
|
||||
|
||||
`manage_canvas_projects/create`, `edit_canvas/register_resource`, and `organize_asset_library/create_folder` accept optional top-level `idempotencyKey`. Other non-generation actions do not accept it; in particular, `create_asset` is not an idempotent generation submission.
|
||||
|
||||
## Local Reference Upload
|
||||
|
||||
Do not ask the user to convert local files to base64. Upload from the Agent/client machine:
|
||||
|
||||
1. Detect the original filename, MIME type, byte length, and image dimensions when relevant.
|
||||
2. Create a ticket with `POST /api/external/v1/assets/direct-upload-tickets`.
|
||||
2. Call `prepare_asset_upload` with `action=create_upload_ticket` and the ticket body in `input` (REST: `POST /api/external/v1/assets/direct-upload-tickets`).
|
||||
3. POST all returned non-null `formFields` and the file part named `file` directly to `upload.host`.
|
||||
4. Confirm the object with `POST /api/external/v1/assets/objects/confirm`.
|
||||
5. Pass the confirmed stable `objectKey` to the selected editor operation.
|
||||
4. Call `prepare_asset_upload` with `action=confirm_upload` and the confirmation body in `input` (REST: `POST /api/external/v1/assets/objects/confirm`).
|
||||
5. Pass the confirmed stable `objectKey` where the selected tool permits it. For operations requiring a registered source, register a project resource or asset first and use its ID.
|
||||
|
||||
The MCP tool does not transfer file bytes and does not accept a local path or base64. The client needs an HTTP/file-transfer capability for step 3. Object ownership comes from the API Key; do not supply `ownerUserId`. Confirmation alone creates neither a library record nor a canvas layer.
|
||||
|
||||
For a private reference image, use a ticket body shaped like:
|
||||
|
||||
@@ -107,7 +113,7 @@ Confirm with the actual file metadata:
|
||||
|
||||
`contentLength` is a JSON number in bytes, not a quoted string. Never invent `sourceWidth` or `sourceHeight`; read them from the local image or ask the user if they cannot be determined.
|
||||
|
||||
For character animation, reuse a real canvas layer ID when available. For a local-only source, derive a stable synthetic `sourceLayerId`, such as `external-reference-hero`, from the filename and keep it unchanged across retries.
|
||||
For character animation, use source identity and dimensions from the actual selected resource; do not invent an existing canvas layer. The bundled local-file helper can maintain its own stable source label, which is not evidence of a persisted canvas layer.
|
||||
|
||||
The bundled helper implements ticket creation, a stdlib multipart upload, confirmation, dimension detection for common formats, and stable source-layer IDs:
|
||||
|
||||
@@ -126,7 +132,7 @@ Do not print the complete confirmation response if it may contain temporary acce
|
||||
- Use `objectKey`, project resource ID, asset ID, or an allowed durable public URL for generation input.
|
||||
- Use a Data URL only when the endpoint explicitly allows it and the caller has a deliberate reason; do not persist it as a durable output.
|
||||
- Never use a Blob URL outside the browser process that created it.
|
||||
- Use `GET /api/external/v1/assets/read-url` to obtain a short-lived `signedUrl` for display/download.
|
||||
- Use `find_assets` with `action=get_download_url` (REST: `GET /api/external/v1/assets/read-url`) to obtain a short-lived `signedUrl` for display/download. This returns a URL; the client still performs any download.
|
||||
- Never store or feed an expiring signed URL back into generation when a stable `objectKey` exists.
|
||||
|
||||
## Logging and Command Safety
|
||||
@@ -139,6 +145,8 @@ Do not print the complete confirmation response if it may contain temporary acce
|
||||
|
||||
## Scope and Retry Guardrails
|
||||
|
||||
- Generation spends account credits. Respect the user's authorized task and scope; do not restart generation merely because a requested derivative is missing.
|
||||
- For `delete_resources`, identify the precise IDs and obtain authorization for the actual deletion scope. Project deletion cascades to its default canvas and project-resource metadata. Folder deletion moves its assets to the default folder; the default folder cannot be deleted. Deleting a folder or asset record does not delete the underlying OSS file.
|
||||
- Do not use account JWT/profile endpoints as the default external integration. Logged-in profile APIs may create/revoke developer keys, but they are outside this external editor contract.
|
||||
- Do not call internal workers, queues, SpacetimeDB, or admin endpoints.
|
||||
- Do not bypass upload confirmation or invent an object key.
|
||||
|
||||
@@ -10,21 +10,17 @@ Use this reference to translate user intent into a hosted MCP tool or its corres
|
||||
- Public contract: `GET /api/external/v1/openapi.json`.
|
||||
- Skill fallback: `GET /api/external/v1/skill/SKILL.md` or `GET /api/external/v1/skill.zip`.
|
||||
|
||||
Prefer MCP when the Agent supports a remote endpoint plus a custom Bearer token. Prefer the complete Skill and Python helper when MCP is unavailable or a client-side local-file upload must be orchestrated. The MCP tool names are derived from OpenAPI `operationId` values in snake case; select by capability instead of memorizing the name.
|
||||
Prefer MCP when the Agent supports a remote endpoint plus a custom Bearer token. Prefer the complete Skill and Python helper when MCP is unavailable or a client-side local-file upload must be orchestrated. Choose a task-oriented tool below for ordinary requests, or the corresponding operation tool in [API Operations](api-operations.md) when the request needs direct control of one REST call. All 44 tools remain available. Discover the live tool schema before calling it; OpenAPI remains the field-level authority. The public discovery and Skill download routes are listed below, but are not MCP tools.
|
||||
|
||||
## Canvas Session
|
||||
## Project and Asset Destination
|
||||
|
||||
Before the first generation in a new conversation, obtain a canvas name unless the user already supplied an existing `projectId` and `assetFolderId`.
|
||||
Use `find_canvas_projects` (`action=list`, `recent`, or `get`) to locate an existing canvas when the request involves one. Use `manage_canvas_projects` (`create` or `rename`) only when the user needs a project created or renamed. A new project does not create an asset folder automatically. Use `find_assets` and `organize_asset_library` when the task involves library records or folders. A project and folder may have different names, and either may be unnecessary for a standalone generation.
|
||||
|
||||
1. List or create a project. When creating one, use the canvas name as `title`.
|
||||
2. Read the asset library. Reuse a folder with the same label or create one with the canvas name.
|
||||
3. Retain `canvasName`, `projectId`, `assetFolderId`, and the current art spec in conversation state.
|
||||
|
||||
Generated artifacts must enter both the current canvas and its same-name library folder whenever the endpoint supports that invariant. Pass `projectId`, `assetFolderId`, the endpoint's label field, and `canvasCompletion`. Character animation returns the final formal resource and asset directly; use those records and never create a duplicate from the first frame.
|
||||
For generation, pass `projectId` with `canvasCompletion` when the result should enter a canvas, and `assetFolderId` with the endpoint's label field when it should enter the library. Use both only when the task requires both destinations. Character animation returns its final resource and asset directly when those destinations are requested; do not duplicate its first frame.
|
||||
|
||||
## Art Spec Routing
|
||||
|
||||
Before art generation, normalize the user's request into:
|
||||
For a series of related art requests, an optional reusable spec can carry the shared requirements:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -43,48 +39,48 @@ Infer what is already clear and ask only for missing fields that block the selec
|
||||
|
||||
## Intent Map
|
||||
|
||||
| User intent | MCP/REST capability |
|
||||
| ----------------------------------------------------------------------- | ---------------------------------------------------------------- |
|
||||
| Generate a background, character, spec, UI mockup, or publication image | Image generation |
|
||||
| Redraw, retouch, or replace an existing image | Image edit |
|
||||
| Remove the background from an existing image | Background removal |
|
||||
| Generate from a local reference | Upload and confirm the local file, then image generation or edit |
|
||||
| Build a reusable transparent icon/game atlas from a visual spec | Icon spritesheet generation |
|
||||
| Extract marked assets from an existing UI design | UI design asset extraction |
|
||||
| Animate a character into frames | Character animation generation |
|
||||
| Generate video | Video generation |
|
||||
| Generate a sound effect | Sound-effect generation |
|
||||
| Generate background music/BGM | Background-music generation |
|
||||
| Upload a local image/audio/video asset | Upload ticket -> OSS form upload -> object confirm |
|
||||
| Save viewport/layers | Canvas save |
|
||||
| Create, load, rename, or delete a canvas | Project operations |
|
||||
| Organize folders and asset records | Asset-library operations |
|
||||
| Obtain temporary access to private media | Signed read URL |
|
||||
| Check generation progress or retrieve its result | Generation query |
|
||||
| User intent | MCP tool and action |
|
||||
| --- | --- |
|
||||
| Find, open, create, or rename a canvas project | `find_canvas_projects` (`list`, `recent`, `get`); `manage_canvas_projects` (`create`, `rename`) |
|
||||
| Read project resources or library records | `find_assets` (`get_project_resources`, `list_library`) |
|
||||
| Create or change folders and asset records | `organize_asset_library` (`create_folder`, `update_folder`, `create_asset`, `update_asset`) |
|
||||
| Upload a local image/audio/video asset | `prepare_asset_upload` (`create_upload_ticket`), client-side OSS form upload, then `prepare_asset_upload` (`confirm_upload`) |
|
||||
| Register existing media in a project, read a canvas, or save its full layout | `edit_canvas` (`register_resource`, `get`, `save_layout`) |
|
||||
| Generate a background, character, spec, UI mockup, or publication image | `generate_image` |
|
||||
| Retouch an existing image, make a reference variation, or remove its background | `modify_image` (`edit`, `variation`, `remove_background`) |
|
||||
| Build a transparent icon/game atlas from a registered visual spec | `generate_icon_spritesheet` |
|
||||
| Generate marked assets from an existing UI design | `extract_ui_assets` |
|
||||
| Animate a character into frames | `generate_character_animation` |
|
||||
| Generate video | `generate_video` |
|
||||
| Generate a sound effect or background music | `generate_audio` (`sound_effect`, `background_music`) |
|
||||
| Check generation progress or retrieve its result | `check_generation` |
|
||||
| Obtain temporary access to private media | `find_assets` (`get_download_url`) |
|
||||
| Delete an exact project, folder, or asset record | `delete_resources` (`delete_project`, `delete_folder`, `delete_asset`) |
|
||||
|
||||
For tools with actions, send `{ "action": "...", "input": { ... } }`; place `idempotencyKey` at the top level when supported or required. Tools without actions accept operation fields directly, with `idempotencyKey` at the top level for generation. The direct operation tools use `body`, `pathParameters`, and `queryParameters` wrappers as shown by their live schemas.
|
||||
|
||||
Do not present an API menu unless the request is genuinely ambiguous. Ask a follow-up when two routes create different artifacts, for example “处理这张图” could mean edit, extract marked UI assets, or use it as a reference for a new generation.
|
||||
|
||||
## Route-Specific Decisions
|
||||
|
||||
- Use image edit when the requested output replaces or modifies a source image. With `projectId`, pass `targetLayerId` to replace an existing layer when no explicit `canvasCompletion` is supplied.
|
||||
- Use `modify_image` `edit` when the requested output modifies a registered source image. Use `variation` when reference images should guide a new `quick-edit` image; it is image generation with fixed `kind="quick-edit"`. Use `remove_background` for a static source image. With `projectId`, `targetLayerId` may replace a matching existing layer when no explicit `canvasCompletion` is supplied.
|
||||
- Use icon spritesheet generation for a transparent reusable atlas when a stable visual-spec reference and concrete `iconDescriptions` exist. Do not use ordinary image generation just because it can draw several objects.
|
||||
- Use UI extraction only for an existing UI design image with red-box annotations. It is not UI generation.
|
||||
- Use a project layer ID as character animation `sourceLayerId` when one exists. For a local-only source, derive a stable synthetic ID from the filename.
|
||||
- Character animation requires a real `sourceLayerId`, source image, and dimensions from an existing resource. A local-only file must first be uploaded and registered where needed; do not invent a layer ID.
|
||||
- For video with image/video/audio references, use a Seedance 2.0-family model; default to `seedance2.0-fast`, `mode: "std"`, and explicit `sound`.
|
||||
- Use `signedUrl` only for preview/download. Feed stable `objectKey` or registered resource/asset identifiers into generation.
|
||||
- `prepare_asset_upload` obtains a ticket and confirms an uploaded object; it does not transfer the file or register a project resource, asset record, or canvas layer. Use `edit_canvas` `register_resource` or `organize_asset_library` `create_asset` only when the task needs those records.
|
||||
- Use temporary signed URLs only for preview/download. Feed stable `objectKey` or registered resource/asset identifiers into generation as each operation permits.
|
||||
|
||||
## AI Game Creator Canonical Visual DAG
|
||||
## Example: Reusable Icon Assets
|
||||
|
||||
Keep the existing autonomous-build task graph. Do not add a parallel task system or collapse these artifacts into one ordinary generation request:
|
||||
When the task needs a visual spec and a reusable icon atlas:
|
||||
|
||||
1. `art-director` generates `assets/art-spec.png` with image generation, `kind: "spec"`, then registers it as `assetKind: "icon-spec"`. This image is the authoritative visual spec; `generationInputs.artSpec` is supporting structured context.
|
||||
2. `design-foundation` generates `assets/ui-prototype.png` with `kind: "ui-design"`, using the registered art-spec resource ID in `referenceImageSrcs`.
|
||||
3. `art-asset-plan` generates transparent `assets/art-spritesheet.png` through icon spritesheet generation, using the same registered art-spec resource ID as `referenceId` plus concrete `iconDescriptions`. `sliceMode` is required and has no default: send `sliceMode: "grid"` with `gridX`/`gridY` only when the requirement itself fixes the slots or names the column/row count, and otherwise send `sliceMode: "connected-components"` (with `sliceCount` when a subject count must be constrained); never invent a grid to express "kinds of assets", and never send `gridX`/`gridY` with `connected-components`.
|
||||
1. Reuse an existing registered `icon-spec`, or generate the requested spec using `generate_image` with `kind=spec` and register it as `assetKind=icon-spec` if necessary.
|
||||
2. Call `generate_icon_spritesheet` with that registered ID as `referenceId`, concrete `iconDescriptions`, and explicit `sliceMode`. Choose `grid` only for requested equal cells or fixed slots and provide those `gridX`/`gridY` values; otherwise use `connected-components`, optionally with `sliceCount`.
|
||||
3. Query `check_generation` and inspect the full sheet and actual slices. Preserve warnings; a usable full sheet does not imply that individual slices exist. Use returned slice identities and dimensions rather than guessing crop coordinates.
|
||||
|
||||
For a playable Canvas game, do not stop at generation. Make `code-prototype` depend on `art-asset-plan` and consume the persisted `iconImageSrcs` slices for core players, blocks or targets, scene obstacles, and feedback. When the requirement fixes grid slots, require the response `sliceMode` to match the declared `grid` request and exactly `gridX × gridY` slices before registering the local runtime sheet; a connected-components request is instead judged by its own `sliceCount` or by the requirement, and both fewer and extra components fail closed. Treat `art-spec.png` as reference-only. A full-sheet `<img>`, CSS background, path-only mention, guessed equal-grid crop, or code-drawn replacement for core entities is not runtime asset use. If slicing produces `sliceWarning`, keep the complete transparent sheet as a valid editor artifact, but fail the playable game asset gate until real slice files or verified atlas coordinates exist; never invent coordinates or replace the icon-spritesheet route with ordinary image generation.
|
||||
|
||||
Never use `assets/ui-prototype.png` as the spritesheet visual-spec reference. UI extraction is outside this canonical DAG.
|
||||
An ordinary UI mockup or uploaded image is not automatically an `icon-spec`. For extracting marked components from a UI design, use `extract_ui_assets`, which includes generation and does not promise pixel-exact cropping.
|
||||
|
||||
## Scope Boundary
|
||||
|
||||
Stay within `/api/external/v1`. Do not invent worker, queue, runtime task-list, admin, profile, or SpacetimeDB calls. The only external generation query is `GET /api/external/v1/generations/{operationId}`.
|
||||
Stay within `/api/external/v1`. Do not invent worker, queue, runtime task-list, admin, profile, or SpacetimeDB calls. The only external generation query is `GET /api/external/v1/generations/{operationId}`. The hosted MCP also exposes the Skill and OpenAPI resources at `genarrative://external-editor/skill`, its four `skill/references/*.md` URIs, and `genarrative://external-editor/openapi`; keep those URI names unchanged.
|
||||
|
||||
@@ -4,15 +4,46 @@ Use this reference to build generation payloads, carry canvas/library context, p
|
||||
|
||||
## Contents
|
||||
|
||||
- [MCP Argument Shapes](#mcp-argument-shapes)
|
||||
- [Asynchronous Submission](#asynchronous-submission)
|
||||
- [Polling State Machine](#polling-state-machine)
|
||||
- [Canvas and Asset-Library Completion](#canvas-and-asset-library-completion)
|
||||
- [Saving Existing Canvas Layout](#saving-existing-canvas-layout)
|
||||
- [Art Spec and Image Request](#art-spec-and-image-request)
|
||||
- [Local Reference Requests](#local-reference-requests)
|
||||
- [Compact Completed Result](#compact-completed-result)
|
||||
- [Warning Semantics](#warning-semantics)
|
||||
- [Output Handling Checklist](#output-handling-checklist)
|
||||
|
||||
## MCP Argument Shapes
|
||||
|
||||
Pass these objects as the `arguments` of the named tool in `tools/call`. They are not REST request envelopes.
|
||||
|
||||
Single-function tools take business fields directly. For example, `generate_image`:
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt": "一张横版幻想森林背景，无文字",
|
||||
"aspectRatio": "16:9",
|
||||
"imageSize": "1K",
|
||||
"idempotencyKey": "forest-image-001"
|
||||
}
|
||||
```
|
||||
|
||||
Multi-function tools take `action` and `input`. A generation key stays outside `input`. For example, `generate_audio`:
|
||||
|
||||
```json
|
||||
{
|
||||
"action": "sound_effect",
|
||||
"input": {"prompt": "轻柔的游戏菜单确认音", "duration": 1},
|
||||
"idempotencyKey": "menu-sound-001"
|
||||
}
|
||||
```
|
||||
|
||||
Keys above identify distinct example requests; create and persist your own key for each new logical generation. An action with no business fields still requires `input: {}`, such as `find_canvas_projects` with `action=list`. Use only fields belonging to the selected action; do not combine branches. REST examples below use the business body directly and put the key in the HTTP header instead.
|
||||
|
||||
MCP returns business data in `structuredContent`, without the REST `data` envelope. Check `isError` before using it; an HTTP-successful MCP exchange can still carry a tool error. Generation acceptance contains an `operationId`, not the final media.
|
||||
|
||||
## Asynchronous Submission
|
||||
|
||||
All nine generation POST routes require `Idempotency-Key` and return HTTP `202` with an `ExternalEditorGenerationSubmissionResponse` shaped like:
|
||||
@@ -50,7 +81,7 @@ Persist the key, exact request body, and `operationId`. If submission outcome is
|
||||
|
||||
## Polling State Machine
|
||||
|
||||
Poll `statusUrl`, or `GET /api/external/v1/generations/{operationId}`, no faster than `pollAfterMs`:
|
||||
With MCP, call `check_generation` with `{"operationId":"<returned operationId>"}`. Each call queries once and does not wait for completion. With REST, poll `statusUrl` or `GET /api/external/v1/generations/{operationId}`. Query no faster than `pollAfterMs`:
|
||||
|
||||
- `queued` / `running`: retain `operationId`; show `phaseLabel`, `phaseDetail`, and `progress` when present; wait before querying again.
|
||||
- `completed`: consume the compact `result` and all warning fields, then stop polling.
|
||||
@@ -75,10 +106,10 @@ Background removal uses the same submission and polling state machine. `sourceIm
|
||||
|
||||
## Canvas and Asset-Library Completion
|
||||
|
||||
For endpoints that support these fields, include:
|
||||
Choose destinations according to the task. Locate an existing project with `find_canvas_projects` and inspect folders with `find_assets/list_library`; create missing destinations with `manage_canvas_projects/create` and `organize_asset_library/create_folder` only when needed. Project creation does not create a folder. For endpoints that support the requested destinations, include:
|
||||
|
||||
- `projectId`: target canvas project.
|
||||
- `assetFolderId`: folder whose label matches the canvas name.
|
||||
- `assetFolderId`: target asset-library folder; its name need not match the project.
|
||||
- `assetLabel` or UI extraction's `spritesheetLabel`: user-visible library label.
|
||||
- `canvasCompletion`: backend canvas placement instructions.
|
||||
|
||||
@@ -102,15 +133,24 @@ A minimal `canvasCompletion` is:
|
||||
|
||||
Background removal preserves the source image dimensions. For normal canvas placement, the Python helper therefore requires the real `source_width` and `source_height` whenever `canvasSession` is used without an explicit `canvasWidth` plus `canvasHeight`; it never substitutes a square default. Passing `targetLayerId` instead selects in-place replacement, so the helper keeps the session's project/library fields without injecting `canvasCompletion` and rejects callers that explicitly combine both placement modes. The request `assetKind` is optional, static-image only, and must equal the authoritative source type when one exists. An in-place target must resolve to the same authoritative source object; a raw object key is bound to that target resource instead of relying on project-list order.
|
||||
|
||||
Character animation accepts `assetFolderId` and `assetLabel` and persists the final transparent sequence directly. Its completed compact result includes the authoritative `assetKind="character-animation"` resource and asset with `imageSequenceFrames` and `imageSequenceDurationMs`. Use those records directly and never synthesize a duplicate asset from the first frame.
|
||||
Character animation accepts `assetFolderId` and `assetLabel` and persists the generated sequence. Consume the returned animation artifacts and persisted identities; do not synthesize a duplicate animation asset from the first frame. Use complete project/library records when complete persisted state is needed.
|
||||
|
||||
For the lower-level asset/resource creation endpoints, `generationInputs` is replayable request context rather than a media-runtime container. When `assetKind` is `character-animation`, the server rejects legacy runtime keys including `characterAnimation`, `frames`, `previewVideoPath`, `frameCount`, `fps`, and `durationSeconds`; send the formal sequence through `imageSequenceFrames` and `imageSequenceDurationMs`. Internal processing audit keys such as `screenColorHex`, `mattingProvider`, and `mattingModel` are removed before persistence.
|
||||
|
||||
## Saving Existing Canvas Layout
|
||||
|
||||
1. Call `edit_canvas` with `action=get` and `input.projectId` to read the latest project and canvas revision.
|
||||
2. Build the intended complete `viewport` and `layers`, preserving unrelated layers. `save_layout` replaces the layout; it is not a one-layer patch.
|
||||
3. Call `edit_canvas` with `action=save_layout` and `input` containing `projectId`, the read `expectedRevision`, and the complete `viewport` and `layers`.
|
||||
4. On a revision conflict, reread and reconcile with the current layout before retrying. Do not blindly resend stale layers with a refreshed revision.
|
||||
|
||||
`edit_canvas/register_resource` registers existing media but does not create a canvas layer. `organize_asset_library/create_asset` creates metadata but does not upload or generate media. For generated media placement, prefer the generation tool's supported `canvasCompletion`; inspect returned identities before registering anything again.
|
||||
|
||||
## Art Spec and Image Request
|
||||
|
||||
Generic External v1 image generation does not expose the main-site structured game-scene contract. `kind: "scene"` and `assetKind: "scene"` are both invalid and return HTTP `400` before any generation job is queued. Do not replace the structured scene fields and server-owned prompt assembly with a generic image prompt.
|
||||
|
||||
Carry the current art spec in `generationInputs.artSpec` and reflect important constraints in the prompt:
|
||||
When maintaining a reusable art spec, carry it in `generationInputs.artSpec` and reflect important constraints in the prompt. This is an example with both canvas and library destinations, not a requirement for every generation:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -177,11 +217,11 @@ Image edit/redraw has a stricter main-source identity rule. After upload confirm
|
||||
|
||||
Icon spritesheet generation has a stricter primary-spec contract. After upload confirmation, create a project resource or asset record with `assetKind: "icon-spec"`, retain its returned `resourceId` or `assetId`, and pass that ID as `referenceId`. The primary spec does not accept the uploaded `objectKey` directly; only additional style references may continue to use stable object keys in `referenceImageSrcs`.
|
||||
|
||||
For character animation from a local-only source, use actual dimensions and a stable synthetic layer ID:
|
||||
For character animation, use the selected source identity and actual dimensions. The following is a business-body example; `generate_character_animation` also requires a top-level `idempotencyKey`:
|
||||
|
||||
```json
|
||||
{
|
||||
"sourceLayerId": "external-reference-hero",
|
||||
"sourceLayerId": "<selected source layer ID>",
|
||||
"sourceImageSrc": "<confirmed objectKey>",
|
||||
"sourceWidth": 720,
|
||||
"sourceHeight": 1280,
|
||||
@@ -209,7 +249,7 @@ The completed `result` may contain stable artifact fields such as:
|
||||
- `spritesheetResource`, `spritesheetAsset`, and stable spritesheet metadata.
|
||||
- `warning` and `sliceWarning` structures.
|
||||
|
||||
It deliberately excludes a complete project/canvas/library snapshot, Data URL, Blob URL, expiring signed URL, worker lease, queue state, and internal provider diagnostics. Use `/assets/read-url` for temporary access to a stable `objectKey`.
|
||||
It deliberately excludes a complete project/canvas/library snapshot, Data URL, Blob URL, expiring signed URL, worker lease, queue state, and internal provider diagnostics. Use `find_assets/get_download_url` for temporary access to a stable `objectKey` (REST: `/assets/read-url`). Reading a record or obtaining a URL does not itself inspect or download the media.
|
||||
|
||||
## Warning Semantics
|
||||
|
||||
|
||||
@@ -37,6 +37,45 @@ function normalizeMarkdownBlankLines(text: string) {
|
||||
.join('');
|
||||
}
|
||||
|
||||
/**
|
||||
* 把粘在正文行里的围栏拆到独立行。
|
||||
*
|
||||
* 模型经常把 ``` 直接粘在上一行末尾（`…实现细节（game.js）：```js`、`… }````），而
|
||||
* CommonMark 只认整行的围栏（最多 3 个空格缩进）：粘着的 ``` 退化成正文，于是正文被当成
|
||||
* 代码渲染，或者代码块一直不闭合、把后面所有内容一起吞进代码块。
|
||||
*
|
||||
* 判据收得很窄：围栏前面必须是非空白字符，且围栏到行尾只允许剩语言标识（可空）。
|
||||
* 这样整行围栏、缩进围栏、行内代码（单个反引号）都不受影响，代码里出现的 ``` 只要后面还有
|
||||
* 别的字符（`const s = "```";`）也不会被拆开。
|
||||
*
|
||||
* 引用块与列表项开头的围栏（`> ```js`、`- ```js`）是**合法结构**，不是粘住的：整行跳过，
|
||||
* 拆开只会把它们从引用块 / 列表项里挪出来。同一行出现第二段围栏串（行内代码
|
||||
* ``文本 ```x``` ``）时同样跳过——末尾那截不是收场围栏。
|
||||
*/
|
||||
const GLUED_FENCE_LINE =
|
||||
/([^\s`~])[ \t]*(`{3,}|~{3,})([A-Za-z0-9+#._-]*)[ \t]*$/;
|
||||
const BLOCK_MARKER_LINE = /^\s*(?:[-*+]|\d+[.)]|>)\s/;
|
||||
const FENCE_RUN_TOKEN = /(`{3,}|~{3,})/g;
|
||||
|
||||
function normalizeMarkdownFences(text: string) {
|
||||
return text
|
||||
.replace(/\r\n?/g, '\n')
|
||||
.split('\n')
|
||||
.map((line) => {
|
||||
if (BLOCK_MARKER_LINE.test(line)) {
|
||||
return line;
|
||||
}
|
||||
// 行里还有第二条围栏串时不动：那是行内代码（`文本 ```x``` `），末尾那截不是收场围栏。
|
||||
// 拆开它会凭空多出一个开场围栏，把后面的正文全变成代码。
|
||||
const runs = line.match(FENCE_RUN_TOKEN);
|
||||
if (runs && runs.length > 1) {
|
||||
return line;
|
||||
}
|
||||
return line.replace(GLUED_FENCE_LINE, '$1\n$2$3');
|
||||
})
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
type MarkdownErrorBoundaryProps = {
|
||||
fallbackText: string;
|
||||
children: ReactNode;
|
||||
@@ -249,7 +288,7 @@ const markdownComponents: Components = {
|
||||
</blockquote>
|
||||
),
|
||||
pre: ({ children }) => (
|
||||
<pre className="m-0 mt-2 max-w-full overflow-x-auto rounded-lg bg-black/6 p-3 text-xs leading-5 first:mt-0">
|
||||
<pre className="m-0 mt-2 max-w-full overflow-x-auto rounded-lg bg-black/6 p-3 text-xs leading-5 break-words whitespace-pre-wrap first:mt-0">
|
||||
<CodeBlockContext.Provider value={true}>
|
||||
{children}
|
||||
</CodeBlockContext.Provider>
|
||||
@@ -260,7 +299,7 @@ const markdownComponents: Components = {
|
||||
return isBlock ? (
|
||||
<code
|
||||
{...props}
|
||||
className={`agc-markdown-code font-mono whitespace-pre ${className ?? ''}`}
|
||||
className={`agc-markdown-code font-mono break-words whitespace-pre-wrap ${className ?? ''}`}
|
||||
>
|
||||
{children}
|
||||
</code>
|
||||
@@ -338,7 +377,10 @@ function ChatMarkdownMessageImpl({
|
||||
streaming ? streamingMarkdownComponents : markdownComponents
|
||||
}
|
||||
>
|
||||
{preserveBlankLines ? text : normalizeMarkdownBlankLines(text)}
|
||||
{/* 顺序有讲究：先拆粘住的围栏，空行压缩的 ` ```…``` ` 配对才认得出真正的代码块。 */}
|
||||
{preserveBlankLines
|
||||
? normalizeMarkdownFences(text)
|
||||
: normalizeMarkdownBlankLines(normalizeMarkdownFences(text))}
|
||||
</ReactMarkdown>
|
||||
</MarkdownErrorBoundary>
|
||||
);
|
||||
|
||||
@@ -12281,6 +12281,21 @@ button.design-workspace-tree__entry:hover,
|
||||
flex: 1 1 auto;
|
||||
}
|
||||
|
||||
/* 过程卡（「陶泥儿正在处理」）渲染在消息列表**外**、紧贴输入盒上方（固定在输入框上面，
|
||||
不随消息滚走）。它不再继承消息列表的左右 16px 内缩，所以要自己补齐，才能与消息内容、
|
||||
输入盒两侧对齐；离开列表后列表的 padding-bottom 也不再作用于它，与最后一条消息的间距
|
||||
同样由这里给。
|
||||
`flex: 0 0 auto` 是这条链上的承重项：卡片带 `overflow: hidden`，flex 项的自动最小尺寸
|
||||
因此归零，窗口变矮时它会先被挤扁、文字被裁掉（实测高度 300px 时压到 33px、240px 时
|
||||
24px），而这一列里只有消息列表该被压缩。 */
|
||||
.game-workbench-chat
|
||||
.project-chat-surface.is-direct-codex
|
||||
.project-chat-conversation
|
||||
> .project-chat-process-card {
|
||||
flex: 0 0 auto;
|
||||
margin: 12px 16px 8px;
|
||||
}
|
||||
|
||||
/* ============================================================
|
||||
工具调用折叠块（2026-09）：一回合一个块，Codex 风格
|
||||
============================================================
|
||||
@@ -12575,17 +12590,6 @@ button.design-workspace-tree__entry:hover,
|
||||
justify-content: flex-end;
|
||||
}
|
||||
|
||||
/* 过程卡（「陶泥儿正在处理」）现在渲染在消息列表**外**、紧贴输入盒上方（固定在输入框上面，
|
||||
不随消息滚走）。它不再继承消息列表的左右 16px 内缩，所以要自己补齐，才能与消息内容、
|
||||
输入盒两侧对齐；离开列表后列表的 padding-bottom 也不再作用于它，与最后一条消息的间距
|
||||
同样由这里给。 */
|
||||
.game-workbench-chat
|
||||
.project-chat-surface.is-direct-codex
|
||||
.project-chat-conversation
|
||||
> .project-chat-process-card {
|
||||
margin: 12px 16px 8px;
|
||||
}
|
||||
|
||||
/* 工具调用折叠块块头改成两行：第一行图标 + 汇总（超长省略），第二行状态 + 用时，箭头右侧跨两行。
|
||||
窄面板里一行塞不下（汇总会被截断、"进行中"还会折成两行），拆成两行后每段都有自己的宽度。 */
|
||||
.game-workbench-chat
|
||||
|
||||
+6
-2
@@ -171,8 +171,12 @@ export function DirectProjectChatView({
|
||||
turnBusy,
|
||||
turns: directTurns,
|
||||
});
|
||||
// 状态条的起点只认**未结束**的最新一轮：`awaiting-start`（本地已发出、宿主还没确认）
|
||||
// 也有用户发送时间，只读 `running` 会让卡片在模型首 token 之前根本不出现。
|
||||
// 只可能是最后一轮：原生 `turnRunning` 只赋给最新一轮，`awaiting-start` 也只判最新一轮。
|
||||
const latestTurn = directTurns.at(-1) ?? null;
|
||||
const activeTurnStartedAt =
|
||||
directTurns.find((turn) => turn.state === 'running')?.startedAt ?? 0;
|
||||
latestTurn && latestTurn.state !== 'finished' ? latestTurn.startedAt : 0;
|
||||
const statusText =
|
||||
runtimeNotice ||
|
||||
statusNotice ||
|
||||
@@ -248,7 +252,7 @@ export function DirectProjectChatView({
|
||||
turns={directTurns}
|
||||
messagesRef={messagesRef}
|
||||
historyHasMore={historyHasMore}
|
||||
nativeRunning={turnStatus.nativeRunning}
|
||||
turnInFlight={turnStatus.displayBusy}
|
||||
activeTurnStartedAt={activeTurnStartedAt}
|
||||
onLoadEarlierHistory={() => void loadEarlierHistory()}
|
||||
onScroll={handleScroll}
|
||||
|
||||
+7
-5
@@ -15,7 +15,7 @@ export function DirectProjectConversation({
|
||||
turns,
|
||||
messagesRef,
|
||||
historyHasMore,
|
||||
nativeRunning,
|
||||
turnInFlight,
|
||||
activeTurnStartedAt,
|
||||
onLoadEarlierHistory,
|
||||
onScroll,
|
||||
@@ -24,10 +24,12 @@ export function DirectProjectConversation({
|
||||
messagesRef: RefObject<HTMLDivElement | null>;
|
||||
historyHasMore: boolean;
|
||||
/**
|
||||
* 原生回合是否在跑（reducer 的 `turnRunning`）：只决定这张"正在处理"卡片。
|
||||
* 本地命令在飞但原生还没认领的窗口见 `DirectProjectTurnStatus`。
|
||||
* 这一轮在飞吗：`DirectProjectTurnStatus.displayBusy`（本地命令在飞 ∪ 原生已确认在跑）。
|
||||
*
|
||||
* 只认原生 `turnRunning` 会让卡片在「命令已发出、`turn.started` 未到」的空窗里不出现——
|
||||
* 模型首 token 之前那段（实测约十秒）界面就没有任何「正在处理」的交代。
|
||||
*/
|
||||
nativeRunning: boolean;
|
||||
turnInFlight: boolean;
|
||||
activeTurnStartedAt: number;
|
||||
onLoadEarlierHistory: () => void;
|
||||
onScroll: UIEventHandler<HTMLDivElement>;
|
||||
@@ -53,7 +55,7 @@ export function DirectProjectConversation({
|
||||
<DirectProjectTurn key={turn.key} turn={turn} />
|
||||
))}
|
||||
</div>
|
||||
{nativeRunning ? (
|
||||
{turnInFlight ? (
|
||||
<AgentMessageContent
|
||||
as="section"
|
||||
tone="process"
|
||||
|
||||
+5
-3
@@ -13,11 +13,13 @@ import type {
|
||||
* 一次讲清楚，组件只读这一个对象：
|
||||
*
|
||||
* - `nativeRunning`：**原生真相**。只由订阅 reducer 的 `turnRunning` 给出（`turn.started`
|
||||
* 已到、`turn.completed` 未到）。它决定"陶泥儿正在处理"这类原生过程提示。
|
||||
* 已到、`turn.completed` 未到）。
|
||||
* - `commandInFlight`：**本地真相**。本次会话的发送命令是否在飞（写权限门 → invoke →
|
||||
* 收尾）；它从按下发送那一刻就为真，与原生是否已经开始无关。
|
||||
* - `displayBusy`：header / composer 该读的忙态，就是两者的并集：只要有一条成立就不能再
|
||||
* 接受新的发送。
|
||||
* - `displayBusy`：header / composer / 「陶泥儿正在处理」卡片该读的忙态，就是两者的并集：
|
||||
* 只要有一条成立就不能再接受新的发送。卡片读它而不是 `nativeRunning`：`turn.started`
|
||||
* 要等宿主应答返回才发出，只认原生真相会让模型首 token 之前那十来秒没有任何「正在处理」
|
||||
* 的交代（2026-09-24 口令）。
|
||||
* - `latestTurnState`：最新一轮在界面上的三态（投影结果）；没有回合时为 null。
|
||||
*
|
||||
* 约定：新增"忙/在跑"类判据一律先落进这里，不要在组件里再拼布尔。
|
||||
|
||||
@@ -159,7 +159,7 @@ describe('ChatMarkdownMessage', () => {
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('无语言标记的多行围栏代码仍使用代码块样式', () => {
|
||||
it('无语言标记的多行围栏代码仍使用代码块样式，且允许自动换行', () => {
|
||||
const { container } = render(
|
||||
<ChatMarkdownMessage
|
||||
role="assistant"
|
||||
@@ -168,8 +168,94 @@ describe('ChatMarkdownMessage', () => {
|
||||
);
|
||||
|
||||
const code = container.querySelector('pre code');
|
||||
expect(code?.className).toContain('whitespace-pre');
|
||||
const pre = container.querySelector('pre');
|
||||
expect(code?.className).not.toContain('rounded');
|
||||
// 代码块必须自动换行：窄面板（280–420px）里长行会把消息拉宽并顶出横向滚动条，
|
||||
// `pre` 与块内 `code` 各自都写过 `white-space`，两处都要给。
|
||||
expect(code?.className).toContain('whitespace-pre-wrap');
|
||||
expect(code?.className).toContain('break-words');
|
||||
expect(pre?.className).toContain('whitespace-pre-wrap');
|
||||
expect(pre?.className).toContain('break-words');
|
||||
});
|
||||
|
||||
it('围栏粘在正文行末尾时仍开在正确位置', () => {
|
||||
const { container } = render(
|
||||
<ChatMarkdownMessage
|
||||
role="assistant"
|
||||
text={'实现细节（game.js）：```js\nconst answer = 42;\n```\n'}
|
||||
/>,
|
||||
);
|
||||
|
||||
// 开场围栏不拆开的话，整行会退化成「一段正文里跟着 ```js」，
|
||||
// 代码块根本不成立、后面的正文又会被当成代码。
|
||||
expect(container.querySelector('pre code')?.textContent).toBe(
|
||||
'const answer = 42;\n',
|
||||
);
|
||||
expect(container.querySelector('pre')?.textContent).not.toContain(
|
||||
'实现细节',
|
||||
);
|
||||
});
|
||||
|
||||
it('收场围栏粘在代码行末尾时不再把后续正文吞进代码块', () => {
|
||||
const { container } = render(
|
||||
<ChatMarkdownMessage
|
||||
role="assistant"
|
||||
text={'```js\nconst a = 1;\n}); }```\n\n后面还是正文\n'}
|
||||
/>,
|
||||
);
|
||||
|
||||
const code = container.querySelector('pre code');
|
||||
expect(code?.textContent).toBe('const a = 1;\n}); }\n');
|
||||
expect(code?.textContent).not.toContain('```');
|
||||
expect(container.querySelectorAll('pre')).toHaveLength(1);
|
||||
// 围栏之后的段落回到正文，而不是继续当代码渲染。
|
||||
expect(container.querySelector('pre')?.textContent).not.toContain(
|
||||
'后面还是正文',
|
||||
);
|
||||
expect(container.textContent).toContain('后面还是正文');
|
||||
});
|
||||
|
||||
it('不拆开代码里出现的 ``` 与行内代码', () => {
|
||||
const { container } = render(
|
||||
<ChatMarkdownMessage
|
||||
role="assistant"
|
||||
text={'```js\nconst s = "```";\n```\n\n行内 `code` 保持原样\n'}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(container.querySelector('pre code')?.textContent).toBe(
|
||||
'const s = "```";\n',
|
||||
);
|
||||
expect(container.querySelectorAll('pre')).toHaveLength(1);
|
||||
expect(container.textContent).toContain('行内 code 保持原样');
|
||||
});
|
||||
|
||||
it('行内代码里的三段反引号保持原样，不凭空造出围栏', () => {
|
||||
const { container } = render(
|
||||
<ChatMarkdownMessage role="assistant" text={'文本 ```x```\n'} />,
|
||||
);
|
||||
|
||||
expect(container.querySelector('p code')?.textContent).toBe('x');
|
||||
expect(container.querySelectorAll('pre')).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('引用块与列表项开头的围栏是合法结构，不做拆分', () => {
|
||||
const { container } = render(
|
||||
<ChatMarkdownMessage
|
||||
role="assistant"
|
||||
text={
|
||||
'> ```js\n> const quoted = 1;\n> ```\n\n- ```js\n const listed = 2;\n ```\n'
|
||||
}
|
||||
/>,
|
||||
);
|
||||
|
||||
// 拆开这两行只会把围栏从引用块 / 列表项里挪出来。
|
||||
expect(
|
||||
container.querySelector('blockquote pre code')?.textContent,
|
||||
).toContain('const quoted = 1;');
|
||||
expect(container.querySelector('li pre code')?.textContent).toContain(
|
||||
'const listed = 2;',
|
||||
);
|
||||
});
|
||||
|
||||
it('保留行内代码中的 HTML 字面量', () => {
|
||||
|
||||
@@ -549,6 +549,12 @@ export function registerChatComposerControlTests() {
|
||||
});
|
||||
expect(within(surface).queryByText(/本轮结束于/)).toBeNull();
|
||||
expect(within(surface).queryByTestId('turn-usage')).toBeNull();
|
||||
// 卡片从「本地命令在飞」起就得出现：只认原生 turn.started 的话，模型首 token 之前
|
||||
// 那段（实测约十秒）界面完全不说"正在处理"。
|
||||
expect(
|
||||
within(surface).getAllByText('陶泥儿正在处理').length,
|
||||
).toBeGreaterThan(0);
|
||||
expect(within(surface).getByText(/^已耗时 /u)).not.toBeNull();
|
||||
|
||||
await act(async () => {
|
||||
pending[0]?.resolve('回复');
|
||||
|
||||
@@ -6373,8 +6373,11 @@ export function registerProjectWorkbenchFoundationTests() {
|
||||
expect(userMessageRule).toContain('color: var(--platform-text-base);');
|
||||
// 过程卡渲染在消息列表的**兄弟**位置（列表外，紧贴输入盒上方），拿不到列表的
|
||||
// `padding: 14px 16px 0`，左右内缩与上下间距只能自己给：左右必须与列表的 16px 对齐。
|
||||
// 同一列里它是定高条目：窗口变矮时只有消息列表可以被压缩，卡片带 `overflow: hidden`
|
||||
// 时自动最小尺寸归零，少了 `flex: 0 0 auto` 就会被挤扁、文字被裁掉。
|
||||
expect(processCardRules.length).toBe(1);
|
||||
expect(processCardRules[0]).toContain('margin: 12px 16px 8px;');
|
||||
expect(processCardRules[0]).toContain('flex: 0 0 auto;');
|
||||
});
|
||||
|
||||
it('enables the run presentation and renders registered images in the resource viewer', async () => {
|
||||
|
||||
@@ -27,7 +27,7 @@ test('运行中状态条的读秒按 100ms 刷新：不足一分钟的耗时以
|
||||
turns={[]}
|
||||
messagesRef={createRef<HTMLDivElement>()}
|
||||
historyHasMore={false}
|
||||
nativeRunning
|
||||
turnInFlight
|
||||
activeTurnStartedAt={STARTED_AT}
|
||||
onLoadEarlierHistory={() => undefined}
|
||||
onScroll={() => undefined}
|
||||
@@ -61,7 +61,7 @@ test('没有运行中的回合时不订阅时钟', () => {
|
||||
turns={[]}
|
||||
messagesRef={createRef<HTMLDivElement>()}
|
||||
historyHasMore={false}
|
||||
nativeRunning={false}
|
||||
turnInFlight={false}
|
||||
activeTurnStartedAt={0}
|
||||
onLoadEarlierHistory={() => undefined}
|
||||
onScroll={() => undefined}
|
||||
|
||||
@@ -21,6 +21,7 @@
|
||||
- [当前产品与工程约束](./【项目基线】当前产品与工程约束-2026-05-15.md)：现役入口、账号钱包、UI 和后端分层。
|
||||
- [平台入口与玩法链路](./【玩法创作】平台入口与玩法链路-2026-05-15.md)：只描述现役平台壳与图片画布编辑器链路。
|
||||
- [外部 OpenAPI 与 API Key 接入方案](./【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md)
|
||||
- [外部 MCP 语义工具说明与参数设计](./technical/【技术方案】外部MCP语义工具说明与参数设计-2026-09-23.md)：15 个新增语义工具与全部原工具并存，复用现有 External API；包含工具说明、action、参数、幂等和兼容合同。
|
||||
- [External v1 OpenAPI](./openapi/genarrative-external-v1.openapi.json)：公开 HTTP 契约唯一机器可读来源。
|
||||
|
||||
## AI 游戏创作与 Agent Runtime
|
||||
|
||||
@@ -1,5 +1,15 @@
|
||||
# 决策记录
|
||||
|
||||
## 2026-09-24 DirectProject 状态条口径翻转、几何约束与对话 Markdown 容错
|
||||
|
||||
- 背景：AGC DirectProject 对话区底部的「陶泥儿正在处理 / 已耗时 12.4秒」状态条同时退化三处：① 读秒 1 秒一跳（耗时文案不足一分钟显示一位小数，小数位却一秒才动一格）；② 窗口压矮时被挤扁（300px 高压到 33px、240px 时 24px，文字被 `overflow: hidden` 裁掉）；③ `turn.started` 之前（模型首 token 前，实测约十秒）整条卡片不出现，界面没有任何「正在处理」的交代。同批还修了对话 Markdown 的两处代码块问题（不换行把消息拉宽、粘在正文行里的围栏导致代码块解析错位）。
|
||||
- 决策（卡片口径翻转，**更正** 2026-09-22「卡片口径取保守」）：卡片与已耗时起点改读 `displayBusy`（本地命令在飞 ∪ 原生已确认在跑）与「最新一个**未结束**回合的用户发送时间」。理由：`turn.started` 要等宿主应答返回才发出，只认原生真相会让首 token 之前那段没有交代；窗口期这一轮确实已经交给宿主（本地命令在飞），文案不虚报「宿主已在跑」之外的东西。
|
||||
- 决策（那条预言的处置）：2026-09-22 那条写「若将来改成窗口期也显示卡片，`running` 在渲染层就没有消费者了，应把投影压成 `unfinished: boolean`」。本次改完后投影三态**仍有**消费者——`DirectProjectTurn` 用 `state !== 'finished'` 做否定式判断、`state === 'running'` 挑流式正文，状态条也用 `state !== 'finished'` 定起点——所以不动 `DirectChatTurnState`，也不压缩成布尔。
|
||||
- 决策（状态条几何）：卡片在 `.project-chat-conversation` 这条定高 flex 列里必须 `flex: 0 0 auto`。它带 `overflow: hidden`，按 flex 规范该项的自动最小尺寸归零，是这条链上唯一还能被压缩的项；压缩只能由消息列表吸收。同一选择器只保留一条规则（几何 + 不可压缩），不留两份。
|
||||
- 决策（对话 Markdown 对模型输出的容错）：解析前先 `normalizeMarkdownFences` 再压缩空行；代码块 `pre` 与块内 `code` 各自都给 `whitespace-pre-wrap` + `break-words`。细则与判据见 `pitfalls.md` 同日两条。
|
||||
- 影响面：`apps/ai-game-creator-shell/src/{styles.css,components/ChatMarkdownMessage/index.tsx,view/project-development/chat/{DirectProjectChatView.tsx,components/DirectProjectConversation/DirectProjectConversation.tsx,controller/useDirectProjectTurnStatus.ts}}`；用例 `tests/{ChatMarkdownMessage.test.tsx,directProjectProcessStatus.test.tsx,appSurface/{chat-composer.suite.ts,project-development.suite.ts}}`。
|
||||
- 验证：`npx vitest run apps/ai-game-creator-shell/tests` 186 passed / 1 skipped（1874 条里 1860 passed / 14 skipped）；真实 Chromium 夹具复核读秒 100ms、窗口 900→220px 高度下卡片恒为 36px 不被裁、代码块换行与两类粘住围栏；变异验证（读秒改回 1000ms、删 `flex: 0 0 auto`、卡片退回 `nativeRunning`、停掉围栏归一化、换行类名退回）逐条变红。
|
||||
|
||||
## 策划 V1/V2 退役的现行边界
|
||||
|
||||
- 旧策划 V1 和 Runtime V2 均已删除，当前策划入口统一使用独立 Design Agent。V1 被 V2 接替只描述历史过程，不表示 V2 仍在使用。
|
||||
|
||||
@@ -21,6 +21,11 @@
|
||||
3. `server-rs/crates/api-server/src/app.rs`、`server-rs/crates/api-server/src/modules.rs` 与对应 crate README / 源码
|
||||
4. `docs/openapi/genarrative-external-v1.openapi.json`（涉及 External v1 时）
|
||||
|
||||
外部 MCP 语义工具设计：
|
||||
|
||||
1. [外部 OpenAPI 与 API Key 接入方案](../../【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md)：现役托管 MCP 与 External API 合同。
|
||||
2. [外部 MCP 语义工具说明与参数设计](../../technical/【技术方案】外部MCP语义工具说明与参数设计-2026-09-23.md)：15 个语义工具与全部旧工具并存，复用现有 API 分派和 schema；多功能入口使用 action/input，结果不裁剪，可选幂等只扩展新入口。语义工具按 operation 显式声明 destructive 风险，并汇总各 action；上传确认包含已有元数据更新风险。instructions 与 resources 已同步当前工具选择、调用流程及结果说明；资源 URI 保留，Skill 文档源与下载包共用，线上状态按实际部署核对。
|
||||
|
||||
AI 游戏创作 / DirectProject / UI workflow：
|
||||
|
||||
1. `docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`
|
||||
|
||||
@@ -2,6 +2,14 @@
|
||||
|
||||
> 策划历史条目边界：旧策划 V1/V2 已全部退役，当前入口仅使用 Design Agent。下文带日期的旧 Planning V2、Fast GDD、`plan.submit_gdd`、旧 IPC/模块记录仅用于追溯，不能作为恢复旧代码、身份门禁或专属测试的依据；共享问题需在现役调用上核查。现行合同见[策划 Agent 生产迁移与工作区浏览](../../technical/【技术方案】策划Agent生产迁移与工作区浏览-2026-09-10.md)。
|
||||
|
||||
## 2026-09-24 模型输出的围栏会粘在正文行里：聊天 Markdown 必须先归一化再解析
|
||||
|
||||
- **现象**：AGC 对话里代码块解析错位——引言行被当成代码渲染（`…实现细节（game.js）：```js`），或者代码块收不住、把后面的正文一起吞进去（`… return centerOn(projection); }````）。文本本身「看起来没问题」，容易被当成渲染器坏了。
|
||||
- **原因**：CommonMark 只认**整行**的围栏（最多 3 个空格缩进）。模型经常把 ``` 直接粘在上一行末尾，那个 ``` 退化成行内文本：开场围栏不成立（后面的正文被当成代码）、收场围栏不生效（代码块不闭合，吞掉剩余内容）。`ChatMarkdownMessage` 原先只做空行压缩，没有这一步归一化。
|
||||
- **处理（现行口径）**：`normalizeMarkdownFences` 在解析前把「围栏前是非空白字符、围栏到行尾只剩语言标识（可空）」的行拆成两行。判据刻意收窄：整行 / 缩进围栏、行内代码（单个反引号）、代码里出现的 ``` （`const s = "```";`）、引用块 / 列表项开头的合法围栏（`> ```js`、`- ```js`）以及同一行里出现第二段围栏串的行内代码（``文本 ```x``` ``）都不动——后三者拆开只会把围栏从引用块 / 列表项里挪出来，或者凭空造出一个开场围栏。归一化对 `preserveBlankLines`（文件预览）同样生效。
|
||||
- **同时**：块级 `pre` 与块内 `code` 都要给 `whitespace-pre-wrap` + `break-words`（两处各写过 `white-space`，只改一处不换行），否则窄面板里长行会把消息拉宽、顶出横向滚动条。
|
||||
- **验证**：`npx vitest run apps/ai-game-creator-shell/tests/ChatMarkdownMessage.test.tsx`（去掉归一化或换行类名即红）；真实 Chromium 夹具里长代码行 `horizontalOverflow: false`、粘住的开场 / 收场围栏都渲染成正确结构。
|
||||
|
||||
## 2026-09-24 对话过程卡的读秒退回 1 秒一跳：刷新粒度必须与显示精度同格
|
||||
|
||||
- **现象**：AGC DirectProject 对话区底部那条「陶泥儿正在处理 / 已耗时 12.4秒」的状态条，小数位一秒才动一格，看着像读数卡住；同一屏里工具卡片的耗时与资源生成侧栏的读秒都在正常走 0.1 秒，只有这一处不动。
|
||||
|
||||
@@ -0,0 +1,387 @@
|
||||
# 外部 MCP 语义工具说明与参数设计
|
||||
|
||||
更新时间：`2026-09-24`
|
||||
|
||||
> 文档状态：`current`（工程实现合同；仓库已增加语义工具，线上可用性以实际部署版本为准）。
|
||||
>
|
||||
> 父规范：[外部 OpenAPI 与 API Key 接入方案](../【后端架构】外部OpenAPI与APIKey接入方案-2026-06-19.md)。
|
||||
>
|
||||
> 当前字段契约：[External v1 OpenAPI](../openapi/genarrative-external-v1.openapi.json)。本文记录工具划分、说明和参数合同；英文工具名、schema 构造、结果与兼容方式见第 5 节。
|
||||
|
||||
## 1. 目标与范围
|
||||
|
||||
在现有 API server 的托管 MCP 层增加按用户任务组织的工具。仅使用现有 External v1 已开放的能力，不直接接入尚未公开的画布功能。
|
||||
|
||||
工具可以存在自然的功能交集：例如参考图快速变体同时属于生成图片和修改图片，读取画布同时服务于项目查找和布局编辑。不以工具数量或 API 唯一归属为目标，不把不相关任务硬合成一个工具。
|
||||
|
||||
本文约定工具说明、操作选择、参数和业务边界，以及与当前工具配套的 instructions 和 resources。开发者发布页、文档存储与独立发布、CLI 改造不属于本轮范围。现有 REST 路由、DTO 和运行行为保持不变，新语义工具与全部原有工具并存。
|
||||
|
||||
## 2. 当前实现与目标调用链
|
||||
|
||||
当前 [external_mcp.rs](../../server-rs/crates/api-server/src/external_mcp.rs) 从 OpenAPI 自动建立原子工具：工具名由 `operationId` 转为 snake_case；description 优先读取 operation 的 `description`，缺省读取 `summary`，附加 HTTP 方法和路径；参数 schema 来自 OpenAPI。服务级 instructions 提供通用工作流，resources 提供详细说明。
|
||||
|
||||
新工具继续部署在同一个 API server 内，复用现有分派和 External REST router：
|
||||
|
||||
```text
|
||||
Agent 调用语义工具
|
||||
→ MCP 层选择对应 operation，并转换参数位置
|
||||
→ 进程内调用现有 External REST router
|
||||
→ 复用鉴权、scope、owner、校验、幂等、计费和业务处理
|
||||
→ MCP 返回业务结果或结构化错误
|
||||
```
|
||||
|
||||
多功能工具的一次调用只选择一个操作。工具聚合不意味着批量执行、自动连续写入或新增跨操作事务。文件上传仍由调用方完成二进制传输。
|
||||
|
||||
## 3. 说明与参数的共同设计
|
||||
|
||||
### 3.1 工具说明
|
||||
|
||||
每份说明包含：
|
||||
|
||||
1. 用途：用户希望完成什么任务时选择本工具。
|
||||
2. 操作选择：多功能工具列出各 action 的用途，解释容易混淆的选择。
|
||||
3. 结果：立即返回业务结果，还是返回异步任务 ID；有哪些重要的降级或结果边界。
|
||||
|
||||
字段格式、枚举、必填关系放入参数 schema 和字段说明，避免把全部接口校验细节堆进工具 description。付费、删除等重要影响应明确表达，但 description 和风险注解不替代后端权限校验；宿主负责结合用户已给出的授权作调用决策。
|
||||
|
||||
### 3.2 多功能与单功能工具
|
||||
|
||||
多功能工具采用 `action + input`，input 必须提供对应 action 的明确字段、类型、必填条件和枚举，不能只是任意 JSON 对象。服务端在分派前校验所选操作，避免不相干分支的参数混入底层请求。
|
||||
|
||||
示意调用（示例 ID 仅为说明）：
|
||||
|
||||
```json
|
||||
{
|
||||
"action": "edit",
|
||||
"input": {
|
||||
"sourceReferenceId": "<resourceId-or-assetId>",
|
||||
"prompt": "把衣服改成红色",
|
||||
"projectId": "<projectId>"
|
||||
},
|
||||
"idempotencyKey": "<stable-logical-request-key>"
|
||||
}
|
||||
```
|
||||
|
||||
单功能工具直接声明业务字段，不强行添加 action 或 input 包装。`kind`、`sliceMode`、视频 `mode` 等原有业务选择保留原名，它们与工具级 action 职责不同。
|
||||
|
||||
工具参数中的 ID 由 MCP 层放入 REST 路径或查询，业务输入放入请求体，`idempotencyKey` 转为 `Idempotency-Key` header。具体参数保留现有命名、类型和条件；下表列的是关键字段，并非完整 schema 或排他白名单。未逐项列出的可选字段按对应 OpenAPI operation 核对，不把站内 DTO 的内部字段顺带开放。
|
||||
|
||||
### 3.3 共用行为
|
||||
|
||||
- 九类生成 POST 继续要求稳定幂等键，并返回 `202 + operationId`；工具不能把受理当成生成完成。
|
||||
- 项目创建、项目资源登记、文件夹创建的 REST API 支持可选幂等键，新语义入口提供并转发这些可选 header；原工具继续保持仅按 required header 推导幂等参数的行为。素材记录创建 API 不在该可选幂等合同中。
|
||||
- 相同 API 能力出现在不同语义入口时，参数约束、底层请求和结果语义保持一致。重试或切换入口恢复同一逻辑请求时，保留原 API 操作、规范请求和幂等键，不因换工具名创建新任务。
|
||||
- `projectId`、`assetFolderId`、`assetLabel`、`canvasCompletion` 等目标参数只在对应 API 支持时提供。UI 素材提取使用 `spritesheetLabel`；各生成族不共享未经核对的字段全集。
|
||||
- owner 由 API Key 确定，不让 Agent 指定身份，不把 API Key 放进工具参数。
|
||||
- 业务结果继续使用结构化返回；业务失败沿用安全错误语义，不把失败包装成成功。生成成功、素材登记、画布落位、切片成功分别按实际结果判断。
|
||||
- 读取、写入、付费与删除应在说明和注解中准确表达。注解作用于整个工具，因此将删除独立成工具；`openWorldHint` 不等同于付费标记。
|
||||
|
||||
## 4. 工具说明与操作映射
|
||||
|
||||
以下共 15 个新增工具。删除从项目管理、素材库整理中移出，统一放入 XV。本节说明仓库实现合同，不表示线上已部署。
|
||||
|
||||
### I. 查找画布项目
|
||||
|
||||
**说明：** 查找已有画布项目，或读取指定项目的完整内容。先通过列表或最近项目确定目标，再读取详情获取画布、图层和资源。
|
||||
|
||||
| action | 关键参数 | 现有 operationId |
|
||||
| --- | --- | --- |
|
||||
| `list` | 无 | `listEditorProjects`，固定 `view=summary` |
|
||||
| `recent` | 无 | `loadRecentEditorProject` |
|
||||
| `get` | 必填 `projectId` | `getEditorProject` |
|
||||
|
||||
列表使用既有摘要视图，不能退回全量项目快照。当前接口没有分页、limit 或名称搜索参数，工具不虚构这些字段。名称匹配基于读取结果；有歧义时向用户明确候选，不静默创建替代项目。
|
||||
|
||||
### II. 管理画布项目
|
||||
|
||||
**说明：** 创建新的画布项目，或修改已有项目的名称。
|
||||
|
||||
| action | 关键参数 | 现有 operationId |
|
||||
| --- | --- | --- |
|
||||
| `create` | 可选 `title`；可选幂等键 | `createEditorProject` |
|
||||
| `rename` | 必填 `projectId`、`title` | `renameEditorProject` |
|
||||
|
||||
一次创建仅创建项目，不默认追加创建同名素材文件夹。删除使用 XV。
|
||||
|
||||
### III. 查找与读取素材
|
||||
|
||||
**说明：** 查看账号素材库或指定项目中的资源，并为已有素材获取临时下载地址。
|
||||
|
||||
| action | 关键参数 | 现有 operationId |
|
||||
| --- | --- | --- |
|
||||
| `list_library` | 无 | `getEditorAssetLibrary` |
|
||||
| `get_project_resources` | 必填 `projectId` | `getEditorProject` |
|
||||
| `get_download_url` | `objectKey` 或兼容的 `legacyPublicPath`；可选 `expireSeconds` | `getExternalAssetReadUrl` |
|
||||
|
||||
项目资源查询复用项目详情，不新增资源查询 API。本次返回完整项目详情，资源位于其中的 resources；保持与项目读取相同的结构，不新增裁剪投影。
|
||||
|
||||
读取素材记录、获取文件地址与查看媒体内容是不同操作。工具不提供本地下载、关键词检索或相似素材搜索。临时签名 URL 用于访问媒体，不作为持久化生成引用。语义入口保留 REST 的 `legacyPublicPath` 可选查询字段；优先使用稳定 objectKey，至少提供一种来源，由现有 API 校验来源，不改 REST 契约。
|
||||
|
||||
### IV. 办理素材上传
|
||||
|
||||
**说明：** 获取文件直传凭证，并在调用方完成上传后确认素材对象。文件传输由调用方执行。
|
||||
|
||||
| action | 必填参数 | 常用可选参数 | 现有 operationId |
|
||||
| --- | --- | --- | --- |
|
||||
| `create_upload_ticket` | `legacyPrefix`、`fileName` | `contentType`、`pathSegments`、`maxSizeBytes` 等 | `createExternalDirectUploadTicket` |
|
||||
| `confirm_upload` | `objectKey`、`assetKind` | `bucket`、`contentType`、`contentLength`、`contentHash` 等 | `confirmExternalAssetObject` |
|
||||
|
||||
流程为“申请凭证 → 调用方按返回的 OSS 表单直传参数上传 → 确认对象”。不把上传协议笼统写成固定 PUT。工具不接受本地路径或 base64，不代上传。对象确认后若需要项目资源或素材库记录，再调用相应登记操作；确认对象不等于完成这些登记。
|
||||
|
||||
`ownerUserId` 由后端账号身份决定，不作为 Agent 可选参数。
|
||||
|
||||
### V. 生成图片
|
||||
|
||||
**说明：** 根据文字和可选参考图生成图片，支持普通图片、视觉规范、角色、UI 设计、宣发素材和快速参考变体。提交后通过任务 ID 查询结果。
|
||||
|
||||
映射 `generateExternalEditorImage`，无需 action。
|
||||
|
||||
| 参数 | 约束与用途 |
|
||||
| --- | --- |
|
||||
| `prompt` | 必填，生成内容描述 |
|
||||
| `kind` | 可选；省略表示普通图，其他用途为 `spec`、`character`、`quick-edit`、`ui-design`、`publication-material` |
|
||||
| `referenceImageSrcs` | 可选参考图；具体引用和组合约束沿用 API |
|
||||
| `model`、`aspectRatio`、`imageSize`、`size` | 可选输出配置，枚举及组合以当前 schema 为准 |
|
||||
| `style` | 可选风格；适用范围沿用对应 kind 的现有处理 |
|
||||
| `screenColor` | 可选纯色抠像背景色，见下文 |
|
||||
| 目标与落位字段 | 按当前接口支持提供项目、素材库和画布目标 |
|
||||
|
||||
**背景色已经通过 API 开放。** screenColor 可传画布支持的颜色 hex，例如 `#CFEFFF`；传 `"auto"`、null 或省略，由服务端自动选择。它描述生成及后续抠图流程的纯色背景，不保证最终产物保留该底色；不能据此承诺“最终图片指定底色”或通用背景替换能力。
|
||||
|
||||
不额外引入与 kind 重复的 action；不添加未开放的 `scene` 生成入口。
|
||||
|
||||
### VI. 修改图片
|
||||
|
||||
**说明：** 对已有图片做定向修改、生成参考变体或移除背景。定向修改使用已登记资源；快速变体通过参考图生成新版本。提交后通过任务 ID 查询结果。
|
||||
|
||||
| action | 关键参数 | 现有 operationId |
|
||||
| --- | --- | --- |
|
||||
| `edit` | 必填 `sourceReferenceId`、`prompt`；其余为编辑 API 的可选字段 | `editExternalEditorImage` |
|
||||
| `variation` | 必填 `prompt`；参考输入使用 `referenceImageSrcs`；其他字段复用图片生成 | `generateExternalEditorImage`，适配层固定 `kind=quick-edit` |
|
||||
| `remove_background` | 必填 `sourceImageSrc`；可选 `backgroundMode`、条件允许的 `screenColor` 及目标字段 | `removeExternalEditorImageBackground` |
|
||||
|
||||
来源参数保留真实差异：
|
||||
|
||||
- edit 的 sourceReferenceId 只接受当前账号已登记的项目资源 ID 或素材 ID，不能用 objectKey 或 URL 代替。
|
||||
- 去背景的 sourceImageSrc 支持当前账号拥有的稳定 objectKey、项目资源 ID 或素材 ID，只处理允许的静态图片。
|
||||
- 变体的参考来源沿用生成 API；不新造含糊的统一 source 字段。其参考输入必填性、有效引用等条件沿用 quick-edit 的现有契约，不从图片生成 schema 只有 prompt 必填就推断所有用途无需前置条件。
|
||||
|
||||
去背景 backgroundMode 省略或 null 时默认为 `complex`。complex 做语义分割；`flat` 用于纯色背景。screenColor 仅在 flat 下可传非 null 值，支持 `auto`、`#RRGGBB`，省略或 null 自动检测；complex 下传非 null screenColor 会被拒绝。不能直接照搬图片生成的全部背景色组合规则。
|
||||
|
||||
需要原位替换时，编辑和去背景沿用各自 API 的 `projectId + targetLayerId` 来源绑定要求；不能用目标图层 ID 绕过账号归属与对象一致性校验。变体入口不额外承诺原位替换。
|
||||
|
||||
variation 与 V 的 kind=quick-edit 是合理交集，复用同一输入定义、底层请求和结果语义，不让调用方重复提供固定 kind。
|
||||
|
||||
### VII. 生成图标图集
|
||||
|
||||
**说明：** 根据已登记的参考规范和图标清单生成图集，并按指定方式尝试拆分独立图标。
|
||||
|
||||
映射 `generateExternalEditorIconSpritesheet`，无需 action。
|
||||
|
||||
- 必填：`referenceId`、`iconDescriptions`、`sliceMode`。
|
||||
- referenceId 为当前账号已登记为 `icon-spec` 的项目资源或素材 ID，不接受 objectKey 或 URL 代替。
|
||||
- `sliceMode=grid`：提供需求对应的 `gridX`、`gridY`，各为 1–32，不传 sliceCount。
|
||||
- `sliceMode=connected-components`：可用 `sliceCount`（1–256）约束张数，不提供 gridX/gridY。
|
||||
- 可选：`referenceImageSrcs`、`screenColor`、`style`、模型、比例、尺寸及目标字段。
|
||||
|
||||
sliceMode 没有默认值。主规范引用使用 referenceId，不用辅助 referenceImageSrcs 替代。此工具包含生成步骤，不是任意已有图片的通用裁切工具。切片未完成时可能仍有完整图集，结果必须保留 sliceWarning。
|
||||
|
||||
### VIII. 提取 UI 素材
|
||||
|
||||
**说明：** 以已有 UI 设计图为参考，生成组件素材图集并尝试拆分，供后续界面制作使用。
|
||||
|
||||
映射 `extractExternalEditorUiDesignAssets`，无需 action。
|
||||
|
||||
- 必填：`sourceImageSrc`、`aspectRatio`、`imageSize`。
|
||||
- 可选：`model`、`referenceImageSrcs`、`screenColor`、`spritesheetLabel` 和对应目标字段。
|
||||
- 结果命名沿用 spritesheetLabel，不强行改成其他生成接口的 assetLabel。
|
||||
|
||||
接口包含生成过程，不保证把原图中的组件逐像素原样裁出。与 VII 的区别是：VII 以规范和明确图标清单生成，VIII 以已有 UI 设计图提取组件语义。两者都须分别判断图集与切片结果。
|
||||
|
||||
### IX. 生成角色动画
|
||||
|
||||
**说明：** 根据角色源图和动作描述，生成角色动画预览与帧序列。
|
||||
|
||||
映射 `generateExternalEditorCharacterAnimation`，无需 action。
|
||||
|
||||
- 来源必填：`sourceLayerId`、`sourceImageSrc`、`sourceWidth`、`sourceHeight`。
|
||||
- 动作必填：`promptText`。
|
||||
- 输出必填：`resolution`、`ratio`、`frameCount`、`durationSeconds`、`model`。
|
||||
- 可选：`screenColor`、`sourceResourceId`、目标与落位字段等。
|
||||
|
||||
来源与尺寸先从真实资源中取得，不为简化调用伪造图层 ID 或源图尺寸。此工具不提供已有动画文件编辑。正式动画结果按现有帧序列合同消费，不重复用第一帧手工登记一份动画。
|
||||
|
||||
### X. 生成视频
|
||||
|
||||
**说明：** 根据文字和模型支持的参考图片、视频或音频生成视频片段。
|
||||
|
||||
映射 `generateExternalEditorVideo`，无需 action。
|
||||
|
||||
- 必填：`prompt`、`model`、`aspectRatio`、`durationSeconds`、`resolution`、`mode`、`sound`。
|
||||
- 可选参考：`referenceImageSrcs`、`referenceVideoSrcs`、`referenceAudioSrcs`。
|
||||
- 其他可选项：当前 API 支持的 `webSearchEnabled`、目标与落位字段等。
|
||||
|
||||
保留 mode 作为视频业务模式，与工具级 action 无关。参考媒体、时长、分辨率、声音和其他选项的组合以所选模型的当前契约为准，不暗示每个模型支持所有组合。
|
||||
|
||||
### XI. 生成音频
|
||||
|
||||
**说明：** 生成音效或背景音乐。短声音、环境声和交互反馈选择音效；配乐选择背景音乐。
|
||||
|
||||
| action | 必填参数 | 常用可选参数 | 现有 operationId |
|
||||
| --- | --- | --- | --- |
|
||||
| `sound_effect` | `prompt` | `duration`、`loop`、`model`、目标字段 | `generateExternalEditorSoundEffect` |
|
||||
| `background_music` | `gptDescriptionPrompt`、`makeInstrumental` | 目标与落位字段 | `generateExternalEditorBackgroundMusic` |
|
||||
|
||||
当前音效 duration 是可选字段，省略或 null 表示自动，手动时长范围为 0.5–30 秒。不沿用“必须填写时长”的过期说法。两个分支保留各自提示词字段，不另外实现一套提示词翻译协议。两者均异步受理。
|
||||
|
||||
### XII. 编辑画布
|
||||
|
||||
**说明：** 读取画布、保存完整布局，或登记供图层引用的项目资源。保存布局需要基于最新画布版本。
|
||||
|
||||
| action | 关键参数 | 现有 operationId |
|
||||
| --- | --- | --- |
|
||||
| `get` | 必填 `projectId` | `getEditorProject` |
|
||||
| `save_layout` | 必填 `projectId`、`viewport`、`layers`、`expectedRevision` | `saveEditorProjectCanvas` |
|
||||
| `register_resource` | 必填 `projectId`、`imageSrc`、`width`、`height`、`sourceType`；支持可选幂等键 | `createEditorProjectResource` |
|
||||
|
||||
save_layout 保存完整默认画布布局，不提供只传一个图层的局部 patch 语义；expectedRevision 来自最新读取，版本冲突按现有 API 处理。
|
||||
|
||||
register_resource 只登记资源，并不自动创建画布图层。正常生成结果落画布优先使用生成接口的 canvasCompletion。可选 objectKey、assetObjectId、assetKind、帧序列等字段按资源登记 schema 提供，不从临时 UI 状态推导正式来源。
|
||||
|
||||
### XIII. 整理素材库
|
||||
|
||||
**说明：** 管理素材文件夹和素材记录，包括新建、改名、移动以及登记已有媒体。
|
||||
|
||||
| action | 关键参数 | 现有 operationId |
|
||||
| --- | --- | --- |
|
||||
| `create_folder` | 必填 `label`；可选 `sortOrder`、幂等键 | `createEditorAssetFolder` |
|
||||
| `update_folder` | 必填 `folderId`；可选 `label`、`collapsed` | `updateEditorAssetFolder` |
|
||||
| `create_asset` | 必填 `folderId`、`label`、`imageSrc`、`width`、`height`、`sourceType` | `createEditorAsset` |
|
||||
| `update_asset` | 必填 `assetId`；可选 `label`、`folderId` | `updateEditorAsset` |
|
||||
|
||||
移动素材复用 update_asset 的 folderId，不额外拆一个重复操作。可选更新字段的有效性按现有 API 处理。create_asset 只登记元数据，不上传文件、不触发生成；生成接口已经完成入库时不重复登记。删除使用 XV。
|
||||
|
||||
### XIV. 查看生成进度与结果
|
||||
|
||||
**说明：** 查询一次已提交生成任务的当前状态。完成时返回结果，失败时返回错误；尚未完成时按建议间隔再次查询。
|
||||
|
||||
映射 `getExternalEditorGenerationJob`，无需 action；必填 `operationId`。
|
||||
|
||||
- 每次调用只查询一次，不在 MCP 服务端长时间阻塞到任务完成。
|
||||
- `queued/running`：读取进度和 pollAfterMs，继续等待。
|
||||
- `completed`：消费 result，并保留 warning 与 sliceWarning 等真实降级信息。
|
||||
- `failed`：返回已有脱敏错误。
|
||||
- 跨账号任务与不存在任务沿用同一不可见语义。
|
||||
|
||||
收到 operationId 后优先直接查询。若提交响应丢失、尚未取得 operationId，不能靠查询工具凭空恢复：按现有幂等合同，复用原 API 操作、原请求和原幂等键重试以取得受理结果；不能换键重提。查询超时不改变服务端任务状态。
|
||||
|
||||
### XV. 删除资源
|
||||
|
||||
**说明：** 删除指定项目、素材文件夹或素材记录。按精确 ID 操作，调用前明确目标及删除范围，并取得对应用户授权。
|
||||
|
||||
| action | 必填参数 | 现有 operationId |
|
||||
| --- | --- | --- |
|
||||
| `delete_project` | `projectId` | `deleteEditorProject` |
|
||||
| `delete_folder` | `folderId` | `deleteEditorAssetFolder` |
|
||||
| `delete_asset` | `assetId` | `deleteEditorAsset` |
|
||||
|
||||
工具统一标记删除风险。项目删除沿用默认画布与项目资源元数据级联清理语义。文件夹删除将其素材记录移到默认文件夹，默认文件夹不能删除；素材删除删除记录并处理关联精选审核状态。文件夹和素材记录删除不等于删除底层 OSS 媒体。
|
||||
|
||||
不增加批量、按名称或模糊匹配删除，不用一个可伪造的 confirm 参数替代真实用户授权和后端鉴权。
|
||||
|
||||
## 5. 实现合同与验收
|
||||
|
||||
### 5.1 工具名与兼容
|
||||
|
||||
| 编号 | 工具名 |
|
||||
| --- | --- |
|
||||
| I | `find_canvas_projects` |
|
||||
| II | `manage_canvas_projects` |
|
||||
| III | `find_assets` |
|
||||
| IV | `prepare_asset_upload` |
|
||||
| V | `generate_image` |
|
||||
| VI | `modify_image` |
|
||||
| VII | `generate_icon_spritesheet` |
|
||||
| VIII | `extract_ui_assets` |
|
||||
| IX | `generate_character_animation` |
|
||||
| X | `generate_video` |
|
||||
| XI | `generate_audio` |
|
||||
| XII | `edit_canvas` |
|
||||
| XIII | `organize_asset_library` |
|
||||
| XIV | `check_generation` |
|
||||
| XV | `delete_resources` |
|
||||
|
||||
现有 MCP 工具和 REST API 全部保留；旧工具名称、schema、注解、调用行为和可见性不变。新工具直接追加进同一个 tools/list，不加开关、不改旧工具前缀、不设隐藏目录。本轮包括工具说明和参数、instructions、resources 及其共用的下载 Skill 文档；独立配套 Skill、CLI、发布页与部署不属于本轮。
|
||||
|
||||
### 5.2 参数与结果
|
||||
|
||||
- 字段 schema 从对应现有 OpenAPI operation 构造，展开本地引用并保留嵌套类型、枚举、条件和字段说明。路径、查询与 body 字段平铺到单功能工具顶层或多功能工具的 input；不另写一份业务字段全集。
|
||||
- 多功能工具以顶层 object 加 oneOf 表达互斥 action，每个分支含 action 常量和完整 input schema。input 必填，无业务参数时传 `{}`。idempotencyKey 仅放在工具顶层：九类生成必填，项目创建、资源登记、文件夹创建可选，其它 action 不接受。
|
||||
- MCP 适配层在分派前校验 action、信封字段、所选分支允许的字段、必填参数及直接字段的类型/枚举/常量；嵌套字段值和业务组合继续由现有 REST DTO/校验处理，不创建第二套业务验证器。协议 schema 与分派校验共同限制错分支参数；未知顶层或 input 字段不能被静默丢弃。
|
||||
- variation 不暴露 kind，固定注入 `quick-edit`；对象确认不暴露或接受 ownerUserId。其余公开字段全部保留。稳定 key、原 API 与规范请求保持一致，重复语义入口不新增幂等命名空间。
|
||||
- 项目列表固定 summary。get_project_resources 返回完整项目响应，不裁剪；下载地址保留 objectKey、legacyPublicPath 与 expireSeconds。所有结果复用现有成功解包与结构化错误处理，不裁剪告警、revision 冲突或异步结果。
|
||||
- 读取工具标记 readOnly；含删除、覆盖或移动已有状态的工具标记 destructive；生成类说明付费及异步语义，可能替换已有图层的生成工具也按潜在破坏性标记。openWorld 仅用于会访问外部服务的能力，不作为付费标志。注解不替代用户授权或后端校验。
|
||||
- 语义工具的 destructive 按 API operation 显式声明，并取各 action 风险的并集，不根据 HTTP 方法或参数字段名推断。`prepare_asset_upload/confirm_upload` 可以更新同一 owner 的已有对象元数据，因此整个上传工具标记 destructive；申请上传票据本身仍属于新增操作。新增 operation 必须补充风险声明，原有工具的注解保持兼容。
|
||||
|
||||
### 5.3 验收
|
||||
|
||||
验证工具目录追加与旧定义一致、全部 action 路由与字段位置、必填和错分支拒绝、可选/必填幂等头、quick-edit 与普通生成入口等价、原 owner/scope 边界、结构化结果/告警和 revision 冲突透传。运行 api-server 定向测试与本地 healthz smoke；真实账号、付费 Provider 和具体 MCP 客户端尚未验证时明确记录，不能将单元测试当作线上验收。
|
||||
|
||||
### 5.4 实现与验收结果（截至 2026-09-24）
|
||||
|
||||
本轮工具、instructions 和 resources 更新完成。新增 15 个语义工具与原有 29 个工具同时可见，7 个资源 URI 保持不变。自动回归与本地业务主路径验证通过；真实生成测试发现的两项既有合同问题已独立记录在 [Issue #495](https://git.genarrative.world/git/GenarrativeAI/Genarrative/issues/495)，不在本轮修复，不能将主路径通过表述为全部合同验收通过。已完成的里程碑和实施计划收口到本节。
|
||||
|
||||
| 验证范围 | 证据与结果 |
|
||||
| --- | --- |
|
||||
| 工具与参数合同 | `external_` 回归中的 25 项 `external_mcp` 测试通过，覆盖 44 个工具、31 个 action、旧定义保留、schema 展开、参数映射、错分支拒绝、必填/可选幂等、等价入口及 15 个语义工具的风险注解 |
|
||||
| 认证与既有 API 回归 | 2026-09-24 在包含最新 master、instructions、resources 和风险注解修复的分支上运行 `cargo test --locked -p api-server external_`：156 项通过，含 MCP 内外认证、scope 拒绝、跨 owner 隔离、异步及 OpenAPI 回归 |
|
||||
| 编译与文本检查 | `cargo check --locked -p api-server`、定向 rustfmt、文档索引、编码和 diff 检查通过 |
|
||||
| 本地服务启动 | 先通过 `npm run dev:spacetime` 启动 SpacetimeDB 2.8.3 并发布隔离数据库，再通过 `npm run dev:api-server` 启动同一目标的 API 和 worker；`/v1/ping`、`/healthz`、`/readyz` 均返回 200 |
|
||||
| MCP 真实 HTTP 链路 | 未认证返回 401；使用隔离数据库中的临时测试 API Key，initialize 成功，tools/list 返回 44 个工具且包含全部 29 个旧工具，resources/list 返回原有 7 个资源 |
|
||||
| 本地数据库读写 | 新工具创建/列表/重命名/读取/删除成功；同键重复创建返回同一项目；旧工具读写与新工具互通；重复语义读取保留完整项目；ownerUserId 额外字段被拒绝且未产生写入 |
|
||||
|
||||
补充验证：
|
||||
|
||||
| 验证范围 | 证据与结果 |
|
||||
| --- | --- |
|
||||
| 全工具业务主路径 | 2026-09-23 通过本地 MCP HTTP 实测 44 个工具，全部至少一次主路径成功；15 个语义工具及其 action 均执行，含实际 OSS 上传/确认/下载、资源登记、素材整理和画布 revision 保存 |
|
||||
| 真实付费生成 | 11 个任务全部 completed，覆盖普通图、编辑、变体、抠图、图标图集、UI 提取、角色动画、视频、音效、背景音乐及旧抠图入口；验证下载、媒体解码和持久化，无 warning/sliceWarning。动画持久化为 32 帧、4000ms；图集与 UI 用例分别产生 2 张和 1 张切片。两项额外合同检查失败另见 Issue #495 |
|
||||
| instructions/resources 与下载包 | 更新后 24 项 MCP 测试、2 项 Skill 归档及 manifest 摘要测试通过；10 个 JSON 示例可解析，其中 6 个业务请求示例通过现有 OpenAPI schema 校验；Skill 格式、编码、文档索引和 diff 检查通过 |
|
||||
|
||||
运行核验使用独立测试身份和数据库，不经过真实用户登录或外部账号开通流程。初次 CRUD smoke 的测试项目和凭证已清理；后续付费全工具测试使用另一组独立凭证与产物，并在该轮结束时保留供复核，凭证不进入仓库。付费测试基于工具实现版本 `23b2a3325`；随后更新说明文档并完成合并后自动回归，未再次触发付费生成。服务当前是否运行需另行检查，本文不作为进程状态记录。
|
||||
|
||||
未验证：真实用户登录/发放 API Key 全流程、外部 MCP 客户端对 oneOf 参数的展示与使用、远端部署、多模型和全部参数组合、容量压测与完整视觉质量。UI 提取使用简单测试图片，不代表复杂 UI 多组件质量验收。合并后仍需部署 API server，并通过实际 MCP 客户端核验工具发现、调用和资源读取。
|
||||
|
||||
### 5.5 instructions 更新（2026-09-24）
|
||||
|
||||
初始化响应的 instructions 只描述服务能力和跨工具共同约定，不引入“语义工具 / 原有工具”分类，也不重复单个工具的参数分支。具体参数以工具 schema 和说明为准，详细流程由 resources 提供。
|
||||
|
||||
当前说明覆盖：按任务需要创建项目和素材文件夹、通过生成目标字段落画布或素材库且避免重复登记、付费异步提交与稳定幂等键、使用 `check_generation` 按返回间隔轮询、本地文件上传票据与实际传输分工、稳定媒体引用、按真实结果和告警判断完成情况。
|
||||
|
||||
不再要求所有任务先创建同名素材文件夹或同时写入画布与素材库。查询超时不代表生成失败，不应因此重新提交或更换幂等键。
|
||||
|
||||
权威文案位于 `external_mcp.rs` 的 `MCP_INSTRUCTIONS`，初始化响应与 `genarrative://external-editor/usage` 共用同一内容。该步骤只更新说明，不改变工具和 API；其它 resources 的内容更新见下一节。
|
||||
|
||||
### 5.6 resources 内容更新（2026-09-24）
|
||||
|
||||
沿用现有 7 个资源 URI。usage 共用 instructions，OpenAPI 继续提供现有 REST 契约；Skill 主入口和四份 references 按当前工具能力更新，不改变工具、路由、鉴权或 DTO。
|
||||
|
||||
- 主入口：服务能力、必要共同约定和按需阅读导航；不要求固定项目、同名文件夹或双重落库。
|
||||
- capability-routing：按意图选择工具与 action，说明上传、登记、生成、落画布之间的边界。
|
||||
- api-operations：工具到现有 REST 操作的映射，保留直接 REST 调用所需信息。
|
||||
- authentication-and-safety：MCP 与 REST 凭证配置、上传分工、幂等键位置、重试与删除范围。
|
||||
- requests-and-outputs：单功能及多功能 MCP 参数示例、异步轮询、按需指定生成目标、完整布局保存和结果/告警读取。
|
||||
|
||||
文档源位于 `.codex/skills/genarrative-external-editor-api/`，由 MCP resources 与下载的 Skill 包共用；本次不修改 Python helper、独立 CLI 或附件中的配套 Skill。说明仍随 API server 编译发布，不新增独立文档托管机制。验收覆盖资源读取、Skill 归档内容及 manifest 摘要一致性，并检查示例与当前 schema。
|
||||
|
||||
## 6. 核对入口
|
||||
|
||||
- [External v1 OpenAPI](../openapi/genarrative-external-v1.openapi.json)：operation、参数、请求体、响应与公开字段权威。
|
||||
- [External API 路由](../../server-rs/crates/api-server/src/modules/external_api.rs)：实际路由挂载。
|
||||
- [MCP 实现](../../server-rs/crates/api-server/src/external_mcp.rs)：工具生成、schema、资源和进程内分派。
|
||||
- [语义工具适配](../../server-rs/crates/api-server/src/external_mcp/semantic.rs)：新增工具、action、同源 schema 与参数映射。
|
||||
- [工具说明](../../server-rs/crates/api-server/prompts/external_mcp/semantic_tools.json)：新增工具的中文用途、操作和结果说明。
|
||||
- [External 编辑器接口](../../server-rs/crates/api-server/src/external_editor_api.rs)：鉴权、请求处理与生成受理。
|
||||
- [AGC 抠图模式与背景色透传](./【技术方案】AGC抠图模式与背景色透传-2026-09-16.md)：去背景模式与背景色已有合同。
|
||||
|
||||
参数合同按 2026-09-23 的实现核对，说明与验收状态于 2026-09-24 收口；后续变更仍以代码与 OpenAPI 为准，并同步修订本文，不维护第二份脱离 API 的字段真相。
|
||||
@@ -78,6 +78,8 @@ provider 原图已保存但透明背景处理最终失败时，worker 保留原
|
||||
|
||||
## 托管远程 MCP
|
||||
|
||||
新增语义工具的合同见 [外部 MCP 语义工具说明与参数设计](./technical/【技术方案】外部MCP语义工具说明与参数设计-2026-09-23.md)。15 个语义工具与原有 OpenAPI 自动映射工具同时可见；旧工具名称、schema、注解和行为保持不变，不删除、隐藏或加前缀。新工具按 action 选择一个既有 operation，平铺业务字段后复用下述 REST 分派；创建项目、项目资源、素材文件夹的新入口额外保留对应 API 的可选幂等键。所有结果沿用相同解包与错误处理，不裁剪告警或项目快照。resources 与 instructions 的内容更新另行处理，REST 契约不变。
|
||||
|
||||
`/api/external/v1/mcp` 是 Genarrative 托管的远程端点，Agent 只需配置 URL 和现有 API Key，不安装本地 MCP server。首版兼容 MCP `2025-11-25` initialize 生命周期，使用 JSON-RPC 2.0 和 Streamable HTTP，支持 `initialize`、`notifications/initialized`、`ping`、`tools/list`、`tools/call`、`resources/list`、`resources/read`。服务端使用无协议 session 的 JSON direct 模式，不依赖 sticky session，也不把 `Mcp-Session-Id` 作为业务身份。
|
||||
|
||||
MCP transport 的 DNS rebinding 防护必须同时允许正式入口 `www.genarrative.world` / `genarrative.world`、开发入口 `dev.genarrative.world` 和本机开发入口；对应 HTTPS Origin 也必须与公开环境同步登记。新增公开环境域名时，必须在发布前使用该域名的真实 `Host` 和 `Origin` 执行 `initialize` 回归，不能只用 `localhost` 单测证明端点可用。
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"find_canvas_projects": {
|
||||
"title": "查找画布项目",
|
||||
"description": "查找已有画布项目。action=list 返回项目摘要，recent 读取最近项目，get 按 projectId 读取完整项目、画布、图层和资源。先定位再读取；名称有歧义时明确候选，不静默创建替代项目。当前不提供分页、limit 或名称搜索参数。"
|
||||
},
|
||||
"manage_canvas_projects": {
|
||||
"title": "管理画布项目",
|
||||
"description": "创建或重命名画布项目。action=create 可选 title 和顶层 idempotencyKey，只创建项目，不自动创建同名素材文件夹；rename 必须提供 projectId 和 title。返回现有 API 的业务结果。删除使用 delete_resources。"
|
||||
},
|
||||
"find_assets": {
|
||||
"title": "查找与读取素材",
|
||||
"description": "查看素材记录或获取临时下载地址。action=list_library 读取账号素材库；get_project_resources 按 projectId 返回完整项目详情（含 resources），不裁剪；get_download_url 通过 objectKey 或兼容的 legacyPublicPath 获取临时访问 URL，可选 expireSeconds。优先稳定 objectKey；临时 URL 不作为持久生成引用。读取记录不等于查看媒体，不提供本地下载、关键词或相似素材搜索。"
|
||||
},
|
||||
"prepare_asset_upload": {
|
||||
"title": "办理素材上传",
|
||||
"description": "办理素材上传的两个独立步骤。action=create_upload_ticket 申请凭证，调用方按返回的 OSS 表单参数传输文件，再用 confirm_upload 确认对象；每次调用只执行一个步骤。工具不接受本地路径或 base64、不代传文件。owner 由 API Key 决定。确认对象不等于登记项目资源、素材库记录或创建画布图层，需要时另外登记。"
|
||||
},
|
||||
"generate_image": {
|
||||
"title": "生成图片",
|
||||
"description": "根据 prompt 和可选参考图付费生成图片。省略 kind 为普通图；spec、character、quick-edit、ui-design、publication-material 分别用于规范、角色、参考变体、UI 设计和宣发，不支持 scene。screenColor 是生成/抠图使用的纯色背景，不保证最终保留底色。使用支持的项目、素材库和 canvasCompletion 字段指定目标。必填稳定 idempotencyKey；返回异步任务 ID，用 check_generation 查询完成结果和告警。"
|
||||
},
|
||||
"modify_image": {
|
||||
"title": "修改图片",
|
||||
"description": "付费修改图片。action=edit 定向修改，sourceReferenceId 必须为已登记项目资源或素材 ID，不能用 objectKey/URL；variation 参考生成新版本，使用 referenceImageSrcs，固定 kind=quick-edit，无需传 kind，与 generate_image 的 quick-edit 相同；remove_background 对静态图片去背景，sourceImageSrc 可用所属 objectKey、资源或素材 ID。去背景默认 complex，只有 flat 可传非 null screenColor。edit/去背景的原位替换遵守 projectId+targetLayerId 来源绑定；变体不承诺原位替换。顶层 idempotencyKey 必填，异步结果用 check_generation 查询。"
|
||||
},
|
||||
"generate_icon_spritesheet": {
|
||||
"title": "生成图标图集",
|
||||
"description": "按已登记 icon-spec 规范 referenceId 与 iconDescriptions 付费生成图集并尝试切片；主 referenceId 不能用 objectKey/URL 或辅助 referenceImageSrcs 替代。sliceMode 必填无默认：grid 提供需求指定的 gridX/gridY（1–32），不传 sliceCount；connected-components 可传 sliceCount（1–256），不传 gridX/gridY。这不是已有图片通用裁切。稳定 idempotencyKey 必填，异步查询 check_generation；保留 warning 与 sliceWarning，图集成功不等于切片成功。"
|
||||
},
|
||||
"extract_ui_assets": {
|
||||
"title": "提取 UI 素材",
|
||||
"description": "以 sourceImageSrc 中的 UI 设计图为参考，付费生成组件素材图集并尝试切片；aspectRatio、imageSize 必填，可选 spritesheetLabel 命名。包含生成步骤，不保证逐像素原样裁出。明确图标清单和规范生成用 generate_icon_spritesheet。稳定 idempotencyKey 必填，异步查询 check_generation，分别判断完整图集与切片结果并保留告警。"
|
||||
},
|
||||
"generate_character_animation": {
|
||||
"title": "生成角色动画",
|
||||
"description": "根据角色源图和 promptText 付费生成动画预览与帧序列。sourceLayerId、sourceImageSrc、sourceWidth、sourceHeight 必须来自真实资源，不伪造；按 schema 提供输出参数。不能编辑已有动画文件。稳定 idempotencyKey 必填，异步查询 check_generation；消费正式帧序列结果，不用第一帧重复登记动画。"
|
||||
},
|
||||
"generate_video": {
|
||||
"title": "生成视频",
|
||||
"description": "根据文字和模型支持的参考图片、视频或音频付费生成视频片段。prompt、model、aspectRatio、durationSeconds、resolution、mode、sound 必填；mode 是视频业务模式。参考媒体、声音与输出组合依所选模型的现有能力，不能假定所有模型均支持。稳定 idempotencyKey 必填，返回异步任务 ID，通过 check_generation 查询。"
|
||||
},
|
||||
"generate_audio": {
|
||||
"title": "生成音频",
|
||||
"description": "付费生成音效或背景音乐。action=sound_effect 用于短声音、环境声和交互反馈，必填 prompt，duration 省略/null 自动，手动 0.5–30 秒；background_music 用于配乐，必填 gptDescriptionPrompt、makeInstrumental，保留各分支原字段。顶层稳定 idempotencyKey 必填，返回异步任务 ID，通过 check_generation 查询结果。"
|
||||
},
|
||||
"edit_canvas": {
|
||||
"title": "编辑画布",
|
||||
"description": "action=get 读取 projectId 对应项目；save_layout 基于最新 expectedRevision 保存完整 viewport 和 layers，不是单图层 patch，冲突时重新读取并处理；register_resource 登记已有媒体为项目资源，可选顶层 idempotencyKey，只登记资源不自动创建图层。正常生成结果落画布优先使用生成工具的 canvasCompletion。返回现有业务结果或真实版本冲突。"
|
||||
},
|
||||
"organize_asset_library": {
|
||||
"title": "整理素材库",
|
||||
"description": "整理素材文件夹和记录。action=create_folder 新建文件夹（可选顶层 idempotencyKey）；update_folder 修改 label/collapsed；create_asset 登记已有媒体元数据，不上传或生成，不支持幂等键；update_asset 修改名称或通过 folderId 移动素材。已经入库的生成结果不要重复登记。删除用 delete_resources。"
|
||||
},
|
||||
"check_generation": {
|
||||
"title": "查看生成进度与结果",
|
||||
"description": "按 operationId 查询一次生成任务状态，不阻塞等待完成。queued/running 按 pollAfterMs 再查；completed 才消费 result，保留 warning、sliceWarning；failed 如实返回安全错误。跨账号与不存在任务同样不可见。已知任务 ID 时直接查询；提交响应丢失而没有 ID 时，用原 API、原请求和原 idempotencyKey 重试提交取得受理结果，不换键重提。查询超时不改变任务状态。"
|
||||
},
|
||||
"delete_resources": {
|
||||
"title": "删除资源",
|
||||
"description": "按精确 ID 删除，调用前明确目标范围并取得相应用户授权。action=delete_project 删除项目并级联清理默认画布和项目资源元数据；delete_folder 将其素材移到默认文件夹后删除文件夹，默认文件夹不可删除；delete_asset 删除素材记录并处理关联精选审核状态。文件夹/素材记录删除不等于删除 OSS 文件。不支持批量、模糊匹配或按名称删除。"
|
||||
}
|
||||
}
|
||||
@@ -25,6 +25,8 @@ use tower::ServiceExt;
|
||||
|
||||
use crate::{modules, request_context::RequestContext, state::AppState};
|
||||
|
||||
mod semantic;
|
||||
|
||||
const OPENAPI_JSON: &str =
|
||||
include_str!("../../../../docs/openapi/genarrative-external-v1.openapi.json");
|
||||
const SKILL_MD: &str =
|
||||
@@ -54,7 +56,15 @@ const SKILL_REQUESTS_AND_OUTPUTS_URI: &str =
|
||||
"genarrative://external-editor/skill/references/requests-and-outputs.md";
|
||||
const MAX_MCP_REST_RESPONSE_BYTES: usize = 4 * 1024 * 1024;
|
||||
|
||||
const MCP_INSTRUCTIONS: &str = r#"陶泥儿外部编辑器工具。先创建或复用画布项目，并创建与画布同名的素材文件夹；生成结果应同时写入画布和素材库。参考本地文件时先走上传票据和对象确认，不要把 Data URL、Blob URL 或临时签名 URL写入生成参数。所有生成工具都是异步提交：必须提供 idempotencyKey，提交后按 pollAfterMs 调用 get_external_editor_generation_job，只有 status=completed 时消费 result；查询超时不能重新提交。图集生成必须显式声明 sliceMode，没有默认值：需求要求等分网格、固定槽位或指定行列数时用 grid 并提供来自需求的 gridX/gridY，自由排布或数量不定时用 connected-components（可用 sliceCount 约束张数），connected-components 不接受 gridX/gridY；缺失、越界或自相矛盾在计费前返回 400。warning 表示主结果可用但存在降级，sliceWarning 表示完整透明图集可用但切片未完成。详细说明、OpenAPI、Skill 主入口和分主题 references 见 resources/list；需要本地文件编排或不支持 MCP 时再下载 skill.zip。"#;
|
||||
const MCP_INSTRUCTIONS: &str = r#"陶泥儿提供画布项目管理、素材库管理，以及图片、角色动画、视频和音频生成能力。
|
||||
|
||||
按用户任务需要创建或复用项目、素材文件夹，不默认创建。生成结果需要进入画布或素材库时，使用对应生成工具支持的目标字段；已有落库结果不要重复登记。
|
||||
|
||||
生成操作会产生费用，采用异步提交。每次独立生成使用稳定的 idempotencyKey；取得任务 ID 后，使用 check_generation 按 pollAfterMs 查询，直到 completed 或 failed。查询超时不代表生成失败，不要因此重新提交或更换幂等键。
|
||||
|
||||
本地参考文件通过 prepare_asset_upload 获取上传票据，由调用方实际上传后确认对象。后续引用遵循各工具要求，使用稳定的对象键或资源、素材 ID；不要把临时下载 URL 当作持久引用。
|
||||
|
||||
以实际返回的结果和告警判断完成情况，部分产物成功不代表所有处理步骤成功。具体参数以工具 schema 和说明为准；需要详细流程、示例或 API 契约时，通过 resources/list 查找相关文档。"#;
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct McpOperation {
|
||||
@@ -125,7 +135,11 @@ impl ServerHandler for GenarrativeExternalMcp {
|
||||
_context: McpRequestContext<RoleServer>,
|
||||
) -> Result<ListToolsResult, ErrorData> {
|
||||
Ok(ListToolsResult::with_all_items(
|
||||
MCP_OPERATIONS.iter().map(mcp_operation_tool).collect(),
|
||||
MCP_OPERATIONS
|
||||
.iter()
|
||||
.map(mcp_operation_tool)
|
||||
.chain(semantic::TOOLS.iter().map(|entry| entry.tool.clone()))
|
||||
.collect(),
|
||||
))
|
||||
}
|
||||
|
||||
@@ -134,6 +148,7 @@ impl ServerHandler for GenarrativeExternalMcp {
|
||||
.iter()
|
||||
.find(|operation| operation.tool_name == name)
|
||||
.map(mcp_operation_tool)
|
||||
.or_else(|| semantic::find(name).map(|entry| entry.tool.clone()))
|
||||
}
|
||||
|
||||
async fn call_tool(
|
||||
@@ -141,12 +156,30 @@ impl ServerHandler for GenarrativeExternalMcp {
|
||||
request: CallToolRequestParams,
|
||||
context: McpRequestContext<RoleServer>,
|
||||
) -> Result<CallToolResult, ErrorData> {
|
||||
if let Some(tool) = semantic::find(request.name.as_ref()) {
|
||||
let result = match tool.prepare(request.arguments.unwrap_or_default()) {
|
||||
Ok(call) => {
|
||||
dispatch_operation(
|
||||
call.operation,
|
||||
call.arguments,
|
||||
&context,
|
||||
call.optional_idempotency_key,
|
||||
)
|
||||
.await
|
||||
}
|
||||
Err(error) => Err(error),
|
||||
};
|
||||
return Ok(match result {
|
||||
Ok(value) => CallToolResult::structured(value),
|
||||
Err(value) => CallToolResult::structured_error(value),
|
||||
});
|
||||
}
|
||||
let operation = MCP_OPERATIONS
|
||||
.iter()
|
||||
.find(|operation| operation.tool_name == request.name.as_ref())
|
||||
.ok_or_else(|| ErrorData::invalid_params("未知的陶泥儿外部 API 工具", None))?;
|
||||
let arguments = request.arguments.unwrap_or_default();
|
||||
match dispatch_operation(operation, arguments, &context).await {
|
||||
match dispatch_operation(operation, arguments, &context, None).await {
|
||||
Ok(value) => Ok(CallToolResult::structured(value)),
|
||||
Err(value) => Ok(CallToolResult::structured_error(value)),
|
||||
}
|
||||
@@ -475,6 +508,7 @@ async fn dispatch_operation(
|
||||
operation: &McpOperation,
|
||||
arguments: Map<String, Value>,
|
||||
context: &McpRequestContext<RoleServer>,
|
||||
optional_idempotency_key: Option<axum::http::HeaderValue>,
|
||||
) -> Result<Value, Value> {
|
||||
validate_required_body(operation, &arguments)?;
|
||||
|
||||
@@ -498,6 +532,51 @@ async fn dispatch_operation(
|
||||
.cloned()
|
||||
.ok_or_else(|| json!({"error": "Authorization 请求头缺失"}))?;
|
||||
|
||||
let request = build_operation_request(
|
||||
operation,
|
||||
&arguments,
|
||||
authorization,
|
||||
request_context,
|
||||
optional_idempotency_key,
|
||||
)?;
|
||||
let response = modules::external_api::router(state.clone())
|
||||
.with_state(state)
|
||||
.oneshot(request)
|
||||
.await
|
||||
.unwrap_or_else(|never| match never {});
|
||||
let status = response.status();
|
||||
let bytes = response
|
||||
.into_body()
|
||||
.collect()
|
||||
.await
|
||||
.map_err(|_| json!({"error": "读取外部 API 响应失败"}))?
|
||||
.to_bytes();
|
||||
if bytes.len() > MAX_MCP_REST_RESPONSE_BYTES {
|
||||
return Err(json!({"error": "外部 API 响应超过 MCP 返回上限"}));
|
||||
}
|
||||
let payload = serde_json::from_slice::<Value>(&bytes).unwrap_or_else(|_| {
|
||||
json!({
|
||||
"status": status.as_u16(),
|
||||
"message": "外部 API 返回了非 JSON 响应"
|
||||
})
|
||||
});
|
||||
if status.is_success() {
|
||||
Ok(unwrap_external_api_success_payload(payload))
|
||||
} else {
|
||||
Err(json!({
|
||||
"status": status.as_u16(),
|
||||
"response": payload,
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
fn build_operation_request(
|
||||
operation: &McpOperation,
|
||||
arguments: &Map<String, Value>,
|
||||
authorization: axum::http::HeaderValue,
|
||||
request_context: RequestContext,
|
||||
optional_idempotency_key: Option<axum::http::HeaderValue>,
|
||||
) -> Result<Request<Body>, Value> {
|
||||
let mut path = operation.path_template.clone();
|
||||
if let Some(path_parameters) = arguments.get("pathParameters").and_then(Value::as_object) {
|
||||
for (name, value) in path_parameters {
|
||||
@@ -532,37 +611,12 @@ async fn dispatch_operation(
|
||||
"application/json".parse().expect("valid content type"),
|
||||
);
|
||||
}
|
||||
apply_operation_headers(operation, &arguments, request.headers_mut())?;
|
||||
apply_operation_headers(operation, arguments, request.headers_mut())?;
|
||||
if let Some(key) = optional_idempotency_key {
|
||||
request.headers_mut().insert("idempotency-key", key);
|
||||
}
|
||||
|
||||
let response = modules::external_api::router(state.clone())
|
||||
.with_state(state)
|
||||
.oneshot(request)
|
||||
.await
|
||||
.unwrap_or_else(|never| match never {});
|
||||
let status = response.status();
|
||||
let bytes = response
|
||||
.into_body()
|
||||
.collect()
|
||||
.await
|
||||
.map_err(|_| json!({"error": "读取外部 API 响应失败"}))?
|
||||
.to_bytes();
|
||||
if bytes.len() > MAX_MCP_REST_RESPONSE_BYTES {
|
||||
return Err(json!({"error": "外部 API 响应超过 MCP 返回上限"}));
|
||||
}
|
||||
let payload = serde_json::from_slice::<Value>(&bytes).unwrap_or_else(|_| {
|
||||
json!({
|
||||
"status": status.as_u16(),
|
||||
"message": "外部 API 返回了非 JSON 响应"
|
||||
})
|
||||
});
|
||||
if status.is_success() {
|
||||
Ok(unwrap_external_api_success_payload(payload))
|
||||
} else {
|
||||
Err(json!({
|
||||
"status": status.as_u16(),
|
||||
"response": payload,
|
||||
}))
|
||||
}
|
||||
Ok(request)
|
||||
}
|
||||
|
||||
fn apply_operation_headers(
|
||||
@@ -977,15 +1031,16 @@ mod tests {
|
||||
let tools = MCP_OPERATIONS
|
||||
.iter()
|
||||
.map(mcp_operation_tool)
|
||||
.chain(semantic::TOOLS.iter().map(|entry| entry.tool.clone()))
|
||||
.collect::<Vec<_>>();
|
||||
let serialized = serde_json::to_vec(&tools).expect("tool catalog should serialize");
|
||||
assert!(serialized.len() < 512 * 1024);
|
||||
for operation in MCP_OPERATIONS.iter() {
|
||||
let serialized = serde_json::to_string(&operation.input_schema)
|
||||
for tool in tools {
|
||||
let serialized = serde_json::to_string(&tool.input_schema)
|
||||
.expect("tool input schema should serialize");
|
||||
assert!(serialized.len() < 64 * 1024, "{}", operation.tool_name);
|
||||
assert!(!serialized.contains("\"$ref\""), "{}", operation.tool_name);
|
||||
assert_eq!(operation.input_schema.get("type"), Some(&json!("object")));
|
||||
assert!(serialized.len() < 64 * 1024, "{}", tool.name);
|
||||
assert!(!serialized.contains("\"$ref\""), "{}", tool.name);
|
||||
assert_eq!(tool.input_schema.get("type"), Some(&json!("object")));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1024,6 +1079,269 @@ mod tests {
|
||||
})),
|
||||
json!({"operationId": "task-1", "status": "queued"})
|
||||
);
|
||||
let result = json!({
|
||||
"operationId": "task-1", "status": "completed",
|
||||
"result": {"objectKey": "media/sheet.png", "warning": {"code": "source_preserved"},
|
||||
"sliceWarning": {"code": "slice_failed"}, "project": {"revision": 7}}
|
||||
});
|
||||
assert_eq!(
|
||||
unwrap_external_api_success_payload(json!({"ok": true, "data": result})),
|
||||
result
|
||||
);
|
||||
}
|
||||
|
||||
fn rpc_request(method: &str, params: Value) -> Request<Body> {
|
||||
Request::builder()
|
||||
.method(Method::POST)
|
||||
.uri("/api/external/v1/mcp")
|
||||
.header(HOST, "localhost")
|
||||
.header(CONTENT_TYPE, "application/json")
|
||||
.header(ACCEPT, "application/json, text/event-stream")
|
||||
.header("mcp-protocol-version", "2025-11-25")
|
||||
.body(Body::from(
|
||||
json!({"jsonrpc": "2.0", "id": 1, "method": method, "params": params}).to_string(),
|
||||
))
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
async fn rpc_payload(response: axum::response::Response) -> Value {
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
serde_json::from_slice(&response.into_body().collect().await.unwrap().to_bytes()).unwrap()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn semantic_catalog_appends_tools_without_changing_legacy_definitions() {
|
||||
let payload = rpc_payload(
|
||||
service()
|
||||
.oneshot(rpc_request("tools/list", json!({})))
|
||||
.await
|
||||
.unwrap()
|
||||
.map(Body::new),
|
||||
)
|
||||
.await;
|
||||
let tools = payload["result"]["tools"].as_array().unwrap();
|
||||
assert_eq!(tools.len(), MCP_OPERATIONS.len() + 15);
|
||||
for op in MCP_OPERATIONS.iter() {
|
||||
let expected = serde_json::to_value(mcp_operation_tool(op)).unwrap();
|
||||
assert_eq!(
|
||||
tools.iter().find(|tool| tool["name"] == op.tool_name),
|
||||
Some(&expected)
|
||||
);
|
||||
}
|
||||
for entry in semantic::TOOLS.iter() {
|
||||
assert_eq!(
|
||||
GenarrativeExternalMcp.get_tool(&entry.tool.name),
|
||||
Some(entry.tool.clone())
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn semantic_invalid_arguments_fail_before_http_context_or_side_effects() {
|
||||
for (name, arguments) in [
|
||||
(
|
||||
"modify_image",
|
||||
json!({"action": "edit", "input": {"prompt": "修改", "sourceImageSrc": "wrong-reference"}, "idempotencyKey": "test"}),
|
||||
),
|
||||
(
|
||||
"delete_resources",
|
||||
json!({"action": "delete_project", "input": {}}),
|
||||
),
|
||||
(
|
||||
"manage_canvas_projects",
|
||||
json!({"action": "rename", "input": {"projectId": "project", "title": "新名"}, "idempotencyKey": "not-supported"}),
|
||||
),
|
||||
("generate_image", json!({"prompt": "test"})),
|
||||
] {
|
||||
let payload = rpc_payload(
|
||||
service()
|
||||
.oneshot(rpc_request(
|
||||
"tools/call",
|
||||
json!({"name": name, "arguments": arguments}),
|
||||
))
|
||||
.await
|
||||
.unwrap()
|
||||
.map(Body::new),
|
||||
)
|
||||
.await;
|
||||
assert_eq!(payload["result"]["isError"], true, "{name}: {payload}");
|
||||
assert!(payload["result"]["structuredContent"]["error"].is_string());
|
||||
assert!(!payload.to_string().contains("上下文缺失"));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn semantic_adapter_builds_real_rest_paths_bodies_and_optional_headers() {
|
||||
for (name, args, method, path, body, key) in [
|
||||
(
|
||||
"manage_canvas_projects",
|
||||
json!({"action":"create","input":{},"idempotencyKey":"create-project"}),
|
||||
Method::POST,
|
||||
"/api/external/v1/editor/projects",
|
||||
json!({}),
|
||||
Some("create-project"),
|
||||
),
|
||||
(
|
||||
"manage_canvas_projects",
|
||||
json!({"action":"rename","input":{"projectId":"project/a","title":"新名"}}),
|
||||
Method::PATCH,
|
||||
"/api/external/v1/editor/projects/project%2Fa/metadata",
|
||||
json!({"title":"新名"}),
|
||||
None,
|
||||
),
|
||||
(
|
||||
"find_assets",
|
||||
json!({"action":"get_download_url","input":{"objectKey":"images/a b.png","expireSeconds":60}}),
|
||||
Method::GET,
|
||||
"/api/external/v1/assets/read-url?expireSeconds=60&objectKey=images%2Fa+b.png",
|
||||
Value::Null,
|
||||
None,
|
||||
),
|
||||
(
|
||||
"find_canvas_projects",
|
||||
json!({"action":"list","input":{}}),
|
||||
Method::GET,
|
||||
"/api/external/v1/editor/projects?view=summary",
|
||||
Value::Null,
|
||||
None,
|
||||
),
|
||||
(
|
||||
"modify_image",
|
||||
json!({"action":"variation","input":{"prompt":"变体","referenceImageSrcs":["ref"]},"idempotencyKey":"same-generation"}),
|
||||
Method::POST,
|
||||
"/api/external/v1/editor/images/generations",
|
||||
json!({"prompt":"变体","referenceImageSrcs":["ref"],"kind":"quick-edit"}),
|
||||
Some("same-generation"),
|
||||
),
|
||||
] {
|
||||
let call = semantic::find(name)
|
||||
.unwrap()
|
||||
.prepare(args.as_object().unwrap().clone())
|
||||
.unwrap();
|
||||
let context = RequestContext::new(
|
||||
"test-request".into(),
|
||||
"POST /api/external/v1/mcp".into(),
|
||||
std::time::Duration::ZERO,
|
||||
false,
|
||||
);
|
||||
let request = build_operation_request(
|
||||
call.operation,
|
||||
&call.arguments,
|
||||
"Bearer fixture".parse().unwrap(),
|
||||
context,
|
||||
call.optional_idempotency_key,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(request.method(), method);
|
||||
assert_eq!(request.uri().to_string(), path);
|
||||
assert_eq!(request.headers()[AUTHORIZATION], "Bearer fixture");
|
||||
assert_eq!(
|
||||
request
|
||||
.headers()
|
||||
.get("idempotency-key")
|
||||
.map(|v| v.to_str().unwrap()),
|
||||
key
|
||||
);
|
||||
assert_eq!(
|
||||
request
|
||||
.extensions()
|
||||
.get::<RequestContext>()
|
||||
.unwrap()
|
||||
.request_id(),
|
||||
"test-request"
|
||||
);
|
||||
let bytes = request.into_body().collect().await.unwrap().to_bytes();
|
||||
if body.is_null() {
|
||||
assert!(bytes.is_empty());
|
||||
} else {
|
||||
assert_eq!(serde_json::from_slice::<Value>(&bytes).unwrap(), body);
|
||||
}
|
||||
}
|
||||
let operation = MCP_OPERATIONS
|
||||
.iter()
|
||||
.find(|op| op.operation_id == "createEditorProject")
|
||||
.unwrap();
|
||||
let mut headers = HeaderMap::new();
|
||||
apply_operation_headers(
|
||||
operation,
|
||||
&json!({"idempotencyKey": "legacy-ignored"})
|
||||
.as_object()
|
||||
.unwrap()
|
||||
.clone(),
|
||||
&mut headers,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(
|
||||
headers.get("idempotency-key").is_none(),
|
||||
"old optional-header behavior must remain unchanged"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn semantic_calls_reuse_rest_scope_checks_and_structured_errors() {
|
||||
use crate::state::external_api_auth::ExternalApiKeyAuthenticator;
|
||||
use futures_util::future::BoxFuture;
|
||||
use spacetime_client::{
|
||||
ExternalApiKeyAuthenticateRecordInput, ExternalApiKeyRecord, SpacetimeClientError,
|
||||
};
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
|
||||
struct NoScopes(AtomicUsize);
|
||||
impl ExternalApiKeyAuthenticator for NoScopes {
|
||||
fn authenticate_external_api_key(
|
||||
&self,
|
||||
_: ExternalApiKeyAuthenticateRecordInput,
|
||||
) -> BoxFuture<'_, Result<ExternalApiKeyRecord, SpacetimeClientError>> {
|
||||
self.0.fetch_add(1, Ordering::Relaxed);
|
||||
Box::pin(async {
|
||||
Ok(ExternalApiKeyRecord {
|
||||
key_id: "fixture-key".into(),
|
||||
owner_user_id: "owner-from-store".into(),
|
||||
name: "测试".into(),
|
||||
key_prefix: "tnr_sk_fixture".into(),
|
||||
scopes: vec![],
|
||||
created_at: "0.000000Z".into(),
|
||||
last_used_at: None,
|
||||
revoked_at: None,
|
||||
updated_at: "0.000000Z".into(),
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
let auth = Arc::new(NoScopes(AtomicUsize::new(0)));
|
||||
let state = AppState::new(AppConfig::default())
|
||||
.unwrap()
|
||||
.with_external_api_auth_state(crate::state::ExternalApiAuthState::new(auth.clone()));
|
||||
let router = modules::external_api::router(state.clone())
|
||||
.with_state(state)
|
||||
.layer(middleware::from_fn(attach_request_context));
|
||||
for (name, arguments) in [
|
||||
(
|
||||
"delete_resources",
|
||||
json!({"action":"delete_project","input":{"projectId":"fixture-project"}}),
|
||||
),
|
||||
(
|
||||
"delete_editor_project",
|
||||
json!({"pathParameters":{"projectId":"fixture-project"}}),
|
||||
),
|
||||
] {
|
||||
let mut request = rpc_request("tools/call", json!({"name":name,"arguments":arguments}));
|
||||
request
|
||||
.headers_mut()
|
||||
.insert(AUTHORIZATION, "Bearer tnr_sk_fixture".parse().unwrap());
|
||||
let payload = rpc_payload(router.clone().oneshot(request).await.unwrap()).await;
|
||||
assert_eq!(payload["result"]["isError"], true, "{payload}");
|
||||
assert_eq!(
|
||||
payload["result"]["structuredContent"]["status"], 403,
|
||||
"{payload}"
|
||||
);
|
||||
assert!(!payload.to_string().contains("owner-from-store"));
|
||||
}
|
||||
assert_eq!(
|
||||
auth.0.load(Ordering::Relaxed),
|
||||
4,
|
||||
"outer MCP and inner REST both authenticate"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -0,0 +1,460 @@
|
||||
//! 语义入口只负责操作选择和参数位置转换，业务校验与副作用仍由 External router 承担。
|
||||
|
||||
use super::*;
|
||||
use axum::http::HeaderValue;
|
||||
|
||||
pub(super) static TOOLS: LazyLock<Vec<SemanticTool>> = LazyLock::new(build_tools);
|
||||
|
||||
pub(super) struct SemanticTool {
|
||||
pub(super) tool: Tool,
|
||||
actions: Vec<Action>,
|
||||
}
|
||||
|
||||
struct Action {
|
||||
name: Option<&'static str>,
|
||||
operation: &'static McpOperation,
|
||||
input_schema: Value,
|
||||
key_schema: Option<Value>,
|
||||
fixed_body: Map<String, Value>,
|
||||
destructive: bool,
|
||||
}
|
||||
|
||||
pub(super) struct PreparedCall {
|
||||
pub(super) operation: &'static McpOperation,
|
||||
pub(super) arguments: Map<String, Value>,
|
||||
pub(super) optional_idempotency_key: Option<HeaderValue>,
|
||||
}
|
||||
|
||||
pub(super) fn find(name: &str) -> Option<&'static SemanticTool> {
|
||||
TOOLS.iter().find(|entry| entry.tool.name == name)
|
||||
}
|
||||
|
||||
fn build_tools() -> Vec<SemanticTool> {
|
||||
let openapi: Value = serde_json::from_str(OPENAPI_JSON).expect("embedded OpenAPI must parse");
|
||||
let descriptions: Value = serde_json::from_str(include_str!(
|
||||
"../../prompts/external_mcp/semantic_tools.json"
|
||||
))
|
||||
.expect("semantic tool descriptions must parse");
|
||||
let definitions: &[(&str, &[(&str, &str)])] = &[
|
||||
(
|
||||
"find_canvas_projects",
|
||||
&[
|
||||
("list", "listEditorProjects"),
|
||||
("recent", "loadRecentEditorProject"),
|
||||
("get", "getEditorProject"),
|
||||
],
|
||||
),
|
||||
(
|
||||
"manage_canvas_projects",
|
||||
&[
|
||||
("create", "createEditorProject"),
|
||||
("rename", "renameEditorProject"),
|
||||
],
|
||||
),
|
||||
(
|
||||
"find_assets",
|
||||
&[
|
||||
("list_library", "getEditorAssetLibrary"),
|
||||
("get_project_resources", "getEditorProject"),
|
||||
("get_download_url", "getExternalAssetReadUrl"),
|
||||
],
|
||||
),
|
||||
(
|
||||
"prepare_asset_upload",
|
||||
&[
|
||||
("create_upload_ticket", "createExternalDirectUploadTicket"),
|
||||
("confirm_upload", "confirmExternalAssetObject"),
|
||||
],
|
||||
),
|
||||
("generate_image", &[("", "generateExternalEditorImage")]),
|
||||
(
|
||||
"modify_image",
|
||||
&[
|
||||
("edit", "editExternalEditorImage"),
|
||||
("variation", "generateExternalEditorImage"),
|
||||
("remove_background", "removeExternalEditorImageBackground"),
|
||||
],
|
||||
),
|
||||
(
|
||||
"generate_icon_spritesheet",
|
||||
&[("", "generateExternalEditorIconSpritesheet")],
|
||||
),
|
||||
(
|
||||
"extract_ui_assets",
|
||||
&[("", "extractExternalEditorUiDesignAssets")],
|
||||
),
|
||||
(
|
||||
"generate_character_animation",
|
||||
&[("", "generateExternalEditorCharacterAnimation")],
|
||||
),
|
||||
("generate_video", &[("", "generateExternalEditorVideo")]),
|
||||
(
|
||||
"generate_audio",
|
||||
&[
|
||||
("sound_effect", "generateExternalEditorSoundEffect"),
|
||||
("background_music", "generateExternalEditorBackgroundMusic"),
|
||||
],
|
||||
),
|
||||
(
|
||||
"edit_canvas",
|
||||
&[
|
||||
("get", "getEditorProject"),
|
||||
("save_layout", "saveEditorProjectCanvas"),
|
||||
("register_resource", "createEditorProjectResource"),
|
||||
],
|
||||
),
|
||||
(
|
||||
"organize_asset_library",
|
||||
&[
|
||||
("create_folder", "createEditorAssetFolder"),
|
||||
("update_folder", "updateEditorAssetFolder"),
|
||||
("create_asset", "createEditorAsset"),
|
||||
("update_asset", "updateEditorAsset"),
|
||||
],
|
||||
),
|
||||
(
|
||||
"check_generation",
|
||||
&[("", "getExternalEditorGenerationJob")],
|
||||
),
|
||||
(
|
||||
"delete_resources",
|
||||
&[
|
||||
("delete_project", "deleteEditorProject"),
|
||||
("delete_folder", "deleteEditorAssetFolder"),
|
||||
("delete_asset", "deleteEditorAsset"),
|
||||
],
|
||||
),
|
||||
];
|
||||
definitions
|
||||
.iter()
|
||||
.map(|(name, operations)| {
|
||||
let actions = operations
|
||||
.iter()
|
||||
.map(|(action, operation)| {
|
||||
Action::new((!action.is_empty()).then_some(*action), operation, &openapi)
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
let read_only = actions.iter().all(|a| a.operation.method == Method::GET);
|
||||
let generation = actions.iter().any(|a| a.operation.requires_idempotency_key);
|
||||
let destructive = actions.iter().any(|a| a.destructive);
|
||||
let schema = tool_schema(&actions);
|
||||
let mut tool = Tool::new(
|
||||
name.to_string(),
|
||||
descriptions[name]["description"]
|
||||
.as_str()
|
||||
.expect("tool description")
|
||||
.to_string(),
|
||||
Arc::new(schema.as_object().expect("object schema").clone()),
|
||||
);
|
||||
tool.title = Some(
|
||||
descriptions[name]["title"]
|
||||
.as_str()
|
||||
.expect("tool title")
|
||||
.to_string(),
|
||||
);
|
||||
tool.annotations = Some(
|
||||
ToolAnnotations::new()
|
||||
.read_only(read_only)
|
||||
.destructive(destructive)
|
||||
.idempotent(
|
||||
read_only || actions.iter().all(|a| a.operation.requires_idempotency_key),
|
||||
)
|
||||
.open_world(
|
||||
generation || *name == "prepare_asset_upload" || *name == "find_assets",
|
||||
),
|
||||
);
|
||||
SemanticTool { tool, actions }
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
impl Action {
|
||||
fn new(name: Option<&'static str>, operation_id: &str, openapi: &Value) -> Self {
|
||||
// 按实际副作用声明；POST 也可能覆盖已有记录，参数名不能代表风险。
|
||||
let destructive = match operation_id {
|
||||
"listEditorProjects"
|
||||
| "loadRecentEditorProject"
|
||||
| "getEditorProject"
|
||||
| "getEditorAssetLibrary"
|
||||
| "getExternalAssetReadUrl"
|
||||
| "getExternalEditorGenerationJob"
|
||||
| "createEditorProject"
|
||||
| "createEditorProjectResource"
|
||||
| "createEditorAssetFolder"
|
||||
| "createEditorAsset"
|
||||
| "createExternalDirectUploadTicket" => false,
|
||||
// 对象确认允许更新同一 owner 的已有对象元数据。
|
||||
"confirmExternalAssetObject"
|
||||
| "renameEditorProject"
|
||||
| "saveEditorProjectCanvas"
|
||||
| "updateEditorAssetFolder"
|
||||
| "updateEditorAsset"
|
||||
| "deleteEditorProject"
|
||||
| "deleteEditorAssetFolder"
|
||||
| "deleteEditorAsset" => true,
|
||||
// 生成完成可修改已有画布状态，编辑与抠图还支持原位替换。
|
||||
"generateExternalEditorImage"
|
||||
| "editExternalEditorImage"
|
||||
| "removeExternalEditorImageBackground"
|
||||
| "generateExternalEditorIconSpritesheet"
|
||||
| "extractExternalEditorUiDesignAssets"
|
||||
| "generateExternalEditorCharacterAnimation"
|
||||
| "generateExternalEditorVideo"
|
||||
| "generateExternalEditorSoundEffect"
|
||||
| "generateExternalEditorBackgroundMusic" => true,
|
||||
_ => panic!("semantic operation must declare destructive risk: {operation_id}"),
|
||||
};
|
||||
let operation = MCP_OPERATIONS
|
||||
.iter()
|
||||
.find(|op| op.operation_id == operation_id)
|
||||
.expect("semantic tools must map to existing operations");
|
||||
let wrapped = &operation.input_schema["properties"];
|
||||
// 保留 body 的 if/then/allOf 等约束；仅合并位置包装，不重建字段定义。
|
||||
let mut input_schema = wrapped
|
||||
.get("body")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!({"type": "object", "properties": {}}));
|
||||
let mut required = input_schema
|
||||
.get("required")
|
||||
.and_then(Value::as_array)
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
for location in ["pathParameters", "queryParameters"] {
|
||||
if let Some(schema) = wrapped.get(location) {
|
||||
for (name, field) in schema["properties"]
|
||||
.as_object()
|
||||
.expect("parameter properties")
|
||||
{
|
||||
assert!(
|
||||
input_schema["properties"].get(name).is_none(),
|
||||
"ambiguous field {name}"
|
||||
);
|
||||
input_schema["properties"][name] = inline_openapi_schema(openapi, field, 0);
|
||||
}
|
||||
required.extend(
|
||||
schema
|
||||
.get("required")
|
||||
.and_then(Value::as_array)
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.cloned(),
|
||||
);
|
||||
}
|
||||
}
|
||||
let mut fixed_body = Map::new();
|
||||
if name == Some("variation") {
|
||||
input_schema["properties"]
|
||||
.as_object_mut()
|
||||
.unwrap()
|
||||
.remove("kind");
|
||||
required.retain(|field| field != "kind");
|
||||
fixed_body.insert("kind".into(), json!("quick-edit"));
|
||||
}
|
||||
if operation_id == "confirmExternalAssetObject" {
|
||||
input_schema["properties"]
|
||||
.as_object_mut()
|
||||
.unwrap()
|
||||
.remove("ownerUserId");
|
||||
}
|
||||
input_schema["required"] = Value::Array(required);
|
||||
input_schema["additionalProperties"] = json!(false);
|
||||
let path = operation.path_template.split('?').next().unwrap();
|
||||
let path_item = &openapi["paths"][path];
|
||||
let rest = &path_item[operation.method.as_str().to_ascii_lowercase()];
|
||||
let key_schema = path_item
|
||||
.get("parameters")
|
||||
.and_then(Value::as_array)
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.chain(
|
||||
rest.get("parameters")
|
||||
.and_then(Value::as_array)
|
||||
.into_iter()
|
||||
.flatten(),
|
||||
)
|
||||
.filter_map(|p| resolve_openapi_reference(openapi, p))
|
||||
.find(|p| p["in"] == "header" && p["name"] == "Idempotency-Key")
|
||||
.map(|p| {
|
||||
let mut schema = inline_openapi_schema(openapi, &p["schema"], 0);
|
||||
if let Some(description) = p.get("description") {
|
||||
schema["description"] = description.clone();
|
||||
}
|
||||
schema
|
||||
});
|
||||
Self {
|
||||
name,
|
||||
operation,
|
||||
input_schema,
|
||||
key_schema,
|
||||
fixed_body,
|
||||
destructive,
|
||||
}
|
||||
}
|
||||
|
||||
fn call_schema(&self) -> Value {
|
||||
let mut schema = match self.name {
|
||||
Some(name) => json!({
|
||||
"type": "object",
|
||||
"properties": {"action": {"type": "string", "const": name}, "input": self.input_schema},
|
||||
"required": ["action", "input"],
|
||||
"additionalProperties": false
|
||||
}),
|
||||
None => self.input_schema.clone(),
|
||||
};
|
||||
if let Some(key) = &self.key_schema {
|
||||
schema["properties"]["idempotencyKey"] = key.clone();
|
||||
if self.operation.requires_idempotency_key {
|
||||
schema["required"]
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.push(json!("idempotencyKey"));
|
||||
}
|
||||
}
|
||||
schema
|
||||
}
|
||||
}
|
||||
|
||||
fn tool_schema(actions: &[Action]) -> Value {
|
||||
if actions[0].name.is_none() {
|
||||
return actions[0].call_schema();
|
||||
}
|
||||
let mut schema = json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"action": {"type": "string", "enum": actions.iter().map(|a| a.name.unwrap()).collect::<Vec<_>>()},
|
||||
"input": {"type": "object"}
|
||||
},
|
||||
"required": ["action", "input"],
|
||||
"additionalProperties": false,
|
||||
"oneOf": actions.iter().map(Action::call_schema).collect::<Vec<_>>()
|
||||
});
|
||||
if let Some(key) = actions.iter().find_map(|a| a.key_schema.as_ref()) {
|
||||
schema["properties"]["idempotencyKey"] = key.clone();
|
||||
}
|
||||
schema
|
||||
}
|
||||
|
||||
impl SemanticTool {
|
||||
pub(super) fn prepare(&self, mut arguments: Map<String, Value>) -> Result<PreparedCall, Value> {
|
||||
let action = if self.actions[0].name.is_none() {
|
||||
&self.actions[0]
|
||||
} else {
|
||||
let name = arguments
|
||||
.get("action")
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| json!({"error": "必须提供字符串 action"}))?;
|
||||
self.actions
|
||||
.iter()
|
||||
.find(|a| a.name == Some(name))
|
||||
.ok_or_else(|| json!({"error": "未知 action"}))?
|
||||
};
|
||||
validate_fields(&action.call_schema(), &arguments)?;
|
||||
let key = arguments.remove("idempotencyKey");
|
||||
let mut optional_idempotency_key = None;
|
||||
if let Some(key) = &key {
|
||||
let key = key
|
||||
.as_str()
|
||||
.ok_or_else(|| json!({"error": "idempotencyKey 必须是字符串"}))?;
|
||||
if key.is_empty() || key.len() > 128 || !key.bytes().all(|c| (b'!'..=b'~').contains(&c))
|
||||
{
|
||||
return Err(json!({"error": "idempotencyKey 必须为 1–128 个非空白 ASCII 字符"}));
|
||||
}
|
||||
if !action.operation.requires_idempotency_key {
|
||||
optional_idempotency_key = Some(
|
||||
HeaderValue::from_str(key)
|
||||
.map_err(|_| json!({"error": "idempotencyKey 不是合法 HTTP 头值"}))?,
|
||||
);
|
||||
}
|
||||
}
|
||||
let input = if action.name.is_some() {
|
||||
arguments
|
||||
.remove("input")
|
||||
.and_then(|value| value.as_object().cloned())
|
||||
.ok_or_else(|| json!({"error": "input 必须是 JSON 对象"}))?
|
||||
} else {
|
||||
arguments
|
||||
};
|
||||
validate_fields(&action.input_schema, &input)?;
|
||||
let wrapped = &action.operation.input_schema["properties"];
|
||||
let mut mapped = Map::new();
|
||||
for location in ["pathParameters", "queryParameters", "body"] {
|
||||
if let Some(schema) = wrapped.get(location) {
|
||||
let mut fields = input
|
||||
.iter()
|
||||
.filter(|(name, _)| schema["properties"].get(*name).is_some())
|
||||
.map(|(name, value)| (name.clone(), value.clone()))
|
||||
.collect::<Map<_, _>>();
|
||||
if location == "body" {
|
||||
fields.extend(action.fixed_body.clone());
|
||||
}
|
||||
// 有请求体的操作始终发送对象，包括无字段的项目创建。
|
||||
if location == "body" || !fields.is_empty() {
|
||||
mapped.insert(location.into(), Value::Object(fields));
|
||||
}
|
||||
}
|
||||
}
|
||||
if action.operation.requires_idempotency_key {
|
||||
if let Some(key) = key {
|
||||
mapped.insert("idempotencyKey".into(), key);
|
||||
}
|
||||
}
|
||||
Ok(PreparedCall {
|
||||
operation: action.operation,
|
||||
arguments: mapped,
|
||||
optional_idempotency_key,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 只校验适配层结构和直接字段，不实现第二套业务 schema 验证器。
|
||||
// 嵌套字段与跨字段条件在现有 REST DTO/业务入口中校验，完整 schema 仍向客户端提供。
|
||||
fn validate_fields(schema: &Value, input: &Map<String, Value>) -> Result<(), Value> {
|
||||
let properties = schema["properties"]
|
||||
.as_object()
|
||||
.expect("input schema properties");
|
||||
for name in schema["required"]
|
||||
.as_array()
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.filter_map(Value::as_str)
|
||||
{
|
||||
if !input.contains_key(name) {
|
||||
return Err(json!({"error": "缺少必填字段", "field": name}));
|
||||
}
|
||||
}
|
||||
for (name, value) in input {
|
||||
let field = properties
|
||||
.get(name)
|
||||
.ok_or_else(|| json!({"error": "当前操作不接受此字段", "field": name}))?;
|
||||
let matches_type = |kind: &str| match kind {
|
||||
"string" => value.is_string(),
|
||||
"object" => value.is_object(),
|
||||
"array" => value.is_array(),
|
||||
"boolean" => value.is_boolean(),
|
||||
"number" => value.is_number(),
|
||||
"integer" => {
|
||||
value.is_i64() || value.is_u64() || value.as_f64().is_some_and(|v| v.fract() == 0.0)
|
||||
}
|
||||
"null" => value.is_null(),
|
||||
_ => true,
|
||||
};
|
||||
let valid_type = match &field["type"] {
|
||||
Value::String(kind) => matches_type(kind),
|
||||
Value::Array(kinds) => kinds.iter().filter_map(Value::as_str).any(matches_type),
|
||||
_ => true,
|
||||
};
|
||||
if !valid_type
|
||||
|| field
|
||||
.get("enum")
|
||||
.and_then(Value::as_array)
|
||||
.is_some_and(|values| !values.contains(value))
|
||||
|| field.get("const").is_some_and(|expected| expected != value)
|
||||
{
|
||||
return Err(json!({"error": "字段类型或取值不符合当前操作", "field": name}));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user