diff --git a/.codex/skills/genarrative-external-editor-api/SKILL.md b/.codex/skills/genarrative-external-editor-api/SKILL.md index c23c63777..d57252df5 100644 --- a/.codex/skills/genarrative-external-editor-api/SKILL.md +++ b/.codex/skills/genarrative-external-editor-api/SKILL.md @@ -16,6 +16,7 @@ Connect to `https://www.genarrative.world/api/external/v1/mcp` using Streamable - Upload local references using `prepare_asset_upload`: request a ticket, transfer the file from the client, then confirm the object. Confirmation does not create a canvas layer or a project/library record. Use the reference type accepted by the target tool; some operations require a registered resource or asset ID rather than an object key. - Generation is paid and asynchronous. Keep one stable `idempotencyKey` per logical generation and retain the returned `operationId`. Call `check_generation` according to `pollAfterMs`; consume `result` only after `completed`, and report the safe error on `failed`. A polling timeout does not justify another generation. - Read actual artifacts and warnings before claiming the requested deliverable is complete. Use project/library reads for complete persisted records, and `find_assets` with `action=get_download_url` for temporary media access. +- Treat `generationInputs` as generation context and provenance metadata, subject to each endpoint's preservation and rebuilding rules. Arbitrary fields, including `artSpec`, do not automatically enter the provider prompt or override request parameters. Put generation requirements in the endpoint's explicit inputs; see [Generation Inputs Metadata](references/requests-and-outputs.md#generation-inputs-metadata) for persistence, reads, and known consumers. - Keep API Keys and temporary upload/download credentials out of chat, repository files, and logs. Business calls operate within the API Key's owner and scopes. ## Documentation Navigation diff --git a/.codex/skills/genarrative-external-editor-api/references/capability-routing.md b/.codex/skills/genarrative-external-editor-api/references/capability-routing.md index 945474c68..2d6da819a 100644 --- a/.codex/skills/genarrative-external-editor-api/references/capability-routing.md +++ b/.codex/skills/genarrative-external-editor-api/references/capability-routing.md @@ -20,7 +20,7 @@ For generation, pass `projectId` with `canvasCompletion` when the result should ## Art Spec Routing -For a series of related art requests, an optional reusable spec can carry the shared requirements: +For a series of related art requests, an optional caller-defined spec can record the shared requirements. `artSpec` is an organizational convention inside `generationInputs`, not a server-defined generation parameter schema: ```json { @@ -35,7 +35,7 @@ For a series of related art requests, an optional reusable spec can carry the sh } ``` -Infer what is already clear and ask only for missing fields that block the selected endpoint. Reuse the current spec unless the user changes style, subject family, palette, format, or constraints. Store structured context under `generationInputs.artSpec` where supported and summarize it in the prompt when useful. +Infer what is already clear and ask only for missing fields that block the selected endpoint. Reuse the current spec unless the user changes style, subject family, palette, format, or constraints. Where the endpoint preserves custom metadata, `generationInputs.artSpec` can retain this context for later retrieval. To affect generation, always translate the relevant requirements into the endpoint's explicit inputs: image `prompt`, scene `sceneContent` / `stylePreset` / `customStyle`, or spritesheet `iconDescriptions`, plus the actual size and reference parameters. Neither `artSpec.references` nor `generationInputs.references` supplies reference media by itself. Scene and sound-effect generation rebuild their metadata and do not preserve an arbitrary `artSpec`; keep a caller-side copy when needed. See [Generation Inputs Metadata](requests-and-outputs.md#generation-inputs-metadata) for the rules applying to the entire `generationInputs` field. ## Intent Map diff --git a/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md b/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md index 10e41f319..37647c6f6 100644 --- a/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md +++ b/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md @@ -9,6 +9,7 @@ Use this reference to build generation payloads, carry canvas/library context, p - [Polling State Machine](#polling-state-machine) - [Canvas and Asset-Library Completion](#canvas-and-asset-library-completion) - [Saving Existing Canvas Layout](#saving-existing-canvas-layout) +- [Generation Inputs Metadata](#generation-inputs-metadata) - [Art Spec and Image Request](#art-spec-and-image-request) - [Local Reference Requests](#local-reference-requests) - [Compact Completed Result](#compact-completed-result) @@ -135,7 +136,7 @@ Background removal preserves the source image dimensions. For normal canvas plac Character animation accepts `assetFolderId` and `assetLabel` and persists the generated sequence. Consume the returned animation artifacts and persisted identities; do not synthesize a duplicate animation asset from the first frame. Use complete project/library records when complete persisted state is needed. -For the lower-level asset/resource creation endpoints, `generationInputs` is replayable request context rather than a media-runtime container. When `assetKind` is `character-animation`, the server rejects legacy runtime keys including `characterAnimation`, `frames`, `previewVideoPath`, `frameCount`, `fps`, and `durationSeconds`; send the formal sequence through `imageSequenceFrames` and `imageSequenceDurationMs`. Internal processing audit keys such as `screenColorHex`, `mattingProvider`, and `mattingModel` are removed before persistence. +For the lower-level asset/resource creation endpoints, `generationInputs` follows the metadata and media-runtime boundaries described in [Generation Inputs Metadata](#generation-inputs-metadata). ## Saving Existing Canvas Layout @@ -146,15 +147,46 @@ For the lower-level asset/resource creation endpoints, `generationInputs` is rep `edit_canvas/register_resource` registers existing media but does not create a canvas layer. `organize_asset_library/create_asset` creates metadata but does not upload or generate media. For generated media placement, prefer the generation tool's supported `canvasCompletion`; inspect returned identities before registering anything again. +## Generation Inputs Metadata + +`generationInputs` is optional JSON generation context: an input snapshot, provenance, and supported application metadata. An object is the useful shape for named fields; accepting `JsonValue` does not promise lossless storage of every JSON value. The selected endpoint may sanitize, augment, or rebuild it. Arbitrary metadata is not automatically included in the provider prompt, used as generation parameters, or applied to a later request. Put requirements into the endpoint's explicit inputs, such as `prompt`, `iconDescriptions`, `sceneContent`, style/size options, and its actual reference-media fields. + +When an operation persists project resources or library assets, their accepted metadata is saved with those records. Retrieve it through `GET /api/external/v1/editor/projects/{projectId}` (`project.resources[].generationInputs`) or `GET /api/external/v1/editor/assets/library` (`library.assets[].generationInputs`), subject to owner/scopes and record existence. MCP equivalents are `find_assets/get_project_resources` and `find_assets/list_library`. A compact generation result is not a complete metadata read. Metadata is not embedded in the image bytes, and uploading an image does not restore a previous record's metadata. + +| Operation | Current handling of `generationInputs` | +| --- | --- | +| Create a project resource or library asset | Preserve accepted metadata after removing client-supplied `references` and internal audit keys. This registers a record; it does not execute a generation recipe. | +| Generate an image | Preserve custom object fields on the provider's original image record after sanitization; rebuild `references` from actual authorized reference inputs. Character transparency processing may create a separate derived record. | +| Generate a scene | Rebuild V2 `version`, `action=scene.generate`, `fields`, and `references` from normalized scene parameters. Only the exact client marker `source=ai-game-creator-client` is retained additionally; arbitrary custom fields such as `artSpec` are discarded. | +| Edit an image | Preserve accepted custom fields and rebuild source/auxiliary `references` from `sourceReferenceId` and the actual reference inputs. | +| Remove a background | Preserve accepted custom fields and rebuild `references` from the actual source. Processing audit metadata remains internal. This metadata does not configure the removal operation. | +| Generate an icon spritesheet or extract UI assets | Preserve accepted custom fields on the provider's original image record. Transparent sheets and slices have separate processing-stage/source metadata and do not automatically inherit all custom fields. Follow the returned source references to read the original context. | +| Generate a character animation | Preserve accepted generation context on the final sequence record; formal frames and sequence duration are separate media fields, not runtime data inside `generationInputs`. | +| Generate a video | Preserve accepted context; object metadata can also receive an added/updated duration display field from normalized request parameters. | +| Generate a sound effect | Rebuild `fields`, empty `references`, and `soundEffect` metadata from the actual generation. Only `source`, `conversationId`, and `toolCallMessageId` are copied from a caller-supplied object; arbitrary fields such as `artSpec` are discarded. | +| Generate background music | Preserve accepted context after sanitization; generation parameters come from the explicit request fields. | + +Preservation does not mean every field is inert. Known consumers include: + +- The canvas reads `fields` / `references` for input display. Recognized V2 `version`, `action`, and stable field/reference IDs support restoring supported generation panels; arbitrary metadata does not guarantee a UI display or a “modify” action. +- Icon spritesheet generation reads the saved reference spec's `fields` entry titled `游戏类型` to select genre-specific prompt text. This does not cause arbitrary fields in the current request to be interpreted as prompts. +- Integrated clients use recognized `source` markers for queue/idempotency namespaces and result projections. These are application markers, not authentication or model instructions. + +Reference metadata does not grant access or select reference media. Image operations rebuild it from actual inputs and authorized records; direct resource/asset creation drops caller-supplied references. Supply the documented `referenceId`, `sourceReferenceId`, or media-reference fields. Do not assume other operations provide the same provenance rebuilding. + +Persisted metadata is bounded to 64 KiB of serialized JSON and cannot contain inline media Data URLs. Top-level internal audit fields `screenColorHex`, `mattingProvider`, and `mattingModel` are stripped from client metadata and owner-facing reads; the server can store its own internal audit values. When `assetKind=character-animation`, legacy runtime keys such as `characterAnimation`, `frames`, `previewVideoPath`, `frameCount`, `fps`, and `durationSeconds` are rejected; use `imageSequenceFrames` and `imageSequenceDurationMs` for formal sequence data. Omitting metadata or sending null does not replace required request parameters; empty objects may normalize to null. + +For reuse, read the saved record, recover the relevant requirements, and explicitly construct the next request. Keep the original complete request and idempotency key for retries: stored metadata, especially derived-asset metadata, is not a complete replayable HTTP payload. + ## Art Spec and Image Request Game scenes have a dedicated structured route: `POST /api/external/v1/editor/scenes/generations` with `sceneContent` and `stylePreset` (`customStyle` required when `stylePreset` is `custom`). The server assembles the full provider prompt; a caller-assembled `prompt` is not accepted. `kind: "scene"` and `assetKind: "scene"` remain invalid on generic image generation and return HTTP `400` before any generation job is queued. -When maintaining a reusable art spec, carry it in `generationInputs.artSpec` and reflect important constraints in the prompt. This is an example with both canvas and library destinations, not a requirement for every generation: +`generationInputs.artSpec` is an optional caller-defined metadata convention with no automatic prompt or parameter effect. In the generic image request below, the prompt repeats the desired style, palette, composition, and exclusions, while `aspectRatio` and `imageSize` set the actual format. The saved spec can help a caller construct later requests. This example uses both canvas and library destinations; neither the spec nor both destinations are required for every generation. Do not copy this metadata expectation to the scene route, which rebuilds its own context. ```json { - "prompt": "一张横版幻想森林背景,适合游戏主视觉,无文字", + "prompt": "一张横版幻想森林背景,适合游戏主视觉,手绘游戏概念图风格,翡翠绿与金色光斑,中心留出角色站位,无文字、无 UI 按钮", "aspectRatio": "16:9", "imageSize": "1K", "projectId": "", diff --git a/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/direct-tools.json b/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/direct-tools.json index b51fa82bd..17a012862 100644 --- a/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/direct-tools.json +++ b/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/direct-tools.json @@ -11,18 +11,17 @@ "agc_update_plan.description": "更新当前回合的进度计划,字段与 update_plan 相同:可选 explanation,以及 plan 中的 step/status(pending、in_progress、completed)。它可与其它独立工具并行;同一计划的连续更新按依赖顺序提交。计划完成只表示进度,不代替宿主交付验收。", "agc_write_file.parameters.path": "当前项目根下的相对路径,例如 game/index.html、assets/manifest.json 或 data/gameplay-spec.md", "agc_write_file.parameters.content": "仅填写目标文件的完整原始 UTF-8 正文", - "taonier_prepare_game_art.description": "创建或恢复当前 AGC 项目的陶泥儿标准游戏美术包。默认复用有效美术包;根据当前对话需要选择 regenerate 重新生成。授权使用 AGC 客户端当前登录会话;遇到 401/403 时报告客户端登录或权限状态异常并停止。", - "taonier_prepare_game_art.parameters.brief": "面向当前游戏的简洁视觉需求", + "taonier_prepare_game_art.description": "创建或恢复当前 AGC 项目的陶泥儿标准游戏美术包,返回总图、实际切片与路径标注预览;需看图识别用途,内容要求不保证切片数量或顺序。默认复用有效美术包;根据当前对话需要选择 regenerate 重新生成。授权使用 AGC 客户端当前登录会话;遇到 401/403 时报告客户端登录或权限状态异常并停止。", + "taonier_prepare_game_art.parameters.brief": "面向当前游戏的简洁视觉需求,不超过 200 字符。写明主题、风格、玩家主体及状态、目标或收集物、障碍或场景元素、反馈特效等具体需要的素材。需求明确时列出各项素材的数量、状态,并要求独立排布、留出切分间距;数量未确定时不编造。工具会补齐沿用规范图和素材独立排布的通用要求;数量是生成目标,内容类别和数量均不保证实际切片数量或返回顺序,须看图确认用途", "taonier_prepare_game_art.parameters.mode": "缺省安全复用有效美术包;Codex 仅在当前对话需要换一套或重新生成时使用 regenerate", "agc_generate_image.description": "生成一张新图片:普通插画、角色立绘、统一视觉规范图、游戏 UI 设计图或透明游戏素材图集。仅在用户明确要求生成新图时调用。", - "agc_generate_image.parameters.prompt": "完整图片描述;普通图片、角色、规范图、UI 设计图或透明图集均可。kind=icon-spritesheet 时,去除首尾空白后的描述须为 1 到 200 个 Unicode 字符,保留内部换行并作为单条 iconDescriptions 原样提交;超限拒绝,不截断、不拆条,客户端不追加生图指令", + "agc_generate_image.parameters.prompt": "完整图片描述;普通图片、角色、规范图、UI 设计图或透明图集均可。kind=icon-spritesheet 时,需求明确则列出各项素材的数量、状态,并要求独立排布、留出切分间距;数量未确定时不编造。数量是生成目标,不保证实际切片数量或返回顺序,须看图确认用途。去除首尾空白后的图集描述须为 1 到 200 个 Unicode 字符,保留内部换行并作为单条 iconDescriptions 原样提交;超限拒绝,不截断、不拆条,客户端不追加生图指令", "agc_generate_image.parameters.kind": "image=普通新图(保留生成原图),character=角色图(纯色底生成后自动抠图,产出透明背景立绘,prompt 只描述角色主体),icon-spec=统一视觉规范图,ui-design=完整 UI 设计图,icon-spritesheet=透明游戏素材图集(纯色底生成后自动抠图并切片,项目须已有 icon-spec 规范图),publication-material=发布宣传图", "agc_generate_image.parameters.assetName": "本地素材的人类可读显示名称", "agc_generate_image.parameters.outputPath": "可选项目相对输出路径,必须位于 assets/ 且不能覆盖已有文件", - "agc_generate_image.parameters.sliceMode": "仅适用于 kind=icon-spritesheet,且必填:需求明确要求等分网格、固定槽位或指定行列数时传 grid,并用 gridX/gridY 传入需求中的行列数;自由排布、数量不定或只要求一张图集时传 connected-components,需要约束素材张数时用 sliceCount。", + "agc_generate_image.parameters.sliceMode": "仅适用于 kind=icon-spritesheet,且必填:需求明确要求等分网格、固定槽位或指定行列数时传 grid,并用 gridX/gridY 传入需求中的行列数;自由排布、数量不定或只要求一张图集时传 connected-components,实际切片数量由图像决定,内容需求不保证数量或用途顺序;查看返回图片后识别用途。", "agc_generate_image.parameters.gridX": "grid 模式横向网格数量,只能与 sliceMode=grid 同时提供", "agc_generate_image.parameters.gridY": "grid 模式纵向网格数量,只能与 sliceMode=grid 同时提供", - "agc_generate_image.parameters.sliceCount": "只与 kind=icon-spritesheet 且 sliceMode=connected-components 同时提供,用于约束目标素材张数;省略时按图像内容自动识别", "agc_generate_image.parameters.screenColor": "抠图纯色背景,仅用于 kind=character(角色形象)和 kind=icon-spritesheet(图标素材)。生成时把主体置于该纯色背景上,回图后据此抠除背景。取值为 auto 或下列色板 hex 之一,传值只填 hex 本身:#CFEFFF(浅雾蓝)、#B0C2E0(浅钢蓝)、#FFD6C2(暖浅桃色)、#E6D8FF(淡薰衣草紫)、#F4D8E8(浅粉灰)、#7FB3FF(中度天蓝)、#FFF2A8(浅柠黄)、#CFFFE1(淡薄荷绿)、#D8DEE8(浅中性灰)、#D8D2E8(淡灰紫)、#A8F7F0(高对比浅青)、#A0BBA0(灰竹绿);auto 时由服务端自动选色。手动指定时选择与主体颜色明显不同的背景色", "agc_edit_image.description": "修改一张已登记图片:换装、改色、换背景或局部重绘。sourceLocalAssetId 必须使用 agc_list_registered_assets 返回的当前项目图片 localAssetId。", "agc_edit_image.parameters.sourceLocalAssetId": "当前项目已登记的图片 localAssetId", diff --git a/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/execution.json b/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/execution.json index b9fd08ea2..44238a227 100644 --- a/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/execution.json +++ b/apps/ai-game-creator-shell/src-tauri/prompts/runtime/texts/execution.json @@ -6,7 +6,7 @@ "playtest.tetris": "完成合同要求 tetris-v1 交互试玩。game/index.html 必须持续更新