Files
Genarrative/.codex/skills/genarrative-external-editor-api/references/api-selection.md
T
lhk229 aecabdacfd 新抠图算法 (#85)
Co-authored-by: 段舒康 <kdletters@qq.com>
Reviewed-on: https://git.genarrative.world/git/GenarrativeAI/Genarrative/pulls/85
Co-authored-by: Linghong <ink29535@proton.me>
Co-committed-by: Linghong <ink29535@proton.me>
2026-07-16 18:33:06 +08:00

14 KiB

External Editor API Routing

Source of truth: docs/openapi/genarrative-external-v1.openapi.json.

Base

  • Fixed base URL: https://www.genarrative.world/.
  • Public contract: GET /api/external/v1/openapi.json.
  • Authenticated calls: Authorization: Bearer <tnr_sk_...>.
  • Default credentials file: ~/.config/genarrative/external-editor-api.json with an apiKey string.
  • Generation clients should allow long-running responses. Use at least 420 seconds for character animation and video; 70 seconds is too short for animation.

Canvas Session and Art Spec

At the start of a new conversation, ask for a canvas name before the first generation call unless the user already supplied projectId and assetFolderId. Create or reuse:

  1. POST /api/external/v1/editor/projects with title = canvas name.
  2. GET /api/external/v1/editor/assets/library; if no folder has the same label, POST /api/external/v1/editor/assets/folders with label = canvas name.
  3. Keep canvasName, projectId, assetFolderId, and the current art spec in conversation state.

Before generating art assets, normalize the user's request into a current art spec with assetType, subject, style, palette, composition, format, constraints, and references. Ask follow-up questions only for missing fields that block the selected endpoint. Reuse the current spec automatically when the user asks for another asset without changing style/spec requirements. Put the spec in generationInputs.artSpec and summarize it in the prompt when useful.

Intent Routing

Infer the endpoint from the user's description. Do not present this as a menu unless the request is genuinely ambiguous.

User says Route
"生成图片", "生图", "做一张背景/角色/宣发图" POST /api/external/v1/editor/images/generations
"重绘", "调整这张图", "基于这张图修改" POST /api/external/v1/editor/images/edits
"用这张参考图", "参考本地图片生成", "基于本地图做图" Upload local image first, then pass returned objectKey into the generation/edit reference field
"按规范图生成图标", "拆图标" POST /api/external/v1/editor/icon-spritesheets/generations
"从 UI 设计图提取素材" POST /api/external/v1/editor/ui-designs/assets/extractions
"让角色动起来", "生成角色动画帧" POST /api/external/v1/editor/character-animations/generations
"生成视频" POST /api/external/v1/editor/videos/generations
"生成音效" POST /api/external/v1/editor/audios/sound-effects/generations
"生成背景音乐/BGM" POST /api/external/v1/editor/audios/background-music/generations
"上传本地素材/图片/音频/视频" Upload flow: direct upload ticket -> OSS form upload -> object confirm
"保存画板布局" PATCH /api/external/v1/editor/projects/{projectId}/canvas
"创建/读取/删除画板项目" Project endpoints
"素材库/文件夹/素材记录" Asset library endpoints
"读取私有素材/拿可访问链接" GET /api/external/v1/assets/read-url

Ask a follow-up only when two routes could both be correct and produce different artifacts, for example "处理这张图" without saying edit, extract UI assets, or use it as a reference for new generation.

Endpoint Map

User intent Endpoint Minimum request
Read contract GET /api/external/v1/openapi.json No auth required
List projects GET /api/external/v1/editor/projects API Key
Create project POST /api/external/v1/editor/projects Optional title
Load recent project GET /api/external/v1/editor/projects/recent API Key
Get/delete project GET or DELETE /api/external/v1/editor/projects/{projectId} projectId
Rename project PATCH /api/external/v1/editor/projects/{projectId}/metadata title
Save canvas layout PATCH /api/external/v1/editor/projects/{projectId}/canvas viewport, layers
Add project resource POST /api/external/v1/editor/projects/{projectId}/resources imageSrc, width, height, sourceType
Create upload ticket POST /api/external/v1/assets/direct-upload-tickets legacyPrefix, fileName
Confirm uploaded object POST /api/external/v1/assets/objects/confirm objectKey, assetKind
Get signed read URL GET /api/external/v1/assets/read-url objectKey or legacyPublicPath
Read asset library GET /api/external/v1/editor/assets/library API Key
Create/update/delete folder POST /api/external/v1/editor/assets/folders, PATCH/DELETE /api/external/v1/editor/assets/folders/{folderId} create: label; update: label or collapsed
Create asset record POST /api/external/v1/editor/assets folderId, label, imageSrc, width, height, sourceType
Update/delete asset PATCH/DELETE /api/external/v1/editor/assets/{assetId} update: label or folderId

Generation Endpoints

User intent Endpoint Required fields Common optional fields
Generate image/spec/character/UI/publication material POST /api/external/v1/editor/images/generations prompt kind, model, aspectRatio, imageSize, size, referenceImageSrcs, projectId, assetFolderId, assetLabel, canvasCompletion, generationInputs
Edit/redraw image POST /api/external/v1/editor/images/edits prompt, sourceImageSrc referenceImageSrcs, model, size, projectId, assetFolderId, assetLabel, sourceResourceId, targetLayerId, canvasCompletion
Generate icon spritesheet POST /api/external/v1/editor/icon-spritesheets/generations referenceImageSrc, iconDescriptions referenceImageSrcs, model, aspectRatio, imageSize, projectId, assetFolderId, canvasCompletion
Extract assets from UI design POST /api/external/v1/editor/ui-designs/assets/extractions sourceImageSrc, aspectRatio, imageSize model, referenceImageSrcs, projectId, assetFolderId, spritesheetLabel, canvasCompletion
Generate character animation POST /api/external/v1/editor/character-animations/generations sourceLayerId, sourceImageSrc, sourceWidth, sourceHeight, promptText, resolution, ratio, frameCount, durationSeconds, model projectId, sourceResourceId, canvasCompletion; then create a library asset from the first returned frame
Generate video POST /api/external/v1/editor/videos/generations prompt, model, aspectRatio, durationSeconds, resolution, mode, sound referenceImageSrcs, referenceVideoSrcs, referenceAudioSrcs, webSearchEnabled, projectId, assetFolderId, assetLabel, canvasCompletion
Generate sound effect POST /api/external/v1/editor/audios/sound-effects/generations prompt, duration model, projectId, assetFolderId, assetLabel, canvasCompletion, generationInputs
Generate background music POST /api/external/v1/editor/audios/background-music/generations gptDescriptionPrompt, makeInstrumental projectId, assetFolderId, assetLabel, canvasCompletion, generationInputs

All generation requests should be placed into both the current canvas and its same-name asset-library folder. For endpoints that support assetLabel, pass it. For UI extraction, use spritesheetLabel. For icon spritesheet, the folder is enough. For character animation, the endpoint does not return asset; after success call POST /api/external/v1/editor/assets using the first returned frame as imageSrc, the session assetFolderId, and assetKind: "character-animation".

HTTP 2xx Warning Handling

Character image generation (including character redraw through kind: "character"), icon spritesheet generation, and UI asset extraction may return HTTP 2xx while carrying a structured warning; completion does not imply that all post-processed derivatives exist.

  • Consume the returned project and media snapshots as authoritative: character responses use resource / asset, while icon spritesheet and UI extraction responses use spritesheetResource / spritesheetAsset. warning.code: "postprocess-failed-source-preserved" means the saved provider source is the main result. Character output has no transparent derivative, while icon spritesheet and UI extraction have no transparent spritesheet and no slices. Display warning.reason directly; do not construct missing assets or retry the provider generation from scratch.
  • sliceWarning is only for a transparent spritesheet that was created successfully but could not be split automatically. Use the complete transparent spritesheet and preserve sliceWarning.reason as the original diagnostic; it is not a post-processing/source-preserved warning.
  • The service contract keeps warning and sliceWarning mutually exclusive. If a malformed response contains both, prioritize the general warning over sliceWarning defensively.

Reference Image Upload

If the user provides a local file as a reference image, run upload before the generation request:

  1. POST /api/external/v1/assets/direct-upload-tickets. Use legacyPrefix: "generated-character-drafts", pathSegments: ["editor", "external-editor-references"], original fileName, detected image contentType, and access: "private".
  2. Upload the file to the returned OSS form endpoint with all returned formFields.
  3. POST /api/external/v1/assets/objects/confirm with returned objectKey, detected contentType, contentLength if known, assetKind: "editor_reference_image", and accessPolicy: "private".
  4. Use the returned objectKey in the actual editor request.

OSS form upload uses upload.host and every non-null upload.formFields entry, then the file part named file. Default responses expose upload; envelope responses expose data.upload. Save the upload ticket response as ticket.json:

node - <<'NODE' ticket.json /path/to/reference.png
const fs = require('fs');
const path = require('path');

(async () => {
  const body = JSON.parse(fs.readFileSync(process.argv[2], 'utf8'));
  const ticket = body.upload || body.data?.upload;
  if (!ticket) throw new Error('Upload ticket response missing upload payload');
  const filePath = process.argv[3];
  const form = new FormData();
  for (const [key, value] of Object.entries(ticket.formFields)) {
    if (value != null) form.append(key, value);
  }
  const bytes = fs.readFileSync(filePath);
  form.append(
    'file',
    new Blob([bytes], { type: ticket.contentType || 'application/octet-stream' }),
    path.basename(filePath),
  );
  const response = await fetch(ticket.host, { method: 'POST', body: form });
  if (!response.ok) {
    throw new Error(`OSS upload failed: ${response.status} ${await response.text()}`);
  }
})().catch((error) => {
  console.error(error.message);
  process.exit(1);
});
NODE

Field mapping after upload:

Target API Put uploaded objectKey in
Image generation referenceImageSrcs
Image edit/redraw sourceImageSrc; additional references in referenceImageSrcs
Icon spritesheet referenceImageSrc; additional style refs in referenceImageSrcs
UI design extraction sourceImageSrc; additional refs in referenceImageSrcs
Character animation sourceImageSrc
Video generation with image references referenceImageSrcs

Do not put the signed read URL into generation fields. Signed URLs are for user-visible preview/download; generation fields should use the stable objectKey for uploaded private references.

Common Enums

  • Image kind: spec, character, quick-edit, ui-design, publication-material.
  • Image model: gpt-image-2, gemini-3.1-flash-image-preview, nanobanana2, nano-banana.
  • Image aspectRatio: 1:1, 2:3, 3:2, 9:16, 16:9.
  • Image imageSize: 0.5K, 1K, 2K.
  • Video model: seedance2.0, seedance2.0-fast, kling3.0, kling3.0-omni, veo3.1, veo3.1-fast.
  • Video aspectRatio: 16:9, 9:16, 1:1, 4:3, 3:4, 21:9.
  • Video resolution: 480p, 720p, 1080p.
  • Video mode: always std.
  • Video sound: on, off.
  • Character animation model: always seedance2.0-fast.
  • Character animation resolution: 480p, 720p; frameCount: 32, 40, 48; durationSeconds: 4, 5, 6; ratio: same, 1:1, 4:3, 16:9, 9:16, 3:4.

Local Reference Media Details

  • contentLength in object confirm is a JSON number from local byte size, not a string.
  • For character animation, use an existing project layer ID as sourceLayerId when available.
  • If the source is only an uploaded local image, derive sourceLayerId from the file name, such as external-reference-hero, and keep it stable across retries.
  • Read sourceWidth and sourceHeight from the local image. If dimensions cannot be read, ask instead of inventing dimensions.
  • UI design extraction uses fixed aspectRatio: "1:1"; choose imageSize: "1K" for normal/small extractions and 2K for dense designs.
  • Video image/video/audio references are supported only by the Seedance 2.0 family; default referenced-media video requests to model: "seedance2.0-fast", mode: "std", and explicit sound.
  • Image edit/redraw can pass targetLayerId with projectId to replace an existing canvas layer when no explicit canvasCompletion is supplied.
  • Image, edit, video, sound effect, and BGM generation can pass assetFolderId and assetLabel; response asset is the created/updated library record.
  • Icon spritesheet and UI extraction can pass assetFolderId; UI extraction can also pass spritesheetLabel.

Canvas Completion

Use canvasCompletion for generation in this skill so the generated result is written back into the project canvas by the backend.

Required:

{
  "title": "素材名称",
  "placeholder": {
    "x": 0,
    "y": 0,
    "width": 512,
    "height": 512,
    "originalWidth": 512,
    "originalHeight": 512
  }
}

dialogId is optional. If the response includes project, resource, or asset, use those snapshots instead of reconstructing canvas/resource/library state locally.

Upload Flow

For a local file that should become a project resource or library asset:

  1. POST /api/external/v1/assets/direct-upload-tickets with legacyPrefix, fileName, and optional contentType, access, maxSizeBytes.
  2. Submit the file to the returned OSS form endpoint with returned formFields.
  3. POST /api/external/v1/assets/objects/confirm with returned objectKey and an assetKind.
  4. Create a project resource or library asset with the confirmed assetObjectId/objectKey.

For reading private/generated assets, call GET /api/external/v1/assets/read-url?objectKey=... and use the returned signedUrl.