补齐外部编辑器生成资产链路
更新外部编辑器 API skill 的画板、素材库和美术规范引导。 扩展 Python helper,自动准备画板会话、长超时生成并补齐角色动画素材入库。 将新建项目和生成任务 ID 前缀调整为 proj/task,并保留旧 ID 兼容。 更新 OpenAPI 说明和项目记忆。
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: genarrative-external-editor-api
|
||||
description: Guide use of Genarrative's external editor/canvas OpenAPI. Use when a user describes a canvas/editor integration need and Codex must infer the right `/api/external/v1` API automatically, draft curl/HTTP/SDK requests, clarify missing generation/upload/project/asset fields, or set up and safely handle a Genarrative developer API Key.
|
||||
description: Guide use of Genarrative's external editor/canvas OpenAPI. Use when a user describes a canvas/editor integration need and Codex must infer the right `/api/external/v1` API automatically, prepare canvas and asset-library context, abstract reusable art specs before generating assets, upload references, draft curl/HTTP/SDK requests, or set up and safely handle a Genarrative developer API Key.
|
||||
---
|
||||
|
||||
# Genarrative External Editor API
|
||||
@@ -11,22 +11,25 @@ Prefer the bundled Python helper for runnable examples: `scripts/genarrative_ext
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Classify the user's natural-language intent first. Do not ask the user to choose an API:
|
||||
1. At the start of a new conversation, ask the user for the canvas name before the first generation call unless an existing session is already provided. Create or use a project with that name and an asset-library folder with the same name. Keep `canvasName`, `projectId`, `assetFolderId`, and the current art spec in conversation state.
|
||||
2. Before any art asset generation, abstract the user's request into a reusable art spec. Ask only for missing spec fields required by the selected asset type. If a current spec already exists and the user does not request a new style/spec, reuse it automatically.
|
||||
3. Classify the user's natural-language intent. Do not ask the user to choose an API:
|
||||
- "生成/生图/做一张图" -> image generation
|
||||
- "重绘/修改这张图" -> image edit
|
||||
- "用这张参考图/基于本地图生成" -> upload local reference image, then generation or edit
|
||||
- "上传本地素材" -> upload ticket, OSS form upload, object confirm
|
||||
- "保存画板/更新布局" -> canvas save
|
||||
- "读取私有素材" -> signed read URL
|
||||
2. Ask only for missing inputs that affect the request body or an actually ambiguous route:
|
||||
4. Ask only for missing inputs that affect the request body or an actually ambiguous route:
|
||||
- credentials JSON path only if the user cannot use the default local path
|
||||
- existing `projectId`, folder/resource IDs, and whether output should update the canvas
|
||||
- canvas name when no current canvas session exists; existing `projectId`, folder/resource IDs only when resuming a known project
|
||||
- media type, prompt, references, dimensions, model, ratio, duration, and resolution
|
||||
- whether referenced media is already uploaded as `objectKey` or still local
|
||||
3. If the user lacks an API Key, guide setup before request design.
|
||||
4. Read `references/api-selection.md` before finalizing any request. Use the core table below for fast routing, then verify details in the reference.
|
||||
5. Use `scripts/genarrative_external_api.py` when the user wants runnable Python, reference image upload, or a chain that should execute with fewer hand-written curl steps.
|
||||
6. Keep to `/api/external/v1` unless the user explicitly asks for internal profile/admin APIs.
|
||||
5. Every external generation must write to both the canvas and the asset library. Include `projectId`, `assetFolderId`, a display label, and `canvasCompletion` whenever the target endpoint supports them. For character animation, use the helper's two-step fallback: generate with `projectId` + `canvasCompletion`, then create a library asset from the first returned frame in the session folder.
|
||||
6. If the user lacks an API Key, guide setup before request design.
|
||||
7. Read `references/api-selection.md` before finalizing any request. Use the core table below for fast routing, then verify details in the reference.
|
||||
8. Use `scripts/genarrative_external_api.py` when the user wants runnable Python, reference image upload, canvas/folder session setup, art-spec carrying, or a chain that should execute with fewer hand-written curl steps.
|
||||
9. Keep to `/api/external/v1` unless the user explicitly asks for internal profile/admin APIs.
|
||||
|
||||
## Core Routes
|
||||
|
||||
@@ -45,6 +48,25 @@ Prefer the bundled Python helper for runnable examples: `scripts/genarrative_ext
|
||||
| Sound effect | `POST /api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` |
|
||||
| Background music | `POST /api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` |
|
||||
|
||||
## Art Spec Interface
|
||||
|
||||
Maintain one current art spec per conversation. A compact spec is enough:
|
||||
|
||||
```json
|
||||
{
|
||||
"assetType": "character | background | prop | ui | icon | animation | video | audio",
|
||||
"subject": "要生成的主体",
|
||||
"style": "画风/材质/时代/参考风格",
|
||||
"palette": "主色与禁用色",
|
||||
"composition": "构图、镜头、姿态或布局",
|
||||
"format": "比例、尺寸、分辨率、帧数、时长",
|
||||
"constraints": "必须保留/禁止出现/透明或绿幕要求",
|
||||
"references": ["objectKey 或本地路径说明"]
|
||||
}
|
||||
```
|
||||
|
||||
For a first spec, infer fields from the user's words and ask only for missing fields that block the selected API. Examples: character animation needs source image/layer, dimensions, motion, ratio, frame count, and duration; UI extraction needs source design image plus target density; icon spritesheet needs reference image and icon descriptions. After a spec exists, reuse it for later assets unless the user changes style, subject family, palette, format, or constraints.
|
||||
|
||||
## API Key
|
||||
|
||||
The external OpenAPI uses:
|
||||
@@ -99,7 +121,25 @@ For Python callers, prefer:
|
||||
from genarrative_external_api import GenarrativeExternalClient
|
||||
|
||||
client = GenarrativeExternalClient()
|
||||
project = client.create_project("新画板")
|
||||
session = client.prepare_canvas_session("新画板")
|
||||
art_spec = {
|
||||
"assetType": "background",
|
||||
"subject": "幻想森林主视觉",
|
||||
"style": "手绘游戏概念图",
|
||||
"palette": "翡翠绿、金色光斑,避免低饱和灰",
|
||||
"composition": "16:9 横版,中心留出角色站位",
|
||||
"format": "16:9, 1K",
|
||||
"constraints": "无文字、无 UI 按钮",
|
||||
"references": [],
|
||||
}
|
||||
client.generate_image(
|
||||
"生成幻想森林背景",
|
||||
canvasSession=session,
|
||||
assetLabel="森林背景",
|
||||
aspectRatio="16:9",
|
||||
imageSize="1K",
|
||||
artSpec=art_spec,
|
||||
)
|
||||
```
|
||||
|
||||
Use the helper directly from this skill path, or copy it into the caller's project. Do not change the fixed base URL or move the API Key into environment variables.
|
||||
@@ -122,7 +162,7 @@ curl -fsS "$api/api/external/v1/editor/projects" \
|
||||
-d '{"title":"新画板"}'
|
||||
```
|
||||
|
||||
Generate an image and save it into a project/canvas when the user supplies placement:
|
||||
Generate an image and save it into both the canvas and the asset-library folder:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -131,6 +171,14 @@ Generate an image and save it into a project/canvas when the user supplies place
|
||||
"aspectRatio": "16:9",
|
||||
"imageSize": "1K",
|
||||
"projectId": "<projectId>",
|
||||
"assetFolderId": "<assetFolderId>",
|
||||
"assetLabel": "森林背景",
|
||||
"generationInputs": {
|
||||
"artSpec": {
|
||||
"assetType": "background",
|
||||
"style": "手绘游戏概念图"
|
||||
}
|
||||
},
|
||||
"canvasCompletion": {
|
||||
"title": "森林背景",
|
||||
"placeholder": {
|
||||
@@ -147,6 +195,8 @@ Generate an image and save it into a project/canvas when the user supplies place
|
||||
|
||||
Then call `POST /api/external/v1/editor/images/generations`.
|
||||
|
||||
For direct HTTP/curl, create or find the folder first with `GET /api/external/v1/editor/assets/library` and `POST /api/external/v1/editor/assets/folders`. The folder label should match the canvas name.
|
||||
|
||||
## Reference Images
|
||||
|
||||
When the user provides a local reference image path/file, upload it first; do not ask the user to convert it to base64.
|
||||
@@ -157,9 +207,12 @@ Python helper path:
|
||||
from genarrative_external_api import GenarrativeExternalClient
|
||||
|
||||
client = GenarrativeExternalClient()
|
||||
session = client.prepare_canvas_session("新画板")
|
||||
ref = client.upload_reference_image("/path/to/reference.png")
|
||||
client.generate_image(
|
||||
"基于参考图生成一张 16:9 游戏背景",
|
||||
canvasSession=session,
|
||||
assetLabel="参考图背景",
|
||||
aspectRatio="16:9",
|
||||
imageSize="1K",
|
||||
referenceImageSrcs=[ref["objectKey"]],
|
||||
@@ -255,6 +308,10 @@ For character animation from an uploaded local image, set:
|
||||
|
||||
Use an existing canvas layer ID when the image came from a project layer. If it came only from a local upload, derive a stable synthetic `sourceLayerId` from the file name, for example `external-reference-hero`. Read `sourceWidth` and `sourceHeight` from the actual image before upload; ask the user only if the dimensions cannot be determined.
|
||||
|
||||
The helper uses a 420 second timeout for generation calls, including character animation and video. Direct HTTP clients should not use a 70 second request timeout for animation.
|
||||
|
||||
Character animation currently returns canvas completion data but not a direct `asset` payload. To keep the "canvas + asset library" invariant, call `client.animate_character(..., canvasSession=session, canvasTitle="...")`; the helper creates a library asset from the first returned frame in the session folder after the animation call succeeds.
|
||||
|
||||
For video generation, always include `mode: "std"`. When using image/video/audio references, default to `model: "seedance2.0-fast"` unless the user asks for another listed model, because reference media support is limited to the Seedance 2.0 family.
|
||||
|
||||
For image edit/redraw that should replace an existing canvas layer, pass `projectId` and `targetLayerId`. If the user instead gives an explicit `canvasCompletion`, let that placement win.
|
||||
@@ -264,6 +321,7 @@ For sound effects and BGM, `assetFolderId` and `assetLabel` can write the genera
|
||||
## Guardrails
|
||||
|
||||
- Do not invent endpoints outside the OpenAPI, especially internal worker or runtime task-list routes.
|
||||
- Do not omit canvas/library context for generation. New generated assets should enter both the canvas and the asset-library folder named after the canvas.
|
||||
- Do not put API Keys in repository files, generated project files, command history snippets with literal secrets, logs, docs, commits, or screenshots. The only default storage is the user's local private JSON credentials file.
|
||||
- Do not use account JWT endpoints as the default external integration path. The profile API can create/revoke keys for logged-in product users, but it is not part of the external editor OpenAPI.
|
||||
- When an endpoint returns `project`, `resource`, or `asset`, treat those as the authoritative updated project/resource/asset snapshots.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
interface:
|
||||
display_name: "Genarrative External Editor API"
|
||||
short_description: "Auto-route external canvas API usage"
|
||||
default_prompt: "Use $genarrative-external-editor-api to infer the right external canvas API and draft a request."
|
||||
short_description: "Auto-route canvas API generation"
|
||||
default_prompt: "Use $genarrative-external-editor-api to prepare a canvas session, infer the right API, and generate assets into the canvas and library."
|
||||
policy:
|
||||
allow_implicit_invocation: true
|
||||
|
||||
@@ -8,6 +8,17 @@ Source of truth: `docs/openapi/genarrative-external-v1.openapi.json`.
|
||||
- Public contract: `GET /api/external/v1/openapi.json`.
|
||||
- Authenticated calls: `Authorization: Bearer <tnr_sk_...>`.
|
||||
- Default credentials file: `~/.config/genarrative/external-editor-api.json` with an `apiKey` string.
|
||||
- Generation clients should allow long-running responses. Use at least 420 seconds for character animation and video; 70 seconds is too short for animation.
|
||||
|
||||
## Canvas Session and Art Spec
|
||||
|
||||
At the start of a new conversation, ask for a canvas name before the first generation call unless the user already supplied `projectId` and `assetFolderId`. Create or reuse:
|
||||
|
||||
1. `POST /api/external/v1/editor/projects` with `title` = canvas name.
|
||||
2. `GET /api/external/v1/editor/assets/library`; if no folder has the same label, `POST /api/external/v1/editor/assets/folders` with `label` = canvas name.
|
||||
3. Keep `canvasName`, `projectId`, `assetFolderId`, and the current art spec in conversation state.
|
||||
|
||||
Before generating art assets, normalize the user's request into a current art spec with `assetType`, `subject`, `style`, `palette`, `composition`, `format`, `constraints`, and `references`. Ask follow-up questions only for missing fields that block the selected endpoint. Reuse the current spec automatically when the user asks for another asset without changing style/spec requirements. Put the spec in `generationInputs.artSpec` and summarize it in the prompt when useful.
|
||||
|
||||
## Intent Routing
|
||||
|
||||
@@ -60,11 +71,13 @@ Ask a follow-up only when two routes could both be correct and produce different
|
||||
| Edit/redraw image | `POST /api/external/v1/editor/images/edits` | `prompt`, `sourceImageSrc` | `referenceImageSrcs`, `model`, `size`, `projectId`, `assetFolderId`, `assetLabel`, `sourceResourceId`, `targetLayerId`, `canvasCompletion` |
|
||||
| Generate icon spritesheet | `POST /api/external/v1/editor/icon-spritesheets/generations` | `referenceImageSrc`, `iconDescriptions` | `referenceImageSrcs`, `model`, `aspectRatio`, `imageSize`, `projectId`, `assetFolderId`, `canvasCompletion` |
|
||||
| Extract assets from UI design | `POST /api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize` | `model`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `spritesheetLabel`, `canvasCompletion` |
|
||||
| Generate character animation | `POST /api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `canvasCompletion` |
|
||||
| Generate character animation | `POST /api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `canvasCompletion`; then create a library asset from the first returned frame |
|
||||
| Generate video | `POST /api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | `referenceImageSrcs`, `referenceVideoSrcs`, `referenceAudioSrcs`, `webSearchEnabled`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` |
|
||||
| Generate sound effect | `POST /api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` | `model`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` |
|
||||
| Generate background music | `POST /api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` | `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` |
|
||||
|
||||
All generation requests should be placed into both the current canvas and its same-name asset-library folder. For endpoints that support `assetLabel`, pass it. For UI extraction, use `spritesheetLabel`. For icon spritesheet, the folder is enough. For character animation, the endpoint does not return `asset`; after success call `POST /api/external/v1/editor/assets` using the first returned frame as `imageSrc`, the session `assetFolderId`, and `assetKind: "character-animation"`.
|
||||
|
||||
## Reference Image Upload
|
||||
|
||||
If the user provides a local file as a reference image, run upload before the generation request:
|
||||
@@ -144,11 +157,12 @@ Do not put the signed read URL into generation fields. Signed URLs are for user-
|
||||
- UI design extraction uses fixed `aspectRatio: "1:1"`; choose `imageSize: "1K"` for normal/small extractions and `2K` for dense designs.
|
||||
- Video image/video/audio references are supported only by the Seedance 2.0 family; default referenced-media video requests to `model: "seedance2.0-fast"`, `mode: "std"`, and explicit `sound`.
|
||||
- Image edit/redraw can pass `targetLayerId` with `projectId` to replace an existing canvas layer when no explicit `canvasCompletion` is supplied.
|
||||
- Image, video, sound effect, and BGM generation can pass `assetFolderId` and `assetLabel`; response `asset` is the created/updated library record.
|
||||
- Image, edit, video, sound effect, and BGM generation can pass `assetFolderId` and `assetLabel`; response `asset` is the created/updated library record.
|
||||
- Icon spritesheet and UI extraction can pass `assetFolderId`; UI extraction can also pass `spritesheetLabel`.
|
||||
|
||||
## Canvas Completion
|
||||
|
||||
Use `canvasCompletion` only when the generated result should be written back into a project canvas by the backend.
|
||||
Use `canvasCompletion` for generation in this skill so the generated result is written back into the project canvas by the backend.
|
||||
|
||||
Required:
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ from typing import Any
|
||||
|
||||
BASE_URL = "https://www.genarrative.world/"
|
||||
DEFAULT_CREDENTIALS_FILE = Path.home() / ".config/genarrative/external-editor-api.json"
|
||||
DEFAULT_REQUEST_TIMEOUT_SECONDS = 60
|
||||
GENERATION_REQUEST_TIMEOUT_SECONDS = 420
|
||||
|
||||
|
||||
class GenarrativeApiError(RuntimeError):
|
||||
@@ -58,6 +60,20 @@ def source_layer_id_from_path(file_path: str | os.PathLike[str]) -> str:
|
||||
return f"external-reference-{slug}"
|
||||
|
||||
|
||||
def normalize_optional_text(value: Any) -> str | None:
|
||||
if not isinstance(value, str):
|
||||
return None
|
||||
stripped = value.strip()
|
||||
return stripped or None
|
||||
|
||||
|
||||
def art_spec_prompt(prompt: str, art_spec: dict[str, Any] | None) -> str:
|
||||
if not art_spec:
|
||||
return prompt
|
||||
spec_json = json.dumps(art_spec, ensure_ascii=False, sort_keys=True)
|
||||
return f"{prompt}\n\n美术规范(JSON): {spec_json}"
|
||||
|
||||
|
||||
def image_dimensions(file_path: str | os.PathLike[str]) -> tuple[int, int] | None:
|
||||
path = Path(file_path)
|
||||
with path.open("rb") as fh:
|
||||
@@ -108,7 +124,7 @@ class GenarrativeExternalClient:
|
||||
body: dict[str, Any] | None = None,
|
||||
query: dict[str, Any] | None = None,
|
||||
auth: bool = True,
|
||||
timeout: int = 60,
|
||||
timeout: int = DEFAULT_REQUEST_TIMEOUT_SECONDS,
|
||||
) -> Any:
|
||||
url = f"{self.base_url}{path}"
|
||||
if query:
|
||||
@@ -140,6 +156,120 @@ class GenarrativeExternalClient:
|
||||
body = {} if title is None else {"title": title}
|
||||
return self.request_json("POST", "/api/external/v1/editor/projects", body)
|
||||
|
||||
def list_asset_library(self) -> Any:
|
||||
return self.request_json("GET", "/api/external/v1/editor/assets/library")
|
||||
|
||||
def create_asset_folder(self, label: str, sort_order: int = 100) -> Any:
|
||||
return self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/assets/folders",
|
||||
{"label": label, "sortOrder": sort_order},
|
||||
)
|
||||
|
||||
def ensure_asset_folder(self, label: str, sort_order: int = 100) -> dict[str, Any]:
|
||||
normalized_label = normalize_optional_text(label) or "新画板"
|
||||
library = self.list_asset_library()
|
||||
folders = unwrap_envelope(library).get("library", {}).get("folders", [])
|
||||
if isinstance(folders, list):
|
||||
for folder in folders:
|
||||
if isinstance(folder, dict) and normalize_optional_text(folder.get("label")) == normalized_label:
|
||||
return folder
|
||||
created = self.create_asset_folder(normalized_label, sort_order=sort_order)
|
||||
folder = unwrap_envelope(created).get("folder")
|
||||
if not isinstance(folder, dict):
|
||||
raise GenarrativeApiError("Create asset folder response missing folder payload.")
|
||||
return folder
|
||||
|
||||
def create_asset(
|
||||
self,
|
||||
folder_id: str,
|
||||
label: str,
|
||||
image_src: str,
|
||||
width: int,
|
||||
height: int,
|
||||
**fields: Any,
|
||||
) -> Any:
|
||||
body = {
|
||||
"folderId": folder_id,
|
||||
"label": label,
|
||||
"imageSrc": image_src,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"sourceType": fields.pop("sourceType", "generated"),
|
||||
**fields,
|
||||
}
|
||||
return self.request_json("POST", "/api/external/v1/editor/assets", body)
|
||||
|
||||
def prepare_canvas_session(self, canvas_name: str) -> dict[str, Any]:
|
||||
normalized_name = normalize_optional_text(canvas_name) or "新画板"
|
||||
project = unwrap_envelope(self.create_project(normalized_name)).get("project")
|
||||
if not isinstance(project, dict) or not normalize_optional_text(project.get("projectId")):
|
||||
raise GenarrativeApiError("Create project response missing projectId.")
|
||||
folder = self.ensure_asset_folder(normalized_name)
|
||||
folder_id = normalize_optional_text(folder.get("folderId"))
|
||||
if not folder_id:
|
||||
raise GenarrativeApiError("Asset folder payload missing folderId.")
|
||||
return {
|
||||
"canvasName": normalized_name,
|
||||
"projectId": project["projectId"],
|
||||
"assetFolderId": folder_id,
|
||||
"project": project,
|
||||
"folder": folder,
|
||||
}
|
||||
|
||||
def build_canvas_completion(
|
||||
self,
|
||||
title: str,
|
||||
width: int,
|
||||
height: int,
|
||||
x: float = 0,
|
||||
y: float = 0,
|
||||
dialog_id: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
payload: dict[str, Any] = {
|
||||
"title": normalize_optional_text(title) or "生成素材",
|
||||
"placeholder": {
|
||||
"x": x,
|
||||
"y": y,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"originalWidth": width,
|
||||
"originalHeight": height,
|
||||
},
|
||||
}
|
||||
if normalize_optional_text(dialog_id):
|
||||
payload["dialogId"] = dialog_id
|
||||
return payload
|
||||
|
||||
def canvas_generation_fields(
|
||||
self,
|
||||
session: dict[str, Any],
|
||||
asset_label: str,
|
||||
width: int = 1024,
|
||||
height: int = 1024,
|
||||
x: float = 0,
|
||||
y: float = 0,
|
||||
dialog_id: str | None = None,
|
||||
asset_label_field: str | None = "assetLabel",
|
||||
) -> dict[str, Any]:
|
||||
fields: dict[str, Any] = {
|
||||
"projectId": session["projectId"],
|
||||
"canvasCompletion": self.build_canvas_completion(
|
||||
asset_label,
|
||||
width=width,
|
||||
height=height,
|
||||
x=x,
|
||||
y=y,
|
||||
dialog_id=dialog_id,
|
||||
),
|
||||
}
|
||||
folder_id = normalize_optional_text(session.get("assetFolderId"))
|
||||
if folder_id:
|
||||
fields["assetFolderId"] = folder_id
|
||||
if asset_label_field:
|
||||
fields[asset_label_field] = normalize_optional_text(asset_label) or "生成素材"
|
||||
return fields
|
||||
|
||||
def save_canvas(self, project_id: str, viewport: dict[str, Any], layers: dict[str, Any]) -> Any:
|
||||
return self.request_json(
|
||||
"PATCH",
|
||||
@@ -147,6 +277,62 @@ class GenarrativeExternalClient:
|
||||
{"viewport": viewport, "layers": layers},
|
||||
)
|
||||
|
||||
def _apply_art_spec(self, fields: dict[str, Any], prompt: str) -> str:
|
||||
art_spec = fields.pop("artSpec", None)
|
||||
if art_spec is None:
|
||||
art_spec = fields.pop("art_spec", None)
|
||||
if not isinstance(art_spec, dict):
|
||||
return prompt
|
||||
generation_inputs = fields.get("generationInputs")
|
||||
if not isinstance(generation_inputs, dict):
|
||||
generation_inputs = {}
|
||||
generation_inputs.setdefault("artSpec", art_spec)
|
||||
fields["generationInputs"] = generation_inputs
|
||||
return art_spec_prompt(prompt, art_spec)
|
||||
|
||||
def _apply_canvas_session_fields(
|
||||
self,
|
||||
fields: dict[str, Any],
|
||||
default_label: str,
|
||||
default_width: int,
|
||||
default_height: int,
|
||||
asset_label_field: str | None = "assetLabel",
|
||||
) -> tuple[dict[str, Any] | None, str]:
|
||||
session = fields.pop("canvasSession", None)
|
||||
if session is None:
|
||||
session = fields.pop("canvas_session", None)
|
||||
if isinstance(session, str):
|
||||
session = self.prepare_canvas_session(session)
|
||||
label = (
|
||||
normalize_optional_text(fields.get(asset_label_field)) if asset_label_field else None
|
||||
) or normalize_optional_text(fields.pop("canvasTitle", None)) or normalize_optional_text(default_label) or "生成素材"
|
||||
if asset_label_field and not normalize_optional_text(fields.get(asset_label_field)):
|
||||
fields.pop(asset_label_field, None)
|
||||
if not isinstance(session, dict):
|
||||
return None, label
|
||||
width = int(fields.pop("canvasWidth", default_width))
|
||||
height = int(fields.pop("canvasHeight", default_height))
|
||||
x = float(fields.pop("canvasX", 0))
|
||||
y = float(fields.pop("canvasY", 0))
|
||||
dialog_id = normalize_optional_text(fields.pop("dialogId", None))
|
||||
fields.update(
|
||||
{
|
||||
key: value
|
||||
for key, value in self.canvas_generation_fields(
|
||||
session,
|
||||
label,
|
||||
width=width,
|
||||
height=height,
|
||||
x=x,
|
||||
y=y,
|
||||
dialog_id=dialog_id,
|
||||
asset_label_field=asset_label_field,
|
||||
).items()
|
||||
if key not in fields or fields[key] is None
|
||||
}
|
||||
)
|
||||
return session, label
|
||||
|
||||
def create_upload_ticket(self, file_path: str | os.PathLike[str], access: str = "private") -> Any:
|
||||
path = Path(file_path)
|
||||
return self.request_json(
|
||||
@@ -245,21 +431,33 @@ class GenarrativeExternalClient:
|
||||
return self.request_json("GET", "/api/external/v1/assets/read-url", query={"objectKey": object_key})
|
||||
|
||||
def generate_image(self, prompt: str, **fields: Any) -> Any:
|
||||
return self.request_json("POST", "/api/external/v1/editor/images/generations", {"prompt": prompt, **fields})
|
||||
self._apply_canvas_session_fields(fields, prompt, 1024, 1024)
|
||||
prompt = self._apply_art_spec(fields, prompt)
|
||||
return self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/images/generations",
|
||||
{"prompt": prompt, **fields},
|
||||
timeout=GENERATION_REQUEST_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
def edit_image(self, prompt: str, source_image_src: str, **fields: Any) -> Any:
|
||||
self._apply_canvas_session_fields(fields, prompt, 1024, 1024)
|
||||
prompt = self._apply_art_spec(fields, prompt)
|
||||
return self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/images/edits",
|
||||
{"prompt": prompt, "sourceImageSrc": source_image_src, **fields},
|
||||
timeout=GENERATION_REQUEST_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
def extract_ui_assets(self, source_image_src: str, image_size: str = "1K", **fields: Any) -> Any:
|
||||
fields.pop("aspectRatio", None)
|
||||
self._apply_canvas_session_fields(fields, fields.get("spritesheetLabel", "UI 素材拆分"), 1024, 1024, "spritesheetLabel")
|
||||
return self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/ui-designs/assets/extractions",
|
||||
{"sourceImageSrc": source_image_src, "imageSize": image_size, **fields, "aspectRatio": "1:1"},
|
||||
timeout=GENERATION_REQUEST_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
def animate_character(
|
||||
@@ -271,6 +469,16 @@ class GenarrativeExternalClient:
|
||||
source_layer_id: str,
|
||||
**fields: Any,
|
||||
) -> Any:
|
||||
session, asset_label = self._apply_canvas_session_fields(
|
||||
fields,
|
||||
fields.get("canvasTitle", "角色动画"),
|
||||
source_width,
|
||||
source_height,
|
||||
asset_label_field=None,
|
||||
)
|
||||
prompt_text = self._apply_art_spec(fields, prompt_text)
|
||||
fields.pop("assetFolderId", None)
|
||||
fields.pop("assetLabel", None)
|
||||
body = {
|
||||
"sourceLayerId": source_layer_id,
|
||||
"sourceImageSrc": source_image_src,
|
||||
@@ -284,10 +492,40 @@ class GenarrativeExternalClient:
|
||||
**fields,
|
||||
"model": "seedance2.0-fast",
|
||||
}
|
||||
return self.request_json("POST", "/api/external/v1/editor/character-animations/generations", body)
|
||||
result = self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/character-animations/generations",
|
||||
body,
|
||||
timeout=GENERATION_REQUEST_TIMEOUT_SECONDS,
|
||||
)
|
||||
if isinstance(session, dict) and isinstance(result, dict) and not result.get("asset"):
|
||||
frames = result.get("frames")
|
||||
first_frame = frames[0] if isinstance(frames, list) and frames else None
|
||||
folder_id = normalize_optional_text(session.get("assetFolderId"))
|
||||
if isinstance(first_frame, dict) and folder_id:
|
||||
asset = self.create_asset(
|
||||
folder_id,
|
||||
asset_label,
|
||||
first_frame["imageSrc"],
|
||||
int(first_frame["width"]),
|
||||
int(first_frame["height"]),
|
||||
prompt=result.get("prompt"),
|
||||
model=result.get("model"),
|
||||
provider="ark",
|
||||
taskId=result.get("taskId"),
|
||||
assetKind="character-animation",
|
||||
generationInputs={
|
||||
"frames": frames,
|
||||
"previewVideoPath": result.get("previewVideoPath"),
|
||||
},
|
||||
)
|
||||
result["asset"] = unwrap_envelope(asset).get("asset")
|
||||
return result
|
||||
|
||||
def generate_video(self, prompt: str, **fields: Any) -> Any:
|
||||
fields.pop("mode", None)
|
||||
self._apply_canvas_session_fields(fields, prompt, 1280, 720)
|
||||
prompt = self._apply_art_spec(fields, prompt)
|
||||
body = {
|
||||
"prompt": prompt,
|
||||
"model": fields.pop("model", "seedance2.0-fast"),
|
||||
@@ -298,20 +536,31 @@ class GenarrativeExternalClient:
|
||||
**fields,
|
||||
"mode": "std",
|
||||
}
|
||||
return self.request_json("POST", "/api/external/v1/editor/videos/generations", body)
|
||||
return self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/videos/generations",
|
||||
body,
|
||||
timeout=GENERATION_REQUEST_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
def generate_sound_effect(self, prompt: str, duration: int, **fields: Any) -> Any:
|
||||
self._apply_canvas_session_fields(fields, prompt, 360, 120)
|
||||
prompt = self._apply_art_spec(fields, prompt)
|
||||
return self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/audios/sound-effects/generations",
|
||||
{"prompt": prompt, "duration": duration, **fields},
|
||||
timeout=GENERATION_REQUEST_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
def generate_background_music(self, description: str, **fields: Any) -> Any:
|
||||
self._apply_canvas_session_fields(fields, description, 360, 120)
|
||||
description = self._apply_art_spec(fields, description)
|
||||
return self.request_json(
|
||||
"POST",
|
||||
"/api/external/v1/editor/audios/background-music/generations",
|
||||
{"gptDescriptionPrompt": description, **fields, "makeInstrumental": True},
|
||||
timeout=GENERATION_REQUEST_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
|
||||
@@ -326,6 +575,50 @@ def _self_test() -> None:
|
||||
assert image_dimensions(fh.name) == (2, 3)
|
||||
assert source_layer_id_from_path(fh.name).startswith("external-reference-")
|
||||
assert unwrap_envelope({"ok": True, "data": {"upload": 1}}) == {"upload": 1}
|
||||
client = GenarrativeExternalClient(api_key="test")
|
||||
session = {"projectId": "proj-demo", "assetFolderId": "editor-asset-folder-demo"}
|
||||
fields = client.canvas_generation_fields(session, "英雄角色", width=512, height=768)
|
||||
assert fields["projectId"] == "proj-demo"
|
||||
assert fields["assetFolderId"] == "editor-asset-folder-demo"
|
||||
assert fields["assetLabel"] == "英雄角色"
|
||||
assert fields["canvasCompletion"]["placeholder"]["height"] == 768
|
||||
assert "美术规范" in art_spec_prompt("生成角色", {"style": "水彩"})
|
||||
calls: list[dict[str, Any]] = []
|
||||
|
||||
def fake_request_json(
|
||||
method: str,
|
||||
path: str,
|
||||
body: dict[str, Any] | None = None,
|
||||
query: dict[str, Any] | None = None,
|
||||
auth: bool = True,
|
||||
timeout: int = DEFAULT_REQUEST_TIMEOUT_SECONDS,
|
||||
) -> Any:
|
||||
calls.append({"method": method, "path": path, "body": body, "timeout": timeout})
|
||||
if path == "/api/external/v1/editor/assets":
|
||||
return {"asset": {"assetId": "editor-asset-demo"}}
|
||||
return {
|
||||
"taskId": "task-demo",
|
||||
"model": "seedance2.0-fast",
|
||||
"prompt": "角色呼吸",
|
||||
"previewVideoPath": "/generated/preview.mp4",
|
||||
"frames": [{"frameIndex": 1, "imageSrc": "/generated/frame01.png", "width": 512, "height": 768}],
|
||||
}
|
||||
|
||||
client.request_json = fake_request_json # type: ignore[method-assign]
|
||||
result = client.animate_character(
|
||||
"/generated/source.png",
|
||||
512,
|
||||
768,
|
||||
"角色呼吸",
|
||||
"layer-hero",
|
||||
canvasSession=session,
|
||||
canvasTitle="角色呼吸动画",
|
||||
)
|
||||
assert calls[0]["timeout"] == GENERATION_REQUEST_TIMEOUT_SECONDS
|
||||
assert calls[0]["body"]["projectId"] == "proj-demo"
|
||||
assert calls[0]["body"]["canvasCompletion"]["title"] == "角色呼吸动画"
|
||||
assert calls[1]["path"] == "/api/external/v1/editor/assets"
|
||||
assert result["asset"]["assetId"] == "editor-asset-demo"
|
||||
print("self-test ok")
|
||||
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"info": {
|
||||
"title": "陶泥儿外部编辑器 OpenAPI",
|
||||
"version": "1.0.0",
|
||||
"description": "外部系统调用陶泥儿图片画布项目、画布布局、素材库,以及图片、视频、音效、音乐等编辑器素材生成/编辑能力的 v1 契约。"
|
||||
"description": "外部系统调用陶泥儿图片画布项目、画布布局、素材库,以及图片、视频、音效、音乐等编辑器素材生成/编辑能力的 v1 契约。新建 projectId 使用 proj- 前缀,新建 taskId / operationId 使用 task- 前缀;历史 editor-project-*、aitask_*、extgen-* ID 仍可作为既有资源标识传入。"
|
||||
},
|
||||
"servers": [
|
||||
{
|
||||
|
||||
@@ -16,6 +16,22 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-03 外部编辑器 API 生成默认写入画布与素材库
|
||||
|
||||
- 背景:外部 API 面向美术 Agent 使用时,需要从自然语言自动选路,并保证生成结果不会只停留在接口回包里;同时后续素材生成需要复用已抽象出的美术规范,避免每次重新追问风格要求。
|
||||
- 决策:外部编辑器 API skill 在新对话首个生成前先确认画布名称,并创建 / 复用同名画布项目和素材库文件夹。所有外部生成请求默认携带 `projectId`、`assetFolderId`、素材展示名和 `canvasCompletion`,使结果进入画布和素材库;角色动画端点当前不直接返回 `asset`,由 helper 在动画成功后用首帧补建素材库记录。skill 先把用户需求抽象为可复用美术规范,缺少目标素材必需信息时再追问;已有规范且用户未提出新规范时自动复用。
|
||||
- 影响范围:`.codex/skills/genarrative-external-editor-api`、外部 OpenAPI 使用说明、外部画布生成集成方。
|
||||
- 验证方式:运行 skill 校验、helper 自测、Python 编译检查、编码检查和 `git diff --check`;真实线上生成 smoke 需要本机 `~/.config/genarrative/external-editor-api.json` 中有有效 API Key。
|
||||
- 关联文档:`docs/openapi/genarrative-external-v1.openapi.json`、`.codex/skills/genarrative-external-editor-api/SKILL.md`。
|
||||
|
||||
## 2026-07-03 新建项目与 AI 任务 ID 使用短前缀
|
||||
|
||||
- 背景:新建外部画布项目和 AI 任务 ID 需要统一以 `proj`、`task` 开头,同时保留旧 ID 兼容读取和路由。
|
||||
- 决策:新建 editor project ID 前缀改为 `proj-`,新建 AI task / 生成任务 / 草稿任务 ID 前缀改为 `task-`。路由和读写仍按字符串处理,不新增拒绝 `editor-project-*`、`aitask_*` 或 `extgen-*` 的校验,历史数据继续兼容。
|
||||
- 影响范围:`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/module-ai/src/domain/ids.rs`、`server-rs/crates/api-server/src/editor_generation_queue.rs`、玩法外部生成入队路径、外部编辑器 API 新建项目返回值、AI 任务创建链路。
|
||||
- 验证方式:运行 `cargo test -p module-ai --manifest-path server-rs/Cargo.toml`、定向 api-server editor project 测试、编码检查和 `git diff --check`。
|
||||
- 关联文档:`docs/openapi/genarrative-external-v1.openapi.json`。
|
||||
|
||||
## 2026-07-01 认证工作集只经 typed projection 同步正式表
|
||||
|
||||
- 背景:同手机号重复账号、兑换码白名单错配和微信资料不回写暴露出 `module-auth` 内存工作集、`auth_store_snapshot` 和正式认证表之间仍有历史互刷路径;旧 JSON 快照会把过期手机号索引或用户资料重新带回运行态。
|
||||
|
||||
@@ -33,7 +33,7 @@ impl AiGenerationDraftContext {
|
||||
Self {
|
||||
// 生成过程草稿使用稳定 task_id,保证同一模板会话操作重试时能继续定位已有内容。
|
||||
task_id: format!(
|
||||
"aitask_draft_{normalized_template}_{normalized_session}_{normalized_operation}"
|
||||
"task-draft-{normalized_template}-{normalized_session}-{normalized_operation}"
|
||||
),
|
||||
owner_user_id: owner_user_id.trim().to_string(),
|
||||
request_label: request_label.trim().to_string(),
|
||||
|
||||
@@ -45,7 +45,7 @@ pub(crate) async fn enqueue_editor_generation_job<T>(
|
||||
where
|
||||
T: Serialize,
|
||||
{
|
||||
let job_id = build_prefixed_uuid_id("extgen-");
|
||||
let job_id = build_prefixed_uuid_id("task-");
|
||||
let request_payload_json = serde_json::to_string(payload).map_err(|error| {
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
|
||||
"provider": EDITOR_GENERATION_QUEUE_PROVIDER,
|
||||
|
||||
@@ -72,7 +72,7 @@ use crate::{
|
||||
|
||||
const EDITOR_SHOWCASE_RESOURCE_PAGE_SIZE: usize = 36;
|
||||
|
||||
pub(crate) const EDITOR_PROJECT_ID_PREFIX: &str = "editor-project-";
|
||||
pub(crate) const EDITOR_PROJECT_ID_PREFIX: &str = "proj-";
|
||||
pub(crate) const EDITOR_RESOURCE_ID_PREFIX: &str = "editor-resource-";
|
||||
pub(crate) const EDITOR_ASSET_FOLDER_ID_PREFIX: &str = "editor-asset-folder-";
|
||||
pub(crate) const EDITOR_ASSET_ID_PREFIX: &str = "editor-asset-";
|
||||
@@ -4963,6 +4963,11 @@ mod tests {
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_project_id_prefix_uses_proj_for_new_ids() {
|
||||
assert_eq!(EDITOR_PROJECT_ID_PREFIX, "proj-");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_record_media_src_prefers_object_key_over_legacy_inline_media() {
|
||||
assert_eq!(
|
||||
|
||||
@@ -300,7 +300,7 @@ async fn enqueue_jump_hop_compile_draft_job(
|
||||
owner_user_id: &str,
|
||||
payload: JumpHopActionRequest,
|
||||
) -> Result<ExternalGenerationJobRecord, Response> {
|
||||
let job_id = build_prefixed_uuid_id("extgen-");
|
||||
let job_id = build_prefixed_uuid_id("task-");
|
||||
let now_micros = current_utc_micros();
|
||||
let price_mud_points = resolve_jump_hop_generation_points_cost(state).await;
|
||||
let request_payload_json = serde_json::to_string(&JumpHopCompileDraftWorkerPayload {
|
||||
|
||||
@@ -764,7 +764,7 @@ pub async fn execute_puzzle_agent_action(
|
||||
Some(&request_context),
|
||||
PuzzleAgentActionResponse {
|
||||
operation: PuzzleAgentOperationResponse {
|
||||
operation_id: build_prefixed_uuid_id("extgen-inline-"),
|
||||
operation_id: build_prefixed_uuid_id("task-inline-"),
|
||||
operation_type: "compile_puzzle_draft".to_string(),
|
||||
status: "completed".to_string(),
|
||||
phase_label: "首关拼图草稿".to_string(),
|
||||
@@ -803,7 +803,7 @@ pub async fn execute_puzzle_agent_action(
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let external_generation_job_id = build_prefixed_uuid_id("extgen-");
|
||||
let external_generation_job_id = build_prefixed_uuid_id("task-");
|
||||
let job = state
|
||||
.spacetime_client()
|
||||
.enqueue_external_generation_job(ExternalGenerationJobEnqueueRecordInput {
|
||||
@@ -1002,7 +1002,7 @@ pub async fn execute_puzzle_agent_action(
|
||||
Some(&request_context),
|
||||
PuzzleAgentActionResponse {
|
||||
operation: PuzzleAgentOperationResponse {
|
||||
operation_id: build_prefixed_uuid_id("extgen-inline-"),
|
||||
operation_id: build_prefixed_uuid_id("task-inline-"),
|
||||
operation_type: "generate_puzzle_images".to_string(),
|
||||
status: "completed".to_string(),
|
||||
phase_label: "拼图图片生成".to_string(),
|
||||
@@ -1025,7 +1025,7 @@ pub async fn execute_puzzle_agent_action(
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let external_generation_job_id = build_prefixed_uuid_id("extgen-");
|
||||
let external_generation_job_id = build_prefixed_uuid_id("task-");
|
||||
let source_entity_id = target_level_id
|
||||
.as_deref()
|
||||
.map(|level_id| format!("{session_id}:{level_id}"))
|
||||
@@ -1145,7 +1145,7 @@ pub async fn execute_puzzle_agent_action(
|
||||
Some(&request_context),
|
||||
PuzzleAgentActionResponse {
|
||||
operation: PuzzleAgentOperationResponse {
|
||||
operation_id: build_prefixed_uuid_id("extgen-inline-"),
|
||||
operation_id: build_prefixed_uuid_id("task-inline-"),
|
||||
operation_type: "generate_puzzle_ui_background".to_string(),
|
||||
status: "completed".to_string(),
|
||||
phase_label: "UI 背景图生成".to_string(),
|
||||
@@ -1168,7 +1168,7 @@ pub async fn execute_puzzle_agent_action(
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let external_generation_job_id = build_prefixed_uuid_id("extgen-");
|
||||
let external_generation_job_id = build_prefixed_uuid_id("task-");
|
||||
let source_entity_id = target_level_id
|
||||
.as_deref()
|
||||
.map(|level_id| format!("{session_id}:{level_id}"))
|
||||
|
||||
@@ -267,7 +267,7 @@ async fn enqueue_puzzle_clear_compile_draft_job(
|
||||
author_display_name: &str,
|
||||
payload: PuzzleClearActionRequest,
|
||||
) -> Result<ExternalGenerationJobRecord, Response> {
|
||||
let job_id = build_prefixed_uuid_id("extgen-");
|
||||
let job_id = build_prefixed_uuid_id("task-");
|
||||
let now_micros = current_utc_micros();
|
||||
let request_payload_json = serde_json::to_string(&PuzzleClearCompileDraftWorkerPayload {
|
||||
session_id: session_id.to_string(),
|
||||
|
||||
@@ -285,7 +285,7 @@ async fn enqueue_wooden_fish_generate_image_assets_job(
|
||||
author_display_name: &str,
|
||||
payload: WoodenFishActionRequest,
|
||||
) -> Result<ExternalGenerationJobRecord, Response> {
|
||||
let job_id = build_prefixed_uuid_id("extgen-");
|
||||
let job_id = build_prefixed_uuid_id("task-");
|
||||
let now_micros = current_utc_micros();
|
||||
let price_mud_points = resolve_wooden_fish_generation_points_cost(state).await;
|
||||
let request_payload_json = serde_json::to_string(&WoodenFishGenerateImageAssetsWorkerPayload {
|
||||
|
||||
@@ -5,7 +5,7 @@ use shared_kernel::{
|
||||
|
||||
use super::types::AiTaskStageKind;
|
||||
|
||||
pub const AI_TASK_ID_PREFIX: &str = "aitask_";
|
||||
pub const AI_TASK_ID_PREFIX: &str = "task-";
|
||||
pub const AI_TASK_STAGE_ID_PREFIX: &str = "aistage_";
|
||||
pub const AI_RESULT_REF_ID_PREFIX: &str = "aires_";
|
||||
pub const AI_TEXT_CHUNK_ID_PREFIX: &str = "aichunk_";
|
||||
|
||||
@@ -50,6 +50,11 @@ fn generate_ai_task_stage_id_contains_task_and_stage_slug() {
|
||||
assert_eq!(stage_id, "aistage_aitask_demo_normalize_result");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generate_ai_task_id_uses_task_prefix_for_new_ids() {
|
||||
assert!(generate_ai_task_id(1_713_680_000_000_000).starts_with("task-"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn create_and_start_task_updates_status() {
|
||||
let service = build_service();
|
||||
|
||||
Reference in New Issue
Block a user