diff --git a/.codex/skills/genarrative-external-editor-api/SKILL.md b/.codex/skills/genarrative-external-editor-api/SKILL.md new file mode 100644 index 000000000..498545105 --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/SKILL.md @@ -0,0 +1,274 @@ +--- +name: genarrative-external-editor-api +description: Guide use of Genarrative's external editor/canvas OpenAPI. Use when a user describes a canvas/editor integration need and Codex must infer the right `/api/external/v1` API automatically, draft curl/HTTP/SDK requests, clarify missing generation/upload/project/asset fields, or set up and safely handle a Genarrative developer API Key. +--- + +# Genarrative External Editor API + +Use the live OpenAPI contract as the source of truth: `GET https://www.genarrative.world/api/external/v1/openapi.json`. In this repository, the same contract is `docs/openapi/genarrative-external-v1.openapi.json`. If exact fields or enums matter, read the contract before emitting final code. + +Prefer the bundled Python helper for runnable examples: `scripts/genarrative_external_api.py`. It uses only Python stdlib, reads the local JSON API Key file, fixes the production base URL, and wraps upload/confirm/generation routes. + +## Workflow + +1. Classify the user's natural-language intent first. Do not ask the user to choose an API: + - "生成/生图/做一张图" -> image generation + - "重绘/修改这张图" -> image edit + - "用这张参考图/基于本地图生成" -> upload local reference image, then generation or edit + - "上传本地素材" -> upload ticket, OSS form upload, object confirm + - "保存画板/更新布局" -> canvas save + - "读取私有素材" -> signed read URL +2. Ask only for missing inputs that affect the request body or an actually ambiguous route: + - credentials JSON path only if the user cannot use the default local path + - existing `projectId`, folder/resource IDs, and whether output should update the canvas + - media type, prompt, references, dimensions, model, ratio, duration, and resolution + - whether referenced media is already uploaded as `objectKey` or still local +3. If the user lacks an API Key, guide setup before request design. +4. Read `references/api-selection.md` before finalizing any request. Use the core table below for fast routing, then verify details in the reference. +5. Use `scripts/genarrative_external_api.py` when the user wants runnable Python, reference image upload, or a chain that should execute with fewer hand-written curl steps. +6. Keep to `/api/external/v1` unless the user explicitly asks for internal profile/admin APIs. + +## Core Routes + +| Intent | Method and path | Required fields | +| --- | --- | --- | +| List/create projects | `GET/POST /api/external/v1/editor/projects` | create: optional `title` | +| Save canvas | `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | `viewport`, `layers` | +| Upload local media | `POST /api/external/v1/assets/direct-upload-tickets` -> OSS form -> `POST /api/external/v1/assets/objects/confirm` | ticket: `legacyPrefix`, `fileName`; confirm: `objectKey`, `assetKind` | +| Read private media | `GET /api/external/v1/assets/read-url` | `objectKey` or `legacyPublicPath` | +| Image generation | `POST /api/external/v1/editor/images/generations` | `prompt` | +| Image edit/redraw | `POST /api/external/v1/editor/images/edits` | `prompt`, `sourceImageSrc` | +| Icon spritesheet | `POST /api/external/v1/editor/icon-spritesheets/generations` | `referenceImageSrc`, `iconDescriptions` | +| UI asset extraction | `POST /api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize`; use `assetFolderId` for library folder | +| Character animation | `POST /api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | +| Video generation | `POST /api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | +| Sound effect | `POST /api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` | +| Background music | `POST /api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` | + +## API Key + +The external OpenAPI uses: + +```text +Authorization: Bearer +``` + +The OpenAPI JSON endpoint is public; every other external endpoint requires the Bearer API Key. + +Use this fixed production base URL: + +```text +https://www.genarrative.world/ +``` + +Guide the user to create a key from the logged-in product UI under `开发者 API Key`. The raw key is shown only once; never ask the user to paste it into chat. Tell them to store it in this local private JSON file, outside the repository: + +```text +~/.config/genarrative/external-editor-api.json +``` + +```json +{ + "apiKey": "tnr_sk_..." +} +``` + +Set the file readable only by the current user where possible: `chmod 600 ~/.config/genarrative/external-editor-api.json`. Do not use environment variables for this API. + +Smoke test by reading the JSON file, without printing the key: + +```bash +api_key="$(node -e 'const fs=require("fs"); const p=process.argv[1]; const c=JSON.parse(fs.readFileSync(p,"utf8")); process.stdout.write(c.apiKey || "");' "$HOME/.config/genarrative/external-editor-api.json")" +curl -fsS "https://www.genarrative.world/api/external/v1/editor/projects" \ + -H "Authorization: Bearer $api_key" +``` + +For generated client code, read `apiKey` from the JSON file, fail with a clear missing-config error, and redact keys in logs. + +Python smoke without printing the key: + +```bash +python3 .codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py list-projects +``` + +## Request Patterns + +For Python callers, prefer: + +```python +from genarrative_external_api import GenarrativeExternalClient + +client = GenarrativeExternalClient() +project = client.create_project("新画板") +``` + +Use the helper directly from this skill path, or copy it into the caller's project. Do not change the fixed base URL or move the API Key into environment variables. + +Use this shared base: + +```bash +api="https://www.genarrative.world" +credentials_file="$HOME/.config/genarrative/external-editor-api.json" +api_key="$(node -e 'const fs=require("fs"); const p=process.argv[1]; const c=JSON.parse(fs.readFileSync(p,"utf8")); process.stdout.write(c.apiKey || "");' "$credentials_file")" +auth=(-H "Authorization: Bearer $api_key") +json=(-H "Content-Type: application/json") +``` + +Create a project: + +```bash +curl -fsS "$api/api/external/v1/editor/projects" \ + "${auth[@]}" "${json[@]}" \ + -d '{"title":"新画板"}' +``` + +Generate an image and save it into a project/canvas when the user supplies placement: + +```json +{ + "prompt": "一张横版幻想森林背景，适合游戏主视觉", + "kind": "spec", + "aspectRatio": "16:9", + "imageSize": "1K", + "projectId": "", + "canvasCompletion": { + "title": "森林背景", + "placeholder": { + "x": 0, + "y": 0, + "width": 1024, + "height": 576, + "originalWidth": 1024, + "originalHeight": 576 + } + } +} +``` + +Then call `POST /api/external/v1/editor/images/generations`. + +## Reference Images + +When the user provides a local reference image path/file, upload it first; do not ask the user to convert it to base64. + +Python helper path: + +```python +from genarrative_external_api import GenarrativeExternalClient + +client = GenarrativeExternalClient() +ref = client.upload_reference_image("/path/to/reference.png") +client.generate_image( + "基于参考图生成一张 16:9 游戏背景", + aspectRatio="16:9", + imageSize="1K", + referenceImageSrcs=[ref["objectKey"]], +) +``` + +Use the normal upload flow with: + +```json +{ + "legacyPrefix": "generated-character-drafts", + "pathSegments": ["editor", "external-editor-references"], + "fileName": "", + "contentType": "image/png", + "access": "private" +} +``` + +After OSS form upload, confirm the object with `assetKind: "editor_reference_image"`. Put the returned `objectKey` into the generation request: + +- image generation: `referenceImageSrcs` +- image edit/redraw: `sourceImageSrc`; extra references go in `referenceImageSrcs` +- icon spritesheet: `referenceImageSrc` +- UI asset extraction: `sourceImageSrc`; extra references go in `referenceImageSrcs` +- character animation: `sourceImageSrc` +- video generation image references: `referenceImageSrcs` + +Use `signedUrl` only for display/download. For generation requests, use `objectKey`, project resource ID, asset ID, public URL, or Data URL as the endpoint allows; prefer uploaded `objectKey` for local/private reference images. + +OSS form upload shape, using the ticket response saved as `ticket.json`. The default response has `upload`; if the caller explicitly requested the API response envelope, use `data.upload`: + +```bash +node - <<'NODE' ticket.json /path/to/reference.png +const fs = require('fs'); +const path = require('path'); + +(async () => { + const body = JSON.parse(fs.readFileSync(process.argv[2], 'utf8')); + const ticket = body.upload || body.data?.upload; + if (!ticket) throw new Error('Upload ticket response missing upload payload'); + const filePath = process.argv[3]; + const form = new FormData(); + for (const [key, value] of Object.entries(ticket.formFields)) { + if (value != null) form.append(key, value); + } + const bytes = fs.readFileSync(filePath); + form.append( + 'file', + new Blob([bytes], { type: ticket.contentType || 'application/octet-stream' }), + path.basename(filePath), + ); + const response = await fetch(ticket.host, { method: 'POST', body: form }); + if (!response.ok) { + throw new Error(`OSS upload failed: ${response.status} ${await response.text()}`); + } +})().catch((error) => { + console.error(error.message); + process.exit(1); +}); +NODE +``` + +Then confirm with `contentLength`: + +```json +{ + "objectKey": "", + "contentType": "image/png", + "contentLength": 12345, + "assetKind": "editor_reference_image", + "accessPolicy": "private" +} +``` + +`contentLength` is a JSON number from the local file byte size, not a quoted string. + +For character animation from an uploaded local image, set: + +```json +{ + "sourceLayerId": "external-reference-hero", + "sourceImageSrc": "", + "sourceWidth": 720, + "sourceHeight": 1280, + "promptText": "让角色自然呼吸并轻微转身", + "resolution": "720p", + "ratio": "9:16", + "frameCount": 40, + "durationSeconds": 5, + "model": "seedance2.0-fast" +} +``` + +Use an existing canvas layer ID when the image came from a project layer. If it came only from a local upload, derive a stable synthetic `sourceLayerId` from the file name, for example `external-reference-hero`. Read `sourceWidth` and `sourceHeight` from the actual image before upload; ask the user only if the dimensions cannot be determined. + +For video generation, always include `mode: "std"`. When using image/video/audio references, default to `model: "seedance2.0-fast"` unless the user asks for another listed model, because reference media support is limited to the Seedance 2.0 family. + +For image edit/redraw that should replace an existing canvas layer, pass `projectId` and `targetLayerId`. If the user instead gives an explicit `canvasCompletion`, let that placement win. + +For sound effects and BGM, `assetFolderId` and `assetLabel` can write the generated audio to the account asset library, same as image/video generation. + +## Guardrails + +- Do not invent endpoints outside the OpenAPI, especially internal worker or runtime task-list routes. +- Do not put API Keys in repository files, generated project files, command history snippets with literal secrets, logs, docs, commits, or screenshots. The only default storage is the user's local private JSON credentials file. +- Do not use account JWT endpoints as the default external integration path. The profile API can create/revoke keys for logged-in product users, but it is not part of the external editor OpenAPI. +- When an endpoint returns `project`, `resource`, or `asset`, treat those as the authoritative updated project/resource/asset snapshots. + +## Resources + +- `references/api-selection.md`: intent routing and required-field cheat sheet. +- `scripts/genarrative_external_api.py`: stdlib Python helper for OpenAPI fetch, API Key loading, local reference upload, object confirm, project/canvas calls, and generation requests. diff --git a/.codex/skills/genarrative-external-editor-api/agents/openai.yaml b/.codex/skills/genarrative-external-editor-api/agents/openai.yaml new file mode 100644 index 000000000..4a23c1b13 --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/agents/openai.yaml @@ -0,0 +1,6 @@ +interface: + display_name: "Genarrative External Editor API" + short_description: "Auto-route external canvas API usage" + default_prompt: "Use $genarrative-external-editor-api to infer the right external canvas API and draft a request." +policy: + allow_implicit_invocation: true diff --git a/.codex/skills/genarrative-external-editor-api/references/api-selection.md b/.codex/skills/genarrative-external-editor-api/references/api-selection.md new file mode 100644 index 000000000..71f2c261b --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/references/api-selection.md @@ -0,0 +1,180 @@ +# External Editor API Routing + +Source of truth: `docs/openapi/genarrative-external-v1.openapi.json`. + +## Base + +- Fixed base URL: `https://www.genarrative.world/`. +- Public contract: `GET /api/external/v1/openapi.json`. +- Authenticated calls: `Authorization: Bearer `. +- Default credentials file: `~/.config/genarrative/external-editor-api.json` with an `apiKey` string. + +## Intent Routing + +Infer the endpoint from the user's description. Do not present this as a menu unless the request is genuinely ambiguous. + +| User says | Route | +| --- | --- | +| "生成图片", "生图", "做一张背景/角色/宣发图" | `POST /api/external/v1/editor/images/generations` | +| "重绘", "调整这张图", "基于这张图修改" | `POST /api/external/v1/editor/images/edits` | +| "用这张参考图", "参考本地图片生成", "基于本地图做图" | Upload local image first, then pass returned `objectKey` into the generation/edit reference field | +| "按规范图生成图标", "拆图标" | `POST /api/external/v1/editor/icon-spritesheets/generations` | +| "从 UI 设计图提取素材" | `POST /api/external/v1/editor/ui-designs/assets/extractions` | +| "让角色动起来", "生成角色动画帧" | `POST /api/external/v1/editor/character-animations/generations` | +| "生成视频" | `POST /api/external/v1/editor/videos/generations` | +| "生成音效" | `POST /api/external/v1/editor/audios/sound-effects/generations` | +| "生成背景音乐/BGM" | `POST /api/external/v1/editor/audios/background-music/generations` | +| "上传本地素材/图片/音频/视频" | Upload flow: direct upload ticket -> OSS form upload -> object confirm | +| "保存画板布局" | `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | +| "创建/读取/删除画板项目" | Project endpoints | +| "素材库/文件夹/素材记录" | Asset library endpoints | +| "读取私有素材/拿可访问链接" | `GET /api/external/v1/assets/read-url` | + +Ask a follow-up only when two routes could both be correct and produce different artifacts, for example "处理这张图" without saying edit, extract UI assets, or use it as a reference for new generation. + +## Endpoint Map + +| User intent | Endpoint | Minimum request | +| --- | --- | --- | +| Read contract | `GET /api/external/v1/openapi.json` | No auth required | +| List projects | `GET /api/external/v1/editor/projects` | API Key | +| Create project | `POST /api/external/v1/editor/projects` | Optional `title` | +| Load recent project | `GET /api/external/v1/editor/projects/recent` | API Key | +| Get/delete project | `GET` or `DELETE /api/external/v1/editor/projects/{projectId}` | `projectId` | +| Rename project | `PATCH /api/external/v1/editor/projects/{projectId}/metadata` | `title` | +| Save canvas layout | `PATCH /api/external/v1/editor/projects/{projectId}/canvas` | `viewport`, `layers` | +| Add project resource | `POST /api/external/v1/editor/projects/{projectId}/resources` | `imageSrc`, `width`, `height`, `sourceType` | +| Create upload ticket | `POST /api/external/v1/assets/direct-upload-tickets` | `legacyPrefix`, `fileName` | +| Confirm uploaded object | `POST /api/external/v1/assets/objects/confirm` | `objectKey`, `assetKind` | +| Get signed read URL | `GET /api/external/v1/assets/read-url` | `objectKey` or `legacyPublicPath` | +| Read asset library | `GET /api/external/v1/editor/assets/library` | API Key | +| Create/update/delete folder | `POST /api/external/v1/editor/assets/folders`, `PATCH`/`DELETE /api/external/v1/editor/assets/folders/{folderId}` | create: `label`; update: `label` or `collapsed` | +| Create asset record | `POST /api/external/v1/editor/assets` | `folderId`, `label`, `imageSrc`, `width`, `height`, `sourceType` | +| Update/delete asset | `PATCH`/`DELETE /api/external/v1/editor/assets/{assetId}` | update: `label` or `folderId` | + +## Generation Endpoints + +| User intent | Endpoint | Required fields | Common optional fields | +| --- | --- | --- | --- | +| Generate image/spec/character/UI/publication material | `POST /api/external/v1/editor/images/generations` | `prompt` | `kind`, `model`, `aspectRatio`, `imageSize`, `size`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | +| Edit/redraw image | `POST /api/external/v1/editor/images/edits` | `prompt`, `sourceImageSrc` | `referenceImageSrcs`, `model`, `size`, `projectId`, `assetFolderId`, `assetLabel`, `sourceResourceId`, `targetLayerId`, `canvasCompletion` | +| Generate icon spritesheet | `POST /api/external/v1/editor/icon-spritesheets/generations` | `referenceImageSrc`, `iconDescriptions` | `referenceImageSrcs`, `model`, `aspectRatio`, `imageSize`, `projectId`, `assetFolderId`, `canvasCompletion` | +| Extract assets from UI design | `POST /api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize` | `model`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `spritesheetLabel`, `canvasCompletion` | +| Generate character animation | `POST /api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `canvasCompletion` | +| Generate video | `POST /api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | `referenceImageSrcs`, `referenceVideoSrcs`, `referenceAudioSrcs`, `webSearchEnabled`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | +| Generate sound effect | `POST /api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` | `model`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | +| Generate background music | `POST /api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` | `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | + +## Reference Image Upload + +If the user provides a local file as a reference image, run upload before the generation request: + +1. `POST /api/external/v1/assets/direct-upload-tickets`. + Use `legacyPrefix: "generated-character-drafts"`, `pathSegments: ["editor", "external-editor-references"]`, original `fileName`, detected image `contentType`, and `access: "private"`. +2. Upload the file to the returned OSS form endpoint with all returned `formFields`. +3. `POST /api/external/v1/assets/objects/confirm` with returned `objectKey`, detected `contentType`, `contentLength` if known, `assetKind: "editor_reference_image"`, and `accessPolicy: "private"`. +4. Use the returned `objectKey` in the actual editor request. + +OSS form upload uses `upload.host` and every non-null `upload.formFields` entry, then the file part named `file`. Default responses expose `upload`; envelope responses expose `data.upload`. Save the upload ticket response as `ticket.json`: + +```bash +node - <<'NODE' ticket.json /path/to/reference.png +const fs = require('fs'); +const path = require('path'); + +(async () => { + const body = JSON.parse(fs.readFileSync(process.argv[2], 'utf8')); + const ticket = body.upload || body.data?.upload; + if (!ticket) throw new Error('Upload ticket response missing upload payload'); + const filePath = process.argv[3]; + const form = new FormData(); + for (const [key, value] of Object.entries(ticket.formFields)) { + if (value != null) form.append(key, value); + } + const bytes = fs.readFileSync(filePath); + form.append( + 'file', + new Blob([bytes], { type: ticket.contentType || 'application/octet-stream' }), + path.basename(filePath), + ); + const response = await fetch(ticket.host, { method: 'POST', body: form }); + if (!response.ok) { + throw new Error(`OSS upload failed: ${response.status} ${await response.text()}`); + } +})().catch((error) => { + console.error(error.message); + process.exit(1); +}); +NODE +``` + +Field mapping after upload: + +| Target API | Put uploaded `objectKey` in | +| --- | --- | +| Image generation | `referenceImageSrcs` | +| Image edit/redraw | `sourceImageSrc`; additional references in `referenceImageSrcs` | +| Icon spritesheet | `referenceImageSrc`; additional style refs in `referenceImageSrcs` | +| UI design extraction | `sourceImageSrc`; additional refs in `referenceImageSrcs` | +| Character animation | `sourceImageSrc` | +| Video generation with image references | `referenceImageSrcs` | + +Do not put the signed read URL into generation fields. Signed URLs are for user-visible preview/download; generation fields should use the stable `objectKey` for uploaded private references. + +## Common Enums + +- Image `kind`: `spec`, `character`, `quick-edit`, `ui-design`, `publication-material`. +- Image `model`: `gpt-image-2`, `gemini-3.1-flash-image-preview`, `nanobanana2`, `nano-banana`. +- Image `aspectRatio`: `1:1`, `2:3`, `3:2`, `9:16`, `16:9`. +- Image `imageSize`: `0.5K`, `1K`, `2K`. +- Video `model`: `seedance2.0`, `seedance2.0-fast`, `kling3.0`, `kling3.0-omni`, `veo3.1`, `veo3.1-fast`. +- Video `aspectRatio`: `16:9`, `9:16`, `1:1`, `4:3`, `3:4`, `21:9`. +- Video `resolution`: `480p`, `720p`, `1080p`. +- Video `mode`: always `std`. +- Video `sound`: `on`, `off`. +- Character animation `model`: always `seedance2.0-fast`. +- Character animation `resolution`: `480p`, `720p`; `frameCount`: `32`, `40`, `48`; `durationSeconds`: `4`, `5`, `6`; `ratio`: `same`, `1:1`, `4:3`, `16:9`, `9:16`, `3:4`. + +## Local Reference Media Details + +- `contentLength` in object confirm is a JSON number from local byte size, not a string. +- For character animation, use an existing project layer ID as `sourceLayerId` when available. +- If the source is only an uploaded local image, derive `sourceLayerId` from the file name, such as `external-reference-hero`, and keep it stable across retries. +- Read `sourceWidth` and `sourceHeight` from the local image. If dimensions cannot be read, ask instead of inventing dimensions. +- UI design extraction uses fixed `aspectRatio: "1:1"`; choose `imageSize: "1K"` for normal/small extractions and `2K` for dense designs. +- Video image/video/audio references are supported only by the Seedance 2.0 family; default referenced-media video requests to `model: "seedance2.0-fast"`, `mode: "std"`, and explicit `sound`. +- Image edit/redraw can pass `targetLayerId` with `projectId` to replace an existing canvas layer when no explicit `canvasCompletion` is supplied. +- Image, video, sound effect, and BGM generation can pass `assetFolderId` and `assetLabel`; response `asset` is the created/updated library record. + +## Canvas Completion + +Use `canvasCompletion` only when the generated result should be written back into a project canvas by the backend. + +Required: + +```json +{ + "title": "素材名称", + "placeholder": { + "x": 0, + "y": 0, + "width": 512, + "height": 512, + "originalWidth": 512, + "originalHeight": 512 + } +} +``` + +`dialogId` is optional. If the response includes `project`, `resource`, or `asset`, use those snapshots instead of reconstructing canvas/resource/library state locally. + +## Upload Flow + +For a local file that should become a project resource or library asset: + +1. `POST /api/external/v1/assets/direct-upload-tickets` with `legacyPrefix`, `fileName`, and optional `contentType`, `access`, `maxSizeBytes`. +2. Submit the file to the returned OSS form endpoint with returned `formFields`. +3. `POST /api/external/v1/assets/objects/confirm` with returned `objectKey` and an `assetKind`. +4. Create a project resource or library asset with the confirmed `assetObjectId`/`objectKey`. + +For reading private/generated assets, call `GET /api/external/v1/assets/read-url?objectKey=...` and use the returned `signedUrl`. diff --git a/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py b/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py new file mode 100644 index 000000000..f6adcce12 --- /dev/null +++ b/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +"""Tiny stdlib client for Genarrative external editor APIs.""" + +from __future__ import annotations + +import argparse +import json +import mimetypes +import os +import re +import struct +import sys +import tempfile +import urllib.error +import urllib.parse +import urllib.request +import uuid +from pathlib import Path +from typing import Any + + +BASE_URL = "https://www.genarrative.world/" +DEFAULT_CREDENTIALS_FILE = Path.home() / ".config/genarrative/external-editor-api.json" + + +class GenarrativeApiError(RuntimeError): + pass + + +def load_api_key(credentials_file: str | os.PathLike[str] = DEFAULT_CREDENTIALS_FILE) -> str: + path = Path(credentials_file).expanduser() + try: + value = json.loads(path.read_text(encoding="utf-8")).get("apiKey", "") + except FileNotFoundError as error: + raise GenarrativeApiError( + f"Missing API key file: {path}. Create JSON like {{\"apiKey\":\"tnr_sk_...\"}}." + ) from error + except json.JSONDecodeError as error: + raise GenarrativeApiError(f"Invalid JSON in API key file: {path}.") from error + if not isinstance(value, str) or not value.strip(): + raise GenarrativeApiError(f"Missing apiKey string in API key file: {path}.") + return value.strip() + + +def unwrap_envelope(body: Any) -> Any: + if isinstance(body, dict) and body.get("ok") is True and "data" in body: + return body["data"] + return body + + +def guess_content_type(file_path: str | os.PathLike[str]) -> str: + return mimetypes.guess_type(str(file_path))[0] or "application/octet-stream" + + +def source_layer_id_from_path(file_path: str | os.PathLike[str]) -> str: + stem = Path(file_path).stem.lower() + slug = re.sub(r"[^a-z0-9]+", "-", stem).strip("-") or "image" + return f"external-reference-{slug}" + + +def image_dimensions(file_path: str | os.PathLike[str]) -> tuple[int, int] | None: + path = Path(file_path) + with path.open("rb") as fh: + header = fh.read(32) + if header.startswith(b"\x89PNG\r\n\x1a\n") and header[12:16] == b"IHDR": + return struct.unpack(">II", header[16:24]) + if not header.startswith(b"\xff\xd8"): + return None + fh.seek(2) + while True: + marker_prefix = fh.read(1) + if not marker_prefix: + return None + if marker_prefix != b"\xff": + continue + marker = fh.read(1) + while marker == b"\xff": + marker = fh.read(1) + if marker in {b"\xd8", b"\xd9"}: + continue + length_bytes = fh.read(2) + if len(length_bytes) != 2: + return None + length = struct.unpack(">H", length_bytes)[0] + if marker and marker[0] in {0xC0, 0xC1, 0xC2, 0xC3, 0xC5, 0xC6, 0xC7, 0xC9, 0xCA, 0xCB, 0xCD, 0xCE, 0xCF}: + data = fh.read(5) + if len(data) != 5: + return None + height, width = struct.unpack(">HH", data[1:5]) + return width, height + fh.seek(max(length - 2, 0), os.SEEK_CUR) + + +class GenarrativeExternalClient: + def __init__( + self, + api_key: str | None = None, + credentials_file: str | os.PathLike[str] = DEFAULT_CREDENTIALS_FILE, + base_url: str = BASE_URL, + ) -> None: + self.base_url = base_url.rstrip("/") + self.api_key = api_key if api_key is not None else load_api_key(credentials_file) + + def request_json( + self, + method: str, + path: str, + body: dict[str, Any] | None = None, + query: dict[str, Any] | None = None, + auth: bool = True, + timeout: int = 60, + ) -> Any: + url = f"{self.base_url}{path}" + if query: + url = f"{url}?{urllib.parse.urlencode({k: v for k, v in query.items() if v is not None})}" + data = None if body is None else json.dumps(body).encode("utf-8") + headers = {"Accept": "application/json"} + if data is not None: + headers["Content-Type"] = "application/json" + if auth: + headers["Authorization"] = f"Bearer {self.api_key}" + request = urllib.request.Request(url, data=data, headers=headers, method=method.upper()) + try: + with urllib.request.urlopen(request, timeout=timeout) as response: + payload = response.read() + except urllib.error.HTTPError as error: + detail = error.read().decode("utf-8", errors="replace") + raise GenarrativeApiError(f"{method.upper()} {path} failed: HTTP {error.code}: {detail}") from error + if not payload: + return None + return unwrap_envelope(json.loads(payload.decode("utf-8"))) + + def openapi(self) -> Any: + return self.request_json("GET", "/api/external/v1/openapi.json", auth=False) + + def list_projects(self) -> Any: + return self.request_json("GET", "/api/external/v1/editor/projects") + + def create_project(self, title: str | None = None) -> Any: + body = {} if title is None else {"title": title} + return self.request_json("POST", "/api/external/v1/editor/projects", body) + + def save_canvas(self, project_id: str, viewport: dict[str, Any], layers: dict[str, Any]) -> Any: + return self.request_json( + "PATCH", + f"/api/external/v1/editor/projects/{urllib.parse.quote(project_id, safe='')}/canvas", + {"viewport": viewport, "layers": layers}, + ) + + def create_upload_ticket(self, file_path: str | os.PathLike[str], access: str = "private") -> Any: + path = Path(file_path) + return self.request_json( + "POST", + "/api/external/v1/assets/direct-upload-tickets", + { + "legacyPrefix": "generated-character-drafts", + "pathSegments": ["editor", "external-editor-references"], + "fileName": path.name, + "contentType": guess_content_type(path), + "access": access, + }, + ) + + def upload_to_oss(self, ticket_response: Any, file_path: str | os.PathLike[str]) -> None: + payload = unwrap_envelope(ticket_response) + ticket = payload.get("upload") if isinstance(payload, dict) else None + if not isinstance(ticket, dict): + raise GenarrativeApiError("Upload ticket response missing upload payload.") + fields = ticket.get("formFields") + if not isinstance(fields, dict): + raise GenarrativeApiError("Upload ticket response missing formFields.") + boundary = f"----genarrative-{uuid.uuid4().hex}" + path = Path(file_path) + content_type = ticket.get("contentType") or guess_content_type(path) + chunks: list[bytes] = [] + for key, value in fields.items(): + if value is None: + continue + chunks.extend( + [ + f"--{boundary}\r\n".encode(), + f'Content-Disposition: form-data; name="{key}"\r\n\r\n'.encode(), + str(value).encode(), + b"\r\n", + ] + ) + chunks.extend( + [ + f"--{boundary}\r\n".encode(), + f'Content-Disposition: form-data; name="file"; filename="{path.name}"\r\n'.encode(), + f"Content-Type: {content_type}\r\n\r\n".encode(), + path.read_bytes(), + b"\r\n", + f"--{boundary}--\r\n".encode(), + ] + ) + request = urllib.request.Request( + ticket["host"], + data=b"".join(chunks), + headers={"Content-Type": f"multipart/form-data; boundary={boundary}"}, + method="POST", + ) + try: + with urllib.request.urlopen(request, timeout=300) as response: + if response.status not in {200, 201, 204}: + raise GenarrativeApiError(f"OSS upload failed: HTTP {response.status}") + except urllib.error.HTTPError as error: + detail = error.read().decode("utf-8", errors="replace") + raise GenarrativeApiError(f"OSS upload failed: HTTP {error.code}: {detail}") from error + + def confirm_asset_object( + self, + object_key: str, + file_path: str | os.PathLike[str], + asset_kind: str = "editor_reference_image", + access_policy: str = "private", + ) -> Any: + path = Path(file_path) + return self.request_json( + "POST", + "/api/external/v1/assets/objects/confirm", + { + "objectKey": object_key, + "contentType": guess_content_type(path), + "contentLength": path.stat().st_size, + "assetKind": asset_kind, + "accessPolicy": access_policy, + }, + ) + + def upload_reference_image(self, file_path: str | os.PathLike[str]) -> dict[str, Any]: + ticket = self.create_upload_ticket(file_path) + upload = unwrap_envelope(ticket)["upload"] + self.upload_to_oss(ticket, file_path) + confirmed = self.confirm_asset_object(upload["objectKey"], file_path) + return { + "objectKey": upload["objectKey"], + "ticket": ticket, + "assetObject": unwrap_envelope(confirmed).get("assetObject"), + "dimensions": image_dimensions(file_path), + "sourceLayerId": source_layer_id_from_path(file_path), + } + + def read_url(self, object_key: str) -> Any: + return self.request_json("GET", "/api/external/v1/assets/read-url", query={"objectKey": object_key}) + + def generate_image(self, prompt: str, **fields: Any) -> Any: + return self.request_json("POST", "/api/external/v1/editor/images/generations", {"prompt": prompt, **fields}) + + def edit_image(self, prompt: str, source_image_src: str, **fields: Any) -> Any: + return self.request_json( + "POST", + "/api/external/v1/editor/images/edits", + {"prompt": prompt, "sourceImageSrc": source_image_src, **fields}, + ) + + def extract_ui_assets(self, source_image_src: str, image_size: str = "1K", **fields: Any) -> Any: + fields.pop("aspectRatio", None) + return self.request_json( + "POST", + "/api/external/v1/editor/ui-designs/assets/extractions", + {"sourceImageSrc": source_image_src, "imageSize": image_size, **fields, "aspectRatio": "1:1"}, + ) + + def animate_character( + self, + source_image_src: str, + source_width: int, + source_height: int, + prompt_text: str, + source_layer_id: str, + **fields: Any, + ) -> Any: + body = { + "sourceLayerId": source_layer_id, + "sourceImageSrc": source_image_src, + "sourceWidth": source_width, + "sourceHeight": source_height, + "promptText": prompt_text, + "resolution": fields.pop("resolution", "720p"), + "ratio": fields.pop("ratio", "same"), + "frameCount": fields.pop("frameCount", 40), + "durationSeconds": fields.pop("durationSeconds", 5), + **fields, + "model": "seedance2.0-fast", + } + return self.request_json("POST", "/api/external/v1/editor/character-animations/generations", body) + + def generate_video(self, prompt: str, **fields: Any) -> Any: + fields.pop("mode", None) + body = { + "prompt": prompt, + "model": fields.pop("model", "seedance2.0-fast"), + "aspectRatio": fields.pop("aspectRatio", "16:9"), + "durationSeconds": fields.pop("durationSeconds", 5), + "resolution": fields.pop("resolution", "720p"), + "sound": fields.pop("sound", "off"), + **fields, + "mode": "std", + } + return self.request_json("POST", "/api/external/v1/editor/videos/generations", body) + + def generate_sound_effect(self, prompt: str, duration: int, **fields: Any) -> Any: + return self.request_json( + "POST", + "/api/external/v1/editor/audios/sound-effects/generations", + {"prompt": prompt, "duration": duration, **fields}, + ) + + def generate_background_music(self, description: str, **fields: Any) -> Any: + return self.request_json( + "POST", + "/api/external/v1/editor/audios/background-music/generations", + {"gptDescriptionPrompt": description, **fields, "makeInstrumental": True}, + ) + + +def _self_test() -> None: + png = ( + b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR" + b"\x00\x00\x00\x02\x00\x00\x00\x03\x08\x06\x00\x00\x00" + ) + with tempfile.NamedTemporaryFile(suffix="Hero Image.png") as fh: + fh.write(png) + fh.flush() + assert image_dimensions(fh.name) == (2, 3) + assert source_layer_id_from_path(fh.name).startswith("external-reference-") + assert unwrap_envelope({"ok": True, "data": {"upload": 1}}) == {"upload": 1} + print("self-test ok") + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--self-test", action="store_true") + parser.add_argument("--credentials-file", default=str(DEFAULT_CREDENTIALS_FILE)) + parser.add_argument("command", nargs="?", choices=["openapi", "list-projects"]) + args = parser.parse_args(argv) + if args.self_test: + _self_test() + return 0 + if args.command is None: + parser.print_help() + return 0 + if args.command == "openapi": + client = GenarrativeExternalClient(api_key="", credentials_file=args.credentials_file) + print(json.dumps(client.openapi(), ensure_ascii=False, indent=2)) + elif args.command == "list-projects": + client = GenarrativeExternalClient(credentials_file=args.credentials_file) + print(json.dumps(client.list_projects(), ensure_ascii=False, indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/docs/openapi/genarrative-external-v1.openapi.json b/docs/openapi/genarrative-external-v1.openapi.json index f0bd895b4..1815df5d6 100644 --- a/docs/openapi/genarrative-external-v1.openapi.json +++ b/docs/openapi/genarrative-external-v1.openapi.json @@ -2719,6 +2719,13 @@ "null" ] }, + "targetLayerId": { + "type": [ + "string", + "null" + ], + "description": "带 projectId 且未提供 canvasCompletion 时，服务端用生成结果替换该画布图层。" + }, "size": { "type": "string" }, @@ -3643,6 +3650,20 @@ }, "generationInputs": { "$ref": "#/components/schemas/JsonValue" + }, + "assetFolderId": { + "type": [ + "string", + "null" + ], + "description": "传 project 时写入默认项目素材文件夹；传具体 folderId 时写入该文件夹。" + }, + "assetLabel": { + "type": [ + "string", + "null" + ], + "description": "写入素材库时使用的素材名称。" } }, "additionalProperties": false @@ -3681,6 +3702,20 @@ }, "generationInputs": { "$ref": "#/components/schemas/JsonValue" + }, + "assetFolderId": { + "type": [ + "string", + "null" + ], + "description": "传 project 时写入默认项目素材文件夹；传具体 folderId 时写入该文件夹。" + }, + "assetLabel": { + "type": [ + "string", + "null" + ], + "description": "写入素材库时使用的素材名称。" } }, "additionalProperties": false @@ -3771,6 +3806,28 @@ ], "description": "当请求携带 canvasCompletion 且服务端成功写入画布布局时返回最新项目快照。" }, + "resource": { + "anyOf": [ + { + "$ref": "#/components/schemas/EditorProjectResource" + }, + { + "type": "null" + } + ], + "description": "请求携带 projectId 时返回写入的画布资源快照。" + }, + "asset": { + "anyOf": [ + { + "$ref": "#/components/schemas/EditorAsset" + }, + { + "type": "null" + } + ], + "description": "请求携带 assetFolderId 时返回写入的账号素材快照。" + }, "queueState": { "anyOf": [ { diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index e403046b1..0cce8ddea 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -66,8 +66,8 @@ - 现象：画板生成、快速编辑、图标素材或 UI 素材提取如果允许直接提交 generated objectKey，用户只要知道其他账号的私有 objectKey，就可能让 api-server 签名读取并送给外部生成供应商。 - 原因：Data URL 参考图可以直接解析，但 objectKey 是服务端私有对象引用；只校验 generated 前缀、mime 和大小不能证明它属于当前账号。 -- 处理：所有编辑器参考图入口统一走 `parse_editor_reference_image(state, owner_user_id, source)`；objectKey 分支必须先在当前账号的项目资源、素材库资产或 `asset_object` 中匹配 owner / bucket / key，再读取 OSS。快速编辑和图标素材额外参考图也必须真实传到 provider，不只写 metadata。 -- 验证：`cargo test -p api-server --manifest-path server-rs/Cargo.toml editor_reference`，并用前端 workflow 测试覆盖 `referenceImageSrcs` 进入快速编辑 / 图标生成请求。 +- 处理：所有编辑器参考图入口统一走 `parse_editor_reference_image(state, owner_user_id, source)`；objectKey 分支必须先在当前账号的项目资源、素材库资产或 `asset_object` 中匹配 owner / bucket / key，再读取 OSS。图标素材等额外参考图必须真实传到 provider，不只写 metadata；图片快速编辑当前不开放额外参考图，若后续重开入口也必须沿用同一归属校验。 +- 验证：`cargo test -p api-server --manifest-path server-rs/Cargo.toml editor_reference`，并用前端 workflow 测试覆盖 `referenceImageSrcs` 进入图标生成请求；若快速编辑重开额外参考图，再补对应请求覆盖。 - 关联：`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/spacetime-client/src/assets.rs`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`。 ## 编辑器生成按钮显示泥点后仍要查真实钱包预扣 @@ -311,14 +311,30 @@ - 验证：`npm run test -- src/components/image-editor/ImageCanvasOverlayModel.test.ts src/components/image-editor/useImageCanvasGenerationSurface.test.tsx`。 - 关联：`src/components/image-editor/ImageCanvasOverlayModel.ts`、`src/components/image-editor/useImageCanvasGenerationSurface.tsx`、`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`。 -## 图片画布图片改造也必须创建独立生成器占位 +## 图片画布重绘创建独立占位，快速编辑不要新建生成器 -- 现象：点击图片图层的“改造”后，输入框直接挂在原图上，提交时既不像其它生成入口一样有独立占位，也容易让用户误以为会覆盖源图。 -- 原因：图片改造复用了旧 `QuickEditPanelState` / redraw 面板路径，只把源图选中并在原图附近打开快速编辑框，没有进入统一的 `CanvasGenerationDialogState` 占位链路。 -- 处理：图片和用户快照图层的“改造”统一创建 `mode="quick-edit"` 的 generation dialog，占位仍走 `ImageCanvasGenerationPlacementModel`；源图作为隐式最后一张参考图提交，并把“当前图”提示词归一到对应参考图编号。音频改造继续走音频生成器路径，非图片 fallback 才保留旧面板。参考图 Data URL 提交前可压缩，但浏览器图片解码卡住时必须超时透传原图，不能阻塞生成请求。 -- 验证：`npm run test -- src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/services/image-editor/editorImageReference.test.ts -- --runInBand`，以及 `npm run test -- src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx -t "hides quick edit and redraw panels|opens generated image info|shows the quick edit generator" -- --runInBand`。 +- 现象：用户点击图片素材的“快速编辑”后，画布上额外出现 `Quick Edit Generator` 占位，像是新建了一个生成器；但用户预期是在原图下方框选区域、填写一个提示词和模型，然后直接修改当前图。 +- 原因：快速编辑入口和提交链路误用了 `createQuickEditGenerationDialogDraft(...)` / `CanvasGenerationDialogState`，把“覆盖源图”的快速编辑伪装成会产出新图层的生成器占位。 +- 处理：图片快速编辑必须走 `QuickEditPanelState`，打开时归档当前 active generation dialog 但不创建新的 `mode="quick-edit"` dialog；提交时调用 `/api/editor/images/edits`，把当前图片或带编号标注的图片作为 `sourceImageSrc`，成功后覆盖源图，失败时保留快速编辑面板。图片重绘、去背景、视频快速编辑等会产出新图层或异步占位的入口仍可走 generation dialog / placement 链路。 +- 验证：`npm run test -- src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/components/image-editor/ImageCanvasQuickEditPanelView.test.tsx -- --runInBand`，以及按需运行 `npm run test -- src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx -t "快速编辑|quick edit" -- --runInBand`。 - 关联：`src/components/image-editor/useImageCanvasGenerationWorkflow.ts`、`src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`src/services/image-editor/editorImageReference.ts`。 +## 图片画布快速编辑完成必须按目标图层回写 + +- 现象：图片快速编辑任务成功后，刷新页面素材库能看到新图，但画布上的源图没有替换。 +- 原因：`/api/editor/images/edits` 只保存生成图、项目资源和素材；没有 `canvasCompletion` 时不会写 `editor_canvas.layers_json`。`sourceResourceId` 只能表示溯源，同一资源可出现在多个图层，不能用它来决定替换哪一层。 +- 处理：图片快速编辑请求必须传 `targetLayerId`；后端在没有 `canvasCompletion` 的快速编辑完成分支里，用目标 layer id 和生成资源写回项目 layout。 +- 验证：`npm run test -- src/services/image-editor/editorProjectClient.test.ts src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx -- --runInBand`；后端验证至少覆盖 `editor_image_edit_request_omits_price_mud_points` 和 `editor_image_edit_can_complete_by_replacing_target_layer`。 +- 关联：`src/services/image-editor/editorProjectClient.ts`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`server-rs/crates/api-server/src/editor_project.rs`。 + +## 图片画布快速编辑元数据必须记录原图引用 + +- 现象：快速编辑生成的新图可以替换画布，但打开图片信息时“生成输入”里看不到被修改的原图。 +- 原因：信息面板直接渲染 `generationInputs.references`；快速编辑虽然把原图作为 `sourceImageSrc` 传给 provider，但如果 `buildQuickEditGenerationInputs(...)` 不把源图写成引用，后端资源和画布层都没有可展示的原图引用。 +- 处理：快速编辑的 `generationInputs.references` 必须始终包含 `原图`，再追加用户额外参考图；关闭额外参考图入口时也不能删除这条源图引用。 +- 验证：`npm run test -- src/components/image-editor/ImageCanvasGenerationModel.test.ts src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx -- --runInBand`。 +- 关联：`src/components/image-editor/ImageCanvasGenerationModel.ts`、`src/components/image-editor/ImageCanvasMetadataModalView.tsx`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`。 + ## 图片画布生成完成应用项目快照后也要刷新素材库 - 现象：部分素材生成成功后画布上已经出现结果，但左侧素材库没有立刻出现新素材，刷新页面后才显示。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 93191f296..1e51204ce 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -20,12 +20,12 @@ - 图片拖拽时显示水平 / 垂直吸附参考线，吸附到其它图层、生成占位框或画板的边缘与中心线；当移动元素接近两个同轴元素形成的等距位置时，支持横向或纵向等距吸附。 - 生成资源右上角显示元数据按钮，点击打开独立元数据窗口。图片信息页不展示后端组装后的生图 Prompt，也不提供复制 Prompt；只展示该图片生成时用户在面板里提交的输入快照，包括普通生成提示词、规范表单字段、角色设定、图标素材描述、快速编辑提示词、重绘提示词，以及角色规范 / 常规参考图 / 图标规范 / 编辑参考图等参考图卡片，并提供“复制信息”复制当前可见字段。参考图输入快照只保存 `refType/refId` 行引用，其中 `refType="project-resource"` 指向 `editor_project_resource.resourceId`，`refType="asset"` 指向 `editor_asset.assetId`；不得把图片 Data URL、普通 URL 或 `objectKey` 写入 `generationInputs.references`。旧数据或上传图片没有输入快照时显示 `-`，禁止回退展示内部 Prompt。 - 对生成资源执行重绘时，在右侧创建新的生成结果图层，并自动调整视图显示原图和新图；重绘面板不因提交成功自动关闭，便于连续改提示词。重绘 / 改造输入框只允许从 `generationInputs.fields` 中恢复用户可见输入快照，例如普通生成提示词、视频描述、音效 `prompt`、背景音乐 `gpt_description_prompt`、角色设定、UI 用户输入、图标素材描述、规范表单和宣发素材字段；禁止回退展示资源 `prompt` / `actualPrompt` 中的后端拼接 Prompt、固定生成模板或模型默认提示词。没有用户输入快照的旧图层打开改造时保持空输入，等待用户重新填写。 -- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine。普通生成、生成规范和重绘保留既有 `gpt-image-2` 路径；图片快速编辑默认从原图模型和分辨率初始化，但提交使用面板当前选择的 `model/aspectRatio/imageSize`；其中生成规范类图片固定 `16:9`、`2K`、`gpt-image-2`，面板底部用与可编辑面板一致的比例 / 尺寸 / 模型胶囊按钮展示固定参数，但按钮为禁用态，不允许在该面板改比例、尺寸或模型。`生成角色形象` 与 `生成图标素材` 支持 `nanobanana2`（`gemini-3.1-flash-image-preview`）和 `gpt-image-2`，默认 `nanobanana2`，并在两类面板之间沿用用户上次选择的模型；两类面板额外提供抠图背景色下拉，展示颜色名称、方块色样和 hex 值，默认 `浅雾蓝 #CFEFFF`，可选 `浅钢蓝 #B0C2E0`、`暖浅桃色 #FFD6C2`、`淡薰衣草紫 #E6D8FF`、`浅粉灰 #F4D8E8`、`中度天蓝 #7FB3FF`。`nanobanana2` 走 `/v1beta/models/{model}:generateContent`，请求体写入 `generationConfig.imageConfig.aspectRatio/imageSize`；`gpt-image-2` 走 `/v1/images/generations` 或 `/v1/images/edits`，请求体按 VectorEngine 文档映射 `size`。宣发素材三个工作流（游戏首图、详情五图、运营海报）固定使用 `gpt-image-2`，面板模型胶囊为禁用态，不提供 `nanobanana2` 入口；前端按 workflow 同时提交 `outputSize`、`aspectRatio` 和 `imageSize`，其中游戏首图为 `720x540 / 4:3`、详情单图为 `720x1280 / 9:16`、运营海报为 `1280x720 / 16:9`；后端收到 `kind: "publication-material"` 时也强制归一为 `gpt-image-2` 生成和计费，生成回填图层优先使用生成占位的 `originalWidth/originalHeight`，即使上游回包尺寸漂移也不得把宣发素材卡片变成随机 `1:1` 或 `4:3`。纯文本生成走 `/api/editor/images/generations`，重绘在前端读入当前图层图片 Data URL 后走同一图片生成 BFF，并在原图右侧生成一张新图；普通图层重绘作为 `quick-edit` 参考图提交，角色图层重绘必须按 `kind: "character"` 提交，继续套用角色生成器提示词限定、透明 PNG 后处理和角色资产持久化。`生成视频` 走 `/api/editor/videos/generations`，前端模型入口仅展示 Seedance 2.0 Fast / Seedance 2.0 / Kling 3.0 / Kling 3.0 Omni，不展示 Veo 入口，默认 Seedance 2.0 Fast；视频参数按当前正式面板支持的比例、时长、清晰度和声音开关提交，且 Seedance Fast 与 Seedance 标准版必须按各自真实模型 ID 独立映射，不得混用。生成结果以视频图层加入画布。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成图片后以当前视口世界中心为目标，经统一 placement 避让后创建选中的灰色占位框，输入框跟随占位框显示；待生成、生成中和失败后保留的占位图都必须继续支持拖动，生成完成时真实生成图或视频落在最新占位框位置，输入框继续跟随新生成图层；占位图失焦时隐藏高亮边框、左上角生成器名称和右上角原始尺寸，重新聚焦时再显示，且名称 / 尺寸在画布缩小时按 viewport 反向缩放保持屏幕尺寸稳定；点击所有图片 / 视频生成入口并确认请求开始后，必须隐藏对应设置面板，只保留画布内占位图或原图预览，并在预览上显示 Lovart 式生成中遮罩，避免“面板仍占屏”或“预览一起消失”。图片快速编辑和重绘在调用图片 BFF 前必须把当前图层图片源读取为图片 Data URL；视频素材快速编辑走视频生成 BFF，不允许走图片模型；角色动作快速编辑固定使用 `seedance2.0-fast` 动作 / 视频模型。前端不持有 provider 密钥；上游失败或配置缺失时恢复当前生成设置面板展示失败，不创建 mock 成功图。 -- 图片画布抠图分两类：手动去除背景面向用户任意图片，走登录态同源 BFF `POST /api/editor/images/background-removals` 并转发远端 BiRefNet；编辑器自己生成的标准纯色背景抠图资产统一走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的纯色背景提示词契约和本地确定性透明化，字节级解码、透明化和 PNG 编码下沉复用 `platform-image::generated_asset_sheets`。角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取都必须把所选 `screenColor` 注入 prompt、请求体和 `generationInputs.fields` 的 `抠图背景色`；后端只接受上述六个 hex，缺省或空值按 `#CFEFFF` 处理，未知值返回 `400`。本地透明化用所选颜色作为 key color，且非绿色 key color 不启用近白背景清理，避免浅色主体被误删。角色动作抽帧后的序列帧暂不接用户背景色选择，仍沿用 legacy `#00FF00` 绿幕和近白兜底。BiRefNet 服务地址为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL/remove-background`，默认 `http://58.87.105.82/remove-background`；可选访问令牌只来自服务端环境变量 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`，前端不持有令牌。api-server 对上游结果做响应字节和图片尺寸上限保护，并先落 OSS / asset object，再返回 `imageSrc/objectKey/assetObjectId/taskId`；queue 模式下手动去背景进入 SpacetimeDB 外部生成队列，画布任务侧栏只展示服务器任务阶段，生成中才显示耗时，不显示百分比；有项目上下文时前端同时创建去背景生成占位并把 `canvasCompletion` 交给后端，完成后由后端写入结果图层和最新项目快照。 -- 快速编辑面板对齐其它生成类面板：首行支持额外参考图，最多 8 张；原图 / 原素材作为 `/api/editor/images/edits` 的 `sourceImageSrc` 直接提交，不作为 `referenceImageSrcs` 的最后一张隐式参考，也不在参考图条里固定展示 `图x`。打开快速编辑时画布必须自动平移缩放，让原素材完整落在可视区上半部分，底部面板固定出现在素材下方且不遮挡内容，竖屏 UI 素材也必须完整展示。快速编辑右侧显示矩形、椭圆、画笔框选工具，但进入时不默认启用；点击工具后显示选中态，再点同一工具取消启用。尺寸和模型默认从原素材配置 / 分辨率推断，底部参数胶囊可点击修改；图片快速编辑左下角统一显示 `x:y·xK`，右下角模型胶囊紧贴生成按钮。提交时保留用户提示词里对 `原图`、`当前图片`、`当前图` 或 `图1` 的原始表述，不再改写为 `图N`。点击快速编辑生成后立即创建独立 `Quick Edit Generator` 画布占位播放生成中动画，不再在原图图层上播放生成中遮罩；生成成功后直接用结果覆盖原图图层，该生成中占位可通过键盘 `Delete` / `Backspace` 删除。 +- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine。普通生成、生成规范和重绘保留既有 `gpt-image-2` 路径；图片快速编辑统一打开框选区域 + 单提示词 + 模型选择面板，默认沿用原图模型，不展示参考图或比例 / 尺寸控件；其中生成规范类图片固定 `16:9`、`2K`、`gpt-image-2`，面板底部用与可编辑面板一致的比例 / 尺寸 / 模型胶囊按钮展示固定参数，但按钮为禁用态，不允许在该面板改比例、尺寸或模型。`生成角色形象` 与 `生成图标素材` 支持 `nanobanana2`（`gemini-3.1-flash-image-preview`）和 `gpt-image-2`，默认 `nanobanana2`，并在两类面板之间沿用用户上次选择的模型；两类面板额外提供抠图背景色下拉，展示颜色名称、方块色样和 hex 值，默认 `浅雾蓝 #CFEFFF`，可选 `浅钢蓝 #B0C2E0`、`暖浅桃色 #FFD6C2`、`淡薰衣草紫 #E6D8FF`、`浅粉灰 #F4D8E8`、`中度天蓝 #7FB3FF`。`nanobanana2` 走 `/v1beta/models/{model}:generateContent`，请求体写入 `generationConfig.imageConfig.aspectRatio/imageSize`；`gpt-image-2` 走 `/v1/images/generations` 或 `/v1/images/edits`，请求体按 VectorEngine 文档映射 `size`。宣发素材三个工作流（游戏首图、详情五图、运营海报）固定使用 `gpt-image-2`，面板模型胶囊为禁用态，不提供 `nanobanana2` 入口；前端按 workflow 同时提交 `outputSize`、`aspectRatio` 和 `imageSize`，其中游戏首图为 `720x540 / 4:3`、详情单图为 `720x1280 / 9:16`、运营海报为 `1280x720 / 16:9`；后端收到 `kind: "publication-material"` 时也强制归一为 `gpt-image-2` 生成和计费，生成回填图层优先使用生成占位的 `originalWidth/originalHeight`，即使上游回包尺寸漂移也不得把宣发素材卡片变成随机 `1:1` 或 `4:3`。纯文本生成走 `/api/editor/images/generations`，重绘在前端读入当前图层图片 Data URL 后走同一图片生成 BFF，并在原图右侧生成一张新图；普通图层重绘作为 `quick-edit` 参考图提交，角色图层重绘必须按 `kind: "character"` 提交，继续套用角色生成器提示词限定、透明 PNG 后处理和角色资产持久化。`生成视频` 走 `/api/editor/videos/generations`，前端模型入口仅展示 Seedance 2.0 Fast / Seedance 2.0 / Kling 3.0 / Kling 3.0 Omni，不展示 Veo 入口，默认 Seedance 2.0 Fast；视频参数按当前正式面板支持的比例、时长、清晰度和声音开关提交，且 Seedance Fast 与 Seedance 标准版必须按各自真实模型 ID 独立映射，不得混用。生成结果以视频图层加入画布。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成图片后以当前视口世界中心为目标，经统一 placement 避让后创建选中的灰色占位框，输入框跟随占位框显示；待生成、生成中和失败后保留的占位图都必须继续支持拖动，生成完成时真实生成图或视频落在最新占位框位置，输入框继续跟随新生成图层；占位图失焦时隐藏高亮边框、左上角生成器名称和右上角原始尺寸，重新聚焦时再显示，且名称 / 尺寸在画布缩小时按 viewport 反向缩放保持屏幕尺寸稳定；点击所有图片 / 视频生成入口并确认请求开始后，必须隐藏对应设置面板，只保留画布内占位图或原图预览，并在预览上显示 Lovart 式生成中遮罩，避免“面板仍占屏”或“预览一起消失”。图片快速编辑和重绘在调用图片 BFF 前必须把当前图层图片源读取为图片 Data URL；视频素材快速编辑走视频生成 BFF，不允许走图片模型；角色动作的 `生成动画` 仍固定使用 `seedance2.0-fast` 动作 / 视频模型，角色动作素材的 `快速编辑` 按当前帧图片走图片编辑。前端不持有 provider 密钥；上游失败或配置缺失时恢复当前生成设置面板展示失败，不创建 mock 成功图。 +- 图片画布抠图分两类：手动去除背景面向用户任意图片，走登录态同源 BFF `POST /api/editor/images/background-removals` 并转发远端 BiRefNet；编辑器自己生成的标准纯色背景抠图资产统一走 `server-rs/crates/api-server/src/editor_green_screen.rs` 的纯色背景提示词契约和本地确定性透明化，字节级解码、透明化和 PNG 编码下沉复用 `platform-image::generated_asset_sheets`。角色形象生成、图标 spritesheet 生成和 UI 设计图素材提取都必须把所选 `screenColor` 注入 prompt、请求体和 `generationInputs.fields` 的 `抠图背景色`；后端只接受上述六个 hex，缺省或空值按 `#CFEFFF` 处理，未知值返回 `400`。本地透明化用所选颜色作为 key color，且非绿色 key color 不启用近白背景清理，避免浅色主体被误删。后端在执行透明化前必须先把带纯色背景 / 绿幕源图写入 OSS。角色动作抽帧后的序列帧暂不接用户背景色选择，仍沿用 legacy `#00FF00` 绿幕和近白兜底，不依赖 BiRefNet。BiRefNet 服务地址为 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL/remove-background`，默认 `http://58.87.105.82/remove-background`；可选访问令牌只来自服务端环境变量 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`，前端不持有令牌。api-server 对上游结果做响应字节和图片尺寸上限保护，并先落 OSS / asset object，再返回 `imageSrc/objectKey/assetObjectId/taskId`；queue 模式下手动去背景进入 SpacetimeDB 外部生成队列，画布任务侧栏只展示服务器任务阶段，生成中才显示耗时，不显示百分比；有项目上下文时前端同时创建去背景生成占位并把 `canvasCompletion` 交给后端，完成后由后端写入结果图层和最新项目快照。 +- 图片快速编辑面板只保留一个提示词输入框和模型选择，不展示额外参考图或比例 / 尺寸控件；原图 / 原素材作为 `/api/editor/images/edits` 的 `sourceImageSrc` 直接提交，不作为 `referenceImageSrcs`。打开快速编辑时画布必须自动平移缩放，让原素材完整落在可视区上半部分，底部面板固定出现在素材下方且不遮挡内容，竖屏 UI 素材也必须完整展示。快速编辑右侧显示矩形、椭圆、画笔框选工具，但进入时不默认启用；点击工具后显示选中态，再点同一工具取消启用。完成框选后，画布红色细框显示连续序号，提示词可按这些编号填写每个区域怎么改。点击 `修改` 后仍停留在当前快速编辑面板显示修改中，不创建独立 `Quick Edit Generator` 画布占位；生成成功后直接用结果覆盖原图图层，失败时保留当前面板并显示错误。 - 底部生成类按钮每次点击都必须创建独立的画布生成对象；新建规范、角色形象或图标素材时，只切换当前编辑面板，不得销毁此前尚未生成或已生成后的其它生成对象状态。归档为非当前编辑对象的生成占位仍可拖动、删除和等待异步完成，完成 / 失败回写必须按生成对象 ID 读取最新占位状态，不能使用提交瞬间的旧快照。 - 画布右上角提供自动隐藏任务侧栏。列表为空且侧栏关闭时只保留图标开关；生成或去背景任务进入时默认打开；用户可手动切换开关状态。 -- 所有会新建画布生成占位的入口必须先创建 draft，再统一经过 `ImageCanvasGenerationPlacementModel` 计算落点，禁止各入口自行使用当前视口中心裸坐标或原图右侧固定偏移。当前覆盖入口包括 `生成图片`、`生成规范`、`生成角色形象`、`生成图标素材`、`生成视频`、`生成UI设计图`、`生成角色动作` 和快速编辑提交后创建的 `Quick Edit Generator`。placement 模型的避让对象为所有未隐藏画布图层，以及当前 active / inactive generation dialogs 中仍存在的 placeholder；每个避让矩形按 32px 画布世界坐标间距外扩。候选落点以当前视口世界中心为距离目标，优先选择离视口中心最近且不重叠的占位位置；若中心被占用，会按上下左右和环形候选继续寻找。打开生成面板时必须把避让后的 placeholder 写入 `openCanvasGenerationDialog(...)`，并立即调用 `centerViewportOnPlacement(...)` 居中到新占位中心，保持原 viewport scale 不变；快速编辑提交时同样必须把避让后的 placeholder 写入独立生成占位，生成结果落在该占位位置。 +- 所有会新建画布生成占位的入口必须先创建 draft，再统一经过 `ImageCanvasGenerationPlacementModel` 计算落点，禁止各入口自行使用当前视口中心裸坐标或原图右侧固定偏移。当前覆盖入口包括 `生成图片`、`生成规范`、`生成角色形象`、`生成图标素材`、`生成视频`、`生成UI设计图` 和 `生成角色动作`。placement 模型的避让对象为所有未隐藏画布图层，以及当前 active / inactive generation dialogs 中仍存在的 placeholder；每个避让矩形按 32px 画布世界坐标间距外扩。候选落点以当前视口世界中心为距离目标，优先选择离视口中心最近且不重叠的占位位置；若中心被占用，会按上下左右和环形候选继续寻找。打开生成面板时必须把避让后的 placeholder 写入 `openCanvasGenerationDialog(...)`，并立即调用 `centerViewportOnPlacement(...)` 居中到新占位中心，保持原 viewport scale 不变；图片快速编辑不属于新建占位入口，提交后覆盖源图。 ## 交互规则 @@ -52,7 +52,7 @@ - 图片、音频、视频和角色动画帧文件本体继续走 OSS / asset object；浏览器读取私有 generated 对象统一经 `/api/assets/read-url` 换签，签名 URL 可在 session 内复用，但不得作为持久化真相。`/api/assets/read-url` 属于页面展示层高频后台请求，前端统一在 `assetReadUrlService` 内做同 key pending 去重、session 缓存和跨组件节流；UI 设计切片、角色动画帧或大量素材恢复时不得绕过该服务并发换签，否则单页可在同一秒内打满发布入口 `genarrative_api_rps` burst。 - 登录态上传和生成结果必须先落 OSS / asset object，再向 `editor_project_resource` / `editor_asset` 写入轻量 `imageSrc: "/"`、`objectKey` 和 `assetObjectId`；未登录演示态可以在内存里使用 Data URL 预览，但项目、素材库、项目资源和 `editor_canvas.layers_json` 不得写入 `data:image/*`、`data:video/*`、`data:audio/*` 或 `blob:`。旧数据读取时如果已有 `objectKey`，`imageSrc` 归一成 `/`；没有 `objectKey` 的旧 Data URL 需要走修复上传并回写轻量引用。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行；没有当前工程 ID 时才创建账号级 `editor_asset` 行，随后把对应 `resourceId` 或 `assetId` 写入参考图临时状态，生成请求仍使用临时状态中的图片源或 `objectKey`。 - 资源表保存资源和素材级元数据；图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组：普通图层按 `layerId/resourceId` 保存，生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存，不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON；刷新时优先从 `editor_project_resource` 恢复，旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`；角色、图标等纯色抠图生成器还必须保存并恢复 `screenColor`，同源重绘可从生成器快照或 `generationInputs.fields` 的 `抠图背景色` 恢复该值；宣发素材生成器还必须保存并恢复 `publicationWorkflowId`、`publicationGameInfo` 和 `publicationReferences`，避免刷新后生成卡片字段或参考图丢失。生成器快照中的参考图同样只保存 `resourceId/sourceAssetId` 行引用和展示所需 label，不保存图片 Data URL、signed URL 或 `objectKey`；刷新时用 `editor_project_resource` / `editor_asset` 行恢复临时生成请求所需图片源。生成成功后仍保存该快照，只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。`generationInputs.references` 是用户可见输入快照中的行级索引，只允许保存 `{ title, label, refType, refId }`；生成接口所需的图片 Data URL、signed URL 或 `objectKey` 只存在于提交前的临时参考图状态和请求体字段，不进入资源 / 素材元数据。图层展示尺寸不再作为独立 `Size` 真相保存，刷新与新建图层均按 `Resolution`（`originalWidth/originalHeight`）原分辨率显示。图层组第一版是画布内布局语义，不单独建表。 -- 图片类和生成视频结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下，画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`，前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照；`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底，不再把同一生成结果二次调用素材创建接口。生成请求失败、inline 完成或 queue 任务终态完成 / 失败后，右上角泥点 chip 必须通过 `/profile/dashboard` 回读余额，不做本地乐观扣减。生成视频会单独抽取首帧封面并写入 `thumbnailSrc`，素材栏和拖回画布时沿用该封面作为 poster。 +- 图片类、生成视频和音频结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下，画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`，前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照；`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底，不再把同一生成结果二次调用素材创建接口。生成请求失败、inline 完成或 queue 任务终态完成 / 失败后，右上角泥点 chip 必须通过 `/profile/dashboard` 回读余额，不做本地乐观扣减。生成视频会单独抽取首帧封面并写入 `thumbnailSrc`，素材栏和拖回画布时沿用该封面作为 poster。 - 前端不直接订阅 SpacetimeDB，统一通过 api-server 的 `/api/editor/projects*` BFF 读写。 - 工程刷新恢复可先应用 session 级轻量项目快照缓存，让画布和素材 chrome 尽快显示；缓存快照必须排除 `data:*` / `blob:` 内联媒体，且在后端项目快照返回前不得触发自动保存。后端快照回来后覆盖本地缓存显示并恢复正常保存队列。 - 未登录用户可以使用本地演示态，但不触发工程自动保存；真实图片生成 / 修改需要登录。编辑器 API 请求允许使用 refresh cookie 静默补 access token，但 401 / 403 只在编辑器局部提示登录，不清空整站登录态，也不把后端 requestId 直接作为生图弹窗主文案。 @@ -78,8 +78,9 @@ - `POST /api/editor/images/background-removals`：接收当前图片源，校验登录态后由 api-server 解析为图片文件并转发到 BiRefNet 去背景服务；请求可携带 `projectId`、`targetLayerId`、`assetFolderId`、`assetLabel`、`sourceResourceId` 和 `canvasCompletion`，有 `canvasCompletion` 时完成后按生成占位写入结果图层，否则沿用旧的目标图层替换路径；响应返回 `imageSrc`、`objectKey`、`assetObjectId`、`width`、`height`、`taskId`、`elapsedMs`、`provider` 和可选 `project` 快照。服务地址由 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL` 配置，令牌只在服务端通过 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 注入。 - `POST /api/editor/icon-spritesheets/generations`：按图标规范图和素材描述数组生成 spritesheet，生成成功后 api-server 先保存带纯色背景 spritesheet 源图，再走 `editor_green_screen` 透明化后处理，并由后端切分为独立透明图标。请求支持 `model`、`screenColor`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置中对应生图模型的尺寸档位（如 `nanobanana2` 的 `0.5K / 1K / 2K` 或 `gpt-image-2` 的 `1K / 2K`），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。后端把 spritesheet 和拆分后的 icon 都保存为 project resource / 账号素材，并随响应返回对应快照。 - `POST /api/editor/ui-designs/assets/extractions`：以前端已绘入红色框选轮廓的 UI 设计图 Data URL 作为参考图，固定 `gpt-image-2` 和 `editor_green_screen` 组装的纯色背景素材提取提示词生成素材 spritesheet，生成成功后 api-server 先保存带纯色背景 spritesheet 源图，再走 `editor_green_screen` 透明化后处理，并按连通域自动拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求必须携带 `screenColor`、`aspectRatio: "1:1"`、`imageSize: "1K" | "2K"` 和 `priceMudPoints`；框选数量不超过 6 个时前端按 `1:1·1K` 与 gpt-image-2 1K 价格提交，超过 6 个时按 `1:1·2K` 与 2K 价格提交。后端必须在调用上游前校验比例、尺寸和泥点价格，只允许 `1:1 / 1K / 2K`。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，后端保存 spritesheet / 拆分素材并返回对应 resource / asset 快照；前端必须把 spritesheet 原图与拆分素材都加入画布。 -- `POST /api/editor/images/edits`：按提示词、当前图片 Data URL 和最多 8 张额外参考图调用 VectorEngine edits，返回新的生成图片元数据；请求携带 project / asset 上下文时由后端创建新 resource / asset，前端只消费响应快照。 +- `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据；接口能力仍可接收明确参考图，但图片快速编辑当前只提交 `sourceImageSrc`，不提交隐藏的 `referenceImageSrcs`。请求携带 project / asset 上下文时由后端创建新 resource / asset，前端只消费响应快照。 - `POST /api/editor/videos/generations`：按视频描述、模型、比例、时长、分辨率、模式、声音、默认联网搜索标记和泥点价格生成视频。前端可选模型为 `seedance2.0-fast`、`seedance2.0`、`kling3.0`、`kling3.0-omni`，默认 `seedance2.0-fast`；后端必须将 `seedance2.0-fast` 映射到 `doubao-seedance-2-0-fast-260128`，将 `seedance2.0` 映射到 `doubao-seedance-2-0-260128`，两者不得混用。后端允许 6 类比例、4 到 15 秒整数、`480p / 720p / 1080p`，并拒绝 `seedance2.0-fast + 1080p`；`sound=on/off` 映射 Ark `generate_audio=true/false`。后端复用 Ark / VectorEngine content generation task 轮询链路，下载最终视频并持久化到 OSS；请求携带 `projectId` / `assetFolderId` 时同步创建 project resource / 账号素材并返回 `project` / `asset` 快照，基础响应返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`。 +- `POST /api/editor/audios/sound-effects/generations` 与 `POST /api/editor/audios/background-music/generations`：按音效 / 背景音乐参数生成音频并持久化到 OSS；请求携带 `projectId` / `assetFolderId` 时同步创建 project resource / 账号素材并返回 `project` / `resource` / `asset` 快照，基础响应返回 `audioSrc`、prompt、model、provider、taskId、duration、歌词和 `priceMudPoints`。 所有写接口都必须校验 Bearer 登录态和 owner；接口只返回当前用户有权读取的工程与资源。 @@ -105,13 +106,13 @@ - 画布多选语义必须同时覆盖普通图层和仍显示占位框的生成器对象：Shift 点选或框选可把生成器加入当前选择；拖动任一已选图层或生成器时，所有已选普通图层和生成器占位框同步移动；删除 / Backspace / Delete 作用于完整选择集合，移除所有已选图层和生成器对象。生成器对象在选择集合中使用稳定 `generation-dialog:` 目标 ID，不把生成器伪装成普通图层，也不新增后端表。 - 生成类入口打开画布内面板时，底部 AI 工具栏必须保持可见；`生成规范`、角色 / 图标规范来源、角色常规参考图来源这类轻量菜单通过页面级 fixed portal 渲染，不能留在底部工具栏或参考图横向滚动容器内部，避免被局部 `overflow` 裁切。角色规范和常规参考图来源菜单必须向上弹出；常规参考图点击后先选择“从画布中选择”或“上传图片”，从画布取图时只绑定参考图，不触发普通画布图层选中、聚焦、面板隐藏或拖拽逻辑，绑定后退出画布选择状态。所有生成面板参考图槽位统一为方形图标组件；角色规范槽位只显示规范 logo 和 `角色规范` 四字，绑定来源标题只保留给可访问名称、悬浮 title 和图片信息。已有参考图槽位只有在 hover / focus 时显示右上角 `×`，点击后只解绑对应参考图。角色形象生成面板每次成功绑定角色规范后，在当前编辑器生命周期内缓存为上一张角色规范；再次新建角色形象时自动带入该缓存。图标素材和 UI 设计图面板每次成功绑定图标规范后，同样缓存为上一张图标规范；再次新建需要图标规范的素材时自动带入该缓存。生成规范菜单里的图标规范对象自身只把首行参考图作为可选参考，不要求必须先绑定图标规范。 - 生成规范类图片面板底部必须以禁用态参数按钮显示 `16:9·2K` 和 `gpt-image-2`，视觉对齐可编辑面板参数控件，提交到 `/api/editor/images/generations` 时也固定携带这些参数。 -- 快速编辑面板底部必须显示当前选择的比例 / 尺寸和模型，参数按钮可展开修改且视觉对齐其它可编辑面板；打开时视口聚焦必须预留底部面板空间，面板位于素材下方，不得遮挡原素材，且素材在当前屏幕内完整可见。快速编辑请求只把原图作为 `sourceImageSrc` 直接提交，信息面板输入快照只展示用户额外选择的参考图。 +- 快速编辑面板底部只显示模型选择和 `修改` 按钮；打开时视口聚焦必须预留底部面板空间，面板位于素材下方，不得遮挡原素材，且素材在当前屏幕内完整可见。快速编辑请求只把原图或红框序号标注图作为 `sourceImageSrc` 直接提交，信息面板输入快照只展示用户填写的快速编辑提示词。 - 点击生成、生成规范、生成角色形象或生成图标素材后创建的占位图可继续保留；点击画布空白区域让当前图片或占位图失焦时，关闭当前生成面板并移除图片选中样式，但不删除占位图本身。 - 生成资源显示元数据按钮，元数据窗口展示来源、生成输入快照、model、task、Resolution 和 OSS 引用；生成输入快照只包含用户面板输入和参考图行引用，不包含后端拼接 Prompt，不再展示独立 Size 字段，也不渲染参考图 Data URL 缩略图。 - 图片选中后的浮动工具栏按钮顺序固定为：快速编辑、分割线、裁扩按钮、去除背景按钮、UI设计图专属提取素材、角色图专属生成动画、分割线、重绘、下载按钮。裁扩通过画布边界拖拉完成，不再展示四边数值输入；默认自由比例，选择固定比例后拖拉边界保持对应比例，完成后在原素材旁边新增裁扩结果图层，扩展区域透明填充。去除背景调用同源 BFF `POST /api/editor/images/background-removals`，由 api-server 代理远端 BiRefNet 服务并持久化结果；有项目上下文时先在画布创建关闭面板的去背景生成占位，完成后由后端通过 `canvasCompletion` 把新 project resource 写入该占位并返回快照，无占位上下文时才用新的 project resource 引用替换当前图层。画布任务侧栏按“排队/生成中”和“已完成”分页，生成中排在排队前，生成中耗时从任务开始时间戳实时计算，排队中不计时；进行中任务只显示阶段文本和已用时，不显示百分比；完成态生成任务副标题显示用户提示词并单行截断；点击任务只聚焦对应画布内容，不激活生成面板或改变任务顺序，聚焦时必须预留图片上方工具栏、底部工具栏和可见生成对话框空间。UI设计图的提取素材必须先进入红框素材框选状态，默认启用矩形框选，右侧框选工具与快速编辑统一且可再次点击取消启用态，当前启用工具按钮必须保持高亮。素材提取面板必须在素材下方，使用与生成新素材一致的面板宽度和底部模型 / 按钮样式，提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选区域截图预览、固定模型 `gpt-image-2`、左下角计划规格 `1:1·1K/2K` 和 `提取 · N泥点` 按钮，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。至少框选一个区域后才可提交，前端把红色轮廓绘入原图后固定走 `gpt-image-2` 和提示词 `仅提取被红色框框选的素材并整理成spritesheet，图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕。绿幕背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具，方便后续扣除背景；素材自身不要出现绿色描边、绿色底板、绿色投影或绿色反光。`，生成的 spritesheet 原图和拆分后的独立素材都作为画布图层保留。 - 重绘生成资源后，右侧出现新生成结果图层，并自动 fit 原图 + 新图，且重绘面板保持打开。 - 快速编辑 / 重绘站内 public 示例图、历史 generated 图或 OSS generated 图时，前端先读取成 `data:image/*;base64,...` 再提交，后端不得再收到 `/creation-type-references/*`、`/generated-*` 或 OSS URL 作为 `referenceImageSrcs/sourceImageSrc`。 -- 快速编辑的额外参考图既进入输入快照，也作为 `/api/editor/images/edits` 的 `referenceImageSrcs` 提交给后端；原图仍只作为 `sourceImageSrc`，不占用额外参考图额度。 +- 快速编辑不保留额外参考图入口；点击修改时只把原图或红框序号标注图作为 `/api/editor/images/edits` 的 `sourceImageSrc` 提交给后端。 - 素材文件夹可以新建、折叠、重命名和删除；删除普通文件夹后，其素材移动到“项目素材”。普通上传默认落入“上传素材”文件夹；素材库缺少该文件夹时，前端在首次普通上传前创建一次并复用，拖到指定文件夹或点击指定文件夹上传时仍进入目标文件夹。 - 上传按钮和拖拽上传都支持多文件；底部工具栏的上传入口选择文件后直接进入“上传素材”并在当前画布视口中心创建画布图层，素材栏文件夹内的上传入口只写入对应素材文件夹、不自动入画布；拖到文件夹或该文件夹内素材时进入目标文件夹；拖到画布时进入“上传素材”并在投放点创建画布图层。上传图片必须在创建占位素材、画布图层和账号级素材记录前先读取原图 Resolution，图层宽高、`originalWidth/originalHeight` 和素材库 `width/height` 都使用图片本身尺寸；上传视频同样在创建素材和图层前读取视频 metadata 宽高，保证单层下载或 ZIP 导出的真实视频文件重新导入后仍按文件自身尺寸入画布；仅在无法解析尺寸时才使用对应媒体兜底尺寸。 - 音频 / 视频素材卡和画布媒体图层必须提供稳定的非文字视觉预览：优先使用 `thumbnailSrc` / 视频 `poster`，没有真实首帧或音频封面时使用由媒体类型、素材名和地址派生的确定性视觉底图。视频图层使用原生 `