refactor: extract platform image provider

This commit is contained in:
2026-05-25 19:03:43 +08:00
parent 0ffbea67fd
commit 080694fb46
15 changed files with 1708 additions and 1546 deletions

View File

@@ -121,6 +121,14 @@
- 验证方式:执行 `cargo test -p api-server external_api_audit --manifest-path server-rs/Cargo.toml -- --nocapture``cargo test -p api-server openai_image_generation --manifest-path server-rs/Cargo.toml -- --nocapture``cargo check -p api-server --manifest-path server-rs/Cargo.toml``npm run check:encoding`
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-05-25 VectorEngine 图片 provider 收到 platform-image
- 背景:`api-server` 里原本同时混着 VectorEngine 创建 / 编辑协议、响应解析、远端图片下载、失败日志和审计落库逻辑Puzzle / Match3D 还各自藏着一份近似实现导致“provider 协议”和“业务编排”边界不清。
- 决策:把 VectorEngine `gpt-image-2` 图片 provider 协议、URL / base64 响应解析、远端图片下载和 provider 侧结构化日志统一收口到 `server-rs/crates/platform-image``api-server` 只保留配置校验、玩法 prompt 编排、OSS / asset object / binding 持久化、计费和外部 API 失败审计桥接;旧 `openai_image_generation.rs` 只作为兼容转接层,不再承担 provider 实现。
- 影响范围:`server-rs/crates/platform-image``server-rs/crates/api-server/src/openai_image_generation.rs``server-rs/crates/api-server/src/puzzle/vector_engine.rs``server-rs/crates/api-server/src/external_api_audit.rs`、后端架构与运维文档。
- 验证方式:`cargo test -p platform-image --manifest-path server-rs/Cargo.toml``cargo test -p api-server openai_image_generation --manifest-path server-rs/Cargo.toml -- --nocapture``cargo test -p api-server puzzle --manifest-path server-rs/Cargo.toml -- --nocapture``cargo check -p api-server --manifest-path server-rs/Cargo.toml``npm run check:encoding`
- 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-05-21 拼图参考图主链改为 OSS assetObjectId 与只读签名 URL
- 背景release 上拼图图生图生成草稿时,旧链路把上传图转成 Data URL/base64 放进创作 action JSON body容易先触发 Nginx `413 Request Entity Too Large`,也让外部模型调用前的 HTTP body 过大。

View File

@@ -234,6 +234,14 @@
- 验证:`SELECT event_id, scope_id AS provider, metadata_json, occurred_at FROM tracking_event WHERE event_key = 'external_api_call_failure' ORDER BY occurred_at DESC LIMIT 50;`;如果查不到同时看 tracking outbox 目录权限和 sealed 文件是否堆积。
- 关联:`server-rs/crates/api-server/src/external_api_audit.rs``server-rs/crates/api-server/src/openai_image_generation.rs``docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## VectorEngine 图片协议先看 platform-image不要先翻 puzzle.rs
- 现象:排查拼图或其它玩法的生图失败时,如果直接在 `api-server` 的大文件里找 `images/generations``images/edits`、base64 解码或下载逻辑,会看到很多历史 helper 和测试桥,看起来像每个玩法都自带一份 provider 实现。
- 原因:旧实现把 VectorEngine 图片 provider 协议、响应解析、下载和日志混在 `api-server` 里,后来虽然迁出到 `platform-image`,但兼容层和测试 helper 仍会让人误判真相源位置。
- 处理:先看 `server-rs/crates/platform-image/src/lib.rs` 的 provider 协议和结构化日志,再看 `server-rs/crates/api-server/src/openai_image_generation.rs` 的兼容桥和 `external_api_audit.rs` 的落库映射;`puzzle/vector_engine.rs` 只保留玩法编排,不再作为 provider 协议真相源。
- 验证:`cargo test -p platform-image --manifest-path server-rs/Cargo.toml``cargo test -p api-server openai_image_generation --manifest-path server-rs/Cargo.toml -- --nocapture``cargo test -p api-server puzzle --manifest-path server-rs/Cargo.toml -- --nocapture` 通过时,排障先按 `platform-image` 的日志字段查 provider / endpoint / failure_stage。
- 关联:`server-rs/crates/platform-image/src/lib.rs``server-rs/crates/api-server/src/openai_image_generation.rs``server-rs/crates/api-server/src/external_api_audit.rs``server-rs/crates/api-server/src/puzzle/vector_engine.rs`
## release 创作接口 413 先查是否还在提交 Data URL
- 现象release 上 `POST /api/runtime/puzzle/agent/sessions/{session_id}/actions` 携带参考图 Data URL 时返回 `413 Request Entity Too Large`access log 显示 `request_time=0.000``upstream_status=-`

View File

@@ -20,7 +20,7 @@ server-rs + Axum + SpacetimeDB
- HTTP 服务:`api-server`
- 领域模块:`module-ai``module-assets``module-auth``module-bark-battle``module-big-fish``module-combat``module-creative-agent``module-custom-world``module-inventory``module-match3d``module-npc``module-progression``module-puzzle``module-quest``module-runtime``module-runtime-item``module-runtime-story``module-square-hole``module-story``module-visual-novel`
- 平台副作用:`platform-agent``platform-auth``platform-llm``platform-oss``platform-speech`
- 平台副作用:`platform-agent``platform-auth``platform-image``platform-llm``platform-oss``platform-speech`
- 共享层:`shared-contracts``shared-kernel``shared-logging`
- SpacetimeDB`spacetime-client``spacetime-module`
- 测试支撑:`tests-support`
@@ -117,9 +117,10 @@ npm run check:server-rs-ddd
2. Adapter 输入应显式包含 provider、prompt、reference images、OSS prefix/path/file name、asset kind、entity kind/id、slot、owner/profile/source job、metadata 和可选透明背景后处理。
3. Adapter 输出应保留 legacy public path、object key、asset object id、MIME、extension、task id 和实际 prompt。
4. Adapter 不负责扣费、退款或钱包读取;计费仍由调用方显式包裹。
5. Puzzle、Match3D、音频、GLB、视频等复杂媒体可以复用 OSS + asset object + binding 的底层持久化能力,但玩法专属处理规则留在各自编排层,不塞进公共接口
6. 拼图入口页与结果页新增关卡的本地参考图不走浏览器直传 OSS前端读取为 Data URL 后随创作 action 提交,并在读取前限制 6MB、显示“图片≤6MB”。`api-server` 必须对 Data URL 实际字节数再次校验;历史图片才提交 `referenceImageAssetObjectId(s)`,后端校验 `asset_object` 的 bucket、kind、图片 MIME、大小和 owner 后签发只读 URL 给 VectorEngine 读取
7. 系列素材图集使用 `server-rs/crates/api-server/src/generated_asset_sheets.rs`:调用方必须传入 `grid_size` 作为 `n*n``n`,可选传入物品名称 prompt 模板和特殊设定 prompt模块负责 sheet prompt 组装、按 `n*n` 切片、透明化、PNG 输出、OSS private upload 请求构造和 sheet / item / special prompt 元数据持久化。玩法只负责规划 slot、调用具体生图 provider、计费、失败回写以及把通用切片结果映射回自己的 DTO / 草稿 / runtime 字段
5. 图片 provider 协议不再放在玩法模块里实现。VectorEngine `gpt-image-2` 创建 / 编辑协议、URL / base64 图片解析、远端图片下载、请求超时 / 上游状态 / 响应解析 / 缺图 / 下载失败的结构化日志统一在 `server-rs/crates/platform-image``api-server` 只负责配置校验、玩法 prompt 编排、OSS / asset object / binding 持久化、计费和外部 API 失败审计落库
6. Puzzle、Match3D、音频、GLB、视频等复杂媒体可以复用 OSS + asset object + binding 的底层持久化能力,但玩法专属处理规则留在各自编排层,不塞进公共接口
7. 拼图入口页与结果页新增关卡的本地参考图不走浏览器直传 OSS前端读取为 Data URL 后随创作 action 提交,并在读取前限制 6MB、显示“图片≤6MB”。`api-server` 必须对 Data URL 实际字节数再次校验;历史图片才提交 `referenceImageAssetObjectId(s)`,后端校验 `asset_object` 的 bucket、kind、图片 MIME、大小和 owner 后签发只读 URL 给 VectorEngine 读取
8. 系列素材图集使用 `server-rs/crates/api-server/src/generated_asset_sheets.rs`:调用方必须传入 `grid_size` 作为 `n*n``n`,可选传入物品名称 prompt 模板和特殊设定 prompt模块负责 sheet prompt 组装、按 `n*n` 切片、透明化、PNG 输出、OSS private upload 请求构造和 sheet / item / special prompt 元数据持久化。玩法只负责规划 slot、调用具体生图 provider、计费、失败回写以及把通用切片结果映射回自己的 DTO / 草稿 / runtime 字段。
## SpacetimeDB schema 变更规则
@@ -156,7 +157,7 @@ npm run check:server-rs-ddd
## 外部服务与资产
- LLM`GENARRATIVE_LLM_*`,创意 Agent 另用 `APIMART_BASE_URL` / `APIMART_API_KEY`
- 图片生成VectorEngine / APIMart / DashScope密钥只在后端环境变量中
- 图片生成VectorEngine `gpt-image-2` 图片 provider 归属 `platform-image`,密钥只在后端环境变量中;`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接,不再承载 provider 协议实现。APIMart 只保留给创意 Agent `gpt-5` Responses 文本 / 多模态链路DashScope 只按仍在使用的历史能力单独处理,不作为 GPT-image-2 兜底
- Match3D 物品 sheet关卡整图完成后走 VectorEngine `/v1/images/edits` multipart `image`,模型为 `gpt-image-2``2K 1:1` 输出 `10*10` spritesheet物品 sheet prompt 固定要求纯绿色绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG并把透明整图写入 `itemSpritesheetImageSrc/itemSpritesheetImageObjectKey`。后端固定从该 sheet 解析并持久化 20 个物品、每个 5 个形态;通用系列素材图集的行列索引按每行 2 个物品计算,必须落在 `1..=10`,难度只决定运行态加载 3 / 9 / 15 / 20 种。
- Match3D UI spritesheet 和背景派生图:关卡整图作为参考图并发生成 `1K 1:1` UI spritesheet 与 `1K 9:16` 背景图,模型均为 `gpt-image-2`。UI spritesheet prompt 固定要求纯绿色绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG背景图必须合成为全画幅不透明 PNG。
- Match3D 1:1 容器 UIVectorEngine `/v1/images/edits` multipart 参考图。该容器参考图是后端生图协议输入,必须通过 `include_bytes!``api-server` 编译进二进制,避免 API 单独发布或运行目录缺少 `public/` 时生成失败。
@@ -164,7 +165,7 @@ npm run check:server-rs-ddd
- Hyper3D / Rodin只保留后端安全代理和旧数据兼容新 Match3D 草稿和批量新增不再生成 GLB。
- 音频:视觉小说专用音频路由保留;拼图、抓大鹅和敲木鱼提示词生成音效入口暂时关闭,通用 `/api/creation/audio/*` 对这些目标返回 `410 Gone`。敲木鱼创作只接收上传 / 录音音频资产;未提供时由 `api-server` 写回内置默认木鱼音 `/wooden-fish/default-hit-sound.mp3`
- OSS私有 generated legacy path 进入浏览器前必须通过 `/api/assets/read-url` 换签;不要裸请求 `/generated-*`
- 外部 API 失败审计:外部供应商调用未成功时,`api-server` 必须发送 OTLP 失败事件并写入 `tracking_event`当前通用 VectorEngine `gpt-image-2-all` 图片生成 / 编辑适配器在 `request_send``response_body``upstream_status``response_parse``missing_image``image_download` 阶段失败时记录 `external_api_call_failure``scope_kind = module``scope_id = provider``module_key = external-api`metadata 固定包含 provider、endpoint、operation、failureStage、statusCode、statusClass、timeout、retryable、errorMessage、latencyMs、promptChars、referenceImageCountimageModel。入库优先复用 tracking outboxoutbox 不可写或保护阈值拒绝时回退同步写 SpacetimeDB不得新增前端兜底或在 SpacetimeDB reducer 内做外部 I/O。
- 外部 API 失败审计:外部供应商调用未成功时,`api-server` 必须发送 OTLP 失败事件并写入 `tracking_event`。VectorEngine 图片 provider 在 `platform-image` 内输出结构化日志和 `PlatformImageFailureAudit`,覆盖 `request_send``response_body``upstream_status``response_parse``missing_image``image_download` 阶段`api-server` 只把该 audit 映射成 `external_api_call_failure``scope_kind = module``scope_id = provider``module_key = external-api`metadata 固定包含 provider、endpoint、operation、failureStage、statusCode、statusClass、timeout、retryable、errorMessage、latencyMs、promptChars、referenceImageCountimageModel 和 rawExcerpt。入库优先复用 tracking outboxoutbox 不可写或保护阈值拒绝时回退同步写 SpacetimeDB不得新增前端兜底或在 SpacetimeDB reducer 内做外部 I/O。
## SpacetimeDB 表目录

View File

@@ -59,7 +59,7 @@ spacetime sql <database> "SELECT * FROM puzzle_gallery_card_view LIMIT 1" --serv
本地 `spacetime` CLI / standalone 版本必须和 `server-rs/Cargo.toml` 里锁定的 `spacetimedb` 版本一致。若版本错配procedure 返回值可能在宿主侧触发 `Failed to BSATN deserialize procedure return value`api-server 最终表现为敲木鱼等创作动作的 `SpacetimeDB procedure 调用超时`。排障时先运行 `spacetime --version`,再对照 `server-rs/Cargo.toml``spacetimedb = "..."`;需要切版本时执行 `spacetime version install <version> && spacetime version use <version>`,然后重新启动 `npm run dev:spacetime`。当前 `scripts/dev.mjs` 会在启动和复用本地 SpacetimeDB 前写入并校验 `dev-spacetime-tool-version`,避免把旧 standalone 继续带进新一轮创作。
本地 `.env``.env.local``.env.secrets.local` 修改后必须重启 `api-server` 才会生效;若已经通过 `npm run dev` 启动完整联调,可在该终端输入 `rs api-server`。排查 RPG / 拼图 / 抓大鹅等 VectorEngine 生图链路时,确认 `VECTOR_ENGINE_BASE_URL``VECTOR_ENGINE_API_KEY``VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 只在本地或服务器密钥文件中配置,不能写入 Git。开局 CG 故事板、首图、背景和图集都属于长耗时图片请求;后端默认会把 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 下限收口到 `1000000`,旧进程仍可能沿用重启前的短超时。若开局 CG 故事板在 `send()` 阶段失败且日志显示 `SendRequest`,先看同一 request_id 的 `request_body_bytes``reference_data_url_bytes``sourceChain``rootSource`;当前开局 CG 会把角色图与首幕背景图压到单边 768 的 JPEG 后再作为 generations `image` 数组发送,`/v1/images/generations` 使用默认 HTTP 协商,只有 multipart `/v1/images/edits` 单独强制 HTTP/1.1。
本地 `.env``.env.local``.env.secrets.local` 修改后必须重启 `api-server` 才会生效;若已经通过 `npm run dev` 启动完整联调,可在该终端输入 `rs api-server`。排查 RPG / 拼图 / 抓大鹅等 VectorEngine 生图链路时,确认 `VECTOR_ENGINE_BASE_URL``VECTOR_ENGINE_API_KEY``VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 只在本地或服务器密钥文件中配置,不能写入 Git。VectorEngine `gpt-image-2` 图片协议、URL / base64 响应解析、远端图片下载和 provider 侧结构化日志在 `server-rs/crates/platform-image``api-server` 只做配置、玩法编排、OSS / asset 持久化、计费和失败审计落库。开局 CG 故事板、首图、背景和图集都属于长耗时图片请求;后端默认会把 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 下限收口到 `1000000`,旧进程仍可能沿用重启前的短超时。若开局 CG 故事板在 `send()` 阶段失败且日志显示 `SendRequest`,先看同一 request_id 的 `request_body_bytes``reference_data_url_bytes``sourceChain``rootSource`;当前开局 CG 会把角色图与首幕背景图压到单边 768 的 JPEG 后再作为 generations `image` 数组发送,`/v1/images/generations` 使用默认 HTTP 协商,只有 multipart `/v1/images/edits` 单独强制 HTTP/1.1。
查看本地 Rust / SpacetimeDB 日志:
@@ -142,6 +142,7 @@ Codex 项目级 hook 已放在 `.codex/config.toml` 与 `.codex/hooks/`
后端代码修改后,按变更范围选择:
- `cargo test -p <crate> --manifest-path server-rs/Cargo.toml`
- `cargo test -p platform-image --manifest-path server-rs/Cargo.toml`
- `cargo check -p api-server --manifest-path server-rs/Cargo.toml`
- `cargo check -p spacetime-client --manifest-path server-rs/Cargo.toml`
- `cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml`
@@ -250,7 +251,7 @@ OpenTelemetry 现阶段默认开启 OTLP traces / metrics / logs但本地日
- debug exporter / Rider 转发都会同时接收 traces、metrics 和 logs。
- api-server 会随 metrics 发送进程级指标:`process.memory.usage``process.memory.virtual``process.cpu.time``genarrative.process.cpu.usage_percent``process.thread.count``genarrative.process.memory.private`Windows 额外发送 `process.windows.handle.count`Linux 额外发送 `process.unix.file_descriptor.count`。这些指标只描述当前进程,不携带请求、用户或作品 label。
- HTTP 运行态补充发送 `genarrative.http.server.response_bodies.in_flight``genarrative.http.server.request_permits.available`,后者带低基数 `pool=default|gallery|detail|admin` label用于区分业务 handler / 背压 permit 是否仍被占用;拼图广场热点缓存补充发送 `genarrative.puzzle_gallery.cache.*` 指标,记录 fresh hit、stale hit、未命中、后台刷新开始 / 失败、重建耗时和预序列化 data JSON 字节数。
- 外部 API 失败统一发送 OTLP 并落库。当前 VectorEngine `gpt-image-2-all` 图片生成 / 编辑失败会输出 `外部 API 调用失败` trace/log记录指标 `genarrative.external_api.failures{provider,failure_stage,status_class,retryable}`;同时写入 `tracking_event``event_key = external_api_call_failure``module_key = external-api``scope_kind = module``scope_id = provider`。排障时先按 provider / failureStage 聚合,再结合 request 日志和上游响应 excerpt 判断是限流、超时、解析失败还是未返回图片。
- 外部 API 失败统一发送 OTLP 并落库。当前 VectorEngine `gpt-image-2` 图片生成 / 编辑失败`platform-image` provider 输出低基数字段结构化日志,字段包括 provider、endpoint、failure_stage、status、status_class、timeout、retryable、latency_ms、prompt_chars、reference_image_count、image_model 和 raw_excerpt`api-server`记录指标 `genarrative.external_api.failures{provider,failure_stage,status_class,retryable}`,并写入 `tracking_event``event_key = external_api_call_failure``module_key = external-api``scope_kind = module``scope_id = provider`。排障时先按 provider / failureStage 聚合,再结合 request 日志和上游响应 excerpt 判断是限流、超时、解析失败还是未返回图片。
- SpacetimeDB 观测分为两类procedure / reducer 调用继续用 `genarrative.spacetime.procedure.*`,订阅本地 cache 读使用 `genarrative.spacetime.read.*``read=list_puzzle_gallery` 表示拼图广场当前从 `puzzle_gallery_card_view` 本地 cache 读取,不再每个 HTTP 请求调用 `list_puzzle_gallery` procedure。
- 本地 Windows 直连压测的内存高水位要结合 K6 VU / 连接数解释。250 RPS 下过高 `PREALLOCATED_VUS` 可能让 300 个本地 Established 连接把 `api-server` private memory 瞬时推到 GB 级,且 `/healthz` 小响应也能复现;若压测结束后回落、`response_bodies.in_flight` 和背压 permit 未显示业务积压,应优先按连接 / 发送链路高水位处理,而不是判断为 SpacetimeDB 或 JSON 缓存泄漏。
- Rider 的 Logs 面板只展示 log event 自身字段,不会自动展开父 span 的全部 attributes请求完成日志会直接带 `request_id``http.request.method``http.route``url.scheme``url.path``http.response.status_code``status_class``latency_ms``slow_request`,完整链路继续到 Traces 面板按 trace/span 查看。

12
server-rs/Cargo.lock generated
View File

@@ -108,6 +108,7 @@ dependencies = [
"opentelemetry",
"platform-agent",
"platform-auth",
"platform-image",
"platform-llm",
"platform-oss",
"platform-speech",
@@ -2321,6 +2322,17 @@ dependencies = [
"urlencoding",
]
[[package]]
name = "platform-image"
version = "0.1.0"
dependencies = [
"base64 0.22.1",
"reqwest 0.12.28",
"serde_json",
"tokio",
"tracing",
]
[[package]]
name = "platform-llm"
version = "0.1.0"

View File

@@ -32,6 +32,7 @@ members = [
"crates/module-visual-novel",
"crates/platform-oss",
"crates/platform-auth",
"crates/platform-image",
"crates/platform-llm",
"crates/platform-speech",
"crates/platform-agent",
@@ -74,6 +75,7 @@ module-story = { path = "crates/module-story", default-features = false }
module-visual-novel = { path = "crates/module-visual-novel", default-features = false }
platform-agent = { path = "crates/platform-agent", default-features = false }
platform-auth = { path = "crates/platform-auth", default-features = false }
platform-image = { path = "crates/platform-image", default-features = false }
platform-llm = { path = "crates/platform-llm", default-features = false }
platform-oss = { path = "crates/platform-oss", default-features = false }
platform-speech = { path = "crates/platform-speech", default-features = false }

View File

@@ -34,6 +34,7 @@ module-story = { workspace = true }
module-visual-novel = { workspace = true }
platform-agent = { workspace = true }
platform-auth = { workspace = true }
platform-image = { workspace = true }
platform-llm = { workspace = true }
platform-oss = { workspace = true }
platform-speech = { workspace = true }

View File

@@ -1,4 +1,5 @@
use axum::http::StatusCode;
use platform_image::PlatformImageFailureAudit;
use module_runtime::RuntimeTrackingScopeKind;
use serde_json::{Value, json};
use time::OffsetDateTime;
@@ -109,6 +110,28 @@ impl ExternalApiFailureDraft {
}
}
pub(crate) fn build_external_api_failure_draft_from_platform_image_audit(
audit: &PlatformImageFailureAudit,
) -> ExternalApiFailureDraft {
ExternalApiFailureDraft::new(
audit.provider,
audit.endpoint.clone(),
audit.operation.clone(),
audit.failure_stage,
audit.error_message.clone(),
)
.with_status_code(audit.status_code)
.with_optional_status_class(audit.status_class)
.with_timeout(audit.timeout)
.with_retryable(audit.retryable)
.with_error_source(audit.error_source.clone())
.with_raw_excerpt(audit.raw_excerpt.clone())
.with_latency_ms(audit.latency_ms)
.with_prompt_chars(audit.prompt_chars)
.with_reference_image_count(audit.reference_image_count)
.with_image_model(audit.image_model)
}
/// 中文注释下载图片、OSS 读写等非标准 HTTP 状态统一显式归类,避免 OTLP 低基数 label 误落到 `transport`。
pub(crate) fn app_error_status_class(status_code: StatusCode) -> &'static str {
status_class(Some(status_code.as_u16()))

View File

@@ -113,6 +113,7 @@ fn resolve_http_error(status_code: StatusCode) -> (&'static str, &'static str) {
StatusCode::NOT_IMPLEMENTED => ("NOT_IMPLEMENTED", "功能暂未实现"),
StatusCode::CONFLICT => ("CONFLICT", "请求冲突"),
StatusCode::TOO_MANY_REQUESTS => ("TOO_MANY_REQUESTS", "请求过于频繁"),
StatusCode::GATEWAY_TIMEOUT => ("GATEWAY_TIMEOUT", "上游服务请求超时"),
StatusCode::BAD_GATEWAY => ("UPSTREAM_ERROR", "上游服务请求失败"),
StatusCode::SERVICE_UNAVAILABLE => ("SERVICE_UNAVAILABLE", "服务暂不可用"),
_ if status_code.is_client_error() => ("BAD_REQUEST", "请求参数不合法"),

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
use std::{
collections::BTreeMap,
time::{Duration, Instant, SystemTime, UNIX_EPOCH},
time::{Instant, SystemTime, UNIX_EPOCH},
};
use axum::{
@@ -103,7 +103,7 @@ use crate::{
},
puzzle_gallery_cache::{build_puzzle_gallery_window_response, puzzle_gallery_cached_json},
request_context::RequestContext,
state::PuzzleApiState,
state::{AppState, PuzzleApiState},
work_author::resolve_puzzle_work_author_by_user_id,
work_play_tracking::{WorkPlayTrackingDraft, record_puzzle_work_play_start_after_success},
};

View File

@@ -1,5 +1,7 @@
use super::*;
use crate::openai_image_generation::GPT_IMAGE_2_MODEL;
use crate::openai_image_generation::{GPT_IMAGE_2_MODEL, map_platform_image_error};
use platform_image::{PlatformImageError, VECTOR_ENGINE_PROVIDER};
use std::time::Duration;
#[test]
fn puzzle_generated_image_size_is_square_1_1() {
@@ -218,45 +220,6 @@ fn puzzle_vector_engine_create_request_never_embeds_signed_reference_url() {
assert!(body.get("image").is_none());
}
#[test]
fn puzzle_vector_engine_generation_url_normalizes_base_url() {
let settings = PuzzleVectorEngineSettings {
base_url: "https://vector.example/v1".to_string(),
api_key: "test-key".to_string(),
};
assert_eq!(
puzzle_vector_engine_images_generation_url(&settings),
"https://vector.example/v1/images/generations"
);
}
#[test]
fn puzzle_vector_engine_edit_url_normalizes_base_url() {
let settings = PuzzleVectorEngineSettings {
base_url: "https://vector.example/v1".to_string(),
api_key: "test-key".to_string(),
};
assert_eq!(
puzzle_vector_engine_images_edit_url(&settings),
"https://vector.example/v1/images/edits"
);
}
#[test]
fn puzzle_vector_engine_edit_response_decodes_b64_image() {
let images = puzzle_images_from_base64(
"edit-1".to_string(),
vec![BASE64_STANDARD.encode(b"\x89PNG\r\n\x1A\nrest")],
1,
);
assert_eq!(images.images.len(), 1);
assert_eq!(images.images[0].mime_type, "image/png");
assert_eq!(images.images[0].extension, "png");
}
#[test]
fn puzzle_vector_engine_prompt_strongly_uses_reference_image() {
let prompt = build_puzzle_vector_engine_generation_prompt("请生成雨夜猫街。", true);
@@ -379,9 +342,18 @@ fn puzzle_asset_object_reference_requires_matching_owner() {
#[test]
fn puzzle_vector_engine_timeout_maps_to_gateway_timeout() {
let error = map_puzzle_vector_engine_request_error(
"创建拼图 VectorEngine 图片生成任务失败operation timed out".to_string(),
);
let error = map_platform_image_error(PlatformImageError::Request {
provider: VECTOR_ENGINE_PROVIDER,
message: "创建拼图 VectorEngine 图片生成任务失败operation timed out".to_string(),
endpoint: Some("https://vector.example/v1/images/generations".to_string()),
timeout: true,
connect: false,
request: true,
body: false,
status_code: None,
source: None,
audit: None,
});
let response = error.into_response();
assert_eq!(response.status(), StatusCode::GATEWAY_TIMEOUT);
@@ -389,11 +361,14 @@ fn puzzle_vector_engine_timeout_maps_to_gateway_timeout() {
#[test]
fn puzzle_vector_engine_upstream_timeout_maps_to_gateway_timeout() {
let error = map_puzzle_vector_engine_upstream_error(
reqwest::StatusCode::GATEWAY_TIMEOUT,
r#"{"error":{"message":"VectorEngine generation endpoint timeout"}}"#,
"创建拼图 VectorEngine 图片生成任务失败",
);
let error = map_platform_image_error(PlatformImageError::Upstream {
provider: VECTOR_ENGINE_PROVIDER,
message: "VectorEngine generation endpoint timeout".to_string(),
upstream_status: reqwest::StatusCode::GATEWAY_TIMEOUT.as_u16(),
raw_excerpt: r#"{"error":{"message":"VectorEngine generation endpoint timeout"}}"#
.to_string(),
audit: None,
});
let response = error.into_response();
assert_eq!(response.status(), StatusCode::GATEWAY_TIMEOUT);

View File

@@ -1,4 +1,7 @@
use super::*;
use crate::openai_image_generation::{
OpenAiReferenceImage, create_openai_image_edit_with_references,
};
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(crate) enum PuzzleImageModel {
@@ -26,6 +29,8 @@ impl PuzzleImageModel {
pub(crate) struct PuzzleVectorEngineSettings {
pub(crate) base_url: String,
pub(crate) api_key: String,
pub(crate) request_timeout_ms: u64,
pub(crate) external_api_audit_state: Option<AppState>,
}
pub(crate) struct PuzzleGeneratedImages {
@@ -78,6 +83,25 @@ impl PuzzleDownloadedImage {
bytes: image.bytes,
}
}
pub(crate) fn from_openai_image(image: DownloadedOpenAiImage) -> Self {
Self {
extension: image.extension,
mime_type: normalize_puzzle_downloaded_image_mime_type(image.mime_type.as_str()),
bytes: image.bytes,
}
}
}
impl PuzzleVectorEngineSettings {
fn to_openai_settings(&self) -> crate::openai_image_generation::OpenAiImageSettings {
crate::openai_image_generation::OpenAiImageSettings {
base_url: self.base_url.clone(),
api_key: self.api_key.clone(),
request_timeout_ms: self.request_timeout_ms,
external_api_audit_state: self.external_api_audit_state.clone(),
}
}
}
pub(crate) struct ParsedPuzzleImageDataUrl {
@@ -151,27 +175,18 @@ pub(crate) fn require_puzzle_vector_engine_settings(
Ok(PuzzleVectorEngineSettings {
base_url: base_url.to_string(),
api_key: api_key.to_string(),
request_timeout_ms: state.vector_engine_image_request_timeout_ms().max(1),
external_api_audit_state: Some(state.root_state().clone()),
})
}
pub(crate) fn build_puzzle_image_http_client(
state: &PuzzleApiState,
image_model: PuzzleImageModel,
_image_model: PuzzleImageModel,
) -> Result<reqwest::Client, AppError> {
let provider = image_model.provider_name();
let request_timeout_ms = state.vector_engine_image_request_timeout_ms();
let settings = require_puzzle_vector_engine_settings(state)?;
reqwest::Client::builder()
.timeout(Duration::from_millis(request_timeout_ms.max(1)))
// 中文注释:参考图走 multipart edits强制 HTTP/1.1 可降低部分网关对长耗时上传流的兼容风险。
.http1_only()
.build()
.map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
"provider": provider,
"message": format!("构造拼图图片生成 HTTP 客户端失败:{error}"),
}))
})
build_openai_image_http_client(&settings.to_openai_settings())
}
pub(crate) fn to_puzzle_generated_image_candidate(
@@ -213,198 +228,66 @@ pub(crate) async fn create_puzzle_vector_engine_image_generation(
.await;
}
let request_body = build_puzzle_vector_engine_image_request_body(
image_model,
let generated = create_openai_image_generation(
http_client,
&settings.to_openai_settings(),
prompt,
negative_prompt,
Some(negative_prompt),
size,
candidate_count,
reference_image,
);
let request_url = puzzle_vector_engine_images_generation_url(settings);
let request_started_at = Instant::now();
let response = http_client
.post(request_url.as_str())
.header(
reqwest::header::AUTHORIZATION,
format!("Bearer {}", settings.api_key),
)
.header(reqwest::header::ACCEPT, "application/json")
.header(reqwest::header::CONTENT_TYPE, "application/json")
.json(&request_body)
.send()
.await
.map_err(|error| {
map_puzzle_vector_engine_request_error(format!(
"创建拼图 VectorEngine 图片生成任务失败:{error}"
))
})?;
let status = response.status();
let upstream_elapsed_ms = request_started_at.elapsed().as_millis() as u64;
tracing::info!(
provider = VECTOR_ENGINE_PROVIDER,
image_model = image_model.request_model_name(),
endpoint = %request_url,
status = status.as_u16(),
prompt_chars = prompt.chars().count(),
size,
has_reference_image = reference_image.is_some(),
elapsed_ms = upstream_elapsed_ms,
"拼图 VectorEngine 图片生成 HTTP 返回"
);
let response_text = response.text().await.map_err(|error| {
map_puzzle_vector_engine_request_error(format!(
"读取拼图 VectorEngine 图片生成响应失败:{error}"
))
})?;
if !status.is_success() {
return Err(map_puzzle_vector_engine_upstream_error(
status,
response_text.as_str(),
"创建拼图 VectorEngine 图片生成任务失败",
));
}
let payload = parse_puzzle_json_payload(
response_text.as_str(),
"解析拼图 VectorEngine 图片生成响应失败",
)?;
let image_urls = extract_puzzle_image_urls(&payload);
if !image_urls.is_empty() {
let download_started_at = Instant::now();
let images = download_puzzle_images_from_urls(
http_client,
format!("vector-engine-{}", current_utc_micros()),
image_urls,
candidate_count,
)
.await?;
tracing::info!(
provider = VECTOR_ENGINE_PROVIDER,
image_model = image_model.request_model_name(),
image_count = images.images.len(),
elapsed_ms = download_started_at.elapsed().as_millis() as u64,
"拼图 VectorEngine 图片下载完成"
);
return Ok(images);
}
let b64_images = extract_puzzle_b64_images(&payload);
if !b64_images.is_empty() {
return Ok(puzzle_images_from_base64(
format!("vector-engine-{}", current_utc_micros()),
b64_images,
candidate_count,
));
}
Err(
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": VECTOR_ENGINE_PROVIDER,
"message": "拼图 VectorEngine 图片生成未返回图片地址",
})),
&[],
"拼图 VectorEngine 图片生成失败",
)
.await?;
Ok(PuzzleGeneratedImages {
task_id: generated.task_id,
images: generated
.images
.into_iter()
.map(PuzzleDownloadedImage::from_openai_image)
.collect(),
})
}
pub(crate) async fn create_puzzle_vector_engine_image_edit(
http_client: &reqwest::Client,
settings: &PuzzleVectorEngineSettings,
image_model: PuzzleImageModel,
_image_model: PuzzleImageModel,
prompt: &str,
negative_prompt: &str,
size: &str,
candidate_count: u32,
reference_image: &PuzzleResolvedReferenceImage,
) -> Result<PuzzleGeneratedImages, AppError> {
let request_url = puzzle_vector_engine_images_edit_url(settings);
let task_id = format!("vector-engine-edit-{}", current_utc_micros());
let file_name = format!(
"puzzle-reference.{}",
puzzle_mime_to_extension(reference_image.mime_type.as_str())
);
let image_part = reqwest::multipart::Part::bytes(reference_image.bytes.clone())
.file_name(file_name)
.mime_str(reference_image.mime_type.as_str())
.map_err(|error| {
map_puzzle_vector_engine_request_error(format!(
"构造拼图 VectorEngine 图片编辑参考图失败:{error}"
))
})?;
let form = reqwest::multipart::Form::new()
.part("image", image_part)
.text("model", image_model.request_model_name().to_string())
.text(
"prompt",
build_puzzle_vector_engine_prompt(prompt, negative_prompt),
)
.text("n", candidate_count.clamp(1, 1).to_string())
.text("size", size.to_string());
let request_started_at = Instant::now();
let response = http_client
.post(request_url.as_str())
.header(
reqwest::header::AUTHORIZATION,
format!("Bearer {}", settings.api_key),
)
.header(reqwest::header::ACCEPT, "application/json")
.multipart(form)
.send()
.await
.map_err(|error| {
map_puzzle_vector_engine_request_error(format!(
"创建拼图 VectorEngine 图片编辑任务失败:{error}"
))
})?;
let status = response.status();
tracing::info!(
provider = VECTOR_ENGINE_PROVIDER,
image_model = image_model.request_model_name(),
endpoint = %request_url,
status = status.as_u16(),
prompt_chars = prompt.chars().count(),
let generated = create_openai_image_edit_with_references(
http_client,
&settings.to_openai_settings(),
prompt,
Some(negative_prompt),
size,
reference_mime = %reference_image.mime_type,
reference_bytes = reference_image.bytes_len,
elapsed_ms = request_started_at.elapsed().as_millis() as u64,
"拼图 VectorEngine 图片编辑 HTTP 返回"
);
let response_text = response.text().await.map_err(|error| {
map_puzzle_vector_engine_request_error(format!(
"读取拼图 VectorEngine 图片编辑响应失败:{error}"
))
})?;
if !status.is_success() {
return Err(map_puzzle_vector_engine_upstream_error(
status,
response_text.as_str(),
"创建拼图 VectorEngine 图片编辑任务失败",
));
}
let payload = parse_puzzle_json_payload(
response_text.as_str(),
"解析拼图 VectorEngine 图片编辑响应失败",
)?;
let image_urls = extract_puzzle_image_urls(&payload);
if !image_urls.is_empty() {
return download_puzzle_images_from_urls(http_client, task_id, image_urls, candidate_count)
.await;
}
let b64_images = extract_puzzle_b64_images(&payload);
if !b64_images.is_empty() {
return Ok(puzzle_images_from_base64(
task_id,
b64_images,
candidate_count,
));
}
Err(
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": VECTOR_ENGINE_PROVIDER,
"message": "拼图 VectorEngine 图片编辑未返回图片",
})),
candidate_count,
&[OpenAiReferenceImage {
bytes: reference_image.bytes.clone(),
mime_type: reference_image.mime_type.clone(),
file_name,
}],
"拼图 VectorEngine 图片编辑失败",
)
.await?;
Ok(PuzzleGeneratedImages {
task_id: generated.task_id,
images: generated
.images
.into_iter()
.map(PuzzleDownloadedImage::from_openai_image)
.collect(),
})
}
pub(crate) fn build_puzzle_downloaded_image_reference(
@@ -569,42 +452,6 @@ pub(crate) fn build_puzzle_vector_engine_prompt(prompt: &str, negative_prompt: &
format!("{prompt}\n避免:{negative_prompt}")
}
pub(crate) fn puzzle_vector_engine_images_generation_url(
settings: &PuzzleVectorEngineSettings,
) -> String {
if settings.base_url.ends_with("/v1") {
format!("{}/images/generations", settings.base_url)
} else {
format!("{}/v1/images/generations", settings.base_url)
}
}
pub(crate) fn puzzle_vector_engine_images_edit_url(
settings: &PuzzleVectorEngineSettings,
) -> String {
if settings.base_url.ends_with("/v1") {
format!("{}/images/edits", settings.base_url)
} else {
format!("{}/v1/images/edits", settings.base_url)
}
}
pub(crate) async fn download_puzzle_images_from_urls(
http_client: &reqwest::Client,
task_id: String,
image_urls: Vec<String>,
candidate_count: u32,
) -> Result<PuzzleGeneratedImages, AppError> {
let mut images = Vec::with_capacity(candidate_count.clamp(1, 1) as usize);
for image_url in image_urls
.into_iter()
.take(candidate_count.clamp(1, 1) as usize)
{
images.push(download_puzzle_remote_image(http_client, image_url.as_str()).await?);
}
Ok(PuzzleGeneratedImages { task_id, images })
}
pub(crate) fn parse_puzzle_asset_object_reference(source: &str) -> Option<&str> {
source
.trim()
@@ -890,40 +737,6 @@ async fn download_signed_puzzle_reference_image(
})
}
pub(crate) async fn download_puzzle_remote_image(
http_client: &reqwest::Client,
image_url: &str,
) -> Result<PuzzleDownloadedImage, AppError> {
let response = http_client.get(image_url).send().await.map_err(|error| {
map_puzzle_image_request_error(format!("下载拼图正式图片失败:{error}"))
})?;
let status = response.status();
let content_type = response
.headers()
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.unwrap_or("image/jpeg")
.to_string();
let bytes = response.bytes().await.map_err(|error| {
map_puzzle_image_request_error(format!("读取拼图正式图片内容失败:{error}"))
})?;
if !status.is_success() {
return Err(
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": "puzzle-image",
"message": "下载拼图正式图片失败",
"status": status.as_u16(),
})),
);
}
let mime_type = normalize_puzzle_downloaded_image_mime_type(content_type.as_str());
Ok(PuzzleDownloadedImage {
extension: puzzle_mime_to_extension(mime_type.as_str()).to_string(),
mime_type,
bytes: bytes.to_vec(),
})
}
pub(crate) async fn persist_puzzle_generated_asset(
state: &PuzzleApiState,
owner_user_id: &str,
@@ -1197,18 +1010,6 @@ pub(crate) fn build_puzzle_level_asset_metadata(
])
}
pub(crate) fn parse_puzzle_json_payload(
raw_text: &str,
fallback_message: &str,
) -> Result<Value, AppError> {
serde_json::from_str::<Value>(raw_text).map_err(|error| {
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": VECTOR_ENGINE_PROVIDER,
"message": format!("{fallback_message}{error}"),
}))
})
}
pub(crate) fn parse_puzzle_image_data_url(value: &str) -> Option<ParsedPuzzleImageDataUrl> {
let body = value.strip_prefix("data:")?;
let (mime_type, data) = body.split_once(";base64,")?;
@@ -1249,49 +1050,6 @@ pub(crate) fn decode_puzzle_base64(value: &str) -> Option<Vec<u8>> {
Some(output)
}
pub(crate) fn extract_puzzle_image_urls(payload: &Value) -> Vec<String> {
let mut urls = Vec::new();
collect_puzzle_strings_by_key(payload, "image", &mut urls);
collect_puzzle_strings_by_key(payload, "url", &mut urls);
let mut deduped = Vec::new();
for url in urls {
if !deduped.contains(&url) {
deduped.push(url);
}
}
deduped
}
pub(crate) fn extract_puzzle_b64_images(payload: &Value) -> Vec<String> {
let mut values = Vec::new();
collect_puzzle_strings_by_key(payload, "b64_json", &mut values);
values
}
pub(crate) fn puzzle_images_from_base64(
task_id: String,
b64_images: Vec<String>,
candidate_count: u32,
) -> PuzzleGeneratedImages {
let images = b64_images
.into_iter()
.take(candidate_count.clamp(1, 1) as usize)
.filter_map(|raw| decode_puzzle_generated_image_base64(raw.as_str()))
.collect();
PuzzleGeneratedImages { task_id, images }
}
pub(crate) fn decode_puzzle_generated_image_base64(raw: &str) -> Option<PuzzleDownloadedImage> {
let bytes = BASE64_STANDARD.decode(raw.trim()).ok()?;
let mime_type = infer_puzzle_image_mime_type(bytes.as_slice());
Some(PuzzleDownloadedImage {
extension: puzzle_mime_to_extension(mime_type.as_str()).to_string(),
mime_type,
bytes,
})
}
pub(crate) fn find_first_puzzle_string_by_key(payload: &Value, target_key: &str) -> Option<String> {
let mut results = Vec::new();
collect_puzzle_strings_by_key(payload, target_key, &mut results);
@@ -1333,22 +1091,6 @@ pub(crate) fn collect_puzzle_string_values(payload: &Value, results: &mut Vec<St
}
}
pub(crate) fn infer_puzzle_image_mime_type(bytes: &[u8]) -> String {
if bytes.starts_with(b"\x89PNG\r\n\x1A\n") {
return "image/png".to_string();
}
if bytes.starts_with(b"\xFF\xD8\xFF") {
return "image/jpeg".to_string();
}
if bytes.starts_with(b"RIFF") && bytes.get(8..12) == Some(b"WEBP") {
return "image/webp".to_string();
}
if bytes.starts_with(b"GIF87a") || bytes.starts_with(b"GIF89a") {
return "image/gif".to_string();
}
"image/png".to_string()
}
pub(crate) fn normalize_puzzle_downloaded_image_mime_type(content_type: &str) -> String {
let mime_type = content_type
.split(';')
@@ -1387,21 +1129,6 @@ pub(crate) fn map_puzzle_image_request_error(message: String) -> AppError {
}))
}
pub(crate) fn map_puzzle_vector_engine_request_error(message: String) -> AppError {
let is_timeout = is_puzzle_request_timeout_message(message.as_str());
let status = if is_timeout {
StatusCode::GATEWAY_TIMEOUT
} else {
StatusCode::BAD_GATEWAY
};
AppError::from_status(status).with_details(json!({
"provider": VECTOR_ENGINE_PROVIDER,
"message": message,
"timeout": is_timeout,
}))
}
pub(crate) fn is_puzzle_request_timeout_message(message: &str) -> bool {
let lower = message.to_ascii_lowercase();
lower.contains("timed out")
@@ -1410,64 +1137,6 @@ pub(crate) fn is_puzzle_request_timeout_message(message: &str) -> bool {
|| lower.contains("deadline has elapsed")
}
pub(crate) fn map_puzzle_vector_engine_upstream_error(
upstream_status: reqwest::StatusCode,
raw_text: &str,
fallback_message: &str,
) -> AppError {
let message = parse_puzzle_api_error_message(raw_text, fallback_message);
let raw_excerpt = trim_puzzle_upstream_excerpt(raw_text, 800);
let is_timeout = is_puzzle_request_timeout_message(message.as_str())
|| is_puzzle_request_timeout_message(raw_excerpt.as_str());
let status = if is_timeout {
StatusCode::GATEWAY_TIMEOUT
} else {
StatusCode::BAD_GATEWAY
};
tracing::warn!(
provider = VECTOR_ENGINE_PROVIDER,
upstream_status = upstream_status.as_u16(),
timeout = is_timeout,
message = %message,
raw_excerpt = %raw_excerpt,
"拼图 VectorEngine 上游请求失败"
);
AppError::from_status(status).with_details(json!({
"provider": VECTOR_ENGINE_PROVIDER,
"upstreamStatus": upstream_status.as_u16(),
"message": message,
"rawExcerpt": raw_excerpt,
"timeout": is_timeout,
}))
}
pub(crate) fn parse_puzzle_api_error_message(raw_text: &str, fallback_message: &str) -> String {
let trimmed = raw_text.trim();
if trimmed.is_empty() {
return fallback_message.to_string();
}
if let Ok(payload) = serde_json::from_str::<Value>(trimmed)
&& let Some(message) = find_first_puzzle_string_by_key(&payload, "message")
{
return message;
}
fallback_message.to_string()
}
pub(crate) fn trim_puzzle_upstream_excerpt(raw_text: &str, max_chars: usize) -> String {
let normalized = raw_text.split_whitespace().collect::<Vec<_>>().join(" ");
if normalized.chars().count() <= max_chars {
return normalized;
}
let keep_chars = max_chars.saturating_sub(3);
format!(
"{}...",
normalized.chars().take(keep_chars).collect::<String>()
)
}
pub(crate) fn map_puzzle_asset_oss_error(error: platform_oss::OssError) -> AppError {
map_oss_error(error, "aliyun-oss")
}

View File

@@ -0,0 +1,12 @@
[package]
name = "platform-image"
edition.workspace = true
version.workspace = true
license.workspace = true
[dependencies]
base64 = { workspace = true }
reqwest = { workspace = true, features = ["json", "multipart", "rustls-tls"] }
serde_json = { workspace = true }
tokio = { workspace = true, features = ["time"] }
tracing = { workspace = true }

File diff suppressed because it is too large Load Diff