GenarrativeAI/Genarrative
11
0
Fork 0
Code Issues 14 Pull Requests 3 Actions 1 Packages Projects 3 Releases Wiki Activity

实现 Raw GPT Image 2 图片编辑代理

Browse Source 
新增受 Bearer 保护的 /api/raw/v1/images/edit JSON 路由

按单图 image 与可选 mask 转发 GPT Image 2 参数

固定 model 与 n 并返回仅含 data[].b64_json 的响应

复用钱包计费与退款边界并补充单元测试
This commit is contained in:
王德宇
2026-09-08 10:36:48 +08:00
parent 302b4addae
commit 4e1316e272
13 changed files with 423 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/technical/【技术方案】Raw GPT Image 2图片编辑代理-2026-09-07.md
+13 -14
View File
@@ -18,12 +18,10 @@ POST /api/raw/v1/images/edit
```json
{
"images": [
{
"data": "<base64>",
"mimeType": "image/png"
}
],
"image": {
"data": "<base64>",
"mimeType": "image/png"
},
"mask": {
"data": "<base64>",
"mimeType": "image/png"
@@ -37,7 +35,7 @@ POST /api/raw/v1/images/edit
}
```
`images` 是必填数组，数组成员结构固定为 `{ data, mimeType }`；`mask` 可选并使用相同结构。输入格式由成员的 MIME 类型和解码后的图片字节共同确定，服务端不把输入格式另建成请求参数。`prompt` 必填。`quality`、`background` 和 `output_format` 采用 GPT Image 模型支持的值。`width`、`height` 为整数，组成发送给 provider 的输出尺寸；不把尺寸改写成业务字符串字段。
`image` 是必填的单图结构 `{ data, mimeType }`；`mask` 可选并使用相同结构。输入格式由 MIME 类型和解码后的图片字节共同确定，服务端不把输入格式另建成请求参数。`prompt` 必填。`quality`、`background` 和 `output_format` 采用 GPT Image 模型支持的值。`width`、`height` 为整数，组成发送给 provider 的输出尺寸；不把尺寸改写成业务字符串字段。
服务端发送给 `platform-image` 时固定注入：
@@ -68,14 +66,15 @@ n = 1
所有请求、JSON、base64、图片结构和 provider 参数检查必须在扣费前完成。预检查失败直接返回 4xx，不产生钱包流水，也不调用 provider。
检查通过后，api-server 调用一个 raw 图片操作的 SpacetimeDB 事务 procedure，在同一事务内完成：
检查通过后，api-server 进入现有资产操作计费边界，通过 SpacetimeDB 钱包事务 procedure 原子完成：
1. 以认证后的用户和请求 ID 建立 raw 操作幂等事实；
2. 按现有图片编辑算法解析价格：GPT Image 2 长边不超过 1536 使用 1K 价格，否则使用 2K 价格；当前默认价格为 3 / 5 泥点；
3. 原子扣除用户泥点并写入 `asset_operation_consume` 流水；
4. 持久化操作状态，供 provider 返回后成功或失败收口。
1. 按现有图片编辑算法解析价格：GPT Image 2 长边不超过 1536 使用 1K 价格，否则使用 2K 价格；当前默认价格为 3 / 5 泥点；
2. 以认证后的用户、`raw-image-edit` 命名空间和请求 ID 组成幂等扣费流水 ID；
3. 原子扣除用户泥点并写入 `asset_operation_consume` 流水。
provider 调用在 SpacetimeDB 事务之外执行。成功后调用同一 raw 操作的完成 procedure；失败后调用失败 procedure，由数据库事务写入退款 outbox / settlement 事实。进程崩溃时不依赖 Rust future `Drop` 才能发现需要退款；恢复处理根据持久化的 raw 操作状态完成退款。
provider 调用在 SpacetimeDB 事务之外执行。失败时由现有计费边界把幂等退款事实写入 SpacetimeDB refund outbox，再由 worker 完成退款。
TODO：新增 raw 操作持久化状态，将“创建 raw 操作事实 + 扣费”收入同一事务，并由恢复 worker 对“已扣费但未收口”状态自动退款，填补进程在扣费后、写入 refund outbox 前崩溃的窗口。
raw 操作使用独立的 operation / ledger 命名空间，例如 `raw-image-edit`，不能复用编辑器资源 ID、编辑器任务 ID 或 `external_generation_job`。
@@ -83,7 +82,7 @@ raw 操作使用独立的 operation / ledger 命名空间，例如 `raw-image-ed
`platform-image` 保留 VectorEngine 协议细节。raw handler 只负责：认证、JSON DTO、base64 解码、预检查、计费编排和响应映射。provider 请求仍由 `platform-image` 统一构造，并携带 `model`、`n`、`quality`、`background`、`output_format`、尺寸及图片参考字节。
provider 返回的原始 `size` 必须沿 `GeneratedImages` 结果传回 api-server；raw handler 从该字段解析响应尺寸，再编码 `data[].b64_json`。
provider 结果统一解码为图片字节；raw handler 只将这些字节编码到 `data[].b64_json`。
## 代码拆分
server-rs/crates/api-server/src/app.rs
+1
View File
@@ -50,6 +50,7 @@ pub fn build_router(state: AppState) -> Router {
.merge(modules::platform::router(state.clone()))
.merge(modules::external_generation::router(state.clone()))
.merge(modules::platform_support::router(state.clone()))
.merge(modules::raw::router(state.clone()))
.merge(crate::error_reports::router(state.clone()))
.route(
"/api/profile/recharge/wechat/notify",
server-rs/crates/api-server/src/main.rs
+1
View File
@@ -67,6 +67,7 @@ mod profile_identity;
mod profile_recharge_expiration_listener;
mod profile_recharge_refund_reconciliation;
mod prompt;
mod raw_image;
mod refresh_session;
mod registration_reward;
mod request_context;
server-rs/crates/api-server/src/modules.rs
+1
View File
@@ -10,3 +10,4 @@ pub mod internal;
pub mod platform;
pub mod platform_support;
pub mod profile;
pub mod raw;
server-rs/crates/api-server/src/modules/raw.rs
+11
View File
@@ -0,0 +1,11 @@
use axum::{Router, middleware, routing::post};
use crate::{auth::require_bearer_auth, raw_image::edit_raw_image, state::AppState};
pub fn router(state: AppState) -> Router<AppState> {
Router::new().route(
"/api/raw/v1/images/edit",
post(edit_raw_image)
.route_layer(middleware::from_fn_with_state(state, require_bearer_auth)),
)
}
server-rs/crates/api-server/src/openai_image_generation.rs
+1 -1
View File
@@ -414,7 +414,7 @@ impl OpenAiImageSettings {
self
}
fn provider_settings(&self) -> VectorEngineImageSettings {
pub(crate) fn provider_settings(&self) -> VectorEngineImageSettings {
VectorEngineImageSettings {
base_url: self.base_url.clone(),
api_key: self.api_key.clone(),
server-rs/crates/api-server/src/raw_image.rs
+219
View File
@@ -0,0 +1,219 @@
use axum::{
Json,
extract::{Extension, State},
http::StatusCode,
};
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64_STANDARD};
use platform_image::{RawImageEditOptions, ReferenceImage, create_vector_engine_raw_image_edit};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use crate::{
asset_billing::{
execute_billable_asset_operation_with_cost, with_editor_generation_durable_billing_boundary,
},
auth::AuthenticatedAccessToken,
http_error::AppError,
openai_image_generation::{
build_openai_image_http_client, map_platform_image_error, require_openai_image_settings,
},
request_context::RequestContext,
state::AppState,
};
#[derive(Clone, Debug, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub(crate) struct RawImageData {
pub(crate) data: String,
pub(crate) mime_type: String,
}
#[derive(Clone, Debug, Deserialize)]
#[serde(rename_all = "snake_case", deny_unknown_fields)]
pub(crate) struct RawImageEditRequest {
pub(crate) image: RawImageData,
pub(crate) mask: Option<RawImageData>,
pub(crate) prompt: String,
pub(crate) quality: Option<String>,
pub(crate) background: Option<String>,
pub(crate) output_format: Option<String>,
pub(crate) width: u32,
pub(crate) height: u32,
}
#[derive(Debug, Serialize)]
pub(crate) struct RawImageEditItem {
pub(crate) b64_json: String,
}
#[derive(Debug, Serialize)]
pub(crate) struct RawImageEditResponse {
pub(crate) data: Vec<RawImageEditItem>,
}
pub(crate) async fn edit_raw_image(
State(state): State<AppState>,
Extension(request_context): Extension<RequestContext>,
Extension(authenticated): Extension<AuthenticatedAccessToken>,
Json(payload): Json<RawImageEditRequest>,
) -> Result<Json<Value>, AppError> {
let prepared = prepare_request(payload)?;
let settings = require_openai_image_settings(&state)?.with_external_api_audit_context(
&request_context,
Some(authenticated.claims().user_id().to_string()),
None,
);
let http_client = build_openai_image_http_client(&settings)?;
let provider_settings = settings.provider_settings();
let user_id = authenticated.claims().user_id().to_string();
let request_id = request_context.request_id().to_string();
let points_cost = raw_image_edit_price(&state, prepared.width, prepared.height).await?;
let operation = async move {
let generated = create_vector_engine_raw_image_edit(
&http_client,
&provider_settings,
prepared.prompt.as_str(),
&prepared.image,
prepared.options,
"raw_image_edit",
)
.await
.map_err(map_platform_image_error)?;
let data = generated
.images
.into_iter()
.map(|image| RawImageEditItem {
b64_json: BASE64_STANDARD.encode(image.bytes),
})
.collect();
Ok::<_, AppError>(RawImageEditResponse { data })
};
let result = with_editor_generation_durable_billing_boundary(
execute_billable_asset_operation_with_cost(
&state,
user_id.as_str(),
"raw-image-edit",
request_id.as_str(),
u64::from(points_cost),
operation,
),
)
.await?;
Ok(Json(serde_json::to_value(result).map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_message(error.to_string())
})?))
}
struct PreparedRawImageEdit {
image: ReferenceImage,
prompt: String,
options: RawImageEditOptions,
width: u32,
height: u32,
}
fn prepare_request(payload: RawImageEditRequest) -> Result<PreparedRawImageEdit, AppError> {
if payload.prompt.trim().is_empty() {
return Err(bad_request("prompt 不能为空"));
}
if payload.width == 0 || payload.height == 0 {
return Err(bad_request("width 和 height 必须为正整数"));
}
let image = decode_image(payload.image, "image")?;
let mask = payload
.mask
.map(|value| decode_image(value, "mask"))
.transpose()?;
Ok(PreparedRawImageEdit {
image,
prompt: payload.prompt,
options: RawImageEditOptions {
quality: payload.quality,
background: payload.background,
output_format: payload.output_format,
width: payload.width,
height: payload.height,
mask,
},
width: payload.width,
height: payload.height,
})
}
fn decode_image(value: RawImageData, field: &str) -> Result<ReferenceImage, AppError> {
let mime_type = value.mime_type.trim().to_string();
if mime_type.is_empty() {
return Err(bad_request(format!("{field}.mimeType 不能为空")));
}
let bytes = BASE64_STANDARD
.decode(value.data.trim())
.map_err(|error| bad_request(format!("{field}.data 必须是有效 base64：{error}")))?;
Ok(ReferenceImage {
bytes,
file_name: format!("{field}.png"),
mime_type,
})
}
async fn raw_image_edit_price(state: &AppState, width: u32, height: u32) -> Result<u32, AppError> {
let tier = if width.max(height) > 1536 { "2K" } else { "1K" };
state
.editor_generation_pricing()
.await
.map(|pricing| {
pricing.image_generation_mud_points(Some("quick-edit"), Some("gpt-image-2"), Some(tier))
})
.map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
"provider": "editor-generation-pricing",
"message": error.to_string(),
}))
})
}
fn bad_request(message: impl Into<String>) -> AppError {
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
"provider": "raw-image-edit",
"message": message.into(),
}))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn request_uses_one_image_object_and_rejects_images_array() {
let payload = serde_json::json!({
"image": {"data": "aGVsbG8=", "mimeType": "image/png"},
"prompt": "edit",
"width": 1024,
"height": 1024
});
let parsed: RawImageEditRequest = serde_json::from_value(payload).expect("image object");
let prepared = prepare_request(parsed).expect("request should prepare");
assert_eq!(prepared.image.bytes, b"hello");
let array_payload = serde_json::json!({
"images": [{"data": "aGVsbG8=", "mimeType": "image/png"}],
"prompt": "edit",
"width": 1024,
"height": 1024
});
assert!(serde_json::from_value::<RawImageEditRequest>(array_payload).is_err());
}
#[test]
fn response_contains_only_data_b64_json() {
let response = serde_json::to_value(RawImageEditResponse {
data: vec![RawImageEditItem {
b64_json: "aGVsbG8=".to_string(),
}],
})
.expect("response should serialize");
assert_eq!(
response,
serde_json::json!({"data": [{"b64_json": "aGVsbG8="}]})
);
}
}
server-rs/crates/platform-image/src/lib.rs
+6 -5
View File
@@ -10,14 +10,15 @@ pub use pixel_art_snapper::{
};
pub use vector_engine::{
DownloadedImage, GPT_IMAGE_2_C_MODEL, GPT_IMAGE_2_MODEL, GeneratedImages, NANOBANANA_2_MODEL,
PlatformImageError, PlatformImageFailureAudit, PlatformImageStatusHint, ReferenceImage,
VECTOR_ENGINE_GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings,
build_vector_engine_image_http_client, build_vector_engine_image_request_body,
PlatformImageError, PlatformImageFailureAudit, PlatformImageStatusHint, RawImageEditOptions,
ReferenceImage, VECTOR_ENGINE_GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER,
VectorEngineImageSettings, build_vector_engine_image_http_client,
build_vector_engine_image_request_body,
build_vector_engine_nanobanana_generate_content_request_body, create_vector_engine_image_edit,
create_vector_engine_image_edit_with_references,
create_vector_engine_image_edit_with_references_and_model,
create_vector_engine_image_generation, create_vector_engine_image_generation_with_model,
create_vector_engine_nanobanana_generate_content, download_remote_image,
vector_engine_images_edit_url, vector_engine_images_generation_url,
create_vector_engine_nanobanana_generate_content, create_vector_engine_raw_image_edit,
download_remote_image, vector_engine_images_edit_url, vector_engine_images_generation_url,
vector_engine_nanobanana_generate_content_url,
};
server-rs/crates/platform-image/src/vector_engine/client.rs
+61 -14
View File
@@ -16,6 +16,7 @@ use super::{
curl_transport::{
map_curl_error, send_vector_engine_json_request_with_curl,
send_vector_engine_multipart_edit_request_with_curl,
send_vector_engine_multipart_edit_request_with_curl_options,
},
error::PlatformImageError,
image_source::resolve_reference_images,
@@ -28,7 +29,7 @@ use super::{
vector_engine_nanobanana_generate_content_url,
},
response::handle_vector_engine_response,
types::{GeneratedImages, ReferenceImage, VectorEngineImageSettings},
types::{GeneratedImages, RawImageEditOptions, ReferenceImage, VectorEngineImageSettings},
util::truncate_raw,
};
@@ -537,6 +538,34 @@ pub async fn create_vector_engine_image_edit_with_references_and_model(
candidate_count: u32,
reference_images: &[ReferenceImage],
failure_context: &str,
) -> Result<GeneratedImages, PlatformImageError> {
create_vector_engine_image_edit_with_references_and_model_and_options(
http_client,
settings,
model,
prompt,
negative_prompt,
size,
candidate_count,
reference_images,
None,
failure_context,
)
.await
}
#[allow(clippy::too_many_arguments)]
pub async fn create_vector_engine_image_edit_with_references_and_model_and_options(
http_client: &reqwest::Client,
settings: &VectorEngineImageSettings,
model: &str,
prompt: &str,
negative_prompt: Option<&str>,
size: &str,
candidate_count: u32,
reference_images: &[ReferenceImage],
options: Option<&RawImageEditOptions>,
failure_context: &str,
) -> Result<GeneratedImages, PlatformImageError> {
let requested_model = normalize_vector_engine_image_model(model);
if reference_images.is_empty() {
@@ -611,19 +640,37 @@ pub async fn create_vector_engine_image_edit_with_references_and_model(
&mut recovered_failure_audits,
));
};
let response = match send_vector_engine_multipart_edit_request_with_curl(
request_url.as_str(),
settings.api_key.as_str(),
upstream_model,
prompt,
negative_prompt,
normalized_size.as_str(),
candidate_count,
reference_images,
attempt_timeout_ms,
)
.await
{
let response = match match options {
Some(options) => {
send_vector_engine_multipart_edit_request_with_curl_options(
request_url.as_str(),
settings.api_key.as_str(),
upstream_model,
prompt,
negative_prompt,
normalized_size.as_str(),
candidate_count,
reference_images,
Some(options),
attempt_timeout_ms,
)
.await
}
None => {
send_vector_engine_multipart_edit_request_with_curl(
request_url.as_str(),
settings.api_key.as_str(),
upstream_model,
prompt,
negative_prompt,
normalized_size.as_str(),
candidate_count,
reference_images,
attempt_timeout_ms,
)
.await
}
} {
Ok(response) => {
if should_retry_vector_engine_upstream_response(
response.status,
server-rs/crates/platform-image/src/vector_engine/curl_transport.rs
+58 -2
View File
@@ -7,8 +7,11 @@ use curl::{
use serde_json::Value;
use super::{
audit::build_failure_audit, constants::VECTOR_ENGINE_PROVIDER, error::PlatformImageError,
request::build_prompt_with_negative, types::ReferenceImage,
audit::build_failure_audit,
constants::VECTOR_ENGINE_PROVIDER,
error::PlatformImageError,
request::build_prompt_with_negative,
types::{RawImageEditOptions, ReferenceImage},
};
#[derive(Debug)]
@@ -119,6 +122,34 @@ pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl(
candidate_count: u32,
reference_images: &[ReferenceImage],
timeout_ms: u64,
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
send_vector_engine_multipart_edit_request_with_curl_options(
request_url,
api_key,
model,
prompt,
negative_prompt,
normalized_size,
candidate_count,
reference_images,
None,
timeout_ms,
)
.await
}
#[allow(clippy::too_many_arguments)]
pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl_options(
request_url: &str,
api_key: &str,
model: &str,
prompt: &str,
negative_prompt: Option<&str>,
normalized_size: &str,
candidate_count: u32,
reference_images: &[ReferenceImage],
options: Option<&RawImageEditOptions>,
timeout_ms: u64,
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
let request_url = request_url.to_string();
let api_key = api_key.to_string();
@@ -127,6 +158,7 @@ pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl(
let negative_prompt = negative_prompt.map(str::to_string);
let normalized_size = normalized_size.to_string();
let reference_images = reference_images.to_vec();
let options = options.cloned();
tokio::task::spawn_blocking(move || {
send_multipart_edit_request_with_curl_blocking(
request_url.as_str(),
@@ -137,6 +169,7 @@ pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl(
normalized_size.as_str(),
candidate_count,
reference_images.as_slice(),
options.as_ref(),
timeout_ms,
)
})
@@ -239,6 +272,7 @@ fn send_multipart_edit_request_with_curl_blocking(
normalized_size: &str,
candidate_count: u32,
reference_images: &[ReferenceImage],
options: Option<&RawImageEditOptions>,
timeout_ms: u64,
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
let mut form = Form::new();
@@ -253,6 +287,28 @@ fn send_multipart_edit_request_with_curl_blocking(
.contents(normalized_size.as_bytes())
.add()?;
if let Some(options) = options {
if let Some(quality) = options.quality.as_deref() {
form.part("quality").contents(quality.as_bytes()).add()?;
}
if let Some(background) = options.background.as_deref() {
form.part("background")
.contents(background.as_bytes())
.add()?;
}
if let Some(output_format) = options.output_format.as_deref() {
form.part("output_format")
.contents(output_format.as_bytes())
.add()?;
}
if let Some(mask) = options.mask.as_ref() {
form.part("mask")
.buffer(mask.file_name.as_str(), mask.bytes.clone())
.content_type(mask.mime_type.as_str())
.add()?;
}
}
for reference_image in reference_images {
form.part("image")
.buffer(
server-rs/crates/platform-image/src/vector_engine/mod.rs
+6 -1
View File
@@ -6,6 +6,7 @@ mod curl_transport;
mod error;
mod image_source;
mod payload;
mod raw_edit;
mod request;
mod response;
mod transport;
@@ -25,6 +26,7 @@ pub use constants::{
};
pub use error::{PlatformImageError, PlatformImageStatusHint};
pub use image_source::download_remote_image;
pub use raw_edit::create_vector_engine_raw_image_edit;
pub use request::{
build_vector_engine_image_request_body, build_vector_engine_image_request_body_with_model,
build_vector_engine_nanobanana_generate_content_request_body, normalize_image_size_for_model,
@@ -32,4 +34,7 @@ pub use request::{
vector_engine_nanobanana_generate_content_url,
};
pub use transport::build_vector_engine_image_http_client;
pub use types::{DownloadedImage, GeneratedImages, ReferenceImage, VectorEngineImageSettings};
pub use types::{
DownloadedImage, GeneratedImages, RawImageEditOptions, ReferenceImage,
VectorEngineImageSettings,
};
server-rs/crates/platform-image/src/vector_engine/raw_edit.rs
+33
View File
@@ -0,0 +1,33 @@
use super::{
client::create_vector_engine_image_edit_with_references_and_model_and_options,
constants::GPT_IMAGE_2_MODEL,
error::PlatformImageError,
types::{GeneratedImages, RawImageEditOptions, ReferenceImage, VectorEngineImageSettings},
};
/// Sends the raw GPT Image 2 edit contract while keeping VectorEngine's
/// multipart transport inside this crate.
#[allow(clippy::too_many_arguments)]
pub async fn create_vector_engine_raw_image_edit(
http_client: &reqwest::Client,
settings: &VectorEngineImageSettings,
prompt: &str,
image: &ReferenceImage,
options: RawImageEditOptions,
failure_context: &str,
) -> Result<GeneratedImages, PlatformImageError> {
let size = format!("{}x{}", options.width, options.height);
create_vector_engine_image_edit_with_references_and_model_and_options(
http_client,
settings,
GPT_IMAGE_2_MODEL,
prompt,
None,
size.as_str(),
1,
std::slice::from_ref(image),
Some(&options),
failure_context,
)
.await
}
server-rs/crates/platform-image/src/vector_engine/types.rs
+12
View File
@@ -29,3 +29,15 @@ pub struct ReferenceImage {
pub mime_type: String,
pub file_name: String,
}
/// Raw GPT Image 2 edit options. The API layer owns validation; this type only
/// carries values that must be forwarded to VectorEngine.
#[derive(Clone, Debug, Default)]
pub struct RawImageEditOptions {
pub quality: Option<String>,
pub background: Option<String>,
pub output_format: Option<String>,
pub width: u32,
pub height: u32,
pub mask: Option<ReferenceImage>,
}