GenarrativeAI/Genarrative
11
0
Fork 0
Code Issues 21 Pull Requests 4 Actions Packages Projects 3 Releases Wiki Activity

补齐背景色决策外部失败审计

Browse Source 
为gpt-5-mini背景色决策记录OTLP和失败事件

透传角色、图标、UI提取和动作生成的审计上下文

补充失败分类测试和后端契约说明
This commit is contained in:
孔令弘
2026-07-10 09:17:48 +00:00
parent a701fa0240
commit f64fff461b
4 changed files with 384 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md
+1 -1
View File
@@ -234,7 +234,7 @@ npm run check:server-rs-ddd
- Hyper3D / Rodin：只保留后端安全代理和旧数据兼容；Rodin 提交、状态、下载和响应解析归属 `platform-hyper3d`，`api-server/src/hyper3d_generation.rs` 只做路由、配置和错误 envelope 映射；新 Match3D 草稿和批量新增不再生成 GLB。
- 音频：视觉小说专用音频路由保留；VectorEngine Suno/Vidu provider 协议、任务提交/查询、音频 URL 提取、下载、MIME/extension 归一和 OSS put 请求准备归属 `platform-audio`。`api-server/src/vector_engine_audio_generation.rs` 只做路由、配置、计费、asset object confirm、entity binding 和错误 envelope 映射；拼图、抓大鹅和敲木鱼提示词生成音效入口暂时关闭，通用 `/api/creation/audio/*` 对这些目标返回 `410 Gone`。敲木鱼创作只接收上传 / 录音音频资产；前端选择或录音阶段只在浏览器本地处理待提交音频，统一限制裁切后最长 1 秒、裁掉前后声音过小片段，并用浏览器端近似响度算法平衡到 `-15 LKFS` 后做峰值保护。点击生成时才直传 OSS 并确认 `asset_object`，创作 JSON 只提交轻量 `WoodenFishAudioAsset`，不得继续上传 Data URL 音频；未提供时由 `api-server` 写回内置默认木鱼音 `/wooden-fish/default-hit-sound.mp3`。
- OSS：私有 generated legacy path 进入浏览器前必须通过 `/api/assets/read-url` 换签；不要裸请求 `/generated-*`。前端如果收到同一 OSS bucket 的完整 `https://*.oss-*.aliyuncs.com/generated-*` 地址，也必须先归一为 legacy path 后走同一换签链路，避免裸连私有 bucket 403 或绕过签名缓存。OSS 签名、读签名、HEAD 和 PUT 的结构化日志由 `platform-oss` 输出，排查资产写入 / 确认失败时优先按 `operation`、`object_key` / `key_prefix`、`status_class`、`error_kind` 和 `elapsed_ms` 下钻。新上传 generated 私有对象默认写入 `Cache-Control: public, max-age=31536000, immutable`；旧对象若缺该头，只能依赖 `ETag` / `Last-Modified` 协商缓存，应通过 OSS 元数据刷新或 CDN 配置补齐，不要恢复 api-server 静态代理。`editor-agent/` 前缀只用于服务端内部读写画布 Agent 会话消息文档，不属于浏览器直传 legacy public prefix；`/api/assets/direct-upload-tickets` 必须拒绝 `legacyPrefix=editor-agent`，内部读取只允许 `editor-agent/{conversationId}.json` 形态。
- 外部 API 失败审计：外部供应商调用未成功时，`api-server` 必须发送 OTLP 失败事件并写入 `tracking_event`。VectorEngine 图片 provider 在 `platform-image` 内输出结构化日志和 `PlatformImageFailureAudit`，覆盖 `request_send`、`response_body`、`upstream_status`、`response_parse`、`missing_image` 和 `image_download` 阶段；`api-server` 只把该 audit 映射成 `external_api_call_failure`，`scope_kind = module`、`scope_id = provider`、`module_key = external-api`。metadata 固定包含 provider、endpoint、operation、failureStage、statusCode、statusClass、timeout、retryable、errorMessage、latencyMs、promptChars、referenceImageCount、imageModel、rawExcerpt，以及在调用方可获得上下文时补充的 `userId`（触发者）和 `profileId`（草稿 / 作品 / 场景作用域）。图片生成入口应优先把 owner user id 和 profile id 透传到失败审计，不要只保留 provider 级聚合，否则很难按“谁触发、哪个作品触发”定位问题。入库优先复用 tracking outbox，outbox 不可写或保护阈值拒绝时回退同步写 SpacetimeDB；不得新增前端兜底或在 SpacetimeDB reducer 内做外部 I/O。
- 外部 API 失败审计：外部供应商调用未成功时，`api-server` 必须发送 OTLP 失败事件并写入 `tracking_event`。VectorEngine 图片 provider 在 `platform-image` 内输出结构化日志和 `PlatformImageFailureAudit`，覆盖 `request_send`、`response_body`、`upstream_status`、`response_parse`、`missing_image` 和 `image_download` 阶段；编辑器 `screenColor=auto` 的 gpt-5-mini 背景色决策同样必须审计每次已发出的 LLM 调用失败，包括传输 / 超时、上游拒绝、响应体解析、空响应和返回候选外颜色；即使随后降级默认背景色并继续主流程也不得只记 warning。`api-server` 将这些失败映射成 `external_api_call_failure`，`scope_kind = module`、`scope_id = provider`、`module_key = external-api`。metadata 固定包含 provider、endpoint、operation、failureStage、statusCode、statusClass、timeout、retryable、errorMessage、latencyMs、promptChars、referenceImageCount、imageModel、rawExcerpt，以及在调用方可获得上下文时补充的 `userId`（触发者）和 `profileId`（草稿 / 作品 / 场景作用域）。图片生成入口应优先把 owner user id 和 profile id 透传到失败审计，不要只保留 provider 级聚合，否则很难按“谁触发、哪个作品触发”定位问题。入库优先复用 tracking outbox，outbox 不可写或保护阈值拒绝时回退同步写 SpacetimeDB；不得新增前端兜底或在 SpacetimeDB reducer 内做外部 I/O。
- 外部生成运行记录：所有外部生成编排的完成态统一写入 `tracking_event`，`event_key = external_generation_run`，`scope_kind = module`，`scope_id = provider`，`module_key = external-generation`。metadata 固定包含 `runId`、`provider`、`operation`、`requestLabel`、`requestPayload`、`status`、`success`、`failureReason`、`providerRequestId`、`resultPayload`、`startedAtMicros`、`completedAtMicros` 和 `durationMs`。这类记录只用于运行审计和排障，不再走 `ai_task` 旧表。
## SpacetimeDB 表目录
server-rs/crates/api-server/src/character_animation_assets.rs
+8
View File
@@ -689,6 +689,14 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
reference_count: 1,
source_image_data_url: Some(source_data_url.clone()),
},
Some(&crate::editor_screen_background_decision::EditorScreenBackgroundDecisionAuditContext {
state: &state,
context: crate::external_api_audit::ExternalApiAuditContext {
user_id: Some(owner_user_id.clone()),
profile_id: project_id.clone(),
request_id: Some(request_context.request_id().to_string()),
},
}),
)
.await?;
// 把实际选定的背景色落进资产记录，供后续以该资产为参考图时反查旧背景色。
server-rs/crates/api-server/src/editor_project.rs
+33
View File
@@ -1467,6 +1467,17 @@ pub(crate) async fn generate_editor_image_for_owner(
reference_count,
source_image_data_url: None,
},
Some(&crate::editor_screen_background_decision::EditorScreenBackgroundDecisionAuditContext {
state,
context: crate::external_api_audit::ExternalApiAuditContext {
user_id: caller.audit_subject_user_id.clone(),
profile_id: caller
.audit_project_id
.clone()
.or_else(|| payload.project_id.clone()),
request_id: Some(request_context.request_id().to_string()),
},
}),
)
.await?,
)
@@ -2980,6 +2991,17 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner(
reference_count,
source_image_data_url: None,
},
Some(&crate::editor_screen_background_decision::EditorScreenBackgroundDecisionAuditContext {
state,
context: crate::external_api_audit::ExternalApiAuditContext {
user_id: caller.audit_subject_user_id.clone(),
profile_id: caller
.audit_project_id
.clone()
.or_else(|| payload.project_id.clone()),
request_id: Some(request_context.request_id().to_string()),
},
}),
)
.await?;
let screen_color = screen_background_decision.color;
@@ -3266,6 +3288,17 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
reference_count,
source_image_data_url: None,
},
Some(&crate::editor_screen_background_decision::EditorScreenBackgroundDecisionAuditContext {
state,
context: crate::external_api_audit::ExternalApiAuditContext {
user_id: caller.audit_subject_user_id.clone(),
profile_id: caller
.audit_project_id
.clone()
.or_else(|| payload.project_id.clone()),
request_id: Some(request_context.request_id().to_string()),
},
}),
)
.await?;
let screen_color = screen_background_decision.color;
server-rs/crates/api-server/src/editor_screen_background_decision.rs
+342 -1
View File
@@ -1,4 +1,6 @@
use platform_llm::{LlmClient, LlmMessage, LlmResponseReasoningEffort, LlmTextRequest};
use platform_llm::{
LlmClient, LlmError, LlmMessage, LlmProvider, LlmResponseReasoningEffort, LlmTextRequest,
};
use serde_json::json;
use tracing::{info, warn};
@@ -11,7 +13,12 @@ use crate::{
ScreenBackgroundColorSafetyReport, decode_image_data_url,
filter_editor_screen_background_colors, safety_report_log_json,
},
external_api_audit::{
ExternalApiAuditContext, ExternalApiFailureDraft, is_retryable_external_api_failure,
record_external_api_failure,
},
http_error::AppError,
state::AppState,
};
const EDITOR_SCREEN_BACKGROUND_AUTO_VALUE: &str = "auto";
@@ -49,6 +56,15 @@ impl EditorScreenBackgroundDecisionKind {
Self::UiDesignAssetExtraction => "UI 设计图素材提取",
}
}
fn audit_operation_key(self) -> &'static str {
match self {
Self::Character => "character",
Self::CharacterAnimation => "character_animation",
Self::IconSpritesheet => "icon_spritesheet",
Self::UiDesignAssetExtraction => "ui_design_asset_extraction",
}
}
}
#[derive(Clone, Debug)]
@@ -63,10 +79,17 @@ pub(crate) struct EditorScreenBackgroundDecisionInput {
pub(crate) source_image_data_url: Option<String>,
}
/// 自动决策虽可业务降级，但已发出的 LLM 调用失败仍须进入 OTLP 和失败事件审计。
pub(crate) struct EditorScreenBackgroundDecisionAuditContext<'a> {
pub(crate) state: &'a AppState,
pub(crate) context: ExternalApiAuditContext,
}
pub(crate) async fn resolve_editor_screen_background_color(
llm_client: Option<&LlmClient>,
vision_llm_client: Option<&LlmClient>,
mut input: EditorScreenBackgroundDecisionInput,
audit: Option<&EditorScreenBackgroundDecisionAuditContext<'_>>,
) -> Result<EditorScreenBackgroundDecision, AppError> {
if !is_auto_screen_background_color(input.screen_color.as_deref()) {
return Ok(EditorScreenBackgroundDecision {
@@ -146,6 +169,11 @@ pub(crate) async fn resolve_editor_screen_background_color(
.as_deref()
.map(str::trim)
.filter(|value| !value.is_empty());
let audit_prompt_chars = system_prompt
.chars()
.count()
.saturating_add(user_prompt.chars().count());
let audit_reference_image_count = usize::from(source_image_data_url.is_some());
let mut last_error: Option<String> = None;
for attempt in 1..=EDITOR_SCREEN_BACKGROUND_DECISION_MAX_ATTEMPTS {
let user_message = match source_image_data_url {
@@ -195,6 +223,24 @@ pub(crate) async fn resolve_editor_screen_background_color(
message = message.as_str(),
"editor_screen_background_auto_decision_invalid_response"
);
record_editor_screen_background_decision_failure(
audit,
llm_client,
input.kind,
decision_model,
true,
attempt,
"response_parse",
Some(200),
false,
false,
message.as_str(),
None,
Some(response.content.as_str()),
audit_prompt_chars,
audit_reference_image_count,
)
.await;
last_error = Some(message);
}
}
@@ -207,6 +253,18 @@ pub(crate) async fn resolve_editor_screen_background_color(
message = message.as_str(),
"editor_screen_background_auto_decision_request_failed"
);
record_editor_screen_background_decision_llm_error(
audit,
llm_client,
input.kind,
decision_model,
true,
attempt,
&error,
audit_prompt_chars,
audit_reference_image_count,
)
.await;
last_error = Some(message);
}
}
@@ -227,6 +285,178 @@ pub(crate) async fn resolve_editor_screen_background_color(
})
}
async fn record_editor_screen_background_decision_llm_error(
audit: Option<&EditorScreenBackgroundDecisionAuditContext<'_>>,
llm_client: &LlmClient,
kind: EditorScreenBackgroundDecisionKind,
decision_model: Option<&'static str>,
uses_responses_api: bool,
attempt: u8,
error: &LlmError,
prompt_chars: usize,
reference_image_count: usize,
) {
let (failure_stage, status_code, timeout, retryable, error_source, raw_excerpt) = match error
{
LlmError::InvalidConfig(_) | LlmError::InvalidRequest(_) => return,
LlmError::Timeout { .. } => ("request_timeout", None, true, true, None, None),
LlmError::Connectivity { message, .. } => (
"request_send",
None,
false,
true,
Some(message.as_str()),
None,
),
LlmError::Upstream {
status_code,
message,
} => (
"upstream_status",
Some(*status_code),
false,
is_retryable_external_api_failure(Some(*status_code), false, false),
None,
Some(message.as_str()),
),
LlmError::Deserialize(message) => (
"response_parse",
Some(200),
false,
false,
Some(message.as_str()),
None,
),
LlmError::EmptyResponse => ("missing_response", Some(200), false, false, None, None),
LlmError::StreamUnavailable => ("response_body", Some(200), false, true, None, None),
LlmError::Transport(message) => (
"transport",
None,
false,
true,
Some(message.as_str()),
None,
),
};
record_editor_screen_background_decision_failure(
audit,
llm_client,
kind,
decision_model,
uses_responses_api,
attempt,
failure_stage,
status_code,
timeout,
retryable,
error.to_string().as_str(),
error_source,
raw_excerpt,
prompt_chars,
reference_image_count,
)
.await;
}
#[allow(clippy::too_many_arguments)]
async fn record_editor_screen_background_decision_failure(
audit: Option<&EditorScreenBackgroundDecisionAuditContext<'_>>,
llm_client: &LlmClient,
kind: EditorScreenBackgroundDecisionKind,
decision_model: Option<&'static str>,
uses_responses_api: bool,
attempt: u8,
failure_stage: &'static str,
status_code: Option<u16>,
timeout: bool,
retryable: bool,
error_message: &str,
error_source: Option<&str>,
raw_excerpt: Option<&str>,
prompt_chars: usize,
reference_image_count: usize,
) {
let Some(audit) = audit else {
return;
};
let draft = build_editor_screen_background_decision_failure_draft(
llm_client,
kind,
decision_model,
uses_responses_api,
attempt,
failure_stage,
status_code,
timeout,
retryable,
error_message,
error_source,
raw_excerpt,
prompt_chars,
reference_image_count,
&audit.context,
);
record_external_api_failure(audit.state, draft).await;
}
#[allow(clippy::too_many_arguments)]
fn build_editor_screen_background_decision_failure_draft(
llm_client: &LlmClient,
kind: EditorScreenBackgroundDecisionKind,
decision_model: Option<&'static str>,
uses_responses_api: bool,
attempt: u8,
failure_stage: &'static str,
status_code: Option<u16>,
timeout: bool,
retryable: bool,
error_message: &str,
error_source: Option<&str>,
raw_excerpt: Option<&str>,
prompt_chars: usize,
reference_image_count: usize,
context: &ExternalApiAuditContext,
) -> ExternalApiFailureDraft {
let endpoint = if uses_responses_api {
llm_client.config().responses_url()
} else {
llm_client.config().chat_completions_url()
};
let model = decision_model.unwrap_or_else(|| llm_client.config().model());
let message = format!(
"{}背景色自动决策第{attempt}/{EDITOR_SCREEN_BACKGROUND_DECISION_MAX_ATTEMPTS}次失败：{error_message}",
kind.label()
);
ExternalApiFailureDraft::new(
editor_screen_background_decision_provider(llm_client.config().provider()),
endpoint,
format!(
"editor_screen_background_decision:{}:{}",
kind.audit_operation_key(),
model
),
failure_stage,
message,
)
.with_status_code(status_code)
.with_timeout(timeout)
.with_retryable(retryable)
.with_error_source(error_source.map(str::to_string))
.with_raw_excerpt(raw_excerpt.map(str::to_string))
.with_prompt_chars(Some(prompt_chars))
.with_reference_image_count(Some(reference_image_count))
.with_image_model(None)
.with_audit_context(context)
}
fn editor_screen_background_decision_provider(provider: LlmProvider) -> &'static str {
match provider {
LlmProvider::OpenAiCompatible => "vector-engine",
LlmProvider::Ark => "ark",
LlmProvider::DashScope => "dash-scope",
}
}
/// 解出参考图并做候选色硬过滤；任何一步失败都返回 None（降级为不过滤）。
fn resolve_screen_background_safety_report(
input: &EditorScreenBackgroundDecisionInput,
@@ -398,6 +628,7 @@ mod tests {
reference_count: 1,
source_image_data_url: None,
},
None,
)
.await
.expect("manual color should parse");
@@ -420,6 +651,7 @@ mod tests {
reference_count: 2,
source_image_data_url: None,
},
None,
)
.await
.expect("auto fallback should succeed");
@@ -442,6 +674,7 @@ mod tests {
reference_count: 1,
source_image_data_url: None,
},
None,
)
.await
.expect("missing color should use auto fallback");
@@ -483,6 +716,111 @@ mod tests {
);
}
#[test]
fn decision_invalid_response_failure_audit_is_success_response_parse() {
use platform_llm::{LlmConfig, LlmProvider};
let client = LlmClient::new(
LlmConfig::new(
LlmProvider::OpenAiCompatible,
"https://vector.example/v1".to_string(),
"test-key".to_string(),
"fallback-model".to_string(),
20_000,
0,
1,
)
.expect("test LLM config should build"),
)
.expect("test LLM client should build");
let audit = build_editor_screen_background_decision_failure_draft(
&client,
EditorScreenBackgroundDecisionKind::Character,
Some("gpt-5-mini"),
true,
3,
"response_parse",
Some(200),
false,
false,
"LLM 返回了无法识别的背景色",
None,
Some("{\"hex\":\"#123456\"}"),
456,
0,
&ExternalApiAuditContext {
user_id: Some("user-1".to_string()),
profile_id: Some("project-1".to_string()),
request_id: Some("request-1".to_string()),
},
);
let tracking =
crate::external_api_audit::build_external_api_failure_tracking_draft(&audit);
assert_eq!(audit.provider, "vector-engine");
assert_eq!(audit.endpoint, "https://vector.example/v1/responses");
assert_eq!(
audit.operation,
"editor_screen_background_decision:character:gpt-5-mini"
);
assert_eq!(audit.failure_stage, "response_parse");
assert_eq!(audit.status_code, Some(200));
assert!(!audit.retryable);
assert_eq!(tracking.metadata["statusClass"], "2xx");
assert_eq!(tracking.metadata["promptChars"], 456);
assert_eq!(tracking.metadata["referenceImageCount"], 0);
assert_eq!(tracking.metadata["rawExcerpt"], "{\"hex\":\"#123456\"}");
assert_eq!(tracking.metadata["userId"], "user-1");
assert_eq!(tracking.metadata["profileId"], "project-1");
assert_eq!(tracking.metadata["requestId"], "request-1");
}
#[test]
fn decision_timeout_failure_audit_keeps_transport_semantics() {
use platform_llm::{LlmConfig, LlmProvider};
let client = LlmClient::new(
LlmConfig::new(
LlmProvider::OpenAiCompatible,
"https://vector.example/v1".to_string(),
"test-key".to_string(),
"fallback-model".to_string(),
20_000,
0,
1,
)
.expect("test LLM config should build"),
)
.expect("test LLM client should build");
let audit = build_editor_screen_background_decision_failure_draft(
&client,
EditorScreenBackgroundDecisionKind::CharacterAnimation,
Some("gpt-5-mini"),
true,
3,
"request_timeout",
None,
true,
true,
"LLM 请求超时，累计尝试 2 次",
None,
None,
789,
1,
&ExternalApiAuditContext::default(),
);
let tracking =
crate::external_api_audit::build_external_api_failure_tracking_draft(&audit);
assert_eq!(audit.failure_stage, "request_timeout");
assert_eq!(audit.status_code, None);
assert!(audit.timeout);
assert!(audit.retryable);
assert_eq!(tracking.metadata["statusClass"], "transport");
assert_eq!(tracking.metadata["promptChars"], 789);
assert_eq!(tracking.metadata["referenceImageCount"], 1);
}
#[tokio::test]
async fn auto_decision_fallback_avoids_colliding_default_color() {
use base64::Engine as _;
@@ -519,6 +857,7 @@ mod tests {
reference_count: 1,
source_image_data_url: Some(data_url),
},
None,
)
.await
.expect("decision should fall back");
@@ -620,6 +959,7 @@ mod tests {
reference_count: 1,
source_image_data_url: None,
},
None,
)
.await
.expect("live 无图决策应成功");
@@ -653,6 +993,7 @@ mod tests {
reference_count: 1,
source_image_data_url: Some(solid_source_image_data_url()),
},
None,
)
.await
.expect("live 有图决策应成功");