GenarrativeAI/Genarrative
10
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects 1 Releases Wiki Activity

修复错误报告权限与输入契约问题

Browse Source 
错误报告查询和管理 procedure 增加 runtime service identity 授权。

删除 user_id 与 now_micros 旧输入字段并改用服务端身份和时间。

增加 fingerprint、source、管理员备注和凭据脱敏校验。

收紧错误报告 OSS 对象键范围并同步生成绑定与技术文档。
This commit is contained in:
王德宇
2026-09-02 11:03:03 +08:00
parent 3cde591b67
commit 5120c7408c
8 changed files with 66 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/project-memory/shared-memory/decision-log.md
+1
View File
@@ -7868,3 +7868,4 @@ CI 上 `background_agent_runtime_recovers_stale_running_before_pending_task` 在
- 2026-09-01 追加：错误报告不依赖 api-server 本地文件、目录锁或同步文件 I/O；ZIP 只在请求内存构建后上传 OSS。管理员详情路由不属于 External OpenAPI；不存在返回 404，归档损坏返回 500。
- 2026-09-01 追加：错误报告不落本地文件；请求内存构建 ZIP 后直接上传 OSS，成功后写入 SpacetimeDB `error_report` 元数据。OSS key 固定为 `agc/error-reports/v1/{batchId}.zip`，不含日期；同一用户 `userId + submissionId` 幂等。管理员查询 DB，详情/下载按 object key 读取 OSS；每日清理先删 OSS，再删 DB，失败留待下次重试。
- 2026-09-01 review minor 修复：错误报告每日清理改为完整分页扫描，DB 删除失败显式返回以便下周期重试；OSS 读取按 `Content-Length` 与流式累计执行大小上限；管理员归档解析限制解压后 `events.jsonl` 为 24 MiB / 100 条事件。
- 2026-09-02 review breaking 项落地：错误报告 create/update procedure 删除 `user_id`、`now_micros` 输入字段，分别改用 `ctx.sender()`、`ctx.timestamp`；首个 fingerprint/source 强制 512 字符上限，备注超过 2,000 字显式拒绝；内部 OSS 对象键收紧为 `agc/error-reports/v1/`，凭据脱敏覆盖空白/分隔符变体。发布需同步 module、spacetime-client bindings 与 api-server。
docs/technical/【技术方案】AGC错误报告与诊断上传-2026-08-31.md
+2 -2
View File
@@ -19,14 +19,14 @@ AI Game Creator Shell 采用 IDEA 风格的当前进程错误报告：错误事
## HTTP 与存储
- 登录态客户端使用 `POST /api/error-reports`，请求 DTO 位于 `shared-contracts::error_reports`。
- api-server 对请求体设置 24 MiB 上限，并校验 schemaVersion、submissionId、事件/日志数量和 20 MiB 压缩包上限；事件字段、用户说明和日志名/内容均做长度限制与基础脱敏，归档使用 `events.jsonl`（每行一个事件）。结构化事件只保存在当前进程内，用户提交时才生成 `events.jsonl`，不在磁盘单独持久化。submissionId 提供重放幂等。
- api-server 对请求体设置 24 MiB 上限，并校验 schemaVersion、submissionId、事件/日志数量和 20 MiB 压缩包上限；事件字段、用户说明和日志名/内容均做长度限制与凭据脱敏，归档使用 `events.jsonl`（每行一个事件）。结构化事件只保存在当前进程内，用户提交时才生成 `events.jsonl`，不在磁盘单独持久化。submissionId 提供重放幂等；SpacetimeDB procedure 不接受调用方 `user_id` 或 `now_micros`，分别使用 `ctx.sender()` 与 `ctx.timestamp`。
- 归档构建只在请求生命周期内使用受 20 MiB 上限约束的内存 `Vec<u8>`，随后直接 PUT 到私有 OSS；服务端不写本地报告文件，也不保留本地索引。OSS 上传失败不写入数据库，调用方可稍后重新提交。
- 归档对象使用固定私有 OSS key：`agc/error-reports/v1/{batchId}.zip`；key 只由报告 UUID 决定，不包含时间戳。上传成功后才写入 SpacetimeDB `error_report` 元数据表；`userId + submissionId` 由唯一幂等键保证重放返回已有记录。完整事件、说明和日志只存在 OSS ZIP。
- `agc` 是服务端专用私有前缀；公共直传票据、通用 object-key 规范化和 legacy 公开路径均拒绝该前缀。归档内同名日志会自动加数字后缀，读取本机诊断日志时拒绝符号链接/非普通文件。
- 后台接口：`GET/PATCH /admin/api/error-reports/{batchId}`、`GET /admin/api/error-reports` 和受保护的 `/download`。列表支持 `limit`/`offset` 分页并返回 `total`、`hasMore`；OSS 读取先检查 `Content-Length` 并在流式累计超过上限时立即中止，不把超限对象完整缓存在内存中。
- 这些是 api-server 内部登录/管理员路由，不属于 `/api/external/v1`，不纳入 External OpenAPI；管理员详情对不存在返回 404，对归档/元数据损坏返回 500。
- admin viewer 仅接受 error-reports Tab 权限，支持列表筛选、分页、详情、状态 `new/in-progress/resolved`、处理备注和受控下载；不存在的更新目标返回 404，存储损坏返回 500。列表行支持键盘 Enter/Space 打开详情，详情事件预览最多显示 20 条，完整内容通过诊断包下载获取。
- 管理员列表、筛选、状态和备注全部读取/更新 SpacetimeDB；详情先读表再从 OSS 下载并解析 ZIP，下载接口直接从 OSS 返回 ZIP。无需新增管理员 DELETE HTTP 接口。每日清理任务按分页扫描全部报告，删除过期 OSS 对象后再删 DB；任一 DB 删除失败会保留错误并在后续周期重试。详情解析对解压后的 `events.jsonl` 设置 24 MiB（与请求体上限一致）与 100 条事件上限。旧本地报告不迁移。
- 管理员列表、筛选、状态和备注全部读取/更新 SpacetimeDB；错误报告的 list/get/update procedure 要求 `require_editor_generation_runtime_service_identity`，详情先读表再从 OSS 下载并解析 ZIP，下载接口直接从 OSS 返回 ZIP。无需新增管理员 DELETE HTTP 接口。每日清理任务按分页扫描全部报告，删除过期 OSS 对象后再删 DB；任一 DB 删除失败会保留错误并在后续周期重试。详情解析对解压后的 `events.jsonl` 设置 24 MiB（与请求体上限一致）与 100 条事件上限。管理员备注超过 2,000 字会被明确拒绝；内部 OSS 读写只允许 `agc/error-reports/v1/`。旧本地报告不迁移。
SpacetimeDB `error_report` 表字段：`batch_id` 主键、`user_id`、`submission_id`、`idempotency_key` 唯一键、`object_key`、`archive_sha256`、`archive_size_bytes`、`event_count`、`log_count`、首个 fingerprint/source、`review_status`、`admin_note`、`created_at`、`updated_at`；索引为 `(user_id, submission_id)`、`created_at`、`review_status`。
server-rs/crates/api-server/src/error_reports.rs
+22 -9
View File
@@ -46,6 +46,7 @@ const MAX_BATCH_BYTES: usize = 20 * 1024 * 1024;
const MAX_REQUEST_BODY_BYTES: usize = 24 * 1024 * 1024;
const MAX_EVENT_FIELD_CHARS: usize = 512;
const MAX_EVENTS_JSONL_BYTES: usize = MAX_REQUEST_BODY_BYTES;
const MAX_NOTE_CHARS: usize = 2_000;
#[derive(Clone, Debug, serde::Serialize)]
#[serde(rename_all = "camelCase")]
@@ -112,7 +113,6 @@ pub async fn create_error_report(
Extension(auth): Extension<AuthenticatedAccessToken>,
Json(mut payload): Json<CreateErrorReportBatchRequest>,
) -> Result<Json<Value>, AppError> {
let user_id = auth.claims().user_id().to_string();
validate_payload(&mut payload)?;
let archive = build_error_report_archive(
&payload.events,
@@ -156,7 +156,6 @@ pub async fn create_error_report(
.spacetime_client()
.create_error_report(ErrorReportCreateRecordInput {
batch_id: batch_id.clone(),
user_id,
submission_id: payload.submission_id.clone(),
idempotency_key: format!("{}:{}", auth.claims().user_id(), payload.submission_id),
object_key: object_key.clone(),
@@ -166,7 +165,6 @@ pub async fn create_error_report(
log_count: payload.logs.len() as u32,
first_fingerprint: payload.events.first().map(|e| e.fingerprint.clone()),
first_source: payload.events.first().map(|e| e.source.clone()),
now_micros: now_micros(),
})
.await
{
@@ -327,13 +325,18 @@ pub async fn admin_update_error_report(
if !matches!(p.status.as_str(), "new" | "in-progress" | "resolved") {
return Err(bad_request("状态必须是 new、in-progress 或 resolved"));
}
if p.note
.as_deref()
.is_some_and(|note| note.chars().count() > MAX_NOTE_CHARS)
{
return Err(bad_request("处理备注不能超过 2000 个字符"));
}
state
.spacetime_client()
.update_error_report(ErrorReportUpdateRecordInput {
batch_id: batch_id.clone(),
status: p.status,
note: p.note,
now_micros: now_micros(),
})
.await
.map_err(internal)?;
@@ -398,9 +401,6 @@ pub async fn admin_download_error_report(
Ok(resp)
}
fn now_micros() -> i64 {
(OffsetDateTime::now_utc().unix_timestamp_nanos() / 1_000) as i64
}
fn bad_request(m: impl Into<String>) -> AppError {
AppError::from_status(StatusCode::BAD_REQUEST).with_message(m)
}
@@ -556,16 +556,29 @@ fn sanitize_report_text(v: &str) -> String {
}
fn sanitize_report_text_with_limit(v: &str, max: usize) -> String {
let l = v.to_ascii_lowercase();
let normalized = l
.chars()
.filter(|c| !c.is_ascii_whitespace())
.collect::<String>();
if [
"authorization:",
"bearer ",
"authorization=",
"bearer",
"token=",
"token:",
"x-api-key:",
"x-api-key=",
"xapikey:",
"xapikey=",
"api-key:",
"api-key=",
"api_key:",
"api_key=",
"apikey:",
"apikey=",
]
.iter()
.any(|m| l.contains(m))
.any(|m| normalized.contains(m))
{
return "[REDACTED]".into();
}
server-rs/crates/platform-oss/src/lib.rs
+1 -1
View File
@@ -1781,7 +1781,7 @@ fn normalize_editor_agent_messages_object_key(raw: &str) -> Result<String, OssEr
fn normalize_internal_object_key(raw: &str) -> Result<String, OssError> {
let normalized = raw.trim().trim_start_matches('/').trim().to_string();
validate_object_key_segments(&normalized)?;
if normalized.starts_with("agc/error-reports/v1/") || normalized.starts_with("editor-agent/") {
if normalized.starts_with("agc/error-reports/v1/") {
Ok(normalized)
} else {
Err(OssError::InvalidRequest(
server-rs/crates/spacetime-client/src/mapper/error_reports.rs
-6
View File
@@ -22,7 +22,6 @@ pub struct ErrorReportRecord {
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ErrorReportCreateRecordInput {
pub batch_id: String,
pub user_id: String,
pub submission_id: String,
pub idempotency_key: String,
pub object_key: String,
@@ -32,7 +31,6 @@ pub struct ErrorReportCreateRecordInput {
pub log_count: u32,
pub first_fingerprint: Option<String>,
pub first_source: Option<String>,
pub now_micros: i64,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ErrorReportListRecordInput {
@@ -47,7 +45,6 @@ pub struct ErrorReportUpdateRecordInput {
pub batch_id: String,
pub status: String,
pub note: Option<String>,
pub now_micros: i64,
}
fn map(s: ErrorReportSnapshot) -> ErrorReportRecord {
@@ -91,7 +88,6 @@ impl From<ErrorReportCreateRecordInput> for ErrorReportCreateInput {
fn from(i: ErrorReportCreateRecordInput) -> Self {
Self {
batch_id: i.batch_id,
user_id: i.user_id,
submission_id: i.submission_id,
idempotency_key: i.idempotency_key,
object_key: i.object_key,
@@ -101,7 +97,6 @@ impl From<ErrorReportCreateRecordInput> for ErrorReportCreateInput {
log_count: i.log_count,
first_fingerprint: i.first_fingerprint,
first_source: i.first_source,
now_micros: i.now_micros,
}
}
}
@@ -122,7 +117,6 @@ impl From<ErrorReportUpdateRecordInput> for ErrorReportUpdateInput {
batch_id: i.batch_id,
status: i.status,
note: i.note,
now_micros: i.now_micros,
}
}
}
server-rs/crates/spacetime-client/src/module_bindings/error_report_create_input_type.rs
-2
View File
@@ -8,7 +8,6 @@ use spacetimedb_sdk::__codegen::{self as __sdk, __lib, __sats, __ws};
#[sats(crate = __lib)]
pub struct ErrorReportCreateInput {
pub batch_id: String,
pub user_id: String,
pub submission_id: String,
pub idempotency_key: String,
pub object_key: String,
@@ -18,7 +17,6 @@ pub struct ErrorReportCreateInput {
pub log_count: u32,
pub first_fingerprint: Option<String>,
pub first_source: Option<String>,
pub now_micros: i64,
}
impl __sdk::InModule for ErrorReportCreateInput {
server-rs/crates/spacetime-client/src/module_bindings/error_report_update_input_type.rs
-1
View File
@@ -10,7 +10,6 @@ pub struct ErrorReportUpdateInput {
pub batch_id: String,
pub status: String,
pub note: Option<String>,
pub now_micros: i64,
}
impl __sdk::InModule for ErrorReportUpdateInput {
server-rs/crates/spacetime-module/src/error_report.rs
+40 -11
View File
@@ -33,7 +33,6 @@ pub struct ErrorReport {
#[derive(Clone, Debug, PartialEq, Eq, SpacetimeType)]
pub struct ErrorReportCreateInput {
pub batch_id: String,
pub user_id: String,
pub submission_id: String,
pub idempotency_key: String,
pub object_key: String,
@@ -43,7 +42,6 @@ pub struct ErrorReportCreateInput {
pub log_count: u32,
pub first_fingerprint: Option<String>,
pub first_source: Option<String>,
pub now_micros: i64,
}
#[derive(Clone, Debug, PartialEq, Eq, SpacetimeType)]
@@ -65,7 +63,6 @@ pub struct ErrorReportUpdateInput {
pub batch_id: String,
pub status: String,
pub note: Option<String>,
pub now_micros: i64,
}
#[derive(Clone, Debug, PartialEq, Eq, SpacetimeType)]
@@ -144,9 +141,11 @@ pub fn create_error_report_and_return(
ctx: &mut ProcedureContext,
input: ErrorReportCreateInput,
) -> ErrorReportProcedureResult {
let caller_user_id = ctx.sender().to_hex().to_string();
let now = ctx.timestamp;
match ctx.try_with_tx(|tx| {
let batch_id = validate_text(&input.batch_id, "batch_id")?;
let user_id = validate_text(&input.user_id, "user_id")?;
let user_id = validate_text(&caller_user_id, "user_id")?;
let submission_id = validate_text(&input.submission_id, "submission_id")?;
let idempotency_key = validate_text(&input.idempotency_key, "idempotency_key")?;
let object_key = validate_text(&input.object_key, "object_key")?;
@@ -172,9 +171,8 @@ pub fn create_error_report_and_return(
if tx.db.error_report().batch_id().find(&batch_id).is_some() {
return Err("错误报告 batch_id 已存在".to_string());
}
let now = Timestamp::from_micros_since_unix_epoch(input.now_micros);
tx.db.error_report().insert(ErrorReport {
batch_id,
batch_id: batch_id.clone(),
user_id,
submission_id,
idempotency_key,
@@ -183,8 +181,16 @@ pub fn create_error_report_and_return(
archive_size_bytes: input.archive_size_bytes,
event_count: input.event_count,
log_count: input.log_count,
first_fingerprint: input.first_fingerprint.clone(),
first_source: input.first_source.clone(),
first_fingerprint: input
.first_fingerprint
.as_deref()
.map(|value| validate_text(value, "first_fingerprint"))
.transpose()?,
first_source: input
.first_source
.as_deref()
.map(|value| validate_text(value, "first_source"))
.transpose()?,
review_status: "new".to_string(),
admin_note: None,
created_at: now,
@@ -193,7 +199,7 @@ pub fn create_error_report_and_return(
tx.db
.error_report()
.batch_id()
.find(&input.batch_id)
.find(&batch_id)
.map(|row| snapshot(&row))
.ok_or_else(|| "错误报告写入后读取失败".to_string())
}) {
@@ -213,7 +219,11 @@ pub fn get_error_report_and_return(
ctx: &mut ProcedureContext,
input: ErrorReportGetInput,
) -> ErrorReportProcedureResult {
let caller = ctx.sender();
match ctx.try_with_tx(|tx| {
crate::editor_project_storage::require_editor_generation_runtime_service_identity(
tx, caller,
)?;
tx.db
.error_report()
.batch_id()
@@ -237,7 +247,11 @@ pub fn list_error_reports_and_return(
ctx: &mut ProcedureContext,
input: ErrorReportListInput,
) -> ErrorReportProcedureResult {
let caller = ctx.sender();
match ctx.try_with_tx(|tx| {
crate::editor_project_storage::require_editor_generation_runtime_service_identity(
tx, caller,
)?;
let mut rows = tx
.db
.error_report()
@@ -294,7 +308,12 @@ pub fn update_error_report_and_return(
ctx: &mut ProcedureContext,
input: ErrorReportUpdateInput,
) -> ErrorReportProcedureResult {
let caller = ctx.sender();
let now = ctx.timestamp;
match ctx.try_with_tx(|tx| {
crate::editor_project_storage::require_editor_generation_runtime_service_identity(
tx, caller,
)?;
if !matches!(input.status.as_str(), "new" | "in-progress" | "resolved") {
return Err("状态必须是 new、in-progress 或 resolved".to_string());
}
@@ -307,11 +326,17 @@ pub fn update_error_report_and_return(
let note = input
.note
.clone()
.map(|value| value.chars().take(MAX_NOTE_CHARS).collect::<String>());
.map(|value| {
if value.chars().count() > MAX_NOTE_CHARS {
return Err("error_report.note 超过 2000 个字符".to_string());
}
Ok(value)
})
.transpose()?;
let updated = ErrorReport {
review_status: input.status.clone(),
admin_note: note,
updated_at: Timestamp::from_micros_since_unix_epoch(input.now_micros),
updated_at: now,
..row.clone()
};
tx.db.error_report().batch_id().delete(&row.batch_id);
@@ -334,7 +359,11 @@ pub fn delete_error_report_and_return(
ctx: &mut ProcedureContext,
input: ErrorReportDeleteInput,
) -> ErrorReportProcedureResult {
let caller = ctx.sender();
match ctx.try_with_tx(|tx| {
crate::editor_project_storage::require_editor_generation_runtime_service_identity(
tx, caller,
)?;
let row = tx
.db
.error_report()