diff --git a/.hermes/shared-memory/decision-log.md b/.hermes/shared-memory/decision-log.md index bb78521f..1b2e161b 100644 --- a/.hermes/shared-memory/decision-log.md +++ b/.hermes/shared-memory/decision-log.md @@ -16,6 +16,14 @@ --- +## 2026-06-05 OSS 平台适配器输出结构化日志 + +- 背景:AI 生成资产、浏览器直传签名、私有读签名和对象确认都依赖 OSS;如果 OSS 侧只有错误字符串,排查资产写入 / 确认失败时很难按操作、对象、状态码和耗时下钻。 +- 决策:`server-rs/crates/platform-oss` 统一为 `sign_post_object`、`sign_get_object_url`、`head_object` 和 `put_object` 输出结构化日志。日志固定携带 `provider=aliyun-oss`、`operation`、`bucket`、`endpoint`、`object_key` / `key_prefix`、`access`、`content_type`、`content_length`、`status`、`status_class`、`error_kind` 和 `elapsed_ms` 等排障字段;禁止输出 AccessKey、policy、signature、Authorization header 或完整 signed URL。 +- 影响范围:`server-rs/crates/platform-oss`、`api-server` 资产签名 / 上传 / 确认链路、OTLP logs、本地 `logs/api-server/` 与运维排障文档。 +- 验证方式:`cargo test -p platform-oss --manifest-path server-rs/Cargo.toml`;真实联调时按 `provider=aliyun-oss` 与 `operation` 过滤日志,确认只出现对象定位和状态字段,不出现签名材料。 +- 关联文档:`server-rs/crates/platform-oss/README.md`、`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`、`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。 + ## 2026-06-03 创作入口关闭不下架已发布作品 - 背景:`creation_entry_disabled` 曾由 api-server 按 runtime 路由前缀统一熔断,导致用户进入平台首页或启动已发布作品时也可能看到“创作入口已关闭”错误。 diff --git a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md index 053fe85c..3fbda7e4 100644 --- a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md +++ b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md @@ -128,9 +128,10 @@ npm run check:server-rs-ddd 3. Adapter 输出应保留 legacy public path、object key、asset object id、MIME、extension、task id 和实际 prompt。 4. Adapter 不负责扣费、退款或钱包读取;计费仍由调用方显式包裹。 5. 图片 provider 协议不再放在玩法模块里实现。VectorEngine `gpt-image-2` 创建 / 编辑协议、URL / base64 图片解析、远端图片下载、请求超时 / 上游状态 / 响应解析 / 缺图 / 下载失败的结构化日志统一在 `server-rs/crates/platform-image/src/vector_engine/`;其中 `client.rs` 只保留 provider 调用编排,`transport.rs` 负责 HTTP client 与 reqwest 错误归一,`request.rs` 负责请求体和路径,`payload.rs` 负责响应 JSON 字段提取,`response.rs` 负责响应状态分流和图片结果归一。`api-server` 只负责配置校验、玩法 prompt 编排、OSS / asset object / binding 持久化、计费和外部 API 失败审计落库。 -6. Puzzle、Match3D、音频、GLB、视频等复杂媒体可以复用 OSS + asset object + binding 的底层持久化能力,但玩法专属处理规则留在各自编排层,不塞进公共接口。 -7. 拼图入口页与结果页新增关卡的本地参考图不走浏览器直传 OSS,前端读取为 Data URL 后随创作 action 提交,并在读取前限制 6MB、显示“图片≤6MB”。`api-server` 必须对 Data URL 实际字节数再次校验;历史图片才提交 `referenceImageAssetObjectId(s)`,后端校验 `asset_object` 的 bucket、kind、图片 MIME、大小和 owner 后签发只读 URL 给 VectorEngine 读取。 -8. 系列素材图集实现真相源在 `server-rs/crates/platform-image/src/generated_asset_sheets/`:调用方必须传入 `grid_size` 作为 `n*n` 的 `n`,可选传入物品名称 prompt 模板和特殊设定 prompt;模块负责 sheet prompt 组装、按 `n*n` 切片、透明化、PNG 输出、OSS private upload 请求构造和 sheet / item / special prompt 元数据持久化。`server-rs/crates/api-server/src/generated_asset_sheets.rs` 只保留 `AppState` / `AppError` 适配和兼容导出。玩法只负责规划 slot、调用具体生图 provider、计费、失败回写,以及把通用切片结果映射回自己的 DTO / 草稿 / runtime 字段。 +6. OSS 平台适配日志统一在 `server-rs/crates/platform-oss` 输出,覆盖 `sign_post_object`、`sign_get_object_url`、`head_object` 和 `put_object`。日志字段固定使用 `provider`、`operation`、`bucket`、`endpoint`、`object_key` / `key_prefix`、`access`、`content_type`、`content_length`、`status`、`status_class`、`error_kind` 和 `elapsed_ms`,只记录对象定位和排障信息;不得输出 AccessKey、policy、signature、Authorization header 或完整 signed URL。 +7. Puzzle、Match3D、音频、GLB、视频等复杂媒体可以复用 OSS + asset object + binding 的底层持久化能力,但玩法专属处理规则留在各自编排层,不塞进公共接口。 +8. 拼图入口页与结果页新增关卡的本地参考图不走浏览器直传 OSS,前端读取为 Data URL 后随创作 action 提交,并在读取前限制 6MB、显示“图片≤6MB”。`api-server` 必须对 Data URL 实际字节数再次校验;历史图片才提交 `referenceImageAssetObjectId(s)`,后端校验 `asset_object` 的 bucket、kind、图片 MIME、大小和 owner 后签发只读 URL 给 VectorEngine 读取。 +9. 系列素材图集实现真相源在 `server-rs/crates/platform-image/src/generated_asset_sheets/`:调用方必须传入 `grid_size` 作为 `n*n` 的 `n`,可选传入物品名称 prompt 模板和特殊设定 prompt;模块负责 sheet prompt 组装、按 `n*n` 切片、透明化、PNG 输出、OSS private upload 请求构造和 sheet / item / special prompt 元数据持久化。`server-rs/crates/api-server/src/generated_asset_sheets.rs` 只保留 `AppState` / `AppError` 适配和兼容导出。玩法只负责规划 slot、调用具体生图 provider、计费、失败回写,以及把通用切片结果映射回自己的 DTO / 草稿 / runtime 字段。 ## SpacetimeDB schema 变更规则 @@ -174,7 +175,7 @@ npm run check:server-rs-ddd - 敲木鱼敲击物和背景环境图:VectorEngine `/v1/images/edits`,模型固定 `gpt-image-2`。敲击物支持 multipart 多参考图,第一张固定为后端内嵌默认木鱼图,用户上传图只作为新主题参考;prompt 必须要求 `1:1` 真实透明 alpha PNG 并禁止黑底、白底、棋盘格和任何实底背景。当前敲击物上传 OSS 前不做服务端抠图后处理,避免误伤玉米等主体像素。背景环境图只使用第一步抠图完成后的透明敲击物图作为参考,prompt 必须要求中央主体预留区保持干净,中央 40% 区域禁止出现主题主体、主体局部特写、轮廓影子或重复元素,主题元素只能作为外围氛围,且必须显式声明不继承任何绿色底色、绿幕底色或纯绿色画布。 - Hyper3D / Rodin:只保留后端安全代理和旧数据兼容;Rodin 提交、状态、下载和响应解析归属 `platform-hyper3d`,`api-server/src/hyper3d_generation.rs` 只做路由、配置和错误 envelope 映射;新 Match3D 草稿和批量新增不再生成 GLB。 - 音频:视觉小说专用音频路由保留;VectorEngine Suno/Vidu provider 协议、任务提交/查询、音频 URL 提取、下载、MIME/extension 归一和 OSS put 请求准备归属 `platform-audio`。`api-server/src/vector_engine_audio_generation.rs` 只做路由、配置、计费、asset object confirm、entity binding 和错误 envelope 映射;拼图、抓大鹅和敲木鱼提示词生成音效入口暂时关闭,通用 `/api/creation/audio/*` 对这些目标返回 `410 Gone`。敲木鱼创作只接收上传 / 录音音频资产;未提供时由 `api-server` 写回内置默认木鱼音 `/wooden-fish/default-hit-sound.mp3`。 -- OSS:私有 generated legacy path 进入浏览器前必须通过 `/api/assets/read-url` 换签;不要裸请求 `/generated-*`。 +- OSS:私有 generated legacy path 进入浏览器前必须通过 `/api/assets/read-url` 换签;不要裸请求 `/generated-*`。OSS 签名、读签名、HEAD 和 PUT 的结构化日志由 `platform-oss` 输出,排查资产写入 / 确认失败时优先按 `operation`、`object_key` / `key_prefix`、`status_class`、`error_kind` 和 `elapsed_ms` 下钻。 - 外部 API 失败审计:外部供应商调用未成功时,`api-server` 必须发送 OTLP 失败事件并写入 `tracking_event`。VectorEngine 图片 provider 在 `platform-image` 内输出结构化日志和 `PlatformImageFailureAudit`,覆盖 `request_send`、`response_body`、`upstream_status`、`response_parse`、`missing_image` 和 `image_download` 阶段;`api-server` 只把该 audit 映射成 `external_api_call_failure`,`scope_kind = module`、`scope_id = provider`、`module_key = external-api`。metadata 固定包含 provider、endpoint、operation、failureStage、statusCode、statusClass、timeout、retryable、errorMessage、latencyMs、promptChars、referenceImageCount、imageModel、rawExcerpt,以及在调用方可获得上下文时补充的 `userId`(触发者)和 `profileId`(草稿 / 作品 / 场景作用域)。图片生成入口应优先把 owner user id 和 profile id 透传到失败审计,不要只保留 provider 级聚合,否则很难按“谁触发、哪个作品触发”定位问题。入库优先复用 tracking outbox,outbox 不可写或保护阈值拒绝时回退同步写 SpacetimeDB;不得新增前端兜底或在 SpacetimeDB reducer 内做外部 I/O。 - 外部生成运行记录:所有外部生成编排的完成态统一写入 `tracking_event`,`event_key = external_generation_run`,`scope_kind = module`,`scope_id = provider`,`module_key = external-generation`。metadata 固定包含 `runId`、`provider`、`operation`、`requestLabel`、`requestPayload`、`status`、`success`、`failureReason`、`providerRequestId`、`resultPayload`、`startedAtMicros`、`completedAtMicros` 和 `durationMs`。这类记录只用于运行审计和排障,不再走 `ai_task` 旧表。 diff --git a/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md b/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md index 55514d35..c22cfd7c 100644 --- a/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md +++ b/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md @@ -293,6 +293,7 @@ OpenTelemetry 现阶段默认开启 OTLP traces / metrics / logs,但本地日 - api-server 会随 metrics 发送进程级指标:`process.memory.usage`、`process.memory.virtual`、`process.cpu.time`、`genarrative.process.cpu.usage_percent`、`process.thread.count`、`genarrative.process.memory.private`;Windows 额外发送 `process.windows.handle.count`,Linux 额外发送 `process.unix.file_descriptor.count`。这些指标只描述当前进程,不携带请求、用户或作品 label。 - HTTP 运行态补充发送 `genarrative.http.server.response_bodies.in_flight` 与 `genarrative.http.server.request_permits.available`,后者带低基数 `pool=default|gallery|detail|admin` label,用于区分业务 handler / 背压 permit 是否仍被占用;拼图广场热点缓存补充发送 `genarrative.puzzle_gallery.cache.*` 指标,记录 fresh hit、stale hit、未命中、后台刷新开始 / 失败、重建耗时和预序列化 data JSON 字节数。 - 外部 API 失败统一发送 OTLP 并落库。当前 VectorEngine `gpt-image-2` 图片生成 / 编辑失败由 `platform-image` provider 输出结构化日志字段,字段包括 provider、endpoint、failure_stage、status、source、source_chain、source_chain_depth、timeout、retryable、latency_ms、prompt_chars、reference_image_count、image_model、request_params 和 raw_excerpt;图片编辑请求参数日志还会带 reference_image_bytes_total,并在 request_params.referenceImages 中记录每个 multipart `image` part 的 fileName、mimeType 和 bytes,不记录 API key 或原始图片 bytes;`api-server` 再记录指标 `genarrative.external_api.failures{provider,failure_stage,status_class,retryable}`,并写入 `tracking_event`,`event_key = external_api_call_failure`、`module_key = external-api`、`scope_kind = module`、`scope_id = provider`。调用方能拿到身份上下文时,失败事件还会在行级 `user_id` / `owner_user_id` / `profile_id` 和 `metadata_json.userId` / `metadata_json.profileId` / `metadata_json.requestId` / `metadata_json.errorSource` 中记录触发者、草稿 / 作品作用域、请求标识和传输错误链。排障时先按 provider / failureStage 聚合,再下钻 userId / profileId,最后结合 request 日志、errorSource 和上游响应 excerpt 判断是限流、超时、解析失败还是未返回图片。 +- OSS 平台适配器也输出结构化日志,覆盖 `sign_post_object`、`sign_get_object_url`、`head_object` 和 `put_object`。排查资产签名、上传或确认失败时,先按 `provider=aliyun-oss` 与 `operation` 过滤,再看 `object_key` / `key_prefix`、`status`、`status_class`、`error_kind`、`content_length`、`content_type` 和 `elapsed_ms`;日志不得包含 AccessKey、policy、signature、Authorization header 或完整 signed URL。 - SpacetimeDB 观测分为两类:procedure / reducer 调用继续用 `genarrative.spacetime.procedure.*`,订阅本地 cache 读使用 `genarrative.spacetime.read.*`。`read=list_puzzle_gallery` 表示拼图广场当前从 `puzzle_gallery_card_view` 本地 cache 读取,不再每个 HTTP 请求调用 `list_puzzle_gallery` procedure。 - 本地 Windows 直连压测的内存高水位要结合 K6 VU / 连接数解释。250 RPS 下过高 `PREALLOCATED_VUS` 可能让 300 个本地 Established 连接把 `api-server` private memory 瞬时推到 GB 级,且 `/healthz` 小响应也能复现;若压测结束后回落、`response_bodies.in_flight` 和背压 permit 未显示业务积压,应优先按连接 / 发送链路高水位处理,而不是判断为 SpacetimeDB 或 JSON 缓存泄漏。 - Rider 的 Logs 面板只展示 log event 自身字段,不会自动展开父 span 的全部 attributes;请求完成日志会直接带 `request_id`、`http.request.method`、`http.route`、`url.scheme`、`url.path`、`http.response.status_code`、`status_class`、`latency_ms` 和 `slow_request`,完整链路继续到 Traces 面板按 trace/span 查看。 diff --git a/server-rs/Cargo.lock b/server-rs/Cargo.lock index e80fbd9c..4024b992 100644 --- a/server-rs/Cargo.lock +++ b/server-rs/Cargo.lock @@ -2436,6 +2436,7 @@ dependencies = [ "sha2", "time", "tokio", + "tracing", ] [[package]] diff --git a/server-rs/crates/platform-oss/Cargo.toml b/server-rs/crates/platform-oss/Cargo.toml index 216e5955..2b8a9b9e 100644 --- a/server-rs/crates/platform-oss/Cargo.toml +++ b/server-rs/crates/platform-oss/Cargo.toml @@ -12,6 +12,7 @@ serde = { workspace = true } serde_json = { workspace = true } sha2 = { workspace = true } time = { workspace = true, features = ["formatting"] } +tracing = { workspace = true } [dev-dependencies] tokio = { workspace = true, features = ["macros", "rt"] } diff --git a/server-rs/crates/platform-oss/README.md b/server-rs/crates/platform-oss/README.md index 025481d3..84d38def 100644 --- a/server-rs/crates/platform-oss/README.md +++ b/server-rs/crates/platform-oss/README.md @@ -22,6 +22,7 @@ 5. 服务端 `PutObject` 上传 helper 6. `x-oss-meta-*` 元数据归一化与大小限制校验 7. `content-type`、`content-length-range`、`success_action_status` policy 条件生成 +8. `PostObject` 签名、`GetObject` 读签名、`HEAD Object` 和 `PutObject` 的结构化日志 当前仍未落地的内容: @@ -34,8 +35,9 @@ 1. 当前产品口径为服务器上传 AI 生成资源、Web 端只负责读取。 2. 因此 `STS` 不作为默认上传主链,`api-server` 只暴露禁用式 contract,避免浏览器拿到 OSS 写权限。 3. 服务端生成资源应优先复用 `OssClient::put_object`,上传成功后再走对象确认链路写入 `asset_object`。 - 4. 读签名和 `HEAD Object` 的入参必须直接传 object_key,不要把 bucket 名拼进路径;例如 `generated-square-hole-assets/.../image.png` 才是正确入参,`xushi-dev/...` 这类前缀不属于 object_key。 - 5. OSS V4 `x-oss-date` 必须固定为 `yyyyMMdd'T'HHmmss'Z'`,不能依赖 `time::Time::to_string()`;后者在小时小于 10 时可能输出非补零时间,导致签名格式错误。 +4. 读签名和 `HEAD Object` 的入参必须直接传 object_key,不要把 bucket 名拼进路径;例如 `generated-square-hole-assets/.../image.png` 才是正确入参,`xushi-dev/...` 这类前缀不属于 object_key。 +5. OSS V4 `x-oss-date` 必须固定为 `yyyyMMdd'T'HHmmss'Z'`,不能依赖 `time::Time::to_string()`;后者在小时小于 10 时可能输出非补零时间,导致签名格式错误。 +6. 结构化日志只记录 `provider`、`operation`、`bucket`、`endpoint`、`object_key` / `key_prefix`、`access`、`content_type`、`content_length`、`status`、`status_class`、`error_kind` 和 `elapsed_ms` 等排障字段;禁止输出 AccessKey、policy、signature、Authorization header 或完整 signed URL。 ## 3. 边界约束 diff --git a/server-rs/crates/platform-oss/src/lib.rs b/server-rs/crates/platform-oss/src/lib.rs index a9b3935e..656db455 100644 --- a/server-rs/crates/platform-oss/src/lib.rs +++ b/server-rs/crates/platform-oss/src/lib.rs @@ -1,4 +1,4 @@ -use std::{collections::BTreeMap, error::Error, fmt}; +use std::{collections::BTreeMap, error::Error, fmt, time::Instant}; use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64_STANDARD}; use hmac::{Hmac, Mac}; @@ -7,6 +7,7 @@ use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; use sha2::{Digest, Sha256}; use time::{Duration, OffsetDateTime, format_description::well_known::Rfc3339}; +use tracing::{info, warn}; type HmacSha256 = Hmac; @@ -19,6 +20,7 @@ const OSS_V4_ALGORITHM: &str = "OSS4-HMAC-SHA256"; const OSS_V4_REQUEST: &str = "aliyun_v4_request"; const OSS_V4_SERVICE: &str = "oss"; const OSS_UNSIGNED_PAYLOAD: &str = "UNSIGNED-PAYLOAD"; +const OSS_PROVIDER: &str = "aliyun-oss"; pub const LEGACY_PUBLIC_PREFIXES: [&str; 13] = [ "generated-character-drafts", @@ -369,105 +371,154 @@ impl OssClient { &self, request: OssPostObjectRequest, ) -> Result { - let max_size_bytes = request - .max_size_bytes - .unwrap_or(self.config.default_post_max_size_bytes); - let expire_seconds = request - .expire_seconds - .unwrap_or(self.config.default_post_expire_seconds); - let success_action_status = request - .success_action_status - .unwrap_or(self.config.default_success_action_status); + let started_at = Instant::now(); + let requested_prefix = request.prefix.as_str(); + let requested_content_type = request + .content_type + .as_deref() + .map(str::trim) + .unwrap_or("") + .to_string(); + let requested_metadata_count = request.metadata.len(); - if max_size_bytes == 0 { - return Err(OssError::InvalidRequest( - "maxSizeBytes 必须大于 0".to_string(), - )); + let result = (|| { + let max_size_bytes = request + .max_size_bytes + .unwrap_or(self.config.default_post_max_size_bytes); + let expire_seconds = request + .expire_seconds + .unwrap_or(self.config.default_post_expire_seconds); + let success_action_status = request + .success_action_status + .unwrap_or(self.config.default_success_action_status); + + if max_size_bytes == 0 { + return Err(OssError::InvalidRequest( + "maxSizeBytes 必须大于 0".to_string(), + )); + } + + if expire_seconds == 0 { + return Err(OssError::InvalidRequest( + "expireSeconds 必须大于 0".to_string(), + )); + } + + if !(100..=999).contains(&success_action_status) { + return Err(OssError::InvalidRequest( + "successActionStatus 必须是三位 HTTP 状态码".to_string(), + )); + } + + let sanitized_segments = request + .path_segments + .iter() + .map(|segment| sanitize_path_segment(segment)) + .filter(|segment| !segment.is_empty()) + .collect::>(); + let file_name = sanitize_file_name(&request.file_name)?; + let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name); + let legacy_public_path = format!("/{}", object_key); + let content_type = normalize_optional_value(request.content_type); + let metadata = normalize_metadata(request.metadata)?; + + let expires_at = OffsetDateTime::now_utc() + .checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err( + |_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()), + )?)) + .ok_or_else(|| { + OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string()) + })?; + let expires_at = expires_at.format(&Rfc3339).map_err(|error| { + OssError::SerializePolicy(format!("格式化过期时间失败:{error}")) + })?; + + let signed_at = OffsetDateTime::now_utc(); + let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?; + let signature_date = build_v4_signature_date(signed_at)?; + let credential = format!("{}/{}", self.config.access_key_id, signature_scope); + let policy_json = build_policy_json( + &self.config.bucket, + &object_key, + &expires_at, + max_size_bytes, + success_action_status, + content_type.as_deref(), + &metadata, + &credential, + &signature_date, + ); + let policy = serde_json::to_string(&policy_json).map_err(|error| { + OssError::SerializePolicy(format!("序列化 policy 失败:{error}")) + })?; + let encoded_policy = BASE64_STANDARD.encode(policy.as_bytes()); + let signature = sign_v4_content( + &self.config.access_key_secret, + &signature_scope, + &encoded_policy, + )?; + + Ok(OssPostObjectResponse { + signature_version: "v4", + provider: OSS_PROVIDER, + bucket: self.config.bucket.clone(), + endpoint: self.config.endpoint.clone(), + host: self.config.upload_host(), + object_key: object_key.clone(), + legacy_public_path, + content_type: content_type.clone(), + access: request.access, + key_prefix: build_key_prefix(request.prefix, &sanitized_segments), + expires_at, + max_size_bytes, + success_action_status, + form_fields: OssPostObjectFormFields { + key: object_key, + policy: encoded_policy, + signature_version: OSS_V4_ALGORITHM.to_string(), + credential, + date: signature_date, + signature, + success_action_status: success_action_status.to_string(), + content_type, + metadata, + }, + }) + })(); + + match &result { + Ok(response) => info!( + provider = OSS_PROVIDER, + operation = "sign_post_object", + bucket = %response.bucket, + endpoint = %response.endpoint, + object_key = %response.object_key, + key_prefix = %response.key_prefix, + access = oss_access_label(response.access), + content_type = %response.content_type.as_deref().unwrap_or(""), + max_size_bytes = response.max_size_bytes, + success_action_status = response.success_action_status, + metadata_count = response.form_fields.metadata.len(), + expires_at = %response.expires_at, + elapsed_ms = elapsed_ms(started_at), + "OSS PostObject 签名完成" + ), + Err(error) => warn!( + provider = OSS_PROVIDER, + operation = "sign_post_object", + bucket = %self.config.bucket(), + endpoint = %self.config.endpoint(), + key_prefix = requested_prefix, + content_type = %requested_content_type, + metadata_count = requested_metadata_count, + error_kind = oss_error_kind_label(error), + message = %error, + elapsed_ms = elapsed_ms(started_at), + "OSS PostObject 签名失败" + ), } - if expire_seconds == 0 { - return Err(OssError::InvalidRequest( - "expireSeconds 必须大于 0".to_string(), - )); - } - - if !(100..=999).contains(&success_action_status) { - return Err(OssError::InvalidRequest( - "successActionStatus 必须是三位 HTTP 状态码".to_string(), - )); - } - - let sanitized_segments = request - .path_segments - .iter() - .map(|segment| sanitize_path_segment(segment)) - .filter(|segment| !segment.is_empty()) - .collect::>(); - let file_name = sanitize_file_name(&request.file_name)?; - let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name); - let legacy_public_path = format!("/{}", object_key); - let content_type = normalize_optional_value(request.content_type); - let metadata = normalize_metadata(request.metadata)?; - - let expires_at = OffsetDateTime::now_utc() - .checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err( - |_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()), - )?)) - .ok_or_else(|| OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string()))?; - let expires_at = expires_at - .format(&Rfc3339) - .map_err(|error| OssError::SerializePolicy(format!("格式化过期时间失败:{error}")))?; - - let signed_at = OffsetDateTime::now_utc(); - let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?; - let signature_date = build_v4_signature_date(signed_at)?; - let credential = format!("{}/{}", self.config.access_key_id, signature_scope); - let policy_json = build_policy_json( - &self.config.bucket, - &object_key, - &expires_at, - max_size_bytes, - success_action_status, - content_type.as_deref(), - &metadata, - &credential, - &signature_date, - ); - let policy = serde_json::to_string(&policy_json) - .map_err(|error| OssError::SerializePolicy(format!("序列化 policy 失败:{error}")))?; - let encoded_policy = BASE64_STANDARD.encode(policy.as_bytes()); - let signature = sign_v4_content( - &self.config.access_key_secret, - &signature_scope, - &encoded_policy, - )?; - - Ok(OssPostObjectResponse { - signature_version: "v4", - provider: "aliyun-oss", - bucket: self.config.bucket.clone(), - endpoint: self.config.endpoint.clone(), - host: self.config.upload_host(), - object_key: object_key.clone(), - legacy_public_path, - content_type: content_type.clone(), - access: request.access, - key_prefix: build_key_prefix(request.prefix, &sanitized_segments), - expires_at, - max_size_bytes, - success_action_status, - form_fields: OssPostObjectFormFields { - key: object_key, - policy: encoded_policy, - signature_version: OSS_V4_ALGORITHM.to_string(), - credential, - date: signature_date, - signature, - success_action_status: success_action_status.to_string(), - content_type, - metadata, - }, - }) + result } // 私有 bucket 的对象读取统一走短期签名 URL,避免把长期主凭证下发给浏览器。 @@ -475,81 +526,119 @@ impl OssClient { &self, request: OssSignedGetObjectUrlRequest, ) -> Result { - let expire_seconds = request - .expire_seconds - .unwrap_or(self.config.default_read_expire_seconds); + let started_at = Instant::now(); + let requested_object_key = request + .object_key + .trim() + .trim_start_matches('/') + .trim() + .to_string(); - if expire_seconds == 0 { - return Err(OssError::InvalidRequest( - "expireSeconds 必须大于 0".to_string(), - )); + let result = (|| { + let expire_seconds = request + .expire_seconds + .unwrap_or(self.config.default_read_expire_seconds); + + if expire_seconds == 0 { + return Err(OssError::InvalidRequest( + "expireSeconds 必须大于 0".to_string(), + )); + } + + let object_key = normalize_object_key(&request.object_key)?; + let expires_at = OffsetDateTime::now_utc() + .checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err( + |_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()), + )?)) + .ok_or_else(|| { + OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string()) + })?; + let expires_at_text = expires_at + .format(&Rfc3339) + .map_err(|error| OssError::Sign(format!("格式化过期时间失败:{error}")))?; + + let signed_at = OffsetDateTime::now_utc(); + let signed_at_text = build_v4_signature_date(signed_at)?; + let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?; + let credential = format!("{}/{}", self.config.access_key_id, signature_scope); + let mut query = BTreeMap::from([ + ("x-oss-additional-headers".to_string(), "host".to_string()), + ( + "x-oss-signature-version".to_string(), + OSS_V4_ALGORITHM.to_string(), + ), + ("x-oss-credential".to_string(), credential), + ("x-oss-date".to_string(), signed_at_text), + ("x-oss-expires".to_string(), expire_seconds.to_string()), + ]); + let canonical_uri = build_v4_canonical_uri(&self.config.bucket, Some(&object_key)); + let object_url_path = format!("/{}", encode_url_path(&object_key)); + let additional_headers = "host"; + let canonical_headers = + format!("host:{}.{}\n", self.config.bucket(), self.config.endpoint()); + let canonical_query = build_canonical_query_string(&query); + let canonical_request = build_v4_canonical_request( + Method::GET.as_str(), + &canonical_uri, + &canonical_query, + &canonical_headers, + additional_headers, + OSS_UNSIGNED_PAYLOAD, + ); + let string_to_sign = build_v4_string_to_sign( + query["x-oss-date"].as_str(), + &signature_scope, + &canonical_request, + ); + let signature = sign_v4_content( + &self.config.access_key_secret, + &signature_scope, + &string_to_sign, + )?; + query.insert("x-oss-signature".to_string(), signature); + let signed_url = format!( + "{}{}?{}", + self.config.upload_host(), + object_url_path, + build_canonical_query_string(&query) + ); + + Ok(OssSignedGetObjectUrlResponse { + provider: OSS_PROVIDER, + bucket: self.config.bucket.clone(), + endpoint: self.config.endpoint.clone(), + host: self.config.upload_host(), + object_key, + expires_at: expires_at_text, + signed_url, + }) + })(); + + match &result { + Ok(response) => info!( + provider = OSS_PROVIDER, + operation = "sign_get_object_url", + bucket = %response.bucket, + endpoint = %response.endpoint, + object_key = %response.object_key, + expires_at = %response.expires_at, + elapsed_ms = elapsed_ms(started_at), + "OSS GetObject 读签名完成" + ), + Err(error) => warn!( + provider = OSS_PROVIDER, + operation = "sign_get_object_url", + bucket = %self.config.bucket(), + endpoint = %self.config.endpoint(), + object_key = %requested_object_key, + error_kind = oss_error_kind_label(error), + message = %error, + elapsed_ms = elapsed_ms(started_at), + "OSS GetObject 读签名失败" + ), } - let object_key = normalize_object_key(&request.object_key)?; - let expires_at = OffsetDateTime::now_utc() - .checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err( - |_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()), - )?)) - .ok_or_else(|| OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string()))?; - let expires_at_text = expires_at - .format(&Rfc3339) - .map_err(|error| OssError::Sign(format!("格式化过期时间失败:{error}")))?; - - let signed_at = OffsetDateTime::now_utc(); - let signed_at_text = build_v4_signature_date(signed_at)?; - let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?; - let credential = format!("{}/{}", self.config.access_key_id, signature_scope); - let mut query = BTreeMap::from([ - ("x-oss-additional-headers".to_string(), "host".to_string()), - ( - "x-oss-signature-version".to_string(), - OSS_V4_ALGORITHM.to_string(), - ), - ("x-oss-credential".to_string(), credential), - ("x-oss-date".to_string(), signed_at_text), - ("x-oss-expires".to_string(), expire_seconds.to_string()), - ]); - let canonical_uri = build_v4_canonical_uri(&self.config.bucket, Some(&object_key)); - let object_url_path = format!("/{}", encode_url_path(&object_key)); - let additional_headers = "host"; - let canonical_headers = - format!("host:{}.{}\n", self.config.bucket(), self.config.endpoint()); - let canonical_query = build_canonical_query_string(&query); - let canonical_request = build_v4_canonical_request( - Method::GET.as_str(), - &canonical_uri, - &canonical_query, - &canonical_headers, - additional_headers, - OSS_UNSIGNED_PAYLOAD, - ); - let string_to_sign = build_v4_string_to_sign( - query["x-oss-date"].as_str(), - &signature_scope, - &canonical_request, - ); - let signature = sign_v4_content( - &self.config.access_key_secret, - &signature_scope, - &string_to_sign, - )?; - query.insert("x-oss-signature".to_string(), signature); - let signed_url = format!( - "{}{}?{}", - self.config.upload_host(), - object_url_path, - build_canonical_query_string(&query) - ); - - Ok(OssSignedGetObjectUrlResponse { - provider: "aliyun-oss", - bucket: self.config.bucket.clone(), - endpoint: self.config.endpoint.clone(), - host: self.config.upload_host(), - object_key, - expires_at: expires_at_text, - signed_url, - }) + result } // 上传完成确认前,服务端必须自己探测一次对象,不能只相信客户端回传的 object_key。 @@ -558,59 +647,107 @@ impl OssClient { client: &reqwest::Client, request: OssHeadObjectRequest, ) -> Result { - let object_key = normalize_object_key(&request.object_key)?; - let target_url = build_object_url(&self.config.bucket, &self.config.endpoint, &object_key) - .map_err(|error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")))?; - let response = send_signed_request( - client, - &self.config, - Method::HEAD, - Some(&object_key), - target_url, - ) - .await?; + let started_at = Instant::now(); + let requested_object_key = request + .object_key + .trim() + .trim_start_matches('/') + .trim() + .to_string(); + let mut response_status = None; - if response.status() == reqwest::StatusCode::NOT_FOUND { - return Err(OssError::ObjectNotFound(format!( - "OSS 对象不存在:{}", - request.object_key - ))); + let result = async { + let object_key = normalize_object_key(&request.object_key)?; + let target_url = + build_object_url(&self.config.bucket, &self.config.endpoint, &object_key).map_err( + |error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")), + )?; + let response = send_signed_request( + client, + &self.config, + Method::HEAD, + Some(&object_key), + target_url, + ) + .await?; + response_status = Some(response.status().as_u16()); + + if response.status() == reqwest::StatusCode::NOT_FOUND { + return Err(OssError::ObjectNotFound(format!( + "OSS 对象不存在:{}", + request.object_key + ))); + } + + if !response.status().is_success() { + return Err(OssError::Request(format!( + "OSS HEAD Object 失败,状态码:{}", + response.status() + ))); + } + + let headers = response.headers(); + let content_length = headers + .get(reqwest::header::CONTENT_LENGTH) + .and_then(|value| value.to_str().ok()) + .and_then(|value| value.parse::().ok()) + .unwrap_or(0); + let content_type = headers + .get(reqwest::header::CONTENT_TYPE) + .and_then(|value| value.to_str().ok()) + .map(|value| value.to_string()); + let etag = headers + .get(reqwest::header::ETAG) + .and_then(|value| value.to_str().ok()) + .map(|value| value.trim_matches('"').to_string()); + let last_modified = headers + .get(reqwest::header::LAST_MODIFIED) + .and_then(|value| value.to_str().ok()) + .map(|value| value.to_string()); + + Ok(OssHeadObjectResponse { + bucket: self.config.bucket.clone(), + object_key, + content_length, + content_type, + etag, + last_modified, + }) + } + .await; + + match &result { + Ok(response) => info!( + provider = OSS_PROVIDER, + operation = "head_object", + bucket = %response.bucket, + endpoint = %self.config.endpoint(), + object_key = %response.object_key, + status = response_status.unwrap_or(reqwest::StatusCode::OK.as_u16()), + status_class = http_status_class_from_option(response_status), + content_length = response.content_length, + content_type = %response.content_type.as_deref().unwrap_or(""), + etag_present = response.etag.is_some(), + last_modified_present = response.last_modified.is_some(), + elapsed_ms = elapsed_ms(started_at), + "OSS HEAD Object 完成" + ), + Err(error) => warn!( + provider = OSS_PROVIDER, + operation = "head_object", + bucket = %self.config.bucket(), + endpoint = %self.config.endpoint(), + object_key = %requested_object_key, + status = response_status.unwrap_or_default(), + status_class = http_status_class_from_option(response_status), + error_kind = oss_error_kind_label(error), + message = %error, + elapsed_ms = elapsed_ms(started_at), + "OSS HEAD Object 失败" + ), } - if !response.status().is_success() { - return Err(OssError::Request(format!( - "OSS HEAD Object 失败,状态码:{}", - response.status() - ))); - } - - let headers = response.headers(); - let content_length = headers - .get(reqwest::header::CONTENT_LENGTH) - .and_then(|value| value.to_str().ok()) - .and_then(|value| value.parse::().ok()) - .unwrap_or(0); - let content_type = headers - .get(reqwest::header::CONTENT_TYPE) - .and_then(|value| value.to_str().ok()) - .map(|value| value.to_string()); - let etag = headers - .get(reqwest::header::ETAG) - .and_then(|value| value.to_str().ok()) - .map(|value| value.trim_matches('"').to_string()); - let last_modified = headers - .get(reqwest::header::LAST_MODIFIED) - .and_then(|value| value.to_str().ok()) - .map(|value| value.to_string()); - - Ok(OssHeadObjectResponse { - bucket: self.config.bucket.clone(), - object_key, - content_length, - content_type, - etag, - last_modified, - }) + result } // AI 生成资源默认由服务端上传 OSS,Web 端只拿签名读地址,不直接持有写权限。 @@ -619,73 +756,128 @@ impl OssClient { client: &reqwest::Client, request: OssPutObjectRequest, ) -> Result { - if request.body.is_empty() { - return Err(OssError::InvalidRequest( - "服务端上传对象内容不能为空".to_string(), - )); + let started_at = Instant::now(); + let requested_prefix = request.prefix.as_str(); + let requested_content_type = request + .content_type + .as_deref() + .map(str::trim) + .unwrap_or("") + .to_string(); + let requested_content_length = request.body.len(); + let requested_metadata_count = request.metadata.len(); + let mut response_status = None; + + let result = async { + if request.body.is_empty() { + return Err(OssError::InvalidRequest( + "服务端上传对象内容不能为空".to_string(), + )); + } + + let sanitized_segments = request + .path_segments + .iter() + .map(|segment| sanitize_path_segment(segment)) + .filter(|segment| !segment.is_empty()) + .collect::>(); + let file_name = sanitize_file_name(&request.file_name)?; + let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name); + let content_type = normalize_optional_value(request.content_type); + let metadata = normalize_metadata(request.metadata)?; + let target_url = + build_object_url(&self.config.bucket, &self.config.endpoint, &object_key).map_err( + |error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")), + )?; + let content_length = u64::try_from(request.body.len()) + .map_err(|_| OssError::InvalidRequest("上传对象大小超出可支持范围".to_string()))?; + let builder = signed_request_builder( + client, + &self.config, + Method::PUT, + Some(&object_key), + target_url, + content_type.as_deref(), + &metadata, + )? + .header(reqwest::header::CONTENT_LENGTH, content_length) + .body(request.body); + + let response = builder + .send() + .await + .map_err(|error| OssError::Request(format!("请求 OSS 失败:{error}")))?; + response_status = Some(response.status().as_u16()); + + if !response.status().is_success() { + return Err(OssError::Request(format!( + "OSS PutObject 失败,状态码:{}", + response.status() + ))); + } + + let headers = response.headers(); + let etag = headers + .get(reqwest::header::ETAG) + .and_then(|value| value.to_str().ok()) + .map(|value| value.trim_matches('"').to_string()); + let last_modified = headers + .get(reqwest::header::LAST_MODIFIED) + .and_then(|value| value.to_str().ok()) + .map(|value| value.to_string()); + + Ok(OssPutObjectResponse { + provider: OSS_PROVIDER, + bucket: self.config.bucket.clone(), + endpoint: self.config.endpoint.clone(), + host: self.config.upload_host(), + legacy_public_path: format!("/{object_key}"), + object_key, + content_type, + content_length, + access: request.access, + etag, + last_modified, + }) + } + .await; + + match &result { + Ok(response) => info!( + provider = OSS_PROVIDER, + operation = "put_object", + bucket = %response.bucket, + endpoint = %response.endpoint, + object_key = %response.object_key, + access = oss_access_label(response.access), + status = response_status.unwrap_or(reqwest::StatusCode::OK.as_u16()), + status_class = http_status_class_from_option(response_status), + content_length = response.content_length, + content_type = %response.content_type.as_deref().unwrap_or(""), + etag_present = response.etag.is_some(), + last_modified_present = response.last_modified.is_some(), + elapsed_ms = elapsed_ms(started_at), + "OSS PutObject 上传完成" + ), + Err(error) => warn!( + provider = OSS_PROVIDER, + operation = "put_object", + bucket = %self.config.bucket(), + endpoint = %self.config.endpoint(), + key_prefix = requested_prefix, + content_length = requested_content_length, + content_type = %requested_content_type, + metadata_count = requested_metadata_count, + status = response_status.unwrap_or_default(), + status_class = http_status_class_from_option(response_status), + error_kind = oss_error_kind_label(error), + message = %error, + elapsed_ms = elapsed_ms(started_at), + "OSS PutObject 上传失败" + ), } - let sanitized_segments = request - .path_segments - .iter() - .map(|segment| sanitize_path_segment(segment)) - .filter(|segment| !segment.is_empty()) - .collect::>(); - let file_name = sanitize_file_name(&request.file_name)?; - let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name); - let content_type = normalize_optional_value(request.content_type); - let metadata = normalize_metadata(request.metadata)?; - let target_url = build_object_url(&self.config.bucket, &self.config.endpoint, &object_key) - .map_err(|error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")))?; - let content_length = u64::try_from(request.body.len()) - .map_err(|_| OssError::InvalidRequest("上传对象大小超出可支持范围".to_string()))?; - let builder = signed_request_builder( - client, - &self.config, - Method::PUT, - Some(&object_key), - target_url, - content_type.as_deref(), - &metadata, - )? - .header(reqwest::header::CONTENT_LENGTH, content_length) - .body(request.body); - - let response = builder - .send() - .await - .map_err(|error| OssError::Request(format!("请求 OSS 失败:{error}")))?; - - if !response.status().is_success() { - return Err(OssError::Request(format!( - "OSS PutObject 失败,状态码:{}", - response.status() - ))); - } - - let headers = response.headers(); - let etag = headers - .get(reqwest::header::ETAG) - .and_then(|value| value.to_str().ok()) - .map(|value| value.trim_matches('"').to_string()); - let last_modified = headers - .get(reqwest::header::LAST_MODIFIED) - .and_then(|value| value.to_str().ok()) - .map(|value| value.to_string()); - - Ok(OssPutObjectResponse { - provider: "aliyun-oss", - bucket: self.config.bucket.clone(), - endpoint: self.config.endpoint.clone(), - host: self.config.upload_host(), - legacy_public_path: format!("/{object_key}"), - object_key, - content_type, - content_length, - access: request.access, - etag, - last_modified, - }) + result } } @@ -717,6 +909,43 @@ impl OssError { } } +fn elapsed_ms(started_at: Instant) -> u64 { + started_at.elapsed().as_millis().min(u64::MAX as u128) as u64 +} + +fn oss_access_label(access: OssObjectAccess) -> &'static str { + match access { + OssObjectAccess::Public => "public", + OssObjectAccess::Private => "private", + } +} + +fn oss_error_kind_label(error: &OssError) -> &'static str { + match error.kind() { + OssErrorKind::InvalidConfig => "invalid_config", + OssErrorKind::InvalidRequest => "invalid_request", + OssErrorKind::ObjectNotFound => "object_not_found", + OssErrorKind::Request => "request", + OssErrorKind::SerializePolicy => "serialize_policy", + OssErrorKind::Sign => "sign", + } +} + +fn http_status_class_from_option(status: Option) -> &'static str { + status.map(http_status_class).unwrap_or("unknown") +} + +fn http_status_class(status: u16) -> &'static str { + match status { + 100..=199 => "1xx", + 200..=299 => "2xx", + 300..=399 => "3xx", + 400..=499 => "4xx", + 500..=599 => "5xx", + _ => "unknown", + } +} + fn build_policy_json( bucket: &str, object_key: &str, @@ -1295,6 +1524,18 @@ mod tests { ); } + #[test] + fn structured_log_labels_are_stable() { + assert_eq!( + oss_error_kind_label(&OssError::InvalidRequest("bad input".to_string())), + "invalid_request" + ); + assert_eq!(oss_access_label(OssObjectAccess::Private), "private"); + assert_eq!(http_status_class(204), "2xx"); + assert_eq!(http_status_class(404), "4xx"); + assert_eq!(http_status_class_from_option(None), "unknown"); + } + fn build_client() -> OssClient { OssClient::new( OssConfig::new(