Merge remote-tracking branch 'origin/master' into codex/tiaoyitiao

This commit is contained in:
2026-06-05 22:46:57 +08:00
49 changed files with 2343 additions and 878 deletions

View File

@@ -16,6 +16,30 @@
--- ---
## 2026-06-05 Server-Provision 全程在目标部署 agent 执行且不安装构建链
- 背景:`Genarrative-Server-Provision``DEPLOY_TARGET=development` 语义是部署到 dev 服务器,不是构建机 dry-run。旧流水线把 development 映射到 `linux && genarrative-build`,还先在 build 节点准备 `provision-tools/` 再 stash 给后续阶段,导致真实 dev 初始化可能跑到 Jenkins controller / build 节点;脚本还安装 clang / lld / pkg-config / OpenSSL headers / sccache 等构建链依赖,超出了服务器初始化职责。
- 决策Server-Provision 只做服务器初始化,全程运行在目标部署 agentdevelopment 使用 `linux && genarrative-dev-deploy`release 使用 `linux && genarrative-release-deploy``Prepare Provision Tools``Provision Server` 在同一个目标 agent workspace 顺序执行,不再切到 `linux && genarrative-build`,不再 `stash/unstash` 工具包。`scripts/jenkins-server-provision.sh` 不再安装 clang / lld / pkg-config / libssl-dev / sccache非 dry-run 仍要求目标 dev / release agent 具备 root 权限,因为 provision 会写 systemd、Nginx、`/etc` 和系统用户。Job 的 `Pipeline script from SCM` 与 Jenkinsfile 参数 `SOURCE_GIT_REMOTE_URL` 都必须使用本机路径或目标 agent 可访问的内网 Git 源,不允许公网 Git fallback。
- 影响范围:`jenkins/Jenkinsfile.production-server-provision``scripts/jenkins-server-provision.sh`、生产运维文档、Server-Provision 排障口径。
- 验证方式Jenkins 日志中 Server-Provision 的 `Prepare``Checkout Provision Files``Prepare Provision Tools``Provision Server` 都在目标 dev / release agent 上执行;日志不出现 `Running on Jenkins``linux && genarrative-build``stash 'server-provision-tools'``Git 主地址拉取失败...改用备用地址``https://git.genarrative.world/GenarrativeAI/Genarrative.git` 或构建依赖 / sccache 安装步骤;`bash -n scripts/jenkins-server-provision.sh` 和编码检查通过。
- 关联文档:`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-06-05 api-server 重启先摘流再排空并持久化 outbox
- 背景:生产部署重启 api-server 时,如果只用 `/healthz` 判断存活并直接停止进程,运行中的 HTTP 请求和本地 tracking outbox active 文件都可能被中断,容易造成用户请求失败或内存/本地缓冲数据延迟丢失。
- 决策:`/healthz` 只表示进程存活,发布和生产接流检查统一使用 `/readyz`。api-server 收到 `SIGINT` / `SIGTERM` 后先把 readiness 标记为不可用,再交给 Axum graceful shutdown 排空已有 HTTP 请求;退出前在 `GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS` 窗口内封存 active tracking outbox 并尽力 flush sealed 文件失败或超时则保留本地文件给下次启动重试。systemd 停机窗口统一放到 `TimeoutStopSec=90`
- 影响范围:`server-rs/crates/api-server``deploy/systemd/genarrative-api.service`、生产 API deploy 脚本、Jenkins API deploy 参数、Nginx 公网健康检查暴露策略、开发运维文档。
- 验证方式:`cargo test -p api-server --manifest-path server-rs/Cargo.toml readyz_reports_readiness_and_draining_state``cargo test -p api-server --manifest-path server-rs/Cargo.toml shutdown_flush_seals_active_file_for_later_retry``cargo check -p api-server --manifest-path server-rs/Cargo.toml`、部署脚本 `bash -n``/readyz` 本机 smoke。
- 关联文档:`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-06-05 OSS 平台适配器输出结构化日志
- 背景AI 生成资产、浏览器直传签名、私有读签名和对象确认都依赖 OSS如果 OSS 侧只有错误字符串,排查资产写入 / 确认失败时很难按操作、对象、状态码和耗时下钻。
- 决策:`server-rs/crates/platform-oss` 统一为 `sign_post_object``sign_get_object_url``head_object``put_object` 输出结构化日志。日志固定携带 `provider=aliyun-oss``operation``bucket``endpoint``object_key` / `key_prefix``access``content_type``content_length``status``status_class``error_kind``elapsed_ms` 等排障字段;禁止输出 AccessKey、policy、signature、Authorization header 或完整 signed URL。
- 影响范围:`server-rs/crates/platform-oss``api-server` 资产签名 / 上传 / 确认链路、OTLP logs、本地 `logs/api-server/` 与运维排障文档。
- 验证方式:`cargo test -p platform-oss --manifest-path server-rs/Cargo.toml`;真实联调时按 `provider=aliyun-oss``operation` 过滤日志,确认只出现对象定位和状态字段,不出现签名材料。
- 关联文档:`server-rs/crates/platform-oss/README.md``docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-06-03 创作入口关闭不下架已发布作品 ## 2026-06-03 创作入口关闭不下架已发布作品
- 背景:`creation_entry_disabled` 曾由 api-server 按 runtime 路由前缀统一熔断,导致用户进入平台首页或启动已发布作品时也可能看到“创作入口已关闭”错误。 - 背景:`creation_entry_disabled` 曾由 api-server 按 runtime 路由前缀统一熔断,导致用户进入平台首页或启动已发布作品时也可能看到“创作入口已关闭”错误。
@@ -183,7 +207,7 @@
## 2026-05-26 推荐页拼图下一关 pending 时保留当前运行态 ## 2026-05-26 推荐页拼图下一关 pending 时保留当前运行态
- 背景:推荐页嵌入拼图在点击“下一关”时,`advancePuzzleNextLevel` 的服务端请求会短暂处于 pending。旧逻辑把推荐卡的 `isStartingRecommendEntry` 和拼图局部 busy 混在一起,导致外层直接切回“加载中...”,把当前 `PuzzleRuntimeShell` 一起卸载,视觉上像是切关闪回。 - 背景:推荐页嵌入拼图在点击“下一关”时,`advancePuzzleNextLevel` 的服务端请求会短暂处于 pending。旧逻辑把推荐卡的 `isStartingRecommendEntry` 和拼图局部 busy 混在一起,导致外层直接切回“加载中...”,把当前 `PuzzleRuntimeShell` 一起卸载,视觉上像是切关闪回。
- 决策:推荐页嵌入拼图切关 pending 期间必须保留当前运行态与棋盘,只让拼图壳内部 busy 表现承接同步;`isStartingRecommendEntry` 只表示推荐作品尚未真正启动出来,不再把已有嵌入拼图 run 的局部 busy 一并当成整卡加载态。若下一关落到相似作品,前端还必须把新作品写回推荐缓存并同步 `activeRecommendEntryKey`,避免运行态进入新作品但推荐卡元信息、分享 / 点赞 / 改造和后续“下一个”仍锚定旧作品。 - 决策:推荐页嵌入拼图切关 pending 期间必须保留当前运行态与棋盘,只让拼图壳内部 busy 表现承接同步;`isStartingRecommendEntry` 只表示推荐作品尚未真正启动出来,不再把已有嵌入拼图 run 的局部 busy 一并当成整卡加载态。若下一关落到相似作品,前端还必须把新作品写回推荐缓存并同步 `activeRecommendEntryKey`,避免运行态进入新作品但推荐卡元信息、分享 / 点赞 / 改造和后续“下一个”仍锚定旧作品;但这个同步仍属于同一个 run 内部推进,不得触发推荐 rail 切卡动画、纵向位移或启动封面重置
- 影响范围:`src/components/platform-entry/PlatformEntryFlowShellImpl.tsx``src/components/rpg-entry/RpgEntryHomeView.tsx`、推荐页拼图切关测试与平台链路文档。 - 影响范围:`src/components/platform-entry/PlatformEntryFlowShellImpl.tsx``src/components/rpg-entry/RpgEntryHomeView.tsx`、推荐页拼图切关测试与平台链路文档。
- 验证方式:点击推荐页拼图“下一关”后,在 `advancePuzzleNextLevel` 未返回前,页面仍应保留 `puzzle-board`,且不出现 `加载中...` 占位;返回相似作品后,当前推荐卡的 `作品信息` 应显示新作品标题。 - 验证方式:点击推荐页拼图“下一关”后,在 `advancePuzzleNextLevel` 未返回前,页面仍应保留 `puzzle-board`,且不出现 `加载中...` 占位;返回相似作品后,当前推荐卡的 `作品信息` 应显示新作品标题。
- 关联文档:`docs/【玩法创作】平台入口与玩法链路-2026-05-15.md` - 关联文档:`docs/【玩法创作】平台入口与玩法链路-2026-05-15.md`
@@ -443,7 +467,7 @@
## 2026-05-19 生产 provision 改为 Windows 下载包后由目标机本地安装 ## 2026-05-19 生产 provision 改为 Windows 下载包后由目标机本地安装
- 后续更新:该口径`2026-06-01 生产 Jenkins 流水线统一改为 Linux 优先并先查 localhost` 取代;当前 `Genarrative-Server-Provision`走 Windows 下载阶段,而是在 Linux build 节点直接准备 `provision-tools/` - 后续更新:该口径被 2026-06-01 Linux 优先方案取代,又在 2026-06-05 被 Server-Provision 专用口径覆盖;当前 `Genarrative-Server-Provision` 不走 Windows 下载阶段,也不在 Linux build 节点中转工具包,而是在目标 dev / release agent 内准备 `provision-tools/`
- 背景:当前 `development` provision 目标实际就是 Linux agent `genarrative-build-01`,之前把 `Prepare Provision Tools` 放在 `linux && genarrative-build` 会让目标机自己连 GitHub 和 `install.spacetimedb.com`违背“Windows 本机先下载再传到目标机”的运维要求。 - 背景:当前 `development` provision 目标实际就是 Linux agent `genarrative-build-01`,之前把 `Prepare Provision Tools` 放在 `linux && genarrative-build` 会让目标机自己连 GitHub 和 `install.spacetimedb.com`违背“Windows 本机先下载再传到目标机”的运维要求。
- 决策:`Genarrative-Server-Provision` 拆成 Windows 下载阶段和 Linux 目标机安装阶段。Windows 节点的 `Download Provision Tool Archives` 只下载 `spacetime-x86_64-unknown-linux-gnu.tar.gz``otelcol-contrib_0.151.0_linux_amd64.tar.gz`,通过 `stash/unstash` 传到目标 Linux 节点;目标机执行 `scripts/prepare-server-provision-tools.sh` 时设置 `PROVISION_REQUIRE_LOCAL_DOWNLOADS=true`,只消费已下载件生成 `provision-tools/`,缺包直接失败,不回退外网下载。 - 决策:`Genarrative-Server-Provision` 拆成 Windows 下载阶段和 Linux 目标机安装阶段。Windows 节点的 `Download Provision Tool Archives` 只下载 `spacetime-x86_64-unknown-linux-gnu.tar.gz``otelcol-contrib_0.151.0_linux_amd64.tar.gz`,通过 `stash/unstash` 传到目标 Linux 节点;目标机执行 `scripts/prepare-server-provision-tools.sh` 时设置 `PROVISION_REQUIRE_LOCAL_DOWNLOADS=true`,只消费已下载件生成 `provision-tools/`,缺包直接失败,不回退外网下载。
- 追加决策Server-Provision 的 Windows helper 不再对 Jenkins `writeFile` 刚写出的 `.ps1` 做原地 UTF-8 BOM 重写,而是由显式 `powershell.exe` 按 UTF-8 读入脚本文本,并用 `ScriptBlock::Create(...)` 在内存中执行;这样既保留中文脚本内容,又避免同一个 workspace 脚本被立即重写时触发 `拒绝访问` - 追加决策Server-Provision 的 Windows helper 不再对 Jenkins `writeFile` 刚写出的 `.ps1` 做原地 UTF-8 BOM 重写,而是由显式 `powershell.exe` 按 UTF-8 读入脚本文本,并用 `ScriptBlock::Create(...)` 在内存中执行;这样既保留中文脚本内容,又避免同一个 workspace 脚本被立即重写时触发 `拒绝访问`
@@ -1101,6 +1125,7 @@
## 2026-06-01 生产 Jenkins 流水线统一改为 Linux 优先并先查 localhost ## 2026-06-01 生产 Jenkins 流水线统一改为 Linux 优先并先查 localhost
- 后续更新:该条仍适用于常规构建 / 发布流水线;`Genarrative-Server-Provision` 已在 2026-06-05 改为目标部署 agent 全程执行,并禁止公网 Git fallback 与 build 节点工具包中转。
- 背景:生产流水线长期混用 Windows、Linux 和公网 Git 入口,导致构建 / 发布 / provision 的 checkout 口径分叉;同时 `Genarrative-Server-Provision` 还残留过 Windows 下载 helper和当前 Linux 构建 / 发布部署路径不一致。 - 背景:生产流水线长期混用 Windows、Linux 和公网 Git 入口,导致构建 / 发布 / provision 的 checkout 口径分叉;同时 `Genarrative-Server-Provision` 还残留过 Windows 下载 helper和当前 Linux 构建 / 发布部署路径不一致。
- 决策:生产 Jenkins 流水线统一把执行节点收口到 Linux label`Pipeline script from SCM` 仍保留公网域名,但所有生产流水线首次 `GitSCM checkout` 先尝试 `http://127.0.0.1:3000/GenarrativeAI/Genarrative.git`,失败后再回退到 `https://git.genarrative.world/GenarrativeAI/Genarrative.git``Genarrative-Stdb-Module-Build``Genarrative-Server-Provision``Genarrative-Notify-Email` 也都切到 Linux 节点。`Genarrative-Server-Provision` 的工具准备不再依赖 Windows helper而是在 Linux build 节点直接生成 `provision-tools/` 后交给后续 Linux 发布阶段。 - 决策:生产 Jenkins 流水线统一把执行节点收口到 Linux label`Pipeline script from SCM` 仍保留公网域名,但所有生产流水线首次 `GitSCM checkout` 先尝试 `http://127.0.0.1:3000/GenarrativeAI/Genarrative.git`,失败后再回退到 `https://git.genarrative.world/GenarrativeAI/Genarrative.git``Genarrative-Stdb-Module-Build``Genarrative-Server-Provision``Genarrative-Notify-Email` 也都切到 Linux 节点。`Genarrative-Server-Provision` 的工具准备不再依赖 Windows helper而是在 Linux build 节点直接生成 `provision-tools/` 后交给后续 Linux 发布阶段。
- 影响范围:`jenkins/Jenkinsfile.production-*``scripts/jenkins-checkout-source.sh``scripts/prepare-server-provision-tools.sh`、生产运维文档。 - 影响范围:`jenkins/Jenkinsfile.production-*``scripts/jenkins-checkout-source.sh``scripts/prepare-server-provision-tools.sh`、生产运维文档。

View File

@@ -205,7 +205,7 @@ npm run check:server-rs-ddd
- 使用 `npm run dev:api-server` 重新拉起后端。 - 使用 `npm run dev:api-server` 重新拉起后端。
- 禁止使用 `npm run api-server:maincloud``npm.cmd run api-server:maincloud` 或任何 `GENARRATIVE_SPACETIME_MAINCLOUD_*` 口径;这些只属于历史残留。 - 禁止使用 `npm run api-server:maincloud``npm.cmd run api-server:maincloud` 或任何 `GENARRATIVE_SPACETIME_MAINCLOUD_*` 口径;这些只属于历史残留。
- 检查 `/healthz` - 本地 smoke 检查 `/healthz`;发布后或确认实例可接生产流量时检查 `/readyz`
- 执行对应自动测试。 - 执行对应自动测试。
- 涉及 SpacetimeDB 表、reducer、procedure、row shape 或绑定变化时,同步更新 `migration.rs`、表目录和生成绑定。 - 涉及 SpacetimeDB 表、reducer、procedure、row shape 或绑定变化时,同步更新 `migration.rs`、表目录和生成绑定。
- SpacetimeDB 已有表新增字段必须放在 Rust 表结构体最后,并设置明确默认值;需要修改字段名时,先询问用户并确认迁移计划,再同步更新 `server-rs/crates/spacetime-module/src/migration.rs`、表目录和生成绑定。 - SpacetimeDB 已有表新增字段必须放在 Rust 表结构体最后,并设置明确默认值;需要修改字段名时,先询问用户并确认迁移计划,再同步更新 `server-rs/crates/spacetime-module/src/migration.rs`、表目录和生成绑定。
@@ -224,7 +224,7 @@ npm run check:server-rs-ddd
## 生产压测与观测默认口径 ## 生产压测与观测默认口径
- 作品列表 50 HTTP req/s 压测使用 `scripts/loadtest/README.md` 中的 K6 命令;当前脚本一次 iteration 请求两个公开列表接口,因此目标 50 HTTP req/s 对应 `PEAK_RPS=25` - 作品列表 50 HTTP req/s 压测使用 `scripts/loadtest/README.md` 中的 K6 命令;当前脚本一次 iteration 请求两个公开列表接口,因此目标 50 HTTP req/s 对应 `PEAK_RPS=25`
- 生产 `api-server` 默认 backlog、worker threads、HTTP 并发背压、systemd 限制、Nginx upstream timing log 和 OTLP 开关以 `docs/【开发运维】本地开发验证与生产运维-2026-05-15.md` 为准。 - 生产 `api-server` 默认 backlog、worker threads、HTTP 并发背压、`/readyz` 接流检查、systemd 优雅停机窗口、Nginx upstream timing log 和 OTLP 开关以 `docs/【开发运维】本地开发验证与生产运维-2026-05-15.md` 为准。
- OpenTelemetry 现阶段可选发送 traces / metrics / logs但不会取代本地 `journalctl -u genarrative-api.service``logs/api-server/``/var/log/nginx/genarrative.*.log` - OpenTelemetry 现阶段可选发送 traces / metrics / logs但不会取代本地 `journalctl -u genarrative-api.service``logs/api-server/``/var/log/nginx/genarrative.*.log`
- 指标 label 不写 raw URI、userId、profileId 或 request_idrequest_id 只用于 trace/log 串联。 - 指标 label 不写 raw URI、userId、profileId 或 request_idrequest_id 只用于 trace/log 串联。

View File

@@ -15,6 +15,14 @@
- 关联:相关文件、文档、提交或 Issue - 关联:相关文件、文档、提交或 Issue
``` ```
## 小程序 H5 导航不能清掉宿主 query
- 现象:微信小程序首次进入 H5 后,点击需要登录的入口没有返回小程序原生授权页,而是弹出 Web 端登录窗口;充值渠道也可能被误判为普通网页环境。
- 原因:小程序 `web-view` 入口通过 `clientType=mini_program``clientRuntime=wechat_mini_program``miniProgramEnv` 标记宿主环境,但 H5 内部 `pushAppHistoryPath(...)` 阶段导航会默认清空 query首点时微信 JS bridge 也可能尚未就绪,导致 `isWechatMiniProgramWebViewRuntime()` 和充值平台判断读不到小程序上下文。
- 处理:路由层统一把 `clientType``clientRuntime``miniProgramEnv` 当作 app runtime context在普通路径归一、显式 query 路由和同一创作流跳转时都跨导航保留;小程序环境识别同时用 `MicroMessenger + miniProgram` User-Agent 兜底首点 bridge 未就绪场景;创作恢复参数仍只在同玩法创作流内保留,离开创作流时继续清理。
- 验证:`npm exec vitest run src/routing/appPageRoutes.test.ts src/components/auth/AuthGate.test.tsx src/services/authService.test.ts src/services/payment/paymentPlatform.test.ts`
- 关联:`src/routing/appPageRoutes.ts``src/services/authService.ts``src/services/payment/paymentPlatform.ts``docs/【项目基线】当前产品与工程约束-2026-05-15.md`
## 平台异步错误必须带来源弹窗,不要只显示裸错误 ## 平台异步错误必须带来源弹窗,不要只显示裸错误
- 现象:用户先后触发多个拼图或草稿生成时,旧请求失败后会在当前页面显示“图片生成失败”等裸错误,容易误判为当前正在看的拼图失败;错误文本也不便复制给开发排查。 - 现象:用户先后触发多个拼图或草稿生成时,旧请求失败后会在当前页面显示“图片生成失败”等裸错误,容易误判为当前正在看的拼图失败;错误文本也不便复制给开发排查。
@@ -35,10 +43,10 @@
- 现象:`external_api_call_failure` 里看到 `failureStage=request_send``timeout=true``statusCode=null``errorSource` 可能是 `client error (SendRequest)` 或更完整的 reqwest 底层错误链,前端只知道图片生成失败。 - 现象:`external_api_call_failure` 里看到 `failureStage=request_send``timeout=true``statusCode=null``errorSource` 可能是 `client error (SendRequest)` 或更完整的 reqwest 底层错误链,前端只知道图片生成失败。
- 原因:`timeout=true` 来自 `reqwest::Error::is_timeout()`,不是业务代码固定写死;`SendRequest` 是 Hyper 发送请求阶段的错误来源标签,只说明请求未拿到可归类的 HTTP 响应,不会包含上游 JSON 错误体。 - 原因:`timeout=true` 来自 `reqwest::Error::is_timeout()`,不是业务代码固定写死;`SendRequest` 是 Hyper 发送请求阶段的错误来源标签,只说明请求未拿到可归类的 HTTP 响应,不会包含上游 JSON 错误体。
- 处理:先按 `provider/failureStage/statusClass` 聚合,再用 `user_id` / `profile_id``metadata_json.userId/profileId/requestId` 定位触发者、草稿 / 作品和同一次 HTTP 请求;`request_send + timeout=true` 优先查 provider 日志的 `source_chain`、请求体大小、参考图数量、出口网络、代理/Nginx、VectorEngine 当时可用性和同一 request_id 日志。若记录有 `502``429 moderation_blocked`,按上游网关或审核失败另行处理,不要归到传输超时。 - 处理:先按 `provider/failureStage/statusClass` 聚合,再用 `user_id` / `profile_id``metadata_json.userId/profileId/requestId` 定位触发者、草稿 / 作品和同一次 HTTP 请求;`request_send + timeout=true` 优先查 provider 日志的 `source_chain`、请求体大小、参考图数量、出口网络、代理/Nginx、VectorEngine 当时可用性和同一 request_id 日志。当前 `platform-image``request_send``timeout` / `connect` 错误最多重试 3 次multipart `/v1/images/edits` 每次重试都必须重建 form看到 `VectorEngine 图片请求发送失败,准备重试` 只是单次 attempt 失败,最终 `external_api_call_failure` 才代表该用户请求整体失败。若记录有 `502``429 moderation_blocked`,按上游网关或审核失败另行处理,不要归到传输超时。
- 拼图关卡资产生成按 `level_scene -> ui_spritesheet -> level_background` 顺序执行,每个资产会输出 `slot``asset_kind``elapsed_ms`;排查拼图草稿失败时优先看同一 request_id 下最后一个失败 slot。 - 拼图关卡资产生成按 `level_scene -> ui_spritesheet -> level_background` 顺序执行,每个资产会输出 `slot``asset_kind``elapsed_ms`;排查拼图草稿失败时优先看同一 request_id 下最后一个失败 slot。
- 验证:`cargo check -p api-server --manifest-path server-rs/Cargo.toml`;查询 `tracking_event` 时失败记录应能看到触发者 `user_id` 和可用的 `profile_id` - 验证:`cargo test -p platform-image --manifest-path server-rs/Cargo.toml vector_engine_image_edit_retries_send_timeout_once_and_succeeds``cargo check -p api-server --manifest-path server-rs/Cargo.toml`;查询 `tracking_event` 时失败记录应能看到触发者 `user_id` 和可用的 `profile_id`
- 关联:`server-rs/crates/api-server/src/external_api_audit.rs``server-rs/crates/api-server/src/openai_image_generation.rs``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md` - 关联:`server-rs/crates/platform-image/src/vector_engine/client.rs``server-rs/crates/api-server/src/external_api_audit.rs``server-rs/crates/api-server/src/openai_image_generation.rs``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## “我的”页每日任务卡不要硬编码进度 ## “我的”页每日任务卡不要硬编码进度
@@ -1187,6 +1195,7 @@
## Jenkins 生产流水线拉 Git 先本机再域名备用 ## Jenkins 生产流水线拉 Git 先本机再域名备用
- 后续更新:该条仍适用于常规构建 / 发布流水线;`Genarrative-Server-Provision` 已在 2026-06-05 改为服务器初始化专用口径,不允许公网 Git fallbackJob 的 `Pipeline script from SCM` 和 Jenkinsfile 内部 checkout 都必须使用本机路径或目标 agent 可访问的内网 Git 源。
- 现象:生产发布、数据库导入导出、服务器配置、构建或 `Genarrative-Full-Build-And-Deploy` 流水线执行 `GitSCM checkout` 时,如果 Jenkins 生成的 fetch 是 `+refs/heads/*:refs/remotes/origin/*`,公网 Git 链路可能在收包阶段以 `git-remote-https died of signal 15``curl 56 GnuTLS recv error (-9)``early EOF``invalid index-pack output` 失败;发布类流水线还可能先遇到 `http://127.0.0.1:3000/GenarrativeAI/Genarrative.git` 不可达。 - 现象:生产发布、数据库导入导出、服务器配置、构建或 `Genarrative-Full-Build-And-Deploy` 流水线执行 `GitSCM checkout` 时,如果 Jenkins 生成的 fetch 是 `+refs/heads/*:refs/remotes/origin/*`,公网 Git 链路可能在收包阶段以 `git-remote-https died of signal 15``curl 56 GnuTLS recv error (-9)``early EOF``invalid index-pack output` 失败;发布类流水线还可能先遇到 `http://127.0.0.1:3000/GenarrativeAI/Genarrative.git` 不可达。
- 原因:`127.0.0.1` 只代表当前执行阶段的 agent 自身;当 release agent 与 Git 服务不在同一台机器,或本机 Git/Web 服务临时不可用时,固定写死 localhost 会阻断 Jenkinsfile 内部源码/脚本 checkout。即使只使用域名 Git如果 `GitSCM` 没有显式 refspec 并开启 `CloneOption honorRefspec=true`Jenkins Git 插件也会拉取所有分支。 - 原因:`127.0.0.1` 只代表当前执行阶段的 agent 自身;当 release agent 与 Git 服务不在同一台机器,或本机 Git/Web 服务临时不可用时,固定写死 localhost 会阻断 Jenkinsfile 内部源码/脚本 checkout。即使只使用域名 Git如果 `GitSCM` 没有显式 refspec 并开启 `CloneOption honorRefspec=true`Jenkins Git 插件也会拉取所有分支。
- 处理Jenkins Job 的 `Pipeline script from SCM` 由 Windows controller 执行SCM URL 使用公网域名 `https://git.genarrative.world/GenarrativeAI/Genarrative.git`。运行于 `linux && genarrative-build``Genarrative-Full-Build-And-Deploy` 源码解析阶段、`Genarrative-Web-Build` checkout 阶段,以及部署/发布类 Linux agent 的 Jenkinsfile 首次 `checkout([$class: 'GitSCM', ...])` 层先尝试 `GIT_REMOTE_URL=http://127.0.0.1:3000/GenarrativeAI/Genarrative.git`,失败后直接尝试 `GIT_REMOTE_FALLBACK_URL=https://git.genarrative.world/GenarrativeAI/Genarrative.git`,不再配置内网 IP fallback这些首次 checkout 都必须使用目标分支 refspec、`CloneOption shallow=true depth=1 noTags=true honorRefspec=true`。后续统一走 `scripts/jenkins-checkout-source.sh`,该脚本也按主地址、域名备用地址顺序重新 fetch 并把 `origin` 切到实际可用地址;`COMMIT_HASH` 为空时继续 `--depth=1 --no-tags`,只有指定 commit 时才允许加深历史做分支归属校验。 - 处理Jenkins Job 的 `Pipeline script from SCM` 由 Windows controller 执行SCM URL 使用公网域名 `https://git.genarrative.world/GenarrativeAI/Genarrative.git`。运行于 `linux && genarrative-build``Genarrative-Full-Build-And-Deploy` 源码解析阶段、`Genarrative-Web-Build` checkout 阶段,以及部署/发布类 Linux agent 的 Jenkinsfile 首次 `checkout([$class: 'GitSCM', ...])` 层先尝试 `GIT_REMOTE_URL=http://127.0.0.1:3000/GenarrativeAI/Genarrative.git`,失败后直接尝试 `GIT_REMOTE_FALLBACK_URL=https://git.genarrative.world/GenarrativeAI/Genarrative.git`,不再配置内网 IP fallback这些首次 checkout 都必须使用目标分支 refspec、`CloneOption shallow=true depth=1 noTags=true honorRefspec=true`。后续统一走 `scripts/jenkins-checkout-source.sh`,该脚本也按主地址、域名备用地址顺序重新 fetch 并把 `origin` 切到实际可用地址;`COMMIT_HASH` 为空时继续 `--depth=1 --no-tags`,只有指定 commit 时才允许加深历史做分支归属校验。
@@ -1209,12 +1218,12 @@
- 验证:运行 `git ls-files --stage scripts/prepare-server-provision-tools.sh`,确认 mode 为 `100755`;重新跑 `Genarrative-Server-Provision` 时应进入工具下载/打包日志,而不是停在 `Permission denied` - 验证:运行 `git ls-files --stage scripts/prepare-server-provision-tools.sh`,确认 mode 为 `100755`;重新跑 `Genarrative-Server-Provision` 时应进入工具下载/打包日志,而不是停在 `Permission denied`
- 关联:`jenkins/Jenkinsfile.production-server-provision``scripts/prepare-server-provision-tools.sh``scripts/jenkins-checkout-source.sh``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md` - 关联:`jenkins/Jenkinsfile.production-server-provision``scripts/prepare-server-provision-tools.sh``scripts/jenkins-checkout-source.sh``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## Server-Provision 工具准备只在 Linux build 节点做一次 ## Server-Provision 工具准备只在目标部署 agent 内做一次
- 现象:`Genarrative-Server-Provision` 在后续目标发布节点重复执行 `scripts/prepare-server-provision-tools.sh`,或日志里出现目标节点继续访问 GitHub release / `install.spacetimedb.com` - 现象:`Genarrative-Server-Provision` 选择 `DEPLOY_TARGET=development` 时如果阶段跑在 `Running on Jenkins``linux && genarrative-build`,真实 provision 会落到构建机 / controller而不是 dev 服务器
- 原因:当前流水线已经改成 Linux build 节点一次性准备 `provision-tools/` 并 stash 给目标发布阶段;如果目标发布阶段又重新准备工具包,就会重复下载并把目标节点暴露到外网依赖 - 原因:Server-Provision 是服务器初始化流水线dev / release 都是目标服务器,不应把 development 当成 build 节点预览目标,也不应通过 build 节点 stash 工具包再切回目标机;同时公网 Git fallback 会让目标 agent 内网源不可达时悄悄改从公网拉源码,掩盖服务器路由问题
- 处理:只允许 `Prepare Provision Tools` 阶段在 `linux && genarrative-build` 节点生成 `provision-tools/`;后续 `Provision Server` 阶段只 `unstash 'server-provision-tools'` 并安装其中的 `spacetime``otelcol-contrib`,不要再运行 `prepare-server-provision-tools.sh` - 处理:Server-Provision 全程运行在目标部署 agentdevelopment 使用 `linux && genarrative-dev-deploy`release 使用 `linux && genarrative-release-deploy``Prepare Provision Tools` `Provision Server` 在同一个目标 agent workspace 内顺序执行,不再使用 `linux && genarrative-build`,也不再 `stash/unstash` 工具包。Job 的 `Pipeline script from SCM`参数 `SOURCE_GIT_REMOTE_URL` 都必须指向本机路径或内网 Git 源,不允许 `https://git.genarrative.world/...` 公网地址
- 验证Jenkins 日志应先在 Linux build 节点出现 `[prepare-provision-tools] 工具包已准备`,后续目标发布节点只出现安装 / systemd / Nginx provision 日志;目标节点不应出现 `下载 otelcol-contrib:``下载 SpacetimeDB 官方安装器脚本:` - 验证Jenkins 日志`Provision Target` 下的 `Prepare``Checkout Provision Files``Prepare Provision Tools``Provision Server` 都应运行在目标 dev / release agent日志不应出现 `stash 'server-provision-tools'`、目标阶段 `unstash``Git 主地址拉取失败...改用备用地址``https://git.genarrative.world/GenarrativeAI/Genarrative.git`
- 关联:`jenkins/Jenkinsfile.production-server-provision``scripts/prepare-server-provision-tools.sh``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md` - 关联:`jenkins/Jenkinsfile.production-server-provision``scripts/prepare-server-provision-tools.sh``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 个人任务 scope 不得扩成 work/site/module ## 个人任务 scope 不得扩成 work/site/module
@@ -1760,6 +1769,26 @@
- 验证:定向测试 `cargo test -p api-server generated_asset_sheet_two_items_per_row --manifest-path server-rs/Cargo.toml -- --nocapture` 应通过,且错位透明样本应按连通域切出完整视图。 - 验证:定向测试 `cargo test -p api-server generated_asset_sheet_two_items_per_row --manifest-path server-rs/Cargo.toml -- --nocapture` 应通过,且错位透明样本应按连通域切出完整视图。
- 关联:`server-rs/crates/api-server/src/generated_asset_sheets.rs``server-rs/crates/api-server/src/match3d/item_assets.rs` - 关联:`server-rs/crates/api-server/src/generated_asset_sheets.rs``server-rs/crates/api-server/src/match3d/item_assets.rs`
## 腾讯云 release 上 VectorEngine `SendRequest` 超时先查出口链路与重试
- 现象release 机器调用 VectorEngine `gpt-image-2``/v1/images/generations``/v1/images/edits` 偶发 `client error (SendRequest) -> connection error -> Connection timed out (os error 110)`,应用层表现为 504本地通常正常。
- 原因:本地 DNS 可能走代理 / 加速出口,而腾讯云 release 直接解析到 VectorEngine 真实边缘节点。实测同一张约 2.37MB PNG、同一 edits 请求,`curl` 5/5 成功,但 `reqwest/hyper` 会间歇性超时;固定 `40.160.33.47` 也只能改善,不能根治。
- 处理:不要优先关闭 multipart也不要直接把 `SendRequest` 解释成上游业务拒绝。VectorEngine 图片 `generations` / `edits` 上游 POST 单独使用 `libcurl`;参考图下载和响应图片 URL 下载仍用 `reqwest`。send 阶段 timeout / connect error 在 `platform-image` 内最多重试 5 次,使用指数退避和短抖动;日志字段 `attempt``max_attempts``retry_delay_ms``reference_image_bytes_total``request_params` 是定位依据。
### api-server libcurl / OpenSSL 3.2 runtime
- 症状release 部署新 `api-server` 后服务反复 `exit-code``LD_TRACE_LOADED_OBJECTS=1 /opt/genarrative/current/api-server``ldd``/lib/x86_64-linux-gnu/libssl.so.3: version 'OPENSSL_3.2.0' not found`
- 根因:`platform-image` 使用 `libcurl`Linux release 构建产物可能直接要求 `OPENSSL_3.2.0` 符号Ubuntu 24.04 apt 默认 OpenSSL 仍是 `3.0.13`,不能满足该符号版本。
- 处理:`Genarrative-Server-Provision` 独立安装 OpenSSL `3.2.0``/opt/genarrative/openssl-3.2.0`,并只通过 `genarrative-api.service``LD_LIBRARY_PATH=/opt/genarrative/openssl-3.2.0/lib64:/opt/genarrative/openssl-3.2.0/lib` 给 api-server 使用,避免替换系统 OpenSSL。
### VectorEngine edits multipart image part
- 症状:拼图参考图链路请求 `/v1/images/edits` 返回 `500 image is required`,但应用日志里 `reference_image_count=1``reference_image_bytes_total>0``request_params.referenceImages[0]` 也有 `field=image`、文件名、MIME 和 bytes。
- 根因Rust `curl::easy::Form``contents(...).filename(...)` 不等价于文件上传 partVectorEngine 转码层会认为没有收到图片。release 上用 curl CLI `-F image=@file` 可成功,证明字段名和上游接口本身没变。
- 处理multipart 参考图必须用 `Form::buffer(file_name, bytes)` 并设置 `content_type(...)`,让 libcurl 生成真正的 `name="image"; filename="..."` 文件 part。
- 验证release 上先看 `journalctl -u genarrative-api.service``VectorEngine 图片请求发送失败,准备重试` 与最终 `HTTP 返回`;若仍失败,再用同一图片分别跑 curl 与最小 reqwest 探针对照。
- 关联:`server-rs/crates/platform-image/src/vector_engine/client.rs``docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`
## 个人中心不再保留直达“存档”按钮入口 ## 个人中心不再保留直达“存档”按钮入口
- 现象2026-05-25 起,移动端“我的”页顶部改为品牌行 + 扫码 / 设置按钮,设置区和次级入口不再提供独立的 `存档` 按钮;用户仍可在“玩过”弹窗里查看可继续存档。 - 现象2026-05-25 起,移动端“我的”页顶部改为品牌行 + 扫码 / 设置按钮,设置区和次级入口不再提供独立的 `存档` 按钮;用户仍可在“玩过”弹窗里查看可继续存档。

View File

@@ -190,7 +190,7 @@ http {
proxy_set_header X-Request-Id $request_id; proxy_set_header X-Request-Id $request_id;
} }
location ~ ^/(generated-|healthz) { location ~ ^/(generated-|healthz|readyz) {
return 404; return 404;
} }

View File

@@ -11,6 +11,7 @@ GENARRATIVE_API_MAX_CONCURRENT_REQUESTS=512
GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS=320 GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS=320
GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS=64 GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS=64
GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS=16 GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS=16
GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS=5000
GENARRATIVE_TRACKING_OUTBOX_ENABLED=true GENARRATIVE_TRACKING_OUTBOX_ENABLED=true
GENARRATIVE_TRACKING_OUTBOX_DIR=/var/lib/genarrative/tracking-outbox GENARRATIVE_TRACKING_OUTBOX_DIR=/var/lib/genarrative/tracking-outbox
GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE=500 GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE=500

View File

@@ -215,7 +215,7 @@ server {
} }
# 开发服仍不恢复旧生成资源代理和健康检查公网入口。 # 开发服仍不恢复旧生成资源代理和健康检查公网入口。
location ~ ^/(generated-|healthz) { location ~ ^/(generated-|healthz|readyz) {
return 404; return 404;
} }

View File

@@ -235,7 +235,7 @@ server {
} }
# 生产公网不再暴露旧生成资源代理和健康检查入口。 # 生产公网不再暴露旧生成资源代理和健康检查入口。
location ~ ^/(generated-|healthz) { location ~ ^/(generated-|healthz|readyz) {
return 404; return 404;
} }

View File

@@ -10,11 +10,12 @@ User=genarrative
Group=genarrative Group=genarrative
WorkingDirectory=/opt/genarrative/current WorkingDirectory=/opt/genarrative/current
EnvironmentFile=/etc/genarrative/api-server.env EnvironmentFile=/etc/genarrative/api-server.env
Environment="LD_LIBRARY_PATH=/opt/genarrative/openssl-3.2.0/lib64:/opt/genarrative/openssl-3.2.0/lib"
ExecStart=/opt/genarrative/current/api-server ExecStart=/opt/genarrative/current/api-server
Restart=always Restart=always
RestartSec=5 RestartSec=5
KillSignal=SIGINT KillSignal=SIGINT
TimeoutStopSec=30 TimeoutStopSec=90
LimitNOFILE=65535 LimitNOFILE=65535
TasksMax=2048 TasksMax=2048

View File

@@ -128,9 +128,10 @@ npm run check:server-rs-ddd
3. Adapter 输出应保留 legacy public path、object key、asset object id、MIME、extension、task id 和实际 prompt。 3. Adapter 输出应保留 legacy public path、object key、asset object id、MIME、extension、task id 和实际 prompt。
4. Adapter 不负责扣费、退款或钱包读取;计费仍由调用方显式包裹。 4. Adapter 不负责扣费、退款或钱包读取;计费仍由调用方显式包裹。
5. 图片 provider 协议不再放在玩法模块里实现。VectorEngine `gpt-image-2` 创建 / 编辑协议、URL / base64 图片解析、远端图片下载、请求超时 / 上游状态 / 响应解析 / 缺图 / 下载失败的结构化日志统一在 `server-rs/crates/platform-image/src/vector_engine/`;其中 `client.rs` 只保留 provider 调用编排,`transport.rs` 负责 HTTP client 与 reqwest 错误归一,`request.rs` 负责请求体和路径,`payload.rs` 负责响应 JSON 字段提取,`response.rs` 负责响应状态分流和图片结果归一。`api-server` 只负责配置校验、玩法 prompt 编排、OSS / asset object / binding 持久化、计费和外部 API 失败审计落库。 5. 图片 provider 协议不再放在玩法模块里实现。VectorEngine `gpt-image-2` 创建 / 编辑协议、URL / base64 图片解析、远端图片下载、请求超时 / 上游状态 / 响应解析 / 缺图 / 下载失败的结构化日志统一在 `server-rs/crates/platform-image/src/vector_engine/`;其中 `client.rs` 只保留 provider 调用编排,`transport.rs` 负责 HTTP client 与 reqwest 错误归一,`request.rs` 负责请求体和路径,`payload.rs` 负责响应 JSON 字段提取,`response.rs` 负责响应状态分流和图片结果归一。`api-server` 只负责配置校验、玩法 prompt 编排、OSS / asset object / binding 持久化、计费和外部 API 失败审计落库。
6. Puzzle、Match3D、音频、GLB、视频等复杂媒体可以复用 OSS + asset object + binding 的底层持久化能力,但玩法专属处理规则留在各自编排层,不塞进公共接口 6. OSS 平台适配日志统一在 `server-rs/crates/platform-oss` 输出,覆盖 `sign_post_object``sign_get_object_url``head_object``put_object`。日志字段固定使用 `provider``operation``bucket``endpoint``object_key` / `key_prefix``access``content_type``content_length``status``status_class``error_kind``elapsed_ms`,只记录对象定位和排障信息;不得输出 AccessKey、policy、signature、Authorization header 或完整 signed URL
7. 拼图入口页与结果页新增关卡的本地参考图不走浏览器直传 OSS前端读取为 Data URL 后随创作 action 提交,并在读取前限制 6MB、显示“图片≤6MB”。`api-server` 必须对 Data URL 实际字节数再次校验;历史图片才提交 `referenceImageAssetObjectId(s)`,后端校验 `asset_object` 的 bucket、kind、图片 MIME、大小和 owner 后签发只读 URL 给 VectorEngine 读取 7. Puzzle、Match3D、音频、GLB、视频等复杂媒体可以复用 OSS + asset object + binding 的底层持久化能力,但玩法专属处理规则留在各自编排层,不塞进公共接口
8. 系列素材图集实现真相源在 `server-rs/crates/platform-image/src/generated_asset_sheets/`:调用方必须传入 `grid_size` 作为 `n*n``n`,可选传入物品名称 prompt 模板和特殊设定 prompt模块负责 sheet prompt 组装、按 `n*n` 切片、透明化、PNG 输出、OSS private upload 请求构造和 sheet / item / special prompt 元数据持久化。`server-rs/crates/api-server/src/generated_asset_sheets.rs` 只保留 `AppState` / `AppError` 适配和兼容导出。玩法只负责规划 slot、调用具体生图 provider、计费、失败回写以及把通用切片结果映射回自己的 DTO / 草稿 / runtime 字段 8. 拼图入口页与结果页新增关卡的本地参考图不走浏览器直传 OSS前端读取为 Data URL 后随创作 action 提交,并在读取前限制 6MB、显示“图片≤6MB”。`api-server` 必须对 Data URL 实际字节数再次校验;历史图片才提交 `referenceImageAssetObjectId(s)`,后端校验 `asset_object` 的 bucket、kind、图片 MIME、大小和 owner 后签发只读 URL 给 VectorEngine 读取
9. 系列素材图集实现真相源在 `server-rs/crates/platform-image/src/generated_asset_sheets/`:调用方必须传入 `grid_size` 作为 `n*n``n`,可选传入物品名称 prompt 模板和特殊设定 prompt模块负责 sheet prompt 组装、按 `n*n` 切片、透明化、PNG 输出、OSS private upload 请求构造和 sheet / item / special prompt 元数据持久化。`server-rs/crates/api-server/src/generated_asset_sheets.rs` 只保留 `AppState` / `AppError` 适配和兼容导出。玩法只负责规划 slot、调用具体生图 provider、计费、失败回写以及把通用切片结果映射回自己的 DTO / 草稿 / runtime 字段。
## SpacetimeDB schema 变更规则 ## SpacetimeDB schema 变更规则
@@ -167,14 +168,14 @@ npm run check:server-rs-ddd
## 外部服务与资产 ## 外部服务与资产
- LLM`GENARRATIVE_LLM_*`,创意 Agent 另用 `APIMART_BASE_URL` / `APIMART_API_KEY` - LLM`GENARRATIVE_LLM_*`,创意 Agent 另用 `APIMART_BASE_URL` / `APIMART_API_KEY`
- 图片生成VectorEngine `gpt-image-2` 图片 provider 归属 `platform-image`,密钥只在后端环境变量中;`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接,不再承载 provider 协议实现。实际外部生成运行记录统一落 `tracking_event``event_key = external_generation_run`metadata 记录开始 / 结束时间、耗时、状态、成功标记、失败原因、provider task id 和结果摘要,不再写回过时的 `ai_task`。APIMart 只保留给创意 Agent `gpt-5` Responses 文本 / 多模态链路DashScope 只按仍在使用的历史能力单独处理,不作为 GPT-image-2 兜底。 - 图片生成VectorEngine `gpt-image-2` 图片 provider 归属 `platform-image`,密钥只在后端环境变量中;`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接,不再承载 provider 协议实现。实际外部生成运行记录统一落 `tracking_event``event_key = external_generation_run`metadata 记录开始 / 结束时间、耗时、状态、成功标记、失败原因、provider task id 和结果摘要,不再写回过时的 `ai_task`。APIMart 只保留给创意 Agent `gpt-5` Responses 文本 / 多模态链路DashScope 只按仍在使用的历史能力单独处理,不作为 GPT-image-2 兜底。VectorEngine `/v1/images/generations``/v1/images/edits` 上游 POST 使用 `libcurl` 发送;`reqwest` 只保留给参考图 URL 下载和响应中图片 URL 下载。`/v1/images/edits` 的 multipart 参考图必须作为 libcurl 文件上传 part 发送,字段名为 `image`,实现上使用 `Form::buffer(file_name, bytes)` 并设置 `Content-Type`;不能只用 `contents(...).filename(...)`,否则上游会把请求转码为缺少图片并返回 `image is required``request_send` 阶段的 curl timeout / connect error 按可重试传输错误处理,最多尝试 5 次,并使用指数退避加短抖动;排障时优先看 `attempt``max_attempts``retry_delay_ms``reference_image_bytes_total``request_params`,不要把 `SendRequest` 当成上游业务错误。
- Match3D 物品 sheet关卡整图完成后走 VectorEngine `/v1/images/edits` multipart `image`,模型为 `gpt-image-2``2K 1:1` 输出 `10*10` spritesheet物品 sheet prompt 固定要求纯绿色绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG并把透明整图写入 `itemSpritesheetImageSrc/itemSpritesheetImageObjectKey`。后端优先按透明 alpha 连通域从该 sheet 识别真实素材矩形并持久化 20 个物品、每个 5 个形态;识别数量不足时才回退 `10*10` 固定网格。通用系列素材图集的行列索引按每行 2 个物品计算,必须落在 `1..=10`,难度只决定运行态加载 3 / 9 / 15 / 20 种。 - Match3D 物品 sheet关卡整图完成后走 VectorEngine `/v1/images/edits` multipart `image`,模型为 `gpt-image-2``2K 1:1` 输出 `10*10` spritesheet物品 sheet prompt 固定要求纯绿色绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG并把透明整图写入 `itemSpritesheetImageSrc/itemSpritesheetImageObjectKey`。后端优先按透明 alpha 连通域从该 sheet 识别真实素材矩形并持久化 20 个物品、每个 5 个形态;识别数量不足时才回退 `10*10` 固定网格。通用系列素材图集的行列索引按每行 2 个物品计算,必须落在 `1..=10`,难度只决定运行态加载 3 / 9 / 15 / 20 种。
- Match3D UI spritesheet 和背景派生图:关卡整图作为参考图并发生成 `1K 1:1` UI spritesheet 与 `1K 9:16` 背景图,模型均为 `gpt-image-2`。UI spritesheet prompt 固定要求纯绿色绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG背景图必须合成为全画幅不透明 PNG。 - Match3D UI spritesheet 和背景派生图:关卡整图作为参考图并发生成 `1K 1:1` UI spritesheet 与 `1K 9:16` 背景图,模型均为 `gpt-image-2`。UI spritesheet prompt 固定要求纯绿色绿幕背景,后端上传 OSS 前必须把绿幕扣成透明 PNG背景图必须合成为全画幅不透明 PNG。
- Match3D 1:1 容器 UIVectorEngine `/v1/images/edits` multipart 参考图。该容器参考图是后端生图协议输入,必须通过 `include_bytes!``api-server` 编译进二进制,避免 API 单独发布或运行目录缺少 `public/` 时生成失败。 - Match3D 1:1 容器 UIVectorEngine `/v1/images/edits` multipart 参考图。该容器参考图是后端生图协议输入,必须通过 `include_bytes!``api-server` 编译进二进制,避免 API 单独发布或运行目录缺少 `public/` 时生成失败。
- 敲木鱼敲击物和背景环境图VectorEngine `/v1/images/edits`,模型固定 `gpt-image-2`。敲击物支持 multipart 多参考图第一张固定为后端内嵌默认木鱼图用户上传图只作为新主题参考prompt 必须要求 `1:1` 真实透明 alpha PNG 并禁止黑底、白底、棋盘格和任何实底背景。当前敲击物上传 OSS 前不做服务端抠图后处理避免误伤玉米等主体像素。背景环境图只使用第一步抠图完成后的透明敲击物图作为参考prompt 必须要求中央主体预留区保持干净,中央 40% 区域禁止出现主题主体、主体局部特写、轮廓影子或重复元素,主题元素只能作为外围氛围,且必须显式声明不继承任何绿色底色、绿幕底色或纯绿色画布。 - 敲木鱼敲击物和背景环境图VectorEngine `/v1/images/edits`,模型固定 `gpt-image-2`。敲击物支持 multipart 多参考图第一张固定为后端内嵌默认木鱼图用户上传图只作为新主题参考prompt 必须要求 `1:1` 真实透明 alpha PNG 并禁止黑底、白底、棋盘格和任何实底背景。当前敲击物上传 OSS 前不做服务端抠图后处理避免误伤玉米等主体像素。背景环境图只使用第一步抠图完成后的透明敲击物图作为参考prompt 必须要求中央主体预留区保持干净,中央 40% 区域禁止出现主题主体、主体局部特写、轮廓影子或重复元素,主题元素只能作为外围氛围,且必须显式声明不继承任何绿色底色、绿幕底色或纯绿色画布。
- Hyper3D / Rodin只保留后端安全代理和旧数据兼容Rodin 提交、状态、下载和响应解析归属 `platform-hyper3d``api-server/src/hyper3d_generation.rs` 只做路由、配置和错误 envelope 映射;新 Match3D 草稿和批量新增不再生成 GLB。 - Hyper3D / Rodin只保留后端安全代理和旧数据兼容Rodin 提交、状态、下载和响应解析归属 `platform-hyper3d``api-server/src/hyper3d_generation.rs` 只做路由、配置和错误 envelope 映射;新 Match3D 草稿和批量新增不再生成 GLB。
- 音频视觉小说专用音频路由保留VectorEngine Suno/Vidu provider 协议、任务提交/查询、音频 URL 提取、下载、MIME/extension 归一和 OSS put 请求准备归属 `platform-audio``api-server/src/vector_engine_audio_generation.rs` 只做路由、配置、计费、asset object confirm、entity binding 和错误 envelope 映射;拼图、抓大鹅和敲木鱼提示词生成音效入口暂时关闭,通用 `/api/creation/audio/*` 对这些目标返回 `410 Gone`。敲木鱼创作只接收上传 / 录音音频资产;未提供时由 `api-server` 写回内置默认木鱼音 `/wooden-fish/default-hit-sound.mp3` - 音频视觉小说专用音频路由保留VectorEngine Suno/Vidu provider 协议、任务提交/查询、音频 URL 提取、下载、MIME/extension 归一和 OSS put 请求准备归属 `platform-audio``api-server/src/vector_engine_audio_generation.rs` 只做路由、配置、计费、asset object confirm、entity binding 和错误 envelope 映射;拼图、抓大鹅和敲木鱼提示词生成音效入口暂时关闭,通用 `/api/creation/audio/*` 对这些目标返回 `410 Gone`。敲木鱼创作只接收上传 / 录音音频资产;未提供时由 `api-server` 写回内置默认木鱼音 `/wooden-fish/default-hit-sound.mp3`
- OSS私有 generated legacy path 进入浏览器前必须通过 `/api/assets/read-url` 换签;不要裸请求 `/generated-*` - OSS私有 generated legacy path 进入浏览器前必须通过 `/api/assets/read-url` 换签;不要裸请求 `/generated-*`OSS 签名、读签名、HEAD 和 PUT 的结构化日志由 `platform-oss` 输出,排查资产写入 / 确认失败时优先按 `operation``object_key` / `key_prefix``status_class``error_kind``elapsed_ms` 下钻。
- 外部 API 失败审计:外部供应商调用未成功时,`api-server` 必须发送 OTLP 失败事件并写入 `tracking_event`。VectorEngine 图片 provider 在 `platform-image` 内输出结构化日志和 `PlatformImageFailureAudit`,覆盖 `request_send``response_body``upstream_status``response_parse``missing_image``image_download` 阶段;`api-server` 只把该 audit 映射成 `external_api_call_failure``scope_kind = module``scope_id = provider``module_key = external-api`。metadata 固定包含 provider、endpoint、operation、failureStage、statusCode、statusClass、timeout、retryable、errorMessage、latencyMs、promptChars、referenceImageCount、imageModel、rawExcerpt以及在调用方可获得上下文时补充的 `userId`(触发者)和 `profileId`(草稿 / 作品 / 场景作用域)。图片生成入口应优先把 owner user id 和 profile id 透传到失败审计,不要只保留 provider 级聚合,否则很难按“谁触发、哪个作品触发”定位问题。入库优先复用 tracking outboxoutbox 不可写或保护阈值拒绝时回退同步写 SpacetimeDB不得新增前端兜底或在 SpacetimeDB reducer 内做外部 I/O。 - 外部 API 失败审计:外部供应商调用未成功时,`api-server` 必须发送 OTLP 失败事件并写入 `tracking_event`。VectorEngine 图片 provider 在 `platform-image` 内输出结构化日志和 `PlatformImageFailureAudit`,覆盖 `request_send``response_body``upstream_status``response_parse``missing_image``image_download` 阶段;`api-server` 只把该 audit 映射成 `external_api_call_failure``scope_kind = module``scope_id = provider``module_key = external-api`。metadata 固定包含 provider、endpoint、operation、failureStage、statusCode、statusClass、timeout、retryable、errorMessage、latencyMs、promptChars、referenceImageCount、imageModel、rawExcerpt以及在调用方可获得上下文时补充的 `userId`(触发者)和 `profileId`(草稿 / 作品 / 场景作用域)。图片生成入口应优先把 owner user id 和 profile id 透传到失败审计,不要只保留 provider 级聚合,否则很难按“谁触发、哪个作品触发”定位问题。入库优先复用 tracking outboxoutbox 不可写或保护阈值拒绝时回退同步写 SpacetimeDB不得新增前端兜底或在 SpacetimeDB reducer 内做外部 I/O。
- 外部生成运行记录:所有外部生成编排的完成态统一写入 `tracking_event``event_key = external_generation_run``scope_kind = module``scope_id = provider``module_key = external-generation`。metadata 固定包含 `runId``provider``operation``requestLabel``requestPayload``status``success``failureReason``providerRequestId``resultPayload``startedAtMicros``completedAtMicros``durationMs`。这类记录只用于运行审计和排障,不再走 `ai_task` 旧表。 - 外部生成运行记录:所有外部生成编排的完成态统一写入 `tracking_event``event_key = external_generation_run``scope_kind = module``scope_id = provider``module_key = external-generation`。metadata 固定包含 `runId``provider``operation``requestLabel``requestPayload``status``success``failureReason``providerRequestId``resultPayload``startedAtMicros``completedAtMicros``durationMs`。这类记录只用于运行审计和排障,不再走 `ai_task` 旧表。

View File

@@ -1,6 +1,6 @@
# 本地开发验证与生产运维 # 本地开发验证与生产运维
更新时间:`2026-05-15` 更新时间:`2026-06-05`
## 标准开发流程 ## 标准开发流程
@@ -47,7 +47,7 @@ npm run dev:api-server
Linux 本机多用户并发开发时,`npm run dev``npm run dev:*` 单模块命令会先在系统级端口段注册表里给当前用户分配一个端口段,再把该段映射为 `web = start``api = start + 1``spacetime = start + 2``admin-web = start + 3`。默认注册表目录是 `/var/tmp/genarrative-dev-port-ranges/`,其中 `registry.json` 记录各用户的活跃段,`registry.lock` 负责串行化分配;可以用 `GENARRATIVE_DEV_PORT_RANGE_REGISTRY_DIR` 覆盖目录。系统自动分配时从 `10000-10099` 开始,每次占用 100 个端口块,后续块按 `10100-10199``10200-10299` 递增;`GENARRATIVE_DEV_PORT_RANGE``--port-range` 只在 Linux 上生效Windows 仍按原来的 3000 / 8082 / 3101 / 3102 端口探测与漂移逻辑运行,不读这个系统级注册表。 Linux 本机多用户并发开发时,`npm run dev``npm run dev:*` 单模块命令会先在系统级端口段注册表里给当前用户分配一个端口段,再把该段映射为 `web = start``api = start + 1``spacetime = start + 2``admin-web = start + 3`。默认注册表目录是 `/var/tmp/genarrative-dev-port-ranges/`,其中 `registry.json` 记录各用户的活跃段,`registry.lock` 负责串行化分配;可以用 `GENARRATIVE_DEV_PORT_RANGE_REGISTRY_DIR` 覆盖目录。系统自动分配时从 `10000-10099` 开始,每次占用 100 个端口块,后续块按 `10100-10199``10200-10299` 递增;`GENARRATIVE_DEV_PORT_RANGE``--port-range` 只在 Linux 上生效Windows 仍按原来的 3000 / 8082 / 3101 / 3102 端口探测与漂移逻辑运行,不读这个系统级注册表。
后端日志默认写入 `logs/api-server/`。后端 API smoke 使用 `npm run dev:api-server` 并检查 `/healthz`;不要使用旧 `api-server:maincloud` 或任何 `GENARRATIVE_SPACETIME_MAINCLOUD_*` 口径。 后端日志默认写入 `logs/api-server/`。后端 API smoke 使用 `npm run dev:api-server` 并检查 `/healthz`需要确认实例可接生产流量时检查 `/readyz`不要使用旧 `api-server:maincloud` 或任何 `GENARRATIVE_SPACETIME_MAINCLOUD_*` 口径。
开发态 `npm run dev``npm run dev:api-server` 会默认注入 `GENARRATIVE_DEV_PASSWORD_ENTRY_AUTO_REGISTER_ENABLED=true`,因此密码登录在本地开发环境可直接注册未知手机号账号;生产环境仍按 `api-server` 配置默认关闭该开关。 开发态 `npm run dev``npm run dev:api-server` 会默认注入 `GENARRATIVE_DEV_PASSWORD_ENTRY_AUTO_REGISTER_ENABLED=true`,因此密码登录在本地开发环境可直接注册未知手机号账号;生产环境仍按 `api-server` 配置默认关闭该开关。
@@ -69,6 +69,8 @@ spacetime sql <database> "SELECT * FROM puzzle_gallery_card_view LIMIT 1" --serv
本地 `.env``.env.local``.env.secrets.local` 修改后必须重启 `api-server` 才会生效;若已经通过 `npm run dev` 启动完整联调,可在该终端输入 `rs api-server`。排查 RPG / 拼图 / 抓大鹅等 VectorEngine 生图链路时,确认 `VECTOR_ENGINE_BASE_URL``VECTOR_ENGINE_API_KEY``VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 只在本地或服务器密钥文件中配置,不能写入 Git。VectorEngine `gpt-image-2` 图片协议、URL / base64 响应解析、远端图片下载和 provider 侧结构化日志在 `server-rs/crates/platform-image``api-server` 只做配置、玩法编排、OSS / asset 持久化、计费和失败审计落库。开局 CG 故事板、首图、背景和图集都属于长耗时图片请求;后端默认会把 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 下限收口到 `1000000`,旧进程仍可能沿用重启前的短超时。若 VectorEngine 在 `send()` 阶段失败且日志显示 `SendRequest`,先看同一 `request_id` 的 provider 日志字段 `source``source_chain``source_chain_depth`,再查 `external_api_call_failure.metadata_json.errorSource`;当前 multipart `/v1/images/edits` 单独强制 HTTP/1.1。拼图关卡资产按 `level_scene -> ui_spritesheet -> level_background` 顺序生成,日志会带 `slot``asset_kind``elapsed_ms` 本地 `.env``.env.local``.env.secrets.local` 修改后必须重启 `api-server` 才会生效;若已经通过 `npm run dev` 启动完整联调,可在该终端输入 `rs api-server`。排查 RPG / 拼图 / 抓大鹅等 VectorEngine 生图链路时,确认 `VECTOR_ENGINE_BASE_URL``VECTOR_ENGINE_API_KEY``VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 只在本地或服务器密钥文件中配置,不能写入 Git。VectorEngine `gpt-image-2` 图片协议、URL / base64 响应解析、远端图片下载和 provider 侧结构化日志在 `server-rs/crates/platform-image``api-server` 只做配置、玩法编排、OSS / asset 持久化、计费和失败审计落库。开局 CG 故事板、首图、背景和图集都属于长耗时图片请求;后端默认会把 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 下限收口到 `1000000`,旧进程仍可能沿用重启前的短超时。若 VectorEngine 在 `send()` 阶段失败且日志显示 `SendRequest`,先看同一 `request_id` 的 provider 日志字段 `source``source_chain``source_chain_depth`,再查 `external_api_call_failure.metadata_json.errorSource`;当前 multipart `/v1/images/edits` 单独强制 HTTP/1.1。拼图关卡资产按 `level_scene -> ui_spritesheet -> level_background` 顺序生成,日志会带 `slot``asset_kind``elapsed_ms`
VectorEngine 图片生成 / 编辑在 `request_send` 阶段出现 `timeout``connect` 错误时,`platform-image` 会对同一请求最多发送 3 次multipart 图片编辑每次重试都会重新构造 form避免复用已消费的 body。日志中 `VectorEngine 图片请求发送失败,准备重试` 表示本次失败已进入下一次尝试;最终仍失败时才会写入 `external_api_call_failure` 并返回 504。排查生产失败时应同时统计 retry 前的尝试日志和最终 audit避免把一次用户请求内的多次发送误判成多个用户请求。
查看本地 Rust / SpacetimeDB 日志: 查看本地 Rust / SpacetimeDB 日志:
```bash ```bash
@@ -242,26 +244,27 @@ Jenkins 按 web / api / Spacetime module / build / deploy / publish 拆分
`Genarrative-Web-Build` 打包 `web.tar.gz` 前、`Genarrative-Web-Deploy` 解包后都会把 Web 静态目录规范为目录 `755`、文件 `644`。如果前端页面能打开但 public 图片、字体或音频返回 `403 Forbidden`,优先检查当前 `/srv/genarrative/web` 指向的 release 中对应文件权限是否被异常归档为 `600`,临时恢复可对该 release 的 `web` 目录执行目录 `755`、文件 `644` 的权限修正。 `Genarrative-Web-Build` 打包 `web.tar.gz` 前、`Genarrative-Web-Deploy` 解包后都会把 Web 静态目录规范为目录 `755`、文件 `644`。如果前端页面能打开但 public 图片、字体或音频返回 `403 Forbidden`,优先检查当前 `/srv/genarrative/web` 指向的 release 中对应文件权限是否被异常归档为 `600`,临时恢复可对该 release 的 `web` 目录执行目录 `755`、文件 `644` 的权限修正。
生产 Jenkins 的 `Pipeline script from SCM` 由 Jenkins controller 读取 JenkinsfileSCM URL 继续使用 `https://git.genarrative.world/GenarrativeAI/Genarrative.git`。现在所有生产流水线 job 的首次 checkout 都先走 `http://127.0.0.1:3000/GenarrativeAI/Genarrative.git`,失败后回退到 `https://git.genarrative.world/GenarrativeAI/Genarrative.git`;两层 checkout 都必须保留单分支 refspec、`shallow=true``depth=1``noTags=true``honorRefspec=true`,后续二次源码确认继续走 `scripts/jenkins-checkout-source.sh` 生产 Jenkins 的 `Pipeline script from SCM` 由 Jenkins controller 读取 Jenkinsfile`Genarrative-Server-Provision` 是服务器初始化流水线Job 配置里的 SCM URL 必须使用 controller 本机可访问的仓库路径或内网 Gitea 地址,不能使用 `https://git.genarrative.world/...`;否则日志一开始的 `Checking out git ... to read jenkins/Jenkinsfile.production-server-provision` 就会先从公网拉 Jenkinsfile。其它构建 / 发布流水线仍按各自 Jenkinsfile 的 checkout 口径执行;所有 `GitSCM checkout` 都必须保留单分支 refspec、`shallow=true``depth=1``noTags=true``honorRefspec=true`
`Genarrative-Stdb-Module-Publish``Pipeline script from SCM` 阶段如果一开始就报 `No such DSL method 'pipeline'`,优先检查 `jenkins/Jenkinsfile.production-stdb-module-publish` 是否带 UTF-8 BOM。Jenkins Declarative Pipeline 的首个 token 必须是纯 `pipeline`;仓库中的 Jenkinsfile 应保存为 UTF-8 without BOM只有临时写给 Windows PowerShell 5.1 `-File` 执行的 `.ps1` 才需要按对应 helper 转成带 BOM。验证时可检查文件前三字节不再是 `EF BB BF`,并运行 `validateDeclarativePipeline` 或重放该流水线。 `Genarrative-Stdb-Module-Publish``Pipeline script from SCM` 阶段如果一开始就报 `No such DSL method 'pipeline'`,优先检查 `jenkins/Jenkinsfile.production-stdb-module-publish` 是否带 UTF-8 BOM。Jenkins Declarative Pipeline 的首个 token 必须是纯 `pipeline`;仓库中的 Jenkinsfile 应保存为 UTF-8 without BOM只有临时写给 Windows PowerShell 5.1 `-File` 执行的 `.ps1` 才需要按对应 helper 转成带 BOM。验证时可检查文件前三字节不再是 `EF BB BF`,并运行 `validateDeclarativePipeline` 或重放该流水线。
`Genarrative-Stdb-Module-Build` 或 SpacetimeDB module 构建失败若出现 Rust `E0425 cannot find function migrate_*`,优先排查 `server-rs/crates/spacetime-module/src/runtime/creation_entry_config.rs` 等同文件内默认种子迁移 helper 是否在分支合并时只保留了调用、漏掉了函数定义。`Genarrative-Stdb-Module-Build` 现在运行在 `linux && genarrative-build` 节点上Checkout 与 Build 都走 bash + cargo + sccache不再依赖 Windows PowerShell 或 Git Bash。修复时不要直接删除迁移调用应恢复只纠偏历史默认种子且不覆盖后台手动配置的 helper并用 `cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml` 复现 Jenkins module 编译路径。 `Genarrative-Stdb-Module-Build` 或 SpacetimeDB module 构建失败若出现 Rust `E0425 cannot find function migrate_*`,优先排查 `server-rs/crates/spacetime-module/src/runtime/creation_entry_config.rs` 等同文件内默认种子迁移 helper 是否在分支合并时只保留了调用、漏掉了函数定义。`Genarrative-Stdb-Module-Build` 现在运行在 `linux && genarrative-build` 节点上Checkout 与 Build 都走 bash + cargo + sccache不再依赖 Windows PowerShell 或 Git Bash。修复时不要直接删除迁移调用应恢复只纠偏历史默认种子且不覆盖后台手动配置的 helper并用 `cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml` 复现 Jenkins module 编译路径。
`Genarrative-Server-Provision` 现在也运行在 `linux && genarrative-build` / `linux && genarrative-release-deploy` 节点上,`Prepare Provision Tools` 会在 Linux build 节点直接准备 SpacetimeDB 与 `otelcol-contrib` 交付件,再 stash 给后续发布阶段;旧 Windows 下载 helper 已退役。`Genarrative-Stdb-Module-Build``Genarrative-Server-Provision` `Genarrative-Notify-Email` 都不再需要单独的 Windows 节点口径 `Genarrative-Server-Provision` 只做服务器初始化,不再承担构建职责。流水线全程运行在目标服务器 agent`DEPLOY_TARGET=development` 使用 `linux && genarrative-dev-deploy``DEPLOY_TARGET=release` 使用 `linux && genarrative-release-deploy``Prepare Provision Tools` 也在同一个目标 agent 工作区内准备 SpacetimeDB 与 `otelcol-contrib` 交付件,不再切到 `linux && genarrative-build`,也不再 stash 给后续阶段。`SOURCE_GIT_REMOTE_URL` 必须显式填写为目标 agent 可访问的本机路径、`file:///` 地址、localhost / 127.0.0.1、RFC1918 内网 HTTP Git 地址、单标签内网主机名或 `.local` / `.lan` / `.internal` 地址;这条流水线不配置公网 Git 备用地址,目标 agent 拉不到内网源就应直接失败。真实初始化会写入 `/etc` / systemd / Nginx、创建系统用户并修改服务目标 dev / release agent 非 dry-run 时都必须具备 root 权限
生产环境变量模板:`deploy/env/api-server.env.example`。真实密钥只放服务器,不提交 Git不写入文档示例。 生产环境变量模板:`deploy/env/api-server.env.example`。真实密钥只放服务器,不提交 Git不写入文档示例。
`Genarrative-Server-Provision` 会安装基础构建依赖、systemd 模板和 Nginx 站点模板。Ubuntu / apt 目标机会额外安装 `libnginx-mod-http-brotli-filter``libnginx-mod-http-brotli-static`,随后由 `scripts/jenkins-server-provision.sh` 通过临时 `nginx -t` 配置探测 Brotli 指令是否可用;该临时配置必须先 `include /etc/nginx/modules-enabled/*.conf`,因为 apt 安装的 Brotli 是动态模块,不会出现在普通 `nginx -V` 编译参数里。探测成功才在渲染后的 `deploy/nginx/genarrative.conf` / `genarrative-dev-http.conf` 中启用 Brotli避免未安装模块的机器直接写入无效配置。Provision 写入 Genarrative Nginx 站点时会把 `/etc/nginx/sites-enabled/default*` 移到 `/etc/nginx/sites-disabled/`,避免 Debian / Certbot 默认站点继续占用 `genarrative.world` / `www.genarrative.world` 并在 `nginx -T` 中出现 `conflicting server name ... ignored`。如果 `nginx -t` 失败,脚本会恢复写入前的 Genarrative 配置和被移动的默认站点。 `Genarrative-Server-Provision` 会安装 systemd 模板和 Nginx 站点模板,不再安装 clang / lld / pkg-config / OpenSSL headers / sccache 等通用构建链依赖。因 VectorEngine 图片上游 POST 已改用 `libcurl`,当前 Linux release 构建出的 `api-server` 运行时需要 `OPENSSL_3.2.0` 符号Ubuntu 24.04 apt 默认只提供 OpenSSL 3.0.x不能直接满足该符号版本。Provision 会把 OpenSSL `3.2.0` 独立安装到 `/opt/genarrative/openssl-3.2.0`,校验官方 tarball SHA256并只通过 `genarrative-api.service``LD_LIBRARY_PATH=/opt/genarrative/openssl-3.2.0/lib64:/opt/genarrative/openssl-3.2.0/lib` 让 api-server 使用,避免替换系统 OpenSSL 或影响 ssh / nginx / apt。Ubuntu / apt 目标机会额外安装 `libnginx-mod-http-brotli-filter``libnginx-mod-http-brotli-static`,随后由 `scripts/jenkins-server-provision.sh` 通过临时 `nginx -t` 配置探测 Brotli 指令是否可用;该临时配置必须先 `include /etc/nginx/modules-enabled/*.conf`,因为 apt 安装的 Brotli 是动态模块,不会出现在普通 `nginx -V` 编译参数里。探测成功才在渲染后的 `deploy/nginx/genarrative.conf` / `genarrative-dev-http.conf` 中启用 Brotli避免未安装模块的机器直接写入无效配置。Provision 写入 Genarrative Nginx 站点时会把 `/etc/nginx/sites-enabled/default*` 移到 `/etc/nginx/sites-disabled/`,避免 Debian / Certbot 默认站点继续占用 `genarrative.world` / `www.genarrative.world` 并在 `nginx -T` 中出现 `conflicting server name ... ignored`。如果 `nginx -t` 失败,脚本会恢复写入前的 Genarrative 配置和被移动的默认站点。
50 HTTP req/s 首版压测优化口径: 50 HTTP req/s 首版压测优化口径:
- `api-server` 生产模板默认 `GENARRATIVE_API_LISTEN_BACKLOG=1024``GENARRATIVE_API_WORKER_THREADS=4`;本地未设置 worker threads 时继续使用 Tokio 默认值。 - `api-server` 生产模板默认 `GENARRATIVE_API_LISTEN_BACKLOG=1024``GENARRATIVE_API_WORKER_THREADS=4`;本地未设置 worker threads 时继续使用 Tokio 默认值。
- `GENARRATIVE_API_MAX_CONCURRENT_REQUESTS=512` 开启应用内 HTTP 并发背压;`GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS=320``GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS=64``GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS=16` 分别限制公开列表、公开详情和后台 API 热路径。超过许可时直接返回 `429 Too Many Requests``Retry-After: 1``/healthz` 不受该限制。这些值不是 RPS 限速;如果压测中 429 上升但内存和 p95 收敛,说明背压正在保护进程。直连 `api-server` 的极高 RPS 压测若出现 `connection refused`,通常已经打到 TCP 监听 / accept 层,应同时检查 backlog、Nginx upstream keepalive 和前置限流。 - `GENARRATIVE_API_MAX_CONCURRENT_REQUESTS=512` 开启应用内 HTTP 并发背压;`GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS=320``GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS=64``GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS=16` 分别限制公开列表、公开详情和后台 API 热路径。超过许可时直接返回 `429 Too Many Requests``Retry-After: 1``/healthz``/readyz` 不受该限制。这些值不是 RPS 限速;如果压测中 429 上升但内存和 p95 收敛,说明背压正在保护进程。直连 `api-server` 的极高 RPS 压测若出现 `connection refused`,通常已经打到 TCP 监听 / accept 层,应同时检查 backlog、Nginx upstream keepalive 和前置限流。
- `genarrative-api.service` 设置 `LimitNOFILE=65535``TasksMax=2048`;上线后用 `systemctl show genarrative-api.service -p LimitNOFILE -p TasksMax``cat /proc/$(pidof api-server)/limits` 核对 - `api-server` 正常运行时 `/healthz` 返回进程存活状态,`/readyz` 返回是否仍接收新流量;收到 `SIGINT` / `SIGTERM` 后会先把 readiness 标记为不可用,再让 Axum 停止接新连接并等待已有 HTTP 请求排空。systemd 仍以 `KillSignal=SIGINT` 停服务,`TimeoutStopSec=90` 作为长请求排空上限
- Server provision 不再通过 Windows helper 下载。`Genarrative-Server-Provision` `Prepare Provision Tools` 在 Linux build 节点直接准备 `spacetime-x86_64-unknown-linux-gnu.tar.gz``otelcol-contrib_0.151.0_linux_amd64.tar.gz`,再 stash `provision-tools/` 给后续发布阶段;如果 build 节点需要代理,在 `PROVISION_DOWNLOAD_PROXY` 配置 Linux 侧可访问的 HTTP 代理。后续 Linux 目标节点只消费 `provision-tools/`,不再回退到外网下载 - `genarrative-api.service` 设置 `LimitNOFILE=65535``TasksMax=2048`;上线后用 `systemctl show genarrative-api.service -p LimitNOFILE -p TasksMax -p TimeoutStopUSec``cat /proc/$(pidof api-server)/limits` 核对
- `Genarrative-Stdb-Module-Build``Genarrative-Web-Build``Genarrative-Api-Build``Genarrative-*Deploy``Genarrative-Database-Import/Export``Genarrative-Full-Build-And-Deploy``Genarrative-Notify-Email` 的生产流水线现都以 Linux agent 为主,统一走 `http://127.0.0.1:3000/GenarrativeAI/Genarrative.git` 优先、`https://git.genarrative.world/GenarrativeAI/Genarrative.git` 备用的 checkout 口径 - Server provision 不再通过 Windows helper 下载,也不再通过 Linux build 节点中转工具包。`Prepare Provision Tools` 在目标 dev / release agent 工作区内准备 `spacetime-x86_64-unknown-linux-gnu.tar.gz``otelcol-contrib_0.151.0_linux_amd64.tar.gz` 并生成 `provision-tools/`;如果目标服务器下载需要代理,在 `PROVISION_DOWNLOAD_PROXY` 配置目标机可访问的 HTTP 代理
-`Genarrative-Server-Provision` 外,`Genarrative-Stdb-Module-Build``Genarrative-Web-Build``Genarrative-Api-Build``Genarrative-*Deploy``Genarrative-Database-Import/Export``Genarrative-Full-Build-And-Deploy``Genarrative-Notify-Email` 的生产流水线现都以 Linux agent 为主,仍按各自 Jenkinsfile 的 checkout 口径执行。Server provision 不使用公网备用 Git 源。
- `otelcol-contrib.service` 作为可选系统服务加入 provision默认监听 `127.0.0.1:4317/4318` 并使用 `deploy/otelcol/genarrative-debug.yaml`。api-server 是否发送 OTLP 仍由 `GENARRATIVE_OTEL_ENABLED` 控制,服务 unit 见 `deploy/systemd/otelcol-contrib.service` - `otelcol-contrib.service` 作为可选系统服务加入 provision默认监听 `127.0.0.1:4317/4318` 并使用 `deploy/otelcol/genarrative-debug.yaml`。api-server 是否发送 OTLP 仍由 `GENARRATIVE_OTEL_ENABLED` 控制,服务 unit 见 `deploy/systemd/otelcol-contrib.service`
- Nginx `/api/``/admin/api/` 通过 `genarrative_api` upstream 代理到 `127.0.0.1:8082`upstream keepalive 为 64`limit_conn` 负责连接 / 并发保护,`limit_req` 负责入口 RPS 快拒绝。当前模板把公开 gallery list 单独放到 `genarrative_gallery_rps`,默认 `rate=5000r/s``burst=4096``limit_conn=320`;公开详情和普通 API 放到 `genarrative_api_rps`,后台 API 放到 `genarrative_admin_rps`。通用 `/api` location 设置 `client_max_body_size 64m` 是反代兜底,防止拼图入口页 / 新增关卡本地参考图 Data URL 或旧兼容请求在到达 `api-server` 前被默认 1 MiB 上限拦截;拼图本地参考图前后端统一限制 6MB历史图片仍提交 `referenceImageAssetObjectId(s)`。若线上出现 `413 Request Entity Too Large` 且 access log 中 `request_time=0.000``upstream_status=-`,说明请求在 Nginx 层被拦截,先用 `nginx -T | grep client_max_body_size` 检查 release 模板是否已渲染并 reload同时检查前端是否超出 6MB 或错误提交了未压缩大图。`limit_conn_status 429``limit_req_status 429` 必须在 HTTP 与 HTTPS server 中同时生效;若线上压测看到 `limiting connections by zone "genarrative_api_conn"` 却返回 503优先检查 `nginx -T` 里 HTTPS server 是否缺少这些状态码,以及 `/api/runtime/puzzle/gallery` 是否误落到通用 `location ~ ^/api``limit_conn=64`。压测时看 `/var/log/nginx/genarrative.access.log` 中的 `request_time``upstream_connect_time``upstream_header_time``upstream_response_time``upstream_status``request_id` - Nginx `/api/``/admin/api/` 通过 `genarrative_api` upstream 代理到 `127.0.0.1:8082`upstream keepalive 为 64`limit_conn` 负责连接 / 并发保护,`limit_req` 负责入口 RPS 快拒绝。当前模板把公开 gallery list 单独放到 `genarrative_gallery_rps`,默认 `rate=5000r/s``burst=4096``limit_conn=320`;公开详情和普通 API 放到 `genarrative_api_rps`,后台 API 放到 `genarrative_admin_rps`。通用 `/api` location 设置 `client_max_body_size 64m` 是反代兜底,防止拼图入口页 / 新增关卡本地参考图 Data URL 或旧兼容请求在到达 `api-server` 前被默认 1 MiB 上限拦截;拼图本地参考图前后端统一限制 6MB历史图片仍提交 `referenceImageAssetObjectId(s)`。若线上出现 `413 Request Entity Too Large` 且 access log 中 `request_time=0.000``upstream_status=-`,说明请求在 Nginx 层被拦截,先用 `nginx -T | grep client_max_body_size` 检查 release 模板是否已渲染并 reload同时检查前端是否超出 6MB 或错误提交了未压缩大图。`limit_conn_status 429``limit_req_status 429` 必须在 HTTP 与 HTTPS server 中同时生效;若线上压测看到 `limiting connections by zone "genarrative_api_conn"` 却返回 503优先检查 `nginx -T` 里 HTTPS server 是否缺少这些状态码,以及 `/api/runtime/puzzle/gallery` 是否误落到通用 `location ~ ^/api``limit_conn=64`。压测时看 `/var/log/nginx/genarrative.access.log` 中的 `request_time``upstream_connect_time``upstream_header_time``upstream_response_time``upstream_status``request_id`
- 作品列表 K6 脚本一次 iteration 默认请求两个公开接口,因此约 50 HTTP req/s 的目标命令使用 `SCENARIO=spike START_RPS=5 PEAK_RPS=25 HOLD=60s END_RPS=5 DETAIL_RATIO=0 npm run loadtest:k6:works` - 作品列表 K6 脚本一次 iteration 默认请求两个公开接口,因此约 50 HTTP req/s 的目标命令使用 `SCENARIO=spike START_RPS=5 PEAK_RPS=25 HOLD=60s END_RPS=5 DETAIL_RATIO=0 npm run loadtest:k6:works`
@@ -279,7 +282,7 @@ npm run container:k6
npm run container:down npm run container:down
``` ```
容器方案默认暴露 `http://127.0.0.1:18080``api-server` 在容器内监听 `0.0.0.0:8082`Nginx 通过 `api-server:8082` upstream 反代 `/api/``/admin/api/`。SpacetimeDB 也纳入 compose容器内由 `spacetimedb:3101` 提供服务,宿主机通过 `http://127.0.0.1:13101` 进行模块发布Collector 镜像使用 `otel/opentelemetry-collector-contrib:0.151.0`。生产 provision 侧现在由 Linux build 节点直接准备 `provision-tools/otelcol-contrib`再交给后续 Linux 发布阶段安装本机 `otelcol-contrib.service`真实库名、token 和外部服务密钥只写本地 `deploy/container/api-server.env`,不提交 Git。完整拓扑、端口、k6 参数和 OTLP debug exporter 使用方法见 `deploy/container/README.md` 容器方案默认暴露 `http://127.0.0.1:18080``api-server` 在容器内监听 `0.0.0.0:8082`Nginx 通过 `api-server:8082` upstream 反代 `/api/``/admin/api/`。SpacetimeDB 也纳入 compose容器内由 `spacetimedb:3101` 提供服务,宿主机通过 `http://127.0.0.1:13101` 进行模块发布Collector 镜像使用 `otel/opentelemetry-collector-contrib:0.151.0`。生产 provision 侧现在由目标 dev / release agent 自己准备 `provision-tools/otelcol-contrib`安装本机 `otelcol-contrib.service`真实库名、token 和外部服务密钥只写本地 `deploy/container/api-server.env`,不提交 Git。完整拓扑、端口、k6 参数和 OTLP debug exporter 使用方法见 `deploy/container/README.md`
`npm run container:config` 默认只做 quiet 校验,避免把本地 env 中的 token 展开到终端;确需排查完整 compose 时再传 `-- --print` `npm run container:config` 默认只做 quiet 校验,避免把本地 env 中的 token 展开到终端;确需排查完整 compose 时再传 `-- --print`
OpenTelemetry 现阶段默认开启 OTLP traces / metrics / logs但本地日志与 Nginx 文件日志仍保留: OpenTelemetry 现阶段默认开启 OTLP traces / metrics / logs但本地日志与 Nginx 文件日志仍保留:
@@ -293,6 +296,7 @@ OpenTelemetry 现阶段默认开启 OTLP traces / metrics / logs但本地日
- api-server 会随 metrics 发送进程级指标:`process.memory.usage``process.memory.virtual``process.cpu.time``genarrative.process.cpu.usage_percent``process.thread.count``genarrative.process.memory.private`Windows 额外发送 `process.windows.handle.count`Linux 额外发送 `process.unix.file_descriptor.count`。这些指标只描述当前进程,不携带请求、用户或作品 label。 - api-server 会随 metrics 发送进程级指标:`process.memory.usage``process.memory.virtual``process.cpu.time``genarrative.process.cpu.usage_percent``process.thread.count``genarrative.process.memory.private`Windows 额外发送 `process.windows.handle.count`Linux 额外发送 `process.unix.file_descriptor.count`。这些指标只描述当前进程,不携带请求、用户或作品 label。
- HTTP 运行态补充发送 `genarrative.http.server.response_bodies.in_flight``genarrative.http.server.request_permits.available`,后者带低基数 `pool=default|gallery|detail|admin` label用于区分业务 handler / 背压 permit 是否仍被占用;拼图广场热点缓存补充发送 `genarrative.puzzle_gallery.cache.*` 指标,记录 fresh hit、stale hit、未命中、后台刷新开始 / 失败、重建耗时和预序列化 data JSON 字节数。 - HTTP 运行态补充发送 `genarrative.http.server.response_bodies.in_flight``genarrative.http.server.request_permits.available`,后者带低基数 `pool=default|gallery|detail|admin` label用于区分业务 handler / 背压 permit 是否仍被占用;拼图广场热点缓存补充发送 `genarrative.puzzle_gallery.cache.*` 指标,记录 fresh hit、stale hit、未命中、后台刷新开始 / 失败、重建耗时和预序列化 data JSON 字节数。
- 外部 API 失败统一发送 OTLP 并落库。当前 VectorEngine `gpt-image-2` 图片生成 / 编辑失败由 `platform-image` provider 输出结构化日志字段,字段包括 provider、endpoint、failure_stage、status、source、source_chain、source_chain_depth、timeout、retryable、latency_ms、prompt_chars、reference_image_count、image_model、request_params 和 raw_excerpt图片编辑请求参数日志还会带 reference_image_bytes_total并在 request_params.referenceImages 中记录每个 multipart `image` part 的 fileName、mimeType 和 bytes不记录 API key 或原始图片 bytes`api-server` 再记录指标 `genarrative.external_api.failures{provider,failure_stage,status_class,retryable}`,并写入 `tracking_event``event_key = external_api_call_failure``module_key = external-api``scope_kind = module``scope_id = provider`。调用方能拿到身份上下文时,失败事件还会在行级 `user_id` / `owner_user_id` / `profile_id``metadata_json.userId` / `metadata_json.profileId` / `metadata_json.requestId` / `metadata_json.errorSource` 中记录触发者、草稿 / 作品作用域、请求标识和传输错误链。排障时先按 provider / failureStage 聚合,再下钻 userId / profileId最后结合 request 日志、errorSource 和上游响应 excerpt 判断是限流、超时、解析失败还是未返回图片。 - 外部 API 失败统一发送 OTLP 并落库。当前 VectorEngine `gpt-image-2` 图片生成 / 编辑失败由 `platform-image` provider 输出结构化日志字段,字段包括 provider、endpoint、failure_stage、status、source、source_chain、source_chain_depth、timeout、retryable、latency_ms、prompt_chars、reference_image_count、image_model、request_params 和 raw_excerpt图片编辑请求参数日志还会带 reference_image_bytes_total并在 request_params.referenceImages 中记录每个 multipart `image` part 的 fileName、mimeType 和 bytes不记录 API key 或原始图片 bytes`api-server` 再记录指标 `genarrative.external_api.failures{provider,failure_stage,status_class,retryable}`,并写入 `tracking_event``event_key = external_api_call_failure``module_key = external-api``scope_kind = module``scope_id = provider`。调用方能拿到身份上下文时,失败事件还会在行级 `user_id` / `owner_user_id` / `profile_id``metadata_json.userId` / `metadata_json.profileId` / `metadata_json.requestId` / `metadata_json.errorSource` 中记录触发者、草稿 / 作品作用域、请求标识和传输错误链。排障时先按 provider / failureStage 聚合,再下钻 userId / profileId最后结合 request 日志、errorSource 和上游响应 excerpt 判断是限流、超时、解析失败还是未返回图片。
- OSS 平台适配器也输出结构化日志,覆盖 `sign_post_object``sign_get_object_url``head_object``put_object`。排查资产签名、上传或确认失败时,先按 `provider=aliyun-oss``operation` 过滤,再看 `object_key` / `key_prefix``status``status_class``error_kind``content_length``content_type``elapsed_ms`;日志不得包含 AccessKey、policy、signature、Authorization header 或完整 signed URL。
- SpacetimeDB 观测分为两类procedure / reducer 调用继续用 `genarrative.spacetime.procedure.*`,订阅本地 cache 读使用 `genarrative.spacetime.read.*``read=list_puzzle_gallery` 表示拼图广场当前从 `puzzle_gallery_card_view` 本地 cache 读取,不再每个 HTTP 请求调用 `list_puzzle_gallery` procedure。 - SpacetimeDB 观测分为两类procedure / reducer 调用继续用 `genarrative.spacetime.procedure.*`,订阅本地 cache 读使用 `genarrative.spacetime.read.*``read=list_puzzle_gallery` 表示拼图广场当前从 `puzzle_gallery_card_view` 本地 cache 读取,不再每个 HTTP 请求调用 `list_puzzle_gallery` procedure。
- 本地 Windows 直连压测的内存高水位要结合 K6 VU / 连接数解释。250 RPS 下过高 `PREALLOCATED_VUS` 可能让 300 个本地 Established 连接把 `api-server` private memory 瞬时推到 GB 级,且 `/healthz` 小响应也能复现;若压测结束后回落、`response_bodies.in_flight` 和背压 permit 未显示业务积压,应优先按连接 / 发送链路高水位处理,而不是判断为 SpacetimeDB 或 JSON 缓存泄漏。 - 本地 Windows 直连压测的内存高水位要结合 K6 VU / 连接数解释。250 RPS 下过高 `PREALLOCATED_VUS` 可能让 300 个本地 Established 连接把 `api-server` private memory 瞬时推到 GB 级,且 `/healthz` 小响应也能复现;若压测结束后回落、`response_bodies.in_flight` 和背压 permit 未显示业务积压,应优先按连接 / 发送链路高水位处理,而不是判断为 SpacetimeDB 或 JSON 缓存泄漏。
- Rider 的 Logs 面板只展示 log event 自身字段,不会自动展开父 span 的全部 attributes请求完成日志会直接带 `request_id``http.request.method``http.route``url.scheme``url.path``http.response.status_code``status_class``latency_ms``slow_request`,完整链路继续到 Traces 面板按 trace/span 查看。 - Rider 的 Logs 面板只展示 log event 自身字段,不会自动展开父 span 的全部 attributes请求完成日志会直接带 `request_id``http.request.method``http.route``url.scheme``url.path``http.response.status_code``status_class``latency_ms``slow_request`,完整链路继续到 Traces 面板按 trace/span 查看。
@@ -388,9 +392,10 @@ GENARRATIVE_TRACKING_OUTBOX_DIR=/var/lib/genarrative/tracking-outbox
GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE=500 GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE=500
GENARRATIVE_TRACKING_OUTBOX_FLUSH_INTERVAL_MS=1000 GENARRATIVE_TRACKING_OUTBOX_FLUSH_INTERVAL_MS=1000
GENARRATIVE_TRACKING_OUTBOX_MAX_BYTES=268435456 GENARRATIVE_TRACKING_OUTBOX_MAX_BYTES=268435456
GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS=5000
``` ```
outbox 采用 NDJSON 文件保存原始事件。达到 `BATCH_SIZE` 时会立刻把当前 active 文件原子封存为 sealed 文件,并马上切到新的 active 继续写入;后台 worker 异步 flush sealed 文件HTTP 请求线程不等待 SpacetimeDB。`FLUSH_INTERVAL_MS` 只负责兜底封存长时间未满批的 active 文件。SpacetimeDB 批量 procedure 返回成功后删除 sealed 文件,失败则保留文件并重试。`MAX_BYTES` 是磁盘保护阈值,不是 flush 阈值;超过后低价值 route tracking 可以被丢弃并记录日志 / 指标关键同步事件不进入该丢弃路径。sealed 文件若出现无法解析的坏行,会重命名为 `corrupt-*` 隔离并记录 `genarrative.tracking_outbox.files.corrupt` 指标,避免一个坏文件阻塞后续批量入库。该机制提供至少一次投递语义,依赖 `tracking_event.event_id` 幂等跳过重复事件。 outbox 采用 NDJSON 文件保存原始事件。达到 `BATCH_SIZE` 时会立刻把当前 active 文件原子封存为 sealed 文件,并马上切到新的 active 继续写入;后台 worker 异步 flush sealed 文件HTTP 请求线程不等待 SpacetimeDB。`FLUSH_INTERVAL_MS` 只负责兜底封存长时间未满批的 active 文件。SpacetimeDB 批量 procedure 返回成功后删除 sealed 文件,失败则保留文件并重试。`MAX_BYTES` 是磁盘保护阈值,不是 flush 阈值;超过后低价值 route tracking 可以被丢弃并记录日志 / 指标关键同步事件不进入该丢弃路径。sealed 文件若出现无法解析的坏行,会重命名为 `corrupt-*` 隔离并记录 `genarrative.tracking_outbox.files.corrupt` 指标,避免一个坏文件阻塞后续批量入库。api-server 收到退出信号后会在 `GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS` 窗口内封存 active 文件并尽力 flush sealed 文件,超时或 SpacetimeDB 暂不可用时保留本地文件给下次启动继续投递。该机制提供至少一次投递语义,依赖 `tracking_event.event_id` 幂等跳过重复事件。
release 机器如果日志每秒刷 `tracking outbox ... Permission denied (os error 13)`,先检查 `/etc/genarrative/api-server.env` 是否缺少 `GENARRATIVE_TRACKING_OUTBOX_DIR`。缺少时 `api-server` 会回退到本地开发默认相对路径 `server-rs/.data/tracking-outbox`,而 systemd 的工作目录是只读发布目录 `/opt/genarrative/releases/<version>``genarrative` 用户无法在其中创建 `server-rs`。修复顺序: release 机器如果日志每秒刷 `tracking outbox ... Permission denied (os error 13)`,先检查 `/etc/genarrative/api-server.env` 是否缺少 `GENARRATIVE_TRACKING_OUTBOX_DIR`。缺少时 `api-server` 会回退到本地开发默认相对路径 `server-rs/.data/tracking-outbox`,而 systemd 的工作目录是只读发布目录 `/opt/genarrative/releases/<version>``genarrative` 用户无法在其中创建 `server-rs`。修复顺序:

View File

@@ -9,7 +9,7 @@
- H5 与桌面微信环境仍分别走 `wechat_h5` / `wechat_native`,不进入虚拟支付链路。 - H5 与桌面微信环境仍分别走 `wechat_h5` / `wechat_native`,不进入虚拟支付链路。
- `session_key` 只保存在后端认证仓储内,用于计算虚拟支付用户态签名,不下发给前端。 - `session_key` 只保存在后端认证仓储内,用于计算虚拟支付用户态签名,不下发给前端。
- 客户端支付成功回调只代表已拉起支付并返回成功;最终到账仍以后端虚拟支付消息推送写入订单为准,普通微信支付订单则继续走微信支付 V3 notify / query。虚拟支付订单的确认接口只读取本地订单真相不再用普通微信支付 V3 查单。 - 客户端支付成功回调只代表已拉起支付并返回成功;最终到账仍以后端虚拟支付消息推送写入订单为准,普通微信支付订单则继续走微信支付 V3 notify / query。虚拟支付订单的确认接口只读取本地订单真相不再用普通微信支付 V3 查单。
- 小程序 WebView 默认进入时会静默调用 `wx.login` 刷新后端微信登录态,避免历史登录用户只有前端 JWT、后端缺少 `session_key` 时无法生成虚拟支付签名 - 小程序 WebView 普通进入不预登录H5 触发受保护入口或支付前必须保留 `clientRuntime=wechat_mini_program` 等宿主上下文,并用 `MicroMessenger + miniProgram` User-Agent 兜底识别首点 bridge 未就绪场景,再跳转小程序原生授权态,确保后端拿到带 `session_key` 的微信登录态
## 关键文件 ## 关键文件
@@ -59,7 +59,7 @@ npm run check:encoding
## 注意事项 ## 注意事项
- 旧微信登录快照可能没有 `session_key`;小程序 WebView 会在普通进入时静默刷新一次微信登录态,刷新失败时仍允许匿名打开 WebView虚拟支付会继续由后端拦截并提示重新登录 - 旧微信登录快照可能没有 `session_key`普通进入小程序 WebView 仍允许匿名打开,虚拟支付会由后端拦截并提示用户在小程序内重新登录。H5 内部导航不得清理 `clientType``clientRuntime``miniProgramEnv`,且首点登录要用小程序 User-Agent 兜底识别,否则登录和支付会误判为普通网页环境
- 小程序充值商品全部映射到虚拟支付;泥点使用 `short_series_coin`,会员使用 `short_series_goods` - 小程序充值商品全部映射到虚拟支付;泥点使用 `short_series_coin`,会员使用 `short_series_goods`
- `short_series_coin` 只用于代币购买,后端从本次下单返回的充值中心商品快照读取 `points_amount` 并写入 `buyQuantity`;不要把 coin 商品当成道具,也不要把 `buyQuantity` 固定为 1。 - `short_series_coin` 只用于代币购买,后端从本次下单返回的充值中心商品快照读取 `points_amount` 并写入 `buyQuantity`;不要把 coin 商品当成道具,也不要把 `buyQuantity` 固定为 1。
- 后台新增的会员类充值商品会直接把商品 `productId` 作为微信 `short_series_goods` 的道具 ID例如微信后台道具 ID 为 `item01` 时,后台会员商品 `productId` 也应配置为 `item01`,且商品价格需要与微信后台道具价格一致。 - 后台新增的会员类充值商品会直接把商品 `productId` 作为微信 `short_series_goods` 的道具 ID例如微信后台道具 ID 为 `item01` 时,后台会员商品 `productId` 也应配置为 `item01`,且商品价格需要与微信后台道具价格一致。

View File

@@ -60,7 +60,7 @@
10. 敲木鱼作品架读取当前用户作品列表时走 `GET /api/creation/wooden-fish/works`;发布成功后平台壳必须同时刷新作品架与公开广场,避免作品刚发布时仍停留在旧列表。 10. 敲木鱼作品架读取当前用户作品列表时走 `GET /api/creation/wooden-fish/works`;发布成功后平台壳必须同时刷新作品架与公开广场,避免作品刚发布时仍停留在旧列表。
11. 移动端草稿页整体禁止长按选择文字,避免误触系统选区;输入框、文本域和可编辑区域仍必须保留文本选择能力。 11. 移动端草稿页整体禁止长按选择文字,避免误触系统选区;输入框、文本域和可编辑区域仍必须保留文本选择能力。
发现页 / 推荐页公开作品卡的作者行只显示可读公开昵称;不得把手机号掩码、`SY-*` 陶泥号或作品号拼接进卡片作者名。陶泥号搜索、作品号复制和完整作品身份只在搜索、详情页或明确的复制入口展示,避免卡片列表暴露账号标识。 发现页 / 推荐页公开作品卡的作者行只显示公开昵称或账号生成的脱敏手机号;不得把纯 `SY-*` 陶泥号或作品号当作卡片作者名。陶泥号搜索、作品号复制和完整作品身份只在搜索、详情页或明确的复制入口展示,避免卡片列表暴露额外账号标识。
发现 Tab、创作 Tab 与草稿 Tab 的页面根内容区不再套 `platform-page-stage` 外层全局卡片壳,让列表、筛选和玩法卡获得更宽的横向空间;推荐页和我的页仍按各自页面设计保留原有全局卡片口径。移动端“我的”页仍按顶部头像 / 昵称 / 陶泥号、会员横幅、三张统计卡、每日任务、五项常用功能宫格、设置入口和法律信息组织,不保留旧的底部“填邀请码”次级入口;常用功能当前只展示四项常驻入口时必须按四列铺满整行,不保留五列网格导致左对齐空位;每日任务卡必须读取 `/api/profile/tasks` 的当前任务摘要并在领取后同步刷新卡片进度。字号必须维持平台普通 UI 档位,不能因为窄屏把卡片标题、功能 label 或法律信息撑成展示级字号;最后一屏内容必须能在底部 dock 上方完整滚动露出,不得被固定底部导航遮挡。 发现 Tab、创作 Tab 与草稿 Tab 的页面根内容区不再套 `platform-page-stage` 外层全局卡片壳,让列表、筛选和玩法卡获得更宽的横向空间;推荐页和我的页仍按各自页面设计保留原有全局卡片口径。移动端“我的”页仍按顶部头像 / 昵称 / 陶泥号、会员横幅、三张统计卡、每日任务、五项常用功能宫格、设置入口和法律信息组织,不保留旧的底部“填邀请码”次级入口;常用功能当前只展示四项常驻入口时必须按四列铺满整行,不保留五列网格导致左对齐空位;每日任务卡必须读取 `/api/profile/tasks` 的当前任务摘要并在领取后同步刷新卡片进度。字号必须维持平台普通 UI 档位,不能因为窄屏把卡片标题、功能 label 或法律信息撑成展示级字号;最后一屏内容必须能在底部 dock 上方完整滚动露出,不得被固定底部导航遮挡。
@@ -126,7 +126,7 @@ RPG / 拼图等运行态存档仍以 `/api/profile/save-archives` 的后端列
- 拼图运行态顶部关卡信息采用游戏化铭牌样式:橘棕横向关卡名牌承载 `第 N 关` 和关卡名,左侧固定使用 `media/logo.png` 卡通形象;倒计时作为下挂米白小牌独立显示,紧贴铭牌但不遮挡棋盘。该样式只改变运行态 HUD 视觉,不改变计时、暂停、失败同步或关卡推进规则。 - 拼图运行态顶部关卡信息采用游戏化铭牌样式:橘棕横向关卡名牌承载 `第 N 关` 和关卡名,左侧固定使用 `media/logo.png` 卡通形象;倒计时作为下挂米白小牌独立显示,紧贴铭牌但不遮挡棋盘。该样式只改变运行态 HUD 视觉,不改变计时、暂停、失败同步或关卡推进规则。
- 拼图运行态进行中关卡的 `elapsedMs` 仍是结算字段,设置面板的“当前用时”必须按 `startedAtMs`、暂停累计和冻结累计实时派生;不要直接把进行中的 `currentLevel.elapsedMs` 当作展示值。 - 拼图运行态进行中关卡的 `elapsedMs` 仍是结算字段,设置面板的“当前用时”必须按 `startedAtMs`、暂停累计和冻结累计实时派生;不要直接把进行中的 `currentLevel.elapsedMs` 当作展示值。
- 推荐页嵌入拼图运行态时,通关结算弹层必须挂到页面级 fixed 浮层,不能留在推荐卡片视觉区内的 absolute 覆盖层;推荐页滑动卡片和运行态视口都使用 `overflow: hidden`,半屏内容区会裁剪排行榜、下一关按钮和相似作品卡。 - 推荐页嵌入拼图运行态时,通关结算弹层必须挂到页面级 fixed 浮层,不能留在推荐卡片视觉区内的 absolute 覆盖层;推荐页滑动卡片和运行态视口都使用 `overflow: hidden`,半屏内容区会裁剪排行榜、下一关按钮和相似作品卡。
- 推荐页嵌入拼图运行态时,“下一关”应优先切到相似作品;如果当前推荐候选为空,才回退到同作品下一关,避免匿名推荐流在多关卡作品上持续停留在同一作品内。下一关请求 pending 期间必须保留当前 `PuzzleRuntimeShell` 和棋盘,不得把推荐卡整体切回 `加载中...` 占位态;局部同步状态由拼图运行态自己的 busy 表现承接。后端返回的新关卡属于其它作品时,前端必须同步 `selectedPuzzleDetail`、推荐页 `puzzleGalleryEntries` 缓存和 `activeRecommendEntryKey`,让底部作品信息、分享 / 点赞 / 改造和下一次“下一个”基准都指向新作品。 - 推荐页嵌入拼图运行态时,“下一关”应优先切到相似作品;如果当前推荐候选为空,才回退到同作品下一关,避免匿名推荐流在多关卡作品上持续停留在同一作品内。下一关请求 pending 期间必须保留当前 `PuzzleRuntimeShell` 和棋盘,不得把推荐卡整体切回 `加载中...` 占位态;局部同步状态由拼图运行态自己的 busy 表现承接。后端返回的新关卡属于其它作品时,前端必须同步 `selectedPuzzleDetail`、推荐页 `puzzleGalleryEntries` 缓存和 `activeRecommendEntryKey`,让底部作品信息、分享 / 点赞 / 改造和下一次“下一个”基准都指向新作品;但这仍属于同一个 runtime run 内部推进,不能触发推荐 rail 切卡动画、纵向位移或启动封面重置,已挂载且 ready 的运行态画面应保持稳定,只静默更新作品信息和操作基准
- 推荐页里的拼图作品如果从运行态进入“改造”结果页,返回平台后要清掉推荐嵌入态的 `activeRecommendEntryKey` / `activeRecommendRuntimeKind` / `isStartingRecommendEntry`,再重新按推荐页自动启动逻辑进入作品,不能复用已经被清空的旧 `puzzleRun` - 推荐页里的拼图作品如果从运行态进入“改造”结果页,返回平台后要清掉推荐嵌入态的 `activeRecommendEntryKey` / `activeRecommendRuntimeKind` / `isStartingRecommendEntry`,再重新按推荐页自动启动逻辑进入作品,不能复用已经被清空的旧 `puzzleRun`
- 拼图运行态允许前端低延迟交互表现,但通关、排行榜、奖励和作品状态仍以后端确认为准。 - 拼图运行态允许前端低延迟交互表现,但通关、排行榜、奖励和作品状态仍以后端确认为准。
@@ -166,7 +166,7 @@ RPG / 拼图等运行态存档仍以 `/api/profile/save-archives` 的后端列
删除等破坏性动作当前未接入 jump-hop 删除 API如果后续要在作品架提供删除入口必须先补齐后端/SpacetimeDB/前端整条删除链路,再开放按钮。 删除等破坏性动作当前未接入 jump-hop 删除 API如果后续要在作品架提供删除入口必须先补齐后端/SpacetimeDB/前端整条删除链路,再开放按钮。
推荐页匿名游玩不再限定为跳一跳。移动端一级 `推荐` Tab 是内嵌运行态刷卡流,会自动选择推荐作品并启动对应玩法;桌面端首页不启动这套移动推荐运行态,而是渲染桌面发现壳,展示 `今日游戏``推荐``作品分类` 等桌面内容。断点事实统一走 `platformEntryResponsive.ts``usePlatformDesktopLayout()`,平台壳和首页视图必须共用同一个判断,避免桌面发现页与移动推荐页同时挂载、重复触发请求或启动运行态。推荐页嵌入运行态启动时按真实身份分流:已登录用户或本地已有 access token 时继续使用账号 Bearer但请求选项必须是 local auth impact避免单卡 401 清空整站登录态;只有确认为匿名访客时才申请短期 Runtime Guest Token并只把它作为局部请求头传给运行态客户端不写入全局登录态、不触发 refresh也不把匿名流量伪装成普通用户。当前覆盖矩阵为跳一跳、视觉小说、抓大鹅 Match3D、方洞挑战、拼图、敲木鱼、大鱼吃小鱼、汪汪声浪。每个模板的启动请求、推荐页内后续运行态动作以及需要上报的 play/finish/leaderboard/next-level 类请求都必须继续按该身份分流公开读取入口仍可匿名读取创作、个人作品、删除、发布、Remix 等账号/所有权动作仍保持普通用户鉴权。 推荐页匿名游玩不再限定为跳一跳。移动端一级 `推荐` Tab 是内嵌运行态刷卡流,会自动选择推荐作品并启动对应玩法;桌面端首页不启动这套移动推荐运行态,而是渲染桌面发现壳,展示 `今日游戏``推荐``作品分类` 等桌面内容。断点事实统一走 `platformEntryResponsive.ts``usePlatformDesktopLayout()`,平台壳和首页视图必须共用同一个判断,避免桌面发现页与移动推荐页同时挂载、重复触发请求或启动运行态。移动端推荐页启动或切换作品时先展示当前作品封面,嵌入 runtime 在封面下层加载;只有对应运行态 run / profile 已准备且 lazy runtime 组件完成挂载后,封面才渐隐,不在中途展示“加载中”文案。拼图下一关在同一个 run 内推进到相似作品时不视为推荐作品切换,不能重新显示启动封面。推荐页嵌入运行态启动时按真实身份分流:已登录用户或本地已有 access token 时继续使用账号 Bearer但请求选项必须是 local auth impact避免单卡 401 清空整站登录态;只有确认为匿名访客时才申请短期 Runtime Guest Token并只把它作为局部请求头传给运行态客户端不写入全局登录态、不触发 refresh也不把匿名流量伪装成普通用户。当前覆盖矩阵为跳一跳、视觉小说、抓大鹅 Match3D、方洞挑战、拼图、敲木鱼、大鱼吃小鱼、汪汪声浪。每个模板的启动请求、推荐页内后续运行态动作以及需要上报的 play/finish/leaderboard/next-level 类请求都必须继续按该身份分流公开读取入口仍可匿名读取创作、个人作品、删除、发布、Remix 等账号/所有权动作仍保持普通用户鉴权。
## 敲木鱼 ## 敲木鱼

View File

@@ -1,6 +1,6 @@
# 当前产品与工程约束 # 当前产品与工程约束
更新时间:`2026-05-15` 更新时间:`2026-06-05`
## 项目定位 ## 项目定位
@@ -46,8 +46,9 @@ Genarrative / 陶泥儿是一个 AI 原生互动内容与小游戏平台。当
3. 登录弹窗继续复用现有独立 modal 和页签结构,不在页面中新增功能说明类文案,也不把邀请码输入放回登录面板。 3. 登录弹窗继续复用现有独立 modal 和页签结构,不在页面中新增功能说明类文案,也不把邀请码输入放回登录面板。
4. 微信小程序 `web-view` 外壳默认不预登录,首次进入直接打开 H5并保持与 Web 端一致的未登录状态;只有 H5 触发 `openLoginModal` / `requireAuth` 等受保护入口时,才跳转小程序原生授权态。 4. 微信小程序 `web-view` 外壳默认不预登录,首次进入直接打开 H5并保持与 Web 端一致的未登录状态;只有 H5 触发 `openLoginModal` / `requireAuth` 等受保护入口时,才跳转小程序原生授权态。
5. 小程序内需要登录时不展示 H5 登录弹窗,也不走手输手机号 / 短信验证码流程;统一通过原生 `button open-type="getPhoneNumber"` 获取微信手机号授权,再调用 `/api/auth/wechat/miniprogram-login``/api/auth/wechat/bind-phone` 换取系统登录态。 5. 小程序内需要登录时不展示 H5 登录弹窗,也不走手输手机号 / 短信验证码流程;统一通过原生 `button open-type="getPhoneNumber"` 获取微信手机号授权,再调用 `/api/auth/wechat/miniprogram-login``/api/auth/wechat/bind-phone` 换取系统登录态。
6. 小程序 `web-view` 页必须启用好友分享与朋友圈分享,分享目标固定回到 `pages/web-view/index`,不把 H5 当前 URL 作为不受控启动参数传回小程序页 6. 小程序外壳注入到 H5 URL 的 `clientType``clientRuntime``miniProgramEnv` 是宿主上下文H5 内部 `pushState` / 阶段导航必须跨页面保留,避免登录和充值误判为普通浏览器;首点时微信 JS bridge 可能尚未就绪,前端还需用 `MicroMessenger + miniProgram` User-Agent 作为小程序识别兜底
7. 小程序 `web-view` 外壳运行时通过 `wx.getAccountInfoSync().miniProgram.envVersion` 自动识别版本:线上版 `release` 使用 `www.genarrative.world`,体验版 `trial` 与开发版 `develop` 使用 `dev.genarrative.world`;传给后端的 `x-mini-program-env` 分别为 `release``trial``dev` 7. 小程序 `web-view` 页必须启用好友分享与朋友圈分享,分享目标固定回到 `pages/web-view/index`,不把 H5 当前 URL 作为不受控启动参数传回小程序页
8. 小程序 `web-view` 外壳运行时通过 `wx.getAccountInfoSync().miniProgram.envVersion` 自动识别版本:线上版 `release` 使用 `www.genarrative.world`,体验版 `trial` 与开发版 `develop` 使用 `dev.genarrative.world`;传给后端的 `x-mini-program-env` 分别为 `release``trial``dev`
## 账户与充值 ## 账户与充值

View File

@@ -88,7 +88,7 @@ pipeline {
chmod +x scripts/jenkins-prepare-cargo-env.sh chmod +x scripts/jenkins-prepare-cargo-env.sh
source scripts/jenkins-prepare-cargo-env.sh source scripts/jenkins-prepare-cargo-env.sh
if ! command -v clang >/dev/null 2>&1 || ! command -v lld >/dev/null 2>&1; then if ! command -v clang >/dev/null 2>&1 || ! command -v lld >/dev/null 2>&1; then
echo "[api-build] 缺少 clang/lld。请先运行 Genarrative-Server-Provision 安装 Linux 构建依赖。" >&2 echo "[api-build] 缺少 clang/lld。请在 genarrative-build 节点预先安装 Linux 构建依赖。" >&2
exit 1 exit 1
fi fi
if ! command -v sccache >/dev/null 2>&1; then if ! command -v sccache >/dev/null 2>&1; then

View File

@@ -24,7 +24,7 @@ pipeline {
string(name: 'RELEASE_ROOT', defaultValue: '/opt/genarrative/releases', description: '生产 release 根目录') string(name: 'RELEASE_ROOT', defaultValue: '/opt/genarrative/releases', description: '生产 release 根目录')
string(name: 'CURRENT_LINK', defaultValue: '/opt/genarrative/current', description: '当前版本软链接') string(name: 'CURRENT_LINK', defaultValue: '/opt/genarrative/current', description: '当前版本软链接')
string(name: 'SERVICE_NAME', defaultValue: 'genarrative-api.service', description: 'systemd 服务名') string(name: 'SERVICE_NAME', defaultValue: 'genarrative-api.service', description: 'systemd 服务名')
string(name: 'HEALTH_URL', defaultValue: 'http://127.0.0.1:8082/healthz', description: '本机健康检查地址') string(name: 'HEALTH_URL', defaultValue: 'http://127.0.0.1:8082/readyz', description: '本机 readiness 检查地址')
string(name: 'API_ENV_FILE', defaultValue: '/etc/genarrative/api-server.env', description: 'api-server 环境文件') string(name: 'API_ENV_FILE', defaultValue: '/etc/genarrative/api-server.env', description: 'api-server 环境文件')
string(name: 'DATABASE', defaultValue: 'genarrative-prod', description: 'api-server 连接的 SpacetimeDB database') string(name: 'DATABASE', defaultValue: 'genarrative-prod', description: 'api-server 连接的 SpacetimeDB database')
string(name: 'SPACETIME_SERVER_URL', defaultValue: 'http://127.0.0.1:3101', description: 'api-server 连接的 SpacetimeDB server URL') string(name: 'SPACETIME_SERVER_URL', defaultValue: 'http://127.0.0.1:3101', description: 'api-server 连接的 SpacetimeDB server URL')

View File

@@ -7,25 +7,21 @@ pipeline {
buildDiscarder(logRotator(numToKeepStr: '20', artifactNumToKeepStr: '20')) buildDiscarder(logRotator(numToKeepStr: '20', artifactNumToKeepStr: '20'))
} }
environment {
GIT_REMOTE_URL = 'http://127.0.0.1:3000/GenarrativeAI/Genarrative.git'
GIT_REMOTE_FALLBACK_URL = 'https://git.genarrative.world/GenarrativeAI/Genarrative.git'
}
parameters { parameters {
choice(name: 'DEPLOY_TARGET', choices: ['development', 'release'], description: '逻辑部署目标development 使用当前 Linux 开发/构建/开发部署 agent') choice(name: 'DEPLOY_TARGET', choices: ['development', 'release'], description: '逻辑部署目标development 使用 dev 服务器部署 agentrelease 使用正式服务器部署 agent')
booleanParam(name: 'CONFIRM_RELEASE_DEPLOY_AGENT', defaultValue: false, description: '确认 release 目标已有独立 release 部署 agent') booleanParam(name: 'CONFIRM_RELEASE_DEPLOY_AGENT', defaultValue: false, description: '确认 release 目标已有独立 release 部署 agent')
string(name: 'NOTIFICATION_EMAILS', defaultValue: '', description: '本次运行追加通知邮箱;会与 Jenkins Secret Text 凭据 genarrative-notification-emails 合并发送') string(name: 'NOTIFICATION_EMAILS', defaultValue: '', description: '本次运行追加通知邮箱;会与 Jenkins Secret Text 凭据 genarrative-notification-emails 合并发送')
booleanParam(name: 'CONFIRM_PROVISION', defaultValue: false, description: '确认执行服务器初始化;未勾选时只允许 dry-run') booleanParam(name: 'CONFIRM_PROVISION', defaultValue: false, description: '确认执行服务器初始化;未勾选时只允许 dry-run')
booleanParam(name: 'DRY_RUN', defaultValue: true, description: '只打印将执行的服务器初始化命令,不写入系统配置') booleanParam(name: 'DRY_RUN', defaultValue: true, description: '只打印将执行的服务器初始化命令,不写入系统配置')
string(name: 'SOURCE_GIT_REMOTE_URL', defaultValue: '', description: '部署脚本 Git 来源;必须是目标 agent 可访问的内网/本机 Gitea 地址,不配置公网备用')
string(name: 'SOURCE_BRANCH', defaultValue: 'master', description: '部署脚本来源分支') string(name: 'SOURCE_BRANCH', defaultValue: 'master', description: '部署脚本来源分支')
string(name: 'COMMIT_HASH', defaultValue: '', description: '部署脚本来源 commit') string(name: 'COMMIT_HASH', defaultValue: '', description: '部署脚本来源 commit')
string(name: 'SERVER_NAME', defaultValue: 'genarrative.example.com', description: '证书主域名;也作为 Nginx server_name 的第一个域名') string(name: 'SERVER_NAME', defaultValue: 'genarrative.example.com', description: '证书主域名;也作为 Nginx server_name 的第一个域名')
string(name: 'SERVER_ALIASES', defaultValue: '', description: '可选,额外 Nginx server_name多个用空格或逗号分隔例如 www.genarrative.world') string(name: 'SERVER_ALIASES', defaultValue: '', description: '可选,额外 Nginx server_name多个用空格或逗号分隔例如 www.genarrative.world')
string(name: 'PROVISION_DOWNLOADS_DIR', defaultValue: 'provision-tool-downloads', description: 'Linux 预下载阶段暂存 SpacetimeDB/otelcol 安装包的工作区相对目录') string(name: 'PROVISION_DOWNLOADS_DIR', defaultValue: 'provision-tool-downloads', description: '目标服务器工作区内暂存 SpacetimeDB/otelcol 安装包的相对目录')
string(name: 'PROVISION_TOOLS_DIR', defaultValue: 'provision-tools', description: '目标机工作区内由已下载安装包生成的工具包目录') string(name: 'PROVISION_TOOLS_DIR', defaultValue: 'provision-tools', description: '目标机工作区内由已下载安装包生成的工具包目录')
string(name: 'PROVISION_DOWNLOAD_PROXY', defaultValue: '', description: '可选,Linux 预下载阶段下载 SpacetimeDB 和 otelcol-contrib 时使用的代理地址,例如 http://127.0.0.1:7890留空不设置代理') string(name: 'PROVISION_DOWNLOAD_PROXY', defaultValue: '', description: '可选,目标服务器下载 SpacetimeDB 和 otelcol-contrib 时使用的代理地址,例如 http://127.0.0.1:7890留空不设置代理')
string(name: 'SPACETIME_DOWNLOAD_ROOT', defaultValue: 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download', description: 'Linux 预下载阶段使用的 SpacetimeDB Linux release tarball 根地址') string(name: 'SPACETIME_DOWNLOAD_ROOT', defaultValue: 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download', description: '目标服务器使用的 SpacetimeDB Linux release tarball 根地址')
string(name: 'SPACETIME_TARGET_HOST', defaultValue: 'x86_64-unknown-linux-gnu', description: 'SpacetimeDB 预编译包 host tripledevelopment/release Linux amd64 使用默认值') string(name: 'SPACETIME_TARGET_HOST', defaultValue: 'x86_64-unknown-linux-gnu', description: 'SpacetimeDB 预编译包 host tripledevelopment/release Linux amd64 使用默认值')
string(name: 'SPACETIME_ROOT', defaultValue: '/stdb', description: 'SpacetimeDB root-dir') string(name: 'SPACETIME_ROOT', defaultValue: '/stdb', description: 'SpacetimeDB root-dir')
string(name: 'RELEASE_ROOT', defaultValue: '/opt/genarrative/releases', description: 'release 根目录') string(name: 'RELEASE_ROOT', defaultValue: '/opt/genarrative/releases', description: 'release 根目录')
@@ -40,205 +36,160 @@ pipeline {
} }
stages { stages {
stage('Prepare') { stage('Provision Target') {
agent { agent {
label 'linux && genarrative-build' label "${params.DEPLOY_TARGET == 'development' ? 'linux && genarrative-dev-deploy' : 'linux && genarrative-release-deploy'}"
} }
steps { stages {
script { stage('Prepare') {
if (params.DEPLOY_TARGET == 'release' && !params.CONFIRM_RELEASE_DEPLOY_AGENT) { steps {
error('release provision 需要先配置独立 release 部署 agent并勾选 CONFIRM_RELEASE_DEPLOY_AGENT。') script {
} if (params.DEPLOY_TARGET == 'release' && !params.CONFIRM_RELEASE_DEPLOY_AGENT) {
if (!params.DRY_RUN && !params.CONFIRM_PROVISION) { error('release provision 需要先配置独立 release 部署 agent并勾选 CONFIRM_RELEASE_DEPLOY_AGENT。')
error('执行服务器初始化前必须勾选 CONFIRM_PROVISION否则请保持 DRY_RUN=true。') }
} if (!params.DRY_RUN && !params.CONFIRM_PROVISION) {
if (!params.SERVER_NAME?.trim()) { error('执行服务器初始化前必须勾选 CONFIRM_PROVISION否则请保持 DRY_RUN=true。')
error('SERVER_NAME 不能为空。') }
} if (!params.SERVER_NAME?.trim()) {
if (!(params.SERVER_NAME.trim() ==~ /^[A-Za-z0-9][A-Za-z0-9.-]*$/)) { error('SERVER_NAME 不能为空。')
error("SERVER_NAME 只能填写单个域名或 IP不能包含空格、路径或协议: ${params.SERVER_NAME}") }
} def sourceGitRemoteUrl = params.SOURCE_GIT_REMOTE_URL?.trim()
def serverAliases = params.SERVER_ALIASES?.trim() if (!sourceGitRemoteUrl) {
if (serverAliases) { error('SOURCE_GIT_REMOTE_URL 不能为空。')
serverAliases.split(/[,\s]+/).findAll { it }.each { aliasName -> }
if (!(aliasName ==~ /^[A-Za-z0-9][A-Za-z0-9.-]*$/)) { def isLocalGitPath = sourceGitRemoteUrl ==~ /^\/[0-9A-Za-z._\/-]+$/
error("SERVER_ALIASES 只能填写域名或 IP多个用空格或逗号分隔: ${aliasName}") def isLocalGitFileUrl = sourceGitRemoteUrl ==~ /^file:\/\/\/\S+$/
def isPrivateHttpGitUrl = sourceGitRemoteUrl ==~ /^https?:\/\/(localhost|127(?:\.[0-9]{1,3}){3}|10(?:\.[0-9]{1,3}){3}|192\.168(?:\.[0-9]{1,3}){2}|172\.(?:1[6-9]|2[0-9]|3[0-1])(?:\.[0-9]{1,3}){2}|[A-Za-z0-9-]+|[A-Za-z0-9.-]+\.(?:local|lan|internal))(?::[0-9]+)?\/\S+$/
if (!isLocalGitPath && !isLocalGitFileUrl && !isPrivateHttpGitUrl) {
error('Genarrative-Server-Provision 不允许使用公网 Git 仓库SOURCE_GIT_REMOTE_URL 只能是目标 agent 可访问的本机路径、file:/// 地址、localhost/127.0.0.1、RFC1918 内网 HTTP 地址、单标签内网主机名或 .local/.lan/.internal 地址。')
}
env.EFFECTIVE_GIT_REMOTE_URL = sourceGitRemoteUrl
if (!(params.SERVER_NAME.trim() ==~ /^[A-Za-z0-9][A-Za-z0-9.-]*$/)) {
error("SERVER_NAME 只能填写单个域名或 IP不能包含空格、路径或协议: ${params.SERVER_NAME}")
}
def serverAliases = params.SERVER_ALIASES?.trim()
if (serverAliases) {
serverAliases.split(/[,\s]+/).findAll { it }.each { aliasName ->
if (!(aliasName ==~ /^[A-Za-z0-9][A-Za-z0-9.-]*$/)) {
error("SERVER_ALIASES 只能填写域名或 IP多个用空格或逗号分隔: ${aliasName}")
}
}
}
if (!params.PROVISION_TOOLS_DIR?.trim()) {
error('PROVISION_TOOLS_DIR 不能为空。')
}
if (!(params.PROVISION_TOOLS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_TOOLS_DIR.startsWith('/') || params.PROVISION_TOOLS_DIR.contains('..') || params.PROVISION_TOOLS_DIR.trim() == '.') {
error("PROVISION_TOOLS_DIR 只能是工作区内的相对目录,不能包含绝对路径或连续点号: ${params.PROVISION_TOOLS_DIR}")
}
if (!params.PROVISION_DOWNLOADS_DIR?.trim()) {
error('PROVISION_DOWNLOADS_DIR 不能为空。')
}
if (!(params.PROVISION_DOWNLOADS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_DOWNLOADS_DIR.startsWith('/') || params.PROVISION_DOWNLOADS_DIR.contains('..') || params.PROVISION_DOWNLOADS_DIR.trim() == '.') {
error("PROVISION_DOWNLOADS_DIR 只能是工作区内的相对目录,不能包含绝对路径或连续点号: ${params.PROVISION_DOWNLOADS_DIR}")
}
def provisionToolsDir = params.PROVISION_TOOLS_DIR.trim()
def provisionDownloadsDir = params.PROVISION_DOWNLOADS_DIR.trim()
if (provisionToolsDir == provisionDownloadsDir || provisionDownloadsDir.startsWith("${provisionToolsDir}/")) {
error("PROVISION_DOWNLOADS_DIR 不能等于或位于 PROVISION_TOOLS_DIR 内,否则目标机生成工具包时会删除下载缓存: ${provisionDownloadsDir}")
}
def provisionDownloadProxy = params.PROVISION_DOWNLOAD_PROXY?.trim()
if (provisionDownloadProxy && !(provisionDownloadProxy ==~ /^https?:\/\/\S+$/)) {
error("PROVISION_DOWNLOAD_PROXY 只能填写 http:// 或 https:// 开头的代理地址,当前值: ${params.PROVISION_DOWNLOAD_PROXY}")
}
if (!(params.OTELCOL_VERSION?.trim() ==~ /^[0-9]+\.[0-9]+\.[0-9]+$/)) {
error("OTELCOL_VERSION 格式应为 x.y.z: ${params.OTELCOL_VERSION}")
}
if (!(params.SPACETIME_DOWNLOAD_ROOT?.trim() ==~ /^https?:\/\/\S+$/)) {
error('SPACETIME_DOWNLOAD_ROOT 不能为空。')
}
if (!(params.SPACETIME_TARGET_HOST?.trim() ==~ /^[0-9A-Za-z._-]+$/)) {
error("SPACETIME_TARGET_HOST 只能包含字母、数字、点号、下划线和短横线: ${params.SPACETIME_TARGET_HOST}")
}
def nginxMode = params.NGINX_CONFIG_MODE?.trim()
if (!(nginxMode in ['none', 'production-https', 'development-http'])) {
error("NGINX_CONFIG_MODE 只能是 none、production-https 或 development-http当前值: ${params.NGINX_CONFIG_MODE}")
}
if (params.DEPLOY_TARGET == 'release' && nginxMode == 'development-http') {
error('release 目标禁止安装 development-http Nginx 配置;无证书初始化请使用 NGINX_CONFIG_MODE=none。')
}
if (!params.DRY_RUN && nginxMode == 'production-https' && params.SERVER_NAME?.trim() == 'genarrative.example.com') {
error('真实初始化安装 Nginx 配置时必须把 SERVER_NAME 改成真实域名,不能使用 genarrative.example.com 占位值。证书未准备好时请先保持 NGINX_CONFIG_MODE=none。')
} }
} }
} }
if (!params.PROVISION_TOOLS_DIR?.trim()) {
error('PROVISION_TOOLS_DIR 不能为空。')
}
if (!(params.PROVISION_TOOLS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_TOOLS_DIR.startsWith('/') || params.PROVISION_TOOLS_DIR.contains('..') || params.PROVISION_TOOLS_DIR.trim() == '.') {
error("PROVISION_TOOLS_DIR 只能是工作区内的相对目录,不能包含绝对路径或连续点号: ${params.PROVISION_TOOLS_DIR}")
}
if (!params.PROVISION_DOWNLOADS_DIR?.trim()) {
error('PROVISION_DOWNLOADS_DIR 不能为空。')
}
if (!(params.PROVISION_DOWNLOADS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_DOWNLOADS_DIR.startsWith('/') || params.PROVISION_DOWNLOADS_DIR.contains('..') || params.PROVISION_DOWNLOADS_DIR.trim() == '.') {
error("PROVISION_DOWNLOADS_DIR 只能是工作区内的相对目录,不能包含绝对路径或连续点号: ${params.PROVISION_DOWNLOADS_DIR}")
}
def provisionToolsDir = params.PROVISION_TOOLS_DIR.trim()
def provisionDownloadsDir = params.PROVISION_DOWNLOADS_DIR.trim()
if (provisionToolsDir == provisionDownloadsDir || provisionDownloadsDir.startsWith("${provisionToolsDir}/")) {
error("PROVISION_DOWNLOADS_DIR 不能等于或位于 PROVISION_TOOLS_DIR 内,否则目标机生成工具包时会删除下载缓存: ${provisionDownloadsDir}")
}
def provisionDownloadProxy = params.PROVISION_DOWNLOAD_PROXY?.trim()
if (provisionDownloadProxy && !(provisionDownloadProxy ==~ /^https?:\/\/\S+$/)) {
error("PROVISION_DOWNLOAD_PROXY 只能填写 http:// 或 https:// 开头的代理地址,当前值: ${params.PROVISION_DOWNLOAD_PROXY}")
}
if (!(params.OTELCOL_VERSION?.trim() ==~ /^[0-9]+\.[0-9]+\.[0-9]+$/)) {
error("OTELCOL_VERSION 格式应为 x.y.z: ${params.OTELCOL_VERSION}")
}
if (!(params.SPACETIME_DOWNLOAD_ROOT?.trim() ==~ /^https?:\/\/\S+$/)) {
error('SPACETIME_DOWNLOAD_ROOT 不能为空。')
}
if (!(params.SPACETIME_TARGET_HOST?.trim() ==~ /^[0-9A-Za-z._-]+$/)) {
error("SPACETIME_TARGET_HOST 只能包含字母、数字、点号、下划线和短横线: ${params.SPACETIME_TARGET_HOST}")
}
def nginxMode = params.NGINX_CONFIG_MODE?.trim()
if (!(nginxMode in ['none', 'production-https', 'development-http'])) {
error("NGINX_CONFIG_MODE 只能是 none、production-https 或 development-http当前值: ${params.NGINX_CONFIG_MODE}")
}
if (params.DEPLOY_TARGET == 'release' && nginxMode == 'development-http') {
error('release 目标禁止安装 development-http Nginx 配置;无证书初始化请使用 NGINX_CONFIG_MODE=none。')
}
if (!params.DRY_RUN && nginxMode == 'production-https' && params.SERVER_NAME?.trim() == 'genarrative.example.com') {
error('真实初始化安装 Nginx 配置时必须把 SERVER_NAME 改成真实域名,不能使用 genarrative.example.com 占位值。证书未准备好时请先保持 NGINX_CONFIG_MODE=none。')
}
} }
}
}
stage('Prepare Provision Tools') { stage('Checkout Provision Files') {
agent { steps {
label 'linux && genarrative-build' script {
} checkout([
steps { $class: 'GitSCM',
script { branches: [[name: "*/${params.SOURCE_BRANCH}"]],
def checkoutFromRemote = { String remoteUrl -> doGenerateSubmoduleConfigurations: false,
checkout([ extensions: [
$class: 'GitSCM', [$class: 'CleanBeforeCheckout'],
branches: [[name: "*/${params.SOURCE_BRANCH}"]], [$class: 'CloneOption', shallow: true, depth: 1, noTags: true, timeout: 30, honorRefspec: true],
doGenerateSubmoduleConfigurations: false, ],
extensions: [ userRemoteConfigs: [[url: env.EFFECTIVE_GIT_REMOTE_URL, refspec: "+refs/heads/${params.SOURCE_BRANCH}:refs/remotes/origin/${params.SOURCE_BRANCH}"]],
[$class: 'CleanBeforeCheckout'], ])
[$class: 'CloneOption', shallow: true, depth: 1, noTags: true, timeout: 30, honorRefspec: true], }
], sh '''
userRemoteConfigs: [[url: remoteUrl, refspec: "+refs/heads/${params.SOURCE_BRANCH}:refs/remotes/origin/${params.SOURCE_BRANCH}"]], bash <<'BASH'
]) set -euo pipefail
} chmod +x scripts/jenkins-checkout-source.sh
try { SOURCE_BRANCH="${SOURCE_BRANCH:-master}" \
checkoutFromRemote(env.GIT_REMOTE_URL) COMMIT_HASH="${COMMIT_HASH:-${SOURCE_COMMIT:-}}" \
env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_URL GIT_REMOTE_URL="${EFFECTIVE_GIT_REMOTE_URL}" \
} catch (error) { SOURCE_COMMIT_FILE=".jenkins-source-commit" \
echo "Git 主地址拉取失败: ${env.GIT_REMOTE_URL},改用备用地址: ${env.GIT_REMOTE_FALLBACK_URL}" scripts/jenkins-checkout-source.sh
checkoutFromRemote(env.GIT_REMOTE_FALLBACK_URL)
env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_FALLBACK_URL
}
}
sh '''
bash <<'BASH'
set -euo pipefail
chmod +x scripts/jenkins-checkout-source.sh
SOURCE_BRANCH="${SOURCE_BRANCH:-master}" \
COMMIT_HASH="${COMMIT_HASH:-}" \
GIT_REMOTE_URL="${EFFECTIVE_GIT_REMOTE_URL:-${GIT_REMOTE_URL}}" \
GIT_REMOTE_FALLBACK_URL="${GIT_REMOTE_FALLBACK_URL:-}" \
SOURCE_COMMIT_FILE=".jenkins-source-commit" \
scripts/jenkins-checkout-source.sh
BASH BASH
''' '''
sh ''' script {
bash -lc ' env.SOURCE_COMMIT = readFile('.jenkins-source-commit').trim()
set -euo pipefail echo "Provision 源码 commit=${env.SOURCE_COMMIT}"
chmod +x scripts/prepare-server-provision-tools.sh }
PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \ }
PROVISION_DOWNLOADS_DIR="${PROVISION_DOWNLOADS_DIR:-provision-tool-downloads}" \
OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}" \
PREPARE_OTELCOL="${ENABLE_OTELCOL:-true}" \
PROVISION_DOWNLOAD_PROXY="${PROVISION_DOWNLOAD_PROXY:-}" \
SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}" \
SPACETIME_TARGET_HOST="${SPACETIME_TARGET_HOST:-x86_64-unknown-linux-gnu}" \
scripts/prepare-server-provision-tools.sh
'
'''
script {
env.SOURCE_COMMIT = readFile('.jenkins-source-commit').trim()
echo "Provision 工具包源码 commit=${env.SOURCE_COMMIT}"
} }
stash name: 'server-provision-tools', includes: "${params.PROVISION_TOOLS_DIR}/**", useDefaultExcludes: false
}
}
stage('Checkout Provision Files') { stage('Prepare Provision Tools') {
agent { steps {
label "${params.DEPLOY_TARGET == 'development' ? 'linux && genarrative-build' : 'linux && genarrative-release-deploy'}" sh '''
} bash -lc '
steps { set -euo pipefail
script { chmod +x scripts/prepare-server-provision-tools.sh
def checkoutFromRemote = { String remoteUrl -> PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \
checkout([ PROVISION_DOWNLOADS_DIR="${PROVISION_DOWNLOADS_DIR:-provision-tool-downloads}" \
$class: 'GitSCM', OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}" \
branches: [[name: "*/${params.SOURCE_BRANCH}"]], PREPARE_OTELCOL="${ENABLE_OTELCOL:-true}" \
doGenerateSubmoduleConfigurations: false, PROVISION_DOWNLOAD_PROXY="${PROVISION_DOWNLOAD_PROXY:-}" \
extensions: [ SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}" \
[$class: 'CleanBeforeCheckout'], SPACETIME_TARGET_HOST="${SPACETIME_TARGET_HOST:-x86_64-unknown-linux-gnu}" \
[$class: 'CloneOption', shallow: true, depth: 1, noTags: true, timeout: 30, honorRefspec: true], scripts/prepare-server-provision-tools.sh
], '
userRemoteConfigs: [[url: remoteUrl, refspec: "+refs/heads/${params.SOURCE_BRANCH}:refs/remotes/origin/${params.SOURCE_BRANCH}"]], '''
])
}
try {
checkoutFromRemote(env.GIT_REMOTE_URL)
env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_URL
} catch (error) {
echo "Git 主地址拉取失败: ${env.GIT_REMOTE_URL},改用备用地址: ${env.GIT_REMOTE_FALLBACK_URL}"
checkoutFromRemote(env.GIT_REMOTE_FALLBACK_URL)
env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_FALLBACK_URL
} }
} }
sh '''
bash <<'BASH'
set -euo pipefail
chmod +x scripts/jenkins-checkout-source.sh
SOURCE_BRANCH="${SOURCE_BRANCH:-master}" \
COMMIT_HASH="${COMMIT_HASH:-${SOURCE_COMMIT:-}}" \
GIT_REMOTE_URL="${EFFECTIVE_GIT_REMOTE_URL:-${GIT_REMOTE_URL}}" \
GIT_REMOTE_FALLBACK_URL="${GIT_REMOTE_FALLBACK_URL:-}" \
SOURCE_COMMIT_FILE=".jenkins-source-commit" \
scripts/jenkins-checkout-source.sh
BASH
'''
script {
env.SOURCE_COMMIT = readFile('.jenkins-source-commit').trim()
echo "Provision 源码 commit=${env.SOURCE_COMMIT}"
}
}
}
stage('Provision Server') { stage('Provision Server') {
agent { steps {
label "${params.DEPLOY_TARGET == 'development' ? 'linux && genarrative-build' : 'linux && genarrative-release-deploy'}" sh '''
} bash <<'BASH'
steps { set -euo pipefail
unstash 'server-provision-tools' if [[ "${ENABLE_OTELCOL:-true}" == "true" ]]; then
sh ''' chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/otelcol-contrib"
bash <<'BASH' fi
set -euo pipefail chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/spacetime" \
if [[ "${ENABLE_OTELCOL:-true}" == "true" ]]; then "${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/bin/current/spacetimedb-cli" \
chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/otelcol-contrib" "${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/bin/current/spacetimedb-standalone"
fi chmod +x scripts/jenkins-server-provision.sh
chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/spacetime" \ PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \
"${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/bin/current/spacetimedb-cli" \ SPACETIME_BIN_SOURCE="${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/spacetime" \
"${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/bin/current/spacetimedb-standalone" OTELCOL_BIN_SOURCE="${PROVISION_TOOLS_DIR:-provision-tools}/otelcol-contrib" \
chmod +x scripts/jenkins-server-provision.sh scripts/jenkins-server-provision.sh
PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \
SPACETIME_BIN_SOURCE="${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/spacetime" \
OTELCOL_BIN_SOURCE="${PROVISION_TOOLS_DIR:-provision-tools}/otelcol-contrib" \
scripts/jenkins-server-provision.sh
BASH BASH
''' '''
}
}
} }
} }
} }
@@ -253,9 +204,7 @@ BASH
string(name: 'SOURCE_RESULT', value: currentBuild.currentResult ?: 'UNKNOWN'), string(name: 'SOURCE_RESULT', value: currentBuild.currentResult ?: 'UNKNOWN'),
string(name: 'SOURCE_BRANCH', value: params.SOURCE_BRANCH ?: ''), string(name: 'SOURCE_BRANCH', value: params.SOURCE_BRANCH ?: ''),
string(name: 'SOURCE_COMMIT', value: env.SOURCE_COMMIT ?: (params.COMMIT_HASH ?: '')), string(name: 'SOURCE_COMMIT', value: env.SOURCE_COMMIT ?: (params.COMMIT_HASH ?: '')),
string(name: 'BUILD_VERSION', value: env.EFFECTIVE_BUILD_VERSION ?: (params.BUILD_VERSION ?: '')),
string(name: 'DEPLOY_TARGET', value: params.DEPLOY_TARGET ?: ''), string(name: 'DEPLOY_TARGET', value: params.DEPLOY_TARGET ?: ''),
string(name: 'DATABASE', value: params.DATABASE ?: ''),
string(name: 'SUMMARY', value: '服务器初始化流水线结束'), string(name: 'SUMMARY', value: '服务器初始化流水线结束'),
] ]
def notificationRecipients = params.NOTIFICATION_EMAILS?.trim() def notificationRecipients = params.NOTIFICATION_EMAILS?.trim()

View File

@@ -5,10 +5,10 @@ set -euo pipefail
usage() { usage() {
cat <<'EOF' cat <<'EOF'
用法: 用法:
./scripts/deploy/production-api-deploy.sh --source-dir build/<version> [--version <version>] [--release-root /opt/genarrative/releases] [--current-link /opt/genarrative/current] [--service genarrative-api.service] [--health-url http://127.0.0.1:8082/healthz] [--api-env-file /etc/genarrative/api-server.env] [--database genarrative-prod] [--spacetime-server-url http://127.0.0.1:3101] ./scripts/deploy/production-api-deploy.sh --source-dir build/<version> [--version <version>] [--release-root /opt/genarrative/releases] [--current-link /opt/genarrative/current] [--service genarrative-api.service] [--health-url http://127.0.0.1:8082/readyz] [--api-env-file /etc/genarrative/api-server.env] [--database genarrative-prod] [--spacetime-server-url http://127.0.0.1:3101]
说明: 说明:
进入维护模式,校验并发布 api-server 单文件,更新 current 链接,重启 systemd 服务并执行 healthz 检查。 进入维护模式,校验并发布 api-server 单文件,更新 current 链接,重启 systemd 服务并执行 readiness 检查。
若传入 --database会在重启前把 GENARRATIVE_SPACETIME_DATABASE 写入 api-server 环境文件,避免服务继续读取旧库。 若传入 --database会在重启前把 GENARRATIVE_SPACETIME_DATABASE 写入 api-server 环境文件,避免服务继续读取旧库。
失败时保留维护模式。 失败时保留维护模式。
EOF EOF
@@ -209,6 +209,7 @@ ensure_runtime_env_and_dirs() {
# 旧生产环境文件会被 server-provision 保留,不一定包含新增的运行态写入路径。 # 旧生产环境文件会被 server-provision 保留,不一定包含新增的运行态写入路径。
# 发布前只补缺省值,不覆盖线上已经定制过的目录或开关。 # 发布前只补缺省值,不覆盖线上已经定制过的目录或开关。
ensure_env_value "${api_env_file}" "GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS" "5000"
ensure_env_value "${api_env_file}" "GENARRATIVE_TRACKING_OUTBOX_ENABLED" "true" ensure_env_value "${api_env_file}" "GENARRATIVE_TRACKING_OUTBOX_ENABLED" "true"
ensure_env_value "${api_env_file}" "GENARRATIVE_TRACKING_OUTBOX_DIR" "/var/lib/genarrative/tracking-outbox" ensure_env_value "${api_env_file}" "GENARRATIVE_TRACKING_OUTBOX_DIR" "/var/lib/genarrative/tracking-outbox"
ensure_env_value "${api_env_file}" "GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE" "500" ensure_env_value "${api_env_file}" "GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE" "500"
@@ -228,7 +229,7 @@ VERSION=""
RELEASE_ROOT="/opt/genarrative/releases" RELEASE_ROOT="/opt/genarrative/releases"
CURRENT_LINK="/opt/genarrative/current" CURRENT_LINK="/opt/genarrative/current"
SERVICE_NAME="genarrative-api.service" SERVICE_NAME="genarrative-api.service"
HEALTH_URL="http://127.0.0.1:8082/healthz" HEALTH_URL="http://127.0.0.1:8082/readyz"
API_ENV_FILE="/etc/genarrative/api-server.env" API_ENV_FILE="/etc/genarrative/api-server.env"
DATABASE="" DATABASE=""
SPACETIME_SERVER_URL="" SPACETIME_SERVER_URL=""
@@ -362,7 +363,7 @@ ln -sfn "${RELEASE_DIR}" "${CURRENT_LINK}"
echo "[production-api-deploy] 重启服务: ${SERVICE_NAME}" echo "[production-api-deploy] 重启服务: ${SERVICE_NAME}"
systemctl restart "${SERVICE_NAME}" systemctl restart "${SERVICE_NAME}"
echo "[production-api-deploy] 等待 healthz: ${HEALTH_URL}" echo "[production-api-deploy] 等待 readiness: ${HEALTH_URL}"
for _ in {1..30}; do for _ in {1..30}; do
if curl -fsS "${HEALTH_URL}" >/dev/null; then if curl -fsS "${HEALTH_URL}" >/dev/null; then
"${SCRIPT_DIR}/maintenance-off.sh" "${SCRIPT_DIR}/maintenance-off.sh"
@@ -373,5 +374,5 @@ for _ in {1..30}; do
sleep 2 sleep 2
done done
echo "[production-api-deploy] healthz 检查超时: ${HEALTH_URL}" >&2 echo "[production-api-deploy] readiness 检查超时: ${HEALTH_URL}" >&2
exit 1 exit 1

View File

@@ -28,7 +28,7 @@ if [[ -z "${RUSTUP_HOME:-}" && -n "${ORIGINAL_HOME}" && -d "${ORIGINAL_HOME}/.ru
fi fi
# HOME 会在下面切到组件级缓存目录,因此这里先把真实用户的 Rust 工具链目录补进 PATH。 # HOME 会在下面切到组件级缓存目录,因此这里先把真实用户的 Rust 工具链目录补进 PATH。
# Server-Provision 通过 cargo install 安装的 sccache 通常会落在 /root/.cargo/bin。 # Jenkins 构建节点预装的 Rust 工具和 sccache 通常会落在 /root/.cargo/bin。
for tool_dir in "${ORIGINAL_HOME}/.cargo/bin" /root/.cargo/bin /usr/local/cargo/bin; do for tool_dir in "${ORIGINAL_HOME}/.cargo/bin" /root/.cargo/bin /usr/local/cargo/bin; do
if [[ -d "${tool_dir}" && ":${PATH}:" != *":${tool_dir}:"* ]]; then if [[ -d "${tool_dir}" && ":${PATH}:" != *":${tool_dir}:"* ]]; then
export PATH="${tool_dir}:${PATH}" export PATH="${tool_dir}:${PATH}"

View File

@@ -4,6 +4,10 @@ set -euo pipefail
PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}"
SPACETIME_BIN_SOURCE="${SPACETIME_BIN_SOURCE:-${PROVISION_TOOLS_DIR}/spacetime/spacetime}" SPACETIME_BIN_SOURCE="${SPACETIME_BIN_SOURCE:-${PROVISION_TOOLS_DIR}/spacetime/spacetime}"
OTELCOL_BIN_SOURCE="${OTELCOL_BIN_SOURCE:-${PROVISION_TOOLS_DIR}/otelcol-contrib}" OTELCOL_BIN_SOURCE="${OTELCOL_BIN_SOURCE:-${PROVISION_TOOLS_DIR}/otelcol-contrib}"
GENARRATIVE_OPENSSL_VERSION="${GENARRATIVE_OPENSSL_VERSION:-3.2.0}"
GENARRATIVE_OPENSSL_PREFIX="${GENARRATIVE_OPENSSL_PREFIX:-/opt/genarrative/openssl-3.2.0}"
GENARRATIVE_OPENSSL_SOURCE_URL="${GENARRATIVE_OPENSSL_SOURCE_URL:-https://github.com/openssl/openssl/releases/download/openssl-${GENARRATIVE_OPENSSL_VERSION}/openssl-${GENARRATIVE_OPENSSL_VERSION}.tar.gz}"
GENARRATIVE_OPENSSL_SOURCE_SHA256="${GENARRATIVE_OPENSSL_SOURCE_SHA256:-14c826f07c7e433706fb5c69fa9e25dab95684844b4c962a2cf1bf183eb4690e}"
require_non_root_relative_path() { require_non_root_relative_path() {
local label="$1" local label="$1"
@@ -27,6 +31,14 @@ require_path() {
fi fi
} }
require_cmd() {
local name="$1"
if ! command -v "${name}" >/dev/null 2>&1; then
echo "[server-provision] 缺少命令: ${name}" >&2
exit 1
fi
}
normalize_server_aliases() { normalize_server_aliases() {
printf "%s" "${SERVER_ALIASES:-}" | tr ',' ' ' | xargs printf "%s" "${SERVER_ALIASES:-}" | tr ',' ' ' | xargs
} }
@@ -56,6 +68,18 @@ run_cmd() {
fi fi
} }
require_root_for_real_provision() {
if [[ "${DRY_RUN}" == "true" ]]; then
return
fi
if [[ "$(id -u)" != "0" ]]; then
echo "[server-provision] 非 dry-run 会安装系统包、写入 systemd/Nginx 和创建系统用户,必须在 root agent 上执行。" >&2
echo "[server-provision] 当前用户: $(id -un) uid=$(id -u)。请确认 DEPLOY_TARGET=${DEPLOY_TARGET:-} 对应的目标服务器 agent 以 root 运行,或保持 DRY_RUN=true。" >&2
exit 1
fi
}
install_file() { install_file() {
local source="$1" local source="$1"
local target="$2" local target="$2"
@@ -66,21 +90,6 @@ install_file() {
fi fi
} }
install_build_dependencies() {
echo "[server-provision] 安装 Linux 构建依赖: clang, lld, pkg-config, OpenSSL headers"
if command -v apt-get >/dev/null 2>&1; then
run_cmd apt-get update
run_cmd apt-get install -y clang lld pkg-config libssl-dev ca-certificates
elif command -v dnf >/dev/null 2>&1; then
run_cmd dnf install -y clang lld pkgconf-pkg-config openssl-devel ca-certificates
elif command -v yum >/dev/null 2>&1; then
run_cmd yum install -y clang lld pkgconf-pkg-config openssl-devel ca-certificates
else
echo "[server-provision] 未找到 apt-get/dnf/yum无法自动安装 clang/lld。请手动安装后重跑构建。" >&2
exit 1
fi
}
install_nginx_brotli_modules() { install_nginx_brotli_modules() {
echo "[server-provision] 安装 Nginx Brotli 动态模块依赖" echo "[server-provision] 安装 Nginx Brotli 动态模块依赖"
if command -v apt-get >/dev/null 2>&1; then if command -v apt-get >/dev/null 2>&1; then
@@ -90,39 +99,111 @@ install_nginx_brotli_modules() {
fi fi
} }
install_sccache() { download_file() {
for tool_dir in "${HOME:-}/.cargo/bin" /root/.cargo/bin /usr/local/cargo/bin; do local url="$1"
if [[ -d "${tool_dir}" && ":${PATH}:" != *":${tool_dir}:"* ]]; then local output="$2"
export PATH="${tool_dir}:${PATH}"
if command -v curl >/dev/null 2>&1; then
curl -fsSL --retry 3 --retry-delay 2 "${url}" -o "${output}"
elif command -v wget >/dev/null 2>&1; then
wget -O "${output}" "${url}"
else
echo "[server-provision] 需要 curl 或 wget 下载: ${url}" >&2
exit 1
fi
}
openssl_lib_dir_candidates() {
printf "%s\n" \
"${GENARRATIVE_OPENSSL_PREFIX}/lib64" \
"${GENARRATIVE_OPENSSL_PREFIX}/lib"
}
find_genarrative_openssl_lib_dir() {
local lib_dir
while IFS= read -r lib_dir; do
if [[ -f "${lib_dir}/libssl.so.3" && -f "${lib_dir}/libcrypto.so.3" ]]; then
printf "%s" "${lib_dir}"
return 0
fi fi
done done < <(openssl_lib_dir_candidates)
return 1
}
if command -v sccache >/dev/null 2>&1; then genarrative_openssl_has_required_symbol() {
echo "[server-provision] sccache 已存在: $(command -v sccache)" local lib_dir
return lib_dir="$(find_genarrative_openssl_lib_dir 2>/dev/null || true)"
if [[ -z "${lib_dir}" ]]; then
return 1
fi fi
grep -a -q "OPENSSL_${GENARRATIVE_OPENSSL_VERSION}" "${lib_dir}/libssl.so.3"
}
if [[ -x /root/.cargo/bin/sccache ]]; then verify_genarrative_openssl_install() {
echo "[server-provision] sccache 已存在: /root/.cargo/bin/sccache" local lib_dir
return lib_dir="$(find_genarrative_openssl_lib_dir 2>/dev/null || true)"
if [[ -z "${lib_dir}" ]]; then
echo "[server-provision] OpenSSL ${GENARRATIVE_OPENSSL_VERSION} 安装后缺少 libssl.so.3/libcrypto.so.3: ${GENARRATIVE_OPENSSL_PREFIX}" >&2
exit 1
fi fi
if ! grep -a -q "OPENSSL_${GENARRATIVE_OPENSSL_VERSION}" "${lib_dir}/libssl.so.3"; then
echo "[server-provision] OpenSSL 动态库缺少 OPENSSL_${GENARRATIVE_OPENSSL_VERSION} 符号: ${lib_dir}/libssl.so.3" >&2
exit 1
fi
if ! env "LD_LIBRARY_PATH=${lib_dir}" "${GENARRATIVE_OPENSSL_PREFIX}/bin/openssl" version | grep -q "OpenSSL ${GENARRATIVE_OPENSSL_VERSION}"; then
echo "[server-provision] OpenSSL ${GENARRATIVE_OPENSSL_VERSION} 安装后命令验证失败: ${GENARRATIVE_OPENSSL_PREFIX}/bin/openssl" >&2
exit 1
fi
echo "[server-provision] OpenSSL ${GENARRATIVE_OPENSSL_VERSION} 已就绪: ${lib_dir}"
}
echo "[server-provision] 未找到 sccache准备通过 cargo install sccache 安装。" install_genarrative_openssl_runtime() {
local tmp_dir archive source_dir jobs lib_dir
echo "[server-provision] 检查 api-server/libcurl 运行时 OpenSSL ${GENARRATIVE_OPENSSL_VERSION}"
if [[ "${DRY_RUN}" == "true" ]]; then if [[ "${DRY_RUN}" == "true" ]]; then
echo "+ cargo install sccache --locked" echo "+ install OpenSSL ${GENARRATIVE_OPENSSL_VERSION} into ${GENARRATIVE_OPENSSL_PREFIX}"
echo "+ verify OPENSSL_${GENARRATIVE_OPENSSL_VERSION} symbol for api-server/libcurl"
return return
fi fi
if ! command -v cargo >/dev/null 2>&1; then if genarrative_openssl_has_required_symbol; then
echo "[server-provision] 未找到 cargo无法自动安装 sccache。请先安装 Rust 工具链后重跑 Server-Provision。" >&2 verify_genarrative_openssl_install
exit 1 return
fi fi
cargo install sccache --locked if command -v apt-get >/dev/null 2>&1; then
if ! command -v sccache >/dev/null 2>&1 && [[ ! -x /root/.cargo/bin/sccache ]]; then run_cmd apt-get install -y build-essential ca-certificates curl perl tar
echo "[server-provision] sccache 安装后仍不可用,请检查 cargo bin 目录是否在 PATH 中。" >&2 else
echo "[server-provision] 当前系统未使用 apt无法自动构建 OpenSSL ${GENARRATIVE_OPENSSL_VERSION};请手动安装到 ${GENARRATIVE_OPENSSL_PREFIX}" >&2
exit 1 exit 1
fi fi
require_cmd sha256sum
require_cmd tar
tmp_dir="$(mktemp -d)"
archive="${tmp_dir}/openssl-${GENARRATIVE_OPENSSL_VERSION}.tar.gz"
echo "[server-provision] 下载 OpenSSL ${GENARRATIVE_OPENSSL_VERSION}: ${GENARRATIVE_OPENSSL_SOURCE_URL}"
download_file "${GENARRATIVE_OPENSSL_SOURCE_URL}" "${archive}"
printf "%s %s\n" "${GENARRATIVE_OPENSSL_SOURCE_SHA256}" "${archive}" | sha256sum -c -
tar -xzf "${archive}" -C "${tmp_dir}"
source_dir="${tmp_dir}/openssl-${GENARRATIVE_OPENSSL_VERSION}"
jobs="$(nproc 2>/dev/null || echo 2)"
(
cd "${source_dir}"
./config --prefix="${GENARRATIVE_OPENSSL_PREFIX}" --openssldir="${GENARRATIVE_OPENSSL_PREFIX}/ssl" shared
make -j "${jobs}"
make install_sw
)
rm -rf "${tmp_dir}"
lib_dir="$(find_genarrative_openssl_lib_dir 2>/dev/null || true)"
if [[ -n "${lib_dir}" ]]; then
chmod 0755 "${GENARRATIVE_OPENSSL_PREFIX}" "${lib_dir}" || true
chmod 0644 "${lib_dir}/libssl.so.3" "${lib_dir}/libcrypto.so.3" || true
fi
verify_genarrative_openssl_install
} }
sync_otelcol_install() { sync_otelcol_install() {
@@ -142,7 +223,7 @@ sync_otelcol_install() {
if [[ ! -x "${resolved_source}" ]]; then if [[ ! -x "${resolved_source}" ]]; then
echo "[server-provision] otelcol-contrib 不存在或不可执行: ${source_bin}" >&2 echo "[server-provision] otelcol-contrib 不存在或不可执行: ${source_bin}" >&2
echo "[server-provision] 请先在构建机准备好 otelcol-contrib ${version},再通过 provision-tools 上传到目标机。" >&2 echo "[server-provision] 请确认 Prepare Provision Tools 已在目标 agent 生成 otelcol-contrib ${version}: ${source_bin}" >&2
exit 1 exit 1
fi fi
@@ -671,9 +752,8 @@ require_non_root_relative_path "PROVISION_TOOLS_DIR" "${PROVISION_TOOLS_DIR}"
echo "[server-provision] target=${DEPLOY_TARGET}, dry_run=${DRY_RUN}, nginx_config_mode=${NGINX_CONFIG_MODE}, source_commit=$(cat .jenkins-source-commit)" echo "[server-provision] target=${DEPLOY_TARGET}, dry_run=${DRY_RUN}, nginx_config_mode=${NGINX_CONFIG_MODE}, source_commit=$(cat .jenkins-source-commit)"
run_cmd id run_cmd id
install_build_dependencies require_root_for_real_provision
install_nginx_brotli_modules install_nginx_brotli_modules
install_sccache
run_cmd mkdir -p "${SPACETIME_ROOT}" "${RELEASE_ROOT}" "$(dirname "${CURRENT_LINK}")" "$(dirname "${WEB_LINK}")" /etc/genarrative /var/lib/genarrative/maintenance /var/lib/genarrative/auth /var/lib/genarrative/tracking-outbox /var/lib/genarrative/database-backups run_cmd mkdir -p "${SPACETIME_ROOT}" "${RELEASE_ROOT}" "$(dirname "${CURRENT_LINK}")" "$(dirname "${WEB_LINK}")" /etc/genarrative /var/lib/genarrative/maintenance /var/lib/genarrative/auth /var/lib/genarrative/tracking-outbox /var/lib/genarrative/database-backups
if ! id spacetimedb >/dev/null 2>&1; then if ! id spacetimedb >/dev/null 2>&1; then
@@ -690,6 +770,7 @@ fi
run_cmd chown -R spacetimedb:spacetimedb "${SPACETIME_ROOT}" run_cmd chown -R spacetimedb:spacetimedb "${SPACETIME_ROOT}"
run_cmd chown -R genarrative:genarrative /opt/genarrative /var/lib/genarrative /srv/genarrative run_cmd chown -R genarrative:genarrative /opt/genarrative /var/lib/genarrative /srv/genarrative
install_genarrative_openssl_runtime
if [[ ! -x "${SPACETIME_BIN_SOURCE}" ]]; then if [[ ! -x "${SPACETIME_BIN_SOURCE}" ]]; then
echo "[server-provision] spacetime CLI 不存在或不可执行: ${SPACETIME_BIN_SOURCE}" >&2 echo "[server-provision] spacetime CLI 不存在或不可执行: ${SPACETIME_BIN_SOURCE}" >&2

61
server-rs/Cargo.lock generated
View File

@@ -636,6 +636,36 @@ dependencies = [
"memchr", "memchr",
] ]
[[package]]
name = "curl"
version = "0.4.49"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "79fc3b6dd0b87ba36e565715bf9a2ced221311db47bd18011676f24a6066edbc"
dependencies = [
"curl-sys",
"libc",
"openssl-probe",
"openssl-sys",
"schannel",
"socket2 0.6.3",
"windows-sys 0.59.0",
]
[[package]]
name = "curl-sys"
version = "0.4.88+curl-8.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "644816de6547255eff4e491a1dda1c19b7237f00b62a61e6e64859ce4f2906d0"
dependencies = [
"cc",
"libc",
"libz-sys",
"openssl-sys",
"pkg-config",
"vcpkg",
"windows-sys 0.61.2",
]
[[package]] [[package]]
name = "darling" name = "darling"
version = "0.23.0" version = "0.23.0"
@@ -1310,7 +1340,7 @@ dependencies = [
"libc", "libc",
"percent-encoding", "percent-encoding",
"pin-project-lite", "pin-project-lite",
"socket2 0.5.10", "socket2 0.6.3",
"tokio", "tokio",
"tower-service", "tower-service",
"tracing", "tracing",
@@ -1668,6 +1698,18 @@ dependencies = [
"glob", "glob",
] ]
[[package]]
name = "libz-sys"
version = "1.1.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85bc9657773828b90eeb625adff10eeac83cc21bbfd8e23a03eaa8a33c9e28d9"
dependencies = [
"cc",
"libc",
"pkg-config",
"vcpkg",
]
[[package]] [[package]]
name = "linux-raw-sys" name = "linux-raw-sys"
version = "0.12.1" version = "0.12.1"
@@ -2405,6 +2447,7 @@ name = "platform-image"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"base64 0.22.1", "base64 0.22.1",
"curl",
"image", "image",
"platform-oss", "platform-oss",
"reqwest 0.12.28", "reqwest 0.12.28",
@@ -2436,6 +2479,7 @@ dependencies = [
"sha2", "sha2",
"time", "time",
"tokio", "tokio",
"tracing",
] ]
[[package]] [[package]]
@@ -2605,7 +2649,7 @@ dependencies = [
"quinn-udp", "quinn-udp",
"rustc-hash", "rustc-hash",
"rustls", "rustls",
"socket2 0.5.10", "socket2 0.6.3",
"thiserror 2.0.18", "thiserror 2.0.18",
"tokio", "tokio",
"tracing", "tracing",
@@ -2642,7 +2686,7 @@ dependencies = [
"cfg_aliases", "cfg_aliases",
"libc", "libc",
"once_cell", "once_cell",
"socket2 0.5.10", "socket2 0.6.3",
"tracing", "tracing",
"windows-sys 0.52.0", "windows-sys 0.52.0",
] ]
@@ -4570,7 +4614,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [ dependencies = [
"windows-sys 0.48.0", "windows-sys 0.61.2",
] ]
[[package]] [[package]]
@@ -4650,6 +4694,15 @@ dependencies = [
"windows-targets 0.52.6", "windows-targets 0.52.6",
] ]
[[package]]
name = "windows-sys"
version = "0.59.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b"
dependencies = [
"windows-targets 0.52.6",
]
[[package]] [[package]]
name = "windows-sys" name = "windows-sys"
version = "0.61.2" version = "0.61.2"

View File

@@ -96,6 +96,7 @@ axum = "0.8"
base64 = "0.22" base64 = "0.22"
cbc = { version = "0.1", features = ["alloc"] } cbc = { version = "0.1", features = ["alloc"] }
bytes = "1" bytes = "1"
curl = "0.4"
dotenvy = "0.15" dotenvy = "0.15"
flate2 = "1" flate2 = "1"
futures-util = "0.3" futures-util = "0.3"

View File

@@ -54,7 +54,7 @@ shared-kernel = { workspace = true }
shared-logging = { workspace = true } shared-logging = { workspace = true }
socket2 = { workspace = true } socket2 = { workspace = true }
spacetime-client = { workspace = true } spacetime-client = { workspace = true }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "net", "time", "sync", "fs", "io-util"] } tokio = { workspace = true, features = ["macros", "rt-multi-thread", "net", "time", "sync", "fs", "io-util", "signal"] }
tokio-stream = { workspace = true } tokio-stream = { workspace = true }
futures-util = { workspace = true } futures-util = { workspace = true }
time = { workspace = true, features = ["formatting"] } time = { workspace = true, features = ["formatting"] }

View File

@@ -877,6 +877,46 @@ mod tests {
); );
} }
#[tokio::test]
async fn readyz_reports_readiness_and_draining_state() {
let state = AppState::new(AppConfig::default()).expect("state should build");
let app = build_router(state.clone());
let ready_response = app
.clone()
.oneshot(
Request::builder()
.uri("/readyz")
.header("x-request-id", "req-ready")
.body(Body::empty())
.expect("readyz request should build"),
)
.await
.expect("readyz request should succeed");
assert_eq!(ready_response.status(), StatusCode::OK);
let ready_body = read_json_response(ready_response).await;
assert_eq!(ready_body["ok"], Value::Bool(true));
assert_eq!(ready_body["ready"], Value::Bool(true));
state.mark_not_ready();
let draining_response = app
.oneshot(
Request::builder()
.uri("/readyz")
.header("x-request-id", "req-draining")
.body(Body::empty())
.expect("readyz request should build"),
)
.await
.expect("readyz request should succeed");
assert_eq!(draining_response.status(), StatusCode::SERVICE_UNAVAILABLE);
let draining_body = read_json_response(draining_response).await;
assert_eq!(
draining_body["error"]["details"]["reason"],
"api_server_draining"
);
}
#[tokio::test] #[tokio::test]
async fn creative_agent_draft_edit_rejects_unconfirmed_template_session() { async fn creative_agent_draft_edit_rejects_unconfirmed_template_session() {
let app = build_internal_creative_agent_app(); let app = build_internal_creative_agent_app();

View File

@@ -102,7 +102,7 @@ fn reject_overloaded_request(request: &Request<Body>) -> Response {
} }
fn should_bypass_backpressure(request: &Request<Body>) -> bool { fn should_bypass_backpressure(request: &Request<Body>) -> bool {
request.uri().path() == "/healthz" matches!(request.uri().path(), "/healthz" | "/readyz")
} }
fn classify_request_permit_pool(path: &str) -> HttpRequestPermitPoolKind { fn classify_request_permit_pool(path: &str) -> HttpRequestPermitPoolKind {
@@ -200,6 +200,7 @@ mod tests {
.route("/held", get(held_request)) .route("/held", get(held_request))
.route("/fast", get(fast_request)) .route("/fast", get(fast_request))
.route("/healthz", get(fast_request)) .route("/healthz", get(fast_request))
.route("/readyz", get(fast_request))
.layer(middleware::from_fn_with_state( .layer(middleware::from_fn_with_state(
backpressure_state, backpressure_state,
limit_concurrent_requests, limit_concurrent_requests,
@@ -297,6 +298,13 @@ mod tests {
.expect("healthz request should complete"); .expect("healthz request should complete");
assert_eq!(health_response.status(), StatusCode::OK); assert_eq!(health_response.status(), StatusCode::OK);
let ready_response = app
.clone()
.oneshot(test_request("/readyz"))
.await
.expect("readyz request should complete");
assert_eq!(ready_response.status(), StatusCode::OK);
gate.release.notify_one(); gate.release.notify_one();
let completed_response = held_response let completed_response = held_response
.await .await

View File

@@ -25,6 +25,7 @@ pub struct AppConfig {
pub gallery_max_concurrent_requests: Option<usize>, pub gallery_max_concurrent_requests: Option<usize>,
pub detail_max_concurrent_requests: Option<usize>, pub detail_max_concurrent_requests: Option<usize>,
pub admin_max_concurrent_requests: Option<usize>, pub admin_max_concurrent_requests: Option<usize>,
pub shutdown_outbox_flush_timeout: Duration,
pub tracking_outbox_enabled: bool, pub tracking_outbox_enabled: bool,
pub tracking_outbox_dir: PathBuf, pub tracking_outbox_dir: PathBuf,
pub tracking_outbox_batch_size: usize, pub tracking_outbox_batch_size: usize,
@@ -169,6 +170,7 @@ impl Default for AppConfig {
gallery_max_concurrent_requests: None, gallery_max_concurrent_requests: None,
detail_max_concurrent_requests: None, detail_max_concurrent_requests: None,
admin_max_concurrent_requests: None, admin_max_concurrent_requests: None,
shutdown_outbox_flush_timeout: Duration::from_millis(5_000),
tracking_outbox_enabled: true, tracking_outbox_enabled: true,
tracking_outbox_dir: PathBuf::from("server-rs/.data/tracking-outbox"), tracking_outbox_dir: PathBuf::from("server-rs/.data/tracking-outbox"),
tracking_outbox_batch_size: 500, tracking_outbox_batch_size: 500,
@@ -365,6 +367,11 @@ impl AppConfig {
{ {
config.admin_max_concurrent_requests = Some(max_concurrent_requests); config.admin_max_concurrent_requests = Some(max_concurrent_requests);
} }
if let Some(timeout_ms) =
read_first_positive_u64_env(&["GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS"])
{
config.shutdown_outbox_flush_timeout = Duration::from_millis(timeout_ms);
}
if let Some(enabled) = read_first_bool_env(&["GENARRATIVE_TRACKING_OUTBOX_ENABLED"]) { if let Some(enabled) = read_first_bool_env(&["GENARRATIVE_TRACKING_OUTBOX_ENABLED"]) {
config.tracking_outbox_enabled = enabled; config.tracking_outbox_enabled = enabled;
} }
@@ -1324,6 +1331,7 @@ mod tests {
std::env::remove_var("GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS"); std::env::remove_var("GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS");
std::env::remove_var("GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS"); std::env::remove_var("GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS");
std::env::remove_var("GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS"); std::env::remove_var("GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS");
std::env::remove_var("GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS");
std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_ENABLED"); std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_ENABLED");
std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_DIR"); std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_DIR");
std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE"); std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE");
@@ -1336,6 +1344,7 @@ mod tests {
std::env::set_var("GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS", "64"); std::env::set_var("GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS", "64");
std::env::set_var("GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS", "32"); std::env::set_var("GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS", "32");
std::env::set_var("GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS", "16"); std::env::set_var("GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS", "16");
std::env::set_var("GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS", "3000");
std::env::set_var("GENARRATIVE_TRACKING_OUTBOX_ENABLED", "false"); std::env::set_var("GENARRATIVE_TRACKING_OUTBOX_ENABLED", "false");
std::env::set_var( std::env::set_var(
"GENARRATIVE_TRACKING_OUTBOX_DIR", "GENARRATIVE_TRACKING_OUTBOX_DIR",
@@ -1354,6 +1363,10 @@ mod tests {
assert_eq!(config.gallery_max_concurrent_requests, Some(64)); assert_eq!(config.gallery_max_concurrent_requests, Some(64));
assert_eq!(config.detail_max_concurrent_requests, Some(32)); assert_eq!(config.detail_max_concurrent_requests, Some(32));
assert_eq!(config.admin_max_concurrent_requests, Some(16)); assert_eq!(config.admin_max_concurrent_requests, Some(16));
assert_eq!(
config.shutdown_outbox_flush_timeout,
std::time::Duration::from_millis(3_000)
);
assert!(!config.tracking_outbox_enabled); assert!(!config.tracking_outbox_enabled);
assert_eq!( assert_eq!(
config.tracking_outbox_dir, config.tracking_outbox_dir,
@@ -1374,6 +1387,7 @@ mod tests {
std::env::remove_var("GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS"); std::env::remove_var("GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS");
std::env::remove_var("GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS"); std::env::remove_var("GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS");
std::env::remove_var("GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS"); std::env::remove_var("GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS");
std::env::remove_var("GENARRATIVE_API_SHUTDOWN_OUTBOX_FLUSH_TIMEOUT_MS");
std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_ENABLED"); std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_ENABLED");
std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_DIR"); std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_DIR");
std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE"); std::env::remove_var("GENARRATIVE_TRACKING_OUTBOX_BATCH_SIZE");

View File

@@ -1,7 +1,15 @@
use axum::{Json, extract::Extension}; use axum::{
Json,
extract::{Extension, State},
http::StatusCode,
response::{IntoResponse, Response},
};
use serde_json::{Value, json}; use serde_json::{Value, json};
use crate::{api_response::json_success_body, request_context::RequestContext}; use crate::{
api_response::json_success_body, http_error::AppError, request_context::RequestContext,
state::AppState,
};
pub async fn health_check(Extension(request_context): Extension<RequestContext>) -> Json<Value> { pub async fn health_check(Extension(request_context): Extension<RequestContext>) -> Json<Value> {
json_success_body( json_success_body(
@@ -12,3 +20,28 @@ pub async fn health_check(Extension(request_context): Extension<RequestContext>)
}), }),
) )
} }
pub async fn readiness_check(
State(state): State<AppState>,
Extension(request_context): Extension<RequestContext>,
) -> Response {
if state.is_ready() {
return json_success_body(
Some(&request_context),
json!({
"ok": true,
"ready": true,
"service": "genarrative-api-server",
}),
)
.into_response();
}
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE)
.with_message("api-server 正在退出,不再接收新流量")
.with_details(json!({
"reason": "api_server_draining",
"ready": false,
}))
.into_response_with_context(Some(&request_context))
}

View File

@@ -99,25 +99,35 @@ use shared_logging::{OtelConfig, init_tracing};
use socket2::{Domain, Protocol, Socket, Type}; use socket2::{Domain, Protocol, Socket, Type};
use std::{ use std::{
collections::HashSet, collections::HashSet,
env, fs, io, env, fs, future, io,
net::{SocketAddr, TcpListener as StdTcpListener}, net::{SocketAddr, TcpListener as StdTcpListener},
panic, thread, panic,
sync::Arc,
thread,
time::Duration, time::Duration,
}; };
use tokio::net::TcpListener; use tokio::net::TcpListener;
use tokio::runtime::Builder as TokioRuntimeBuilder; use tokio::runtime::Builder as TokioRuntimeBuilder;
use tokio::time::timeout; use tokio::time::timeout;
use tracing::{error, info}; use tracing::{error, info, warn};
use crate::{ use crate::{
app::{build_router, build_spacetime_unavailable_router}, app::{build_router, build_spacetime_unavailable_router},
config::AppConfig, config::AppConfig,
state::{AppState, AppStateInitError}, state::{AppState, AppStateInitError},
tracking_outbox::TrackingOutbox,
}; };
const API_SERVER_STARTUP_STACK_SIZE_BYTES: usize = 32 * 1024 * 1024; const API_SERVER_STARTUP_STACK_SIZE_BYTES: usize = 32 * 1024 * 1024;
const AUTH_STORE_STARTUP_RESTORE_TIMEOUT: Duration = Duration::from_secs(8); const AUTH_STORE_STARTUP_RESTORE_TIMEOUT: Duration = Duration::from_secs(8);
#[derive(Clone)]
struct ShutdownContext {
app_state: Option<AppState>,
tracking_outbox: Option<Arc<TrackingOutbox>>,
outbox_flush_timeout: Duration,
}
fn main() -> Result<(), io::Error> { fn main() -> Result<(), io::Error> {
// Windows 本地调试下 Axum 路由树和启动恢复链较重,显式放大启动线程栈,避免 debug 构建在进入监听前栈溢出。 // Windows 本地调试下 Axum 路由树和启动恢复链较重,显式放大启动线程栈,避免 debug 构建在进入监听前栈溢出。
let server_thread = thread::Builder::new() let server_thread = thread::Builder::new()
@@ -158,19 +168,33 @@ async fn run_server(config: AppConfig) -> Result<(), io::Error> {
let listen_backlog = config.listen_backlog; let listen_backlog = config.listen_backlog;
let worker_threads = config.worker_threads; let worker_threads = config.worker_threads;
let otel_enabled = config.otel_enabled; let otel_enabled = config.otel_enabled;
let outbox_flush_timeout = config.shutdown_outbox_flush_timeout;
let listener = build_tcp_listener(bind_address, listen_backlog)?; let listener = build_tcp_listener(bind_address, listen_backlog)?;
let router = match restore_app_state_for_startup(config).await { let (router, shutdown_context) = match restore_app_state_for_startup(config).await {
Ok(state) => { Ok(state) => {
state.puzzle_gallery_cache().spawn_cleanup_task(); state.puzzle_gallery_cache().spawn_cleanup_task();
if let Some(outbox) = state.tracking_outbox() { let tracking_outbox = state.tracking_outbox();
if let Some(outbox) = tracking_outbox.clone() {
outbox.spawn_worker(); outbox.spawn_worker();
} }
build_router(state) (
} build_router(state.clone()),
Err(AppStateInitError::DependencyUnavailable(message)) => { ShutdownContext {
build_spacetime_unavailable_router(message) app_state: Some(state),
tracking_outbox,
outbox_flush_timeout,
},
)
} }
Err(AppStateInitError::DependencyUnavailable(message)) => (
build_spacetime_unavailable_router(message),
ShutdownContext {
app_state: None,
tracking_outbox: None,
outbox_flush_timeout,
},
),
Err(error) => { Err(error) => {
return Err(std::io::Error::other(format!( return Err(std::io::Error::other(format!(
"初始化应用状态失败:{error}" "初始化应用状态失败:{error}"
@@ -186,7 +210,98 @@ async fn run_server(config: AppConfig) -> Result<(), io::Error> {
"api-server 已完成 tracing 初始化并开始监听" "api-server 已完成 tracing 初始化并开始监听"
); );
axum::serve(listener, router).await let result = axum::serve(listener, router)
.with_graceful_shutdown(shutdown_signal(shutdown_context.clone()))
.await;
finalize_shutdown(shutdown_context).await;
result
}
async fn shutdown_signal(context: ShutdownContext) {
let signal = wait_for_shutdown_signal().await;
if let Some(state) = context.app_state.as_ref() {
state.mark_not_ready();
}
info!(
signal,
"api-server 收到退出信号,已标记 readiness 不可用并开始排空 HTTP 请求"
);
}
async fn wait_for_shutdown_signal() -> &'static str {
#[cfg(unix)]
{
tokio::select! {
signal = wait_for_ctrl_c_signal() => signal,
signal = wait_for_sigterm_signal() => signal,
}
}
#[cfg(not(unix))]
{
wait_for_ctrl_c_signal().await
}
}
async fn wait_for_ctrl_c_signal() -> &'static str {
if let Err(error) = tokio::signal::ctrl_c().await {
error!(error = %error, "监听 SIGINT 失败,无法通过 Ctrl-C 触发优雅退出");
future::pending::<()>().await;
}
"sigint"
}
#[cfg(unix)]
async fn wait_for_sigterm_signal() -> &'static str {
let mut signal = match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())
{
Ok(signal) => signal,
Err(error) => {
error!(error = %error, "监听 SIGTERM 失败,无法通过 systemd terminate 触发优雅退出");
future::pending::<()>().await;
unreachable!("pending future never returns");
}
};
signal.recv().await;
"sigterm"
}
async fn finalize_shutdown(context: ShutdownContext) {
if let Some(state) = context.app_state.as_ref() {
state.mark_not_ready();
}
let Some(outbox) = context.tracking_outbox else {
return;
};
if context.outbox_flush_timeout.is_zero() {
warn!("api-server 退出时 tracking outbox flush timeout 为 0跳过主动 flush");
return;
}
let timeout_ms = context
.outbox_flush_timeout
.as_millis()
.min(u128::from(u64::MAX)) as u64;
info!(timeout_ms, "api-server 退出前封存并 flush tracking outbox");
match timeout(context.outbox_flush_timeout, outbox.flush_for_shutdown()).await {
Ok(Ok(())) => {
info!("api-server 退出前 tracking outbox flush 完成");
}
Ok(Err(error)) => {
warn!(
error = %error,
"api-server 退出前 tracking outbox flush 未完成,已保留本地文件等待下次启动重试"
);
}
Err(_) => {
warn!(
timeout_ms,
"api-server 退出前 tracking outbox flush 超时,已保留本地文件等待下次启动重试"
);
}
}
} }
fn build_tcp_listener( fn build_tcp_listener(

View File

@@ -1,7 +1,12 @@
use axum::{Router, routing::get}; use axum::{Router, routing::get};
use crate::{health::health_check, state::AppState}; use crate::{
health::{health_check, readiness_check},
state::AppState,
};
pub fn router(_state: AppState) -> Router<AppState> { pub fn router(_state: AppState) -> Router<AppState> {
Router::new().route("/healthz", get(health_check)) Router::new()
.route("/healthz", get(health_check))
.route("/readyz", get(readiness_check))
} }

View File

@@ -2,7 +2,10 @@ use std::{
collections::HashMap, collections::HashMap,
error::Error, error::Error,
fmt, fmt,
sync::{Arc, Mutex}, sync::{
Arc, Mutex,
atomic::{AtomicBool, Ordering},
},
}; };
use axum::extract::FromRef; use axum::extract::FromRef;
@@ -229,6 +232,7 @@ pub struct AppStateInner {
// 配置会在后续中间件、路由和平台适配接入时逐步消费。 // 配置会在后续中间件、路由和平台适配接入时逐步消费。
#[allow(dead_code)] #[allow(dead_code)]
pub config: AppConfig, pub config: AppConfig,
ready: AtomicBool,
http_request_permit_pools: HttpRequestPermitPools, http_request_permit_pools: HttpRequestPermitPools,
auth_jwt_config: JwtConfig, auth_jwt_config: JwtConfig,
admin_runtime: Option<AdminRuntime>, admin_runtime: Option<AdminRuntime>,
@@ -399,6 +403,7 @@ impl AppState {
Ok(Self(Arc::new(AppStateInner { Ok(Self(Arc::new(AppStateInner {
config, config,
ready: AtomicBool::new(true),
http_request_permit_pools, http_request_permit_pools,
auth_jwt_config, auth_jwt_config,
admin_runtime, admin_runtime,
@@ -447,6 +452,14 @@ impl AppState {
self.http_request_permit_pools.clone() self.http_request_permit_pools.clone()
} }
pub fn is_ready(&self) -> bool {
self.ready.load(Ordering::Acquire)
}
pub fn mark_not_ready(&self) {
self.ready.store(false, Ordering::Release);
}
pub async fn upsert_creation_entry_type_config( pub async fn upsert_creation_entry_type_config(
&self, &self,
input: module_runtime::CreationEntryTypeAdminUpsertInput, input: module_runtime::CreationEntryTypeAdminUpsertInput,

View File

@@ -159,6 +159,16 @@ impl TrackingOutbox {
}); });
} }
pub async fn flush_for_shutdown(&self) -> Result<(), TrackingOutboxError> {
{
let mut inner = self.inner.lock().await;
self.ensure_initialized_locked(&mut inner).await?;
self.seal_active_locked(&mut inner, "shutdown").await?;
}
self.flush_sealed_files_once().await
}
async fn seal_active_if_due(&self) -> Result<(), TrackingOutboxError> { async fn seal_active_if_due(&self) -> Result<(), TrackingOutboxError> {
let mut inner = self.inner.lock().await; let mut inner = self.inner.lock().await;
self.ensure_initialized_locked(&mut inner).await?; self.ensure_initialized_locked(&mut inner).await?;
@@ -176,7 +186,11 @@ impl TrackingOutbox {
crate::telemetry::update_tracking_outbox_pending_files(sealed_files.len()); crate::telemetry::update_tracking_outbox_pending_files(sealed_files.len());
for path in sealed_files { for path in sealed_files {
let started_at = Instant::now(); let started_at = Instant::now();
let metadata = fs::metadata(&path).await?; let metadata = match fs::metadata(&path).await {
Ok(metadata) => metadata,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue,
Err(error) => return Err(error.into()),
};
let file_bytes = metadata.len(); let file_bytes = metadata.len();
let events = match read_outbox_events(&path).await { let events = match read_outbox_events(&path).await {
Ok(events) => events, Ok(events) => events,
@@ -203,7 +217,11 @@ impl TrackingOutbox {
match self.spacetime_client.record_tracking_events(events).await { match self.spacetime_client.record_tracking_events(events).await {
Ok(accepted_count) => { Ok(accepted_count) => {
fs::remove_file(&path).await?; match fs::remove_file(&path).await {
Ok(()) => {}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}
Err(error) => return Err(error.into()),
}
self.subtract_total_bytes(file_bytes).await; self.subtract_total_bytes(file_bytes).await;
crate::telemetry::record_tracking_outbox_flush( crate::telemetry::record_tracking_outbox_flush(
started_at.elapsed(), started_at.elapsed(),
@@ -596,6 +614,34 @@ mod tests {
let _ = std::fs::remove_dir_all(dir); let _ = std::fs::remove_dir_all(dir);
} }
#[tokio::test]
async fn shutdown_flush_seals_active_file_for_later_retry() {
let dir = test_dir("shutdown");
let outbox = test_outbox(dir.clone(), 500, 1024 * 1024);
outbox.enqueue(sample_event("event-1")).await.unwrap();
let result = outbox.flush_for_shutdown().await;
assert!(
matches!(result, Err(TrackingOutboxError::Spacetime(_))),
"missing test SpacetimeDB should keep sealed file for retry"
);
assert!(!dir.join(ACTIVE_FILE_NAME).exists());
let sealed_count = std::fs::read_dir(&dir)
.unwrap()
.filter_map(Result::ok)
.filter(|entry| {
entry
.file_name()
.to_str()
.is_some_and(|name| name.starts_with(SEALED_FILE_PREFIX))
})
.count();
assert_eq!(sealed_count, 1);
let _ = std::fs::remove_dir_all(dir);
}
#[test] #[test]
fn directory_size_excludes_quarantined_corrupt_files() { fn directory_size_excludes_quarantined_corrupt_files() {
let dir = test_dir("directory-size"); let dir = test_dir("directory-size");

View File

@@ -6,9 +6,10 @@ license.workspace = true
[dependencies] [dependencies]
base64 = { workspace = true } base64 = { workspace = true }
curl = { workspace = true }
image = { workspace = true, features = ["jpeg", "png", "webp"] } image = { workspace = true, features = ["jpeg", "png", "webp"] }
reqwest = { workspace = true, features = ["json", "multipart", "rustls-tls"] } reqwest = { workspace = true, features = ["json", "multipart", "rustls-tls"] }
serde_json = { workspace = true } serde_json = { workspace = true }
tokio = { workspace = true, features = ["time"] } tokio = { workspace = true, features = ["io-util", "macros", "net", "time"] }
tracing = { workspace = true } tracing = { workspace = true }
platform-oss = { workspace = true } platform-oss = { workspace = true }

View File

@@ -1,16 +1,22 @@
use reqwest::header; use std::time::{SystemTime, UNIX_EPOCH};
const VECTOR_ENGINE_SEND_MAX_ATTEMPTS: u32 = 5;
const VECTOR_ENGINE_SEND_RETRY_BASE_DELAY_MS: u64 = 500;
const VECTOR_ENGINE_SEND_RETRY_MAX_JITTER_MS: u64 = 999;
use super::{ use super::{
constants::{GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER}, constants::{GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER},
curl_transport::{
map_curl_error, send_vector_engine_json_request_with_curl,
send_vector_engine_multipart_edit_request_with_curl,
},
error::PlatformImageError, error::PlatformImageError,
image_source::resolve_reference_images, image_source::resolve_reference_images,
request::{ request::{
build_prompt_with_negative, build_vector_engine_image_edit_request_log_params, build_vector_engine_image_edit_request_log_params, build_vector_engine_image_request_body,
build_vector_engine_image_request_body, normalize_image_size, normalize_image_size, vector_engine_images_edit_url, vector_engine_images_generation_url,
vector_engine_images_edit_url, vector_engine_images_generation_url,
}, },
response::handle_vector_engine_response, response::handle_vector_engine_response,
transport::map_reqwest_error,
types::{GeneratedImages, ReferenceImage, VectorEngineImageSettings}, types::{GeneratedImages, ReferenceImage, VectorEngineImageSettings},
}; };
@@ -50,63 +56,69 @@ pub async fn create_vector_engine_image_generation(
reference_images, reference_images,
); );
let started_at = std::time::Instant::now(); let started_at = std::time::Instant::now();
let response = match http_client let mut attempt = 1;
.post(request_url.as_str()) let response = loop {
.header( match send_vector_engine_json_request_with_curl(
header::AUTHORIZATION, request_url.as_str(),
format!("Bearer {}", settings.api_key), settings.api_key.as_str(),
&request_body,
settings.request_timeout_ms,
) )
.header(header::ACCEPT, "application/json")
.header(header::CONTENT_TYPE, "application/json")
.json(&request_body)
.send()
.await .await
{ {
Ok(response) => response, Ok(response) => break response,
Err(error) => { Err(error) => {
return Err(map_reqwest_error( if should_retry_vector_engine_curl_send_error(&error, attempt) {
format!("{failure_context}:创建图片生成任务失败").as_str(), retry_vector_engine_send_after_delay(
request_url.as_str(), "generation",
"request_send", request_url.as_str(),
error, "request_send",
started_at.elapsed().as_millis() as u64, attempt,
Some(prompt.chars().count()), error.is_timeout(),
Some(reference_images.len()), error.is_connect(),
Some(&request_body), true,
)); false,
error.to_string().as_str(),
started_at.elapsed().as_millis() as u64,
Some(prompt.chars().count()),
Some(reference_images.len()),
Some(&request_body),
)
.await;
attempt += 1;
continue;
}
return Err(map_curl_error(
format!("{failure_context}:创建图片生成任务失败").as_str(),
request_url.as_str(),
"request_send",
error,
started_at.elapsed().as_millis() as u64,
Some(prompt.chars().count()),
Some(reference_images.len()),
Some(&request_body),
));
}
} }
}; };
let response_status = response.status(); let response_status = response.status;
tracing::info!( tracing::info!(
provider = VECTOR_ENGINE_PROVIDER, provider = VECTOR_ENGINE_PROVIDER,
endpoint = %request_url, endpoint = %request_url,
status = response_status.as_u16(), status = response_status,
prompt_chars = prompt.chars().count(), prompt_chars = prompt.chars().count(),
size = %normalized_size, size = %normalized_size,
reference_image_count = reference_images.len(), reference_image_count = reference_images.len(),
attempt,
elapsed_ms = started_at.elapsed().as_millis() as u64, elapsed_ms = started_at.elapsed().as_millis() as u64,
failure_context, failure_context,
"VectorEngine 图片生成 HTTP 返回" "VectorEngine 图片生成 HTTP 返回"
); );
let response_text = match response.text().await { let response_text = response.body;
Ok(response_text) => response_text,
Err(error) => {
return Err(map_reqwest_error(
format!("{failure_context}:读取图片生成响应失败").as_str(),
request_url.as_str(),
"response_body",
error,
started_at.elapsed().as_millis() as u64,
Some(prompt.chars().count()),
Some(reference_images.len()),
Some(&request_body),
));
}
};
handle_vector_engine_response( handle_vector_engine_response(
http_client, http_client,
request_url.as_str(), request_url.as_str(),
response_status.as_u16(), response_status,
response_text.as_str(), response_text.as_str(),
failure_context, failure_context,
started_at.elapsed().as_millis() as u64, started_at.elapsed().as_millis() as u64,
@@ -167,26 +179,6 @@ pub async fn create_vector_engine_image_edit_with_references(
reference_images, reference_images,
); );
let mut form = reqwest::multipart::Form::new()
.text("model", GPT_IMAGE_2_MODEL.to_string())
.text(
"prompt",
build_prompt_with_negative(prompt, negative_prompt),
)
.text("n", candidate_count.clamp(1, 4).to_string())
.text("size", normalized_size.clone());
for reference_image in reference_images.iter().take(5) {
let image_part = reqwest::multipart::Part::bytes(reference_image.bytes.clone())
.file_name(reference_image.file_name.clone())
.mime_str(reference_image.mime_type.as_str())
.map_err(|error| PlatformImageError::InvalidRequest {
provider: VECTOR_ENGINE_PROVIDER,
message: format!("{failure_context}:构造参考图失败:{error}"),
})?;
form = form.part("image", image_part);
}
let reference_image_count = reference_images.iter().take(5).count(); let reference_image_count = reference_images.iter().take(5).count();
let reference_image_bytes_total: usize = reference_images let reference_image_bytes_total: usize = reference_images
.iter() .iter()
@@ -214,64 +206,75 @@ pub async fn create_vector_engine_image_edit_with_references(
failure_context, failure_context,
"VectorEngine 图片编辑请求参数" "VectorEngine 图片编辑请求参数"
); );
let response = match http_client let mut attempt = 1;
.post(request_url.as_str()) let response = loop {
.header( match send_vector_engine_multipart_edit_request_with_curl(
header::AUTHORIZATION, request_url.as_str(),
format!("Bearer {}", settings.api_key), settings.api_key.as_str(),
prompt,
negative_prompt,
normalized_size.as_str(),
candidate_count,
reference_images,
settings.request_timeout_ms,
) )
.header(header::ACCEPT, "application/json")
.multipart(form)
.send()
.await .await
{ {
Ok(response) => response, Ok(response) => break response,
Err(error) => { Err(error) => {
return Err(map_reqwest_error( if should_retry_vector_engine_curl_send_error(&error, attempt) {
format!("{failure_context}:创建图片编辑任务失败").as_str(), retry_vector_engine_send_after_delay(
request_url.as_str(), "edit",
"request_send", request_url.as_str(),
error, "request_send",
started_at.elapsed().as_millis() as u64, attempt,
Some(prompt.chars().count()), error.is_timeout(),
Some(reference_image_count), error.is_connect(),
Some(&request_params), true,
)); false,
error.to_string().as_str(),
started_at.elapsed().as_millis() as u64,
Some(prompt.chars().count()),
Some(reference_image_count),
Some(&request_params),
)
.await;
attempt += 1;
continue;
}
return Err(map_curl_error(
format!("{failure_context}:创建图片编辑任务失败").as_str(),
request_url.as_str(),
"request_send",
error,
started_at.elapsed().as_millis() as u64,
Some(prompt.chars().count()),
Some(reference_image_count),
Some(&request_params),
));
}
} }
}; };
let response_status = response.status(); let response_status = response.status;
tracing::info!( tracing::info!(
provider = VECTOR_ENGINE_PROVIDER, provider = VECTOR_ENGINE_PROVIDER,
endpoint = %request_url, endpoint = %request_url,
status = response_status.as_u16(), status = response_status,
prompt_chars = prompt.chars().count(), prompt_chars = prompt.chars().count(),
size = %normalized_size, size = %normalized_size,
reference_image_count, reference_image_count,
reference_image_bytes_total, reference_image_bytes_total,
request_params = %request_params, request_params = %request_params,
attempt,
elapsed_ms = started_at.elapsed().as_millis() as u64, elapsed_ms = started_at.elapsed().as_millis() as u64,
failure_context, failure_context,
"VectorEngine 图片编辑 HTTP 返回" "VectorEngine 图片编辑 HTTP 返回"
); );
let response_text = match response.text().await { let response_text = response.body;
Ok(response_text) => response_text,
Err(error) => {
return Err(map_reqwest_error(
format!("{failure_context}:读取图片编辑响应失败").as_str(),
request_url.as_str(),
"response_body",
error,
started_at.elapsed().as_millis() as u64,
Some(prompt.chars().count()),
Some(reference_image_count),
Some(&request_params),
));
}
};
handle_vector_engine_response( handle_vector_engine_response(
http_client, http_client,
request_url.as_str(), request_url.as_str(),
response_status.as_u16(), response_status,
response_text.as_str(), response_text.as_str(),
failure_context, failure_context,
started_at.elapsed().as_millis() as u64, started_at.elapsed().as_millis() as u64,
@@ -282,3 +285,84 @@ pub async fn create_vector_engine_image_edit_with_references(
) )
.await .await
} }
fn should_retry_vector_engine_curl_send_error(
error: &super::curl_transport::VectorEngineCurlError,
attempt: u32,
) -> bool {
attempt < VECTOR_ENGINE_SEND_MAX_ATTEMPTS && (error.is_timeout() || error.is_connect())
}
async fn retry_vector_engine_send_after_delay(
request_kind: &'static str,
request_url: &str,
failure_stage: &'static str,
attempt: u32,
timeout: bool,
connect: bool,
request: bool,
body: bool,
error: &str,
elapsed_ms: u64,
prompt_chars: Option<usize>,
reference_image_count: Option<usize>,
request_params: Option<&serde_json::Value>,
) {
let delay_ms = vector_engine_send_retry_delay_ms(attempt, vector_engine_send_retry_jitter_ms());
tracing::warn!(
provider = VECTOR_ENGINE_PROVIDER,
endpoint = %request_url,
request_kind,
failure_stage,
attempt,
max_attempts = VECTOR_ENGINE_SEND_MAX_ATTEMPTS,
retry_delay_ms = delay_ms,
timeout,
connect,
request,
body,
status = 0,
error,
elapsed_ms,
prompt_chars,
reference_image_count,
request_params = %request_params
.map(|value| value.to_string())
.unwrap_or_default(),
"VectorEngine 图片请求发送失败,准备重试"
);
tokio::time::sleep(std::time::Duration::from_millis(delay_ms)).await;
}
fn vector_engine_send_retry_delay_ms(attempt: u32, jitter_ms: u64) -> u64 {
let exponential_factor = 1_u64 << attempt.saturating_sub(1).min(10);
let bounded_jitter_ms = jitter_ms.min(VECTOR_ENGINE_SEND_RETRY_MAX_JITTER_MS);
VECTOR_ENGINE_SEND_RETRY_BASE_DELAY_MS * exponential_factor + bounded_jitter_ms
}
fn vector_engine_send_retry_jitter_ms() -> u64 {
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|duration| duration.subsec_nanos())
.unwrap_or_default();
u64::from(nanos) % (VECTOR_ENGINE_SEND_RETRY_MAX_JITTER_MS + 1)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn vector_engine_send_retry_policy_allows_four_retries_before_final_attempt() {
assert_eq!(VECTOR_ENGINE_SEND_MAX_ATTEMPTS, 5);
}
#[test]
fn vector_engine_send_retry_delay_uses_exponential_backoff_with_bounded_jitter() {
assert_eq!(vector_engine_send_retry_delay_ms(1, 0), 500);
assert_eq!(vector_engine_send_retry_delay_ms(2, 0), 1_000);
assert_eq!(vector_engine_send_retry_delay_ms(3, 0), 2_000);
assert_eq!(vector_engine_send_retry_delay_ms(4, 0), 4_000);
assert_eq!(vector_engine_send_retry_delay_ms(4, 999), 4_999);
}
}

View File

@@ -0,0 +1,406 @@
use std::{error::Error, fmt, time::Duration};
use curl::{
FormError,
easy::{Easy, Form, List},
};
use serde_json::Value;
use super::{
audit::build_failure_audit,
constants::{GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER},
error::PlatformImageError,
request::build_prompt_with_negative,
types::ReferenceImage,
};
#[derive(Debug)]
pub(crate) struct VectorEngineCurlResponse {
pub(crate) status: u16,
pub(crate) body: String,
}
#[derive(Debug)]
pub(crate) enum VectorEngineCurlError {
Curl(curl::Error),
Form(FormError),
WorkerJoin(tokio::task::JoinError),
}
impl VectorEngineCurlError {
pub(crate) fn is_timeout(&self) -> bool {
match self {
Self::Curl(error) => error.is_operation_timedout(),
Self::Form(_) | Self::WorkerJoin(_) => false,
}
}
pub(crate) fn is_connect(&self) -> bool {
match self {
Self::Curl(error) => {
error.is_couldnt_connect()
|| error.is_couldnt_resolve_host()
|| error.is_couldnt_resolve_proxy()
}
Self::Form(_) | Self::WorkerJoin(_) => false,
}
}
}
impl fmt::Display for VectorEngineCurlError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Curl(error) => write!(formatter, "{error}"),
Self::Form(error) => write!(formatter, "multipart form error: {error}"),
Self::WorkerJoin(error) => write!(formatter, "curl worker join failed: {error}"),
}
}
}
impl Error for VectorEngineCurlError {}
impl From<curl::Error> for VectorEngineCurlError {
fn from(error: curl::Error) -> Self {
Self::Curl(error)
}
}
impl From<FormError> for VectorEngineCurlError {
fn from(error: FormError) -> Self {
Self::Form(error)
}
}
pub(crate) async fn send_vector_engine_json_request_with_curl(
request_url: &str,
api_key: &str,
request_body: &Value,
timeout_ms: u64,
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
let request_url = request_url.to_string();
let api_key = api_key.to_string();
let request_body = request_body.to_string();
tokio::task::spawn_blocking(move || {
send_json_request_with_curl_blocking(
request_url.as_str(),
api_key.as_str(),
request_body.as_str(),
timeout_ms,
)
})
.await
.map_err(VectorEngineCurlError::WorkerJoin)?
}
#[allow(clippy::too_many_arguments)]
pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl(
request_url: &str,
api_key: &str,
prompt: &str,
negative_prompt: Option<&str>,
normalized_size: &str,
candidate_count: u32,
reference_images: &[ReferenceImage],
timeout_ms: u64,
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
let request_url = request_url.to_string();
let api_key = api_key.to_string();
let prompt = prompt.to_string();
let negative_prompt = negative_prompt.map(str::to_string);
let normalized_size = normalized_size.to_string();
let reference_images = reference_images.iter().take(5).cloned().collect::<Vec<_>>();
tokio::task::spawn_blocking(move || {
send_multipart_edit_request_with_curl_blocking(
request_url.as_str(),
api_key.as_str(),
prompt.as_str(),
negative_prompt.as_deref(),
normalized_size.as_str(),
candidate_count,
reference_images.as_slice(),
timeout_ms,
)
})
.await
.map_err(VectorEngineCurlError::WorkerJoin)?
}
pub(crate) fn map_curl_error(
context: &str,
request_url: &str,
failure_stage: &'static str,
error: VectorEngineCurlError,
latency_ms: u64,
prompt_chars: Option<usize>,
reference_image_count: Option<usize>,
request_params: Option<&Value>,
) -> PlatformImageError {
let is_timeout = error.is_timeout();
let is_connect = error.is_connect();
let source = error.to_string();
let message = format!("{context}{source}");
let audit = build_failure_audit(
request_url,
context,
failure_stage,
None,
None,
is_timeout,
is_connect,
message.as_str(),
Some(source.clone()),
None,
Some(latency_ms),
prompt_chars,
reference_image_count,
);
tracing::warn!(
provider = VECTOR_ENGINE_PROVIDER,
endpoint = %request_url,
failure_stage,
timeout = is_timeout,
connect = is_connect,
request = true,
body = false,
status = 0,
source = %source,
source_chain = %source,
source_chain_depth = 1,
message = %message,
elapsed_ms = latency_ms,
prompt_chars,
reference_image_count,
request_params = %request_params
.map(|value| value.to_string())
.unwrap_or_default(),
"VectorEngine 图片 libcurl 请求失败"
);
PlatformImageError::Request {
provider: VECTOR_ENGINE_PROVIDER,
message,
endpoint: Some(request_url.to_string()),
timeout: is_timeout,
connect: is_connect,
request: true,
body: false,
status_code: None,
source: Some(source),
audit: Some(audit),
}
}
fn send_json_request_with_curl_blocking(
request_url: &str,
api_key: &str,
request_body: &str,
timeout_ms: u64,
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
let mut headers = vector_engine_curl_headers(api_key)?;
headers.append("Content-Type: application/json")?;
let mut easy = Easy::new();
easy.url(request_url)?;
easy.post(true)?;
easy.http_headers(headers)?;
easy.timeout(Duration::from_millis(timeout_ms.max(1)))?;
easy.post_fields_copy(request_body.as_bytes())?;
Ok(perform_curl_request(easy)?)
}
#[allow(clippy::too_many_arguments)]
fn send_multipart_edit_request_with_curl_blocking(
request_url: &str,
api_key: &str,
prompt: &str,
negative_prompt: Option<&str>,
normalized_size: &str,
candidate_count: u32,
reference_images: &[ReferenceImage],
timeout_ms: u64,
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
let mut form = Form::new();
form.part("model")
.contents(GPT_IMAGE_2_MODEL.as_bytes())
.add()?;
form.part("prompt")
.contents(build_prompt_with_negative(prompt, negative_prompt).as_bytes())
.add()?;
form.part("n")
.contents(candidate_count.clamp(1, 4).to_string().as_bytes())
.add()?;
form.part("size")
.contents(normalized_size.as_bytes())
.add()?;
for reference_image in reference_images {
form.part("image")
.buffer(
reference_image.file_name.as_str(),
reference_image.bytes.clone(),
)
.content_type(reference_image.mime_type.as_str())
.add()?;
}
let headers = vector_engine_curl_headers(api_key)?;
let mut easy = Easy::new();
easy.url(request_url)?;
easy.httppost(form)?;
easy.http_headers(headers)?;
easy.timeout(Duration::from_millis(timeout_ms.max(1)))?;
Ok(perform_curl_request(easy)?)
}
fn vector_engine_curl_headers(api_key: &str) -> Result<List, curl::Error> {
let mut headers = List::new();
headers.append(format!("Authorization: Bearer {api_key}").as_str())?;
headers.append("Accept: application/json")?;
Ok(headers)
}
fn perform_curl_request(mut easy: Easy) -> Result<VectorEngineCurlResponse, curl::Error> {
let mut body = Vec::new();
{
let mut transfer = easy.transfer();
transfer.write_function(|data| {
body.extend_from_slice(data);
Ok(data.len())
})?;
transfer.perform()?;
}
let status = easy.response_code()? as u16;
let body = String::from_utf8_lossy(body.as_slice()).into_owned();
Ok(VectorEngineCurlResponse { status, body })
}
#[cfg(test)]
mod tests {
use super::*;
use crate::vector_engine::types::ReferenceImage;
use tokio::{
io::{AsyncReadExt, AsyncWriteExt},
net::TcpListener,
sync::oneshot,
};
#[tokio::test]
async fn vector_engine_curl_transport_posts_json_request() {
let (base_url, server, request_rx) = start_single_response_server().await;
let response = send_vector_engine_json_request_with_curl(
format!("{base_url}/v1/images/generations").as_str(),
"test-key",
&serde_json::json!({"model":"gpt-image-2","prompt":"测试"}),
1_000,
)
.await
.expect("curl json request should succeed");
assert_eq!(response.status, 200);
assert_eq!(response.body, "{\"data\":[]}");
let request = request_rx
.await
.expect("mock server should capture request");
let request_text = String::from_utf8_lossy(request.as_slice());
assert!(request_text.contains("Content-Type: application/json"));
server.abort();
}
#[tokio::test]
async fn vector_engine_curl_transport_posts_multipart_request() {
let (base_url, server, request_rx) = start_single_response_server().await;
let response = send_vector_engine_multipart_edit_request_with_curl(
format!("{base_url}/v1/images/edits").as_str(),
"test-key",
"测试提示词",
None,
"1024x1024",
1,
&[ReferenceImage {
bytes: b"reference".to_vec(),
mime_type: "image/png".to_string(),
file_name: "reference.png".to_string(),
}],
1_000,
)
.await
.expect("curl multipart request should succeed");
assert_eq!(response.status, 200);
assert_eq!(response.body, "{\"data\":[]}");
let request = request_rx
.await
.expect("mock server should capture request");
let request_text = String::from_utf8_lossy(request.as_slice());
assert!(request_text.contains("name=\"image\"; filename=\"reference.png\""));
assert!(request_text.contains("Content-Type: image/png"));
assert!(request_text.contains("reference"));
server.abort();
}
async fn start_single_response_server() -> (
String,
tokio::task::JoinHandle<()>,
oneshot::Receiver<Vec<u8>>,
) {
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("mock server should bind");
let addr = listener
.local_addr()
.expect("mock server addr should be readable");
let (request_tx, request_rx) = oneshot::channel();
let server = tokio::spawn(async move {
let Ok((mut stream, _)) = listener.accept().await else {
return;
};
let mut request = Vec::new();
let mut buffer = [0_u8; 4096];
loop {
let Ok(read) = stream.read(&mut buffer).await else {
return;
};
if read == 0 {
return;
}
request.extend_from_slice(&buffer[..read]);
if request.windows(4).any(|window| window == b"\r\n\r\n") {
break;
}
}
let header_end = request
.windows(4)
.position(|window| window == b"\r\n\r\n")
.map(|index| index + 4)
.unwrap_or(request.len());
let headers = String::from_utf8_lossy(&request[..header_end]);
let content_length = headers
.lines()
.find_map(|line| {
line.strip_prefix("Content-Length:")
.or_else(|| line.strip_prefix("content-length:"))
})
.and_then(|value| value.trim().parse::<usize>().ok())
.unwrap_or_default();
let expected_len = header_end + content_length;
while request.len() < expected_len {
let Ok(read) = stream.read(&mut buffer).await else {
return;
};
if read == 0 {
break;
}
request.extend_from_slice(&buffer[..read]);
}
let _ = request_tx.send(request);
let body = "{\"data\":[]}";
let response = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{}",
body.len(),
body
);
let _ = stream.write_all(response.as_bytes()).await;
});
(format!("http://{addr}"), server, request_rx)
}
}

View File

@@ -1,6 +1,7 @@
mod audit; mod audit;
mod client; mod client;
mod constants; mod constants;
mod curl_transport;
mod error; mod error;
mod image_source; mod image_source;
mod payload; mod payload;

View File

@@ -1,10 +1,7 @@
use std::{error::Error, time::Duration}; use std::time::Duration;
use serde_json::Value;
use super::{ use super::{
audit::build_failure_audit, constants::VECTOR_ENGINE_PROVIDER, error::PlatformImageError, constants::VECTOR_ENGINE_PROVIDER, error::PlatformImageError, types::VectorEngineImageSettings,
types::VectorEngineImageSettings,
}; };
pub fn build_vector_engine_image_http_client( pub fn build_vector_engine_image_http_client(
@@ -20,130 +17,3 @@ pub fn build_vector_engine_image_http_client(
message: format!("构造 VectorEngine 图片生成 HTTP 客户端失败:{error}"), message: format!("构造 VectorEngine 图片生成 HTTP 客户端失败:{error}"),
}) })
} }
pub(super) fn map_reqwest_error(
context: &str,
request_url: &str,
failure_stage: &'static str,
error: reqwest::Error,
latency_ms: u64,
prompt_chars: Option<usize>,
reference_image_count: Option<usize>,
request_params: Option<&Value>,
) -> PlatformImageError {
let is_timeout = error.is_timeout();
let is_connect = error.is_connect();
let source_chain_parts = collect_error_source_chain(&error);
let source = source_chain_parts.first().cloned();
let source_chain_depth = source_chain_parts.len();
let source_chain = if source_chain_parts.is_empty() {
None
} else {
Some(source_chain_parts.join(" -> "))
};
let message = format!("{context}{error}");
let audit = build_failure_audit(
request_url,
context,
failure_stage,
error.status().map(|status| status.as_u16()),
None,
is_timeout,
is_connect,
message.as_str(),
source_chain.clone().or_else(|| source.clone()),
None,
Some(latency_ms),
prompt_chars,
reference_image_count,
);
tracing::warn!(
provider = VECTOR_ENGINE_PROVIDER,
endpoint = %request_url,
failure_stage,
timeout = is_timeout,
connect = is_connect,
request = error.is_request(),
body = error.is_body(),
status = error.status().map(|status| status.as_u16()).unwrap_or_default(),
source = %source.clone().unwrap_or_default(),
source_chain = %source_chain.clone().unwrap_or_default(),
source_chain_depth,
message = %message,
elapsed_ms = latency_ms,
prompt_chars,
reference_image_count,
request_params = %request_params
.map(|value| value.to_string())
.unwrap_or_default(),
"VectorEngine 图片请求发送失败"
);
PlatformImageError::Request {
provider: VECTOR_ENGINE_PROVIDER,
message,
endpoint: Some(request_url.to_string()),
timeout: is_timeout,
connect: is_connect,
request: error.is_request(),
body: error.is_body(),
status_code: error.status().map(|status| status.as_u16()),
source: source_chain.or(source),
audit: Some(audit),
}
}
fn collect_error_source_chain(error: &(dyn Error + 'static)) -> Vec<String> {
let mut chain = Vec::new();
let mut next = error.source();
while let Some(source) = next {
chain.push(source.to_string());
next = source.source();
}
chain
}
#[cfg(test)]
mod tests {
use super::*;
use std::fmt;
#[derive(Debug)]
struct TestError {
message: &'static str,
source: Option<Box<TestError>>,
}
impl fmt::Display for TestError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str(self.message)
}
}
impl Error for TestError {
fn source(&self) -> Option<&(dyn Error + 'static)> {
self.source
.as_deref()
.map(|source| source as &(dyn Error + 'static))
}
}
#[test]
fn collect_error_source_chain_keeps_nested_causes() {
let error = TestError {
message: "top",
source: Some(Box::new(TestError {
message: "middle",
source: Some(Box::new(TestError {
message: "bottom",
source: None,
})),
})),
};
assert_eq!(
collect_error_source_chain(&error),
vec!["middle".to_string(), "bottom".to_string()]
);
}
}

View File

@@ -1,8 +1,20 @@
use platform_image::vector_engine::{ use platform_image::vector_engine::{
GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings, GPT_IMAGE_2_MODEL, ReferenceImage, VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings,
build_vector_engine_image_request_body, vector_engine_images_edit_url, build_vector_engine_image_http_client, build_vector_engine_image_request_body,
create_vector_engine_image_edit, vector_engine_images_edit_url,
vector_engine_images_generation_url, vector_engine_images_generation_url,
}; };
use std::{
sync::{
Arc,
atomic::{AtomicUsize, Ordering},
},
time::Duration,
};
use tokio::{
io::{AsyncReadExt, AsyncWriteExt},
net::TcpListener,
};
#[test] #[test]
fn vector_engine_module_exposes_provider_protocol_helpers() { fn vector_engine_module_exposes_provider_protocol_helpers() {
@@ -30,3 +42,70 @@ fn vector_engine_module_exposes_provider_protocol_helpers() {
"https://vector.example/v1/images/edits" "https://vector.example/v1/images/edits"
); );
} }
#[tokio::test]
async fn vector_engine_image_edit_retries_send_timeout_once_and_succeeds() {
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("mock server should bind");
let server_addr = listener
.local_addr()
.expect("mock server address should be readable");
let request_count = Arc::new(AtomicUsize::new(0));
let request_count_for_server = Arc::clone(&request_count);
let server = tokio::spawn(async move {
loop {
let Ok((mut stream, _)) = listener.accept().await else {
break;
};
let request_index = request_count_for_server.fetch_add(1, Ordering::SeqCst);
tokio::spawn(async move {
let mut buffer = [0_u8; 4096];
let _ = stream.read(&mut buffer).await;
if request_index == 0 {
tokio::time::sleep(Duration::from_millis(120)).await;
return;
}
let body = r#"{"data":[{"b64_json":"iVBORw0KGgpyZXN0"}]}"#;
let response = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{}",
body.len(),
body
);
let _ = stream.write_all(response.as_bytes()).await;
});
}
});
let settings = VectorEngineImageSettings {
base_url: format!("http://{server_addr}/v1"),
api_key: "test-key".to_string(),
request_timeout_ms: 40,
};
let http_client =
build_vector_engine_image_http_client(&settings).expect("client should build");
let reference_image = ReferenceImage {
bytes: b"reference".to_vec(),
mime_type: "image/png".to_string(),
file_name: "reference.png".to_string(),
};
let generated = create_vector_engine_image_edit(
&http_client,
&settings,
"测试提示词",
None,
"1024x1024",
&reference_image,
"测试 VectorEngine 图片编辑失败",
)
.await
.expect("second attempt should return generated image");
assert_eq!(generated.images.len(), 1);
assert_eq!(generated.images[0].mime_type, "image/png");
assert_eq!(request_count.load(Ordering::SeqCst), 2);
server.abort();
}

View File

@@ -12,6 +12,7 @@ serde = { workspace = true }
serde_json = { workspace = true } serde_json = { workspace = true }
sha2 = { workspace = true } sha2 = { workspace = true }
time = { workspace = true, features = ["formatting"] } time = { workspace = true, features = ["formatting"] }
tracing = { workspace = true }
[dev-dependencies] [dev-dependencies]
tokio = { workspace = true, features = ["macros", "rt"] } tokio = { workspace = true, features = ["macros", "rt"] }

View File

@@ -22,6 +22,7 @@
5. 服务端 `PutObject` 上传 helper 5. 服务端 `PutObject` 上传 helper
6. `x-oss-meta-*` 元数据归一化与大小限制校验 6. `x-oss-meta-*` 元数据归一化与大小限制校验
7. `content-type``content-length-range``success_action_status` policy 条件生成 7. `content-type``content-length-range``success_action_status` policy 条件生成
8. `PostObject` 签名、`GetObject` 读签名、`HEAD Object``PutObject` 的结构化日志
当前仍未落地的内容: 当前仍未落地的内容:
@@ -34,8 +35,9 @@
1. 当前产品口径为服务器上传 AI 生成资源、Web 端只负责读取。 1. 当前产品口径为服务器上传 AI 生成资源、Web 端只负责读取。
2. 因此 `STS` 不作为默认上传主链,`api-server` 只暴露禁用式 contract避免浏览器拿到 OSS 写权限。 2. 因此 `STS` 不作为默认上传主链,`api-server` 只暴露禁用式 contract避免浏览器拿到 OSS 写权限。
3. 服务端生成资源应优先复用 `OssClient::put_object`,上传成功后再走对象确认链路写入 `asset_object` 3. 服务端生成资源应优先复用 `OssClient::put_object`,上传成功后再走对象确认链路写入 `asset_object`
4. 读签名和 `HEAD Object` 的入参必须直接传 object_key不要把 bucket 名拼进路径;例如 `generated-square-hole-assets/.../image.png` 才是正确入参,`xushi-dev/...` 这类前缀不属于 object_key。 4. 读签名和 `HEAD Object` 的入参必须直接传 object_key不要把 bucket 名拼进路径;例如 `generated-square-hole-assets/.../image.png` 才是正确入参,`xushi-dev/...` 这类前缀不属于 object_key。
5. OSS V4 `x-oss-date` 必须固定为 `yyyyMMdd'T'HHmmss'Z'`,不能依赖 `time::Time::to_string()`;后者在小时小于 10 时可能输出非补零时间,导致签名格式错误。 5. OSS V4 `x-oss-date` 必须固定为 `yyyyMMdd'T'HHmmss'Z'`,不能依赖 `time::Time::to_string()`;后者在小时小于 10 时可能输出非补零时间,导致签名格式错误。
6. 结构化日志只记录 `provider``operation``bucket``endpoint``object_key` / `key_prefix``access``content_type``content_length``status``status_class``error_kind``elapsed_ms` 等排障字段;禁止输出 AccessKey、policy、signature、Authorization header 或完整 signed URL。
## 3. 边界约束 ## 3. 边界约束

View File

@@ -1,4 +1,4 @@
use std::{collections::BTreeMap, error::Error, fmt}; use std::{collections::BTreeMap, error::Error, fmt, time::Instant};
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64_STANDARD}; use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64_STANDARD};
use hmac::{Hmac, Mac}; use hmac::{Hmac, Mac};
@@ -7,6 +7,7 @@ use serde::{Deserialize, Serialize};
use serde_json::{Value, json}; use serde_json::{Value, json};
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use time::{Duration, OffsetDateTime, format_description::well_known::Rfc3339}; use time::{Duration, OffsetDateTime, format_description::well_known::Rfc3339};
use tracing::{info, warn};
type HmacSha256 = Hmac<Sha256>; type HmacSha256 = Hmac<Sha256>;
@@ -19,6 +20,7 @@ const OSS_V4_ALGORITHM: &str = "OSS4-HMAC-SHA256";
const OSS_V4_REQUEST: &str = "aliyun_v4_request"; const OSS_V4_REQUEST: &str = "aliyun_v4_request";
const OSS_V4_SERVICE: &str = "oss"; const OSS_V4_SERVICE: &str = "oss";
const OSS_UNSIGNED_PAYLOAD: &str = "UNSIGNED-PAYLOAD"; const OSS_UNSIGNED_PAYLOAD: &str = "UNSIGNED-PAYLOAD";
const OSS_PROVIDER: &str = "aliyun-oss";
pub const LEGACY_PUBLIC_PREFIXES: [&str; 13] = [ pub const LEGACY_PUBLIC_PREFIXES: [&str; 13] = [
"generated-character-drafts", "generated-character-drafts",
@@ -369,105 +371,154 @@ impl OssClient {
&self, &self,
request: OssPostObjectRequest, request: OssPostObjectRequest,
) -> Result<OssPostObjectResponse, OssError> { ) -> Result<OssPostObjectResponse, OssError> {
let max_size_bytes = request let started_at = Instant::now();
.max_size_bytes let requested_prefix = request.prefix.as_str();
.unwrap_or(self.config.default_post_max_size_bytes); let requested_content_type = request
let expire_seconds = request .content_type
.expire_seconds .as_deref()
.unwrap_or(self.config.default_post_expire_seconds); .map(str::trim)
let success_action_status = request .unwrap_or("")
.success_action_status .to_string();
.unwrap_or(self.config.default_success_action_status); let requested_metadata_count = request.metadata.len();
if max_size_bytes == 0 { let result = (|| {
return Err(OssError::InvalidRequest( let max_size_bytes = request
"maxSizeBytes 必须大于 0".to_string(), .max_size_bytes
)); .unwrap_or(self.config.default_post_max_size_bytes);
let expire_seconds = request
.expire_seconds
.unwrap_or(self.config.default_post_expire_seconds);
let success_action_status = request
.success_action_status
.unwrap_or(self.config.default_success_action_status);
if max_size_bytes == 0 {
return Err(OssError::InvalidRequest(
"maxSizeBytes 必须大于 0".to_string(),
));
}
if expire_seconds == 0 {
return Err(OssError::InvalidRequest(
"expireSeconds 必须大于 0".to_string(),
));
}
if !(100..=999).contains(&success_action_status) {
return Err(OssError::InvalidRequest(
"successActionStatus 必须是三位 HTTP 状态码".to_string(),
));
}
let sanitized_segments = request
.path_segments
.iter()
.map(|segment| sanitize_path_segment(segment))
.filter(|segment| !segment.is_empty())
.collect::<Vec<_>>();
let file_name = sanitize_file_name(&request.file_name)?;
let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name);
let legacy_public_path = format!("/{}", object_key);
let content_type = normalize_optional_value(request.content_type);
let metadata = normalize_metadata(request.metadata)?;
let expires_at = OffsetDateTime::now_utc()
.checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err(
|_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()),
)?))
.ok_or_else(|| {
OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string())
})?;
let expires_at = expires_at.format(&Rfc3339).map_err(|error| {
OssError::SerializePolicy(format!("格式化过期时间失败:{error}"))
})?;
let signed_at = OffsetDateTime::now_utc();
let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?;
let signature_date = build_v4_signature_date(signed_at)?;
let credential = format!("{}/{}", self.config.access_key_id, signature_scope);
let policy_json = build_policy_json(
&self.config.bucket,
&object_key,
&expires_at,
max_size_bytes,
success_action_status,
content_type.as_deref(),
&metadata,
&credential,
&signature_date,
);
let policy = serde_json::to_string(&policy_json).map_err(|error| {
OssError::SerializePolicy(format!("序列化 policy 失败:{error}"))
})?;
let encoded_policy = BASE64_STANDARD.encode(policy.as_bytes());
let signature = sign_v4_content(
&self.config.access_key_secret,
&signature_scope,
&encoded_policy,
)?;
Ok(OssPostObjectResponse {
signature_version: "v4",
provider: OSS_PROVIDER,
bucket: self.config.bucket.clone(),
endpoint: self.config.endpoint.clone(),
host: self.config.upload_host(),
object_key: object_key.clone(),
legacy_public_path,
content_type: content_type.clone(),
access: request.access,
key_prefix: build_key_prefix(request.prefix, &sanitized_segments),
expires_at,
max_size_bytes,
success_action_status,
form_fields: OssPostObjectFormFields {
key: object_key,
policy: encoded_policy,
signature_version: OSS_V4_ALGORITHM.to_string(),
credential,
date: signature_date,
signature,
success_action_status: success_action_status.to_string(),
content_type,
metadata,
},
})
})();
match &result {
Ok(response) => info!(
provider = OSS_PROVIDER,
operation = "sign_post_object",
bucket = %response.bucket,
endpoint = %response.endpoint,
object_key = %response.object_key,
key_prefix = %response.key_prefix,
access = oss_access_label(response.access),
content_type = %response.content_type.as_deref().unwrap_or(""),
max_size_bytes = response.max_size_bytes,
success_action_status = response.success_action_status,
metadata_count = response.form_fields.metadata.len(),
expires_at = %response.expires_at,
elapsed_ms = elapsed_ms(started_at),
"OSS PostObject 签名完成"
),
Err(error) => warn!(
provider = OSS_PROVIDER,
operation = "sign_post_object",
bucket = %self.config.bucket(),
endpoint = %self.config.endpoint(),
key_prefix = requested_prefix,
content_type = %requested_content_type,
metadata_count = requested_metadata_count,
error_kind = oss_error_kind_label(error),
message = %error,
elapsed_ms = elapsed_ms(started_at),
"OSS PostObject 签名失败"
),
} }
if expire_seconds == 0 { result
return Err(OssError::InvalidRequest(
"expireSeconds 必须大于 0".to_string(),
));
}
if !(100..=999).contains(&success_action_status) {
return Err(OssError::InvalidRequest(
"successActionStatus 必须是三位 HTTP 状态码".to_string(),
));
}
let sanitized_segments = request
.path_segments
.iter()
.map(|segment| sanitize_path_segment(segment))
.filter(|segment| !segment.is_empty())
.collect::<Vec<_>>();
let file_name = sanitize_file_name(&request.file_name)?;
let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name);
let legacy_public_path = format!("/{}", object_key);
let content_type = normalize_optional_value(request.content_type);
let metadata = normalize_metadata(request.metadata)?;
let expires_at = OffsetDateTime::now_utc()
.checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err(
|_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()),
)?))
.ok_or_else(|| OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string()))?;
let expires_at = expires_at
.format(&Rfc3339)
.map_err(|error| OssError::SerializePolicy(format!("格式化过期时间失败:{error}")))?;
let signed_at = OffsetDateTime::now_utc();
let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?;
let signature_date = build_v4_signature_date(signed_at)?;
let credential = format!("{}/{}", self.config.access_key_id, signature_scope);
let policy_json = build_policy_json(
&self.config.bucket,
&object_key,
&expires_at,
max_size_bytes,
success_action_status,
content_type.as_deref(),
&metadata,
&credential,
&signature_date,
);
let policy = serde_json::to_string(&policy_json)
.map_err(|error| OssError::SerializePolicy(format!("序列化 policy 失败:{error}")))?;
let encoded_policy = BASE64_STANDARD.encode(policy.as_bytes());
let signature = sign_v4_content(
&self.config.access_key_secret,
&signature_scope,
&encoded_policy,
)?;
Ok(OssPostObjectResponse {
signature_version: "v4",
provider: "aliyun-oss",
bucket: self.config.bucket.clone(),
endpoint: self.config.endpoint.clone(),
host: self.config.upload_host(),
object_key: object_key.clone(),
legacy_public_path,
content_type: content_type.clone(),
access: request.access,
key_prefix: build_key_prefix(request.prefix, &sanitized_segments),
expires_at,
max_size_bytes,
success_action_status,
form_fields: OssPostObjectFormFields {
key: object_key,
policy: encoded_policy,
signature_version: OSS_V4_ALGORITHM.to_string(),
credential,
date: signature_date,
signature,
success_action_status: success_action_status.to_string(),
content_type,
metadata,
},
})
} }
// 私有 bucket 的对象读取统一走短期签名 URL避免把长期主凭证下发给浏览器。 // 私有 bucket 的对象读取统一走短期签名 URL避免把长期主凭证下发给浏览器。
@@ -475,81 +526,119 @@ impl OssClient {
&self, &self,
request: OssSignedGetObjectUrlRequest, request: OssSignedGetObjectUrlRequest,
) -> Result<OssSignedGetObjectUrlResponse, OssError> { ) -> Result<OssSignedGetObjectUrlResponse, OssError> {
let expire_seconds = request let started_at = Instant::now();
.expire_seconds let requested_object_key = request
.unwrap_or(self.config.default_read_expire_seconds); .object_key
.trim()
.trim_start_matches('/')
.trim()
.to_string();
if expire_seconds == 0 { let result = (|| {
return Err(OssError::InvalidRequest( let expire_seconds = request
"expireSeconds 必须大于 0".to_string(), .expire_seconds
)); .unwrap_or(self.config.default_read_expire_seconds);
if expire_seconds == 0 {
return Err(OssError::InvalidRequest(
"expireSeconds 必须大于 0".to_string(),
));
}
let object_key = normalize_object_key(&request.object_key)?;
let expires_at = OffsetDateTime::now_utc()
.checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err(
|_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()),
)?))
.ok_or_else(|| {
OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string())
})?;
let expires_at_text = expires_at
.format(&Rfc3339)
.map_err(|error| OssError::Sign(format!("格式化过期时间失败:{error}")))?;
let signed_at = OffsetDateTime::now_utc();
let signed_at_text = build_v4_signature_date(signed_at)?;
let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?;
let credential = format!("{}/{}", self.config.access_key_id, signature_scope);
let mut query = BTreeMap::from([
("x-oss-additional-headers".to_string(), "host".to_string()),
(
"x-oss-signature-version".to_string(),
OSS_V4_ALGORITHM.to_string(),
),
("x-oss-credential".to_string(), credential),
("x-oss-date".to_string(), signed_at_text),
("x-oss-expires".to_string(), expire_seconds.to_string()),
]);
let canonical_uri = build_v4_canonical_uri(&self.config.bucket, Some(&object_key));
let object_url_path = format!("/{}", encode_url_path(&object_key));
let additional_headers = "host";
let canonical_headers =
format!("host:{}.{}\n", self.config.bucket(), self.config.endpoint());
let canonical_query = build_canonical_query_string(&query);
let canonical_request = build_v4_canonical_request(
Method::GET.as_str(),
&canonical_uri,
&canonical_query,
&canonical_headers,
additional_headers,
OSS_UNSIGNED_PAYLOAD,
);
let string_to_sign = build_v4_string_to_sign(
query["x-oss-date"].as_str(),
&signature_scope,
&canonical_request,
);
let signature = sign_v4_content(
&self.config.access_key_secret,
&signature_scope,
&string_to_sign,
)?;
query.insert("x-oss-signature".to_string(), signature);
let signed_url = format!(
"{}{}?{}",
self.config.upload_host(),
object_url_path,
build_canonical_query_string(&query)
);
Ok(OssSignedGetObjectUrlResponse {
provider: OSS_PROVIDER,
bucket: self.config.bucket.clone(),
endpoint: self.config.endpoint.clone(),
host: self.config.upload_host(),
object_key,
expires_at: expires_at_text,
signed_url,
})
})();
match &result {
Ok(response) => info!(
provider = OSS_PROVIDER,
operation = "sign_get_object_url",
bucket = %response.bucket,
endpoint = %response.endpoint,
object_key = %response.object_key,
expires_at = %response.expires_at,
elapsed_ms = elapsed_ms(started_at),
"OSS GetObject 读签名完成"
),
Err(error) => warn!(
provider = OSS_PROVIDER,
operation = "sign_get_object_url",
bucket = %self.config.bucket(),
endpoint = %self.config.endpoint(),
object_key = %requested_object_key,
error_kind = oss_error_kind_label(error),
message = %error,
elapsed_ms = elapsed_ms(started_at),
"OSS GetObject 读签名失败"
),
} }
let object_key = normalize_object_key(&request.object_key)?; result
let expires_at = OffsetDateTime::now_utc()
.checked_add(Duration::seconds(i64::try_from(expire_seconds).map_err(
|_| OssError::InvalidRequest("expireSeconds 超出可支持范围".to_string()),
)?))
.ok_or_else(|| OssError::InvalidRequest("expireSeconds 计算结果溢出".to_string()))?;
let expires_at_text = expires_at
.format(&Rfc3339)
.map_err(|error| OssError::Sign(format!("格式化过期时间失败:{error}")))?;
let signed_at = OffsetDateTime::now_utc();
let signed_at_text = build_v4_signature_date(signed_at)?;
let signature_scope = build_v4_signature_scope(&self.config.endpoint, signed_at)?;
let credential = format!("{}/{}", self.config.access_key_id, signature_scope);
let mut query = BTreeMap::from([
("x-oss-additional-headers".to_string(), "host".to_string()),
(
"x-oss-signature-version".to_string(),
OSS_V4_ALGORITHM.to_string(),
),
("x-oss-credential".to_string(), credential),
("x-oss-date".to_string(), signed_at_text),
("x-oss-expires".to_string(), expire_seconds.to_string()),
]);
let canonical_uri = build_v4_canonical_uri(&self.config.bucket, Some(&object_key));
let object_url_path = format!("/{}", encode_url_path(&object_key));
let additional_headers = "host";
let canonical_headers =
format!("host:{}.{}\n", self.config.bucket(), self.config.endpoint());
let canonical_query = build_canonical_query_string(&query);
let canonical_request = build_v4_canonical_request(
Method::GET.as_str(),
&canonical_uri,
&canonical_query,
&canonical_headers,
additional_headers,
OSS_UNSIGNED_PAYLOAD,
);
let string_to_sign = build_v4_string_to_sign(
query["x-oss-date"].as_str(),
&signature_scope,
&canonical_request,
);
let signature = sign_v4_content(
&self.config.access_key_secret,
&signature_scope,
&string_to_sign,
)?;
query.insert("x-oss-signature".to_string(), signature);
let signed_url = format!(
"{}{}?{}",
self.config.upload_host(),
object_url_path,
build_canonical_query_string(&query)
);
Ok(OssSignedGetObjectUrlResponse {
provider: "aliyun-oss",
bucket: self.config.bucket.clone(),
endpoint: self.config.endpoint.clone(),
host: self.config.upload_host(),
object_key,
expires_at: expires_at_text,
signed_url,
})
} }
// 上传完成确认前,服务端必须自己探测一次对象,不能只相信客户端回传的 object_key。 // 上传完成确认前,服务端必须自己探测一次对象,不能只相信客户端回传的 object_key。
@@ -558,59 +647,107 @@ impl OssClient {
client: &reqwest::Client, client: &reqwest::Client,
request: OssHeadObjectRequest, request: OssHeadObjectRequest,
) -> Result<OssHeadObjectResponse, OssError> { ) -> Result<OssHeadObjectResponse, OssError> {
let object_key = normalize_object_key(&request.object_key)?; let started_at = Instant::now();
let target_url = build_object_url(&self.config.bucket, &self.config.endpoint, &object_key) let requested_object_key = request
.map_err(|error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")))?; .object_key
let response = send_signed_request( .trim()
client, .trim_start_matches('/')
&self.config, .trim()
Method::HEAD, .to_string();
Some(&object_key), let mut response_status = None;
target_url,
)
.await?;
if response.status() == reqwest::StatusCode::NOT_FOUND { let result = async {
return Err(OssError::ObjectNotFound(format!( let object_key = normalize_object_key(&request.object_key)?;
"OSS 对象不存在:{}", let target_url =
request.object_key build_object_url(&self.config.bucket, &self.config.endpoint, &object_key).map_err(
))); |error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")),
)?;
let response = send_signed_request(
client,
&self.config,
Method::HEAD,
Some(&object_key),
target_url,
)
.await?;
response_status = Some(response.status().as_u16());
if response.status() == reqwest::StatusCode::NOT_FOUND {
return Err(OssError::ObjectNotFound(format!(
"OSS 对象不存在:{}",
request.object_key
)));
}
if !response.status().is_success() {
return Err(OssError::Request(format!(
"OSS HEAD Object 失败,状态码:{}",
response.status()
)));
}
let headers = response.headers();
let content_length = headers
.get(reqwest::header::CONTENT_LENGTH)
.and_then(|value| value.to_str().ok())
.and_then(|value| value.parse::<u64>().ok())
.unwrap_or(0);
let content_type = headers
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.map(|value| value.to_string());
let etag = headers
.get(reqwest::header::ETAG)
.and_then(|value| value.to_str().ok())
.map(|value| value.trim_matches('"').to_string());
let last_modified = headers
.get(reqwest::header::LAST_MODIFIED)
.and_then(|value| value.to_str().ok())
.map(|value| value.to_string());
Ok(OssHeadObjectResponse {
bucket: self.config.bucket.clone(),
object_key,
content_length,
content_type,
etag,
last_modified,
})
}
.await;
match &result {
Ok(response) => info!(
provider = OSS_PROVIDER,
operation = "head_object",
bucket = %response.bucket,
endpoint = %self.config.endpoint(),
object_key = %response.object_key,
status = response_status.unwrap_or(reqwest::StatusCode::OK.as_u16()),
status_class = http_status_class_from_option(response_status),
content_length = response.content_length,
content_type = %response.content_type.as_deref().unwrap_or(""),
etag_present = response.etag.is_some(),
last_modified_present = response.last_modified.is_some(),
elapsed_ms = elapsed_ms(started_at),
"OSS HEAD Object 完成"
),
Err(error) => warn!(
provider = OSS_PROVIDER,
operation = "head_object",
bucket = %self.config.bucket(),
endpoint = %self.config.endpoint(),
object_key = %requested_object_key,
status = response_status.unwrap_or_default(),
status_class = http_status_class_from_option(response_status),
error_kind = oss_error_kind_label(error),
message = %error,
elapsed_ms = elapsed_ms(started_at),
"OSS HEAD Object 失败"
),
} }
if !response.status().is_success() { result
return Err(OssError::Request(format!(
"OSS HEAD Object 失败,状态码:{}",
response.status()
)));
}
let headers = response.headers();
let content_length = headers
.get(reqwest::header::CONTENT_LENGTH)
.and_then(|value| value.to_str().ok())
.and_then(|value| value.parse::<u64>().ok())
.unwrap_or(0);
let content_type = headers
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.map(|value| value.to_string());
let etag = headers
.get(reqwest::header::ETAG)
.and_then(|value| value.to_str().ok())
.map(|value| value.trim_matches('"').to_string());
let last_modified = headers
.get(reqwest::header::LAST_MODIFIED)
.and_then(|value| value.to_str().ok())
.map(|value| value.to_string());
Ok(OssHeadObjectResponse {
bucket: self.config.bucket.clone(),
object_key,
content_length,
content_type,
etag,
last_modified,
})
} }
// AI 生成资源默认由服务端上传 OSSWeb 端只拿签名读地址,不直接持有写权限。 // AI 生成资源默认由服务端上传 OSSWeb 端只拿签名读地址,不直接持有写权限。
@@ -619,73 +756,128 @@ impl OssClient {
client: &reqwest::Client, client: &reqwest::Client,
request: OssPutObjectRequest, request: OssPutObjectRequest,
) -> Result<OssPutObjectResponse, OssError> { ) -> Result<OssPutObjectResponse, OssError> {
if request.body.is_empty() { let started_at = Instant::now();
return Err(OssError::InvalidRequest( let requested_prefix = request.prefix.as_str();
"服务端上传对象内容不能为空".to_string(), let requested_content_type = request
)); .content_type
.as_deref()
.map(str::trim)
.unwrap_or("")
.to_string();
let requested_content_length = request.body.len();
let requested_metadata_count = request.metadata.len();
let mut response_status = None;
let result = async {
if request.body.is_empty() {
return Err(OssError::InvalidRequest(
"服务端上传对象内容不能为空".to_string(),
));
}
let sanitized_segments = request
.path_segments
.iter()
.map(|segment| sanitize_path_segment(segment))
.filter(|segment| !segment.is_empty())
.collect::<Vec<_>>();
let file_name = sanitize_file_name(&request.file_name)?;
let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name);
let content_type = normalize_optional_value(request.content_type);
let metadata = normalize_metadata(request.metadata)?;
let target_url =
build_object_url(&self.config.bucket, &self.config.endpoint, &object_key).map_err(
|error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")),
)?;
let content_length = u64::try_from(request.body.len())
.map_err(|_| OssError::InvalidRequest("上传对象大小超出可支持范围".to_string()))?;
let builder = signed_request_builder(
client,
&self.config,
Method::PUT,
Some(&object_key),
target_url,
content_type.as_deref(),
&metadata,
)?
.header(reqwest::header::CONTENT_LENGTH, content_length)
.body(request.body);
let response = builder
.send()
.await
.map_err(|error| OssError::Request(format!("请求 OSS 失败:{error}")))?;
response_status = Some(response.status().as_u16());
if !response.status().is_success() {
return Err(OssError::Request(format!(
"OSS PutObject 失败,状态码:{}",
response.status()
)));
}
let headers = response.headers();
let etag = headers
.get(reqwest::header::ETAG)
.and_then(|value| value.to_str().ok())
.map(|value| value.trim_matches('"').to_string());
let last_modified = headers
.get(reqwest::header::LAST_MODIFIED)
.and_then(|value| value.to_str().ok())
.map(|value| value.to_string());
Ok(OssPutObjectResponse {
provider: OSS_PROVIDER,
bucket: self.config.bucket.clone(),
endpoint: self.config.endpoint.clone(),
host: self.config.upload_host(),
legacy_public_path: format!("/{object_key}"),
object_key,
content_type,
content_length,
access: request.access,
etag,
last_modified,
})
}
.await;
match &result {
Ok(response) => info!(
provider = OSS_PROVIDER,
operation = "put_object",
bucket = %response.bucket,
endpoint = %response.endpoint,
object_key = %response.object_key,
access = oss_access_label(response.access),
status = response_status.unwrap_or(reqwest::StatusCode::OK.as_u16()),
status_class = http_status_class_from_option(response_status),
content_length = response.content_length,
content_type = %response.content_type.as_deref().unwrap_or(""),
etag_present = response.etag.is_some(),
last_modified_present = response.last_modified.is_some(),
elapsed_ms = elapsed_ms(started_at),
"OSS PutObject 上传完成"
),
Err(error) => warn!(
provider = OSS_PROVIDER,
operation = "put_object",
bucket = %self.config.bucket(),
endpoint = %self.config.endpoint(),
key_prefix = requested_prefix,
content_length = requested_content_length,
content_type = %requested_content_type,
metadata_count = requested_metadata_count,
status = response_status.unwrap_or_default(),
status_class = http_status_class_from_option(response_status),
error_kind = oss_error_kind_label(error),
message = %error,
elapsed_ms = elapsed_ms(started_at),
"OSS PutObject 上传失败"
),
} }
let sanitized_segments = request result
.path_segments
.iter()
.map(|segment| sanitize_path_segment(segment))
.filter(|segment| !segment.is_empty())
.collect::<Vec<_>>();
let file_name = sanitize_file_name(&request.file_name)?;
let object_key = build_object_key(request.prefix, &sanitized_segments, &file_name);
let content_type = normalize_optional_value(request.content_type);
let metadata = normalize_metadata(request.metadata)?;
let target_url = build_object_url(&self.config.bucket, &self.config.endpoint, &object_key)
.map_err(|error| OssError::Request(format!("构造 OSS 对象 URL 失败:{error}")))?;
let content_length = u64::try_from(request.body.len())
.map_err(|_| OssError::InvalidRequest("上传对象大小超出可支持范围".to_string()))?;
let builder = signed_request_builder(
client,
&self.config,
Method::PUT,
Some(&object_key),
target_url,
content_type.as_deref(),
&metadata,
)?
.header(reqwest::header::CONTENT_LENGTH, content_length)
.body(request.body);
let response = builder
.send()
.await
.map_err(|error| OssError::Request(format!("请求 OSS 失败:{error}")))?;
if !response.status().is_success() {
return Err(OssError::Request(format!(
"OSS PutObject 失败,状态码:{}",
response.status()
)));
}
let headers = response.headers();
let etag = headers
.get(reqwest::header::ETAG)
.and_then(|value| value.to_str().ok())
.map(|value| value.trim_matches('"').to_string());
let last_modified = headers
.get(reqwest::header::LAST_MODIFIED)
.and_then(|value| value.to_str().ok())
.map(|value| value.to_string());
Ok(OssPutObjectResponse {
provider: "aliyun-oss",
bucket: self.config.bucket.clone(),
endpoint: self.config.endpoint.clone(),
host: self.config.upload_host(),
legacy_public_path: format!("/{object_key}"),
object_key,
content_type,
content_length,
access: request.access,
etag,
last_modified,
})
} }
} }
@@ -717,6 +909,43 @@ impl OssError {
} }
} }
fn elapsed_ms(started_at: Instant) -> u64 {
started_at.elapsed().as_millis().min(u64::MAX as u128) as u64
}
fn oss_access_label(access: OssObjectAccess) -> &'static str {
match access {
OssObjectAccess::Public => "public",
OssObjectAccess::Private => "private",
}
}
fn oss_error_kind_label(error: &OssError) -> &'static str {
match error.kind() {
OssErrorKind::InvalidConfig => "invalid_config",
OssErrorKind::InvalidRequest => "invalid_request",
OssErrorKind::ObjectNotFound => "object_not_found",
OssErrorKind::Request => "request",
OssErrorKind::SerializePolicy => "serialize_policy",
OssErrorKind::Sign => "sign",
}
}
fn http_status_class_from_option(status: Option<u16>) -> &'static str {
status.map(http_status_class).unwrap_or("unknown")
}
fn http_status_class(status: u16) -> &'static str {
match status {
100..=199 => "1xx",
200..=299 => "2xx",
300..=399 => "3xx",
400..=499 => "4xx",
500..=599 => "5xx",
_ => "unknown",
}
}
fn build_policy_json( fn build_policy_json(
bucket: &str, bucket: &str,
object_key: &str, object_key: &str,
@@ -1295,6 +1524,18 @@ mod tests {
); );
} }
#[test]
fn structured_log_labels_are_stable() {
assert_eq!(
oss_error_kind_label(&OssError::InvalidRequest("bad input".to_string())),
"invalid_request"
);
assert_eq!(oss_access_label(OssObjectAccess::Private), "private");
assert_eq!(http_status_class(204), "2xx");
assert_eq!(http_status_class(404), "4xx");
assert_eq!(http_status_class_from_option(None), "unknown");
}
fn build_client() -> OssClient { fn build_client() -> OssClient {
OssClient::new( OssClient::new(
OssConfig::new( OssConfig::new(

View File

@@ -573,6 +573,7 @@ type BabyObjectMatchGenerationPhase = 'generating' | 'ready' | 'failed';
type RecommendRuntimeState = { type RecommendRuntimeState = {
activeKind: RecommendRuntimeKind | null; activeKind: RecommendRuntimeKind | null;
barkBattlePublishedConfig: BarkBattlePublishedConfig | null;
babyObjectMatchDraft: BabyObjectMatchDraft | null; babyObjectMatchDraft: BabyObjectMatchDraft | null;
bigFishRun: BigFishRuntimeSnapshotResponse | null; bigFishRun: BigFishRuntimeSnapshotResponse | null;
jumpHopRun: JumpHopRunResponse['run'] | null; jumpHopRun: JumpHopRunResponse['run'] | null;
@@ -758,7 +759,7 @@ function isRecommendRuntimeReadyForEntry(
return Boolean(state.visualNovelRun); return Boolean(state.visualNovelRun);
} }
if (expectedKind === 'bark-battle') { if (expectedKind === 'bark-battle') {
return true; return Boolean(state.barkBattlePublishedConfig);
} }
if (expectedKind === 'edutainment') { if (expectedKind === 'edutainment') {
return Boolean(state.babyObjectMatchDraft); return Boolean(state.babyObjectMatchDraft);
@@ -15050,6 +15051,29 @@ export function PlatformEntryFlowShellImpl({
isDesktopLayout, isDesktopLayout,
]); ]);
const activeRecommendEntry =
activeRecommendEntryKey && !isDesktopLayout
? (recommendRuntimeEntries.find(
(entry) =>
getPlatformPublicGalleryEntryKey(entry) ===
activeRecommendEntryKey,
) ?? null)
: null;
const isActiveRecommendRuntimeReady =
activeRecommendEntry !== null &&
isRecommendRuntimeReadyForEntry(activeRecommendEntry, {
activeKind: activeRecommendRuntimeKind,
barkBattlePublishedConfig,
babyObjectMatchDraft,
bigFishRun,
jumpHopRun,
match3dRun,
puzzleRun,
squareHoleRun,
visualNovelRun,
woodenFishRun,
});
useEffect(() => { useEffect(() => {
if ( if (
isDesktopLayout || isDesktopLayout ||
@@ -15067,25 +15091,6 @@ export function PlatformEntryFlowShellImpl({
return; return;
} }
const activeRecommendEntry = activeRecommendEntryKey
? (recommendRuntimeEntries.find(
(entry) =>
getPlatformPublicGalleryEntryKey(entry) === activeRecommendEntryKey,
) ?? null)
: null;
const isActiveRecommendRuntimeReady =
activeRecommendEntry !== null &&
isRecommendRuntimeReadyForEntry(activeRecommendEntry, {
activeKind: activeRecommendRuntimeKind,
babyObjectMatchDraft,
bigFishRun,
jumpHopRun,
match3dRun,
puzzleRun,
squareHoleRun,
visualNovelRun,
woodenFishRun,
});
if ( if (
(activeRecommendEntry !== null && isActiveRecommendRuntimeReady) || (activeRecommendEntry !== null && isActiveRecommendRuntimeReady) ||
isStartingRecommendEntry isStartingRecommendEntry
@@ -15101,9 +15106,12 @@ export function PlatformEntryFlowShellImpl({
}, [ }, [
activeRecommendEntryKey, activeRecommendEntryKey,
activeRecommendRuntimeKind, activeRecommendRuntimeKind,
activeRecommendEntry,
barkBattlePublishedConfig,
babyObjectMatchDraft, babyObjectMatchDraft,
bigFishRun, bigFishRun,
jumpHopRun, jumpHopRun,
isActiveRecommendRuntimeReady,
isStartingRecommendEntry, isStartingRecommendEntry,
match3dRun, match3dRun,
platformBootstrap.isLoadingPlatform, platformBootstrap.isLoadingPlatform,
@@ -16446,6 +16454,7 @@ export function PlatformEntryFlowShellImpl({
onOpenRecommendGalleryDetail={openRecommendGalleryDetail} onOpenRecommendGalleryDetail={openRecommendGalleryDetail}
recommendRuntimeContent={recommendRuntimeContent} recommendRuntimeContent={recommendRuntimeContent}
activeRecommendEntryKey={activeRecommendEntryKey} activeRecommendEntryKey={activeRecommendEntryKey}
isRecommendRuntimeReady={isActiveRecommendRuntimeReady}
isStartingRecommendEntry={ isStartingRecommendEntry={
isStartingRecommendEntry || isStartingRecommendEntry ||
isBigFishBusy || isBigFishBusy ||

View File

@@ -823,6 +823,7 @@ function renderLoggedOutHomeView(
| 'recommendRuntimeContent' | 'recommendRuntimeContent'
| 'activeRecommendEntryKey' | 'activeRecommendEntryKey'
| 'isStartingRecommendEntry' | 'isStartingRecommendEntry'
| 'isRecommendRuntimeReady'
| 'recommendRuntimeError' | 'recommendRuntimeError'
| 'onSelectNextRecommendEntry' | 'onSelectNextRecommendEntry'
| 'onSelectPreviousRecommendEntry' | 'onSelectPreviousRecommendEntry'
@@ -883,6 +884,7 @@ function renderLoggedOutHomeView(
} }
activeRecommendEntryKey={overrides.activeRecommendEntryKey} activeRecommendEntryKey={overrides.activeRecommendEntryKey}
isStartingRecommendEntry={overrides.isStartingRecommendEntry} isStartingRecommendEntry={overrides.isStartingRecommendEntry}
isRecommendRuntimeReady={overrides.isRecommendRuntimeReady}
recommendRuntimeError={overrides.recommendRuntimeError} recommendRuntimeError={overrides.recommendRuntimeError}
onSelectNextRecommendEntry={overrides.onSelectNextRecommendEntry} onSelectNextRecommendEntry={overrides.onSelectNextRecommendEntry}
onSelectPreviousRecommendEntry={ onSelectPreviousRecommendEntry={
@@ -3703,7 +3705,10 @@ test('logged out mobile recommend page renders runtime instead of cover', () =>
); );
expect(screen.getByTestId('recommend-runtime')).toBeTruthy(); expect(screen.getByTestId('recommend-runtime')).toBeTruthy();
expect(document.querySelector('.platform-recommend-cover-only')).toBeNull(); expect(
document.querySelector('.platform-recommend-runtime-cover'),
).toBeTruthy();
expect(screen.queryByText('加载中...')).toBeNull();
expect( expect(
document.querySelector('.platform-public-work-card__cover'), document.querySelector('.platform-public-work-card__cover'),
).toBeNull(); ).toBeNull();
@@ -3712,7 +3717,7 @@ test('logged out mobile recommend page renders runtime instead of cover', () =>
expect(onOpenGalleryDetail).not.toHaveBeenCalled(); expect(onOpenGalleryDetail).not.toHaveBeenCalled();
}); });
test('mobile recommend loading state is themed instead of hardcoded black', () => { test('mobile recommend startup keeps cover visible without loading copy', () => {
renderLoggedOutHomeView(vi.fn(), { renderLoggedOutHomeView(vi.fn(), {
latestEntries: [puzzlePublicEntry], latestEntries: [puzzlePublicEntry],
activeRecommendEntryKey: 'puzzle:user-2:puzzle-profile-public-1', activeRecommendEntryKey: 'puzzle:user-2:puzzle-profile-public-1',
@@ -3720,8 +3725,123 @@ test('mobile recommend loading state is themed instead of hardcoded black', () =
recommendRuntimeContent: null, recommendRuntimeContent: null,
}); });
expect(document.querySelector('.platform-recommend-cover-only')).toBeNull(); expect(
expect(screen.getByText('加载中...')).toBeTruthy(); document.querySelector('.platform-recommend-runtime-cover'),
).toBeTruthy();
expect(screen.queryByText('加载中...')).toBeNull();
expect(screen.getAllByText('奇幻拼图').length).toBeGreaterThan(0);
});
test('mobile recommend next level keeps runtime visual stable when active work changes', async () => {
const animationCallbacks: FrameRequestCallback[] = [];
Object.defineProperty(window, 'requestAnimationFrame', {
configurable: true,
writable: true,
value: vi.fn((callback: FrameRequestCallback) => {
animationCallbacks.push(callback);
return animationCallbacks.length;
}),
});
Object.defineProperty(window, 'cancelAnimationFrame', {
configurable: true,
writable: true,
value: vi.fn(),
});
const firstEntry = {
...puzzlePublicEntry,
workId: 'puzzle-work-feed-1',
profileId: 'puzzle-profile-feed-1',
ownerUserId: 'user-feed-1',
publicWorkCode: 'PZ-FEED1',
worldName: '当前拼图',
coverImageSrc: 'current-cover.png',
} satisfies PlatformPublicGalleryCard;
const similarEntry = {
...puzzlePublicEntry,
workId: 'puzzle-work-similar-1',
profileId: 'puzzle-profile-similar-1',
ownerUserId: 'user-feed-2',
publicWorkCode: 'PZ-SIMILAR1',
worldName: '相似拼图',
coverImageSrc: 'similar-cover.png',
} satisfies PlatformPublicGalleryCard;
const { rerender } = renderLoggedOutHomeView(vi.fn(), {
latestEntries: [firstEntry, similarEntry],
activeRecommendEntryKey: 'puzzle:user-feed-1:puzzle-profile-feed-1',
isRecommendRuntimeReady: true,
});
act(() => {
animationCallbacks.splice(0).forEach((callback) => callback(16));
});
await waitFor(() => {
expect(
document.querySelector('.platform-recommend-runtime-cover')?.className,
).toContain('platform-recommend-runtime-cover--hidden');
});
rerender(
<AuthUiContext.Provider
value={{
user: null,
canAccessProtectedData: false,
openLoginModal: vi.fn(),
requireAuth: vi.fn(),
openSettingsModal: vi.fn(),
openAccountModal: vi.fn(),
setCurrentUser: vi.fn(),
logout: vi.fn(async () => undefined),
musicVolume: 0.42,
setMusicVolume: vi.fn(),
platformTheme: 'light',
setPlatformTheme: vi.fn(),
isHydratingSettings: false,
isPersistingSettings: false,
settingsError: null,
}}
>
<RpgEntryHomeView
activeTab="home"
isDesktopLayout={false}
onTabChange={vi.fn()}
hasSavedGame={false}
savedSnapshot={null}
saveEntries={[]}
saveError={null}
featuredEntries={[]}
latestEntries={[firstEntry, similarEntry]}
myEntries={[]}
historyEntries={[]}
profileDashboard={null}
isLoadingPlatform={false}
isLoadingDashboard={false}
isResumingSaveWorldKey={null}
platformError={null}
dashboardError={null}
onContinueGame={vi.fn()}
onResumeSave={vi.fn()}
onOpenCreateWorld={vi.fn()}
onOpenCreateTypePicker={vi.fn()}
onOpenGalleryDetail={vi.fn()}
recommendRuntimeContent={<div data-testid="recommend-runtime" />}
activeRecommendEntryKey="puzzle:user-feed-2:puzzle-profile-similar-1"
isRecommendRuntimeReady
onOpenLibraryDetail={vi.fn()}
onSearchPublicCode={vi.fn()}
/>
</AuthUiContext.Provider>,
);
const rail = document.querySelector(
'.platform-recommend-swipe-rail',
) as HTMLElement | null;
expect(rail?.className).toContain('platform-recommend-swipe-rail--settled');
expect(rail?.style.transform).toBe('translate3d(0, 0px, 0)');
expect(screen.getByLabelText('相似拼图 作品信息')).toBeTruthy();
expect(
document.querySelector('.platform-recommend-runtime-cover')?.className,
).toContain('platform-recommend-runtime-cover--hidden');
}); });
test('logged in recommend runtime preloads adjacent work previews and drag switches like video feed', () => { test('logged in recommend runtime preloads adjacent work previews and drag switches like video feed', () => {

View File

@@ -39,6 +39,7 @@ import {
type CSSProperties, type CSSProperties,
type PointerEvent, type PointerEvent,
type ReactNode, type ReactNode,
Suspense,
useCallback, useCallback,
useEffect, useEffect,
useMemo, useMemo,
@@ -195,6 +196,7 @@ export interface RpgEntryHomeViewProps {
recommendRuntimeContent?: ReactNode; recommendRuntimeContent?: ReactNode;
activeRecommendEntryKey?: string | null; activeRecommendEntryKey?: string | null;
isStartingRecommendEntry?: boolean; isStartingRecommendEntry?: boolean;
isRecommendRuntimeReady?: boolean;
recommendRuntimeError?: string | null; recommendRuntimeError?: string | null;
onSelectNextRecommendEntry?: (activeEntryKey?: string | null) => void; onSelectNextRecommendEntry?: (activeEntryKey?: string | null) => void;
onSelectPreviousRecommendEntry?: (activeEntryKey?: string | null) => void; onSelectPreviousRecommendEntry?: (activeEntryKey?: string | null) => void;
@@ -946,6 +948,115 @@ function RecommendRuntimePreviewCard({
); );
} }
function RecommendRuntimeCover({
entry,
className = '',
}: {
entry: PlatformPublicGalleryCard;
className?: string;
}) {
const coverImage = resolvePlatformWorldCoverImage(entry);
const fallbackCoverImage = resolvePlatformWorldFallbackCoverImage(entry);
return (
<div
className={`platform-recommend-runtime-cover ${className}`}
aria-hidden="true"
>
{coverImage || fallbackCoverImage ? (
<PlatformWorkCoverArtwork
entry={entry}
imageSrc={coverImage}
fallbackSrc={fallbackCoverImage}
alt=""
className="absolute inset-0 h-full w-full object-cover"
/>
) : (
<div className="absolute inset-0 bg-[radial-gradient(circle_at_22%_18%,rgba(255,255,255,0.28),transparent_30%),linear-gradient(135deg,rgba(255,118,117,0.42),rgba(89,164,255,0.34))]" />
)}
<div className="absolute inset-0 bg-[linear-gradient(180deg,rgba(0,0,0,0.05),rgba(0,0,0,0.34))]" />
</div>
);
}
function RecommendRuntimeMountedProbe({
onMounted,
}: {
onMounted: () => void;
}) {
useEffect(() => {
const animationFrameId = window.requestAnimationFrame(onMounted);
return () => window.cancelAnimationFrame(animationFrameId);
}, [onMounted]);
return null;
}
function RecommendRuntimeVisual({
entry,
runtimeContent,
isStarting,
isRuntimeReady,
}: {
entry: PlatformPublicGalleryCard;
runtimeContent?: ReactNode;
isStarting: boolean;
isRuntimeReady: boolean;
}) {
const [isRuntimeMounted, setIsRuntimeMounted] = useState(false);
const activeEntryKey = buildPublicGalleryCardKey(entry);
const previousEntryKeyRef = useRef(activeEntryKey);
useEffect(() => {
if (previousEntryKeyRef.current === activeEntryKey) {
return;
}
previousEntryKeyRef.current = activeEntryKey;
setIsRuntimeMounted((currentValue) => {
// 中文注释:拼图推荐流“下一关”会在同一个 run 内切到相似作品;
// 此时只更新作品信息和分享基准,不应重显封面造成运行态闪跳。
if (currentValue && !isStarting && isRuntimeReady) {
return currentValue;
}
return false;
});
}, [activeEntryKey, isRuntimeReady, isStarting]);
const handleRuntimeMounted = useCallback(() => {
if (!isStarting && isRuntimeReady) {
setIsRuntimeMounted(true);
}
}, [isRuntimeReady, isStarting]);
const shouldShowCover =
!runtimeContent || isStarting || !isRuntimeReady || !isRuntimeMounted;
return (
<div className="platform-recommend-runtime-visual">
{runtimeContent ? (
<Suspense fallback={null}>
<div
className="platform-recommend-runtime-viewport"
aria-hidden={shouldShowCover}
>
{runtimeContent}
</div>
<RecommendRuntimeMountedProbe
key={activeEntryKey}
onMounted={handleRuntimeMounted}
/>
</Suspense>
) : null}
<RecommendRuntimeCover
entry={entry}
className={
shouldShowCover ? '' : 'platform-recommend-runtime-cover--hidden'
}
/>
</div>
);
}
function RecommendSwipeCard({ function RecommendSwipeCard({
entry, entry,
authorAvatarUrl, authorAvatarUrl,
@@ -4023,6 +4134,7 @@ export function RpgEntryHomeView({
recommendRuntimeContent, recommendRuntimeContent,
activeRecommendEntryKey = null, activeRecommendEntryKey = null,
isStartingRecommendEntry = false, isStartingRecommendEntry = false,
isRecommendRuntimeReady = false,
recommendRuntimeError = null, recommendRuntimeError = null,
onSelectNextRecommendEntry, onSelectNextRecommendEntry,
onSelectPreviousRecommendEntry, onSelectPreviousRecommendEntry,
@@ -5687,10 +5799,6 @@ export function RpgEntryHomeView({
{recommendRuntimeError} {recommendRuntimeError}
</button> </button>
</section> </section>
) : isStartingRecommendEntry ? (
<section className="platform-recommend-runtime-panel">
<div className="platform-recommend-runtime-state">...</div>
</section>
) : activeRecommendEntry ? ( ) : activeRecommendEntry ? (
<div <div
ref={recommendCardStageRef} ref={recommendCardStageRef}
@@ -5732,9 +5840,12 @@ export function RpgEntryHomeView({
)} )}
isActive isActive
visual={ visual={
<div className="platform-recommend-runtime-viewport"> <RecommendRuntimeVisual
{recommendRuntimeContent} entry={activeRecommendEntry}
</div> runtimeContent={recommendRuntimeContent}
isStarting={isStartingRecommendEntry}
isRuntimeReady={isRecommendRuntimeReady}
/>
} }
onDragPointerDown={beginRecommendDrag} onDragPointerDown={beginRecommendDrag}
onDragPointerMove={moveRecommendDrag} onDragPointerMove={moveRecommendDrag}

View File

@@ -237,7 +237,7 @@ test('resolves public work author from display name and public user code before
expect(resolvePlatformWorkAuthorDisplayName(card, null)).toBe('敲木鱼玩家'); expect(resolvePlatformWorkAuthorDisplayName(card, null)).toBe('敲木鱼玩家');
}); });
test('public work author display hides phone masks and public user codes on cards', () => { test('public work author display keeps phone masks and hides bare public user codes on cards', () => {
const card = mapWoodenFishWorkToPlatformGalleryCard({ const card = mapWoodenFishWorkToPlatformGalleryCard({
publicWorkCode: 'WF-AUTHOR2', publicWorkCode: 'WF-AUTHOR2',
workId: 'wooden-fish-work-author-mask', workId: 'wooden-fish-work-author-mask',
@@ -263,8 +263,18 @@ test('public work author display hides phone masks and public user codes on card
displayName: '158****3533', displayName: '158****3533',
avatarUrl: null, avatarUrl: null,
}), }),
).toBe('玩家'); ).toBe('158****3533');
expect(resolvePlatformWorkAuthorDisplayName(card, null)).toBe('玩家'); expect(resolvePlatformWorkAuthorDisplayName(card, null)).toBe(
'158****3533 · SY-00000003',
);
const publicCodeOnlyCard = {
...card,
authorDisplayName: 'SY-00000003',
};
expect(resolvePlatformWorkAuthorDisplayName(publicCodeOnlyCard, null)).toBe(
'玩家',
);
}); });
test('keeps baby object match public card code and template label intact', () => { test('keeps baby object match public card code and template label intact', () => {

View File

@@ -884,9 +884,6 @@ function normalizePlatformPublicAuthorName(value: string | null | undefined) {
} }
const compact = normalized.replace(/\s+/gu, ''); const compact = normalized.replace(/\s+/gu, '');
if (/^\d+\*+\d+(?:[·.-]?SY-\d+)?$/iu.test(compact)) {
return '';
}
if (/^SY-\d+$/iu.test(compact)) { if (/^SY-\d+$/iu.test(compact)) {
return ''; return '';
} }

View File

@@ -4831,6 +4831,31 @@ html[data-mobile-keyboard-open='true'] .platform-mobile-bottom-dock {
pointer-events: auto; pointer-events: auto;
} }
.platform-recommend-runtime-visual {
position: absolute;
inset: 0;
min-width: 0;
overflow: hidden;
background: var(--platform-recommend-runtime-fill);
}
.platform-recommend-runtime-cover {
position: absolute;
inset: 0;
z-index: 3;
overflow: hidden;
background: var(--platform-recommend-runtime-fill);
opacity: 1;
pointer-events: auto;
transition: opacity 420ms ease;
will-change: opacity;
}
.platform-recommend-runtime-cover--hidden {
opacity: 0;
pointer-events: none;
}
.platform-recommend-swipe-stage { .platform-recommend-swipe-stage {
position: relative; position: relative;
flex: 1 1 auto; flex: 1 1 auto;

View File

@@ -132,7 +132,37 @@ describe('appPageRoutes', () => {
expect(window.location.pathname).toBe('/creation/rpg/result'); expect(window.location.pathname).toBe('/creation/rpg/result');
expect(window.location.search).toBe( expect(window.location.search).toBe(
'?sessionId=session-1&profileId=profile-1&draftId=draft-1&workId=work-1', '?sessionId=session-1&profileId=profile-1&draftId=draft-1&workId=work-1&clientRuntime=wechat_mini_program',
);
});
it('preserves mini program runtime context while normalizing app paths', () => {
window.history.replaceState(
null,
'',
'/?clientType=mini_program&clientRuntime=wechat_mini_program&miniProgramEnv=trial',
);
pushAppHistoryPath('/');
expect(window.location.pathname).toBe('/');
expect(window.location.search).toBe(
'?clientType=mini_program&clientRuntime=wechat_mini_program&miniProgramEnv=trial',
);
});
it('keeps mini program runtime context when navigating to explicit query routes', () => {
window.history.replaceState(
null,
'',
'/?clientRuntime=wechat_mini_program',
);
pushAppHistoryPath('/works/detail?work=PZ-7A7B18D9');
expect(window.location.pathname).toBe('/works/detail');
expect(window.location.search).toBe(
'?work=PZ-7A7B18D9&clientRuntime=wechat_mini_program',
); );
}); });

View File

@@ -72,6 +72,12 @@ const ROUTE_STAGE_BY_PATH = new Map(
STAGE_ROUTE_ENTRIES.map(([stage, path]) => [path, stage] as const), STAGE_ROUTE_ENTRIES.map(([stage, path]) => [path, stage] as const),
) as Map<string, SelectionStage>; ) as Map<string, SelectionStage>;
const APP_RUNTIME_CONTEXT_QUERY_KEYS = [
'clientType',
'clientRuntime',
'miniProgramEnv',
] as const;
export function normalizeAppPath(pathname: string) { export function normalizeAppPath(pathname: string) {
const trimmedPathname = pathname.trim().toLowerCase(); const trimmedPathname = pathname.trim().toLowerCase();
@@ -135,13 +141,12 @@ export function isKnownMainAppPagePath(pathname: string) {
export function pushAppHistoryPath(path: string) { export function pushAppHistoryPath(path: string) {
const nextUrl = new URL(path, window.location.origin); const nextUrl = new URL(path, window.location.origin);
const normalizedPath = normalizeAppPath(nextUrl.pathname); const normalizedPath = normalizeAppPath(nextUrl.pathname);
const nextSearch = const nextSearch = buildPreservedAppSearch(
nextUrl.search || nextUrl.search,
buildPreservedAppSearch( window.location.pathname,
window.location.pathname, normalizedPath,
normalizedPath, window.location.search,
window.location.search, );
);
const nextRelativeUrl = `${normalizedPath}${nextSearch}`; const nextRelativeUrl = `${normalizedPath}${nextSearch}`;
const currentRelativeUrl = `${normalizeAppPath(window.location.pathname)}${window.location.search}`; const currentRelativeUrl = `${normalizeAppPath(window.location.pathname)}${window.location.search}`;
if (currentRelativeUrl === nextRelativeUrl) { if (currentRelativeUrl === nextRelativeUrl) {
@@ -153,16 +158,39 @@ export function pushAppHistoryPath(path: string) {
} }
function buildPreservedAppSearch( function buildPreservedAppSearch(
explicitNextSearch: string,
currentPathname: string, currentPathname: string,
normalizedPath: string, normalizedPath: string,
search: string, search: string,
) { ) {
const preservedParams = new URLSearchParams(explicitNextSearch);
if ( if (
!isCreationRestorePath(normalizedPath) || !explicitNextSearch &&
!isSameCreationFlowPath(currentPathname, normalizedPath) isCreationRestorePath(normalizedPath) &&
isSameCreationFlowPath(currentPathname, normalizedPath)
) { ) {
return ''; const creationParams = new URLSearchParams(
buildCreationUrlSearchFromParams(search),
);
creationParams.forEach((value, key) => {
preservedParams.set(key, value);
});
} }
return buildCreationUrlSearchFromParams(search); const currentParams = new URLSearchParams(search);
// 中文注释:小程序 WebView 依赖这些宿主上下文判断登录和充值通道,不能被前端阶段导航清掉。
APP_RUNTIME_CONTEXT_QUERY_KEYS.forEach((key) => {
if (preservedParams.has(key)) {
return;
}
const value = currentParams.get(key)?.trim();
if (value) {
preservedParams.set(key, value);
}
});
const queryString = preservedParams.toString();
return queryString ? `?${queryString}` : '';
} }

View File

@@ -31,6 +31,7 @@ import {
getCurrentAuthUser, getCurrentAuthUser,
getPublicAuthUserById, getPublicAuthUserById,
liftAuthRiskBlock, liftAuthRiskBlock,
isWechatMiniProgramWebViewRuntime,
loginWithPhoneCode, loginWithPhoneCode,
logoutAllAuthSessions, logoutAllAuthSessions,
redeemRegistrationInviteCode, redeemRegistrationInviteCode,
@@ -80,6 +81,7 @@ function createWindowMock(overrides: Record<string, unknown> = {}) {
describe('authService', () => { describe('authService', () => {
beforeEach(() => { beforeEach(() => {
vi.unstubAllGlobals();
vi.clearAllMocks(); vi.clearAllMocks();
vi.stubGlobal('window', createWindowMock()); vi.stubGlobal('window', createWindowMock());
clearStoredAccessToken({ emit: false }); clearStoredAccessToken({ emit: false });
@@ -428,6 +430,26 @@ describe('authService', () => {
}); });
}); });
it('detects mini program user agent before the WeChat bridge is ready', () => {
vi.stubGlobal('navigator', {
userAgent:
'Mozilla/5.0 iPhone MicroMessenger/8.0.49 NetType/WIFI Language/zh_CN miniProgram',
});
vi.stubGlobal(
'window',
createWindowMock({
location: {
pathname: '/',
hash: '',
search: '',
assign: vi.fn(),
},
}),
);
expect(isWechatMiniProgramWebViewRuntime()).toBe(true);
});
it('waits for an existing WeChat JS SDK script before opening the native auth page', async () => { it('waits for an existing WeChat JS SDK script before opening the native auth page', async () => {
const navigateTo = vi.fn((options: { url: string; success?: () => void }) => { const navigateTo = vi.fn((options: { url: string; success?: () => void }) => {
options.success?.(); options.success?.();

View File

@@ -90,9 +90,15 @@ export function isWechatMiniProgramWebViewRuntime() {
} }
const params = new URLSearchParams(window.location.search || ''); const params = new URLSearchParams(window.location.search || '');
const userAgent =
typeof navigator === 'undefined' ? '' : navigator.userAgent || '';
const normalizedUserAgent = userAgent.toLowerCase();
return ( return (
params.get('clientRuntime') === 'wechat_mini_program' || params.get('clientRuntime') === 'wechat_mini_program' ||
params.get('clientType') === 'mini_program' || params.get('clientType') === 'mini_program' ||
(normalizedUserAgent.includes('micromessenger') &&
normalizedUserAgent.includes('miniprogram')) ||
Boolean(window.wx?.miniProgram?.postMessage) Boolean(window.wx?.miniProgram?.postMessage)
); );
} }