Merge branch 'master' into release

# Conflicts:
#	jenkins/Jenkinsfile.production-server-provision
This commit is contained in:
2026-05-19 22:40:44 +08:00
15 changed files with 521 additions and 35409 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 514 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 510 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.8 KiB

View File

@@ -1,4 +0,0 @@
Set-Location 'C:\Genarrative'
$env:RUST_SERVER_TARGET = 'http://127.0.0.1:8082'
$env:GENARRATIVE_RUNTIME_SERVER_TARGET = 'http://127.0.0.1:8082'
npm.cmd run dev:web *> 'C:\Genarrative\.codex\logs\dev-web-final.out.log'

View File

@@ -1 +0,0 @@
../../.hermes/skills/behavior-driven-development/

File diff suppressed because it is too large Load Diff

6
.gitignore vendored
View File

@@ -5,10 +5,16 @@ coverage/
.DS_Store
*.log
/.codex-logs/
/.codex/logs/
/.codex/run-logs/
/.codex/tmp-schema.json
/.codex-home-*.png
/.codex-cargo-home-*/
/.codex-cache*/
/.tmp*/
/.idea/
/bash.exe.stackdump
/test-code.ps1
.preview.*
tmp_*
tmp/

View File

@@ -87,12 +87,14 @@
- 验证方式:执行 `npm run container:config` 展开 compose 配置;需要真实运行时再执行 `npm run container:build``npm run container:up``npm run container:k6`,并结合容器 Nginx log 与 OTLP debug exporter 判断瓶颈。
- 关联文档:`deploy/container/README.md``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-05-18 生产 provision 改为构建机准备工具包再上传安装
## 2026-05-19 生产 provision 改为 Windows 下载包后由目标机本地安装
- 背景:目标 release 服务器无法访问 GitHub之前的 server provision 默认仍假设 `spacetime``otelcol-contrib` 已经存在于目标机本地路径,和真实运维条件不符
- 决策:Jenkins 新增 `Prepare Provision Tools` 阶段,在 `linux && genarrative-build` 构建机执行 `scripts/prepare-server-provision-tools.sh`,通过官方 SpacetimeDB 安装入口和 OpenTelemetry release 包生成 `provision-tools/`再用 `stash/unstash` 带到 release 部署 agent`scripts/jenkins-server-provision.sh` 只从工作区工具包复制安装,不再要求目标机自己下载或预装二进制
- 背景:当前 `development` provision 目标实际就是 Linux agent `genarrative-build-01`,之前把 `Prepare Provision Tools` 放在 `linux && genarrative-build` 会让目标机自己连 GitHub 和 `install.spacetimedb.com`违背“Windows 本机先下载再传到目标机”的运维要求
- 决策:`Genarrative-Server-Provision` 拆成 Windows 下载阶段和 Linux 目标机安装阶段。Windows 节点的 `Download Provision Tool Archives` 只下载 `spacetime-x86_64-unknown-linux-gnu.tar.gz``otelcol-contrib_0.151.0_linux_amd64.tar.gz`通过 `stash/unstash` 传到目标 Linux 节点;目标机执行 `scripts/prepare-server-provision-tools.sh` 时设置 `PROVISION_REQUIRE_LOCAL_DOWNLOADS=true`,只消费已下载件生成 `provision-tools/`,缺包直接失败,不回退外网下载
- 追加决策Server-Provision 的 Windows helper 不再对 Jenkins `writeFile` 刚写出的 `.ps1` 做原地 UTF-8 BOM 重写,而是由显式 `powershell.exe` 按 UTF-8 读入脚本文本,并用 `ScriptBlock::Create(...)` 在内存中执行;这样既保留中文脚本内容,又避免同一个 workspace 脚本被立即重写时触发 `拒绝访问`
- 追加决策GitHub release asset 的可用校验信息使用 `digest` 字段,实际是 `sha256:...`,不是 MD5Windows 下载阶段先查 digest再决定是否复用已有文件。
- 影响范围:`jenkins/Jenkinsfile.production-server-provision``scripts/prepare-server-provision-tools.sh``scripts/jenkins-server-provision.sh`、生产运维文档。
- 验证方式Jenkins 构建机可完成工具包准备release 部署 agent 只消费工作区文件;目标机不再依赖 GitHub 外网下载
- 验证方式Jenkins 日志应先出现 Windows 节点的 `[jenkins-powershell] workspace:``[jenkins-powershell] loaded bytes:``[prepare-provision-downloads]` 下载日志,再在 `genarrative-build-01` 上出现“使用已下载的 ...”日志;目标机不应出现直接访问 `install.spacetimedb.com` 或 OpenTelemetry GitHub release 下载地址的回退日志,且不再需要 `spacetimedb-update-*` 作为离线交付包
- 关联文档:`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-05-19 公开 gallery 入口发布限流以快拒绝保护后端
@@ -621,6 +623,14 @@
- 验证方式server provision 跑过后,目标机应同时具备 Brotli 模块包与 `nginx -t` 可接受的 brotli 指令;再由 Nginx 模板启用对应指令。
- 关联文档:`deploy/nginx/README.md``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-05-19 server provision 下载件固定由 Windows 节点断点续传
- 背景:`SpacetimeDB``otelcol-contrib` release 资产在 Linux 目标机直接下载很慢;改到 Windows Jenkins 节点下载后GitHub 大文件仍可能出现 `curl: (18)` 响应体截断。
- 决策:`Genarrative-Server-Provision``Download Provision Tool Archives` 阶段继续只在 Windows 节点下载,再通过 `stash/unstash` 交给目标 Linux agent下载前查 GitHub release asset `digest`,本地最终文件 SHA256 命中即跳过,`.download` 临时文件用于 `curl -C -` 断点续传,完整返回但 digest 不匹配才清理重下。
- 影响范围:`jenkins/Jenkinsfile.production-server-provision`、目标机 `scripts/prepare-server-provision-tools.sh` 的本地下载件消费路径、生产 provision 运维排障。
- 验证方式Windows 下载日志应出现 digest 查询、已存在校验跳过或 `curl 断点续传`Linux 目标机阶段只使用 `provision-tool-downloads/` 中的 tarball不访问 GitHub 下载地址。
- 关联文档:`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 个人任务与埋点首版边界冻结
- 背景“我的”Tab、任务、奖励、钱包和埋点涉及用户、运营、分析多条链路需要避免范围泛化。

View File

@@ -22,6 +22,14 @@
- 验证:拼图入口测试仍可通过,且新组件可通过不同页面复用而不需要复制上传卡实现。
- 关联:`src/components/common/CreativeImageInputPanel.tsx``src/components/puzzle-agent/PuzzleAgentWorkspace.tsx`
## Windows provision 下载截断要断点续传而不是回退目标机下载
- 现象:`Genarrative-Server-Provision``Download Provision Tool Archives` 阶段出现 `curl: (18) end of response ... bytes missing`,常见于 `otelcol-contrib_0.151.0_linux_amd64.tar.gz` 等 GitHub release 大文件。
- 原因:这是 Windows Jenkins 节点到 GitHub 的响应体被截断;若每轮都删除 `.download` 临时文件,就会丢掉已下载部分,下一次又从头开始。
- 处理Windows 下载函数保留 `${Output}.download``curl` 失败时下一轮使用 `-C -` 断点续传;最终只以 GitHub release asset 的 SHA256 `digest` 作为放行条件,完整返回但 digest 不匹配才删除临时文件重新下载。不要把 SpacetimeDB 或 `otelcol-contrib` 下载挪回 Linux 目标机。
- 验证:日志应显示 `curl 断点续传 ... resumeBytes=...`,最终出现 `已下载 ... bytes=...`;目标 Linux 阶段只消费 `stash/unstash` 带过去的下载件。
- 关联:`jenkins/Jenkinsfile.production-server-provision``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## OTLP 端点只填 Collector HTTP base endpoint
- 现象:生产或容器 env 里把 `OTEL_EXPORTER_OTLP_ENDPOINT` 填成 `4317`、Rider 端口或别的非 HTTP base endpoint 后api-server 发不出 OTLP或者链路被错误转发。
@@ -736,6 +744,22 @@
- 验证:扫描 `jenkins/Jenkinsfile.production-database-export``jenkins/Jenkinsfile.production-database-import`,确认 `INCLUDE_TABLES``CHUNK_SIZE``SERVER_BACKUP_DIRECTORY``SMOKE_HEALTH_URL` 等可选参数不再裸读。
- 关联:`docs/technical/PRODUCTION_DEPLOYMENT_PLAN_2026-05-02.md``jenkins/Jenkinsfile.production-database-export``jenkins/Jenkinsfile.production-database-import`
## Jenkins 二次 checkout 后脚本执行位会被 Git 还原
- 现象:`Genarrative-Server-Provision` 已在 shell 块前面对脚本执行 `chmod +x`,但进入 `Prepare Provision Tools` 后仍报 `scripts/prepare-server-provision-tools.sh: Permission denied` / `exit code 126`
- 原因:该阶段会先运行 `scripts/jenkins-checkout-source.sh`,脚本内部执行 `git reset --hard HEAD``git clean -fd`,会把前面临时 `chmod` 的执行位还原为 Git 记录的 mode若被直接执行的脚本在仓库里是 `100644`,二次 checkout 后仍不可执行。
- 处理:需要直接以 `scripts/*.sh` 方式执行的 Jenkins 脚本应提交为 Git `100755`;如果只想临时授权,必须放在 `scripts/jenkins-checkout-source.sh` 完成之后。
- 验证:运行 `git ls-files --stage scripts/prepare-server-provision-tools.sh`,确认 mode 为 `100755`;重新跑 `Genarrative-Server-Provision` 时应进入工具下载/打包日志,而不是停在 `Permission denied`
- 关联:`jenkins/Jenkinsfile.production-server-provision``scripts/prepare-server-provision-tools.sh``scripts/jenkins-checkout-source.sh``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## Server-Provision 下载阶段不要放回 genarrative-build-01
- 现象:`Genarrative-Server-Provision` 日志里 `Prepare Provision Tools` 显示 `Running on genarrative-build-01 in /root/...`,随后在该节点上下载 GitHub release 或 `install.spacetimedb.com` 失败。
- 原因:`genarrative-build-01` 在当前 provision 流程里是 Linux 目标发布机/目标 agent不是用户本地 Windows 下载环境;把下载阶段放在 `linux && genarrative-build` 等于让目标机自己外连。
- 处理:下载必须发生在 Jenkins `windows` 节点的 `Download Provision Tool Archives` 阶段,先下载 SpacetimeDB Linux release tarball 和 `otelcol-contrib` Linux amd64 包,再 `stash/unstash` 到目标 Linux 节点。目标机执行 `scripts/prepare-server-provision-tools.sh` 时设置 `PROVISION_REQUIRE_LOCAL_DOWNLOADS=true`,缺少下载件直接失败,不回退联网下载。
- 验证Jenkins 日志应先出现 `Running on ... windows``[prepare-provision-downloads] 下载 ...`,目标节点只出现 `[prepare-provision-tools] 使用已下载的 ...`;如果目标节点出现 `下载 otelcol-contrib:``下载 SpacetimeDB 官方安装器脚本:`,说明又回退到错误路径。
- 关联:`jenkins/Jenkinsfile.production-server-provision``scripts/prepare-server-provision-tools.sh``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 个人任务 scope 不得扩成 work/site/module
- 现象:个人任务配置为 `work` / `site` / `module` 后进度串桶或静默按 0 处理。
@@ -1016,6 +1040,22 @@
- 验证:检查 Jenkins build log 中是否出现 `[jenkins-powershell] user:``[jenkins-powershell] exe:`,以及 `[stdb-checkout] current HEAD:`。上游 Full Build 传下来的 `COMMIT_HASH` 若已等于当前 GitSCM checkout日志应显示 `requested commit already matches Jenkins GitSCM checkout` 并继续进入构建阶段;同时确认 `builds/<n>/log` 不再停在 `PipelineNodeTreeScanner... Cannot run program "powershell"` 或 Checkout 内部 exit code 5。
- 关联:`jenkins/Jenkinsfile.production-stdb-module-build``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## Server-Provision Windows 下载 helper 不要原地重写临时 ps1
- 现象:`Genarrative-Server-Provision` 的 Windows 下载阶段已经打印了 `[jenkins-powershell] user:``[jenkins-powershell] exe:`,但在 `.ps1` 原地 BOM 重写前后仍然返回 `exit code 5` / `拒绝访问`,且下载目录还没创建。
- 原因Jenkins `writeFile` 生成的临时 `.ps1` 正被同一个 workspace 里的 PowerShell 进程马上重写成 BOM 文件,这个原地改写在本地 Windows Jenkins 环境里比直接脚本执行更容易碰到 workspace 占用或 ACL 拒绝。对这条流水线来说BOM 不是必须的执行条件。
- 处理:`runWindowsPowerShell(...)` 改成先 `writeFile`,再由显式 `powershell.exe` 读取脚本文本并用 `ScriptBlock::Create(...)` 直接在内存中执行,不再对同一个 `.ps1` 做 BOM 重写。Windows 下载脚本里先把 `PROVISION_DOWNLOADS_DIR` 归一到 workspace 绝对路径,并补 `Windows workspace` / `download dir` / `已创建下载目录` 三段日志,方便区分是路径问题还是下载问题。
- 验证Jenkins log 应先出现 `[jenkins-powershell] workspace:``[jenkins-powershell] loaded bytes:`,再出现 `[prepare-provision-downloads] Windows workspace:``[prepare-provision-downloads] 已创建下载目录:`;如果下载 URL 故意指到不可达地址,应该只在 `curl 下载失败` 处结束,而不是卡在 BOM 重写前。
- 关联:`jenkins/Jenkinsfile.production-server-provision``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## SpacetimeDB update installer 不要按带 host 后缀的下载文件名执行
- 现象Server-Provision 目标机阶段已经显示“使用已下载的 SpacetimeDB Linux update installer”随后报 `Error: unexpected argument '-y' found` 或前置 `unknown command name for spacetimedb-update multicall binary`
- 原因:`spacetimedb-update-*` 不是当前离线交付的最终形态GitHub release 页面真正可比较的缓存对象是 `spacetime-x86_64-unknown-linux-gnu.tar.gz` 这种 release tarballGitHub release asset API 暴露的是 `digest` / SHA256不是 MD5。
- 处理Windows 下载阶段应直接缓存 release tarball 和 `otelcol-contrib_0.151.0_linux_amd64.tar.gz`,目标机 `scripts/prepare-server-provision-tools.sh` 只解压本地 tarball 生成 `bin/current/spacetimedb-cli``bin/current/spacetimedb-standalone`,不要再把 update installer 当成最终离线包执行。
- 验证Jenkins 目标机日志不再出现 `unexpected argument '-y'``unknown command name for spacetimedb-update multicall binary`,后续应继续检查 `bin/current/spacetimedb-cli``bin/current/spacetimedb-standalone` 是否生成。
- 关联:`scripts/prepare-server-provision-tools.sh``jenkins/Jenkinsfile.production-server-provision`
## QQ 浏览器发现页推荐封面全不显示先查 aspect-ratio 兜底
- 现象:发现页的“推荐”子频道作品卡标题、作者和数据正常,但所有封面图不显示,常见于 QQ 浏览器 / X5 等旧移动内核。

View File

@@ -1,28 +0,0 @@
Stack trace:
Frame Function Args
0007FFFFB520 00021005FE8E (000210285F68, 00021026AB6E, 000000000000, 0007FFFFA420) msys-2.0.dll+0x1FE8E
0007FFFFB520 0002100467F9 (000000000000, 000000000000, 000000000000, 0007FFFFB7F8) msys-2.0.dll+0x67F9
0007FFFFB520 000210046832 (000210286019, 0007FFFFB3D8, 000000000000, 000000000000) msys-2.0.dll+0x6832
0007FFFFB520 000210068CF6 (000000000000, 000000000000, 000000000000, 000000000000) msys-2.0.dll+0x28CF6
0007FFFFB520 000210068E24 (0007FFFFB530, 000000000000, 000000000000, 000000000000) msys-2.0.dll+0x28E24
0007FFFFB800 00021006A225 (0007FFFFB530, 000000000000, 000000000000, 000000000000) msys-2.0.dll+0x2A225
End of stack trace
Loaded modules:
000100400000 bash.exe
7FFA3C060000 ntdll.dll
7FFA3B490000 KERNEL32.DLL
7FFA390F0000 KERNELBASE.dll
7FFA3BE50000 USER32.dll
7FFA38E90000 win32u.dll
7FFA3A230000 GDI32.dll
7FFA38D60000 gdi32full.dll
7FFA38EC0000 msvcp_win.dll
7FFA38930000 ucrtbase.dll
000210040000 msys-2.0.dll
7FFA39EB0000 advapi32.dll
7FFA3A180000 msvcrt.dll
7FFA3BCA0000 sechost.dll
7FFA3B5F0000 RPCRT4.dll
7FFA37D70000 CRYPTBASE.DLL
7FFA38B40000 bcryptPrimitives.dll
7FFA3A260000 IMM32.DLL

View File

@@ -160,7 +160,9 @@ Windows Stdb module 构建流水线运行在 Jenkins `windows` 节点上。该
- `api-server` 生产模板默认 `GENARRATIVE_API_LISTEN_BACKLOG=1024``GENARRATIVE_API_WORKER_THREADS=4`;本地未设置 worker threads 时继续使用 Tokio 默认值。
- `GENARRATIVE_API_MAX_CONCURRENT_REQUESTS=512` 开启应用内 HTTP 并发背压;`GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS=320``GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS=64``GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS=16` 分别限制公开列表、公开详情和后台 API 热路径。超过许可时直接返回 `429 Too Many Requests``Retry-After: 1``/healthz` 不受该限制。这些值不是 RPS 限速;如果压测中 429 上升但内存和 p95 收敛,说明背压正在保护进程。直连 `api-server` 的极高 RPS 压测若出现 `connection refused`,通常已经打到 TCP 监听 / accept 层,应同时检查 backlog、Nginx upstream keepalive 和前置限流。
- `genarrative-api.service` 设置 `LimitNOFILE=65535``TasksMax=2048`;上线后用 `systemctl show genarrative-api.service -p LimitNOFILE -p TasksMax``cat /proc/$(pidof api-server)/limits` 核对。
- Server provision 不在目标机下载 SpacetimeDB 或 `otelcol-contrib`Jenkins 的 `Prepare Provision Tools` 阶段在 `linux && genarrative-build` 构建机执行 `scripts/prepare-server-provision-tools.sh`,通过官方 SpacetimeDB 安装入口 `https://install.spacetimedb.com` 和 OpenTelemetry release 包生成 `provision-tools/`,再通过 `stash/unstash` 上传到 release 部署 agent。目标机上的 `scripts/jenkins-server-provision.sh` 只从该工作区工具包安装 `/stdb/spacetime``/stdb/bin/current/*``/usr/local/bin/otelcol-contrib`
- Server provision 不在目标机联网下载 SpacetimeDB 或 `otelcol-contrib``Genarrative-Server-Provision` 先在 Windows Jenkins 节点执行 `Download Provision Tool Archives`,把 `spacetime-x86_64-unknown-linux-gnu.tar.gz``otelcol-contrib_0.151.0_linux_amd64.tar.gz` 先下载到工作区,再通过 `stash/unstash` 带到 `genarrative-build-01`Windows 下载前会先查 GitHub release asset 的 `digest` 字段做 SHA256 校验,已有本地文件且 digest 一致就直接复用,不再重复下载。目标 Linux 节点上的 `scripts/prepare-server-provision-tools.sh` 只消费这些本地下载件生成 `provision-tools/`,再交给 `scripts/jenkins-server-provision.sh` 安装 `/stdb/spacetime``/stdb/bin/current/*``/usr/local/bin/otelcol-contrib`Windows 侧的 `runWindowsPowerShell(...)` 现在会先 `writeFile` 生成 UTF-8 `.ps1`,再直接把脚本文本读入内存并通过 `ScriptBlock::Create(...)` 执行,避免对同一个 workspace 脚本做原地 BOM 重写。排查时除了看下载日志,还要看 `[jenkins-powershell] workspace:``[jenkins-powershell] script:``[jenkins-powershell] loaded bytes:`。注意 `scripts/jenkins-checkout-source.sh` 会执行 `git reset --hard` / `git clean`,因此被直接执行的新增脚本必须以 Git `100755` 模式提交,或在二次 checkout 之后再补 `chmod +x`
- Windows 下载阶段如果出现 `curl: (18)` 或响应体截断,流水线会保留同名 `.download` 临时文件并用 `curl -C -` 断点续传;只有完整返回但 SHA256 digest 仍不匹配时才删除临时文件后重新下载。目标 Linux 节点仍只接收 `stash/unstash` 带过去的本地下载件,不回退外网下载。
- Windows 下载阶段如果走代理,在 `Genarrative-Server-Provision` 参数 `PROVISION_DOWNLOAD_PROXY` 填写 Windows Jenkins 节点可访问的 HTTP 代理,例如 `http://127.0.0.1:7890`;不要填写目标 release 机器视角的 `127.0.0.1`,除非代理确实运行在该 Windows 节点本机。Linux 目标机阶段会强制要求使用本地下载件,缺少文件直接失败,不再回退到外网下载。
- `otelcol-contrib.service` 作为可选系统服务加入 provision默认监听 `127.0.0.1:4317/4318` 并使用 `deploy/otelcol/genarrative-debug.yaml`。api-server 是否发送 OTLP 仍由 `GENARRATIVE_OTEL_ENABLED` 控制,服务 unit 见 `deploy/systemd/otelcol-contrib.service`
- Nginx `/api/``/admin/api/` 通过 `genarrative_api` upstream 代理到 `127.0.0.1:8082`upstream keepalive 为 64`limit_conn` 负责连接 / 并发保护,`limit_req` 负责入口 RPS 快拒绝。当前模板把公开 gallery list 单独放到 `genarrative_gallery_rps`,默认 `rate=5000r/s``burst=4096``limit_conn=320`;公开详情和普通 API 放到 `genarrative_api_rps`,后台 API 放到 `genarrative_admin_rps`。压测时看 `/var/log/nginx/genarrative.access.log` 中的 `request_time``upstream_connect_time``upstream_header_time``upstream_response_time``upstream_status``request_id`
- 作品列表 K6 脚本一次 iteration 默认请求两个公开接口,因此约 50 HTTP req/s 的目标命令使用 `SCENARIO=spike START_RPS=5 PEAK_RPS=25 HOLD=60s END_RPS=5 DETAIL_RATIO=0 npm run loadtest:k6:works`
@@ -178,7 +180,7 @@ npm run container:k6
npm run container:down
```
容器方案默认暴露 `http://127.0.0.1:18080``api-server` 在容器内监听 `0.0.0.0:8082`Nginx 通过 `api-server:8082` upstream 反代 `/api/``/admin/api/`。SpacetimeDB 也纳入 compose容器内由 `spacetimedb:3101` 提供服务,宿主机通过 `http://127.0.0.1:13101` 进行模块发布Collector 镜像使用 `otel/opentelemetry-collector-contrib:0.151.0`。生产 provision 侧则通过 Jenkins 构建机准备的 `provision-tools/otelcol-contrib` 安装本机 `otelcol-contrib.service`真实库名、token 和外部服务密钥只写本地 `deploy/container/api-server.env`,不提交 Git。完整拓扑、端口、k6 参数和 OTLP debug exporter 使用方法见 `deploy/container/README.md`
容器方案默认暴露 `http://127.0.0.1:18080``api-server` 在容器内监听 `0.0.0.0:8082`Nginx 通过 `api-server:8082` upstream 反代 `/api/``/admin/api/`。SpacetimeDB 也纳入 compose容器内由 `spacetimedb:3101` 提供服务,宿主机通过 `http://127.0.0.1:13101` 进行模块发布Collector 镜像使用 `otel/opentelemetry-collector-contrib:0.151.0`。生产 provision 侧则通过 Windows Jenkins 下载件在目标 Linux 节点生成 `provision-tools/otelcol-contrib`,再安装本机 `otelcol-contrib.service`真实库名、token 和外部服务密钥只写本地 `deploy/container/api-server.env`,不提交 Git。完整拓扑、端口、k6 参数和 OTLP debug exporter 使用方法见 `deploy/container/README.md`
`npm run container:config` 默认只做 quiet 校验,避免把本地 env 中的 token 展开到终端;确需排查完整 compose 时再传 `-- --print`
OpenTelemetry 现阶段默认开启 OTLP traces / metrics / logs但本地日志与 Nginx 文件日志仍保留:

View File

@@ -1,3 +1,27 @@
def runWindowsPowerShell(String scriptName, String scriptBody) {
def scriptPath = ".jenkins-${scriptName}.ps1"
writeFile file: scriptPath, text: scriptBody, encoding: 'UTF-8'
bat label: "PowerShell ${scriptName}", script: """
@echo off
setlocal
set "GENARRATIVE_POWERSHELL=%SystemRoot%\\System32\\WindowsPowerShell\\v1.0\\powershell.exe"
if not exist "%GENARRATIVE_POWERSHELL%" (
echo [jenkins-powershell] powershell.exe not found: %GENARRATIVE_POWERSHELL%
exit /b 1
)
echo [jenkins-powershell] user:
whoami
echo [jenkins-powershell] workspace: %CD%
echo [jenkins-powershell] exe: %GENARRATIVE_POWERSHELL%
if not exist "%CD%\\${scriptPath}" (
echo [jenkins-powershell] script not found: %CD%\\${scriptPath}
exit /b 1
)
"%GENARRATIVE_POWERSHELL%" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass -Command "try { \$path = Join-Path (Get-Location).ProviderPath '${scriptPath}'; Write-Host '[jenkins-powershell] script:' \$path; \$text = [System.IO.File]::ReadAllText(\$path, [System.Text.Encoding]::UTF8); Write-Host '[jenkins-powershell] loaded bytes:' ([System.IO.File]::ReadAllBytes(\$path).Length); \$scriptBlock = [ScriptBlock]::Create(\$text); & \$scriptBlock; if (\$LASTEXITCODE -is [int] -and \$LASTEXITCODE -ne 0) { exit \$LASTEXITCODE } } catch { Write-Host '[jenkins-powershell] failed:' \$_.Exception.Message; if (\$_.ScriptStackTrace) { Write-Host \$_.ScriptStackTrace }; exit 1 }"
exit /b %ERRORLEVEL%
"""
}
pipeline {
agent none
@@ -22,8 +46,11 @@ pipeline {
string(name: 'COMMIT_HASH', defaultValue: '', description: '部署脚本来源 commit')
string(name: 'SERVER_NAME', defaultValue: 'genarrative.example.com', description: '证书主域名;也作为 Nginx server_name 的第一个域名')
string(name: 'SERVER_ALIASES', defaultValue: '', description: '可选,额外 Nginx server_name多个用空格或逗号分隔例如 www.genarrative.world')
string(name: 'PROVISION_TOOLS_DIR', defaultValue: 'provision-tools', description: '构建机准备并上传到目标机工作区的工具包目录')
string(name: 'SPACETIME_DOWNLOAD_ROOT', defaultValue: 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download', description: '构建机下载 SpacetimeDB 官方安装产物的根地址;目标机不访问该地址')
string(name: 'PROVISION_DOWNLOADS_DIR', defaultValue: 'provision-tool-downloads', description: 'Windows 下载阶段暂存 SpacetimeDB/otelcol 安装包的工作区相对目录')
string(name: 'PROVISION_TOOLS_DIR', defaultValue: 'provision-tools', description: '目标机工作区内由已下载安装包生成的工具包目录')
string(name: 'PROVISION_DOWNLOAD_PROXY', defaultValue: '', description: '可选Windows 下载 SpacetimeDB 和 otelcol-contrib 时使用的代理地址,例如 http://127.0.0.1:7890留空不设置代理')
string(name: 'SPACETIME_DOWNLOAD_ROOT', defaultValue: 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download', description: 'Windows 下载 SpacetimeDB Linux release tarball 的根地址;目标机不访问该地址')
string(name: 'SPACETIME_TARGET_HOST', defaultValue: 'x86_64-unknown-linux-gnu', description: '目标机 SpacetimeDB 预编译包 host tripledevelopment/release Linux amd64 使用默认值')
string(name: 'SPACETIME_ROOT', defaultValue: '/stdb', description: 'SpacetimeDB root-dir')
string(name: 'RELEASE_ROOT', defaultValue: '/opt/genarrative/releases', description: 'release 根目录')
string(name: 'CURRENT_LINK', defaultValue: '/opt/genarrative/current', description: '当前版本软链接')
@@ -39,7 +66,7 @@ pipeline {
stages {
stage('Prepare') {
agent {
label 'linux && genarrative-build'
label 'windows'
}
steps {
script {
@@ -66,15 +93,33 @@ pipeline {
if (!params.PROVISION_TOOLS_DIR?.trim()) {
error('PROVISION_TOOLS_DIR 不能为空。')
}
if (!(params.PROVISION_TOOLS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_TOOLS_DIR.startsWith('/') || params.PROVISION_TOOLS_DIR.contains('..')) {
if (!(params.PROVISION_TOOLS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_TOOLS_DIR.startsWith('/') || params.PROVISION_TOOLS_DIR.contains('..') || params.PROVISION_TOOLS_DIR.trim() == '.') {
error("PROVISION_TOOLS_DIR 只能是工作区内的相对目录,不能包含绝对路径或连续点号: ${params.PROVISION_TOOLS_DIR}")
}
if (!params.PROVISION_DOWNLOADS_DIR?.trim()) {
error('PROVISION_DOWNLOADS_DIR 不能为空。')
}
if (!(params.PROVISION_DOWNLOADS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_DOWNLOADS_DIR.startsWith('/') || params.PROVISION_DOWNLOADS_DIR.contains('..') || params.PROVISION_DOWNLOADS_DIR.trim() == '.') {
error("PROVISION_DOWNLOADS_DIR 只能是工作区内的相对目录,不能包含绝对路径或连续点号: ${params.PROVISION_DOWNLOADS_DIR}")
}
def provisionToolsDir = params.PROVISION_TOOLS_DIR.trim()
def provisionDownloadsDir = params.PROVISION_DOWNLOADS_DIR.trim()
if (provisionToolsDir == provisionDownloadsDir || provisionDownloadsDir.startsWith("${provisionToolsDir}/")) {
error("PROVISION_DOWNLOADS_DIR 不能等于或位于 PROVISION_TOOLS_DIR 内,否则目标机生成工具包时会删除下载缓存: ${provisionDownloadsDir}")
}
def provisionDownloadProxy = params.PROVISION_DOWNLOAD_PROXY?.trim()
if (provisionDownloadProxy && !(provisionDownloadProxy ==~ /^https?:\/\/\S+$/)) {
error("PROVISION_DOWNLOAD_PROXY 只能填写 http:// 或 https:// 开头的代理地址,当前值: ${params.PROVISION_DOWNLOAD_PROXY}")
}
if (!(params.OTELCOL_VERSION?.trim() ==~ /^[0-9]+\.[0-9]+\.[0-9]+$/)) {
error("OTELCOL_VERSION 格式应为 x.y.z: ${params.OTELCOL_VERSION}")
}
if (!params.SPACETIME_DOWNLOAD_ROOT?.trim()) {
if (!(params.SPACETIME_DOWNLOAD_ROOT?.trim() ==~ /^https?:\/\/\S+$/)) {
error('SPACETIME_DOWNLOAD_ROOT 不能为空。')
}
if (!(params.SPACETIME_TARGET_HOST?.trim() ==~ /^[0-9A-Za-z._-]+$/)) {
error("SPACETIME_TARGET_HOST 只能包含字母、数字、点号、下划线和短横线: ${params.SPACETIME_TARGET_HOST}")
}
def nginxMode = params.NGINX_CONFIG_MODE?.trim()
if (!(nginxMode in ['none', 'production-https', 'development-http'])) {
error("NGINX_CONFIG_MODE 只能是 none、production-https 或 development-http当前值: ${params.NGINX_CONFIG_MODE}")
@@ -89,55 +134,258 @@ pipeline {
}
}
stage('Prepare Provision Tools') {
stage('Download Provision Tool Archives') {
agent {
label 'linux && genarrative-build'
label 'windows'
}
steps {
script {
def checkoutFromRemote = { String remoteUrl ->
checkout([
$class: 'GitSCM',
branches: [[name: "*/${params.SOURCE_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [
[$class: 'CleanBeforeCheckout'],
[$class: 'CloneOption', shallow: true, depth: 1, noTags: true, timeout: 30, honorRefspec: true],
],
userRemoteConfigs: [[url: remoteUrl, refspec: "+refs/heads/${params.SOURCE_BRANCH}:refs/remotes/origin/${params.SOURCE_BRANCH}"]],
])
runWindowsPowerShell('server-provision-tool-downloads', '''
$ErrorActionPreference = 'Stop'
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$downloadsDir = if ($env:PROVISION_DOWNLOADS_DIR) { $env:PROVISION_DOWNLOADS_DIR } else { 'provision-tool-downloads' }
$otelVersion = if ($env:OTELCOL_VERSION) { $env:OTELCOL_VERSION } else { '0.151.0' }
$prepareOtel = if ($env:ENABLE_OTELCOL) { $env:ENABLE_OTELCOL } else { 'true' }
$otelRoot = if ($env:OTELCOL_DOWNLOAD_ROOT) { $env:OTELCOL_DOWNLOAD_ROOT.TrimEnd('/') } else { 'https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download' }
$spacetimeDownloadRoot = if ($env:SPACETIME_DOWNLOAD_ROOT) { $env:SPACETIME_DOWNLOAD_ROOT.TrimEnd('/') } else { 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download' }
$spacetimeTargetHost = if ($env:SPACETIME_TARGET_HOST) { $env:SPACETIME_TARGET_HOST } else { 'x86_64-unknown-linux-gnu' }
$downloadProxy = if ($env:PROVISION_DOWNLOAD_PROXY) { $env:PROVISION_DOWNLOAD_PROXY } else { '' }
$workspace = (Get-Location).ProviderPath
if ([System.IO.Path]::IsPathRooted($downloadsDir)) {
throw "[prepare-provision-downloads] PROVISION_DOWNLOADS_DIR 只能是工作区内相对路径: ${downloadsDir}"
}
$downloadsDir = Join-Path $workspace $downloadsDir
Write-Host "[prepare-provision-downloads] Windows workspace: ${workspace}"
Write-Host "[prepare-provision-downloads] download dir: ${downloadsDir}"
if (Test-Path -LiteralPath $downloadsDir) {
Write-Host "[prepare-provision-downloads] 复用已有下载目录: ${downloadsDir}"
} else {
New-Item -ItemType Directory -Force -Path $downloadsDir | Out-Null
Write-Host "[prepare-provision-downloads] 已创建下载目录: ${downloadsDir}"
}
if ($downloadProxy) {
$env:HTTP_PROXY = $downloadProxy
$env:HTTPS_PROXY = $downloadProxy
$env:ALL_PROXY = $downloadProxy
Write-Host "[prepare-provision-downloads] 已配置 Windows 下载代理: $($downloadProxy -replace '://.*', '://***')"
}
function Get-GithubReleaseAssetDigest {
param(
[Parameter(Mandatory=$true)][string]$Repository,
[Parameter(Mandatory=$true)][string]$ReleaseSelector,
[Parameter(Mandatory=$true)][string]$AssetName
)
$request = @{
Uri = "https://api.github.com/repos/${Repository}/${ReleaseSelector}"
Headers = @{
Accept = 'application/vnd.github+json'
'User-Agent' = 'Genarrative-Server-Provision'
}
ErrorAction = 'Stop'
}
if ($downloadProxy) {
$request.Proxy = $downloadProxy
}
Write-Host "[prepare-provision-downloads] 查询 GitHub digest: repo=${Repository} release=${ReleaseSelector} asset=${AssetName}"
$release = Invoke-RestMethod @request
$asset = $release.assets | Where-Object { $_.name -eq $AssetName } | Select-Object -First 1
if (-not $asset) {
throw "[prepare-provision-downloads] GitHub release 未找到资产: ${Repository}/${AssetName}"
}
if (-not $asset.digest) {
throw "[prepare-provision-downloads] GitHub release 未返回 digest: ${Repository}/${AssetName}"
}
Write-Host "[prepare-provision-downloads] GitHub digest ${AssetName}: $($asset.digest)"
return $asset.digest
}
function Get-FileDigest {
param(
[Parameter(Mandatory=$true)][string]$Path,
[Parameter(Mandatory=$true)][string]$Algorithm
)
$fileHash = Get-FileHash -Algorithm $Algorithm -LiteralPath $Path
return $fileHash.Hash.ToLowerInvariant()
}
function Test-DownloadDigestMatch {
param(
[Parameter(Mandatory=$true)][string]$Path,
[Parameter(Mandatory=$true)][string]$ExpectedDigest
)
$parts = $ExpectedDigest.Split(':', 2)
if ($parts.Length -ne 2) {
throw "[prepare-provision-downloads] 无法解析 GitHub digest: ${ExpectedDigest}"
}
$algorithm = $parts[0].Trim().ToLowerInvariant()
$expectedHash = $parts[1].Trim().ToLowerInvariant()
if ($algorithm -ne 'sha256') {
throw "[prepare-provision-downloads] 暂不支持的 GitHub digest 算法: ${algorithm}"
}
$localHash = Get-FileDigest -Path $Path -Algorithm 'SHA256'
return $localHash -eq $expectedHash
}
function Invoke-ProvisionDownload {
param(
[Parameter(Mandatory=$true)][string]$Label,
[Parameter(Mandatory=$true)][string]$Url,
[Parameter(Mandatory=$true)][string]$Output,
[string]$ExpectedDigest = ''
)
if ($ExpectedDigest) {
if (Test-Path -LiteralPath $Output) {
if (Test-DownloadDigestMatch -Path $Output -ExpectedDigest $ExpectedDigest) {
$existingItem = Get-Item -LiteralPath $Output
Write-Host "[prepare-provision-downloads] 已存在且校验一致,跳过下载: ${Label} bytes=$($existingItem.Length) path=${Output}"
return
}
Write-Host "[prepare-provision-downloads] 已存在但校验不一致,重新下载: ${Label} path=${Output}"
}
}
Write-Host "[prepare-provision-downloads] 下载 ${Label}: ${Url}"
$tempOutput = "${Output}.download"
if (Test-Path -LiteralPath $tempOutput) {
$tempItem = Get-Item -LiteralPath $tempOutput
if ($ExpectedDigest -and $tempItem.Length -gt 0 -and (Test-DownloadDigestMatch -Path $tempOutput -ExpectedDigest $ExpectedDigest)) {
Move-Item -LiteralPath $tempOutput -Destination $Output -Force
$finalItem = Get-Item -LiteralPath $Output
Write-Host "[prepare-provision-downloads] 已复用校验通过的临时下载: ${Label} bytes=$($finalItem.Length) path=${Output}"
return
}
if ($tempItem.Length -gt 0) {
Write-Host "[prepare-provision-downloads] 发现未完成临时文件,后续尝试断点续传: ${Label} bytes=$($tempItem.Length) path=${tempOutput}"
} else {
Remove-Item -LiteralPath $tempOutput -Force
}
}
$curl = Get-Command curl.exe -ErrorAction SilentlyContinue
$maxAttempts = 8
for ($attempt = 1; $attempt -le $maxAttempts; $attempt++) {
$resumeBytes = 0
if (Test-Path -LiteralPath $tempOutput) {
$resumeBytes = (Get-Item -LiteralPath $tempOutput).Length
}
try {
checkoutFromRemote(env.GIT_REMOTE_URL)
env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_URL
} catch (error) {
echo "Git 主地址拉取失败: ${env.GIT_REMOTE_URL},改用备用地址: ${env.GIT_REMOTE_FALLBACK_URL}"
checkoutFromRemote(env.GIT_REMOTE_FALLBACK_URL)
env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_FALLBACK_URL
if ($curl) {
$arguments = @('-fL', '--retry', '3', '--retry-delay', '3', '--retry-all-errors', '--connect-timeout', '30', '--speed-time', '60', '--speed-limit', '1024')
if ($resumeBytes -gt 0) {
$arguments += @('-C', '-')
Write-Host "[prepare-provision-downloads] curl 断点续传 ${Label}: attempt=${attempt}/${maxAttempts} resumeBytes=${resumeBytes}"
} else {
Write-Host "[prepare-provision-downloads] curl 下载 ${Label}: attempt=${attempt}/${maxAttempts}"
}
$arguments += @('-o', $tempOutput)
if ($downloadProxy) {
$arguments += @('--proxy', $downloadProxy)
}
$arguments += $Url
& $curl.Source @arguments
$exitCode = $LASTEXITCODE
if ($exitCode -ne 0) {
$currentBytes = if (Test-Path -LiteralPath $tempOutput) { (Get-Item -LiteralPath $tempOutput).Length } else { 0 }
Write-Host "[prepare-provision-downloads] curl 下载未完成: ${Label}, attempt=${attempt}/${maxAttempts}, exit=${exitCode}, tempBytes=${currentBytes}"
if ($attempt -lt $maxAttempts) {
Start-Sleep -Seconds ([Math]::Min(30, 3 * $attempt))
continue
}
throw "[prepare-provision-downloads] curl 下载失败: ${Label}, exit=${exitCode}, temp=${tempOutput}"
}
} else {
Write-Host "[prepare-provision-downloads] Invoke-WebRequest 下载 ${Label}: attempt=${attempt}/${maxAttempts}"
if ($resumeBytes -gt 0) {
Write-Host "[prepare-provision-downloads] Invoke-WebRequest 不支持断点续传,删除临时文件后重新下载: ${Label}, bytes=${resumeBytes}"
Remove-Item -LiteralPath $tempOutput -Force
}
$parameters = @{
Uri = $Url
OutFile = $tempOutput
UseBasicParsing = $true
}
if ($downloadProxy) {
$parameters.Proxy = $downloadProxy
}
Invoke-WebRequest @parameters
}
} catch {
$currentBytes = if (Test-Path -LiteralPath $tempOutput) { (Get-Item -LiteralPath $tempOutput).Length } else { 0 }
Write-Host "[prepare-provision-downloads] 下载尝试失败: ${Label}, attempt=${attempt}/${maxAttempts}, tempBytes=${currentBytes}, error=$($_.Exception.Message)"
if ($attempt -lt $maxAttempts) {
Start-Sleep -Seconds ([Math]::Min(30, 3 * $attempt))
continue
}
throw
}
sh '''
bash <<'BASH'
set -euo pipefail
chmod +x scripts/jenkins-checkout-source.sh scripts/prepare-server-provision-tools.sh
SOURCE_BRANCH="${SOURCE_BRANCH:-master}" \
COMMIT_HASH="${COMMIT_HASH:-}" \
GIT_REMOTE_URL="${EFFECTIVE_GIT_REMOTE_URL:-${GIT_REMOTE_URL}}" \
GIT_REMOTE_FALLBACK_URL="${GIT_REMOTE_FALLBACK_URL:-}" \
SOURCE_COMMIT_FILE=".jenkins-source-commit" \
scripts/jenkins-checkout-source.sh
PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \
OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}" \
SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}" \
scripts/prepare-server-provision-tools.sh
BASH
'''
script {
env.SOURCE_COMMIT = readFile('.jenkins-source-commit').trim()
echo "Provision 工具包已准备,源码 commit=${env.SOURCE_COMMIT}"
if (-not (Test-Path -LiteralPath $tempOutput)) {
throw "[prepare-provision-downloads] 下载未生成临时文件: ${tempOutput}"
}
stash name: 'server-provision-tools', includes: "${params.PROVISION_TOOLS_DIR}/**", useDefaultExcludes: false
$item = Get-Item -LiteralPath $tempOutput
if ($item.Length -le 0) {
if ($attempt -lt $maxAttempts) {
Write-Host "[prepare-provision-downloads] 下载结果为空,将重试: ${Label}"
Start-Sleep -Seconds ([Math]::Min(30, 3 * $attempt))
continue
}
throw "[prepare-provision-downloads] 下载结果为空: ${tempOutput}"
}
if ($ExpectedDigest) {
if (-not (Test-DownloadDigestMatch -Path $tempOutput -ExpectedDigest $ExpectedDigest)) {
Write-Host "[prepare-provision-downloads] 下载结果校验未通过,将继续重试: ${Label}, attempt=${attempt}/${maxAttempts}, tempBytes=$($item.Length)"
if ($attempt -lt $maxAttempts) {
Remove-Item -LiteralPath $tempOutput -Force
Start-Sleep -Seconds ([Math]::Min(30, 3 * $attempt))
continue
}
throw "[prepare-provision-downloads] 下载结果校验失败: ${Label}, temp=${tempOutput}"
}
}
Move-Item -LiteralPath $tempOutput -Destination $Output -Force
$finalItem = Get-Item -LiteralPath $Output
Write-Host "[prepare-provision-downloads] 已下载 ${Label}: bytes=$($finalItem.Length) path=${Output}"
return
}
throw "[prepare-provision-downloads] 下载重试耗尽: ${Label}"
}
$spacetimeArchiveName = "spacetime-${spacetimeTargetHost}.tar.gz"
$spacetimeArchiveUrl = "${spacetimeDownloadRoot}/${spacetimeArchiveName}"
$spacetimeArchiveDigest = Get-GithubReleaseAssetDigest -Repository 'clockworklabs/SpacetimeDB' -ReleaseSelector 'releases/latest' -AssetName $spacetimeArchiveName
Invoke-ProvisionDownload -Label "SpacetimeDB release tarball ${spacetimeTargetHost}" -Url $spacetimeArchiveUrl -Output (Join-Path $downloadsDir $spacetimeArchiveName) -ExpectedDigest $spacetimeArchiveDigest
if ($prepareOtel -eq 'true') {
$otelArchiveName = "otelcol-contrib_${otelVersion}_linux_amd64.tar.gz"
$otelUrl = "${otelRoot}/v${otelVersion}/${otelArchiveName}"
$otelDigest = Get-GithubReleaseAssetDigest -Repository 'open-telemetry/opentelemetry-collector-releases' -ReleaseSelector "releases/tags/v${otelVersion}" -AssetName $otelArchiveName
Invoke-ProvisionDownload -Label "otelcol-contrib ${otelVersion} linux amd64" -Url $otelUrl -Output (Join-Path $downloadsDir $otelArchiveName) -ExpectedDigest $otelDigest
} else {
Write-Host "[prepare-provision-downloads] ENABLE_OTELCOL=${prepareOtel},跳过 otelcol-contrib 下载。"
}
$utf8NoBom = New-Object System.Text.UTF8Encoding $false
$manifest = @(
"spacetime release tarball ${spacetimeArchiveUrl}",
"spacetime target host ${spacetimeTargetHost}",
"otelcol-contrib ${otelVersion} prepare=${prepareOtel}"
)
[System.IO.File]::WriteAllLines((Join-Path $downloadsDir 'DOWNLOADS-MANIFEST.txt'), $manifest, $utf8NoBom)
Get-ChildItem -LiteralPath $downloadsDir | Sort-Object Name | ForEach-Object {
Write-Host "[prepare-provision-downloads] artifact $($_.Length) $($_.Name)"
}
''')
}
stash name: 'server-provision-tool-downloads', includes: "${params.PROVISION_DOWNLOADS_DIR}/**", useDefaultExcludes: false
}
}
@@ -180,6 +428,10 @@ BASH
scripts/jenkins-checkout-source.sh
BASH
'''
script {
env.SOURCE_COMMIT = readFile('.jenkins-source-commit').trim()
echo "Provision 源码 commit=${env.SOURCE_COMMIT}"
}
}
}
@@ -188,12 +440,24 @@ BASH
label "${params.DEPLOY_TARGET == 'development' ? 'linux && genarrative-build' : 'linux && genarrative-release-deploy'}"
}
steps {
unstash 'server-provision-tools'
unstash 'server-provision-tool-downloads'
sh '''
bash <<'BASH'
set -euo pipefail
chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/otelcol-contrib" \
"${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/spacetime" \
chmod +x scripts/prepare-server-provision-tools.sh
PROVISION_DOWNLOADS_DIR="${PROVISION_DOWNLOADS_DIR:-provision-tool-downloads}" \
PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \
OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}" \
PREPARE_OTELCOL="${ENABLE_OTELCOL:-true}" \
PROVISION_REQUIRE_LOCAL_DOWNLOADS="true" \
SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}" \
SPACETIME_TARGET_HOST="${SPACETIME_TARGET_HOST:-x86_64-unknown-linux-gnu}" \
scripts/prepare-server-provision-tools.sh
if [[ "${ENABLE_OTELCOL:-true}" == "true" ]]; then
chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/otelcol-contrib"
fi
chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/spacetime" \
"${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/bin/current/spacetimedb-cli" \
"${PROVISION_TOOLS_DIR:-provision-tools}/spacetime/bin/current/spacetimedb-standalone"
chmod +x scripts/jenkins-server-provision.sh

141
scripts/prepare-server-provision-tools.sh Normal file → Executable file
View File

@@ -2,12 +2,24 @@
set -euo pipefail
PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}"
PROVISION_DOWNLOADS_DIR="${PROVISION_DOWNLOADS_DIR:-}"
OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}"
PREPARE_OTELCOL="${PREPARE_OTELCOL:-${ENABLE_OTELCOL:-true}}"
OTELCOL_DOWNLOAD_ROOT="${OTELCOL_DOWNLOAD_ROOT:-https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download}"
OTELCOL_ARCHIVE_PATH="${OTELCOL_ARCHIVE_PATH:-}"
SPACETIME_INSTALLER_URL="${SPACETIME_INSTALLER_URL:-https://install.spacetimedb.com}"
SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}"
SPACETIME_TARGET_HOST="${SPACETIME_TARGET_HOST:-x86_64-unknown-linux-gnu}"
SPACETIME_ARCHIVE_PATH="${SPACETIME_ARCHIVE_PATH:-}"
SPACETIME_INSTALLER_PATH="${SPACETIME_INSTALLER_PATH:-}"
SPACETIME_UPDATE_INSTALLER_PATH="${SPACETIME_UPDATE_INSTALLER_PATH:-}"
PROVISION_DOWNLOAD_PROXY="${PROVISION_DOWNLOAD_PROXY:-}"
PROVISION_NO_PROXY="${PROVISION_NO_PROXY:-127.0.0.1,localhost}"
PROVISION_REQUIRE_LOCAL_DOWNLOADS="${PROVISION_REQUIRE_LOCAL_DOWNLOADS:-false}"
PROVISION_TOOLS_TMP_PARENT="${PROVISION_TOOLS_TMP_PARENT:-${WORKSPACE:-$(pwd)}/.tmp/server-provision-tools}"
TMP_DIR_TO_CLEAN=""
OTELCOL_SOURCE_DESCRIPTION="skipped"
SPACETIME_SOURCE_DESCRIPTION="unset"
cleanup_tmp_dir() {
if [[ -n "${TMP_DIR_TO_CLEAN}" ]]; then
@@ -23,6 +35,22 @@ require_cmd() {
fi
}
configure_download_proxy() {
if [[ -z "${PROVISION_DOWNLOAD_PROXY}" ]]; then
return
fi
export HTTP_PROXY="${PROVISION_DOWNLOAD_PROXY}"
export HTTPS_PROXY="${PROVISION_DOWNLOAD_PROXY}"
export ALL_PROXY="${PROVISION_DOWNLOAD_PROXY}"
export http_proxy="${PROVISION_DOWNLOAD_PROXY}"
export https_proxy="${PROVISION_DOWNLOAD_PROXY}"
export all_proxy="${PROVISION_DOWNLOAD_PROXY}"
export NO_PROXY="${PROVISION_NO_PROXY}"
export no_proxy="${PROVISION_NO_PROXY}"
echo "[prepare-provision-tools] 已配置下载代理: ${PROVISION_DOWNLOAD_PROXY%%://*}://***"
}
download_file() {
local url="$1"
local output="$2"
@@ -37,6 +65,19 @@ download_file() {
fi
}
validate_relative_dir() {
local label="$1"
local path="$2"
if [[ -z "${path}" ]]; then
return
fi
if [[ "${path}" == /* || "${path}" == *..* || "${path}" == "." ]]; then
echo "[prepare-provision-tools] ${label} 只能是工作区内的相对路径: ${path}" >&2
exit 1
fi
}
make_spacetime_wrapper() {
local target="$1"
@@ -54,13 +95,32 @@ prepare_otelcol() {
local archive="${tmp_dir}/otelcol-contrib.tar.gz"
local extract_dir="${tmp_dir}/otelcol-contrib"
local url="${OTELCOL_DOWNLOAD_ROOT}/v${OTELCOL_VERSION}/otelcol-contrib_${OTELCOL_VERSION}_linux_amd64.tar.gz"
local downloaded_archive="${PROVISION_DOWNLOADS_DIR}/otelcol-contrib_${OTELCOL_VERSION}_linux_amd64.tar.gz"
local source_archive=""
local target="${PROVISION_TOOLS_DIR}/otelcol-contrib"
require_cmd tar
echo "[prepare-provision-tools] 下载 otelcol-contrib: ${url}"
if [[ -n "${OTELCOL_ARCHIVE_PATH}" && -f "${OTELCOL_ARCHIVE_PATH}" ]]; then
source_archive="${OTELCOL_ARCHIVE_PATH}"
elif [[ -n "${PROVISION_DOWNLOADS_DIR}" && -f "${downloaded_archive}" ]]; then
source_archive="${downloaded_archive}"
fi
if [[ "${PROVISION_REQUIRE_LOCAL_DOWNLOADS}" == "true" && -z "${source_archive}" ]]; then
echo "[prepare-provision-tools] 要求使用 Windows 已下载的 otelcol-contrib 包,但未找到: ${downloaded_archive}" >&2
exit 1
fi
mkdir -p "${extract_dir}"
if [[ -n "${source_archive}" ]]; then
echo "[prepare-provision-tools] 使用已下载的 otelcol-contrib 包: ${source_archive}"
cp "${source_archive}" "${archive}"
OTELCOL_SOURCE_DESCRIPTION="local ${source_archive}"
else
echo "[prepare-provision-tools] 下载 otelcol-contrib: ${url}"
download_file "${url}" "${archive}"
OTELCOL_SOURCE_DESCRIPTION="download ${url}"
fi
tar -xzf "${archive}" -C "${extract_dir}"
if [[ ! -x "${extract_dir}/otelcol-contrib" ]]; then
@@ -76,12 +136,68 @@ prepare_spacetime() {
local tmp_dir="$1"
local install_root="${tmp_dir}/spacetime-root"
local target_dir="${PROVISION_TOOLS_DIR}/spacetime"
local archive_name="spacetime-${SPACETIME_TARGET_HOST}.tar.gz"
local downloaded_archive="${PROVISION_DOWNLOADS_DIR}/${archive_name}"
local source_archive=""
local update_name="spacetimedb-update-${SPACETIME_TARGET_HOST}"
local downloaded_update="${PROVISION_DOWNLOADS_DIR}/${update_name}"
local source_update=""
local prepared_update="${tmp_dir}/spacetimedb-update"
local downloaded_installer="${PROVISION_DOWNLOADS_DIR}/spacetime-install.sh"
local source_installer=""
echo "[prepare-provision-tools] 使用官方安装器准备 SpacetimeDB: ${SPACETIME_INSTALLER_URL}"
mkdir -p "${install_root}"
if [[ -n "${SPACETIME_ARCHIVE_PATH}" && -f "${SPACETIME_ARCHIVE_PATH}" ]]; then
source_archive="${SPACETIME_ARCHIVE_PATH}"
elif [[ -n "${PROVISION_DOWNLOADS_DIR}" && -f "${downloaded_archive}" ]]; then
source_archive="${downloaded_archive}"
fi
if [[ -n "${source_archive}" ]]; then
echo "[prepare-provision-tools] 使用已下载的 SpacetimeDB release tarball: ${source_archive}"
mkdir -p "${install_root}/bin/current"
tar -xzf "${source_archive}" -C "${install_root}/bin/current"
chmod 0755 "${install_root}/bin/current/spacetimedb-cli" "${install_root}/bin/current/spacetimedb-standalone"
SPACETIME_SOURCE_DESCRIPTION="local archive ${source_archive}"
elif [[ -n "${SPACETIME_UPDATE_INSTALLER_PATH}" && -f "${SPACETIME_UPDATE_INSTALLER_PATH}" ]]; then
source_update="${SPACETIME_UPDATE_INSTALLER_PATH}"
elif [[ -n "${PROVISION_DOWNLOADS_DIR}" && -f "${downloaded_update}" ]]; then
source_update="${downloaded_update}"
fi
if [[ "${PROVISION_REQUIRE_LOCAL_DOWNLOADS}" == "true" && -z "${source_archive}" ]]; then
echo "[prepare-provision-tools] 要求使用 Windows 已下载的 SpacetimeDB release tarball但未找到: ${downloaded_archive}" >&2
exit 1
fi
if [[ -n "${source_archive}" ]]; then
:
elif [[ -n "${source_update}" ]]; then
echo "[prepare-provision-tools] 使用已下载的 SpacetimeDB Linux update installer: ${source_update}"
cp "${source_update}" "${prepared_update}"
chmod 0755 "${prepared_update}"
TMPDIR="${tmp_dir}" "${prepared_update}" --root-dir "${install_root}" -y
SPACETIME_SOURCE_DESCRIPTION="local update ${source_update}"
else
if [[ -n "${SPACETIME_INSTALLER_PATH}" && -f "${SPACETIME_INSTALLER_PATH}" ]]; then
source_installer="${SPACETIME_INSTALLER_PATH}"
elif [[ -n "${PROVISION_DOWNLOADS_DIR}" && -f "${downloaded_installer}" ]]; then
source_installer="${downloaded_installer}"
fi
if [[ "${PROVISION_REQUIRE_LOCAL_DOWNLOADS}" == "true" && -z "${source_installer}" ]]; then
echo "[prepare-provision-tools] 要求使用 Windows 已下载的 SpacetimeDB 官方安装器脚本,但未找到: ${downloaded_installer}" >&2
exit 1
elif [[ -n "${source_installer}" ]]; then
echo "[prepare-provision-tools] 使用已下载的 SpacetimeDB 官方安装器脚本: ${source_installer}"
cp "${source_installer}" "${tmp_dir}/spacetime-install.sh"
else
echo "[prepare-provision-tools] 下载 SpacetimeDB 官方安装器脚本: ${SPACETIME_INSTALLER_URL}"
download_file "${SPACETIME_INSTALLER_URL}" "${tmp_dir}/spacetime-install.sh"
fi
chmod 0755 "${tmp_dir}/spacetime-install.sh"
TMPDIR="${tmp_dir}" SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT}" sh "${tmp_dir}/spacetime-install.sh" --root-dir "${install_root}" -y
SPACETIME_SOURCE_DESCRIPTION="installer ${SPACETIME_INSTALLER_URL}; download root ${SPACETIME_DOWNLOAD_ROOT}"
fi
if [[ ! -x "${install_root}/bin/current/spacetimedb-cli" ]]; then
echo "[prepare-provision-tools] SpacetimeDB 安装结果缺少 bin/current/spacetimedb-cli。" >&2
@@ -107,6 +223,16 @@ main() {
require_cmd install
require_cmd mktemp
require_cmd rm
require_cmd tar
validate_relative_dir "PROVISION_TOOLS_DIR" "${PROVISION_TOOLS_DIR}"
validate_relative_dir "PROVISION_DOWNLOADS_DIR" "${PROVISION_DOWNLOADS_DIR}"
if [[ -n "${PROVISION_DOWNLOADS_DIR}" && "${PROVISION_DOWNLOADS_DIR%/}" == "${PROVISION_TOOLS_DIR%/}" ]]; then
echo "[prepare-provision-tools] PROVISION_DOWNLOADS_DIR 不能等于 PROVISION_TOOLS_DIR否则会被清理: ${PROVISION_DOWNLOADS_DIR}" >&2
exit 1
fi
configure_download_proxy
mkdir -p "${PROVISION_TOOLS_TMP_PARENT}"
tmp_dir="$(mktemp -d "${PROVISION_TOOLS_TMP_PARENT%/}/run.XXXXXX")"
@@ -116,13 +242,18 @@ main() {
rm -rf "${PROVISION_TOOLS_DIR}"
mkdir -p "${PROVISION_TOOLS_DIR}"
if [[ "${PREPARE_OTELCOL}" == "true" ]]; then
prepare_otelcol "${tmp_dir}"
else
echo "[prepare-provision-tools] PREPARE_OTELCOL=${PREPARE_OTELCOL},跳过 otelcol-contrib 工具包准备。"
fi
prepare_spacetime "${tmp_dir}"
cat >"${PROVISION_TOOLS_DIR}/MANIFEST.txt" <<EOF
otelcol-contrib ${OTELCOL_VERSION}
spacetime installer ${SPACETIME_INSTALLER_URL}
spacetime download root ${SPACETIME_DOWNLOAD_ROOT}
otelcol-contrib ${OTELCOL_VERSION} ${OTELCOL_SOURCE_DESCRIPTION}
spacetime ${SPACETIME_SOURCE_DESCRIPTION}
spacetime target host ${SPACETIME_TARGET_HOST}
require local downloads ${PROVISION_REQUIRE_LOCAL_DOWNLOADS}
EOF
echo "[prepare-provision-tools] 工具包已准备: ${PROVISION_TOOLS_DIR}"

View File

@@ -1,27 +0,0 @@
$response =(Invoke-RestMethod `
-Method Post `
-Uri "http://127.0.0.1:8082/admin/api/login" `
-ContentType "application/json" `
-Body '{"username":"admin","password":"123456"}')
$adminToken = $response.token
Invoke-RestMethod `
-Method Post `
-Uri http://127.0.0.1:8082/admin/api/profile/redeem-codes `
-Headers @{ Authorization = "Bearer $adminToken" } `
-ContentType 'application/json' `
-Body '{
"code": "TEST100",
"mode": "public",
"rewardPoints": 100,
"maxUses": 1,
"enabled": true
}'
$userToken = "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2F1dGguZ2VuYXJyYXRpdmUubG9jYWwiLCJzdWIiOiJ1c2VyXzAwMDAwMDAxIiwic2lkIjoidXNlc3NfNWVhMWU3NWIxOTUxNGU2ZTg5ZTdmYWFkZDEyYTE0NTEiLCJwcm92aWRlciI6InBhc3N3b3JkIiwicm9sZXMiOlsidXNlciJdLCJ2ZXIiOjIsInBob25lX3ZlcmlmaWVkIjp0cnVlLCJiaW5kaW5nX3N0YXR1cyI6ImFjdGl2ZSIsImRpc3BsYXlfbmFtZSI6IjE1OCoqKiozNTMzIiwiaWF0IjoxNzc3MzU0NjA4LCJleHAiOjE3Nzc5NTk0MDh9.O6US67MOVD62kPliBlp7qDV3Pyo3pJlnBoGv0fFuGfA"
Invoke-RestMethod `
-Method Post `
-Uri http://127.0.0.1:8082/api/profile/redeem-codes/redeem `
-Headers @{ Authorization = "Bearer $userToken" } `
-ContentType 'application/json' `
-Body '{"code":"TEST100"}'