fix(jenkins): download provision tools on windows

2026-05-19 17:57:55 +08:00
parent a9c54b0e1a
commit b4a6cb2465
5 changed files with 266 additions and 84 deletions
--- a/.hermes/shared-memory/decision-log.md
+++ b/.hermes/shared-memory/decision-log.md
@@ -87,20 +87,12 @@
 - 验证方式：执行 `npm run container:config` 展开 compose 配置；需要真实运行时再执行 `npm run container:build`、`npm run container:up`、`npm run container:k6`，并结合容器 Nginx log 与 OTLP debug exporter 判断瓶颈。
 - 关联文档：`deploy/container/README.md`、`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。

-## 2026-05-18 生产 provision 改为构建机准备工具包再上传安装
+## 2026-05-19 生产 provision 改为 Windows 下载包后由目标机本地安装

- 背景：目标 release 服务器无法访问 GitHub，之前的 server provision 默认仍假设 `spacetime` 和 `otelcol-contrib` 已经存在于目标机本地路径，和真实运维条件不符。
- 决策：Jenkins 新增 `Prepare Provision Tools` 阶段，在 `linux && genarrative-build` 构建机执行 `scripts/prepare-server-provision-tools.sh`，通过官方 SpacetimeDB 安装入口和 OpenTelemetry release 包生成 `provision-tools/`，再用 `stash/unstash` 带到 release 部署 agent；`scripts/jenkins-server-provision.sh` 只从工作区工具包复制安装，不再要求目标机自己下载或预装二进制。
+- 背景：当前 `development` provision 目标实际就是 Linux agent `genarrative-build-01`，之前把 `Prepare Provision Tools` 放在 `linux && genarrative-build` 会让目标机自己连 GitHub 和 `install.spacetimedb.com`，违背“Windows 本机先下载再传到目标机”的运维要求。
+- 决策：`Genarrative-Server-Provision` 拆成 Windows 下载阶段和 Linux 目标机安装阶段。Windows 节点的 `Download Provision Tool Archives` 只下载 SpacetimeDB 官方安装脚本、Linux update installer 和 `otelcol-contrib_0.151.0_linux_amd64.tar.gz`，通过 `stash/unstash` 传到目标 Linux 节点；目标机执行 `scripts/prepare-server-provision-tools.sh` 时设置 `PROVISION_REQUIRE_LOCAL_DOWNLOADS=true`，只消费已下载件生成 `provision-tools/`，缺包直接失败，不回退外网下载。
 - 影响范围：`jenkins/Jenkinsfile.production-server-provision`、`scripts/prepare-server-provision-tools.sh`、`scripts/jenkins-server-provision.sh`、生产运维文档。
- 验证方式：Jenkins 构建机可完成工具包准备，release 部署 agent 只消费工作区文件；目标机不再依赖 GitHub 外网下载。
- 关联文档：`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。
-
-## 2026-05-19 provision 工具包由 Jenkins 构建机本地准备后上传
-
- 背景：目标 release 机器不应自己下载 SpacetimeDB 或 `otelcol-contrib`，但 Jenkins 构建机可以先准备二进制工具包，再把结果带到目标部署 agent。
- 决策：`jenkins/Jenkinsfile.production-server-provision` 不要求人工上传安装包；`Prepare Provision Tools` 阶段在 `linux && genarrative-build` 构建机本地执行 `scripts/prepare-server-provision-tools.sh`，下载 SpacetimeDB 官方安装器和 OpenTelemetry release 包并生成 `provision-tools/`，随后通过 `stash/unstash` 上传到 release 部署 agent。若构建机下载较慢，可通过 `PROVISION_DOWNLOAD_PROXY` 显式指定该构建机可访问的代理地址。
- 影响范围：`jenkins/Jenkinsfile.production-server-provision`、`scripts/prepare-server-provision-tools.sh`、`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。
- 验证方式：Jenkins 日志应显示目标部署 agent 只消费 `server-provision-tools` stash；目标机不直接访问 `install.spacetimedb.com` 或 OpenTelemetry GitHub release 下载地址。
+- 验证方式：Jenkins 日志应先出现 Windows 节点的 `[prepare-provision-downloads]` 下载日志，再在 `genarrative-build-01` 上出现“使用已下载的 ...”日志；目标机不应出现直接访问 `install.spacetimedb.com` 或 OpenTelemetry GitHub release 下载地址的回退日志。
 - 关联文档：`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。

 ## 2026-05-19 公开 gallery 入口发布限流以快拒绝保护后端
--- a/.hermes/shared-memory/pitfalls.md
+++ b/.hermes/shared-memory/pitfalls.md
@@ -744,6 +744,14 @@
 - 验证：运行 `git ls-files --stage scripts/prepare-server-provision-tools.sh`，确认 mode 为 `100755`；重新跑 `Genarrative-Server-Provision` 时应进入工具下载/打包日志，而不是停在 `Permission denied`。
 - 关联：`jenkins/Jenkinsfile.production-server-provision`、`scripts/prepare-server-provision-tools.sh`、`scripts/jenkins-checkout-source.sh`、`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。

+## Server-Provision 下载阶段不要放回 genarrative-build-01
+
+- 现象：`Genarrative-Server-Provision` 日志里 `Prepare Provision Tools` 显示 `Running on genarrative-build-01 in /root/...`，随后在该节点上下载 GitHub release 或 `install.spacetimedb.com` 失败。
+- 原因：`genarrative-build-01` 在当前 provision 流程里是 Linux 目标发布机/目标 agent，不是用户本地 Windows 下载环境；把下载阶段放在 `linux && genarrative-build` 等于让目标机自己外连。
+- 处理：下载必须发生在 Jenkins `windows` 节点的 `Download Provision Tool Archives` 阶段，先下载 SpacetimeDB Linux update installer 和 `otelcol-contrib` Linux amd64 包，再 `stash/unstash` 到目标 Linux 节点。目标机执行 `scripts/prepare-server-provision-tools.sh` 时设置 `PROVISION_REQUIRE_LOCAL_DOWNLOADS=true`，缺少下载件直接失败，不回退联网下载。
+- 验证：Jenkins 日志应先出现 `Running on ... windows` 和 `[prepare-provision-downloads] 下载 ...`，目标节点只出现 `[prepare-provision-tools] 使用已下载的 ...`；如果目标节点出现 `下载 otelcol-contrib:` 或 `下载 SpacetimeDB 官方安装器脚本:`，说明又回退到错误路径。
+- 关联：`jenkins/Jenkinsfile.production-server-provision`、`scripts/prepare-server-provision-tools.sh`、`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。
+
 ## 个人任务 scope 不得扩成 work/site/module

 - 现象：个人任务配置为 `work` / `site` / `module` 后进度串桶或静默按 0 处理。
--- a/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md
+++ b/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md
@@ -160,8 +160,8 @@ Windows Stdb module 构建流水线运行在 Jenkins `windows` 节点上。该
 - `api-server` 生产模板默认 `GENARRATIVE_API_LISTEN_BACKLOG=1024`、`GENARRATIVE_API_WORKER_THREADS=4`；本地未设置 worker threads 时继续使用 Tokio 默认值。
 - `GENARRATIVE_API_MAX_CONCURRENT_REQUESTS=512` 开启应用内 HTTP 并发背压；`GENARRATIVE_API_GALLERY_MAX_CONCURRENT_REQUESTS=320`、`GENARRATIVE_API_DETAIL_MAX_CONCURRENT_REQUESTS=64`、`GENARRATIVE_API_ADMIN_MAX_CONCURRENT_REQUESTS=16` 分别限制公开列表、公开详情和后台 API 热路径。超过许可时直接返回 `429 Too Many Requests` 和 `Retry-After: 1`，`/healthz` 不受该限制。这些值不是 RPS 限速；如果压测中 429 上升但内存和 p95 收敛，说明背压正在保护进程。直连 `api-server` 的极高 RPS 压测若出现 `connection refused`，通常已经打到 TCP 监听 / accept 层，应同时检查 backlog、Nginx upstream keepalive 和前置限流。
 - `genarrative-api.service` 设置 `LimitNOFILE=65535`、`TasksMax=2048`；上线后用 `systemctl show genarrative-api.service -p LimitNOFILE -p TasksMax` 和 `cat /proc/$(pidof api-server)/limits` 核对。
- Server provision 不在目标机下载 SpacetimeDB 或 `otelcol-contrib`。Jenkins 的 `Prepare Provision Tools` 阶段在 `linux && genarrative-build` 构建机执行 `scripts/prepare-server-provision-tools.sh`，先在该构建机本地通过官方 SpacetimeDB 安装入口 `https://install.spacetimedb.com` 和 OpenTelemetry release 包生成 `provision-tools/`，再通过 `stash/unstash` 上传到 release 部署 agent。目标机上的 `scripts/jenkins-server-provision.sh` 只从该工作区工具包安装 `/stdb/spacetime`、`/stdb/bin/current/*` 和 `/usr/local/bin/otelcol-contrib`。注意 `scripts/jenkins-checkout-source.sh` 会执行 `git reset --hard` / `git clean`，因此被直接执行的新增脚本必须以 Git `100755` 模式提交，或在二次 checkout 之后再补 `chmod +x`。
- 构建机下载慢时，在 `Genarrative-Server-Provision` 参数 `PROVISION_DOWNLOAD_PROXY` 填写构建机可访问的 HTTP 代理，例如 `http://<proxy-host>:7890`。不要填写目标 release 机器视角的 `127.0.0.1`，除非代理确实运行在该构建机本机。
+- Server provision 不在目标机联网下载 SpacetimeDB 或 `otelcol-contrib`。`Genarrative-Server-Provision` 先在 Windows Jenkins 节点执行 `Download Provision Tool Archives`，把 `https://install.spacetimedb.com`、SpacetimeDB Linux update installer 和 `otelcol-contrib_0.151.0_linux_amd64.tar.gz` 先下载到工作区，再通过 `stash/unstash` 带到 `genarrative-build-01`；目标 Linux 节点上的 `scripts/prepare-server-provision-tools.sh` 只消费这些本地下载件生成 `provision-tools/`，再交给 `scripts/jenkins-server-provision.sh` 安装 `/stdb/spacetime`、`/stdb/bin/current/*` 和 `/usr/local/bin/otelcol-contrib`。注意 `scripts/jenkins-checkout-source.sh` 会执行 `git reset --hard` / `git clean`，因此被直接执行的新增脚本必须以 Git `100755` 模式提交，或在二次 checkout 之后再补 `chmod +x`。
+- Windows 下载阶段如果走代理，在 `Genarrative-Server-Provision` 参数 `PROVISION_DOWNLOAD_PROXY` 填写 Windows Jenkins 节点可访问的 HTTP 代理，例如 `http://127.0.0.1:7890`；不要填写目标 release 机器视角的 `127.0.0.1`，除非代理确实运行在该 Windows 节点本机。Linux 目标机阶段会强制要求使用本地下载件，缺少文件直接失败，不再回退到外网下载。
 - `otelcol-contrib.service` 作为可选系统服务加入 provision，默认监听 `127.0.0.1:4317/4318` 并使用 `deploy/otelcol/genarrative-debug.yaml`。api-server 是否发送 OTLP 仍由 `GENARRATIVE_OTEL_ENABLED` 控制，服务 unit 见 `deploy/systemd/otelcol-contrib.service`。
 - Nginx `/api/` 与 `/admin/api/` 通过 `genarrative_api` upstream 代理到 `127.0.0.1:8082`，upstream keepalive 为 64；`limit_conn` 负责连接 / 并发保护，`limit_req` 负责入口 RPS 快拒绝。当前模板把公开 gallery list 单独放到 `genarrative_gallery_rps`，默认 `rate=5000r/s`、`burst=4096`、`limit_conn=320`；公开详情和普通 API 放到 `genarrative_api_rps`，后台 API 放到 `genarrative_admin_rps`。压测时看 `/var/log/nginx/genarrative.access.log` 中的 `request_time`、`upstream_connect_time`、`upstream_header_time`、`upstream_response_time`、`upstream_status`、`request_id`。
 - 作品列表 K6 脚本一次 iteration 默认请求两个公开接口，因此约 50 HTTP req/s 的目标命令使用 `SCENARIO=spike START_RPS=5 PEAK_RPS=25 HOLD=60s END_RPS=5 DETAIL_RATIO=0 npm run loadtest:k6:works`。
@@ -179,7 +179,7 @@ npm run container:k6
 npm run container:down
 ```

-容器方案默认暴露 `http://127.0.0.1:18080`，`api-server` 在容器内监听 `0.0.0.0:8082`，Nginx 通过 `api-server:8082` upstream 反代 `/api/` 和 `/admin/api/`。SpacetimeDB 也纳入 compose，容器内由 `spacetimedb:3101` 提供服务，宿主机通过 `http://127.0.0.1:13101` 进行模块发布；Collector 镜像使用 `otel/opentelemetry-collector-contrib:0.151.0`。生产 provision 侧则通过 Jenkins 构建机准备的 `provision-tools/otelcol-contrib` 安装本机 `otelcol-contrib.service`，真实库名、token 和外部服务密钥只写本地 `deploy/container/api-server.env`，不提交 Git。完整拓扑、端口、k6 参数和 OTLP debug exporter 使用方法见 `deploy/container/README.md`。
+容器方案默认暴露 `http://127.0.0.1:18080`，`api-server` 在容器内监听 `0.0.0.0:8082`，Nginx 通过 `api-server:8082` upstream 反代 `/api/` 和 `/admin/api/`。SpacetimeDB 也纳入 compose，容器内由 `spacetimedb:3101` 提供服务，宿主机通过 `http://127.0.0.1:13101` 进行模块发布；Collector 镜像使用 `otel/opentelemetry-collector-contrib:0.151.0`。生产 provision 侧则通过 Windows Jenkins 下载件在目标 Linux 节点生成 `provision-tools/otelcol-contrib`，再安装本机 `otelcol-contrib.service`，真实库名、token 和外部服务密钥只写本地 `deploy/container/api-server.env`，不提交 Git。完整拓扑、端口、k6 参数和 OTLP debug exporter 使用方法见 `deploy/container/README.md`。
 `npm run container:config` 默认只做 quiet 校验，避免把本地 env 中的 token 展开到终端；确需排查完整 compose 时再传 `-- --print`。

 OpenTelemetry 现阶段默认开启 OTLP traces / metrics / logs，但本地日志与 Nginx 文件日志仍保留：
--- a/jenkins/Jenkinsfile.production-server-provision
+++ b/jenkins/Jenkinsfile.production-server-provision
@@ -1,3 +1,24 @@
+def runWindowsPowerShell(String scriptName, String scriptBody) {
+  def scriptPath = ".jenkins-${scriptName}.ps1"
+  writeFile file: scriptPath, text: scriptBody, encoding: 'UTF-8'
+  bat label: "PowerShell ${scriptName}", script: """
+@echo off
+setlocal
+set "GENARRATIVE_POWERSHELL=%SystemRoot%\\System32\\WindowsPowerShell\\v1.0\\powershell.exe"
+if not exist "%GENARRATIVE_POWERSHELL%" (
+  echo [jenkins-powershell] powershell.exe not found: %GENARRATIVE_POWERSHELL%
+  exit /b 1
+)
+echo [jenkins-powershell] user:
+whoami
+echo [jenkins-powershell] exe: %GENARRATIVE_POWERSHELL%
+"%GENARRATIVE_POWERSHELL%" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass -Command "\$path = '%CD%\\${scriptPath}'; \$text = [System.IO.File]::ReadAllText(\$path, [System.Text.Encoding]::UTF8); \$utf8Bom = New-Object System.Text.UTF8Encoding(\$true); [System.IO.File]::WriteAllText(\$path, \$text, \$utf8Bom)"
+if errorlevel 1 exit /b %ERRORLEVEL%
+"%GENARRATIVE_POWERSHELL%" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass -File "%CD%\\${scriptPath}"
+exit /b %ERRORLEVEL%
+"""
+}
+
 pipeline {
  agent none

@@ -22,9 +43,12 @@ pipeline {
    string(name: 'COMMIT_HASH', defaultValue: '', description: '部署脚本来源 commit')
    string(name: 'SERVER_NAME', defaultValue: 'genarrative.example.com', description: '证书主域名；也作为 Nginx server_name 的第一个域名')
    string(name: 'SERVER_ALIASES', defaultValue: '', description: '可选，额外 Nginx server_name，多个用空格或逗号分隔，例如 www.genarrative.world')
-    string(name: 'PROVISION_TOOLS_DIR', defaultValue: 'provision-tools', description: '构建机准备并上传到目标机工作区的工具包目录')
-    string(name: 'PROVISION_DOWNLOAD_PROXY', defaultValue: '', description: '可选，构建机下载 SpacetimeDB 和 otelcol-contrib 时使用的代理地址，例如 http://proxy-host:7890；留空不设置代理')
-    string(name: 'SPACETIME_DOWNLOAD_ROOT', defaultValue: 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download', description: '构建机下载 SpacetimeDB 官方安装产物的根地址；目标机不访问该地址')
+    string(name: 'PROVISION_DOWNLOADS_DIR', defaultValue: 'provision-tool-downloads', description: 'Windows 下载阶段暂存 SpacetimeDB/otelcol 安装包的工作区相对目录')
+    string(name: 'PROVISION_TOOLS_DIR', defaultValue: 'provision-tools', description: '目标机工作区内由已下载安装包生成的工具包目录')
+    string(name: 'PROVISION_DOWNLOAD_PROXY', defaultValue: '', description: '可选，Windows 下载 SpacetimeDB 和 otelcol-contrib 时使用的代理地址，例如 http://127.0.0.1:7890；留空不设置代理')
+    string(name: 'SPACETIME_INSTALLER_URL', defaultValue: 'https://install.spacetimedb.com', description: 'Windows 下载 SpacetimeDB 官方安装脚本的地址；目标机不访问该地址')
+    string(name: 'SPACETIME_DOWNLOAD_ROOT', defaultValue: 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download', description: 'Windows 下载 SpacetimeDB Linux update installer 的根地址；目标机不访问该地址')
+    string(name: 'SPACETIME_TARGET_HOST', defaultValue: 'x86_64-unknown-linux-gnu', description: '目标机 SpacetimeDB 预编译包 host triple，development/release Linux amd64 使用默认值')
    string(name: 'SPACETIME_ROOT', defaultValue: '/stdb', description: 'SpacetimeDB root-dir')
    string(name: 'RELEASE_ROOT', defaultValue: '/opt/genarrative/releases', description: 'release 根目录')
    string(name: 'CURRENT_LINK', defaultValue: '/opt/genarrative/current', description: '当前版本软链接')
@@ -40,7 +64,7 @@ pipeline {
  stages {
    stage('Prepare') {
      agent {
-        label 'linux && genarrative-build'
+        label 'windows'
      }
      steps {
        script {
@@ -67,9 +91,20 @@ pipeline {
          if (!params.PROVISION_TOOLS_DIR?.trim()) {
            error('PROVISION_TOOLS_DIR 不能为空。')
          }
-          if (!(params.PROVISION_TOOLS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_TOOLS_DIR.startsWith('/') || params.PROVISION_TOOLS_DIR.contains('..')) {
+          if (!(params.PROVISION_TOOLS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_TOOLS_DIR.startsWith('/') || params.PROVISION_TOOLS_DIR.contains('..') || params.PROVISION_TOOLS_DIR.trim() == '.') {
            error("PROVISION_TOOLS_DIR 只能是工作区内的相对目录，不能包含绝对路径或连续点号: ${params.PROVISION_TOOLS_DIR}")
          }
+          if (!params.PROVISION_DOWNLOADS_DIR?.trim()) {
+            error('PROVISION_DOWNLOADS_DIR 不能为空。')
+          }
+          if (!(params.PROVISION_DOWNLOADS_DIR.trim() ==~ /^[0-9A-Za-z._\/-]+$/) || params.PROVISION_DOWNLOADS_DIR.startsWith('/') || params.PROVISION_DOWNLOADS_DIR.contains('..') || params.PROVISION_DOWNLOADS_DIR.trim() == '.') {
+            error("PROVISION_DOWNLOADS_DIR 只能是工作区内的相对目录，不能包含绝对路径或连续点号: ${params.PROVISION_DOWNLOADS_DIR}")
+          }
+          def provisionToolsDir = params.PROVISION_TOOLS_DIR.trim()
+          def provisionDownloadsDir = params.PROVISION_DOWNLOADS_DIR.trim()
+          if (provisionToolsDir == provisionDownloadsDir || provisionDownloadsDir.startsWith("${provisionToolsDir}/")) {
+            error("PROVISION_DOWNLOADS_DIR 不能等于或位于 PROVISION_TOOLS_DIR 内，否则目标机生成工具包时会删除下载缓存: ${provisionDownloadsDir}")
+          }
          def provisionDownloadProxy = params.PROVISION_DOWNLOAD_PROXY?.trim()
          if (provisionDownloadProxy && !(provisionDownloadProxy ==~ /^https?:\/\/\S+$/)) {
            error("PROVISION_DOWNLOAD_PROXY 只能填写 http:// 或 https:// 开头的代理地址，当前值: ${params.PROVISION_DOWNLOAD_PROXY}")
@@ -77,9 +112,15 @@ pipeline {
          if (!(params.OTELCOL_VERSION?.trim() ==~ /^[0-9]+\.[0-9]+\.[0-9]+$/)) {
            error("OTELCOL_VERSION 格式应为 x.y.z: ${params.OTELCOL_VERSION}")
          }
-          if (!params.SPACETIME_DOWNLOAD_ROOT?.trim()) {
+          if (!(params.SPACETIME_INSTALLER_URL?.trim() ==~ /^https?:\/\/\S+$/)) {
+            error("SPACETIME_INSTALLER_URL 只能填写 http:// 或 https:// 开头的地址: ${params.SPACETIME_INSTALLER_URL}")
+          }
+          if (!(params.SPACETIME_DOWNLOAD_ROOT?.trim() ==~ /^https?:\/\/\S+$/)) {
            error('SPACETIME_DOWNLOAD_ROOT 不能为空。')
          }
+          if (!(params.SPACETIME_TARGET_HOST?.trim() ==~ /^[0-9A-Za-z._-]+$/)) {
+            error("SPACETIME_TARGET_HOST 只能包含字母、数字、点号、下划线和短横线: ${params.SPACETIME_TARGET_HOST}")
+          }
          def nginxMode = params.NGINX_CONFIG_MODE?.trim()
          if (!(nginxMode in ['none', 'production-https', 'development-http'])) {
            error("NGINX_CONFIG_MODE 只能是 none、production-https 或 development-http，当前值: ${params.NGINX_CONFIG_MODE}")
@@ -94,65 +135,106 @@ pipeline {
      }
    }

-    stage('Prepare Provision Tools') {
+    stage('Download Provision Tool Archives') {
      agent {
-        label 'linux && genarrative-build'
+        label 'windows'
      }
      steps {
        script {
-          def checkoutFromRemote = { String remoteUrl ->
-            checkout([
-              $class: 'GitSCM',
-              branches: [[name: "*/${params.SOURCE_BRANCH}"]],
-              doGenerateSubmoduleConfigurations: false,
-              extensions: [
-                [$class: 'CleanBeforeCheckout'],
-                [$class: 'CloneOption', shallow: true, depth: 1, noTags: true, timeout: 30, honorRefspec: true],
-              ],
-              userRemoteConfigs: [[url: remoteUrl, refspec: "+refs/heads/${params.SOURCE_BRANCH}:refs/remotes/origin/${params.SOURCE_BRANCH}"]],
-            ])
-          }
-          try {
-            checkoutFromRemote(env.GIT_REMOTE_URL)
-            env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_URL
-          } catch (error) {
-            echo "Git 主地址拉取失败: ${env.GIT_REMOTE_URL}，改用备用地址: ${env.GIT_REMOTE_FALLBACK_URL}"
-            checkoutFromRemote(env.GIT_REMOTE_FALLBACK_URL)
-            env.EFFECTIVE_GIT_REMOTE_URL = env.GIT_REMOTE_FALLBACK_URL
-          }
-        }
-        sh '''
-          bash <<'BASH'
-            set -euo pipefail
-            chmod +x scripts/jenkins-checkout-source.sh
-            SOURCE_BRANCH="${SOURCE_BRANCH:-master}" \
-            COMMIT_HASH="${COMMIT_HASH:-}" \
-            GIT_REMOTE_URL="${EFFECTIVE_GIT_REMOTE_URL:-${GIT_REMOTE_URL}}" \
-            GIT_REMOTE_FALLBACK_URL="${GIT_REMOTE_FALLBACK_URL:-}" \
-            SOURCE_COMMIT_FILE=".jenkins-source-commit" \
-            scripts/jenkins-checkout-source.sh
-            # jenkins-checkout-source.sh 会 reset/clean 到目标 commit，前面的临时 chmod 可能被 Git mode 还原；
-            # 直接执行脚本前在二次 checkout 之后再补执行位，避免 Linux agent 报 Permission denied。
-            chmod +x scripts/prepare-server-provision-tools.sh
-BASH
-        '''
-        sh '''
-          bash <<'BASH'
-            set -euo pipefail
+          runWindowsPowerShell('server-provision-tool-downloads', '''
+            $ErrorActionPreference = 'Stop'
+            [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

-            PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \
-            OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}" \
-            PREPARE_OTELCOL="${ENABLE_OTELCOL:-true}" \
-            PROVISION_DOWNLOAD_PROXY="${PROVISION_DOWNLOAD_PROXY:-}" \
-            SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}" \
-            scripts/prepare-server-provision-tools.sh
-BASH
-        '''
-        script {
-          env.SOURCE_COMMIT = readFile('.jenkins-source-commit').trim()
-          echo "Provision 工具包已准备，源码 commit=${env.SOURCE_COMMIT}"
+            $downloadsDir = if ($env:PROVISION_DOWNLOADS_DIR) { $env:PROVISION_DOWNLOADS_DIR } else { 'provision-tool-downloads' }
+            $otelVersion = if ($env:OTELCOL_VERSION) { $env:OTELCOL_VERSION } else { '0.151.0' }
+            $prepareOtel = if ($env:ENABLE_OTELCOL) { $env:ENABLE_OTELCOL } else { 'true' }
+            $otelRoot = if ($env:OTELCOL_DOWNLOAD_ROOT) { $env:OTELCOL_DOWNLOAD_ROOT.TrimEnd('/') } else { 'https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download' }
+            $spacetimeInstallerUrl = if ($env:SPACETIME_INSTALLER_URL) { $env:SPACETIME_INSTALLER_URL } else { 'https://install.spacetimedb.com' }
+            $spacetimeDownloadRoot = if ($env:SPACETIME_DOWNLOAD_ROOT) { $env:SPACETIME_DOWNLOAD_ROOT.TrimEnd('/') } else { 'https://github.com/clockworklabs/SpacetimeDB/releases/latest/download' }
+            $spacetimeTargetHost = if ($env:SPACETIME_TARGET_HOST) { $env:SPACETIME_TARGET_HOST } else { 'x86_64-unknown-linux-gnu' }
+            $downloadProxy = if ($env:PROVISION_DOWNLOAD_PROXY) { $env:PROVISION_DOWNLOAD_PROXY } else { '' }
+
+            if (Test-Path -LiteralPath $downloadsDir) {
+              Remove-Item -LiteralPath $downloadsDir -Recurse -Force
+            }
+            New-Item -ItemType Directory -Force -Path $downloadsDir | Out-Null
+
+            if ($downloadProxy) {
+              $env:HTTP_PROXY = $downloadProxy
+              $env:HTTPS_PROXY = $downloadProxy
+              $env:ALL_PROXY = $downloadProxy
+              Write-Host "[prepare-provision-downloads] 已配置 Windows 下载代理: $($downloadProxy -replace '://.*', '://***')"
+            }
+
+            function Invoke-ProvisionDownload {
+              param(
+                [Parameter(Mandatory=$true)][string]$Label,
+                [Parameter(Mandatory=$true)][string]$Url,
+                [Parameter(Mandatory=$true)][string]$Output
+              )
+
+              Write-Host "[prepare-provision-downloads] 下载 ${Label}: ${Url}"
+              $curl = Get-Command curl.exe -ErrorAction SilentlyContinue
+              if ($curl) {
+                $arguments = @('-fL', '--retry', '3', '--retry-delay', '2', '-o', $Output)
+                if ($downloadProxy) {
+                  $arguments += @('--proxy', $downloadProxy)
+                }
+                $arguments += $Url
+                & $curl.Source @arguments
+                $exitCode = $LASTEXITCODE
+                if ($exitCode -ne 0) {
+                  throw "[prepare-provision-downloads] curl 下载失败: ${Label}, exit=${exitCode}"
+                }
+              } else {
+                $parameters = @{
+                  Uri = $Url
+                  OutFile = $Output
+                  UseBasicParsing = $true
+                }
+                if ($downloadProxy) {
+                  $parameters.Proxy = $downloadProxy
+                }
+                Invoke-WebRequest @parameters
+              }
+
+              $item = Get-Item -LiteralPath $Output
+              if ($item.Length -le 0) {
+                throw "[prepare-provision-downloads] 下载结果为空: ${Output}"
+              }
+              Write-Host "[prepare-provision-downloads] 已下载 ${Label}: bytes=$($item.Length) path=${Output}"
+            }
+
+            $installerPath = Join-Path $downloadsDir 'spacetime-install.sh'
+            Invoke-ProvisionDownload -Label 'SpacetimeDB install script' -Url $spacetimeInstallerUrl -Output $installerPath
+
+            $spacetimeUpdateName = "spacetimedb-update-${spacetimeTargetHost}"
+            $spacetimeUpdateUrl = "${spacetimeDownloadRoot}/${spacetimeUpdateName}"
+            Invoke-ProvisionDownload -Label "SpacetimeDB Linux update installer ${spacetimeTargetHost}" -Url $spacetimeUpdateUrl -Output (Join-Path $downloadsDir $spacetimeUpdateName)
+
+            if ($prepareOtel -eq 'true') {
+              $otelArchiveName = "otelcol-contrib_${otelVersion}_linux_amd64.tar.gz"
+              $otelUrl = "${otelRoot}/v${otelVersion}/${otelArchiveName}"
+              Invoke-ProvisionDownload -Label "otelcol-contrib ${otelVersion} linux amd64" -Url $otelUrl -Output (Join-Path $downloadsDir $otelArchiveName)
+            } else {
+              Write-Host "[prepare-provision-downloads] ENABLE_OTELCOL=${prepareOtel}，跳过 otelcol-contrib 下载。"
+            }
+
+            $utf8NoBom = New-Object System.Text.UTF8Encoding $false
+            $manifest = @(
+              "spacetime installer ${spacetimeInstallerUrl}",
+              "spacetime update ${spacetimeDownloadRoot}/${spacetimeUpdateName}",
+              "spacetime target host ${spacetimeTargetHost}",
+              "otelcol-contrib ${otelVersion} prepare=${prepareOtel}"
+            )
+            [System.IO.File]::WriteAllLines((Join-Path $downloadsDir 'DOWNLOADS-MANIFEST.txt'), $manifest, $utf8NoBom)
+
+            Get-ChildItem -LiteralPath $downloadsDir | Sort-Object Name | ForEach-Object {
+              Write-Host "[prepare-provision-downloads] artifact $($_.Length) $($_.Name)"
+            }
+          ''')
        }
-        stash name: 'server-provision-tools', includes: "${params.PROVISION_TOOLS_DIR}/**", useDefaultExcludes: false
+        stash name: 'server-provision-tool-downloads', includes: "${params.PROVISION_DOWNLOADS_DIR}/**", useDefaultExcludes: false
      }
    }

@@ -195,6 +277,10 @@ BASH
            scripts/jenkins-checkout-source.sh
 BASH
        '''
+        script {
+          env.SOURCE_COMMIT = readFile('.jenkins-source-commit').trim()
+          echo "Provision 源码 commit=${env.SOURCE_COMMIT}"
+        }
      }
    }

@@ -203,10 +289,21 @@ BASH
        label "${params.DEPLOY_TARGET == 'development' ? 'linux && genarrative-build' : 'linux && genarrative-release-deploy'}"
      }
      steps {
-        unstash 'server-provision-tools'
+        unstash 'server-provision-tool-downloads'
        sh '''
          bash <<'BASH'
            set -euo pipefail
+            chmod +x scripts/prepare-server-provision-tools.sh
+            PROVISION_DOWNLOADS_DIR="${PROVISION_DOWNLOADS_DIR:-provision-tool-downloads}" \
+            PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}" \
+            OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}" \
+            PREPARE_OTELCOL="${ENABLE_OTELCOL:-true}" \
+            PROVISION_REQUIRE_LOCAL_DOWNLOADS="true" \
+            SPACETIME_INSTALLER_URL="${SPACETIME_INSTALLER_URL:-https://install.spacetimedb.com}" \
+            SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}" \
+            SPACETIME_TARGET_HOST="${SPACETIME_TARGET_HOST:-x86_64-unknown-linux-gnu}" \
+            scripts/prepare-server-provision-tools.sh
+
            if [[ "${ENABLE_OTELCOL:-true}" == "true" ]]; then
              chmod +x "${PROVISION_TOOLS_DIR:-provision-tools}/otelcol-contrib"
            fi
--- a/scripts/prepare-server-provision-tools.sh
+++ b/scripts/prepare-server-provision-tools.sh
@@ -2,16 +2,23 @@
 set -euo pipefail

 PROVISION_TOOLS_DIR="${PROVISION_TOOLS_DIR:-provision-tools}"
+PROVISION_DOWNLOADS_DIR="${PROVISION_DOWNLOADS_DIR:-}"
 OTELCOL_VERSION="${OTELCOL_VERSION:-0.151.0}"
 PREPARE_OTELCOL="${PREPARE_OTELCOL:-${ENABLE_OTELCOL:-true}}"
 OTELCOL_DOWNLOAD_ROOT="${OTELCOL_DOWNLOAD_ROOT:-https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download}"
+OTELCOL_ARCHIVE_PATH="${OTELCOL_ARCHIVE_PATH:-}"
 SPACETIME_INSTALLER_URL="${SPACETIME_INSTALLER_URL:-https://install.spacetimedb.com}"
 SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT:-https://github.com/clockworklabs/SpacetimeDB/releases/latest/download}"
+SPACETIME_TARGET_HOST="${SPACETIME_TARGET_HOST:-x86_64-unknown-linux-gnu}"
+SPACETIME_INSTALLER_PATH="${SPACETIME_INSTALLER_PATH:-}"
+SPACETIME_UPDATE_INSTALLER_PATH="${SPACETIME_UPDATE_INSTALLER_PATH:-}"
 PROVISION_DOWNLOAD_PROXY="${PROVISION_DOWNLOAD_PROXY:-}"
 PROVISION_NO_PROXY="${PROVISION_NO_PROXY:-127.0.0.1,localhost}"
+PROVISION_REQUIRE_LOCAL_DOWNLOADS="${PROVISION_REQUIRE_LOCAL_DOWNLOADS:-false}"
 PROVISION_TOOLS_TMP_PARENT="${PROVISION_TOOLS_TMP_PARENT:-${WORKSPACE:-$(pwd)}/.tmp/server-provision-tools}"
 TMP_DIR_TO_CLEAN=""
 OTELCOL_SOURCE_DESCRIPTION="skipped"
+SPACETIME_SOURCE_DESCRIPTION="unset"

 cleanup_tmp_dir() {
  if [[ -n "${TMP_DIR_TO_CLEAN}" ]]; then
@@ -57,6 +64,19 @@ download_file() {
  fi
 }

+validate_relative_dir() {
+  local label="$1"
+  local path="$2"
+
+  if [[ -z "${path}" ]]; then
+    return
+  fi
+  if [[ "${path}" == /* || "${path}" == *..* || "${path}" == "." ]]; then
+    echo "[prepare-provision-tools] ${label} 只能是工作区内的相对路径: ${path}" >&2
+    exit 1
+  fi
+}
+
 make_spacetime_wrapper() {
  local target="$1"

@@ -74,14 +94,32 @@ prepare_otelcol() {
  local archive="${tmp_dir}/otelcol-contrib.tar.gz"
  local extract_dir="${tmp_dir}/otelcol-contrib"
  local url="${OTELCOL_DOWNLOAD_ROOT}/v${OTELCOL_VERSION}/otelcol-contrib_${OTELCOL_VERSION}_linux_amd64.tar.gz"
+  local downloaded_archive="${PROVISION_DOWNLOADS_DIR}/otelcol-contrib_${OTELCOL_VERSION}_linux_amd64.tar.gz"
+  local source_archive=""
  local target="${PROVISION_TOOLS_DIR}/otelcol-contrib"

  require_cmd tar

-  echo "[prepare-provision-tools] 下载 otelcol-contrib: ${url}"
+  if [[ -n "${OTELCOL_ARCHIVE_PATH}" && -f "${OTELCOL_ARCHIVE_PATH}" ]]; then
+    source_archive="${OTELCOL_ARCHIVE_PATH}"
+  elif [[ -n "${PROVISION_DOWNLOADS_DIR}" && -f "${downloaded_archive}" ]]; then
+    source_archive="${downloaded_archive}"
+  fi
+  if [[ "${PROVISION_REQUIRE_LOCAL_DOWNLOADS}" == "true" && -z "${source_archive}" ]]; then
+    echo "[prepare-provision-tools] 要求使用 Windows 已下载的 otelcol-contrib 包，但未找到: ${downloaded_archive}" >&2
+    exit 1
+  fi
+
  mkdir -p "${extract_dir}"
-  download_file "${url}" "${archive}"
-  OTELCOL_SOURCE_DESCRIPTION="download ${url}"
+  if [[ -n "${source_archive}" ]]; then
+    echo "[prepare-provision-tools] 使用已下载的 otelcol-contrib 包: ${source_archive}"
+    cp "${source_archive}" "${archive}"
+    OTELCOL_SOURCE_DESCRIPTION="local ${source_archive}"
+  else
+    echo "[prepare-provision-tools] 下载 otelcol-contrib: ${url}"
+    download_file "${url}" "${archive}"
+    OTELCOL_SOURCE_DESCRIPTION="download ${url}"
+  fi
  tar -xzf "${archive}" -C "${extract_dir}"

  if [[ ! -x "${extract_dir}/otelcol-contrib" ]]; then
@@ -97,12 +135,51 @@ prepare_spacetime() {
  local tmp_dir="$1"
  local install_root="${tmp_dir}/spacetime-root"
  local target_dir="${PROVISION_TOOLS_DIR}/spacetime"
+  local update_name="spacetimedb-update-${SPACETIME_TARGET_HOST}"
+  local downloaded_update="${PROVISION_DOWNLOADS_DIR}/${update_name}"
+  local source_update=""
+  local prepared_update="${tmp_dir}/${update_name}"
+  local downloaded_installer="${PROVISION_DOWNLOADS_DIR}/spacetime-install.sh"
+  local source_installer=""

-  echo "[prepare-provision-tools] 使用官方安装器准备 SpacetimeDB: ${SPACETIME_INSTALLER_URL}"
  mkdir -p "${install_root}"
-  download_file "${SPACETIME_INSTALLER_URL}" "${tmp_dir}/spacetime-install.sh"
-  chmod 0755 "${tmp_dir}/spacetime-install.sh"
-  TMPDIR="${tmp_dir}" SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT}" sh "${tmp_dir}/spacetime-install.sh" --root-dir "${install_root}" -y
+  if [[ -n "${SPACETIME_UPDATE_INSTALLER_PATH}" && -f "${SPACETIME_UPDATE_INSTALLER_PATH}" ]]; then
+    source_update="${SPACETIME_UPDATE_INSTALLER_PATH}"
+  elif [[ -n "${PROVISION_DOWNLOADS_DIR}" && -f "${downloaded_update}" ]]; then
+    source_update="${downloaded_update}"
+  fi
+  if [[ "${PROVISION_REQUIRE_LOCAL_DOWNLOADS}" == "true" && -z "${source_update}" ]]; then
+    echo "[prepare-provision-tools] 要求使用 Windows 已下载的 SpacetimeDB Linux update installer，但未找到: ${downloaded_update}" >&2
+    exit 1
+  fi
+
+  if [[ -n "${source_update}" ]]; then
+    echo "[prepare-provision-tools] 使用已下载的 SpacetimeDB Linux update installer: ${source_update}"
+    cp "${source_update}" "${prepared_update}"
+    chmod 0755 "${prepared_update}"
+    TMPDIR="${tmp_dir}" "${prepared_update}" --root-dir "${install_root}" -y
+    SPACETIME_SOURCE_DESCRIPTION="local update ${source_update}"
+  else
+    if [[ -n "${SPACETIME_INSTALLER_PATH}" && -f "${SPACETIME_INSTALLER_PATH}" ]]; then
+      source_installer="${SPACETIME_INSTALLER_PATH}"
+    elif [[ -n "${PROVISION_DOWNLOADS_DIR}" && -f "${downloaded_installer}" ]]; then
+      source_installer="${downloaded_installer}"
+    fi
+
+    if [[ "${PROVISION_REQUIRE_LOCAL_DOWNLOADS}" == "true" && -z "${source_installer}" ]]; then
+      echo "[prepare-provision-tools] 要求使用 Windows 已下载的 SpacetimeDB 官方安装器脚本，但未找到: ${downloaded_installer}" >&2
+      exit 1
+    elif [[ -n "${source_installer}" ]]; then
+      echo "[prepare-provision-tools] 使用已下载的 SpacetimeDB 官方安装器脚本: ${source_installer}"
+      cp "${source_installer}" "${tmp_dir}/spacetime-install.sh"
+    else
+      echo "[prepare-provision-tools] 下载 SpacetimeDB 官方安装器脚本: ${SPACETIME_INSTALLER_URL}"
+      download_file "${SPACETIME_INSTALLER_URL}" "${tmp_dir}/spacetime-install.sh"
+    fi
+    chmod 0755 "${tmp_dir}/spacetime-install.sh"
+    TMPDIR="${tmp_dir}" SPACETIME_DOWNLOAD_ROOT="${SPACETIME_DOWNLOAD_ROOT}" sh "${tmp_dir}/spacetime-install.sh" --root-dir "${install_root}" -y
+    SPACETIME_SOURCE_DESCRIPTION="installer ${SPACETIME_INSTALLER_URL}; download root ${SPACETIME_DOWNLOAD_ROOT}"
+  fi

  if [[ ! -x "${install_root}/bin/current/spacetimedb-cli" ]]; then
    echo "[prepare-provision-tools] SpacetimeDB 安装结果缺少 bin/current/spacetimedb-cli。" >&2
@@ -129,6 +206,13 @@ main() {
  require_cmd mktemp
  require_cmd rm

+  validate_relative_dir "PROVISION_TOOLS_DIR" "${PROVISION_TOOLS_DIR}"
+  validate_relative_dir "PROVISION_DOWNLOADS_DIR" "${PROVISION_DOWNLOADS_DIR}"
+  if [[ -n "${PROVISION_DOWNLOADS_DIR}" && "${PROVISION_DOWNLOADS_DIR%/}" == "${PROVISION_TOOLS_DIR%/}" ]]; then
+    echo "[prepare-provision-tools] PROVISION_DOWNLOADS_DIR 不能等于 PROVISION_TOOLS_DIR，否则会被清理: ${PROVISION_DOWNLOADS_DIR}" >&2
+    exit 1
+  fi
+
  configure_download_proxy

  mkdir -p "${PROVISION_TOOLS_TMP_PARENT}"
@@ -148,8 +232,9 @@ main() {

  cat >"${PROVISION_TOOLS_DIR}/MANIFEST.txt" <<EOF
 otelcol-contrib ${OTELCOL_VERSION} ${OTELCOL_SOURCE_DESCRIPTION}
-spacetime installer ${SPACETIME_INSTALLER_URL}
-spacetime download root ${SPACETIME_DOWNLOAD_ROOT}
+spacetime ${SPACETIME_SOURCE_DESCRIPTION}
+spacetime target host ${SPACETIME_TARGET_HOST}
+require local downloads ${PROVISION_REQUIRE_LOCAL_DOWNLOADS}
 EOF

  echo "[prepare-provision-tools] 工具包已准备: ${PROVISION_TOOLS_DIR}"