Files
Genarrative/apps/ai-game-creator-shell/src-tauri/src/browser/cdp.rs
T
kdletters 485ed50b26
Project CI / AI game creator shell Rust smoke (push) Has been cancelled
Project CI / AI game creator shell Rust crates (push) Has been cancelled
Project CI / Backend tests (push) Has been cancelled
Project CI / Native shell tests (push) Has been cancelled
Project CI / Frontend tests (push) Has been cancelled
Project CI / Repository checks (push) Has been cancelled
Project CI / AI game creator shell web tests (push) Has been cancelled
Project CI / AI game creator shell Rust shard 2/4 (push) Has been cancelled
Project CI / AI game creator shell Rust shard 1/4 (push) Has been cancelled
Project CI / AI game creator shell Rust shard 3/4 (push) Has been cancelled
Project CI / AI game creator shell Rust shard 4/4 (push) Has been cancelled
AGC 七项交付效率优化并升级捆绑 Codex 到 0.155.1 (#439)
## 背景

净月潭案例(2026-09-19 21:53 → 09-20 01:26,3 小时 33 分)的问题不是模型慢,而是环境未就绪、验收靠模型自述、预算只约束单个工具入口、工具调用被 SDK 全局串行闸门卡住。本 PR 落地确认后的七项交付效率合同,并把捆绑 Codex 升到当前 npm latest。

## 结果

- 新建 Web 游戏在正式生成前由宿主自动预检:捆绑 Node/npm、真实 Vite 构建、受限浏览器桌面/移动截图;失败不启动生成或付费素材。
- 首个副作用前冻结交付合同,宿主保存权威证据与预算;证据齐全后先封口、排空并取得执行器完整退出证明,再产出交付报告,模型回复不再当作验收。
- 原生 shell、内置浏览器与托管命令、第三方 MCP 共用同一执行许可和累计执行时间;`validation.maxRuns` 改为执行/返修批次语义,另设 `maxTurnSeconds` 墙钟上限。
- 所有工具可并发:移除 SDK 全局串行的 `apply_patch`/`update_plan` 注册,改由宿主 `agc_apply_patch`(官方 parser + 当前回合写许可 + 受控进程树 + 短项目事务)和 `agc_update_plan` 提供等价能力;真实请求目录里已无串行注册,长 MCP 与补丁/计划实测在同一响应内重叠。
- 付费提交与本地写入绑定原回合原租约:容量与同动作锁等待可取消,封口、取消或预算耗尽后零新增提交;已越过提交边界的请求保留 operation ID 走 GET 对账,不自动重放。
- 新增请求与工具分段计时、有界并行批读和首轮上下文预取,未知耗时不补零。
- 捆绑 Codex 0.147.0 → 0.155.1:固定版本收敛到 `build_support/codex_bundle.rs` 单一声明,vendor 解析源码按 `rust-v0.155.1` 逐字节重取并更新 UPSTREAM 证据,适配 0.155 统一 exec(`exec_command`/`write_stdin`);macOS 侧车最小系统版本仍为 15.0。

## 验证

- 生产 CLI → 捆绑 Codex 0.155.1 → 本地 Responses/MCP 夹具 9/9:补丁、完成收尾、批次、只读/可写 MCP 并发、原生命令并发、原生资源并发、统一 exec 会话、期限终止(真实重叠 775 / 764 / 999 ms)。
- 真实模型目录 2/2;vendor 上游库 111 项;宿主定向与回归 106 项;真实 Windows 进程与取消 21 项;Node 侧门禁 52 项。
- 发行载荷:artifact-only 重建 NSIS,解包验证侧车清单 `codex-cli 0.155.1`、6 个组件哈希、打包后 `bin/codex.exe --version`、Node 运行时 2130 文件与双端 PNG、`--environment-check` ready。
- `check-config`、TypeScript、编码(4991 文件)、文档索引、`git diff --check`、`cargo fmt --check` 全部通过。

## 未覆盖

- 真实陶泥儿登录态 Provider 生成尚未执行(本机 `--llm-status` 返回 `authentication-required`)。
- macOS 侧车只在本机做静态 Mach-O 与清单解析,未在 macOS 上跑 `check-macos-bundle.mjs`。
- 全仓库聚合套件在本机仍因负载敏感的后台 mock-provider 用例而红,与本次改动无关:改动前的旧二进制同样失败,换单线程后相关用例 3/3 通过。

---------

Co-authored-by: kdletters <61648117+kdletters@users.noreply.github.com>
Reviewed-on: http://192.168.35.82/git/GenarrativeAI/Genarrative/pulls/439
2026-09-21 02:29:29 +08:00

363 lines
14 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
use std::time::Duration;
use chromiumoxide::browser::Browser;
use chromiumoxide::cdp::browser_protocol::browser::{
SetDownloadBehaviorBehavior, SetDownloadBehaviorParams,
};
use chromiumoxide::cdp::browser_protocol::emulation::{
SetDeviceMetricsOverrideParams, SetTouchEmulationEnabledParams,
};
use chromiumoxide::cdp::browser_protocol::network::SetBypassServiceWorkerParams;
use chromiumoxide::cdp::browser_protocol::page::CaptureScreenshotFormat;
use chromiumoxide::page::ScreenshotParams;
use url::Url;
use super::capture::{
build_snapshot_script, canvas_validation_diagnostic, sanitize_url, start_capture_tasks,
truncate_chars, BrowserCanvasSnapshot, PageSnapshot, RENDERED_IMAGE_CAPTURE_SCRIPT,
RESTRICTION_SCRIPT,
};
use super::evidence::write_atomic;
use super::model::{
required_viewport_playtests_passed, BrowserExpectedTextMatch, BrowserIdentity,
BrowserPlaytestResult, BrowserValidationEvidencePaths, BrowserValidationInput,
BrowserValidationResult, BrowserValidationViewport, BrowserViewportPlaytestResult,
BrowserViewportValidationResult, DiscoveredBrowser, BROWSER_TIMEOUT, REQUIRED_VIEWPORTS,
RESULT_SCHEMA_VERSION,
};
use super::network_policy::same_preview_origin;
use super::playtest::run_viewport_playtest;
struct BrowserViewportValidationOutcome {
result: BrowserViewportValidationResult,
playtest: Option<BrowserPlaytestResult>,
}
pub(super) async fn run_browser_validation(
browser: &Browser,
discovered: &DiscoveredBrowser,
preview_url: &Url,
input: &BrowserValidationInput,
advisory_interaction: bool,
) -> Result<BrowserValidationResult, String> {
browser
.execute(SetDownloadBehaviorParams::new(
SetDownloadBehaviorBehavior::Deny,
))
.await
.map_err(|error| format!("禁用浏览器下载失败:{error}"))?;
let version = browser
.version()
.await
.map_err(|error| format!("读取浏览器版本失败:{error}"))?;
let mut viewport_results = Vec::with_capacity(REQUIRED_VIEWPORTS.len());
let mut playtest = None;
let mut viewport_playtests = Vec::new();
for viewport in REQUIRED_VIEWPORTS {
let outcome =
validate_viewport(browser, preview_url, input, viewport, advisory_interaction).await?;
if let Some(result) = outcome.playtest {
if viewport == BrowserValidationViewport::Desktop {
playtest = Some(result.clone());
}
viewport_playtests.push(BrowserViewportPlaytestResult { viewport, result });
}
viewport_results.push(outcome.result);
}
let mut diagnostics = viewport_results
.iter()
.flat_map(|result| {
result
.diagnostics
.iter()
.map(move |message| format!("{}: {message}", result.viewport.file_stem()))
})
.collect::<Vec<_>>();
let playtest_passed = if let Some(scenario) = input.playtest_scenario {
for viewport in REQUIRED_VIEWPORTS {
match viewport_playtests
.iter()
.find(|entry| entry.viewport == viewport)
{
Some(entry) if !entry.result.passed => diagnostics.extend(
entry
.result
.diagnostics
.iter()
.map(|message| format!("playtest {}: {message}", viewport.file_stem())),
),
None => {
diagnostics.push(format!("playtest: {} 试玩结果缺失", viewport.file_stem()))
}
_ => {}
}
}
required_viewport_playtests_passed(scenario, &viewport_playtests)
} else {
viewport_playtests.is_empty()
};
let passed = viewport_results.iter().all(|result| result.passed) && playtest_passed;
let report_path = input.evidence_root.join("validation.json");
Ok(BrowserValidationResult {
schema_version: RESULT_SCHEMA_VERSION.to_string(),
url: preview_url.as_str().to_string(),
browser: BrowserIdentity {
kind: discovered.kind,
product: version.product,
protocol_version: version.protocol_version,
},
passed,
viewport_results,
playtest,
viewport_playtests,
diagnostics,
evidence: BrowserValidationEvidencePaths {
root: input.evidence_root.clone(),
report_path,
},
completed_at_unix_ms: 0,
})
}
async fn validate_viewport(
browser: &Browser,
preview_url: &Url,
input: &BrowserValidationInput,
viewport: BrowserValidationViewport,
advisory_interaction: bool,
) -> Result<BrowserViewportValidationOutcome, String> {
let (width, height, mobile) = viewport.dimensions();
let page = browser
.new_page("about:blank")
.await
.map_err(|error| format!("创建 {} 页面失败:{error}", viewport.file_stem()))?;
page.execute(SetDeviceMetricsOverrideParams::new(
i64::from(width),
i64::from(height),
1.0,
mobile,
))
.await
.map_err(|error| format!("设置 {} 视口失败:{error}", viewport.file_stem()))?;
page.execute(SetTouchEmulationEnabledParams::new(mobile))
.await
.map_err(|error| format!("设置触摸模拟失败:{error}"))?;
page.execute(SetBypassServiceWorkerParams::new(true))
.await
.map_err(|error| format!("绕过 Service Worker 失败:{error}"))?;
page.evaluate_on_new_document(RESTRICTION_SCRIPT)
.await
.map_err(|error| format!("安装浏览器限制脚本失败:{error}"))?;
page.evaluate_on_new_document(RENDERED_IMAGE_CAPTURE_SCRIPT)
.await
.map_err(|error| {
format!(
"安装 {} 平台素材渲染观察失败:{error}",
viewport.file_stem()
)
})?;
let tasks = start_capture_tasks(&page, preview_url).await?;
let mut navigation_url = preview_url.clone();
if input.playtest_scenario == Some(super::model::BrowserPlaytestScenario::RunnerV1) {
navigation_url = super::playtest::runner::seeded_preview_url(preview_url);
}
let navigation = tokio::time::timeout(BROWSER_TIMEOUT, page.goto(navigation_url.as_str()))
.await
.map_err(|_| format!("{} 页面导航超时", viewport.file_stem()))?;
if let Err(error) = navigation {
let _ = tasks.stop().await;
let _ = page.close().await;
return Err(format!("{} 页面导航失败:{error}", viewport.file_stem()));
}
tokio::time::sleep(Duration::from_millis(input.settle_ms)).await;
let advisory_interaction_diagnostic = if advisory_interaction {
Some(run_advisory_interaction_probe(&page).await)
} else {
None
};
let playtest = match input.playtest_scenario {
Some(scenario) => Some(run_viewport_playtest(&page, scenario, viewport).await),
None => None,
};
let snapshot_script = build_snapshot_script(&input.expected_text)?;
let snapshot: PageSnapshot = page
.evaluate(snapshot_script)
.await
.map_err(|error| format!("采集 {} 页面状态失败:{error}", viewport.file_stem()))?
.into_value()
.map_err(|error| format!("解析 {} 页面状态失败:{error}", viewport.file_stem()))?;
let screenshot = page
.screenshot(
ScreenshotParams::builder()
.format(CaptureScreenshotFormat::Png)
.full_page(false)
.capture_beyond_viewport(false)
.build(),
)
.await
.map_err(|error| format!("采集 {} PNG 失败:{error}", viewport.file_stem()))?;
if !screenshot.starts_with(b"\x89PNG\r\n\x1a\n") {
return Err(format!("{} 截图不是有效 PNG", viewport.file_stem()));
}
let screenshot_path = input
.evidence_root
.join(format!("{}.png", viewport.file_stem()));
write_atomic(&screenshot_path, &screenshot)?;
tokio::task::yield_now().await;
let capture = tasks.stop().await?;
let _ = page.close().await;
if !capture.infrastructure_errors.is_empty() {
return Err(format!(
"浏览器安全拦截失败:{}",
capture.infrastructure_errors.join(";")
));
}
let expected_text = input
.expected_text
.iter()
.enumerate()
.map(|(index, text)| BrowserExpectedTextMatch {
text: text.clone(),
found: snapshot
.expected_text_matches
.get(index)
.copied()
.unwrap_or(false),
})
.collect::<Vec<_>>();
let mut diagnostics = Vec::new();
if snapshot.ready_state != "complete" {
diagnostics.push(format!("document.readyState={}", snapshot.ready_state));
}
let missing_text = expected_text
.iter()
.filter(|item| !item.found)
.map(|item| item.text.as_str())
.collect::<Vec<_>>();
if !missing_text.is_empty() {
diagnostics.push(format!("缺少可见文本:{}", missing_text.join("、")));
}
if input.fail_on_console_error && !capture.console_errors.is_empty() {
diagnostics.push(format!("console error {} 条", capture.console_errors.len()));
}
if !capture.exceptions.is_empty() {
diagnostics.push(format!("未捕获异常 {} 条", capture.exceptions.len()));
}
let fatal_request_count = capture
.failed_requests
.iter()
.filter(|request| request.fatal)
.count();
if fatal_request_count > 0 {
diagnostics.push(format!("失败请求 {} 条", fatal_request_count));
}
if !same_preview_origin(&snapshot.final_url, preview_url) {
diagnostics.push("页面最终 URL 已离开当前预览 origin".to_string());
}
let canvases = snapshot
.canvases
.into_iter()
.map(BrowserCanvasSnapshot::into_evidence)
.collect::<Vec<_>>();
if let Some(diagnostic) = canvas_validation_diagnostic(&canvases) {
diagnostics.push(diagnostic.to_string());
}
if !snapshot.canvas_or_webgl_image_paths.is_empty() {
diagnostics.push(format!(
"[advisory] rendered-local-images:{}",
snapshot.canvas_or_webgl_image_paths.join("|")
));
}
if let Some(diagnostic) = advisory_interaction_diagnostic {
diagnostics.push(format!("[advisory] {diagnostic}"));
}
Ok(BrowserViewportValidationOutcome {
result: BrowserViewportValidationResult {
viewport,
width,
height,
final_url: sanitize_url(&snapshot.final_url),
title: truncate_chars(&snapshot.title, 512),
ready_state: snapshot.ready_state,
visible_text_summary: snapshot.visible_text_summary,
visible_text_character_count: snapshot.visible_text_character_count,
dom_character_count: snapshot.dom_character_count,
expected_text,
console_errors: capture.console_errors,
console_warnings: capture.console_warnings,
exceptions: capture.exceptions,
failed_requests: capture.failed_requests,
canvases,
blocked_popup_count: snapshot.blocked_popup_count,
blocked_dialog_count: capture.blocked_dialog_count,
blocked_download_count: snapshot.blocked_download_count,
blocked_permission_count: snapshot.blocked_permission_count,
blocked_service_worker_count: snapshot.blocked_service_worker_count,
screenshot_path,
passed: diagnostics.is_empty()
|| diagnostics
.iter()
.all(|diagnostic| diagnostic.starts_with("[advisory]")),
diagnostics,
},
playtest,
})
}
async fn run_advisory_interaction_probe(page: &chromiumoxide::Page) -> String {
const PROBE: &str = r#"function() {
const candidates = Array.from(document.querySelectorAll(
'button:not([disabled]), a[href], input[type="button"]:not([disabled]), input[type="submit"]:not([disabled])'
));
const item = candidates.find((element) => {
const style = getComputedStyle(element);
const rect = element.getBoundingClientRect();
return style.display !== 'none' && style.visibility !== 'hidden' &&
rect.width > 0 && rect.height > 0 &&
element.getAttribute('aria-disabled') !== 'true';
});
if (!item) return JSON.stringify({ ok: false, reason: 'no-visible-enabled-control' });
item.scrollIntoView({ block: 'center', inline: 'center' });
return JSON.stringify({ ok: true, tag: item.tagName.toLowerCase() });
}"#;
let mut elements = match page.find_elements(
"button:not([disabled]), a[href], input[type=button]:not([disabled]), input[type=submit]:not([disabled])",
)
.await
{
Ok(elements) => elements,
Err(_) => return "真实交互探针无法查询可见控件".to_string(),
};
for element in elements.drain(..).take(8) {
let evaluated = match element.call_js_fn(PROBE, false).await {
Ok(value) => value,
Err(_) => continue,
};
let Some(value) = evaluated.result.value else {
continue;
};
let Some(text) = value.as_str() else {
continue;
};
let Ok(probe) = serde_json::from_str::<serde_json::Value>(text) else {
continue;
};
if probe.get("ok").and_then(serde_json::Value::as_bool) != Some(true) {
continue;
}
return match element.click().await {
Ok(_) => "真实交互探针已点击一个可见启用控件".to_string(),
Err(_) => "真实交互探针发现控件但点击未完成".to_string(),
};
}
"真实交互探针未发现可见启用控件".to_string()
}