修复试玩回包遗漏双端玩法失败证据(#548)
补全逐视口玩法诊断、断言和状态投影,并区分未执行与失败 补充关键回归测试,核对最终工具回包、截图及真实浏览器持久报告 同步试玩证据合同、提示词和技能包指纹
This commit is contained in:
@@ -17,8 +17,14 @@
|
||||
"browser.status": "整体 passed={}",
|
||||
"browser.notStarted": "本次未启动 Chromium",
|
||||
"browser.evidence": "[AGC 浏览器事实证据]\nattempt={attempt}; completionError={completion_status}; browser={browser_status}; codeFingerprintChanged={output_changed}\n{}\n诊断与平台素材运行时观察:{}\n交互探针:{}\n客户端已保存结构化证据;它是事实输入,不代表 Codex 已阅读截图或已经完成修复。当前 cwd 是项目根目录;按需读取实际文件,并自行决定是否修改、再次试玩或直接回复。AGC 只负责启动浏览器、采集证据和执行项目边界,Codex 负责解释结果。",
|
||||
"browser.noFailureDetails": "无额外硬失败详情",
|
||||
"browser.noVisibleControls": "未找到可执行的可见控件",
|
||||
"browser.noFailureDetails": "无额外视觉检查失败详情",
|
||||
"browser.noVisibleControls": "无建议性交互探针记录",
|
||||
"browser.gameplayNotRun": "玩法检查:未执行(visual 仅提供视觉证据)。",
|
||||
"browser.gameplayMissing": "固定场景玩法结果缺失,不能确认通过。",
|
||||
"browser.gameplayNoDiagnostic": "无诊断记录",
|
||||
"browser.gameplayNoUnmetAssertion": "无",
|
||||
"browser.gameplayViewport": "{viewport}: gameplayPassed={passed};首条玩法诊断:{diagnostic};首个未通过断言:{assertion}",
|
||||
"browser.gameplayEvidence": "[固定场景玩法证据]\n{}\n断言 passed=false 表示未通过,也可能因前序阻断而未执行;修复以具体诊断为准,不把所有未通过项视为独立故障。固定场景通过不代表完整关卡已验证;reportPath 仅为宿主证据引用,无需读取私有报告。",
|
||||
"system.role": "你是陶泥儿,是 Genarrative 面向用户的游戏创作助手,负责当前任务的执行。先理解用户意图:普通对话直接回答,项目请求按需要检查、修改、运行和验证,并用简洁中文报告真实结果。",
|
||||
"system.workspaceBoundary": "工作区边界:只在当前项目目录内工作;不要读取或输出凭据、Token、Cookie、auth.json、.env 或宿主私密路径。遇到阻断必须说明具体原因、文件和下一步,不要声称未验证的成功。",
|
||||
"system.toolAuthorization": "AGC 工具授权:agc_tools 使用当前登录会话。工具返回 401/403 时,报告登录或权限状态异常并停止,交由用户处理登录和权限。",
|
||||
|
||||
+2
@@ -10,6 +10,8 @@ Canvas/WebGL probes can show whether registered image sources entered a core ren
|
||||
|
||||
`mode=visual` captures desktop/mobile screenshots, loading, layout and diagnostics without clicking gameplay controls. It does not prove that the game loop works. `mode=gameplay` uses the fixed approved scenario (`generic-v1` by default, or the existing `tetris-v1` / `lane-defense-v1` contract), not an agent-written autopilot. It exercises the scenario independently in desktop and mobile viewports and requires both results. The receipt declares `gameplayViewports` separately from `visualViewports`. Older reports with only a desktop `playtest` result do not prove mobile gameplay. These bounded scenarios do not establish completion of an arbitrary long level.
|
||||
|
||||
Use each viewport's `gameplayResults` to diagnose failures: start with its first `diagnostics` entry, then inspect `assertions` and the recorded initial/final phase, sequence and level. An assertion with `passed=false` may not have run because an earlier step blocked; do not treat every unmet assertion as an independent defect. Missing viewport evidence is not a pass. In visual mode, `gameplayResults` is empty because gameplay was not run. `reportPath` is a private host evidence reference; use the returned diagnostics instead of trying to read that report.
|
||||
|
||||
For `generic-v1`, expose exactly one `script#playable-web-game-state[type="application/json"]` whose JSON is a projection of actual game state:
|
||||
|
||||
```json
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"schemaVersion": "agc-skill-pack.v1",
|
||||
"version": "2026-08-26.35",
|
||||
"version": "2026-08-26.37",
|
||||
"skills": [
|
||||
{
|
||||
"name": "agc-unity-editor",
|
||||
@@ -140,7 +140,7 @@
|
||||
"references/browser-evidence-contract.md",
|
||||
"references/runner-physics.mjs"
|
||||
],
|
||||
"sha256": "e0e7bdca529e068ce2974a5e312a267108ab5acb20e8f9d7c0c22d049d90b796"
|
||||
"sha256": "6800059c4e7bae70b0b42ec47175fc85e38d789e1759dca5df2067f715dc0657"
|
||||
},
|
||||
{
|
||||
"name": "agc-client-projection",
|
||||
|
||||
@@ -30,6 +30,8 @@ mod direct_runtime;
|
||||
mod direct_tool_bridge;
|
||||
mod direct_tools_mcp;
|
||||
mod direct_validation;
|
||||
#[cfg(test)]
|
||||
pub(crate) use direct_validation::browser_tool_result;
|
||||
mod generation;
|
||||
pub(crate) mod json_sidecar;
|
||||
pub(crate) mod redaction;
|
||||
|
||||
@@ -4421,6 +4421,70 @@ pub(super) fn drain_analytics_test_writer(
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn browser_gameplay_failure_reaches_model_with_both_viewports_and_screenshots() {
|
||||
let temp = tempfile::tempdir().unwrap();
|
||||
let evidence =
|
||||
super::super::direct_validation::tests::failed_browser_evidence_fixture(temp.path());
|
||||
for viewport in &evidence.viewport_results {
|
||||
image::RgbaImage::new(2, 2)
|
||||
.save(&viewport.screenshot_path)
|
||||
.unwrap();
|
||||
}
|
||||
let receipt = super::super::direct_validation::browser_tool_result(
|
||||
temp.path(),
|
||||
&evidence,
|
||||
Some(crate::browser::BrowserPlaytestScenario::GenericV1),
|
||||
1,
|
||||
);
|
||||
assert!(evidence.viewport_results.iter().all(|result| result.passed));
|
||||
assert_eq!(receipt["passed"], false);
|
||||
let failure =
|
||||
bridge_validation_result(temp.path(), Ok(receipt.clone()), |report, screenshots| {
|
||||
BrowserPlaytestError::PlaytestNotPassed {
|
||||
report,
|
||||
screenshots,
|
||||
}
|
||||
})
|
||||
.unwrap_err();
|
||||
let error = ToolCallError::from(&failure);
|
||||
assert_eq!(error.images.len(), 2);
|
||||
assert_eq!(error.error["PlaytestNotPassed"]["report"], receipt);
|
||||
assert!(error.error["PlaytestNotPassed"]
|
||||
.get("screenshots")
|
||||
.is_none());
|
||||
let state = direct_tool_bridge_state(temp.path().to_path_buf());
|
||||
let response =
|
||||
compose_direct_tool_outcome(&state, "agc_browser_playtest", "{}", false, Err(error));
|
||||
assert_eq!(response["isError"], true);
|
||||
let text = response["content"][0]["text"].as_str().unwrap();
|
||||
assert_eq!(
|
||||
response["content"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter(|block| block["type"] == "image")
|
||||
.count(),
|
||||
2
|
||||
);
|
||||
for (index, entry) in evidence.viewport_playtests.iter().enumerate() {
|
||||
let report = &receipt["gameplayResults"][index];
|
||||
assert_eq!(
|
||||
report["assertions"],
|
||||
serde_json::to_value(&entry.result.assertions).unwrap()
|
||||
);
|
||||
assert_eq!(report["finalPhase"], "ready");
|
||||
assert_eq!(report["finalSequence"], index as u64);
|
||||
assert!(text.contains(&entry.result.diagnostics[0]));
|
||||
}
|
||||
let summary = receipt["summary"].as_str().unwrap();
|
||||
assert!(summary.contains("desktop: gameplayPassed=false"));
|
||||
assert!(summary.contains("mobile: gameplayPassed=false"));
|
||||
assert!(summary.contains("start-control-clicked"));
|
||||
assert!(summary.contains("start-phase-playing"));
|
||||
assert!(summary.contains("未执行"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn host_cancellation_releases_queued_image_without_a_new_post() {
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
|
||||
@@ -395,6 +395,137 @@ async fn finish_async(
|
||||
.map_err(|_| "验证回执任务退出".to_string())?
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
struct GameplayViewportEvidence {
|
||||
viewport: crate::browser::BrowserValidationViewport,
|
||||
passed: bool,
|
||||
diagnostics: Vec<String>,
|
||||
assertions: Vec<crate::browser::BrowserPlaytestAssertion>,
|
||||
initial_phase: Option<crate::browser::BrowserPlaytestPhase>,
|
||||
initial_sequence: Option<u64>,
|
||||
initial_level: Option<u64>,
|
||||
final_phase: Option<crate::browser::BrowserPlaytestPhase>,
|
||||
final_sequence: Option<u64>,
|
||||
final_level: Option<u64>,
|
||||
}
|
||||
|
||||
/// 只投影模型修复所需的玩法事实;完整报告继续留在宿主私有目录。
|
||||
pub(crate) fn browser_tool_result(
|
||||
root: &Path,
|
||||
evidence: &crate::browser::BrowserValidationResult,
|
||||
scenario: Option<crate::browser::BrowserPlaytestScenario>,
|
||||
sequence: usize,
|
||||
) -> Value {
|
||||
use crate::browser::BrowserValidationViewport::{Desktop, Mobile};
|
||||
|
||||
let gameplay_results = if scenario.is_some() {
|
||||
[Desktop, Mobile]
|
||||
.into_iter()
|
||||
.map(|viewport| {
|
||||
let result = evidence
|
||||
.viewport_playtests
|
||||
.iter()
|
||||
.find(|entry| entry.viewport == viewport)
|
||||
.map(|entry| &entry.result);
|
||||
GameplayViewportEvidence {
|
||||
viewport,
|
||||
passed: result.is_some_and(|result| result.passed),
|
||||
diagnostics: result
|
||||
.map(|result| {
|
||||
result
|
||||
.diagnostics
|
||||
.iter()
|
||||
.take(4)
|
||||
.map(|message| truncate_agent_runtime_text(message, 511))
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_else(|| {
|
||||
vec![prompt_text!("direct.browser.gameplayMissing").into()]
|
||||
}),
|
||||
assertions: result
|
||||
.map(|result| result.assertions.clone())
|
||||
.unwrap_or_default(),
|
||||
initial_phase: result.and_then(|result| result.initial_phase),
|
||||
initial_sequence: result.and_then(|result| result.initial_sequence),
|
||||
initial_level: result.and_then(|result| result.initial_level),
|
||||
final_phase: result.and_then(|result| result.final_phase),
|
||||
final_sequence: result.and_then(|result| result.final_sequence),
|
||||
final_level: result.and_then(|result| result.final_level),
|
||||
}
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
} else {
|
||||
Vec::new()
|
||||
};
|
||||
let gameplay_summary = if scenario.is_none() {
|
||||
prompt_text!("direct.browser.gameplayNotRun").to_string()
|
||||
} else {
|
||||
let lines = gameplay_results
|
||||
.iter()
|
||||
.map(|result| {
|
||||
format!(
|
||||
prompt_text!("direct.browser.gameplayViewport"),
|
||||
viewport = match result.viewport {
|
||||
Desktop => "desktop",
|
||||
Mobile => "mobile",
|
||||
},
|
||||
passed = result.passed,
|
||||
diagnostic = result
|
||||
.diagnostics
|
||||
.first()
|
||||
.map(String::as_str)
|
||||
.unwrap_or(prompt_text!("direct.browser.gameplayNoDiagnostic")),
|
||||
assertion = result
|
||||
.assertions
|
||||
.iter()
|
||||
.find(|assertion| !assertion.passed)
|
||||
.map(|assertion| assertion.name.as_str())
|
||||
.unwrap_or(prompt_text!("direct.browser.gameplayNoUnmetAssertion")),
|
||||
)
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
format!(
|
||||
prompt_text!("direct.browser.gameplayEvidence"),
|
||||
lines.join("\n")
|
||||
)
|
||||
};
|
||||
let summary = format!(
|
||||
"{}\n{}",
|
||||
render_direct_browser_tool_evidence(root, evidence, sequence),
|
||||
gameplay_summary
|
||||
);
|
||||
let screenshots: Vec<_> = evidence
|
||||
.viewport_results
|
||||
.iter()
|
||||
.filter_map(|viewport| {
|
||||
viewport
|
||||
.screenshot_path
|
||||
.strip_prefix(root)
|
||||
.ok()
|
||||
.map(|path| path.to_string_lossy().replace('\\', "/"))
|
||||
})
|
||||
.collect();
|
||||
let passed = evidence.passed
|
||||
&& scenario.is_none_or(|scenario| {
|
||||
crate::browser::required_viewport_playtests_passed(
|
||||
scenario,
|
||||
&evidence.viewport_playtests,
|
||||
)
|
||||
});
|
||||
json!({
|
||||
"kind":"browser", "mode":if scenario.is_some() {"gameplay"} else {"visual"},
|
||||
"scenario":scenario, "passed":passed,
|
||||
"scenarioFingerprint":scenario.map(crate::browser::browser_playtest_scenario_fingerprint),
|
||||
"summary":summary, "screenshots":screenshots,
|
||||
"reportPath":evidence.evidence.report_path.strip_prefix(root).ok().map(|path|path.to_string_lossy().replace('\\', "/")),
|
||||
"visualViewports":["desktop","mobile"],
|
||||
"gameplayViewports":evidence.viewport_playtests.iter().map(|entry|entry.viewport).collect::<Vec<_>>(),
|
||||
"gameplayResults":gameplay_results,
|
||||
"coverage":if scenario.is_some() {"dual-viewport-fixed-scenario"} else {"visual-only"}
|
||||
})
|
||||
}
|
||||
|
||||
pub(super) async fn run_browser(root: &Path, arguments: &Value) -> Result<Value, String> {
|
||||
run_browser_with_budget(root, arguments, true).await
|
||||
}
|
||||
@@ -466,33 +597,9 @@ async fn run_browser_with_budget(
|
||||
.await;
|
||||
let (result, passed) = match evidence {
|
||||
Ok(evidence) => {
|
||||
let screenshots: Vec<_> = evidence
|
||||
.viewport_results
|
||||
.iter()
|
||||
.filter_map(|v| {
|
||||
v.screenshot_path
|
||||
.strip_prefix(root)
|
||||
.ok()
|
||||
.map(|p| p.to_string_lossy().replace('\\', "/"))
|
||||
})
|
||||
.collect();
|
||||
let gameplay_passed = scenario.is_none_or(|scenario| {
|
||||
crate::browser::required_viewport_playtests_passed(
|
||||
scenario,
|
||||
&evidence.viewport_playtests,
|
||||
)
|
||||
});
|
||||
let passed = evidence.passed && gameplay_passed;
|
||||
let summary = render_direct_browser_tool_evidence(root, &evidence, sequence);
|
||||
(
|
||||
json!({"kind":"browser", "mode":input.mode, "scenario":scenario,
|
||||
"scenarioFingerprint":scenario.map(crate::browser::browser_playtest_scenario_fingerprint), "summary":summary, "screenshots":screenshots,
|
||||
"reportPath":evidence.evidence.report_path.strip_prefix(root).ok().map(|p|p.to_string_lossy().replace('\\', "/")),
|
||||
"visualViewports":["desktop","mobile"],
|
||||
"gameplayViewports":evidence.viewport_playtests.iter().map(|v|v.viewport).collect::<Vec<_>>(),
|
||||
"coverage":if scenario.is_some() {"dual-viewport-fixed-scenario"} else {"visual-only"}}),
|
||||
passed,
|
||||
)
|
||||
let result = browser_tool_result(root, &evidence, scenario, sequence);
|
||||
let passed = result["passed"] == true;
|
||||
(result, passed)
|
||||
}
|
||||
Err(error) => (
|
||||
json!({"mode":input.mode,"needsReconciliation":error.starts_with("browser-cleanup-unconfirmed:"),"error":truncate_agent_runtime_text(&error, 1800)}),
|
||||
@@ -599,9 +706,145 @@ pub(super) async fn run_command(root: &Path, arguments: &Value) -> Result<Value,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
pub(super) mod tests {
|
||||
use super::*;
|
||||
|
||||
pub(crate) fn browser_evidence_fixture(root: &Path) -> crate::browser::BrowserValidationResult {
|
||||
let mut viewports = Vec::new();
|
||||
let mut playtests = Vec::new();
|
||||
for viewport in ["desktop", "mobile"] {
|
||||
viewports.push(json!({
|
||||
"viewport":viewport, "width":1280, "height":720,
|
||||
"finalUrl":"http://127.0.0.1/", "title":"fixture", "readyState":"complete",
|
||||
"visibleTextSummary":"", "visibleTextCharacterCount":0, "domCharacterCount":0,
|
||||
"expectedText":[], "consoleErrors":[], "consoleWarnings":[], "exceptions":[],
|
||||
"failedRequests":[], "canvases":[], "diagnostics":[], "passed":true,
|
||||
"blockedPopupCount":0, "blockedDialogCount":0, "blockedDownloadCount":0,
|
||||
"blockedPermissionCount":0, "blockedServiceWorkerCount":0,
|
||||
"screenshotPath":root.join(format!("{viewport}.png"))
|
||||
}));
|
||||
playtests.push(json!({"viewport":viewport, "result":{
|
||||
"scenario":"generic-v1", "scenarioFingerprint":"fixture", "passed":true,
|
||||
"initialSequence":0, "initialPhase":"ready", "initialLevel":1,
|
||||
"finalSequence":3, "finalPhase":"ready", "finalLevel":1,
|
||||
"assertions":[
|
||||
{"name":"start-control-clicked", "passed":true},
|
||||
{"name":"start-phase-playing", "passed":true},
|
||||
{"name":"restart-control-clicked", "passed":true}
|
||||
], "diagnostics":[]
|
||||
}}));
|
||||
}
|
||||
serde_json::from_value(json!({
|
||||
"schemaVersion":"browser-validation.v1", "url":"http://127.0.0.1/",
|
||||
"browser":{"kind":"chrome", "product":"fixture", "protocolVersion":"1.3"},
|
||||
"passed":true, "viewportResults":viewports, "viewportPlaytests":playtests,
|
||||
"diagnostics":[], "completedAtUnixMs":1,
|
||||
"evidence":{"root":root, "reportPath":root.join("validation.json")}
|
||||
}))
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
pub(crate) fn failed_browser_evidence_fixture(
|
||||
root: &Path,
|
||||
) -> crate::browser::BrowserValidationResult {
|
||||
let mut evidence = browser_evidence_fixture(root);
|
||||
evidence.passed = false;
|
||||
for (index, entry) in evidence.viewport_playtests.iter_mut().enumerate() {
|
||||
entry.result.passed = false;
|
||||
entry.result.final_phase = Some(crate::browser::BrowserPlaytestPhase::Ready);
|
||||
entry.result.final_sequence = Some(index as u64);
|
||||
entry.result.diagnostics = vec![if index == 0 {
|
||||
"固定试玩控件 start 处于 disabled 状态".into()
|
||||
} else {
|
||||
"generic-v1 start 后必须进入 playing".into()
|
||||
}];
|
||||
for (assertion_index, assertion) in entry.result.assertions.iter_mut().enumerate() {
|
||||
assertion.passed = index == 1 && assertion_index == 0;
|
||||
}
|
||||
}
|
||||
evidence
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn browser_tool_projection_distinguishes_gameplay_success_visual_only_and_missing_viewport() {
|
||||
use crate::browser::BrowserPlaytestScenario::GenericV1;
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
for mode in ["gameplay", "visual", "missing-mobile"] {
|
||||
let mut evidence = browser_evidence_fixture(root.path());
|
||||
let scenario = if mode == "visual" {
|
||||
evidence.viewport_playtests.clear();
|
||||
None
|
||||
} else {
|
||||
Some(GenericV1)
|
||||
};
|
||||
if mode == "missing-mobile" {
|
||||
evidence.viewport_playtests.pop();
|
||||
evidence.passed = false;
|
||||
}
|
||||
let receipt = browser_tool_result(root.path(), &evidence, scenario, 1);
|
||||
let summary = receipt["summary"].as_str().unwrap();
|
||||
assert_eq!(receipt["passed"], mode != "missing-mobile");
|
||||
if mode == "visual" {
|
||||
assert_eq!(receipt["gameplayResults"], json!([]));
|
||||
assert!(summary.contains("未执行"));
|
||||
assert_eq!(receipt["coverage"], "visual-only");
|
||||
continue;
|
||||
}
|
||||
for (index, viewport) in ["desktop", "mobile"].iter().enumerate() {
|
||||
let result = &receipt["gameplayResults"][index];
|
||||
assert_eq!(result["viewport"], *viewport);
|
||||
if mode == "missing-mobile" && index == 1 {
|
||||
assert_eq!(result["passed"], false);
|
||||
assert!(result["diagnostics"][0].as_str().unwrap().contains("缺失"));
|
||||
assert!(summary.contains("mobile: gameplayPassed=false"));
|
||||
assert_eq!(result["assertions"], json!([]));
|
||||
assert_eq!(result["finalPhase"], Value::Null);
|
||||
} else {
|
||||
assert_eq!(result["passed"], true);
|
||||
assert_eq!(result["diagnostics"], json!([]));
|
||||
assert!(result["assertions"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.all(|a| a["passed"] == true));
|
||||
assert!(summary.contains(&format!("{viewport}: gameplayPassed=true")));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn browser_tool_projection_bounds_diagnostics_without_losing_either_blocker() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let mut evidence = failed_browser_evidence_fixture(root.path());
|
||||
for viewport in &mut evidence.viewport_playtests {
|
||||
viewport
|
||||
.result
|
||||
.diagnostics
|
||||
.extend((0..8).map(|_| "额外诊断".repeat(200)));
|
||||
}
|
||||
let receipt = browser_tool_result(
|
||||
root.path(),
|
||||
&evidence,
|
||||
Some(crate::browser::BrowserPlaytestScenario::GenericV1),
|
||||
1,
|
||||
);
|
||||
for (index, entry) in evidence.viewport_playtests.iter().enumerate() {
|
||||
let diagnostics = receipt["gameplayResults"][index]["diagnostics"]
|
||||
.as_array()
|
||||
.unwrap();
|
||||
assert_eq!(diagnostics.len(), 4);
|
||||
assert_eq!(diagnostics[0], entry.result.diagnostics[0]);
|
||||
assert!(diagnostics
|
||||
.iter()
|
||||
.all(|d| d.as_str().unwrap().chars().count() <= 512));
|
||||
assert!(receipt["summary"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.contains(&entry.result.diagnostics[0]));
|
||||
}
|
||||
}
|
||||
|
||||
fn project() -> (tempfile::TempDir, std::path::PathBuf) {
|
||||
let temp = tempfile::tempdir().unwrap();
|
||||
let root = temp.path().join("project");
|
||||
|
||||
@@ -1641,6 +1641,38 @@ async fn run_stable_generic_viewport_fixture(
|
||||
let persisted: serde_json::Value =
|
||||
serde_json::from_slice(&fs::read(&result.evidence.report_path).unwrap()).unwrap();
|
||||
assert_eq!(persisted["viewportPlaytests"].as_array().unwrap().len(), 2);
|
||||
let receipt = crate::agent::browser_tool_result(
|
||||
evidence.path(),
|
||||
&result,
|
||||
Some(BrowserPlaytestScenario::GenericV1),
|
||||
1,
|
||||
);
|
||||
assert_eq!(receipt["passed"], persisted["passed"]);
|
||||
for (index, entry) in persisted["viewportPlaytests"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.enumerate()
|
||||
{
|
||||
let projected = &receipt["gameplayResults"][index];
|
||||
assert_eq!(projected["viewport"], entry["viewport"]);
|
||||
for field in [
|
||||
"passed",
|
||||
"assertions",
|
||||
"initialPhase",
|
||||
"initialSequence",
|
||||
"initialLevel",
|
||||
"finalPhase",
|
||||
"finalSequence",
|
||||
"finalLevel",
|
||||
] {
|
||||
assert_eq!(projected[field], entry["result"][field], "{field}");
|
||||
}
|
||||
if let Some(reason) = entry["result"]["diagnostics"][0].as_str() {
|
||||
assert_eq!(projected["diagnostics"][0], reason);
|
||||
assert!(receipt["summary"].as_str().unwrap().contains(reason));
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
|
||||
@@ -195,6 +195,7 @@ UI 编辑器的“分析参考图”步骤、Rust 命令 `suggest_ui_design_sema
|
||||
### 分层验证与预算
|
||||
|
||||
- 复用客户端浏览器与现有固定玩法场景。视觉检查采集双端画面/布局/资源/诊断;玩法检查分别在 desktop/mobile 执行明确的固定场景和真实输入,按视口保存结果。旧报告缺少移动端玩法结果时保持未知,不补通过。报告必须声明检查层级;视觉通过不能宣称玩法通过,固定场景通过也不能宣称覆盖未执行的完整关卡。
|
||||
- 浏览器工具回执在 `mode=gameplay` 时新增 `gameplayResults` 双端有界投影,每个视口包含 `viewport`、`passed`、`diagnostics`、`assertions` 及 `initialPhase` / `initialSequence` / `initialLevel`、`finalPhase` / `finalSequence` / `finalLevel`。`assertions` 仅列当前固定场景的断言名称和通过状态;每端最多公开 4 条诊断,每条最多 512 字符。缺少某端结果时仍返回该视口 `passed=false`,诊断说明证据缺失,断言为空且六个状态字段为 `null`;断言的 `passed=false` 可包含未执行,不能据此断言该断言已独立执行且失败。`mode=visual` 的 `gameplayResults` 为空,并在摘要中明确玩法未执行。摘要从同一投影给出每端首条诊断和首个未通过断言,不能与结构化结果相矛盾。工具回执不得展开完整宿主报告;`reportPath` 只作宿主证据引用。宿主报告结构、验收门禁、持久化预算和私有路径保护保持原合同,旧回执不回填新字段。
|
||||
- 输入或碰撞改变先做定点玩法检查;纯图像/颜色变化做视觉检查;首次交付和影响闭环的修改做所需玩法验证。新增失败或相关代码变化才重跑对应层,不因改说明文字重复完整验证。
|
||||
- 内置试玩和客户端托管的外部 Node/npm 验证共用当前 clientTurnId 的持久化预算。客户端分配递增执行序号,模型提供的旧 attempt 仅作兼容输入,不能减少计数或重置预算;同一轮错误反馈、工具切换和进程重启均不能刷新已消费次数。
|
||||
- 新增本地 validation.maxRuns(默认 3,正整数)独立于 llm.maxRetries;显式配置原样使用,不按角色或运行模式改写。超限直接返回已用/上限和最近证据,停止新的验证。预检与正常构建不计作重复试玩。
|
||||
|
||||
Reference in New Issue
Block a user