支持素材生成上传参考图

图标素材生成支持上传普通参考图并提交到后端参考图数组。

UI设计图生成和UI素材提取支持上传参考图并写入生成输入记录。

同步前后端契约、OpenAPI、文档和相关单测。
This commit is contained in:
2026-06-27 20:27:55 +08:00
parent f20f4e0f8e
commit 5976c2b337
31 changed files with 1232 additions and 235 deletions
@@ -2778,7 +2778,14 @@
"properties": {
"referenceImageSrc": {
"type": "string",
"description": "规范图/参考图 Data URL。"
"description": "规范图/参考图 Data URL 或 objectKey;上传参考图优先提交 objectKey。"
},
"referenceImageSrcs": {
"type": "array",
"items": {
"type": "string",
"description": "额外图标素材参考图 Data URL 或 objectKey;上传参考图优先提交 objectKey。"
}
},
"iconDescriptions": {
"type": "array",
@@ -2840,13 +2847,20 @@
"properties": {
"sourceImageSrc": {
"type": "string",
"description": "UI 设计图 Data URL。"
"description": "UI 设计图 Data URL 或 objectKey。"
},
"model": {
"type": "string",
"default": "gemini-3.1-flash-image-preview",
"description": "支持 gpt-image-2、gemini-3.1-flash-image-preview、nanobanana2、nano-banana。未传时默认使用 nanobanana。"
},
"referenceImageSrcs": {
"type": "array",
"items": {
"type": "string",
"description": "额外 UI 素材参考图 Data URL 或 objectKey;上传参考图优先提交 objectKey。"
}
},
"aspectRatio": {
"type": "string",
"const": "1:1",
@@ -21,6 +21,7 @@
- `从画布中选择` 只接受 `assetKind: "icon-spec"` 的图层;普通图片、其他类别图层和不携带标签的图片不绑定。
- `新建图标规范` 复用现有图标规范生成表单。
- `上传图片` 仅绑定到当前 UI 设计面板的图标规范参考,不自动添加为画布图层。
- 图标规范之外可继续上传普通参考图,用于约束 UI 设计图的布局风格、配色或材质;普通参考图不替代图标规范。
## 生成参数
@@ -44,12 +45,14 @@
```
生成请求固定使用 `gpt-image-2`。有参考图时走图片编辑请求;无参考图时走图片生成请求。
前端提交时会把图标规范作为第一张参考图,并把用户上传的普通参考图按顺序追加到同一个 `referenceImageSrcs` 数组;上传参考图优先提交 `objectKey`
## 提取素材
- 点击浮动工具栏 `提取素材` 后,进入 UI 设计图素材框选编辑状态,不立即提交提取接口。
- 框选状态参考微信截图工具栏布局,默认矩形框选,并支持矩形框选、椭圆框选和画笔自由框选三种工具。
- 用户可在同一张 UI 设计图上多次框选;未至少框选一个区域时,工具栏 `提取` 按钮不可点击。
- 可上传普通参考图辅助 UI 素材提取,用于约束被框选素材的风格、配色或材质;普通参考图会以 `objectKey` 随合成后的红框 UI 设计图一起提交。
- 点击 `提取` 后,前端把所有框选的红色轮廓绘入原 UI 设计图,生成合成图片 Data URL,再提交到 `POST /api/editor/ui-designs/assets/extractions`
- 后端固定使用 `gpt-image-2` 图片编辑链路,并固定提示词:
@@ -66,5 +69,6 @@
- 面板第一模块为图标规范,来源菜单包含三个动作。
- 从画布选择时只能绑定图标规范图片。
- 请求参数包含 `kind: "ui-design"``model: "gpt-image-2"`、比例、大小与可选参考图。
- 生成图层信息面板展示 `用户输入``图标规范`
- 上传普通参考图后,请求参考图数组同时包含图标规范和普通参考图,生成图层信息面板展示 `用户输入``图标规范` 与普通参考图
- 选中 UI 设计图时浮动工具栏显示 `提取素材`;点击后进入红框素材框选状态,至少框选一个区域后才能调用固定 `gpt-image-2` 提取接口,画布同时出现 spritesheet 图集和拆分后的独立素材。
- UI 素材提取面板上传普通参考图后,提取请求参考图数组同时包含红框 UI 设计图和普通参考图,生成图层信息面板展示 `UI设计图` 与普通参考图。
@@ -74,5 +74,6 @@
- 默认 6 个素材描述会进入 prompt;用户在单个文本框中继续输入时最多解析 100 个素材描述。
- 默认打开图标素材面板时选中 `nanobanana2 / 1:1 / 1K`;模型切换后,角色和图标素材面板之间沿用上次选择的模型。
- 图标素材生成请求必须带 `model``aspectRatio``imageSize``nanobanana2` 请求体必须包含 `generationConfig.imageConfig.aspectRatio/imageSize``gpt-image-2` 请求必须包含文档映射后的 `size`
- 图标素材生成可以上传普通参考图;提交时图标规范图仍走 `referenceImageSrc`,普通参考图走 `referenceImageSrcs`,上传参考图优先提交 `objectKey`,并写入 `generationInputs.references`
- 生成成功后画布只出现一个已扣绿的透明 spritesheet 图集,不出现额外独立图标图层。
- 生成图标素材提交体包含按模型和尺寸计算的 `priceMudPoints``nanobanana2 1K` 应为 `12``gpt-image-2 1K` 应为 `3``gpt-image-2 2K` 应为 `5`。若前端传入与后端计费配置不一致的值,后端返回 `priceMudPoints` 校验错误,不继续调用上游生成。
+205 -14
View File
@@ -18,7 +18,9 @@ use platform_image::{
slice_generated_icon_spritesheet_all_by_connected_components,
},
};
use platform_oss::{LegacyAssetPrefix, OssHeadObjectRequest, OssObjectAccess};
use platform_oss::{
LegacyAssetPrefix, OssHeadObjectRequest, OssObjectAccess, OssSignedGetObjectUrlRequest,
};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use shared_contracts::assets::{
@@ -241,6 +243,7 @@ pub struct EditorIconSpritesheetGenerationRequest {
pub struct EditorUiDesignAssetExtractionRequest {
pub(crate) source_image_src: String,
pub(crate) model: Option<String>,
pub(crate) reference_image_srcs: Option<Vec<String>>,
pub(crate) aspect_ratio: String,
pub(crate) image_size: String,
pub(crate) project_id: Option<String>,
@@ -1172,10 +1175,12 @@ pub(crate) async fn generate_editor_image_for_owner(
let reference_images = if reference_sources.is_empty() {
Vec::new()
} else {
reference_sources
.iter()
.map(|source| parse_editor_reference_image(source.as_str()))
.collect::<Result<Vec<_>, _>>()?
let mut parsed_reference_images = Vec::with_capacity(reference_sources.len());
for source in &reference_sources {
parsed_reference_images
.push(parse_editor_reference_image(state, source.as_str()).await?);
}
parsed_reference_images
};
let image_size = if is_ui_design_generation {
Cow::Owned(generation_options.size.clone())
@@ -1577,7 +1582,8 @@ pub(crate) async fn edit_editor_image_for_owner(
})),
);
}
let reference_image = parse_editor_reference_image(payload.source_image_src.as_str())?;
let reference_image =
parse_editor_reference_image(state, payload.source_image_src.as_str()).await?;
let expected_price_mud_points = resolve_editor_image_edit_price(state)?;
let settings = require_openai_image_settings(state)?.with_external_api_audit_context(
@@ -1735,12 +1741,13 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner(
payload: EditorIconSpritesheetGenerationRequest,
) -> Result<Json<Value>, AppError> {
let icon_descriptions = normalize_icon_descriptions(payload.icon_descriptions)?;
let reference_image = parse_editor_reference_image(payload.reference_image_src.as_str())
let reference_image = parse_editor_reference_image(state, payload.reference_image_src.as_str())
.await
.map_err(|error| {
error.with_details(json!({
"provider": "editor-icon-spritesheet",
"field": "referenceImageSrc",
"message": "图标素材规范必须是图片 Data URL。",
"message": "图标素材规范必须是图片 Data URL 或 objectKey",
}))
})?;
let generation_options = normalize_editor_generation_options(
@@ -1942,14 +1949,29 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
caller: EditorGenerationCaller,
payload: EditorUiDesignAssetExtractionRequest,
) -> Result<Json<Value>, AppError> {
let reference_image =
parse_editor_reference_image(payload.source_image_src.as_str()).map_err(|error| {
let reference_image = parse_editor_reference_image(state, payload.source_image_src.as_str())
.await
.map_err(|error| {
error.with_details(json!({
"provider": "editor-ui-design-asset-extraction",
"field": "sourceImageSrc",
"message": "UI设计图必须是图片 Data URL。",
"message": "UI设计图必须是图片 Data URL 或 objectKey",
}))
})?;
let mut reference_images = vec![reference_image];
for source in payload.reference_image_srcs.as_deref().unwrap_or(&[]) {
reference_images.push(
parse_editor_reference_image(state, source.as_str())
.await
.map_err(|error| {
error.with_details(json!({
"provider": "editor-ui-design-asset-extraction",
"field": "referenceImageSrcs",
"message": "UI素材参考图必须是图片 Data URL 或 objectKey。",
}))
})?,
);
}
let generation_options = normalize_editor_ui_design_asset_extraction_options(
payload.model.as_deref(),
payload.aspect_ratio.as_str(),
@@ -1985,7 +2007,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
None,
generation_options.aspect_ratio,
generation_options.provider_image_size,
&[reference_image],
reference_images.as_slice(),
"图片画布提取UI设计图素材 spritesheet",
)
.await
@@ -1998,7 +2020,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
None,
generation_options.size.as_str(),
1,
&[reference_image],
reference_images.as_slice(),
"图片画布提取UI设计图素材 spritesheet",
)
.await
@@ -3711,7 +3733,21 @@ pub(crate) async fn persist_editor_generated_image(
})
}
pub(crate) fn parse_editor_reference_image(source: &str) -> Result<OpenAiReferenceImage, AppError> {
const EDITOR_REFERENCE_IMAGE_READ_EXPIRE_SECONDS: u64 = 300;
const EDITOR_REFERENCE_IMAGE_MAX_SIZE_BYTES: u64 = 32 * 1024 * 1024;
pub(crate) async fn parse_editor_reference_image(
state: &AppState,
source: &str,
) -> Result<OpenAiReferenceImage, AppError> {
let trimmed = source.trim();
if trimmed.starts_with("data:") {
return parse_editor_reference_image_data_url(trimmed);
}
read_editor_reference_image_object(state, trimmed).await
}
fn parse_editor_reference_image_data_url(source: &str) -> Result<OpenAiReferenceImage, AppError> {
let Some((header, data)) = source.trim().split_once(',') else {
return Err(
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
@@ -3763,6 +3799,134 @@ pub(crate) fn parse_editor_reference_image(source: &str) -> Result<OpenAiReferen
})
}
async fn read_editor_reference_image_object(
state: &AppState,
source: &str,
) -> Result<OpenAiReferenceImage, AppError> {
if source.is_empty() {
return Err(
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
"provider": "editor-reference-image",
"field": "sourceImageSrc",
"message": "图片参考图不能为空。",
})),
);
}
let oss_client = state.oss_client().ok_or_else(|| {
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
"provider": "aliyun-oss",
"reason": "OSS 未完成环境变量配置",
}))
})?;
let signed = oss_client
.sign_get_object_url(OssSignedGetObjectUrlRequest {
object_key: source.to_string(),
expire_seconds: Some(EDITOR_REFERENCE_IMAGE_READ_EXPIRE_SECONDS),
})
.map_err(|error| map_oss_error(error, "aliyun-oss"))?;
let response = reqwest::Client::new()
.get(signed.signed_url.as_str())
.send()
.await
.map_err(|error| {
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": "aliyun-oss",
"message": format!("读取图片参考图失败:{error}"),
}))
})?;
let status = response.status();
if !status.is_success() {
return Err(
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": "aliyun-oss",
"message": format!("读取图片参考图返回非成功状态:{}", status.as_u16()),
"objectKey": signed.object_key,
})),
);
}
if response
.content_length()
.is_some_and(|size| size > EDITOR_REFERENCE_IMAGE_MAX_SIZE_BYTES)
{
return Err(editor_reference_image_too_large());
}
let content_type_mime = response
.headers()
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.and_then(normalize_editor_reference_image_mime_type)
.map(ToOwned::to_owned);
let bytes = response.bytes().await.map_err(|error| {
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": "aliyun-oss",
"message": format!("读取图片参考图内容失败:{error}"),
}))
})?;
if bytes.is_empty() {
return Err(
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
"provider": "editor-reference-image",
"field": "sourceImageSrc",
"message": "图片参考图为空。",
})),
);
}
if bytes.len() as u64 > EDITOR_REFERENCE_IMAGE_MAX_SIZE_BYTES {
return Err(editor_reference_image_too_large());
}
let mime_type = content_type_mime
.or_else(|| infer_editor_reference_image_mime_type(bytes.as_ref()).map(ToOwned::to_owned))
.ok_or_else(|| {
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
"provider": "editor-reference-image",
"field": "sourceImageSrc",
"message": "图片参考图 objectKey 指向的资源不是图片。",
"objectKey": signed.object_key,
}))
})?
.to_string();
let extension = editor_reference_image_extension(mime_type.as_str());
Ok(OpenAiReferenceImage {
bytes: bytes.to_vec(),
mime_type,
file_name: format!("editor-reference.{extension}"),
})
}
fn normalize_editor_reference_image_mime_type(content_type: &str) -> Option<&str> {
let mime_type = content_type.split(';').next()?.trim();
mime_type.starts_with("image/").then_some(mime_type)
}
fn infer_editor_reference_image_mime_type(bytes: &[u8]) -> Option<&'static str> {
if bytes.starts_with(b"\x89PNG\r\n\x1a\n") {
return Some("image/png");
}
if bytes.starts_with(&[0xff, 0xd8, 0xff]) {
return Some("image/jpeg");
}
if bytes.len() >= 12 && bytes.starts_with(b"RIFF") && &bytes[8..12] == b"WEBP" {
return Some("image/webp");
}
None
}
fn editor_reference_image_extension(mime_type: &str) -> &'static str {
match mime_type {
"image/jpeg" => "jpg",
"image/png" => "png",
"image/webp" => "webp",
_ => "png",
}
}
fn editor_reference_image_too_large() -> AppError {
AppError::from_status(StatusCode::PAYLOAD_TOO_LARGE).with_details(json!({
"provider": "editor-reference-image",
"message": "图片参考图不能超过 32MB。",
}))
}
pub(crate) fn map_editor_project_error(error: SpacetimeClientError) -> AppError {
match error {
SpacetimeClientError::Procedure(message) if message.contains("无权") => {
@@ -5011,6 +5175,33 @@ mod tests {
assert!(normalize_icon_descriptions(too_many).is_err());
}
#[test]
fn editor_reference_image_data_url_stays_compatible() {
let parsed = parse_editor_reference_image_data_url("data:image/png;base64,AAAA")
.expect("legacy data URL reference should still parse");
assert_eq!(parsed.mime_type, "image/png");
assert_eq!(parsed.file_name, "editor-reference.png");
assert_eq!(parsed.bytes, vec![0, 0, 0]);
}
#[test]
fn editor_reference_image_mime_type_falls_back_to_magic_bytes() {
assert_eq!(
infer_editor_reference_image_mime_type(b"\x89PNG\r\n\x1a\nrest"),
Some("image/png")
);
assert_eq!(
infer_editor_reference_image_mime_type(&[0xff, 0xd8, 0xff, 0xe0]),
Some("image/jpeg")
);
assert_eq!(
infer_editor_reference_image_mime_type(b"RIFFxxxxWEBPrest"),
Some("image/webp")
);
assert_eq!(infer_editor_reference_image_mime_type(b"not image"), None);
}
#[test]
fn editor_icon_spritesheet_price_is_resolved_from_generation_config() {
let state = AppState::new(AppConfig::default()).expect("state should build");
@@ -184,6 +184,8 @@ function createStageProps(): ImageCanvasStageViewProps {
onUiAssetExtractionPointerStart: vi.fn(),
onUiAssetExtractionPointerMove: vi.fn(),
onUiAssetExtractionPointerEnd: vi.fn(),
onRequestUiAssetExtractionReferenceUpload: vi.fn(),
onRemoveUiAssetExtractionReference: vi.fn(),
onSubmitUiAssetExtraction: vi.fn(),
onQuickEditSelectionToolChange: vi.fn(),
onQuickEditSelectionPointerStart: vi.fn(),
@@ -297,6 +299,7 @@ describe('ImageCanvasEditorShellView', () => {
tool: 'rect' as const,
model: 'gemini-3.1-flash-image-preview',
marks: [],
references: [],
draftMark: null,
status: 'idle' as const,
},
@@ -342,6 +342,7 @@ export type CharacterAnimationPanelState = {
export type UploadTarget =
| 'asset'
| 'generation-reference'
| 'ui-asset-extraction-reference'
| 'quick-edit-reference'
| 'video-reference-image'
| 'video-reference-video'
@@ -43,6 +43,7 @@ import type {
CanvasLayer,
CanvasTool,
CanvasViewport,
CharacterReferenceImage,
ImageContextMenuState,
QuickEditPanelState,
} from './ImageCanvasEditorTypes';
@@ -164,6 +165,9 @@ export function ImageCanvasEditorView() {
const setQuickEditPanelRef = useRef<
Dispatch<SetStateAction<QuickEditPanelState | null>>
>(() => {});
const appendUiAssetExtractionReferencesRef = useRef<
(references: CharacterReferenceImage[]) => void
>(() => {});
const deleteLayerByIdRef = useRef<(targetLayerId: string | null) => void>(
() => {},
);
@@ -312,7 +316,9 @@ export function ImageCanvasEditorView() {
if (!isMounted || currentRequestId !== requestId) {
return;
}
setWalletBalanceLabel(`${formatDashboardCount(dashboard.walletBalance)}`);
setWalletBalanceLabel(
`${formatDashboardCount(dashboard.walletBalance)}`,
);
})
.catch(() => {
if (!isMounted || currentRequestId !== requestId) {
@@ -790,6 +796,8 @@ export function ImageCanvasEditorView() {
setLayers,
setGenerateDialog,
setQuickEditPanel: (updater) => setQuickEditPanelRef.current(updater),
appendUiAssetExtractionReferences: (references) =>
appendUiAssetExtractionReferencesRef.current(references),
appendCanvasLayersWithResources,
selectSingleLayer,
});
@@ -889,12 +897,16 @@ export function ImageCanvasEditorView() {
endUiAssetExtractionPointer,
endQuickEditSelectionPointer,
cancelUiAssetExtraction,
appendUiAssetExtractionReferences,
removeUiAssetExtractionReference,
submitUiAssetExtraction,
generationComposerStyle,
generationComposerNode,
switchGenerationTool,
} = generationSurface;
setQuickEditPanelRef.current = setQuickEditPanel;
appendUiAssetExtractionReferencesRef.current =
appendUiAssetExtractionReferences;
const clearGenerationPanelsAfterBlur = useCallback(() => {
setQuickEditPanel(null);
setCropExpandPanel(null);
@@ -927,7 +939,7 @@ export function ImageCanvasEditorView() {
});
const contextMenuLayer =
contextMenu?.kind === 'layer'
? layers.find((layer) => layer.id === contextMenu.layerId) ?? null
? (layers.find((layer) => layer.id === contextMenu.layerId) ?? null)
: null;
const {
canvasClipboard,
@@ -1078,7 +1090,9 @@ export function ImageCanvasEditorView() {
}, [pendingGenerationDeleteDialog, removeCanvasGenerationDialog]);
const updateLayerAssetKind = useCallback(
(layerId: string, assetKind: CanvasAssetKind | null) => {
const targetLayer = layersRef.current.find((layer) => layer.id === layerId);
const targetLayer = layersRef.current.find(
(layer) => layer.id === layerId,
);
if (!targetLayer || targetLayer.assetKind === assetKind) {
return;
}
@@ -1490,8 +1504,7 @@ export function ImageCanvasEditorView() {
onUpdateLayerAssetKind: updateLayerAssetKind,
onGenerationFramePointerDown: handleGenerationFramePointerDown,
onActivateGenerationDialog: activateCanvasGenerationDialog,
onCropExpandHandlePointerDown:
generationSurface.startCropExpandFrameResize,
onCropExpandHandlePointerDown: generationSurface.startCropExpandFrameResize,
onOpenQuickEditPanel: openQuickEditPanel,
onOpenRedrawPanel: openRedrawPanel,
onOpenCropExpandPanel: openCropExpandPanel,
@@ -1502,6 +1515,9 @@ export function ImageCanvasEditorView() {
onUiAssetExtractionPointerStart: startUiAssetExtractionPointer,
onUiAssetExtractionPointerMove: moveUiAssetExtractionPointer,
onUiAssetExtractionPointerEnd: endUiAssetExtractionPointer,
onRequestUiAssetExtractionReferenceUpload: () =>
requestUpload('ui-asset-extraction-reference'),
onRemoveUiAssetExtractionReference: removeUiAssetExtractionReference,
onSubmitUiAssetExtraction: () => void submitUiAssetExtraction(),
contextMenuLayer,
onQuickEditSelectionToolChange: changeQuickEditSelectionTool,
@@ -101,7 +101,9 @@ export function formatIconDescriptionsText(descriptions: string[]): string {
function isSeedanceVideoModel(model: string | undefined) {
const resolvedModel = model ?? DEFAULT_VIDEO_MODEL;
return resolvedModel === 'seedance2.0' || resolvedModel === 'seedance2.0-fast';
return (
resolvedModel === 'seedance2.0' || resolvedModel === 'seedance2.0-fast'
);
}
function getReferenceMediaType(layer: CanvasLayer) {
@@ -301,6 +303,7 @@ export function createIconGenerationDialogDraft({
status: 'idle',
composerOpen: true,
iconSpecReference: null,
generationReferences: [],
iconDescriptions: [],
imageModel: normalizedImageModel,
aspectRatio: dimensionDefaults.aspectRatio,
@@ -501,9 +504,7 @@ function resolveUserGenerationPromptSnapshot(sourceLayer: CanvasLayer) {
return structuredLines.join('\n');
}
function resolveAudioRedrawPrompt(
sourceLayer: CanvasLayer,
) {
function resolveAudioRedrawPrompt(sourceLayer: CanvasLayer) {
return resolveUserGenerationPromptSnapshot(sourceLayer);
}
@@ -525,8 +526,9 @@ function resolveAudioRedrawMode(sourceLayer: CanvasLayer) {
function resolveAudioRedrawSoundModel(
sourceLayer: CanvasLayer,
): NonNullable<GenerateDialogState['soundModel']> {
const fieldValue = findGenerationInputFieldValue(sourceLayer, ['model'])
?.trim();
const fieldValue = findGenerationInputFieldValue(sourceLayer, [
'model',
])?.trim();
// 中文注释:编辑器音效改造入口当前只暴露 Vidu audio1.0,历史快照里的其他模型统一回落到默认模型。
return fieldValue === DEFAULT_SOUND_EFFECT_MODEL ? 'audio1.0' : 'audio1.0';
}
@@ -578,7 +580,6 @@ export function createAudioRedrawGenerationDialogDraft(
};
}
export function createVideoRedrawGenerationDialogDraft(
sourceLayer: CanvasLayer,
): Omit<CanvasGenerationDialogState, 'id'> | null {
@@ -671,10 +672,9 @@ function buildSourceSidePlaceholder(sourceLayer: CanvasLayer) {
};
}
function placeDraftBesideSourceLayer<T extends Omit<CanvasGenerationDialogState, 'id'>>(
draft: T,
sourceLayer: CanvasLayer,
): T {
function placeDraftBesideSourceLayer<
T extends Omit<CanvasGenerationDialogState, 'id'>,
>(draft: T, sourceLayer: CanvasLayer): T {
if (!draft.placeholder) {
return draft;
}
@@ -719,7 +719,10 @@ function resolveSpecTypeFromSourceLayer(
if (sourceDialog?.mode === 'spec' && sourceDialog.specType) {
return sourceDialog.specType;
}
if (sourceLayer.assetKind === 'icon-spec' || isIconSpecReferenceLayer(sourceLayer)) {
if (
sourceLayer.assetKind === 'icon-spec' ||
isIconSpecReferenceLayer(sourceLayer)
) {
return 'icon';
}
const fields = getGenerationInputFieldValues(sourceLayer);
@@ -773,7 +776,10 @@ function resolvePublicationWorkflowIdFromLayer(
sourceLayer: CanvasLayer,
sourceDialog?: CanvasGenerationDialogState | null,
): PublicationMaterialsWorkflowId {
if (sourceDialog?.mode === 'publication' && sourceDialog.publicationWorkflowId) {
if (
sourceDialog?.mode === 'publication' &&
sourceDialog.publicationWorkflowId
) {
return sourceDialog.publicationWorkflowId;
}
const ratio = inferEditorImageAspectRatio(
@@ -833,7 +839,9 @@ function restoreIconDescriptionsFromLayer(
iconDescriptions: descriptions.length
? descriptions
: (sourceDialog?.iconDescriptions ?? draft.iconDescriptions),
iconSpecReference: sourceDialog?.iconSpecReference ?? draft.iconSpecReference,
iconSpecReference:
sourceDialog?.iconSpecReference ?? draft.iconSpecReference,
generationReferences: sourceDialog?.generationReferences ?? [],
};
}
@@ -843,7 +851,10 @@ function createSameSourceCharacterAnimationDraft({
viewport,
sourceAnimationLayer,
mode,
}: SourceGenerationDialogDraftContext): Omit<CanvasGenerationDialogState, 'id'> | null {
}: SourceGenerationDialogDraftContext): Omit<
CanvasGenerationDialogState,
'id'
> | null {
const animationSourceLayer = sourceAnimationLayer ?? sourceLayer;
const draft = createCharacterAnimationGenerationDialogDraft({
canvasSize,
@@ -871,7 +882,10 @@ export function createSameSourceGenerationDialogDraft({
sourceDialog,
mode,
sourceAnimationLayer,
}: SourceGenerationDialogDraftContext): Omit<CanvasGenerationDialogState, 'id'> | null {
}: SourceGenerationDialogDraftContext): Omit<
CanvasGenerationDialogState,
'id'
> | null {
const sourceMode = resolveGeneratedSourceDialogMode({
sourceLayer,
sourceDialog,
@@ -919,7 +933,10 @@ export function createSameSourceGenerationDialogDraft({
...createCharacterGenerationDialogDraft({
canvasSize,
viewport,
imageModel: sourceDialog?.imageModel ?? sourceLayer.model ?? DEFAULT_IMAGE_MODEL,
imageModel:
sourceDialog?.imageModel ??
sourceLayer.model ??
DEFAULT_IMAGE_MODEL,
}),
prompt,
sourceLayerId: sourceLayer.id,
@@ -994,7 +1011,9 @@ export function createSameSourceGenerationDialogDraft({
canvasSize,
viewport,
imageModel:
sourceDialog?.imageModel ?? sourceLayer.model ?? DEFAULT_IMAGE_MODEL,
sourceDialog?.imageModel ??
sourceLayer.model ??
DEFAULT_IMAGE_MODEL,
}),
sourceLayerId: sourceLayer.id,
},
@@ -1080,10 +1099,16 @@ export function createEditDialogDraft(
);
const aspectRatio =
options.aspectRatio ??
inferEditorImageAspectRatio(sourceLayer.originalWidth, sourceLayer.originalHeight);
inferEditorImageAspectRatio(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
);
const imageSize =
options.imageSize ??
inferEditorImageSizeLabel(sourceLayer.originalWidth, sourceLayer.originalHeight);
inferEditorImageSizeLabel(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
);
return {
mode: 'edit',
prompt: resolveUserGenerationPromptSnapshot(sourceLayer),
@@ -1104,12 +1129,14 @@ export function createQuickEditPanelDraft(
imageSize?: string;
} = {},
): QuickEditPanelState {
const aspectRatio = options.aspectRatio ??
const aspectRatio =
options.aspectRatio ??
inferEditorImageAspectRatio(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
);
const imageSize = options.imageSize ??
const imageSize =
options.imageSize ??
inferEditorImageSizeLabel(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
@@ -1258,7 +1285,8 @@ export function appendCharacterReference(
dialog: GenerateDialogState | null,
layer: CanvasLayer,
): GenerateDialogState | null {
return dialog?.mode === 'character' && getReferenceMediaType(layer) === 'image'
return dialog?.mode === 'character' &&
getReferenceMediaType(layer) === 'image'
? {
...resetFailedGenerationDialog(dialog),
characterReferences: [
@@ -1287,7 +1315,12 @@ export function appendGenerationReference(
if (dialog?.mode === 'video' && !isSeedanceVideoModel(dialog.videoModel)) {
return dialog;
}
if (dialog?.mode === 'generate' || dialog?.mode === 'quick-edit') {
if (
dialog?.mode === 'generate' ||
dialog?.mode === 'quick-edit' ||
dialog?.mode === 'icon' ||
dialog?.mode === 'ui-design'
) {
if (getReferenceMediaType(layer) !== 'image') {
return dialog;
}
@@ -1316,7 +1349,8 @@ export function appendPublicationReference(
dialog: GenerateDialogState | null,
layer: CanvasLayer,
): GenerateDialogState | null {
return dialog?.mode === 'publication' && getReferenceMediaType(layer) === 'image'
return dialog?.mode === 'publication' &&
getReferenceMediaType(layer) === 'image'
? {
...resetFailedGenerationDialog(dialog),
publicationReferences: [
@@ -1396,7 +1430,11 @@ export function appendQuickEditReference(
panel: QuickEditPanelState | null,
layer: CanvasLayer,
): QuickEditPanelState | null {
if (!panel || panel.mode === 'redraw' || getReferenceMediaType(layer) !== 'image') {
if (
!panel ||
panel.mode === 'redraw' ||
getReferenceMediaType(layer) !== 'image'
) {
return panel;
}
return {
@@ -17,6 +17,7 @@ import {
buildSoundEffectGenerationInputs,
buildSpecGenerationInputs,
buildSpecPrompt,
buildUiDesignAssetExtractionGenerationInputs,
calculateCharacterAnimationPrice,
calculateEditorBackgroundMusicPrice,
calculateEditorIconSpritesheetPrice,
@@ -73,17 +74,27 @@ describe('ImageCanvasGenerationModel', () => {
(model) => !(model in EDITOR_MODEL_MUD_POINT_CONFIG),
),
).toEqual([]);
expect(EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG[IMAGE_MODEL_GPT_IMAGE_2]['1K']).toBe(3);
expect(EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG[IMAGE_MODEL_GPT_IMAGE_2]['2K']).toBe(5);
expect(EDITOR_MODEL_MUD_POINT_CONFIG[CHARACTER_ANIMATION_MODEL]).toBeTruthy();
expect(
EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG[IMAGE_MODEL_GPT_IMAGE_2]['1K'],
).toBe(3);
expect(
EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG[IMAGE_MODEL_GPT_IMAGE_2]['2K'],
).toBe(5);
expect(
EDITOR_MODEL_MUD_POINT_CONFIG[CHARACTER_ANIMATION_MODEL],
).toBeTruthy();
expect(
EDITOR_SOUND_EFFECT_MODEL_MUD_POINT_CONFIG[DEFAULT_SOUND_EFFECT_MODEL],
).toBe(5);
expect(
EDITOR_BACKGROUND_MUSIC_MODEL_MUD_POINT_CONFIG[BACKGROUND_MUSIC_MODEL_SUNO],
EDITOR_BACKGROUND_MUSIC_MODEL_MUD_POINT_CONFIG[
BACKGROUND_MUSIC_MODEL_SUNO
],
).toBe(12);
// 中文注释:Veo 已从前端入口移除,但客户端类型和后端兼容层仍保留,定价表也必须覆盖,避免旧布局回放或测试请求价格漂移。
expect(EDITOR_VIDEO_MODEL_MUD_POINT_CONFIG[VIDEO_MODEL_VEO_3_1]).toBeTruthy();
expect(
EDITOR_VIDEO_MODEL_MUD_POINT_CONFIG[VIDEO_MODEL_VEO_3_1],
).toBeTruthy();
expect(
EDITOR_VIDEO_MODEL_MUD_POINT_CONFIG[VIDEO_MODEL_VEO_3_1_FAST],
).toBeTruthy();
@@ -93,8 +104,12 @@ describe('ImageCanvasGenerationModel', () => {
expect(calculateEditorImageModelPrice(DEFAULT_IMAGE_MODEL)).toBe(
EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG[DEFAULT_IMAGE_MODEL]['1K'],
);
expect(calculateEditorImageModelPrice(IMAGE_MODEL_GPT_IMAGE_2, '1K')).toBe(3);
expect(calculateEditorImageModelPrice(IMAGE_MODEL_GPT_IMAGE_2, '2K')).toBe(5);
expect(calculateEditorImageModelPrice(IMAGE_MODEL_GPT_IMAGE_2, '1K')).toBe(
3,
);
expect(calculateEditorImageModelPrice(IMAGE_MODEL_GPT_IMAGE_2, '2K')).toBe(
5,
);
expect(
calculateEditorImageGenerationPrice({
kind: 'character',
@@ -120,21 +135,25 @@ describe('ImageCanvasGenerationModel', () => {
expect(calculateEditorVideoPrice(VIDEO_MODEL_KLING_3, '480p', 4)).toBe(
EDITOR_VIDEO_MODEL_MUD_POINT_CONFIG[VIDEO_MODEL_KLING_3]['480p'] * 4,
);
expect(calculateEditorVideoPrice(VIDEO_MODEL_KLING_3, '720p', 5)).toBe(
150,
);
expect(calculateEditorVideoPrice(VIDEO_MODEL_KLING_3, '720p', 5)).toBe(150);
expect(calculateEditorVideoPrice(VIDEO_MODEL_KLING_3_OMNI, '720p', 5)).toBe(
200,
);
expect(calculateCharacterAnimationPrice(CHARACTER_ANIMATION_MODEL, '720p', 6))
.toBe(
EDITOR_VIDEO_MODEL_MUD_POINT_CONFIG[CHARACTER_ANIMATION_MODEL]['720p'] * 6,
);
expect(
calculateCharacterAnimationPrice(CHARACTER_ANIMATION_MODEL, '720p', 6),
).toBe(
EDITOR_VIDEO_MODEL_MUD_POINT_CONFIG[CHARACTER_ANIMATION_MODEL]['720p'] *
6,
);
expect(calculateEditorSoundEffectPrice(DEFAULT_SOUND_EFFECT_MODEL)).toBe(
EDITOR_SOUND_EFFECT_MODEL_MUD_POINT_CONFIG[DEFAULT_SOUND_EFFECT_MODEL],
);
expect(calculateEditorBackgroundMusicPrice(BACKGROUND_MUSIC_MODEL_SUNO)).toBe(
EDITOR_BACKGROUND_MUSIC_MODEL_MUD_POINT_CONFIG[BACKGROUND_MUSIC_MODEL_SUNO],
expect(
calculateEditorBackgroundMusicPrice(BACKGROUND_MUSIC_MODEL_SUNO),
).toBe(
EDITOR_BACKGROUND_MUSIC_MODEL_MUD_POINT_CONFIG[
BACKGROUND_MUSIC_MODEL_SUNO
],
);
});
@@ -167,15 +186,21 @@ describe('ImageCanvasGenerationModel', () => {
},
});
expect(calculateEditorImageModelPrice(IMAGE_MODEL_GPT_IMAGE_2, '1K')).toBe(29);
expect(calculateEditorSpecGenerationPrice(IMAGE_MODEL_GPT_IMAGE_2)).toBe(58);
expect(calculateEditorImageModelPrice(IMAGE_MODEL_GPT_IMAGE_2, '1K')).toBe(
29,
);
expect(calculateEditorSpecGenerationPrice(IMAGE_MODEL_GPT_IMAGE_2)).toBe(
58,
);
expect(calculateEditorVideoPrice(VIDEO_MODEL_SEEDANCE_2, '720p', 5)).toBe(
130,
);
expect(calculateEditorSoundEffectPrice(DEFAULT_SOUND_EFFECT_MODEL)).toBe(15);
expect(calculateEditorBackgroundMusicPrice(BACKGROUND_MUSIC_MODEL_SUNO)).toBe(
9,
expect(calculateEditorSoundEffectPrice(DEFAULT_SOUND_EFFECT_MODEL)).toBe(
15,
);
expect(
calculateEditorBackgroundMusicPrice(BACKGROUND_MUSIC_MODEL_SUNO),
).toBe(9);
});
it('builds user-facing generation input snapshots instead of backend prompts', () => {
@@ -250,9 +275,7 @@ describe('ImageCanvasGenerationModel', () => {
resourceId: 'resource-icon-spec',
}),
).toEqual({
fields: [
{ title: '素材描述', value: '返回按钮\n设置按钮' },
],
fields: [{ title: '素材描述', value: '返回按钮\n设置按钮' }],
references: [
{
title: '图标规范',
@@ -277,14 +300,51 @@ describe('ImageCanvasGenerationModel', () => {
],
});
expect(
buildQuickEditGenerationInputs('快速编辑提示词', '让图2下雨', sourceLayer, [
buildUiDesignAssetExtractionGenerationInputs(sourceLayer, [
{
id: 'ref-1',
label: '天空',
src: '/sky.png',
resourceId: 'resource-sky',
id: 'ref-ui',
label: 'UI风格参考',
src: '/ui-style.png',
resourceId: 'resource-ui-style',
},
]),
).toEqual({
fields: [
{
title: '提取提示词',
value:
'仅提取被红色框框选的素材并整理成spritesheet,图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕,方便后续扣除背景;素材自身不要出现绿色描边、绿色底板或绿色阴影。',
},
],
references: [
{
title: 'UI设计图',
label: '原图',
refType: 'project-resource',
refId: 'resource-source',
},
{
title: '参考图 1',
label: 'UI风格参考',
refType: 'project-resource',
refId: 'resource-ui-style',
},
],
});
expect(
buildQuickEditGenerationInputs(
'快速编辑提示词',
'让图2下雨',
sourceLayer,
[
{
id: 'ref-1',
label: '天空',
src: '/sky.png',
resourceId: 'resource-sky',
},
],
),
).toEqual({
fields: [{ title: '快速编辑提示词', value: '让图2下雨' }],
references: [
@@ -309,8 +369,9 @@ describe('ImageCanvasGenerationModel', () => {
expect(prompt).toContain('生成一张完整游戏UI规范汇总设定展板');
expect(prompt).toContain('玩法设定:消除玩法');
expect(buildSpecPrompt('custom', { ...blankSpecValues, customPrompt: '自定义' }))
.toBe('自定义');
expect(
buildSpecPrompt('custom', { ...blankSpecValues, customPrompt: '自定义' }),
).toBe('自定义');
expect(buildQuickEditModelOptions('nano-banana')).toEqual([
{ label: 'nanobanana2', value: DEFAULT_IMAGE_MODEL },
{ label: 'gpt-image-2', value: 'gpt-image-2' },
@@ -80,7 +80,9 @@ export const EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG = {
},
} as const;
export const SPEC_GENERATION_COST =
EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG[SPEC_GENERATION_MODEL][SPEC_GENERATION_IMAGE_SIZE];
EDITOR_IMAGE_MODEL_MUD_POINT_CONFIG[SPEC_GENERATION_MODEL][
SPEC_GENERATION_IMAGE_SIZE
];
export const ICON_DESCRIPTION_LIMIT = 100;
export const DEFAULT_PUBLICATION_GAME_INFO: PublicationMaterialsGameInfo = {
gameName: '',
@@ -194,15 +196,18 @@ export function inferEditorImageAspectRatio(width: number, height: number) {
const safeWidth = Number.isFinite(width) && width > 0 ? Math.round(width) : 1;
const safeHeight =
Number.isFinite(height) && height > 0 ? Math.round(height) : 1;
const commonRatios = EDITOR_IMAGE_DIMENSION_OPTIONS[
IMAGE_MODEL_NANOBANANA2
].aspectRatios as readonly string[];
const commonRatios = EDITOR_IMAGE_DIMENSION_OPTIONS[IMAGE_MODEL_NANOBANANA2]
.aspectRatios as readonly string[];
const sourceRatio = safeWidth / safeHeight;
return commonRatios.reduce((bestRatio, ratio) => {
const bestNumbers = resolveAspectRatioNumbers(bestRatio);
const ratioNumbers = resolveAspectRatioNumbers(ratio);
const bestDiff = Math.abs(bestNumbers.width / bestNumbers.height - sourceRatio);
const ratioDiff = Math.abs(ratioNumbers.width / ratioNumbers.height - sourceRatio);
const bestDiff = Math.abs(
bestNumbers.width / bestNumbers.height - sourceRatio,
);
const ratioDiff = Math.abs(
ratioNumbers.width / ratioNumbers.height - sourceRatio,
);
return ratioDiff < bestDiff ? ratio : bestRatio;
}, commonRatios[0] ?? '1:1');
}
@@ -245,9 +250,15 @@ export function resizeGenerationPlaceholderToImageSelection(
};
}
function resolveEvenVideoWidth(height: number, aspectRatio: string | null | undefined) {
function resolveEvenVideoWidth(
height: number,
aspectRatio: string | null | undefined,
) {
const ratio = resolveAspectRatioNumbers(aspectRatio ?? '16:9');
const rawWidth = Math.max(1, Math.round((height * ratio.width) / ratio.height));
const rawWidth = Math.max(
1,
Math.round((height * ratio.width) / ratio.height),
);
return rawWidth % 2 === 0 ? rawWidth : rawWidth + 1;
}
@@ -382,9 +393,10 @@ export const EDITOR_MODEL_MUD_POINT_CONFIG = {
},
[BACKGROUND_MUSIC_MODEL_SUNO]: {
unit: 'perGeneration',
price: EDITOR_BACKGROUND_MUSIC_MODEL_MUD_POINT_CONFIG[
BACKGROUND_MUSIC_MODEL_SUNO
],
price:
EDITOR_BACKGROUND_MUSIC_MODEL_MUD_POINT_CONFIG[
BACKGROUND_MUSIC_MODEL_SUNO
],
},
} satisfies EditorGenerationPricingConfig['models'];
let runtimeEditorGenerationPricingConfig: EditorGenerationPricingConfig = {
@@ -446,7 +458,11 @@ export const EDITOR_VIDEO_ASPECT_RATIO_OPTIONS = [
'3:4',
'21:9',
] as const;
export const EDITOR_VIDEO_RESOLUTION_OPTIONS = ['480p', '720p', '1080p'] as const;
export const EDITOR_VIDEO_RESOLUTION_OPTIONS = [
'480p',
'720p',
'1080p',
] as const;
export const DEFAULT_VIDEO_ASPECT_RATIO = '16:9';
export const DEFAULT_VIDEO_DURATION_SECONDS = 4;
export const DEFAULT_VIDEO_SOUND = 'on';
@@ -509,9 +525,7 @@ export function buildQuickEditSizeOptions(currentSize: string) {
return Array.from(new Set([currentSize, ...QUICK_EDIT_SIZE_PRESETS]));
}
export function normalizeEditorImageModel(
model: string | null | undefined,
) {
export function normalizeEditorImageModel(model: string | null | undefined) {
const normalizedModel = model?.trim();
if (!normalizedModel) {
return DEFAULT_IMAGE_MODEL;
@@ -527,8 +541,9 @@ export function getEditorImageModelDisplayName(
) {
const normalizedModel = normalizeEditorImageModel(model);
return (
EDITOR_IMAGE_MODEL_OPTIONS.find((option) => option.value === normalizedModel)
?.label ?? normalizedModel
EDITOR_IMAGE_MODEL_OPTIONS.find(
(option) => option.value === normalizedModel,
)?.label ?? normalizedModel
);
}
@@ -537,7 +552,10 @@ export function buildQuickEditModelOptions(currentModel: string) {
const options = [...QUICK_EDIT_MODEL_OPTIONS];
return options.some((option) => option.value === normalizedCurrentModel)
? options
: [{ label: normalizedCurrentModel, value: normalizedCurrentModel }, ...options];
: [
{ label: normalizedCurrentModel, value: normalizedCurrentModel },
...options,
];
}
export function buildCharacterSpecPrompt(values: SpecFormValues) {
@@ -734,9 +752,7 @@ function readModelTierPrice({
function readModelFlatPrice(model: string, fallbackModel: string) {
return (
getModelPrices(model)?.price ??
getModelPrices(fallbackModel)?.price ??
0
getModelPrices(model)?.price ?? getModelPrices(fallbackModel)?.price ?? 0
);
}
@@ -789,7 +805,10 @@ export function calculateEditorImageModelPrice(
export function calculateEditorSpecGenerationPrice(
model: string | null | undefined = SPEC_GENERATION_MODEL,
) {
return calculateEditorImageModelPrice(model ?? SPEC_GENERATION_MODEL, SPEC_GENERATION_IMAGE_SIZE);
return calculateEditorImageModelPrice(
model ?? SPEC_GENERATION_MODEL,
SPEC_GENERATION_IMAGE_SIZE,
);
}
export function calculateEditorImageGenerationPrice({
@@ -870,9 +889,10 @@ export function resolveCharacterAnimationSourceImageSrc(layer: CanvasLayer) {
export function createCanvasLayerReference(
layer: CanvasLayer,
): CharacterReferenceImage {
const mediaType = layer.mediaType === 'video' || layer.mediaType === 'audio'
? layer.mediaType
: undefined;
const mediaType =
layer.mediaType === 'video' || layer.mediaType === 'audio'
? layer.mediaType
: undefined;
return {
id: `canvas-${layer.id}`,
label: layer.title,
@@ -938,10 +958,10 @@ export function appendLimitedQuickEditReferences(
const imageReferences = nextReferences.filter(
(reference) => (reference.mediaType ?? 'image') === 'image',
);
return [
...(references ?? []),
...imageReferences,
].slice(0, QUICK_EDIT_REFERENCE_LIMIT);
return [...(references ?? []), ...imageReferences].slice(
0,
QUICK_EDIT_REFERENCE_LIMIT,
);
}
export function createGenerationInputField(
@@ -1029,7 +1049,10 @@ export function buildSpecGenerationInputs(
: [];
if (specType === 'custom') {
return {
fields: createGenerationInputField('自定义规范提示词', values.customPrompt),
fields: createGenerationInputField(
'自定义规范提示词',
values.customPrompt,
),
references,
};
}
@@ -1071,12 +1094,18 @@ export function buildCharacterGenerationInputs(
export function buildUiDesignGenerationInputs(
prompt: string,
specReference: CharacterReferenceImage | null | undefined,
references?: CharacterReferenceImage[],
): CanvasGenerationInputs {
return {
fields: createGenerationInputField('用户输入', prompt),
references: specReference
? createGenerationInputReference('图标规范', specReference)
: [],
references: [
...(specReference
? createGenerationInputReference('图标规范', specReference)
: []),
...(references ?? []).flatMap((reference, index) =>
createGenerationInputReference(`参考图 ${index + 1}`, reference),
),
],
};
}
@@ -1096,7 +1125,10 @@ export function buildPublicationMaterialsPrompt(
const normalizedGameInfo = normalizePublicationGameInfo(gameInfo);
return [
...createGenerationInputField('游戏名', normalizedGameInfo.gameName),
...createGenerationInputField('游戏分类', normalizedGameInfo.gameCategories),
...createGenerationInputField(
'游戏分类',
normalizedGameInfo.gameCategories,
),
...createGenerationInputField(
'一句话描述游戏',
normalizedGameInfo.gameDescription,
@@ -1116,7 +1148,9 @@ export function buildPublicationMaterialsGenerationPrompt({
const inputPrompt = buildPublicationMaterialsPrompt(gameInfo);
return [
`【宣发素材类型】${workflow.promptLabel ?? workflow.label}`,
inputPrompt ? `【游戏输入】\n${inputPrompt}` : '【游戏输入】\n请根据参考图生成宣发素材。',
inputPrompt
? `【游戏输入】\n${inputPrompt}`
: '【游戏输入】\n请根据参考图生成宣发素材。',
`【生图约束】\n${workflow.promptConstraints.join('\n')}`,
`【参考图约束】\n${workflow.referenceConstraints.join('\n')}`,
`【输出检查】\n${workflow.postProcessConstraints.join('\n')}`,
@@ -1131,7 +1165,10 @@ export function buildPublicationMaterialsGenerationInputs(
return {
fields: [
...createGenerationInputField('游戏名', normalizedGameInfo.gameName),
...createGenerationInputField('游戏分类', normalizedGameInfo.gameCategories),
...createGenerationInputField(
'游戏分类',
normalizedGameInfo.gameCategories,
),
...createGenerationInputField(
'一句话描述游戏',
normalizedGameInfo.gameDescription,
@@ -1146,6 +1183,7 @@ export function buildPublicationMaterialsGenerationInputs(
export function buildIconGenerationInputs(
iconDescriptions: string[],
specReference: CharacterReferenceImage,
references?: CharacterReferenceImage[],
): CanvasGenerationInputs {
return {
fields: [
@@ -1154,19 +1192,30 @@ export function buildIconGenerationInputs(
value: iconDescriptions.join('\n'),
},
],
references: createGenerationInputReference('图标规范', specReference),
references: [
...createGenerationInputReference('图标规范', specReference),
...(references ?? []).flatMap((reference, index) =>
createGenerationInputReference(`参考图 ${index + 1}`, reference),
),
],
};
}
export function buildUiDesignAssetExtractionGenerationInputs(
sourceLayer: CanvasLayer,
references?: CharacterReferenceImage[],
): CanvasGenerationInputs {
return {
fields: createGenerationInputField(
'提取提示词',
UI_DESIGN_ASSET_EXTRACTION_PROMPT,
),
references: createLayerGenerationInputReference('UI设计图', sourceLayer),
references: [
...createLayerGenerationInputReference('UI设计图', sourceLayer),
...(references ?? []).flatMap((reference, index) =>
createGenerationInputReference(`参考图 ${index + 1}`, reference),
),
],
};
}
@@ -1267,9 +1316,11 @@ export function getGenerationFrameLabel(dialog: CanvasGenerationDialogState) {
return 'Icon Generator';
}
if (dialog.mode === 'publication') {
return `${getPublicationMaterialsWorkflow(
dialog.publicationWorkflowId ?? 'publication-cover-image',
).englishName} Generator`;
return `${
getPublicationMaterialsWorkflow(
dialog.publicationWorkflowId ?? 'publication-cover-image',
).englishName
} Generator`;
}
if (dialog.mode === 'ui-design') {
return 'UI Design Generator';
@@ -343,6 +343,14 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
src: 'data:image/png;base64,ui-spec',
resourceId: 'resource-ui-spec',
},
generationReferences: [
{
id: 'ui-ref',
label: '界面风格参考',
src: 'data:image/png;base64,ui-ref',
resourceId: 'resource-ui-ref',
},
],
},
layers: [],
nextGeneratedIndex: 6,
@@ -356,11 +364,31 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
model: IMAGE_MODEL_GPT_IMAGE_2,
aspectRatio: '16:9',
imageSize: '1K',
referenceImageSrcs: ['data:image/png;base64,ui-spec'],
referenceImageSrcs: [
'data:image/png;base64,ui-spec',
'data:image/png;base64,ui-ref',
],
},
result: {
assetKind: 'ui-design',
title: 'UI设计图 6',
generationInputs: {
fields: [{ title: '用户输入', value: '设计移动端商城首页' }],
references: [
{
title: '图标规范',
label: 'UI图标规范',
refType: 'project-resource',
refId: 'resource-ui-spec',
},
{
title: '参考图 1',
label: '界面风格参考',
refType: 'project-resource',
refId: 'resource-ui-ref',
},
],
},
},
rememberImageModel: IMAGE_MODEL_GPT_IMAGE_2,
});
@@ -611,6 +639,14 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
src: 'data:image/png;base64,spec',
resourceId: 'resource-icon-spec',
},
generationReferences: [
{
id: 'icon-ref',
label: '按钮风格参考',
src: 'data:image/png;base64,ref',
resourceId: 'resource-icon-ref',
},
],
iconDescriptions: ['旧返回', '旧设置'],
});
@@ -619,6 +655,7 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
iconDescriptions: ['返回按钮', '设置按钮'],
input: {
referenceImageSrc: 'data:image/png;base64,spec',
referenceImageSrcs: ['data:image/png;base64,ref'],
iconDescriptions: ['返回按钮', '设置按钮'],
model: 'gpt-image-2',
aspectRatio: '3:2',
@@ -633,6 +670,12 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
refType: 'project-resource',
refId: 'resource-icon-spec',
},
{
title: '参考图 1',
label: '按钮风格参考',
refType: 'project-resource',
refId: 'resource-icon-ref',
},
],
},
rememberImageModel: 'gpt-image-2',
@@ -72,7 +72,7 @@ function normalizeGptImageSize(imageSize: string | null | undefined) {
return imageSize?.trim().toUpperCase() === '2K' ? '2K' : '1K';
}
function resolveImageReferenceSubmissionSource(reference: {
export function resolveImageReferenceSubmissionSource(reference: {
src: string;
objectKey?: string | null;
}) {
@@ -378,6 +378,9 @@ export function buildImageGenerationSubmissionPlan({
dialog.uiDesignSpecReference
? resolveImageReferenceSubmissionSource(dialog.uiDesignSpecReference)
: null,
...(dialog.generationReferences ?? []).map((reference) =>
resolveImageReferenceSubmissionSource(reference),
),
].filter((src): src is string => Boolean(src));
return {
kind: 'image',
@@ -396,6 +399,7 @@ export function buildImageGenerationSubmissionPlan({
generationInputs: buildUiDesignGenerationInputs(
normalizedPrompt,
dialog.uiDesignSpecReference,
dialog.generationReferences,
),
},
rememberImageModel: imageModel,
@@ -546,8 +550,8 @@ export function buildImageGenerationSubmissionPlan({
imageSize: dialog.imageSize ?? '1K',
...(dialog.generationReferences?.length
? {
referenceImageSrcs: dialog.generationReferences.map(
(reference) => resolveImageReferenceSubmissionSource(reference),
referenceImageSrcs: dialog.generationReferences.map((reference) =>
resolveImageReferenceSubmissionSource(reference),
),
}
: {}),
@@ -597,6 +601,13 @@ export function buildIconSpritesheetGenerationSubmissionPlan(
referenceImageSrc: resolveImageReferenceSubmissionSource(
dialog.iconSpecReference,
),
...(dialog.generationReferences?.length
? {
referenceImageSrcs: dialog.generationReferences.map((reference) =>
resolveImageReferenceSubmissionSource(reference),
),
}
: {}),
iconDescriptions,
model: rememberImageModel,
aspectRatio: dialog.aspectRatio ?? '1:1',
@@ -605,6 +616,7 @@ export function buildIconSpritesheetGenerationSubmissionPlan(
generationInputs: buildIconGenerationInputs(
iconDescriptions,
dialog.iconSpecReference,
dialog.generationReferences,
),
rememberImageModel,
};
@@ -119,7 +119,9 @@ describe('ImageCanvasIconSpritesheetComposerView', () => {
render(<IconComposerHarness initialDialog={createIconDialog()} />);
const panel = screen.getByRole('dialog', { name: '生成图标素材' });
const firstRow = panel.querySelector('.image-canvas-editor__reference-strip');
const firstRow = panel.querySelector(
'.image-canvas-editor__reference-strip',
);
expect(firstRow).toBeTruthy();
expect(
@@ -154,6 +156,31 @@ describe('ImageCanvasIconSpritesheetComposerView', () => {
expect(screen.queryByRole('button', { name: '图标规范A' })).toBeNull();
});
it('uploads and removes icon generation references', () => {
const requestUpload = vi.fn();
render(
<IconComposerHarness
initialDialog={createIconDialog({
generationReferences: [
{
id: 'ref-a',
label: '按钮风格参考',
src: 'data:image/png;base64,ref',
},
],
})}
onRequestUpload={requestUpload}
/>,
);
expect(screen.getByLabelText('按钮风格参考')).toBeTruthy();
fireEvent.click(screen.getByRole('button', { name: '上传图标素材参考图' }));
expect(requestUpload).toHaveBeenCalledWith('generation-reference');
fireEvent.click(screen.getByRole('button', { name: '删除按钮风格参考' }));
expect(screen.queryByLabelText('按钮风格参考')).toBeNull();
});
it('updates multi-line descriptions and submits', () => {
const updateIconDescriptionText = vi.fn();
const submitIconGeneration = vi.fn();
@@ -214,9 +241,9 @@ describe('ImageCanvasIconSpritesheetComposerView', () => {
fireEvent.click(screen.getByRole('button', { name: '生成' }));
expect((screen.getByLabelText('素材描述') as HTMLInputElement).disabled).toBe(
true,
);
expect(
(screen.getByLabelText('素材描述') as HTMLInputElement).disabled,
).toBe(true);
expect(submitIconGeneration).not.toHaveBeenCalled();
rerender(
@@ -1,4 +1,4 @@
import { ImageIcon } from 'lucide-react';
import { ImageIcon, ImagePlus } from 'lucide-react';
import {
type CSSProperties,
type Dispatch,
@@ -84,6 +84,7 @@ export function ImageCanvasIconSpritesheetComposerView({
setIsIconSpecMenuOpen(false);
onRequestUpload('icon-spec');
};
const references = dialog.generationReferences ?? [];
return (
<form
@@ -142,10 +143,7 @@ export function ImageCanvasIconSpritesheetComposerView({
className="image-canvas-editor__character-spec-menu image-canvas-editor__portal-menu"
label="图标规范来源"
placement="top-start"
style={buildPortalMenuStyle(
iconSpecButtonRef.current,
'above',
)}
style={buildPortalMenuStyle(iconSpecButtonRef.current, 'above')}
>
<PlatformFloatingMenuItem
className="image-canvas-editor__context-menu-item"
@@ -168,6 +166,49 @@ export function ImageCanvasIconSpritesheetComposerView({
</PlatformFloatingMenu>,
)
: null}
{references.map((reference, index) => (
<ImageCanvasReferenceSlot
key={reference.id}
tone="default"
icon={<ImageIcon className="h-4 w-4" aria-hidden="true" />}
imageSrc={reference.src}
objectKey={reference.objectKey}
label={`参考图${index + 1}`}
ariaLabel={reference.label || `参考图${index + 1}`}
title={reference.label}
disabled={dialog.status === 'generating'}
onRemove={() =>
setGenerateDialog((currentDialog) =>
currentDialog?.mode === 'icon'
? {
...currentDialog,
status:
currentDialog.status === 'failed'
? 'idle'
: currentDialog.status,
errorMessage:
currentDialog.status === 'failed'
? undefined
: currentDialog.errorMessage,
generationReferences: (
currentDialog.generationReferences ?? []
).filter((item) => item.id !== reference.id),
}
: currentDialog,
)
}
removeLabel={`删除${reference.label || `参考图${index + 1}`}`}
/>
))}
<ImageCanvasReferenceSlot
tone="default"
icon={<ImagePlus className="h-4 w-4" aria-hidden="true" />}
label="参考图"
ariaLabel="上传图标素材参考图"
disabled={dialog.status === 'generating'}
isAdd
onClick={() => onRequestUpload('generation-reference')}
/>
</div>
<label className="image-canvas-editor__field-block image-canvas-editor__field-block--single">
<PlatformTextField
@@ -110,13 +110,17 @@ describe('ImageCanvasQuickEditPanelView', () => {
);
expect(screen.getByLabelText('当前状态').textContent).toBe('idle');
expect(screen.getByLabelText('当前错误').textContent).toBe('-');
fireEvent.click(screen.getByRole('button', { name: '快速编辑尺寸 1:1·1K' }));
fireEvent.click(
screen.getByRole('button', { name: '快速编辑尺寸 1:1·1K' }),
);
fireEvent.click(screen.getByRole('button', { name: '比例 16:9' }));
fireEvent.click(screen.getByRole('button', { name: '尺寸 2K' }));
expect(screen.getByLabelText('当前比例').textContent).toBe('16:9');
expect(screen.getByLabelText('当前图片规格').textContent).toBe('2K');
fireEvent.click(screen.getByRole('button', { name: '快速编辑模型 nanobanana2' }));
fireEvent.click(
screen.getByRole('button', { name: '快速编辑模型 nanobanana2' }),
);
fireEvent.click(screen.getByRole('button', { name: 'gpt-image-2' }));
expect(screen.getByLabelText('当前模型').textContent).toBe('gpt-image-2');
expect(rememberImageModel).toHaveBeenCalledWith('gpt-image-2');
@@ -247,9 +251,11 @@ describe('ImageCanvasQuickEditPanelView', () => {
);
expect(
(screen.getByRole('button', {
name: '添加快速编辑参考图',
}) as HTMLButtonElement).disabled,
(
screen.getByRole('button', {
name: '添加快速编辑参考图',
}) as HTMLButtonElement
).disabled,
).toBe(true);
expect(screen.queryByLabelText('原图 图9')).toBeNull();
});
@@ -49,7 +49,7 @@ function createQuickEditDialog(
panel: QuickEditPanelState,
): GenerateDialogState {
return {
mode: 'quick-edit',
mode: panel.mode === 'redraw' ? 'generate' : 'quick-edit',
prompt: panel.prompt,
status: panel.status,
sourceLayerId: panel.sourceLayerId,
@@ -54,8 +54,10 @@ function renderPanel({
function UiDesignHarness({
initialDialog,
onRequestUpload = vi.fn(),
}: {
initialDialog: GenerateDialogState;
onRequestUpload?: (target: UploadTarget) => void;
}) {
const [dialog, setDialog] = useState<GenerateDialogState | null>(
initialDialog,
@@ -71,7 +73,7 @@ function UiDesignHarness({
renderEditorPortal={(node) => node}
buildPortalMenuStyle={() => ({ position: 'fixed', left: 0, top: 0 })}
onUpdateSpecFormValue={vi.fn()}
onRequestUpload={vi.fn()}
onRequestUpload={onRequestUpload}
onSubmit={vi.fn()}
/>
) : null;
@@ -154,6 +156,39 @@ describe('ImageCanvasSpecGenerationPanelView', () => {
expect(screen.queryByText('nanobanana2')).toBeNull();
});
it('uploads and removes UI design generation references', () => {
const requestUpload = vi.fn();
render(
<UiDesignHarness
onRequestUpload={requestUpload}
initialDialog={{
mode: 'ui-design',
prompt: '',
status: 'idle',
composerOpen: true,
uiDesignSpecReference: null,
generationReferences: [
{
id: 'ref-a',
label: '界面风格参考',
src: 'data:image/png;base64,ref',
},
],
imageModel: 'gpt-image-2',
aspectRatio: '16:9',
imageSize: '1K',
}}
/>,
);
expect(screen.getByLabelText('界面风格参考')).toBeTruthy();
fireEvent.click(screen.getByRole('button', { name: '上传UI设计参考图' }));
expect(requestUpload).toHaveBeenCalledWith('generation-reference');
fireEvent.click(screen.getByRole('button', { name: '删除界面风格参考' }));
expect(screen.queryByLabelText('界面风格参考')).toBeNull();
});
it('renders character spec fields and forwards updates', () => {
const updateSpecFormValue = vi.fn();
renderPanel({
@@ -192,6 +192,59 @@ export function ImageCanvasSpecGenerationPanelView({
removeLabel={`删除${referenceLabel}`}
/>
) : null}
{isUiDesignDialog
? (dialog.generationReferences ?? []).map((item, index) => (
<ImageCanvasReferenceSlot
key={item.id}
tone="default"
icon={<ImagePlus className="h-4 w-4" aria-hidden="true" />}
imageSrc={item.src}
objectKey={item.objectKey}
label={`参考图${index + 1}`}
ariaLabel={item.label || `参考图${index + 1}`}
title={item.label}
disabled={isGenerating}
onRemove={
setGenerateDialog
? () =>
setGenerateDialog((currentDialog) =>
currentDialog?.mode === 'ui-design'
? {
...currentDialog,
status:
currentDialog.status === 'failed'
? 'idle'
: currentDialog.status,
errorMessage:
currentDialog.status === 'failed'
? undefined
: currentDialog.errorMessage,
generationReferences: (
currentDialog.generationReferences ?? []
).filter(
(referenceItem) =>
referenceItem.id !== item.id,
),
}
: currentDialog,
)
: undefined
}
removeLabel={`删除${item.label || `参考图${index + 1}`}`}
/>
))
: null}
{isUiDesignDialog ? (
<ImageCanvasReferenceSlot
tone="default"
icon={<ImagePlus className="h-4 w-4" aria-hidden="true" />}
label="参考图"
ariaLabel="上传UI设计参考图"
disabled={isGenerating}
isAdd
onClick={() => onRequestUpload('generation-reference')}
/>
) : null}
</div>
<div className="image-canvas-editor__spec-fields">
{isUiDesignDialog ? (
@@ -123,6 +123,8 @@ export type ImageCanvasStageViewProps = {
onUiAssetExtractionPointerStart: (point: { x: number; y: number }) => void;
onUiAssetExtractionPointerMove: (point: { x: number; y: number }) => void;
onUiAssetExtractionPointerEnd: () => void;
onRequestUiAssetExtractionReferenceUpload: () => void;
onRemoveUiAssetExtractionReference: (referenceId: string) => void;
onSubmitUiAssetExtraction: () => void;
onQuickEditSelectionToolChange: (tool: UiAssetExtractionTool | null) => void;
onQuickEditSelectionPointerStart: (point: { x: number; y: number }) => void;
@@ -237,6 +239,8 @@ export function ImageCanvasStageView({
onUiAssetExtractionPointerStart,
onUiAssetExtractionPointerMove,
onUiAssetExtractionPointerEnd,
onRequestUiAssetExtractionReferenceUpload,
onRemoveUiAssetExtractionReference,
onSubmitUiAssetExtraction,
onQuickEditSelectionToolChange,
onQuickEditSelectionPointerStart,
@@ -348,6 +352,8 @@ export function ImageCanvasStageView({
onPointerMove={onUiAssetExtractionPointerMove}
onPointerEnd={onUiAssetExtractionPointerEnd}
onModelChange={onUiAssetExtractionModelChange}
onRequestUpload={onRequestUiAssetExtractionReferenceUpload}
onRemoveReference={onRemoveUiAssetExtractionReference}
onSubmit={onSubmitUiAssetExtraction}
/>
@@ -1,3 +1,5 @@
import type { CharacterReferenceImage } from './ImageCanvasEditorTypes';
export type UiAssetExtractionTool = 'rect' | 'ellipse' | 'brush';
export type UiAssetExtractionPoint = {
@@ -29,6 +31,7 @@ export type UiAssetExtractionState = {
tool: UiAssetExtractionTool | null;
model: string;
marks: UiAssetExtractionMark[];
references: CharacterReferenceImage[];
draftMark: UiAssetExtractionMark | null;
status: 'idle' | 'extracting' | 'failed';
errorMessage?: string;
@@ -52,6 +55,7 @@ export function createUiAssetExtractionState(
tool: options.initialTool === undefined ? 'rect' : options.initialTool,
model: options.model,
marks: [],
references: [],
draftMark: null,
status: 'idle',
};
@@ -66,6 +66,7 @@ function createState(
tool: 'rect',
model: IMAGE_MODEL_NANOBANANA2,
marks: [],
references: [],
draftMark: null,
status: 'idle',
...overrides,
@@ -129,15 +130,29 @@ describe('ImageCanvasUiAssetExtractionOverlayView', () => {
height: 36,
}));
const onRequestUpload = vi.fn();
const onRemoveReference = vi.fn();
render(
<ImageCanvasUiAssetExtractionOverlayView
sourceLayer={createLayer({ src: 'data:image/png;base64,ui' })}
viewport={{ x: 10, y: 20, scale: 1 }}
state={createState({ marks })}
state={createState({
marks,
references: [
{
id: 'ui-ref-1',
label: '参考UI.png',
src: 'data:image/png;base64,ui-ref',
},
],
})}
onToolChange={vi.fn()}
onPointerStart={vi.fn()}
onPointerMove={vi.fn()}
onPointerEnd={vi.fn()}
onRequestUpload={onRequestUpload}
onRemoveReference={onRemoveReference}
onSubmit={vi.fn()}
/>,
);
@@ -157,6 +172,15 @@ describe('ImageCanvasUiAssetExtractionOverlayView', () => {
expect(
within(panel).getByRole('button', { name: '提取素材模型 nanobanana2' }),
).toBeTruthy();
expect(within(panel).getByLabelText('参考UI.png')).toBeTruthy();
fireEvent.click(
within(panel).getByRole('button', { name: '上传UI素材参考图' }),
);
fireEvent.click(
within(panel).getByRole('button', { name: '删除参考UI.png' }),
);
expect(onRequestUpload).toHaveBeenCalledTimes(1);
expect(onRemoveReference).toHaveBeenCalledWith('ui-ref-1');
expect(
within(panel).queryByRole('button', { name: '取消框选' }),
).toBeNull();
@@ -3,6 +3,8 @@ import {
ChevronDown,
Circle,
Cpu,
ImageIcon,
ImagePlus,
Paintbrush,
Square,
} from 'lucide-react';
@@ -21,6 +23,7 @@ import {
EDITOR_IMAGE_MODEL_OPTIONS,
getEditorImageModelDisplayName,
} from './ImageCanvasGenerationModel';
import { ImageCanvasReferenceSlot } from './ImageCanvasReferenceSlot';
import type {
UiAssetExtractionMark,
UiAssetExtractionState,
@@ -42,6 +45,8 @@ type ImageCanvasUiAssetExtractionOverlayViewProps = {
onPointerMove: (point: { x: number; y: number }) => void;
onPointerEnd: () => void;
onModelChange?: (model: string) => void;
onRequestUpload?: () => void;
onRemoveReference?: (referenceId: string) => void;
onSubmit?: () => void;
};
@@ -267,6 +272,8 @@ export function ImageCanvasUiAssetExtractionOverlayView({
onPointerMove,
onPointerEnd,
onModelChange = () => {},
onRequestUpload,
onRemoveReference,
onSubmit,
}: ImageCanvasUiAssetExtractionOverlayViewProps) {
const [isModelMenuOpen, setIsModelMenuOpen] = useState(false);
@@ -391,6 +398,38 @@ export function ImageCanvasUiAssetExtractionOverlayView({
<div className="image-canvas-editor__ui-extraction-panel-head">
使
</div>
<div className="image-canvas-editor__reference-strip">
{state.references.map((reference, index) => (
<ImageCanvasReferenceSlot
key={reference.id}
tone="default"
icon={<ImageIcon className="h-4 w-4" aria-hidden="true" />}
imageSrc={reference.src}
objectKey={reference.objectKey}
label={`参考图${index + 1}`}
ariaLabel={reference.label || `参考图${index + 1}`}
title={reference.label}
disabled={state.status === 'extracting'}
onRemove={
onRemoveReference
? () => onRemoveReference(reference.id)
: undefined
}
removeLabel={`删除${reference.label || `参考图${index + 1}`}`}
/>
))}
{onRequestUpload ? (
<ImageCanvasReferenceSlot
tone="default"
icon={<ImagePlus className="h-4 w-4" aria-hidden="true" />}
label="参考图"
ariaLabel="上传UI素材参考图"
disabled={state.status === 'extracting'}
isAdd
onClick={onRequestUpload}
/>
) : null}
</div>
<div
className="image-canvas-editor__ui-extraction-preview-list"
aria-label="预览列表"
@@ -109,7 +109,11 @@ describe('ImageCanvasUploadModel', () => {
resolveUploadFolderId({
assetFolders: [
...folders,
createFolder({ id: 'uploaded', label: '上传素材', systemDefault: false }),
createFolder({
id: 'uploaded',
label: '上传素材',
systemDefault: false,
}),
],
activeUploadFolderId: 'characters',
preferUploadFolder: true,
@@ -157,7 +161,11 @@ describe('ImageCanvasUploadModel', () => {
expandUploadFolder({
folders: [
createFolder({ id: 'project', collapsed: true }),
createFolder({ id: 'characters', label: '角色素材', collapsed: true }),
createFolder({
id: 'characters',
label: '角色素材',
collapsed: true,
}),
],
folderId: 'characters',
}),
@@ -429,6 +437,37 @@ describe('ImageCanvasUploadModel', () => {
).toBe(iconDialog);
});
it('appends generation reference uploads to icon and UI design dialogs', () => {
const reference = {
id: 'ref-new',
label: '参考图',
src: 'data:image/png;base64,ref',
};
expect(
applyGenerationReferenceUpload({
dialog: createDialog({ mode: 'icon' }),
target: 'generation-reference',
references: [reference],
}),
).toMatchObject({
mode: 'icon',
status: 'idle',
generationReferences: [reference],
});
expect(
applyGenerationReferenceUpload({
dialog: createDialog({ mode: 'ui-design' }),
target: 'generation-reference',
references: [reference],
}),
).toMatchObject({
mode: 'ui-design',
status: 'idle',
generationReferences: [reference],
});
});
it('creates upload canvas layers centered on the target canvas point', () => {
const layer = createUploadCanvasLayer({
uploadIndex: 1,
@@ -459,7 +498,8 @@ describe('ImageCanvasUploadModel', () => {
uploadIndex: 1,
fileName: '画布素材.png',
imageSrc: 'data:image/png;base64,canvas',
objectKey: 'generated-character-drafts/editor/asset-library/image/画布素材.png',
objectKey:
'generated-character-drafts/editor/asset-library/image/画布素材.png',
assetObjectId: 'assetobj-image',
imageDimensions: { width: 1536, height: 1024 },
canvasPoint: { x: 110, y: 120 },
@@ -207,7 +207,9 @@ export function applyGenerationReferenceUpload({
if (target === 'generation-reference') {
return dialog?.mode === 'generate' ||
dialog?.mode === 'video' ||
dialog?.mode === 'spec'
dialog?.mode === 'spec' ||
dialog?.mode === 'icon' ||
dialog?.mode === 'ui-design'
? {
...setFailedGenerationIdle(dialog),
generationReferences: [
@@ -317,7 +319,13 @@ export function createUploadingAssetPlaceholder({
: fallbackSize;
return {
id: `upload-${uploadIndex}`,
label: fileName || (mediaType === 'audio' ? '上传音频' : mediaType === 'video' ? '上传视频' : '上传图片'),
label:
fileName ||
(mediaType === 'audio'
? '上传音频'
: mediaType === 'video'
? '上传视频'
: '上传图片'),
src: '',
mediaType,
width: size.width,
@@ -430,7 +438,8 @@ export function applyPersistedUploadAsset({
height: persistedAsset.height,
objectKey: persistedAsset.objectKey ?? undefined,
assetObjectId: persistedAsset.assetObjectId ?? undefined,
durationSeconds: persistedAsset.durationSeconds ?? asset.durationSeconds,
durationSeconds:
persistedAsset.durationSeconds ?? asset.durationSeconds,
persisted: true,
uploadStatus: undefined,
uploadProgress: undefined,
@@ -567,7 +576,13 @@ export function createUploadCanvasLayer({
return {
id: `layer-upload-${uploadIndex}`,
resourceId: `local-resource-upload-${uploadIndex}`,
title: fileName || (mediaType === 'audio' ? '上传音频' : mediaType === 'video' ? '上传视频' : '上传图片'),
title:
fileName ||
(mediaType === 'audio'
? '上传音频'
: mediaType === 'video'
? '上传视频'
: '上传图片'),
src: imageSrc,
mediaType,
assetKind:
@@ -606,8 +621,7 @@ export function resizeUploadCanvasLayerToImage({
canvasSize: CanvasSize;
viewport: CanvasViewport;
}): CanvasLayer {
const safeOriginalWidth =
originalWidth || UPLOAD_LAYER_FALLBACK_SIZE.width;
const safeOriginalWidth = originalWidth || UPLOAD_LAYER_FALLBACK_SIZE.width;
const safeOriginalHeight =
originalHeight || UPLOAD_LAYER_FALLBACK_SIZE.height;
const { width, height } = resolveLayerResolutionSize(
@@ -373,6 +373,36 @@ function SubmissionWorkflowHarness({
>
UI素材
</button>
<button
type="button"
onClick={() => {
if (sourceLayer) {
void workflow.extractUiDesignAssets(sourceLayer, {
marks: [
{
id: 'mark-ref',
tool: 'rect',
x: 10,
y: 20,
width: 80,
height: 60,
},
],
references: [
{
id: 'ref-a',
label: '风格参考',
src: '/generated/ref-fallback.png',
objectKey: 'generated/ref-style.png',
resourceId: 'resource-style-ref',
},
],
});
}
}}
>
UI素材
</button>
<button
type="button"
onClick={() => void workflow.submitCharacterAnimation()}
@@ -612,6 +642,7 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
height: 120,
},
],
references: [],
draftMark: null,
status: 'idle',
}}
@@ -1566,10 +1597,7 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
);
});
it('resolves uploaded objectKey icon spec before submitting spritesheets', async () => {
resolveEditorImageReferenceDataUrlForGenerationMock.mockResolvedValueOnce(
'data:image/jpeg;base64,normalized-icon-spec',
);
it('submits uploaded objectKey icon references before submitting spritesheets', async () => {
generateEditorIconSpritesheetMock.mockResolvedValueOnce({
spritesheetImageSrc: 'data:image/png;base64,sheet',
spritesheetWidth: 512,
@@ -1597,6 +1625,15 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
objectKey: 'generated-character-drafts/editor/specs/icon-spec.png',
assetObjectId: 'assetobj-icon-spec',
},
generationReferences: [
{
id: 'ref-style',
label: '风格参考',
src: 'https://signed.example.test/icon-style.png?expires=1',
objectKey: 'generated-character-drafts/editor/refs/icon-style.png',
assetObjectId: 'assetobj-icon-style',
},
],
}}
/>,
);
@@ -1605,14 +1642,14 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
fireEvent.click(screen.getByRole('button', { name: '提交图标' }));
await waitFor(() => {
expect(
resolveEditorImageReferenceDataUrlForGenerationMock,
).toHaveBeenCalledWith(
'generated-character-drafts/editor/specs/icon-spec.png',
);
expect(resolveEditorImageReferenceDataUrlForGenerationMock).not.toHaveBeenCalled();
expect(generateEditorIconSpritesheetMock).toHaveBeenCalledWith(
expect.objectContaining({
referenceImageSrc: 'data:image/jpeg;base64,normalized-icon-spec',
referenceImageSrc:
'generated-character-drafts/editor/specs/icon-spec.png',
referenceImageSrcs: [
'generated-character-drafts/editor/refs/icon-style.png',
],
iconDescriptions: ['返回按钮'],
}),
);
@@ -1648,19 +1685,17 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
fireEvent.click(screen.getByRole('button', { name: '提取UI素材' }));
await waitFor(() => {
expect(resolveEditorImageReferenceDataUrlMock).toHaveBeenCalledWith(
'generated/ui-design.png',
);
});
await waitFor(() => {
expect(extractEditorUiDesignAssetsMock).toHaveBeenCalledWith(
expect.objectContaining({
sourceImageSrc: 'data:image/png;base64,ui-design.png',
sourceImageSrc: 'generated/ui-design.png',
model: 'gemini-3.1-flash-image-preview',
}),
);
});
expect(resolveEditorImageReferenceDataUrlMock).not.toHaveBeenCalledWith(
'generated/ui-design.png',
);
await waitFor(() => {
expect(screen.getByTestId('layers').textContent).toContain(
'layer-icon-spritesheet-1:源图 素材图集:-:icon-spritesheet',
@@ -1785,6 +1820,53 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
});
});
it('submits UI asset extraction with uploaded references', async () => {
extractEditorUiDesignAssetsMock.mockResolvedValueOnce({
spritesheetImageSrc: 'data:image/png;base64,ui-sheet',
spritesheetWidth: 512,
spritesheetHeight: 512,
prompt:
'仅提取被红色框框选的素材并整理成spritesheet,图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕,方便后续扣除背景;素材自身不要出现绿色描边、绿色底板或绿色阴影。',
actualPrompt:
'仅提取被红色框框选的素材并整理成spritesheet,图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕,方便后续扣除背景;素材自身不要出现绿色描边、绿色底板或绿色阴影。',
model: 'gpt-image-2',
provider: 'VectorEngine',
taskId: 'task-ui-assets',
});
render(
<SubmissionWorkflowHarness
initialLayers={[
createLayer({
assetKind: 'ui-design',
objectKey: 'generated/ui-design.png',
}),
]}
/>,
);
fireEvent.click(screen.getByRole('button', { name: '提取带参考图UI素材' }));
await waitFor(() => {
expect(resolveEditorImageReferenceDataUrlForGenerationMock).not.toHaveBeenCalled();
});
await waitFor(() => {
expect(extractEditorUiDesignAssetsMock).toHaveBeenCalledWith(
expect.objectContaining({
referenceImageSrcs: ['generated/ref-style.png'],
generationInputs: expect.objectContaining({
references: expect.arrayContaining([
expect.objectContaining({
title: '参考图 1',
label: '风格参考',
refType: 'project-resource',
}),
]),
}),
}),
);
});
});
it('keeps character generation dialog data after the result is created', async () => {
generateEditorImageMock.mockResolvedValueOnce(
createGenerated({ width: 768, height: 768 }),

Some files were not shown because too many files have changed in this diff Show More