接入创作图片上传宿主导入

CreativeImageInputPanel 在原生壳优先使用 file.importImage

宿主图片导入结果转换为现有 File 上传回调

浏览器和小程序继续保留原文件输入路径

更新测试和原生壳架构文档
This commit is contained in:
2026-06-18 03:19:00 +08:00
parent 14f838c414
commit d5cab219a1
5 changed files with 332 additions and 49 deletions
@@ -34,6 +34,7 @@
- 2026-06-18 原生壳网络状态:新增 `network.status``network.statusChanged` HostBridge capabilityExpo 壳通过 `expo-network` 查询和订阅真实系统网络状态,Tauri 壳通过短超时连接 `app.genarrative.world:443` 查询主站可达性,并通过 WebView `online` / `offline` 注入变化事件;H5 统一使用 `getHostNetworkStatus()` / `subscribeHostNetworkStatusChange()`,不得直接读取 Expo / Tauri 私有网络 API。
- 2026-06-18 桌面图片导入:新增 `file.importImage``file.imageDropped` HostBridge capabilityTauri 壳通过系统文件选择框和主窗口拖拽事件读取用户选择 / 拖入的真实图片,只允许 `image/png``image/jpeg``image/webp` 且单次不超过 10 MiB;H5 统一使用 `importHostImageFile()` / `subscribeHostImageDrop()`,宿主只回传文件名、MIME、base64 内容、字节数和可选坐标,不暴露本地绝对路径,也不开放通用文件系统。
- 2026-06-18 移动图片导入:Expo 壳开始声明并实现 `file.importImage`,通过 `expo-image-picker` 请求相册权限并打开系统相册选择器,只允许 `image/png``image/jpeg``image/webp` 且单次不超过 10 MiB;成功只回传清洗后的文件名、MIME、base64 内容和字节数,不暴露设备本地 URI,不请求相机或麦克风权限,用户取消返回 `cancelled` 并由 H5 facade 归为 `false`
- 2026-06-18 H5 图片上传接入宿主导入:`CreativeImageInputPanel``native_app` 且声明 `file.importImage` 时,主图上传和描述参考图上传优先调用 `importHostImageFile()`,并把宿主返回的 base64 图片转换为现有 `File` 回调;浏览器、小程序和未声明能力的裁剪壳继续走原生 `<input type="file">` 路径,不新增玩法侧上传分叉。
- 影响范围:`src/services/host-bridge/`、未来 `apps/mobile-shell/`、未来 `apps/desktop-shell/`、移动端支付 / 分享 / 深链 / 推送、桌面端系统能力、AI H5 sandbox 的 GameBridge 边界。
- 验证方式:普通浏览器、小程序、Expo 壳、Tauri 壳都能返回正确 `getHostRuntime()`;未支持能力能回退 H5;固定玩法在各宿主中读取同一作品数据和运行态 snapshot;AI sandbox 无法直接调用 HostBridgeTauri release 不允许任意远端页面调用桌面命令。
- 关联文档:`docs/【前端架构】ExpoReactNative与Tauri宿主壳方案-2026-06-17.md``docs/【前端架构】宿主壳能力统一协议-2026-06-17.md`
File diff suppressed because one or more lines are too long
@@ -1,6 +1,6 @@
# 宿主壳能力统一协议
更新时间:`2026-06-17`
更新时间:`2026-06-18`
## 背景
@@ -56,7 +56,7 @@ AI H5 sandbox
- `navigateHostNativePage()`:受控跳转宿主页,供订阅授权、支付、登录等 adapter 复用。Expo 移动壳首版只接受同源 H5 route 并切换 WebView URLTauri 桌面壳同样只接受 `https://app.genarrative.world` 同源 H5 route 并在主窗口内跳转。真正原生页面、登录和支付能力必须等对应 SDK / 页面接入后再声明支持。
- `exportHostTextFile()`:原生 App 宿主的受控文本导出入口。Expo 移动壳通过 `file.exportText` 写入缓存文本文件并交给系统分享 / 保存面板;Tauri 桌面壳通过 `file.exportText` 打开系统保存对话框并写入用户选择的文件。文件名必须清洗,单次文本不超过 5 MiB,成功只返回文件名和字节数,不把本机绝对路径暴露给 H5;系统分享不可用或用户取消时返回明确错误,由 H5 fallback 承接。
- `exportHostImageFile()`:原生 App 宿主的受控图片导出入口。H5 只传自己生成的图片 `base64Data`、清洗后的文件名和允许的 `image/png` / `image/jpeg` / `image/webp` MIME;Expo 移动壳写入缓存图片后交给系统分享 / 保存面板,Tauri 桌面壳打开系统保存对话框并写入图片字节。单次图片不超过 5 MiB,成功只返回文件名和字节数,不回传本机绝对路径。当前分享卡下载在 native app 中优先走 `file.exportImage`,宿主未声明时保留浏览器下载路径。
- `importHostImageFile()` / `subscribeHostImageDrop()`:原生 App 宿主的受控图片导入入口。Expo 移动壳通过 Expo ImagePicker 请求相册权限并打开系统相册选择器,Tauri 壳通过系统文件选择框或主窗口拖拽事件读取用户选择 / 拖入的图片;两端都只接受 `image/png``image/jpeg``image/webp`,单次不超过 10 MiB,成功只返回文件名、MIME、base64 内容、字节数和可选拖入坐标,不暴露设备本地 URI 或本机绝对路径,也不开放通用文件系统能力。
- `importHostImageFile()` / `subscribeHostImageDrop()`:原生 App 宿主的受控图片导入入口。Expo 移动壳通过 Expo ImagePicker 请求相册权限并打开系统相册选择器,Tauri 壳通过系统文件选择框或主窗口拖拽事件读取用户选择 / 拖入的图片;两端都只接受 `image/png``image/jpeg``image/webp`,单次不超过 10 MiB,成功只返回文件名、MIME、base64 内容、字节数和可选拖入坐标,不暴露设备本地 URI 或本机绝对路径,也不开放通用文件系统能力。H5 的通用图片输入面板 `CreativeImageInputPanel``native_app` 且声明 `file.importImage` 时优先调用宿主导入,并把结果转换成现有 `File` 回调;普通浏览器、小程序和未声明能力的裁剪壳继续使用浏览器文件输入。
## 迁移顺序
@@ -1,10 +1,25 @@
/* @vitest-environment jsdom */
import { fireEvent, render, screen, within } from '@testing-library/react';
import { expect, test, vi } from 'vitest';
import {
fireEvent,
render,
screen,
waitFor,
within,
} from '@testing-library/react';
import { afterEach, expect, test, vi } from 'vitest';
import {
canUseNativeHostCapability,
importHostImageFile,
} from '../../services/host-bridge/hostBridge';
import { CreativeImageInputPanel } from './CreativeImageInputPanel';
vi.mock('../../services/host-bridge/hostBridge', () => ({
canUseNativeHostCapability: vi.fn(() => false),
importHostImageFile: vi.fn(),
}));
vi.mock('../ResolvedAssetImage', () => ({
ResolvedAssetImage: ({
src,
@@ -17,6 +32,14 @@ vi.mock('../ResolvedAssetImage', () => ({
}) => (src ? <img src={src} alt={alt} className={className} /> : null),
}));
const canUseNativeHostCapabilityMock = vi.mocked(canUseNativeHostCapability);
const importHostImageFileMock = vi.mocked(importHostImageFile);
afterEach(() => {
vi.clearAllMocks();
canUseNativeHostCapabilityMock.mockReturnValue(false);
});
test('creative image input panel handles reference uploads and preview', () => {
const onPromptReferenceFilesSelect = vi.fn();
const onPromptReferenceRemove = vi.fn();
@@ -711,3 +734,170 @@ test('creative image input panel can upload prompt references while showing a ma
screen.getByRole('button', { name: '预览参考图 描述参考图 1' }),
).toBeTruthy();
});
test('creative image input panel imports the main image from native host capability', async () => {
const onMainImageFileSelect = vi.fn();
const inputClickSpy = vi
.spyOn(HTMLInputElement.prototype, 'click')
.mockImplementation(() => undefined);
canUseNativeHostCapabilityMock.mockReturnValue(true);
importHostImageFileMock.mockResolvedValue({
action: 'selected',
fileName: '参考图.png',
mimeType: 'image/png',
base64Data: 'aW1hZ2U=',
bytes: 5,
});
try {
render(
<CreativeImageInputPanel
uploadedImageSrc=""
uploadedImageAlt="拼图图片"
mainImageInputId="native-main-image-upload-input"
promptTextareaId="native-image-prompt-input"
prompt="旧街灯牌下的猫。"
promptLabel="画面描述"
aiRedraw
promptReferenceImages={[]}
imageModelPicker={null}
submitLabel="生成"
submitDisabled={false}
labels={{
imageField: '拼图画面',
uploadImage: '上传拼图图片',
replaceImage: '更换拼图图片',
emptyImageHint: '上传图片/填写画面描述',
removeImage: '移除拼图图片',
removeImageConfirmTitle: '移除拼图图片?',
removeImageConfirmBody: '移除后需要重新上传图片。',
promptReferenceUpload: '上传参考图',
promptReferencePreviewAlt: '参考图预览',
closePromptReferencePreview: '关闭参考图预览',
}}
onMainImageFileSelect={onMainImageFileSelect}
onMainImageRemove={() => {}}
onAiRedrawChange={() => {}}
onPromptChange={() => {}}
onSubmit={() => {}}
/>,
);
fireEvent.click(screen.getByRole('button', { name: '上传拼图图片' }));
await waitFor(() => {
expect(onMainImageFileSelect).toHaveBeenCalledWith(expect.any(File));
});
const selectedFile = onMainImageFileSelect.mock.calls[0]?.[0] as File;
expect(selectedFile.name).toBe('参考图.png');
expect(selectedFile.type).toBe('image/png');
expect(inputClickSpy).not.toHaveBeenCalled();
} finally {
inputClickSpy.mockRestore();
}
});
test('creative image input panel imports prompt reference image from native host capability', async () => {
const onPromptReferenceFilesSelect = vi.fn();
canUseNativeHostCapabilityMock.mockReturnValue(true);
importHostImageFileMock.mockResolvedValue({
action: 'selected',
fileName: '描述参考.webp',
mimeType: 'image/webp',
base64Data: 'cmVm',
bytes: 3,
});
render(
<CreativeImageInputPanel
uploadedImageSrc=""
uploadedImageAlt="拼图图片"
mainImageInputId="native-reference-main-image-upload-input"
promptTextareaId="native-reference-prompt-input"
prompt="旧街灯牌下的猫。"
promptLabel="画面描述"
aiRedraw
promptReferenceImages={[]}
imageModelPicker={null}
submitLabel="生成"
submitDisabled={false}
labels={{
imageField: '拼图画面',
uploadImage: '上传拼图图片',
replaceImage: '更换拼图图片',
emptyImageHint: '上传图片/填写画面描述',
removeImage: '移除拼图图片',
removeImageConfirmTitle: '移除拼图图片?',
removeImageConfirmBody: '移除后需要重新上传图片。',
promptReferenceUpload: '上传参考图',
promptReferencePreviewAlt: '参考图预览',
closePromptReferencePreview: '关闭参考图预览',
}}
onMainImageFileSelect={() => {}}
onMainImageRemove={() => {}}
onAiRedrawChange={() => {}}
onPromptChange={() => {}}
onPromptReferenceFilesSelect={onPromptReferenceFilesSelect}
onSubmit={() => {}}
/>,
);
fireEvent.click(screen.getByRole('button', { name: '上传参考图' }));
await waitFor(() => {
expect(onPromptReferenceFilesSelect).toHaveBeenCalledWith([
expect.any(File),
]);
});
const selectedFile = onPromptReferenceFilesSelect.mock.calls[0]?.[0]?.[0] as
| File
| undefined;
expect(selectedFile?.name).toBe('描述参考.webp');
expect(selectedFile?.type).toBe('image/webp');
});
test('creative image input panel keeps callbacks quiet when native image import is cancelled', async () => {
const onMainImageFileSelect = vi.fn();
canUseNativeHostCapabilityMock.mockReturnValue(true);
importHostImageFileMock.mockResolvedValue(false);
render(
<CreativeImageInputPanel
uploadedImageSrc=""
uploadedImageAlt="拼图图片"
mainImageInputId="cancelled-native-image-upload-input"
promptTextareaId="cancelled-native-prompt-input"
prompt="旧街灯牌下的猫。"
promptLabel="画面描述"
aiRedraw
promptReferenceImages={[]}
imageModelPicker={null}
submitLabel="生成"
submitDisabled={false}
labels={{
imageField: '拼图画面',
uploadImage: '上传拼图图片',
replaceImage: '更换拼图图片',
emptyImageHint: '上传图片/填写画面描述',
removeImage: '移除拼图图片',
removeImageConfirmTitle: '移除拼图图片?',
removeImageConfirmBody: '移除后需要重新上传图片。',
promptReferenceUpload: '上传参考图',
promptReferencePreviewAlt: '参考图预览',
closePromptReferencePreview: '关闭参考图预览',
}}
onMainImageFileSelect={onMainImageFileSelect}
onMainImageRemove={() => {}}
onAiRedrawChange={() => {}}
onPromptChange={() => {}}
onSubmit={() => {}}
/>,
);
fireEvent.click(screen.getByRole('button', { name: '上传拼图图片' }));
await waitFor(() => {
expect(importHostImageFileMock).toHaveBeenCalledTimes(1);
});
expect(onMainImageFileSelect).not.toHaveBeenCalled();
});
+134 -42
View File
@@ -1,6 +1,12 @@
import { History, ImagePlus, Loader2, Sparkles, Trash2 } from 'lucide-react';
import { type ReactNode, useEffect, useRef, useState } from 'react';
import {
canUseNativeHostCapability,
type HostFileImportImageResult,
importHostImageFile,
} from '../../services/host-bridge/hostBridge';
import { puzzleReferenceImageDataUrlToFile } from '../../services/puzzleReferenceImage';
import { ResolvedAssetImage } from '../ResolvedAssetImage';
import { PlatformActionButton } from './PlatformActionButton';
import { PlatformFieldLabel } from './PlatformFieldLabel';
@@ -86,6 +92,13 @@ export type CreativeImageInputPanelProps = {
const DEFAULT_IMAGE_ACCEPT = 'image/png,image/jpeg,image/webp';
const DEFAULT_PROMPT_REFERENCE_LIMIT = 5;
function hostImageImportResultToFile(result: HostFileImportImageResult) {
return puzzleReferenceImageDataUrlToFile(
`data:${result.mimeType};base64,${result.base64Data}`,
result.fileName,
);
}
export function CreativeImageInputPanel({
className = '',
fillHeight = true,
@@ -132,6 +145,7 @@ export function CreativeImageInputPanel({
onSubmit,
}: CreativeImageInputPanelProps) {
const mainImageInputRef = useRef<HTMLInputElement | null>(null);
const promptReferenceInputRef = useRef<HTMLInputElement | null>(null);
const [previewReferenceImage, setPreviewReferenceImage] =
useState<CreativeImageInputReferenceImage | null>(null);
const [isMainImagePreviewOpen, setIsMainImagePreviewOpen] = useState(false);
@@ -151,6 +165,8 @@ export function CreativeImageInputPanel({
mainImageClickMode === 'preview' && Boolean(uploadedImageSrc);
const shouldShowMainImageUploadButton =
isMainImageUploadEnabled && shouldPreviewMainImage;
const canImportHostImage = canUseNativeHostCapability('file.importImage');
const promptReferenceInputId = `${mainImageInputId}-prompt-reference`;
useEffect(() => {
if (uploadedImageSrc) {
@@ -188,6 +204,60 @@ export function CreativeImageInputPanel({
? 'creative-image-input-panel__image-card puzzle-image-upload-card relative aspect-square h-full min-h-[14rem] max-h-full max-w-full overflow-hidden rounded-[1.25rem] border border-[var(--platform-subpanel-border)] bg-white/90 shadow-[0_12px_28px_rgba(15,23,42,0.08)] transition sm:min-h-[18rem] lg:h-auto lg:w-full'
: 'creative-image-input-panel__image-card puzzle-image-upload-card relative aspect-square w-full min-h-[14rem] max-w-full overflow-hidden rounded-[1.25rem] border border-[var(--platform-subpanel-border)] bg-white/90 shadow-[0_12px_28px_rgba(15,23,42,0.08)] transition sm:min-h-[18rem]';
const importHostImageAsFile = async () => {
const result = await importHostImageFile();
if (!result) {
return null;
}
return hostImageImportResultToFile(result);
};
const handleMainImageUploadClick = () => {
if (disabled || !isMainImageUploadEnabled) {
return;
}
if (!canImportHostImage) {
mainImageInputRef.current?.click();
return;
}
void (async () => {
try {
const file = await importHostImageAsFile();
if (file) {
onMainImageFileSelect(file);
}
} catch {
// 中文注释:宿主导入失败时不再弹浏览器文件框,避免权限失败后重复打扰用户。
}
})();
};
const handlePromptReferenceUploadClick = () => {
if (
promptReferenceUploadDisabled ||
!shouldShowPromptReferences ||
!onPromptReferenceFilesSelect
) {
return;
}
if (!canImportHostImage) {
promptReferenceInputRef.current?.click();
return;
}
void (async () => {
try {
const file = await importHostImageAsFile();
if (file) {
onPromptReferenceFilesSelect([file]);
}
} catch {
// 中文注释:宿主导入失败时保持当前表单状态,由外层错误通道继续承接后续重试。
}
})();
};
return (
<div
className={`creative-image-input-panel flex min-h-0 flex-col ${
@@ -240,21 +310,28 @@ export function CreativeImageInputPanel({
onClick={() => setIsMainImagePreviewOpen(true)}
/>
) : isMainImageUploadEnabled ? (
<label
htmlFor={mainImageInputId}
className={`absolute inset-0 z-0 cursor-pointer ${disabled ? 'cursor-not-allowed' : ''}`}
<button
type="button"
className={`absolute inset-0 z-0 border-0 bg-transparent p-0 ${disabled ? 'cursor-not-allowed' : 'cursor-pointer'}`}
disabled={disabled}
aria-label={
uploadedImageSrc
? labels.replaceImage
: labels.uploadImage
}
title={
uploadedImageSrc
? labels.replaceImage
: labels.uploadImage
}
onClick={handleMainImageUploadClick}
>
<span className="sr-only">
{uploadedImageSrc
? labels.replaceImage
: labels.uploadImage}
</span>
</label>
</button>
) : null}
{uploadedImageSrc ? (
<ResolvedAssetImage
@@ -280,7 +357,7 @@ export function CreativeImageInputPanel({
label={labels.replaceImage}
title={labels.replaceImage}
disabled={disabled}
onClick={() => mainImageInputRef.current?.click()}
onClick={handleMainImageUploadClick}
icon={<ImagePlus className="h-4 w-4" />}
className="absolute bottom-3 right-3 z-10 h-10 w-10"
/>
@@ -323,16 +400,18 @@ export function CreativeImageInputPanel({
className="absolute left-3 top-3 z-10 h-10 w-10"
/>
) : isMainImageUploadEnabled && !uploadedImageSrc ? (
<label
htmlFor={mainImageInputId}
className={`absolute bottom-9 left-1/2 z-10 -translate-x-1/2 whitespace-nowrap text-center text-sm font-black text-[var(--platform-text-strong)] drop-shadow-[0_1px_0_rgba(255,255,255,0.82)] transition hover:text-[var(--platform-accent)] sm:bottom-10 ${
<button
type="button"
disabled={disabled}
onClick={handleMainImageUploadClick}
className={`absolute bottom-9 left-1/2 z-10 -translate-x-1/2 whitespace-nowrap border-0 bg-transparent p-0 text-center text-sm font-black text-[var(--platform-text-strong)] drop-shadow-[0_1px_0_rgba(255,255,255,0.82)] transition hover:text-[var(--platform-accent)] sm:bottom-10 ${
disabled
? 'cursor-not-allowed opacity-55'
: 'cursor-pointer'
}`}
>
{labels.emptyImageHint}
</label>
</button>
) : null}
</div>
</div>
@@ -370,39 +449,52 @@ export function CreativeImageInputPanel({
{imageModelPicker}
{shouldShowPromptReferences &&
onPromptReferenceFilesSelect ? (
<PlatformIconButton
asChild="label"
variant="surfaceFloating"
label={labels.promptReferenceUpload}
title={labels.promptReferenceUpload}
icon={
<>
<ImagePlus className="h-4 w-4" />
<input
type="file"
accept={mainImageAccept}
multiple
aria-label={labels.promptReferenceUpload}
disabled={promptReferenceUploadDisabled}
onChange={(event) => {
const files = Array.from(
event.currentTarget.files ?? [],
);
event.currentTarget.value = '';
if (files.length > 0) {
onPromptReferenceFilesSelect(files);
}
}}
className="sr-only"
/>
</>
}
className={`absolute bottom-3 right-3 z-10 h-8 w-8 border-[var(--platform-subpanel-border)] bg-white/96 hover:bg-[var(--platform-subpanel-fill)] ${
promptReferenceUploadDisabled
? 'cursor-not-allowed opacity-55'
: 'cursor-pointer'
}`}
/>
<>
<input
ref={promptReferenceInputRef}
id={promptReferenceInputId}
type="file"
accept={mainImageAccept}
multiple
aria-label={labels.promptReferenceUpload}
disabled={promptReferenceUploadDisabled}
onChange={(event) => {
const files = Array.from(
event.currentTarget.files ?? [],
);
event.currentTarget.value = '';
if (files.length > 0) {
onPromptReferenceFilesSelect(files);
}
}}
className="sr-only"
/>
{canImportHostImage ? (
<PlatformIconButton
variant="surfaceFloating"
label={labels.promptReferenceUpload}
title={labels.promptReferenceUpload}
disabled={promptReferenceUploadDisabled}
onClick={handlePromptReferenceUploadClick}
icon={<ImagePlus className="h-4 w-4" />}
className="absolute bottom-3 right-3 z-10 h-8 w-8 border-[var(--platform-subpanel-border)] bg-white/96 hover:bg-[var(--platform-subpanel-fill)]"
/>
) : (
<PlatformIconButton
asChild="label"
htmlFor={promptReferenceInputId}
variant="surfaceFloating"
label={labels.promptReferenceUpload}
title={labels.promptReferenceUpload}
icon={<ImagePlus className="h-4 w-4" />}
className={`absolute bottom-3 right-3 z-10 h-8 w-8 border-[var(--platform-subpanel-border)] bg-white/96 hover:bg-[var(--platform-subpanel-fill)] ${
promptReferenceUploadDisabled
? 'cursor-not-allowed opacity-55'
: 'cursor-pointer'
}`}
/>
)}
</>
) : null}
</div>
{shouldShowPromptReferences &&