Editor agent 右键菜单支持复制, 引用,下载 #95

Merged
kdletters merged 20 commits from editor-agent-right-click into master 2026-07-22 18:00:27 +08:00
12 changed files with 2696 additions and 429 deletions
@@ -1,6 +1,6 @@
# 画布Agent对话面板
日期:`2026-07-16`
日期:`2026-07-20`
## 定位与边界
@@ -74,6 +74,7 @@
- 附件选择弹窗使用 `PlatformToolModalShell` 承接 portal 主题变量和不透明 panel 背景;不能直接把未注入 `platform-theme``UnifiedModal` portal 到 `document.body`,否则 `--platform-modal-fill` 失效后面板会变透明。
- 应用后附件以胶囊 chip 挂在输入框上方;发出的消息内附件渲染为纯文本胶囊 chip(名称 + 小图标),**默认无缩略图,鼠标悬浮才浮出缩略图预览**。
- 附件领域形状:统一为画布资源 / 素材库对象引用(`resourceId` / `assetId` + 可选 `objectKey`),不存在只属于对话的第三种图;单条消息上限 9 张(前后端共同校验)。前端可携带展示用 `imageSrc` / `thumbnailSrc`,后端必须按当前工程和当前账号重新归一、校验归属与 `objectKey`
- 输入区附件临时状态统一收口到 `useConversationAttachments`,选择弹窗由独立的 `AttachmentPicker` 负责纯展示;选择、引用、粘贴上传完成、移除、发送清空和失败恢复都必须经同一最新状态更新入口。引用历史消息附件时先保留消息中的展示快照,最终发送前再按 `source + referenceId` 从当前画布和素材库选项刷新,避免提前刷新后又被失败恢复的旧快照覆盖。发送失败时,已发送附件必须与等待期间新增的附件去重合并,不得因输入区已非空而丢弃;同一 `source + referenceId` 冲突时保留等待期间的当前草稿快照,失败请求快照只补充缺失 identity。合并后超过 9 张时优先保留等待期间的最新附件,不恢复失败请求的附件,并立即显示上限错误。异步粘贴完成时基于当时的最新附件去重并重新校验 9 张上限,不能用上传开始时捕获的旧列表覆盖期间新增的引用。
## 工具调用确认展示契约
@@ -116,11 +117,13 @@
5. 生成中的进行中动画;
6. 错误气泡(失败/余额不足,带原因);
7. 普通消息请求等待期间禁用发送按钮,不提供客户端停止操作;前端持续等待后端响应,超过 120 秒但 POST 仍 pending 时在思考气泡中显示“仍在处理中,请耐心等待”,最终成功或失败后自动移除,避免后端已持久化消息但前端中断请求后产生会话状态错位。
8. 桌面端右键消息正文可复制该条可见文本;右键消息附件或生成结果可下载素材,图片额外支持复制图片本体和“引用”到当前输入区。引用复用附件去重、9 张上限和发送链路;
9. 消息右键菜单遵循 Canva 式单实例交互:任一菜单已打开时,下一次右键必须先关闭旧菜单;新落点是消息正文或素材时再在新位置打开对应菜单,新落点没有右键动作时仅收起旧菜单,不允许多个消息菜单并存。复制、引用或下载成功后自动关闭菜单;失败时保留菜单和失败状态,避免错误无提示消失。
不做(明确排除,防止后人补齐):
- 点赞/点踩反馈按钮;
- 消息复制、分享/导出对话;
- 分享/导出整段对话;
- Agent 模式切换下拉(固定单一 Agent);
- 语音输入、@引用、多 Agent 协作;
- Lovart 的积分/加速档位显示(泥点扣费只在生成动作上体现)。
@@ -158,3 +161,4 @@
- 发送消息时先本地追加用户消息,再应用 JSON 响应中的 `deltaMessages`;请求等待期间发送按钮保持禁用,前端不主动中断当前回合。
- Agent 消息内生成结果缩略图只用于预览,不显示名称,也不点击跳转图层;轮询到任务终态并完成会话懒回填后统一刷新工程快照和素材库。
- 对话内容可被用户选中复制;用户从输入框或对话内容点击回画布图层 / 生成器时,焦点应回到画布对象,Backspace / Delete 等画布快捷键继续生效。
- 对话正文右键菜单只复制当前气泡展示的完整文本,隐藏的内部 system 文本不得进入菜单;素材右键菜单优先于正文菜单,私有素材继续通过既有读取链路换签或代理下载,不复制会过期的临时链接。
@@ -3,16 +3,33 @@ import { Image as ImageIcon, X } from 'lucide-react';
import type { EditorAgentAttachmentRef } from '@/packages/shared/src/contracts';
import { ResolvedAssetImage } from '@/src/components/ResolvedAssetImage.tsx';
import type { RightClickMenuHandler } from './common.ts';
function AttachmentChip({
attachment,
onRemove,
onRightClickMenu,
}: {
attachment: EditorAgentAttachmentRef;
onRemove?: () => void;
onRightClickMenu?: RightClickMenuHandler;
}) {
const label = attachment.label?.trim() || attachment.referenceId;
return (
<span className="group relative inline-flex max-w-full items-center gap-1.5 rounded-full border border-slate-200 bg-white px-2.5 py-1 text-xs text-slate-600 shadow-sm">
<span
className="group relative inline-flex max-w-full items-center gap-1.5 rounded-full border border-slate-200 bg-white px-2.5 py-1 text-xs text-slate-600 shadow-sm"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
...attachment,
kind: 'attachment',
mediaType: 'image',
suggestedFileName: label,
})
: undefined
}
>
<ImageIcon className="h-3.5 w-3.5 shrink-0" aria-hidden="true" />
<span className="truncate">{label}</span>
{onRemove ? (
@@ -0,0 +1,116 @@
import { PlatformActionButton } from '@/src/components/common/PlatformActionButton.tsx';
import { PlatformToolModalShell } from '@/src/components/common/PlatformToolModalShell.tsx';
import { ResolvedAssetImage } from '@/src/components/ResolvedAssetImage.tsx';
import type {
AttachmentPickerTab,
EditorAgentAttachmentOption,
} from './useConversationAttachments.ts';
export function AttachmentPicker({
open,
tab,
canvasOptions,
libraryOptions,
selectedKeys,
attachmentError,
onTabChange,
onToggleKey,
onApply,
onClose,
}: {
open: boolean;
tab: AttachmentPickerTab;
canvasOptions: EditorAgentAttachmentOption[];
libraryOptions: EditorAgentAttachmentOption[];
selectedKeys: Set<string>;
attachmentError: string | null;
onTabChange: (tab: AttachmentPickerTab) => void;
onToggleKey: (key: string) => void;
onApply: () => void;
onClose: () => void;
}) {
const visibleOptions = tab === 'canvas' ? canvasOptions : libraryOptions;
return (
<PlatformToolModalShell
open={open}
title="选择图片附件"
size="md"
onClose={onClose}
footer={
<>
<PlatformActionButton tone="ghost" size="sm" onClick={onClose}>
</PlatformActionButton>
<PlatformActionButton tone="primary" size="sm" onClick={onApply}>
</PlatformActionButton>
</>
}
>
<div className="flex min-h-[18rem] flex-col gap-3">
<div className="flex gap-2">
<button
type="button"
className={`rounded-full px-3 py-1.5 text-sm ${
tab === 'canvas'
? 'bg-slate-900 text-white'
: 'bg-slate-100 text-slate-600'
}`}
onClick={() => onTabChange('canvas')}
>
</button>
<button
type="button"
className={`rounded-full px-3 py-1.5 text-sm ${
tab === 'library'
? 'bg-slate-900 text-white'
: 'bg-slate-100 text-slate-600'
}`}
onClick={() => onTabChange('library')}
>
</button>
</div>
{attachmentError ? (
<div className="rounded-2xl bg-amber-50 px-3 py-2 text-sm text-amber-700">
{attachmentError}
</div>
) : null}
<div className="grid grid-cols-2 gap-2 overflow-y-auto sm:grid-cols-3">
{visibleOptions.map((option) => {
const label =
option.attachment.label?.trim() || option.attachment.referenceId;
return (
<label
key={option.key}
className="flex cursor-pointer flex-col gap-2 rounded-2xl border border-slate-200 bg-white p-2 text-sm text-slate-700 shadow-sm"
>
<ResolvedAssetImage
src={
option.attachment.thumbnailSrc ?? option.attachment.imageSrc
}
objectKey={option.attachment.objectKey}
refreshKey={option.attachment.referenceId}
alt=""
className="aspect-square rounded-xl bg-slate-100 object-cover"
/>
<span className="flex items-center gap-2">
<input
type="checkbox"
aria-label={`选择${option.sourceLabel}图片 ${label}`}
checked={selectedKeys.has(option.key)}
onChange={() => onToggleKey(option.key)}
/>
<span className="truncate">{label}</span>
</span>
</label>
);
})}
</div>
</div>
</PlatformToolModalShell>
);
}
@@ -9,22 +9,15 @@ import {
X,
} from 'lucide-react';
import {
type ClipboardEvent as ReactClipboardEvent,
type FormEvent,
useEffect,
useMemo,
useState,
type WheelEvent as ReactWheelEvent,
} from 'react';
import {
EDITOR_AGENT_MAX_ATTACHMENTS,
type EditorAgentAttachmentRef,
} from '@/packages/shared/src/contracts';
import { PlatformActionButton } from '@/src/components/common/PlatformActionButton.tsx';
import { PlatformDangerConfirmDialog } from '@/src/components/common/PlatformDangerConfirmDialog.tsx';
import { PlatformToolModalShell } from '@/src/components/common/PlatformToolModalShell.tsx';
import AttachmentChip from '@/src/components/image-editor/EditorAgentConversation/AttachmentChip.tsx';
import { AttachmentPicker } from '@/src/components/image-editor/EditorAgentConversation/AttachmentPicker.tsx';
import { attachmentKey } from '@/src/components/image-editor/EditorAgentConversation/common.ts';
import {
MessageBubble,
@@ -34,25 +27,14 @@ import type {
CanvasLayer,
EditorAsset,
} from '@/src/components/image-editor/ImageCanvasEditorTypes.ts';
import { probeImageFileDimensions } from '@/src/components/image-editor/ImageCanvasFileModel.ts';
import { ResolvedAssetImage } from '@/src/components/ResolvedAssetImage.tsx';
import { uploadEditorMediaAssetFile } from '@/src/services/image-editor/editorMediaAssetUploadClient.ts';
import { createEditorProjectResource } from '@/src/services/image-editor/editorProjectClient.ts';
import { useImageCanvasContextStore } from '../useImageCanvasContextStore.ts';
import { useConversationAttachments } from './useConversationAttachments.ts';
import {
type EditorAgentConversationClient,
useEditorAgentConversation,
} from './useEditorAgentConversation';
type AttachmentPickerTab = 'canvas' | 'library';
type EditorAgentAttachmentOption = {
key: string;
sourceLabel: string;
attachment: EditorAgentAttachmentRef;
};
type EditorAgentConversationPanelViewProps = {
open: boolean;
onToggleOpen: () => void;
@@ -68,170 +50,6 @@ function stopAgentPanelWheel(event: ReactWheelEvent<HTMLElement>) {
event.stopPropagation();
}
function isImageLayer(layer: CanvasLayer) {
return (
(layer.mediaType ?? 'image') === 'image' &&
Boolean(layer.resourceId?.trim()) &&
layer.src.trim()
);
}
function isImageAsset(asset: EditorAsset) {
return (asset.mediaType ?? 'image') === 'image' && asset.src.trim();
}
function createCanvasAttachmentOptions(
layers: CanvasLayer[] = [],
): EditorAgentAttachmentOption[] {
return layers.filter(isImageLayer).map((layer) => {
const attachment: EditorAgentAttachmentRef = {
source: 'canvas_resource',
referenceId: layer.resourceId || layer.id,
objectKey: layer.objectKey ?? null,
imageSrc: layer.src,
thumbnailSrc: layer.thumbnailSrc ?? null,
label: layer.title,
width: layer.width,
height: layer.height,
};
return {
key: attachmentKey(attachment),
sourceLabel: '画布',
attachment,
};
});
}
function createLibraryAttachmentOptions(
assets: EditorAsset[] = [],
): EditorAgentAttachmentOption[] {
return assets.filter(isImageAsset).map((asset) => {
const attachment: EditorAgentAttachmentRef = {
source: 'library_asset',
referenceId: asset.id,
objectKey: asset.objectKey ?? null,
imageSrc: asset.src,
thumbnailSrc: asset.thumbnailSrc ?? null,
label: asset.label,
width: asset.width,
height: asset.height,
};
return {
key: attachmentKey(attachment),
sourceLabel: '素材库',
attachment,
};
});
}
function AttachmentPickerModal({
open,
tab,
canvasOptions,
libraryOptions,
selectedKeys,
attachmentError,
onTabChange,
onToggleKey,
onApply,
onClose,
}: {
open: boolean;
tab: AttachmentPickerTab;
canvasOptions: EditorAgentAttachmentOption[];
libraryOptions: EditorAgentAttachmentOption[];
selectedKeys: Set<string>;
attachmentError: string | null;
onTabChange: (tab: AttachmentPickerTab) => void;
onToggleKey: (key: string) => void;
onApply: () => void;
onClose: () => void;
}) {
const visibleOptions = tab === 'canvas' ? canvasOptions : libraryOptions;
return (
<PlatformToolModalShell
open={open}
title="选择图片附件"
size="md"
onClose={onClose}
footer={
<>
<PlatformActionButton tone="ghost" size="sm" onClick={onClose}>
</PlatformActionButton>
<PlatformActionButton tone="primary" size="sm" onClick={onApply}>
</PlatformActionButton>
</>
}
>
<div className="flex min-h-[18rem] flex-col gap-3">
<div className="flex gap-2">
<button
type="button"
className={`rounded-full px-3 py-1.5 text-sm ${
tab === 'canvas'
? 'bg-slate-900 text-white'
: 'bg-slate-100 text-slate-600'
}`}
onClick={() => onTabChange('canvas')}
>
</button>
<button
type="button"
className={`rounded-full px-3 py-1.5 text-sm ${
tab === 'library'
? 'bg-slate-900 text-white'
: 'bg-slate-100 text-slate-600'
}`}
onClick={() => onTabChange('library')}
>
</button>
</div>
{attachmentError ? (
<div className="rounded-2xl bg-amber-50 px-3 py-2 text-sm text-amber-700">
{attachmentError}
</div>
) : null}
<div className="grid grid-cols-2 gap-2 overflow-y-auto sm:grid-cols-3">
{visibleOptions.map((option) => {
const label =
option.attachment.label?.trim() || option.attachment.referenceId;
return (
<label
key={option.key}
className="flex cursor-pointer flex-col gap-2 rounded-2xl border border-slate-200 bg-white p-2 text-sm text-slate-700 shadow-sm"
>
<ResolvedAssetImage
src={
option.attachment.thumbnailSrc ?? option.attachment.imageSrc
}
objectKey={option.attachment.objectKey}
refreshKey={option.attachment.referenceId}
alt=""
className="aspect-square rounded-xl bg-slate-100 object-cover"
/>
<span className="flex items-center gap-2">
<input
type="checkbox"
aria-label={`选择${option.sourceLabel}图片 ${label}`}
checked={selectedKeys.has(option.key)}
onChange={() => onToggleKey(option.key)}
/>
<span className="truncate">{label}</span>
</span>
</label>
);
})}
</div>
</div>
</PlatformToolModalShell>
);
}
export function EditorAgentConversationPanelView({
open,
onToggleOpen,
@@ -276,70 +94,31 @@ export function EditorAgentConversationPanelView({
onConfirmSent,
});
const [draftText, setDraftText] = useState('');
const [attachments, setAttachments] = useState<EditorAgentAttachmentRef[]>(
[],
);
const [attachmentPickerOpen, setAttachmentPickerOpen] = useState(false);
const [attachmentPickerTab, setAttachmentPickerTab] =
useState<AttachmentPickerTab>('canvas');
const [attachmentError, setAttachmentError] = useState<string | null>(null);
const [isPastingAttachment, setIsPastingAttachment] = useState(false);
const [draftAttachmentKeys, setDraftAttachmentKeys] = useState<Set<string>>(
() => new Set(),
);
const {
attachments,
attachmentError,
isPastingAttachment,
attachmentPickerOpen,
attachmentPickerTab,
draftAttachmentKeys,
canvasAttachmentOptions,
libraryAttachmentOptions,
setAttachmentPickerTab,
openAttachmentPicker,
closeAttachmentPicker,
toggleAttachmentKey,
applyAttachmentSelection,
referenceContextAsset,
handleInputPaste,
removeAttachment,
consumeAttachments,
restoreAttachments,
} = useConversationAttachments({ projectId, layers, assets });
const [deleteConfirmOpen, setDeleteConfirmOpen] = useState(false);
const canvasAttachmentOptions = useMemo(
() => createCanvasAttachmentOptions(layers),
[layers],
);
const libraryAttachmentOptions = useMemo(
() => createLibraryAttachmentOptions(assets),
[assets],
);
const attachmentOptionsByKey = useMemo(() => {
const optionMap = new Map<string, EditorAgentAttachmentOption>();
[...canvasAttachmentOptions, ...libraryAttachmentOptions].forEach(
(option) => optionMap.set(option.key, option),
);
return optionMap;
}, [canvasAttachmentOptions, libraryAttachmentOptions]);
const hasProject = Boolean(projectId?.trim());
const isConversationBusy = isWaiting || isToolCallActionPending;
const openAttachmentPicker = () => {
setAttachmentError(null);
setDraftAttachmentKeys(new Set(attachments.map(attachmentKey)));
setAttachmentPickerOpen(true);
};
const toggleAttachmentKey = (key: string) => {
setDraftAttachmentKeys((currentKeys) => {
const nextKeys = new Set(currentKeys);
if (nextKeys.has(key)) {
nextKeys.delete(key);
} else {
nextKeys.add(key);
}
return nextKeys;
});
};
const applyAttachmentSelection = () => {
const nextAttachments = Array.from(draftAttachmentKeys)
.map((key) => attachmentOptionsByKey.get(key)?.attachment)
.filter((attachment): attachment is EditorAgentAttachmentRef =>
Boolean(attachment),
);
if (nextAttachments.length > EDITOR_AGENT_MAX_ATTACHMENTS) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
return;
}
setAttachments(nextAttachments);
setAttachmentPickerOpen(false);
};
const submitMessage = (event: FormEvent<HTMLFormElement>) => {
event.preventDefault();
if (isWaiting) {
@@ -353,139 +132,12 @@ export function EditorAgentConversationPanelView({
return;
}
setDraftText('');
const nextAttachments = attachments;
setAttachments([]);
const nextAttachments = consumeAttachments();
void sendMessage(text, nextAttachments).catch(() => {
setDraftText((currentText) => (currentText ? currentText : text));
setAttachments((currentAttachments) =>
currentAttachments.length ? currentAttachments : nextAttachments,
);
restoreAttachments(nextAttachments);
});
};
const appendAttachments = (nextAttachments: EditorAgentAttachmentRef[]) => {
function mergeAttachments(
currentAttachments: EditorAgentAttachmentRef[],
nextAttachments: EditorAgentAttachmentRef[],
) {
const merged = [...currentAttachments];
const existingKeys = new Set(currentAttachments.map(attachmentKey));
nextAttachments.forEach((attachment) => {
const key = attachmentKey(attachment);
if (!existingKeys.has(key)) {
existingKeys.add(key);
merged.push(attachment);
}
});
return merged;
}
const mergedAttachments = mergeAttachments(attachments, nextAttachments);
if (mergedAttachments.length > EDITOR_AGENT_MAX_ATTACHMENTS) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
return false;
}
setAttachments(mergedAttachments);
setAttachmentError(null);
return true;
};
const createPastedAgentImageAttachment = async (
file: File,
): Promise<EditorAgentAttachmentRef> => {
if (!projectId?.trim()) {
throw new Error('缺少画布项目');
}
const [upload, dimensions] = await Promise.all([
uploadEditorMediaAssetFile(file, 'image', {
pathSegments: ['editor', 'agent-paste', 'image', `${Date.now()}`],
entityId: projectId,
metadata: {
source: 'agent-input-paste',
},
}),
probeImageFileDimensions(file),
]);
const width = dimensions?.width ?? 1;
const height = dimensions?.height ?? 1;
const resource = await createEditorProjectResource(projectId, {
imageSrc: upload.src,
objectKey: upload.objectKey,
assetObjectId: upload.assetObjectId,
width,
height,
sourceType: 'uploaded',
});
return {
source: 'canvas_resource',
referenceId: resource.resourceId,
objectKey: resource.objectKey ?? upload.objectKey,
imageSrc: resource.imageSrc,
thumbnailSrc: null,
label: resource.label ?? '粘贴图片',
width: resource.width,
height: resource.height,
};
};
function extractClipboardImageFiles(
clipboardData: DataTransfer | null,
): File[] {
if (!clipboardData) {
return [];
}
const fileItems = Array.from(clipboardData.files ?? []).filter((file) =>
file.type.startsWith('image/'),
);
return [...fileItems];
}
const handleInputPaste = (
event: ReactClipboardEvent<HTMLTextAreaElement>,
) => {
const imageFiles = extractClipboardImageFiles(event.clipboardData);
if (!imageFiles.length) {
return;
}
if (isPastingAttachment) {
return;
}
event.preventDefault();
if (attachments.length >= EDITOR_AGENT_MAX_ATTACHMENTS) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
return;
}
const remainingAttachmentSlots =
EDITOR_AGENT_MAX_ATTACHMENTS - attachments.length;
const uploadFiles = imageFiles.slice(0, remainingAttachmentSlots);
const hasOverflow = uploadFiles.length < imageFiles.length;
// TODO: deduplicate those existing assets
setIsPastingAttachment(true);
setAttachmentError('图片上传中');
void Promise.all(
uploadFiles.map((file) => createPastedAgentImageAttachment(file)),
)
.then((pastedAttachments) => {
if (appendAttachments(pastedAttachments) && hasOverflow) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
}
})
.catch(() => {
setAttachmentError('图片粘贴失败,请重试');
})
.finally(() => {
setIsPastingAttachment(false);
});
};
const removeAttachment = (targetAttachment: EditorAgentAttachmentRef) => {
const key = attachmentKey(targetAttachment);
setAttachments((currentAttachments) =>
currentAttachments.filter(
(attachment) => attachmentKey(attachment) !== key,
),
);
};
if (!open) {
return (
@@ -609,6 +261,7 @@ export function EditorAgentConversationPanelView({
}
onConfirmToolCall={confirmToolCall}
onCancelToolCall={cancelToolCall}
onReferenceImage={referenceContextAsset}
onJobCompleted={() => {
void refreshActiveConversation();
onCanvasRefreshRequested?.();
@@ -689,7 +342,7 @@ export function EditorAgentConversationPanelView({
</div>
</form>
</aside>
<AttachmentPickerModal
<AttachmentPicker
open={attachmentPickerOpen}
tab={attachmentPickerTab}
canvasOptions={canvasAttachmentOptions}
@@ -699,7 +352,7 @@ export function EditorAgentConversationPanelView({
onTabChange={setAttachmentPickerTab}
onToggleKey={toggleAttachmentKey}
onApply={applyAttachmentSelection}
onClose={() => setAttachmentPickerOpen(false)}
onClose={closeAttachmentPicker}
/>
<PlatformDangerConfirmDialog
open={deleteConfirmOpen}
File diff suppressed because it is too large Load Diff
@@ -3,10 +3,16 @@ import {
type EditorAgentMessage,
} from '@/packages/shared/src/contracts';
import AttachmentChip from '@/src/components/image-editor/EditorAgentConversation/AttachmentChip.tsx';
import { attachmentKey } from '@/src/components/image-editor/EditorAgentConversation/common.ts';
import {
attachmentKey,
type EditorAgentContextAsset,
} from '@/src/components/image-editor/EditorAgentConversation/common.ts';
import { PendingToolCall } from '@/src/components/image-editor/EditorAgentConversation/PendingToolCall.tsx';
import ToolCallView from '@/src/components/image-editor/EditorAgentConversation/ToolCallView.tsx';
import { MessageBubbleRightClickMenu } from './MessageBubbleRightClickMenu.tsx';
import { useRightClickMenu } from './useRightClickMenu.ts';
function messageRoleLabel(role: EditorAgentMessage['role']) {
if (role === 'user') {
return '你';
@@ -55,6 +61,8 @@ type MessageBubbleProps = {
onConfirmToolCall: (messageId: number) => Promise<void>;
onCancelToolCall: (messageId: number) => Promise<void>;
onJobCompleted?: () => void;
// TODO prop drilling
onReferenceImage?: (asset: EditorAgentContextAsset) => boolean;
};
export function MessageBubble({
@@ -63,7 +71,14 @@ export function MessageBubble({
onConfirmToolCall,
onCancelToolCall,
onJobCompleted,
onReferenceImage,
}: MessageBubbleProps) {
const {
rightClickMenu,
openRightClickMenu,
closeRightClickMenu,
runRightClickAction,
} = useRightClickMenu({ onReferenceImage });
const systemErrorText =
message.role === 'system' &&
!message.toolCall &&
@@ -71,6 +86,11 @@ export function MessageBubble({
? message.text.slice(EDITOR_AGENT_ERROR_MESSAGE_PREFIX.length)
: null;
const isUser = message.role === 'user';
const isSystem = message.role === 'system';
const isSystemError = systemErrorText !== null;
const visibleText = systemErrorText ?? (!isSystem ? message.text : '');
if (
message.role === 'system' &&
!message.toolCall &&
@@ -95,56 +115,78 @@ export function MessageBubble({
);
}
const isUser = message.role === 'user';
const isSystem = message.role === 'system';
const isSystemError = systemErrorText !== null;
return (
<article
className={`flex ${isUser ? 'justify-end' : 'justify-start'}`}
aria-label={
isSystemError
? 'Agent错误'
: isSystem
? 'Agent操作'
: `${messageRoleLabel(message.role)}消息`
}
>
<div
className={
isSystem && !isSystemError
? 'max-w-[86%]'
: `max-w-[86%] rounded-3xl px-3.5 py-3 text-sm leading-6 shadow-sm ${
isUser
? 'bg-slate-900 text-white'
: isSystemError || message.toolCall?.status === 'failed'
? 'border border-red-200 bg-red-50 text-red-700'
: 'border border-slate-200 bg-white text-slate-700'
}`
<>
<article
className={`flex ${isUser ? 'justify-end' : 'justify-start'}`}
aria-label={
isSystemError
? 'Agent错误'
: isSystem
? 'Agent操作'
: `${messageRoleLabel(message.role)}消息`
}
onContextMenu={
visibleText.trim()
? (event) =>
openRightClickMenu(event, { kind: 'text', text: visibleText })
: undefined
}
>
{(!isSystem || isSystemError) && (systemErrorText ?? message.text) ? (
<div className="whitespace-pre-wrap break-words">
{systemErrorText ?? message.text}
</div>
) : null}
{!isSystem && message.attachments.length ? (
<div className="mt-2 flex flex-wrap gap-1.5">
{message.attachments.map((attachment) => (
<AttachmentChip
key={attachmentKey(attachment)}
attachment={attachment}
/>
))}
</div>
) : null}
{message.toolCall ? (
<ToolCallView
toolCall={message.toolCall}
onJobCompleted={onJobCompleted}
/>
) : null}
</div>
</article>
<div
className={
isSystem && !isSystemError
? 'max-w-[86%]'
: `max-w-[86%] rounded-3xl px-3.5 py-3 text-sm leading-6 shadow-sm ${
isUser
? 'bg-slate-900 text-white'
: isSystemError || message.toolCall?.status === 'failed'
? 'border border-red-200 bg-red-50 text-red-700'
: 'border border-slate-200 bg-white text-slate-700'
}`
}
>
{(!isSystem || isSystemError) && (systemErrorText ?? message.text) ? (
<div className="whitespace-pre-wrap break-words">
{systemErrorText ?? message.text}
</div>
) : null}
{!isSystem && message.attachments.length ? (
<div className="mt-2 flex flex-wrap gap-1.5">
{message.attachments.map((attachment) => (
<AttachmentChip
key={attachmentKey(attachment)}
attachment={attachment}
onRightClickMenu={(event, asset) =>
openRightClickMenu(event, { kind: 'asset', asset })
}
/>
))}
</div>
) : null}
{message.toolCall ? (
<ToolCallView
toolCall={message.toolCall}
onJobCompleted={onJobCompleted}
onRightClickMenu={(event, asset) =>
openRightClickMenu(event, { kind: 'asset', asset })
}
/>
) : null}
</div>
</article>
{rightClickMenu ? (
<MessageBubbleRightClickMenu
x={rightClickMenu.x}
y={rightClickMenu.y}
target={rightClickMenu.target}
pendingAction={rightClickMenu.pendingAction}
resultAction={rightClickMenu.resultAction}
result={rightClickMenu.result}
onAction={(action) => void runRightClickAction(action)}
onClose={closeRightClickMenu}
/>
) : null}
</>
);
}
@@ -0,0 +1,233 @@
import { useEffect, useLayoutEffect, useRef, useState } from 'react';
import { createPortal } from 'react-dom';
import {
contextAssetMediaSrc,
type EditorAgentContextAsset,
EditorAgentRightClickAction,
type RightClickMenuTarget,
} from './common.ts';
type MessageBubbleRightClickMenuProps = {
x: number;
y: number;
target: RightClickMenuTarget;
pendingAction: EditorAgentRightClickAction | null;
resultAction: EditorAgentRightClickAction | null;
result: 'success' | 'error' | null;
onAction: (action: EditorAgentRightClickAction) => void;
onClose: () => void;
};
const VIEWPORT_MARGIN = 8;
function actionLabel({
action,
idleLabel,
pendingAction,
resultAction,
result,
}: {
action: EditorAgentRightClickAction;
idleLabel: string;
pendingAction: EditorAgentRightClickAction | null;
resultAction: EditorAgentRightClickAction | null;
result: 'success' | 'error' | null;
}) {
if (pendingAction === action) {
return action === EditorAgentRightClickAction.DownloadAsset
? '下载中'
: action === EditorAgentRightClickAction.ReferenceImage
? '引用中'
: '复制中';
}
if (resultAction !== action) {
return idleLabel;
}
if (result === 'success') {
return action === EditorAgentRightClickAction.DownloadAsset
? '已下载'
: action === EditorAgentRightClickAction.ReferenceImage
? '已引用'
: '已复制';
}
if (result === 'error') {
return action === EditorAgentRightClickAction.DownloadAsset
? '下载失败'
: action === EditorAgentRightClickAction.ReferenceImage
? '引用失败'
: '复制失败';
}
return idleLabel;
}
function assetDownloadLabel(asset: EditorAgentContextAsset) {
return `下载${
asset.mediaType === 'image'
? '图片'
: asset.mediaType === 'video'
? '视频'
: '音频'
}`;
}
export function MessageBubbleRightClickMenu({
x,
y,
target,
pendingAction,
resultAction,
result,
onAction,
onClose,
}: MessageBubbleRightClickMenuProps) {
const menuRef = useRef<HTMLDivElement | null>(null);
const [position, setPosition] = useState<{ x: number; y: number } | null>(
null,
);
useLayoutEffect(() => {
const menu = menuRef.current;
if (!menu || typeof window === 'undefined') {
return;
}
const rect = menu.getBoundingClientRect();
setPosition({
x: Math.min(
Math.max(x, VIEWPORT_MARGIN),
Math.max(
VIEWPORT_MARGIN,
window.innerWidth - rect.width - VIEWPORT_MARGIN,
),
),
y: Math.min(
Math.max(y, VIEWPORT_MARGIN),
Math.max(
VIEWPORT_MARGIN,
window.innerHeight - rect.height - VIEWPORT_MARGIN,
),
),
});
}, [target.kind, x, y]);
useEffect(() => {
const handlePointerDown = (event: PointerEvent) => {
if (!menuRef.current?.contains(event.target as Node)) {
onClose();
}
};
const handleContextMenu = (event: MouseEvent) => {
if (menuRef.current?.contains(event.target as Node)) {
event.preventDefault();
}
onClose();
};
const handleKeyDown = (event: KeyboardEvent) => {
if (event.key === 'Escape') {
onClose();
}
};
window.addEventListener('pointerdown', handlePointerDown);
window.addEventListener('contextmenu', handleContextMenu, true);
window.addEventListener('keydown', handleKeyDown);
window.addEventListener('scroll', onClose, true);
window.addEventListener('resize', onClose);
return () => {
window.removeEventListener('pointerdown', handlePointerDown);
window.removeEventListener('contextmenu', handleContextMenu, true);
window.removeEventListener('keydown', handleKeyDown);
window.removeEventListener('scroll', onClose, true);
window.removeEventListener('resize', onClose);
};
}, [onClose]);
if (typeof document === 'undefined') {
return null;
}
return createPortal(
<div
ref={menuRef}
className="image-canvas-editor__context-menu"
role="menu"
aria-label={target.kind === 'text' ? '消息右键菜单' : '消息素材右键菜单'}
style={{
left: position?.x ?? x,
top: position?.y ?? y,
zIndex: 60,
}}
onContextMenu={(event) => event.preventDefault()}
>
{target.kind === 'text' ? (
<button
type="button"
role="menuitem"
disabled={pendingAction !== null}
onClick={() => onAction(EditorAgentRightClickAction.CopyText)}
>
{actionLabel({
action: EditorAgentRightClickAction.CopyText,
idleLabel: '复制文本',
pendingAction,
resultAction,
result,
})}
</button>
) : (
<>
{target.asset.mediaType === 'image' ? (
<>
{contextAssetMediaSrc(target.asset).trim() ? (
<button
type="button"
role="menuitem"
disabled={pendingAction !== null}
onClick={() =>
onAction(EditorAgentRightClickAction.ReferenceImage)
}
>
{actionLabel({
action: EditorAgentRightClickAction.ReferenceImage,
idleLabel: '引用',
pendingAction,
resultAction,
result,
})}
</button>
) : null}
<button
type="button"
role="menuitem"
disabled={pendingAction !== null}
onClick={() => onAction(EditorAgentRightClickAction.CopyImage)}
>
{actionLabel({
action: EditorAgentRightClickAction.CopyImage,
idleLabel: '复制图片',
pendingAction,
resultAction,
result,
})}
</button>
</>
) : null}
<button
type="button"
role="menuitem"
disabled={pendingAction !== null}
onClick={() => onAction(EditorAgentRightClickAction.DownloadAsset)}
>
{actionLabel({
action: EditorAgentRightClickAction.DownloadAsset,
idleLabel: assetDownloadLabel(target.asset),
pendingAction,
resultAction,
result,
})}
</button>
</>
)}
</div>,
document.body,
);
}
@@ -8,12 +8,16 @@ import { ResolvedAssetImage } from '@/src/components/ResolvedAssetImage.tsx';
import { ResolvedAssetVideo } from '@/src/components/ResolvedAssetVideo.tsx';
import { getExternalGenerationJobStatus } from '@/src/services/external-generation';
import type { RightClickMenuHandler } from './common.ts';
function ToolCallView({
toolCall,
onJobCompleted,
onRightClickMenu,
}: {
toolCall: EditorAgentToolCall;
onJobCompleted?: () => void;
onRightClickMenu?: RightClickMenuHandler;
}) {
const videos = toolCall.videos ?? [];
const audios = toolCall.audios ?? [];
@@ -69,6 +73,8 @@ function ToolCallView({
disposed = true;
if (timeoutId) clearTimeout(timeoutId);
};
// 轮询只允许新的 job/status source 重置;其余值通过当前 source 对应的闭包读取。
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [initialDisplayError, initialDisplayStatus, jobId, shouldPoll]);
const isCancelled = displayStatus === 'cancelled';
const isCompleted = displayStatus === 'completed';
@@ -107,6 +113,18 @@ function ToolCallView({
<div
key={`${toolCall.toolName}-${image.resourceId ?? index}`}
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'image',
mediaSrc: image.imageSrc,
objectKey: image.objectKey,
suggestedFileName: `Agent生成图片-${index + 1}`,
})
: undefined
}
>
<ResolvedAssetImage
src={image.thumbnailSrc ?? image.imageSrc}
@@ -125,6 +143,18 @@ function ToolCallView({
<div
key={`${toolCall.toolName}-${video.resourceId ?? video.objectKey ?? index}`}
className="overflow-hidden rounded-xl border border-slate-200 bg-slate-100"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'video',
mediaSrc: video.videoSrc,
objectKey: video.objectKey,
suggestedFileName: `Agent生成视频-${index + 1}`,
})
: undefined
}
>
<ResolvedAssetVideo
src={video.videoSrc}
@@ -148,6 +178,18 @@ function ToolCallView({
<div
key={`${toolCall.toolName}-${audio.resourceId ?? audio.objectKey ?? index}`}
className="flex items-center gap-2 rounded-xl border border-slate-200 bg-slate-50 p-2"
onContextMenu={
onRightClickMenu
? (event) =>
onRightClickMenu(event, {
kind: 'generated_media',
mediaType: 'audio',
mediaSrc: audio.audioSrc,
objectKey: audio.objectKey,
suggestedFileName: `Agent生成音频-${index + 1}`,
})
: undefined
}
>
<Volume2
className="h-4 w-4 shrink-0 text-slate-500"
@@ -1,5 +1,41 @@
import type { MouseEvent as ReactMouseEvent } from 'react';
import type { EditorAgentAttachmentRef } from '@/packages/shared/src/contracts';
export enum EditorAgentRightClickAction {
CopyText = 'copy_text',
CopyImage = 'copy_image',
ReferenceImage = 'reference_image',
DownloadAsset = 'download_asset',
}
export type EditorAgentContextAsset =
| (EditorAgentAttachmentRef & {
kind: 'attachment';
mediaType: 'image';
suggestedFileName: string;
})
| {
kind: 'generated_media';
mediaType: 'image' | 'video' | 'audio';
mediaSrc: string;
objectKey?: string | null;
suggestedFileName: string;
};
export type RightClickMenuTarget =
| { kind: 'text'; text: string }
| { kind: 'asset'; asset: EditorAgentContextAsset };
export type RightClickMenuHandler = (
event: ReactMouseEvent<HTMLElement>,
asset: EditorAgentContextAsset,
) => void;
export function attachmentKey(attachment: EditorAgentAttachmentRef) {
return `${attachment.source}:${attachment.referenceId}`;
}
export function contextAssetMediaSrc(asset: EditorAgentContextAsset) {
return asset.kind === 'attachment' ? asset.imageSrc : asset.mediaSrc;
}
@@ -0,0 +1,413 @@
import {
type ClipboardEvent as ReactClipboardEvent,
useCallback,
useMemo,
useRef,
useState,
} from 'react';
import {
EDITOR_AGENT_MAX_ATTACHMENTS,
type EditorAgentAttachmentRef,
} from '@/packages/shared/src/contracts';
import type {
CanvasLayer,
EditorAsset,
} from '@/src/components/image-editor/ImageCanvasEditorTypes.ts';
import { probeImageFileDimensions } from '@/src/components/image-editor/ImageCanvasFileModel.ts';
import { uploadEditorMediaAssetFile } from '@/src/services/image-editor/editorMediaAssetUploadClient.ts';
import { createEditorProjectResource } from '@/src/services/image-editor/editorProjectClient.ts';
import {
attachmentKey,
contextAssetMediaSrc,
type EditorAgentContextAsset,
} from './common.ts';
export type AttachmentPickerTab = 'canvas' | 'library';
export type EditorAgentAttachmentOption = {
key: string;
sourceLabel: string;
attachment: EditorAgentAttachmentRef;
};
type AttachmentUpdater =
| EditorAgentAttachmentRef[]
| ((
currentAttachments: EditorAgentAttachmentRef[],
) => EditorAgentAttachmentRef[]);
function isImageLayer(layer: CanvasLayer) {
return (
(layer.mediaType ?? 'image') === 'image' &&
Boolean(layer.resourceId?.trim()) &&
layer.src.trim()
);
}
function isImageAsset(asset: EditorAsset) {
return (asset.mediaType ?? 'image') === 'image' && asset.src.trim();
}
function createCanvasAttachmentOptions(
layers: CanvasLayer[],
): EditorAgentAttachmentOption[] {
return layers.filter(isImageLayer).map((layer) => {
const attachment: EditorAgentAttachmentRef = {
source: 'canvas_resource',
referenceId: layer.resourceId || layer.id,
objectKey: layer.objectKey ?? null,
imageSrc: layer.src,
thumbnailSrc: layer.thumbnailSrc ?? null,
label: layer.title,
width: layer.width,
height: layer.height,
};
return {
key: attachmentKey(attachment),
sourceLabel: '画布',
attachment,
};
});
}
function createLibraryAttachmentOptions(
assets: EditorAsset[],
): EditorAgentAttachmentOption[] {
return assets.filter(isImageAsset).map((asset) => {
const attachment: EditorAgentAttachmentRef = {
source: 'library_asset',
referenceId: asset.id,
objectKey: asset.objectKey ?? null,
imageSrc: asset.src,
thumbnailSrc: asset.thumbnailSrc ?? null,
label: asset.label,
width: asset.width,
height: asset.height,
};
return {
key: attachmentKey(attachment),
sourceLabel: '素材库',
attachment,
};
});
}
function mergeAttachments(
currentAttachments: EditorAgentAttachmentRef[],
nextAttachments: EditorAgentAttachmentRef[],
) {
const merged = [...currentAttachments];
const existingKeys = new Set(currentAttachments.map(attachmentKey));
nextAttachments.forEach((attachment) => {
const key = attachmentKey(attachment);
if (!existingKeys.has(key)) {
existingKeys.add(key);
merged.push(attachment);
}
});
return merged;
}
function extractClipboardImageFiles(clipboardData: DataTransfer | null) {
if (!clipboardData) {
return [];
}
return Array.from(clipboardData.files ?? []).filter((file) =>
file.type.startsWith('image/'),
);
}
async function createPastedAgentImageAttachment(
projectId: string,
file: File,
): Promise<EditorAgentAttachmentRef> {
const [upload, dimensions] = await Promise.all([
uploadEditorMediaAssetFile(file, 'image', {
pathSegments: ['editor', 'agent-paste', 'image', `${Date.now()}`],
entityId: projectId,
metadata: {
source: 'agent-input-paste',
},
}),
probeImageFileDimensions(file),
]);
const width = dimensions?.width ?? 1;
const height = dimensions?.height ?? 1;
const resource = await createEditorProjectResource(projectId, {
imageSrc: upload.src,
objectKey: upload.objectKey,
assetObjectId: upload.assetObjectId,
width,
height,
sourceType: 'uploaded',
});
return {
source: 'canvas_resource',
referenceId: resource.resourceId,
objectKey: resource.objectKey ?? upload.objectKey,
imageSrc: resource.imageSrc,
thumbnailSrc: null,
label: resource.label ?? '粘贴图片',
width: resource.width,
height: resource.height,
};
}
export function useConversationAttachments({
projectId,
layers,
assets,
}: {
projectId: string | null;
layers: CanvasLayer[];
assets: EditorAsset[];
}) {
const [attachments, setAttachments] = useState<EditorAgentAttachmentRef[]>(
[],
);
const attachmentsRef = useRef<EditorAgentAttachmentRef[]>([]);
const [attachmentPickerOpen, setAttachmentPickerOpen] = useState(false);
const [attachmentPickerTab, setAttachmentPickerTab] =
useState<AttachmentPickerTab>('canvas');
const [attachmentError, setAttachmentError] = useState<string | null>(null);
const [isPastingAttachment, setIsPastingAttachment] = useState(false);
const isPastingAttachmentRef = useRef(false);
const [draftAttachmentKeys, setDraftAttachmentKeys] = useState<Set<string>>(
() => new Set(),
);
const canvasAttachmentOptions = useMemo(
() => createCanvasAttachmentOptions(layers),
[layers],
);
const libraryAttachmentOptions = useMemo(
() => createLibraryAttachmentOptions(assets),
[assets],
);
const attachmentOptions = useMemo(
() => [...canvasAttachmentOptions, ...libraryAttachmentOptions],
[canvasAttachmentOptions, libraryAttachmentOptions],
);
const attachmentOptionsByKey = useMemo(() => {
const optionMap = new Map<string, EditorAgentAttachmentOption>();
attachmentOptions.forEach((option) => optionMap.set(option.key, option));
return optionMap;
}, [attachmentOptions]);
const updateAttachments = useCallback((updater: AttachmentUpdater) => {
const nextAttachments =
typeof updater === 'function' ? updater(attachmentsRef.current) : updater;
attachmentsRef.current = nextAttachments;
setAttachments(nextAttachments);
return nextAttachments;
}, []);
const appendAttachments = useCallback(
(nextAttachments: EditorAgentAttachmentRef[]) => {
const mergedAttachments = mergeAttachments(
attachmentsRef.current,
nextAttachments,
);
if (mergedAttachments.length > EDITOR_AGENT_MAX_ATTACHMENTS) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
return false;
}
updateAttachments(mergedAttachments);
setAttachmentError(null);
return true;
},
[updateAttachments],
);
const openAttachmentPicker = useCallback(() => {
setAttachmentError(null);
setDraftAttachmentKeys(new Set(attachmentsRef.current.map(attachmentKey)));
setAttachmentPickerOpen(true);
}, []);
const closeAttachmentPicker = useCallback(() => {
setAttachmentPickerOpen(false);
}, []);
const toggleAttachmentKey = useCallback((key: string) => {
setDraftAttachmentKeys((currentKeys) => {
const nextKeys = new Set(currentKeys);
if (nextKeys.has(key)) {
nextKeys.delete(key);
} else {
nextKeys.add(key);
}
return nextKeys;
});
}, []);
const applyAttachmentSelection = useCallback(() => {
const nextAttachments = Array.from(draftAttachmentKeys)
.map((key) => attachmentOptionsByKey.get(key)?.attachment)
.filter((attachment): attachment is EditorAgentAttachmentRef =>
Boolean(attachment),
);
if (nextAttachments.length > EDITOR_AGENT_MAX_ATTACHMENTS) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
return;
}
updateAttachments(nextAttachments);
setAttachmentPickerOpen(false);
}, [attachmentOptionsByKey, draftAttachmentKeys, updateAttachments]);
const referenceContextAsset = useCallback(
(asset: EditorAgentContextAsset) => {
if (asset.kind === 'attachment') {
const directAttachment: EditorAgentAttachmentRef = {
source: asset.source,
referenceId: asset.referenceId,
objectKey: asset.objectKey,
imageSrc: asset.imageSrc,
thumbnailSrc: asset.thumbnailSrc,
label: asset.label,
width: asset.width,
height: asset.height,
};
return directAttachment.referenceId.trim() &&
directAttachment.imageSrc.trim()
? appendAttachments([directAttachment])
: false;
}
const objectKey = asset.objectKey?.trim();
const mediaSrc = contextAssetMediaSrc(asset).trim();
const objectKeyOption = objectKey
? attachmentOptions.find(
({ attachment }) => attachment.objectKey?.trim() === objectKey,
)
: undefined;
const option =
objectKeyOption ||
attachmentOptions.find(({ attachment }) => {
return (
attachment.imageSrc.trim() === mediaSrc ||
attachment.thumbnailSrc?.trim() === mediaSrc
);
});
const matchedAttachment = option?.attachment;
return matchedAttachment?.referenceId.trim() &&
matchedAttachment.imageSrc.trim()
? appendAttachments([matchedAttachment])
: false;
},
[appendAttachments, attachmentOptions],
);
const handleInputPaste = useCallback(
(event: ReactClipboardEvent<HTMLTextAreaElement>) => {
const imageFiles = extractClipboardImageFiles(event.clipboardData);
if (!imageFiles.length || isPastingAttachmentRef.current) {
return;
}
event.preventDefault();
const remainingAttachmentSlots =
EDITOR_AGENT_MAX_ATTACHMENTS - attachmentsRef.current.length;
if (remainingAttachmentSlots <= 0) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
return;
}
const uploadFiles = imageFiles.slice(0, remainingAttachmentSlots);
const hasOverflow = uploadFiles.length < imageFiles.length;
const currentProjectId = projectId?.trim();
if (!currentProjectId) {
setAttachmentError('缺少画布项目');
return;
}
isPastingAttachmentRef.current = true;
setIsPastingAttachment(true);
setAttachmentError('图片上传中');
void Promise.all(
uploadFiles.map((file) =>
createPastedAgentImageAttachment(currentProjectId, file),
),
)
.then((pastedAttachments) => {
if (appendAttachments(pastedAttachments) && hasOverflow) {
setAttachmentError(`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS}`);
}
})
.catch(() => {
setAttachmentError('图片粘贴失败,请重试');
})
.finally(() => {
isPastingAttachmentRef.current = false;
setIsPastingAttachment(false);
});
},
[appendAttachments, projectId],
);
const removeAttachment = useCallback(
(targetAttachment: EditorAgentAttachmentRef) => {
const key = attachmentKey(targetAttachment);
updateAttachments((currentAttachments) =>
currentAttachments.filter(
(attachment) => attachmentKey(attachment) !== key,
),
);
},
[updateAttachments],
);
const consumeAttachments = useCallback(() => {
const currentAttachments = attachmentsRef.current;
const refreshedAttachments = currentAttachments.map(
(attachment) =>
attachmentOptionsByKey.get(attachmentKey(attachment))?.attachment ??
attachment,
);
updateAttachments([]);
return refreshedAttachments;
}, [attachmentOptionsByKey, updateAttachments]);
const restoreAttachments = useCallback(
(failedAttachments: EditorAgentAttachmentRef[]) => {
updateAttachments((currentAttachments) => {
// respect current (version) when duplicated with old ones
const restoredAttachments = mergeAttachments(
currentAttachments,
failedAttachments,
);
if (restoredAttachments.length > EDITOR_AGENT_MAX_ATTACHMENTS) {
setAttachmentError(
`最多 ${EDITOR_AGENT_MAX_ATTACHMENTS} 张,发送失败的附件未恢复`,
);
return currentAttachments;
}
setAttachmentError(null);
return restoredAttachments;
});
},
[updateAttachments],
);
return {
attachments,
attachmentError,
isPastingAttachment,
attachmentPickerOpen,
attachmentPickerTab,
draftAttachmentKeys,
canvasAttachmentOptions,
libraryAttachmentOptions,
setAttachmentPickerTab,
openAttachmentPicker,
closeAttachmentPicker,
toggleAttachmentKey,
applyAttachmentSelection,
referenceContextAsset,
handleInputPaste,
removeAttachment,
consumeAttachments,
restoreAttachments,
};
}
@@ -0,0 +1,318 @@
import {
type MouseEvent as ReactMouseEvent,
useCallback,
useState,
} from 'react';
import { readAssetBytes } from '@/src/services/assetReadUrlService.ts';
import { copyTextToClipboard } from '@/src/services/clipboard.ts';
import {
canUseNativeHostCapability,
exportHostAudioFile,
exportHostImageFile,
type HostFileExportAudioRequest,
type HostFileExportImageRequest,
isNativeAppRuntime,
} from '@/src/services/host-bridge/hostBridge.ts';
import { getLayerAssetExtensionFromTypeOrSrc } from '../ImageCanvasExportModel.ts';
import {
contextAssetMediaSrc,
type EditorAgentContextAsset,
EditorAgentRightClickAction,
type RightClickMenuTarget,
} from './common.ts';
type RightClickMenuState = {
x: number;
y: number;
target: RightClickMenuTarget;
pendingAction: EditorAgentRightClickAction | null;
resultAction: EditorAgentRightClickAction | null;
result: 'success' | 'error' | null;
};
function sanitizeDownloadName(value: string, extension: string) {
const normalized = value
.trim()
.replace(/[<>:"/\\|?*]/gu, '-')
.replace(/\p{Cc}/gu, '-')
.replace(/[. ]+$/u, '')
.slice(0, 100);
const baseName = (normalized || 'Agent素材').replace(
/\.[a-z0-9]{1,8}$/iu,
'',
);
return extension ? `${baseName}.${extension}` : baseName;
}
function blobToBase64Data(blob: Blob) {
return new Promise<string>((resolve, reject) => {
if (typeof FileReader === 'undefined') {
reject(new Error('当前环境不支持文件编码'));
return;
}
const reader = new FileReader();
reader.onerror = () => reject(new Error('文件编码失败'));
reader.onload = () => {
const result = typeof reader.result === 'string' ? reader.result : '';
const base64Data = result.split(',')[1] ?? '';
if (base64Data) {
resolve(base64Data);
} else {
reject(new Error('文件编码失败'));
}
};
reader.readAsDataURL(blob);
});
}
function hostImageMimeTypeFromExtension(
extension: string,
): HostFileExportImageRequest['mimeType'] | null {
if (extension === 'png') {
return 'image/png';
}
if (extension === 'jpg' || extension === 'jpeg') {
return 'image/jpeg';
}
if (extension === 'webp') {
return 'image/webp';
}
return null;
}
function hostAudioMimeTypeFromExtension(
extension: string,
): HostFileExportAudioRequest['mimeType'] | null {
if (extension === 'mp3') {
return 'audio/mpeg';
}
if (extension === 'm4a' || extension === 'mp4') {
return 'audio/mp4';
}
if (extension === 'wav') {
return 'audio/wav';
}
if (extension === 'ogg') {
return 'audio/ogg';
}
if (extension === 'webm') {
return 'audio/webm';
}
return null;
}
function triggerBrowserDownload(blob: Blob, fileName: string) {
if (
typeof document === 'undefined' ||
typeof URL.createObjectURL !== 'function' ||
typeof URL.revokeObjectURL !== 'function'
) {
return false;
}
const downloadUrl = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = downloadUrl;
link.download = fileName;
link.style.display = 'none';
document.body.appendChild(link);
link.click();
link.remove();
window.setTimeout(() => URL.revokeObjectURL(downloadUrl), 0);
return true;
}
async function convertImageBlobToPng(blob: Blob) {
if (blob.type.toLowerCase() === 'image/png') {
return blob;
}
if (
typeof createImageBitmap !== 'function' ||
typeof document === 'undefined'
) {
throw new Error('当前浏览器不支持复制此图片格式');
}
const bitmap = await createImageBitmap(blob);
try {
const canvas = document.createElement('canvas');
canvas.width = bitmap.width;
canvas.height = bitmap.height;
const context = canvas.getContext('2d');
if (!context) {
throw new Error('图片转换失败');
}
context.drawImage(bitmap, 0, 0);
return await new Promise<Blob>((resolve, reject) => {
canvas.toBlob((pngBlob) => {
if (pngBlob) {
resolve(pngBlob);
} else {
reject(new Error('图片转换失败'));
}
}, 'image/png');
});
} finally {
bitmap.close();
}
}
async function copyAssetImage(asset: EditorAgentContextAsset) {
if (
typeof navigator === 'undefined' ||
typeof navigator.clipboard?.write !== 'function' ||
typeof ClipboardItem === 'undefined'
) {
return false;
}
try {
const pngBlob = readAssetBytes(contextAssetMediaSrc(asset), {
objectKey: asset.objectKey,
}).then(async (response) => convertImageBlobToPng(await response.blob()));
// Clipboard write must start in the menu click's user-activation stack.
// Safari accepts promised ClipboardItem data and resolves it after asset loading/conversion.
await navigator.clipboard.write([
new ClipboardItem({ 'image/png': pngBlob }),
]);
return true;
} catch {
return false;
}
}
async function downloadAsset(asset: EditorAgentContextAsset) {
try {
const source = contextAssetMediaSrc(asset);
const response = await readAssetBytes(source, {
objectKey: asset.objectKey,
});
const blob = await response.blob();
const extension = getLayerAssetExtensionFromTypeOrSrc(
asset.mediaType,
// application/octet-stream is impossible by contract
blob.type,
asset.objectKey ?? source,
);
const fileName = sanitizeDownloadName(asset.suggestedFileName, extension);
if (!isNativeAppRuntime()) {
return triggerBrowserDownload(blob, fileName);
}
if (
asset.mediaType === 'image' &&
canUseNativeHostCapability('file.exportImage')
) {
const mimeType = hostImageMimeTypeFromExtension(extension);
if (!mimeType) {
return false;
}
const base64Data = await blobToBase64Data(blob);
return Boolean(
await exportHostImageFile({ fileName, base64Data, mimeType }),
);
}
if (
asset.mediaType === 'audio' &&
canUseNativeHostCapability('file.exportAudio')
) {
const mimeType = hostAudioMimeTypeFromExtension(extension);
if (!mimeType) {
return false;
}
const base64Data = await blobToBase64Data(blob);
return Boolean(
await exportHostAudioFile({ fileName, base64Data, mimeType }),
);
}
// video is not yet supported to export on native shell
return false;
} catch {
return false;
}
}
export function useRightClickMenu({
onReferenceImage,
}: {
onReferenceImage?: (asset: EditorAgentContextAsset) => boolean;
} = {}) {
const [rightClickMenu, setRightClickMenu] =
useState<RightClickMenuState | null>(null);
const closeRightClickMenu = useCallback(() => {
setRightClickMenu(null);
}, []);
const openRightClickMenu = useCallback(
(event: ReactMouseEvent<HTMLElement>, target: RightClickMenuTarget) => {
event.preventDefault();
event.stopPropagation();
setRightClickMenu({
x: event.clientX,
y: event.clientY,
target,
pendingAction: null,
resultAction: null,
result: null,
});
},
[],
);
const runRightClickAction = useCallback(
async (action: EditorAgentRightClickAction) => {
if (!rightClickMenu || rightClickMenu.pendingAction) {
return;
}
const target = rightClickMenu.target;
setRightClickMenu((current) =>
current?.target === target
? {
...current,
pendingAction: action,
resultAction: null,
result: null,
}
: current,
);
const succeeded =
action === EditorAgentRightClickAction.CopyText &&
target.kind === 'text'
? await copyTextToClipboard(target.text)
: action === EditorAgentRightClickAction.CopyImage &&
target.kind === 'asset' &&
target.asset.mediaType === 'image'
? await copyAssetImage(target.asset)
: action === EditorAgentRightClickAction.ReferenceImage &&
target.kind === 'asset' &&
target.asset.mediaType === 'image' &&
contextAssetMediaSrc(target.asset).trim()
? (onReferenceImage?.(target.asset) ?? false)
: action === EditorAgentRightClickAction.DownloadAsset &&
target.kind === 'asset'
? await downloadAsset(target.asset)
: false;
setRightClickMenu((current) =>
current?.target === target && current.pendingAction === action
? succeeded
? null
: {
...current,
pendingAction: null,
resultAction: action,
result: 'error',
}
: current,
);
},
[onReferenceImage, rightClickMenu],
);
return {
rightClickMenu,
openRightClickMenu,
closeRightClickMenu,
runRightClickAction,
};
}