WIP: UI编辑器自动分图层切图标 #304

Draft
k88936 wants to merge 100 commits from feat/ui-editor-auto-seperation into master
2 changed files with 16 additions and 6 deletions
Showing only changes of commit 1c992c69fe - Show all commits
@@ -44,13 +44,13 @@ const SYSTEM_PROMPT: &str = r#"
* 由于每个截图未必是完整的, 可能是局部的, 每棵树描述清楚每个截图上UI的层次结构即可
* 不同树的共用框架/层次/...请使用使用相同的名称描述. 不同状态/变体名称使用相同的前缀, 用后缀区别
* 粒度要求: 尽可能细致, 以可交互,方便程序化控制的最小单位为准. 包括不限于: icon, 进度条的底槽、填充和外框; slider的底槽, dragger等.
* 为每个节点直接返回 component.
* 为每个节点直接返回 component.
无背景的逻辑容器返回 "PureNode",不要返回 null.
有背景的容器推荐使用Simple+不锁定宽高比的Image component.
目前我们只做识别, 不要求图片字体参数.
每个节点最多返回一个 component;需要多个视觉层时拆成多个节点。
文字组件要求: 艺术字等作为图片组件, 其余正常文字要作为单独的节点识别.
* 不鼓励兄弟节点相互重叠, 对于背景等元素, 请promote为父节点的组件, 不要再建单独的兄弟节点承载.
"#;
#[derive(Clone, Debug, Deserialize, PartialEq, Serialize, JsonSchema)]
@@ -35,10 +35,6 @@ pub(super) fn gen_binding_prompt(nodes: Vec<&SeparationNode>) -> String {
let binding_system_prompt = format!(
r#"
You will be given a src UI design image and a processed image, where some ui elements are separated.
Here were the separation requirements:
```
{SHARED_SEPARATION_REQ}
```
You need to recognize and review the separation using the given tool.
field notes:
* extracted_area MUST be the recognized area from the processed image, INSTEAD OF from the src image.
@@ -47,6 +43,20 @@ pub(super) fn gen_binding_prompt(nodes: Vec<&SeparationNode>) -> String {
Do not copy, infer, or reuse the source node rectangle.
The src image is only for identifying which semantic UI element belongs to to_node.
Here were the separation requirements:
```
{SHARED_SEPARATION_REQ}
```
And you should also review if the extracted's successfully meet the src image:
* shape
* color
* style
* edge process
...
if not, use `NeedRework` data structure in the tool to indicate the (it should be from which) node id and advice.
your advice (less than 20 words) will be used to improve the separation in the next time.
these node need handle:
"#
);