feat(chat): 结构化用户附件并支持图片预览
This commit is contained in:
parent
c4418711ab
commit
054aaf8e8c
|
|
@ -4,6 +4,10 @@
|
|||
> 维护口径:只记**用户可感知**的变化("能干什么了"而不是"改了哪个模块"),纯内部重构 / 修复不记,
|
||||
> 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。
|
||||
> 工程口径的完整记录见 `PROGRESS.md` / git log。
|
||||
## 0.65.2 — 2026-08-13
|
||||
|
||||
- 粘贴、拖拽或选择附件发送后，对话中的用户消息会保留附件卡片；图片同时显示可点击放大的缩略图，刷新历史对话后也不会消失。
|
||||
|
||||
|
||||
## 0.65.1 — 2026-08-12
|
||||
|
||||
|
|
|
|||
|
|
@ -175,6 +175,8 @@ Eval 与生产 core 解耦，通过现有 `/v1` API 创建专用任务、监听
|
|||
|
||||
**对话产物与生命周期(0025/0028)**:真实文件仍是内容事实源；`artifacts` 表记录已发布产物的稳定身份和生命周期，包含 user-root 相对当前路径、来源 task、复制来源、哈希/大小及 active/deleted、回收路径。新 `messages.artifact_refs` 使用 `{version:2, artifact_id, scope:"working_dir", path:"reports/a.pdf", label?:"最终报告"}`；`path` 是兼容快照，预览/下载优先按 `artifact_id` 找当前路径，因此移动或重命名后历史卡片仍有效。version 1 和 `NULL` 旧消息继续走原 task-scoped 兼容链。普通源码树、中间文件和配套资源不登记；agent 仅用 `publish_artifacts` 显式提升少量最终文件。移动保持身份，复制为每个副本创建新身份并记录直接来源；删除将文件移入 `.zcbot_artifact_trash/` 并软删记录，普通文件仍物理删除。
|
||||
|
||||
**用户消息附件(0031)**:`messages.attachment_refs` 与助手产物分开表达输入附件，元素为 `{version:1, scope:"working_dir", path, label, kind, media_type, size_bytes}`；文件仍是事实源，不登记为已发布 artifact，也不承诺独立生命周期。`payload.content` 只保存用户自然语言，模型上下文在内存中按附件类型补兼容提示，避免 UI 协议污染正文。`NULL` 表示旧客户端/旧历史，前端继续解析正文标记；`[]` 表示新消息明确无附件。独立 attachment 表暂缓，只有出现跨消息复用、稳定身份、版本快照或附件级审计需求时再抽象 `file_assets + message_attachments`。
|
||||
|
||||
### 7.2 资源模型(/v1)
|
||||
|
||||
统一 `/v1` 前缀返 JSON;UI 由 platform 实现(§7.9),本地 dev SPA dogfood。要点(细节见 `web/app.py`):
|
||||
|
|
@ -222,8 +224,8 @@ tasks(task_id pk, user_id fk, name NOT NULL, auto_title_pending default false,
|
|||
context_base_idx, -- 0019 §8.8 软重置窗口起点
|
||||
deleted_at, -- 0010 软删
|
||||
created_at, updated_at)
|
||||
messages(pk, task_id fk, idx, payload jsonb, artifact_refs jsonb null, -- 0025,task-relative UI 元数据
|
||||
tokens_in/out, model_profile, kind, -- kind=push 等
|
||||
messages(pk, task_id fk, idx, payload jsonb, artifact_refs jsonb null, attachment_refs jsonb null,
|
||||
tokens_in/out, model_profile, kind, -- 0025/0031 UI 元数据;kind=push 等
|
||||
unique(task_id, idx); gin(payload))
|
||||
usage_events(pk, user_id, task_id, message_id, kind, -- chat/image/video/vision/... 自由文本
|
||||
model_profile, units jsonb, cost numeric, created_at) -- 多态用量,加媒体不动 schema
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`。
|
||||
|
||||
最后更新:2026-08-12(Windows Node 列表与生命周期管理,bump 0.65.1)
|
||||
最后更新:2026-08-13(用户消息结构化附件与图片预览,bump 0.65.2)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -18,9 +18,12 @@
|
|||
| §7 SaaS | DESIGN §7 路线 | 🟡 | A 事件流化 ✅;B 完工 ✅;D `/v1` JSON API ✅;D' 过渡 auth + dev SPA ✅;单活 run 锁 + cancel ✅;0004 schema 瘦身 ✅;入口归位 ✅;真 OIDC 待;**C Step 1-3 + 3d ✅(Executor + Docker 池 + DockerExecutor + fs 工具进容器)+ Step 5 部署前置对账 ✅ + 容器资源 yaml + 应用层磁盘配额 ✅ + dogfood 网络放开 + 容器内 pip/npm 源持久化 ✅**;**Step 4 完整 egress proxy + Step 3b PGID kill 协议延后到外部用户开放前**(还需 egress proxy + xfs project quota OS 层硬化,§7.5 落地清单 #2 #4)。 |
|
||||
|
||||
---
|
||||
|
||||
## 已完成关键能力
|
||||
|
||||
### 2026-08-13
|
||||
|
||||
- **08-13 / 0.65.2 / 用户消息结构化附件 + 对话内图片预览**:新增 0031 `messages.attachment_refs`，新客户端将附件作为结构化字段发送，数据库正文仅保留用户自然语言；后端按 task working_dir 校验路径并在内存模型上下文中补附件提示，旧客户端与历史正文标记继续兼容。用户消息即时态和历史态统一展示附件 chip，图片额外显示可点击缩略图；相关 Python 34 项、全部前端 Node 26 项、Python/JavaScript 语法、Alembic 单 head 与 diff 检查通过，未连接或迁移生产 DB。
|
||||
|
||||
### 2026-08-12
|
||||
|
||||
- **08-12 / 0.65.1 / Windows Node 列表与生命周期管理**:管理后台按 10 秒轮询展示节点名称/ID、在线状态、系统与桌面会话、版本、能力和最近心跳，并提供可恢复的禁用/启用与永久删除；禁用或删除在线节点先主动关闭 WebSocket，删除后的本机身份必须清除并重新注册。生产实测补齐 nginx 节点 WebSocket Upgrade 专用 location 与故障说明，避免注册成功后长连接被普通 HTTP location 当作 GET。相关后端、鉴权路由和管理前端 45 项 unittest、Python 编译、JavaScript 语法与 diff 检查通过；无 schema、migration 或依赖变化，未连接或写入生产 DB。
|
||||
|
|
|
|||
5
RUN.md
5
RUN.md
|
|
@ -152,6 +152,7 @@
|
|||
- **PG**:`ZCBOT_DB_URL` 必填。本地 docker compose / 远端 dev / 生产任选;未设置时启动清晰报错,不引导 docker(§7.4)。
|
||||
- **OpenAPI / MCP 外部系统**:① `.env` 配置独立的 `ZCBOT_CREDENTIAL_MASTER_KEY`，可选 `ZCBOT_CREDENTIAL_KEY_ID` 标识当前密钥；轮换时把旧 key 以 JSON 对象放入 `ZCBOT_CREDENTIAL_PREVIOUS_KEYS`，待用户凭据完成重写后再移除。② 执行 `main.py db upgrade head`。③ admin 进入管理后台「外部系统」，选择通用 OpenAPI 或通用 MCP；具体 MES/ERP/LIMS 都作为数据库 definition 配置，不新增专用 provider。MCP 填写与登录 Base URL 同源的 Streamable HTTP URL，可选填写期望 Server 名称；连接后以 `tools/list` 为事实源。④ 普通用户点击左栏 **「外部」**，页面按 definition 动态显示用户名密码、API Key 或 Bearer Token；目标、Server 身份、登录、认证绑定或 TLS 变化后保留密文并暂停调用，重新测试成功后恢复。OpenAPI spec 和 MCP tool catalog 只在进程内按连接身份有界缓存，登录与业务响应均限长，普通用户和模型不能传任意 URL。
|
||||
- **Artifact 生命周期(0028)**:部署本版本必须先执行 `.venv/Scripts/python.exe main.py db upgrade head`。migration 会从存量 `messages.artifact_refs` 回填 active artifact 身份；删除后的已发布产物保存在用户根目录隐藏区 `.zcbot_artifact_trash/`，默认不自动清理且继续计入磁盘配额。普通文件删除语义不变。
|
||||
- **用户消息结构化附件(0031)**:部署新版前执行 `.venv/Scripts/python.exe main.py db upgrade head`，为 `messages` 增加 nullable `attachment_refs`。migration 不回写或改动存量正文；旧客户端仍可只传 `content`，新版会发送结构化 `attachments`。
|
||||
- **旧 `factory_mes` definition 一次性转换**:新版代码不再识别 `factory_mes`；部署时保持旧服务进程运行，先从新代码目录执行数据脚本，转换成功后再重启到新版。脚本不加载 `.env`、不读取 `ZCBOT_DB_URL`，只认显式的 `ZCBOT_MIGRATION_DB_URL`；默认 dry-run，检查同名冲突与配置合法性。确认输出后加 `--apply`，脚本把 definition 转为 `generic_openapi + query`、物化 JWT/提示/只读 POST 配置并同步 active connection revision，不解密或改写用户凭据。
|
||||
```powershell
|
||||
$env:ZCBOT_MIGRATION_DB_URL="postgresql+psycopg://user:pass@host:5432/zcbot"
|
||||
|
|
@ -361,8 +362,8 @@ $env:ZCBOT_EVAL_TOKEN = "<dedicated-eval-user-jwt>"
|
|||
| `DELETE /v1/external-systems/{id}` | 清除当前用户连接及密文凭据；指定用户模式保留管理员授予的可见权 | 必填 |
|
||||
| `GET/POST /v1/admin/external-system-definitions` | 管理员列出或新增可信外部系统目录 | admin |
|
||||
| `PUT/DELETE /v1/admin/external-system-definitions/{id}` | 管理员编辑、停用或删除目录项；已有用户连接时拒绝删除 | admin |
|
||||
| `GET /v1/tasks/{id}/messages` | LiteLLM payload 透传；0025 起每条另带 `artifact_refs`：`null`=旧消息、`[]`=新消息无产物、非空数组=相对该 task 当前 working_dir 的结构化产物引用 | 必填 |
|
||||
| `POST /v1/tasks/{id}/messages` | `{content, image_model?=""}` 发消息;返 `{events_url}`;**`run_status` 是 running/cancelling → 409**(单活 run;error 起新 run 时清);`image_model` 是 `config/media/doubao.yaml` image 段的 variant key(空 → 沿用 yaml 第一个),仅本 run 装配 SeedreamTool 时使用,不入 DB;UI 应 disable send 直到 SSE `done` | 必填 |
|
||||
| `GET /v1/tasks/{id}/messages` | LiteLLM payload 透传；另带 `artifact_refs`(助手产物)与 `attachment_refs`(用户附件)。两者均以 `null` 表示旧消息、`[]` 表示新消息明确为空、非空数组表示 task-relative 结构化引用 | 必填 |
|
||||
| `POST /v1/tasks/{id}/messages` | `{content, attachments?:[{path,kind,label?}], image_model?=""}` 发消息；`attachments` 路径由后端按当前 task working_dir 校验并规范化，允许纯附件消息；旧客户端省略该字段继续兼容。返 `{events_url}`；**`run_status` 是 running/cancelling → 409**；UI 应 disable send 直到 SSE `done` | 必填 |
|
||||
| `GET /v1/tasks/{id}/events` | SSE 流(`event: <type>` + `data: <json>`);订阅 task 当前活动 | 必填 |
|
||||
| `POST /v1/tasks/{id}/cancel` | 协作式 cancel;`run_status != running` → 409;LLM 走 streaming,chunk 间 poll cancel — 延迟 100ms 级,基本秒退 | 必填 |
|
||||
| `GET /v1/procs` | 当前用户全部后台进程(bg proc,§8.12;shell/run_python `background=true` 启动);纯文件系统读取,前端运行条 5s 轮询用 | 必填 |
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
|
||||
# 改版本只动这一行。
|
||||
__version__ = "0.65.1"
|
||||
__version__ = "0.65.2"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,76 @@
|
|||
"""Structured user-message attachments and model-context compatibility rendering."""
|
||||
from __future__ import annotations
|
||||
|
||||
import mimetypes
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .artifacts import ArtifactPathError, resolve_artifact_path
|
||||
|
||||
ATTACHMENT_REF_VERSION = 1
|
||||
MAX_ATTACHMENTS_PER_MESSAGE = 10
|
||||
_IMAGE_EXTENSIONS = {
|
||||
".avif", ".bmp", ".gif", ".heic", ".heif", ".jpeg", ".jpg",
|
||||
".png", ".svg", ".tif", ".tiff", ".webp",
|
||||
}
|
||||
|
||||
|
||||
def _attachment_kind(requested: str, path: Path, media_type: str) -> str:
|
||||
detected_image = media_type.startswith("image/") or path.suffix.lower() in _IMAGE_EXTENSIONS
|
||||
if requested == "image" and detected_image:
|
||||
return "image"
|
||||
return "file"
|
||||
|
||||
|
||||
def normalize_attachment_refs(
|
||||
refs: Iterable[Any], *, working_dir: Path, user_root: Path,
|
||||
) -> list[dict]:
|
||||
"""Validate client refs and return task-relative, display-ready metadata."""
|
||||
output: list[dict] = []
|
||||
seen: set[str] = set()
|
||||
for raw in refs:
|
||||
if len(output) >= MAX_ATTACHMENTS_PER_MESSAGE:
|
||||
raise ArtifactPathError(
|
||||
f"at most {MAX_ATTACHMENTS_PER_MESSAGE} attachments are allowed"
|
||||
)
|
||||
data = raw.model_dump() if hasattr(raw, "model_dump") else dict(raw or {})
|
||||
absolute, rel = resolve_artifact_path(
|
||||
str(data.get("path") or ""),
|
||||
working_dir=working_dir,
|
||||
user_root=user_root,
|
||||
require_file=True,
|
||||
)
|
||||
if rel in seen:
|
||||
continue
|
||||
seen.add(rel)
|
||||
media_type = mimetypes.guess_type(absolute.name)[0] or "application/octet-stream"
|
||||
label = str(data.get("label") or absolute.name).strip() or absolute.name
|
||||
output.append({
|
||||
"version": ATTACHMENT_REF_VERSION,
|
||||
"scope": "working_dir",
|
||||
"path": rel,
|
||||
"label": label,
|
||||
"kind": _attachment_kind(str(data.get("kind") or "file"), absolute, media_type),
|
||||
"media_type": media_type,
|
||||
"size_bytes": absolute.stat().st_size,
|
||||
})
|
||||
return output
|
||||
|
||||
|
||||
def content_for_model(content: str, refs: Iterable[dict], working_dir_name: str) -> str:
|
||||
"""Append the legacy-readable file hints only to the provider-bound message."""
|
||||
lines: list[str] = []
|
||||
wd = str(working_dir_name or "").strip().strip("/\\")
|
||||
for ref in refs or ():
|
||||
path = str((ref or {}).get("path") or "").strip().replace("\\", "/")
|
||||
if not path:
|
||||
continue
|
||||
full_path = f"{wd}/{path}" if wd else path
|
||||
marker = "[用户上传的参考图]" if (ref or {}).get("kind") == "image" else "[用户上传的文件]"
|
||||
lines.append(f"{marker} {full_path}")
|
||||
text = str(content or "").strip()
|
||||
if not lines:
|
||||
return text
|
||||
suffix = "\n".join(lines)
|
||||
return f"{text}\n\n{suffix}" if text else suffix
|
||||
|
|
@ -226,6 +226,16 @@ class Session:
|
|||
).scalars().all()
|
||||
for row in rows:
|
||||
payload = dict(row.payload)
|
||||
attachment_refs = getattr(row, "attachment_refs", None)
|
||||
if payload.get("role") == "user" and attachment_refs:
|
||||
from .attachments import content_for_model
|
||||
working_dir = str((sess.meta or {}).get("working_dir") or "")
|
||||
wd_name = working_dir.replace("\\", "/").rstrip("/").rsplit("/", 1)[-1]
|
||||
payload["content"] = content_for_model(
|
||||
str(payload.get("content") or ""),
|
||||
attachment_refs,
|
||||
wd_name,
|
||||
)
|
||||
if payload.get("role") == "assistant" and isinstance(payload.get("content"), str):
|
||||
# 历史行不回写生产库；只在重建 LLM 上下文时应用同一窄修复，
|
||||
# 与 Web 展示层保持一致，避免旧坏围栏继续污染后续轮次。
|
||||
|
|
|
|||
|
|
@ -192,6 +192,10 @@ class Message(Base):
|
|||
# fall back to path extraction); [] means a new message explicitly published no artifacts.
|
||||
# Kept outside payload so provider-bound conversation messages remain protocol-clean.
|
||||
artifact_refs: Mapped[Optional[list[dict[str, Any]]]] = mapped_column(JSONB, nullable=True)
|
||||
# Structured user input attachments. NULL means legacy; [] means explicitly none.
|
||||
attachment_refs: Mapped[Optional[list[dict[str, Any]]]] = mapped_column(
|
||||
JSONB, nullable=True
|
||||
)
|
||||
tokens_in: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
|
||||
tokens_out: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
|
||||
# 0006:产生该 message 的模型(只在 assistant 行有值;user/tool/system 为 NULL)。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
"""Add structured user attachment references to messages.
|
||||
|
||||
Revision ID: 0031
|
||||
Revises: 0030
|
||||
Create Date: 2026-08-13
|
||||
"""
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision: str = "0031"
|
||||
down_revision: str | None = "0030"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"messages",
|
||||
sa.Column("attachment_refs", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("messages", "attachment_refs")
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
import { parseUserAttachments } from "../web/static/js/attachments.js";
|
||||
|
||||
const chatJs = readFileSync("web/static/js/chat.js", "utf8");
|
||||
|
||||
test("user attachment protocol lines are separated from visible message text", () => {
|
||||
const parsed = parseUserAttachments(
|
||||
"请比较这张图\n\n[用户上传的参考图] 项目/显微照片.png\n[用户上传的文件] 项目/数据表.xlsx",
|
||||
);
|
||||
assert.equal(parsed.content, "请比较这张图");
|
||||
assert.deepEqual(parsed.attachments, [
|
||||
{ path: "项目/显微照片.png", kind: "image" },
|
||||
{ path: "项目/数据表.xlsx", kind: "file" },
|
||||
]);
|
||||
});
|
||||
|
||||
test("attachment-only messages have an empty visible body", () => {
|
||||
const parsed = parseUserAttachments("[用户上传的参考图] 项目/照片 01.png");
|
||||
assert.equal(parsed.content, "");
|
||||
assert.deepEqual(parsed.attachments, [{ path: "项目/照片 01.png", kind: "image" }]);
|
||||
});
|
||||
|
||||
test("ordinary bracketed text remains visible", () => {
|
||||
const content = "[备注] 这不是附件\n继续处理";
|
||||
assert.deepEqual(parseUserAttachments(content), { content, attachments: [] });
|
||||
});
|
||||
|
||||
test("new sends use structured attachments and history keeps a legacy fallback", () => {
|
||||
assert.match(chatJs, /attachments:\s*pendingAttachments\.map/);
|
||||
assert.match(chatJs, /Array\.isArray\(m\.attachment_refs\)/);
|
||||
assert.match(chatJs, /structuredAttachments === null/);
|
||||
assert.doesNotMatch(chatJs, /content = content \? `\$\{content\}\\n\\n\$\{lines\}`/);
|
||||
});
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from core.artifacts import ArtifactPathError
|
||||
from core.attachments import content_for_model, normalize_attachment_refs
|
||||
|
||||
|
||||
class AttachmentRefTests(unittest.TestCase):
|
||||
def test_normalizes_user_relative_path_to_task_relative_metadata(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
wd = root / "实验分析"
|
||||
wd.mkdir()
|
||||
image = wd / "显微照片.png"
|
||||
image.write_bytes(b"png")
|
||||
refs = normalize_attachment_refs(
|
||||
[{"path": "实验分析/显微照片.png", "kind": "image"}],
|
||||
working_dir=wd,
|
||||
user_root=root,
|
||||
)
|
||||
self.assertEqual(refs[0]["path"], "显微照片.png")
|
||||
self.assertEqual(refs[0]["kind"], "image")
|
||||
self.assertEqual(refs[0]["size_bytes"], 3)
|
||||
self.assertEqual(refs[0]["scope"], "working_dir")
|
||||
|
||||
def test_rejects_attachment_outside_working_dir(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
wd = root / "任务"
|
||||
wd.mkdir()
|
||||
(root / "outside.txt").write_text("x", encoding="utf-8")
|
||||
with self.assertRaises(ArtifactPathError):
|
||||
normalize_attachment_refs(
|
||||
[{"path": "../outside.txt", "kind": "file"}],
|
||||
working_dir=wd,
|
||||
user_root=root,
|
||||
)
|
||||
|
||||
def test_builds_provider_content_without_changing_stored_text(self):
|
||||
refs = [{"path": "figures/a.png", "kind": "image"}]
|
||||
self.assertEqual(
|
||||
content_for_model("请分析", refs, "实验分析"),
|
||||
"请分析\n\n[用户上传的参考图] 实验分析/figures/a.png",
|
||||
)
|
||||
self.assertEqual(
|
||||
content_for_model("", refs, "实验分析"),
|
||||
"[用户上传的参考图] 实验分析/figures/a.png",
|
||||
)
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
import unittest
|
||||
|
||||
from pydantic import ValidationError
|
||||
|
||||
from web.schemas import MessageRequest
|
||||
|
||||
|
||||
class MessageAttachmentSchemaTests(unittest.TestCase):
|
||||
def test_omitted_attachments_remains_distinguishable_for_legacy_clients(self):
|
||||
body = MessageRequest(content="旧客户端正文")
|
||||
self.assertEqual(body.attachments, [])
|
||||
self.assertNotIn("attachments", body.model_fields_set)
|
||||
|
||||
def test_explicit_structured_attachments_are_accepted(self):
|
||||
body = MessageRequest(
|
||||
content="分析图片",
|
||||
attachments=[{"path": "任务/a.png", "kind": "image"}],
|
||||
)
|
||||
self.assertIn("attachments", body.model_fields_set)
|
||||
self.assertEqual(body.attachments[0].path, "任务/a.png")
|
||||
|
||||
def test_attachment_count_is_limited(self):
|
||||
with self.assertRaises(ValidationError):
|
||||
MessageRequest(
|
||||
content="太多",
|
||||
attachments=[{"path": f"任务/{i}.txt"} for i in range(11)],
|
||||
)
|
||||
|
|
@ -38,6 +38,7 @@ class ClaimRunTests(unittest.TestCase):
|
|||
with patch.object(run_lifecycle, "session_scope", fake_scope):
|
||||
claim = run_lifecycle.claim_run_with_message(
|
||||
tid, uid, "持久化消息", prepare=prepare,
|
||||
attachment_refs=[{"path": "a.png", "kind": "image"}],
|
||||
)
|
||||
|
||||
self.assertEqual(claim.metadata, {"marker": "ok"})
|
||||
|
|
@ -48,6 +49,7 @@ class ClaimRunTests(unittest.TestCase):
|
|||
added.payload,
|
||||
{"role": "user", "content": "持久化消息"},
|
||||
)
|
||||
self.assertEqual(added.attachment_refs, [{"path": "a.png", "kind": "image"}])
|
||||
update_stmt = session.execute.call_args_list[-1].args[0]
|
||||
params = update_stmt.compile().params
|
||||
self.assertEqual(params["run_status"], "running")
|
||||
|
|
|
|||
|
|
@ -123,7 +123,8 @@ class StaticVendorTests(unittest.TestCase):
|
|||
self.assertIn(".composer-tool .composer-label", css)
|
||||
self.assertIn('aria-label="添加附件"', html)
|
||||
self.assertIn("await ensureTaskFromDraft();", frontend)
|
||||
self.assertIn('"[用户上传的文件]"', frontend)
|
||||
self.assertIn("attachments: pendingAttachments.map", frontend)
|
||||
self.assertIn("parseUserAttachments", frontend)
|
||||
self.assertNotIn("发送第一条消息创建对话后即可添加附件", frontend)
|
||||
self.assertIn('aria-label="语音输入"', html)
|
||||
self.assertIn('aria-label="润色提示词"', html)
|
||||
|
|
|
|||
|
|
@ -100,6 +100,7 @@ def register_message_routes(app, *, require_user) -> None:
|
|||
Message.idx, Message.payload, Message.tokens_in,
|
||||
Message.tokens_out, Message.model_profile, Message.created_at,
|
||||
Message.artifact_refs,
|
||||
Message.attachment_refs,
|
||||
)
|
||||
if limit is None:
|
||||
# 旧行为:升序全量
|
||||
|
|
@ -148,6 +149,7 @@ def register_message_routes(app, *, require_user) -> None:
|
|||
"model_profile": r.model_profile, # 0006:assistant 行非空,标产生该 msg 的模型
|
||||
"created_at": iso(r.created_at),
|
||||
"artifact_refs": r.artifact_refs,
|
||||
"attachment_refs": r.attachment_refs,
|
||||
}
|
||||
for r in rows
|
||||
]
|
||||
|
|
@ -205,7 +207,7 @@ def register_message_routes(app, *, require_user) -> None:
|
|||
headers={"Retry-After": "3"},
|
||||
)
|
||||
content = (body.content or "").strip()
|
||||
if not content:
|
||||
if not content and not body.attachments:
|
||||
raise HTTPException(400, "empty content")
|
||||
# 快捷指令展开(与渠道入口共用 core/shortcuts.py):整条精确命中触发词 → 换成完整
|
||||
# 指令。在起 run 之前、落库之前展开,模型看到的就是完整指令(不进上下文、不问模型)。
|
||||
|
|
@ -219,6 +221,31 @@ def register_message_routes(app, *, require_user) -> None:
|
|||
# running 后才发现参数非法，会留下一个实际上没有 worker 的假活跃任务。
|
||||
image_variant = resolve_image_model(body.image_model, user_id=user_id)
|
||||
video_variant = resolve_video_model(body.video_model, user_id=user_id)
|
||||
# Omitted means a legacy client; explicit [] means a new message has no attachments.
|
||||
attachments_supplied = "attachments" in body.model_fields_set
|
||||
attachment_refs: Optional[list[dict]] = [] if attachments_supplied else None
|
||||
if body.attachments:
|
||||
from core.artifacts import ArtifactPathError
|
||||
from core.attachments import normalize_attachment_refs
|
||||
from core.paths import from_db_path
|
||||
root = load_user_root(user_id)
|
||||
with session_scope() as s:
|
||||
task_working_dir = s.execute(
|
||||
select(Task.working_dir).where(
|
||||
Task.task_id == tid, Task.user_id == user_id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not task_working_dir:
|
||||
raise HTTPException(404, f"task not found: {tid}")
|
||||
working_dir = from_db_path(task_working_dir).resolve()
|
||||
try:
|
||||
attachment_refs = normalize_attachment_refs(
|
||||
body.attachments,
|
||||
working_dir=working_dir,
|
||||
user_root=root,
|
||||
)
|
||||
except ArtifactPathError as e:
|
||||
raise HTTPException(400, str(e))
|
||||
def _prepare_claim(s, task):
|
||||
values: dict = {}
|
||||
# 档位门控:存量 task 的模型已不在用户档位内(如管理员下调了档位)→ 本次起
|
||||
|
|
@ -248,6 +275,7 @@ def register_message_routes(app, *, require_user) -> None:
|
|||
try:
|
||||
claim = claim_run_with_message(
|
||||
tid, user_id, content, prepare=_prepare_claim,
|
||||
attachment_refs=attachment_refs,
|
||||
)
|
||||
except RunTaskNotFound:
|
||||
raise HTTPException(404, f"task not found: {tid}")
|
||||
|
|
@ -276,7 +304,7 @@ def register_message_routes(app, *, require_user) -> None:
|
|||
should_auto_title = claim.metadata["should_auto_title"]
|
||||
if should_auto_title:
|
||||
from core.task_title import is_attachment_only_message
|
||||
should_auto_title = not is_attachment_only_message(content)
|
||||
should_auto_title = bool(content) and not is_attachment_only_message(content)
|
||||
# 快速入口只在首条消息时 pending=true。辅助标题与主 run 并行，不阻塞
|
||||
# TTFT/SSE；纯附件消息保留 pending，等下一条自然语言再命名，避免标题暴露
|
||||
# `[用户上传的文件]` 等内部标记。generate_task_title 内再次查闸并用条件 UPDATE
|
||||
|
|
|
|||
|
|
@ -53,6 +53,7 @@ def claim_run_with_message(
|
|||
user_message: str,
|
||||
*,
|
||||
prepare: Optional[PrepareClaim] = None,
|
||||
attachment_refs: Optional[list[dict[str, Any]]] = None,
|
||||
) -> RunClaim:
|
||||
"""在 task 行锁下原子提交 user 消息和 `run_status=running`。
|
||||
|
||||
|
|
@ -82,6 +83,7 @@ def claim_run_with_message(
|
|||
task_id=task_id,
|
||||
idx=int(next_idx),
|
||||
payload={"role": "user", "content": user_message},
|
||||
attachment_refs=attachment_refs,
|
||||
))
|
||||
values = {
|
||||
"run_status": "running",
|
||||
|
|
|
|||
|
|
@ -33,8 +33,15 @@ class SchedulePatchRequest(BaseModel):
|
|||
enabled: Optional[bool] = None
|
||||
|
||||
|
||||
class MessageAttachmentRequest(BaseModel):
|
||||
path: str
|
||||
kind: str = "file"
|
||||
label: str = ""
|
||||
|
||||
|
||||
class MessageRequest(BaseModel):
|
||||
content: str
|
||||
attachments: list[MessageAttachmentRequest] = Field(default_factory=list, max_length=10)
|
||||
# 该条消息触发的生图 / 生视频模型 variant key(config/media/doubao.yaml image/video 段)。
|
||||
# 空 → 对应 tool 走 yaml 第一个 variant;非空 → 本次 run 装配指定 variant。
|
||||
# 仅作用于本 run,不入 DB,UI 下拉的选择跟在消息 POST body 上。
|
||||
|
|
|
|||
|
|
@ -1132,6 +1132,14 @@
|
|||
width: auto; height: auto; cursor: zoom-in;
|
||||
}
|
||||
.art-media-video { flex: 0 1 720px; width: min(100%, 720px); }
|
||||
/* 用户发出的图片先以紧凑缩略图确认内容，下面仍保留统一附件 chip。 */
|
||||
.user-attachments { margin-top: 6px; }
|
||||
.user-attachments .artifact-bar { margin-top: 4px; }
|
||||
.user-attachments .art-media-image { border-radius: 8px; }
|
||||
.user-attachments .art-media-image img {
|
||||
width: auto; height: auto; max-width: min(240px, 100%); max-height: 180px;
|
||||
object-fit: contain;
|
||||
}
|
||||
.art-media video {
|
||||
display: block; width: 100%; height: auto; max-height: 405px; background: #000;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,19 @@
|
|||
// 新消息使用结构化 attachment_refs；这里只解析旧客户端与存量消息的正文标记。
|
||||
const USER_ATTACHMENT_RE = /^\[(用户上传的参考图|用户上传的文件)\]\s+(.+?)\s*$/;
|
||||
|
||||
export function parseUserAttachments(content) {
|
||||
const attachments = [];
|
||||
const bodyLines = [];
|
||||
for (const line of String(content || "").split("\n")) {
|
||||
const match = line.match(USER_ATTACHMENT_RE);
|
||||
if (!match) {
|
||||
bodyLines.push(line);
|
||||
continue;
|
||||
}
|
||||
attachments.push({
|
||||
path: match[2],
|
||||
kind: match[1] === "用户上传的参考图" ? "image" : "file",
|
||||
});
|
||||
}
|
||||
return { content: bodyLines.join("\n").trim(), attachments };
|
||||
}
|
||||
|
|
@ -33,6 +33,7 @@ import {
|
|||
} from "./preview.js";
|
||||
import { loadFiles, scheduleFilesRefresh, uploadFiles, formatUploadProgress } from "./files.js";
|
||||
import { toolActivityLabel, _workingDirName, extractMediaBanner, extractArtifactRels, renderArtifactBarHtml, renderArtifactChipContent, upgradeMediaArtifacts, ARTIFACT_PRODUCING_TOOLS, _flushMediaArtifactCache } from "./media.js";
|
||||
import { parseUserAttachments } from "./attachments.js";
|
||||
import { applyProgressAction, cloneProgressSteps, progressActionsFromToolCalls } from "./progress.js";
|
||||
import { refreshProcs, decorateBgprocCard, hasRunningProc, killTaskProcs } from "./procs.js";
|
||||
|
||||
|
|
@ -1708,8 +1709,13 @@ function renderMessages(msgs, { stickBottom = true } = {}) {
|
|||
hasVisibleAssistantContent = true;
|
||||
}
|
||||
}
|
||||
if (typeof p.content === "string" && p.content) {
|
||||
html += `<div class="body">${renderMd(p.content)}</div>`;
|
||||
const structuredAttachments = role === "user" && Array.isArray(m.attachment_refs)
|
||||
? m.attachment_refs : null;
|
||||
const userParts = role === "user" && structuredAttachments === null
|
||||
? parseUserAttachments(p.content) : null;
|
||||
const visibleContent = userParts ? userParts.content : p.content;
|
||||
if (typeof visibleContent === "string" && visibleContent) {
|
||||
html += `<div class="body">${renderMd(visibleContent)}</div>`;
|
||||
if (role === "assistant") hasVisibleAssistantContent = true;
|
||||
// assistant 正文里 echo 的 <wd>/... 路径**永远**展示(绕开 seenRels)。图片/视频
|
||||
// 已可能在产物工具结果中内联，仍用 chip 防重复；HTML 通常由 write/shell 产出，
|
||||
|
|
@ -1719,6 +1725,12 @@ function renderMessages(msgs, { stickBottom = true } = {}) {
|
|||
html += renderArtifactBarHtml(extractArtifactRels(p.content, wd), "html", state.taskId || "", true);
|
||||
}
|
||||
}
|
||||
const visibleAttachments = structuredAttachments || (userParts && userParts.attachments) || [];
|
||||
if (visibleAttachments.length) {
|
||||
html += renderUserAttachmentsHtml(
|
||||
visibleAttachments, state.taskId || "", structuredAttachments === null,
|
||||
);
|
||||
}
|
||||
if (role === "assistant" && Array.isArray(m.artifact_refs)) {
|
||||
html += renderArtifactBarHtml(m.artifact_refs, true, state.taskId || "");
|
||||
if (m.artifact_refs.length) hasVisibleAssistantContent = true;
|
||||
|
|
@ -1784,6 +1796,7 @@ function renderMessages(msgs, { stickBottom = true } = {}) {
|
|||
wrap.appendChild(sb);
|
||||
_msgScrollObserver.observe(sb);
|
||||
}
|
||||
|
||||
if (stickBottom) wrap.scrollTop = wrap.scrollHeight;
|
||||
setTaskProgress(state.taskId, currentProgressSteps);
|
||||
upgradeMediaArtifacts(wrap);
|
||||
|
|
@ -1791,6 +1804,18 @@ function renderMessages(msgs, { stickBottom = true } = {}) {
|
|||
renderLiveRunIfVisible();
|
||||
}
|
||||
|
||||
// 用户附件与助手产物语义不同：每个附件都保留可辨认、可点击的 chip；图片另外在
|
||||
// chip 上方给缩略图，兼顾快速确认内容和统一的预览/下载入口。
|
||||
function renderUserAttachmentsHtml(attachments, taskId = "", legacy = false) {
|
||||
if (!attachments || !attachments.length) return "";
|
||||
const images = attachments.filter((item) => item.kind === "image");
|
||||
const refs = attachments.map((item) => ({ path: item.path, label: item.label || "" }));
|
||||
return `<div class="user-attachments">`
|
||||
+ (images.length ? renderArtifactBarHtml(images, true, taskId, legacy) : "")
|
||||
+ renderArtifactBarHtml(refs, false, taskId, legacy)
|
||||
+ `</div>`;
|
||||
}
|
||||
|
||||
// ───── send + SSE ─────
|
||||
// 发送 / 停止 单按钮:idle → 发送(primary 红实心);streaming → 停止(danger 红边);
|
||||
// cancelling 是过渡态 — 用户点过停止后到 SSE 收到 cancelled/done 之间。
|
||||
|
|
@ -2661,16 +2686,9 @@ async function sendMessage(overrideText) {
|
|||
$("chat-hint").textContent = `${_chCfg.label}对话请在${_chCfg.label}里进行 — web 端为只读镜像`;
|
||||
return;
|
||||
}
|
||||
// 附件路径按类型注入正文：图片走视觉/改图工具，其他文件走文档读取、音频转写等工具。
|
||||
const pendingAttachments = fromInput ? takePendingAttachments() : [];
|
||||
const attachmentOnly = !content && pendingAttachments.length > 0;
|
||||
if (pendingAttachments.length) {
|
||||
const lines = pendingAttachments.map(({ rel, kind }) =>
|
||||
`${kind === "image" ? "[用户上传的参考图]" : "[用户上传的文件]"} ${rel}`
|
||||
).join("\n");
|
||||
content = content ? `${content}\n\n${lines}` : lines;
|
||||
}
|
||||
if (!content) return;
|
||||
if (!content && !pendingAttachments.length) return;
|
||||
setActionMode("cancelling"); // 临时锁住,等 events_url 拿到再切 streaming
|
||||
$("chat-hint").textContent = "发送中…";
|
||||
const taskId = state.taskId;
|
||||
|
|
@ -2680,8 +2698,13 @@ async function sendMessage(overrideText) {
|
|||
const wrap = $("chat-stream");
|
||||
const userCard = document.createElement("div");
|
||||
userCard.className = "msg user";
|
||||
userCard.innerHTML = `<div class="role">我</div><div class="body">${escapeHtml(content)}</div>`;
|
||||
userCard.innerHTML = `<div class="role">我</div>`
|
||||
+ (content ? `<div class="body">${escapeHtml(content)}</div>` : "")
|
||||
+ renderUserAttachmentsHtml(
|
||||
pendingAttachments.map(({ rel, kind }) => ({ path: rel, kind })), taskId, true,
|
||||
);
|
||||
wrap.appendChild(userCard);
|
||||
upgradeMediaArtifacts(userCard);
|
||||
|
||||
// assistant 流式占位卡
|
||||
const asstCard = document.createElement("div");
|
||||
|
|
@ -2692,6 +2715,10 @@ async function sendMessage(overrideText) {
|
|||
|
||||
const r = await postMessageWithRetry(taskId, {
|
||||
content,
|
||||
attachments: pendingAttachments.map(({ rel, kind }) => ({
|
||||
path: rel,
|
||||
kind,
|
||||
})),
|
||||
image_model: state.imageModel || "",
|
||||
video_model: state.videoModel || "",
|
||||
});
|
||||
|
|
|
|||
Loading…
Reference in New Issue