feat(models): add GLM-5.3 Flash multimodal support

This commit is contained in:
caoqianming 2026-08-27 14:55:17 +08:00
parent a475207781
commit 9728fc3dd2
29 changed files with 460 additions and 100 deletions

View File

@ -6,6 +6,10 @@
> 开发中的用户文案可先写入 `## Unreleased`;该区不会被前端解析,正式发布时再替换为数字版本和日期。
> 工程口径的完整记录见 `PROGRESS.md` / git log。
## Unreleased
- GLM-5.3 Flash 替换旧版 GLM 并向默认档位开放;上传图片会直接交给主模型理解,长工具任务可延续既有分析状态。
## 0.69.1 — 2026-08-27
- 专业软件任务的进度与结果改为在当前对话左侧集中堆叠,不再用多条结果消息打断正文;可直接打开 Blender 等软件的多张预览,手机端从“软件结果”入口统一查看。

View File

@ -136,7 +136,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
默认 `deepseek_v4.flash`;复杂 bug / 终稿升 pro + reasoning_effort=max;fallback 手动切 Claude。成本量级:修 bug flash ~$0.01 / 完整申报书 flash ~$0.30(pro-max ~$1.5,Opus ~$10+)。99% 任务 flash 够用。
模型思考参数由 profile 统一表达:`thinking_enabled` 只表示开关,`thinking_transport` 只表示已验证的传输协议,`reasoning_effort` 只表示开启后的推理强度,`reasoning_replay` 只表示历史 reasoning 的 provider 回传策略(`none` / `tool_calls` / `all``core/llm_params.py` 是请求参数构造唯一入口,`core/context.py` 是历史消息清洗唯一入口。原始 assistant 响应仍完整落库供展示与导出,发模型前才按 profile 裁剪且上下文统计、压缩与折叠都使用裁剪后的请求视图。DeepSeek、GLM、方舟当前共享 `extra_body` 协议DeepSeek V4 仅为带工具调用的 assistant 消息保留 reasoning未验证网关明确用 `none`、不猜参数协议,主循环不再按 family 分支。`/v1/models` 只返回语义明确的 `thinking_enabled`
模型思考参数由 profile 统一表达:`thinking_enabled` 只表示开关,`thinking_transport` 只表示已验证的传输协议,`reasoning_effort` 只表示开启后的推理强度,`thinking_clear` 表示 provider 是否清除历史思考,`reasoning_replay` 表示状态生命周期(`none` / `tool_turn` / `conversation` / `provider_managed``core/llm_params.py` 是请求参数构造唯一入口,`core/context.py` 是历史消息清洗唯一入口。原始 assistant 响应完整落库provider-bound 副本只向相同生产模型回放未改写的 reasoningDeepSeek V4 仅保留当前用户轮次的工具链状态GLM-5.3 Flash 在同模型会话内保留完整状态,未验证网关明确用 `none`,未来签名/加密 block 走 `provider_managed`。模型切换、上下文折叠和普通压缩都在同一入口应用隔离;上下文统计使用裁剪后的请求视图,原生图片 token 不反向污染 chars/token 校准
---
@ -184,7 +184,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
**对话产物与生命周期(0025/0028/0033)**:真实文件仍是内容事实源;`artifacts` 表记录已发布产物的稳定身份和生命周期,包含 user-root 相对当前路径、来源 task、复制来源、可空的 `software_job_id`、哈希/大小及 active/deleted、回收路径。`software_job_id` 不设外键,非空即表示该正式产物由对应专业软件 Job 生成复制品保留该来源Job 或 task 生命周期结束也不抹除来源标识。新 `messages.artifact_refs` 使用 `{version:2, artifact_id, scope:"working_dir", path:"reports/a.pdf", label?:"最终报告"}``path` 是兼容快照,预览/下载优先按 `artifact_id` 找当前路径因此移动或重命名后历史卡片仍有效。version 1 和 `NULL` 旧消息继续走原 task-scoped 兼容链。普通源码树、中间文件、`.meta/` 技术信息和配套资源不登记agent 仅用 `publish_artifacts` 显式提升少量最终文件。移动保持身份,复制为每个副本创建新身份并记录直接来源;删除将文件移入 `.zcbot_artifact_trash/` 并软删记录,普通文件仍物理删除。
**用户消息附件(0031)**:`messages.attachment_refs` 与助手产物分开表达输入附件,元素为 `{version:1, scope:"working_dir", path, label, kind, media_type, size_bytes}`;文件仍是事实源,不登记为已发布 artifact也不承诺独立生命周期。`payload.content` 只保存用户自然语言,模型上下文在内存中按附件类型补兼容提示,避免 UI 协议污染正文。`NULL` 表示旧客户端/旧历史,前端继续解析正文标记;`[]` 表示新消息明确无附件。独立 attachment 表暂缓,只有出现跨消息复用、稳定身份、版本快照或附件级审计需求时再抽象 `file_assets + message_attachments`
**用户消息附件(0031)**:`messages.attachment_refs` 与助手产物分开表达输入附件,元素为 `{version:1, scope:"working_dir", path, label, kind, media_type, size_bytes}`;文件仍是事实源,不登记为已发布 artifact也不承诺独立生命周期。`payload.content` 只保存用户自然语言,Session 内存补兼容路径提示并保留私有附件引用;发模型前按 `input_modalities` 决定纯文本退化或把图片物化为 OpenAI-compatible Base64 `image_url` blockBase64 永不落库、物化仍经过 working_dir/user_root 越界与大小校验。原生看图模型不注册 `look_at_image``read_document` 继续承担扫描 PDF 等文档摄取。`NULL` 表示旧客户端/旧历史,`[]` 表示新消息明确无附件;独立 attachment 表仍待跨消息复用、稳定身份、版本快照或附件级审计需求出现后再抽象
### 7.2 资源模型(/v1)
@ -316,7 +316,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
### 8.1 图像理解 + Seedream i2i(✅ 2026-06-16)
缺口:主模型纯文本;t2i 无法"改已生成图"或"读上传图"。**选 E+C 组合**:seedream 加 `reference_images` 走 i2i + 新增 `look_at_image`(Doubao Seed 2.0 Lite,一次读图 <¥0.01)让 DeepSeek 自决何时"借眼睛"。**不选 A**(主模型换多模态:V4 code/tool-calling 是核心,换=降能力+改 loop 引 multimodal,工程 5×);**不选 B**(每条消息隐式 vision 路由:烧 token+失 agentic 控制权)。关键实测:ARK 接受 base64 data URL → 内网无需对象存储。**升级到 A 的信号**:用户要"贴图直接对话读图"成高频——当前假设"图是工具调用对象"而非"对话内容"
历史基线是纯文本主模型配 `look_at_image` 借眼睛,生图改图独立走 `reference_images`。2026-08-27 随 GLM-5.3 Flash 原生多模态进入默认档位,升级信号已满足:附件输入改为 capability-driven原生图片模型直接收到用户上传图片纯文本模型继续使用路径提示与辅助视觉工具不会把所有文件或工具产物隐式送入模型避免无关 token、越界读取和缓存污染。生图/改图仍是显式外部动作,原生理解不替代图像生成工具;运行中产生的新图片若未作为后续用户附件加入,不自动注入下一轮
### 8.2 Token 优化与上下文治理(✅ 2026-06-04 起)

View File

@ -2,7 +2,7 @@
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`
最后更新:2026-08-27(专业软件结果轨与 Web 终态收敛bump 0.69.1)
最后更新:2026-08-27(GLM-5.3 Flash、原生图片输入与 reasoning 生命周期统一;未发版)
---
@ -20,6 +20,8 @@
---
## 已完成关键能力
- **08-27 / Unreleased / GLM-5.3 Flash + 通用 reasoning/多模态能力**:下架 GLM-5.1/5.2 的可选入口并以隐藏别名将存量 `glm.pro/pro52` 统一解析到 `glm.flash53`Flash53 加入默认与专业档位;模型能力新增原生输入模态和 `none/tool_turn/conversation/provider_managed` reasoning 生命周期历史状态仅向同一生产模型回放DeepSeek 限当前工具轮、GLM 保留同模型会话并发送 `clear_thinking=false`。结构化图片附件仍以文件引用为事实源,只在 provider 请求边界安全物化为 Base64 `image_url`GLM 不再注册 `look_at_image`,文档读取与生图/视频工具保持独立;无 schema/migration/依赖变化,未调用真实模型或连接生产数据库。
- **08-27 / 0.69.1 / 专业软件结果轨与终态收敛**:当前对话的专业软件 Job 按独立状态点堆叠在聊天左侧,悬停展示名称与进度,成功的 Workspace 任务可直接打开多预览画廊,移动端收敛为“软件结果”入口;主消息流只保留用户与 Agent 叙事,默认 `report` 完成后只更新 Job 账本和视图,显式 `analyze` 才重新进入 Agent 对话。Web SSE 终态统一延后到任务状态落库之后发送,结果刷新会清理不再活动的 live card避免软件已出图但“思考中”被重新点亮。既有历史软件报告消息继续兼容显示无 schema、migration 或 HTTP API 变化。
- **08-26 / 0.69.0 / Origin 科研分析 MVP**:新增非默认的 `origin.analysis@v1` / adapter 0.1.0,与稳定的 `origin.plot@v2` 分离;首批提供数据体检、最大值/面积/参考点归一化、固定局部多项式平滑、一/二阶导数、区间/累计梯形积分和带 95% 置信区间的线性拟合,固定输出可编辑 OPJU、CSV、诊断、分析规格与溯源可选 XLSX线性拟合工程同时保留 Origin 原生报告。六个合成真值用例在 Origin 2024 / originpro 1.1.15 / numpy 2.2.6 上完成生产 Worker、数值 oracle、OPJU 重开和进程释放门;前三例来自组合执行的逐例通过记录,后三例分别通过,目标机 COM 完整退出约需 2 分钟。Origin/合同/Job/Node 专项 159 项 unittest、Release .NET build、完整 Node publish 与独立 adapter ZIP 校验通过,未连接或写入数据库。

2
RUN.md
View File

@ -215,7 +215,7 @@ python -m venv .venv
```bash
# 模型能力对账(费 token)
.venv/Scripts/python.exe main.py probe --model deepseek_v4.flash
.venv/Scripts/python.exe main.py probe --model glm.pro # 智谱 GLM-5.1(走 litellm zai provider + 国内站 bigmodel.cn)
.venv/Scripts/python.exe main.py probe --model glm.flash53 # 智谱 GLM-5.3 Flash(原生多模态,国内站 bigmodel.cn)
.venv/Scripts/python.exe main.py probe --model local.r1 # 内网 DeepSeek-R1(满血,128K),涉密任务用;需 .env 设 LOCAL_LLM_API_KEY
.venv/Scripts/python.exe main.py probe --model local.qwen3 # 内网 Qwen3-30B-A3B(MoE,原生 32K);共享 LOCAL_LLM_API_KEY

View File

@ -13,6 +13,7 @@ model_tiers:
default: # 基线:所有未分配档位的用户(= 公测期默认可用)
- deepseek_v4.flash
- deepseek_v4.pro
- glm.flash53
- local.r1 # 内网模型(涉密任务)
- local.qwen3
- seedream_5 # 图(config/media/doubao.yaml image 段)
@ -26,8 +27,7 @@ model_tiers:
- doubao.turbo
- doubao.pro
- doubao.evolving
- glm.pro
- glm.pro52
- glm.flash53
- unifyllm.fable5 # unifyllm 网关(海外旗舰,走代理出口)
- unifyllm.opus48
- unifyllm.sonnet46

View File

@ -16,7 +16,7 @@ variants:
thinking_transport: extra_body
reasoning_effort_levels: [low, high, max]
default_reasoning_effort: high
reasoning_replay: tool_calls # 普通跨轮剥离;工具调用 reasoning 按官方协议完整回传
reasoning_replay: tool_turn # 只在当前用户轮次的工具链内原样回传 reasoning
code_quality: good
enable_run_python: true
max_iterations: 120 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
@ -42,7 +42,7 @@ variants:
thinking_transport: extra_body
reasoning_effort_levels: [low, medium, high, max]
default_reasoning_effort: medium
reasoning_replay: tool_calls
reasoning_replay: tool_turn
code_quality: excellent
enable_run_python: true
max_iterations: 150 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard

View File

@ -1,55 +1,43 @@
# 智谱 GLM 模型档案
# 走 litellm 原生 zai provider(1.83+ 内置)。litellm 默认 api_base 是国际站 api.z.ai;
# 国内站 bigmodel.cn 通过 yaml 的 api_base 字段覆盖。两站 API key 不通用,env 也分开命名。
# thinking 走统一 extra_body 协议,由本档 `thinking_enabled` 决定开关。当前均 false=禁用 ——
# 因网关侧默认开 thinking,重任务上会把输出预算烧在 reasoning_content 上撞满 65536 上限
# 被截断→空响应(task 35744bea 案);禁用后线上探针实测 reasoning_content 归零、正文照常。
# 走 LiteLLM 原生 zai provider。国内站 bigmodel.cn 与国际站 API key 不通用。
# GLM-5.3-Flash 强制 thinkingzcbot 保留同模型完整 reasoning 以维持长工具链连续性。
family: glm
variants:
pro:
display_name: GLM 5.1
model_id: zai/glm-5.1
flash53:
display_name: GLM-5.3 Flash
model_id: zai/glm-5.3-flash
api_base: https://open.bigmodel.cn/api/paas/v4
api_key_env: ZHIPUAI_API_KEY
max_context: 200000
reliable_context: 100000
max_output: 8192
parallel_tools: false
tool_calling_quality: good
thinking_enabled: false
thinking_transport: extra_body
reasoning_effort_levels: []
default_reasoning_effort: ""
code_quality: good
enable_run_python: true
max_iterations: 50
optimal_temperature: 0.3
prompt_caching: false
extended_thinking: false
# GLM 5.2:与 5.1 并存(新增 variant,不动 glm.pro,线上 task 仍引 5.1 不受影响)。
# 旗舰基座,真正可用的 1M 上下文,适合大仓库/长链路工程任务。thinking 同 pro 留 false(协议同 5.1)。
pro52:
display_name: GLM 5.2
model_id: zai/glm-5.2
api_base: https://open.bigmodel.cn/api/paas/v4
api_key_env: ZHIPUAI_API_KEY
max_context: 1000000 # 真 1M
max_context: 1000000
reliable_context: 262144
max_output: 8192
max_output: 131072
input_modalities: [text, image]
parallel_tools: false
tool_calling_quality: good
thinking_enabled: false
tool_calling_quality: excellent
thinking_enabled: true
thinking_transport: extra_body
reasoning_effort_levels: []
default_reasoning_effort: ""
thinking_clear: false
reasoning_effort_levels: [low, high, max]
default_reasoning_effort: high
reasoning_replay: conversation
code_quality: excellent
enable_run_python: true
max_iterations: 50
optimal_temperature: 0.3
max_iterations: 120
optimal_temperature: 1.0
prompt_caching: false
extended_thinking: false
input_cny_per_mtoken: 8.0
output_cny_per_mtoken: 28.0
cache_hit_cny_per_mtoken: 2.0
# 官方国际站标价按 1 USD ~= 7.2 CNY 折算;促销价不写入长期兜底。
input_cny_per_mtoken: 1.08
output_cny_per_mtoken: 3.6
cache_hit_cny_per_mtoken: 0.216
# 隐藏兼容别名:真实调用、能力和记账身份全部归 glm.flash53。
# 待一个兼容版本确认无旧 profile 活跃后删除。
pro:
hidden: true
alias_of: flash53
pro52:
hidden: true
alias_of: flash53

View File

@ -56,10 +56,12 @@ _MEDIA_LOOK_SEG = """\
- **何时调**:用户消息里出现 `[用户上传的参考图] <路径>` 且需要据图内容回答("这图里写了啥 / 是什么 / 表格数据多少");或要基于 task 内某张图(`figures/xxx.png`)**实际内容**做事(不是改图,改图走 seedream) `image=<路径>`并且**必须在 `question` 里一次写清完成当前任务所需的具体信息**别分多次看
- **问题范围**:只问完成用户任务所需内容例如问仪表读数就只读数问某列数据就只提取该列仅当用户明确要求"全文 OCR / 转文字 / 完整识别整张图"才要求逐字识别全部文字不要无条件同时要求画面描述全文 OCR 和图表解析
- **何时不调**:用户只是要改图( seedream i2i)/ 只要文件名不关心内容 / 图是你自己刚生成的且 prompt 已知(无需再读)/ 图的内容对当前任务可有可无**绝不对同一张图反复看**(每看一次都是几十秒等待)"""
_MEDIA_NATIVE_LOOK_SEG = """\
- **原生看图**:你可以直接理解用户随消息上传的图片图片已作为多模态内容随请求送入直接据图分析**不要调用辅助看图工具**消息中的 `[用户上传的参考图]` 路径仅用于定位同一附件或交给改图工具"""
_MEDIA_READDOC_SEG = """\
- `read_document` PDF(豆包 Seed 2.0 Lite 文档理解),**专治扫描件**:markitdown 对某 PDF 转出** / 近空**(纯图页无文本层) 用它逐页 OCR markdown每页约 1-2 厘钱,单次上限 100 (更长先拆分卷)
- **多页整本 OCR 必传 `save_md`**( `save_md='source/xxx.md'`):全文落文件只返回预览,后续直接 read 那个 md;只问局部内容("第3章指标是什么") `question` 即可不落盘
- **何时不调**:PDF 有文本层且 markitdown 已转出正文(直接用那份 md,别重复花钱);DOCX/PPTX/XLSX( markitdown);单张图片( look_at_image)"""
- **何时不调**:PDF 有文本层且 markitdown 已转出正文(直接用那份 md,别重复花钱);DOCX/PPTX/XLSX( markitdown);单张图片由主模型原生读取或走本轮实际提供的看图能力"""
# PDF 摄取是跨 skill 的基础入口,始终注入,不能依赖 agent 是否正确加载 paper / proposal。
# read_document 只在配了 ARK 时挂载;没挂时扫描件明确停在“需 OCR”,不诱导现场装包或
@ -87,15 +89,20 @@ _MEDIA_SEEDANCE_SEG = """\
- 兜底硬约束:用户没主动要视频就别装饰性生成(比生图更严重的红线);同一目的不满意**绝不连发**(1 次错 = ¥4+60s,连发 2 = ¥8+2min);固定开场图传 `image`,多张主体/产品/场景参考图传 `reference_images`当前不支持 video-to-video"""
def _media_tools_block(has_ark: bool, image_tool: str) -> str:
def _media_tools_block(
has_ark: bool, image_tool: str, native_image_input: bool = False
) -> str:
"""按本 run 实际挂载的媒体工具拼指引段;一个都没挂 → 返 ""(prompt 不出现媒体段)。
image_tool: "seedream" / "gpt_image" / ""(无图像工具)look_at_image seedance
只跟豆包(ARK_API_KEY);图像工具二选一,prompt 只描述实际存在的那个
"""
segs: list[str] = []
if native_image_input:
segs.append(_MEDIA_NATIVE_LOOK_SEG)
if has_ark:
segs.append(_MEDIA_LOOK_SEG)
if not native_image_input:
segs.append(_MEDIA_LOOK_SEG)
segs.append(_MEDIA_READDOC_SEG)
if image_tool == "seedream":
segs.append(_MEDIA_SEEDREAM_SEG)
@ -107,7 +114,7 @@ def _media_tools_block(has_ark: bool, image_tool: str) -> str:
segs.append(_MEDIA_SEEDANCE_SEG)
if not segs:
return ""
return "## 媒体工具(图像生成 / 视频生成 / 看图)\n" + "\n".join(segs)
return "## 媒体能力(图像生成 / 视频生成 / 看图)\n" + "\n".join(segs)
def _choose_image_variant(
@ -525,6 +532,7 @@ def build_agent(
model = cfg["default_model"]
caps = ModelCapabilities.load(model, ROOT / cfg["models_dir"])
model = caps.profile
llm = LLM(caps)
workspace_dir = resolve_workspace(workspace, cfg)
@ -603,7 +611,9 @@ def build_agent(
system_prompt = _build_system_prompt(
cfg, skills, workspace_dir, tool_base, working_dir_path, uid,
task_id, task_state.name, task_state.skill,
media_block=_media_tools_block(ark_cfg is not None, image_tool_name),
media_block=_media_tools_block(
ark_cfg is not None, image_tool_name, caps.native_image_input
),
allow_working_dir_rename=not scheduled_run,
office_to_pdf_available=office_pdf_available,
)
@ -671,6 +681,7 @@ def build_agent(
if current_profile.split(".", 1)[0] == "unifyllm":
return None
new_caps = ModelCapabilities.load(profile, ROOT / cfg["models_dir"])
profile = new_caps.profile
new_llm = LLM(new_caps) # 缺 api key 会 raise → loop 捕获转 warn,不切换
from sqlalchemy import update as _update
from core.storage import session_scope as _scope

View File

@ -1,7 +1,9 @@
"""Structured user-message attachments and model-context compatibility rendering."""
from __future__ import annotations
import base64
import mimetypes
from copy import deepcopy
from collections.abc import Iterable
from pathlib import Path
from typing import Any
@ -10,6 +12,14 @@ from .artifacts import ArtifactPathError, resolve_artifact_path
ATTACHMENT_REF_VERSION = 1
MAX_ATTACHMENTS_PER_MESSAGE = 10
MAX_NATIVE_IMAGE_BYTES = 10 * 1024 * 1024
_NATIVE_IMAGE_MIME = {
".gif": "image/gif",
".jpeg": "image/jpeg",
".jpg": "image/jpeg",
".png": "image/png",
".webp": "image/webp",
}
_IMAGE_EXTENSIONS = {
".avif", ".bmp", ".gif", ".heic", ".heif", ".jpeg", ".jpg",
".png", ".svg", ".tif", ".tiff", ".webp",
@ -74,3 +84,60 @@ def content_for_model(content: str, refs: Iterable[dict], working_dir_name: str)
return text
suffix = "\n".join(lines)
return f"{text}\n\n{suffix}" if text else suffix
def materialize_native_images(
messages: Iterable[dict],
*,
enabled: bool,
working_dir: Path,
user_root: Path,
) -> tuple[list[dict], int]:
"""把内存附件引用按需物化为 OpenAI-compatible image_url blocks。
Base64 只存在于本次 provider-bound 副本不写数据库历史文件已删除越界
格式不受支持或超过上限时保留原有文字路径提示并跳过图片块不能让旧附件阻断续聊
所有 ``_`` 内部字段在返回前剥离
"""
output: list[dict] = []
image_count = 0
for message in messages:
new_msg = deepcopy(message)
refs = list(new_msg.pop("_attachment_refs", []) or [])
new_msg.pop("_model_profile", None)
if not enabled or new_msg.get("role") != "user" or not refs:
output.append(new_msg)
continue
blocks: list[dict] = []
text = new_msg.get("content")
if isinstance(text, str) and text:
blocks.append({"type": "text", "text": text})
message_image_count = 0
for ref in refs:
if not isinstance(ref, dict) or ref.get("kind") != "image":
continue
raw_path = str(ref.get("path") or "")
try:
path, _ = resolve_artifact_path(
raw_path,
working_dir=working_dir,
user_root=user_root,
require_file=True,
)
mime = _NATIVE_IMAGE_MIME.get(path.suffix.lower())
if mime is None or path.stat().st_size > MAX_NATIVE_IMAGE_BYTES:
continue
data = base64.b64encode(path.read_bytes()).decode("ascii")
except (ArtifactPathError, OSError):
continue
blocks.append({
"type": "image_url",
"image_url": {"url": f"data:{mime};base64,{data}"},
})
image_count += 1
message_image_count += 1
if message_image_count:
new_msg["content"] = blocks
output.append(new_msg)
return output, image_count

View File

@ -3,13 +3,26 @@ from __future__ import annotations
from dataclasses import dataclass, field, fields
from pathlib import Path
from typing import List
from typing import List, Optional
import yaml
from .llm_params import THINKING_TRANSPORTS
REASONING_REPLAY_POLICIES = {"none", "tool_calls", "all"}
REASONING_REPLAY_POLICIES = {
"none",
"tool_turn",
"conversation",
"provider_managed",
}
def model_profile_of(caps: object) -> str:
"""返回能力对象的 canonical profile兼容测试和旧调用方的鸭子类型。"""
profile = getattr(caps, "profile", "")
if profile:
return str(profile)
return f"{getattr(caps, 'family', '')}.{getattr(caps, 'variant', '')}".strip(".")
@dataclass
@ -34,10 +47,16 @@ class ModelCapabilities:
thinking_transport: str = "none"
reasoning_effort_levels: List[str] = field(default_factory=list)
default_reasoning_effort: str = ""
# 是否清除历史 thinking。None=不发送该 provider 可选字段False=保留历史推理。
thinking_clear: Optional[bool] = None
# 历史 reasoning 发回模型的策略。持久化原始响应与 provider-bound 输入分离:
# none=全部剥离tool_calls=仅工具调用 assistant 保留all=完整原样回传。
# none=剥离tool_turn=仅当前工具链conversation=同模型会话;
# provider_managed=同模型不透明签名/block。
reasoning_replay: str = "none"
# 原生输入模态。附件仍以文件引用落库,发模型前才物化成 provider content blocks。
input_modalities: List[str] = field(default_factory=lambda: ["text"])
# 代码 / 沙盒
code_quality: str = "good"
enable_run_python: bool = False
@ -81,6 +100,16 @@ class ModelCapabilities:
)
var = dict(variants[variant])
# 已下架 profile 可保留为一版隐藏别名,确保存量 task / scheduled job 能续跑;
# 真正能力和记账身份统一归目标 variant不维持旧模型双轨。
seen_aliases = {variant}
while var.get("alias_of"):
target = str(var["alias_of"]).strip()
if not target or target in seen_aliases or target not in variants:
raise ValueError(f"档案 {path} 的 variant={variant} alias_of 无效")
seen_aliases.add(target)
variant = target
var = dict(variants[target])
valid_keys = {f.name for f in fields(cls)}
kwargs = {k: v for k, v in var.items() if k in valid_keys}
kwargs["family"] = data.get("family", family)
@ -109,3 +138,11 @@ class ModelCapabilities:
f"{caps.default_reasoning_effort!r} 不在 reasoning_effort_levels 中"
)
return caps
@property
def profile(self) -> str:
return f"{self.family}.{self.variant}"
@property
def native_image_input(self) -> bool:
return "image" in self.input_modalities

View File

@ -14,6 +14,8 @@ from typing import Any, List
import json
import re
from .capabilities import REASONING_REPLAY_POLICIES
# chars↔tokens 粗折算(CJK+代码+json 混合保守按 ~2.5 char/token)与压缩触发比例。
# loop(50% 压缩门槛)与 context_fold(85% 折叠门槛)共用,单一事实源;压缩/折叠是
# 成本/安全优化、非正确性关键,估算粗糙无妨。
@ -40,29 +42,41 @@ _REASONING_FIELDS = {
def filter_reasoning_for_replay(
messages: List[dict[str, Any]],
policy: str = "none",
target_model_profile: str = "",
) -> tuple[List[dict[str, Any]], dict[str, int]]:
"""构造 provider-bound 消息副本,按能力档案裁剪历史 reasoning。
原始 Session/DB payload 不动继续供 UI导出和排障使用工具调用需要 reasoning
连续性的 provider 可用 ``tool_calls``未声明能力的 provider 默认 ``none``避免
跨模型发送私有状态保留时把 LiteLLM 可能放在 provider_specific_fields 里的
reasoning_content 提升到 DeepSeek/OpenAI-compatible 所需的顶层字段
``tool_turn`` 只保留当前用户轮次内工具链的状态``conversation`` 保留同模型
会话状态``provider_managed`` 保留同模型的不透明签名/block未声明能力默认
``none``只要历史消息的生产模型与目标模型不同就无条件剥离私有状态
"""
if policy not in {"none", "tool_calls", "all"}:
if policy not in REASONING_REPLAY_POLICIES:
raise ValueError(f"unsupported reasoning replay policy: {policy!r}")
last_user = max(
(i for i, msg in enumerate(messages) if msg.get("role") == "user"),
default=-1,
)
filtered: List[dict[str, Any]] = []
stripped_messages = 0
stripped_chars = 0
for message in messages:
for idx, message in enumerate(messages):
new_msg = deepcopy(message)
if policy == "all":
filtered.append(new_msg)
continue
source_profile = str(new_msg.pop("_model_profile", "") or "")
compatible = not target_model_profile or source_profile == target_model_profile
preserve = (
policy == "tool_calls"
compatible
and new_msg.get("role") == "assistant"
and bool(new_msg.get("tool_calls"))
and (
policy in {"conversation", "provider_managed"}
or (
policy == "tool_turn"
and idx > last_user
and bool(new_msg.get("tool_calls"))
)
)
)
if preserve:
provider_fields = new_msg.get("provider_specific_fields")
@ -242,6 +256,7 @@ def prepare_messages_for_llm(
old_tool_chars: int = 2_000,
compact_threshold_chars: int = 0,
reasoning_replay: str = "none",
target_model_profile: str = "",
) -> List[dict[str, Any]]:
"""返回发给 LLM 的 messages 副本。
@ -257,6 +272,7 @@ def prepare_messages_for_llm(
old_tool_chars=old_tool_chars,
compact_threshold_chars=compact_threshold_chars,
reasoning_replay=reasoning_replay,
target_model_profile=target_model_profile,
)
return prepared
@ -268,6 +284,7 @@ def prepare_messages_with_stats(
old_tool_chars: int = 2_000,
compact_threshold_chars: int = 0,
reasoning_replay: str = "none",
target_model_profile: str = "",
) -> tuple[List[dict[str, Any]], dict[str, int]]:
"""返回发给 LLM 的 messages 副本和压缩统计。
@ -279,7 +296,9 @@ def prepare_messages_with_stats(
"""
if keep_recent < 0:
keep_recent = 0
messages, reasoning_stats = filter_reasoning_for_replay(messages, reasoning_replay)
messages, reasoning_stats = filter_reasoning_for_replay(
messages, reasoning_replay, target_model_profile
)
# 先补齐被中断 run 留下的悬空 tool_calls(否则原样发给模型会被拒,见函数注释)。
messages, repaired_tool_calls = _repair_dangling_tool_calls(messages)
original_chars = sum(_message_chars(m) for m in messages)

View File

@ -25,6 +25,7 @@ from uuid import UUID
from sqlalchemy import update
from .capabilities import model_profile_of
from .context import (
CHARS_PER_TOKEN,
COMPACT_CONTEXT_RATIO,
@ -144,7 +145,10 @@ def maybe_fold(
"""
msgs = session.messages
reasoning_replay = getattr(caps, "reasoning_replay", "none")
provider_msgs, _ = filter_reasoning_for_replay(msgs, reasoning_replay)
target_profile = model_profile_of(caps)
provider_msgs, _ = filter_reasoning_for_replay(
msgs, reasoning_replay, target_profile
)
before_chars = sum(_message_chars(m) for m in provider_msgs)
est_tokens = estimate_window_tokens(provider_msgs, session.last_measured_usage())
if est_tokens < caps.reliable_context * FOLD_TRIGGER_RATIO:
@ -164,6 +168,7 @@ def maybe_fold(
msgs,
compact_threshold_chars=compact_threshold,
reasoning_replay=reasoning_replay,
target_model_profile=target_profile,
)
p_cut = _kth_user_index(prepared, _user_ordinal(msgs, cutoff))
if p_cut is None: # 理论不可达,防御
@ -200,7 +205,7 @@ def maybe_fold(
persist_fold(session.task_id, new_base, summary)
session.apply_fold(cutoff, summary)
after_provider_msgs, _ = filter_reasoning_for_replay(
session.messages, reasoning_replay
session.messages, reasoning_replay, target_profile
)
after_chars = sum(_message_chars(m) for m in after_provider_msgs)
stats = {

View File

@ -54,9 +54,15 @@ class LLM:
parallel_tool_calls: Optional[bool],
reasoning_effort: Optional[str],
) -> dict:
# Session 私有元数据永不进入 provider 请求。原生图片由主 loop 在此之前
# 物化;折叠/标题等旁路则安全退化为纯文本路径提示。
provider_messages = [
{k: v for k, v in message.items() if not str(k).startswith("_")}
for message in messages
]
kwargs: dict = {
"model": self.caps.model_id,
"messages": messages,
"messages": provider_messages,
"temperature": self.caps.optimal_temperature,
"api_key": self.api_key,
"timeout": _REQUEST_TIMEOUT_S,
@ -72,6 +78,7 @@ class LLM:
enabled=self.caps.thinking_enabled,
transport=self.caps.thinking_transport,
reasoning_effort=reasoning_effort,
clear_thinking=self.caps.thinking_clear,
)
)
if self.caps.prompt_caching:

View File

@ -7,7 +7,11 @@ THINKING_TRANSPORTS = {"none", "extra_body"}
def build_thinking_kwargs(
*, enabled: bool, transport: str, reasoning_effort: str | None
*,
enabled: bool,
transport: str,
reasoning_effort: str | None,
clear_thinking: bool | None = None,
) -> dict[str, Any]:
"""把统一的 thinking 配置转换为 LiteLLM 调用参数。
@ -22,9 +26,12 @@ def build_thinking_kwargs(
f"不支持的 thinking_transport={transport!r};可选: {sorted(THINKING_TRANSPORTS)}"
)
body: dict[str, Any] = {
"thinking": {"type": "enabled" if enabled else "disabled"}
thinking: dict[str, Any] = {
"type": "enabled" if enabled else "disabled"
}
if enabled and clear_thinking is not None:
thinking["clear_thinking"] = clear_thinking
body: dict[str, Any] = {"thinking": thinking}
if enabled and reasoning_effort:
body["reasoning_effort"] = reasoning_effort
return {"extra_body": body}

View File

@ -22,7 +22,8 @@ import litellm
from . import pptx_guard
from .artifacts import MAX_ARTIFACTS_PER_MESSAGE
from .capabilities import ModelCapabilities
from .attachments import materialize_native_images
from .capabilities import ModelCapabilities, model_profile_of
from .context import (
CHARS_PER_TOKEN,
COMPACT_CONTEXT_RATIO,
@ -312,20 +313,21 @@ class AgentLoop:
assert response is not None
msg = response.choices[0].message
tool_calls = getattr(msg, "tool_calls", None) or []
model_profile = model_profile_of(self.caps)
asst_msg_id = self.session.append(
msg,
artifact_refs=(list(self._pending_artifact_refs) if not tool_calls else None),
source_model_profile=model_profile,
)
usage_details = extract_usage_details(getattr(response, "usage", None))
pt, ct = usage_details["tokens_in"], usage_details["tokens_out"]
# 用本轮实报 prompt_tokens 刷新 chars/token 校准比值(下一轮门槛/占用环即用)。
if pt > 0 and self._last_sent_chars > 0:
if pt > 0 and self._last_sent_chars > 0 and not self._last_had_native_images:
self._ctx_chars_per_token = clamp_ratio(self._last_sent_chars / pt)
# 记账(0006):一行 usage_event + 回填 messages.tokens_in/out + model_profile。
# 任何失败都吞掉(litellm cost map miss / DB 异常),不阻塞主 loop;
# message 仍在 session/DB 里,后续重启不影响。
model_profile = f"{self.caps.family}.{self.caps.variant}"
try:
record_chat_usage(
task_id=self.session.task_id,
@ -456,6 +458,7 @@ class AgentLoop:
# 无实测时的回退(详 context.py 常数注释)。
_ctx_chars_per_token: Optional[float] = None
_last_sent_chars: int = 0
_last_had_native_images: bool = False
def _context_ratio(self) -> float:
"""当前窗口的 chars/token 换算比值(校准态)。
@ -470,6 +473,7 @@ class AgentLoop:
provider_messages, _ = filter_reasoning_for_replay(
self.session.messages,
getattr(self.caps, "reasoning_replay", "none"),
model_profile_of(self.caps),
)
self._ctx_chars_per_token = calibrated_chars_per_token(
provider_messages, self.session.last_measured_usage()
@ -514,7 +518,18 @@ class AgentLoop:
self.session.messages,
compact_threshold_chars=compact_threshold,
reasoning_replay=getattr(self.caps, "reasoning_replay", "none"),
target_model_profile=model_profile_of(self.caps),
)
native_images = 0
if self.user_root is not None:
llm_messages, native_images = materialize_native_images(
llm_messages,
enabled=self.caps.native_image_input,
working_dir=self.working_dir,
user_root=self.user_root,
)
self._last_had_native_images = native_images > 0
context_stats["native_images"] = native_images
self._last_sent_chars = context_stats.get("sent_chars", 0)
llm_start_event = {
"type": "llm_start",

View File

@ -6,7 +6,7 @@
- 某档成员里出现 `"*"` 该档全开(含未来新增模型),给内部档用
模型 id 约定( list 端点 / resolve 校验一致):
- 文本模型 = `family.variant`(config/models/<family>.yaml), `doubao.pro``glm.pro52`
- 文本模型 = `family.variant`(config/models/<family>.yaml), `doubao.pro``glm.flash53`
- / 视频模型 = variant key(config/media/doubao.yaml), `seedream_5``seedance_2_fast`
两者命名不冲突(文本带点媒体 variant 不带点),同一档集合里混放即可

View File

@ -65,6 +65,7 @@ class Session:
msg: Any,
*,
artifact_refs: Optional[list[dict]] = None,
source_model_profile: str = "",
) -> Optional[UUID]:
"""追加消息;非 system 落 DB,system 仅内存。返回新落库行的 message_id。
@ -91,7 +92,10 @@ class Session:
f"[markdown:fence-warning] task={self.task_id} unclosed=1",
flush=True,
)
self.messages.append(msg_dict)
memory_msg = dict(msg_dict)
if source_model_profile and memory_msg.get("role") == "assistant":
memory_msg["_model_profile"] = source_model_profile
self.messages.append(memory_msg)
if msg_dict.get("role") == "system":
return None
@ -236,6 +240,9 @@ class Session:
attachment_refs,
wd_name,
)
payload["_attachment_refs"] = list(attachment_refs)
if payload.get("role") == "assistant" and row.model_profile:
payload["_model_profile"] = row.model_profile
if payload.get("role") == "assistant" and isinstance(payload.get("content"), str):
# 历史行不回写生产库;只在重建 LLM 上下文时应用同一窄修复,
# 与 Web 展示层保持一致,避免旧坏围栏继续污染后续轮次。

View File

@ -269,17 +269,24 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
cancel_check=ctx.cancel_check, daily_limit=videos_per_day, **base,
)]
def _vision() -> list:
def _vision_kwargs() -> Optional[dict]:
# 看图 + 文档理解共用 vision variant(读图不分档,固定第一个)。
key, v = _pick_variant(ctx.ark_cfg.raw.get("vision") or {})
if v is None:
return []
kw = dict(
return None
return dict(
ark_cfg=ctx.ark_cfg, vision_variant_cfg=v, variant_key=key,
working_dir=ctx.working_dir_path, task_id=ctx.task_id, user_id=ctx.uid,
**base,
)
return [LookAtImageTool(**kw), ReadDocumentTool(**kw)]
def _look_at_image() -> list:
kw = _vision_kwargs()
return [LookAtImageTool(**kw)] if kw is not None else []
def _read_document() -> list:
kw = _vision_kwargs()
return [ReadDocumentTool(**kw)] if kw is not None else []
def _transcribe() -> list:
# 录音文件转写(讯飞 LFASR,与 IAT 听写是两个服务两套 key)。
@ -316,7 +323,13 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
("office_to_pdf", lambda: ctx.office_to_pdf_available, _office_to_pdf),
("image", lambda: ctx.img_cfg is not None, _image),
("video", lambda: ctx.ark_cfg is not None, _video),
("vision", lambda: ctx.ark_cfg is not None, _vision),
(
"look_at_image",
lambda: ctx.ark_cfg is not None
and not getattr(ctx.caps, "native_image_input", False),
_look_at_image,
),
("read_document", lambda: ctx.ark_cfg is not None, _read_document),
("transcribe_audio", lfasr_configured, _transcribe),
("web_search", lambda: BochaConfig.load() is not None, _web_search),
]

View File

@ -3,7 +3,11 @@ import unittest
from pathlib import Path
from core.artifacts import ArtifactPathError
from core.attachments import content_for_model, normalize_attachment_refs
from core.attachments import (
content_for_model,
materialize_native_images,
normalize_attachment_refs,
)
class AttachmentRefTests(unittest.TestCase):
@ -47,3 +51,27 @@ class AttachmentRefTests(unittest.TestCase):
content_for_model("", refs, "实验分析"),
"[用户上传的参考图] 实验分析/figures/a.png",
)
def test_materializes_native_image_without_persisting_internal_fields(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
wd = root / "任务"
wd.mkdir()
(wd / "a.png").write_bytes(b"png")
messages = [{
"role": "user",
"content": "请分析\n\n[用户上传的参考图] 任务/a.png",
"_attachment_refs": [{"path": "a.png", "kind": "image"}],
}]
prepared, count = materialize_native_images(
messages, enabled=True, working_dir=wd, user_root=root,
)
self.assertEqual(count, 1)
self.assertNotIn("_attachment_refs", prepared[0])
self.assertEqual(prepared[0]["content"][0]["type"], "text")
self.assertEqual(prepared[0]["content"][1]["type"], "image_url")
self.assertTrue(
prepared[0]["content"][1]["image_url"]["url"].startswith(
"data:image/png;base64,"
)
)

View File

@ -58,7 +58,7 @@ class ContextCompactionTests(unittest.TestCase):
prepared, stats = prepare_messages_with_stats(
messages,
reasoning_replay="tool_calls",
reasoning_replay="tool_turn",
compact_threshold_chars=10_000,
)
@ -78,7 +78,7 @@ class ContextCompactionTests(unittest.TestCase):
}, {"role": "tool", "tool_call_id": "tc1", "content": "ok"}]
prepared = prepare_messages_for_llm(
messages, reasoning_replay="tool_calls",
messages, reasoning_replay="tool_turn",
)
self.assertEqual(prepared[0]["reasoning_content"], "nested reasoning")
@ -91,10 +91,54 @@ class ContextCompactionTests(unittest.TestCase):
"provider_specific_fields": {"signature": "opaque"},
}
prepared = prepare_messages_for_llm([message], reasoning_replay="all")
prepared = prepare_messages_for_llm(
[message], reasoning_replay="conversation"
)
self.assertEqual(prepared[0], message)
def test_conversation_policy_strips_reasoning_from_other_model(self) -> None:
messages = [{
"role": "assistant",
"content": "answer",
"reasoning_content": "deepseek reasoning",
"_model_profile": "deepseek_v4.flash",
}]
prepared = prepare_messages_for_llm(
messages,
reasoning_replay="conversation",
target_model_profile="glm.flash53",
)
self.assertNotIn("reasoning_content", prepared[0])
self.assertNotIn("_model_profile", prepared[0])
def test_tool_turn_keeps_only_reasoning_after_latest_user(self) -> None:
call = lambda cid, reasoning: {
"role": "assistant", "content": None,
"reasoning_content": reasoning,
"_model_profile": "deepseek_v4.flash",
"tool_calls": [{
"id": cid, "type": "function",
"function": {"name": "shell", "arguments": "{}"},
}],
}
messages = [
{"role": "user", "content": "old"},
call("old", "old reasoning"),
{"role": "tool", "tool_call_id": "old", "content": "old result"},
{"role": "assistant", "content": "done"},
{"role": "user", "content": "new"},
call("new", "new reasoning"),
{"role": "tool", "tool_call_id": "new", "content": "new result"},
]
prepared = prepare_messages_for_llm(
messages,
reasoning_replay="tool_turn",
target_model_profile="deepseek_v4.flash",
)
self.assertNotIn("reasoning_content", prepared[1])
self.assertEqual(prepared[5]["reasoning_content"], "new reasoning")
def test_preserves_system_and_recent_messages(self) -> None:
messages = [
{"role": "system", "content": "rules"},

View File

@ -146,7 +146,7 @@ class _FakeLLM:
_FAKE_CAPS = SimpleNamespace(
reliable_context=100, # 极小上限 → 体量必然超阈值,不用造几十万字符
family="deepseek_v4", variant="flash",
reasoning_replay="tool_calls",
reasoning_replay="tool_turn",
input_cny_per_mtoken=1.0, output_cny_per_mtoken=2.0,
)
@ -227,7 +227,7 @@ class MaybeFoldTests(unittest.TestCase):
self.assertEqual(sess.messages, before) # 内存零污染
self.assertTrue(any(e.get("type") == "warn" for e in events))
def test_fold_input_strips_final_reasoning_but_keeps_tool_reasoning(self) -> None:
def test_fold_input_strips_reasoning_from_completed_turns(self) -> None:
sess = self._session()
sess.messages[2]["reasoning_content"] = "final reasoning"
sess.messages[4] = {
@ -254,7 +254,7 @@ class MaybeFoldTests(unittest.TestCase):
final_msg = next(m for m in sent if m.get("content", "").startswith("a000"))
tool_msg = next(m for m in sent if m.get("tool_calls"))
self.assertNotIn("reasoning_content", final_msg)
self.assertEqual(tool_msg["reasoning_content"], "tool reasoning")
self.assertNotIn("reasoning_content", tool_msg)
if __name__ == "__main__":

View File

@ -75,6 +75,22 @@ class LLMKwargsTests(unittest.TestCase):
kwargs["extra_body"], {"thinking": {"type": "enabled"}}
)
def test_glm_preserved_thinking_is_nested_in_thinking_body(self) -> None:
caps = ModelCapabilities(
family="glm", variant="flash53", model_id="zai/glm-5.3-flash",
api_key_env="TEST_LLM_API_KEY", thinking_enabled=True,
thinking_transport="extra_body", thinking_clear=False,
default_reasoning_effort="high",
)
with patch.dict(os.environ, {"TEST_LLM_API_KEY": "test-key"}):
kwargs = LLM(caps)._build_kwargs(
[{"role": "user", "content": "hello"}], None, None, "high"
)
self.assertEqual(kwargs["extra_body"], {
"thinking": {"type": "enabled", "clear_thinking": False},
"reasoning_effort": "high",
})
def test_invalid_transport_is_rejected_before_request(self) -> None:
llm = self._llm(
family="test", thinking_enabled=True, thinking_transport="unknown"
@ -97,7 +113,7 @@ class LLMKwargsTests(unittest.TestCase):
self.assertEqual(caps.output_cny_per_mtoken, 2.0)
self.assertEqual(caps.cache_hit_cny_per_mtoken, 0.02)
self.assertEqual(caps.thinking_transport, "extra_body")
self.assertEqual(caps.reasoning_replay, "tool_calls")
self.assertEqual(caps.reasoning_replay, "tool_turn")
def test_other_controllable_profiles_declare_transport(self) -> None:
models_dir = Path(__file__).resolve().parents[1] / "config" / "models"
@ -105,8 +121,12 @@ class LLMKwargsTests(unittest.TestCase):
glm = ModelCapabilities.load("glm.pro52", models_dir)
doubao = ModelCapabilities.load("doubao.turbo", models_dir)
self.assertFalse(glm.thinking_enabled)
self.assertEqual(glm.profile, "glm.flash53")
self.assertTrue(glm.thinking_enabled)
self.assertEqual(glm.thinking_transport, "extra_body")
self.assertFalse(glm.thinking_clear)
self.assertEqual(glm.reasoning_replay, "conversation")
self.assertTrue(glm.native_image_input)
self.assertTrue(doubao.thinking_enabled)
self.assertEqual(doubao.thinking_transport, "extra_body")
self.assertEqual(doubao.default_reasoning_effort, "")

View File

@ -1,9 +1,12 @@
"""look_at_image 的问题收敛与兼容兜底测试(不碰网络和数据库)。"""
from __future__ import annotations
import tempfile
import unittest
import uuid
from contextlib import ExitStack
from pathlib import Path
from types import SimpleNamespace
from unittest import mock
from core.ark_client import ArkConfig
@ -27,6 +30,69 @@ class LookAtImageQuestionTests(unittest.TestCase):
user_root=Path("."),
)
def test_native_vision_model_omits_look_tool_but_keeps_document_ocr(self):
from core.tool_registry import ToolContext, build_tools
ark = ArkConfig(
api_key="test",
base_url="https://example.invalid",
raw={"vision": {"test": {"model_id": "vision-test"}}},
)
def build(native_image_input: bool) -> dict:
with tempfile.TemporaryDirectory() as tmp, ExitStack() as stack:
root = Path(tmp)
stack.enter_context(mock.patch(
"core.tool_registry._external_system_status_available",
return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry._external_systems_available",
return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.smtp_configured", return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.wechat_push_available", return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.lfasr_configured", return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.BochaConfig.load", return_value=None,
))
return build_tools(ToolContext(
tool_base=root,
ur_path=root,
working_dir_path=root,
task_id=uuid.uuid4(),
uid=uuid.uuid4(),
cfg={},
caps=SimpleNamespace(
enable_run_python=False,
native_image_input=native_image_input,
),
skills=SimpleNamespace(skills={}),
cancel_check=None,
scheduled_run=True,
deferred_actions=SimpleNamespace(),
ark_cfg=ark,
img_provider="",
img_key="",
img_cfg=None,
img_provider_cfg=None,
video_variant="",
office_to_pdf_available=False,
))
text_only = build(False)
native = build(True)
self.assertIn("look_at_image", text_only)
self.assertIn("read_document", text_only)
self.assertNotIn("look_at_image", native)
self.assertIn("read_document", native)
def _execute_and_question(self, question=None) -> str:
captured = {}

View File

@ -155,7 +155,9 @@ def _model_catalog() -> list[dict]:
except Exception:
continue
family = data.get("family") or path.stem
for variant in (data.get("variants") or {}).keys():
for variant, variant_cfg in (data.get("variants") or {}).items():
if isinstance(variant_cfg, dict) and variant_cfg.get("hidden"):
continue
profile = f"{family}.{variant}"
try:
caps = ModelCapabilities.load(profile, models_dir)

View File

@ -70,8 +70,9 @@ def resolve_model_profile(profile: str, user_id: Optional[UUID] = None) -> tuple
caps = ModelCapabilities.load(name, ROOT / cfg["models_dir"])
except (FileNotFoundError, ValueError) as e:
raise HTTPException(400, f"invalid model_profile {name!r}: {e}")
assert_model_allowed(name, user_id, "text")
return name, caps.model_id
canonical = caps.profile
assert_model_allowed(canonical, user_id, "text")
return canonical, caps.model_id
def skill_pinned_profiles() -> set:

View File

@ -261,6 +261,16 @@ def register_message_routes(app, *, require_user) -> None:
cur_profile = task.model_profile or ""
if cur_profile:
from core.model_access import is_allowed
# 旧模型 profile 先解析成隐藏兼容别名的正式目标,避免被误判成档外
# 后降到 DeepSeek本次持久化 canonical profile后续直接走新模型。
try:
canonical_profile, canonical_model = resolve_model_profile(cur_profile)
except HTTPException:
canonical_profile, canonical_model = cur_profile, task.model
if canonical_profile != cur_profile:
values["model_profile"] = canonical_profile
values["model"] = canonical_model
cur_profile = canonical_profile
urow = s.execute(
select(User.plan, User.role).where(User.user_id == user_id)
).first()
@ -511,6 +521,10 @@ def register_message_routes(app, *, require_user) -> None:
cfg = load_config()
chosen_profile = task_model_profile or cfg["default_model"]
try:
chosen_profile, _ = resolve_model_profile(chosen_profile)
except HTTPException:
pass
# 档位门控:task 存量模型已不在用户档位内 → 润色也落回 flash(与 send 路径一致,
# 不持久改 task,仅本次润色调用降级)。
if chosen_profile and not model_allowed_for_user(chosen_profile, user_id):

View File

@ -41,7 +41,9 @@ def register_model_routes(app, *, require_user) -> None:
except Exception:
continue
family = data.get("family") or path.stem
for variant in (data.get("variants") or {}).keys():
for variant, variant_cfg in (data.get("variants") or {}).items():
if isinstance(variant_cfg, dict) and variant_cfg.get("hidden"):
continue
profile = f"{family}.{variant}"
if allowed is not None and profile not in allowed:
continue

View File

@ -322,6 +322,7 @@ def register_task_routes(app, *, require_user) -> None:
window_messages,
compact_threshold_chars=compact_threshold,
reasoning_replay=caps.reasoning_replay,
target_model_profile=caps.profile,
)
window_chars = context_stats["original_chars"]
except Exception:

View File

@ -950,7 +950,7 @@ function renderModelDropdown(t) {
let opts = state.models.map(m =>
`<option value="${escapeHtml(m.profile)}" ${m.profile === cur ? "selected" : ""}>${escapeHtml(m.display_name)}</option>`
).join("");
// 当前模型不在该用户档位清单里(skill 定向模型,如 ppt → glm.pro52)→ 补一项,
// 当前模型不在该用户档位清单里(skill 定向模型)→ 补一项,
// 下拉如实显示;用户仍可切回档内模型。
if (cur && !state.models.some(m => m.profile === cur)) {
opts += `<option value="${escapeHtml(cur)}" selected>${escapeHtml(cur)}</option>`;