feat(models): unify thinking controls

This commit is contained in:
caoqianming 2026-08-03 14:36:50 +08:00
parent dfda6fd772
commit bc45753f77
15 changed files with 241 additions and 58 deletions

View File

@ -5,6 +5,10 @@
> 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。 > 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。
> 工程口径的完整记录见 `PROGRESS.md` / git log。 > 工程口径的完整记录见 `PROGRESS.md` / git log。
## 0.60.26 — 2026-08-03
- 默认 DeepSeek Flash 已使用官方 0731 API 升级,继续展示思考过程并显式控制推理强度;不同模型的思考开关不再依赖服务端默认值,切换模型时行为更稳定。
## 0.60.25 — 2026-08-03 ## 0.60.25 — 2026-08-03
- HTML 产物现在可以在对话中直接显示进入可视区域后自动加载交互内容也可放大到弹窗查看HTML 与 Markdown 预览均可在渲染结果和源文件之间切换,常见 HTTPS 图表、地图及网页资源可正常加载。 - HTML 产物现在可以在对话中直接显示进入可视区域后自动加载交互内容也可放大到弹窗查看HTML 与 Markdown 预览均可在渲染结果和源文件之间切换,常见 HTTPS 图表、地图及网页资源可正常加载。

View File

@ -108,6 +108,8 @@ Session = 消息列表,ORM 直写 PG `messages`(append-only,jsonb 存 LiteLLM
默认 `deepseek_v4.flash`;复杂 bug / 终稿升 pro + reasoning_effort=max;fallback 手动切 Claude。成本量级:修 bug flash ~$0.01 / 完整申报书 flash ~$0.30(pro-max ~$1.5,Opus ~$10+)。99% 任务 flash 够用。 默认 `deepseek_v4.flash`;复杂 bug / 终稿升 pro + reasoning_effort=max;fallback 手动切 Claude。成本量级:修 bug flash ~$0.01 / 完整申报书 flash ~$0.30(pro-max ~$1.5,Opus ~$10+)。99% 任务 flash 够用。
模型思考参数由 profile 统一表达:`thinking_enabled` 只表示开关,`thinking_transport` 只表示已验证的传输协议,`reasoning_effort` 只表示开启后的推理强度;`core/llm_params.py` 是请求构造唯一入口。DeepSeek、GLM、方舟当前共享 `extra_body` 协议,未验证网关明确用 `none`、不猜参数协议,主循环不再按 family 分支。`/v1/models` 只返回语义明确的 `thinking_enabled`
--- ---
## 5. 设计哲学 ## 5. 设计哲学

View File

@ -2,7 +2,7 @@
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9` > 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`
最后更新:2026-08-03(交互式 HTML 预览 + 对话内嵌,bump 0.60.25) 最后更新:2026-08-03(DeepSeek Flash-0731 + thinking 参数统一,bump 0.60.26)
--- ---
@ -23,6 +23,7 @@
### 2026-08-03 ### 2026-08-03
- **08-03 / 0.60.26 / DeepSeek Flash-0731 + thinking 参数统一**:默认 `deepseek-v4-flash` 无需换模型 ID 即接入官方 0731 后训练升级Flash 改为显式开启 thinking 并透传 `reasoning_effort=high`,校准当前常规时段 token 成本但保留 8K 稳定输出预算。模型档案统一用 `thinking_enabled`(开关)+`thinking_transport`(协议)+`reasoning_effort`(强度)DeepSeek/GLM/方舟共用纯函数请求构造,移除 family 分支与旧 `thinking_mode` 字段方舟保持既有思考开启GLM 保持生产验证过的显式关闭,未验证网关标记 `none`。`/v1/models` 同步只返回新字段445 项 unittest 全绿(17 skip)Ruff 与 diff 检查通过;未连生产 DB、未发真实模型请求无 schema/migration/依赖变化。
- **08-03 / 0.60.25 / 交互式 HTML 预览 + 对话内嵌**:文件预览将 HTML 从普通源码提升为可切换“预览 / 源文件”的 sandbox iframe允许脚本与 HTTPS CDN/接口但保持 opaque origin禁止宿主权限、表单和顶层跳转助手最终答复中的 HTML 产物改为进入可视区才加载的内嵌卡片并可放大复用完整预览Markdown 同步补源文件切换。Node 14 项、Python 27 项、JavaScript 语法及 diff 检查通过;当前环境无可用浏览器实例,真实页面点击/截图留部署后冒烟;无 schema、migration、HTTP API 或依赖变化。 - **08-03 / 0.60.25 / 交互式 HTML 预览 + 对话内嵌**:文件预览将 HTML 从普通源码提升为可切换“预览 / 源文件”的 sandbox iframe允许脚本与 HTTPS CDN/接口但保持 opaque origin禁止宿主权限、表单和顶层跳转助手最终答复中的 HTML 产物改为进入可视区才加载的内嵌卡片并可放大复用完整预览Markdown 同步补源文件切换。Node 14 项、Python 27 项、JavaScript 语法及 diff 检查通过;当前环境无可用浏览器实例,真实页面点击/截图留部署后冒烟;无 schema、migration、HTTP API 或依赖变化。
- **08-03 / 0.60.24 / Web Mermaid 直出 + Markdown 围栏容错**:模型偶发用同长度围栏嵌套 Markdown/Mermaid 示例CommonMark 会把后续正文吞进未闭合代码块;新增仅针对该明确形态的前后端确定性修复,提示词统一要求外层使用更长异类围栏,历史上下文加载时同样修正且不批量回写生产数据。聊天页本地 vendoring Mermaid 11.16.0,仅在助手文字段定稿后顺序渲染 `language-mermaid`,采用 strict 安全级别、文本/边数上限,语法错误或组件不可用时保留源码并提示;真实 Edge 冒烟确认中文流程图与 XYChart 柱线组合图可生成 SVG。Python 27 项、Node 9 项、Ruff、JS/Python 语法及 diff 检查通过;无 schema、migration、HTTP API 或 Python 依赖变化。 - **08-03 / 0.60.24 / Web Mermaid 直出 + Markdown 围栏容错**:模型偶发用同长度围栏嵌套 Markdown/Mermaid 示例CommonMark 会把后续正文吞进未闭合代码块;新增仅针对该明确形态的前后端确定性修复,提示词统一要求外层使用更长异类围栏,历史上下文加载时同样修正且不批量回写生产数据。聊天页本地 vendoring Mermaid 11.16.0,仅在助手文字段定稿后顺序渲染 `language-mermaid`,采用 strict 安全级别、文本/边数上限,语法错误或组件不可用时保留源码并提示;真实 Edge 冒烟确认中文流程图与 XYChart 柱线组合图可生成 SVG。Python 27 项、Node 9 项、Ruff、JS/Python 语法及 diff 检查通过;无 schema、migration、HTTP API 或 Python 依赖变化。
- **08-03 / 0.60.23 / Office→PDF 组件感知 + 展示路径兼容**:生产 task `92ac20cf` 暴露两层问题:上传消息给出 user-root 相对的 `测试pdf/x.doc`host tool 又按 task_dir 拼接导致首次找不到改用裸文件名后host 仅安装 `libreoffice-impress` 却因只检测 `soffice` 而错误宣称支持 DOC最终 Writer 导入返回 `source file could not be loaded`。现 `office_to_pdf` 同时解析 task 相对、user-root 展示路径与 `/workspace` 路径Debian/Ubuntu 按 Writer/Calc/Impress 实际安装包缓存支持后缀,工具 schema 只声明可用格式缺组件在启动转换前给出明确管理员提示。RUN bootstrap 与故障表同步要求 host 安装三组件;相关 30 项 unittest、Python 编译及 diff 格式检查通过,本机无 LibreOffice真实 `.doc/.xlsx/.pptx` 冒烟留部署 host无 schema、migration、HTTP API 或 Python 依赖变化,无需重建沙箱镜像。 - **08-03 / 0.60.23 / Office→PDF 组件感知 + 展示路径兼容**:生产 task `92ac20cf` 暴露两层问题:上传消息给出 user-root 相对的 `测试pdf/x.doc`host tool 又按 task_dir 拼接导致首次找不到改用裸文件名后host 仅安装 `libreoffice-impress` 却因只检测 `soffice` 而错误宣称支持 DOC最终 Writer 导入返回 `source file could not be loaded`。现 `office_to_pdf` 同时解析 task 相对、user-root 展示路径与 `/workspace` 路径Debian/Ubuntu 按 Writer/Calc/Impress 实际安装包缓存支持后缀,工具 schema 只声明可用格式缺组件在启动转换前给出明确管理员提示。RUN bootstrap 与故障表同步要求 host 安装三组件;相关 30 项 unittest、Python 编译及 diff 格式检查通过,本机无 LibreOffice真实 `.doc/.xlsx/.pptx` 冒烟留部署 host无 schema、migration、HTTP API 或 Python 依赖变化,无需重建沙箱镜像。

View File

@ -9,21 +9,23 @@ variants:
api_key_env: DEEPSEEK_API_KEY api_key_env: DEEPSEEK_API_KEY
max_context: 1048576 max_context: 1048576
reliable_context: 262144 reliable_context: 262144
max_output: 8192 max_output: 8192 # 官方上限 384Kzcbot 先保留稳定输出预算,压测后再放大
parallel_tools: false parallel_tools: false
tool_calling_quality: good tool_calling_quality: good
thinking_mode: false thinking_enabled: true
reasoning_effort_levels: [] thinking_transport: extra_body
default_reasoning_effort: "" reasoning_effort_levels: [low, high, max]
default_reasoning_effort: high
code_quality: good code_quality: good
enable_run_python: true enable_run_python: true
max_iterations: 120 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard max_iterations: 120 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
optimal_temperature: 0.3 optimal_temperature: 0.3
prompt_caching: false prompt_caching: false
extended_thinking: false extended_thinking: false
# Flash-0731 官方美元价按 1 USD ~= 7.2 CNY 折算;峰谷价正式生效后再同步。
input_cny_per_mtoken: 1.0 input_cny_per_mtoken: 1.0
output_cny_per_mtoken: 12.0 output_cny_per_mtoken: 2.0
cache_hit_cny_per_mtoken: 0.1 # DeepSeek 前缀缓存命中价(input 的 ~0.1x) cache_hit_cny_per_mtoken: 0.02 # 官方 $0.0028 / M tokens
pro: pro:
display_name: DeepSeek V4 Pro display_name: DeepSeek V4 Pro
@ -35,7 +37,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: true thinking_enabled: true
thinking_transport: extra_body
reasoning_effort_levels: [low, medium, high, max] reasoning_effort_levels: [low, medium, high, max]
default_reasoning_effort: medium default_reasoning_effort: medium
code_quality: excellent code_quality: excellent

View File

@ -3,10 +3,8 @@
# 与 config/models/local.yaml 同范式(避免 litellm volcengine provider 的版本/字段差异)。 # 与 config/models/local.yaml 同范式(避免 litellm volcengine provider 的版本/字段差异)。
# api_key 复用媒体侧的 ARK_API_KEY(同一火山账号),env 见 RUN.md。 # api_key 复用媒体侧的 ARK_API_KEY(同一火山账号),env 见 RUN.md。
# #
# thinking_mode 暂设 false:Seed 2.1 是深度思考模型,但开关走 Ark body `thinking:{type:enabled}`, # Seed 2.1 的 thinking 走统一 extra_body 协议;保持既有深度思考行为,但由服务端默认
# 与 OpenAI/DeepSeek 的 `reasoning_effort` 等级协议不同 —— 同 glm.yaml 的处理,要 core/llm.py # 改为显式 enabled。方舟未声明 effort 等级,故只传 thinking、不猜 reasoning_effort。
# 加 family 分支才能透传等级,留 TODO。设 false 只是不发 reasoning_effort 字段;模型默认仍会
# 深度思考并返回 reasoning_content,不影响调用。
# 单价见各 variant(元/百万 tokens,来源:火山方舟 2026-06 发布价)。 # 单价见各 variant(元/百万 tokens,来源:火山方舟 2026-06 发布价)。
family: doubao family: doubao
@ -21,7 +19,8 @@ variants:
max_output: 16384 # 模型上限 128K(含思考),这里保守取值,需要长输出可调高 max_output: 16384 # 模型上限 128K(含思考),这里保守取值,需要长输出可调高
parallel_tools: true # Ark 兼容 parallel_tool_calls,默认 true parallel_tools: true # Ark 兼容 parallel_tool_calls,默认 true
tool_calling_quality: good tool_calling_quality: good
thinking_mode: false thinking_enabled: true
thinking_transport: extra_body
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: good code_quality: good
@ -44,7 +43,8 @@ variants:
max_output: 16384 # 模型上限 128K(含思考),这里保守取值,需要长输出可调高 max_output: 16384 # 模型上限 128K(含思考),这里保守取值,需要长输出可调高
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: false thinking_enabled: true
thinking_transport: extra_body
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent
@ -69,7 +69,8 @@ variants:
max_output: 16384 max_output: 16384
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: false thinking_enabled: true
thinking_transport: extra_body
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent

View File

@ -1,9 +1,7 @@
# 智谱 GLM 模型档案 # 智谱 GLM 模型档案
# 走 litellm 原生 zai provider(1.83+ 内置)。litellm 默认 api_base 是国际站 api.z.ai; # 走 litellm 原生 zai provider(1.83+ 内置)。litellm 默认 api_base 是国际站 api.z.ai;
# 国内站 bigmodel.cn 通过 yaml 的 api_base 字段覆盖。两站 API key 不通用,env 也分开命名。 # 国内站 bigmodel.cn 通过 yaml 的 api_base 字段覆盖。两站 API key 不通用,env 也分开命名。
# thinking 已接(core/llm.py _build_kwargs 的 family=="glm" 分支):GLM 协议是 body # thinking 走统一 extra_body 协议,由本档 `thinking_enabled` 决定开关。当前均 false=禁用 ——
# `{"thinking":{"type":"enabled|disabled"}}`(与 OpenAI/DeepSeek 的 `reasoning_effort`
# 等级不同族),走 extra_body 透传,由本档 `thinking_mode` 决定开关。当前均 false=禁用 ——
# 因网关侧默认开 thinking,重任务上会把输出预算烧在 reasoning_content 上撞满 65536 上限 # 因网关侧默认开 thinking,重任务上会把输出预算烧在 reasoning_content 上撞满 65536 上限
# 被截断→空响应(task 35744bea 案);禁用后线上探针实测 reasoning_content 归零、正文照常。 # 被截断→空响应(task 35744bea 案);禁用后线上探针实测 reasoning_content 归零、正文照常。
family: glm family: glm
@ -19,7 +17,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: false parallel_tools: false
tool_calling_quality: good tool_calling_quality: good
thinking_mode: false thinking_enabled: false
thinking_transport: extra_body
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: good code_quality: good
@ -41,7 +40,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: false parallel_tools: false
tool_calling_quality: good tool_calling_quality: good
thinking_mode: false thinking_enabled: false
thinking_transport: extra_body
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent

View File

@ -2,7 +2,7 @@
# 走 OpenAI 兼容协议(litellm provider 前缀 `openai/`,后段为实际 model 字段透传给 base_url)。 # 走 OpenAI 兼容协议(litellm provider 前缀 `openai/`,后段为实际 model 字段透传给 base_url)。
# 涉密任务时用户显式选 local.r1 / local.qwq 代替默认 deepseek_v4.flash;不走自动路由。 # 涉密任务时用户显式选 local.r1 / local.qwq 代替默认 deepseek_v4.flash;不走自动路由。
# 两个 variant 共用同一台推理服务器(api_base 同),api_key_env 也共用 LOCAL_LLM_API_KEY。 # 两个 variant 共用同一台推理服务器(api_base 同),api_key_env 也共用 LOCAL_LLM_API_KEY。
# thinking_mode=false:R1 / QwQ 是天生推理模型,默认就思考,不通过 reasoning_effort 等级控制 # thinking_enabled=false:R1 / QwQ 是天生推理模型,默认就思考,不通过 reasoning_effort 等级控制
# (那是 OpenAI / DeepSeek V4 风格);设 true 会发 reasoning_effort 字段,本地 vLLM / sglang # (那是 OpenAI / DeepSeek V4 风格);设 true 会发 reasoning_effort 字段,本地 vLLM / sglang
# 多半不认,报 400。 # 多半不认,报 400。
family: local family: local
@ -18,7 +18,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: false parallel_tools: false
tool_calling_quality: fair tool_calling_quality: fair
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: good code_quality: good
@ -41,7 +42,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: false parallel_tools: false
tool_calling_quality: fair tool_calling_quality: fair
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: good code_quality: good

View File

@ -23,7 +23,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent
@ -43,7 +44,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent
@ -63,7 +65,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent
@ -83,7 +86,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent
@ -103,7 +107,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: true parallel_tools: true
tool_calling_quality: excellent tool_calling_quality: excellent
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent
@ -123,7 +128,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: true parallel_tools: true
tool_calling_quality: good tool_calling_quality: good
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: good code_quality: good
@ -143,7 +149,8 @@ variants:
max_output: 8192 max_output: 8192
parallel_tools: false # gemini 走网关未实测该参数,保守关闭 parallel_tools: false # gemini 走网关未实测该参数,保守关闭
tool_calling_quality: good tool_calling_quality: good
thinking_mode: false thinking_enabled: false
thinking_transport: none
reasoning_effort_levels: [] reasoning_effort_levels: []
default_reasoning_effort: "" default_reasoning_effort: ""
code_quality: excellent code_quality: excellent

View File

@ -1,3 +1,3 @@
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。 # zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
# 改版本只动这一行。 # 改版本只动这一行。
__version__ = "0.60.25" __version__ = "0.60.26"

View File

@ -7,6 +7,8 @@ from typing import List
import yaml import yaml
from .llm_params import THINKING_TRANSPORTS
@dataclass @dataclass
class ModelCapabilities: class ModelCapabilities:
@ -24,8 +26,10 @@ class ModelCapabilities:
parallel_tools: bool = False parallel_tools: bool = False
tool_calling_quality: str = "good" tool_calling_quality: str = "good"
# 思考模式 # 思考开关
thinking_mode: bool = False thinking_enabled: bool = False
# none=不猜 provider 默认值extra_body=显式发送 thinking.typeeffort 同体透传。
thinking_transport: str = "none"
reasoning_effort_levels: List[str] = field(default_factory=list) reasoning_effort_levels: List[str] = field(default_factory=list)
default_reasoning_effort: str = "" default_reasoning_effort: str = ""
@ -71,9 +75,27 @@ class ModelCapabilities:
f"档案 {path} 没有 variant={variant};可选: {list(variants)}" f"档案 {path} 没有 variant={variant};可选: {list(variants)}"
) )
var = variants[variant] var = dict(variants[variant])
valid_keys = {f.name for f in fields(cls)} valid_keys = {f.name for f in fields(cls)}
kwargs = {k: v for k, v in var.items() if k in valid_keys} kwargs = {k: v for k, v in var.items() if k in valid_keys}
kwargs["family"] = data.get("family", family) kwargs["family"] = data.get("family", family)
kwargs["variant"] = variant kwargs["variant"] = variant
return cls(**kwargs) caps = cls(**kwargs)
if caps.thinking_transport not in THINKING_TRANSPORTS:
raise ValueError(
f"档案 {path} 的 thinking_transport={caps.thinking_transport!r} 无效;"
f"可选: {sorted(THINKING_TRANSPORTS)}"
)
if caps.thinking_enabled and caps.thinking_transport == "none":
raise ValueError(
f"档案 {path} 开启 thinking 时必须声明可验证的 thinking_transport"
)
if (
caps.default_reasoning_effort
and caps.default_reasoning_effort not in caps.reasoning_effort_levels
):
raise ValueError(
f"档案 {path} 的 default_reasoning_effort="
f"{caps.default_reasoning_effort!r} 不在 reasoning_effort_levels 中"
)
return caps

View File

@ -25,6 +25,7 @@ from litellm.exceptions import (
) )
from .capabilities import ModelCapabilities from .capabilities import ModelCapabilities
from .llm_params import build_thinking_kwargs
# 单次 LLM 请求超时(秒),默认与 litellm 一致(600s)但显式化 + env 可调 ── # 单次 LLM 请求超时(秒),默认与 litellm 一致(600s)但显式化 + env 可调 ──
# 长思考模型真被掐("600s 无字节 → run 标 error")时调大 ZCBOT_LLM_TIMEOUT_S 即可, # 长思考模型真被掐("600s 无字节 → run 标 error")时调大 ZCBOT_LLM_TIMEOUT_S 即可,
@ -64,18 +65,13 @@ class LLM:
kwargs["tools"] = tools kwargs["tools"] = tools
if self.caps.parallel_tools and parallel_tool_calls is not False: if self.caps.parallel_tools and parallel_tool_calls is not False:
kwargs["parallel_tool_calls"] = True kwargs["parallel_tool_calls"] = True
if self.caps.thinking_mode and reasoning_effort: kwargs.update(
kwargs["reasoning_effort"] = reasoning_effort build_thinking_kwargs(
# GLM(zai)的 thinking 网关侧默认开着 —— 重任务上把整个输出预算烧在 reasoning_content enabled=self.caps.thinking_enabled,
# 上、撞满模型自带输出上限(65536)被截断,回来 content 空 + 无 tool_call,被 loop 判空 transport=self.caps.thinking_transport,
# 响应整轮丢弃 + 无效重试(task 35744bea 案:5 次 empty 全 tokens_out=65536,重试同上下文 reasoning_effort=reasoning_effort,
# 再撞)。GLM 的 thinking 协议是 body {"thinking":{"type":...}}(与 OpenAI 的 )
# reasoning_effort 等级协议不同族),走 extra_body 透传 —— 线上探针实测 disabled 后 )
# reasoning_content 归零、正文/工具照常。config thinking_mode 决定开关(当前 glm 档均 false)。
if self.caps.family == "glm":
kwargs["extra_body"] = {
"thinking": {"type": "enabled" if self.caps.thinking_mode else "disabled"}
}
if self.caps.prompt_caching: if self.caps.prompt_caching:
kwargs["extra_headers"] = {"anthropic-beta": "prompt-caching-2024-07-31"} kwargs["extra_headers"] = {"anthropic-beta": "prompt-caching-2024-07-31"}
return kwargs return kwargs

30
core/llm_params.py Normal file
View File

@ -0,0 +1,30 @@
"""跨 provider 的可选 LLM 请求参数构造。"""
from __future__ import annotations
from typing import Any
THINKING_TRANSPORTS = {"none", "extra_body"}
def build_thinking_kwargs(
*, enabled: bool, transport: str, reasoning_effort: str | None
) -> dict[str, Any]:
"""把统一的 thinking 配置转换为 LiteLLM 调用参数。
``none`` 表示该模型没有经过验证的显式控制协议不猜 provider 默认值
``extra_body`` 对应当前 DeepSeekGLM 与方舟 ChatCompletions 的共同协议
effort 仅在开启且档案提供非空值时发送
"""
if transport == "none":
return {}
if transport != "extra_body":
raise ValueError(
f"不支持的 thinking_transport={transport!r};可选: {sorted(THINKING_TRANSPORTS)}"
)
body: dict[str, Any] = {
"thinking": {"type": "enabled" if enabled else "disabled"}
}
if enabled and reasoning_effort:
body["reasoning_effort"] = reasoning_effort
return {"extra_body": body}

View File

@ -6,7 +6,7 @@
四项探测: 四项探测:
- basic_chat:连通性失败则跳过其余 - basic_chat:连通性失败则跳过其余
- parallel_tools:给两个独立工具, single response tool_calls 数量 - parallel_tools:给两个独立工具, single response tool_calls 数量
- thinking_mode: declared=True 的模型传 reasoning_effort, API 是否接受 + 是否产出 thinking - thinking: enabled=True 的模型传统一参数, API 是否接受 + 是否产出 thinking
- long_context(opt-in):needle-in-haystack 简化版,默认探 reliable_context 1/8 - long_context(opt-in):needle-in-haystack 简化版,默认探 reliable_context 1/8
""" """
from __future__ import annotations from __future__ import annotations
@ -133,11 +133,11 @@ def probe_parallel_tools(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
) )
def probe_thinking_mode(llm: LLM, caps: ModelCapabilities) -> ProbeResult: def probe_thinking(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
declared = caps.thinking_mode declared = caps.thinking_enabled
if not declared: if not declared:
return ProbeResult( return ProbeResult(
name="thinking_mode", name="thinking",
declared=False, declared=False,
observed=None, observed=None,
status="skip", status="skip",
@ -145,7 +145,7 @@ def probe_thinking_mode(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
) )
effort = ( effort = (
caps.default_reasoning_effort caps.default_reasoning_effort
or (caps.reasoning_effort_levels[0] if caps.reasoning_effort_levels else "medium") or (caps.reasoning_effort_levels[0] if caps.reasoning_effort_levels else None)
) )
try: try:
resp = llm.chat( resp = llm.chat(
@ -162,22 +162,22 @@ def probe_thinking_mode(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
) )
observed = bool(rc) observed = bool(rc)
return ProbeResult( return ProbeResult(
name="thinking_mode", name="thinking",
declared=True, declared=True,
observed=observed, observed=observed,
status="ok" if observed else "mismatch", status="ok" if observed else "mismatch",
detail=( detail=(
f"reasoning_effort={effort} accepted; " (f"reasoning_effort={effort} accepted; " if effort else "thinking enabled; ")
+ ("thinking content returned" if observed else "no thinking content in response") + ("thinking content returned" if observed else "no thinking content in response")
), ),
) )
except Exception as e: except Exception as e:
return ProbeResult( return ProbeResult(
name="thinking_mode", name="thinking",
declared=True, declared=True,
observed=False, observed=False,
status="mismatch", status="mismatch",
detail=f"reasoning_effort rejected: {type(e).__name__}: {e}", detail=f"thinking parameters rejected: {type(e).__name__}: {e}",
) )
@ -237,7 +237,7 @@ def probe_capabilities(
if report.results[0].status == "error": if report.results[0].status == "error":
return report return report
report.add(probe_parallel_tools(llm, caps)) report.add(probe_parallel_tools(llm, caps))
report.add(probe_thinking_mode(llm, caps)) report.add(probe_thinking(llm, caps))
if include_long_context: if include_long_context:
report.add(probe_long_context(llm, caps)) report.add(probe_long_context(llm, caps))
return report return report

115
tests/test_llm_kwargs.py Normal file
View File

@ -0,0 +1,115 @@
import os
import unittest
from pathlib import Path
from unittest.mock import patch
from core.capabilities import ModelCapabilities
from core.llm import LLM
class LLMKwargsTests(unittest.TestCase):
def _llm(
self, *, family: str, thinking_enabled: bool, thinking_transport: str
) -> LLM:
caps = ModelCapabilities(
family=family,
model_id=f"{family}/model",
api_key_env="TEST_LLM_API_KEY",
thinking_enabled=thinking_enabled,
thinking_transport=thinking_transport,
optimal_temperature=0.3,
)
with patch.dict(os.environ, {"TEST_LLM_API_KEY": "test-key"}):
return LLM(caps)
def test_deepseek_explicitly_enables_thinking_and_sets_effort(self) -> None:
llm = self._llm(
family="deepseek_v4", thinking_enabled=True, thinking_transport="extra_body"
)
kwargs = llm._build_kwargs(
[{"role": "user", "content": "hello"}], None, None, "high"
)
self.assertNotIn("reasoning_effort", kwargs)
self.assertEqual(
kwargs["extra_body"],
{"thinking": {"type": "enabled"}, "reasoning_effort": "high"},
)
def test_deepseek_explicitly_disables_thinking_without_effort(self) -> None:
llm = self._llm(
family="deepseek_v4", thinking_enabled=False, thinking_transport="extra_body"
)
kwargs = llm._build_kwargs(
[{"role": "user", "content": "hello"}], None, None, "high"
)
self.assertNotIn("reasoning_effort", kwargs)
self.assertEqual(
kwargs["extra_body"], {"thinking": {"type": "disabled"}}
)
def test_other_openai_compatible_provider_gets_no_thinking_body(self) -> None:
llm = self._llm(
family="unifyllm", thinking_enabled=False, thinking_transport="none"
)
kwargs = llm._build_kwargs(
[{"role": "user", "content": "hello"}], None, None, None
)
self.assertNotIn("extra_body", kwargs)
def test_extra_body_transport_enables_thinking_without_effort(self) -> None:
llm = self._llm(
family="doubao", thinking_enabled=True, thinking_transport="extra_body"
)
kwargs = llm._build_kwargs(
[{"role": "user", "content": "hello"}], None, None, None
)
self.assertEqual(
kwargs["extra_body"], {"thinking": {"type": "enabled"}}
)
def test_invalid_transport_is_rejected_before_request(self) -> None:
llm = self._llm(
family="test", thinking_enabled=True, thinking_transport="unknown"
)
with self.assertRaisesRegex(ValueError, "thinking_transport"):
llm._build_kwargs(
[{"role": "user", "content": "hello"}], None, None, "high"
)
def test_flash_profile_matches_0731_capabilities(self) -> None:
caps = ModelCapabilities.load(
"deepseek_v4.flash", Path(__file__).resolve().parents[1] / "config" / "models"
)
self.assertTrue(caps.thinking_enabled)
self.assertEqual(caps.reasoning_effort_levels, ["low", "high", "max"])
self.assertEqual(caps.default_reasoning_effort, "high")
self.assertEqual(caps.max_output, 8192)
self.assertEqual(caps.output_cny_per_mtoken, 2.0)
self.assertEqual(caps.cache_hit_cny_per_mtoken, 0.02)
self.assertEqual(caps.thinking_transport, "extra_body")
def test_other_controllable_profiles_declare_transport(self) -> None:
models_dir = Path(__file__).resolve().parents[1] / "config" / "models"
glm = ModelCapabilities.load("glm.pro52", models_dir)
doubao = ModelCapabilities.load("doubao.turbo", models_dir)
self.assertFalse(glm.thinking_enabled)
self.assertEqual(glm.thinking_transport, "extra_body")
self.assertTrue(doubao.thinking_enabled)
self.assertEqual(doubao.thinking_transport, "extra_body")
self.assertEqual(doubao.default_reasoning_effort, "")
if __name__ == "__main__":
unittest.main()

View File

@ -54,7 +54,7 @@ def register_model_routes(app, *, require_user) -> None:
"display_name": caps.display_name or profile, "display_name": caps.display_name or profile,
"family": caps.family, "family": caps.family,
"variant": caps.variant, "variant": caps.variant,
"thinking_mode": caps.thinking_mode, "thinking_enabled": caps.thinking_enabled,
"is_default": profile == default, "is_default": profile == default,
}) })
return {"models": out} return {"models": out}