feat(models): unify thinking controls
This commit is contained in:
parent
dfda6fd772
commit
bc45753f77
|
|
@ -5,6 +5,10 @@
|
|||
> 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。
|
||||
> 工程口径的完整记录见 `PROGRESS.md` / git log。
|
||||
|
||||
## 0.60.26 — 2026-08-03
|
||||
|
||||
- 默认 DeepSeek Flash 已使用官方 0731 API 升级，继续展示思考过程并显式控制推理强度；不同模型的思考开关不再依赖服务端默认值，切换模型时行为更稳定。
|
||||
|
||||
## 0.60.25 — 2026-08-03
|
||||
|
||||
- HTML 产物现在可以在对话中直接显示，进入可视区域后自动加载交互内容，也可放大到弹窗查看；HTML 与 Markdown 预览均可在渲染结果和源文件之间切换，常见 HTTPS 图表、地图及网页资源可正常加载。
|
||||
|
|
|
|||
|
|
@ -108,6 +108,8 @@ Session = 消息列表,ORM 直写 PG `messages`(append-only,jsonb 存 LiteLLM
|
|||
|
||||
默认 `deepseek_v4.flash`;复杂 bug / 终稿升 pro + reasoning_effort=max;fallback 手动切 Claude。成本量级:修 bug flash ~$0.01 / 完整申报书 flash ~$0.30(pro-max ~$1.5,Opus ~$10+)。99% 任务 flash 够用。
|
||||
|
||||
模型思考参数由 profile 统一表达：`thinking_enabled` 只表示开关，`thinking_transport` 只表示已验证的传输协议，`reasoning_effort` 只表示开启后的推理强度；`core/llm_params.py` 是请求构造唯一入口。DeepSeek、GLM、方舟当前共享 `extra_body` 协议，未验证网关明确用 `none`、不猜参数协议，主循环不再按 family 分支。`/v1/models` 只返回语义明确的 `thinking_enabled`。
|
||||
|
||||
---
|
||||
|
||||
## 5. 设计哲学
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`。
|
||||
|
||||
最后更新:2026-08-03(交互式 HTML 预览 + 对话内嵌,bump 0.60.25)
|
||||
最后更新:2026-08-03(DeepSeek Flash-0731 + thinking 参数统一,bump 0.60.26)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -23,6 +23,7 @@
|
|||
|
||||
### 2026-08-03
|
||||
|
||||
- **08-03 / 0.60.26 / DeepSeek Flash-0731 + thinking 参数统一**:默认 `deepseek-v4-flash` 无需换模型 ID 即接入官方 0731 后训练升级；Flash 改为显式开启 thinking 并透传 `reasoning_effort=high`，校准当前常规时段 token 成本但保留 8K 稳定输出预算。模型档案统一用 `thinking_enabled`(开关)+`thinking_transport`(协议)+`reasoning_effort`(强度)，DeepSeek/GLM/方舟共用纯函数请求构造，移除 family 分支与旧 `thinking_mode` 字段；方舟保持既有思考开启，GLM 保持生产验证过的显式关闭，未验证网关标记 `none`。`/v1/models` 同步只返回新字段；445 项 unittest 全绿(17 skip)，Ruff 与 diff 检查通过；未连生产 DB、未发真实模型请求，无 schema/migration/依赖变化。
|
||||
- **08-03 / 0.60.25 / 交互式 HTML 预览 + 对话内嵌**:文件预览将 HTML 从普通源码提升为可切换“预览 / 源文件”的 sandbox iframe，允许脚本与 HTTPS CDN/接口但保持 opaque origin，禁止宿主权限、表单和顶层跳转；助手最终答复中的 HTML 产物改为进入可视区才加载的内嵌卡片，并可放大复用完整预览，Markdown 同步补源文件切换。Node 14 项、Python 27 项、JavaScript 语法及 diff 检查通过；当前环境无可用浏览器实例，真实页面点击/截图留部署后冒烟；无 schema、migration、HTTP API 或依赖变化。
|
||||
- **08-03 / 0.60.24 / Web Mermaid 直出 + Markdown 围栏容错**:模型偶发用同长度围栏嵌套 Markdown/Mermaid 示例，CommonMark 会把后续正文吞进未闭合代码块；新增仅针对该明确形态的前后端确定性修复，提示词统一要求外层使用更长异类围栏，历史上下文加载时同样修正且不批量回写生产数据。聊天页本地 vendoring Mermaid 11.16.0，仅在助手文字段定稿后顺序渲染 `language-mermaid`，采用 strict 安全级别、文本/边数上限，语法错误或组件不可用时保留源码并提示；真实 Edge 冒烟确认中文流程图与 XYChart 柱线组合图可生成 SVG。Python 27 项、Node 9 项、Ruff、JS/Python 语法及 diff 检查通过；无 schema、migration、HTTP API 或 Python 依赖变化。
|
||||
- **08-03 / 0.60.23 / Office→PDF 组件感知 + 展示路径兼容**:生产 task `92ac20cf` 暴露两层问题：上传消息给出 user-root 相对的 `测试pdf/x.doc`，host tool 又按 task_dir 拼接导致首次找不到；改用裸文件名后，host 仅安装 `libreoffice-impress` 却因只检测 `soffice` 而错误宣称支持 DOC，最终 Writer 导入返回 `source file could not be loaded`。现 `office_to_pdf` 同时解析 task 相对、user-root 展示路径与 `/workspace` 路径；Debian/Ubuntu 按 Writer/Calc/Impress 实际安装包缓存支持后缀，工具 schema 只声明可用格式，缺组件在启动转换前给出明确管理员提示。RUN bootstrap 与故障表同步要求 host 安装三组件；相关 30 项 unittest、Python 编译及 diff 格式检查通过，本机无 LibreOffice，真实 `.doc/.xlsx/.pptx` 冒烟留部署 host；无 schema、migration、HTTP API 或 Python 依赖变化，无需重建沙箱镜像。
|
||||
|
|
|
|||
|
|
@ -9,21 +9,23 @@ variants:
|
|||
api_key_env: DEEPSEEK_API_KEY
|
||||
max_context: 1048576
|
||||
reliable_context: 262144
|
||||
max_output: 8192
|
||||
max_output: 8192 # 官方上限 384K；zcbot 先保留稳定输出预算，压测后再放大
|
||||
parallel_tools: false
|
||||
tool_calling_quality: good
|
||||
thinking_mode: false
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
thinking_enabled: true
|
||||
thinking_transport: extra_body
|
||||
reasoning_effort_levels: [low, high, max]
|
||||
default_reasoning_effort: high
|
||||
code_quality: good
|
||||
enable_run_python: true
|
||||
max_iterations: 120 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
|
||||
optimal_temperature: 0.3
|
||||
prompt_caching: false
|
||||
extended_thinking: false
|
||||
# Flash-0731 官方美元价按 1 USD ~= 7.2 CNY 折算；峰谷价正式生效后再同步。
|
||||
input_cny_per_mtoken: 1.0
|
||||
output_cny_per_mtoken: 12.0
|
||||
cache_hit_cny_per_mtoken: 0.1 # DeepSeek 前缀缓存命中价(input 的 ~0.1x)
|
||||
output_cny_per_mtoken: 2.0
|
||||
cache_hit_cny_per_mtoken: 0.02 # 官方 $0.0028 / M tokens
|
||||
|
||||
pro:
|
||||
display_name: DeepSeek V4 Pro
|
||||
|
|
@ -35,7 +37,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: true
|
||||
thinking_enabled: true
|
||||
thinking_transport: extra_body
|
||||
reasoning_effort_levels: [low, medium, high, max]
|
||||
default_reasoning_effort: medium
|
||||
code_quality: excellent
|
||||
|
|
|
|||
|
|
@ -3,10 +3,8 @@
|
|||
# 与 config/models/local.yaml 同范式(避免 litellm volcengine provider 的版本/字段差异)。
|
||||
# api_key 复用媒体侧的 ARK_API_KEY(同一火山账号),env 见 RUN.md。
|
||||
#
|
||||
# thinking_mode 暂设 false:Seed 2.1 是深度思考模型,但开关走 Ark body `thinking:{type:enabled}`,
|
||||
# 与 OpenAI/DeepSeek 的 `reasoning_effort` 等级协议不同 —— 同 glm.yaml 的处理,要 core/llm.py
|
||||
# 加 family 分支才能透传等级,留 TODO。设 false 只是不发 reasoning_effort 字段;模型默认仍会
|
||||
# 深度思考并返回 reasoning_content,不影响调用。
|
||||
# Seed 2.1 的 thinking 走统一 extra_body 协议；保持既有深度思考行为，但由服务端默认
|
||||
# 改为显式 enabled。方舟未声明 effort 等级，故只传 thinking、不猜 reasoning_effort。
|
||||
# 单价见各 variant(元/百万 tokens,来源:火山方舟 2026-06 发布价)。
|
||||
family: doubao
|
||||
|
||||
|
|
@ -21,7 +19,8 @@ variants:
|
|||
max_output: 16384 # 模型上限 128K(含思考),这里保守取值,需要长输出可调高
|
||||
parallel_tools: true # Ark 兼容 parallel_tool_calls,默认 true
|
||||
tool_calling_quality: good
|
||||
thinking_mode: false
|
||||
thinking_enabled: true
|
||||
thinking_transport: extra_body
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: good
|
||||
|
|
@ -44,7 +43,8 @@ variants:
|
|||
max_output: 16384 # 模型上限 128K(含思考),这里保守取值,需要长输出可调高
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: false
|
||||
thinking_enabled: true
|
||||
thinking_transport: extra_body
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
@ -69,7 +69,8 @@ variants:
|
|||
max_output: 16384
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: false
|
||||
thinking_enabled: true
|
||||
thinking_transport: extra_body
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
|
|||
|
|
@ -1,9 +1,7 @@
|
|||
# 智谱 GLM 模型档案
|
||||
# 走 litellm 原生 zai provider(1.83+ 内置)。litellm 默认 api_base 是国际站 api.z.ai;
|
||||
# 国内站 bigmodel.cn 通过 yaml 的 api_base 字段覆盖。两站 API key 不通用,env 也分开命名。
|
||||
# thinking 已接(core/llm.py _build_kwargs 的 family=="glm" 分支):GLM 协议是 body
|
||||
# `{"thinking":{"type":"enabled|disabled"}}`(与 OpenAI/DeepSeek 的 `reasoning_effort`
|
||||
# 等级不同族),走 extra_body 透传,由本档 `thinking_mode` 决定开关。当前均 false=禁用 ——
|
||||
# thinking 走统一 extra_body 协议,由本档 `thinking_enabled` 决定开关。当前均 false=禁用 ——
|
||||
# 因网关侧默认开 thinking,重任务上会把输出预算烧在 reasoning_content 上撞满 65536 上限
|
||||
# 被截断→空响应(task 35744bea 案);禁用后线上探针实测 reasoning_content 归零、正文照常。
|
||||
family: glm
|
||||
|
|
@ -19,7 +17,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: false
|
||||
tool_calling_quality: good
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: extra_body
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: good
|
||||
|
|
@ -41,7 +40,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: false
|
||||
tool_calling_quality: good
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: extra_body
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
# 走 OpenAI 兼容协议(litellm provider 前缀 `openai/`,后段为实际 model 字段透传给 base_url)。
|
||||
# 涉密任务时用户显式选 local.r1 / local.qwq 代替默认 deepseek_v4.flash;不走自动路由。
|
||||
# 两个 variant 共用同一台推理服务器(api_base 同),api_key_env 也共用 LOCAL_LLM_API_KEY。
|
||||
# thinking_mode=false:R1 / QwQ 是天生推理模型,默认就思考,不通过 reasoning_effort 等级控制
|
||||
# thinking_enabled=false:R1 / QwQ 是天生推理模型,默认就思考,不通过 reasoning_effort 等级控制
|
||||
# (那是 OpenAI / DeepSeek V4 风格);设 true 会发 reasoning_effort 字段,本地 vLLM / sglang
|
||||
# 多半不认,报 400。
|
||||
family: local
|
||||
|
|
@ -18,7 +18,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: false
|
||||
tool_calling_quality: fair
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: good
|
||||
|
|
@ -41,7 +42,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: false
|
||||
tool_calling_quality: fair
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: good
|
||||
|
|
|
|||
|
|
@ -23,7 +23,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
@ -43,7 +44,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
@ -63,7 +65,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
@ -83,7 +86,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
@ -103,7 +107,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: true
|
||||
tool_calling_quality: excellent
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
@ -123,7 +128,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: true
|
||||
tool_calling_quality: good
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: good
|
||||
|
|
@ -143,7 +149,8 @@ variants:
|
|||
max_output: 8192
|
||||
parallel_tools: false # gemini 走网关未实测该参数,保守关闭
|
||||
tool_calling_quality: good
|
||||
thinking_mode: false
|
||||
thinking_enabled: false
|
||||
thinking_transport: none
|
||||
reasoning_effort_levels: []
|
||||
default_reasoning_effort: ""
|
||||
code_quality: excellent
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
|
||||
# 改版本只动这一行。
|
||||
__version__ = "0.60.25"
|
||||
__version__ = "0.60.26"
|
||||
|
|
|
|||
|
|
@ -7,6 +7,8 @@ from typing import List
|
|||
|
||||
import yaml
|
||||
|
||||
from .llm_params import THINKING_TRANSPORTS
|
||||
|
||||
|
||||
@dataclass
|
||||
class ModelCapabilities:
|
||||
|
|
@ -24,8 +26,10 @@ class ModelCapabilities:
|
|||
parallel_tools: bool = False
|
||||
tool_calling_quality: str = "good"
|
||||
|
||||
# 思考模式
|
||||
thinking_mode: bool = False
|
||||
# 思考开关
|
||||
thinking_enabled: bool = False
|
||||
# none=不猜 provider 默认值；extra_body=显式发送 thinking.type，effort 同体透传。
|
||||
thinking_transport: str = "none"
|
||||
reasoning_effort_levels: List[str] = field(default_factory=list)
|
||||
default_reasoning_effort: str = ""
|
||||
|
||||
|
|
@ -71,9 +75,27 @@ class ModelCapabilities:
|
|||
f"档案 {path} 没有 variant={variant};可选: {list(variants)}"
|
||||
)
|
||||
|
||||
var = variants[variant]
|
||||
var = dict(variants[variant])
|
||||
valid_keys = {f.name for f in fields(cls)}
|
||||
kwargs = {k: v for k, v in var.items() if k in valid_keys}
|
||||
kwargs["family"] = data.get("family", family)
|
||||
kwargs["variant"] = variant
|
||||
return cls(**kwargs)
|
||||
caps = cls(**kwargs)
|
||||
if caps.thinking_transport not in THINKING_TRANSPORTS:
|
||||
raise ValueError(
|
||||
f"档案 {path} 的 thinking_transport={caps.thinking_transport!r} 无效;"
|
||||
f"可选: {sorted(THINKING_TRANSPORTS)}"
|
||||
)
|
||||
if caps.thinking_enabled and caps.thinking_transport == "none":
|
||||
raise ValueError(
|
||||
f"档案 {path} 开启 thinking 时必须声明可验证的 thinking_transport"
|
||||
)
|
||||
if (
|
||||
caps.default_reasoning_effort
|
||||
and caps.default_reasoning_effort not in caps.reasoning_effort_levels
|
||||
):
|
||||
raise ValueError(
|
||||
f"档案 {path} 的 default_reasoning_effort="
|
||||
f"{caps.default_reasoning_effort!r} 不在 reasoning_effort_levels 中"
|
||||
)
|
||||
return caps
|
||||
|
|
|
|||
20
core/llm.py
20
core/llm.py
|
|
@ -25,6 +25,7 @@ from litellm.exceptions import (
|
|||
)
|
||||
|
||||
from .capabilities import ModelCapabilities
|
||||
from .llm_params import build_thinking_kwargs
|
||||
|
||||
# 单次 LLM 请求超时(秒),默认与 litellm 一致(600s)但显式化 + env 可调 ──
|
||||
# 长思考模型真被掐("600s 无字节 → run 标 error")时调大 ZCBOT_LLM_TIMEOUT_S 即可,
|
||||
|
|
@ -64,18 +65,13 @@ class LLM:
|
|||
kwargs["tools"] = tools
|
||||
if self.caps.parallel_tools and parallel_tool_calls is not False:
|
||||
kwargs["parallel_tool_calls"] = True
|
||||
if self.caps.thinking_mode and reasoning_effort:
|
||||
kwargs["reasoning_effort"] = reasoning_effort
|
||||
# GLM(zai)的 thinking 网关侧默认开着 —— 重任务上把整个输出预算烧在 reasoning_content
|
||||
# 上、撞满模型自带输出上限(65536)被截断,回来 content 空 + 无 tool_call,被 loop 判空
|
||||
# 响应整轮丢弃 + 无效重试(task 35744bea 案:5 次 empty 全 tokens_out=65536,重试同上下文
|
||||
# 再撞)。GLM 的 thinking 协议是 body {"thinking":{"type":...}}(与 OpenAI 的
|
||||
# reasoning_effort 等级协议不同族),走 extra_body 透传 —— 线上探针实测 disabled 后
|
||||
# reasoning_content 归零、正文/工具照常。config thinking_mode 决定开关(当前 glm 档均 false)。
|
||||
if self.caps.family == "glm":
|
||||
kwargs["extra_body"] = {
|
||||
"thinking": {"type": "enabled" if self.caps.thinking_mode else "disabled"}
|
||||
}
|
||||
kwargs.update(
|
||||
build_thinking_kwargs(
|
||||
enabled=self.caps.thinking_enabled,
|
||||
transport=self.caps.thinking_transport,
|
||||
reasoning_effort=reasoning_effort,
|
||||
)
|
||||
)
|
||||
if self.caps.prompt_caching:
|
||||
kwargs["extra_headers"] = {"anthropic-beta": "prompt-caching-2024-07-31"}
|
||||
return kwargs
|
||||
|
|
|
|||
|
|
@ -0,0 +1,30 @@
|
|||
"""跨 provider 的可选 LLM 请求参数构造。"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
THINKING_TRANSPORTS = {"none", "extra_body"}
|
||||
|
||||
|
||||
def build_thinking_kwargs(
|
||||
*, enabled: bool, transport: str, reasoning_effort: str | None
|
||||
) -> dict[str, Any]:
|
||||
"""把统一的 thinking 配置转换为 LiteLLM 调用参数。
|
||||
|
||||
``none`` 表示该模型没有经过验证的显式控制协议，不猜 provider 默认值。
|
||||
``extra_body`` 对应当前 DeepSeek、GLM 与方舟 ChatCompletions 的共同协议；
|
||||
effort 仅在开启且档案提供非空值时发送。
|
||||
"""
|
||||
if transport == "none":
|
||||
return {}
|
||||
if transport != "extra_body":
|
||||
raise ValueError(
|
||||
f"不支持的 thinking_transport={transport!r};可选: {sorted(THINKING_TRANSPORTS)}"
|
||||
)
|
||||
|
||||
body: dict[str, Any] = {
|
||||
"thinking": {"type": "enabled" if enabled else "disabled"}
|
||||
}
|
||||
if enabled and reasoning_effort:
|
||||
body["reasoning_effort"] = reasoning_effort
|
||||
return {"extra_body": body}
|
||||
|
|
@ -6,7 +6,7 @@
|
|||
四项探测:
|
||||
- basic_chat:连通性。失败则跳过其余。
|
||||
- parallel_tools:给两个独立工具,看 single response 里 tool_calls 数量。
|
||||
- thinking_mode:对 declared=True 的模型传 reasoning_effort,看 API 是否接受 + 是否产出 thinking。
|
||||
- thinking:对 enabled=True 的模型传统一参数,看 API 是否接受 + 是否产出 thinking。
|
||||
- long_context(opt-in):needle-in-haystack 简化版,默认探 reliable_context 的 1/8。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
|
@ -133,11 +133,11 @@ def probe_parallel_tools(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
|
|||
)
|
||||
|
||||
|
||||
def probe_thinking_mode(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
|
||||
declared = caps.thinking_mode
|
||||
def probe_thinking(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
|
||||
declared = caps.thinking_enabled
|
||||
if not declared:
|
||||
return ProbeResult(
|
||||
name="thinking_mode",
|
||||
name="thinking",
|
||||
declared=False,
|
||||
observed=None,
|
||||
status="skip",
|
||||
|
|
@ -145,7 +145,7 @@ def probe_thinking_mode(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
|
|||
)
|
||||
effort = (
|
||||
caps.default_reasoning_effort
|
||||
or (caps.reasoning_effort_levels[0] if caps.reasoning_effort_levels else "medium")
|
||||
or (caps.reasoning_effort_levels[0] if caps.reasoning_effort_levels else None)
|
||||
)
|
||||
try:
|
||||
resp = llm.chat(
|
||||
|
|
@ -162,22 +162,22 @@ def probe_thinking_mode(llm: LLM, caps: ModelCapabilities) -> ProbeResult:
|
|||
)
|
||||
observed = bool(rc)
|
||||
return ProbeResult(
|
||||
name="thinking_mode",
|
||||
name="thinking",
|
||||
declared=True,
|
||||
observed=observed,
|
||||
status="ok" if observed else "mismatch",
|
||||
detail=(
|
||||
f"reasoning_effort={effort} accepted; "
|
||||
(f"reasoning_effort={effort} accepted; " if effort else "thinking enabled; ")
|
||||
+ ("thinking content returned" if observed else "no thinking content in response")
|
||||
),
|
||||
)
|
||||
except Exception as e:
|
||||
return ProbeResult(
|
||||
name="thinking_mode",
|
||||
name="thinking",
|
||||
declared=True,
|
||||
observed=False,
|
||||
status="mismatch",
|
||||
detail=f"reasoning_effort rejected: {type(e).__name__}: {e}",
|
||||
detail=f"thinking parameters rejected: {type(e).__name__}: {e}",
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -237,7 +237,7 @@ def probe_capabilities(
|
|||
if report.results[0].status == "error":
|
||||
return report
|
||||
report.add(probe_parallel_tools(llm, caps))
|
||||
report.add(probe_thinking_mode(llm, caps))
|
||||
report.add(probe_thinking(llm, caps))
|
||||
if include_long_context:
|
||||
report.add(probe_long_context(llm, caps))
|
||||
return report
|
||||
|
|
|
|||
|
|
@ -0,0 +1,115 @@
|
|||
import os
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from core.capabilities import ModelCapabilities
|
||||
from core.llm import LLM
|
||||
|
||||
|
||||
class LLMKwargsTests(unittest.TestCase):
|
||||
def _llm(
|
||||
self, *, family: str, thinking_enabled: bool, thinking_transport: str
|
||||
) -> LLM:
|
||||
caps = ModelCapabilities(
|
||||
family=family,
|
||||
model_id=f"{family}/model",
|
||||
api_key_env="TEST_LLM_API_KEY",
|
||||
thinking_enabled=thinking_enabled,
|
||||
thinking_transport=thinking_transport,
|
||||
optimal_temperature=0.3,
|
||||
)
|
||||
with patch.dict(os.environ, {"TEST_LLM_API_KEY": "test-key"}):
|
||||
return LLM(caps)
|
||||
|
||||
def test_deepseek_explicitly_enables_thinking_and_sets_effort(self) -> None:
|
||||
llm = self._llm(
|
||||
family="deepseek_v4", thinking_enabled=True, thinking_transport="extra_body"
|
||||
)
|
||||
|
||||
kwargs = llm._build_kwargs(
|
||||
[{"role": "user", "content": "hello"}], None, None, "high"
|
||||
)
|
||||
|
||||
self.assertNotIn("reasoning_effort", kwargs)
|
||||
self.assertEqual(
|
||||
kwargs["extra_body"],
|
||||
{"thinking": {"type": "enabled"}, "reasoning_effort": "high"},
|
||||
)
|
||||
|
||||
def test_deepseek_explicitly_disables_thinking_without_effort(self) -> None:
|
||||
llm = self._llm(
|
||||
family="deepseek_v4", thinking_enabled=False, thinking_transport="extra_body"
|
||||
)
|
||||
|
||||
kwargs = llm._build_kwargs(
|
||||
[{"role": "user", "content": "hello"}], None, None, "high"
|
||||
)
|
||||
|
||||
self.assertNotIn("reasoning_effort", kwargs)
|
||||
self.assertEqual(
|
||||
kwargs["extra_body"], {"thinking": {"type": "disabled"}}
|
||||
)
|
||||
|
||||
def test_other_openai_compatible_provider_gets_no_thinking_body(self) -> None:
|
||||
llm = self._llm(
|
||||
family="unifyllm", thinking_enabled=False, thinking_transport="none"
|
||||
)
|
||||
|
||||
kwargs = llm._build_kwargs(
|
||||
[{"role": "user", "content": "hello"}], None, None, None
|
||||
)
|
||||
|
||||
self.assertNotIn("extra_body", kwargs)
|
||||
|
||||
def test_extra_body_transport_enables_thinking_without_effort(self) -> None:
|
||||
llm = self._llm(
|
||||
family="doubao", thinking_enabled=True, thinking_transport="extra_body"
|
||||
)
|
||||
|
||||
kwargs = llm._build_kwargs(
|
||||
[{"role": "user", "content": "hello"}], None, None, None
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
kwargs["extra_body"], {"thinking": {"type": "enabled"}}
|
||||
)
|
||||
|
||||
def test_invalid_transport_is_rejected_before_request(self) -> None:
|
||||
llm = self._llm(
|
||||
family="test", thinking_enabled=True, thinking_transport="unknown"
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(ValueError, "thinking_transport"):
|
||||
llm._build_kwargs(
|
||||
[{"role": "user", "content": "hello"}], None, None, "high"
|
||||
)
|
||||
|
||||
def test_flash_profile_matches_0731_capabilities(self) -> None:
|
||||
caps = ModelCapabilities.load(
|
||||
"deepseek_v4.flash", Path(__file__).resolve().parents[1] / "config" / "models"
|
||||
)
|
||||
|
||||
self.assertTrue(caps.thinking_enabled)
|
||||
self.assertEqual(caps.reasoning_effort_levels, ["low", "high", "max"])
|
||||
self.assertEqual(caps.default_reasoning_effort, "high")
|
||||
self.assertEqual(caps.max_output, 8192)
|
||||
self.assertEqual(caps.output_cny_per_mtoken, 2.0)
|
||||
self.assertEqual(caps.cache_hit_cny_per_mtoken, 0.02)
|
||||
self.assertEqual(caps.thinking_transport, "extra_body")
|
||||
|
||||
def test_other_controllable_profiles_declare_transport(self) -> None:
|
||||
models_dir = Path(__file__).resolve().parents[1] / "config" / "models"
|
||||
|
||||
glm = ModelCapabilities.load("glm.pro52", models_dir)
|
||||
doubao = ModelCapabilities.load("doubao.turbo", models_dir)
|
||||
|
||||
self.assertFalse(glm.thinking_enabled)
|
||||
self.assertEqual(glm.thinking_transport, "extra_body")
|
||||
self.assertTrue(doubao.thinking_enabled)
|
||||
self.assertEqual(doubao.thinking_transport, "extra_body")
|
||||
self.assertEqual(doubao.default_reasoning_effort, "")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -54,7 +54,7 @@ def register_model_routes(app, *, require_user) -> None:
|
|||
"display_name": caps.display_name or profile,
|
||||
"family": caps.family,
|
||||
"variant": caps.variant,
|
||||
"thinking_mode": caps.thinking_mode,
|
||||
"thinking_enabled": caps.thinking_enabled,
|
||||
"is_default": profile == default,
|
||||
})
|
||||
return {"models": out}
|
||||
|
|
|
|||
Loading…
Reference in New Issue