fix(security): 强化工程类型检查与依赖审计

This commit is contained in:
caoqianming 2026-08-03 16:59:32 +08:00
parent cee4eb4c14
commit 137d27acdd
40 changed files with 259 additions and 95 deletions

View File

@ -5,6 +5,10 @@
> 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。 > 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。
> 工程口径的完整记录见 `PROGRESS.md` / git log。 > 工程口径的完整记录见 `PROGRESS.md` / git log。
## 0.60.27 — 2026-08-03
- 加强服务端命令执行、第三方依赖和文件处理链路的安全防护,并补齐持续类型检查,降低已知依赖漏洞与边界类型错误导致服务异常的风险。
## 0.60.26 — 2026-08-03 ## 0.60.26 — 2026-08-03
- 默认 DeepSeek Flash 已使用官方 0731 API 升级,继续展示思考过程并显式控制推理强度;不同模型的思考开关不再依赖服务端默认值,切换模型时行为更稳定。 - 默认 DeepSeek Flash 已使用官方 0731 API 升级,继续展示思考过程并显式控制推理强度;不同模型的思考开关不再依赖服务端默认值,切换模型时行为更稳定。

View File

@ -2,7 +2,7 @@
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9` > 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`
最后更新:2026-08-03(DeepSeek Flash-0731 + thinking 参数统一,bump 0.60.26) 最后更新:2026-08-03(工程类型门禁 + 安全审计清零,bump 0.60.27)
--- ---
@ -23,6 +23,7 @@
### 2026-08-03 ### 2026-08-03
- **08-03 / 0.60.27 / 工程类型门禁 + 安全审计清零**:修复 141 个源文件中的 203 项 mypy 错误,新增按第三方无类型依赖与动态工具签名精确收口的 mypy 配置外部协议规定的弱哈希显式标注非本地密码用途host shell 前后台执行均改用明确解释器 argv移除隐式 `shell=True`。提高 aiohttp、Pillow、cryptography、Starlette、python-multipart、pydantic-settings 与 pip 的安全版本下限,`pip check` 无冲突、`pip-audit` 无已知漏洞;新增显式 shell 回归测试,完整审计 448 项 unittest 全绿(17 skip)Ruff/mypy/Bandit/pip-audit 通过engineering 80/100 PASS、security 100/100 PASS。无 schema/migration/API 变化,未连接生产 DB。
- **08-03 / 0.60.26 / DeepSeek Flash-0731 + thinking 参数统一**:默认 `deepseek-v4-flash` 无需换模型 ID 即接入官方 0731 后训练升级Flash 改为显式开启 thinking 并透传 `reasoning_effort=high`,校准当前常规时段 token 成本但保留 8K 稳定输出预算。模型档案统一用 `thinking_enabled`(开关)+`thinking_transport`(协议)+`reasoning_effort`(强度)DeepSeek/GLM/方舟共用纯函数请求构造,移除 family 分支与旧 `thinking_mode` 字段方舟保持既有思考开启GLM 保持生产验证过的显式关闭,未验证网关标记 `none`。`/v1/models` 同步只返回新字段445 项 unittest 全绿(17 skip)Ruff 与 diff 检查通过;未连生产 DB、未发真实模型请求无 schema/migration/依赖变化。 - **08-03 / 0.60.26 / DeepSeek Flash-0731 + thinking 参数统一**:默认 `deepseek-v4-flash` 无需换模型 ID 即接入官方 0731 后训练升级Flash 改为显式开启 thinking 并透传 `reasoning_effort=high`,校准当前常规时段 token 成本但保留 8K 稳定输出预算。模型档案统一用 `thinking_enabled`(开关)+`thinking_transport`(协议)+`reasoning_effort`(强度)DeepSeek/GLM/方舟共用纯函数请求构造,移除 family 分支与旧 `thinking_mode` 字段方舟保持既有思考开启GLM 保持生产验证过的显式关闭,未验证网关标记 `none`。`/v1/models` 同步只返回新字段445 项 unittest 全绿(17 skip)Ruff 与 diff 检查通过;未连生产 DB、未发真实模型请求无 schema/migration/依赖变化。
- **08-03 / 0.60.25 / 交互式 HTML 预览 + 对话内嵌**:文件预览将 HTML 从普通源码提升为可切换“预览 / 源文件”的 sandbox iframe允许脚本与 HTTPS CDN/接口但保持 opaque origin禁止宿主权限、表单和顶层跳转助手最终答复中的 HTML 产物改为进入可视区才加载的内嵌卡片并可放大复用完整预览Markdown 同步补源文件切换。Node 14 项、Python 27 项、JavaScript 语法及 diff 检查通过;当前环境无可用浏览器实例,真实页面点击/截图留部署后冒烟;无 schema、migration、HTTP API 或依赖变化。 - **08-03 / 0.60.25 / 交互式 HTML 预览 + 对话内嵌**:文件预览将 HTML 从普通源码提升为可切换“预览 / 源文件”的 sandbox iframe允许脚本与 HTTPS CDN/接口但保持 opaque origin禁止宿主权限、表单和顶层跳转助手最终答复中的 HTML 产物改为进入可视区才加载的内嵌卡片并可放大复用完整预览Markdown 同步补源文件切换。Node 14 项、Python 27 项、JavaScript 语法及 diff 检查通过;当前环境无可用浏览器实例,真实页面点击/截图留部署后冒烟;无 schema、migration、HTTP API 或依赖变化。
- **08-03 / 0.60.24 / Web Mermaid 直出 + Markdown 围栏容错**:模型偶发用同长度围栏嵌套 Markdown/Mermaid 示例CommonMark 会把后续正文吞进未闭合代码块;新增仅针对该明确形态的前后端确定性修复,提示词统一要求外层使用更长异类围栏,历史上下文加载时同样修正且不批量回写生产数据。聊天页本地 vendoring Mermaid 11.16.0,仅在助手文字段定稿后顺序渲染 `language-mermaid`,采用 strict 安全级别、文本/边数上限,语法错误或组件不可用时保留源码并提示;真实 Edge 冒烟确认中文流程图与 XYChart 柱线组合图可生成 SVG。Python 27 项、Node 9 项、Ruff、JS/Python 语法及 diff 检查通过;无 schema、migration、HTTP API 或 Python 依赖变化。 - **08-03 / 0.60.24 / Web Mermaid 直出 + Markdown 围栏容错**:模型偶发用同长度围栏嵌套 Markdown/Mermaid 示例CommonMark 会把后续正文吞进未闭合代码块;新增仅针对该明确形态的前后端确定性修复,提示词统一要求外层使用更长异类围栏,历史上下文加载时同样修正且不批量回写生产数据。聊天页本地 vendoring Mermaid 11.16.0,仅在助手文字段定稿后顺序渲染 `language-mermaid`,采用 strict 安全级别、文本/边数上限,语法错误或组件不可用时保留源码并提示;真实 Edge 冒烟确认中文流程图与 XYChart 柱线组合图可生成 SVG。Python 27 项、Node 9 项、Ruff、JS/Python 语法及 diff 检查通过;无 schema、migration、HTTP API 或 Python 依赖变化。

View File

@ -1,3 +1,3 @@
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。 # zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
# 改版本只动这一行。 # 改版本只动这一行。
__version__ = "0.60.26" __version__ = "0.60.27"

View File

@ -77,7 +77,11 @@ def _load_credentials() -> tuple[str, str]:
def _auth_params() -> dict: def _auth_params() -> dict:
appid, secret = _load_credentials() appid, secret = _load_credentials()
ts = str(int(time.time())) ts = str(int(time.time()))
md5hex = hashlib.md5((appid + ts).encode()).hexdigest() # 讯飞 LFASR 协议固定要求 MD5(appid+ts) 后再做 HMAC-SHA1此处不能
# 替换算法,且 MD5 仅作为协议输入,不承担本地密码学安全用途。
md5hex = hashlib.md5(
(appid + ts).encode(), usedforsecurity=False
).hexdigest()
signa = base64.b64encode( signa = base64.b64encode(
hmac.new(secret.encode(), md5hex.encode(), hashlib.sha1).digest() hmac.new(secret.encode(), md5hex.encode(), hashlib.sha1).digest()
).decode() ).decode()
@ -180,7 +184,8 @@ def format_transcript(segments: list[tuple[str, str]]) -> str:
if len(roles) < 2: if len(roles) < 2:
return "".join(t for _, t in segments).strip() return "".join(t for _, t in segments).strip()
lines: list[str] = [] lines: list[str] = []
cur_role, buf = None, [] cur_role: Optional[str] = None
buf: list[str] = []
for role, text in segments: for role, text in segments:
if role != cur_role: if role != cur_role:
if buf: if buf:
@ -232,7 +237,8 @@ def transcribe_file(
status = info.get("status") status = info.get("status")
if status == -1: if status == -1:
ft = info.get("failType") ft = info.get("failType")
hint = _FAIL_HINTS.get(ft, f"转写失败(failType={ft})") fail_type = ft if isinstance(ft, int) else -1
hint = _FAIL_HINTS.get(fail_type, f"转写失败(failType={ft})")
raise LfasrError(f"讯飞录音转写失败:{hint}") raise LfasrError(f"讯飞录音转写失败:{hint}")
if status != 4: if status != 4:
continue # 0 已创建 / 3 处理中 continue # 0 已创建 / 3 处理中

View File

@ -21,6 +21,7 @@ import json
import os import os
import time import time
from contextlib import suppress from contextlib import suppress
from typing import Any
from urllib.parse import urlencode from urllib.parse import urlencode
from wsgiref.handlers import format_date_time from wsgiref.handlers import format_date_time
@ -109,7 +110,7 @@ class XfyunStream:
def __init__(self, on_text, *, language: str = "zh_cn"): def __init__(self, on_text, *, language: str = "zh_cn"):
self._on_text = on_text self._on_text = on_text
self._language = language self._language = language
self._ws = None self._ws: Any = None
self._recv_task: asyncio.Task | None = None self._recv_task: asyncio.Task | None = None
self._segs: dict[int, str] = {} # sn → 该段文本;pgs=rpl 按 rg 区间删旧段 self._segs: dict[int, str] = {} # sn → 该段文本;pgs=rpl 按 rg 区间删旧段
self._done = asyncio.Event() self._done = asyncio.Event()

View File

@ -212,7 +212,7 @@ def prepare_messages_with_stats(
# 未到上下文压力门槛 → 原样发,零压缩(缓存全暖 + 不丢信息)。压缩是"放不下"才做的事。 # 未到上下文压力门槛 → 原样发,零压缩(缓存全暖 + 不丢信息)。压缩是"放不下"才做的事。
if original_chars < compact_threshold_chars: if original_chars < compact_threshold_chars:
prepared = [deepcopy(m) for m in messages] unchanged = [deepcopy(m) for m in messages]
stats = { stats = {
"original_chars": original_chars, "original_chars": original_chars,
"sent_chars": original_chars, "sent_chars": original_chars,
@ -222,7 +222,7 @@ def prepare_messages_with_stats(
"compaction_skipped": 1, "compaction_skipped": 1,
"repaired_tool_calls": repaired_tool_calls, "repaired_tool_calls": repaired_tool_calls,
} }
return prepared, stats return unchanged, stats
recent_start = max(0, len(messages) - keep_recent) recent_start = max(0, len(messages) - keep_recent)
prepared: List[dict[str, Any]] = [] prepared: List[dict[str, Any]] = []

View File

@ -23,7 +23,8 @@ from uuid import UUID
from core.task import TaskState from core.task import TaskState
from docx import Document from docx import Document as create_document
from docx.document import Document
from docx.enum.text import WD_ALIGN_PARAGRAPH from docx.enum.text import WD_ALIGN_PARAGRAPH
from docx.oxml.ns import qn from docx.oxml.ns import qn
from docx.shared import Cm, Pt, RGBColor from docx.shared import Cm, Pt, RGBColor
@ -56,7 +57,7 @@ def _preserve_spaces(run) -> None:
# ───────────────────────── 文档骨架 ───────────────────────── # ───────────────────────── 文档骨架 ─────────────────────────
def _init_doc() -> Document: def _init_doc() -> Document:
doc = Document() doc = create_document()
section = doc.sections[0] section = doc.sections[0]
section.page_height = Cm(29.7) section.page_height = Cm(29.7)
section.page_width = Cm(21) section.page_width = Cm(21)

View File

@ -83,7 +83,11 @@ def _try_lock(f) -> bool:
import fcntl import fcntl
try: try:
fcntl.flock(f.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) flock = getattr(fcntl, "flock")
flock(
f.fileno(),
getattr(fcntl, "LOCK_EX") | getattr(fcntl, "LOCK_NB"),
)
return True return True
except BlockingIOError: except BlockingIOError:
return False return False
@ -99,7 +103,7 @@ def _unlock(f) -> None:
import fcntl import fcntl
fcntl.flock(f.fileno(), fcntl.LOCK_UN) getattr(fcntl, "flock")(f.fileno(), getattr(fcntl, "LOCK_UN"))
@contextmanager @contextmanager

View File

@ -216,7 +216,9 @@ def _doc_name_for(d: Path, source_name: str) -> str:
doc_rel = f"docs/{stem}.md" doc_rel = f"docs/{stem}.md"
for e in entries: for e in entries:
if e["doc"] == doc_rel and e["source"] != f"sources/{source_name}": if e["doc"] == doc_rel and e["source"] != f"sources/{source_name}":
suffix = hashlib.md5(source_name.encode("utf-8")).hexdigest()[:6] suffix = hashlib.md5(
source_name.encode("utf-8"), usedforsecurity=False
).hexdigest()[:6]
return f"{stem}_{suffix}.md" return f"{stem}_{suffix}.md"
return f"{stem}.md" return f"{stem}.md"
@ -301,8 +303,8 @@ def _run_ingest_locked(
print(f"[kb_ingest] {kb_name}/{name} failed: {msg}", flush=True) print(f"[kb_ingest] {kb_name}/{name} failed: {msg}", flush=True)
return True return True
finally: finally:
st = _status.get(key) final_state = _status.get(key)
if st is not None: if final_state is not None:
st["running"] = False final_state["running"] = False
st["current"] = None final_state["current"] = None
st["finished_at"] = datetime.now().isoformat(timespec="seconds") final_state["finished_at"] = datetime.now().isoformat(timespec="seconds")

View File

@ -162,7 +162,9 @@ class _RepeatGuard:
一步里只要有一次净产出就算在推进 一步里只要有一次净产出就算在推进
""" """
st = self._state(name, args) st = self._state(name, args)
h = hashlib.sha1(result.encode("utf-8", "replace")).hexdigest() h = hashlib.sha1(
result.encode("utf-8", "replace"), usedforsecurity=False
).hexdigest()
esig = _tool_error_signature(name, result) esig = _tool_error_signature(name, result)
is_err = esig is not None is_err = esig is not None
dup = h in st["hashes"] dup = h in st["hashes"]
@ -303,6 +305,7 @@ class AgentLoop:
self._emit({"type": "cancelled"}) self._emit({"type": "cancelled"})
return "[cancelled]" return "[cancelled]"
assert response is not None
msg = response.choices[0].message msg = response.choices[0].message
tool_calls = getattr(msg, "tool_calls", None) or [] tool_calls = getattr(msg, "tool_calls", None) or []
asst_msg_id = self.session.append( asst_msg_id = self.session.append(
@ -624,7 +627,9 @@ class AgentLoop:
self._emit({"type": "reasoning", "delta": delta_reasoning}) self._emit({"type": "reasoning", "delta": delta_reasoning})
finally: finally:
# generator 提前 break 时 GeneratorExit 触发 chat_stream finally → close 底层连接 # generator 提前 break 时 GeneratorExit 触发 chat_stream finally → close 底层连接
stream.close() close = getattr(stream, "close", None)
if callable(close):
close()
if cancelled: if cancelled:
return None, True return None, True

View File

@ -92,6 +92,7 @@ def normalize_markdown_fences(text: str) -> MarkdownFenceResult:
): ):
idx += 1 idx += 1
continue continue
assert inner_idx is not None
inner_close_idx = None inner_close_idx = None
for candidate in range(inner_idx + 1, len(lines)): for candidate in range(inner_idx + 1, len(lines)):
@ -118,6 +119,7 @@ def normalize_markdown_fences(text: str) -> MarkdownFenceResult:
): ):
idx += 1 idx += 1
continue continue
assert outer_close_idx is not None
repaired_length = max(outer[2], inner[2]) + 1 repaired_length = max(outer[2], inner[2]) + 1
lines[idx] = _replace_fence(lines[idx], repaired_length) lines[idx] = _replace_fence(lines[idx], repaired_length)

View File

@ -15,11 +15,16 @@ svg_to_pptx.py,用外部渲染器把每页渲成整页 PNG 贴进幻灯片交付
from __future__ import annotations from __future__ import annotations
from pathlib import Path from pathlib import Path
from typing import Iterator, List, Optional from typing import Any, Iterator, List, Optional
Presentation: Any
MSO_SHAPE_TYPE: Any
try: try:
from pptx import Presentation from pptx import Presentation as _Presentation
from pptx.enum.shapes import MSO_SHAPE_TYPE from pptx.enum.shapes import MSO_SHAPE_TYPE as _MSO_SHAPE_TYPE
Presentation = _Presentation
MSO_SHAPE_TYPE = _MSO_SHAPE_TYPE
except Exception: # pragma: no cover - 宿主缺 python-pptx 时静默降级为不检 except Exception: # pragma: no cover - 宿主缺 python-pptx 时静默降级为不检
Presentation = None Presentation = None
MSO_SHAPE_TYPE = None MSO_SHAPE_TYPE = None

View File

@ -23,6 +23,13 @@ from pathlib import Path
MAX_LOG_BYTES = 10 * 1024 * 1024 MAX_LOG_BYTES = 10 * 1024 * 1024
def _shell_argv(command: str) -> list[str]:
"""兼容升级前已落盘的 shell_cmd 元数据,不使用 subprocess 隐式 shell。"""
if os.name == "nt":
return [os.environ.get("COMSPEC") or "cmd.exe", "/d", "/s", "/c", command]
return ["/bin/sh", "-c", command]
def _read_meta(d: Path) -> dict: def _read_meta(d: Path) -> dict:
return json.loads((d / "proc.json").read_text(encoding="utf-8")) return json.loads((d / "proc.json").read_text(encoding="utf-8"))
@ -37,7 +44,7 @@ def _kill_tree(child: subprocess.Popen) -> None:
if os.name == "posix": if os.name == "posix":
import signal import signal
try: try:
os.killpg(child.pid, signal.SIGKILL) getattr(os, "killpg")(child.pid, getattr(signal, "SIGKILL"))
except (ProcessLookupError, PermissionError, OSError): except (ProcessLookupError, PermissionError, OSError):
try: try:
child.kill() child.kill()
@ -87,7 +94,7 @@ def main() -> int:
if meta.get("argv"): if meta.get("argv"):
child = subprocess.Popen(meta["argv"], **spawn_kw) child = subprocess.Popen(meta["argv"], **spawn_kw)
else: else:
child = subprocess.Popen(meta["shell_cmd"], shell=True, **spawn_kw) child = subprocess.Popen(_shell_argv(meta["shell_cmd"]), **spawn_kw)
except Exception as e: # 启动失败也要写 exit_code,否则状态永远悬着 except Exception as e: # 启动失败也要写 exit_code,否则状态永远悬着
log.write(f"[proc_wrapper] spawn failed: {type(e).__name__}: {e}\n".encode("utf-8")) log.write(f"[proc_wrapper] spawn failed: {type(e).__name__}: {e}\n".encode("utf-8"))
log.close() log.close()

View File

@ -225,6 +225,12 @@ def finished_at(d: Path) -> Optional[float]:
# ───────────── host 模式启动 / 终止 ───────────── # ───────────── host 模式启动 / 终止 ─────────────
def shell_argv(command: str) -> List[str]:
"""用明确的系统解释器执行 shell 语法,避免 subprocess 隐式 shell。"""
if os.name == "nt":
return [os.environ.get("COMSPEC") or "cmd.exe", "/d", "/s", "/c", command]
return ["/bin/sh", "-c", command]
def launch_host( def launch_host(
anchor: Path, anchor: Path,
task_id: str, task_id: str,
@ -310,10 +316,10 @@ def kill_proc(meta: Dict[str, Any], d: Path) -> str:
for pid in (child_pid, wrapper_pid): for pid in (child_pid, wrapper_pid):
if pid > 0: if pid > 0:
try: try:
os.killpg(pid, signal.SIGKILL) getattr(os, "killpg")(pid, getattr(signal, "SIGKILL"))
except (ProcessLookupError, PermissionError, OSError): except (ProcessLookupError, PermissionError, OSError):
try: try:
os.kill(pid, signal.SIGKILL) os.kill(pid, getattr(signal, "SIGKILL"))
except OSError: except OSError:
pass pass
else: else:

View File

@ -141,7 +141,9 @@ class SandboxPool:
""" """
self.user_root_base = user_root_base self.user_root_base = user_root_base
self.repo_root = repo_root self.repo_root = repo_root
self.image = image or os.getenv("ZCBOT_SANDBOX_IMAGE", DEFAULT_IMAGE) self.image: str = (
image or os.getenv("ZCBOT_SANDBOX_IMAGE") or DEFAULT_IMAGE
)
self.runtime = runtime or os.getenv("ZCBOT_SANDBOX_RUNTIME") or "" self.runtime = runtime or os.getenv("ZCBOT_SANDBOX_RUNTIME") or ""
self.idle_ttl = idle_ttl if idle_ttl is not None else int( self.idle_ttl = idle_ttl if idle_ttl is not None else int(
os.getenv("ZCBOT_SANDBOX_IDLE_TTL", str(DEFAULT_IDLE_TTL_SECONDS)) os.getenv("ZCBOT_SANDBOX_IDLE_TTL", str(DEFAULT_IDLE_TTL_SECONDS))
@ -370,12 +372,16 @@ def setup_pool(
dns_cfg = cfg.get("dns") or [] dns_cfg = cfg.get("dns") or []
if not isinstance(dns_cfg, list): if not isinstance(dns_cfg, list):
dns_cfg = [] dns_cfg = []
memory = cfg.get("memory")
cpus = cfg.get("cpus")
pids_limit = cfg.get("pids_limit")
shm_size = cfg.get("shm_size")
return SandboxPool( return SandboxPool(
user_root_base=user_root_base, user_root_base=user_root_base,
repo_root=repo_root, repo_root=repo_root,
memory=cfg.get("memory") if isinstance(cfg.get("memory"), str) else None, memory=memory if isinstance(memory, str) else None,
cpus=str(cfg["cpus"]) if cfg.get("cpus") is not None else None, cpus=str(cpus) if cpus is not None else None,
pids_limit=int(cfg["pids_limit"]) if cfg.get("pids_limit") is not None else None, pids_limit=int(str(pids_limit)) if pids_limit is not None else None,
shm_size=cfg.get("shm_size") if isinstance(cfg.get("shm_size"), str) else None, shm_size=shm_size if isinstance(shm_size, str) else None,
dns=[str(x) for x in dns_cfg], dns=[str(x) for x in dns_cfg],
) )

View File

@ -20,6 +20,7 @@ import os
import sys import sys
import traceback import traceback
from pathlib import Path from pathlib import Path
from typing import Any
# 镜像里 /sandbox/ 下放了 tools/ 的拷贝,让 import 走 /sandbox/ # 镜像里 /sandbox/ 下放了 tools/ 的拷贝,让 import 走 /sandbox/
@ -66,7 +67,8 @@ def main() -> int:
) )
return 2 return 2
tool = cls(base_dir=Path(os.getcwd()), user_root=Path("/workspace")) tool_cls: Any = cls
tool = tool_cls(base_dir=Path(os.getcwd()), user_root=Path("/workspace"))
try: try:
result = tool.execute(**args) result = tool.execute(**args)
except TypeError as e: except TypeError as e:

View File

@ -20,6 +20,7 @@ from typing import Optional
from rich.console import Console from rich.console import Console
from rich.markdown import Markdown from rich.markdown import Markdown
from rich.status import Status
class ConsoleEventSink: class ConsoleEventSink:
@ -28,7 +29,7 @@ class ConsoleEventSink:
def __init__(self, console: Console) -> None: def __init__(self, console: Console) -> None:
self.console = console self.console = console
self._status = None self._status: Optional[Status] = None
self._stop: Optional[threading.Event] = None self._stop: Optional[threading.Event] = None
self._thread: Optional[threading.Thread] = None self._thread: Optional[threading.Thread] = None
self._start = 0.0 self._start = 0.0
@ -59,19 +60,21 @@ class ConsoleEventSink:
def _spinner_start(self) -> None: def _spinner_start(self) -> None:
self._start = time.monotonic() self._start = time.monotonic()
self._stop = threading.Event() stop = threading.Event()
self._stop = stop
def fmt() -> str: def fmt() -> str:
elapsed = time.monotonic() - self._start elapsed = time.monotonic() - self._start
return f"[muted]thinking... {elapsed:.1f}s[/muted]" return f"[muted]thinking... {elapsed:.1f}s[/muted]"
self._status = self.console.status(fmt(), spinner="dots") status = self.console.status(fmt(), spinner="dots")
self._status.__enter__() self._status = status
status.__enter__()
def tick() -> None: def tick() -> None:
while not self._stop.wait(0.1): while not stop.wait(0.1):
try: try:
self._status.update(fmt()) status.update(fmt())
except Exception: except Exception:
return return

View File

@ -101,7 +101,7 @@ def upsert_task(
""" """
values = {"task_id": task_id, "user_id": user_id, **fields} values = {"task_id": task_id, "user_id": user_id, **fields}
stmt = insert(Task).values(**values) stmt = insert(Task).values(**values)
update_cols = {k: stmt.excluded[k] for k in fields} update_cols: dict[str, Any] = {k: stmt.excluded[k] for k in fields}
if update_cols: if update_cols:
# ORM 的 onupdate=func.now() 只在 ORM-level UPDATE 触发,DO UPDATE 是 raw DML # ORM 的 onupdate=func.now() 只在 ORM-level UPDATE 触发,DO UPDATE 是 raw DML
# 不会自动刷 updated_at —— 这里显式追加。 # 不会自动刷 updated_at —— 这里显式追加。
@ -126,7 +126,7 @@ def update_task(task_id: UUID, **fields: Any) -> int:
result = s.execute( result = s.execute(
update(Task).where(Task.task_id == task_id).values(**fields) update(Task).where(Task.task_id == task_id).values(**fields)
) )
return result.rowcount or 0 return int(getattr(result, "rowcount", 0) or 0)
def get_task(task_id: UUID) -> Optional[Task]: def get_task(task_id: UUID) -> Optional[Task]:

View File

@ -121,7 +121,7 @@ def generate_task_title(
) )
.values(**values) .values(**values)
) )
applied = bool(result.rowcount) applied = bool(getattr(result, "rowcount", 0))
if response is not None and caps is not None: if response is not None and caps is not None:
usage = getattr(response, "usage", None) usage = getattr(response, "usage", None)
try: try:

View File

@ -63,7 +63,7 @@ class ToolContext:
tool_base: Path # fs/shell 类工具的 base_dir(cwd / task 目录) tool_base: Path # fs/shell 类工具的 base_dir(cwd / task 目录)
ur_path: Path # user_root(输出渲染相对路径 + host-side 落点) ur_path: Path # user_root(输出渲染相对路径 + host-side 落点)
working_dir_path: Path # 该 task 的宿主工作目录绝对路径 working_dir_path: Path # 该 task 的宿主工作目录绝对路径
task_id: str task_id: UUID
uid: UUID uid: UUID
cfg: dict # config/agent.yaml(quotas 段) cfg: dict # config/agent.yaml(quotas 段)
caps: Any # ModelCapabilities(enable_run_python) caps: Any # ModelCapabilities(enable_run_python)
@ -115,8 +115,8 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
AskUserTool(**base), AskUserTool(**base),
ReadTool(**base), WriteTool(**base), EditTool(**base), ReadTool(**base), WriteTool(**base), EditTool(**base),
GlobTool(**base), GrepTool(**base), GlobTool(**base), GrepTool(**base),
ShellTool(task_id=ctx.task_id, **base), ShellTool(task_id=str(ctx.task_id), **base),
CheckProcessTool(task_id=ctx.task_id, **base), CheckProcessTool(task_id=str(ctx.task_id), **base),
WebFetchTool(**base), WebFetchTool(**base),
] ]
@ -183,7 +183,7 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
return [WechatPushTool(ctx.uid, task_id=ctx.task_id, **wd_base)] return [WechatPushTool(ctx.uid, task_id=ctx.task_id, **wd_base)]
def _run_python() -> list: def _run_python() -> list:
return [RunPythonTool(task_id=ctx.task_id, **base)] return [RunPythonTool(task_id=str(ctx.task_id), **base)]
def _office_to_pdf() -> list: def _office_to_pdf() -> list:
# LibreOffice 只装在 backend hostDocker agent 通过 typed tool 转换用户目录内 # LibreOffice 只装在 backend hostDocker agent 通过 typed tool 转换用户目录内
@ -195,7 +195,7 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
# 媒体段同源);本次 run 锁定该 variant,下一条消息可重选。 # 媒体段同源);本次 run 锁定该 variant,下一条消息可重选。
if ctx.img_cfg is None: if ctx.img_cfg is None:
return [] return []
cls_kwargs = dict( cls_kwargs: dict[str, Any] = dict(
image_variant_cfg=ctx.img_cfg, variant_key=ctx.img_key, image_variant_cfg=ctx.img_cfg, variant_key=ctx.img_key,
working_dir=ctx.working_dir_path, task_id=ctx.task_id, user_id=ctx.uid, working_dir=ctx.working_dir_path, task_id=ctx.task_id, user_id=ctx.uid,
daily_limit=images_per_day, **base, daily_limit=images_per_day, **base,
@ -235,7 +235,9 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
)] )]
def _web_search() -> list: def _web_search() -> list:
return [WebSearchTool(cfg=BochaConfig.load())] cfg = BochaConfig.load()
assert cfg is not None
return [WebSearchTool(cfg=cfg)]
# ── 注册表:(组名, gate, factory)。gate 判定统一零参 bool;新工具在此加行 ── # ── 注册表:(组名, gate, factory)。gate 判定统一零参 bool;新工具在此加行 ──
registry: list[tuple[str, Callable[[], bool], Callable[[], list]]] = [ registry: list[tuple[str, Callable[[], bool], Callable[[], list]]] = [

View File

@ -351,8 +351,8 @@ def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
{"cutoff": cutoff, "cutoff_24h": cutoff_24h}, {"cutoff": cutoff, "cutoff_24h": cutoff_24h},
).fetchall() ).fetchall()
out = [] out: list[dict[str, Any]] = []
totals = { counts: dict[str, int] = {
"salvaged": 0, "salvaged": 0,
"malformed": 0, "malformed": 0,
"salvaged_24h": 0, "salvaged_24h": 0,
@ -377,7 +377,7 @@ def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
("salvaged_24h", saved_24h), ("salvaged_24h", saved_24h),
("malformed_24h", residual_24h), ("malformed_24h", residual_24h),
): ):
totals[key] += value counts[key] += value
out.append({ out.append({
"model_profile": model_profile or "?", "model_profile": model_profile or "?",
"tool": tool or "?", "tool": tool or "?",
@ -399,9 +399,10 @@ def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
), ),
reverse=True, reverse=True,
) )
totals["recovery_rate"] = _rate(totals["salvaged"], totals["malformed"]) totals: dict[str, int | float | None] = dict(counts)
totals["recovery_rate"] = _rate(counts["salvaged"], counts["malformed"])
totals["recovery_rate_24h"] = _rate( totals["recovery_rate_24h"] = _rate(
totals["salvaged_24h"], totals["malformed_24h"] counts["salvaged_24h"], counts["malformed_24h"]
) )
return {"days": days, "rows": out, "total": totals} return {"days": days, "rows": out, "total": totals}

View File

@ -333,7 +333,9 @@ class ILinkClient:
# —— 发文件(getuploadurl → AES-128-ECB → CDN → file_item)—— # —— 发文件(getuploadurl → AES-128-ECB → CDN → file_item)——
def _upload_file(self, to_user_id: str, data: bytes) -> dict[str, Any]: def _upload_file(self, to_user_id: str, data: bytes) -> dict[str, Any]:
rawsize = len(data) rawsize = len(data)
rawmd5 = hashlib.md5(data).hexdigest() # iLink 上传协议字段名即 rawfilemd5算法由服务端契约固定这里只做
# 传输完整性字段,不用于本地认证或密码存储。
rawmd5 = hashlib.md5(data, usedforsecurity=False).hexdigest()
aeskey = os.urandom(16) aeskey = os.urandom(16)
filekey = os.urandom(16).hex() filekey = os.urandom(16).hex()
ciphertext = _aes_ecb_pkcs7(data, aeskey) ciphertext = _aes_ecb_pkcs7(data, aeskey)

View File

@ -42,7 +42,10 @@ def _aes_key() -> bytes:
def _signature(timestamp: str, nonce: str, encrypt: str) -> str: def _signature(timestamp: str, nonce: str, encrypt: str) -> str:
arr = sorted([callback_token(), timestamp, nonce, encrypt]) arr = sorted([callback_token(), timestamp, nonce, encrypt])
return hashlib.sha1("".join(arr).encode("utf-8")).hexdigest() # 企业微信回调协议固定使用 SHA-1算法不可单方面升级。
return hashlib.sha1(
"".join(arr).encode("utf-8"), usedforsecurity=False
).hexdigest()
def _aes_decrypt(encrypt_b64: str) -> bytes: def _aes_decrypt(encrypt_b64: str) -> bytes:

View File

@ -87,9 +87,11 @@ def evaluate_assertion(
text = observation.artifact(spec.path).decode("utf-8") text = observation.artifact(spec.path).decode("utf-8")
passed = spec.value in text passed = spec.value in text
elif spec.type == "max_cost_cny": elif spec.type == "max_cost_cny":
assert spec.max_value is not None
passed = observation.cost_cny <= float(spec.max_value) passed = observation.cost_cny <= float(spec.max_value)
detail = f"{observation.cost_cny:.6f} CNY" detail = f"{observation.cost_cny:.6f} CNY"
elif spec.type == "max_duration_s": elif spec.type == "max_duration_s":
assert spec.max_value is not None
passed = observation.duration_s <= float(spec.max_value) passed = observation.duration_s <= float(spec.max_value)
detail = f"{observation.duration_s:.3f}s" detail = f"{observation.duration_s:.3f}s"
except (FileNotFoundError, UnicodeDecodeError, zipfile.BadZipFile, OSError) as exc: except (FileNotFoundError, UnicodeDecodeError, zipfile.BadZipFile, OSError) as exc:

24
mypy.ini Normal file
View File

@ -0,0 +1,24 @@
[mypy]
python_version = 3.12
# Tool implementations expose heterogeneous, JSON-schema-defined signatures.
# They are invoked dynamically by ExecutorHost from JSON arguments, so the
# uniform Tool.execute(**kwargs) declaration is an interface marker rather than
# a substitutable Python call signature.
[mypy-tools.*]
disable_error_code = override
# These runtime dependencies do not publish typing metadata. Keep the exception
# scoped to their import namespaces instead of suppressing missing imports for
# application modules.
[mypy-yaml.*]
ignore_missing_imports = True
[mypy-markdown.*]
ignore_missing_imports = True
[mypy-pilk.*]
ignore_missing_imports = True
[mypy-croniter.*]
ignore_missing_imports = True

View File

@ -11,7 +11,8 @@ import re
import sys import sys
from pathlib import Path from pathlib import Path
from docx import Document from docx import Document as create_document
from docx.document import Document
from docx.enum.text import WD_ALIGN_PARAGRAPH from docx.enum.text import WD_ALIGN_PARAGRAPH
from docx.opc.constants import RELATIONSHIP_TYPE as RT from docx.opc.constants import RELATIONSHIP_TYPE as RT
from docx.oxml import OxmlElement from docx.oxml import OxmlElement
@ -141,7 +142,7 @@ def add_external_link(paragraph, url: str, text: str, *, size_pt: float) -> None
# ───────────────────────── 文档初始化 ───────────────────────── # ───────────────────────── 文档初始化 ─────────────────────────
def init_doc(color: bool) -> Document: def init_doc(color: bool) -> Document:
doc = Document() doc = create_document()
section = doc.sections[0] section = doc.sections[0]
section.page_height = Cm(29.7) section.page_height = Cm(29.7)
section.page_width = Cm(21) section.page_width = Cm(21)

View File

@ -13,7 +13,8 @@ import re
import sys import sys
from pathlib import Path from pathlib import Path
from docx import Document from docx import Document as create_document
from docx.document import Document
from docx.enum.text import WD_ALIGN_PARAGRAPH from docx.enum.text import WD_ALIGN_PARAGRAPH
from docx.oxml import OxmlElement from docx.oxml import OxmlElement
from docx.oxml.ns import qn from docx.oxml.ns import qn
@ -79,7 +80,7 @@ PROFILES = {
# ───────────────────────── 文档初始化 ───────────────────────── # ───────────────────────── 文档初始化 ─────────────────────────
def init_doc(prof: dict) -> Document: def init_doc(prof: dict) -> Document:
doc = Document() doc = create_document()
section = doc.sections[0] section = doc.sections[0]
section.page_height = Cm(29.7) section.page_height = Cm(29.7)

View File

@ -1,4 +1,9 @@
# 打包工具也纳入 pip-audit安全下限需随部署一起升级避免应用依赖干净但
# 虚拟环境自带的 pip 仍触发已知漏洞。
pip>=26.1.2
litellm>=1.83.0 # zai provider(GLM)要 ≥1.83;PR #17307 merge 后才内置 litellm>=1.83.0 # zai provider(GLM)要 ≥1.83;PR #17307 merge 后才内置
aiohttp>=3.14.1 # litellm 传递依赖3.14.1 修复 2026-08 pip-audit 命中项
pyyaml>=6.0 pyyaml>=6.0
click>=8.1.0 click>=8.1.0
rich>=13.7.0 rich>=13.7.0
@ -7,7 +12,7 @@ rich>=13.7.0
python-pptx>=0.6.21 python-pptx>=0.6.21
python-docx>=1.1.0 python-docx>=1.1.0
matplotlib>=3.8.0 matplotlib>=3.8.0
Pillow>=9.0.0 # ppt skill(SVG-first)svg_finalize:配图裁切/内嵌 Pillow>=12.3.0 # ppt skill(SVG-first)svg_finalize:配图裁切/内嵌 + 安全修复
# ppt skill 可选 —— 老版 Office(<2019)的 SVG→PNG 兜底;现代 PowerPoint 直接渲 SVG 无需,核心不依赖: # ppt skill 可选 —— 老版 Office(<2019)的 SVG→PNG 兜底;现代 PowerPoint 直接渲 SVG 无需,核心不依赖:
# svglib>=1.5.0 # svglib>=1.5.0
# reportlab>=4.0.0 # reportlab>=4.0.0
@ -40,7 +45,7 @@ croniter>=2.0
# 微信接入(§8.7 ClawBot):segno 渲绑定二维码;cryptography 做凭据列加密 + 文件 AES-128-ECB # 微信接入(§8.7 ClawBot):segno 渲绑定二维码;cryptography 做凭据列加密 + 文件 AES-128-ECB
segno>=1.6 segno>=1.6
cryptography>=42.0 cryptography>=48.0.1
# broker 外置(§7.0,蓝绿双实例跨进程 SSE/cancel):ZCBOT_REDIS_URL 设了才用, # broker 外置(§7.0,蓝绿双实例跨进程 SSE/cancel):ZCBOT_REDIS_URL 设了才用,
# 不设走进程内 broker(dev 不需要起 redis);fakeredis 供单测(无需真 redis 服务) # 不设走进程内 broker(dev 不需要起 redis);fakeredis 供单测(无需真 redis 服务)
@ -54,8 +59,9 @@ alembic>=1.13.0
# §7 Phase G / D: 纯 JSON API(FastAPI + 原生 SSE),前端由 platform 提供 # §7 Phase G / D: 纯 JSON API(FastAPI + 原生 SSE),前端由 platform 提供
fastapi>=0.111.0 fastapi>=0.111.0
starlette>=1.3.1 # FastAPI 底层 ASGI显式固定安全下限
uvicorn[standard]>=0.30.0 uvicorn[standard]>=0.30.0
python-multipart>=0.0.9 # files upload multipart 解析 python-multipart>=0.0.31 # files upload multipart 解析 + 安全修复
pyjwt>=2.8.0 # /v1/auth/login HS256 token mint/verify(§7 D' 过渡形态) pyjwt>=2.8.0 # /v1/auth/login HS256 token mint/verify(§7 D' 过渡形态)
bcrypt>=4.1.0 # /v1/auth/login_password 密码哈希(users.password_hash) bcrypt>=4.1.0 # /v1/auth/login_password 密码哈希(users.password_hash)
@ -63,6 +69,7 @@ bcrypt>=4.1.0 # /v1/auth/login_password 密码哈希(users.password_
# pymatgen skill: 无机材料计算(晶体结构/XRD/相图/Materials Project) # pymatgen skill: 无机材料计算(晶体结构/XRD/相图/Materials Project)
pymatgen>=2024.0 pymatgen>=2024.0
mp-api>=0.41.0 mp-api>=0.41.0
pydantic-settings>=2.14.2 # mp-api 传递依赖;固定安全下限
# stats_ml skill: 统计建模与 ML(sklearn 必装,statsmodels 必装,PyMC 可选) # stats_ml skill: 统计建模与 ML(sklearn 必装,statsmodels 必装,PyMC 可选)
scikit-learn>=1.4.0 scikit-learn>=1.4.0
statsmodels>=0.14.0 statsmodels>=0.14.0

View File

@ -0,0 +1,45 @@
from __future__ import annotations
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from core import procs
from core.proc_wrapper import _shell_argv as wrapper_shell_argv
from tools.shell import ShellTool
class ShellSecurityTests(unittest.TestCase):
def test_shell_argv_uses_explicit_interpreter(self) -> None:
argv = procs.shell_argv("echo ok")
wrapper_argv = wrapper_shell_argv("echo ok")
self.assertEqual(argv, wrapper_argv)
self.assertEqual(argv[-1], "echo ok")
if os.name == "nt":
self.assertEqual(argv[1:4], ["/d", "/s", "/c"])
else:
self.assertEqual(argv[:2], ["/bin/sh", "-c"])
def test_foreground_shell_still_executes_command(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
result = ShellTool(base_dir=Path(tmp)).execute("echo shell-ok")
self.assertIn("shell-ok", result)
self.assertIn("[exit 0]", result)
def test_background_shell_persists_explicit_argv(self) -> None:
with tempfile.TemporaryDirectory() as tmp, patch(
"tools.shell.procs.count_running", return_value=0
), patch("tools.shell.procs.launch_host", return_value=("proc-1", Path(tmp))) as launch:
result = ShellTool(base_dir=Path(tmp)).execute(
"echo background-ok", background=True
)
self.assertIn("proc_id=proc-1", result)
kwargs = launch.call_args.kwargs
self.assertEqual(kwargs["argv"], procs.shell_argv("echo background-ok"))
self.assertNotIn("shell_cmd", kwargs)
if __name__ == "__main__":
unittest.main()

View File

@ -126,6 +126,7 @@ class LookAtImageTool(Tool):
) )
if api_err: if api_err:
return api_err return api_err
assert resp is not None
answer, _truncated = extract_chat_answer(resp) answer, _truncated = extract_chat_answer(resp)
if not answer: if not answer:

View File

@ -140,7 +140,11 @@ class MaterialsProjectSearchSummaryTool(Tool):
f"[Error] 一次最多 {self._MAX_BATCH} 个化学式(收到 {len(formulas)});请分批调用。" f"[Error] 一次最多 {self._MAX_BATCH} 个化学式(收到 {len(formulas)});请分批调用。"
) )
seen: set[str] = set() seen: set[str] = set()
uniq = [f for f in formulas if not (f in seen or seen.add(f))] uniq = []
for formula in formulas:
if formula not in seen:
seen.add(formula)
uniq.append(formula)
try: try:
session = _mpr() session = _mpr()
except Exception as e: except Exception as e:
@ -172,7 +176,8 @@ class MaterialsProjectSearchSummaryTool(Tool):
) )
if self.working_dir is not None: if self.working_dir is not None:
h = hashlib.sha1( h = hashlib.sha1(
",".join(a["formula"] for a in agg).encode("utf-8") ",".join(a["formula"] for a in agg).encode("utf-8"),
usedforsecurity=False,
).hexdigest()[:8] ).hexdigest()[:8]
dest = self.working_dir / "materials" / f"mp_search_batch_{h}.json" dest = self.working_dir / "materials" / f"mp_search_batch_{h}.json"
try: try:

View File

@ -168,6 +168,7 @@ class ReadDocumentTool(Tool):
) )
if api_err: if api_err:
return api_err return api_err
assert resp is not None
answer, truncated = extract_chat_answer(resp) answer, truncated = extract_chat_answer(resp)
if not answer: if not answer:

View File

@ -102,7 +102,7 @@ class ShellTool(Tool):
anchor, self.task_id, anchor, self.task_id,
kind="shell", kind="shell",
command_display=command, command_display=command,
shell_cmd=command, argv=procs.shell_argv(command),
cwd=self.base_dir, timeout_s=timeout_s, env=None, cwd=self.base_dir, timeout_s=timeout_s, env=None,
) )
return ( return (
@ -113,8 +113,7 @@ class ShellTool(Tool):
try: try:
result = subprocess.run( result = subprocess.run(
command, procs.shell_argv(command),
shell=True,
cwd=str(self.base_dir), cwd=str(self.base_dir),
capture_output=True, capture_output=True,
timeout=timeout, timeout=timeout,

View File

@ -12,6 +12,8 @@ app.state 内存(轻);其余走 DB 聚合(GROUP BY,无 N+1)。指标只读、不
""" """
from __future__ import annotations from __future__ import annotations
import importlib
from types import ModuleType
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from typing import Any from typing import Any
from uuid import UUID from uuid import UUID
@ -26,8 +28,9 @@ from core.storage.models import Task, UsageEvent, User, UserDiskUsage
from .broker import broker from .broker import broker
resource: ModuleType | None
try: try:
import resource # Unix only;Windows dev 无此模块,RSS 监控降级跳过 resource = importlib.import_module("resource")
except ImportError: # pragma: no cover - Windows except ImportError: # pragma: no cover - Windows
resource = None resource = None
@ -302,6 +305,6 @@ def register_admin_routes(app: FastAPI, require_admin) -> None:
result = s.execute( result = s.execute(
update(User).where(User.user_id == target).values(plan=plan or None) update(User).where(User.user_id == target).values(plan=plan or None)
) )
if result.rowcount == 0: if getattr(result, "rowcount", 0) == 0:
raise HTTPException(404, f"user not found: {uid}") raise HTTPException(404, f"user not found: {uid}")
return {"user_id": str(target), "plan": plan} return {"user_id": str(target), "plan": plan}

View File

@ -8,7 +8,9 @@ bg proc 清扫 / 孤儿 run 收割 / 优雅 drain。每个 start_* 返回 asynci
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import importlib
import os import os
from types import ModuleType
from typing import Optional from typing import Optional
from sqlalchemy import or_, update from sqlalchemy import or_, update
@ -19,8 +21,9 @@ from core.storage.models import Task
from .broker import broker from .broker import broker
from .common import INSTANCE from .common import INSTANCE
resource: ModuleType | None
try: try:
import resource # Unix only;Windows dev 无此模块,RSS 监控自动降级跳过 resource = importlib.import_module("resource")
except ImportError: # pragma: no cover - Windows except ImportError: # pragma: no cover - Windows
resource = None resource = None
@ -55,8 +58,9 @@ def reap_stale_runs() -> None:
run_error="server restarted before run finished", run_error="server restarted before run finished",
) )
) )
if result.rowcount: rowcount = int(getattr(result, "rowcount", 0) or 0)
print(f"[startup] reaped {result.rowcount} stale active run(s)") if rowcount:
print(f"[startup] reaped {rowcount} stale active run(s)")
def start_disk_scanner(cfg: dict) -> asyncio.Task: def start_disk_scanner(cfg: dict) -> asyncio.Task:

View File

@ -170,7 +170,7 @@ class RedisRunBroker:
self._aredis = async_client if async_client is not None else _aredis.from_url( self._aredis = async_client if async_client is not None else _aredis.from_url(
url, decode_responses=True, url, decode_responses=True,
) )
self._pubsub = None self._pubsub: Any = None
self._reader_task: Optional[asyncio.Task] = None self._reader_task: Optional[asyncio.Task] = None
self._subs: dict[UUID, set[asyncio.Queue]] = defaultdict(set) self._subs: dict[UUID, set[asyncio.Queue]] = defaultdict(set)
self._loop: Optional[asyncio.AbstractEventLoop] = None self._loop: Optional[asyncio.AbstractEventLoop] = None
@ -231,11 +231,12 @@ class RedisRunBroker:
断线 1s 退避重连,并把 _subs 里所有活跃 channel 重挂上(订阅状态不丢)""" 断线 1s 退避重连,并把 _subs 里所有活跃 channel 重挂上(订阅状态不丢)"""
while True: while True:
try: try:
self._pubsub = self._aredis.pubsub() pubsub = self._aredis.pubsub()
self._pubsub = pubsub
channels = [self._CTL_CHANNEL] + [self._chan(t) for t in self._subs] channels = [self._CTL_CHANNEL] + [self._chan(t) for t in self._subs]
await self._pubsub.subscribe(*channels) await pubsub.subscribe(*channels)
while True: while True:
msg = await self._pubsub.get_message( msg = await pubsub.get_message(
ignore_subscribe_messages=True, timeout=5.0 ignore_subscribe_messages=True, timeout=5.0
) )
if msg is None or msg.get("type") != "message": if msg is None or msg.get("type") != "message":

View File

@ -63,7 +63,7 @@ def _cache_pdf_path(pptx_path: Path) -> Path:
""" """
st = pptx_path.stat() st = pptx_path.stat()
sig = f"{st.st_mtime_ns}-{st.st_size}".encode("utf-8") sig = f"{st.st_mtime_ns}-{st.st_size}".encode("utf-8")
digest = hashlib.sha1(sig).hexdigest()[:12] digest = hashlib.sha1(sig, usedforsecurity=False).hexdigest()[:12]
return pptx_path.parent / _PREVIEW_DIRNAME / f"{pptx_path.stem}.{digest}.pdf" return pptx_path.parent / _PREVIEW_DIRNAME / f"{pptx_path.stem}.{digest}.pdf"

View File

@ -7,7 +7,7 @@ from __future__ import annotations
import asyncio import asyncio
import time import time
from typing import Any from typing import Any, Optional
from uuid import UUID from uuid import UUID
from fastapi import Depends, HTTPException from fastapi import Depends, HTTPException
@ -76,9 +76,9 @@ def register_message_routes(app, *, require_user) -> None:
@app.get("/v1/tasks/{task_id}/messages", tags=["messages"]) @app.get("/v1/tasks/{task_id}/messages", tags=["messages"])
def list_messages( def list_messages(
task_id: str, task_id: str,
limit: int = None, limit: Optional[int] = None,
before_idx: int = None, before_idx: Optional[int] = None,
after_idx: int = None, after_idx: Optional[int] = None,
user_id: UUID = Depends(require_user), user_id: UUID = Depends(require_user),
): ):
"""task 历史消息(idx 升序);LiteLLM 原 payload 透传给前端,自行渲染。 """task 历史消息(idx 升序);LiteLLM 原 payload 透传给前端,自行渲染。

View File

@ -84,14 +84,16 @@ def register_schedule_routes(app, *, require_user) -> None:
.limit(page_size).offset(offset) .limit(page_size).offset(offset)
).scalars().all() ).scalars().all()
tids = [r.task_id for r in rows] tids = [r.task_id for r in rows]
msg_counts = ( msg_counts: dict[UUID, int] = {}
dict(s.execute( if tids:
count_rows = s.execute(
select(Message.task_id, func.count()) select(Message.task_id, func.count())
.where(Message.task_id.in_(tids)) .where(Message.task_id.in_(tids))
.group_by(Message.task_id) .group_by(Message.task_id)
).all()) ).all()
if tids else {} msg_counts = {
) task_id: int(count) for task_id, count in count_rows
}
usage = usage_aggregates(s, tids) usage = usage_aggregates(s, tids)
return { return {

View File

@ -168,14 +168,16 @@ def register_task_routes(app, *, require_user) -> None:
).scalars().all() ).scalars().all()
tids = [r.task_id for r in rows] tids = [r.task_id for r in rows]
msg_counts = ( msg_counts: dict[UUID, int] = {}
dict(s.execute( if tids:
count_rows = s.execute(
select(Message.task_id, func.count()) select(Message.task_id, func.count())
.where(Message.task_id.in_(tids)) .where(Message.task_id.in_(tids))
.group_by(Message.task_id) .group_by(Message.task_id)
).all()) ).all()
if tids else {} msg_counts = {
) task_id: int(count) for task_id, count in count_rows
}
usage = usage_aggregates(s, tids) usage = usage_aggregates(s, tids)
return { return {
@ -227,13 +229,16 @@ def register_task_routes(app, *, require_user) -> None:
) )
).scalars().all() ).scalars().all()
} }
msg_counts = dict( msg_counts: dict[UUID, int] = {}
s.execute( if rows:
count_rows = s.execute(
select(Message.task_id, func.count()) select(Message.task_id, func.count())
.where(Message.task_id.in_(list(rows.keys()))) .where(Message.task_id.in_(list(rows.keys())))
.group_by(Message.task_id) .group_by(Message.task_id)
).all() ).all()
) if rows else {} msg_counts = {
task_id: int(count) for task_id, count in count_rows
}
usage = usage_aggregates(s, list(rows.keys())) usage = usage_aggregates(s, list(rows.keys()))
for kind, tid in tids.items(): for kind, tid in tids.items():
row = rows.get(tid) if tid else None row = rows.get(tid) if tid else None
@ -456,7 +461,7 @@ def register_task_routes(app, *, require_user) -> None:
.where(Task.task_id == tid, Task.user_id == user_id) .where(Task.task_id == tid, Task.user_id == user_id)
.values(**updates) .values(**updates)
) )
if result.rowcount == 0: if getattr(result, "rowcount", 0) == 0:
raise HTTPException(404, f"task not found: {tid}") raise HTTPException(404, f"task not found: {tid}")
row = s.execute(select(Task).where(Task.task_id == tid)).scalar_one() row = s.execute(select(Task).where(Task.task_id == tid)).scalar_one()
n = s.execute( n = s.execute(