Compare commits
16 Commits
b4df60062e
...
0c577ba0a5
| Author | SHA1 | Date |
|---|---|---|
|
|
0c577ba0a5 | |
|
|
4a6aaaf34d | |
|
|
02a69058df | |
|
|
1035b12847 | |
|
|
7356d25652 | |
|
|
514d36c481 | |
|
|
80a658eba4 | |
|
|
91202b6172 | |
|
|
e8dbfa57a5 | |
|
|
2b3692c8bf | |
|
|
aeecc7f0f3 | |
|
|
4f87bf14ee | |
|
|
5fbf3746be | |
|
|
425ea59937 | |
|
|
55dc8eb99d | |
|
|
efe4a91c33 |
33
CLAUDE.md
|
|
@ -5,3 +5,36 @@
|
|||
- **Python 虚拟环境**: `.venv/`(项目根目录下),所有依赖装在里面
|
||||
- 跑脚本 / 测试一律用 `.venv/Scripts/python.exe ...`,**不要用全局 `python`**(没装 litellm/python-pptx 等会报 ModuleNotFoundError)
|
||||
- requirements 见 `requirements.txt`
|
||||
|
||||
## 开发阶段心智
|
||||
|
||||
当前处于开发阶段(尚未发布给真实用户)。改需求 / 重构时,**以最优实现为准,不为旧数据 / 旧字段 / 旧 API 留兼容层**:
|
||||
- DB schema 变 → 直接改 model + 写一条干净的 migration(必要时清空旧 row,不写双向兼容代码)
|
||||
- 字段语义变 → 全量替换,不留 `legacy_xxx` / `*_v2` 并存
|
||||
- CLI / REPL 选项变 → 直接改,不留 deprecated 别名
|
||||
- 只有当用户明确说"这条要保留兼容"时才写兼容代码
|
||||
|
||||
理由:兼容层就是技术债,开发期写了之后忘记删反而拖累;真上线后再视情况补迁移路径。
|
||||
|
||||
## 文档维护
|
||||
|
||||
每完成一步实现(commit 前),**必须更新 `PROGRESS.md`**:
|
||||
- "已完成关键能力" 段加一条 `YYYY-MM-DD / <短标题>:<改了什么>`
|
||||
- 状态表(§7 B Step 几 / Phase 几)若变化跟着改
|
||||
- 文件清单若新增 / 删除模块跟着改
|
||||
|
||||
**只有以下情况才动 `DESIGN.md`**(避免把工程笔记沉淀成设计):
|
||||
- 架构 / 心智模型变化(如 §7.1 task-primary 重写)
|
||||
- 取舍决策推翻或新增(§5 / §7.9 类内容)
|
||||
- API / schema 字段语义变化(§7.2 / §7.4)
|
||||
- 实施中发现 DESIGN 描述与代码偏离 → 同步改回
|
||||
|
||||
bug 修复、重构、新加 skill、调参 —— **不动 DESIGN**,只更 PROGRESS。
|
||||
|
||||
**改任何对外行为(CLI 选项 / REPL 命令 / env 变量 / 文件布局 / migration 步骤)→ 同步更新 `RUN.md`**:
|
||||
- 新加 / 改 / 删 CLI 子命令 + 选项时,改"日常命令"段
|
||||
- env 变量 / 启动初始化变化时,改"环境" / "一次性初始化"段
|
||||
- 真实踩过的坑(用户报或自己跑出来),加一行到"故障兜底"表
|
||||
- 纯内部重构 / 不影响用户怎么跑的 —— **不动 RUN**
|
||||
|
||||
三文档边界:`DESIGN`=为什么(架构 / 取舍),`PROGRESS`=做到哪(状态 / 历史),`RUN`=怎么跑(命令 / env / 兜底)。一次改动可能动多个,但每个动的理由要符合上述边界。
|
||||
|
|
|
|||
601
DESIGN.md
|
|
@ -1,520 +1,445 @@
|
|||
# 设计文档
|
||||
|
||||
> 一个本地运行的个人任务 agent。覆盖三类工作:写汇报 PPT、写科研申报书、写代码。
|
||||
> 模型自由(LiteLLM 接 OpenAI-compatible),代码可控(目标 1500-2000 行 Python,自己读得懂)。
|
||||
> 本地运行的个人任务 agent,覆盖三类工作:汇报 PPT、科研申报书、代码。
|
||||
> 模型自由(LiteLLM 接 OpenAI-compatible),代码可控(目标 1500-2000 行 Python)。
|
||||
|
||||
---
|
||||
|
||||
## 1. 边界
|
||||
|
||||
### 做什么
|
||||
- **PPT**:文本 / 会议纪要 → `.pptx`(用 `python-pptx`)
|
||||
- **科研申报**:课题信息 → 分章节 `.docx`(用 `python-docx`)
|
||||
- **编码**:文件编辑、shell 执行、迭代验证
|
||||
**做**:PPT(`python-pptx`)/ 申报书(`python-docx`)/ 编码(读写文件 + shell + 迭代验证)。
|
||||
**不做**:子 agent / IM 渠道 / 自定义 RAG / 锁定 Anthropic / Eval Suite(个人工具 dogfooding 替代)。多用户 / Web UI 归 §7。
|
||||
|
||||
### 不做什么
|
||||
- 子 agent / IM 渠道 / 自定义 RAG / 锁定 Anthropic(注:多用户 / Web UI 是 §7 SaaS 化路线,personal-tool 阶段不做)
|
||||
- **Eval Suite**:个人工具用 dogfooding 判断模型升级,造作 case 没区分度
|
||||
|
||||
### 关键约束
|
||||
- 模型自由:LiteLLM 接 OpenAI-compatible 任意 provider(默认 DeepSeek V4)
|
||||
**关键约束**:
|
||||
- 模型自由:LiteLLM + OpenAI-compatible(默认 DeepSeek V4)
|
||||
- 任务持久化:任意时刻关机,下次能恢复
|
||||
- 演化性:模型升级时 agent 跟着升级,不需要大改架构
|
||||
- **形态兼容**:本地 CLI 与 SaaS 共享同一份 core 和同一种 storage(PG,无 SQLite / JSON 分支);CLI 长期保留(本地直跑 + `--remote` API client 双模式),不会被 HTTP API 取代(详 §7.0)
|
||||
- 演化性:模型升级不需要大改架构
|
||||
- **形态兼容**:本地 CLI 与 SaaS 共享同一份 core 和 storage(PG,无 SQLite / JSON 分支);CLI 长期保留(本地直跑 + `--remote` API client 双模式)
|
||||
|
||||
---
|
||||
|
||||
## 2. 架构
|
||||
|
||||
### 目录树(实际)
|
||||
|
||||
```
|
||||
zcbot/
|
||||
├── core/
|
||||
│ ├── capabilities.py # ModelCapabilities,从 yaml 加载
|
||||
│ ├── llm.py # LiteLLM 封装,按 capabilities 自动启用 features
|
||||
│ ├── llm.py # LiteLLM 封装,按 capabilities 自动启 features
|
||||
│ ├── loop.py # ReAct 主循环
|
||||
│ ├── probe.py # 真实探测对账 yaml 声称的能力
|
||||
│ ├── session.py # 消息列表 + meta + 落盘 messages.json
|
||||
│ ├── skills.py # SkillRegistry (Anthropic 渐进披露格式)
|
||||
│ └── task.py # TaskState (mode/desc/status/tokens/timestamps)
|
||||
│ ├── session.py # 消息列表 + meta + 落盘
|
||||
│ ├── skills.py # SkillRegistry (Anthropic 渐进披露)
|
||||
│ └── task.py # TaskState
|
||||
├── tools/
|
||||
│ ├── base.py # Tool 基类 + _resolve 路径
|
||||
│ ├── base.py # Tool 基类 + _resolve
|
||||
│ ├── fs.py # read / write / edit (唯一匹配) / glob / grep
|
||||
│ ├── shell.py # subprocess + 黑名单
|
||||
│ ├── run_python.py # tmp .py + subprocess,过滤敏感 env
|
||||
│ └── skill_tool.py # load_skill
|
||||
├── skills/
|
||||
│ ├── coding/ # SKILL.md
|
||||
│ ├── ppt/ # SKILL.md + references/ + scripts/ + assets/
|
||||
│ └── proposal/ # SKILL.md
|
||||
├── prompts/system/
|
||||
│ └── general_v1.md
|
||||
├── config/
|
||||
│ ├── agent.yaml
|
||||
│ └── models/
|
||||
│ └── deepseek_v4.yaml # flash + pro 两档
|
||||
├── skills/{coding,ppt,proposal}/ # SKILL.md + references / scripts / assets
|
||||
├── prompts/system/general_v1.md
|
||||
├── config/{agent.yaml, models/deepseek_v4.yaml}
|
||||
├── workspace/
|
||||
│ ├── memory/ # 双层记忆 (workspace 级,跨 task 共享)
|
||||
│ │ ├── core.md # 注 system prompt,常驻
|
||||
│ │ └── extended/ # 索引(标题+绝对路径)注 prompt,内容靠 read 工具按需拉
|
||||
│ │ └── *.md
|
||||
│ └── tasks/<task_id>/ # task_dir:仅 skill 产物,state/messages 在 PG
|
||||
│ ├── spec_lock.md # skill 阶段一产物 (proposal/ppt)
|
||||
│ ├── source/ # proposal 用户素材 (PDF / 团队介绍)
|
||||
│ ├── source.md # ppt 转过的素材
|
||||
│ ├── sections/ # proposal 逐章 md (01_summary.md ... 12_appendix.md)
|
||||
│ ├── slides/ # ppt 中间素材 (chart_p?.png)
|
||||
│ └── <topic>.docx / .pptx # 最终产物
|
||||
├── main.py # 装配 (build_agent)
|
||||
└── cli.py # CLI: chat / tasks / probe
|
||||
│ └── users/<user_id>/
|
||||
│ ├── .memory/{core.md, extended/*.md} # 跨 task 共享记忆(user 级,dotfile 隔离)
|
||||
│ └── <working_dir>/ # 工作目录,用户起名(同 working_dir 多 task 共享),仅 skill 产物
|
||||
└── {main.py, cli.py}
|
||||
```
|
||||
|
||||
**task_dir = `workspace/tasks/<task_id>/`,所有 skill 产物都写到这里**。task_dir 绝对路径在 system prompt 里显式给 agent,SKILL.md 的 `<task_dir>` 占位符指向它。如果 agent 写错位置(写到 cwd / `skills/` / repo 根),git status 会立刻报红 —— `.gitignore` 不再用无锚通配规则盖住污染。
|
||||
**工作目录(working_dir) = `workspace/users/<user_id>/<working_dir>/`,所有 skill 产物写到这里**,绝对路径在 system prompt 显式给 agent(prompt 里仍叫 `task_dir` 占位符,跟 SKILL.md DSL 一致)。写错位置(cwd / `skills/` / repo 根)git status 立刻报红。本地 CLI user_id 固定为 SENTINEL(`00000000-...`);web/JWT 路径用 `sub`。**`name`(任务显示名)必填**,**`working_dir` 可选**(留空 → 用 name 作目录名);两者都是简单名(不含 `/\..`、不以 `.` 起头,挡 `.memory`);同 `working_dir` 多 task 自动共享同目录(§7.1)。SaaS 化只是把 `workspace/` 换 `<storage_root>/`,布局不变。
|
||||
|
||||
### 启动时拼装顺序
|
||||
1. 读 `config/agent.yaml` 拿 default_model;`ZCBOT_DB_URL` 环境变量指向 PG(本地 dev 连远端测试 PG 或 docker compose 起的本地 PG;两形态同一种 schema)
|
||||
2. `ModelCapabilities.load("deepseek_v4.flash", config/models/)` 拿能力档案
|
||||
3. `LLM(caps)` 构造,从 env 读 API key
|
||||
4. 解析 task_dir(新建 or resume)
|
||||
5. 拼 system prompt:`prompts/system/general_v1.md` + `SkillRegistry.discovery_block()`(skill 列表)+ cwd + **task_dir 绝对路径**(产物根)
|
||||
6. 装配工具集(fs / shell / load_skill / run_python)
|
||||
7. 启动 REPL —— **新建路径不预占文件**(懒创建,见 §3.6)
|
||||
**启动**:读 `agent.yaml` → 加载 `ModelCapabilities` → `LLM(caps)` → 解析 task_dir → 拼 system prompt(general_v1.md + skill discovery + cwd + task_dir 绝对路径)→ 装配工具 → REPL。新建路径**懒创建**,不预占文件(§3.6)。`ZCBOT_DB_URL` 指 PG(本地 docker compose / 远端 dev / 生产)。
|
||||
|
||||
---
|
||||
|
||||
## 3. 核心组件
|
||||
|
||||
### 3.1 主循环(`core/loop.py`)
|
||||
ReAct 风格:LLM → 若有 tool_calls 就执行 → 把结果塞回消息列表 → 再调 LLM。无 tool_call 即返回。
|
||||
- 工具结果对模型截断到 16K 字符,对用户预览 400 字符
|
||||
- thinking spinner 由后台 daemon 线程每 100ms 刷新文本:`thinking... 1.3s ctx 12,345 tok`(累计 token 反映上下文大小)
|
||||
- 每轮 LLM 返回追加 dim 一行 `[in N out N t Xs]` —— 留痕本轮成本
|
||||
- assistant 文字走 `rich.markdown.Markdown` 渲染,粗体/列表/表格/代码块正常展示(非流式,整段渲染)
|
||||
- `max_iterations` 从 capabilities 读,不同模型不同
|
||||
ReAct:LLM → 若有 tool_calls 就执行 → 结果塞回消息 → 再调 LLM。无 tool_call 即返回。
|
||||
- 工具结果对模型截 16K 字符,用户预览 400 字符
|
||||
- 后台 daemon 线程每 100ms 刷 spinner:`thinking... 1.3s ctx 12,345 tok`
|
||||
- 每轮 LLM 返回追加 dim 一行 `[in N out N t Xs]`
|
||||
- assistant 文本走 `rich.markdown.Markdown` 整段渲染(非流式)
|
||||
- `max_iterations` 从 capabilities 读
|
||||
|
||||
### 3.2 Model Profile(`core/capabilities.py` + `config/models/*.yaml`)
|
||||
**核心思想**:每个模型一份 yaml 档案,agent 行为按档案动态调整。新模型 5 分钟接入,不改代码。
|
||||
|
||||
`ModelCapabilities` 字段:max/reliable_context、max_output、parallel_tools、tool_calling_quality、thinking_mode、reasoning_effort_levels、code_quality、enable_run_python、max_iterations、optimal_temperature、prompt_caching、extended_thinking、api_base、api_key_env。
|
||||
|
||||
每模型一份 yaml,agent 行为按档案动态调整。新模型 5 分钟接入,不改代码。
|
||||
字段:max/reliable_context、max_output、parallel_tools、tool_calling_quality、thinking_mode、reasoning_effort_levels、code_quality、enable_run_python、max_iterations、optimal_temperature、prompt_caching、extended_thinking、api_base、api_key_env。
|
||||
`LLM.chat` 按 capabilities 自动启 `parallel_tool_calls` / `reasoning_effort` / Anthropic prompt-caching header。
|
||||
|
||||
### 3.3 Capability Probing(`core/probe.py` + `cli.py probe`)
|
||||
yaml 是手填的,可能错。`probe` 用真实 LLM 调用对账:
|
||||
- `basic_chat`:连通性
|
||||
- `parallel_tools`:给两个独立工具,看 single response 是否 ≥2 个 tool_calls
|
||||
- `thinking_mode`:对 declared=True 的模型试 reasoning_effort,看 API 是否接受 + 是否产 reasoning_content
|
||||
- `long_context`:needle-in-haystack 简化版(opt-in,默认关)
|
||||
|
||||
不修改 yaml,只输出 rich Table 报告。退出码 0/2/3 区分 ok / mismatch / error。**显式触发,不进启动路径**(每次启动跑会烧 API)。
|
||||
yaml 是手填的,probe 用真实调用对账:`basic_chat` / `parallel_tools` / `thinking_mode` / `long_context`(opt-in)。不改 yaml,只出 rich Table 报告。**显式触发,不进启动路径**(避免烧 API)。
|
||||
|
||||
### 3.4 工具系统(Hybrid 范式)
|
||||
**JSON tool call**(`tools/`):read / write / edit / glob / grep / shell / run_python / load_skill —— 离散操作。
|
||||
**Code execution**(`run_python`):tmp `.py` + subprocess + 工作目录限制 + 敏感 env 过滤(`*API_KEY *TOKEN *SECRET *PASSWORD *PRIVATE_KEY`)—— 批处理 / 算数据 / 生成文档。
|
||||
关键设计:`edit` **唯一匹配**(CoreCoder 风格,old_str 重复即报错);工具按**原子操作**切分,不做 `make_pptx()` 这种高级封装。
|
||||
|
||||
**两类工具并存**:
|
||||
- **JSON tool call**(`tools/`):read / write / edit / glob / grep / shell / run_python / load_skill —— 处理离散操作
|
||||
- **Code execution**(`run_python`):tmp `.py` + subprocess + 工作目录限制 + 敏感 env 过滤(`*API_KEY *TOKEN *SECRET *PASSWORD *PRIVATE_KEY`)—— 处理批处理 / 算数据 / 生成文档
|
||||
|
||||
**关键设计**:
|
||||
- `edit` 用 **唯一匹配**约束(CoreCoder 风格):old_str 必须只出现一次,否则报错。防 LLM 改错地方。
|
||||
- 工具按**原子操作**切分,不做高级封装。`make_pptx()` ❌,`run_python(code)` 调 `python-pptx` ✅。粒度太粗会接收不到模型升级红利。
|
||||
|
||||
### 3.5 Skill 系统(Anthropic 渐进披露标准)
|
||||
对齐 Anthropic 2025-12 开放标准,跨平台兼容(Claude Code / Codex CLI / Gemini CLI 都用)。
|
||||
|
||||
**三层加载**:
|
||||
| 层 | 时机 | 内容 | Token |
|
||||
|---|------|------|------|
|
||||
| Discovery | agent 启动 | 仅 `name + description`,所有 skill 都读 | 几百 |
|
||||
| Activation | `load_skill(name)` | 完整 SKILL.md | 1000-5000 |
|
||||
| Execution | SKILL.md 指 `references/xxx` | 单个 reference 文件 | 视情况 |
|
||||
|
||||
**Skill 设计原则**:写 WHY+WHAT,不写 Step 1/2/3。让模型自己规划。description 要明确具体——决定模型能否触发。
|
||||
### 3.5 Skill 系统(Anthropic 渐进披露)
|
||||
对齐 Anthropic 2025-12 开放标准。三层加载:Discovery(`name + description`,几百 token)→ Activation(`load_skill(name)` 加载完整 SKILL.md,1-5K)→ Execution(SKILL.md 指 `references/xxx` 按需拉)。
|
||||
原则:写 WHY+WHAT,不写 Step 1/2/3。description 决定模型能否触发。
|
||||
|
||||
### 3.6 Session 与 Task
|
||||
|
||||
**Session**(`core/session.py`)= 消息列表 + meta,**直接 ORM 写 PG `messages` 表**(append-only,`jsonb` 存 LiteLLM 原样 payload)。
|
||||
**Task**(`core/task.py`)= Session 上层,含 name / working_dir / skill / description / status / model / reasoning_effort / 时间戳 / tokens。**直接 ORM 写 PG `tasks` 表**。working_dir FS 目录只存 skill 产物,无 `state.json` / `messages.json`。本地 + SaaS **同一份 schema 和 ORM**,差别只在 `ZCBOT_DB_URL`。
|
||||
|
||||
**Task**(`core/task.py`)= Session 的上层概念,含 mode / description / status (active/completed/abandoned) / model / reasoning_effort / task_dir / created_at / updated_at / tokens_prompt / tokens_completion。**直接 ORM 写 PG `tasks` 表**。
|
||||
**字段三件套语义**:
|
||||
- `name`(NOT NULL) = 任务显示名,UI 列表 / 标题 / docx 导出文件名用;独立于工作目录
|
||||
- `working_dir` = 工作目录(相对 ROOT posix 串),同 working_dir 多 task 共享同物理目录
|
||||
- `skill` = 智能体类型标签(coding / ppt / proposal / ...自由形式,后续可对齐 `skills/` 注册表强校验)
|
||||
|
||||
存储:Session / Task → PG;task_dir FS 目录只存 skill 产物(spec_lock / sections / *.docx / *.pptx 等),不再有 `state.json` / `messages.json`。每轮 `agent.run` 后 `sync_task_tokens` UPDATE 累计 tokens。**本地 + SaaS 同一份 schema 和 ORM 实现,无 adapter 抽象层**,差别只在 `ZCBOT_DB_URL`(本地连 docker compose 起的 PG / 远端 dev PG,SaaS 连生产 PG)。
|
||||
**创建语义** —— working_dir 目录在 task 创建入口立即 `mkdir(parents=True, exist_ok=True)`(`name` 必填代表"显式声明项目";`working_dir` 留空 → fallback 用 name 作目录名)。`Task` 行在 web `/v1/tasks` POST 时即写;CLI 内仍走 `Session.append` 首条 user 消息触发的占位 INSERT(`ensure_local_task_row` idempotent,`name` 透传给 NOT NULL 列)—— REPL 启动后立刻 `/exit` 不留 DB 行(目录留着无害,跨 task 复用)。
|
||||
|
||||
**懒创建** —— `build_agent` 新建分支不立刻 INSERT,Task / Session 在第一条 user 消息触发 `Session.append` 时才 INSERT;task_dir FS 目录在 skill 第一次落产物时 `mkdir(parents=True)`。启动 REPL 后立刻 `/exit` 不留 DB 行 + 不留 FS 目录,跨进程安全。
|
||||
**REPL 内 task 切换** —— `/new` / `/resume [last|<id>]`(无参列最近 10 个)/ `/done /abandon` / `/desc`。切走前 `_cleanup_if_empty` 守门:无 user message → DELETE DB 行;**FS 一律不动**(同 name 跨 task 共享,绝不 rmtree)。
|
||||
|
||||
**REPL 内 task 切换** —— `/new` 开新 task,`/resume [last|<id>]` 切到已有 task(无参数列最近 10 个表格让用户选),`/done /abandon` 改状态,`/desc` 改描述。切走前 `_cleanup_if_empty` 守门:DB 里该 task 没 messages 行 **且** FS task_dir 没产物 → DELETE tasks 行 + rmdir task_dir;任一痕迹存在则保留。
|
||||
**原子性** —— PG INSERT 天然原子;skill 产物走 `core.session.atomic_write_text`(tmp + fsync + replace)。
|
||||
|
||||
**原子性** —— PG INSERT 天然原子,messages / tasks 写入无 0 字节风险。skill 产物(spec_lock.md / sections/*.md 等)仍走 `core.session.atomic_write_text`(tmp + fsync + replace),避免大文件写一半留半文件。
|
||||
|
||||
CLI:`chat --mode coding --desc "..." [--resume last|<id>] [--remote <url>]`;`tasks [--status active|completed|abandoned]` 列任务。
|
||||
CLI:`chat --name "<任务名>" [--working-dir <目录名>] [--skill coding] [--desc "..."] [--resume last|<id>] [--remote <url>]`;`tasks [--status ...]`。
|
||||
|
||||
### 3.7 双层记忆(`core/memory.py`)
|
||||
|
||||
跨 task 共享的事实(用户偏好 / 项目约定 / 模型 quirk 备忘)放 `workspace/memory/`,两层切法:
|
||||
跨 task 共享的事实(用户偏好 / 项目约定 / 模型 quirk)放 `workspace/users/<user_id>/.memory/`(per-user,dotfile 隔离):
|
||||
|
||||
| 层 | 文件 | 加载时机 | 适合内容 |
|
||||
|---|------|---------|---------|
|
||||
| Core | `workspace/memory/core.md` | 每次 build_agent 拼进 system prompt | 跨任务高频用的精炼事实(几百 token 内) |
|
||||
| Extended | `workspace/memory/extended/*.md` | 索引(标题+绝对路径)进 prompt,内容靠 `read` 工具按需拉 | 大量低频专题(API 速查 / 历史事件) |
|
||||
| 层 | 文件 | 加载 | 适合 |
|
||||
|---|---|---|---|
|
||||
| Core | `core.md` | 每次 build_agent 进 system prompt | 跨任务高频精炼事实(几百 token) |
|
||||
| Extended | `extended/*.md` | 索引(标题+绝对路径)进 prompt,内容靠 `read` 工具按需拉 | 大量低频专题 |
|
||||
|
||||
**system prompt 每次 build_agent 重建**,resume 也走 `_build_system_prompt` 并覆盖 `messages[0]` —— memory 演化即时生效。代价:resume 时上下文里的 system 段可能和上一轮不一样,但跨轮强一致性不是个人 agent 的痛点,memory 时效性更重要。
|
||||
**system prompt 每次 build_agent 重建**,resume 也走 `_build_system_prompt` 并覆盖 `messages[0]` —— memory 演化即时生效。
|
||||
|
||||
memory 文件由人填(也允许 agent 用 `write` 写)。系统不自动维护 —— 这是和"auto memory"框架的关键差异:**事实由用户判断,不由 LLM 自动总结**(后者噪音和误判风险高)。
|
||||
memory 由人填(也允许 agent 用 `write` 写),系统不自动维护 —— 关键差异:**事实由用户判断,不由 LLM 自动总结**。
|
||||
|
||||
**形态兼容** —— memory **永远在 FS,不入 DB**:
|
||||
- 本地形态:`workspace/memory/{core.md, extended/}`
|
||||
- SaaS 形态:`<storage_root>/users/<user_id>/memory/{core.md, extended/}`(bind mount 进容器)
|
||||
|
||||
理由:① memory 本质是"用户笔记",FS 读写 + 编辑器手编是产品语义的一部分,DB 化反而要造一层 UI 让用户改 md;② 跨 task 共享靠"同一 user 看同一份目录"语义自动达成,不需要 schema 设计;③ 不参与 §7.4 表结构,task 删/folder 删都不连带 memory。memory 不分 folder,是 per-user 单一命名空间。
|
||||
**memory 永远在 FS,不入 DB**:统一 `<workspace_or_storage_root>/users/<user_id>/.memory/`(本地直接是 `workspace/`,SaaS 是 `<storage_root>/`,bind mount 进容器)。本地 CLI 走 SENTINEL user;web/JWT 走 `sub`。**dotfile `.memory/` 命名**:跟用户起的项目目录(同样落 `<uid>/` 下)区分,避免项目名取 `memory` 时撞名;`validate_task_name` 拒 `.` 起头双向防呆。理由:用户笔记语义,FS 读写 + 编辑器手编是产品的一部分;跨 task 共享靠"同一 user 同一目录"自动达成,无需 schema。
|
||||
|
||||
---
|
||||
|
||||
## 4. 模型路由
|
||||
|
||||
### 默认配置(`config/agent.yaml`)
|
||||
```yaml
|
||||
default_model: deepseek_v4.flash
|
||||
```
|
||||
默认 `default_model: deepseek_v4.flash`。后续分模式路由思路:
|
||||
|
||||
设计上的分模式路由(后续要做)思路:
|
||||
| 模式 | 模型 | 理由 |
|
||||
|-----|-----|------|
|
||||
| 通用 / 编码 / PPT / 提案初稿 | flash | flash SWE-Bench 80.6,够用 |
|
||||
| 复杂 bug / 提案终稿 | pro + reasoning_effort=max | 关键产出值得花 |
|
||||
|---|---|---|
|
||||
| 通用 / 编码 / PPT / 提案初稿 | flash | SWE-Bench 80.6,够用 |
|
||||
| 复杂 bug / 提案终稿 | pro + reasoning_effort=max | 关键产出 |
|
||||
| fallback | claude_4_7.opus | V4 不行时手动切 |
|
||||
|
||||
### 成本量级
|
||||
| 任务 | flash | pro-max | Claude Opus 4.7 |
|
||||
|-----|------|--------|------|
|
||||
| 修一个 bug(~10 轮) | $0.01 | $0.05 | $0.30 |
|
||||
| 5 页汇报 PPT | $0.05 | $0.20 | $1.50 |
|
||||
成本量级(对比):
|
||||
|
||||
| 任务 | flash | pro-max | Opus 4.7 |
|
||||
|---|---|---|---|
|
||||
| 修 bug(~10 轮) | $0.01 | $0.05 | $0.30 |
|
||||
| 5 页 PPT | $0.05 | $0.20 | $1.50 |
|
||||
| 完整申报书 | $0.30 | $1.50 | $10-15 |
|
||||
|
||||
99% 任务 flash 够用,关键终稿升级 Pro。
|
||||
99% 任务 flash 够用,关键终稿升 Pro。
|
||||
|
||||
---
|
||||
|
||||
## 5. 设计哲学
|
||||
|
||||
### 核心原则:Less Scaffolding, More Trust
|
||||
老 agent 框架(早期 LangChain、AutoGPT)失败的核心:给 LLM 太多脚手架,模型升级后这些脚手架成枷锁。
|
||||
|
||||
**正确做法**:把 LLM 当一个**会持续变强的同事**对待,告诉它目标,不告诉它步骤。
|
||||
老 agent 框架失败的核心:给 LLM 太多脚手架,模型升级后这些脚手架成枷锁。**正确做法**:把 LLM 当一个**会持续变强的同事**,告诉它目标,不告诉它步骤。
|
||||
|
||||
### 七条具体原则
|
||||
1. **Prompt 用 WHY+WHAT,不用 HOW** —— 详细教"应该怎么思考"会降智强模型
|
||||
2. **Skill 渐进披露,不写完整流程** —— 对齐 Anthropic 标准
|
||||
3. **工具按原子操作切分,不做高级封装** —— 留出组合空间给模型
|
||||
4. **Model Profile 化,不硬编码** —— 新模型 5 分钟接入
|
||||
5. **Capability Probing** —— yaml 是手填的,跑探测对账实际行为
|
||||
6. **版本化 Prompt** —— `prompts/system/active.md` 软链接(尚未做,等真要切版本时再做)
|
||||
7. **eval 评估** —— 设计阶段曾认为是关键,落地后判断:个人工具 dogfooding 更有效;**已删**
|
||||
1. Prompt 用 WHY+WHAT 不用 HOW —— 教"怎么思考"会降智强模型
|
||||
2. Skill 渐进披露,不写完整流程
|
||||
3. 工具按原子操作切分,不做高级封装 —— 留组合空间
|
||||
4. Model Profile 化,不硬编码
|
||||
5. Capability Probing 对账实际行为
|
||||
6. 版本化 Prompt(等真要切版本时再做)
|
||||
7. ~~eval 评估~~ —— 已删,dogfooding 更有效
|
||||
|
||||
### 借鉴自(简版)
|
||||
### 借鉴
|
||||
| 来源 | 借鉴 |
|
||||
|-----|------|
|
||||
| CoreCoder | 主循环简洁实现 + Edit 唯一匹配约束 |
|
||||
| Anthropic Agent Skills | SKILL.md + 渐进披露标准 |
|
||||
|---|---|
|
||||
| CoreCoder | 主循环简洁实现 + Edit 唯一匹配 |
|
||||
| Anthropic Skills | SKILL.md 渐进披露 |
|
||||
| nanobot | Workspace + 任务隔离 |
|
||||
| smolagents | LiteLLM 做模型层 + CodeAct 范式启发 run_python |
|
||||
| smolagents | LiteLLM + CodeAct 启发 run_python |
|
||||
|
||||
---
|
||||
|
||||
## 6. 风险与取舍
|
||||
|
||||
### 已知风险
|
||||
| 风险 | 缓解 |
|
||||
|-----|------|
|
||||
| run_python subprocess 沙盒不够强(本地形态非真隔离) | 限制工作目录 + 敏感 env 过滤;SaaS 形态走 docker exec(§7.6 #6),本地依赖用户对模型生成代码的最终审阅 |
|
||||
| V4 在某些复杂任务不如 Claude | dogfooding 判断,fallback 手动切 |
|
||||
| Skill description 不够好 → 触发不准 | 用 Pro 优化 description,实战观察 |
|
||||
| Long context 退化 | `probe --long-context` 探测可靠 ceiling,不依赖宣称值 |
|
||||
| 本地 PG 连接不稳定 / 离线 dogfood | `docker compose up -d` 一行起本地 PG 兜底;也可连远端 dev / staging PG;CI 用 ephemeral PG container |
|
||||
|---|---|
|
||||
| run_python sandbox 不够强(本地非真隔离) | 工作目录限制 + 敏感 env 过滤;SaaS 走 docker exec(§7.5);本地依赖用户最终审阅 |
|
||||
| V4 某些复杂任务不如 Claude | dogfooding 判断,fallback 手动切 |
|
||||
| Skill description 不准 → 触发不到 | Pro 优化描述,实战观察 |
|
||||
| Long context 退化 | `probe --long-context` 探测可靠 ceiling |
|
||||
| 本地 PG 离线 | `docker compose up -d` 起本地 PG 兜底;也可连远端 dev / staging PG |
|
||||
|
||||
### 取舍说明
|
||||
**为什么用 Hybrid 范式而不是纯 CodeAgent**:V4 JSON tool call 已稳定;沙盒成本只在需要时付;兼容 thinking 模式。
|
||||
|
||||
**为什么用 Anthropic Skill 标准而不是自创**:行业标准已成,跨 SDK 兼容;直接拿 Anthropic 现成 skills repo。
|
||||
|
||||
**为什么不做 subagent**:状态管理复杂度爆炸;单 agent + skill 已覆盖 95% 场景。
|
||||
|
||||
**为什么不做 Eval Suite**:DESIGN 旧版按团队/产品场景设计;个人单用户场景里,跑两个真实任务的 dogfooding 比造作 case 信号更强,probe 已覆盖健康检查。
|
||||
**Hybrid 范式而非纯 CodeAgent**:V4 JSON tool call 已稳定;sandbox 成本只在需要时付;兼容 thinking。
|
||||
**Anthropic Skill 标准**:行业标准已成,跨 SDK 兼容。
|
||||
**不做 subagent**:状态管理爆炸;单 agent + skill 已覆盖 95% 场景。
|
||||
**不做 Eval Suite**:个人单用户场景,dogfooding 信号比造作 case 强,probe 覆盖健康检查。
|
||||
|
||||
---
|
||||
|
||||
## 7. SaaS 化(草案,status=design,2026-05-12)
|
||||
|
||||
> §1-§6 是 **本地 dogfood 形态**;本节是 **SaaS 形态**,把 core 包成多用户在线服务。
|
||||
> 不引入 platform/core 切分 —— core 就是后端,直接对用户做 auth(原"平台签 JWT、core 验签"多租户方案废弃)。两条形态共享同一份 core,差别只在 CLI 入口 vs HTTP 入口。本节落地前 §1-§6 路线照走,不阻塞 dogfood。
|
||||
> §1-§6 是**本地 dogfood 形态**;本节是**SaaS 形态**,把 core 包成多用户在线服务。
|
||||
> 不引入 platform/core 切分 —— core 就是后端,直接对用户做 auth。两条形态共享同一份 core,差别只在 CLI 入口 vs HTTP 入口。本节落地前 §1-§6 路线照走,不阻塞 dogfood。
|
||||
|
||||
### 7.0 与本地形态的兼容性
|
||||
|
||||
SaaS 化不是"重写"也不是"取代 CLI",而是**给同一份 core 加一个 HTTP 入口**。落地过程中本地 CLI 必须始终可用。
|
||||
SaaS 化不是"重写"也不是"取代 CLI",而是**给同一份 core 加一个 HTTP 入口**。落地过程中本地 CLI 始终可用。
|
||||
|
||||
**两条形态共享**:
|
||||
- 同一份 `core/`(loop / capabilities / skills / memory / storage 接口)
|
||||
- 同一份 `tools/`(底层 executor 从 subprocess 换 docker exec,接口不变)
|
||||
- 同一份 SKILL.md 和 prompts
|
||||
**共享**:同一份 `core/` / `tools/` / SKILL.md / prompts。
|
||||
**差别**:
|
||||
|
||||
**两条形态差别**:
|
||||
|
||||
| 维度 | 本地形态 | SaaS 形态 |
|
||||
| 维度 | 本地 | SaaS |
|
||||
|---|---|---|
|
||||
| 入口 | `cli.py chat ...` 直调 core | HTTP `/v1/...` + SSE |
|
||||
| Storage | **PG**(`ZCBOT_DB_URL` 指 docker compose / 远端 dev PG) | **PG**(`ZCBOT_DB_URL` 指生产 PG) |
|
||||
| task_dir 根 | `workspace/tasks/<task_id>/`(派生,task 私有) | `<storage_root>/users/<user_id>/<task_dir>/`(用户给,可共享) |
|
||||
| Memory | `workspace/memory/`(FS) | `<storage_root>/users/<user_id>/memory/`(仍是 FS) |
|
||||
| Sandbox | subprocess + env 过滤(非真隔离) | per-task docker exec |
|
||||
| Auth | 无(单用户 `user_id='local'`) | OIDC + JWT(user_id) |
|
||||
| 入口 | `cli.py chat` 直调 core | HTTP `/v1/...` + SSE |
|
||||
| Storage | **PG**(`ZCBOT_DB_URL` 指 docker compose / 远端 dev PG) | **PG**(指生产 PG) |
|
||||
| task_dir 派生 | `workspace/users/<sentinel>/<name>/`(`name` 必填,简单名) | `<storage_root>/users/<user_id>/<name>/`(`name` 必填,简单名) |
|
||||
| Memory | `workspace/users/<sentinel>/.memory/`(FS,dotfile) | `<storage_root>/users/<user_id>/.memory/`(仍是 FS,dotfile) |
|
||||
| Sandbox | subprocess + env 过滤 | per-task docker exec |
|
||||
| Auth | 无(`user_id='local'`) | PLATFORM_KEY → JWT(过渡)→ OIDC |
|
||||
|
||||
**CLI 长期双模式**:
|
||||
- **本地直跑**:`cli.py chat`(默认),直接调 core in-process,直连 PG。适合 dogfood / 调 core 内部状态
|
||||
- **API client**:`cli.py chat --remote https://...`,走 HTTP /v1,跟前端用户路径一致
|
||||
**CLI 长期双模式**:本地直跑(默认,in-process,直连 PG,适合调内部状态)/ `--remote https://...`(HTTP 走 `/v1`,等价真实用户路径)。两模式共用 `cli.py`,差别只在 transport 层。
|
||||
|
||||
两模式共用 `cli.py` 入口,差别只在 transport 层(in-process call vs HTTP)。dogfood ≡ 真实用户路径只在 `--remote` 模式下成立;**本地直跑模式永久保留**(调试 core 内部状态比 HTTP roundtrip 顺手)。
|
||||
`workspace/` 仅存 skill 产物,state / messages 全在 PG。本地 vs SaaS 共用 `users/<user_id>/` 子树布局,差别只在外层根目录(`workspace/` vs `<storage_root>/`),不在 storage 形态。
|
||||
|
||||
**本地 PG 连接** —— `ZCBOT_DB_URL` 指向 docker compose 起的本地 PG(`docker compose up -d` 一行起,repo 自带 `docker-compose.yml`)或远端 dev / staging PG。**离线场景靠本地 docker compose 兜底**,不靠"零依赖"幻觉。
|
||||
### 7.1 心智模型:Task 一等公民 + Dir 文件副视图
|
||||
|
||||
`workspace/` 目录:仅存 skill 产物(spec_lock / sections / *.docx / *.pptx),state / messages 全在 PG。本地 vs SaaS 差别只在 task_dir 根路径,不在 storage 形态。
|
||||
两个并列入口,正交不嵌套:
|
||||
|
||||
### 7.1 心智模型:Folder-centric,task-as-DB-record
|
||||
| 视图 | 入口语义 | 适用场景 | API |
|
||||
|---|---|---|---|
|
||||
| **Task list**(主) | "我的对话历史" | 任务驱动:"继续昨天那个 bug fix" | `GET /v1/tasks?status=&task_dir=` |
|
||||
| **Dir tree**(辅) | "我的文件资产" | 项目驱动:"看汇报项目里所有素材 + 关联对话" | `GET /v1/folders` |
|
||||
|
||||
参考 Claude Code(cwd 是 anchor,状态存别处)+ OpenAI Assistants(stateful agent service)。
|
||||
类比:macOS Finder + 最近使用 / Apple Notes 文件夹视图 + 全部备忘录。两个视图查同一份数据的不同切面,**dir 不是 task 的父容器**。
|
||||
|
||||
- **Folder** = 用户的"硬盘",路径 `users/<user_id>/<user-defined>/...`。能浏览、新建、改名、上传、下载,**和本地文件管理器体感一致**。folder 没 ID,**path 就是标识**;改名走 prefix cascade。
|
||||
- **Task** = DB 一行,带 `task_dir` 指向 folder(相对 user root)。同 folder 允许多 task,但 task 之间**不允许嵌套**(no-subtask)。
|
||||
- **Messages** = DB 表,append-only,`jsonb` 存 LiteLLM 原样 payload。
|
||||
- **Skill 运行产物** 全落 cwd,不引入 artifacts 表;终稿后 SKILL.md 指示 agent 清中间件。
|
||||
- **Skill 定义** 是项目代码,跟部署走,所有用户共享,不入用户 folder。
|
||||
- **Task** = DB 一行,一等公民,自带 `task_dir text` 字段:
|
||||
- **新建必给 `name`**(简单名),`task_dir = workspace/users/<user_id>/<name>/`。同 name 多 task 共享 → "同一项目多对话"语义;不再支持空 task_dir / 自动 UUID 派生(原 ChatGPT thread 模式取消,纯对话也得起个项目名)
|
||||
- **指定 → 项目化 task**,同 task_dir 多 task 自动共享 `source/` / `sections/` / 终稿(无需建"项目"实体)
|
||||
- **Dir** = FS 路径,**无 DB 实体,path 即标识**;无父子结构,改名走 prefix cascade(§7.4)
|
||||
- **No-subtask**:同 task_dir 允许(同项目多对话),前缀嵌套拒
|
||||
- **Messages** = DB 表,append-only,`jsonb` 存 LiteLLM 原样 payload
|
||||
- **Skill 产物**全落 task_dir,不引入 artifacts 表;SKILL.md 指示 agent 清中间件
|
||||
- **Skill 定义**是项目代码,跟部署走,所有用户共享
|
||||
|
||||
**task_dir 在两形态的对应**(§7.0 总览的展开):
|
||||
- 本地形态:`task_dir = workspace/tasks/<task_id>/`(派生,task 私有,无并发写冲突)
|
||||
- SaaS 形态:`task_dir = <storage_root>/users/<user_id>/<user-given-path>/`(用户给,可被同 user 多 task 共享)
|
||||
**空 dir**(用户上传素材但还没开 task)在 dir tree 视图正常展示 —— 上传本身是有效产品行为;UI 上跟"有 task 的 dir"做轻量区分(如 task 数 badge)。
|
||||
|
||||
state / messages **两形态都在 PG**,FS 只承担 skill 产物(sections / *.docx / 中间件)。多 task 共享同 folder 时由 §7.8 文件级悲观锁兜底(并发写同名文件冲突早失败,推到模型自纠)。
|
||||
state / messages 两形态都在 PG,FS 只承担 skill 产物。多 task 共享同 task_dir 时由 §7.8 文件级悲观锁兜底。
|
||||
|
||||
### 7.2 资源模型与接口(/v1)
|
||||
### 7.2 资源模型(/v1)
|
||||
|
||||
Task 一等公民,files 是其副视图(经 `task_dir` 暴露,无独立 folder 实体)。所有路由统一 `/v1` 前缀,**返 JSON**;前端 / UI 由 platform 端实现,本仓库不维护(§7.9 取舍)。本地开发用 FastAPI 自带 `/docs` Swagger UI 自查;`GET /` 302 跳 `/docs`。
|
||||
|
||||
```
|
||||
POST /v1/folders 创建
|
||||
GET /v1/folders 列树
|
||||
GET /v1/folders/{path} 详情(task 列表 + 文件列表)
|
||||
PATCH /v1/folders/{path} 改名/移动(prefix cascade)
|
||||
DELETE /v1/folders/{path} hard cascade(连带 task+messages,前端二确认)
|
||||
Tasks
|
||||
POST /v1/tasks 创建 {name(必填), working_dir?, description?, skill?};
|
||||
留空 working_dir → 用 name 作目录名;
|
||||
working_dir 派生 workspace/users/<user_id>/<working_dir>/;
|
||||
name/working_dir 不合法 → 400
|
||||
GET /v1/tasks?page=&page_size=&status=&skill=&working_dir=&q=&ordering=
|
||||
列表,返 `{page, page_size, count, results}`
|
||||
分页 1-based;page_size 1–100 clamp;status active/completed/abandoned;
|
||||
skill 精确;working_dir 末段名(后端拼前缀比对);q 在 name+description ILIKE;
|
||||
ordering DRF 风格逗号分隔,`-field` 倒序;allowlist
|
||||
created_at/updated_at/name/status;**默认 `-created_at`**
|
||||
GET /v1/tasks/{id} 单 task meta + 完整 messages
|
||||
PATCH /v1/tasks/{id} {status?,description?,name?,skill?};status 从 web 不让切回 active(走 CLI)
|
||||
DELETE /v1/tasks/{id} 硬删:DB 行 + messages(CASCADE);**FS working_dir 保留**
|
||||
(同 working_dir 多 task 共享,文件由用户经 /files/delete 单独清)
|
||||
GET /v1/folders 列当前 user 的 working_dir(FS 是 source of truth + 关联 task 计数 + 最后使用时间)
|
||||
GET /v1/tasks/{id}/messages 历史(后续 ?search= 走 jsonb GIN / tsvector)
|
||||
POST /v1/tasks/{id}/messages {content} 发消息 + 起 run,返 {run_id}
|
||||
GET /v1/tasks/{id}/runs/{rid}/events SSE 流(见下)
|
||||
POST /v1/tasks/{id}/runs/{rid}/cancel (待)
|
||||
|
||||
POST /v1/folders/{path}/files 上传(multipart)
|
||||
GET /v1/folders/{path}/files[/{name}] 列 / 下载
|
||||
DELETE /v1/folders/{path}/files/{name}
|
||||
Files(per-task,task_dir 副视图)
|
||||
GET /v1/tasks/{id}/files?path= 列子目录 {entries, crumbs, exists, root}
|
||||
POST /v1/tasks/{id}/files/upload multipart;path 通过 query 或 form;严格拒含 / \\ .. 的 filename
|
||||
GET /v1/tasks/{id}/files/download?path= 下载单文件;`..` / 绝对 / symlink 越界 400
|
||||
POST /v1/tasks/{id}/files/delete {path} 文件或空目录;非空目录 400
|
||||
|
||||
POST /v1/tasks 创建({task_dir, mode, desc, model})
|
||||
GET /v1/tasks 列(?task_dir= ?status= 过滤)
|
||||
GET /v1/tasks/{id} 详情
|
||||
PATCH /v1/tasks/{id} 改 mode/desc/status
|
||||
DELETE /v1/tasks/{id} 删 task(messages 一起删,不动 cwd 文件)
|
||||
Export
|
||||
GET /v1/tasks/{id}/export docx 临时文件下载,BackgroundTask 删 tmp
|
||||
|
||||
POST /v1/tasks/{id}/messages 发消息,返回 {run_id}
|
||||
GET /v1/tasks/{id}/messages 历史(?search= 走 jsonb GIN / tsvector)
|
||||
GET /v1/tasks/{id}/runs/{run_id}/events SSE 事件流
|
||||
POST /v1/tasks/{id}/runs/{run_id}/cancel
|
||||
|
||||
GET /v1/skills | /v1/models | /v1/usage
|
||||
POST /v1/probe (admin) 跑 capability probe
|
||||
Misc
|
||||
GET /healthz {"status":"ok"}
|
||||
GET / 302 → /docs (Swagger UI 自查,本地形态便利)
|
||||
```
|
||||
|
||||
**SSE 事件**:`tool_call` / `tool_result` / `text` (delta) / `usage` / `done`,带 `run_id`。
|
||||
|
||||
**版本化**:`/v1` minor 半年内向后兼容,major 6 个月 deprecation。
|
||||
|
||||
### 7.3 认证模型
|
||||
|
||||
OIDC / Clerk / 自建邮箱登录,JWT 只带 `user_id` claim:
|
||||
**SSE 事件**(`Content-Type: text/event-stream`,响应头带 `X-Accel-Buffering: no` 给 nginx 反代友好;每事件 `event: <type>` + `data: <JSON>`):
|
||||
|
||||
```
|
||||
Authorization: Bearer <user_jwt>
|
||||
X-Request-Id: <uuid>
|
||||
run_start {}
|
||||
llm_start {}
|
||||
text {"content":"<delta 或全量,取决于 model streaming 配置>"}
|
||||
tool_call {"name":"...","args":{...},"args_preview":"..."}
|
||||
tool_result {"name":"...","preview":"...","truncated":bool} # 完整 result 走 DB,SSE 只送预览给 UI
|
||||
llm_end {"prompt_tokens":N,"completion_tokens":N}
|
||||
error {"msg":"<type>: <detail>"}
|
||||
done {}
|
||||
```
|
||||
|
||||
所有 storage/executor 调用 scoped by `user_id`。**无 tenant 层** —— 个人 SaaS 用不上,日后做企业版加 `org_id` claim 等价隔离。
|
||||
订阅 fan-out:同 run 多订阅者(刷新 / 多 tab / 多设备)每订阅 1 独立 queue。订阅迟到(run 已 done)立刻收 done 不挂。事件不持久化 —— messages 走 PG,未来要"刷新继续看流式"再加 event log。
|
||||
|
||||
**版本化**:`/v1` minor 半年向后兼容,major 6 个月 deprecation。`/v1internal` 实验位(未启)。
|
||||
|
||||
**CORS**:本地 dev `allow_origins=["*"]`;部署 platform 时收紧到 platform 域名 allowlist。
|
||||
|
||||
**Auth**:PLATFORM_KEY → JWT 兑换(过渡形态,见 §7.3);`Authorization: Bearer <jwt>` 走所有 `/v1/tasks*`;`/healthz`、`/docs`、`/openapi.json`、`/`、`/v1/auth/login`、`/static/*` 豁免。
|
||||
|
||||
### 7.3 认证
|
||||
|
||||
**当前形态(D' 过渡,2026-05-15 落地)**:platform 服务端(或 dev 浏览器)持有 `PLATFORM_KEY` 共享密钥,调 `POST /v1/auth/login {user_id, platform_key}` → 后端校验 key 匹配 → 签 HS256 JWT(`sub=user_id`,默 7d TTL,`JWT_SECRET` env 签)→ 返 `{token, expires_at, user_id, ttl_seconds}`。后续 `Authorization: Bearer <jwt>` 走所有 /v1/tasks*,FastAPI `Depends(require_user)` 验签 → 提取 user_id → SELECT/UPDATE 全带 `Task.user_id == user_id` 条件做隔离。`PLATFORM_KEY` / `JWT_SECRET` 任一缺失 → app 启动 fail-fast。**信任模型**:platform 是单点可信中间层(持 KEY = 可为任意 user_id 签 token,等同 user 身份由 platform 注入);风险与"platform 服务端泄漏 = 用户身份泄漏"同级,可接受。
|
||||
|
||||
**未来形态(真 OIDC,D 阶段后期)**:OIDC / Clerk / 自建邮箱登录,Provider 签 ID token,zcbot `/v1/auth/login` 内部从"校验 PLATFORM_KEY"换成"校验 ID token 签名 + 提取 sub" —— **路由层 Depends 不动**,Bearer JWT 契约不变。所有 storage/executor scoped by `user_id`。**无 tenant 层** —— 个人 SaaS 用不上,做企业版再加 `org_id` 等价隔离。
|
||||
|
||||
### 7.4 存储:Postgres + 本地文件系统
|
||||
|
||||
```sql
|
||||
users(user_id uuid pk, email null, password_hash | oidc_subject null, plan null, created_at)
|
||||
-- 本地形态固定 INSERT 一行 sentinel: user_id = '00000000-0000-0000-0000-000000000000',
|
||||
-- email / auth / plan 全 NULL;CLI 启动时若不存在则建,tasks 全部 FK 到它
|
||||
-- 本地形态固定 INSERT sentinel: user_id = '00000000-...',email/auth/plan 全 NULL
|
||||
|
||||
tasks(
|
||||
task_id uuid pk,
|
||||
user_id uuid fk,
|
||||
task_dir text not null, -- 相对 user root,如 "project_a/sub"
|
||||
mode text, -- coding / proposal / ppt / chat
|
||||
description text,
|
||||
status text, -- pending / running / paused / done
|
||||
model_profile text,
|
||||
tokens_prompt int default 0,
|
||||
tokens_completion int default 0,
|
||||
cost_usd numeric default 0,
|
||||
created_at timestamptz,
|
||||
updated_at timestamptz
|
||||
);
|
||||
create index on tasks (user_id, task_dir);
|
||||
tasks(task_id uuid pk, user_id fk, name text not null, working_dir text not null, skill, description,
|
||||
status, model_profile, tokens_prompt, tokens_completion, cost_usd,
|
||||
created_at, updated_at);
|
||||
create index on tasks (user_id, working_dir);
|
||||
-- working_dir 存储约定:本地 ROOT 内 → 相对 ROOT 的 posix 串
|
||||
-- (`workspace/users/<user_id>/<name>`,name 是简单名,无 /\..);
|
||||
-- 新建强制 `name` 必填,空串只可能在 legacy 数据(开发期已 wipe)。
|
||||
-- SaaS 阶段同理(基础是 <storage_root>/users/<uid>/)。
|
||||
-- 读写边界统一过 core/paths.py::{to_db_path,from_db_path}。
|
||||
-- 入口校验 main.py::validate_task_name(): 拒空 / 含 /\NUL / `.` 起头 / >255。
|
||||
|
||||
messages(
|
||||
message_id uuid pk,
|
||||
task_id uuid fk,
|
||||
idx int not null,
|
||||
payload jsonb not null, -- LiteLLM dict 原样
|
||||
tokens_in int, tokens_out int,
|
||||
created_at timestamptz,
|
||||
unique (task_id, idx)
|
||||
);
|
||||
messages(message_id uuid pk, task_id fk, idx int not null,
|
||||
payload jsonb not null, tokens_in, tokens_out, created_at,
|
||||
unique (task_id, idx));
|
||||
create index on messages using gin (payload jsonb_path_ops);
|
||||
-- 对话全文搜按需加 tsvector + GIN(中文起步 simple + pg_trgm)
|
||||
-- 全文搜按需加 tsvector + GIN(中文 simple + pg_trgm 起步)
|
||||
|
||||
runs(run_id uuid pk, task_id fk, status, started_at, finished_at, error, tokens_p, tokens_c)
|
||||
runs(run_id pk, task_id fk, status, started_at, finished_at, error, tokens_p, tokens_c)
|
||||
usage_events(id, user_id, task_id uuid, run_id uuid, kind, value, ts)
|
||||
-- append-only。task_id/run_id 不 FK,task 硬删后审计记录仍存活
|
||||
-- append-only。task_id/run_id 不 FK,task 硬删后审计仍存活
|
||||
```
|
||||
|
||||
**No-subtask 校验**(`create_task` 入口):
|
||||
**No-subtask 校验**(`create_task`):查同 user 下是否存在 `new LIKE existing/%` 或 `existing LIKE new/%`,中一则拒;同 task_dir 允许。**两侧先用 `from_db_path` 归一到 absolute posix 再比前缀**(混合存储形态 [相对+绝对] 不会漏判),数量小直接 Python 端比对,不在 SQL 里拼分隔符。
|
||||
|
||||
```sql
|
||||
SELECT 1 FROM tasks
|
||||
WHERE user_id = ?
|
||||
AND ( ? LIKE task_dir || '/%' -- new 在已有之下 → 拒
|
||||
OR task_dir LIKE ? || '/%' ); -- 已有在 new 之下 → 拒
|
||||
-- 同 task_dir 允许(同 folder 多 task)
|
||||
```
|
||||
**Folder rename**(`old → new`,FS rename 成功后):`UPDATE tasks SET task_dir = new || substring(task_dir from len(old)+1) WHERE user_id=? AND (task_dir = old OR task_dir LIKE old||'/%')`。**用 `old/%` 而非 `old%`**,避免 `project_a` 误中 `project_a_other`。running task 引用时禁 rename / delete。
|
||||
|
||||
**Folder rename**(改名 `old → new`,FS rename 成功后跑):
|
||||
|
||||
```sql
|
||||
UPDATE tasks
|
||||
SET task_dir = ? || substring(task_dir from char_length(?) + 1) -- new, old
|
||||
WHERE user_id = ? AND (task_dir = ? OR task_dir LIKE ? || '/%'); -- old, old
|
||||
```
|
||||
|
||||
LIKE 用 `old/%` 而非 `old%`,避免 `project_a` 误中 `project_a_other`。**running task 引用该 folder 时禁 rename / delete**(后端校验 + UI 禁按钮)。
|
||||
|
||||
**Folder delete**:hard cascade,前端 modal 列影响面("将删 N 个对话、M 条消息、K 个文件")+ 输入 folder 名二确认。
|
||||
|
||||
```sql
|
||||
-- 先 DB 后 FS;DB 失败 FS 不动一致;DB 成功 FS 失败由后台 GC 兜底清孤儿目录
|
||||
DELETE FROM messages
|
||||
WHERE task_id IN (SELECT task_id FROM tasks
|
||||
WHERE user_id=? AND (task_dir=? OR task_dir LIKE ?||'/%'));
|
||||
DELETE FROM tasks
|
||||
WHERE user_id=? AND (task_dir=? OR task_dir LIKE ?||'/%');
|
||||
-- 然后 FS 递归删 folder
|
||||
```
|
||||
|
||||
`usage_events` 不参与 cascade(审计 append-only)。
|
||||
|
||||
**文件系统**:
|
||||
**Folder delete**:hard cascade,前端 modal 列影响面 + 输入 folder 名二确认。先 DELETE messages → DELETE tasks → FS 递归删;DB 成功 FS 失败由后台 GC 兜底清孤儿目录。`usage_events` 不参与 cascade。
|
||||
|
||||
**文件系统**(本地 `<storage_root>` = `workspace/`,SaaS 替换为部署根,布局不变):
|
||||
```
|
||||
<storage_root>/users/<user_id>/
|
||||
memory/{core.md, extended/} # 跨 task 的 per-user 记忆,不入 DB
|
||||
project_a/source/ sections/ proposal.docx
|
||||
project_b/...
|
||||
.memory/{core.md, extended/} # per-user 记忆,dotfile 隔离,不入 DB
|
||||
<name>/ # 项目目录,name 用户起(必填),task_dir 直接落这
|
||||
<name>/... # 同 name 多 task 共享同目录(§7.1)
|
||||
```
|
||||
本地优先 S3(部署简化 / 低延迟),storage 抽象层留好后续可换。
|
||||
|
||||
本地优先 S3(简化部署 / 低延迟),storage 抽象层留好后续可换 backend。
|
||||
|
||||
**Storage 实现:单一 PG ORM**(本地 + SaaS 共用):
|
||||
- 一份 schema、一份 ORM(SQLAlchemy)、一份查询代码,无 adapter 抽象层,无 SQL 方言适配,无契约测试
|
||||
- 本地 dev 连接:`ZCBOT_DB_URL=postgresql://...` 环境变量;repo 自带 `docker-compose.yml` 起本地 PG(零配置)或连远端 dev / staging PG
|
||||
- Schema 演化:alembic 管理 migration,`db/migrations/*.py` 与代码一同版本化;CLI 启动校验当前 schema 版本,落后报错让用户跑 `cli db upgrade`(本地)或部署管线自动 `alembic upgrade head`(SaaS)
|
||||
- 旧 workspace JSON 一次性迁移:`cli migrate-from-fs --workspace ./workspace` 把 `state.json` / `messages.json` 导入 PG,完成后 workspace 进只读 archive 模式
|
||||
- 本地单用户 sentinel:DB init 时若 users 表无 sentinel 行则 INSERT;本地 CLI 所有 tasks 全 FK 到这一行,无 auth 流程,但 schema 与 SaaS 完全一致
|
||||
- memory 不参与:per-user FS,两形态都不入 DB
|
||||
**Storage 实现:单一 PG ORM**(本地 + SaaS 共用):一份 schema、一份 SQLAlchemy、一份查询,无 adapter,无 SQL 方言适配,无契约测试。alembic 管 migration;CLI 启动校验 schema 版本,落后报错让用户跑 `cli db upgrade`(本地)或部署管线自动 `alembic upgrade head`(SaaS)。`cli migrate-from-fs --workspace ./workspace` 一次性导旧 JSON。
|
||||
|
||||
### 7.5 沙盒:Per-task 容器 + Per-run exec
|
||||
|
||||
| 选择 | 理由 |
|
||||
|---|---|
|
||||
| 每 task 长驻容器 | 起容器 ~300ms 太慢;多轮 tool call 共享划算 |
|
||||
| 每 run 一次 `docker exec` | exec 级 timeout/资源限制 |
|
||||
| 每 run 一次 `docker exec` | exec 级 timeout / 资源限制 |
|
||||
| 空闲 N 分钟回收 | 不浪费,resume 时拉起 |
|
||||
| **bind mount = user root** | `<storage_root>/users/<user_id>/` → 容器 `/workspace`;同用户多 task 不互隔(协作方便),跨用户由独立容器实例隔离 |
|
||||
| bind mount = user root | `<storage_root>/users/<user_id>/` → `/workspace`;同 user 多 task 不互隔(协作方便),跨 user 由独立实例隔离 |
|
||||
|
||||
**资源限制**:cgroup CPU/mem、磁盘配额、egress allowlist(只放 LLM + PyPI 镜像)、root fs read-only、no-new-privileges、drop ALL caps。
|
||||
|
||||
**选型**:起步 Docker(运维门槛低);流量起来后视情况换 gVisor / Firecracker / e2b。Executor Protocol 抽象后切换成本低。
|
||||
**选型**:起步 Docker;流量起来后视情况换 gVisor / Firecracker / e2b。Executor Protocol 抽象后切换成本低。
|
||||
|
||||
### 7.6 Core 代码改造(按依赖顺序)
|
||||
|
||||
| # | 项 | 影响文件 | 估时 |
|
||||
|---|---|---|---|
|
||||
| 1 | ~~事件流化 `loop.py`~~ | 已完成(commit `375bb29`) | — |
|
||||
| 2 | **Storage 落 PG**:`Session` / `TaskState` 改 SQLAlchemy ORM 写 PG `messages` / `tasks` 表(单一实现,无 adapter 抽象);alembic 管 schema migration;`cli migrate-from-fs` 一次性把现有 workspace JSON 导入;repo 加 `docker-compose.yml` 起本地 PG 用于 dev | `core/session.py` `core/task.py` 新增 `core/storage/` `db/migrations/` `cli.py::migrate_from_fs` `cli.py::db_upgrade` `docker-compose.yml` `requirements.txt` | 3 天 |
|
||||
| 3 | **task_dir 双形态共存**:`TaskState.task_dir` 可显式指定(本地默认 `workspace/tasks/<task_id>/`,SaaS = 用户给路径);`tools/fs.py::_resolve` 接受 task_dir 注入;system prompt 注入逻辑两形态共用 | `core/task.py` `tools/fs.py` `main.py` `prompts/system/general_v1.md` | 1 天 |
|
||||
| 4 | **Folder API**:list / create / rename(cascade + 锁 running task) / delete(hard cascade,前端二确认强校验) / upload / download | 新增 `core/folders/` | 2 天 |
|
||||
| 5 | **No-subtask 校验**:`create_task` 入口跑 §7.4 的 SQL | `core/task.py` | 0.5 天 |
|
||||
| 6 | **Executor + 沙箱**:`run_python`/`shell` → `Executor.run(...)`,`docker exec` 到 per-user/per-task 容器;`api_key_env` → `KeyProvider`(运行时注入);**本地形态保留 subprocess executor**,SaaS 形态走 docker executor | `tools/run_python.py` `tools/shell.py` `core/capabilities.py` `core/llm.py` 新增 `core/executor/` | 2-3 天 |
|
||||
| 7 | **HTTP /v1**:FastAPI + SSE + OIDC | 新增 `core/api/` `core/auth/` | 4 天 |
|
||||
| 8 | **CLI 双模式**:加 transport 层抽象 —— 无 `--remote` 时走 in-process 直调 core(本地形态);`--remote <url>` 走 HTTP API client(dogfood ≡ 真实用户路径);**不删除本地直跑** | `cli.py` 加 `core/transport/` | 1.5 天 |
|
||||
| # | 项 | 估时 |
|
||||
|---|---|---|
|
||||
| 1 | ~~事件流化 `loop.py`~~(commit `375bb29`) | done |
|
||||
| 2 | **Storage 落 PG**:`Session` / `TaskState` 改 SQLAlchemy 写 PG;alembic;`cli migrate-from-fs`;`docker-compose.yml` 起本地 PG | 3 天 |
|
||||
| 3 | **task_dir 字段语义**:新建必给 `name`(简单名),task_dir 派生为 `<storage_root>/users/<user_id>/<name>/`(本地 `<storage_root>` = `workspace/`,sentinel user);同 name 多 task 共享同目录;`tools/fs.py::_resolve` 接 task_dir 注入;system prompt 注入 | 1 天 |
|
||||
| 4 | **Folder API**:list / create / rename(cascade + 锁 running) / delete(hard cascade) / upload / download | 2 天 |
|
||||
| 5 | **No-subtask 校验**:`create_task` 入口跑 §7.4 SQL | 0.5 天 |
|
||||
| 6 | **Executor + sandbox**:`run_python`/`shell` → `Executor.run(...)`;本地保留 subprocess executor,SaaS 走 docker;`api_key_env` → `KeyProvider` 运行时注入 | 2-3 天 |
|
||||
| 7 | **HTTP /v1**:FastAPI + SSE + OIDC | 4 天 |
|
||||
| 8 | **CLI 双模式**:transport 层抽象,默认 in-process;`--remote` 走 HTTP;**本地直跑不删** | 1.5 天 |
|
||||
| 9 | ~~Web UI 简洁版(Jinja2+HTMX)~~ → 改为 **API surface 完工**:Phase G 落地的模板 / HTMX / 服务端 markdown 渲染删除,所有路由切纯 JSON;UI 由 platform 端实现(§7.9 取舍) | 已落 |
|
||||
|
||||
代码量增量:**+1000~1500 行**(单一 PG 实现比双 adapter 方案省 500-800 行;无契约测试集 / 无方言适配层)。
|
||||
代码量增量:**+1000~1500 行**(单一 PG 比双 adapter 省 500-800 行;UI 不计入,本仓库只维护 API)。
|
||||
|
||||
### 7.7 分阶段落地
|
||||
|
||||
| 阶段 | 范围 | 工作量 | 验收 |
|
||||
|---|---|---|---|
|
||||
| A | §7.6 #1 | done | ✅ |
|
||||
| B | §7.6 #2 #3 #4 #5(Storage 落 PG + task_dir 双形态 + Folder API + no-subtask) | ~1 周 | 本地 CLI 走 PG,messages 进 DB 可全文搜;多 task + folder rename 单测过;`migrate-from-fs` 跑通 |
|
||||
| C | §7.6 #6(Executor + sandbox) | 3 天 | 两本地账号互不可见对方 folder,本地 subprocess executor 仍可用 |
|
||||
| D | §7.6 #7(HTTP /v1 + auth) | 4 天 | curl/Postman 跑通主流程 |
|
||||
| E | §7.6 #8(CLI transport 双模式) | 1.5 天 | CLI 默认本地直跑保留,`--remote` 走 HTTP 也跑通 |
|
||||
| F | 上线打磨(限流 / 监控 / 告警 / HA) | 持续 | SLO 99.5% |
|
||||
| A | #1 事件流化 | done ✅ | sink 协议铺 SSE 路 |
|
||||
| B | #2 #3 #4 #5(Storage 落 PG + task_dir 双形态 + no-subtask)| done ✅ | 本地走 PG,messages 进 DB,任务/消息/状态全在 PG;task_dir 改相对存储(§7.4 注释)|
|
||||
| D | #7 HTTP /v1 surface(无 auth)| done ✅ | `/v1/tasks/*` + SSE JSON + files 4 路由 + export + Swagger;本地形态 sentinel user 跑通 |
|
||||
| D' 过渡 | PLATFORM_KEY → JWT 兑换 + user_id 数据隔离 + dev SPA | done ✅ | `POST /v1/auth/login` 拿 token,`Authorization: Bearer` 走全部 /v1/tasks*;`web/static/dev.html` 单文件 3 栏 SPA 给开发自验。详 §7.3 |
|
||||
| D' 真 OIDC | 替换 /v1/auth/login 内部为 ID token 校验 + CORS allowlist 收紧 | 1 天 | 真发布给真实用户前补;路由层 Depends 不动,只换 login 内部 |
|
||||
| C | #6 Executor + sandbox | 3 天 | 两本地账号互不可见对方 folder,本地 subprocess executor 仍可用 |
|
||||
| E | #8 CLI transport 双模式 | 1.5 天 | 默认本地直跑保留,`--remote` 走 HTTP 跑通 |
|
||||
| ~~G~~ | ~~Web UI 简洁版~~ —— **删除**,前端由 platform 端实现 | — | 本仓库不维护 UI |
|
||||
| F | 上线打磨(限流 / 监控 / 告警 / HA)| 持续 | SLO 99.5% |
|
||||
|
||||
**B 阶段一次性切换** —— 切到 PG 后本地与 SaaS 走相同代码路径,无回退、无双轨。**dogfood 即生效**(messages 进 DB → 全文搜、jsonb 查询立刻可用)。前置:repo 提供 `docker-compose.yml`,作者本机 `docker compose up -d postgres` 一行准备好 dev DB。
|
||||
**B 阶段一次性切换** —— 切到 PG 后本地与 SaaS 走相同代码路径,无回退、无双轨。**dogfood 即生效**(messages 进 DB → 全文搜、jsonb 查询立刻可用)。
|
||||
|
||||
**D 落在 G 前面** —— 原排期 D 在 G 后(以为 dogfood 用 UI 跑),实际转向"platform 端联调"后,API surface 反而成阻塞;G 的 Jinja2+HTMX 投入(G1-G6 ~3 天)沉淀 = 删除前的 dogfood 价值,留下的 sink 协议 / broker / no-subtask / files 路径安全归一 / task_dir 相对存储仍被 D 复用。
|
||||
|
||||
### 7.8 已知风险
|
||||
|
||||
| 风险 | 缓解 |
|
||||
|---|---|
|
||||
| 过早抽象违背 §5 哲学 | B 阶段单一 PG 实现无 adapter 抽象层;C-E 各阶段独立 dogfood 价值,"先有场景再加" |
|
||||
| 本地 PG 连接 / 离线 dogfood | `docker compose up -d` 本地起 PG 兜底;也支持连远端 dev / staging PG;CI 用 ephemeral PG container |
|
||||
| CLI 双模式分叉、本地直跑被忽略 | transport 层抽象统一接口;CI 跑 in-process 和 HTTP 两路径同一组用例 |
|
||||
| 过早抽象违背 §5 | B 阶段单一 PG 无 adapter;C-E 各阶段独立 dogfood 价值 |
|
||||
| CLI 双模式分叉、本地直跑被忽略 | transport 层抽象统一接口;CI 跑两路径同一组用例 |
|
||||
| `/v1` 冻死后演化慢 | minor 半年兼容,major 6 个月 deprecation;`/v1internal` 实验 |
|
||||
| Rename 误命中前缀 / 漏改子 task | cascade SQL + 单测覆盖 `project_a` 不中 `project_a_other` |
|
||||
| 运行中 task 被 rename / delete | 后端校验 + UI 禁按钮 |
|
||||
| 误删 folder 丢对话 | 前端二确认 + 输入 folder 名;真要再加 trash bin(延迟 cascade) |
|
||||
| DB-then-FS 中断留孤儿目录 | 后台 GC 周期扫 "FS 有但 DB 无引用" 的目录 |
|
||||
| 同 folder 多 task 并发写同名文件 | 文件级悲观锁,冲突早失败 |
|
||||
| Sandbox 出站越权 | egress allowlist 起步只放 LLM + PyPI 镜像 |
|
||||
| 资源滥用(LLM / 存储) | BYO key 默认;月度 token & 存储配额;cold task LRU 清 |
|
||||
| Rename 误中前缀 / 漏改子 task | cascade SQL 用 `old/%` + 单测覆盖 |
|
||||
| Running task 被 rename / delete | 后端校验 + UI 禁按钮 |
|
||||
| 误删 folder | 二确认 + 输入 folder 名;真要再加 trash bin |
|
||||
| DB-then-FS 中断留孤儿目录 | 后台 GC 周期扫"FS 有但 DB 无引用" |
|
||||
| 同 folder 多 task 并发写同名 | 文件级悲观锁,冲突早失败 |
|
||||
| Sandbox 出站越权 | egress allowlist 起步只放 LLM + PyPI |
|
||||
| 资源滥用 | BYO key 默认;月度配额;cold task LRU 清 |
|
||||
|
||||
### 7.9 取舍说明
|
||||
|
||||
**path-as-identity 而非 folder_id**:folder 真实存在于 FS,folder_id 等于造两份 source of truth(易不一致)。rename 是 UI 主动动作,cascade 单事务搞定。
|
||||
**path-as-identity 而非 folder_id**:folder 真实存在于 FS,folder_id 等于造两份 source of truth。rename 是 UI 主动动作,cascade 单事务搞定。
|
||||
|
||||
**user auth 而非 tenant 层**:个人 SaaS 用不上。日后做企业版加 `org_id` claim,数据隔离规则等价。提前抽象 MVP 多 NULL 一层。
|
||||
**user auth 而非 tenant 层**:个人 SaaS 用不上。企业版加 `org_id` claim 等价。
|
||||
|
||||
**skill 中间件全落 cwd 不引入 artifacts 表**:中间件是用户花 token 生成的资产,可下载可替换;artifacts 表 + 分类是为不确定的 UX 收益预付架构成本。真嫌乱 UI 加折叠视图。
|
||||
**skill 产物全落 cwd 不引入 artifacts 表**:中间件是用户花 token 生成的资产,可下载可替换;artifacts 表是为不确定 UX 收益预付架构成本。真嫌乱 UI 加折叠视图。
|
||||
|
||||
**hard cascade 而非 soft orphan**:`orphaned` 让 list/resume/UI 都多一种特殊 case,代码长尾;"删 folder = 删项目" 比 "留对话残骸" 自然。`usage_events` append-only 不 FK,task 硬删后月账仍存活。
|
||||
**hard cascade 而非 soft orphan**:`orphaned` 让 list / resume / UI 都多一种特殊 case,"删 folder = 删项目"比"留对话残骸"自然。`usage_events` append-only 不 FK,task 硬删后月账仍存活。
|
||||
|
||||
**Docker + Postgres 起步**:运维门槛最低,Executor 抽象层留好,切 microVM / S3 都是 backend 替换不动接口。
|
||||
**本地也用 PG,不用 SQLite / JSON**:① dogfood ≡ 真实用户路径,bug 在 dogfood 就能复现;② Docker 已是必然依赖(§7.5),`docker compose up postgres` 零增量门槛;③ 双 adapter 维护税远高于 PG 一次性配置成本;④ 本地 dev 也能连远端测试服。
|
||||
|
||||
**本地也用 PG,不用 SQLite / JSON**:
|
||||
1. **dogfood ≡ 真实用户路径** —— 本地与 SaaS 走相同 SQL 方言、相同事务语义、相同 ORM,bug 在 dogfood 阶段就能复现,不会等到生产
|
||||
2. **Docker 已经是必然依赖** —— §7.6 #6 沙盒走 docker exec;装 Docker 是前提,顺手 `docker compose up postgres` 是零增量门槛
|
||||
3. **双 adapter 维护税远高于 PG 一次性配置成本** —— 一份 schema、一份 ORM、一份查询;SaaS 起步即终态,切换成本归零
|
||||
4. **本地 dev 也能连测试服** —— 不强迫本机起 PG,作者可直接连远端 dev / staging PG 跑 dogfood,体感跟连 SaaS 几乎一致
|
||||
**API-only,UI 由 platform 实现**(2026-05-15 决策):
|
||||
- **原计划**:Phase G 用 Jinja2 + HTMX 在本仓库做"简洁 Web UI",dogfood 用,真上线再做正经前端。已落地 G1-G6:task list / chat 流式 / files 浏览 / new / done/abandon/export/toast,共 ~600 行 HTML+CSS+SSE-HTML-片段。
|
||||
- **触发**:用户决定与已有 platform 联调,前端用 platform 的框架,本仓库再维护 HTML / CSS / HTMX 就是双套 UI 浪费。
|
||||
- **取舍**:
|
||||
- 删 `web/templates/*` `web/static/*` + jinja2/markdown-it-py/pygments/mdit-py-plugins 依赖
|
||||
- SSE 事件 payload 从 HTML 片段切 JSON(`{"type":"text","content":"..."}` 等);前端自渲染 markdown / tool_call 折叠
|
||||
- 路由统一 `/v1` 前缀,响应全 JSON,FastAPI 自带 `/docs` Swagger UI 接替"对内调试"角色(本地形态 `GET /` 302→ `/docs`)
|
||||
- 本地 sentinel user 形态保留;auth 走 D' 过渡形态(PLATFORM_KEY → JWT,见 §7.3),真 OIDC 留到联调约定 token 形态后接
|
||||
- CORS `allow_origins=["*"]` 本地宽松,platform 部署时按 platform 域名收紧
|
||||
- **沉淀**:G 阶段的 sink 协议(§7 A)/ RunBroker fan-out / no-subtask 校验 / files 路径安全归一 / task_dir 相对存储 全部保留,不在 UI 层不被牵连
|
||||
|
||||
**CLI 不被 API 取代,而是双模式共存**:本地直跑模式调 core 内部状态比 HTTP roundtrip 顺手;前端用户路径靠 `--remote` 模式打通。transport 层抽象代价小、长期价值高 —— 删本地直跑省不下多少代码,反而失去最便利的调试入口。**离线**靠本地 docker compose PG 兜底,不靠"全栈零依赖"幻觉。
|
||||
**dev SPA 留一份**(2026-05-15 决策):`web/static/dev.html` 单文件 vanilla JS,3 栏布局(task list + chat + files),~600 行无构建链。**与"UI 由 platform 实现"不冲突**:platform UI 是给真用户的、生产形态;dev.html 是给本仓库开发者自验 /v1 API + SSE 流的开发期工具。platform 未上线 / 网络断 / 凌晨随手验时不需要拉 platform。理由:① SSE 调试在 curl 里看不到 UI 反应,需要可视端;② Swagger 不发 SSE 流也没流式视图;③ 一个静态文件维护成本可忽略,删了再补不如留着。形态:登录页填 user_id(默 sentinel)+ platform_key → localStorage 存 JWT → fetch+Bearer。
|
||||
|
||||
**Memory 不入 DB**:跨 task 共享靠"同一 user 看同一份 FS 目录"的语义自动达成,不需要 schema。md 文件用户直接编辑器改,DB 化反而要造 UI、违反 §3.7 "事实由用户判断" 原则。两形态 memory 行为一致(只是根目录不同),迁移零成本。
|
||||
**CLI 不被 API 取代,而是双模式共存**:本地直跑调 core 内部状态比 HTTP roundtrip 顺手;前端用户路径靠 `--remote` 打通。离线靠本地 docker compose PG 兜底,不靠"全栈零依赖"幻觉。
|
||||
|
||||
**为什么 Tasks/Messages 在 PG 但 skill 产物在 FS**:tasks / messages 是元数据 + 对话流,需要查询、过滤、全文搜、跨 task 统计 —— 都是 DB 强项,jsonb GIN / pg_trgm 让查询代码不爆炸。skill 产物(`*.pptx` / `*.docx` / `sections/*.md`)是终用户拿走的文件,期望直接在文件管理器看到、用 Office 打开、邮件附件发出去 —— 进 DB 就要做"导出"这一步多余操作,且二进制 BLOB 在 PG 里没 GIN 索引价值。**FS 是产物的天然存储,DB 是元数据 / 状态 / 查询索引的天然存储,各司其职**。同理 §7.5 沙盒 bind mount = user root,容器里看到的就是用户在 Web UI 里看到的目录,无中间层翻译。
|
||||
**Memory 不入 DB**:跨 task 共享靠"同一 user 同一 FS 目录"自动达成。md 用户直接编辑器改,DB 化反而要造 UI、违反 §3.7"事实由用户判断"。
|
||||
|
||||
**Web UI 走 server-render + HTMX 不上 SPA**:① 与 §5 "Less Scaffolding" 一致,不引入 React/Vue 构建链 / node_modules / 双语言双 lint;② chat 主交互是 SSE 流式追加 + 表单提交,HTMX `hx-swap` / `sse-swap` 原生覆盖,无需客户端状态管理;③ FastAPI 单进程既出 `/v1` JSON 也出 HTML 模板,部署单容器;④ 上限低(协作 / 实时多光标 / 复杂表单态做不动),真要做重前端再换栈,届时 `/v1` 已稳定可直接对接 SPA。
|
||||
|
||||
**Tasks/Messages 在 PG 但 skill 产物在 FS**:tasks / messages 需要查询、过滤、全文搜、跨 task 统计 —— DB 强项;skill 产物(`*.pptx` / `*.docx` / `sections/*.md`)终用户拿走,期望文件管理器看到、Office 打开、邮件发出 —— 进 DB 要做"导出"多余操作。**FS 是产物天然存储,DB 是元数据 / 状态 / 索引天然存储**。同理 §7.5 bind mount = user root,容器里 ≡ 用户在 Web UI 看到的目录,无中间层翻译。
|
||||
|
||||
---
|
||||
|
||||
## 附录:DeepSeek V4 关键事实(2026-04-24)
|
||||
|
||||
- **V4-Pro**:1.6T 总 / 49B 激活,1M context,SWE-Bench 80.6 / Terminal-Bench 67.9 / MCPAtlas 73.6
|
||||
- **V4-Flash**:284B 总 / 13B 激活,1M context
|
||||
- 三种推理模式:non-thinking / thinking / thinking-max
|
||||
- 价格:输入 ~$0.145/M,输出 ~$1.74/M(约 Claude Opus 1/6 ~ 1/7)
|
||||
- `deepseek-chat` / `deepseek-reasoner` 在 2026-07-24 后下线 → 必须迁 `deepseek-v4-flash` / `deepseek-v4-pro`
|
||||
- **V4-Pro**:1.6T / 49B 激活,1M context,SWE-Bench 80.6 / Terminal-Bench 67.9 / MCPAtlas 73.6
|
||||
- **V4-Flash**:284B / 13B 激活,1M context
|
||||
- 推理模式:non-thinking / thinking / thinking-max
|
||||
- 价格:in ~$0.145/M,out ~$1.74/M(约 Claude Opus 1/6 ~ 1/7)
|
||||
- `deepseek-chat` / `deepseek-reasoner` 2026-07-24 下线 → 必须迁 `deepseek-v4-flash` / `deepseek-v4-pro`
|
||||
|
|
|
|||
111
PROGRESS.md
|
|
@ -1,88 +1,125 @@
|
|||
# 实施进度
|
||||
|
||||
> 配合 `DESIGN.md` 阅读。本文件只记录 phase 状态、决策偏差、文件量、下一步。
|
||||
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。
|
||||
|
||||
最后更新:2026-05-12(§7 改写为 user-direct SaaS 草案)
|
||||
最后更新:2026-05-17(`GET /v1/tasks` 分页 + 多维筛选 + ordering(DRF 风格,默认 `-created_at`):`{page, page_size, count, results}` 标准壳 + status/skill/working_dir/q 过滤 + 排序;dev SPA prev/next 翻页 + 搜索框 + 工作目录筛选 + 排序 dropdown;schema 重构:`name`(必填,显示名)+ `working_dir`(可选,留空 fallback name)解耦;`task_dir → working_dir` + `mode → skill` 列重命名)
|
||||
|
||||
---
|
||||
|
||||
## 状态
|
||||
|
||||
| Phase | 标题 | 状态 | 备注 |
|
||||
|------|-----|-----|------|
|
||||
|---|---|---|---|
|
||||
| 1-3 | 骨架 + Skill + run_python | ✅ | 三个 skill;CoreCoder 唯一匹配 edit;敏感 env 过滤 |
|
||||
| 4 | 演化性能力 | 🟡 | Model Profile + Probing ✅;版本化 prompt 未做 |
|
||||
| 5 | Eval Suite | ⏸ 不做 | dogfooding 替代,probe 覆盖健康检查 |
|
||||
| 6 | 长任务工程化 | 🟡 | task + state.json + 恢复 ✅;双层记忆 ✅;context 压缩未做 |
|
||||
| 6 | 长任务工程化 | 🟡 | task + 恢复 ✅;双层记忆 ✅;context 压缩未做 |
|
||||
| 7 | 打磨 | ❌ | Docker 沙盒 / 更多 skill |
|
||||
| §7 SaaS 化 | DESIGN §7 路线 | 🟡 | A 事件流化 ✅;B (Storage 落 PG + task_dir 双形态 + Folder API + no-subtask) 可立刻开,本地与 SaaS 共用同一种 storage |
|
||||
| §7 SaaS | DESIGN §7 路线 | 🟡 | A 事件流化 ✅;B 完工;**D `/v1` JSON API 完工 ✅**(原 Phase G Jinja2/HTMX UI 撤,改 platform 端实现);**D' 过渡 auth(PLATFORM_KEY → JWT)+ dev SPA ✅**;真 OIDC 待;C(Executor)待;E(CLI 双模式)待。 |
|
||||
|
||||
---
|
||||
|
||||
## 已完成关键能力
|
||||
|
||||
**2026-Q1 ~ 05-06:Phase 1-4** —— 骨架 / 三个 skill(coding/ppt/proposal)/ run_python 范式 / Model Profile + Capability Probing。`ppt` v3:商务红约束 + apply_brand + Iconify + render_icon/quality_check;素材摄取改 markitdown CLI。
|
||||
|
||||
**2026-05-06:Phase 6 部分** —— task + state.json + tokens 累计;CLI `tasks` + REPL `/status /done /abandon /desc`;移除 legacy `workspace/sessions/`。
|
||||
|
||||
**2026-05-07:TUI 打磨 + task_dir 落地** —— rich Markdown 渲染;thinking spinner 显实时耗时+累计 token;system prompt 注入 task_dir 绝对路径,skill 产物全收敛 `workspace/tasks/<id>/`;`.gitignore` 删 bandaid 行。
|
||||
|
||||
**2026-05-08:REPL task 切换 + 懒创建** —— `/resume [last|<id>]`;`build_agent` 不预占文件,首条 user 消息触发 save;`_cleanup_if_empty` 三条件守门防误删。
|
||||
|
||||
**2026-05-09 → 05-10:§7 草案 + 对话导出** —— DESIGN §7 初版 SaaS 草案(后于 05-12 重写);`cli.py export <task_id>` + `core/export_docx.py` 导对话成 docx。
|
||||
|
||||
**2026-05-11:原子写 + 双层记忆 + §7 A** —— `atomic_write_text` 接管 save;`core/memory.py` 双层记忆(core.md 入 prompt,extended/* 走索引);loop 事件流化(`sink.emit`)铺 SSE 路。
|
||||
|
||||
**2026-05-12:§7 改写** —— 原 platform/core 多租户方案废弃,改 user-direct(folder-centric,task/messages 入 PG,no-subtask 约束,hard cascade delete)。
|
||||
- **Q1 → 05-06 / Phase 1-4**:骨架 / 三 skill / run_python / Model Profile + Probing。ppt v3 加商务红 + apply_brand + Iconify;素材摄取改 markitdown CLI。
|
||||
- **05-06 / Phase 6 部分**:task + state.json + tokens 累计;CLI `tasks` + REPL `/status /done /abandon /desc`;移除 legacy `workspace/sessions/`。
|
||||
- **05-07 / TUI + task_dir**:rich Markdown 渲染;spinner 显实时耗时 + 累计 token;system prompt 注入 task_dir 绝对路径,产物收敛 `workspace/tasks/<id>/`;`.gitignore` 删 bandaid。
|
||||
- **05-08 / REPL 切换 + 懒创建**:`/resume [last|<id>]`;`build_agent` 不预占文件;`_cleanup_if_empty` 三条件守门。
|
||||
- **05-09 → 05-10 / §7 草案 + 导出**:DESIGN §7 初版(05-12 重写);`cli.py export <task_id>` + `core/export_docx.py`。
|
||||
- **05-11 / 原子写 + 双层记忆 + §7 A**:`atomic_write_text` 接管 save;`core/memory.py`(core.md 入 prompt,extended/* 走索引);loop 事件流化(`sink.emit`)铺 SSE 路。
|
||||
- **05-12 / §7 改写**:platform/core 多租户方案废弃,改 user-direct(folder-centric、task/messages 入 PG、no-subtask、hard cascade)。
|
||||
- **05-14 / §7.1 心智模型修正**:`Folder-centric` → **Task 一等公民 + Dir 文件副视图**(双视图正交,dir 不是 task 父容器);task_dir 留空=一次性对话 / 指定=项目化二分语义入文。
|
||||
- **05-14 / §7 B Step 1 基建**:`core/storage/{engine,models}.py` SQLAlchemy 2.x ORM(users/tasks/messages/runs/usage_events 5 表)+ alembic(初版 migration `0001_initial_schema`,GIN/复合索引)+ `cli db {upgrade,downgrade,current}` 子命令组 + 本地 sentinel user(`00000000-...`)+ `ZCBOT_DB_URL` 必填(未设给清晰报错,不引导 docker)。已在远端测试 PG 跑通 `db upgrade head`。
|
||||
- **05-14 / §7 B Step 2 Session ORM**:`core/session.py` 重写,messages 走 PG(append-only,jsonb,idx 严格递增);system prompt 不入库(每次 build_agent 重建);`Session.load(task_id, system_prompt=...)` resume 接口;`ensure_local_task_row` idempotent UPSERT(`INSERT ... ON CONFLICT DO NOTHING`)在首条非 system 消息前打底 tasks 行。task_id 切换为 UUID(原时间戳格式废弃,旧 workspace **不做兼容**)。main.py/cli.py 适配:`resolve_task_id`(UUID 前缀解析)、`_cleanup_if_empty` 双检查(DB messages + FS 产物)、`_list_task_rows` 改读 PG。`core/export_docx.py` 改从 PG 读 messages。端到端 build/append/resume/cleanup smoke 全绿。**取消 Step 5 migrate-from-fs**(用户决定不兼容旧 workspace)。
|
||||
- **05-14 / §7 B Step 3 TaskState ORM**:`core/task.py` 重写,TaskState dataclass 保留为内存 DTO 但落地走 PG —— `save()` 调 `upsert_task`(INSERT ON CONFLICT DO UPDATE,显式 set `updated_at=func.now()`),`load(task_id)` 走 SELECT;**字段去掉 `cwd`**(改读 task_dir,§7 SaaS task_dir-as-identity)。`state.json` 文件**全面废除**,task_dir 只承担 skill 产物。`core/storage/utils.py` 加 `upsert_task` / `update_task` 工具。`main.py::sync_task_tokens` 改 `update_task(tokens_p,tokens_c)` 单字段 UPDATE(ORM-level update 自带 onupdate=func.now())。`core/session.py::Session.append` 的 ensure 调用补传 `mode/description/reasoning_effort`,避免首次 INSERT 后 _list_task_rows 看到空 meta。`cli.py` 全字段从 ORM Task 列读;`_cleanup_if_empty` 去 state.json 特例(任何 FS 文件 / 子目录都算实质痕迹);`/done /abandon /desc` 走 PG。`core/export_docx.py` meta 改从 `TaskState.load(tid)` 读(asdict 拿到 dict),去 CWD 字段。端到端 smoke:storage UPSERT/UPDATE round-trip + build_agent 懒创建 + Session.append 自动 INSERT 完整 meta + sync_task_tokens 局部 UPDATE + task_state.save UPSERT 保留 task_dir/tokens + export → .docx 37KB 全绿。
|
||||
- **05-14 / §7 B Step 4 task_dir 双形态**:CLI `chat --task-dir <path>` 支持用户显式指定项目目录(§7.1 task-primary + dir 副视图心智模型落地)—— 留空走默认派生 `workspace/tasks/<uuid>/`,显式走用户路径(绝对或相对 cwd,Path.resolve())。`main.py::resolve_task_id` 增 `task_dir_arg`;resume 时从 PG `tasks.task_dir` 读(`SELECT task_dir WHERE task_id=?`),空则降级默认派生。新增 `is_managed_task_dir(td, ws)` 判断是否在 `workspace/tasks/<uuid>/` 模板下,作 `_cleanup_if_empty` 保护开关 —— 用户自指定的项目目录**绝不 rmtree**(可能含用户已有文件);DB 行该删还是删。`core/export_docx.py::export_chat_to_docx` 重构:task_id 升一等参数(从 `task_dir.name` 提取改入参传入),task_dir 留空时自动从 PG 读,支持用户目录(非 UUID 命名)正常导出。cli `/export` 与 `cli.py export` 子命令均改走 `_resolve_uuid_or_prefix` + task_id 直传。Smoke 4 路径全绿:default-derived(managed=True, cleanup rmtree)/ --task-dir(managed=False, FS preserved)/ resume reads DB / export 自动 PG 查路径。
|
||||
- **05-14 / §7 B Step 6 no-subtask 校验**:`core/storage/utils.py::check_no_subtask(task_dir, user_id=SENTINEL)` —— 同 user 下查 `new LIKE existing||'/%' OR existing LIKE new||'/%'`(`task_dir != new` 过滤掉同 task_dir 同项目多对话场景)。冲突抛 `NoSubtaskError`(`ValueError` 子类),消息带冲突 task 的 UUID 前 8 位 + 它的 task_dir。**分隔符容差**:SQL 里 `replace(task_dir, :bs, '/')` 把存的 Windows `\` 在比较前归一,新值也 `replace('\\', '/')`,跨 OS / 历史数据混合分隔符不漏判;`bs` 通过 bind 参数传(绕开 SQL 字符串转义陷阱)。空 / whitespace `task_dir` 直接 return(legacy / 未绑项目)。`main.py::build_agent` 在 `resolve_task_id` 后、TaskState 构造前调,`if not resume` 单层闸 —— resume 跳过(目录改名走未来 Folder API cascade,这里只拦新建)。`cli.py` 三处 build_agent 调用现有 try/except 直接接住 NoSubtaskError 并友好打印。Smoke 全绿:同 dir 允许 / child 拒 / parent 拒 / sibling 允许 / `proj_a_other` 不误中 `proj_a`(因为用 `/%` 而非 `%`)/ 空跳过 / Win `\` 子目录拒 / 混合分隔符(`\` 存 + `/` 查)仍拒 / build_agent 端到端三分支(child raise / same pass / resume bypass)。
|
||||
- **05-14 / §7 Phase G G1 Web UI 脚手架**:新增 `web/` 包(`app.py` FastAPI 工厂 + `templates/{base,home}.html` + `static/style.css`),`cli.py web --host --port --reload` 子命令(默认 127.0.0.1:8765,本地形态 sentinel user 无 auth,Phase D 才上 OIDC)。模板用 Jinja2 + HTMX/HTMX-SSE 走 CDN(无 node 链路),`base.html` 留 `{% block nav %}` 让 G2+ 扩。**Starlette 新版 `TemplateResponse` 签名**:`(request, name, context)`,旧式塞 context 里会让 jinja 用 dict 当 cache key 报 `unhashable type`,踩过修了。requirements 加 `fastapi>=0.111 uvicorn[standard] jinja2>=3.1 python-multipart`(后者为 G5 文件上传留)。Smoke 四路径全绿(in-process via Starlette `TestClient`):`/healthz` → "ok" / `/` → 1063B(title + static link + version) / `/static/style.css` → 1624B / `/nonexistent` → 404。**Linux portability 顺手**:模板里 path 显示约定用 `Path.as_posix()`(G3+ 模板落地);SSE 响应头 G4 上时带 `X-Accel-Buffering: no`(nginx 反代友好)。
|
||||
- **05-14 / §7 Phase G G2 task list 页**:`web/app.py::list_tasks(limit, status)` 读 PG `tasks` + `messages` count(updated_at 降序),返回模板友好的 dict 列表;**不复用 `cli.py::_list_task_rows`** —— CLI 拿 tuple, Web 拿 dict,数据形状有别,等真有 schema 变更同步成本时再抽(避免预付抽象)。`/` 路由换成 task 表渲染,filter via `?status=active|completed|abandoned`(无效值静默降级为 all);`/tasks/{task_id}` 占位路由 UUID 校验 + DB 存在性校验,缺一则 404,有效则渲染 `task_placeholder.html`(G3 来填消息流)。**Linux portability 落地**:`_norm_path()` 把存的 backslash 在显示时全替成 forward slash(`Path.as_posix()` 在 Linux 读 Win backslash 串时不归一,所以直接 `replace('\\','/')`);Win Path.resolve() 存 `D:\projects\...`、Linux 存 `/home/user/...`,都能正确显示。template:`home.html` 表格(id/updated/status/mode/model/msgs/tokens/desc-dir),status 用 badge(`status-active/completed/abandoned` 配色),hover 高亮;空态文案。CSS:table 紧凑(.9rem)+ `tabular-nums` 对齐 + accent-soft placeholder note。Smoke 18 路径全绿(in-process):3 task seed(active/completed/abandoned)+ Win\Linux 双路径形态 → / 渲染对、status filter 正/反向、garbage status 静默 all、UUID 占位、notauuid 404、ghost UUID 404、limit 生效、/healthz 不退化。版本 0.1 → 0.2。
|
||||
- **05-15 / §7 Phase G G3 chat 只读页**:`web/app.py` 加 `_get_md()` 单例 MarkdownIt(`gfm-like` 预设 + linkify + breaks,`html=False` 禁内联 HTML 防 XSS),fenced code 走 pygments `_pygments_highlight()` 回调(`codehilite` cssclass)。`load_chat_messages(tid)` 读 PG idx asc;`build_chat_blocks(messages)` 聚合显示块 —— system / tool 不入 block(tool 内嵌进 assistant 的 tool_call.result),user / assistant text 走 markdown 渲染,assistant.tool_calls 配对 tool result(orphan tool_call → `[no result]`)。`_args_preview` 60 字符截断,`_pretty_json` 解析失败 fallback 原串。`/tasks/{id}` 替换占位为 `chat.html` 渲染,删 `task_placeholder.html`。template:`.msg` 卡片(user 浅蓝 / assistant 白底),`.body` markdown 区(`<pre>` / `<code>` / `<table>` / `<blockquote>` / `<s>` 全 GFM 样式),tool_call 用 `<details>` 默认折叠(无 JS,浏览器原生开闭;`summary` 显示 tool 名 + args 前 60 字预览,展开看 args_pretty + result)。CSS 加 `.codehilite` 浅色 token 配色(keyword / string / comment / function / number / operator 6 类,余下黑色)。Smoke 28 路径全绿:4 display blocks(user/assistant×3,system/tool 跳过)+ markdown 特性(table / fence / autolink / strikethrough / bold)+ tool 配对(call_1 命中、orphan 走 `[no result]`)+ HTML 含 `<details>`/`tool-badge`/`codehilite`/`<s>` + 空 task 文案 + invalid UUID 404 + util 单测(args_preview / pretty_json / render_md 边界)。版本 0.2 → 0.3。requirements 加 `markdown-it-py[linkify]` / `mdit-py-plugins` / `pygments`。
|
||||
- **05-15 / §7 Phase G G6 部分:/new 入口(提前于 G5 落)**:用户反馈 Web 没"新建对话"入口 — 加 `GET /new` 表单页(description / mode / task_dir 三字段)+ `POST /new` 处理(strip 校验 + `description` 与 `task_dir` 至少填一个否则 400 + `check_no_subtask` 同 CLI / build_agent 一致拦前缀嵌套 → 409 + `ensure_local_task_row` 写占位行 + 303 See Other 跳转 `/tasks/{tid}`)。task_dir 空 → 默认派生 `workspace/tasks/<uuid>/`(同 `_default_task_dir`),显式 → `Path.expanduser().resolve()` 同 cli.py `--task-dir`。模板 `new_task.html` 加表单 + error 渲染(400/409 重渲带 form_state 不丢用户填的值);home.html 加 `+ new task` 主按钮 + nav 加 `new` 链接;base.html 默认 nav 也带 tasks/new。CSS 加 `.btn-primary` / `.new-task-form` / `.navlinks .active` 配色。**懒创建保留语义**:Task 在 /new POST 时入库,后续 build_agent 走 resume 路径(已存在,不冲突);CLI REPL `/new` 仍走 build_agent 懒创建路径,不互相干扰。Smoke 21 路径全绿:GET 表单 200 + 三字段 / POST happy(description-only / custom task_dir)→ 303 + Location 正确 / DB 行字段对 + default-derived task_dir 含 uuid / 空描述空 task_dir → 400 重渲表单带 error / no-subtask 父子嵌套 → 409 + 错误文案 / home 页 `+ new task` 按钮 + nav 链接 / `/new` nav 链接 active 标记。版本 0.4 → 0.5。
|
||||
- **05-15 / litellm 启动 cost map 网络警告兜底**:litellm 启动会去 GitHub 拉 `model_prices_and_context_window.json`,墙内 SSL 握手常超时,虽然有本地 backup 不影响功能,但 stdout 一行 WARNING 噪声大。`core/llm.py` 在 `import litellm` 之前 `os.environ.setdefault("LITELLM_LOCAL_MODEL_COST_MAP", "True")`(setdefault 不覆盖用户已显式设的值),走 litellm 的 `LITELLM_LOCAL_MODEL_COST_MAP=True` 路径直接用打包的本地 cost map,跳过 httpx.get。CLI / Web 都经 `core.llm` 走这条单点,不需要在多个入口分别设。冷启动从原来 ~5s SSL 超时降到 <1s。
|
||||
- **05-15 / task_dir 改相对存储**:DB `tasks.task_dir` 原存绝对(`D:\projects\zcbot\workspace\tasks\<uuid>`),改为 **ROOT 内→相对 posix(`workspace/tasks/<uuid>`)、ROOT 外→保留绝对**(用户 `--task-dir` 指外部项目的场景)。新增 `core/paths.py` 提供 `ROOT` / `to_db_path` / `from_db_path` 三个出口,所有读写边界统一过这里。读端:`resolve_task_id` resume 分支 `from_db_path(db_dir)`(相对走 `ROOT/.`,绝对原样 resolve);`export_chat_to_docx` 自动从 PG 读时同样过 `from_db_path`。写端:`build_agent` 构造 `meta` 和 `TaskState` 时 `to_db_path(task_dir)`,`web/app.py::/new` 同步。`check_no_subtask` 抛掉原来 SQL 里 `replace(task_dir, :bs, '/')` 的拼接,改 Python 端 fetch + 双侧 `from_db_path` 归一到 absolute posix 后比前缀,逻辑更清晰且天然支持混合形态(老绝对 + 新相对 DB row 并存也对)。alembic `0002_task_dir_relative` 一次 UPDATE 把现有 ROOT-prefix 行转相对(本机两条 active row 已 migrate 完);downgrade 反向用 `_:%` / `/%` LIKE 区分相对 vs 绝对。Smoke 四段全绿:round-trip(ROOT-内 / 外 / 空 / Windows backslash)/ check_no_subtask 混合形态 7 case(same / child / parent / sibling / outside-child / 绝对串新值 vs 相对串老 row 仍能拦 / 空跳过)/ resolve_task_id resume 还原一致 / build_agent 端到端写 DB 验证默认派生→相对、`--task-dir` 外部→绝对。`CLAUDE.md` 加"开发阶段不写兼容层"心智(用户指示)。
|
||||
- **05-15 / §7 D 阶段:`/v1` JSON API 落地;Phase G Jinja2/HTMX 路线撤掉**:用户决定与已有 platform 联调,前端用 platform 框架,本仓库再维护 HTML 就是双套 UI 浪费(DESIGN §7.9 新增取舍说明)。**删除**:`web/templates/*` 9 个模板 + `web/static/*` CSS 全去;`requirements.txt` 拿掉 jinja2 / markdown-it-py / mdit-py-plugins / pygments(`python-multipart` upload 还要用,保留)。**重写 `web/app.py` 全 `/v1/` 前缀,JSON 响应**:`POST /v1/tasks`(创建,Pydantic body)/ `GET /v1/tasks?status=&limit=`(列表)/ `GET /v1/tasks/{id}`(单 meta,不含 messages 走 /messages 拿)/ `PATCH /v1/tasks/{id}`(`{status?,description?,mode?}` 部分更新,active 不让从 web 切回)/ `GET /v1/tasks/{id}/messages`(LiteLLM 原 payload 透传)/ `POST /v1/tasks/{id}/messages`(JSON `{content}`,返 `{run_id, events_url}` + 起 BG run)/ `GET /v1/tasks/{id}/runs/{rid}/events`(SSE)/ files 4 路由全 `/v1/` + JSON 返回 / `GET /v1/tasks/{id}/export`(.docx 下载不变)/ `GET /healthz`(`{"status":"ok"}`)/ `GET /` 302→`/docs`(Swagger UI)。**SSE 事件 payload 由 HTML 片段切 JSON**:每帧 `event: <type>` + `data: <JSON dict>`,前端自渲染;event types `run_start / llm_start / text / tool_call / tool_result / llm_end / error / done`(去掉 `type` 键的剩余字段进 data)。**Pydantic 请求体** 给 FastAPI auto-docs 自动出 schema。**CORS** `allow_origins=["*"]` 起步(部署 platform 时收紧)。**没动**:`core/loop.py` event shape(已是 dict)/ `web/broker.py` fan-out / `web/sinks.py` WebEventSink / 文件路径安全归一 / no-subtask 校验。**Smoke 50+ case 全绿**(in-process TestClient + 真实 HTTP):root 302、healthz JSON、docs/openapi 暴露、tasks CRUD 全分支(create happy + custom dir + 双空 400 + 嵌套 409 + 列表 + 单 get + ghost/非 UUID 404 + PATCH 多分支 + 空 PATCH 400)、messages list/post(payload 透传 + run_id 返 + events_url 拼对 + 空 content 400)、files list/upload/download/delete(攻击名 400、路径越界 400、root 拒、size raw int、mtime ISO)、export PK\x03\x04 magic、CORS preflight `Access-Control-Allow-Origin: *`。真实 HTTP `cli.py web` 起服务 → curl `/healthz` `/v1/tasks` `/openapi.json` 全 200 + 干净 JSON。版本 0.7 → 0.7(API surface 完工)。`_smoke_api.py` ad-hoc 跑完即删。**沉淀的 Phase G 工作**:sink 协议 / RunBroker fan-out / no-subtask 校验 / files 路径安全归一 / task_dir 相对存储 全部保留 —— 删的只是 UI 层。
|
||||
- **05-15 / §7 Phase G G5 文件浏览 + 上传 / 下载 / 删**:`web/app.py` 加四件套路由 — `GET /tasks/{id}/files?path=<rel>`(列目录树,面包屑 + 目录在前文件在后 + size humanize + mtime 格式化)/ `GET /tasks/{id}/files/download?path=<rel>`(FileResponse + Content-Disposition)/ `POST /tasks/{id}/files/upload`(multipart `list[UploadFile]`,`?path=` 指目标子目录,自动 `mkdir(parents=True)`,303 回浏览页)/ `POST /tasks/{id}/files/delete`(form `path=...`,文件 / 空目录可删,非空目录 → 400,root → 400)。**核心:`_safe_join(root, rel)` 路径安全归一**——空 / "."→ root;`/` `\\` 起头 → 400(absolute-style 拒);Path.is_absolute → 400;`(root / rel).resolve().relative_to(root.resolve())` 校验仍在 root 内(防 `../` / symlink 逃逸)。上传文件名 strict 拒带 path 痕迹(`/` `\\` `..` parts)—— 现代浏览器只给 basename,异常 client 直接 400 不悄悄 sanitize。task_dir 不存在(skill 还没产物)→ 200 + 空文案,不报错。task_dir 空(legacy / 未绑)→ 400。`_load_task_dir(task_id)` 共用入口:404 if 非 UUID / task 不存在,400 if task_dir 空,否则返 `(tid, abs_path)`。**模板**:新增 `files.html`(面包屑 nav + upload-form `multipart/form-data` + `<table class="file-list">` 行渲染目录用蓝色 + `/` 后缀,文件用 `download` 链 + size + mtime + 删除按钮);`chat.html` 在 page-head 加 `files` 按钮(task_dir 非空时显示)。**CSS**:`.crumbs` / `.upload-form`(虚线红框 accent-soft 区)/ `.file-list` 表 / `.btn-mini` mini 按钮 + `.btn-mini-danger` 红 hover / `.ico-dir` `.ico-file` 文件类型标识。**Smoke 50+ case 全绿**:task_dir 不存在 200(2) / 列文件 + 子目录(12) / download 文件 + 子目录 + 404 + 目录-是非文件 400(7) / path 安全 6 case(`../` 越界 + POSIX 绝对 + Win 绝对 + `\\` 越界 + `/tmp`) / upload 单文件 + multi-file + nested mkdir + 攻击名 `../escape.txt` / `../../boom` / empty 全拒 + 目标 path 是文件 400 + 文件落 FS 内容一致(13) / delete 文件 + 空目录 + 非空 400 + ghost 404 + root 拒 + 越界拒(9) / chat.html files 链接 + ghost task_id 全 404(5) / task_dir 空 400(2)。版本 0.6 → 0.7。`_smoke_g5.py` ad-hoc 跑完即删。
|
||||
- **05-15 / §7 Phase G G6 三件套:/done /abandon 按钮 + /export 下载 + 全局 toast**:① `POST /tasks/{id}/status`(`status=completed|abandoned`,active 不让从 web 切回 → 400)走 `UPDATE tasks SET status`,303 redirect 回 `/tasks/{id}` —— 浏览器全页刷新,聊天流不重发。chat.html active task 渲两个 `<form method="post">` 按钮(原生 `confirm()` 防误操,无 HTMX 依赖),completed/abandoned 自动隐藏按钮只显 status badge。② `GET /tasks/{id}/export` 走 `tempfile.mkstemp(suffix=.docx)` → `export_chat_to_docx(tid, out_path=tmp)` → `FileResponse(..., background=BackgroundTask(tmp.unlink, missing_ok=True))` 响应完成自动删 tmpfile;无 messages → 400 / ghost UUID → 404 / 失败 → 500 带错文。chat.html 在 `n_messages > 0` 时渲 `<a class="btn">export .docx</a>`(浏览器原生下载,无 HTMX 干预 Content-Disposition)。**`export_chat_to_docx` 顺手修了一个 bug**:`task_dir is None` 且 PG 也空时旧逻辑硬抛 `ValueError`,即便 `out_path` 已经显式传入 —— 现在 `task_dir` 改为可选(None 时 meta 段显示 `(未绑)`),只在 `out_path` 也 None 时才报错。③ `base.html` 末尾加 `<div id="toast-region">` + inline JS 监听 `htmx:responseError`(4xx/5xx 抓 responseText 截 200 字)和 `htmx:sendError`(网络层挂),自动 5-6s dismiss + 手动 × 关。CSS `.toast` / `.toast-error` 右上角 fixed 区 + `@keyframes toast-in/out` 滑入滑出;`#toast-region` z-index 9999 + `pointer-events: none`(容器穿透,toast 自身可点)。Smoke 32 case 全绿:status 6 case(completed/abandoned 303 + DB UPDATE + GET 不再渲按钮、invalid status 400、active 400 拒切回、非 UUID 404、ghost 404)+ export 7 case(200 + Content-Disposition attachment + filename `chat_<8>.docx` + media-type docx + size > 8KB + magic `PK\x03\x04` + no messages 400 + 404 双路径)+ toast 6 case(div / 两 listener / CSS)+ chat.html 7 case(active 渲 done/abandon/export + confirm 文案 + completed 不渲)。版本 0.5 → 0.6。`_smoke_g6.py` ad-hoc 跑完即删(不入 git)。**TODO**:并发同 task 多 run lock 还没做(留到 D 阶段或下次)。
|
||||
- **05-15 / §7 D' 过渡 auth + dev SPA**:platform 联调前需要 auth,但完整 OIDC 还要等;落地 **PLATFORM_KEY → JWT 兑换** 过渡形态(`web/auth.py`),前后端走完全同一条流(platform 服务端 / dev 浏览器都持有 PLATFORM_KEY、调 `/v1/auth/login` 换 token、后续 `Authorization: Bearer <jwt>`)。**实现**:`pyjwt` HS256,`AuthConfig.from_env()` 启动校验 `PLATFORM_KEY` / `JWT_SECRET` 必填(任一缺失 fail-fast)、`ZCBOT_JWT_TTL_SECONDS` 默认 7d、`mint_token` / `verify_token` / `ensure_user_row`(任意 user_id 幂等 INSERT users 行避免 FK 失败)。`HTTPBearer(auto_error=False)` Depends 拿凭证 → `verify_token` → UUID;`make_require_user(cfg)` 工厂闭包持 cfg,FastAPI Depends 抽签到每个 /v1/tasks* 路由。**数据隔离**:所有 `SELECT Task` / `UPDATE Task` 增 `Task.user_id == user_id` 条件;`_load_task_dir(task_id, user_id)` 跨 user 视为 404(不暴露存在性);`check_no_subtask(... user_id=user_id)`、`ensure_local_task_row(... user_id=user_id)` 同 user 隔离 no-subtask 校验。新增 `_assert_owns_task(s, tid, user_id)` helper 复用 messages / SSE / export 三处所有权校验。**豁免**:`/`、`/healthz`、`/docs`、`/openapi.json`、`/v1/auth/login`、`/static/*` 不验 token。**dev SPA**(`web/static/dev.html` ~600 行单文件 vanilla JS):login overlay(user_id 默 SENTINEL 全 0 + platform_key) → localStorage 存 token → 3 栏布局(左 task 列表 + 状态 filter + 新建按钮;中 chat meta + 流式消息卡 + send 表单;右 file 浏览 + 面包屑 + 下载)+ 顶 bar(user 显示 + logout)+ new task modal。**SSE 走 fetch + ReadableStream**(不用 EventSource,因为 EventSource API 不支持自定义 header,token 没法塞;改用 fetch + 手解 SSE frame `\n\n` 切帧、`event:` `data:` 行解析、JSON.parse data 字段)。/ 302 → /static/dev.html(Swagger 仍在 /docs)。**Smoke 32 case 全绿**(TestClient + 真实 HTTP via uvicorn @8767):基本路由(/healthz / / 302 / dev.html 28KB / /docs 仍 200)+ 未带 token 8 路径全 401 + login 路径(bad key 403 / bad user_id 400 / happy 200 + token/expires_at/user_id 回显)+ 带 token CRUD 200/201 + 跨 user 隔离 4 case(other 看 sentinel 404 / 列表不串 / 各自创建独立 / sentinel 看 other 404)+ token 异常(garbled / Basic scheme / wrong-secret / expired 全 401)+ 真实 HTTP login + bearer call + dev.html 静态服务 29KB + root 302 Location 正确 + /docs 仍开放。版本 0.7 → 0.8。requirements 加 `pyjwt>=2.8.0`。**没动**:`core/*`、`build_agent`、`Session.append`、CLI 全链(本地 SENTINEL 单 user 默认走通,不进 web auth)。**TODO**:真 OIDC 接入(替换 /v1/auth/login 内部为 ID token 校验,路由层不动)。
|
||||
- **05-17 / `GET /v1/tasks` + ordering 排序(DRF 风格)**:加 `ordering` query 参数,逗号分隔多字段,`-field` 倒序;allowlist `created_at/updated_at/name/status`;非法字段静默丢弃,全非法 fallback 默认。**默认从 `-updated_at` 改 `-created_at`**(用户要求,创建时间倒序更稳定)。`_parse_ordering(s)` helper 返 sqlalchemy `order_by` 列表直接 `*expand`。dev SPA 加 ordering dropdown(7 个常用选项:创建/更新时间双向 + 名称双向 + 状态分组),默认值 `-created_at` 不发送 URL(参数干净);onchange 同 filter 一样 reset page=1。Smoke 7 case:default = `-created_at`(mu/alpha/zeta) / `created_at` asc 反向 / `name` asc(alpha/mu/zeta) / `-name` desc 反向 / 多字段 `status,-created_at`(状态 alpha 排序 abandoned→active→completed) / 非法字段 `garbage` → fallback default / 混合 `garbage,-name` → 仅 `-name` 生效。文档 DESIGN §7.2 / RUN 路由表同步。
|
||||
- **05-17 / `GET /v1/tasks` 分页 + 多维筛选 + dev SPA 翻页/搜索**:用户反馈 list 接口缺分页和 status 等筛选。改 `list_tasks_route`:① **标准分页壳** `{page, page_size, count, results}`(响应键固定顺序,前端契约稳定);② **6 个 query 参数** —— `page`(default 1, ≥1 clamp)/ `page_size`(default 20,1–100 clamp)/ `status` 单值 active|completed|abandoned(非法值静默忽略)/ `skill` 精确匹配 / `working_dir` **末段目录名**(后端自动拼 `workspace/users/<uid>/<name>` 比对,客户端不用知道完整 db form)/ `q` 走 PG `ILIKE '%q%'` 同时打 `name` + `description` 两列;③ 实现 select 出 conditions list 一把过 + 单 `COUNT(*)` + 单 `SELECT LIMIT/OFFSET`,无 N+1。**dev SPA** 改 `loadTaskList`:从老的"无分页 ?status=" 改成构造 URLSearchParams 传 page+filters;state 新增 `taskPage / taskPageSize / taskTotal`;新增 `renderPager()` 显示 `from–to / count (第 P/L 页)` + prev/next 按钮(`disabled` 边界态);筛选输入框(`#filter-q` `#filter-wd`)debounce 300ms 后 reset page=1 重拉;`#filter-wd` autocomplete 复用 `<datalist id="folders-datalist">`(focus 时 lazy 拉 `/v1/folders`)。task list pane 改成三段:① label + status select + 刷新 ② q 搜索 + 工作目录筛选 ③ pager(条件态显示)。**Smoke 12 case 全绿**:无 filter (count=25, page1=20条) / page=2 (count=25, 5条) / page_size clamp 500 → 100 / page=0 → 1 / status 单维 (10) / skill 单维 (10) / working_dir CJK '水泥申报' (10) / q 'AI' (10) / q CJK '废弃' (5) / status+skill 组合 (10) / status+skill 不命中 (0+[]) / 非法 status 静默 → 全集 25。新 envelope 验证:`list(data.keys()) == ['page','page_size','count','results']` 顺序固定。文档 DESIGN §7.2 路由签名 + RUN 路由表同步。
|
||||
- **05-17 / 0003 schema:name + working_dir + skill 三件套(去掉 task_dir / mode 旧名)**:用户反馈"name 应该自动 / 可改;现在的 name 其实是工作目录(可建可选)"——也就是要把任务标识和工作目录解耦。同时观察到 `mode` 命名抽象,跟项目 `skills/` 注册表对不上,顺手改 `skill`。**alembic 0003**:用户授权清表(`TRUNCATE tasks CASCADE`)+ `task_dir → working_dir` + `mode → skill` + 加 `name TEXT NOT NULL`(空表上 NOT NULL 不需要 backfill)。**ORM** `Task` 三列同步;**TaskState** 加 `name` 字段、`task_dir → working_dir`、`mode → skill`;**ensure_local_task_row / upsert_task** 签名重排(name 必传 INSERT 路径);**check_no_subtask** ORM 引用 + 形参 `task_dir → working_dir`;**core/paths.py / export_docx.py / session.py** 同步刷;**main.py::build_agent** 重构:new task 必传 `name`(任务名)+ 可选 `working_dir`(留空 → fallback name 作目录),两者都过 `validate_task_name`;`working_dir_from_name` 取代 `task_dir_from_name`(纯路径派生);`resolve_task_id` 形参 `working_dir_name`;mkdir 在新建分支后 + check_no_subtask 后立即落盘;**meta dict** 多 `name` 字段、原 `task_dir`/`mode` 改 `working_dir`/`skill`。**CLI**:`chat --name <必填>` + `--working-dir <可选>` + `--skill <coding/ppt/...>`;`/new <name>` 自动复用当前 working_dir(取上层 task_dir 末段);`/new` 无参 → 自动 gen `新任务_HH-MM-SS`;`/status` 显示 name + skill + working_dir 全套;`/resume` 列表加 name + skill 两列。**web /v1**:`TaskCreateRequest` 字段 `name`(req)+ `working_dir`(opt) + `description` + `skill`;`TaskPatchRequest` 加 `name` + `skill`(去 `mode`);`create_task` working_dir 留空 fallback 用 name + mkdir + check_no_subtask;**新增 `GET /v1/folders`**(列 user 下 FS 非 dotfile 子目录 + 关联 task 计数 + 最后使用时间,sort `last_used desc, name asc`);`_load_task_dir → _load_working_dir`;`_task_dict` 返回 dict 加 `name` 字段、`task_dir → working_dir`、`mode → skill`;路径越界错文案 task_dir → working_dir。**dev SPA** modal:任务名 + 工作目录(配 `<datalist>` autocomplete 走 `/v1/folders`)+ skill + description 四字段;`hd-new` 打开 modal 时拉 folders;`nt-wd` 输入时实时提示"→ 复用已有目录 (N 个 task)" / "→ 新建目录 X" / "留空 → 用任务名 fallback";`renderTaskList` 主行从"working_dir 末段"改为 `t.name`(任务名优先),`📁 工作目录名`+ skill + description 走副行;`renderChatMeta` 同步把 name 顶头 + 📁 + skill + tid + desc + 计数。**Smoke**:9 case `/v1/tasks` POST 全绿(name+working_dir 双填 / 同 working_dir 二次共享 / 留空 fallback / name 缺 → 422 / name 非法 → 400 / working_dir 非法 → 400 / GET 列表含三新字段 / PATCH name+skill / `/v1/folders` 含 水泥申报 n_tasks=2 last_used 非空);CLI build_agent 4 case(new 双填 / append 后 reloaded 字段对 / resume 还原 / fallback)。文档:DESIGN §3.1 目录树注释 / §3.6 三件套字段语义 + 创建语义 / §7.2 POST + PATCH + DELETE + 新 GET /v1/folders / §7.4 schema 块 + index 同步;PROGRESS 单条记录;RUN 待刷。
|
||||
- **05-17 / files 面板 UX 让用户清楚"我在哪个项目里" + 修 root crumb bug**:用户反馈"web 右侧 files 看不到文件夹"——实际场景是用户建了 task name=水泥申报,FS `workspace/users/.../水泥申报/` 已建,但里面是空的,files 面板只显示"(空目录)"——用户混淆为"看不到 水泥申报 这个文件夹本身"。真因是 UI 没把"现在面板内部就是 水泥申报 的内容"说清楚。修两处:① 后端 `_enumerate_files`:`cur_rel == "."`(target == root)时不再追加一个无意义 "." crumb(原来 `if cur_rel:` 把 "." 当真值,会塞 `{label: ".", rel: "."}` 进 crumbs[1]);改为 `if cur_rel and cur_rel != "."`。② dev SPA `renderFiles`:`pane-head` 旁加 `<span id="files-proj">`(`muted small` 样式 + ellipsis),`textContent = "· " + projName`(取 `task_dir.split('/').filter(Boolean).pop()`);crumbs 第一格 label 从 "/" 替换为项目名(`projName`),整条路径直观为 `水泥申报 / 草稿 / draft.md`。`deleteCurrentTask` 清面板时也 reset `files-proj`。Smoke:root 路径 crumbs 长度 == 1(原 == 2);进 `水泥申报/草稿` 子目录 crumbs == 2 且第二格 label == "草稿"(CJK 透传 OK);GBK 控制台显示乱码确认是 stdout encoding 而非 PG 存储问题(`task_dir.encode('utf-8')` 字节正确 + codepoints 是 [0x6c34, 0x6ce5, 0x7533, 0x62a5])。
|
||||
- **05-17 / task 硬删 API + dev SPA delete 按钮 + 文件 per-row 删**:用户反馈缺 task 删除入口(原本只有 PATCH status=abandoned 软态)。新增 `DELETE /v1/tasks/{id}`:user_id ownership 校验(跨 user → 404 不暴露存在性)+ DB 行 DELETE(messages / runs CASCADE)+ **FS task_dir 不动**(同 name 多 task 共享语义下"最后一个 task 删了顺便 rmtree"的判断有边界 case,易擦用户素材;让用户经 /files/delete 或文件管理器显式清更安全)。返 204 No Content。dev SPA:chat 面板 head 加 `btn-delete-task`(`small danger` 样式,title 说明"清 DB 行 + messages,FS 文件不动"),`disabled` 仅在没选 task 时 true —— 任何 status 都可删(active / completed / abandoned 不限,confirm 弹窗带项目名 + 消息条数二次确认)。点击后清空 chat 面板 + files 面板 + state reset + reload task list。file 面板 per-row 加红 `×` 按钮(`del-file` class),click stopPropagation 不触发行的下载/进目录;调原有 `POST /v1/tasks/{id}/files/delete`(API 没改,非空目录仍 400 拒,弹错文)。Smoke 6 case 全绿:happy 路径 204 + DB 行 gone + FS `should_survive.txt` 保留 / messages CASCADE 真生效(idx=1 user msg INSERT 后 DELETE task → messages count = 0)/ ghost UUID 404 / 非 UUID 字符串 404 / 跨 user delete 404 + 原 user 仍可删 + 原 task 行未被擦 / 无 token 401。文档:DESIGN §7.2 资源模型加 `DELETE /v1/tasks/{id}` 行 + 注释 "FS task_dir 保留"。
|
||||
- **05-17 / task_dir 改 eager mkdir + dev SPA 列表显示项目名**:用户反馈"创建 task 给了名字也聊了天,文件夹没建出来"——原"懒 mkdir(skill 第一次写产物时建)"是 UUID-named 派生目录时代的设计,现在 task_dir 是用户给的项目名(`workspace/users/<uid>/<name>/`),**name = 项目声明**,目录就该在 task 创建时存在(用户可立刻往里塞素材文件,而非等 LLM 触发 skill)。改两处入口:`main.py::build_agent` 新建分支(`not resume` && no-subtask 校验后)+ `web/app.py::create_task`(`fs_dir = task_dir_from_name(...)` 之后),都加 `mkdir(parents=True, exist_ok=True)`。同 name 多 task 共享同目录(§7.1),`exist_ok=True` 无冲突 + 已有内容(其他 task 产物或用户素材)不被擦。`task_dir_from_name` 仍保持纯路径派生(docstring 同步)。`cli.py::_cleanup_if_empty` 注释里"未触发 lazy mkdir"过时表述修正。**dev SPA `dev.html`**:`renderTaskList` 主行原本 `t.description || "(no desc)"`,description 空时一片"(no desc)"丑;改为 **主行 = 项目名(`task_dir.split('/').filter(Boolean).pop()`)+ description 移到副行(空则不渲)+ `task_id[:8]` 移到 badge 行末段**,信息密度更高且每条都有标识。`renderChatMeta` 同步同样规则。**DESIGN §3.6 同步**:删"task_dir 在 skill 第一次落产物时 mkdir"+ 修过时的 `_cleanup_if_empty` 描述("DB 无 messages 且 FS 无产物 → DELETE + rmdir" → 实际现在 "无 user msg → DELETE DB 行;FS 一律不动")。Smoke:`task_dir_from_name` 纯路径不预 mkdir + `mkdir` idempotent + POST /v1/tasks 后 FS 真存在 + 同 name 二次 create reuse 不擦已落入文件(`user_marker.txt` 保留)+ DB 双行 task_id 不同 task_dir 同。
|
||||
- **05-17 / task = name-based 项目目录 + memory dotfile**:废弃自动 UUID 派生 + `tasks/` 中间层。新建 task **必须给 `name`(简单名,项目目录名)**,task_dir 派生为 `workspace/users/<uid>/<name>/`;同 name 多 task 自动共享同目录(§7.1 task-primary)。**`name` 校验**(`main.py::validate_task_name`):非空 / 不含 `/\NUL` / 不以 `.` 起头(挡 `.memory` 等系统区)/ ≤ 255 字符;允许 CJK 与其他 Unicode。**memory 搬 dotfile**:`workspace/users/<uid>/.memory/{core.md, extended/}`,跟用户项目目录扁平共存不撞名;`validate_task_name` 拒 `.` 起头双向防呆。**删函数**:`_default_task_dir` / `is_managed_task_dir` / `tasks_dir` 全删,`build_agent.task_dir_arg` 改 `name`,`cli.py --task-dir` 改 `--name`;web `TaskCreateRequest.task_dir` → `name`(必填),`POST /v1/tasks` 缺 name → 422 (Pydantic) / 不合法 → 400(`InvalidTaskName` 文案)/ 同名共享 task_dir 不触发 no-subtask。**`_cleanup_if_empty` 简化**:FS 一律不动(项目目录跨 task 复用,绝不 rmtree),空 task 只删 DB 行;原"managed 派生模板"概念整个废弃。**dev SPA**:新建 task modal 字段 `task_dir`(留空)→ `name`(必填),task 列表行末段显示项目名(`task_dir.split('/').pop()`)而非两段 UUID/path。**清旧数据**:`workspace/users/*/tasks/` 上轮白建的中间层空目录 + `users/<uid>/memory/`(非 dotfile)全 rm,DB 已空。Smoke 全绿:validate_task_name 14 case 边界(简单名 / 中间含 `.` / 空白 strip / CJK / 含 `/\NUL` 拒 / `.` 起头四种拒 / 长度边界 255 vs 256)+ 路径派生 + memory dotfile 路径 + CLI build_agent name 必填强制 + web `/v1/tasks` 4 + name 不合法 400 全分支 + 同 name 多 task 共享同 task_dir + resume + dotfile memory 注入 + 跨 user 隔离。文档同步:DESIGN.md §3.1 目录树 / §3.7 memory 路径 + dotfile 说明 / §7.0 表 / §7.1 留空派生改"必给 name" / §7.2 /v1/tasks POST 入参 / §7.4 schema 注释 + 文件系统块 / §7.6 Step 3 描述 全部刷新。**未来形态**:外部绝对路径(项目目录在 workspace 外的场景)暂未保留,日后真需要再开 `--external-path` 单独通道。
|
||||
- **05-15 / workspace 布局统一 per-user**:DESIGN §7.0 / §7.4 落地补 —— 原默认 task_dir `workspace/tasks/<uuid>/` + 全局 `workspace/memory/` 改为 **`workspace/users/<user_id>/{tasks/<uuid>,memory/}/`**(本地 CLI user_id = SENTINEL,web/JWT user_id = JWT sub)。`main.py::_default_task_dir(workspace, tid, user_id)` / `tasks_dir(workspace, user_id)` / 新增 `user_root(workspace, user_id)` / `is_managed_task_dir(td, ws, user_id)` 都接 user_id;`resolve_task_id` / `build_agent` / `_build_system_prompt` 透传(`build_agent.user_id` 留 Optional 默认 SENTINEL,CLI 不需要显式传)。`core/memory.py::memory_block(workspace, user_id)` per-user 子树读 `users/<uid>/memory/`,prompt 段从"workspace 级"改"user 级"。`web/app.py::create_task` 把 Depends 拿到的 JWT user_id 喂进 `_default_task_dir`;`_run_agent_bg(task_id, run_id, user_id, msg)` 加 user_id 参数透传给 `build_agent(resume=True, user_id=...)` —— 确保 web resume 时 memory_block 读对 per-user 子树。`cli.py::_cleanup_if_empty` 调 `is_managed_task_dir(..., SENTINEL_USER_ID)`;`tasks_dir(ws, SENTINEL_USER_ID)` 显示路径。**没动 session.py** —— `Session.append → ensure_local_task_row(user_id=SENTINEL)` 默认值对 CLI 正确;web 路径 `create_task` 已提前预 INSERT 真 user_id 的占位行,后续 ON CONFLICT DO NOTHING 不会落 SENTINEL 默认值。开发期心态(CLAUDE.md):**清旧数据不留兼容** —— DELETE FROM tasks(CASCADE messages/runs)+ usage_events;`rm -rf workspace/tasks/`;保留 users 表 2 行(sentinel + 已登录 web user)避免 JWT FK 失败。文档同步:DESIGN.md §3.1 目录树 / §3.7 memory 路径 / §7.0 task_dir & memory 默认值表 / §7.1 留空派生路径 / §7.2 /v1/tasks POST task_dir 默认值 / §7.4 task_dir 存储约定注释 / §7.4 文件系统块布局 / §7.6 Step 3 描述 全部刷新。
|
||||
- **05-15 / §7 Phase G G4 chat 发送 + SSE 流式**:新增 `web/broker.py::RunBroker`(in-process pub/sub,`subscribe/emit/close/unsubscribe`)+ `web/sinks.py::WebEventSink` 实现 §7 A 的 sink 协议,把 `AgentLoop._emit` 桥到 broker。**异步策略 = `asyncio.to_thread`**(不改 core):POST `/tasks/{tid}/messages` async handler → 校验 task + INSERT `runs` 行 + `asyncio.create_task(asyncio.to_thread(_run_agent_bg, ...))`,`_run_agent_bg` 在工作线程跑 `build_agent(resume=True) + agent.run`,sink 通过 `loop.call_soon_threadsafe(q.put_nowait, ev)` 跨线程桥事件回 asyncio queue。**多访问策略 = fan-out**:每订阅一个独立 `asyncio.Queue`,同 run 多 tab / 刷新 / 桌面+移动都看得到流;`_done` 集合让晚到订阅者立即收 `done`(不挂)。GET `/tasks/{tid}/runs/{rid}/events` 返 `StreamingResponse` async gen,响应头带 `text/event-stream / Cache-Control: no-cache / X-Accel-Buffering: no`(nginx 反代友好);第一帧发 `: connected\nretry: 3000\n\n` 让 EventSource 立即建立,30s 无 event 发 `: ping` 注释心跳。**SSE multi-line data**:HTML 片段含换行,每行加 `data: ` 前缀(SSE spec),EventSource API 还原成 `\n` 拼接的 HTML 字符串。**Event → HTML 片段**:`_render_event_fragment` 渲染 `text`/`tool_call`/`tool_result`/`error` 四种,`run_start/llm_start/llm_end/done` 发空 data(只让客户端识别 event type)。新 fragment 模板 `_frag_text.html` / `_frag_tool_call.html` / `_frag_tool_result.html` / `_frag_error.html` + `_send_response.html`(POST 响应:user msg 卡 + `msg-assistant streaming` 容器带 `sse-connect/sse-swap/sse-close`)。`chat.html` 加 send 表单(Enter 发送、Shift+Enter 换行,HTMX `hx-post / hx-target=#chat-stream / hx-swap=beforeend / hx-on::after-request reset`);`chat` section 改 `id="chat-stream"` 让 SSE 追加进同一容器;非 active task 隐藏表单。CSS 加 `.streaming .run-indicator` 红点脉冲 / `.send-form` 表单样式 / `.tool-result-inline` 追加式样式 / `.msg-error` 错误卡。**Run 状态写 PG `runs` 表**:POST 时 status=running,正常完结 status=ok + tokens_p/c,异常 status=error + error 文本;DB 写失败不放大噪声(已 emit error 给前端)。**lifespan** `bind_loop(asyncio.get_running_loop())` 让 broker 拿到 asyncio loop 引用。Smoke 双层全绿:broker 单元 8 case(subscribe/emit/get、fan-out 双订阅、跨 run_id 隔离、close 派 done、late subscribe 立刻收 done、unsubscribe 后失联、WebEventSink 桥、unbinded loop silent drop);端到端 24 case(POST 200 + HTML 含 sse-connect + run_id 抽出 + SSE stream content-type/x-accel-buffering/cache-control 头对、event types 序列 `run_start/llm_start/text/tool_call/tool_result/llm_end/done`、text fragment 含 `<strong>` markdown、tool_call 含 `<details>`、tool_result 含 preview、empty body 400、invalid/ghost UUID 404、late subscribe 立刻 done、PG runs 行 INSERT)。版本 0.3 → 0.4。**TODO**:并发同 task 多 run 互锁(messages idx UniqueConstraint 在并发 POST 下会冲突 — 用户连续点 send 暂时不会触发,但需要在 G6 或 D 阶段加 lock_for_update);event log 持久化(刷新继续看流式)留到未来。
|
||||
|
||||
---
|
||||
|
||||
## 关键决策与偏差
|
||||
|
||||
| 项 | 决策 | 备注 |
|
||||
|---|------|------|
|
||||
|---|---|---|
|
||||
| 工具基目录 | cwd(读)+ task_dir(写) | system prompt 同时注入两者绝对路径 |
|
||||
| Workspace 布局 | `tasks/<id>/` + `memory/{core.md, extended/}` | memory 跨 task 共享 |
|
||||
| Eval Suite | 不做 | 个人工具用 dogfooding |
|
||||
| Workspace 布局 | `workspace/users/<user_id>/{.memory/, <name>/}` | per-user 隔离;memory dotfile 防撞;`<name>` 用户起项目名,同 name 多 task 共享;CLI sentinel = `00000000-...` |
|
||||
| Eval Suite | 不做 | 个人工具 dogfooding |
|
||||
| 版本化 prompt | 直接 `general_v1.md` | Windows 软链接麻烦,真要切再做 |
|
||||
| run_python 沙盒 | subprocess + env 过滤 | Docker 在 §7 C 阶段 |
|
||||
|
||||
---
|
||||
|
||||
## 文件清单(代码量)
|
||||
## 文件清单
|
||||
|
||||
```
|
||||
core/capabilities.py 71
|
||||
core/llm.py 89
|
||||
core/llm.py 93 ← +litellm 离线 cost map env
|
||||
core/loop.py 152 ← §7 A: sink.emit
|
||||
core/sinks.py 101 ← §7 A
|
||||
core/ui.py 38
|
||||
core/paths.py 50 ← task_dir db form 归一(to_db_path / from_db_path)
|
||||
core/probe.py 243
|
||||
core/session.py 93 ← +atomic_write_text
|
||||
core/session.py 153 ← §7 B Step 2-3: ORM + ensure 补 meta
|
||||
core/skills.py 81
|
||||
core/task.py 64
|
||||
core/memory.py 76
|
||||
core/export_docx.py 372
|
||||
core/task.py 82 ← §7 B Step 3: PG-backed TaskState,去 cwd
|
||||
core/memory.py 81 ← per-user `.memory/` dotfile
|
||||
core/export_docx.py 383 ← §7 B Step 2-4 + from_db_path 还原 + task_dir Optional
|
||||
core/storage/__init__.py 27 ← §7 B Step 1-3
|
||||
core/storage/engine.py 80 ← §7 B Step 1
|
||||
core/storage/models.py 124 ← §7 B Step 1
|
||||
core/storage/utils.py 136 ← check_no_subtask 改 Python 端归一
|
||||
tools/base.py 34
|
||||
tools/fs.py 182
|
||||
tools/shell.py 94
|
||||
tools/run_python.py 84
|
||||
tools/skill_tool.py 45
|
||||
main.py 210
|
||||
cli.py 439
|
||||
main.py 285 ← user_root / task_dir_from_name / validate_task_name(删 auto-derive 三件套)
|
||||
cli.py 558 ← §7 B Step 4 / Phase G G1: --task-dir / web 子命令
|
||||
db/migrations/env.py 61 ← §7 B Step 1
|
||||
db/migrations/versions/
|
||||
0001_initial_schema.py 125 ← §7 B Step 1
|
||||
0002_task_dir_relative.py 61 ← 现有 ROOT-prefix 绝对 → 相对
|
||||
web/__init__.py 5 ← Phase G G1
|
||||
web/app.py 660 ← /v1/ JSON API + user_id 隔离(D' 过渡 auth)
|
||||
web/auth.py 115 ← D' 过渡:PLATFORM_KEY → JWT 兑换
|
||||
web/broker.py 88 ← Phase G G4: in-process pub/sub
|
||||
web/sinks.py 20 ← Phase G G4: WebEventSink (§7 A sink 协议)
|
||||
web/static/dev.html ~600 ← D' dev SPA(login + 3-pane,vanilla JS)
|
||||
─────────────────────────────────
|
||||
Python 合计 ~2429 行
|
||||
Python 合计 ~3700 行(+ dev.html ~600 静态)
|
||||
```
|
||||
|
||||
加 skills/ppt 脚本 ~600 行 + SKILL.md / references / config / prompts,总仓库约 3000 行。
|
||||
加 skills/ppt 脚本 ~600 行 + SKILL.md / references / config / prompts + alembic.ini,总仓库约 3500 行。
|
||||
|
||||
---
|
||||
|
||||
## 下一步候选(性价比排序)
|
||||
|
||||
1. **§7 B 阶段**(~1 周)—— Storage 落 PG(单一实现,无 adapter 抽象)+ task_dir 双形态 + Folder API + No-subtask。**dogfood 即生效**(messages 进 DB → 全文搜立刻可用)。
|
||||
- 前置:repo 加 `docker-compose.yml`(`docker compose up -d postgres` 起本地 dev PG)或 `ZCBOT_DB_URL` 指向远端测试 PG
|
||||
- 里程碑:① schema + alembic 初版迁移 ② SQLAlchemy ORM 接入 `Session` / `TaskState` ③ CLI 适配(去 `.json` 读写,加 `_cleanup_if_empty` 新逻辑)④ `cli migrate-from-fs` 工具(把现有 `workspace/tasks/*/` 导入 PG)⑤ Folder API + no-subtask SQL 校验 ⑥ 本地单用户 sentinel(`user_id='00000000-...'`)init 流程
|
||||
2. **Phase 6 context 三层压缩**(~1 天)—— 兜底,V4 长上下文一般用不到
|
||||
3. **Phase 7 更多 skill / 模型档案**(持续)
|
||||
4. **Proposal mermaid 流程图预渲染**(~半天)—— ASCII 透传不够用时再上 `mmdc`(Node.js 依赖)
|
||||
1. **platform 端起 API 联调**(~?)—— platform 服务端持 `PLATFORM_KEY` 调 `POST /v1/auth/login {user_id, platform_key}` 拿 token,后续走 `Authorization: Bearer <jwt>`。Swagger UI(`http://127.0.0.1:8765/docs`)生成 client stub。
|
||||
2. **dev.html 浏览器手验**(~10 分钟)—— `cli.py web` 起后访问 `http://127.0.0.1:8765/`,login(填 sentinel UUID + PLATFORM_KEY)→ 看 3 栏布局 + 新建 task + 发消息 SSE 流式 + 文件浏览。
|
||||
3. **真 OIDC 接入 + CORS 收紧**(~1 天)—— 把 `/v1/auth/login` 内部从 platform_key 校验换成 OIDC ID token 校验(路由层 Depends 不动);CORS 改成 platform 域名 allowlist。
|
||||
4. **§7 C Executor + sandbox**(~2-3 天)—— D 完工,继续 C。
|
||||
5. **并发 run 互锁**(~2 小时)—— 用户连发两条消息 messages idx UniqueConstraint 在 race 下会冲;PG `SELECT ... FOR UPDATE` 锁 tasks 行,或 advisory lock。
|
||||
6. **§7 E CLI transport 双模式**(~1.5 天)—— `cli.py chat --remote https://...` 走 HTTP 替代 in-process。
|
||||
7. **Phase 6 context 三层压缩**(~1 天)—— 兜底,V4 长上下文一般用不到。
|
||||
|
||||
> §7 B + D + D'(过渡 auth)主体已完工。剩余路线:真 OIDC → C(Executor)→ E(CLI 双模式)→ F(deploy / billing)。原 Phase G Web UI 路线撤(DESIGN §7.9),UI 改 platform 端实现;`web/static/dev.html` 是开发期单文件 SPA,跟 platform UI 并存不冲突。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,206 @@
|
|||
# 运行手册
|
||||
|
||||
> 怎么把 zcbot 跑起来。env / 常用命令 / 故障兜底。设计看 `DESIGN.md`,进度看 `PROGRESS.md`。
|
||||
|
||||
最后更新:2026-05-17(task 拆 `--name`(必填,任务名)+ `--working-dir`(可选,目录名);`--mode → --skill`;`/v1/folders` 列已有目录;0003 migration)
|
||||
|
||||
---
|
||||
|
||||
## 环境
|
||||
|
||||
- **Python**:虚拟环境 `.venv/`,所有依赖装在里面。一律用 `.venv/Scripts/python.exe ...`(Windows)/`.venv/bin/python ...`(Unix),不要全局 `python`(litellm/python-pptx 等会 ModuleNotFoundError)。
|
||||
- **配置文件 `.env`**(项目根,git 忽略,litellm 自动加载):
|
||||
```
|
||||
DEEPSEEK_API_KEY=sk-...
|
||||
ZCBOT_DB_URL=postgresql://user:pass@host:5432/zcbot
|
||||
# cli.py web 必填(纯 CLI 用不到,只在起 web 时校验)
|
||||
PLATFORM_KEY=<至少 16 字符的随机串,platform 服务端 / dev 浏览器持有,登录时校验>
|
||||
JWT_SECRET=<≥32 字符随机串,HS256 签 session token;泄漏 = 任意伪造,与 PLATFORM_KEY 同级保护>
|
||||
# 可选:覆盖默认 7d
|
||||
# ZCBOT_JWT_TTL_SECONDS=604800
|
||||
```
|
||||
> litellm 在 import 时副作用加载 .env;CLI 入口直接走 `cli.py`,`.env` 会自动生效。直跑 `python -c "from core.storage import ..."` 不经 litellm 链路时记得自己 `import litellm` 触发,或手动 `export ZCBOT_DB_URL=...`。
|
||||
- **依赖**:`pip install -r requirements.txt`(已在 `.venv` 里)。
|
||||
- **PG**:`ZCBOT_DB_URL` 必填。本地 docker compose 起 / 远端 dev / 生产任选。未设置时启动会清晰报错,不引导 docker(§7.4)。
|
||||
- **Auth env**(`cli.py web` 必填):`PLATFORM_KEY` + `JWT_SECRET`,任一缺失 web 启动会 fail-fast。生成随机串可用 `python -c "import secrets; print(secrets.token_urlsafe(48))"`。CLI(`chat / tasks / probe / db`)不验,不要这两个 env 也能跑。
|
||||
|
||||
---
|
||||
|
||||
## 一次性初始化
|
||||
|
||||
```bash
|
||||
# 1) 装依赖(若 .venv 不在)
|
||||
python -m venv .venv
|
||||
.venv/Scripts/python.exe -m pip install -r requirements.txt
|
||||
|
||||
# 2) 准备 .env(见上)
|
||||
|
||||
# 3) DB schema 上车
|
||||
.venv/Scripts/python.exe cli.py db upgrade head
|
||||
.venv/Scripts/python.exe cli.py db current # 应输出 0003 (head)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 日常命令
|
||||
|
||||
### 聊天 / 任务
|
||||
|
||||
```bash
|
||||
# 新建 task —— `--name` 必填(任务显示名),`--working-dir` 可选(目录名,留空 → 用 --name)
|
||||
.venv/Scripts/python.exe cli.py chat --name "初稿大纲" --working-dir proposal_v3
|
||||
|
||||
# 只给 name → working_dir fallback 用 name
|
||||
.venv/Scripts/python.exe cli.py chat --name proposal_v3
|
||||
|
||||
# 带 skill + 描述(便于后续 list 识别)
|
||||
.venv/Scripts/python.exe cli.py chat --name "修登录 401" --working-dir fix_login_bug --skill coding --desc "登录返回 401 排查"
|
||||
|
||||
# 同 working_dir 多 task(共享 workspace/users/<sentinel>/proposal_v3/ 目录,name 各不同)
|
||||
.venv/Scripts/python.exe cli.py chat --name "补充资料" --working-dir proposal_v3
|
||||
|
||||
# 恢复最近一个 task(resume 时 --name / --working-dir 都忽略)
|
||||
.venv/Scripts/python.exe cli.py chat --resume last
|
||||
|
||||
# 恢复指定 task(UUID 完整或 ≥8 字符前缀)
|
||||
.venv/Scripts/python.exe cli.py chat --resume 76c6bd25
|
||||
|
||||
# 切模型
|
||||
.venv/Scripts/python.exe cli.py chat --name x --model deepseek_v4.pro
|
||||
```
|
||||
|
||||
REPL 内命令:`/exit /reset /new [<name>] /resume [last|<id>] /id /status /done /abandon /desc <文本> /export [<id>]`(`/new <name>` 用新任务名 + 沿用当前 working_dir;`/new` 无参 → 自动 gen `新任务_HH-MM-SS`)
|
||||
|
||||
### 列表 / 导出
|
||||
|
||||
```bash
|
||||
# 看最近 20 个 task
|
||||
.venv/Scripts/python.exe cli.py tasks
|
||||
|
||||
# 只看 active
|
||||
.venv/Scripts/python.exe cli.py tasks --status active --limit 50
|
||||
|
||||
# 导出某 task 的对话为 .docx(自动从 PG 找 task_dir 作为输出目录)
|
||||
.venv/Scripts/python.exe cli.py export 76c6bd25
|
||||
|
||||
# 导出最近的
|
||||
.venv/Scripts/python.exe cli.py export last -o /tmp/chat.docx
|
||||
```
|
||||
|
||||
### 能力探测 / DB 管理
|
||||
|
||||
```bash
|
||||
# 实测对账模型 yaml 声称的能力(费 token,有 API 开销)
|
||||
.venv/Scripts/python.exe cli.py probe --model deepseek_v4.flash
|
||||
|
||||
# DB migration
|
||||
.venv/Scripts/python.exe cli.py db upgrade head
|
||||
.venv/Scripts/python.exe cli.py db downgrade -1
|
||||
.venv/Scripts/python.exe cli.py db current
|
||||
```
|
||||
|
||||
### Web API(§7 D + D' 过渡 auth)
|
||||
|
||||
```bash
|
||||
# 默认 127.0.0.1:8765 启;dev SPA 在 /,Swagger UI 在 /docs
|
||||
.venv/Scripts/python.exe cli.py web
|
||||
|
||||
# 自定义端口 / 监听 0.0.0.0(慎用,部署形态走反代不直暴)
|
||||
.venv/Scripts/python.exe cli.py web --port 9000
|
||||
|
||||
# dev:文件改动自动重启(uvicorn 工厂模式 reload)
|
||||
.venv/Scripts/python.exe cli.py web --reload
|
||||
```
|
||||
|
||||
**Auth**:所有 `/v1/tasks*` 需 `Authorization: Bearer <jwt>`;先走 `/v1/auth/login` 拿 token:
|
||||
|
||||
```bash
|
||||
# 登录 → 拿 token(本地默 user_id = sentinel 全 0)
|
||||
curl --noproxy '*' -s -X POST http://127.0.0.1:8765/v1/auth/login \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"user_id":"00000000-0000-0000-0000-000000000000","platform_key":"<value of $PLATFORM_KEY>"}'
|
||||
# → {"token":"eyJ...","expires_at":"...","user_id":"...","ttl_seconds":604800}
|
||||
|
||||
# 用 token 调 /v1/*
|
||||
TOKEN="eyJ..."
|
||||
curl --noproxy '*' -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8765/v1/tasks
|
||||
```
|
||||
|
||||
**dev SPA**:打开 `http://127.0.0.1:8765/`(自动 302 → `/static/dev.html`),login 表单填 user_id(默 sentinel)+ PLATFORM_KEY 进入 3 栏(task 列表 / chat / files)。仅给开发自验,不发布给真用户。
|
||||
|
||||
**路由表**(全 JSON,CORS `allow_origins=["*"]`;详细 schema 见 `http://127.0.0.1:8765/docs`):
|
||||
|
||||
| 方法 + 路径 | 用途 | Auth |
|
||||
|---|---|---|
|
||||
| `GET /healthz` | `{"status":"ok"}` 健康检查 | 豁免 |
|
||||
| `GET /` | 302 → `/static/dev.html` dev SPA | 豁免 |
|
||||
| `GET /docs` `/openapi.json` | Swagger UI / OpenAPI schema | 豁免 |
|
||||
| `GET /static/*` | dev.html 等静态文件 | 豁免 |
|
||||
| `POST /v1/auth/login` | body `{user_id, platform_key}` → `{token,expires_at,user_id,ttl_seconds}` | 豁免 |
|
||||
| `POST /v1/tasks` | 创建 task,body `{name(req), working_dir?, description?, skill?}` | 必填 |
|
||||
| `GET /v1/tasks?page=&page_size=&status=&skill=&working_dir=&q=&ordering=` | 列任务,默认 `-created_at`;响应 `{page, page_size, count, results}`;`page` 1-based,`page_size` 1–100;`working_dir` 末段名;`q` ILIKE name+desc;`ordering` DRF 风格逗号分隔 `-field` 倒序,allowlist created_at/updated_at/name/status | 必填 |
|
||||
| `GET /v1/tasks/{id}` | 单 task meta + `n_messages`;跨 user → 404 | 必填 |
|
||||
| `PATCH /v1/tasks/{id}` | `{status?,description?,name?,skill?}` 部分更新;active 走 CLI 切回 | 必填 |
|
||||
| `DELETE /v1/tasks/{id}` | 硬删 DB 行(messages CASCADE),FS working_dir 保留 | 必填 |
|
||||
| `GET /v1/folders` | 列当前 user 工作目录 + n_tasks + last_used(供创建 task 自动补全用) | 必填 |
|
||||
| `GET /v1/tasks/{id}/messages` | LiteLLM payload 透传 | 必填 |
|
||||
| `POST /v1/tasks/{id}/messages` | `{content}` 发消息;返 `{run_id, events_url}` | 必填 |
|
||||
| `GET /v1/tasks/{id}/runs/{rid}/events` | SSE 流(`event: <type>` + `data: <json>`) | 必填 |
|
||||
| `GET /v1/tasks/{id}/files?path=` | 列子目录条目 + 面包屑 | 必填 |
|
||||
| `GET /v1/tasks/{id}/files/download?path=` | 下单文件 | 必填 |
|
||||
| `POST /v1/tasks/{id}/files/upload` | multipart 上传,`path` 走 form | 必填 |
|
||||
| `POST /v1/tasks/{id}/files/delete` | body `{path}`;文件或空目录 | 必填 |
|
||||
| `GET /v1/tasks/{id}/export` | 对话导出 .docx | 必填 |
|
||||
|
||||
**SSE 事件 schema**(每帧 `event: <type>` + `data: <JSON>`):`run_start{}` → `llm_start{}` → `text{content}` / `tool_call{name,args,args_preview}` / `tool_result{name,preview,truncated}` → `llm_end{prompt_tokens,completion_tokens}` → `done{}`;异常路径走 `error{msg}`。30s 无 event 服务端发 `: ping` 注释心跳。SSE 经 nginx 反代记得关 buffering(响应头已带 `X-Accel-Buffering: no` 默认起效)。
|
||||
|
||||
**SSE 客户端注意**:浏览器原生 `EventSource` 不支持自定义 header,无法塞 Bearer token。要么走 `fetch + ReadableStream` 自解 SSE 帧(dev.html 走的就是这条),要么后端日后加 `?token=...` query 路径(目前不支持,避免 token 进 access log)。
|
||||
|
||||
> 原 Phase G Jinja2 + HTMX Web UI 路线撤(DESIGN §7.9 取舍说明)—— UI 改 platform 端实现,本仓库只维护 API + 一个 dev SPA。`cli.py web` 跑的是 API + Swagger + dev.html。
|
||||
|
||||
---
|
||||
|
||||
## 故障兜底
|
||||
|
||||
| 现象 | 原因 / 处理 |
|
||||
|---|---|
|
||||
| `ZCBOT_DB_URL is not set` | `.env` 没写 / litellm 链路没触发。直跑脚本时 `import litellm`,或 `export ZCBOT_DB_URL=...` |
|
||||
| `ModuleNotFoundError: litellm` | 用了全局 `python`,改 `.venv/Scripts/python.exe ...` |
|
||||
| Windows 控制台 emoji 崩 | Python stdout 是 GBK,emoji 不能直 print。用 `[OK]` / `[ng]` 等 ASCII 标签(见 memory) |
|
||||
| `db upgrade` 报 `column already exists` | DB 已被改过,先 `db current` 确认 revision,必要时手 ALTER 或 `db downgrade base` 重来 |
|
||||
| Resume 找不到 task | `cli.py tasks` 看 task_id 是否在;前缀冲突报 ambiguous 时给完整 UUID |
|
||||
| `--working-dir` 指定后 `/exit` 没清目录 | 设计如此 —— 工作目录绝不 rmtree(同 working_dir 多 task 共享);DB 行该删还是删。要彻底删手动 `rm -rf <dir>` |
|
||||
| Export 报 "无可导出内容" | task 没 messages(只 system 不算);先在 REPL 发条消息再 export |
|
||||
| `NoSubtaskError: working_dir ... 与已有 task ... 前缀嵌套` | §7.4 no-subtask:同 user 不允许 working_dir 嵌套(child 或 parent)。**同项目多对话**请传**完全相同**的 `--working-dir`;否则改路径成 sibling(平级) |
|
||||
| `cli.py web` 启动后 curl 连不上 | 检查 proxy(`HTTP_PROXY` / `HTTPS_PROXY`):本地服务在 127.0.0.1,系统 proxy 拦截会 502。临时 `unset HTTP_PROXY HTTPS_PROXY` 或加 `curl --noproxy '*'`。验通:`curl --noproxy '*' http://127.0.0.1:8765/healthz` → `{"status":"ok"}` |
|
||||
| SSE 卡住不流(经 nginx) | 反代要关 buffering — 后端响应头已带 `X-Accel-Buffering: no`,nginx ≥ 1.5.6 默认认。仍卡看 nginx 配 `proxy_buffering off; proxy_read_timeout 3600s;` |
|
||||
| platform 端 CORS preflight 失败 | 本地 dev `allow_origins=["*"]` 应该没事;部署后看是否按 platform 域名收紧过头(`access-control-allow-origin` 响应头要含 platform 域名 或 `*`)|
|
||||
| `UniqueViolation idx already exists` from messages | 同 task 并发 POST messages,idx 冲突。**已知 TODO**:加 task 级 `SELECT ... FOR UPDATE` 或 advisory lock(留到 D' / 真发布前) |
|
||||
| `cli.py web` 启动报 `PLATFORM_KEY env not set` / `JWT_SECRET env not set` | D' 过渡 auth 强制双 env 必填。生成 `python -c "import secrets;print(secrets.token_urlsafe(48))"` 各填一,写进 `.env` 重起 |
|
||||
| `/v1/*` 全返 401 `missing Authorization: Bearer` | 没拿 token 或没带 header。先 `POST /v1/auth/login` 拿 token,curl 加 `-H "Authorization: Bearer $TOKEN"` |
|
||||
| `/v1/*` 返 401 `token expired` | JWT 默 7d TTL 到期,重 login。要更长改 `ZCBOT_JWT_TTL_SECONDS` env |
|
||||
| dev.html SSE 收不到流(消息发出去但 UI 没动) | EventSource 不支持 header,dev.html 走 `fetch + ReadableStream`。看浏览器 devtools Network,POST /messages 是否 202 + Network 看 events_url GET 是否 200 + Content-Type 是 text/event-stream;若 401,token 过期了 — logout 重 login |
|
||||
| dev.html 显示 "load failed" 且立刻回登录页 | token 过期或 JWT_SECRET 服务端变了,localStorage 旧 token 失效。已自动跳登录页,重新填 platform_key 即可 |
|
||||
|
||||
---
|
||||
|
||||
## 关键路径与文件
|
||||
|
||||
- **入口**:`cli.py`(REPL + `chat / tasks / probe / db / web` 子命令)→ `main.py::build_agent`(装配)
|
||||
- **核心**:`core/loop.py`(ReAct)/ `core/session.py`(PG messages)/ `core/task.py`(PG tasks)/ `core/llm.py`(LiteLLM 封装)
|
||||
- **工具**:`tools/{fs,shell,run_python,skill_tool}.py`
|
||||
- **存储**:`core/storage/{engine,models,utils}.py`(SQLAlchemy 2.x ORM)+ `db/migrations/`(alembic)
|
||||
- **Web**:`web/{app.py, auth.py, broker.py, sinks.py}`(FastAPI + /v1 JSON API + SSE + PLATFORM_KEY→JWT)+ `web/static/dev.html`(dev SPA,单文件 vanilla JS)
|
||||
- **配置**:`config/agent.yaml`(全局)/ `config/models/*.yaml`(模型档案,§3.2 Model Profile)
|
||||
- **Skill**:`skills/{coding,ppt,proposal}/SKILL.md`(渐进披露,§3.5)
|
||||
- **Workspace**(per-user 子树,本地 CLI sentinel = `00000000-0000-0000-0000-000000000000`,web/JWT 用 sub):
|
||||
- `workspace/users/<user_id>/.memory/{core.md, extended/}` —— 跨 task 记忆,FS 永久,dotfile 隔离
|
||||
- `workspace/users/<user_id>/<working_dir>/` —— 工作目录,用户起的目录名(`cli chat --working-dir` 或留空 fallback `--name` / API `POST /v1/tasks {working_dir?}`),同 working_dir 多 task 共享
|
||||
|
||||
---
|
||||
|
||||
## 维护约定
|
||||
|
||||
- **每改一个对外行为(CLI 选项 / REPL 命令 / env 变量 / 文件布局)→ 同步更新本文档**。bug 修不动这个,只动 PROGRESS。
|
||||
- 故障兜底表新增条目:用过一次的真实坑,写一行(现象 + 处理),不预测。
|
||||
- 跟 DESIGN/PROGRESS 的边界:DESIGN 写"为什么",PROGRESS 写"做到哪",RUN 写"怎么跑"。
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# Alembic config. DB URL is read from ZCBOT_DB_URL env var in env.py
|
||||
# (NOT hardcoded here, so SaaS/local share the same alembic.ini).
|
||||
|
||||
[alembic]
|
||||
script_location = db/migrations
|
||||
prepend_sys_path = .
|
||||
version_path_separator = os
|
||||
# timestamp + rev + slug, sortable by name
|
||||
file_template = %%(year)d%%(month).2d%%(day).2d_%%(hour).2d%%(minute).2d_%%(rev)s_%%(slug)s
|
||||
|
||||
[loggers]
|
||||
keys = root,sqlalchemy,alembic
|
||||
|
||||
[handlers]
|
||||
keys = console
|
||||
|
||||
[formatters]
|
||||
keys = generic
|
||||
|
||||
[logger_root]
|
||||
level = WARN
|
||||
handlers = console
|
||||
qualname =
|
||||
|
||||
[logger_sqlalchemy]
|
||||
level = WARN
|
||||
handlers =
|
||||
qualname = sqlalchemy.engine
|
||||
|
||||
[logger_alembic]
|
||||
level = INFO
|
||||
handlers =
|
||||
qualname = alembic
|
||||
|
||||
[handler_console]
|
||||
class = StreamHandler
|
||||
args = (sys.stderr,)
|
||||
level = NOTSET
|
||||
formatter = generic
|
||||
|
||||
[formatter_generic]
|
||||
format = %(levelname)-5.5s [%(name)s] %(message)s
|
||||
datefmt = %H:%M:%S
|
||||
327
cli.py
|
|
@ -4,31 +4,33 @@
|
|||
python cli.py chat # 新建一个 task
|
||||
python cli.py chat --mode coding --desc "修一处 bug" # 带元数据建任务
|
||||
python cli.py chat --resume last # 恢复最近一个 task
|
||||
python cli.py chat --resume 20260506_141523 # 显式 task_id
|
||||
python cli.py chat --resume <uuid-or-prefix> # 显式 task_id(前缀 ≥8 字符)
|
||||
python cli.py chat --model deepseek_v4.pro
|
||||
python cli.py tasks # 列出 task
|
||||
python cli.py probe # 实测对账 yaml 声称的能力
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from rich.prompt import Prompt
|
||||
from rich.table import Table
|
||||
|
||||
from core.task import TaskState
|
||||
from core.storage import SENTINEL_USER_ID
|
||||
from core.ui import make_console
|
||||
from main import (
|
||||
ROOT,
|
||||
InvalidTaskName,
|
||||
_resolve_uuid_or_prefix,
|
||||
build_agent,
|
||||
load_config,
|
||||
resolve_workspace,
|
||||
sync_task_tokens,
|
||||
tasks_dir,
|
||||
user_root,
|
||||
validate_task_name,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -37,70 +39,163 @@ def cli() -> None:
|
|||
"""zcbot - 个人任务 agent"""
|
||||
|
||||
|
||||
def _cleanup_if_empty(task_dir, session, console=None) -> bool:
|
||||
"""切走前清理 task_dir。三条都满足才删:
|
||||
1) session 没有 user 消息
|
||||
2) task_dir 在磁盘上(懒创建后,没说话就没目录,直接 no-op)
|
||||
3) 目录里只剩 messages.json(state.json 存在 = `/done /abandon /desc` 留下的显式痕迹,要保)
|
||||
原子写留下的 `*.tmp` 孤儿不算痕迹,放过。
|
||||
"""
|
||||
if any(m.get("role") == "user" for m in session.messages):
|
||||
return False
|
||||
@cli.group()
|
||||
def db() -> None:
|
||||
"""数据库管理 (alembic upgrade/downgrade/current)。需先 export ZCBOT_DB_URL。"""
|
||||
|
||||
|
||||
def _alembic_cfg():
|
||||
from alembic.config import Config
|
||||
return Config(str(ROOT / "alembic.ini"))
|
||||
|
||||
|
||||
def _run_alembic(fn, *args) -> None:
|
||||
"""统一包一层友好出错(ZCBOT_DB_URL 未设置 / 连不上 → 简洁报错,不打 traceback)。"""
|
||||
try:
|
||||
entries = list(task_dir.iterdir())
|
||||
except FileNotFoundError:
|
||||
fn(_alembic_cfg(), *args)
|
||||
except RuntimeError as e:
|
||||
click.echo(f"[err] {e}", err=True)
|
||||
sys.exit(2)
|
||||
except Exception as e:
|
||||
click.echo(f"[err] {type(e).__name__}: {e}", err=True)
|
||||
sys.exit(3)
|
||||
|
||||
|
||||
@db.command("upgrade")
|
||||
@click.argument("revision", default="head")
|
||||
def db_upgrade(revision: str) -> None:
|
||||
"""alembic upgrade <revision> (default head)."""
|
||||
from alembic import command
|
||||
_run_alembic(command.upgrade, revision)
|
||||
|
||||
|
||||
@db.command("downgrade")
|
||||
@click.argument("revision")
|
||||
def db_downgrade(revision: str) -> None:
|
||||
"""alembic downgrade <revision> (use -1 for one step, base for all)."""
|
||||
from alembic import command
|
||||
_run_alembic(command.downgrade, revision)
|
||||
|
||||
|
||||
@db.command("current")
|
||||
def db_current() -> None:
|
||||
"""alembic current -- show currently applied revision."""
|
||||
from alembic import command
|
||||
_run_alembic(command.current)
|
||||
|
||||
|
||||
def _cleanup_if_empty(working_dir, session, workspace_dir, console=None) -> bool:
|
||||
"""切走前清理空 task。
|
||||
|
||||
DB 行无条件删除(若存在且 session 内存无 user 消息)。
|
||||
FS **绝不 rmtree** —— working_dir 是用户起的项目目录名,同 working_dir 跨 task 复用,
|
||||
可能里面已有别的产物;空 task 只清 DB 行。
|
||||
"""
|
||||
_ = workspace_dir # 不再用,签名保留向后兼容
|
||||
_ = working_dir # FS 不动,只清 DB
|
||||
if session.n_user_msgs() > 0:
|
||||
return False
|
||||
if any(p.is_dir() for p in entries):
|
||||
return False
|
||||
meaningful = {
|
||||
p.name for p in entries
|
||||
if p.is_file() and not p.name.endswith(".tmp")
|
||||
}
|
||||
if meaningful - {"messages.json"}:
|
||||
return False
|
||||
shutil.rmtree(task_dir, ignore_errors=True)
|
||||
|
||||
_delete_task_db_row(session.task_id)
|
||||
if console is not None:
|
||||
console.print(f"[muted]清理空 task {task_dir.name}[/muted]")
|
||||
console.print(
|
||||
f"[muted]cleaned empty task {str(session.task_id)[:8]} (kept FS dir)[/muted]"
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
def _delete_task_db_row(task_id) -> None:
|
||||
"""删 PG tasks 行(messages 走 CASCADE)。task_id 可能从未入库,DELETE 0 行无副作用。"""
|
||||
from sqlalchemy import delete
|
||||
from core.storage import session_scope
|
||||
from core.storage.models import Task
|
||||
with session_scope() as s:
|
||||
s.execute(delete(Task).where(Task.task_id == task_id))
|
||||
|
||||
|
||||
def _task_has_messages(task_id_str: str) -> bool:
|
||||
"""PG 里该 task_id 有至少一条 message。task_id 字符串(UUID 完整形式)。"""
|
||||
from uuid import UUID
|
||||
from sqlalchemy import select
|
||||
from core.storage import session_scope
|
||||
from core.storage.models import Message
|
||||
try:
|
||||
tid = UUID(task_id_str)
|
||||
except ValueError:
|
||||
return False
|
||||
with session_scope() as s:
|
||||
row = s.execute(
|
||||
select(Message.message_id).where(Message.task_id == tid).limit(1)
|
||||
).scalar_one_or_none()
|
||||
return row is not None
|
||||
|
||||
|
||||
def _list_task_rows(workspace_dir, limit=20, status=None):
|
||||
"""返回 [(mtime, task_id, status, mode, model, tokens, n_msgs, desc), ...] mtime 降序。"""
|
||||
tdir = tasks_dir(workspace_dir)
|
||||
"""返回 [(updated_at, task_id_str, status, name, skill, model, tokens, n_msgs, desc), ...] 时间降序。
|
||||
|
||||
Step 3 后:全字段从 PG tasks 表读,messages 数从 PG 数;workspace_dir 仅用于
|
||||
保持签名向后兼容(不再读 state.json)。status 过滤走 SQL WHERE。
|
||||
"""
|
||||
from sqlalchemy import func, select
|
||||
from core.storage import session_scope
|
||||
from core.storage.models import Message, Task
|
||||
|
||||
_ = workspace_dir # 签名占位,Step 3 后已不需要
|
||||
with session_scope() as s:
|
||||
q = select(
|
||||
Task.task_id, Task.updated_at, Task.status, Task.name, Task.skill,
|
||||
Task.model, Task.model_profile, Task.tokens_prompt,
|
||||
Task.tokens_completion, Task.description,
|
||||
).order_by(Task.updated_at.desc())
|
||||
if status:
|
||||
q = q.where(Task.status == status)
|
||||
rows_db = s.execute(q.limit(limit)).all()
|
||||
msg_counts = dict(s.execute(
|
||||
select(Message.task_id, func.count()).group_by(Message.task_id)
|
||||
).all())
|
||||
|
||||
rows = []
|
||||
for d in tdir.iterdir():
|
||||
if not d.is_dir():
|
||||
continue
|
||||
msg_path = d / "messages.json"
|
||||
if not msg_path.exists():
|
||||
continue
|
||||
st = TaskState.load(d)
|
||||
if st is None:
|
||||
continue
|
||||
if status and st.status != status:
|
||||
continue
|
||||
try:
|
||||
data = json.loads(msg_path.read_text(encoding="utf-8"))
|
||||
n = len(data.get("messages", []))
|
||||
except Exception:
|
||||
n = -1
|
||||
for tid, updated_at, st_, nm, sk, mdl, prof, tp, tc, desc in rows_db:
|
||||
n = msg_counts.get(tid, 0)
|
||||
rows.append((
|
||||
msg_path.stat().st_mtime, st.task_id, st.status, st.mode,
|
||||
st.model_profile or st.model, st.tokens_total, n, st.description,
|
||||
updated_at, str(tid), st_, nm, sk,
|
||||
prof or mdl, (tp or 0) + (tc or 0), n, desc,
|
||||
))
|
||||
rows.sort(reverse=True)
|
||||
return rows[:limit]
|
||||
return rows
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.option("--model", default=None, help="模型档案,如 deepseek_v4.flash 或 deepseek_v4.pro")
|
||||
@click.option("--workspace", default=None, help="工作目录(存 tasks/ 和 sessions/)")
|
||||
@click.option("--workspace", default=None, help="工作目录根(默认 ./workspace)")
|
||||
@click.option("--resume", default=None, help="恢复 task: 'last' 或 task_id")
|
||||
@click.option("--mode", default="", help="任务模式标签(coding/ppt/proposal/...自由形式)")
|
||||
@click.option("--skill", default="", help="智能体类型标签(coding/ppt/proposal/...自由形式,对齐 skills/)")
|
||||
@click.option("--desc", default="", help="一句话任务描述,便于 tasks 列表识别")
|
||||
def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
||||
"""启动交互式 REPL。每次启动默认开新 task,用 --resume 接老的。"""
|
||||
@click.option("--name", default=None,
|
||||
help="任务名(必填,DB 存,UI 显示用)。resume 时忽略。")
|
||||
@click.option("--working-dir", default=None,
|
||||
help="工作目录名(简单名,不含 / \\ .. 也不能以 . 起头);留空 → 用 --name。"
|
||||
"工作目录落 workspace/users/<sentinel>/<working_dir>/,同名多 task 共享。"
|
||||
"resume 时忽略。")
|
||||
def chat(model: str, workspace: str, resume: str, skill: str, desc: str,
|
||||
name: str, working_dir: str) -> None:
|
||||
"""启动交互式 REPL。新建必填 `--name`,可选 `--working-dir`;用 --resume 接老的。"""
|
||||
console = make_console()
|
||||
ws_dir = resolve_workspace(workspace)
|
||||
if not resume:
|
||||
if not name:
|
||||
console.print("[err]新建 task 需要 --name <任务名>[/err]")
|
||||
sys.exit(1)
|
||||
try:
|
||||
name = validate_task_name(name)
|
||||
except InvalidTaskName as e:
|
||||
console.print(f"[err]name 不合法:[/err] {e}")
|
||||
sys.exit(1)
|
||||
if working_dir:
|
||||
try:
|
||||
working_dir = validate_task_name(working_dir)
|
||||
except InvalidTaskName as e:
|
||||
console.print(f"[err]working_dir 不合法:[/err] {e}")
|
||||
sys.exit(1)
|
||||
try:
|
||||
agent, session, sid, task_state, task_dir = build_agent(
|
||||
model_name=model,
|
||||
|
|
@ -108,8 +203,10 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
console=console,
|
||||
session_id=resume,
|
||||
resume=bool(resume),
|
||||
mode=mode,
|
||||
skill=skill,
|
||||
description=desc,
|
||||
name=name if not resume else None,
|
||||
working_dir=working_dir if not resume else None,
|
||||
)
|
||||
except Exception as e:
|
||||
console.print(f"[err]启动失败:[/err] {type(e).__name__}: {e}")
|
||||
|
|
@ -117,15 +214,16 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
|
||||
if resume:
|
||||
console.print(
|
||||
f"[ok]恢复 task[/ok] [bold]{sid}[/bold] ({len(session.messages)} 条消息) "
|
||||
f"[ok]恢复 task[/ok] [bold]{sid[:8]}[/bold] ({len(session.messages)} 条消息) "
|
||||
f"name: [accent]{task_state.name}[/accent] "
|
||||
f"model: [accent]{agent.caps.model_id}[/accent]"
|
||||
)
|
||||
else:
|
||||
meta_tail = ""
|
||||
if task_state.mode or task_state.description:
|
||||
meta_tail = f" mode={task_state.mode!r} desc={task_state.description!r}"
|
||||
if task_state.skill or task_state.description:
|
||||
meta_tail = f" skill={task_state.skill!r} desc={task_state.description!r}"
|
||||
console.print(
|
||||
f"[ok]新 task[/ok] [bold]{sid}[/bold] "
|
||||
f"[ok]新 task[/ok] [bold]{sid[:8]}[/bold] name=[accent]{task_state.name}[/accent] "
|
||||
f"model: [accent]{agent.caps.model_id}[/accent]{meta_tail}"
|
||||
)
|
||||
console.print(
|
||||
|
|
@ -140,32 +238,46 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
user_input = Prompt.ask("[user]you[/user]", console=console)
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
console.print("\n[muted]bye[/muted]")
|
||||
_cleanup_if_empty(task_dir, session, console)
|
||||
_cleanup_if_empty(task_dir, session, ws_dir, console)
|
||||
break
|
||||
|
||||
cmd = user_input.strip()
|
||||
if cmd in ("/exit", "/quit"):
|
||||
_cleanup_if_empty(task_dir, session, console)
|
||||
_cleanup_if_empty(task_dir, session, ws_dir, console)
|
||||
break
|
||||
if cmd == "/reset":
|
||||
session.reset(keep_system=True)
|
||||
console.print("[info]当前 task 对话已重置(保留 system 和 state)[/info]")
|
||||
continue
|
||||
if cmd == "/new":
|
||||
_cleanup_if_empty(task_dir, session, console)
|
||||
if cmd.startswith("/new"):
|
||||
_cleanup_if_empty(task_dir, session, ws_dir, console)
|
||||
# `/new <name>` → 新 task,name = 参数;`/new` 无参 → 自动生成名(时间戳)
|
||||
arg = cmd[len("/new"):].strip()
|
||||
new_name = arg or f"新任务_{datetime.now().strftime('%H-%M-%S')}"
|
||||
try:
|
||||
new_name = validate_task_name(new_name)
|
||||
except InvalidTaskName as e:
|
||||
console.print(f"[err]name 不合法:[/err] {e}")
|
||||
continue
|
||||
# 沿用当前 task 的 working_dir(同项目多对话);取上层 task_dir 末段作为 dir name
|
||||
current_wd = task_dir.name # 例如 `水泥申报` 或 `proposal_v3`
|
||||
try:
|
||||
agent, session, sid, task_state, task_dir = build_agent(
|
||||
model_name=model, workspace=workspace, console=console,
|
||||
mode=mode, description=desc,
|
||||
skill=skill, description=desc,
|
||||
name=new_name, working_dir=current_wd,
|
||||
)
|
||||
except Exception as e:
|
||||
console.print(f"[err]新建失败:[/err] {type(e).__name__}: {e}")
|
||||
continue
|
||||
console.print(f"[ok]新 task[/ok] [bold]{sid}[/bold]")
|
||||
name = new_name # 更新当前 name
|
||||
console.print(
|
||||
f"[ok]新 task[/ok] [bold]{sid[:8]}[/bold] name=[accent]{name}[/accent] "
|
||||
f"working_dir=[accent]{current_wd}[/accent]"
|
||||
)
|
||||
continue
|
||||
if cmd.startswith("/resume"):
|
||||
arg = cmd[len("/resume"):].strip()
|
||||
ws_dir = resolve_workspace(workspace)
|
||||
target_id = None
|
||||
if arg == "last":
|
||||
rs = _list_task_rows(ws_dir, limit=1)
|
||||
|
|
@ -184,14 +296,15 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
tbl.add_column("#", style="bold")
|
||||
tbl.add_column("task id")
|
||||
tbl.add_column("status")
|
||||
tbl.add_column("mode")
|
||||
tbl.add_column("name")
|
||||
tbl.add_column("skill")
|
||||
tbl.add_column("msgs", justify="right")
|
||||
tbl.add_column("desc")
|
||||
sc = {"active": "status.active", "completed": "status.completed", "abandoned": "status.abandoned"}
|
||||
for i, (_, tid, st, md, _mdl, _tok, n, dsc) in enumerate(rs, 1):
|
||||
for i, (_, tid, st, nm, sk, _mdl, _tok, n, dsc) in enumerate(rs, 1):
|
||||
c = sc.get(st, "info")
|
||||
d_show = dsc if len(dsc) <= 50 else dsc[:47] + "..."
|
||||
tbl.add_row(str(i), tid, f"[{c}]{st}[/{c}]", md, str(n), d_show)
|
||||
tbl.add_row(str(i), tid[:8], f"[{c}]{st}[/{c}]", nm, sk, str(n), d_show)
|
||||
console.print(tbl)
|
||||
try:
|
||||
sel = Prompt.ask("[user]选编号或输入 task_id (回车取消)[/user]", console=console, default="")
|
||||
|
|
@ -212,7 +325,7 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
if target_id == sid:
|
||||
console.print(f"[info]已是当前 task: {sid}[/info]")
|
||||
continue
|
||||
_cleanup_if_empty(task_dir, session, console)
|
||||
_cleanup_if_empty(task_dir, session, ws_dir, console)
|
||||
try:
|
||||
agent, session, sid, task_state, task_dir = build_agent(
|
||||
model_name=model, workspace=workspace, console=console,
|
||||
|
|
@ -222,7 +335,7 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
console.print(f"[err]恢复失败:[/err] {type(e).__name__}: {e}")
|
||||
continue
|
||||
console.print(
|
||||
f"[ok]切到 task[/ok] [bold]{sid}[/bold] ({len(session.messages)} 条消息) "
|
||||
f"[ok]切到 task[/ok] [bold]{sid[:8]}[/bold] ({len(session.messages)} 条消息) "
|
||||
f"model: [accent]{agent.caps.model_id}[/accent]"
|
||||
)
|
||||
continue
|
||||
|
|
@ -233,8 +346,10 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
continue
|
||||
if cmd == "/status":
|
||||
console.print(
|
||||
f"[info]task {task_state.task_id} status={task_state.status} "
|
||||
f"mode={task_state.mode!r} desc={task_state.description!r}\n"
|
||||
f"[info]task {task_state.task_id} name={task_state.name!r} "
|
||||
f"status={task_state.status} skill={task_state.skill!r} "
|
||||
f"desc={task_state.description!r}\n"
|
||||
f" working_dir={task_state.working_dir}\n"
|
||||
f" model={task_state.model} tokens={task_state.tokens_total} "
|
||||
f"(p={task_state.tokens_prompt}/c={task_state.tokens_completion}) "
|
||||
f"created={task_state.created_at} updated={task_state.updated_at}[/info]"
|
||||
|
|
@ -242,40 +357,47 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
continue
|
||||
if cmd == "/done":
|
||||
task_state.status = "completed"
|
||||
task_state.save(task_dir)
|
||||
task_state.save()
|
||||
console.print(f"[ok]task {sid} marked completed[/ok]")
|
||||
break
|
||||
if cmd == "/abandon":
|
||||
task_state.status = "abandoned"
|
||||
task_state.save(task_dir)
|
||||
task_state.save()
|
||||
console.print(f"[warn]task {sid} marked abandoned[/warn]")
|
||||
break
|
||||
if cmd.startswith("/desc"):
|
||||
new_desc = cmd[len("/desc"):].strip()
|
||||
task_state.description = new_desc
|
||||
task_state.save(task_dir)
|
||||
task_state.save()
|
||||
console.print(f"[info]description set: {new_desc!r}[/info]")
|
||||
continue
|
||||
if cmd.startswith("/export"):
|
||||
arg = cmd[len("/export"):].strip()
|
||||
target_dir = task_dir
|
||||
from uuid import UUID
|
||||
if arg:
|
||||
ws_dir = resolve_workspace(workspace)
|
||||
if arg == "last":
|
||||
rs = _list_task_rows(ws_dir, limit=1)
|
||||
if not rs:
|
||||
console.print("[warn]没有 task 可导出[/warn]")
|
||||
continue
|
||||
arg = rs[0][1]
|
||||
target_dir = tasks_dir(ws_dir) / arg
|
||||
if not (target_dir / "messages.json").exists():
|
||||
try:
|
||||
target_tid = _resolve_uuid_or_prefix(arg)
|
||||
except Exception as e:
|
||||
console.print(f"[err]task_id 解析失败:[/err] {type(e).__name__}: {e}")
|
||||
continue
|
||||
target_dir = None # 让 export_chat_to_docx 从 PG 读 task_dir
|
||||
else:
|
||||
target_tid = UUID(sid)
|
||||
target_dir = task_dir
|
||||
if not _task_has_messages(str(target_tid)):
|
||||
console.print(
|
||||
f"[warn]无可导出内容: {target_dir.name} 还没有消息[/warn]"
|
||||
f"[warn]无可导出内容: {str(target_tid)[:8]} 还没有消息[/warn]"
|
||||
)
|
||||
continue
|
||||
try:
|
||||
from core.export_docx import export_chat_to_docx
|
||||
out = export_chat_to_docx(target_dir)
|
||||
out = export_chat_to_docx(target_tid, target_dir)
|
||||
except Exception as e:
|
||||
console.print(f"[err]导出失败:[/err] {type(e).__name__}: {e}")
|
||||
continue
|
||||
|
|
@ -291,7 +413,7 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
except Exception as e:
|
||||
console.print(f"[err]运行错误:[/err] {type(e).__name__}: {e}")
|
||||
finally:
|
||||
sync_task_tokens(task_state, task_dir, agent.llm)
|
||||
sync_task_tokens(task_state, agent.llm)
|
||||
|
||||
|
||||
@cli.command()
|
||||
|
|
@ -299,27 +421,28 @@ def chat(model: str, workspace: str, resume: str, mode: str, desc: str) -> None:
|
|||
@click.option("--limit", default=20, help="显示最近 N 个")
|
||||
@click.option("--status", default=None, help="只看某状态: active / completed / abandoned")
|
||||
def tasks(workspace: str, limit: int, status: str) -> None:
|
||||
"""列出已有 task(新格式,workspace/tasks/<id>/state.json)。"""
|
||||
"""列出已有 task(从 PG tasks 表读,按 updated_at 降序)。"""
|
||||
cfg = load_config()
|
||||
ws = resolve_workspace(workspace, cfg)
|
||||
rows = _list_task_rows(ws, limit=limit, status=status)
|
||||
|
||||
if not rows:
|
||||
click.echo(f"(no tasks in {tasks_dir(ws)})")
|
||||
click.echo(f"(no tasks under {user_root(ws, SENTINEL_USER_ID)})")
|
||||
return
|
||||
tbl = Table(show_lines=False)
|
||||
tbl.add_column("task id", style="bold")
|
||||
tbl.add_column("status")
|
||||
tbl.add_column("mode")
|
||||
tbl.add_column("name")
|
||||
tbl.add_column("skill")
|
||||
tbl.add_column("model")
|
||||
tbl.add_column("msgs", justify="right")
|
||||
tbl.add_column("tokens", justify="right")
|
||||
tbl.add_column("desc")
|
||||
sc = {"active": "status.active", "completed": "status.completed", "abandoned": "status.abandoned"}
|
||||
for _, tid, st, mode, model, tok, n, desc in rows:
|
||||
for _, tid, st, nm, sk, model, tok, n, desc in rows:
|
||||
c = sc.get(st, "info")
|
||||
d_show = desc if len(desc) <= 50 else desc[:47] + "..."
|
||||
tbl.add_row(tid, f"[{c}]{st}[/{c}]", mode, model, str(n), str(tok), d_show)
|
||||
tbl.add_row(tid[:8], f"[{c}]{st}[/{c}]", nm, sk, model, str(n), str(tok), d_show)
|
||||
make_console().print(tbl)
|
||||
|
||||
|
||||
|
|
@ -352,15 +475,19 @@ def export(task_id: str, workspace: str, output: str, include_system: bool,
|
|||
sys.exit(1)
|
||||
task_id = rs[0][1]
|
||||
|
||||
td = tasks_dir(ws) / task_id
|
||||
if not (td / "messages.json").exists():
|
||||
console.print(f"[err]task 不存在或无 messages.json:[/err] {td}")
|
||||
try:
|
||||
tid = _resolve_uuid_or_prefix(task_id)
|
||||
except Exception as e:
|
||||
console.print(f"[err]task_id 解析失败:[/err] {type(e).__name__}: {e}")
|
||||
sys.exit(1)
|
||||
if not _task_has_messages(str(tid)):
|
||||
console.print(f"[err]task 不存在或无 messages:[/err] {tid}")
|
||||
sys.exit(1)
|
||||
|
||||
out = Path(output).resolve() if output else None
|
||||
try:
|
||||
path = export_chat_to_docx(
|
||||
td, out,
|
||||
tid, None, out,
|
||||
include_system=include_system,
|
||||
include_reasoning=not no_reasoning,
|
||||
tool_head=tool_head,
|
||||
|
|
@ -435,5 +562,25 @@ def probe(model: str, long_context: bool) -> None:
|
|||
console.print("\n[ok]全部能力声明与实测一致。[/ok]")
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.option("--host", default="127.0.0.1", show_default=True,
|
||||
help="监听地址。本地形态默认 127.0.0.1,不对外暴露")
|
||||
@click.option("--port", default=8765, show_default=True, type=int,
|
||||
help="监听端口")
|
||||
@click.option("--reload/--no-reload", default=False,
|
||||
help="dev:文件改动自动重启(uvicorn 工厂模式)")
|
||||
def web(host: str, port: int, reload: bool) -> None:
|
||||
"""启动 Web UI(§7 Phase G,本地形态 sentinel user 无 auth)。"""
|
||||
import uvicorn
|
||||
|
||||
if reload:
|
||||
# reload 模式需要 import string + factory,uvicorn 才能监听文件
|
||||
uvicorn.run("web.app:create_app", host=host, port=port,
|
||||
reload=True, factory=True, log_level="info")
|
||||
else:
|
||||
from web.app import create_app
|
||||
uvicorn.run(create_app(), host=host, port=port, log_level="info")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
"""把 task 的 messages.json 渲染为 .docx 对话稿。
|
||||
"""把 task 的 PG messages 表 + tasks 元数据 渲染为 .docx 对话稿。
|
||||
|
||||
布局:
|
||||
- 文档开头 meta 表(task_id / 模式 / 描述 / 模型 / 创建时间 / 消息数 / tokens / 导出时间)
|
||||
|
|
@ -8,8 +8,10 @@
|
|||
- tool_calls 把 function 名 + 参数 JSON 单列展示
|
||||
|
||||
调用入口:
|
||||
- 顶层函数 export_chat_to_docx(task_dir, out_path=None, ...)
|
||||
- 顶层函数 export_chat_to_docx(task_id, task_dir=None, out_path=None, ...)
|
||||
- CLI 子命令 `python cli.py export <task_id>` 与 REPL `/export [<task_id>]` 都走它
|
||||
|
||||
§7 B Step 3 后:meta 和 messages 都从 PG 读(state.json 已废除)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
@ -17,6 +19,9 @@ import json
|
|||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from uuid import UUID
|
||||
|
||||
from core.task import TaskState
|
||||
|
||||
from docx import Document
|
||||
from docx.enum.text import WD_ALIGN_PARAGRAPH
|
||||
|
|
@ -163,7 +168,7 @@ def _format_args(args_str: str) -> str:
|
|||
# ───────────────────────── Meta 区块 ─────────────────────────
|
||||
|
||||
def _add_meta_block(
|
||||
doc: Document, meta: dict, task_state: dict, n_msgs: int, source_path: Path
|
||||
doc: Document, meta: dict, task_state: dict, n_msgs: int, working_dir: Optional[Path]
|
||||
) -> None:
|
||||
p = doc.add_paragraph()
|
||||
p.alignment = WD_ALIGN_PARAGRAPH.LEFT
|
||||
|
|
@ -176,12 +181,12 @@ def _add_meta_block(
|
|||
run.font.bold = True
|
||||
_set_run_fonts(run, cn_font="黑体", en_font="Consolas")
|
||||
|
||||
name = task_state.get("name") or ""
|
||||
desc = task_state.get("description") or ""
|
||||
mode = task_state.get("mode") or ""
|
||||
skill = task_state.get("skill") or ""
|
||||
status = task_state.get("status") or ""
|
||||
model = meta.get("model") or task_state.get("model") or ""
|
||||
profile = meta.get("model_profile") or task_state.get("model_profile") or ""
|
||||
cwd = meta.get("cwd") or task_state.get("cwd") or ""
|
||||
created = meta.get("created_at") or task_state.get("created_at") or ""
|
||||
updated = task_state.get("updated_at") or ""
|
||||
tp = task_state.get("tokens_prompt", 0)
|
||||
|
|
@ -189,17 +194,17 @@ def _add_meta_block(
|
|||
|
||||
rows = [
|
||||
("Task ID", meta.get("id") or task_state.get("task_id") or "?"),
|
||||
("模式", mode),
|
||||
("任务名", name),
|
||||
("Skill", skill),
|
||||
("描述", desc),
|
||||
("状态", status),
|
||||
("模型", model),
|
||||
("Profile", profile),
|
||||
("CWD", cwd),
|
||||
("创建时间", created),
|
||||
("更新时间", updated),
|
||||
("消息数", str(n_msgs)),
|
||||
("Tokens", f"{tp} prompt / {tc} completion / {tp + tc} total"),
|
||||
("源文件", str(source_path)),
|
||||
("工作目录", str(working_dir) if working_dir else "(未绑)"),
|
||||
("导出时间", datetime.now().isoformat(timespec="seconds")),
|
||||
]
|
||||
|
||||
|
|
@ -312,7 +317,8 @@ def _render_message(
|
|||
# ───────────────────────── 顶层入口 ─────────────────────────
|
||||
|
||||
def export_chat_to_docx(
|
||||
task_dir: Path,
|
||||
task_id: UUID,
|
||||
working_dir: Optional[Path] = None,
|
||||
out_path: Optional[Path] = None,
|
||||
*,
|
||||
include_system: bool = False,
|
||||
|
|
@ -320,41 +326,48 @@ def export_chat_to_docx(
|
|||
tool_head: int = 1000,
|
||||
tool_tail: int = 500,
|
||||
) -> Path:
|
||||
"""渲染 task_dir 下的 messages.json 为 .docx,返回写入路径。
|
||||
"""渲染 task 对话为 .docx,返回写入路径。
|
||||
|
||||
out_path 缺省落到 task_dir/chat_<task_id>.docx。
|
||||
include_system 默认 False(system prompt 信息密度低,默认跳过)。
|
||||
include_reasoning 默认 True(模型思考过程,有观察价值)。
|
||||
tool 结果默认前 1000 + 后 500,中间省略。
|
||||
task_id 是主标识(从 PG 读 messages + 元数据)。
|
||||
working_dir 留空 → 用 PG tasks.working_dir(用户指定模式可能不在默认派生路径下);
|
||||
DB 也空 → 报错(无处放产物)。out_path 留空 → working_dir / chat_<uuid>.docx。
|
||||
"""
|
||||
msg_path = task_dir / "messages.json"
|
||||
if not msg_path.exists():
|
||||
raise FileNotFoundError(f"messages.json 不存在: {msg_path}")
|
||||
from dataclasses import asdict
|
||||
from sqlalchemy import select
|
||||
from core.storage import session_scope
|
||||
from core.storage.models import Message as MessageRow
|
||||
|
||||
data = json.loads(msg_path.read_text(encoding="utf-8"))
|
||||
if isinstance(data, list):
|
||||
meta = {}
|
||||
messages = data
|
||||
elif isinstance(data, dict):
|
||||
meta = data.get("meta") or {}
|
||||
messages = data.get("messages") or []
|
||||
else:
|
||||
raise ValueError(f"messages.json 格式不识别: {type(data).__name__}")
|
||||
with session_scope() as s:
|
||||
rows = s.execute(
|
||||
select(MessageRow).where(MessageRow.task_id == task_id).order_by(MessageRow.idx)
|
||||
).scalars().all()
|
||||
messages = [dict(r.payload) for r in rows]
|
||||
|
||||
state_path = task_dir / "state.json"
|
||||
task_state: dict = {}
|
||||
if state_path.exists():
|
||||
try:
|
||||
task_state = json.loads(state_path.read_text(encoding="utf-8")) or {}
|
||||
except Exception:
|
||||
task_state = {}
|
||||
st = TaskState.load(task_id)
|
||||
task_state: dict = asdict(st) if st is not None else {}
|
||||
|
||||
if working_dir is None:
|
||||
wd_str = task_state.get("working_dir", "")
|
||||
if wd_str:
|
||||
# wd_str 是 db 形态(相对 ROOT 或绝对),走 from_db_path 还原 absolute Path
|
||||
from core.paths import from_db_path
|
||||
working_dir = from_db_path(wd_str)
|
||||
# else: working_dir 留 None,只在 out_path 也 None 时报错(不能没地方落 .docx)
|
||||
|
||||
if out_path is None:
|
||||
tid = meta.get("id") or task_state.get("task_id") or task_dir.name
|
||||
out_path = task_dir / f"chat_{tid}.docx"
|
||||
if working_dir is None:
|
||||
raise ValueError(f"task {task_id} 无 working_dir 且未指定 out_path —— 无处放 .docx")
|
||||
out_path = working_dir / f"chat_{task_id}.docx"
|
||||
|
||||
meta = {
|
||||
"id": str(task_id),
|
||||
"model": task_state.get("model", ""),
|
||||
"model_profile": task_state.get("model_profile", ""),
|
||||
"created_at": task_state.get("created_at", ""),
|
||||
}
|
||||
|
||||
doc = _init_doc()
|
||||
_add_meta_block(doc, meta, task_state, len(messages), msg_path)
|
||||
_add_meta_block(doc, meta, task_state, len(messages), working_dir)
|
||||
doc.add_paragraph() # 与 meta 表保持一行间距
|
||||
|
||||
for msg in messages:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,11 @@ import os
|
|||
import time
|
||||
from typing import Any, List, Optional
|
||||
|
||||
import litellm
|
||||
# 跳过启动时从 GitHub 拉 model_prices 的网络请求,直接用 litellm 打包的本地副本。
|
||||
# 必须在 `import litellm` 之前设置,否则 get_model_cost_map() 已经跑过了。
|
||||
os.environ.setdefault("LITELLM_LOCAL_MODEL_COST_MAP", "True")
|
||||
|
||||
import litellm # noqa: E402
|
||||
from litellm.exceptions import (
|
||||
APIConnectionError,
|
||||
APIError,
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
"""双层记忆: `workspace/memory/`。
|
||||
"""双层记忆: `workspace/users/<user_id>/.memory/` (§3.7 / §7.4)。
|
||||
|
||||
core.md —— 注 system prompt,每次都看到。装稳定事实
|
||||
(用户偏好 / 常用命令 / 项目约定 / 模型 quirk 备忘等)
|
||||
|
|
@ -9,17 +9,21 @@
|
|||
core 一直挂在上下文里,token 成本固定 ⇒ 只放跨任务高频用的精炼内容
|
||||
extended 索引只占几行,内容按需付费 ⇒ 适合大量低频专题
|
||||
|
||||
memory 是 workspace 级别(不是 task 级别)。同一 workspace 的所有 task 共享。
|
||||
SaaS 化(§7)后会按 tenant 隔离 —— 接口不变,只换 storage backend。
|
||||
memory 是 per-user(同一 workspace 内按 user_id 隔离),同 user 的所有 task 共享。
|
||||
**dotfile `.memory/` 命名**:跟用户起的项目目录(同样落 `<uid>/` 下)区分,避免
|
||||
项目名取 `memory` 时撞名;`.` 起头也被 `validate_task_name` 拒,双向防呆。
|
||||
本地 CLI = SENTINEL user;web/JWT 用 sub。SaaS 化时 `<storage_root>` 替换
|
||||
`workspace`,布局不变(§7.0)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import List, Tuple
|
||||
from uuid import UUID
|
||||
|
||||
|
||||
def _memory_dir(workspace_dir: Path) -> Path:
|
||||
return workspace_dir / "memory"
|
||||
def _memory_dir(workspace_dir: Path, user_id: UUID) -> Path:
|
||||
return workspace_dir / "users" / str(user_id) / ".memory"
|
||||
|
||||
|
||||
def _read_first_title(p: Path) -> str:
|
||||
|
|
@ -36,8 +40,8 @@ def _read_first_title(p: Path) -> str:
|
|||
return p.stem
|
||||
|
||||
|
||||
def _load_core(workspace_dir: Path) -> str:
|
||||
p = _memory_dir(workspace_dir) / "core.md"
|
||||
def _load_core(workspace_dir: Path, user_id: UUID) -> str:
|
||||
p = _memory_dir(workspace_dir, user_id) / "core.md"
|
||||
if not p.is_file():
|
||||
return ""
|
||||
try:
|
||||
|
|
@ -46,9 +50,9 @@ def _load_core(workspace_dir: Path) -> str:
|
|||
return ""
|
||||
|
||||
|
||||
def _extended_index(workspace_dir: Path) -> List[Tuple[str, Path]]:
|
||||
def _extended_index(workspace_dir: Path, user_id: UUID) -> List[Tuple[str, Path]]:
|
||||
"""返回 [(title, abs_path), ...],按文件名排序。"""
|
||||
ext_dir = _memory_dir(workspace_dir) / "extended"
|
||||
ext_dir = _memory_dir(workspace_dir, user_id) / "extended"
|
||||
if not ext_dir.is_dir():
|
||||
return []
|
||||
items: List[Tuple[str, Path]] = []
|
||||
|
|
@ -58,14 +62,14 @@ def _extended_index(workspace_dir: Path) -> List[Tuple[str, Path]]:
|
|||
return items
|
||||
|
||||
|
||||
def memory_block(workspace_dir: Path) -> str:
|
||||
def memory_block(workspace_dir: Path, user_id: UUID) -> str:
|
||||
"""构造注入 system prompt 的记忆段;两块都空就返回空串。"""
|
||||
core = _load_core(workspace_dir)
|
||||
ext = _extended_index(workspace_dir)
|
||||
core = _load_core(workspace_dir, user_id)
|
||||
ext = _extended_index(workspace_dir, user_id)
|
||||
if not core and not ext:
|
||||
return ""
|
||||
|
||||
parts = ["\n\n## 记忆 (workspace 级,跨 task 共享)"]
|
||||
parts = ["\n\n## 记忆 (user 级,跨 task 共享)"]
|
||||
if core:
|
||||
parts.append("\n### Core (常驻 prompt)\n")
|
||||
parts.append(core)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,50 @@
|
|||
"""working_dir 在 DB 与文件系统两种形态之间的归一(原 `task_dir` 已改名)。
|
||||
|
||||
存储约定(DESIGN §7.4):
|
||||
- working_dir 在 ROOT 内 → 相对 ROOT 的 posix 串(如 `workspace/users/<uid>/<name>`)
|
||||
- working_dir 在 ROOT 外 → 绝对 str(如 `D:\\projects\\other\\proj` 或 `/home/u/proj`)
|
||||
- 空串 → 空串(legacy / 未绑项目)
|
||||
|
||||
跨机器迁移 / 切 OS / 移 repo 后,ROOT-内路径仍能 resolve;ROOT-外仍存绝对是务实选择
|
||||
—— 用户自指定的项目目录没有更好的归一基。
|
||||
|
||||
Read 端两种来源走两个入口:
|
||||
- DB tasks.working_dir → `from_db_path(s)` → absolute Path
|
||||
- 用户 CLI `--working-dir` / Web `/v1/tasks` 表单 → `Path(arg).expanduser().resolve()`
|
||||
|
||||
Write 端只通过 `to_db_path(absolute Path)` → DB 串。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Union
|
||||
|
||||
ROOT: Path = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def to_db_path(p: Union[Path, str, None]) -> str:
|
||||
"""absolute Path / str → DB 串。
|
||||
|
||||
输入应已是绝对路径(build_agent / web 路由那一层都 .resolve() 过)。
|
||||
ROOT 内 → 相对 posix(`workspace/users/<uid>/<name>`)
|
||||
ROOT 外 → str(Path)(保留 OS 原生分隔符)
|
||||
空 → ""
|
||||
"""
|
||||
if not p:
|
||||
return ""
|
||||
pp = Path(p).resolve()
|
||||
try:
|
||||
return pp.relative_to(ROOT).as_posix()
|
||||
except ValueError:
|
||||
return str(pp)
|
||||
|
||||
|
||||
def from_db_path(s: str) -> Path:
|
||||
"""DB 串 → absolute Path。
|
||||
|
||||
相对串 → ROOT / s(再 resolve);绝对串 → resolve();空 → Path("")(调用方判)。
|
||||
"""
|
||||
if not s or not s.strip():
|
||||
return Path("")
|
||||
p = Path(s)
|
||||
return p.resolve() if p.is_absolute() else (ROOT / p).resolve()
|
||||
140
core/session.py
|
|
@ -1,19 +1,24 @@
|
|||
"""会话: 内存中的消息列表 + meta(cwd / model / created_at) + 落盘 json。
|
||||
"""会话: 内存中的消息列表 + meta + 落 PG `messages` 表。
|
||||
|
||||
文件格式:
|
||||
{
|
||||
"meta": {"id": "...", "created_at": "...", "cwd": "...", "model": "..."},
|
||||
"messages": [...]
|
||||
}
|
||||
§7 B Step 2:消息走 ORM(append-only, idx 严格递增,payload jsonb)。
|
||||
|
||||
兼容老格式: 如果文件根是 list,就当 messages 处理,meta 为空。
|
||||
system prompt **不入库** —— 每次 build_agent 重建拼到 messages[0](§3.7
|
||||
"memory 演化即时生效")。Session 内存里仍维持 [system, user_1, assistant_1, ...]
|
||||
全列表;DB idx 从 0 开始数第一条非 system 消息。
|
||||
|
||||
保留 `atomic_write_text` 给 skill 产物 / 其他 .md 文件写入使用。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import delete, select
|
||||
|
||||
from .storage import session_scope
|
||||
from .storage.models import Message, Task
|
||||
|
||||
|
||||
def _to_dict(msg: Any) -> Any:
|
||||
|
|
@ -30,8 +35,7 @@ def atomic_write_text(path: Path, text: str, encoding: str = "utf-8") -> None:
|
|||
"""原子写: 先写到 path.tmp 再 os.replace 到 path。
|
||||
|
||||
防止写中途异常(磁盘满 / surrogate 编码错 / 进程被杀)留下 0 字节或半文件。
|
||||
单 REPL 单 task 假设下 .tmp 名固定;若上次写崩留下孤儿,本次写会覆盖它。
|
||||
`_cleanup_if_empty` 已配合放过 `*.tmp` 文件。
|
||||
skill 产物(spec_lock.md / sections/*.md 等)走这里,messages 已改走 PG。
|
||||
"""
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
|
|
@ -43,51 +47,109 @@ def atomic_write_text(path: Path, text: str, encoding: str = "utf-8") -> None:
|
|||
|
||||
|
||||
class Session:
|
||||
"""消息列表 anchored on task_id。
|
||||
|
||||
Lazy-persist: 构造时不动 DB,第一条非 system 消息 append 时:
|
||||
1) 调 ensure_task_row 保证 tasks 行存在(Step 2 用占位值,Step 3 由 TaskState 提供完整值)
|
||||
2) INSERT 一行 messages
|
||||
|
||||
系统 reset 走 DB DELETE 该 task 全部 messages。
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
task_id: UUID,
|
||||
system_prompt: str = "",
|
||||
path: Optional[Path] = None,
|
||||
meta: Optional[dict] = None,
|
||||
) -> None:
|
||||
self.task_id: UUID = task_id
|
||||
self.messages: List[dict] = []
|
||||
self.path = path
|
||||
self.meta: Dict[str, Any] = dict(meta or {})
|
||||
self._db_idx: int = 0 # 下一条要写 DB 的 idx
|
||||
if system_prompt:
|
||||
self.messages.append({"role": "system", "content": system_prompt})
|
||||
|
||||
def append(self, msg: Any) -> None:
|
||||
self.messages.append(_to_dict(msg))
|
||||
if self.path is not None:
|
||||
self.save()
|
||||
"""追加消息;非 system 落 DB,system 仅内存。"""
|
||||
msg_dict = _to_dict(msg)
|
||||
self.messages.append(msg_dict)
|
||||
if msg_dict.get("role") == "system":
|
||||
return
|
||||
|
||||
# 首次写入前,让 tasks 行就位。`ensure_local_task_row` 在 storage 层 idempotent。
|
||||
# meta 字段(name/working_dir/skill/description/reasoning_effort)走 INSERT 一次性带入,
|
||||
# 避免首次 append 后 _list_task_rows 看到空 meta;后续 task_state.save() 走 UPSERT 覆盖。
|
||||
# name 是 NOT NULL,build_agent 必须放进 meta(新建 / resume 都已就位)。
|
||||
from .storage.utils import ensure_local_task_row
|
||||
ensure_local_task_row(
|
||||
task_id=self.task_id,
|
||||
name=self.meta.get("name", ""),
|
||||
working_dir=self.meta.get("working_dir", ""),
|
||||
skill=self.meta.get("skill", ""),
|
||||
description=self.meta.get("description", ""),
|
||||
model=self.meta.get("model", ""),
|
||||
model_profile=self.meta.get("model_profile", ""),
|
||||
reasoning_effort=self.meta.get("reasoning_effort", ""),
|
||||
)
|
||||
|
||||
with session_scope() as s:
|
||||
s.add(Message(
|
||||
task_id=self.task_id,
|
||||
idx=self._db_idx,
|
||||
payload=msg_dict,
|
||||
))
|
||||
self._db_idx += 1
|
||||
|
||||
def reset(self, keep_system: bool = True) -> None:
|
||||
"""清空消息。keep_system 仅影响内存(system 本来就不在 DB)。"""
|
||||
if keep_system and self.messages and self.messages[0].get("role") == "system":
|
||||
self.messages = [self.messages[0]]
|
||||
else:
|
||||
self.messages = []
|
||||
if self.path is not None:
|
||||
self.save()
|
||||
|
||||
def save(self) -> None:
|
||||
if self.path is None:
|
||||
return
|
||||
payload = {"meta": self.meta, "messages": self.messages}
|
||||
atomic_write_text(
|
||||
self.path,
|
||||
json.dumps(payload, ensure_ascii=False, indent=2),
|
||||
)
|
||||
with session_scope() as s:
|
||||
s.execute(delete(Message).where(Message.task_id == self.task_id))
|
||||
self._db_idx = 0
|
||||
|
||||
@classmethod
|
||||
def load(cls, path: Path) -> "Session":
|
||||
s = cls(path=path)
|
||||
if not path.exists():
|
||||
return s
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(data, list):
|
||||
# 老格式: 纯消息列表
|
||||
s.messages = data
|
||||
s.meta = {}
|
||||
elif isinstance(data, dict):
|
||||
s.messages = data.get("messages", []) or []
|
||||
s.meta = data.get("meta", {}) or {}
|
||||
return s
|
||||
def load(
|
||||
cls,
|
||||
task_id: UUID,
|
||||
system_prompt: str = "",
|
||||
meta: Optional[dict] = None,
|
||||
) -> "Session":
|
||||
"""从 DB 读历史 messages。system_prompt 由调用方注入(memory 演化即时生效)。
|
||||
|
||||
若 task_id 在 DB 不存在,返回空 Session(messages 只含 system,_db_idx=0);
|
||||
调用方判断该不该报错。
|
||||
"""
|
||||
sess = cls(task_id=task_id, system_prompt=system_prompt, meta=meta)
|
||||
with session_scope() as s:
|
||||
rows = s.execute(
|
||||
select(Message)
|
||||
.where(Message.task_id == task_id)
|
||||
.order_by(Message.idx)
|
||||
).scalars().all()
|
||||
for row in rows:
|
||||
sess.messages.append(dict(row.payload))
|
||||
sess._db_idx = len(rows)
|
||||
return sess
|
||||
|
||||
@classmethod
|
||||
def task_exists(cls, task_id: UUID) -> bool:
|
||||
"""tasks 行 + messages 至少 1 条 → 该 task 真存在(不是 lazy 占位)。"""
|
||||
with session_scope() as s:
|
||||
row = s.execute(
|
||||
select(Task.task_id).where(Task.task_id == task_id)
|
||||
).scalar_one_or_none()
|
||||
if row is None:
|
||||
return False
|
||||
cnt = s.execute(
|
||||
select(Message.message_id)
|
||||
.where(Message.task_id == task_id)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
return cnt is not None
|
||||
|
||||
def n_user_msgs(self) -> int:
|
||||
"""内存里 user 消息数,用于 _cleanup_if_empty 守门(避免回 DB)。"""
|
||||
return sum(1 for m in self.messages if m.get("role") == "user")
|
||||
|
|
|
|||
|
|
@ -0,0 +1,36 @@
|
|||
"""§7 B 阶段:Storage 落 PG。
|
||||
|
||||
入口:
|
||||
from core.storage import get_engine, session_scope, ensure_local_sentinel
|
||||
from core.storage.models import User, Task, Message, Run, UsageEvent
|
||||
|
||||
ZCBOT_DB_URL 环境变量必填(本地连测试 / staging PG;SaaS 连生产 PG)。
|
||||
未设置时 get_engine() 抛 RuntimeError 并指引设置。
|
||||
"""
|
||||
from .engine import (
|
||||
ensure_local_sentinel,
|
||||
get_engine,
|
||||
session_scope,
|
||||
)
|
||||
from .models import SENTINEL_USER_ID
|
||||
from .utils import (
|
||||
NoSubtaskError,
|
||||
check_no_subtask,
|
||||
ensure_local_task_row,
|
||||
get_task,
|
||||
update_task,
|
||||
upsert_task,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"NoSubtaskError",
|
||||
"SENTINEL_USER_ID",
|
||||
"check_no_subtask",
|
||||
"ensure_local_sentinel",
|
||||
"ensure_local_task_row",
|
||||
"get_engine",
|
||||
"get_task",
|
||||
"session_scope",
|
||||
"update_task",
|
||||
"upsert_task",
|
||||
]
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
"""PG 连接 + Session factory + 本地 sentinel 初始化。
|
||||
|
||||
`ZCBOT_DB_URL` 必填,标准 SQLAlchemy URL,如:
|
||||
postgresql+psycopg://user:pass@host:5432/zcbot
|
||||
|
||||
未设置时 get_engine() 抛 RuntimeError 并打印指引(不引导 docker)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from contextlib import contextmanager
|
||||
from typing import Iterator, Optional
|
||||
|
||||
from sqlalchemy import Engine, create_engine, select
|
||||
from sqlalchemy.orm import Session, sessionmaker
|
||||
|
||||
from .models import SENTINEL_USER_ID, User
|
||||
|
||||
_engine: Optional[Engine] = None
|
||||
_SessionLocal: Optional[sessionmaker[Session]] = None
|
||||
|
||||
|
||||
_DB_URL_HINT = (
|
||||
"ZCBOT_DB_URL is not set.\n"
|
||||
" export ZCBOT_DB_URL='postgresql+psycopg://user:pass@host:5432/dbname'\n"
|
||||
" (local: dev/staging PG; SaaS: production PG)"
|
||||
)
|
||||
|
||||
|
||||
def _read_db_url() -> str:
|
||||
url = os.environ.get("ZCBOT_DB_URL", "").strip()
|
||||
if not url:
|
||||
raise RuntimeError(_DB_URL_HINT)
|
||||
return url
|
||||
|
||||
|
||||
def get_engine() -> Engine:
|
||||
"""单例 engine。线程安全(SQLAlchemy 内置 pool)。"""
|
||||
global _engine, _SessionLocal
|
||||
if _engine is None:
|
||||
url = _read_db_url()
|
||||
_engine = create_engine(url, pool_pre_ping=True, future=True)
|
||||
_SessionLocal = sessionmaker(bind=_engine, expire_on_commit=False, future=True)
|
||||
return _engine
|
||||
|
||||
|
||||
def get_sessionmaker() -> sessionmaker[Session]:
|
||||
if _SessionLocal is None:
|
||||
get_engine()
|
||||
assert _SessionLocal is not None
|
||||
return _SessionLocal
|
||||
|
||||
|
||||
@contextmanager
|
||||
def session_scope() -> Iterator[Session]:
|
||||
"""事务上下文:成功 commit,异常 rollback,总是 close。"""
|
||||
sm = get_sessionmaker()
|
||||
s = sm()
|
||||
try:
|
||||
yield s
|
||||
s.commit()
|
||||
except Exception:
|
||||
s.rollback()
|
||||
raise
|
||||
finally:
|
||||
s.close()
|
||||
|
||||
|
||||
def ensure_local_sentinel() -> None:
|
||||
"""本地形态:若 users 表无 sentinel 行则 INSERT。
|
||||
|
||||
本地 CLI 启动时调用一次,SaaS 形态不调用(用户由 auth 流程创建)。
|
||||
幂等。
|
||||
"""
|
||||
with session_scope() as s:
|
||||
existing = s.execute(
|
||||
select(User).where(User.user_id == SENTINEL_USER_ID)
|
||||
).scalar_one_or_none()
|
||||
if existing is None:
|
||||
s.add(User(user_id=SENTINEL_USER_ID))
|
||||
|
|
@ -0,0 +1,125 @@
|
|||
"""SQLAlchemy 2.x ORM models,对应 DESIGN.md §7.4 schema。
|
||||
|
||||
5 张表:users / tasks / messages / runs / usage_events。
|
||||
- users 本地形态固定 INSERT sentinel(`00000000-...`)
|
||||
- messages.payload 用 jsonb,GIN 索引在 migration 里建
|
||||
- runs / usage_events 在 B 阶段先建表,真正写入要等 D 阶段(HTTP /v1 + run 生命周期)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from typing import Any, Optional
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
Numeric,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB, UUID as PG_UUID
|
||||
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
|
||||
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
pass
|
||||
|
||||
|
||||
# 本地单用户 sentinel —— 所有本地 task 都 FK 到这一行
|
||||
SENTINEL_USER_ID: UUID = UUID("00000000-0000-0000-0000-000000000000")
|
||||
|
||||
|
||||
class User(Base):
|
||||
__tablename__ = "users"
|
||||
|
||||
user_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||
email: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||
oidc_subject: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||
password_hash: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||
plan: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
|
||||
|
||||
class Task(Base):
|
||||
__tablename__ = "tasks"
|
||||
|
||||
task_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||
user_id: Mapped[UUID] = mapped_column(
|
||||
PG_UUID(as_uuid=True), ForeignKey("users.user_id"), nullable=False
|
||||
)
|
||||
name: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
working_dir: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
skill: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
description: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
status: Mapped[str] = mapped_column(Text, nullable=False, default="active")
|
||||
model: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
model_profile: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
reasoning_effort: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
tokens_prompt: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
tokens_completion: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
cost_usd: Mapped[Decimal] = mapped_column(Numeric(12, 6), nullable=False, default=0)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False
|
||||
)
|
||||
|
||||
|
||||
class Message(Base):
|
||||
__tablename__ = "messages"
|
||||
__table_args__ = (UniqueConstraint("task_id", "idx", name="uq_messages_task_idx"),)
|
||||
|
||||
message_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||
task_id: Mapped[UUID] = mapped_column(
|
||||
PG_UUID(as_uuid=True),
|
||||
ForeignKey("tasks.task_id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
idx: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
payload: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
||||
tokens_in: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
|
||||
tokens_out: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
|
||||
|
||||
class Run(Base):
|
||||
__tablename__ = "runs"
|
||||
|
||||
run_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||
task_id: Mapped[UUID] = mapped_column(
|
||||
PG_UUID(as_uuid=True),
|
||||
ForeignKey("tasks.task_id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
status: Mapped[str] = mapped_column(Text, nullable=False, default="pending")
|
||||
started_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
finished_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
error: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||
tokens_p: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
tokens_c: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
|
||||
|
||||
class UsageEvent(Base):
|
||||
"""append-only 审计。task_id / run_id 不 FK,task 硬删后审计仍存活(§7.4)。"""
|
||||
|
||||
__tablename__ = "usage_events"
|
||||
|
||||
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
|
||||
user_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), nullable=False)
|
||||
task_id: Mapped[Optional[UUID]] = mapped_column(PG_UUID(as_uuid=True), nullable=True)
|
||||
run_id: Mapped[Optional[UUID]] = mapped_column(PG_UUID(as_uuid=True), nullable=True)
|
||||
kind: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
value: Mapped[Decimal] = mapped_column(Numeric(20, 8), nullable=False)
|
||||
ts: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
"""Storage 辅助:tasks 表的 idempotent 创建 / UPSERT / UPDATE / no-subtask 校验。"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert
|
||||
|
||||
from .engine import session_scope
|
||||
from .models import SENTINEL_USER_ID, Task
|
||||
|
||||
|
||||
class NoSubtaskError(ValueError):
|
||||
"""working_dir 与同 user 已有 task 形成前缀嵌套(§7.4 no-subtask 策略)。"""
|
||||
|
||||
|
||||
def ensure_local_task_row(
|
||||
task_id: UUID,
|
||||
name: str,
|
||||
working_dir: str = "",
|
||||
skill: str = "",
|
||||
description: str = "",
|
||||
model: str = "",
|
||||
model_profile: str = "",
|
||||
reasoning_effort: str = "",
|
||||
user_id: UUID = SENTINEL_USER_ID,
|
||||
) -> None:
|
||||
"""占位 INSERT(ON CONFLICT DO NOTHING)—— 不覆盖已有字段。
|
||||
|
||||
用于 `Session.append` 在首条非 system 消息前打底 tasks 行,避免 messages
|
||||
FK 违反。字段是 build_agent 阶段已知的最小集;TaskState.save 之后会通过
|
||||
`upsert_task` 把真实字段(desc/status/tokens 等)写进去。`name` 必填(列 NOT NULL),
|
||||
调用方应已 validate。
|
||||
"""
|
||||
stmt = (
|
||||
insert(Task)
|
||||
.values(
|
||||
task_id=task_id,
|
||||
user_id=user_id,
|
||||
name=name,
|
||||
working_dir=working_dir,
|
||||
skill=skill,
|
||||
description=description,
|
||||
model=model,
|
||||
model_profile=model_profile,
|
||||
reasoning_effort=reasoning_effort,
|
||||
)
|
||||
.on_conflict_do_nothing(index_elements=["task_id"])
|
||||
)
|
||||
with session_scope() as s:
|
||||
s.execute(stmt)
|
||||
|
||||
|
||||
def upsert_task(
|
||||
task_id: UUID,
|
||||
*,
|
||||
user_id: UUID = SENTINEL_USER_ID,
|
||||
**fields: Any,
|
||||
) -> None:
|
||||
"""INSERT ... ON CONFLICT DO UPDATE —— TaskState.save 的落地点。
|
||||
|
||||
fields 可包含 tasks 表任意可写列(name/working_dir/skill/description/status/model/
|
||||
model_profile/reasoning_effort/tokens_prompt/tokens_completion/cost_usd)。
|
||||
不传的字段在 INSERT 时走 ORM 默认值,UPDATE 时不动。
|
||||
INSERT 路径需要 name(NOT NULL)+ working_dir;纯 UPDATE 路径(行已存在)不强制。
|
||||
"""
|
||||
values = {"task_id": task_id, "user_id": user_id, **fields}
|
||||
stmt = insert(Task).values(**values)
|
||||
update_cols = {k: stmt.excluded[k] for k in fields}
|
||||
if update_cols:
|
||||
# ORM 的 onupdate=func.now() 只在 ORM-level UPDATE 触发,DO UPDATE 是 raw DML
|
||||
# 不会自动刷 updated_at —— 这里显式追加。
|
||||
update_cols["updated_at"] = func.now()
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["task_id"], set_=update_cols
|
||||
)
|
||||
else:
|
||||
stmt = stmt.on_conflict_do_nothing(index_elements=["task_id"])
|
||||
with session_scope() as s:
|
||||
s.execute(stmt)
|
||||
|
||||
|
||||
def update_task(task_id: UUID, **fields: Any) -> int:
|
||||
"""UPDATE 已有 tasks 行;不存在则 no-op(返回 0)。
|
||||
|
||||
ORM-level update 会带 onupdate=func.now() 自动刷 updated_at,无需显式传。
|
||||
"""
|
||||
if not fields:
|
||||
return 0
|
||||
with session_scope() as s:
|
||||
result = s.execute(
|
||||
update(Task).where(Task.task_id == task_id).values(**fields)
|
||||
)
|
||||
return result.rowcount or 0
|
||||
|
||||
|
||||
def get_task(task_id: UUID) -> Optional[Task]:
|
||||
"""读 tasks 行,不存在返回 None。"""
|
||||
with session_scope() as s:
|
||||
return s.execute(
|
||||
select(Task).where(Task.task_id == task_id)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def check_no_subtask(
|
||||
working_dir: str,
|
||||
user_id: UUID = SENTINEL_USER_ID,
|
||||
) -> None:
|
||||
"""§7.4 no-subtask:同 user 下校验 working_dir 不能与已有 working_dir 形成前缀嵌套。
|
||||
|
||||
允许:同 working_dir(同项目多对话)、完全无关路径(平级或不相关)。
|
||||
拒绝:new 是 existing 的子目录、existing 是 new 的子目录。
|
||||
空 working_dir / 仅 whitespace 跳过(legacy / 未绑项目)。
|
||||
|
||||
`working_dir` 入参既可以是 db 形态(相对 ROOT)也可以是 absolute str,内部统一用
|
||||
`from_db_path` 归一到 absolute posix 后再比前缀;DB 里行的两种形态同样归一。
|
||||
数量小(per user 几十量级),全量拉到 Python 端比对,不在 SQL 里拼分隔符 / 前缀。
|
||||
"""
|
||||
if not working_dir or not working_dir.strip():
|
||||
return
|
||||
from core.paths import from_db_path
|
||||
|
||||
new_abs = from_db_path(working_dir).as_posix()
|
||||
if not new_abs:
|
||||
return
|
||||
with session_scope() as s:
|
||||
rows = s.execute(
|
||||
select(Task.task_id, Task.working_dir)
|
||||
.where(Task.user_id == user_id, Task.working_dir != "")
|
||||
).all()
|
||||
for existing_id, existing_dir in rows:
|
||||
existing_abs = from_db_path(existing_dir).as_posix()
|
||||
if not existing_abs or existing_abs == new_abs:
|
||||
continue
|
||||
if new_abs.startswith(existing_abs + "/") or existing_abs.startswith(new_abs + "/"):
|
||||
raise NoSubtaskError(
|
||||
f"working_dir {working_dir!r} 与已有 task {str(existing_id)[:8]} 的 "
|
||||
f"working_dir {existing_dir!r} 前缀嵌套 — 同项目多对话请用相同 working_dir"
|
||||
)
|
||||
95
core/task.py
|
|
@ -1,64 +1,85 @@
|
|||
"""任务状态: DESIGN.md §7.1 规约,落 `<task_dir>/state.json`。
|
||||
"""任务元数据: Session 上层,落 PG `tasks` 表(§7 B Step 3)。
|
||||
|
||||
Task 是 Session 的上层概念 —— Session 只管对话消息,Task 还管 mode/description/
|
||||
status/tokens/cost/timestamps,这些是跨轮次共享、和文件系统状态对齐的元数据。
|
||||
Session 只管对话消息;Task 管 mode/description/status/model/tokens/cost/时间戳
|
||||
—— 跨轮次共享的元数据,DESIGN.md §7.1 / §7.4 规约。
|
||||
|
||||
文件路径约定(workspace/ 下):
|
||||
tasks/<task_id>/state.json ← 此模块负责
|
||||
tasks/<task_id>/messages.json ← Session 落
|
||||
state.json 已废除;字段从 PG 读出,save() 走 INSERT ... ON CONFLICT DO UPDATE。
|
||||
created_at / updated_at 由 PG server_default / onupdate 管,Python 侧只读。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import asdict, dataclass, fields
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from uuid import UUID
|
||||
|
||||
from .session import atomic_write_text
|
||||
from .storage import upsert_task
|
||||
from .storage.models import Task as TaskRow
|
||||
from .storage.utils import get_task
|
||||
|
||||
|
||||
def _iso(dt: Optional[datetime]) -> str:
|
||||
return dt.isoformat(timespec="seconds") if dt else ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class TaskState:
|
||||
task_id: str
|
||||
mode: str = "" # 自由形式: coding / ppt / proposal / general / 自定
|
||||
task_id: str # UUID 字符串形式(对外展示用,DB 仍是 UUID)
|
||||
name: str = "" # 任务显示名(列 NOT NULL,新建必填;resume 时从 DB 读)
|
||||
working_dir: str = "" # 工作目录(db 形态:ROOT 内相对 / ROOT 外绝对;空=未绑)
|
||||
skill: str = "" # 智能体类型(coding / ppt / proposal / 自由形式,后续可对齐 skills/ 注册表)
|
||||
description: str = "" # 一句话描述,便于列表识别
|
||||
status: str = "active" # active / completed / abandoned
|
||||
model: str = "" # caps.model_id
|
||||
model_profile: str = "" # 档案名,如 deepseek_v4.flash
|
||||
reasoning_effort: str = ""
|
||||
cwd: str = "" # 任务的工作基目录
|
||||
created_at: str = "" # ISO 时间戳
|
||||
updated_at: str = ""
|
||||
tokens_prompt: int = 0
|
||||
tokens_completion: int = 0
|
||||
cost_usd: float = 0.0 # 暂不算,留位
|
||||
cost_usd: float = 0.0
|
||||
created_at: str = "" # PG server_default 填,Python 侧只读
|
||||
updated_at: str = ""
|
||||
|
||||
@property
|
||||
def tokens_total(self) -> int:
|
||||
return self.tokens_prompt + self.tokens_completion
|
||||
|
||||
def save(self, task_dir: Path) -> None:
|
||||
self.updated_at = datetime.now().isoformat(timespec="seconds")
|
||||
atomic_write_text(
|
||||
task_dir / "state.json",
|
||||
json.dumps(asdict(self), ensure_ascii=False, indent=2),
|
||||
def save(self) -> None:
|
||||
"""UPSERT 到 PG。created_at / updated_at 不参与写入(PG 自动管)。"""
|
||||
upsert_task(
|
||||
UUID(self.task_id),
|
||||
name=self.name,
|
||||
working_dir=self.working_dir,
|
||||
skill=self.skill,
|
||||
description=self.description,
|
||||
status=self.status,
|
||||
model=self.model,
|
||||
model_profile=self.model_profile,
|
||||
reasoning_effort=self.reasoning_effort,
|
||||
tokens_prompt=self.tokens_prompt,
|
||||
tokens_completion=self.tokens_completion,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def load(cls, task_dir: Path) -> Optional["TaskState"]:
|
||||
p = task_dir / "state.json"
|
||||
if not p.exists():
|
||||
return None
|
||||
try:
|
||||
data = json.loads(p.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
return None
|
||||
if not isinstance(data, dict):
|
||||
return None
|
||||
# 容忍 schema 演化:只取已知字段,缺失字段用 dataclass 默认
|
||||
known = {f.name for f in fields(cls)}
|
||||
kwargs = {k: v for k, v in data.items() if k in known}
|
||||
if "task_id" not in kwargs:
|
||||
kwargs["task_id"] = task_dir.name
|
||||
return cls(**kwargs)
|
||||
def from_row(cls, row: TaskRow) -> "TaskState":
|
||||
return cls(
|
||||
task_id=str(row.task_id),
|
||||
name=row.name,
|
||||
working_dir=row.working_dir,
|
||||
skill=row.skill,
|
||||
description=row.description,
|
||||
status=row.status,
|
||||
model=row.model,
|
||||
model_profile=row.model_profile,
|
||||
reasoning_effort=row.reasoning_effort,
|
||||
tokens_prompt=row.tokens_prompt,
|
||||
tokens_completion=row.tokens_completion,
|
||||
cost_usd=float(row.cost_usd or 0),
|
||||
created_at=_iso(row.created_at),
|
||||
updated_at=_iso(row.updated_at),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def load(cls, task_id: UUID) -> Optional["TaskState"]:
|
||||
"""从 PG 读;不存在返回 None。"""
|
||||
row = get_task(task_id)
|
||||
return cls.from_row(row) if row is not None else None
|
||||
|
|
|
|||
|
|
@ -0,0 +1,61 @@
|
|||
"""Alembic env.py -- read DB URL from ZCBOT_DB_URL, metadata from core.storage.models."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from logging.config import fileConfig
|
||||
from pathlib import Path
|
||||
|
||||
from alembic import context
|
||||
from sqlalchemy import engine_from_config, pool
|
||||
|
||||
# Make project root importable so we can pull in core.storage.models
|
||||
ROOT = Path(__file__).resolve().parent.parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from core.storage.models import Base # noqa: E402
|
||||
|
||||
config = context.config
|
||||
|
||||
if config.config_file_name is not None:
|
||||
fileConfig(config.config_file_name)
|
||||
|
||||
# Inject URL from env var (not hardcoded in alembic.ini)
|
||||
db_url = os.environ.get("ZCBOT_DB_URL", "").strip()
|
||||
if not db_url:
|
||||
raise RuntimeError(
|
||||
"ZCBOT_DB_URL is not set.\n"
|
||||
" export ZCBOT_DB_URL='postgresql+psycopg://user:pass@host:5432/dbname'"
|
||||
)
|
||||
config.set_main_option("sqlalchemy.url", db_url)
|
||||
|
||||
target_metadata = Base.metadata
|
||||
|
||||
|
||||
def run_migrations_offline() -> None:
|
||||
context.configure(
|
||||
url=db_url,
|
||||
target_metadata=target_metadata,
|
||||
literal_binds=True,
|
||||
dialect_opts={"paramstyle": "named"},
|
||||
)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
def run_migrations_online() -> None:
|
||||
connectable = engine_from_config(
|
||||
config.get_section(config.config_ini_section, {}),
|
||||
prefix="sqlalchemy.",
|
||||
poolclass=pool.NullPool,
|
||||
)
|
||||
with connectable.connect() as connection:
|
||||
context.configure(connection=connection, target_metadata=target_metadata)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
if context.is_offline_mode():
|
||||
run_migrations_offline()
|
||||
else:
|
||||
run_migrations_online()
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
"""${message}
|
||||
|
||||
Revision ID: ${up_revision}
|
||||
Revises: ${down_revision | comma,n}
|
||||
Create Date: ${create_date}
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
${imports if imports else ""}
|
||||
|
||||
revision: str = ${repr(up_revision)}
|
||||
down_revision: Union[str, None] = ${repr(down_revision)}
|
||||
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
|
||||
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
${upgrades if upgrades else "pass"}
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
${downgrades if downgrades else "pass"}
|
||||
|
|
@ -0,0 +1,125 @@
|
|||
"""initial schema -- users / tasks / messages / runs / usage_events
|
||||
|
||||
Revision ID: 0001
|
||||
Revises:
|
||||
Create Date: 2026-05-14
|
||||
|
||||
DESIGN.md section 7.4 schema. First migration.
|
||||
- pgcrypto extension fallback (PG 13+ has gen_random_uuid built-in;
|
||||
older versions need the extension).
|
||||
- messages.payload GIN index (jsonb_path_ops).
|
||||
- tasks (user_id, task_dir) and (user_id, status) composite indexes.
|
||||
- Local sentinel user is INSERTed by core.storage.ensure_local_sentinel
|
||||
at CLI startup, NOT in this migration (avoids stray sentinel rows on
|
||||
the SaaS instance).
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision: str = "0001"
|
||||
down_revision: Union[str, None] = None
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute("CREATE EXTENSION IF NOT EXISTS pgcrypto")
|
||||
|
||||
op.create_table(
|
||||
"users",
|
||||
sa.Column("user_id", postgresql.UUID(as_uuid=True), primary_key=True,
|
||||
server_default=sa.text("gen_random_uuid()")),
|
||||
sa.Column("email", sa.Text(), nullable=True),
|
||||
sa.Column("oidc_subject", sa.Text(), nullable=True),
|
||||
sa.Column("password_hash", sa.Text(), nullable=True),
|
||||
sa.Column("plan", sa.Text(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"), nullable=False),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"tasks",
|
||||
sa.Column("task_id", postgresql.UUID(as_uuid=True), primary_key=True,
|
||||
server_default=sa.text("gen_random_uuid()")),
|
||||
sa.Column("user_id", postgresql.UUID(as_uuid=True),
|
||||
sa.ForeignKey("users.user_id"), nullable=False),
|
||||
sa.Column("task_dir", sa.Text(), nullable=False),
|
||||
sa.Column("mode", sa.Text(), nullable=False, server_default=""),
|
||||
sa.Column("description", sa.Text(), nullable=False, server_default=""),
|
||||
sa.Column("status", sa.Text(), nullable=False, server_default="active"),
|
||||
sa.Column("model", sa.Text(), nullable=False, server_default=""),
|
||||
sa.Column("model_profile", sa.Text(), nullable=False, server_default=""),
|
||||
sa.Column("reasoning_effort", sa.Text(), nullable=False, server_default=""),
|
||||
sa.Column("tokens_prompt", sa.Integer(), nullable=False, server_default="0"),
|
||||
sa.Column("tokens_completion", sa.Integer(), nullable=False, server_default="0"),
|
||||
sa.Column("cost_usd", sa.Numeric(12, 6), nullable=False, server_default="0"),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"), nullable=False),
|
||||
)
|
||||
op.create_index("ix_tasks_user_task_dir", "tasks", ["user_id", "task_dir"])
|
||||
op.create_index("ix_tasks_user_status", "tasks", ["user_id", "status"])
|
||||
|
||||
op.create_table(
|
||||
"messages",
|
||||
sa.Column("message_id", postgresql.UUID(as_uuid=True), primary_key=True,
|
||||
server_default=sa.text("gen_random_uuid()")),
|
||||
sa.Column("task_id", postgresql.UUID(as_uuid=True),
|
||||
sa.ForeignKey("tasks.task_id", ondelete="CASCADE"), nullable=False),
|
||||
sa.Column("idx", sa.Integer(), nullable=False),
|
||||
sa.Column("payload", postgresql.JSONB(), nullable=False),
|
||||
sa.Column("tokens_in", sa.Integer(), nullable=True),
|
||||
sa.Column("tokens_out", sa.Integer(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"), nullable=False),
|
||||
sa.UniqueConstraint("task_id", "idx", name="uq_messages_task_idx"),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_messages_payload_gin", "messages", ["payload"],
|
||||
postgresql_using="gin", postgresql_ops={"payload": "jsonb_path_ops"},
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"runs",
|
||||
sa.Column("run_id", postgresql.UUID(as_uuid=True), primary_key=True,
|
||||
server_default=sa.text("gen_random_uuid()")),
|
||||
sa.Column("task_id", postgresql.UUID(as_uuid=True),
|
||||
sa.ForeignKey("tasks.task_id", ondelete="CASCADE"), nullable=False),
|
||||
sa.Column("status", sa.Text(), nullable=False, server_default="pending"),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("error", sa.Text(), nullable=True),
|
||||
sa.Column("tokens_p", sa.Integer(), nullable=False, server_default="0"),
|
||||
sa.Column("tokens_c", sa.Integer(), nullable=False, server_default="0"),
|
||||
)
|
||||
op.create_index("ix_runs_task", "runs", ["task_id"])
|
||||
|
||||
op.create_table(
|
||||
"usage_events",
|
||||
sa.Column("id", sa.BigInteger(), primary_key=True, autoincrement=True),
|
||||
sa.Column("user_id", postgresql.UUID(as_uuid=True), nullable=False),
|
||||
sa.Column("task_id", postgresql.UUID(as_uuid=True), nullable=True),
|
||||
sa.Column("run_id", postgresql.UUID(as_uuid=True), nullable=True),
|
||||
sa.Column("kind", sa.Text(), nullable=False),
|
||||
sa.Column("value", sa.Numeric(20, 8), nullable=False),
|
||||
sa.Column("ts", sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"), nullable=False),
|
||||
)
|
||||
op.create_index("ix_usage_user_ts", "usage_events", ["user_id", "ts"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_usage_user_ts", table_name="usage_events")
|
||||
op.drop_table("usage_events")
|
||||
op.drop_index("ix_runs_task", table_name="runs")
|
||||
op.drop_table("runs")
|
||||
op.drop_index("ix_messages_payload_gin", table_name="messages")
|
||||
op.drop_table("messages")
|
||||
op.drop_index("ix_tasks_user_status", table_name="tasks")
|
||||
op.drop_index("ix_tasks_user_task_dir", table_name="tasks")
|
||||
op.drop_table("tasks")
|
||||
op.drop_table("users")
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
"""task_dir: ROOT-prefix absolute → relative posix.
|
||||
|
||||
Revision ID: 0002
|
||||
Revises: 0001
|
||||
Create Date: 2026-05-15
|
||||
|
||||
把 `tasks.task_dir` 在 ROOT(本机仓库根)内的绝对路径统一改成相对 ROOT 的 posix 串;
|
||||
ROOT 外的绝对路径(用户自指定的项目目录)保持原样。
|
||||
|
||||
ROOT 从 `core.paths` 读 —— alembic env.py 把项目根注入 sys.path,可正常 import。
|
||||
|
||||
存储约定见 DESIGN.md §7.4 / core/paths.py 头部注释。
|
||||
|
||||
UPDATE 逻辑:
|
||||
- 把 task_dir 的 backslash 归一成 `/`(`replace`),与 ROOT 的 posix 串比前缀
|
||||
- 命中 → 截掉前缀 + 一个分隔符,得到相对 posix
|
||||
- 没命中 → 不动
|
||||
|
||||
downgrade 反向 —— 相对(无盘符 / 不以 `/` 起头)拼回 ROOT。
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
from sqlalchemy import text
|
||||
|
||||
revision: str = "0002"
|
||||
down_revision: Union[str, None] = "0001"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def _root_posix() -> str:
|
||||
from core.paths import ROOT
|
||||
return str(ROOT).replace("\\", "/")
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
root = _root_posix()
|
||||
# SUBSTRING from N 是 1-indexed;`<root>/<rel>` 长度 = len(root)+1+len(rel),想取 rel 从 len+2 起
|
||||
op.execute(
|
||||
text(
|
||||
"UPDATE tasks "
|
||||
"SET task_dir = substring(replace(task_dir, '\\', '/') from :off) "
|
||||
"WHERE replace(task_dir, '\\', '/') LIKE :prefix"
|
||||
).bindparams(off=len(root) + 2, prefix=root + "/%")
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
root = _root_posix()
|
||||
# 把"看起来是相对"的行拼回 ROOT 绝对。绝对 = 以 `/` 起头(Linux/posix)或盘符
|
||||
# `<letter>:` 起头(Windows);LIKE 里 `_` 通配单字符,正好可匹配盘符。
|
||||
op.execute(
|
||||
text(
|
||||
"UPDATE tasks "
|
||||
"SET task_dir = :prefix || task_dir "
|
||||
"WHERE task_dir <> '' "
|
||||
" AND task_dir NOT LIKE '/%' "
|
||||
" AND task_dir NOT LIKE '_:%'"
|
||||
).bindparams(prefix=root + "/")
|
||||
)
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
"""task name + rename task_dir → working_dir + rename mode → skill.
|
||||
|
||||
Revision ID: 0003
|
||||
Revises: 0002
|
||||
Create Date: 2026-05-17
|
||||
|
||||
三件事一把改:
|
||||
- 加 `name` 列(必填,任务显示名,与工作目录解耦 —— 同 working_dir 可有多个 task)
|
||||
- `task_dir` → `working_dir`(同目录跨 task 共享,语义就是"工作目录",DESIGN §7.1)
|
||||
- `mode` → `skill`(跟项目 `skills/` 注册表对齐,语义"选用哪个智能体")
|
||||
|
||||
开发期 + 用户授权清表:TRUNCATE tasks CASCADE(messages / runs 跟着清)。
|
||||
新加 `name` 列 NOT NULL,空表上加 NOT NULL 不需要 server_default + backfill 两步。
|
||||
|
||||
downgrade 反向同样 TRUNCATE + 删列 + 改名;数据不可恢复。
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision: str = "0003"
|
||||
down_revision: Union[str, None] = "0002"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# 清掉旧数据 —— 用户授权(开发期,新 name 列 NOT NULL 不写 backfill)
|
||||
op.execute("TRUNCATE TABLE tasks CASCADE")
|
||||
|
||||
# task_dir → working_dir + 索引同步
|
||||
op.drop_index("ix_tasks_user_task_dir", table_name="tasks")
|
||||
op.alter_column("tasks", "task_dir", new_column_name="working_dir")
|
||||
op.create_index("ix_tasks_user_working_dir", "tasks", ["user_id", "working_dir"])
|
||||
|
||||
# mode → skill(对齐 skills/ 注册表语义)
|
||||
op.alter_column("tasks", "mode", new_column_name="skill")
|
||||
|
||||
# name 列必填,空表上 NOT NULL 加列不需要 default + UPDATE 两步
|
||||
op.add_column("tasks", sa.Column("name", sa.Text(), nullable=False))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.execute("TRUNCATE TABLE tasks CASCADE")
|
||||
op.drop_column("tasks", "name")
|
||||
op.alter_column("tasks", "skill", new_column_name="mode")
|
||||
op.drop_index("ix_tasks_user_working_dir", table_name="tasks")
|
||||
op.alter_column("tasks", "working_dir", new_column_name="task_dir")
|
||||
op.create_index("ix_tasks_user_task_dir", "tasks", ["user_id", "task_dir"])
|
||||
313
main.py
|
|
@ -1,14 +1,25 @@
|
|||
"""装配入口: 读 config → 加载 capabilities/skills → 构造 LLM/tools/session/loop。
|
||||
|
||||
存储布局:
|
||||
workspace/tasks/<task_id>/state.json ← TaskState
|
||||
workspace/tasks/<task_id>/messages.json ← Session 消息
|
||||
存储布局(§7.0 / §7.4):本地 + SaaS 共用 `workspace/` 根,只差 user_id:
|
||||
|
||||
PG tasks / messages ← 元数据 + 消息
|
||||
workspace/users/<user_id>/<working_dir>/ ← 工作目录(用户起名,可多 task 共享)
|
||||
workspace/users/<user_id>/.memory/{core.md, extended/} ← per-user 记忆(dotfile 隔离)
|
||||
|
||||
本地 CLI user_id = SENTINEL(`00000000-...`),web/JWT user_id = sub。
|
||||
task_id / user_id 全 UUID;state.json 已删除(元数据全在 PG)。
|
||||
|
||||
**新建 task 必须给 `name`**(任务显示名,DB 列 NOT NULL);**`working_dir` 可选**
|
||||
(留空 → 用 name 作目录名;同 working_dir 多 task 自动共享 §7.1)。name 和 working_dir
|
||||
都过同一份 `validate_task_name` 校验(简单名,不含 `/\\..`、不以 `.` 起头)。
|
||||
`_cleanup_if_empty` 不 rmtree FS —— 同 working_dir 跨 task 复用,空 task 只删 DB 行。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional, Tuple
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
import yaml
|
||||
from rich.console import Console
|
||||
|
|
@ -17,17 +28,17 @@ from core.capabilities import ModelCapabilities
|
|||
from core.llm import LLM
|
||||
from core.loop import AgentLoop
|
||||
from core.memory import memory_block
|
||||
from core.paths import ROOT, from_db_path, to_db_path
|
||||
from core.session import Session
|
||||
from core.sinks import ConsoleEventSink
|
||||
from core.skills import SkillRegistry
|
||||
from core.storage import SENTINEL_USER_ID, check_no_subtask, ensure_local_sentinel
|
||||
from core.task import TaskState
|
||||
from tools.fs import EditTool, GlobTool, GrepTool, ReadTool, WriteTool
|
||||
from tools.run_python import RunPythonTool
|
||||
from tools.shell import ShellTool
|
||||
from tools.skill_tool import LoadSkillTool
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
|
||||
|
||||
def load_config() -> dict:
|
||||
return yaml.safe_load((ROOT / "config" / "agent.yaml").read_text(encoding="utf-8")) or {}
|
||||
|
|
@ -40,38 +51,115 @@ def resolve_workspace(workspace: Optional[str], cfg: Optional[dict] = None) -> P
|
|||
return p
|
||||
|
||||
|
||||
def tasks_dir(workspace_dir: Path) -> Path:
|
||||
d = workspace_dir / "tasks"
|
||||
def user_root(workspace_dir: Path, user_id: UUID) -> Path:
|
||||
"""per-user 子树根:`<workspace>/users/<user_id>/`。working_dir / `.memory/` 都在下面。"""
|
||||
d = workspace_dir / "users" / str(user_id)
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def resolve_task_messages_path(
|
||||
workspace_dir: Path, task_id: Optional[str], resume: bool
|
||||
) -> Tuple[Path, str]:
|
||||
"""返回 (messages_file_path, task_id)。
|
||||
新建:tasks/<id>/messages.json;Resume:tasks/<id>/messages.json,'last' 取最新。
|
||||
"""
|
||||
tdir = tasks_dir(workspace_dir)
|
||||
if resume:
|
||||
if task_id in (None, "", "last"):
|
||||
candidates = []
|
||||
for d in tdir.iterdir():
|
||||
mf = d / "messages.json"
|
||||
if mf.is_file():
|
||||
candidates.append((mf.stat().st_mtime, mf, d.name))
|
||||
if not candidates:
|
||||
raise FileNotFoundError(f"无可恢复的 task: {tdir} 下无 task")
|
||||
candidates.sort(key=lambda x: x[0], reverse=True)
|
||||
_, path, sid = candidates[0]
|
||||
return path, sid
|
||||
task_msg = tdir / task_id / "messages.json"
|
||||
if not task_msg.exists():
|
||||
raise FileNotFoundError(f"task 不存在: {task_msg}")
|
||||
return task_msg, task_id
|
||||
class InvalidTaskName(ValueError):
|
||||
"""task name / working_dir 不合法(空 / 含分隔符 / dotfile 起头 / 超长)。"""
|
||||
|
||||
sid = task_id or datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
return tdir / sid / "messages.json", sid
|
||||
|
||||
def validate_task_name(name: str) -> str:
|
||||
"""返回 stripped name;非法抛 InvalidTaskName。
|
||||
|
||||
name 和 working_dir 共用一份规则:非空 / 不含 `/\\` 和 NUL / 不以 `.` 起头
|
||||
(挡 `.memory` 等系统区)/ ≤ 255 字符。允许 CJK 与其他 Unicode 字符。
|
||||
"""
|
||||
n = (name or "").strip()
|
||||
if not n:
|
||||
raise InvalidTaskName("name 不能为空")
|
||||
if len(n) > 255:
|
||||
raise InvalidTaskName(f"name 超长(>255 字符): {n[:40]!r}...")
|
||||
if any(c in n for c in ("/", "\\", "\x00")):
|
||||
raise InvalidTaskName(f"name 不能含 `/` `\\` 或 NUL: {n!r}")
|
||||
if n.startswith("."):
|
||||
raise InvalidTaskName(
|
||||
f"name 不能以 `.` 起头(保留给 .memory 等系统区): {n!r}"
|
||||
)
|
||||
return n
|
||||
|
||||
|
||||
def working_dir_from_name(workspace_dir: Path, user_id: UUID, dir_name: str) -> Path:
|
||||
"""`<workspace>/users/<user_id>/<dir_name>` 绝对路径。
|
||||
|
||||
入参 dir_name 由 `validate_task_name` 在入口校验过;本函数只拼路径,不 mkdir
|
||||
(目录创建放在 task 创建入口 build_agent / web `/v1/tasks`,函数保持纯)。
|
||||
"""
|
||||
return user_root(workspace_dir, user_id) / dir_name
|
||||
|
||||
|
||||
def resolve_task_id(
|
||||
workspace_dir: Path,
|
||||
task_id_arg: Optional[str],
|
||||
resume: bool,
|
||||
user_id: UUID,
|
||||
working_dir_name: Optional[str] = None,
|
||||
) -> Tuple[UUID, Path]:
|
||||
"""返回 (task_id, working_dir 绝对路径)。
|
||||
|
||||
新建:`working_dir_name` 必填(调用方应已 fallback 到 name + 校验过),
|
||||
工作目录 = `<workspace>/users/<uid>/<working_dir_name>/`。
|
||||
Resume:`task_id` 从前缀/UUID/'last' 解析,working_dir 从 PG `tasks.working_dir`
|
||||
读还原;`working_dir_name` 在 resume 时被忽略。
|
||||
"""
|
||||
if resume:
|
||||
from sqlalchemy import select
|
||||
from core.storage import session_scope
|
||||
from core.storage.models import Task
|
||||
|
||||
if task_id_arg in (None, "", "last"):
|
||||
with session_scope() as s:
|
||||
row = s.execute(
|
||||
select(Task.task_id, Task.working_dir)
|
||||
.order_by(Task.updated_at.desc()).limit(1)
|
||||
).first()
|
||||
if row is None:
|
||||
raise FileNotFoundError("no recoverable task: PG tasks 表为空")
|
||||
tid, db_dir = row
|
||||
else:
|
||||
tid = _resolve_uuid_or_prefix(task_id_arg)
|
||||
with session_scope() as s:
|
||||
db_dir = s.execute(
|
||||
select(Task.working_dir).where(Task.task_id == tid)
|
||||
).scalar_one_or_none() or ""
|
||||
|
||||
if not db_dir:
|
||||
raise ValueError(
|
||||
f"task {tid} has empty working_dir in DB — should not happen "
|
||||
"(new tasks require name + working_dir; legacy empty data was wiped)"
|
||||
)
|
||||
# DB 存的是 db 形态(相对 ROOT 或绝对),走 from_db_path 还原绝对
|
||||
fs_dir = from_db_path(db_dir)
|
||||
return tid, fs_dir
|
||||
|
||||
if not working_dir_name:
|
||||
raise InvalidTaskName("new task 必须指定 working_dir(或留空 fallback 用 name)")
|
||||
safe = validate_task_name(working_dir_name)
|
||||
return uuid4(), working_dir_from_name(workspace_dir, user_id, safe)
|
||||
|
||||
|
||||
def _resolve_uuid_or_prefix(s: str) -> UUID:
|
||||
"""完整 UUID 字符串直接解析;否则当前缀,从 tasks 表精确匹配一个。"""
|
||||
try:
|
||||
return UUID(s)
|
||||
except ValueError:
|
||||
pass
|
||||
from sqlalchemy import cast, String, select
|
||||
from core.storage import session_scope
|
||||
from core.storage.models import Task
|
||||
|
||||
with session_scope() as sess:
|
||||
matches = sess.execute(
|
||||
select(Task.task_id).where(cast(Task.task_id, String).like(f"{s}%"))
|
||||
).scalars().all()
|
||||
if not matches:
|
||||
raise FileNotFoundError(f"no task matching prefix: {s}")
|
||||
if len(matches) > 1:
|
||||
raise ValueError(f"ambiguous prefix {s!r}, matched {len(matches)} tasks")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def _build_system_prompt(
|
||||
|
|
@ -79,25 +167,26 @@ def _build_system_prompt(
|
|||
skills: SkillRegistry,
|
||||
workspace_dir: Path,
|
||||
tool_base: Path,
|
||||
task_dir: Path,
|
||||
working_dir: Path,
|
||||
user_id: UUID,
|
||||
) -> str:
|
||||
"""拼 system prompt: 模板 + skill 列表 + memory + 工作目录段。
|
||||
|
||||
new task 和 resume task 都走这里,memory 演化即时生效。
|
||||
new task 和 resume task 都走这里,memory 演化即时生效。memory 按 user_id 隔离。
|
||||
"""
|
||||
prompt = (ROOT / cfg["system_prompt"]).read_text(encoding="utf-8")
|
||||
if skills.skills:
|
||||
prompt += f"\n\n## 可用 skill (用 load_skill 加载完整指引)\n{skills.discovery_block()}"
|
||||
prompt += memory_block(workspace_dir)
|
||||
task_dir_abs = task_dir.resolve()
|
||||
prompt += memory_block(workspace_dir, user_id)
|
||||
wd_abs = working_dir.resolve()
|
||||
prompt += (
|
||||
f"\n\n## 工作目录\n"
|
||||
f"- cwd(用户启动时所在目录,只读用): `{tool_base}`\n"
|
||||
f"- **task_dir(所有产物写到这里)**: `{task_dir_abs}`\n\n"
|
||||
f"- **task_dir(所有产物写到这里)**: `{wd_abs}`\n\n"
|
||||
f"SKILL 文档里出现的 `<task_dir>` 占位符,一律指上面这个绝对路径。"
|
||||
f"产物示例: `{task_dir_abs}/spec_lock.md`、"
|
||||
f"`{task_dir_abs}/sections/01_summary.md`、"
|
||||
f"`{task_dir_abs}/slides/`、最终 .docx/.pptx。\n"
|
||||
f"产物示例: `{wd_abs}/spec_lock.md`、"
|
||||
f"`{wd_abs}/sections/01_summary.md`、"
|
||||
f"`{wd_abs}/slides/`、最终 .docx/.pptx。\n"
|
||||
f"⛔ 不要把产物写到 cwd / `skills/` / repo 根 —— 只写到 task_dir。"
|
||||
)
|
||||
return prompt
|
||||
|
|
@ -110,78 +199,106 @@ def build_agent(
|
|||
session_id: Optional[str] = None,
|
||||
resume: bool = False,
|
||||
tool_base: Optional[Path] = None,
|
||||
mode: str = "",
|
||||
skill: str = "",
|
||||
description: str = "",
|
||||
name: Optional[str] = None,
|
||||
working_dir: Optional[str] = None,
|
||||
user_id: Optional[UUID] = None,
|
||||
) -> Tuple[AgentLoop, Session, str, TaskState, Path]:
|
||||
"""返回 (agent, session, task_id, task_state, task_dir)。"""
|
||||
"""返回 (agent, session, task_id_str, task_state, working_dir_path)。
|
||||
|
||||
新建 task:
|
||||
- `name` 必填(任务显示名,DB 列 NOT NULL,走 validate_task_name)
|
||||
- `working_dir` 可选(留空 → fallback 用 name 作目录名;非空也走 validate_task_name)
|
||||
Resume:name / working_dir 都忽略(从 DB 读)。
|
||||
|
||||
`user_id` 决定 working_dir 根、memory 子树、no-subtask 校验作用域。
|
||||
None → SENTINEL(本地 CLI)。web 入口必须显式传入 JWT user_id。
|
||||
"""
|
||||
cfg = load_config()
|
||||
model = model_name or cfg["default_model"]
|
||||
uid = user_id or SENTINEL_USER_ID
|
||||
|
||||
# 本地 sentinel user 入库(idempotent);build_agent 是所有 task 操作的入口
|
||||
ensure_local_sentinel()
|
||||
|
||||
caps = ModelCapabilities.load(model, ROOT / cfg["models_dir"])
|
||||
llm = LLM(caps)
|
||||
|
||||
workspace_dir = resolve_workspace(workspace, cfg)
|
||||
session_path, sid = resolve_task_messages_path(workspace_dir, session_id, resume)
|
||||
|
||||
# 新建时校验 name + 解析 working_dir(留空 fallback 用 name);resume 跳过
|
||||
task_name_safe = ""
|
||||
wd_name_for_resolve: Optional[str] = None
|
||||
if not resume:
|
||||
if not name:
|
||||
raise InvalidTaskName("new task 必须指定 name(任务显示名)")
|
||||
task_name_safe = validate_task_name(name)
|
||||
wd_raw = (working_dir or "").strip()
|
||||
wd_name = wd_raw if wd_raw else task_name_safe
|
||||
wd_name_for_resolve = validate_task_name(wd_name)
|
||||
|
||||
task_id, working_dir_path = resolve_task_id(
|
||||
workspace_dir, session_id, resume, uid, wd_name_for_resolve
|
||||
)
|
||||
sid = str(task_id)
|
||||
|
||||
# §7.4 no-subtask:新建 task 时校验 working_dir 不与同 user 已有 task 形成前缀嵌套
|
||||
# (resume 跳过 —— 该 task 已落库,改名走 Folder API 的 cascade)
|
||||
if not resume:
|
||||
check_no_subtask(str(working_dir_path), user_id=uid)
|
||||
# 新建 task 立刻建工作目录 —— 用户已声明项目,目录就该存在
|
||||
# (同 working_dir 多 task 共享,exist_ok=True 不冲突)
|
||||
working_dir_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
tool_base = Path(tool_base) if tool_base else Path.cwd()
|
||||
|
||||
skills = SkillRegistry(ROOT / cfg.get("skills_dir", "skills"))
|
||||
|
||||
task_dir = session_path.parent
|
||||
system_prompt = _build_system_prompt(
|
||||
cfg, skills, workspace_dir, tool_base, working_dir_path, uid
|
||||
)
|
||||
|
||||
system_prompt = _build_system_prompt(cfg, skills, workspace_dir, tool_base, task_dir)
|
||||
now_iso = datetime.now().isoformat(timespec="seconds")
|
||||
# meta["working_dir"] 是 db 形态(相对 ROOT 或绝对);Session.append → ensure_local_task_row
|
||||
# 把它直接落 PG tasks.working_dir,所以这里就转好。文件系统操作仍用 working_dir_path(absolute)。
|
||||
wd_db = to_db_path(working_dir_path)
|
||||
meta = {
|
||||
"id": sid,
|
||||
"created_at": now_iso,
|
||||
"cwd": str(tool_base),
|
||||
"name": task_name_safe, # resume 时空字符串(Session.load 会从 DB 拿不到 -- 不要紧,ensure 走 ON CONFLICT DO NOTHING)
|
||||
"working_dir": wd_db,
|
||||
"model": caps.model_id,
|
||||
"model_profile": model,
|
||||
"skill": skill,
|
||||
"description": description,
|
||||
"reasoning_effort": caps.default_reasoning_effort or "",
|
||||
}
|
||||
|
||||
if resume:
|
||||
session = Session.load(session_path)
|
||||
# 用最新 memory + skill 列表刷新 system prompt(messages[0]),memory 演化即时生效
|
||||
if session.messages and session.messages[0].get("role") == "system":
|
||||
session.messages[0]["content"] = system_prompt
|
||||
else:
|
||||
session.messages.insert(0, {"role": "system", "content": system_prompt})
|
||||
saved_cwd = session.meta.get("cwd")
|
||||
if saved_cwd and console is not None and saved_cwd != str(tool_base):
|
||||
console.print(
|
||||
f"[warn]提示:[/warn] 当前 cwd 与 task 记录不同 —— "
|
||||
f"工具基于 current cwd,不会自动切回。\n"
|
||||
f" task cwd: [info]{saved_cwd}[/info]\n"
|
||||
f" current cwd: [info]{tool_base}[/info]"
|
||||
)
|
||||
task_state = TaskState.load(task_dir)
|
||||
session = Session.load(task_id, system_prompt=system_prompt, meta=meta)
|
||||
task_state = TaskState.load(task_id)
|
||||
if task_state is None:
|
||||
# messages.json 存在但 state.json 缺失:用 session.meta 兜底重建
|
||||
# tasks 行不存在 —— 理论上 resolve_task_id 已经定位到 DB 行了,走到这里
|
||||
# 说明被并发删了,兜底构造空 state(不主动 save,等下条 append / 命令)
|
||||
task_state = TaskState(
|
||||
task_id=sid,
|
||||
mode=mode,
|
||||
description=description,
|
||||
status="active",
|
||||
model=session.meta.get("model", caps.model_id),
|
||||
model_profile=session.meta.get("model_profile", model),
|
||||
cwd=session.meta.get("cwd", str(tool_base)),
|
||||
created_at=session.meta.get("created_at", datetime.now().isoformat(timespec="seconds")),
|
||||
task_id=sid, name="", working_dir=wd_db,
|
||||
skill=skill, description=description, status="active",
|
||||
model=caps.model_id, model_profile=model,
|
||||
)
|
||||
task_state.save(task_dir)
|
||||
# resume 时 meta name 用 DB 里读出来的真值(给 Session.append → ensure 用,避免落空串)
|
||||
meta["name"] = task_state.name
|
||||
else:
|
||||
now_iso = datetime.now().isoformat(timespec="seconds")
|
||||
meta = {
|
||||
"id": sid,
|
||||
"created_at": now_iso,
|
||||
"cwd": str(tool_base),
|
||||
"model": caps.model_id,
|
||||
"model_profile": model,
|
||||
}
|
||||
session = Session(system_prompt=system_prompt, path=session_path, meta=meta)
|
||||
# 懒创建:不预占文件。首条 user 消息触发 Session.append → save() 才会 mkdir + 落盘。
|
||||
# task_state 同步推迟到首轮 sync_task_tokens。直到那一刻为止,task_dir 在磁盘上不存在。
|
||||
session = Session(task_id=task_id, system_prompt=system_prompt, meta=meta)
|
||||
# 懒创建:TaskState 仅内存。tasks 行在首条 user 消息 append 时由
|
||||
# ensure_local_task_row 占位 INSERT(name 已就位);首次 sync_task_tokens
|
||||
# 或 /done /desc 走 upsert 覆盖完整字段。
|
||||
task_state = TaskState(
|
||||
task_id=sid,
|
||||
mode=mode,
|
||||
description=description,
|
||||
status="active",
|
||||
model=caps.model_id,
|
||||
model_profile=model,
|
||||
task_id=sid, name=task_name_safe, working_dir=wd_db,
|
||||
skill=skill, description=description, status="active",
|
||||
model=caps.model_id, model_profile=model,
|
||||
reasoning_effort=caps.default_reasoning_effort or "",
|
||||
cwd=str(tool_base),
|
||||
created_at=now_iso,
|
||||
)
|
||||
|
||||
tools = {}
|
||||
|
|
@ -199,12 +316,22 @@ def build_agent(
|
|||
|
||||
sink = ConsoleEventSink(console, token_counter=lambda: llm.token_counter.total) if console else None
|
||||
agent = AgentLoop(llm, tools, session, caps, sink=sink)
|
||||
return agent, session, sid, task_state, task_dir
|
||||
return agent, session, sid, task_state, working_dir_path
|
||||
|
||||
|
||||
def sync_task_tokens(task_state: TaskState, task_dir: Path, llm: LLM) -> None:
|
||||
"""每轮 agent.run 后调,把 LLM 累计 tokens 写回 state.json。"""
|
||||
def sync_task_tokens(task_state: TaskState, llm: LLM) -> None:
|
||||
"""每轮 agent.run 后调,把 LLM 累计 tokens UPDATE 到 PG tasks 表。
|
||||
|
||||
走 update_task 而非 task_state.save() —— 只更 tokens 两列,避免无谓全字段 UPSERT
|
||||
且 ORM-level update 自动刷 updated_at。
|
||||
"""
|
||||
from uuid import UUID
|
||||
from core.storage import update_task
|
||||
tc = llm.token_counter
|
||||
task_state.tokens_prompt = tc.prompt_tokens
|
||||
task_state.tokens_completion = tc.completion_tokens
|
||||
task_state.save(task_dir)
|
||||
update_task(
|
||||
UUID(task_state.task_id),
|
||||
tokens_prompt=tc.prompt_tokens,
|
||||
tokens_completion=tc.completion_tokens,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -10,3 +10,14 @@ matplotlib>=3.8.0
|
|||
|
||||
# 素材摄取: PDF/DOCX/PPTX/XLSX/HTML/URL → Markdown (ppt 阶段零 + proposal 阶段零)
|
||||
markitdown[pdf,docx,pptx,xlsx]>=0.0.1
|
||||
|
||||
# §7 B 阶段: Storage 落 PG
|
||||
sqlalchemy>=2.0.0
|
||||
psycopg[binary]>=3.1.0
|
||||
alembic>=1.13.0
|
||||
|
||||
# §7 Phase G / D: 纯 JSON API(FastAPI + 原生 SSE),前端由 platform 提供
|
||||
fastapi>=0.111.0
|
||||
uvicorn[standard]>=0.30.0
|
||||
python-multipart>=0.0.9 # files upload multipart 解析
|
||||
pyjwt>=2.8.0 # /v1/auth/login HS256 token mint/verify(§7 D' 过渡形态)
|
||||
|
|
|
|||
|
After Width: | Height: | Size: 2.1 KiB |
|
After Width: | Height: | Size: 1.4 KiB |
|
After Width: | Height: | Size: 1.9 KiB |
|
After Width: | Height: | Size: 3.8 KiB |
|
After Width: | Height: | Size: 2.7 KiB |
|
After Width: | Height: | Size: 3.3 KiB |
|
Before Width: | Height: | Size: 4.3 KiB After Width: | Height: | Size: 3.6 KiB |
|
|
@ -0,0 +1,5 @@
|
|||
"""§7 Phase G: Web UI 简洁版(FastAPI + Jinja2 + HTMX + 原生 SSE)。
|
||||
|
||||
入口:`cli.py web` → `web.app.create_app()` → uvicorn 起。
|
||||
本地形态固定 sentinel user(无 auth);Phase D 加 OIDC 后才有真正用户态。
|
||||
"""
|
||||
|
|
@ -0,0 +1,827 @@
|
|||
"""FastAPI app: 纯 /v1 JSON API(2026-05-15 切换 — 详见 DESIGN §7.9)。
|
||||
|
||||
设计要点:
|
||||
- 所有路由 `/v1/*` 前缀,响应 JSON;模板 / HTMX / 服务端 markdown 渲染全删
|
||||
- SSE 事件 payload 是 JSON dict 而非 HTML 片段(`event: <type>` + `data: <json>`)
|
||||
- Auth: PLATFORM_KEY → JWT 兑换(§7 D' 过渡形态,见 web/auth.py);OIDC 替换时只动 /v1/auth/login 内部
|
||||
- 所有 /v1/tasks* 路由 Depends(require_user),按 user_id 隔离数据
|
||||
- 豁免:/healthz、/docs、/openapi.json、/、/v1/auth/login、/static/*
|
||||
- CORS allow_origins=["*"] 本地宽松;真发布按 platform 域名收紧
|
||||
- `GET /` 302 → /static/dev.html(本地 dev SPA)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime as _dt
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
from fastapi import Depends, FastAPI, File, Form, HTTPException, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, RedirectResponse, StreamingResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func, select, update
|
||||
from starlette.background import BackgroundTask
|
||||
|
||||
from core.paths import from_db_path, to_db_path
|
||||
from core.storage import (
|
||||
NoSubtaskError,
|
||||
check_no_subtask,
|
||||
session_scope,
|
||||
)
|
||||
from core.storage.models import Message, Run, Task
|
||||
from core.storage.utils import ensure_local_task_row
|
||||
|
||||
from .auth import AuthConfig, ensure_user_row, make_require_user, mint_token
|
||||
from .broker import broker
|
||||
from .sinks import WebEventSink
|
||||
|
||||
|
||||
STATUS_FILTERS = ("active", "completed", "abandoned")
|
||||
STATUS_WRITABLE = ("completed", "abandoned") # web 不让从 web 端切回 active(走 CLI)
|
||||
ORDER_FIELDS = ("created_at", "updated_at", "name", "status")
|
||||
ORDER_DEFAULT = "-created_at"
|
||||
|
||||
|
||||
# ─────────────────────────── helpers ───────────────────────────
|
||||
|
||||
def _norm_path(p: str) -> str:
|
||||
"""跨 OS 显示归一:backslash → forward slash。"""
|
||||
return (p or "").replace("\\", "/")
|
||||
|
||||
|
||||
def _iso(dt: Optional[Any]) -> Optional[str]:
|
||||
return dt.isoformat() if dt else None
|
||||
|
||||
|
||||
def _parse_ordering(s: Optional[str]) -> list:
|
||||
"""DRF 风格 `ordering` 解析:逗号分隔多字段,`-` 前缀代表 desc。
|
||||
|
||||
allowlist 见 `ORDER_FIELDS`;非法字段静默丢弃。全部非法或空串 → `ORDER_DEFAULT`(`-created_at`)。
|
||||
返回 sqlalchemy `order_by` 列表(可直接 `*expand`)。
|
||||
"""
|
||||
spec = (s or "").strip() or ORDER_DEFAULT
|
||||
cols = []
|
||||
for part in spec.split(","):
|
||||
p = part.strip()
|
||||
if not p:
|
||||
continue
|
||||
asc = True
|
||||
if p.startswith("-"):
|
||||
asc = False
|
||||
p = p[1:]
|
||||
if p in ORDER_FIELDS:
|
||||
col = getattr(Task, p)
|
||||
cols.append(col.asc() if asc else col.desc())
|
||||
if not cols:
|
||||
# 用户传了全无效字段 → fallback 默认
|
||||
cols = [Task.created_at.desc()]
|
||||
return cols
|
||||
|
||||
|
||||
def _task_dict(row: Any, *, n_messages: Optional[int] = None) -> dict:
|
||||
"""Task ORM row → API JSON dict。"""
|
||||
d = {
|
||||
"task_id": str(row.task_id),
|
||||
"name": row.name or "",
|
||||
"description": row.description or "",
|
||||
"working_dir": _norm_path(row.working_dir or ""),
|
||||
"status": row.status,
|
||||
"skill": row.skill or "",
|
||||
"model": row.model or "",
|
||||
"model_profile": row.model_profile or "",
|
||||
"tokens_prompt": row.tokens_prompt or 0,
|
||||
"tokens_completion": row.tokens_completion or 0,
|
||||
"tokens": (row.tokens_prompt or 0) + (row.tokens_completion or 0),
|
||||
"created_at": _iso(getattr(row, "created_at", None)),
|
||||
"updated_at": _iso(getattr(row, "updated_at", None)),
|
||||
}
|
||||
if n_messages is not None:
|
||||
d["n_messages"] = n_messages
|
||||
return d
|
||||
|
||||
|
||||
# ─────────────────────── files helpers ───────────────────────
|
||||
|
||||
def _load_working_dir(task_id: str, user_id: UUID) -> tuple[UUID, Path]:
|
||||
"""task_id 解析 + 查 PG 拿 working_dir db form + 还原 absolute Path。
|
||||
404 / 400 if 非 UUID / task 不存在 / 不属于 user / working_dir 空。
|
||||
跨 user 视为 not found(不暴露 task 存在性)。
|
||||
"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, f"invalid task id: {task_id!r}")
|
||||
with session_scope() as s:
|
||||
row = s.execute(
|
||||
select(Task.working_dir).where(
|
||||
Task.task_id == tid, Task.user_id == user_id
|
||||
)
|
||||
).first()
|
||||
if row is None:
|
||||
raise HTTPException(404, f"task not found: {tid}")
|
||||
wd = row[0] or ""
|
||||
if not wd:
|
||||
raise HTTPException(400, f"task {tid} has no working_dir, files browsing unavailable")
|
||||
return tid, from_db_path(wd)
|
||||
|
||||
|
||||
def _safe_join(root: Path, rel: str) -> Path:
|
||||
"""归一用户路径到 absolute,并校验仍在 root 内。防 `../` / 绝对 path / symlink 越界。"""
|
||||
rel = (rel or "").strip()
|
||||
if not rel:
|
||||
return root.resolve()
|
||||
if rel[0] in ("/", "\\"):
|
||||
raise HTTPException(400, f"absolute-style path not allowed: {rel!r}")
|
||||
if Path(rel).is_absolute():
|
||||
raise HTTPException(400, f"absolute path not allowed: {rel!r}")
|
||||
target = (root / rel).resolve()
|
||||
try:
|
||||
target.relative_to(root.resolve())
|
||||
except ValueError:
|
||||
raise HTTPException(400, f"path escapes working_dir: {rel!r}")
|
||||
return target
|
||||
|
||||
|
||||
def _rel_to(root: Path, target: Path) -> str:
|
||||
try:
|
||||
return target.resolve().relative_to(root.resolve()).as_posix()
|
||||
except ValueError:
|
||||
return ""
|
||||
|
||||
|
||||
def _enumerate_files(root: Path, current: Path) -> tuple[list[dict], list[dict], bool]:
|
||||
"""枚举 current 下条目 + 拼面包屑。size raw bytes,mtime ISO 串(前端 humanize)。"""
|
||||
entries: list[dict] = []
|
||||
exists = current.exists()
|
||||
if exists and current.is_dir():
|
||||
try:
|
||||
raw = sorted(current.iterdir(), key=lambda p: (p.is_file(), p.name.lower()))
|
||||
except OSError:
|
||||
raw = []
|
||||
for p in raw:
|
||||
try:
|
||||
st = p.stat()
|
||||
except OSError:
|
||||
continue
|
||||
entries.append({
|
||||
"name": p.name,
|
||||
"is_dir": p.is_dir(),
|
||||
"size": st.st_size if p.is_file() else None,
|
||||
"mtime": _dt.fromtimestamp(st.st_mtime).isoformat(timespec="seconds"),
|
||||
"rel": _rel_to(root, p),
|
||||
})
|
||||
cur_rel = _rel_to(root, current)
|
||||
crumbs = [{"label": "/", "rel": ""}]
|
||||
# cur_rel == "." 表示当前就在 root(target.relative_to(root) 返 Path(".")),
|
||||
# 不该再追加一个无意义的 "." crumb
|
||||
if cur_rel and cur_rel != ".":
|
||||
acc = ""
|
||||
for part in cur_rel.split("/"):
|
||||
acc = f"{acc}/{part}" if acc else part
|
||||
crumbs.append({"label": part, "rel": acc})
|
||||
return entries, crumbs, exists
|
||||
|
||||
|
||||
# ─────────────────── Run 启动 + SSE 帧格式 ───────────────────
|
||||
|
||||
def _run_agent_bg(task_id: UUID, run_id: UUID, user_id: UUID, user_message: str) -> None:
|
||||
"""工作线程:`build_agent(resume=True)` → 装 WebEventSink → `agent.run` → 写 runs 状态。
|
||||
|
||||
sink 通过 broker.emit 桥事件回 asyncio loop;agent.run 是 sync,所以在 to_thread 跑。
|
||||
user_id 必须从 JWT 那侧透传过来 —— 决定 memory_block 读哪个 per-user 子树。
|
||||
"""
|
||||
from main import build_agent, sync_task_tokens
|
||||
try:
|
||||
broker.emit(run_id, {"type": "run_start"})
|
||||
agent, session, sid, task_state, task_dir = build_agent(
|
||||
session_id=str(task_id), resume=True, user_id=user_id,
|
||||
)
|
||||
agent.sink = WebEventSink(broker, run_id)
|
||||
agent.run(user_message)
|
||||
sync_task_tokens(task_state, agent.llm)
|
||||
with session_scope() as s:
|
||||
s.execute(
|
||||
update(Run).where(Run.run_id == run_id).values(
|
||||
status="ok",
|
||||
finished_at=func.now(),
|
||||
tokens_p=agent.llm.token_counter.prompt_tokens,
|
||||
tokens_c=agent.llm.token_counter.completion_tokens,
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
err = f"{type(e).__name__}: {e}"
|
||||
broker.emit(run_id, {"type": "error", "msg": err})
|
||||
try:
|
||||
with session_scope() as s:
|
||||
s.execute(
|
||||
update(Run).where(Run.run_id == run_id).values(
|
||||
status="error", error=err, finished_at=func.now()
|
||||
)
|
||||
)
|
||||
except Exception:
|
||||
pass # 已 emit error 给前端,DB 写失败不放大噪声
|
||||
finally:
|
||||
broker.close(run_id)
|
||||
|
||||
|
||||
def _sse_event(event_type: str, payload: dict) -> bytes:
|
||||
"""格式化 SSE 一帧:`event: <type>` + `data: <json single-line>`。"""
|
||||
body = json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
|
||||
return f"event: {event_type}\ndata: {body}\n\n".encode("utf-8")
|
||||
|
||||
|
||||
# ────────────────────── Pydantic 请求体 ──────────────────────
|
||||
|
||||
class TaskCreateRequest(BaseModel):
|
||||
name: str # 任务显示名(必填,DB 列 NOT NULL)
|
||||
working_dir: str = "" # 工作目录名(可选,留空 → 用 name 作目录名)
|
||||
description: str = ""
|
||||
skill: str = ""
|
||||
|
||||
|
||||
class TaskPatchRequest(BaseModel):
|
||||
status: Optional[str] = None
|
||||
description: Optional[str] = None
|
||||
name: Optional[str] = None
|
||||
skill: Optional[str] = None
|
||||
|
||||
|
||||
class MessageRequest(BaseModel):
|
||||
content: str
|
||||
|
||||
|
||||
class FileDeleteRequest(BaseModel):
|
||||
path: str
|
||||
|
||||
|
||||
class LoginRequest(BaseModel):
|
||||
user_id: str
|
||||
platform_key: str
|
||||
|
||||
|
||||
# ────────────────────── App 工厂 ──────────────────────
|
||||
|
||||
# web/static 目录路径 — /static 静态挂载用,dev.html 也放这
|
||||
_STATIC_DIR = Path(__file__).parent / "static"
|
||||
|
||||
|
||||
def create_app() -> FastAPI:
|
||||
# fail-fast:env 缺失直接抛,不裸跑无密
|
||||
auth_cfg = AuthConfig.from_env()
|
||||
require_user = make_require_user(auth_cfg)
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
broker.bind_loop(asyncio.get_running_loop())
|
||||
yield
|
||||
|
||||
app = FastAPI(
|
||||
title="zcbot api",
|
||||
version="0.8",
|
||||
description=(
|
||||
"zcbot 后端 — /v1 JSON API + SSE。Auth: PLATFORM_KEY → JWT(§7 D' 过渡)。"
|
||||
"本地 dev SPA: /static/dev.html。"
|
||||
),
|
||||
lifespan=lifespan,
|
||||
)
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"], # 本地宽松,部署 platform 时按域名收紧
|
||||
allow_credentials=False,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
if _STATIC_DIR.is_dir():
|
||||
app.mount("/static", StaticFiles(directory=str(_STATIC_DIR)), name="static")
|
||||
|
||||
# ───────────── Misc ─────────────
|
||||
|
||||
@app.get("/", include_in_schema=False)
|
||||
def root():
|
||||
# 本地 dev SPA;Swagger UI 仍在 /docs
|
||||
return RedirectResponse(url="/static/dev.html", status_code=302)
|
||||
|
||||
@app.get("/healthz", tags=["misc"])
|
||||
def healthz():
|
||||
return {"status": "ok"}
|
||||
|
||||
# ───────────── Auth ─────────────
|
||||
|
||||
@app.post("/v1/auth/login", tags=["auth"])
|
||||
def login(body: LoginRequest):
|
||||
"""platform_key 校验通过 → 签 JWT(user_id 作为 sub)。
|
||||
|
||||
platform_key 错 → 403;user_id 非 UUID → 400。
|
||||
user_id 未存在则幂等创建 users 行(避免下游 FK 失败)。
|
||||
"""
|
||||
if body.platform_key != auth_cfg.platform_key:
|
||||
raise HTTPException(403, "invalid platform_key")
|
||||
try:
|
||||
uid = UUID(body.user_id)
|
||||
except (ValueError, TypeError):
|
||||
raise HTTPException(400, f"invalid user_id (must be UUID): {body.user_id!r}")
|
||||
ensure_user_row(uid)
|
||||
token, exp = mint_token(auth_cfg, uid)
|
||||
return {
|
||||
"token": token,
|
||||
"expires_at": _dt.fromtimestamp(exp).isoformat(),
|
||||
"user_id": str(uid),
|
||||
"ttl_seconds": auth_cfg.ttl_seconds,
|
||||
}
|
||||
|
||||
# ───────────── Tasks CRUD ─────────────
|
||||
|
||||
@app.post("/v1/tasks", status_code=201, tags=["tasks"])
|
||||
def create_task(body: TaskCreateRequest, user_id: UUID = Depends(require_user)):
|
||||
"""新建 task。
|
||||
|
||||
- `name` 必填(任务显示名,DB 列 NOT NULL,UI 列表 / 标题用)
|
||||
- `working_dir` 可选(留空 → 用 name 作目录名);同 working_dir 多 task 共享同目录(§7.1)
|
||||
- name / working_dir 都过 validate_task_name(简单名,无 `/\\..`,非 `.` 起头,≤255)
|
||||
- 前缀嵌套(no-subtask,同 user 内)→ 409
|
||||
"""
|
||||
from main import InvalidTaskName, resolve_workspace, validate_task_name, working_dir_from_name
|
||||
try:
|
||||
name = validate_task_name(body.name)
|
||||
except InvalidTaskName as e:
|
||||
raise HTTPException(400, f"name 不合法: {e}")
|
||||
# working_dir 留空 → fallback 用 name
|
||||
wd_raw = (body.working_dir or "").strip()
|
||||
wd_name = wd_raw if wd_raw else name
|
||||
try:
|
||||
wd_name = validate_task_name(wd_name)
|
||||
except InvalidTaskName as e:
|
||||
raise HTTPException(400, f"working_dir 不合法: {e}")
|
||||
description = body.description.strip()
|
||||
skill = body.skill.strip()
|
||||
|
||||
tid = uuid4()
|
||||
ws = resolve_workspace(None)
|
||||
fs_dir = working_dir_from_name(ws, user_id, wd_name)
|
||||
fs_dir_db = to_db_path(fs_dir)
|
||||
|
||||
try:
|
||||
check_no_subtask(fs_dir_db, user_id=user_id)
|
||||
except NoSubtaskError as e:
|
||||
raise HTTPException(409, str(e))
|
||||
|
||||
# 工作目录立刻建出(同 working_dir 多 task 共享,exist_ok=True)
|
||||
fs_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ensure_local_task_row(
|
||||
task_id=tid, name=name, working_dir=fs_dir_db, skill=skill,
|
||||
description=description, user_id=user_id,
|
||||
)
|
||||
with session_scope() as s:
|
||||
row = s.execute(select(Task).where(Task.task_id == tid)).scalar_one()
|
||||
return _task_dict(row, n_messages=0)
|
||||
|
||||
@app.get("/v1/tasks", tags=["tasks"])
|
||||
def list_tasks_route(
|
||||
page: int = 1,
|
||||
page_size: int = 20,
|
||||
status: Optional[str] = None,
|
||||
skill: Optional[str] = None,
|
||||
working_dir: Optional[str] = None,
|
||||
q: Optional[str] = None,
|
||||
ordering: Optional[str] = None,
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""列出当前 user 的 task,分页 + 多维筛选 + 排序。
|
||||
|
||||
- `page` ≥ 1(1-based);`page_size` 1–100(超界 clamp)
|
||||
- `status` 在 active/completed/abandoned;非法值静默忽略
|
||||
- `skill` 精确匹配(空忽略)
|
||||
- `working_dir` 末段目录名(如 `水泥申报`);后端自动拼 `workspace/users/<uid>/<name>` 比对
|
||||
- `q` 模糊搜索 name + description(ILIKE,大小写不敏感)
|
||||
- `ordering` DRF 风格,逗号分隔,`-field` 倒序;allowlist `created_at/updated_at/name/status`;
|
||||
非法字段静默忽略;**默认 `-created_at`**(创建时间倒序)
|
||||
返回标准分页壳 `{page, page_size, count, results}` —— count 供前端算总页数。
|
||||
"""
|
||||
# clamp + sanitize
|
||||
page = max(1, page)
|
||||
page_size = max(1, min(page_size, 100))
|
||||
status = status if status in STATUS_FILTERS else None
|
||||
skill = (skill or "").strip() or None
|
||||
wd_name = (working_dir or "").strip() or None
|
||||
q_text = (q or "").strip() or None
|
||||
|
||||
# 组装 WHERE
|
||||
conditions = [Task.user_id == user_id]
|
||||
if status:
|
||||
conditions.append(Task.status == status)
|
||||
if skill:
|
||||
conditions.append(Task.skill == skill)
|
||||
if wd_name:
|
||||
# 末段 → 完整 db form。同 working_dir 多 task 共享时,这是命中入口。
|
||||
wd_db = f"workspace/users/{user_id}/{wd_name}"
|
||||
conditions.append(Task.working_dir == wd_db)
|
||||
if q_text:
|
||||
pat = f"%{q_text}%"
|
||||
conditions.append(Task.name.ilike(pat) | Task.description.ilike(pat))
|
||||
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
with session_scope() as s:
|
||||
cnt = s.execute(
|
||||
select(func.count()).select_from(Task).where(*conditions)
|
||||
).scalar_one() or 0
|
||||
|
||||
rows = s.execute(
|
||||
select(Task).where(*conditions)
|
||||
.order_by(*_parse_ordering(ordering))
|
||||
.limit(page_size).offset(offset)
|
||||
).scalars().all()
|
||||
|
||||
tids = [r.task_id for r in rows]
|
||||
msg_counts = (
|
||||
dict(s.execute(
|
||||
select(Message.task_id, func.count())
|
||||
.where(Message.task_id.in_(tids))
|
||||
.group_by(Message.task_id)
|
||||
).all())
|
||||
if tids else {}
|
||||
)
|
||||
|
||||
return {
|
||||
"page": page,
|
||||
"page_size": page_size,
|
||||
"count": int(cnt),
|
||||
"results": [
|
||||
_task_dict(r, n_messages=msg_counts.get(r.task_id, 0))
|
||||
for r in rows
|
||||
],
|
||||
}
|
||||
|
||||
@app.get("/v1/tasks/{task_id}", tags=["tasks"])
|
||||
def get_task(task_id: str, user_id: UUID = Depends(require_user)):
|
||||
"""单 task meta(不含 messages;走 /messages 拿)。跨 user → 404。"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, f"invalid task id: {task_id!r}")
|
||||
with session_scope() as s:
|
||||
row = s.execute(
|
||||
select(Task).where(Task.task_id == tid, Task.user_id == user_id)
|
||||
).scalar_one_or_none()
|
||||
if row is None:
|
||||
raise HTTPException(404, f"task not found: {tid}")
|
||||
n = s.execute(
|
||||
select(func.count()).select_from(Message).where(Message.task_id == tid)
|
||||
).scalar_one()
|
||||
return _task_dict(row, n_messages=n)
|
||||
|
||||
@app.get("/v1/folders", tags=["folders"])
|
||||
def list_folders(user_id: UUID = Depends(require_user)):
|
||||
"""列出当前 user 的工作目录(`workspace/users/<uid>/` 下非 dotfile 子目录)。
|
||||
供新建 task 时自动补全 / 选已有目录用。FS 是 source of truth(也含手动创建
|
||||
但还无关联 task 的目录)。每项带 n_tasks(关联 task 数)+ last_used(最近使用 ISO)。
|
||||
排序:有 last_used 的按降序,无 last_used 的排最后,同列 by name asc。
|
||||
"""
|
||||
from main import resolve_workspace, user_root
|
||||
ws = resolve_workspace(None)
|
||||
root = user_root(ws, user_id)
|
||||
|
||||
folder_names: list[str] = []
|
||||
if root.is_dir():
|
||||
for p in sorted(root.iterdir(), key=lambda x: x.name.lower()):
|
||||
if p.is_dir() and not p.name.startswith("."):
|
||||
folder_names.append(p.name)
|
||||
|
||||
folders: list[dict] = []
|
||||
if folder_names:
|
||||
with session_scope() as s:
|
||||
for name in folder_names:
|
||||
db_form = f"workspace/users/{user_id}/{name}"
|
||||
stat = s.execute(
|
||||
select(func.count(), func.max(Task.updated_at))
|
||||
.where(Task.user_id == user_id, Task.working_dir == db_form)
|
||||
).first()
|
||||
n = int((stat[0] if stat else 0) or 0)
|
||||
lu = stat[1] if stat else None
|
||||
folders.append({
|
||||
"name": name,
|
||||
"n_tasks": n,
|
||||
"last_used": _iso(lu),
|
||||
})
|
||||
|
||||
folders.sort(key=lambda f: f["name"])
|
||||
folders.sort(key=lambda f: f["last_used"] or "", reverse=True)
|
||||
return {"folders": folders}
|
||||
|
||||
@app.delete("/v1/tasks/{task_id}", status_code=204, tags=["tasks"])
|
||||
def delete_task(task_id: str, user_id: UUID = Depends(require_user)):
|
||||
"""硬删除:DELETE DB 行(messages / runs CASCADE)。**FS task_dir 不动**
|
||||
(同 name 多 task 共享,文件由用户经 /files/delete 单独清)。跨 user → 404。
|
||||
"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, f"invalid task id: {task_id!r}")
|
||||
from sqlalchemy import delete as _delete
|
||||
with session_scope() as s:
|
||||
result = s.execute(
|
||||
_delete(Task).where(Task.task_id == tid, Task.user_id == user_id)
|
||||
)
|
||||
if result.rowcount == 0:
|
||||
raise HTTPException(404, f"task not found: {tid}")
|
||||
return None # 204
|
||||
|
||||
@app.patch("/v1/tasks/{task_id}", tags=["tasks"])
|
||||
def patch_task(
|
||||
task_id: str,
|
||||
body: TaskPatchRequest,
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""更新 task 字段。`status` 仅允许 completed/abandoned(active 走 CLI 切回)。"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, f"invalid task id: {task_id!r}")
|
||||
updates: dict[str, Any] = {}
|
||||
if body.status is not None:
|
||||
if body.status not in STATUS_WRITABLE:
|
||||
raise HTTPException(
|
||||
400, f"invalid status {body.status!r}; allowed: {STATUS_WRITABLE}"
|
||||
)
|
||||
updates["status"] = body.status
|
||||
if body.description is not None:
|
||||
updates["description"] = body.description
|
||||
if body.skill is not None:
|
||||
updates["skill"] = body.skill
|
||||
if body.name is not None:
|
||||
from main import InvalidTaskName, validate_task_name
|
||||
try:
|
||||
updates["name"] = validate_task_name(body.name)
|
||||
except InvalidTaskName as e:
|
||||
raise HTTPException(400, f"name 不合法: {e}")
|
||||
if not updates:
|
||||
raise HTTPException(400, "no fields to update")
|
||||
with session_scope() as s:
|
||||
result = s.execute(
|
||||
update(Task)
|
||||
.where(Task.task_id == tid, Task.user_id == user_id)
|
||||
.values(**updates)
|
||||
)
|
||||
if result.rowcount == 0:
|
||||
raise HTTPException(404, f"task not found: {tid}")
|
||||
row = s.execute(select(Task).where(Task.task_id == tid)).scalar_one()
|
||||
n = s.execute(
|
||||
select(func.count()).select_from(Message).where(Message.task_id == tid)
|
||||
).scalar_one()
|
||||
return _task_dict(row, n_messages=n)
|
||||
|
||||
# ───────────── Messages ─────────────
|
||||
|
||||
def _assert_owns_task(s, tid: UUID, user_id: UUID) -> None:
|
||||
ok = s.execute(
|
||||
select(Task.task_id).where(Task.task_id == tid, Task.user_id == user_id)
|
||||
).first()
|
||||
if ok is None:
|
||||
raise HTTPException(404, f"task not found: {tid}")
|
||||
|
||||
@app.get("/v1/tasks/{task_id}/messages", tags=["messages"])
|
||||
def list_messages(task_id: str, user_id: UUID = Depends(require_user)):
|
||||
"""task 历史消息(idx 升序);LiteLLM 原 payload 透传给前端,自行渲染。"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, f"invalid task id: {task_id!r}")
|
||||
with session_scope() as s:
|
||||
_assert_owns_task(s, tid, user_id)
|
||||
rows = s.execute(
|
||||
select(
|
||||
Message.idx, Message.payload, Message.tokens_in,
|
||||
Message.tokens_out, Message.created_at,
|
||||
).where(Message.task_id == tid).order_by(Message.idx)
|
||||
).all()
|
||||
return {
|
||||
"messages": [
|
||||
{
|
||||
"idx": r.idx,
|
||||
"payload": dict(r.payload),
|
||||
"tokens_in": r.tokens_in,
|
||||
"tokens_out": r.tokens_out,
|
||||
"created_at": _iso(r.created_at),
|
||||
}
|
||||
for r in rows
|
||||
]
|
||||
}
|
||||
|
||||
@app.post("/v1/tasks/{task_id}/messages", status_code=202, tags=["messages"])
|
||||
async def post_message(
|
||||
task_id: str,
|
||||
body: MessageRequest,
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""发消息 + 起 BG run。返 `{run_id, events_url}`,客户端立刻订阅 SSE 拿流式。"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, f"invalid task id: {task_id!r}")
|
||||
content = (body.content or "").strip()
|
||||
if not content:
|
||||
raise HTTPException(400, "empty content")
|
||||
with session_scope() as s:
|
||||
_assert_owns_task(s, tid, user_id)
|
||||
|
||||
run_id = uuid4()
|
||||
with session_scope() as s:
|
||||
s.add(Run(run_id=run_id, task_id=tid, status="running", started_at=func.now()))
|
||||
# to_thread 跑 sync agent.run;sink 通过 broker 把 event 桥回 asyncio
|
||||
asyncio.create_task(asyncio.to_thread(_run_agent_bg, tid, run_id, user_id, content))
|
||||
return {
|
||||
"run_id": str(run_id),
|
||||
"events_url": f"/v1/tasks/{tid}/runs/{run_id}/events",
|
||||
}
|
||||
|
||||
# ───────────── SSE events ─────────────
|
||||
|
||||
@app.get("/v1/tasks/{task_id}/runs/{run_id}/events", tags=["runs"])
|
||||
async def stream_events(
|
||||
task_id: str,
|
||||
run_id: str,
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""SSE 流。事件类型:run_start / llm_start / text / tool_call / tool_result /
|
||||
llm_end / error / done。data 是 JSON dict(已剔除 `type` 字段,移到 event 名)。
|
||||
"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
rid = UUID(run_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, "invalid id")
|
||||
with session_scope() as s:
|
||||
_assert_owns_task(s, tid, user_id)
|
||||
|
||||
async def gen():
|
||||
q = broker.subscribe(rid)
|
||||
try:
|
||||
# 第一帧 retry 注释 + 心跳:让 EventSource 立即建立(不被 buffer 卡)
|
||||
yield b": connected\nretry: 3000\n\n"
|
||||
while True:
|
||||
try:
|
||||
ev = await asyncio.wait_for(q.get(), timeout=30.0)
|
||||
except asyncio.TimeoutError:
|
||||
yield b": ping\n\n"
|
||||
continue
|
||||
ev_type = ev.get("type", "msg")
|
||||
payload = {k: v for k, v in ev.items() if k != "type"}
|
||||
yield _sse_event(ev_type, payload)
|
||||
if ev_type in ("done", "error"):
|
||||
break
|
||||
except asyncio.CancelledError:
|
||||
pass # 客户端断开,静默退
|
||||
finally:
|
||||
broker.unsubscribe(rid, q)
|
||||
|
||||
return StreamingResponse(
|
||||
gen(),
|
||||
media_type="text/event-stream",
|
||||
headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"X-Accel-Buffering": "no",
|
||||
},
|
||||
)
|
||||
|
||||
# ───────────── Files ─────────────
|
||||
|
||||
@app.get("/v1/tasks/{task_id}/files", tags=["files"])
|
||||
def list_files(
|
||||
task_id: str,
|
||||
path: str = "",
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""列子目录条目 + 面包屑。`path` 留空 → root;`../` / 绝对 → 400。"""
|
||||
tid, root = _load_working_dir(task_id, user_id)
|
||||
current = _safe_join(root, path)
|
||||
entries, crumbs, exists = _enumerate_files(root, current)
|
||||
return {
|
||||
"task_id": str(tid),
|
||||
"root": _norm_path(str(root)),
|
||||
"current": _rel_to(root, current),
|
||||
"exists": exists,
|
||||
"crumbs": crumbs,
|
||||
"entries": entries,
|
||||
}
|
||||
|
||||
@app.get("/v1/tasks/{task_id}/files/download", tags=["files"])
|
||||
def download_file(
|
||||
task_id: str,
|
||||
path: str,
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""下载单个 regular file(目录 → 400 / 不存在 → 404)。"""
|
||||
tid, root = _load_working_dir(task_id, user_id)
|
||||
target = _safe_join(root, path)
|
||||
if not target.exists():
|
||||
raise HTTPException(404, f"file not found: {path}")
|
||||
if not target.is_file():
|
||||
raise HTTPException(400, f"not a file: {path}")
|
||||
return FileResponse(path=str(target), filename=target.name)
|
||||
|
||||
@app.post("/v1/tasks/{task_id}/files/upload", tags=["files"])
|
||||
async def upload_files(
|
||||
task_id: str,
|
||||
path: str = Form(""),
|
||||
files: list[UploadFile] = File(...),
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""multipart 多文件上传到 `<task_dir>/<path>/`。
|
||||
路径不存在自动 mkdir(parents=True);重名直接覆盖。
|
||||
文件名严格校验(含 `/ \\ ..` 或为空 → 400)。
|
||||
"""
|
||||
tid, root = _load_working_dir(task_id, user_id)
|
||||
dest_dir = _safe_join(root, path)
|
||||
if dest_dir.exists() and not dest_dir.is_dir():
|
||||
raise HTTPException(400, f"upload target is a file, not a directory: {path}")
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
saved: list[dict] = []
|
||||
for up in files or []:
|
||||
raw_name = up.filename or ""
|
||||
if (
|
||||
not raw_name
|
||||
or raw_name in (".", "..")
|
||||
or "/" in raw_name or "\\" in raw_name
|
||||
or any(part in (".", "..") for part in Path(raw_name).parts)
|
||||
):
|
||||
raise HTTPException(400, f"invalid filename: {raw_name!r}")
|
||||
dest = dest_dir / raw_name
|
||||
try:
|
||||
dest.resolve().relative_to(root.resolve())
|
||||
except ValueError:
|
||||
raise HTTPException(400, f"path escapes task_dir: {raw_name!r}")
|
||||
data = await up.read()
|
||||
dest.write_bytes(data)
|
||||
saved.append({"name": raw_name, "size": len(data), "rel": _rel_to(root, dest)})
|
||||
if not saved:
|
||||
raise HTTPException(400, "no files uploaded")
|
||||
return {"count": len(saved), "saved": saved}
|
||||
|
||||
@app.post("/v1/tasks/{task_id}/files/delete", tags=["files"])
|
||||
def delete_file(
|
||||
task_id: str,
|
||||
body: FileDeleteRequest,
|
||||
user_id: UUID = Depends(require_user),
|
||||
):
|
||||
"""删 task_dir 下文件或**空**目录。非空目录 → 400(避免误操);root → 400。"""
|
||||
tid, root = _load_working_dir(task_id, user_id)
|
||||
target = _safe_join(root, body.path)
|
||||
if target.resolve() == root.resolve():
|
||||
raise HTTPException(400, "cannot delete task_dir root")
|
||||
if not target.exists():
|
||||
raise HTTPException(404, f"path not found: {body.path}")
|
||||
try:
|
||||
if target.is_dir():
|
||||
target.rmdir() # 非空目录会触发 OSError
|
||||
else:
|
||||
target.unlink()
|
||||
except OSError as e:
|
||||
raise HTTPException(400, f"delete failed: {e}")
|
||||
return {"ok": True, "path": body.path}
|
||||
|
||||
# ───────────── Export ─────────────
|
||||
|
||||
@app.get("/v1/tasks/{task_id}/export", tags=["export"])
|
||||
def export_task(task_id: str, user_id: UUID = Depends(require_user)):
|
||||
"""导出对话为 .docx,临时文件下载完后 BackgroundTask 删 tmp。"""
|
||||
try:
|
||||
tid = UUID(task_id)
|
||||
except ValueError:
|
||||
raise HTTPException(404, f"invalid task id: {task_id!r}")
|
||||
with session_scope() as s:
|
||||
_assert_owns_task(s, tid, user_id)
|
||||
has_msg = s.execute(
|
||||
select(Message.message_id).where(Message.task_id == tid).limit(1)
|
||||
).first()
|
||||
if not has_msg:
|
||||
raise HTTPException(400, "no messages to export")
|
||||
|
||||
fd, tmp_str = tempfile.mkstemp(suffix=".docx", prefix="zcbot-export-")
|
||||
os.close(fd)
|
||||
tmp_path = Path(tmp_str)
|
||||
try:
|
||||
from core.export_docx import export_chat_to_docx
|
||||
export_chat_to_docx(tid, out_path=tmp_path)
|
||||
except Exception as e:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
raise HTTPException(500, f"export failed: {type(e).__name__}: {e}")
|
||||
|
||||
return FileResponse(
|
||||
path=str(tmp_path),
|
||||
media_type="application/vnd.openxmlformats-officedocument.wordprocessingml.document",
|
||||
filename=f"chat_{str(tid)[:8]}.docx",
|
||||
background=BackgroundTask(tmp_path.unlink, missing_ok=True),
|
||||
)
|
||||
|
||||
return app
|
||||
|
|
@ -0,0 +1,138 @@
|
|||
"""Auth: PLATFORM_KEY → JWT token 兑换(§7 D' 过渡形态)。
|
||||
|
||||
模型:
|
||||
- `PLATFORM_KEY` env(必填)是 platform/本仓库间的共享密钥;platform 服务端 / dev 页持有它
|
||||
- `JWT_SECRET` env(必填)用于 HS256 签 token;泄漏 = 任意伪造,与 PLATFORM_KEY 同级保护
|
||||
- `POST /v1/auth/login {user_id, platform_key}` → `{token, expires_at}`(后端校验 key 对 → 签 JWT)
|
||||
- 后续 `/v1/*`(除 /healthz、/docs、/openapi.json、/、/v1/auth/login)走 `Authorization: Bearer <jwt>`
|
||||
- Token TTL: `ZCBOT_JWT_TTL_SECONDS` env 覆盖,默 7 天
|
||||
|
||||
OIDC(D')替换:只动 `/v1/auth/login` 实现(校验 ID token 代替 key),路由层 Depends 不变。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from typing import Optional
|
||||
from uuid import UUID
|
||||
|
||||
import jwt
|
||||
from fastapi import Depends, HTTPException, Request
|
||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||
|
||||
from core.storage import session_scope
|
||||
from core.storage.models import SENTINEL_USER_ID, User
|
||||
|
||||
|
||||
_DEFAULT_TTL_SECONDS = 7 * 24 * 3600 # 7d
|
||||
|
||||
|
||||
class AuthConfig:
|
||||
"""App 启动时一次性读 env + 校验存在性;create_app 调 `AuthConfig.from_env()` 拿到。"""
|
||||
|
||||
def __init__(self, platform_key: str, jwt_secret: str, ttl_seconds: int):
|
||||
self.platform_key = platform_key
|
||||
self.jwt_secret = jwt_secret
|
||||
self.ttl_seconds = ttl_seconds
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> "AuthConfig":
|
||||
key = os.environ.get("PLATFORM_KEY", "").strip()
|
||||
secret = os.environ.get("JWT_SECRET", "").strip()
|
||||
missing = []
|
||||
if not key:
|
||||
missing.append("PLATFORM_KEY")
|
||||
if not secret:
|
||||
missing.append("JWT_SECRET")
|
||||
if missing:
|
||||
raise RuntimeError(
|
||||
f"{', '.join(missing)} env not set. zcbot web requires both:\n"
|
||||
" PLATFORM_KEY=<shared secret between platform and zcbot>\n"
|
||||
" JWT_SECRET=<HMAC secret used to sign session tokens>"
|
||||
)
|
||||
ttl_raw = os.environ.get("ZCBOT_JWT_TTL_SECONDS", "").strip()
|
||||
try:
|
||||
ttl = int(ttl_raw) if ttl_raw else _DEFAULT_TTL_SECONDS
|
||||
except ValueError:
|
||||
raise RuntimeError(
|
||||
f"ZCBOT_JWT_TTL_SECONDS must be int seconds, got {ttl_raw!r}"
|
||||
)
|
||||
if ttl <= 0:
|
||||
raise RuntimeError(f"ZCBOT_JWT_TTL_SECONDS must be > 0, got {ttl}")
|
||||
return cls(platform_key=key, jwt_secret=secret, ttl_seconds=ttl)
|
||||
|
||||
|
||||
def mint_token(cfg: AuthConfig, user_id: UUID) -> tuple[str, int]:
|
||||
"""签 JWT。返回 `(token, exp_unix_seconds)`。"""
|
||||
now = int(time.time())
|
||||
exp = now + cfg.ttl_seconds
|
||||
payload = {"sub": str(user_id), "iat": now, "exp": exp}
|
||||
token = jwt.encode(payload, cfg.jwt_secret, algorithm="HS256")
|
||||
return token, exp
|
||||
|
||||
|
||||
def verify_token(cfg: AuthConfig, token: str) -> UUID:
|
||||
"""验签 + 取 sub。失败抛 HTTPException 401。"""
|
||||
try:
|
||||
payload = jwt.decode(token, cfg.jwt_secret, algorithms=["HS256"])
|
||||
except jwt.ExpiredSignatureError:
|
||||
raise HTTPException(401, "token expired")
|
||||
except jwt.InvalidTokenError as e:
|
||||
raise HTTPException(401, f"invalid token: {e}")
|
||||
sub = payload.get("sub", "")
|
||||
try:
|
||||
return UUID(sub)
|
||||
except (ValueError, TypeError):
|
||||
raise HTTPException(401, f"invalid sub in token: {sub!r}")
|
||||
|
||||
|
||||
def ensure_user_row(user_id: UUID) -> None:
|
||||
"""幂等 INSERT 一行 users 占位(`ON CONFLICT DO NOTHING`)。
|
||||
|
||||
dev 用 SENTINEL,platform 注入的 user_id 也走这条 — 无论是新用户首次登录还是
|
||||
既有用户复登,都安全。真用户 profile(email/oidc_subject 等)在 D' OIDC 阶段
|
||||
再走专门的 register/sync 路径。
|
||||
"""
|
||||
from sqlalchemy.dialects.postgresql import insert
|
||||
stmt = insert(User).values(user_id=user_id).on_conflict_do_nothing(
|
||||
index_elements=["user_id"]
|
||||
)
|
||||
with session_scope() as s:
|
||||
s.execute(stmt)
|
||||
|
||||
|
||||
# ──────────────── FastAPI Depends ────────────────
|
||||
# auto_error=False 让我们自己出 401 文案,而不是 FastAPI 默认 "Not authenticated"
|
||||
_bearer = HTTPBearer(auto_error=False)
|
||||
|
||||
|
||||
def make_require_user(cfg: AuthConfig):
|
||||
"""工厂:返回一个 Depends 函数,闭包持有 cfg(避免 app 启动后改 env)。
|
||||
|
||||
用法:
|
||||
require_user = make_require_user(cfg)
|
||||
@app.get("/v1/...", dependencies=[Depends(require_user)])
|
||||
def route(user_id: UUID = Depends(require_user)):
|
||||
...
|
||||
实际使用建议直接 `user_id: UUID = Depends(require_user)`,既验签又拿到 user_id。
|
||||
"""
|
||||
async def require_user(
|
||||
creds: Optional[HTTPAuthorizationCredentials] = Depends(_bearer),
|
||||
) -> UUID:
|
||||
if creds is None or not creds.credentials:
|
||||
raise HTTPException(401, "missing Authorization: Bearer <token>")
|
||||
if creds.scheme.lower() != "bearer":
|
||||
raise HTTPException(401, f"unsupported auth scheme: {creds.scheme!r}")
|
||||
return verify_token(cfg, creds.credentials)
|
||||
|
||||
return require_user
|
||||
|
||||
|
||||
__all__ = [
|
||||
"AuthConfig",
|
||||
"SENTINEL_USER_ID",
|
||||
"ensure_user_row",
|
||||
"make_require_user",
|
||||
"mint_token",
|
||||
"verify_token",
|
||||
]
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
"""RunBroker:in-process pub/sub,把 agent run 产生的 event fan-out 给所有 SSE 订阅者。
|
||||
|
||||
设计:
|
||||
- emit() 从工作线程调(agent.run 在 to_thread 跑),用 loop.call_soon_threadsafe
|
||||
桥到 asyncio queue;SSE generator await queue.get() 拉出来推流。
|
||||
- 同一 run_id 多个订阅者(刷新页面 / 多 tab / 桌面+移动)— 每个订阅 1 个独立 queue。
|
||||
- run 结束 → broker.close(run_id) 给所有订阅者派一条 done;新订阅者(在 done 后到的)
|
||||
立即收到 done 并断流(不漏不挂)。
|
||||
- 进程内单实例 / 多进程不共享 — 个人 SaaS 单 worker 够用;真要扩多 worker 再上 Redis。
|
||||
- 不持久化 event — messages 已落 PG,刷新页面走 G3 静态视图能看历史;真要"刷新继续看
|
||||
实时流"未来加 event log 表 + backfill。
|
||||
|
||||
线程模型:
|
||||
- broker.bind_loop(loop) 在 FastAPI startup 调一次,记录 asyncio loop 引用。
|
||||
- emit() 调用方可能在任意线程;put_nowait 是 thread-unsafe(asyncio.Queue 设计前提
|
||||
是单 loop),所以走 call_soon_threadsafe 跨回 loop 线程再 put。
|
||||
- subscribe / unsubscribe / close 也都用 call_soon_threadsafe 包,避免 race
|
||||
(实测 SSE generator 在 finally 里 unsubscribe,这个就在 loop 线程,直接调也行)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from collections import defaultdict
|
||||
from typing import Any, Optional
|
||||
from uuid import UUID
|
||||
|
||||
|
||||
class RunBroker:
|
||||
def __init__(self) -> None:
|
||||
self._subs: dict[UUID, set[asyncio.Queue]] = defaultdict(set)
|
||||
# 已经发完 done 的 run — 后来订阅者直接收到 done,避免无限等
|
||||
self._done: set[UUID] = set()
|
||||
self._loop: Optional[asyncio.AbstractEventLoop] = None
|
||||
|
||||
def bind_loop(self, loop: asyncio.AbstractEventLoop) -> None:
|
||||
"""FastAPI startup 调一次。"""
|
||||
self._loop = loop
|
||||
|
||||
def subscribe(self, run_id: UUID) -> asyncio.Queue:
|
||||
"""订阅 run 的 event 流。已 done 的 run 立刻在 queue 放一条 done。
|
||||
|
||||
调用方:SSE handler(在 asyncio loop 线程内)。
|
||||
"""
|
||||
q: asyncio.Queue = asyncio.Queue()
|
||||
if run_id in self._done:
|
||||
q.put_nowait({"type": "done"})
|
||||
else:
|
||||
self._subs[run_id].add(q)
|
||||
return q
|
||||
|
||||
def unsubscribe(self, run_id: UUID, q: asyncio.Queue) -> None:
|
||||
"""SSE generator finally 清理。"""
|
||||
self._subs.get(run_id, set()).discard(q)
|
||||
if run_id in self._subs and not self._subs[run_id]:
|
||||
del self._subs[run_id]
|
||||
|
||||
def emit(self, run_id: UUID, event: dict[str, Any]) -> None:
|
||||
"""从工作线程调:把 event 推给所有订阅者。
|
||||
|
||||
如果没人订阅(run 在跑但没浏览器连上),event 丢弃 — 这是设计选择
|
||||
(event 不持久化,messages 走 PG)。
|
||||
"""
|
||||
loop = self._loop
|
||||
if loop is None:
|
||||
return # 还没 bind,丢弃(测试 / 启动竞态)
|
||||
for q in list(self._subs.get(run_id, [])):
|
||||
loop.call_soon_threadsafe(q.put_nowait, event)
|
||||
|
||||
def close(self, run_id: UUID) -> None:
|
||||
"""run 结束:派 done 给所有订阅者,标记 run_id 为已完成。
|
||||
|
||||
从工作线程调(agent.run 完成 / 抛异常 finally 清理)。
|
||||
"""
|
||||
self.emit(run_id, {"type": "done"})
|
||||
self._done.add(run_id)
|
||||
# subs 不在这里立即删 — SSE generator 会先收到 done、yield 它、走到
|
||||
# finally unsubscribe;此处 emit 后立即删会让那次 emit 之后的清理无的放矢。
|
||||
|
||||
def n_subscribers(self, run_id: UUID) -> int:
|
||||
"""供测试 / 监控用。"""
|
||||
return len(self._subs.get(run_id, set()))
|
||||
|
||||
def is_done(self, run_id: UUID) -> bool:
|
||||
return run_id in self._done
|
||||
|
||||
|
||||
# 进程内单例 — FastAPI lifespan 里 bind_loop;agent / sink / SSE handler 共享。
|
||||
broker = RunBroker()
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
"""WebEventSink:实现 §7 A 的 sink 协议,把 AgentLoop.emit 桥到 RunBroker。
|
||||
|
||||
每次 run 一个 sink 实例,绑死 run_id。`emit({type, ...})` 直接转 broker.emit(run_id, event)。
|
||||
sink 实例由 web 层在启 run 时创建,传进 AgentLoop;loop 完全不知 web 存在(§5 Less Scaffolding)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from .broker import RunBroker
|
||||
|
||||
|
||||
class WebEventSink:
|
||||
def __init__(self, broker: RunBroker, run_id: UUID) -> None:
|
||||
self._broker = broker
|
||||
self._run_id = run_id
|
||||
|
||||
def emit(self, event: dict[str, Any]) -> None:
|
||||
self._broker.emit(self._run_id, event)
|
||||