From e9e549b606a6e2845daa53d6c9687b841b85749b Mon Sep 17 00:00:00 2001 From: caoqianming Date: Tue, 7 Jul 2026 15:42:20 +0800 Subject: [PATCH] =?UTF-8?q?feat(web):=20=E8=AF=AD=E9=9F=B3=E8=BE=93?= =?UTF-8?q?=E5=85=A5=E6=B5=81=E5=BC=8F=E5=8C=96:=E8=BE=B9=E8=AF=B4?= =?UTF-8?q?=E8=BE=B9=E5=87=BA=E5=AD=97,=E6=9D=BE=E6=89=8B=E7=A7=92?= =?UTF-8?q?=E5=87=BA=E6=9C=80=E7=BB=88=E6=96=87=E6=9C=AC(bump=200.49.0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 根因:整段模式按 40ms/帧回放音频给讯飞 = 按录音时长等比例白等(16.8s 实测等 4.6s)。 - 新增 WS /v1/asr/stream 流式代理:首消息 JWT 鉴权(不走 query 防进 access log), 二进制帧实时转发讯飞(dwa=wpgs),动态修正(sn/pgs/rg)在服务端 XfyunStream 合并, 回推 {"text","final"} 增量全文;引擎提前收尾(静音/60s)推 final 前端自动结束 - 前端:录音即连 WS,增量降采样(carry 跨片连续)实时上行,录音面板实时字幕; WS 失败静默退回整段 POST 兜底。实测 16.8s 音频 end→final 179ms(原 ~2.5s+) - 整段兜底路径帧间隔 40ms→8ms(实测 4.6s→1.75s,识别文本一致) - nginx 示例加 /v1/asr/stream WS Upgrade 段;RUN 故障兜底补讯飞 IP 白名单 403 与 反代挡 WS 两条 Co-Authored-By: Claude Fable 5 --- PROGRESS.md | 1 + RUN.md | 5 +- core/__init__.py | 2 +- core/asr_xfyun.py | 152 ++++++++++++++++++++++++++++++-- deploy/nginx/zcbot.conf.example | 17 ++++ web/app.py | 77 +++++++++++++++- web/static/dev.html | 22 +++-- web/static/js/chat.js | 119 +++++++++++++++++++++---- 8 files changed, 364 insertions(+), 31 deletions(-) diff --git a/PROGRESS.md b/PROGRESS.md index b47c82a..2125435 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -23,6 +23,7 @@ ### 2026-07 +- **07-07 / 0.49.0**:语音输入流式化(边说边出字,消掉松手后停顿)——用户报"录完停顿几秒才出字",根因:整段模式按讯飞文档 40ms/帧节奏回放音频 = 按录音时长等比例白等(16.8s 音频实测等 4.6s)。改双通道:① **流式主通道** `WS /v1/asr/stream`——录音开始即连(首消息 JWT 鉴权,不走 query 防进 access log),前端增量降采样(carry 缓冲跨片连续)实时上行,服务端 `XfyunStream` 转发讯飞(dwa=wpgs)并**在服务端合并动态修正**(sn/pgs/rg),回推 partial 全文,录音面板实时字幕,松手只等最终帧——实测 16.8s 音频 **end→final 179ms**(61 个增量帧,文本与整段一致);引擎提前收尾(静音 10s/60s 上限)推 final 前端自动结束。② **整段 POST 兜底**(WS 连不上/中途挂/超时),帧间隔实测调优 40ms→8ms(4.6s→1.75s,文本一致,已到讯飞处理地板)。nginx 示例加 `location = /v1/asr/stream` WS Upgrade 段(不配不坏,退兜底);踩坑:讯飞控制台 IP 白名单(握手 403 "Your IP address is not allowed"),进 RUN 故障兜底。 - **07-07 / 0.48.1**:语音手势两端统一(用户提出 PC 也要能按住说话)——不再按 pointerType 分流,改按**按下时长**判别,PC 鼠标/手机触屏同一心智模型:按住 ≥300ms = 按住说话(松开即转写,上滑取消);快速点一下 <300ms = 转成连续录音免持(说完 Enter / 再点结束,Esc 取消;鼠标按住 60s 会累且易失手,长口述场景必须保留免持)。实现要点:press 发起的录音松手后合成 click 全压掉(防"松开转写完又开新录音"/"短按刚转免持被自己 click 停掉"),"点按钮停止连续录音"的 press 不压(`_pressActive` 区分);getUserMedia 完成前松手按同一时长规则收尾;键盘激活按钮(无 pointer 事件)兜底走 click 起录。 - **07-07 / 0.48.0**:语音输入手势升级(纯前端,后端零改动)——业界对齐调研后两端分治:① **触屏按住说话**(微信/DeepSeek 式):`pointerdown`(pointerType≠mouse)按住录音、松开转写、上滑 60px 进入"松开取消"(面板变红),pointer capture 保证滑出按钮仍收事件,touch 后合成 click 用时间窗压掉;② **桌面点击开停 + Enter/Esc 收尾**:说完按 Enter 结束转写、Esc 取消(capture 拦截防落回 textarea 触发发送)。**明确不做桌面 VAD 静音自动停**:口述需求"边想边说"停顿 3~5s 是常态,自动截断比多按一下更伤,业界听写类(ChatGPT/Google Docs/Win+H)也全是手动收尾,自动停只属于 Siri 式短指令场景。③ 录音期 `#voice-rec-panel` 替换 textarea:实时波浪(采集 chunk 顺手算 RMS,1/8 抽样,零额外开销)+ 计时 + 手势提示;iOS Safari 手势内 `ctx.resume()`;授权弹窗期间松手 → 启动完成立即收尾。 - **07-07 / 0.47.0**:web 端语音输入(讯飞语音听写 IAT)——聊天输入区新增「🎙 语音」按钮:点击录音(Web Audio ScriptProcessor 采 PCM → 前端线性插值降采样 16k/16bit/mono)→ 再点停止 → `POST /v1/asr/transcribe`(裸 PCM body,require_user)→ 后端 `core/asr_xfyun.py` 连讯飞 wss(HMAC-SHA256 签名,9600B/帧 40ms 节奏,vad_eos=10000 防句间停顿截尾,未开 dwa 整段拼接)→ 文本 insertText 插输入框光标处(接原生 undo 栈,**不自动发送**)。60s 上限自动停(讯飞单会话限制);录音需 HTTPS/localhost;渠道只读镜像 task 按钮锁定;`XFYUN_APPID/API_KEY/API_SECRET` env 未配返 501(**当前 .env 为测试 key,待换正式**)。诊断 `scripts/diag_asr.py`(静音冒烟 / 带 PCM 文件真测);requirements 显式加 `websockets`。已实测:TTS 合成中文 5.2s 音频经端点转写逐字正确。企微语音消息(AMR 需 ffmpeg 转码)留待二期;将来流式逐字上屏可复用 `build_auth_url` 签临时 URL。 diff --git a/RUN.md b/RUN.md index febe8f0..096c120 100644 --- a/RUN.md +++ b/RUN.md @@ -232,7 +232,8 @@ curl --noproxy '*' -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8765/v1/ta | `POST /v1/tasks/{id}/cancel` | 协作式 cancel;`run_status != running` → 409;LLM 走 streaming,chunk 间 poll cancel — 延迟 100ms 级,基本秒退 | 必填 | | `POST /v1/tasks/{id}/clear` | 清空当前 task 全部 messages + reset `tasks.tokens_prompt/completion/cost_cny` 三列累计 + `run_status='idle'`;`usage_events`(账单记账)**不动**,只 `message_id` 列变 NULL;run 活跃中(running/cancelling)→ 409(先 cancel);FS 文件保留 | 必填 | | `POST /v1/tasks/{id}/optimize_prompt` | body `{text(req, ≤4000), image_model?=""}`;同步调当前 task model 润色草稿,返 `{optimized, model_profile, tokens_in, tokens_out, cost_cny}`;**不**写 messages、**不**累计 task 三列(顶栏数字不污染),只在 `usage_events` 写一行 `kind="prompt_optimize"`(对账可见);不与主对话 run 互斥(允许 streaming 中并行润色) | 必填 | -| `POST /v1/asr/transcribe` | body 为裸 PCM(16kHz/16bit/单声道/小端,`application/octet-stream`)→ 讯飞 IAT 整段转写,返 `{text}`;>60s → 413;`XFYUN_*` env 未配 → 501;讯飞侧错误 → 502(带错误码提示)。前端「🎙 语音」按钮用 | 必填 | +| `WS /v1/asr/stream` | 流式语音转写(前端「🎙 语音」主通道):连上后首条文本帧发 `{"token":""}`(WS 塞不了 header,token 不走 query 防进 access log,失败 close 4401)→ 之后二进制帧 = PCM 分片(16k/16bit/mono)实时转发讯飞;任意文本帧(约定 `{"type":"end"}`)= 说完。服务端回推 `{"text","final"}` 增量全文(wpgs 动态修正已合并),final=true 为最终结果;错误回 `{"error"}` 后关闭。nginx 反代该路径需支持 WS Upgrade(`proxy_set_header Upgrade/Connection`) | 首消息 | +| `POST /v1/asr/transcribe` | body 为裸 PCM(16kHz/16bit/单声道/小端,`application/octet-stream`)→ 讯飞 IAT 整段转写,返 `{text}`;>60s → 413;`XFYUN_*` env 未配 → 501;讯飞侧错误 → 502(带错误码提示)。流式 WS 连不上时前端的兜底通道 | 必填 | | `GET /v1/files?path=` | 列 user_root 下条目 + 面包屑;dotfile 隐藏 | 必填 | | `GET /v1/files/download?path=` | 下单文件 | 必填 | | `POST /v1/files/upload` | multipart 上传到 `//`;路径不存在自动 mkdir,重名覆盖 | 必填 | @@ -824,6 +825,8 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_" /opt | `NoSubtaskError: working_dir ... 前缀嵌套` | §7.4 no-subtask:同 user 不允许 working_dir 嵌套(child / parent)。**同项目多对话**用**完全相同**的 working_dir;否则改成 sibling(平级) | | `main.py web` 启动后 curl 连不上 | 检查 proxy(`HTTP_PROXY` / `HTTPS_PROXY`):本地服务 127.0.0.1,系统 proxy 拦截会 502。临时 `unset HTTP_PROXY HTTPS_PROXY` 或 `curl --noproxy '*'`。验通:`curl --noproxy '*' http://127.0.0.1:8765/healthz` | | 「🎙 语音」点了没反应 / 提示"此环境不支持录音" | 浏览器只在**安全上下文**(HTTPS 或 localhost)开放麦克风;http://IP 访问必现。上 HTTPS 或本机 localhost 试。凭据/链路问题跑 `scripts/diag_asr.py` 定位(11200=讯飞服务未开通或量用完,10105/10313=appid 错) | +| 讯飞握手 403 `Your IP address is not allowed` | 讯飞控制台该应用开了 **IP 白名单**,当前机器出口 IP 不在名单(服务器好用、开发机忽然全挂就是这个)。console.xfyun.cn → 应用 → 语音听写 → IP 白名单加出口 IP 或关闭 | +| 语音输入没有实时字幕 / 松手后仍要等几秒 | 流式 WS(`/v1/asr/stream`)被反代挡了(nginx 未透传 Upgrade 头),前端静默退回整段 POST 兜底 —— 功能不坏但慢。nginx 加 WS 反代段(`deploy/nginx/zcbot.conf.example` 已带 `location = /v1/asr/stream`),`nginx -t && systemctl reload nginx` | | SSE 卡住不流(经 nginx) | 反代要关 buffering — 后端响应头已带 `X-Accel-Buffering: no`(nginx ≥1.5.6 默认认)。仍卡看 nginx 配 `proxy_buffering off; proxy_read_timeout 3600s;` | | platform CORS preflight 失败 | 本地 dev `allow_origins=["*"]` 应该没事;部署后看是否按 platform 域名收紧过头 | | `POST /v1/tasks/{id}/messages` 返 409 `task already has an active run` | 上一条消息的 BG run 还没跑完;等流式 done 或点 stop / `POST .../cancel`;服务异常下 `run_status` 卡 `running`/`cancelling`,启动 reaper 会清 | diff --git a/core/__init__.py b/core/__init__.py index d5e6571..4b8c221 100644 --- a/core/__init__.py +++ b/core/__init__.py @@ -1,3 +1,3 @@ # zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。 # 改版本只动这一行。 -__version__ = "0.48.1" +__version__ = "0.49.0" diff --git a/core/asr_xfyun.py b/core/asr_xfyun.py index e8e7143..886b9bb 100644 --- a/core/asr_xfyun.py +++ b/core/asr_xfyun.py @@ -5,9 +5,11 @@ - 音频: 仅收 16kHz/16bit/单声道 PCM(raw),单会话上限 60 秒 - 凭据: env XFYUN_APPID / XFYUN_API_KEY / XFYUN_API_SECRET(换正式 key 只改 .env) -本模块只做"整段 PCM → 最终文本"(web 语音输入按钮用);未开 dwa 动态修正 —— -那是实时逐字上屏才需要的,整段转写直接按序拼接 result.ws 即可。将来做流式 -上屏时复用 build_auth_url() 签临时 URL 给前端直连。 +两条使用路径: +- `XfyunStream`:流式会话(web /v1/asr/stream 代理用)—— 边录边喂 PCM 分片,开 + dwa=wpgs 动态修正,**服务端**合并 sn/pgs/rg 增量,partial 全文经回调推给前端, + 松手后只等最终帧,消掉"整段回放"的等待。 +- `transcribe()`:整段 PCM → 最终文本(POST /v1/asr/transcribe,流式失败的兜底)。 """ from __future__ import annotations @@ -30,10 +32,12 @@ BYTES_PER_SECOND = SAMPLE_RATE * 2 # 16bit mono MAX_SECONDS = 60 # 讯飞单会话硬上限 MAX_PCM_BYTES = MAX_SECONDS * BYTES_PER_SECOND -# 每帧 9600B(=300ms 音频),base64 后 12800B < 讯飞单帧上限 13000B; -# 帧间 40ms 为文档推荐节奏,整段上传比实时快 7.5 倍(60s 音频 ≈ 8s 传完) +# 每帧 9600B(=300ms 音频),base64 后 12800B < 讯飞单帧上限 13000B。 +# 帧间隔:文档推荐 40ms 是给实时麦克风流的节奏;整段转写(transcribe 兜底路径)实测 +# 2026-07-07(16.8s 音频):40ms→4.6s / 20ms→2.6s / 8ms→1.75s / 0ms→1.77s(识别文本 +# 全一致,0ms 也不报错)—— 取 8ms:已到讯飞处理耗时地板,再压没有收益。 _FRAME_BYTES = 9600 -_FRAME_INTERVAL = 0.04 +_FRAME_INTERVAL = 0.008 _RECV_TIMEOUT = 30 # 单次等响应上限;讯飞 10s 无数据也会主动断 # 常见错误码 → 给用户/运维看的中文提示(全量见文档错误码页) @@ -89,6 +93,142 @@ def _load_credentials() -> tuple[str, str, str]: return appid, api_key, api_secret +class XfyunStream: + """流式会话:实时喂 PCM 分片,partial 全文经 on_text 异步回调推出(wpgs 已合并)。 + + 用法(web /v1/asr/stream 代理): + st = XfyunStream(on_text=cb) # await cb(text, final) 从内部 recv task 调 + await st.start() + await st.feed(chunk); ... # 16k/16bit/mono PCM,任意大小(内部按帧上限分片) + text = await st.finish() # 发 status=2,等最终文本 + await st.close() # 幂等清理(finally 里调) + 引擎可能先于 finish 收尾(静音超 vad_eos / 60s 上限)→ on_text(text, True) 已推, + 此后 feed 静默丢弃,finish 直接返缓存结果。 + """ + + def __init__(self, on_text, *, language: str = "zh_cn"): + self._on_text = on_text + self._language = language + self._ws = None + self._recv_task: asyncio.Task | None = None + self._segs: dict[int, str] = {} # sn → 该段文本;pgs=rpl 按 rg 区间删旧段 + self._done = asyncio.Event() + self._error: XfyunASRError | None = None + self._final_text: str | None = None + self._first = True + self._appid = "" + + async def start(self) -> None: + self._appid, api_key, api_secret = _load_credentials() + import websockets + + try: + self._ws = await websockets.connect( + build_auth_url(api_key, api_secret), open_timeout=10 + ) + except Exception as e: + raise XfyunASRError(f"讯飞连接失败:{type(e).__name__}: {e}") + self._recv_task = asyncio.create_task(self._recv_loop()) + + def _assemble(self, result: dict) -> str: + """合并一帧 wpgs 结果,返回当前完整文本。""" + sn = result.get("sn") or 0 + if result.get("pgs") == "rpl": + a, b = (result.get("rg") or [sn, sn])[:2] + for k in [k for k in self._segs if a <= k <= b]: + del self._segs[k] + self._segs[sn] = "".join( + cw.get("w", "") + for w in (result.get("ws") or []) + for cw in (w.get("cw") or []) + ) + return "".join(self._segs[k] for k in sorted(self._segs)).strip() + + async def _recv_loop(self) -> None: + try: + while True: + msg = json.loads(await self._ws.recv()) + code = msg.get("code") + if code: + hint = _ERR_HINTS.get(code, msg.get("message") or "未知错误") + raise XfyunASRError(f"讯飞识别失败({code}):{hint}", code=code) + data = msg.get("data") or {} + text = self._assemble(data.get("result") or {}) + final = data.get("status") == 2 + if final: + self._final_text = text + await self._on_text(text, final) + if final: + return + except XfyunASRError as e: + self._error = e + except asyncio.CancelledError: + raise + except Exception as e: # 连接中断 / 回调方(客户端 ws)挂了 + self._error = XfyunASRError(f"讯飞连接中断:{type(e).__name__}: {e}") + finally: + self._done.set() + + def _frame(self, chunk: bytes) -> str: + frame: dict = {"data": { + "status": 0 if self._first else 1, + "format": f"audio/L16;rate={SAMPLE_RATE}", + "encoding": "raw", + "audio": base64.b64encode(chunk).decode(), + }} + if self._first: + frame["common"] = {"app_id": self._appid} + frame["business"] = { + "language": self._language, + "domain": "iat", + "accent": "mandarin", + "vad_eos": 10000, + "ptt": 1, + "dwa": "wpgs", # 动态修正:边说边出字,回改在服务端 _assemble 消化 + } + self._first = False + return json.dumps(frame) + + async def feed(self, chunk: bytes) -> None: + if self._done.is_set() or not chunk: + return # 引擎已收尾(提前结束/出错)→ 静默丢弃 + try: + for off in range(0, len(chunk), _FRAME_BYTES): + await self._ws.send(self._frame(chunk[off : off + _FRAME_BYTES])) + except Exception: + pass # 发送失败由 recv loop 报连接错误,这里不重复抛 + + async def finish(self, timeout: float = 15) -> str: + if not self._done.is_set(): + try: + if self._first: + await self._ws.send(self._frame(b"")) # 一帧音频都没喂过:先补参数帧 + await self._ws.send(json.dumps({"data": { + "status": 2, + "format": f"audio/L16;rate={SAMPLE_RATE}", + "encoding": "raw", + "audio": "", + }})) + except Exception: + pass + try: + await asyncio.wait_for(self._done.wait(), timeout=timeout) + except asyncio.TimeoutError: + raise XfyunASRError("讯飞识别超时(结束后无最终结果)") + if self._error: + raise self._error + return self._final_text or "" + + async def close(self) -> None: + if self._recv_task and not self._recv_task.done(): + self._recv_task.cancel() + with suppress(asyncio.CancelledError): + await self._recv_task + if self._ws: + with suppress(Exception): + await self._ws.close() + + async def transcribe(pcm: bytes, *, language: str = "zh_cn") -> str: """整段 PCM(16k/16bit/mono, 小端)→ 识别文本。空音频/纯静音返回 ""。""" appid, api_key, api_secret = _load_credentials() diff --git a/deploy/nginx/zcbot.conf.example b/deploy/nginx/zcbot.conf.example index 54a3f5d..58f77ca 100644 --- a/deploy/nginx/zcbot.conf.example +++ b/deploy/nginx/zcbot.conf.example @@ -41,6 +41,23 @@ server { # 上传接口(/v1/files/upload)会传大文件;与后端无硬限制,给足余量 client_max_body_size 512m; + # ★ WebSocket(流式语音转写):必须透传 Upgrade/Connection 握手头,否则前端 + # 连不上 WS、自动退回整段 POST 兜底(功能不坏但没了实时字幕 + 松手秒出)。 + # 单独 location 精确匹配,不影响主 location 的 keepalive(Connection "")。 + location = /v1/asr/stream { + proxy_pass http://zcbot_backend; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_buffering off; + proxy_read_timeout 300s; # 单次录音 ≤60s + 余量;别沿用 3600 占着连接 + proxy_send_timeout 300s; + } + location / { proxy_pass http://zcbot_backend; proxy_http_version 1.1; diff --git a/web/app.py b/web/app.py index f72b650..33947e0 100644 --- a/web/app.py +++ b/web/app.py @@ -28,7 +28,16 @@ try: except ImportError: # pragma: no cover - Windows resource = None -from fastapi import Depends, FastAPI, File, Form, HTTPException, Request, UploadFile +from fastapi import ( + Depends, + FastAPI, + File, + Form, + HTTPException, + Request, + UploadFile, + WebSocket, +) from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import FileResponse, RedirectResponse, StreamingResponse from pydantic import BaseModel @@ -57,6 +66,7 @@ from .auth import ( make_require_user, mint_token, resolve_user_by_email, + verify_token, ) from .admin import register_admin_routes from .broker import broker @@ -2883,6 +2893,71 @@ def create_app() -> FastAPI: raise HTTPException(502, str(e)) return {"text": text} + @app.websocket("/v1/asr/stream") + async def asr_stream(ws: WebSocket): + """流式语音转写代理:浏览器边录边推,服务端实时转发讯飞并回推 partial 全文。 + + 协议(浏览器 WebSocket 塞不了 Authorization header;token 走首消息而非 URL + query,避免进 access log): + 1. client → 文本帧 `{"token": ""}`(连上后 10s 内) + 2. client → 二进制帧 = PCM 分片(16kHz/16bit/mono/小端,实时节奏); + 文本帧(任意内容,约定 `{"type":"end"}`)= 说完 + 3. server → `{"text","final"}` 增量全文(wpgs 动态修正已在服务端合并,前端 + 直接整体替换显示);final=true 为最终结果,随后正常关闭 + 错误:`{"error": msg}` 后关闭;鉴权失败 close code 4401。引擎提前收尾 + (静音超 10s / 60s 上限)同样推 final=true,前端应就此收尾。 + """ + from core.asr_xfyun import XfyunASRError, XfyunStream + + await ws.accept() + try: + first = await asyncio.wait_for(ws.receive_json(), timeout=10) + verify_token(auth_cfg, str((first or {}).get("token") or "")) + except HTTPException as e: + try: + await ws.send_json({"error": f"auth: {e.detail}"}) + except Exception: + pass + await ws.close(code=4401) + return + except Exception: # 非 JSON 首帧 / 超时 / 断开 + try: + await ws.close(code=4401) + except Exception: + pass + return + + async def push(text: str, final: bool): + await ws.send_json({"text": text, "final": final}) + + stream = XfyunStream(on_text=push) + try: + try: + await stream.start() # 未配 XFYUN_* / 连不上讯飞 → 直接报错关闭 + except XfyunASRError as e: + await ws.send_json({"error": str(e)}) + await ws.close() + return + while True: + # 90s 上限兜底(讯飞 60s + 静音余量):客户端挂着不说话不结束也不无限占用 + msg = await asyncio.wait_for(ws.receive(), timeout=90) + if msg.get("type") == "websocket.disconnect": + return # 客户端取消/刷新:直接丢弃会话 + if msg.get("bytes"): + await stream.feed(msg["bytes"]) + elif msg.get("text") is not None: + break # 唯一文本指令:end + try: + await stream.finish() # 最终文本已由 recv loop 经 push(final=True) 推出 + except XfyunASRError as e: + await ws.send_json({"error": str(e)}) + await ws.close() + except Exception: + # 客户端断开 / 发送失败等:静默收尾(recv loop 的错误已尽力 push 过) + pass + finally: + await stream.close() + # ───────────── SSE events ───────────── @app.get("/v1/tasks/{task_id}/events", tags=["tasks"]) diff --git a/web/static/dev.html b/web/static/dev.html index e3ca7c0..b2013c0 100644 --- a/web/static/dev.html +++ b/web/static/dev.html @@ -894,16 +894,21 @@ #chat-mic { touch-action: none; -webkit-user-select: none; user-select: none; } #chat-mic.recording { color: #fff; background: #d33; border-color: #d33; animation: mic-pulse 1.6s ease-in-out infinite; } @keyframes mic-pulse { 50% { opacity: .65; } } - /* 录音面板:录音期替换 textarea(波浪 + 计时 + 手势提示);上滑取消预备时整体变红 */ - #voice-rec-panel { display: none; align-items: center; gap: 10px; height: 72px; padding: 0 12px; + /* 录音面板:录音期替换 textarea(波浪 + 计时 + 手势提示 + 实时字幕);上滑取消预备时整体变红 */ + #voice-rec-panel { display: none; flex-direction: column; gap: 2px; padding: 6px 12px; border: 1px solid #d33; border-radius: 6px; background: #fff5f5; } #chat-form.recording #voice-rec-panel { display: flex; } #chat-form.recording #chat-input { display: none; } + #voice-rec-panel .vr-row { display: flex; align-items: center; gap: 10px; height: 48px; } #voice-rec-panel .vr-time { font-variant-numeric: tabular-nums; font-weight: 600; color: #d33; } - #voice-rec-panel canvas { flex: 1; height: 44px; min-width: 0; } + #voice-rec-panel canvas { flex: 1; height: 40px; min-width: 0; } #voice-rec-panel .vr-hint { font-size: 11px; color: var(--muted); white-space: nowrap; } + /* 实时字幕:流式转写边说边出字;空时不占位(流式不可用退回整段模式时无字幕) */ + #voice-rec-panel .vr-live { font-size: 13px; line-height: 1.5; max-height: 60px; overflow-y: auto; } + #voice-rec-panel .vr-live:empty { display: none; } #voice-rec-panel.cancel-armed { background: #d33; border-color: #b22; } - #voice-rec-panel.cancel-armed .vr-time, #voice-rec-panel.cancel-armed .vr-hint { color: #fff; } + #voice-rec-panel.cancel-armed .vr-time, #voice-rec-panel.cancel-armed .vr-hint, + #voice-rec-panel.cancel-armed .vr-live { color: #fff; } #chat-form .hint { font-size: 11px; color: var(--muted); } /* 附件托盘:粘贴 / 拖拽的文件 chip 独占一行,累积展示,不与状态文字抢容器 */ #chat-attach { display: none; flex-wrap: wrap; gap: 4px 0; align-items: center; } @@ -1574,9 +1579,12 @@