806 行
48 KiB
Markdown
806 行
48 KiB
Markdown
|
|
# C4 - 横切关注点(错误处理 / 日志 / 可观测性 / 性能 / 一致性)
|
|||
|
|
|
|||
|
|
> 审计范围(实际 Read / Grep 过的文件):
|
|||
|
|
> `main.py`、`backend/dto/common.py`、`backend/routes/common.py`、`backend/route_registry.py`、
|
|||
|
|
> `backend/services/worker_service.py`、`backend/services/queue_service.py`、`backend/services/log_service.py`、
|
|||
|
|
> `backend/services/cost_service.py`、`backend/services/event_stream_service.py`、`backend/services/dashboard_service.py`、
|
|||
|
|
> `backend/services/generation_service.py`、`backend/services/advanced_material_service.py`、
|
|||
|
|
> `backend/services/advanced_style_set_service.py`、`backend/services/material_service.py`、
|
|||
|
|
> `backend/services/api_discovery_service.py`、`backend/services/settings_service.py`、
|
|||
|
|
> `backend/services/chat_session_service.py`、`backend/services/chat_workflow_service.py`、
|
|||
|
|
> `backend/services/integration_token_service.py`、`backend/services/workflow_round_worker_runtime.py`、
|
|||
|
|
> `backend/services/advanced_material_worker_runtime.py`、
|
|||
|
|
> `core/generation_errors.py`、`core/identity.py`、`core/audit_service.py`、`core/event_bus.py`、`core/remote_fetch.py`、
|
|||
|
|
> `core/interop.py`、`core/astrbot_provider_bridge.py`、
|
|||
|
|
> `database/db_helper.py`、`database/repositories/job_repository.py`、
|
|||
|
|
> `database/repositories/advanced_material_repository.py`、`database/migrations/*.sql`、
|
|||
|
|
> `adapters/astrbot_adapter.py`、`adapters/webui_adapter.py`、
|
|||
|
|
> `engines/image_provider.py`、
|
|||
|
|
> `frontend/shared/api/client.ts`、`frontend/standalone-web/src/**`(grep 全量)
|
|||
|
|
>
|
|||
|
|
> 已按要求完全跳过 `.tmp*/`、`release/`、`node_modules/`、`__pycache__/`、`web/assets/`、`pages/imagestudio/assets/`。
|
|||
|
|
|
|||
|
|
## 摘要
|
|||
|
|
|
|||
|
|
本仓库在**局部**做得相当克制:时间处理完全统一为 aware UTC(`utcnow()` 出现 0 次、裸 `datetime.now()` 出现 0 次)、ID 生成统一用 `前缀_uuid4().hex`、没有 `except:` 裸捕获、没有同步 `requests`/`time.sleep`/`subprocess` 混进 async、文件句柄都走 `Path.read_bytes` / `with` 上下文管理器。真正的问题集中在**"看不见"和"扛不住"**两条线上。
|
|||
|
|
|
|||
|
|
最致命的是可观测性接近于零:178 个 Python 源文件中只有 `worker_service.py` 一个文件配置了 logger(全仓 6 处 `logger.` 调用);同时 200+ 个 API 端点全部经由 `handle_request` 吞掉非 `ApiError` 异常,只回一个 `{"exception": "KeyError"}`,既不打日志也不留栈——一旦线上出内部错误,没有任何手段定位。后端生成的 `request_id` 又在前端 `unwrapApiResponse` 中被剥掉,聊天侧展示的 `chatreq_xxx` 更是从未写入任何存储,三条关联链路全断。
|
|||
|
|
|
|||
|
|
其次是 `ApiError` 被定义成继承 `Exception` 的 `@dataclass`,导致 `str(exc)` 返回 `args` 元组的 repr(含完整 `details` 字典);`main.py` 有 40 处直接把 `{exc}` 拼进聊天回复,用户看到的是 Python 元组而非中文错误信息。
|
|||
|
|
|
|||
|
|
性能上,SQLite 未开启 WAL 却跑 13 个并发 worker 线程,且每次 DB 操作都新建连接(全仓 151 处 `.connect()`),空转时约 43 次 connect/秒;`cost_service` 对 `generation_images` 做无 LIMIT 全表扫描后再在 Python 里过滤日期;`advanced_material_service` 存在按 attempt 循环查子表的 N+1。
|
|||
|
|
|
|||
|
|
## 发现
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P1-严重] 全后端 178 个源文件只有 1 个配置了日志,生产环境完全不可观测
|
|||
|
|
|
|||
|
|
- **位置**:`backend/services/worker_service.py:5,17`(全仓唯一 logger)
|
|||
|
|
- **问题**:对整个真实源码树做统计,`import logging` / `getLogger` 只出现在 `worker_service.py` 一处,全仓 `logger.` 调用共 **6 次**,全部是 `logger.exception`,全部在这一个文件里。`main.py`(121 KB)、`backend/routes/*`(32 个文件)、`backend/services/*`(60+ 个文件)、`database/`、`engines/`、`core/`、`adapters/` **零日志**。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
$ grep -rn "import logging|getLogger|logger\." --include="*.py" main.py backend core database engines adapters
|
|||
|
|
backend/services/worker_service.py:5:import logging
|
|||
|
|
backend/services/worker_service.py:17:logger = logging.getLogger(__name__)
|
|||
|
|
backend/services/worker_service.py:198: logger.exception(
|
|||
|
|
backend/services/worker_service.py:235: logger.exception(
|
|||
|
|
backend/services/worker_service.py:253: logger.exception(
|
|||
|
|
backend/services/worker_service.py:290: logger.exception(
|
|||
|
|
backend/services/worker_service.py:337: logger.exception(
|
|||
|
|
backend/services/worker_service.py:352: logger.exception(
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
作为对照,全仓有 **180 处 `except Exception`**,其中 **33 处**直接 `pass` / `return None` / `continue`。
|
|||
|
|
- **影响**:只有队列 worker 崩溃这一条路径会在 AstrBot 日志里留痕。生成失败、素材识别失败、配置写入失败、媒体签名失败、Provider 探测失败全部无声。用户报"点了没反应"时,运维手上除了 `logs/errors` 里那条 `error_message` 文本外没有任何上下文(无栈、无入参、无 provider、无 request_id)。这也是下面 P1-2、P2-1、P2-2 三条问题无法自愈的根因。
|
|||
|
|
- **修复建议**:
|
|||
|
|
1. 新建 `core/logging.py`,导出 `get_logger(name)`,内部优先桥接 AstrBot 的 `astrbot.api.logger`,回退到 `logging.getLogger(f"imagestudio.{name}")`;
|
|||
|
|
2. 在 `backend/routes/common.py:handle_request` 的 `except` 中强制 `logger.exception(...)`(见下条);
|
|||
|
|
3. 至少给 `generation_service`、`advanced_material_service`、`advanced_style_set_service`、`model_provider_runtime`、`astrbot_provider_bridge` 这 5 个含外部 IO 的模块补上 `logger.warning/exception`,把当前 33 处静默 `pass` 全部改成 `logger.debug/warning` + 原有兜底逻辑。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P1-严重] `handle_request` 吞掉全部非 ApiError 异常且不记日志,200+ 端点内部错误彻底静默
|
|||
|
|
|
|||
|
|
- **位置**:`backend/routes/common.py:12-16`、`backend/dto/common.py:75-83`
|
|||
|
|
- **问题**:全部 32 个 route 文件、166 条路由(`backend/route_registry.py:55-342` 的 `ROUTES`)都通过 `handle_request` 包装 handler。它 `except Exception` 后交给 `response_from_exception`,后者对任何非 `ApiError` 异常一律返回固定文案 + **仅异常类名**,栈、消息、上下文全部丢弃,且**没有任何日志输出**。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/routes/common.py:12-16
|
|||
|
|
def handle_request(ctx: RequestContext, callback: Callable[[], Any]) -> dict[str, Any]:
|
|||
|
|
try:
|
|||
|
|
return success_response(callback(), request_id=ctx.request_id)
|
|||
|
|
except Exception as exc:
|
|||
|
|
return response_from_exception(exc, request_id=ctx.request_id)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/dto/common.py:75-83
|
|||
|
|
def response_from_exception(exc: Exception, request_id: str | None = None) -> dict[str, Any]:
|
|||
|
|
if isinstance(exc, ApiError):
|
|||
|
|
return exc.to_response(request_id=request_id)
|
|||
|
|
return error_response(
|
|||
|
|
error_code=ErrorCode.UNKNOWN_ERROR,
|
|||
|
|
message="服务器内部错误",
|
|||
|
|
details={"exception": exc.__class__.__name__}, # 只有类名
|
|||
|
|
request_id=request_id,
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
各 route 文件的 `handle_request` 使用次数(合计 200+ 处):`advanced_material_routes.py:20`、`material_routes.py:17`、`settings_routes.py:12`、`job_routes.py:11`、`image_routes.py:10`、`node_routes.py:10`……
|
|||
|
|
- **影响**:任何 `KeyError` / `TypeError` / `sqlite3.OperationalError` / `json.JSONDecodeError` 都变成前端一句"服务器内部错误",服务端没有任何记录。配合 P1-1(全仓无日志),这类 bug 在生产中**不可能被诊断**,只能靠复现。`log_service.py` 提供的 `logs/errors` 端点只覆盖 `generation_tasks` 和 `integration_logs` 两张表,完全不含 API 层错误。
|
|||
|
|
- **修复建议**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/routes/common.py
|
|||
|
|
logger = get_logger(__name__)
|
|||
|
|
|
|||
|
|
def handle_request(ctx: RequestContext, callback: Callable[[], Any]) -> dict[str, Any]:
|
|||
|
|
try:
|
|||
|
|
return success_response(callback(), request_id=ctx.request_id)
|
|||
|
|
except ApiError as exc:
|
|||
|
|
logger.info("api business error request_id=%s code=%s", ctx.request_id, exc.error_code)
|
|||
|
|
return exc.to_response(request_id=ctx.request_id)
|
|||
|
|
except Exception:
|
|||
|
|
logger.exception("api internal error request_id=%s actor=%s", ctx.request_id, ctx.actor_id)
|
|||
|
|
return response_from_exception(exc, request_id=ctx.request_id)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
同时在 `response_from_exception` 的 `details` 中补 `{"request_id": request_id}`,让前端展示的 ID 可以直接反查日志。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P1-严重] `ApiError` 是继承 Exception 的 dataclass,`str(exc)` 泄漏完整 details 元组;main.py 有 40 处直接回显给聊天用户
|
|||
|
|
|
|||
|
|
- **位置**:`backend/dto/common.py:29-33`(定义)、`main.py:515,544,574,...,1591,1618,1646`(共 40 处 `{exc}`)
|
|||
|
|
- **问题**:`ApiError` 用 `@dataclass(frozen=True)` 继承 `Exception`。`Exception.__new__` 会把构造时的位置参数塞进 `self.args`,而 `Exception.__str__` 直接返回 `args` 的 repr。全仓 **169 处** `ApiError(ErrorCode.X, "消息", {...})` 全部用位置参数构造,因此 `str(exc)` 得到的是 **`(<ErrorCode.X: 'X'>, '消息', {details 全量})` 这个元组的 repr**,而不是 `message`。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/dto/common.py:29-33
|
|||
|
|
@dataclass(frozen=True)
|
|||
|
|
class ApiError(Exception):
|
|||
|
|
error_code: ErrorCode
|
|||
|
|
message: str
|
|||
|
|
details: dict[str, Any] | None = None
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
实测(本机 Python 运行验证):
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
str(exc) => "(<ErrorCode.VALIDATION_FAILED: 'VALIDATION_FAILED'>, '内容审核未通过,未创建生成任务', {'a': 1})"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
而 `main.py` 里 40 处这样用:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# main.py:1591
|
|||
|
|
await self._send_proactive(context.message_origin, f"❌ 单图生成失败\n请求:{request_id}\n错误:{exc}", [])
|
|||
|
|
# main.py:669
|
|||
|
|
yield event.plain_result(f"❌ 节点生成失败:{exc}")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
同一仓库的 `worker_service.py:514-534` `_safe_error_message()` 已经专门为 `ApiError` 做了正确的格式化(提取 `message` + `[code]` + 白名单 hint),说明作者知道这个坑,但聊天路径没有复用。
|
|||
|
|
- **影响**:
|
|||
|
|
1. **可用性**:QQ/微信用户收到的是 `❌ 节点生成失败:(<ErrorCode.GENERATION_FAILED: 'GENERATION_FAILED'>, '...', {...})`,中文提示被 Python 元组语法淹没。
|
|||
|
|
2. **信息泄漏**:`details` 里包含 `provider_id`、`model_id`、`route_name`、`body_preview`(`engines/image_provider.py:253,289,324` 会塞入上游响应体前 500 字节)、内部 `field` 名等,这些本来只该出现在管理端,现在直接推送到群聊。`core/generation_errors.py:_safe_details` 的脱敏逻辑在这条路径上完全没有生效。
|
|||
|
|
- **修复建议**:
|
|||
|
|
1. 给 `ApiError` 加显式 `__str__`:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
@dataclass(frozen=True)
|
|||
|
|
class ApiError(Exception):
|
|||
|
|
...
|
|||
|
|
def __str__(self) -> str:
|
|||
|
|
return f"{self.message} [{self.error_code.value}]"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
2. 把 `main.py` 的 40 处 `{exc}` 统一改为调用一个共享的 `user_facing_error(exc)`(可直接复用 `worker_service._safe_error_message`,把它提到 `backend/dto/common.py` 或 `core/generation_errors.py`),确保聊天侧永远只看到 message,不看到 details。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] SQLite 未开启 WAL,却跑 13 个并发写线程,全靠 30 秒 busy_timeout 硬扛
|
|||
|
|
|
|||
|
|
- **位置**:`database/db_helper.py:159-171`
|
|||
|
|
- **问题**:默认后端是 SQLite(`DatabaseSettings.selected_backend` 在未配置 PG 时返回 `"sqlite"`)。`connect()` 设了 `foreign_keys` 和 `busy_timeout`,但**没有设 `journal_mode=WAL`**。全仓 grep `PRAGMA` 只有这两行 + 一处 `PRAGMA table_info`。同时系统默认启动 5 个 round worker(`workflow_round_worker_runtime.py:25`)+ 8 个 advanced material worker(`advanced_material_worker_runtime.py:22`),共 13 个后台线程持续读写同一个 db 文件。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# database/db_helper.py:159-171
|
|||
|
|
def connect(self):
|
|||
|
|
if self.backend == "sqlite":
|
|||
|
|
db_path = self._resolve_sqlite_path()
|
|||
|
|
db_path.parent.mkdir(parents=True, exist_ok=True)
|
|||
|
|
# Parallel image tasks briefly overlap while recording queue,
|
|||
|
|
# node, image, and audit rows. Give SQLite enough time to hand the
|
|||
|
|
# writer lock to the next task instead of failing immediately.
|
|||
|
|
conn = sqlite3.connect(db_path, timeout=30.0)
|
|||
|
|
conn.row_factory = sqlite3.Row
|
|||
|
|
conn.execute("PRAGMA foreign_keys = ON")
|
|||
|
|
conn.execute("PRAGMA busy_timeout = 30000")
|
|||
|
|
return conn
|
|||
|
|
return self._connect_postgresql()
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
注释本身就承认了"并行图片任务会重叠写入",但给出的方案是"给 SQLite 足够时间"(把超时拉到 30 秒),而不是消除读写互斥。
|
|||
|
|
- **影响**:在默认 rollback journal 模式下,任何一个写事务(`audit_logs` 插入、`generation_tasks` 状态更新、`generation_images` 落库)会**阻塞所有读**。13 个 worker + 前端轮询 + 每次 `audit_service.record` 都是一次独立写事务,实际表现是 UI 列表接口在批量生成期间随机卡顿数百毫秒到数秒;极端情况下触发 30 秒超时后抛 `sqlite3.OperationalError: database is locked`,而这个异常会被 P1-2 的 `handle_request` 吞成"服务器内部错误"且无日志。
|
|||
|
|
- **修复建议**:在 `connect()` 中补两行(WAL 是持久化设置,只需生效一次,但每次设置代价极低):
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
conn.execute("PRAGMA journal_mode = WAL")
|
|||
|
|
conn.execute("PRAGMA synchronous = NORMAL")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
另外把 `initialize()` 里做一次 `PRAGMA wal_autocheckpoint` 配置,避免 WAL 文件无限增长。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] 每次队列 claim 开 2 个新连接;13 个 worker 空转时约 43 次 connect + COUNT(\*) / 秒
|
|||
|
|
|
|||
|
|
- **位置**:`backend/services/worker_service.py:207-221`、`backend/services/queue_service.py:280-310`、`database/repositories/job_repository.py:90-100,139-157`
|
|||
|
|
- **问题**:全仓 **151 处** `self.database.connect()`,没有任何连接池,每个仓储方法自己 `connect()` → `execute()` → `close()`。队列 worker 的轮询链路把这个模式放大了:
|
|||
|
|
- `_claim_next` 每轮调 `list_jobs(status="queued", page=1, page_size=claim_batch_size)`,`job_repository.list()` 里**额外跑一次 `COUNT(*)`**,而 worker 只用 `.items`,`total` 被丢弃;
|
|||
|
|
- 然后对每个候选调 `claim_queued_task`,它先 `get_job`(1 个连接)再 `update_if_status`(第 2 个连接),**每次 claim 尝试 = 2 个连接**;
|
|||
|
|
- 多 worker 抢同一批 queued 行时,失败的 claim 会继续尝试下一个候选,最坏 25 个候选 × 2 = 50 个连接/轮。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/worker_service.py:207-221
|
|||
|
|
def _claim_next(self, ctx: RequestContext) -> GenerationTask | None:
|
|||
|
|
queued = self.queue_service.list_jobs(status="queued", page=1, page_size=self.claim_batch_size).items
|
|||
|
|
for candidate in queued:
|
|||
|
|
if _retry_deferred(candidate):
|
|||
|
|
continue
|
|||
|
|
...
|
|||
|
|
claimed = self.queue_service.claim_queued_task(candidate.task_id, ctx, worker_id=self.worker_id)
|
|||
|
|
if claimed is not None:
|
|||
|
|
return claimed
|
|||
|
|
return None
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/queue_service.py:289-294
|
|||
|
|
before = self.get_job(task_id) # 连接 1(job_repository.get -> connect/close)
|
|||
|
|
if before.status != "queued":
|
|||
|
|
return None
|
|||
|
|
self._validate_task_status_transition(before.status, "running")
|
|||
|
|
attempt = ...
|
|||
|
|
after = self.repository.update_if_status(...) # 连接 2
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# database/repositories/job_repository.py:139-157(list 内部多跑一次 COUNT(*))
|
|||
|
|
conn = self.database.connect()
|
|||
|
|
try:
|
|||
|
|
total_row = conn.execute(f"SELECT COUNT(*) AS total FROM generation_tasks {where_clause}", tuple(params)).fetchone()
|
|||
|
|
rows = conn.execute(f"SELECT {TASK_COLUMNS} FROM generation_tasks {where_clause} ORDER BY created_at DESC LIMIT ...").fetchall()
|
|||
|
|
finally:
|
|||
|
|
conn.close()
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
轮询频率:round pool `interval_seconds=0.25` × 5 worker = 20 次/秒;advanced pool `interval_seconds=0.35` × 8 worker ≈ 23 次/秒。**空载合计约 43 次/秒**,每次都是完整的 `mkdir` 系统调用 + `sqlite3.connect` + 2 条 PRAGMA + 1 次 COUNT + 1 次 SELECT。
|
|||
|
|
- **影响**:完全空闲的实例也在持续消耗 CPU 和 IO;配合 P2-1(无 WAL)会显著抬高真实任务的写锁等待概率。在 PostgreSQL 后端下更糟——每次 `connect()` 都是一次完整的 TCP + 认证握手,43 次/秒的建连会直接打满 `max_connections`。
|
|||
|
|
- **修复建议**:
|
|||
|
|
1. 给 `JobRepository` 加 `list_items_only(...)`(或给 `list()` 加 `with_total: bool = True` 参数),worker 路径跳过 `COUNT(*)`;
|
|||
|
|
2. 把 `claim_queued_task` 的 `get_job` + `update_if_status` 合成一次 `UPDATE ... WHERE task_id=? AND status='queued' RETURNING *`(PG)/ `UPDATE` + `changes()`(SQLite),单连接单语句完成 CAS;
|
|||
|
|
3. 在 `Database` 上加一个线程局部连接缓存或最小连接池(`threading.local()` + 复用),至少让同一个 worker 线程的连续操作共用一条连接;
|
|||
|
|
4. worker 空闲时做指数退避(当前固定 0.25s/0.35s),比如连续 N 轮 idle 后把间隔拉到 2 秒。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] 前端 direct-fetch 路径完全没有超时,与 plugin-bridge 路径行为不一致
|
|||
|
|
|
|||
|
|
- **位置**:`frontend/shared/api/client.ts:81-85` vs `frontend/shared/api/client.ts:132-144,186-200`
|
|||
|
|
- **问题**:同一个 `request<T>()` 有两条传输路径。plugin-bridge 路径给每个调用都套了 `withClientTimeout`(GET 15s / POST 30s / 长任务 `LONG_TASK_TIMEOUT_MS`),而 direct-fetch(独立部署的 standalone-web)路径的 `fetch()` **没有传 `signal`,也没有任何 AbortController**。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
// frontend/shared/api/client.ts:80-93 —— 无超时
|
|||
|
|
try {
|
|||
|
|
response = await fetch(url, {
|
|||
|
|
method: options.method ?? "GET",
|
|||
|
|
headers: options.body === undefined ? undefined : { "Content-Type": "application/json" },
|
|||
|
|
body: options.body === undefined ? undefined : JSON.stringify(options.body),
|
|||
|
|
})
|
|||
|
|
} catch (err) {
|
|||
|
|
throw new ApiClientError("network_unreachable", "无法连接 ImageStudio 后端接口", { ... })
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
// frontend/shared/api/client.ts:131-144 —— 有超时
|
|||
|
|
if (method === "GET" && typeof bridge.apiGet === "function") {
|
|||
|
|
return withClientTimeout(
|
|||
|
|
bridge.apiGet<...>(endpoint, compactQuery(options.query)),
|
|||
|
|
PLUGIN_BRIDGE_GET_TIMEOUT_MS,
|
|||
|
|
"AstrBot Plugin Page GET request timed out",
|
|||
|
|
)
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
- **影响**:standalone-web 模式下,只要后端某个请求 hang 住(结合 P2-1 的 SQLite 写锁等待,或上游 provider 180 秒超时),对应的 UI 按钮会永远停在 loading 状态,没有任何反馈也无法取消。同一份代码在 AstrBot 内嵌模式下 15 秒就报错,两种部署形态行为不一致,问题在 standalone 上更难复现和归因。
|
|||
|
|
- **修复建议**:在 direct-fetch 分支复用同一套超时常量:
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
const controller = new AbortController()
|
|||
|
|
const timeoutMs = options.method === "GET"
|
|||
|
|
? PLUGIN_BRIDGE_GET_TIMEOUT_MS
|
|||
|
|
: pluginBridgePostTimeout(endpoint)
|
|||
|
|
const timer = window.setTimeout(() => controller.abort(), timeoutMs)
|
|||
|
|
try {
|
|||
|
|
response = await fetch(url, { ..., signal: controller.signal })
|
|||
|
|
} finally {
|
|||
|
|
window.clearTimeout(timer)
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
并把 `AbortError` 映射成与 bridge 一致的 `ApiClientError("bridge_timeout" | "request_timeout", ...)`。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] request_id 三条关联链路全断:前端剥掉、错误对象不带、聊天侧的 ID 从未落库
|
|||
|
|
|
|||
|
|
- **位置**:`frontend/shared/api/client.ts:202-230`、`main.py:1210-1218,1579-1591`、`core/identity.py:10-16`
|
|||
|
|
- **问题**:后端为每个请求生成 `req_{uuid4().hex}` 并写进 `audit_logs.request_id`(`core/audit_service.py:51`),响应体里也带了 `request_id`(`backend/dto/common.py:54,71`)。但:
|
|||
|
|
1. 前端 `unwrapApiResponse` 成功时直接 `current = current.data`,**request_id 被丢弃**;失败时构造 `ApiClientError(current.error_code, current.message, current.details ?? {})`,**同样不带 request_id**(它在响应的顶层而非 `details` 里)。
|
|||
|
|
2. 聊天命令自己造了一套完全独立的 ID(`chatreq_` / `chatfast_` / `chathires_`),只用于回显给用户,**从未传入 `RequestContext`,也从未写入任何表**。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
// frontend/shared/api/client.ts:205-210 —— request_id 在这里蒸发
|
|||
|
|
if (isApiResponse(current)) {
|
|||
|
|
if (!current.ok) {
|
|||
|
|
throw new ApiClientError(current.error_code, current.message, current.details ?? {})
|
|||
|
|
}
|
|||
|
|
current = current.data
|
|||
|
|
continue
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# main.py:1210-1218 —— 造一个只给人看的 ID
|
|||
|
|
request_id = f"chatreq_{uuid4().hex}"
|
|||
|
|
task = asyncio.create_task(self._run_single_image_background(command, context, request_id))
|
|||
|
|
...
|
|||
|
|
yield event.plain_result(
|
|||
|
|
"✅ 单图请求已进入后台执行\n\n"
|
|||
|
|
f"请求:{request_id}\n尺寸:{command.size}\n数量:{command.count}\n" ...
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# main.py:1579-1591 —— request_id 只参与字符串拼接,没有进 submit_single_image
|
|||
|
|
async def _run_single_image_background(self, command: Any, context: Any, request_id: str) -> None:
|
|||
|
|
try:
|
|||
|
|
result = await asyncio.to_thread(self.chat_workflow_service.submit_single_image, command, context)
|
|||
|
|
...
|
|||
|
|
f"✅ 单图生成完成\n请求:{request_id}\n任务:{generation.get('task_id') or '—'}\n"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
注:ID 前缀本身也不一致——`core/identity.py:new_request_id()` 用 `req_`,聊天侧用 `chatreq_`/`chatfast_`/`chathires_`,三者互不相干。
|
|||
|
|
- **影响**:用户报障时给出的唯一凭据(Web 端的错误提示 / 聊天里的"请求:chatreq_xxx")在 `audit_logs`、`generation_tasks`、`workflow_history` 里**一条都查不到**。配合 P1-1(无日志)和 P1-2(异常吞掉),线上问题定位手段为零。
|
|||
|
|
- **修复建议**:
|
|||
|
|
1. `client.ts`:`ApiClientError` 增加 `readonly requestId?: string`,在 `unwrapApiResponse` 里从 `current.request_id` 取;UI 的错误 toast 追加 `(追踪号 req_xxx)`。
|
|||
|
|
2. `main.py`:删掉 `chatreq_`/`chatfast_`/`chathires_` 这套平行 ID,改为先构造 `ctx = RequestContext(...)`,把 `ctx.request_id` 同时用于回显和下游调用,让聊天生成的所有审计行都能被这个 ID 反查。
|
|||
|
|
3. `RequestContext` 支持从入站 `X-Request-Id` 头继承(`_request_context_from_astrbot_request` 目前只取 username/user_id/ip),便于跨 AstrBot 网关追踪。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] `cost_service` 对 generation_images 做无 LIMIT 全表扫描,days 过滤放在 Python 侧
|
|||
|
|
|
|||
|
|
- **位置**:`backend/services/cost_service.py:168-200`(`_image_rows`)、`40-43`(Python 侧过滤)、`202-234`(另两个同样无 LIMIT 的查询)
|
|||
|
|
- **问题**:`GET costs/usage` 是 `read` 权限的普通查询接口。`_image_rows()` 的 SQL **没有 LIMIT、没有时间条件**,把 `generation_images` 全表(含 `metadata_json` 大字段)拉进内存,`days` 过滤在 Python 循环里做;`_advanced_material_attempt_rows()` 和 `_advanced_style_validation_case_rows()` 同样无 LIMIT。每行还要跑一次 `json.loads`。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/cost_service.py:186-198
|
|||
|
|
where_sql = f"WHERE {' AND '.join(where)}" if where else ""
|
|||
|
|
conn = self.database.connect()
|
|||
|
|
try:
|
|||
|
|
return conn.execute(
|
|||
|
|
f"""
|
|||
|
|
SELECT image_id, task_id, profile_id, workflow_id, node_id,
|
|||
|
|
model_route, metadata_json, created_at
|
|||
|
|
FROM generation_images
|
|||
|
|
{where_sql}
|
|||
|
|
ORDER BY created_at DESC
|
|||
|
|
""", # <- 无 LIMIT,无 created_at 过滤
|
|||
|
|
tuple(params),
|
|||
|
|
).fetchall()
|
|||
|
|
finally:
|
|||
|
|
conn.close()
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/cost_service.py:40-44 —— 时间过滤在拉完全表之后
|
|||
|
|
for row in self._image_rows(profile_id=..., workflow_id=..., node_id=...):
|
|||
|
|
created_at = _parse_datetime(row["created_at"])
|
|||
|
|
if since and created_at and created_at < since:
|
|||
|
|
continue
|
|||
|
|
metadata = _json_loads(row["metadata_json"])
|
|||
|
|
```
|
|||
|
|
- **影响**:仪表盘/成本页每次刷新都把全部历史图片记录 + 全部 metadata JSON 载入内存并逐行解析。跑几个月后 `generation_images` 到十万级时,单次请求就是几百 MB 内存峰值 + 秒级 CPU,而且这次查询会长时间持有 SQLite 读锁(P2-1 无 WAL 时会阻塞写)。`days=7` 这类"只看最近一周"的查询完全没有得到任何加速。
|
|||
|
|
- **修复建议**:把时间窗口下推到 SQL 并加上限:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
if since is not None:
|
|||
|
|
where.append(f"created_at >= {placeholder}")
|
|||
|
|
params.append(since.isoformat())
|
|||
|
|
...
|
|||
|
|
ORDER BY created_at DESC
|
|||
|
|
LIMIT {placeholder} # 例如 20000,超出时在响应里标 truncated=True
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
长期建议在写入侧就把 `usage` 的 token 数拆成独立数值列(或建一张 `generation_cost_facts` 汇总表),避免每次聚合都解析 JSON。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] 高级素材详情存在 N+1:按 attempt 循环查子表,每次新开一个数据库连接
|
|||
|
|
|
|||
|
|
- **位置**:`backend/services/advanced_material_service.py:228-239`、`292-303`;`backend/services/advanced_style_set_service.py:1215-1216`、`509-510`;被调方 `database/repositories/advanced_material_repository.py:1147-1157`
|
|||
|
|
- **问题**:`get_import` / `list_attempts` 对每个 attempt 单独调 `list_style_validation_cases(attempt.attempt_id)`,而该仓储方法自己 `connect()` / `close()`。attempt 有 N 条就产生 **N 次独立连接 + N 次查询**。`advanced_style_set_service.py` 里还有两层嵌套(`for attempt: for case:`)。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/advanced_material_service.py:225-240
|
|||
|
|
attempts = self.repository.list_attempts(import_id)
|
|||
|
|
is_style_set = self.style_sets.is_style_set(record)
|
|||
|
|
style_detail = self.style_sets.detail(record) if is_style_set else {}
|
|||
|
|
attempt_payloads = [
|
|||
|
|
{
|
|||
|
|
**attempt.to_dict(),
|
|||
|
|
**(
|
|||
|
|
{
|
|||
|
|
"validation_cases": [
|
|||
|
|
case.to_dict()
|
|||
|
|
for case in self.repository.list_style_validation_cases(attempt.attempt_id) # N+1
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
if is_style_set
|
|||
|
|
else {}
|
|||
|
|
),
|
|||
|
|
}
|
|||
|
|
for attempt in attempts
|
|||
|
|
]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# database/repositories/advanced_material_repository.py:1147-1157
|
|||
|
|
def list_style_validation_cases(self, attempt_id: str) -> list[AdvancedStyleValidationCase]:
|
|||
|
|
conn = self.database.connect() # 每次调用一条新连接
|
|||
|
|
try:
|
|||
|
|
rows = conn.execute(
|
|||
|
|
f"SELECT {STYLE_VALIDATION_CASE_COLUMNS} FROM advanced_style_validation_cases "
|
|||
|
|
f"WHERE attempt_id = {self.placeholder} ORDER BY sort_order ASC, case_id ASC",
|
|||
|
|
(attempt_id,),
|
|||
|
|
).fetchall()
|
|||
|
|
finally:
|
|||
|
|
conn.close()
|
|||
|
|
return [_row_to_style_validation_case(row) for row in rows]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
另外 `advanced_material_service.py:258-263` 把 `get_material` 调了**两次**(一次在条件里,一次在取值里),又是两个连接,还引入了 TOCTOU:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
"accepted_material": (
|
|||
|
|
self.repository.get_material(record.accepted_material_id).to_dict()
|
|||
|
|
if record.accepted_material_id and self.repository.get_material(record.accepted_material_id)
|
|||
|
|
else None
|
|||
|
|
),
|
|||
|
|
```
|
|||
|
|
- **影响**:一个做过 20 轮验证的风格集导入,打开详情页就是 20+ 次建连 + 20+ 次查询(PG 后端下是 20+ 次 TCP 握手)。这是前端轮询的接口,会被反复调用。
|
|||
|
|
- **修复建议**:加一个批量方法:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
def list_style_validation_cases_by_attempts(self, attempt_ids: Sequence[str]) -> dict[str, list[...]]:
|
|||
|
|
if not attempt_ids:
|
|||
|
|
return {}
|
|||
|
|
marks = ", ".join([self.placeholder] * len(attempt_ids))
|
|||
|
|
conn = self.database.connect()
|
|||
|
|
try:
|
|||
|
|
rows = conn.execute(
|
|||
|
|
f"SELECT {STYLE_VALIDATION_CASE_COLUMNS} FROM advanced_style_validation_cases "
|
|||
|
|
f"WHERE attempt_id IN ({marks}) ORDER BY attempt_id, sort_order ASC, case_id ASC",
|
|||
|
|
tuple(attempt_ids),
|
|||
|
|
).fetchall()
|
|||
|
|
finally:
|
|||
|
|
conn.close()
|
|||
|
|
grouped: dict[str, list] = defaultdict(list)
|
|||
|
|
for row in rows:
|
|||
|
|
grouped[row["attempt_id"]].append(_row_to_style_validation_case(row))
|
|||
|
|
return grouped
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`accepted_material` 处改成先 `material = self.repository.get_material(...) if record.accepted_material_id else None`,再用局部变量。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] `ChatSessionStore` 构造期无保护地做 naive/aware datetime 比较,脏持久化文件会让插件启动直接崩
|
|||
|
|
|
|||
|
|
- **位置**:`backend/services/chat_session_service.py:32,57-60,62-76`
|
|||
|
|
- **问题**:`__init__` 无保护地调 `self._load()`。`_load()` 内部对每一行用 `except (KeyError, TypeError, ValueError): continue` 做了容错——但 `datetime.fromisoformat("2026-07-27T10:00:00")`(无时区偏移)**不会抛异常**,它会安静地返回一个 naive datetime 存进 `self._items`。真正爆炸的地方在循环外的 `self._prune()`(第 76 行),那里拿 naive 和 aware 的 `now` 比较,抛 `TypeError: can't compare offset-naive and offset-aware datetimes`,而这个位置**不在任何 try 内**。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/chat_session_service.py:57-76
|
|||
|
|
def _prune(self) -> None:
|
|||
|
|
now = datetime.now(timezone.utc) # aware
|
|||
|
|
for key in [key for key, item in self._items.items() if item.expires_at <= now]: # <- 这里炸
|
|||
|
|
self._items.pop(key, None)
|
|||
|
|
|
|||
|
|
def _load(self) -> None:
|
|||
|
|
try:
|
|||
|
|
payload = json.loads(self.persistence_path.read_text(encoding="utf-8"))
|
|||
|
|
except (OSError, ValueError, TypeError):
|
|||
|
|
return
|
|||
|
|
for item in payload if isinstance(payload, list) else []:
|
|||
|
|
try:
|
|||
|
|
session = ChatSession(
|
|||
|
|
key=str(item["key"]), action=str(item["action"]), payload=dict(item.get("payload") or {}),
|
|||
|
|
expires_at=datetime.fromisoformat(item["expires_at"]), created_at=datetime.fromisoformat(item["created_at"]),
|
|||
|
|
) # naive 字符串在这里不会报错
|
|||
|
|
self._items[session.key] = session
|
|||
|
|
except (KeyError, TypeError, ValueError):
|
|||
|
|
continue
|
|||
|
|
self._prune() # 无 try 保护
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
同样的问题也在 `put()` 里:`min(self._items.values(), key=lambda item: item.created_at)`(第 38 行)遇到 naive/aware 混排也会 `TypeError`。
|
|||
|
|
- **影响**:只要 `data/chat_sessions.json` 里出现一个不带时区的时间戳(旧版本写入、手工编辑、外部工具生成、跨环境迁移),`ChatSessionStore()` 构造就抛异常,整个聊天服务乃至插件加载失败。作者显然想让这个文件"坏了就忽略"(`_load` 开头的 `except (OSError, ValueError, TypeError): return` 已经表明了意图),但这层防护被绕过了。
|
|||
|
|
- **修复建议**:在解析处强制归一化时区,并给 `_prune()` 加保护:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
def _aware(value: str) -> datetime:
|
|||
|
|
parsed = datetime.fromisoformat(value)
|
|||
|
|
return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`expires_at=_aware(item["expires_at"])`、`created_at=_aware(item["created_at"])`。仓库里 `cost_service._parse_datetime`(`cost_service.py:319-329`)、`worker_service._retry_deferred`(`worker_service.py:487-493`)、`integration_token_service._parse_time`(`integration_token_service.py:186-198`)三处都已经写对了这段归一化逻辑,应该抽成 `core/` 里的共享工具函数,消除这处漏网。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] 路由权限矩阵声明了但 dispatch 从不校验,只有 5 处散落的手工检查
|
|||
|
|
|
|||
|
|
- **位置**:`adapters/astrbot_adapter.py:171-187`、`backend/route_registry.py:9-29,32-52`
|
|||
|
|
- **问题**:`route_registry.py` 定义了完整的 `PERMISSION_MATRIX`(read/write/config/materials/retry/public_token),166 条路由每条都标了 `permission`,`RouteSpec.to_dict()` 还会把 `auth_required` / `required_permissions` 导出到 manifest。但唯一的请求入口 `dispatch_extension_request` 拿到 `RouteMatch.permission` 之后**直接调 handler,从不做任何校验**。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# adapters/astrbot_adapter.py:179-187
|
|||
|
|
match = resolve_route(method, path)
|
|||
|
|
if match is None:
|
|||
|
|
return error_response(error_code=ErrorCode.RESOURCE_NOT_FOUND, ...)
|
|||
|
|
return match.handler(*_handler_args(match, ctx, query or {}, payload or {}))
|
|||
|
|
# ^ match.permission 被解析出来了,但这里没有任何 ctx.has_permission 判断
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
全仓 `has_permission` 只出现 5 次,集中在 2 个文件:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
backend/routes/image_routes.py:42,70,90,156
|
|||
|
|
backend/routes/job_routes.py:36
|
|||
|
|
```
|
|||
|
|
- **影响**:当前之所以没出安全事故,是因为 `main.py:2337` 给所有已认证用户一次性授予了全部 4 个权限(`permissions = BRIDGE_PERMISSIONS if username or user_id else frozenset()`),矩阵在运行时是恒真的。但这意味着:(1) 这套权限声明是**纯装饰**,任何人读代码会误以为已生效;(2) 一旦将来 AstrBot 暴露细粒度用户角色、或有人把 `BRIDGE_PERMISSIONS` 收窄,166 条路由里只有 5 处会真正拦截,其余 161 条静默放行。这是典型的"策略集中声明、执行处处缺失"的横切一致性缺陷。
|
|||
|
|
- **修复建议**:在 dispatch 层统一执行,让声明成为唯一事实来源:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
match = resolve_route(method, path)
|
|||
|
|
if match is None: ...
|
|||
|
|
requirement = permission_requirement(match.permission)
|
|||
|
|
if requirement["auth_required"] and not ctx.is_authenticated:
|
|||
|
|
return error_response(error_code=ErrorCode.UNAUTHORIZED, ..., request_id=ctx.request_id)
|
|||
|
|
for needed in requirement.get("required_permissions") or []:
|
|||
|
|
if not ctx.has_permission(needed):
|
|||
|
|
return error_response(error_code=ErrorCode.FORBIDDEN,
|
|||
|
|
message="缺少所需权限",
|
|||
|
|
details={"required_permission": needed},
|
|||
|
|
request_id=ctx.request_id)
|
|||
|
|
return match.handler(...)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
然后把 `image_routes` / `job_routes` 里那 5 处手工检查降级为纯业务判断(如"受限图片")或直接删除。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P2-中等] 主动通知全链路静默失败:签名失败返回 None,发送失败返回 False,一行日志都没有
|
|||
|
|
|
|||
|
|
- **位置**:`main.py:1650-1670`(`_send_proactive`)、`backend/services/workflow_round_worker_runtime.py:257-263`(`_signed_url`)、`backend/services/worker_service.py:366-380`(`_notify_terminal`)
|
|||
|
|
- **问题**:轮次完成后主动推图到聊天的这条链路上,有三个连续的静默吞异常点,任何一环失败用户和运维都得不到任何信号:
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/workflow_round_worker_runtime.py:257-263
|
|||
|
|
def _signed_url(media_service: MediaService, uri: Any) -> str | None:
|
|||
|
|
if not isinstance(uri, str) or not uri:
|
|||
|
|
return None
|
|||
|
|
try:
|
|||
|
|
return media_service.sign_media(uri).get("url")
|
|||
|
|
except Exception:
|
|||
|
|
return None # 签名失败 -> 通知里图片 URL 变 None -> 图片静默消失
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# main.py:1664-1670
|
|||
|
|
message = AstrBotMessageChain(components) if AstrBotMessageChain is not None else components
|
|||
|
|
result = sender(message_origin, message)
|
|||
|
|
if inspect.isawaitable(result):
|
|||
|
|
await result
|
|||
|
|
return True
|
|||
|
|
except Exception:
|
|||
|
|
return False # 发送失败 -> 返回 False -> 调用方也不看返回值
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/worker_service.py:372-380
|
|||
|
|
if self.terminal_callback is None:
|
|||
|
|
return
|
|||
|
|
try:
|
|||
|
|
self.terminal_callback(claimed_task, current_task, exc)
|
|||
|
|
except Exception:
|
|||
|
|
# Queue state is authoritative. ...
|
|||
|
|
return # 同一文件其余 6 处失败都 logger.exception,唯独这里静默
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`worker_service.py` 是全仓唯一有 logger 的文件,其他 6 个 `except` 全部 `logger.exception(...)`,`_notify_terminal` 是唯一的例外,明显是遗漏而非有意为之。
|
|||
|
|
- **影响**:用户提交生成后一直收不到结果推送,服务端三个环节都"成功"返回(`None` / `False` / 静默 return),队列状态显示 succeeded。这类问题在没有日志的前提下**只能靠加打印重现**。
|
|||
|
|
- **修复建议**:三处都改为 `logger.warning(...)` / `logger.exception(...)` 后再返回兜底值。`_send_proactive` 的返回值当前在 `_deliver_round_terminal_notification` 中被忽略,应该至少在返回 `False` 时记一条带 `round_id` 和 `message_origin` 的 warning。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P3-轻微] `_Psycopg2ConnectionAdapter.execute` 每次新建 cursor 且从不关闭
|
|||
|
|
|
|||
|
|
- **位置**:`database/db_helper.py:301-317`
|
|||
|
|
- **问题**:psycopg2 回退适配器的 `execute()` 每次调用都 `self.connection.cursor()` 新建游标,返回后没有任何地方 `close()` 它。适配器只暴露 `commit` / `rollback` / `close`,没有 cursor 生命周期管理。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
class _Psycopg2ConnectionAdapter:
|
|||
|
|
def __init__(self, connection):
|
|||
|
|
self.connection = connection
|
|||
|
|
|
|||
|
|
def execute(self, query: str, params: tuple[Any, ...] | list[Any] | None = None):
|
|||
|
|
cursor = self.connection.cursor()
|
|||
|
|
cursor.execute(query, tuple(params or ()))
|
|||
|
|
return cursor # 调用方拿去 .fetchall()/.fetchone() 后就丢弃,永不 close
|
|||
|
|
```
|
|||
|
|
- **影响**:单个连接内每执行一条 SQL 就泄漏一个游标对象,直到 `conn.close()` 才随连接释放。因为本仓库是"每次操作新建连接"(见 P2-2),泄漏窗口很短,所以实际危害有限;但一旦按 P2-2 的建议引入连接复用/连接池,这个泄漏会立刻变成真实的内存和服务端资源问题。而且 psycopg2 只是 `import psycopg` 失败时的回退路径,问题更难被发现。
|
|||
|
|
- **修复建议**:返回一个包装对象,在 `fetchall`/`fetchone` 取完后自动 `close`;或者让 `execute` 直接返回结果集:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
def execute(self, query, params=None):
|
|||
|
|
cursor = self.connection.cursor()
|
|||
|
|
try:
|
|||
|
|
cursor.execute(query, tuple(params or ()))
|
|||
|
|
return _CursorResult(cursor) # __enter__/__exit__ 或 fetch 后自动 close
|
|||
|
|
except Exception:
|
|||
|
|
cursor.close()
|
|||
|
|
raise
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P3-轻微] `.part` 临时文件在 `replace()` 失败时残留,无清理
|
|||
|
|
|
|||
|
|
- **位置**:`backend/services/material_service.py:326-336`、`backend/services/advanced_material_service.py:1584-1594`
|
|||
|
|
- **问题**:两处上传落盘都用"先写 `.part` 再 `replace`"的原子写模式,但 `except OSError` 里只抛 `ApiError`,**不删除已经写出的 `.part` 文件**。如果 `write_bytes` 成功而 `replace` 失败(目标被占用、跨设备、权限、磁盘满导致部分写入后失败),临时文件永久留在存储目录。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/material_service.py:326-336
|
|||
|
|
try:
|
|||
|
|
persisted_path.parent.mkdir(parents=True, exist_ok=True)
|
|||
|
|
temporary_path = persisted_path.with_suffix(f"{persisted_path.suffix}.part")
|
|||
|
|
temporary_path.write_bytes(content)
|
|||
|
|
temporary_path.replace(persisted_path)
|
|||
|
|
except OSError as exc:
|
|||
|
|
raise ApiError(
|
|||
|
|
ErrorCode.STORAGE_WRITE_FAILED,
|
|||
|
|
"material image could not be stored",
|
|||
|
|
{"uri": image_uri, "exception_type": exc.__class__.__name__},
|
|||
|
|
) from None
|
|||
|
|
# temporary_path 从未清理
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`advanced_material_service.py:1584-1594` 是同一段代码的复制。
|
|||
|
|
- **影响**:磁盘写失败(最典型是磁盘满)时,每次重试都留下一个和原图等大的 `.part` 残骸,把本就紧张的空间进一步耗尽,形成正反馈。`orphan_media_service` 扫描的是 `storage://` URI 与 DB 的差集,识别不了 `.part` 后缀的孤儿。
|
|||
|
|
- **修复建议**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
temporary_path = persisted_path.with_suffix(f"{persisted_path.suffix}.part")
|
|||
|
|
try:
|
|||
|
|
persisted_path.parent.mkdir(parents=True, exist_ok=True)
|
|||
|
|
temporary_path.write_bytes(content)
|
|||
|
|
temporary_path.replace(persisted_path)
|
|||
|
|
except OSError as exc:
|
|||
|
|
temporary_path.unlink(missing_ok=True)
|
|||
|
|
raise ApiError(...) from None
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
建议把这段抽成 `core/storage.py` 的 `atomic_write_bytes(path, content)`,两处共用。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P3-轻微] 每个请求重排 166 条路由 + 逐条编译正则 + `inspect.signature`,全程无缓存
|
|||
|
|
|
|||
|
|
- **位置**:`adapters/astrbot_adapter.py:44-64`(`resolve_route`)、`224-230`(`_match_endpoint`)、`233-252`(`_handler_args`)
|
|||
|
|
- **问题**:唯一的请求入口每次都做纯静态的重复计算:`ROUTES` 是模块级常量,路由表和 handler 签名在进程生命周期内不变,但每个请求都重跑一遍。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# adapters/astrbot_adapter.py:44-64
|
|||
|
|
def resolve_route(method: str, path: str) -> RouteMatch | None:
|
|||
|
|
normalized_method = str(method or "").strip().upper()
|
|||
|
|
endpoint = normalize_extension_path(path)
|
|||
|
|
candidates = sorted( # 每请求排序
|
|||
|
|
(route for route in ROUTES if route.method == normalized_method),
|
|||
|
|
key=_route_specificity,
|
|||
|
|
reverse=True,
|
|||
|
|
)
|
|||
|
|
for route in candidates:
|
|||
|
|
params = _match_endpoint(route.endpoint, endpoint) # 每候选编译一次正则
|
|||
|
|
...
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# adapters/astrbot_adapter.py:224-230
|
|||
|
|
def _match_endpoint(pattern: str, endpoint: str) -> dict[str, str] | None:
|
|||
|
|
param_names = re.findall(r"\{([^{}]+)\}", pattern)
|
|||
|
|
regex = "^" + re.sub(r"\{([^{}]+)\}", r"([^/]+)", pattern) + "$"
|
|||
|
|
match = re.match(regex, endpoint) # 每次都走 re 的模式缓存查找
|
|||
|
|
...
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# adapters/astrbot_adapter.py:240
|
|||
|
|
for index, parameter in enumerate(inspect.signature(match.handler).parameters.values()):
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
实测(本机 2000 次循环):`resolve_route + _handler_args` 约 **0.178 ms / 次**,路由总数 166。
|
|||
|
|
- **影响**:0.18 ms 本身不致命,但这是所有 166 个端点的固定入口开销,且是纯浪费。前端存在多处轮询(队列状态、导入进度、事件流),这部分开销会持续累加;随着路由表增长呈线性劣化。
|
|||
|
|
- **修复建议**:
|
|||
|
|
1. 模块加载时预编译一张 `dict[str, list[tuple[re.Pattern, RouteSpec]]]`(按 method 分桶、已按 `_route_specificity` 排好序);
|
|||
|
|
2. `_load_handler` 加 `@functools.lru_cache(maxsize=None)`;
|
|||
|
|
3. handler 的参数绑定方案在导入时算一次并缓存到 `RouteSpec` 上,避免每请求 `inspect.signature`。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P3-轻微] `log_service` / `cost_service` 的 `_json_loads` 无异常保护,单条脏数据可让整个日志端点 500
|
|||
|
|
|
|||
|
|
- **位置**:`backend/services/log_service.py:297-300`、`backend/services/cost_service.py:332-336`
|
|||
|
|
- **问题**:日志查询接口对每一行的 JSON 列做 `json.loads`,没有任何保护。只要 `integration_logs.request_json` / `audit_logs.metadata_json` / `generation_images.metadata_json` 里有一条被截断或写坏的记录,整个列表接口就抛 `json.JSONDecodeError`。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# backend/services/log_service.py:297-300
|
|||
|
|
def _json_loads(value: str | None) -> Any:
|
|||
|
|
if not value:
|
|||
|
|
return None
|
|||
|
|
return json.loads(value) # 无 try
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
调用点(每行一次):`log_service.py:162-163`(`request` / `response`)、`284-286`(`before_snapshot` / `after_snapshot` / `metadata`)。
|
|||
|
|
- **影响**:日志查询是排障工具,恰恰应该对脏数据最宽容。现在一条坏记录就能让 `logs/integrations` 和 `security/audit-logs` 两个端点永久 500,而且因为 P1-2 该异常被吞成"服务器内部错误"、因为 P1-1 没有日志,运维根本不知道是哪条记录坏了——排障工具自己坏了且无法排障。
|
|||
|
|
- **修复建议**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
def _json_loads(value: str | None) -> Any:
|
|||
|
|
if not value:
|
|||
|
|
return None
|
|||
|
|
try:
|
|||
|
|
return json.loads(value)
|
|||
|
|
except (ValueError, TypeError):
|
|||
|
|
return {"_parse_error": True, "raw_preview": str(value)[:200]}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`cost_service._json_loads` 同理(它返回 `dict`,失败时返回 `{}` 即可)。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### [P3-轻微] `EventBus.publish` 不隔离订阅者异常,一个 handler 抛错会中断发布方并跳过后续 handler
|
|||
|
|
|
|||
|
|
- **位置**:`core/event_bus.py:45-50`
|
|||
|
|
- **问题**:`publish` 在循环里直接调 `handler(event)`,没有 try/except。任一订阅者抛异常会向上冒泡到发布方(`generation_service._publish`,`generation_service.py:1701-1703`),同时后续 handler 全部不执行。
|
|||
|
|
- **代码**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# core/event_bus.py:45-50
|
|||
|
|
def publish(self, event_type: str, payload: dict[str, Any] | None = None) -> Event:
|
|||
|
|
event = Event(event_type=_normalize_event_type(event_type), payload=dict(payload or {}))
|
|||
|
|
self._history.append(event)
|
|||
|
|
for handler in [*self._handlers.get(event.event_type, []), *self._handlers.get("*", [])]:
|
|||
|
|
handler(event) # 无隔离
|
|||
|
|
return event
|
|||
|
|
```
|
|||
|
|
- **影响**:当前全仓 grep 不到任何 `event_bus.subscribe(...)` 调用(事件总线只被当作 `deque(maxlen=500)` 的历史缓冲,供 `event_stream_service.recent_events` 轮询读取),所以这条现在**不会触发**。但 `_publish` 已被 `generation_service` 在生成成功/失败等关键路径调用,一旦将来接入第一个订阅者(例如 webhook 推送),一个下游 HTTP 超时就会把生成流程本身打断。
|
|||
|
|
- **修复建议**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
for handler in [...]:
|
|||
|
|
try:
|
|||
|
|
handler(event)
|
|||
|
|
except Exception:
|
|||
|
|
logger.exception("event handler failed event_type=%s event_id=%s", event.event_type, event.event_id)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
顺带修掉 `unsubscribe`(`event_bus.py:41-43`)对未订阅类型会因 `defaultdict` 而凭空建键的小问题。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 附:确认无问题的横切项(供后续审计参考)
|
|||
|
|
|
|||
|
|
以下项经实际 grep/阅读确认**不存在**问题,无需再查:
|
|||
|
|
|
|||
|
|
| 检查项 | 结论 | 证据 |
|
|||
|
|
|---|---|---|
|
|||
|
|
| 裸 `except:` | 0 处 | `grep -rn "except:"` 全树无匹配 |
|
|||
|
|
| naive datetime | 0 处 | `utcnow()` 0 次;`datetime.now()`(无参)0 次;`datetime.now(timezone.utc)` / `datetime.now(UTC)` 共 29 次 |
|
|||
|
|
| async 函数内同步阻塞 | 无 | `requests` / `time.sleep` / `subprocess` 全树 0 次导入;`main.py` 的同步服务调用全部走 `asyncio.to_thread`(30+ 处) |
|
|||
|
|
| 裸 `open()` 文件句柄泄漏 | 无 | `with open(` / `= open(` 全树 0 处,统一用 `Path.read_bytes/write_bytes/read_text` |
|
|||
|
|
| 临时目录清理 | 正确 | `data_backup_service.py:49,126` 用 `tempfile.TemporaryDirectory` 上下文管理器 |
|
|||
|
|
| ID 生成方式 | 统一 | 全部 `f"{前缀}_{uuid4().hex}"`,前缀按实体命名(`req_`/`audit_`/`task_`/`round_`/`img_`…),唯一例外见 P2-3 的聊天侧 ID |
|
|||
|
|
| 敏感信息脱敏 | 已实现 | `core/secret_service.redact`、`settings_service._mask_sensitive`(审计快照 `settings_service.py:453-454,564-565`)、`generation_errors._safe_message/_safe_details`、`api_discovery_service._redact_probe_error`、`prompt_governance_service._redact` |
|
|||
|
|
| SSRF 防护 | 已实现 | `core/remote_fetch.py` 校验协议/凭据/私网地址/重定向同源,并在跨源重定向时移除 `Authorization`(`remote_fetch.py:124-126`) |
|
|||
|
|
| 内存缓存无上限 | 无 | `_NODE_GENERATION_LOCKS` / `_QUICK_CHARACTER_WORKFLOW_LOCKS` 有引用计数并在归零时 `pop`(`generation_service.py:1947-1950,1966-1969`);`EventBus._history` 是 `deque(maxlen=500)`;`ChatSessionStore` 有 `max_sessions=1000` 上限 |
|
|||
|
|
| 前端 catch 静默 | 可接受 | 20 处 `catch {}` 均为"降级到默认值"的有意设计(如 `ModelAssignmentsPanel.vue:153` `runtimeModels.value = []`),非错误吞没 |
|