diff --git a/.gitignore b/.gitignore index c839b08..2a2843e 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,7 @@ __pycache__/ .mypy_cache/ .venv/ venv/ +pytest-cache-files-*/ # Node / frontend node_modules/ @@ -32,6 +33,7 @@ data/exports/* !data/exports/.gitkeep # Local machine/editor noise +.tmp/ .env .env.* !.env.example diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index e8a2dc8..3af56de 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,77 +1,91 @@ -# Development Environment +# 开发约定 -## Scope +## AstrBot 运行契约 -The repository has completed the non-UI backend queue through `S120`. UI-only work remains separate. +ImageStudio 的生产宿主是 AstrBot `4.25.0` 及以上版本。 -Prepared: +- 插件入口是 `main.py` 中的 `Star` 子类。 +- AstrBot 调用 `initialize()` 和 `terminate()` 管理生命周期。 +- `initialize()` 通过 `context.register_web_api` 注册 API。 +- `pages/imagestudio/index.html` 由 AstrBot 自动发现为插件页面。 +- 插件页面 bridge 只接收 `ui/bootstrap` 这类相对 endpoint。 +- v4.25 legacy bridge 前缀为 `/api/plug/astrbot_plugin_imagestudio/`;当前直接 API 前缀为 `/api/v1/plugins/extensions/astrbot_plugin_imagestudio/`。 -- AstrBot plugin root files, runtime registration helpers, and v3 configuration placeholders. -- Backend API routes, route registry, permission matrix, contract export, and pure dispatch helpers. -- SQLite default database with optional PostgreSQL selection from `config.yaml`. -- `psycopg` and `psycopg2` PostgreSQL driver compatibility. -- Core repositories and services for profiles, workflows, nodes, jobs, images, materials, wardrobe, integrations, settings, review, costs, media, and model providers. -- Real image-provider path, VLM adapter interface, worker/retry/cancel infrastructure, budget guard, prompt expansion, token rotation, deletion safety, media indexing, and orphan scanning. -- Runtime data directories with `.gitkeep` files only. -- Generated data, local virtual environments, node modules, and build artifacts are ignored. +不要把内部辅助函数或自定义 web mount 当作 AstrBot 官方回调。也不要依赖 `/api/v1/plugins/extensions/astrbot_plugin_imagestudio/web` 存在;`web/` 仅作为独立构建/兼容产物,生产插件页面以 `pages/imagestudio/` 为准。 -## Local Tools +## 目录职责 -On Windows, PowerShell may block `npm.ps1`; use `npm.cmd` when needed. +- `main.py`:AstrBot Star 生命周期与 Web API 接入。 +- `backend/routes/`:endpoint 适配层。 +- `backend/services/`:业务服务。 +- `database/`:数据库连接、仓储和迁移。 +- `frontend/plugin-page/`:AstrBot 插件页面源码,构建到 `pages/imagestudio/`。 +- `frontend/standalone-web/`:独立预览/兼容页面源码,构建到 `web/`。 +- `frontend/shared/`:两套前端共享的 API client、endpoint 和类型。 -Codex bundled tools are usually available under the local Codex runtime cache. The project itself does not require a standalone backend server for normal plugin use. +后端 endpoint 使用不带宿主前缀的形式维护,例如 `nodes/{node_id}/generate`。宿主适配层负责 legacy bridge 和当前 `/api/v1` 前缀的兼容,业务服务不应感知 URL 前缀。 -## Verification +## 持久化规则 -Run backend tests and non-UI smoke: +运行数据必须位于: ```text -python -m unittest discover -s tests -python -m compileall adapters backend core database engines scripts tests main.py -python scripts/imagestudio_non_ui_smoke.py -python scripts/imagestudio_pg_smoke.py --dry-run +data/plugin_data/astrbot_plugin_imagestudio/ ``` -Set `IMAGESTUDIO_TEST_PG_DSN` before `python -m unittest tests.test_pg_smoke` to run a live PostgreSQL smoke. +不要向插件源码目录写数据库、图片、日志或缓存。测试可使用临时目录,且不能把真实运行数据加入发布包。 -## API Contract +`config.yaml` 保留 `data/db`、`data/images` 等逻辑路径;运行时路径解析层负责把前导 `data/` 替换为 AstrBot 插件数据根目录。本地开发则继续映射到仓库的 `data/`。不要在配置默认值中重复写完整 `data/plugin_data/astrbot_plugin_imagestudio/` 前缀。 -The backend contract is available from: +## 本地前端开发 + +Windows 上若 PowerShell 阻止 `npm.ps1`,使用 `npm.cmd`: + +```powershell +cd frontend +npm.cmd install +npm.cmd run typecheck +npm.cmd run dev:plugin +``` + +`dev:plugin` 只用于本地开发。生产构建: + +```powershell +npm.cmd run build:plugin +``` + +它会更新 `pages/imagestudio/`。`dev:web` 与 `build:web` 只用于独立预览或兼容产物,不代表 AstrBot 注册了 `/web` 生产路由。 + +## 数据库后端 + +默认配置选择 SQLite。`backend: auto` 仅在 PostgreSQL 已启用且连接字段完整时切换到 PG;`backend: sqlite` 强制 SQLite;`backend: postgresql` 在驱动或配置缺失时快速失败。 + +PG 驱动是可选依赖,优先使用 `psycopg`,兼容回退到 `psycopg2`。不要在默认 `requirements.txt` 中启用 PG 驱动,以免 SQLite 用户被强制安装额外二进制依赖。 + +## 验证 + +```powershell +python -m compileall adapters backend core database engines scripts tests main.py + +cd frontend +npm.cmd run typecheck +npm.cmd run build:plugin +npm.cmd run build:web +``` + + +生产构建必须同时提交 `pages/imagestudio/` 的最新插件页面产物。`web/` 仅是独立预览兼容产物,但若仓库保留它,也应与同一份前端源码同步构建,避免两个产物行为漂移。 + +## API 契约 + +契约 endpoint 为相对路径: ```text GET contract ``` -The exported contract includes `api_base`, routes, permissions, error codes, and the standard success/error response shape. - -## Database Backend - -Default behavior: +插件页面通过 bridge 调用时传 `contract`;直接请求时使用: ```text -database.backend = auto -postgresql.enabled = false -selected backend = sqlite +GET /api/v1/plugins/extensions/astrbot_plugin_imagestudio/contract ``` - -When PostgreSQL is configured: - -```yaml -database: - backend: auto - postgresql: - enabled: true - host: 127.0.0.1 - port: 5432 - database: imagestudio - user: imagestudio - password: "" -``` - -With `backend: auto`, enabling a complete PostgreSQL config makes PostgreSQL the selected backend. Use `backend: sqlite` to force SQLite even when PostgreSQL fields exist, or `backend: postgresql` to fail fast when PG config/driver is missing. - -PostgreSQL support is optional and imports `psycopg` first, then falls back to `psycopg2`. - -## Runtime Rule - -Do not start a separate FastAPI, Flask, or Vite production port for the plugin. Dev servers may only be used for local frontend development before build output is copied into AstrBot static paths. diff --git a/INSTALL.md b/INSTALL.md index 15ebf07..e9c42e0 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,40 +1,164 @@ -# Installation and Packaging +# 安装与打包 -ImageStudio is an AstrBot plugin. Do not run it as a standalone FastAPI, Flask, or Vite production service. +ImageStudio 是 AstrBot 插件,不是独立 Web 服务。 -## Install Into AstrBot +## 兼容性 -1. Clone or copy this repository into AstrBot's plugin directory as `astrbot_plugin_imagestudio`. -2. Keep the repository root files in place: `metadata.yaml`, `main.py`, `config.yaml`, `_conf_schema.json`. -3. Restart or reload AstrBot so it can discover the plugin metadata. -4. AstrBot should call `main.get_plugin_manifest()` or `main.register_plugin(runtime)` to discover extension routes and the web mount. +- AstrBot `>= 4.25.0`。 +- Windows、Linux、macOS 与 Docker(以 AstrBot 本身支持的平台为准)。 +- Python 版本跟随对应 AstrBot 发行版。 +- SQLite 默认可用;PostgreSQL 需要额外安装驱动。 -The backend API is mounted under: +## 安装到 AstrBot + +推荐通过 AstrBot 插件管理器安装。手工安装时,将仓库完整复制到 AstrBot 配置的插件目录,目录名保持为 `astrbot_plugin_imagestudio`;常见安装位置为: ```text -/api/v1/plugins/extensions/astrbot_plugin_imagestudio +data/plugins/astrbot_plugin_imagestudio/ ``` -The standalone web entry is mounted under: +至少保留以下运行文件和目录: ```text -/api/v1/plugins/extensions/astrbot_plugin_imagestudio/web +main.py +metadata.yaml +_conf_schema.json +config.yaml +pages/imagestudio/index.html +adapters/ +backend/ +core/ +database/ +engines/ ``` -## Database +重启或在 AstrBot 中重载插件。AstrBot 会发现 `Star` 插件类,调用其 `initialize()`;停用或重载时调用 `terminate()`。插件在 `initialize()` 中使用 `context.register_web_api` 注册 API。 -SQLite is the default and requires no extra service: +## 插件页面与 API + +AstrBot 自动发现 `pages/imagestudio/index.html`。请从 AstrBot 插件管理界面打开页面,不要把未注册的 `/web` 静态目录当作生产入口。 + +页面 bridge 使用相对 endpoint: + +```text +ui/bootstrap +jobs +settings/current +``` + +兼容路由如下: + +```text +# AstrBot v4.25 legacy bridge +/api/plug/astrbot_plugin_imagestudio/ + +# 当前直接 API +/api/v1/plugins/extensions/astrbot_plugin_imagestudio/ +``` + +`window.AstrBotPluginPage.apiGet("ui/bootstrap")` 这类 bridge 调用只传相对 endpoint;直接请求才拼接当前 `/api/v1/plugins/extensions/...` 前缀。 + +## 配置图片生成 Provider + +ImageStudio 不内置可直接用于生产的云端模型。首次生图前,请在 AstrBot 的插件配置页填写: + +```text +default_generation_model: image-main +generation_provider_id: imagestudio_default +generation_provider_base_url: https://provider.example.com +generation_provider_image_endpoint: /v1/images/generations +generation_provider_model: provider-model-id +generation_provider_route: image-main +generation_provider_api_key_env_var: IMAGESTUDIO_PROVIDER_API_KEY +generation_provider_timeout_seconds: 180 +``` + +并在启动 AstrBot 的进程环境中提供密钥,例如: + +```powershell +$env:IMAGESTUDIO_PROVIDER_API_KEY = "replace-with-secret" +``` + +对应的 `config.yaml` 结构为: + +```yaml +generation: + default_model: image-main + provider: + provider_id: imagestudio_default + base_url: https://provider.example.com + image_endpoint: /v1/images/generations + model_id: provider-model-id + route_name: image-main + api_key_env_var: IMAGESTUDIO_PROVIDER_API_KEY + timeout_seconds: 180 +``` + +`route_name` 是 ImageStudio 使用的逻辑路由名,`model_id` 是 Provider 接收的真实模型名。无鉴权的本地 Provider 可以留空 `api_key_env_var`;除此之外不要把 API Key 直接写入仓库或 `config.yaml`。配置不完整时,生图接口会返回 `MODEL_CAPABILITY_UNCONFIRMED`;Provider 请求失败时任务和节点会进入失败状态,不会使用 mock 结果冒充生产成功。 + +## 配置素材视觉识别 Provider + +素材识别需要一个支持图片输入的 OpenAI 兼容多模态模型。在 AstrBot 插件配置页至少填写模型和路由;连接字段留空时会复用上面的生图 Provider。若 Vision 地址、密钥或超时不同而 ID 留空,插件会自动派生独立 Provider ID,避免覆盖生图配置: + +```text +vision_provider_model: vision-model-id +vision_provider_route: material_vlm +vision_provider_endpoint: /v1/chat/completions +``` + +如需使用独立的视觉服务,可再填写: + +```text +vision_provider_id: vision_provider +vision_provider_base_url: https://vision-provider.example.com +vision_provider_api_key_env_var: IMAGESTUDIO_VISION_API_KEY +vision_provider_timeout_seconds: 180 +vision_provider_enabled: true +``` + +等价的 `config.yaml` 配置如下。`provider_id`、`base_url`、`api_key_env_var` 和超时留空时均复用 `generation.provider`,但 `model_id` 必须显式配置: + +```yaml +vision: + default_route: material_vlm + provider: + provider_id: "" + base_url: "" + vision_endpoint: /v1/chat/completions + model_id: vision-model-id + route_name: material_vlm + api_key_env_var: "" + timeout_seconds: 180 + enabled: true +``` + +识别时 ImageStudio 会从插件 VFS 读取 `storage://` 素材文件,校验实际大小、扩展名和 PNG/JPEG/GIF/WebP 文件签名,再编码为 `data:image/...;base64` 发送给视觉 Provider。未配置有效路由或密钥时接口返回 `MODEL_CAPABILITY_UNCONFIRMED`;Provider 调用失败时素材保持 `pending_recognition`,只记录脱敏后的失败类型,不会生成或允许确认 mock 提示词与标签。 + +## 运行数据 + +AstrBot 插件的持久化数据根目录为: + +```text +data/plugin_data/astrbot_plugin_imagestudio/ +``` + +默认 SQLite 配置: ```yaml database: backend: auto sqlite: + # 逻辑路径;AstrBot 运行时映射到插件数据目录 path: data/db/imagestudio.db postgresql: enabled: false ``` -PostgreSQL is selected when `backend: postgresql`, or when `backend: auto` and `postgresql.enabled` plus connection fields are configured. +配置中的 `data/...` 是可移植的逻辑路径,AstrBot 运行时会把它映射到上述插件数据根目录;不要在配置里再次手工拼接 `data/plugin_data/astrbot_plugin_imagestudio`,否则可能产生重复目录。图片、素材、缩略图、归档和临时文件也位于同一个插件数据根目录。升级插件源码时不要删除该目录;备份时应单独备份它。 + +## 可选 PostgreSQL + +选择 `backend: postgresql`,或在 `backend: auto` 下启用并完整配置 PostgreSQL 后,ImageStudio 会使用 PG: ```yaml database: @@ -48,59 +172,43 @@ database: password: "" ``` -The plugin supports either `psycopg` or `psycopg2` when PostgreSQL is selected. +默认 SQLite 不需要第三方数据库驱动。为避免给所有安装者强制安装 PG 驱动,`requirements.txt` 中不启用 PostgreSQL 依赖;需要 PG 时任选其一安装: -## Secrets +```powershell +python -m pip install "psycopg[binary]>=3.1" +# 或兼容驱动 +python -m pip install "psycopg2-binary>=2.9" +``` -Do not commit `.env*`, API keys, DSNs, or integration tokens. Runtime tokens are rotated through: +## 密钥 + +不要提交 `.env*`、API Key、DSN、数据库密码或集成 token。私伴集成 token 的轮换 endpoint 为: ```text POST integrations/private-companion/token/rotate ``` -The rotate response returns the new token once. Config stores only salt and hash. +返回的新 token 只展示一次,配置仅保存 salt 与 hash。 -## Smoke Checks - -Run the non-UI backend smoke: +## 验证 ```powershell -python scripts/imagestudio_non_ui_smoke.py +python -m compileall adapters backend core database engines main.py ``` -Run the PostgreSQL plan without connecting: - ```powershell -python scripts/imagestudio_pg_smoke.py --dry-run ``` -Run full tests: +## 打包 -```powershell -python -m unittest discover -s tests -python -m compileall adapters backend core database engines scripts tests main.py -``` - -Optional live PostgreSQL smoke: - -```powershell -$env:IMAGESTUDIO_TEST_PG_DSN = "postgresql://user:password@127.0.0.1:5432/imagestudio_test" -python -m unittest tests.test_pg_smoke -``` - -## Package - -Use Git so ignored local data and secrets are excluded: +使用 Git 归档可自动排除未跟踪的本地数据和多数开发产物: ```powershell git archive --format=zip --output astrbot_plugin_imagestudio.zip HEAD ``` -The package should include source modules, migrations, built/static assets, config templates, and docs. It should not include: +发布包应包含源码、迁移、`pages/imagestudio/` 构建产物、配置模板和文档;不应包含: -- `.git/` -- `.env*` -- `data/*` runtime contents -- `node_modules/` -- `__pycache__/` -- local test databases or generated media +- `.git/`、`.env*`、`node_modules/`、`__pycache__/`。 +- `data/plugin_data/astrbot_plugin_imagestudio/` 中的任何运行数据。 +- 本地测试数据库、生成图片、日志、缓存或密钥。 diff --git a/README.md b/README.md index 61bc060..1de8443 100644 --- a/README.md +++ b/README.md @@ -1,38 +1,114 @@ # AstrBot ImageStudio -ImageStudio 是一个给 AstrBot 使用的角色资产与生图工作台插件。它把角色档案、工作流、素材库、衣橱、队列、日志和模型能力探测放在同一个插件里,方便后续为角色持续生成、筛选和管理图片。 +ImageStudio 是运行在 AstrBot 内的角色资产与生图工作台插件,提供角色档案、工作流、素材库、衣橱、任务队列、日志和模型能力档案等功能。 -## 现在能用什么 +## 兼容范围 -- 查看插件运行概览、队列状态、最近任务和最近图片。 -- 创建角色档案,维护基础设定、外观提示词、风格提示词和默认模型路由。 -- 创建角色工作流,按节点生成、重试、采纳、选择或归档图片。 -- 管理素材库、素材分类、衣橱条目,并做每日穿搭匹配。 -- 查看任务队列、失败任务、死信队列、生成日志、错误日志和审计日志。 -- 探测 OpenAI 兼容生图接口的能力,保存模型/路线能力档案。 -- 默认使用 SQLite;在配置 PostgreSQL 后可以切换到 PG 数据库。 -- 提供 AstrBot 插件卡片概览页,也提供独立工作台入口。 +- AstrBot:`4.25.0` 及以上。 +- 运行平台:AstrBot 支持的 Windows、Linux、macOS 和 Docker 环境。 +- 数据库:默认 SQLite;PostgreSQL 为可选后端。 +- 生产运行方式:由 AstrBot 加载插件,不启动独立 FastAPI、Flask 或 Vite 服务。 -## 怎么打开 +## AstrBot 加载方式 -在 AstrBot 插件页面打开 ImageStudio 时,默认看到的是轻量概览页:它用于确认插件状态、队列、最近任务、权限和快捷入口。 +AstrBot 通过 `main.py` 中的 `Star` 插件类自动发现 ImageStudio,并管理其生命周期: -完整工作台在概览页的“工作台”入口里,路径由 AstrBot 挂载到: +- `initialize()`:读取 AstrBot 配置、初始化持久化数据,并通过 `context.register_web_api` 注册插件 API。 +- `terminate()`:插件停用或重载时释放运行资源。 +- `pages/imagestudio/index.html`:由 AstrBot 自动发现为插件页面,无需手工注册静态 `/web` 入口。 + +## 打开界面 + +安装并重载插件后,从 AstrBot 的插件管理界面打开 ImageStudio。生产页面来自: ```text -/api/v1/plugins/extensions/astrbot_plugin_imagestudio/web +pages/imagestudio/index.html ``` -## 当前 UI 状态 +仓库中的 `web/` 是独立前端构建/兼容产物。除非宿主显式注册对应静态路由,否则 `/api/v1/plugins/extensions/astrbot_plugin_imagestudio/web` 不是 AstrBot 保证提供的生产入口。 -插件卡片页目前是概览页,这是有意保留的轻量入口,不代表后端功能只有这一页。 +## API 兼容模式 -完整工作台代码已经包含概览、角色、工作流、素材、衣橱、队列、能力探测、日志和设置等模块。后续还需要继续做 UI 体验整理、布局打磨和更顺手的操作流。 +插件页面优先使用 `window.AstrBotPluginPage` bridge。传给 `apiGet`、`apiPost`、`apiPut` 或 `apiDelete` 的值必须是相对 endpoint,例如: -## 配置说明 +```text +ui/bootstrap +dashboard/summary +nodes//generate +``` -AstrBot 小齿轮设置用于安装后最常改的项目:默认模型、私伴插件集成、SQLite/PG 数据库配置。 +ImageStudio 同时兼容两套路由前缀: -更完整的配置、导出、导入、校验、差异对比和快照回滚在独立工作台的“设置”页里。 +- AstrBot v4.25 legacy bridge:`/api/plug/astrbot_plugin_imagestudio/`。 +- 当前扩展 API:`/api/v1/plugins/extensions/astrbot_plugin_imagestudio/`。 -安装、打包、数据库和 smoke 测试命令见 [INSTALL.md](INSTALL.md)。 +bridge 会处理宿主前缀;前端不应把完整 `/api/plug/...` 或 `/api/v1/...` 地址传给 bridge。直接 `fetch` 时才使用当前 `/api/v1` 前缀。 + +## 运行与权限边界 + +ImageStudio 当前定位为 AstrBot WebUI 内的管理工作台。Bridge 只接受 AstrBot 已认证请求,并为插件页面授予显式的写入、配置、素材和任务重试能力;不会信任浏览器自行提交的用户名、用户 ID 或转发 IP 请求头。 + +角色档案列表和档案读写已按 AstrBot 登录身份隔离,但工作流、图片、任务、衣橱、Dashboard 和媒体索引尚未形成完整的跨资源多租户授权链。因此当前版本应仅暴露给可信 AstrBot 管理员,不应作为面向多个互不信任用户的公共工作台部署。 + +## 配置生图 Provider + +生产生图需要一个 OpenAI 兼容的图片生成接口。请在 AstrBot 插件配置中填写至少以下字段: + +```text +default_generation_model = image-main +generation_provider_base_url = https://provider.example.com +generation_provider_model = provider-model-id +generation_provider_route = image-main +generation_provider_api_key_env_var = IMAGESTUDIO_PROVIDER_API_KEY +``` + +然后在 AstrBot 进程环境中设置对应 API Key。无鉴权的本地 Provider 可以把 `generation_provider_api_key_env_var` 留空。Provider 未配置完整或调用失败时,生产模式会返回明确错误并把任务/节点标记为失败,不会把 mock 图片伪装成成功结果。完整示例见 [INSTALL.md](INSTALL.md)。 + +素材识别还需配置 `vision_provider_model`、`vision_provider_route` 和 `vision_provider_endpoint`。Vision 的 Provider ID、根地址、API Key 环境变量和超时可留空以复用生图 Provider;如果连接或超时不同但 ID 留空,插件会自动派生独立 Vision Provider ID,避免覆盖生图配置。识别模型必须显式填写。识别链会读取并校验真实 `storage://` 图片后发送 data URL,无有效路由或 Provider 失败时不会回退到 mock 结果。完整配置见 [INSTALL.md](INSTALL.md)。 + +## 主要功能 + +- 查看运行概览、队列、最近任务、最近图片和日志。 +- 创建角色档案及角色工作流,按节点生成、重试、审查、采纳和归档图片。 +- 管理素材分类、素材库、衣橱与穿搭匹配。 +- 登记、确认和校验 OpenAI 兼容模型能力档案;当前 discovery endpoint 是安全的元数据检查,不会主动向任意地址发起探测请求。 +- 使用 SQLite 零配置运行,或按需切换 PostgreSQL。 + +## 聊天命令 + +插件提供 `/影棚` 和短别名 `/yp`,两套命令行为一致。聊天权限在设置页“聊天权限”中配置。 + +```text +/yp 帮助 +/yp 状态 +/yp 单图 <提示词> [--角色 名称] [--模型 路由] [--尺寸 1K|2K|4K] [--数量 1-4] +/yp 快速角色 <角色名和描述> +/yp 确认 +/yp 工作流 <角色名或ID> +/yp 节点 +/yp 生成 +/yp 下一步 +/yp 候选图 +/yp 选择 <序号> +/yp 审核 +/yp 高清 2K|4K +/yp 素材导入 <分类名称或ID> (命令消息需附带图片) +/yp 加入衣橱 <角色名或ID> <衣橱分类> +/yp 任务 [task_id] +/yp 取消 +/yp 重试 +``` + +快速角色和详细角色工作流要求群聊、用户白名单同时命中;单图允许任意一个白名单命中。聊天生成还受用户/群聊并发、每日额度和 2K/4K 开关约束。单图在后台线程执行,命令会立即返回请求 ID,完成后通过 AstrBot 统一消息来源主动发送图片。 + +## 持久化数据 + +运行数据统一放在 AstrBot 数据目录中: + +```text +data/plugin_data/astrbot_plugin_imagestudio/ +``` + +数据库、图片、缩略图、素材、归档和临时文件都应位于该目录下,不能依赖插件源码目录可写。安装、升级和打包细节见 [INSTALL.md](INSTALL.md),开发约定见 [DEVELOPMENT.md](DEVELOPMENT.md)。 + +本轮 AstrBot 兼容、后端、前端、功能闭环与安全审计记录见 [docs/code-audit-2026-07-10.md](docs/code-audit-2026-07-10.md)。 diff --git a/_conf_schema.json b/_conf_schema.json index c3ef70d..250d583 100644 --- a/_conf_schema.json +++ b/_conf_schema.json @@ -1,9 +1,104 @@ { "default_generation_model": { - "description": "默认生图模型或模型路由名。留空时由工作流或能力探测结果选择。", + "description": "默认生图路由名。generation_provider_route 留空时也使用此值。", "type": "string", "default": "" }, + "generation_provider_id": { + "description": "OpenAI 兼容生图提供方 ID。", + "type": "string", + "default": "imagestudio_default" + }, + "generation_provider_base_url": { + "description": "OpenAI 兼容生图 API 根地址,例如 https://api.openai.com。", + "type": "string", + "default": "" + }, + "generation_provider_image_endpoint": { + "description": "生图 endpoint。", + "type": "string", + "default": "/v1/images/generations" + }, + "generation_provider_model": { + "description": "提供方实际模型 ID;留空时复用默认生图路由名。", + "type": "string", + "default": "" + }, + "generation_provider_route": { + "description": "ImageStudio 模型路由名;留空时复用 default_generation_model。", + "type": "string", + "default": "" + }, + "generation_provider_api_key_env_var": { + "description": "保存 API Key 的环境变量名。无鉴权的本地提供方可留空。", + "type": "string", + "default": "" + }, + "generation_provider_timeout_seconds": { + "description": "生图请求超时秒数。", + "type": "string", + "default": "180" + }, + "vision_provider_id": { + "description": "素材视觉识别提供方 ID;连接配置与生图 Provider 一致时留空可复用,否则会自动派生独立 ID。", + "type": "string", + "default": "" + }, + "vision_provider_base_url": { + "description": "素材视觉识别 API 根地址;留空时复用生图提供方地址。", + "type": "string", + "default": "" + }, + "vision_provider_endpoint": { + "description": "OpenAI 兼容视觉识别 endpoint。", + "type": "string", + "default": "/v1/chat/completions" + }, + "vision_provider_model": { + "description": "素材视觉识别使用的真实多模态模型 ID。该字段必须显式配置。", + "type": "string", + "default": "" + }, + "vision_provider_route": { + "description": "ImageStudio 素材视觉识别逻辑路由名。", + "type": "string", + "default": "material_vlm" + }, + "vision_provider_api_key_env_var": { + "description": "视觉识别 API Key 的环境变量名;留空时复用生图提供方配置。", + "type": "string", + "default": "" + }, + "vision_provider_timeout_seconds": { + "description": "素材视觉识别请求超时秒数。", + "type": "string", + "default": "180" + }, + "vision_provider_enabled": { + "description": "是否启用素材视觉识别路由。", + "type": "bool", + "default": true + }, + "model_material_recognition": {"description":"素材导入解析使用的 Vision 路由。","type":"string","default":"material_vlm"}, + "model_profile_parse": {"description":"外部角色导入解析使用的文本模型路由。","type":"string","default":""}, + "model_persona_split": {"description":"角色设定拆解使用的文本模型路由。","type":"string","default":""}, + "model_prompt_planning": {"description":"提示词规划与扩写使用的文本模型路由。","type":"string","default":""}, + "model_cross_review_primary": {"description":"候选图交叉审核主 Vision 路由。","type":"string","default":"material_vlm"}, + "model_cross_review_secondary": {"description":"候选图交叉审核复核 Vision 路由。","type":"string","default":"material_vlm"}, + "model_quick_generation": {"description":"快速生图使用的模型路由。","type":"string","default":""}, + "model_portrait_generation": {"description":"头像草稿使用的生图路由。","type":"string","default":""}, + "model_halfbody_generation": {"description":"半身角色图使用的生图路由。","type":"string","default":""}, + "model_fullbody_generation": {"description":"全身角色图使用的生图路由。","type":"string","default":""}, + "model_sheet_generation": {"description":"设定图与姿势图使用的生图路由。","type":"string","default":""}, + "model_hires_fix": {"description":"高清修复使用的生图路由。","type":"string","default":""}, + "chat_group_allowlist": {"description":"允许使用 ImageStudio 聊天工作流的群聊 ID 白名单。","type":"list","default":[]}, + "chat_user_allowlist": {"description":"允许使用 ImageStudio 聊天功能的用户 ID 白名单。","type":"list","default":[]}, + "chat_user_concurrency_limit": {"description":"单个聊天用户同时运行的生成任务上限。","type":"int","default":2}, + "chat_group_concurrency_limit": {"description":"单个群聊同时运行的生成任务上限。","type":"int","default":5}, + "chat_daily_user_generation_limit": {"description":"单个用户每日聊天生成次数上限。","type":"int","default":30}, + "chat_daily_group_generation_limit": {"description":"单个群聊每日聊天生成次数上限。","type":"int","default":100}, + "chat_allow_2k": {"description":"允许聊天命令申请 2K 图片。","type":"bool","default":true}, + "chat_allow_4k": {"description":"允许聊天命令申请 4K 图片。","type":"bool","default":false}, "enable_private_companion_integration": { "description": "允许 astrbot_plugin_private_companion 调用 ImageStudio 的穿搭和素材集成接口。", "type": "bool", @@ -15,7 +110,7 @@ "default": "auto" }, "sqlite_path": { - "description": "SQLite 数据库文件路径。默认放在插件目录的 data/db/imagestudio.db。", + "description": "SQLite 逻辑路径。AstrBot 运行时会把 data/... 映射到 data/plugin_data/astrbot_plugin_imagestudio/...;默认物理落点为该插件数据目录下的 db/imagestudio.db。", "type": "string", "default": "data/db/imagestudio.db" }, diff --git a/adapters/astrbot_adapter.py b/adapters/astrbot_adapter.py index e1808f6..1f35b42 100644 --- a/adapters/astrbot_adapter.py +++ b/adapters/astrbot_adapter.py @@ -44,9 +44,12 @@ def normalize_extension_path(path: str) -> str: def resolve_route(method: str, path: str) -> RouteMatch | None: normalized_method = str(method or "").strip().upper() endpoint = normalize_extension_path(path) - for route in ROUTES: - if route.method != normalized_method: - continue + candidates = sorted( + (route for route in ROUTES if route.method == normalized_method), + key=_route_specificity, + reverse=True, + ) + for route in candidates: params = _match_endpoint(route.endpoint, endpoint) if params is None: continue @@ -61,6 +64,21 @@ def resolve_route(method: str, path: str) -> RouteMatch | None: return None +def _route_specificity(route: RouteSpec) -> tuple[int, int, int]: + """Prefer concrete routes over parameterized routes. + + Route manifests are maintained by feature area, so relying on declaration + order makes a generic route such as ``jobs/{task_id}/retry`` shadow + ``jobs/dead-letter/retry``. Literal segment count is the primary signal; + total segment count and fewer parameters break ties deterministically. + """ + + segments = [segment for segment in route.endpoint.split("/") if segment] + parameter_count = sum(1 for segment in segments if segment.startswith("{") and segment.endswith("}")) + literal_count = len(segments) - parameter_count + return literal_count, len(segments), -parameter_count + + def runtime_route_table() -> list[dict[str, Any]]: return [ { diff --git a/adapters/astrbot_card_renderer.py b/adapters/astrbot_card_renderer.py new file mode 100644 index 0000000..ba2949e --- /dev/null +++ b/adapters/astrbot_card_renderer.py @@ -0,0 +1,84 @@ +"""Render compact PNG cards for AstrBot chat responses.""" + +from __future__ import annotations + +import hashlib +from pathlib import Path +from typing import Iterable + + +CARD_DIR = Path(__file__).resolve().parent.parent / "data" / "chat_cards" + + +def render_help_card(lines: Iterable[str]) -> Path: + return _render_card( + key="help-v1", + eyebrow="IMAGESTUDIO COMMANDS", + title="影棚命令帮助", + subtitle="使用 /影棚 或 /yp 调用", + lines=list(lines), + accent=(40, 169, 151), + ) + + +def render_status_card(*, state: str, version: str, database: str, api: str) -> Path: + return _render_card( + key=f"status-{state}-{version}-{database}-{api}", + eyebrow="IMAGESTUDIO STATUS", + title=f"运行状态 · {state}", + subtitle=f"版本 {version}", + lines=[f"数据库 {database}", f"接口 {api}", "聊天命令框架已加载"], + accent=(40, 169, 151) if state == "正常" else (224, 157, 55), + ) + + +def render_profile_card(*, name: str, base: str, appearance: str, style: str) -> Path: + return _render_card( + key=f"profile-{name}-{base}-{appearance}-{style}", eyebrow="IMAGESTUDIO QUICK PROFILE", + title=name or "快速角色草稿", subtitle="确认后将创建角色档案与工作流", + lines=[f"基础 {base or '—'}", f"外观 {appearance or '—'}", f"风格 {style or '—'}", "回复:确认 / 修改 / 放弃"], + accent=(112, 91, 220), + ) + + +def _render_card(*, key: str, eyebrow: str, title: str, subtitle: str, lines: list[str], accent: tuple[int, int, int]) -> Path: + from PIL import Image, ImageDraw, ImageFont + + CARD_DIR.mkdir(parents=True, exist_ok=True) + target = CARD_DIR / f"{hashlib.sha256(key.encode('utf-8')).hexdigest()[:16]}.png" + if target.exists(): + return target + width = 1000 + height = 300 + max(len(lines), 1) * 66 + image = Image.new("RGB", (width, height), (242, 247, 249)) + draw = ImageDraw.Draw(image) + draw.rounded_rectangle((28, 28, width - 28, height - 28), radius=34, fill=(255, 255, 255), outline=(211, 224, 228), width=2) + draw.rounded_rectangle((28, 28, 52, height - 28), radius=12, fill=accent) + fonts = _fonts(ImageFont) + draw.text((86, 70), eyebrow, font=fonts["small"], fill=accent) + draw.text((86, 112), title, font=fonts["title"], fill=(24, 36, 44)) + draw.text((88, 180), subtitle, font=fonts["body"], fill=(101, 118, 128)) + y = 245 + for index, line in enumerate(lines): + draw.rounded_rectangle((82, y, width - 82, y + 48), radius=12, fill=(244, 249, 250) if index % 2 == 0 else (248, 250, 252)) + draw.text((102, y + 9), line, font=fonts["body"], fill=(43, 59, 68)) + y += 66 + image.save(target, format="PNG", optimize=True) + return target + + +def _fonts(image_font): + candidates = [ + Path("C:/Windows/Fonts/msyh.ttc"), + Path("C:/Windows/Fonts/simhei.ttf"), + Path("/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc"), + Path("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf"), + ] + path = next((item for item in candidates if item.exists()), None) + if path is None: + return {"small": image_font.load_default(), "title": image_font.load_default(), "body": image_font.load_default()} + return { + "small": image_font.truetype(str(path), 20), + "title": image_font.truetype(str(path), 42), + "body": image_font.truetype(str(path), 25), + } diff --git a/adapters/astrbot_command_adapter.py b/adapters/astrbot_command_adapter.py new file mode 100644 index 0000000..265e52f --- /dev/null +++ b/adapters/astrbot_command_adapter.py @@ -0,0 +1,102 @@ +"""Pure helpers for the AstrBot chat command surface.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any + +from core.chat_access_policy import chat_action_allowed + + +@dataclass(frozen=True) +class ChatCommandContext: + user_id: str + group_id: str | None + message_origin: str + platform: str + message_id: str + + +def command_context_from_event(event: Any) -> ChatCommandContext: + return ChatCommandContext( + user_id=_event_value(event, "get_sender_id", "sender_id", "user_id"), + group_id=_optional_event_value(event, "get_group_id", "group_id"), + message_origin=_event_value(event, "unified_msg_origin", "message_origin"), + platform=_event_value(event, "get_platform_name", "platform_name", fallback="unknown"), + message_id=_event_value(event, "message_id", "get_message_id"), + ) + + +def command_allowed(config: dict[str, Any], action: str, context: ChatCommandContext) -> bool: + return chat_action_allowed( + config, + action, + group_id=context.group_id, + user_id=context.user_id, + ) + + +def message_text_from_event(event: Any) -> str: + for name in ("message_str", "get_message_str", "raw_message"): + value = getattr(event, name, None) + if callable(value): + try: + value = value() + except (TypeError, AttributeError): + continue + text = str(value or "").strip() + if text: + return text + return "" + + +def help_text() -> str: + return """🎨 ImageStudio 命令 + +/影棚 帮助 或 /yp 帮助 +/影棚 状态 或 /yp 状态 +/影棚 快速角色 <角色描述> +/影棚 工作流 <角色名或角色 ID> +/影棚 单图 <提示词> + +权限规则: +• 快速角色、角色工作流:群聊和用户白名单必须同时命中 +• 单图:群聊或用户任一白名单命中即可 + +当前为命令框架版本,后续将逐步接入角色选择、节点生成、选图和任务回执。""" + + +def access_denied_text(action: str, context: ChatCommandContext) -> str: + scope = f"群聊 {context.group_id}" if context.group_id else "私聊" + return f"⛔ 当前{scope}或用户 {context.user_id or '未知'}不满足「{action}」的白名单规则。" + + +def accepted_text(action: str, argument: str, context: ChatCommandContext) -> str: + value = argument.strip() or "未提供" + return ( + f"✅ ImageStudio 已接收「{action}」命令框架请求\n\n" + f"输入:{value}\n" + f"平台:{context.platform}\n" + f"用户:{context.user_id or '未知'}\n" + f"群聊:{context.group_id or '私聊'}\n\n" + "当前阶段已完成命令注册、参数接收与白名单校验;业务执行器将在下一阶段接入。" + ) + + +def _event_value(event: Any, *names: str, fallback: str = "") -> str: + for name in names: + value = getattr(event, name, None) + if callable(value): + try: + value = value() + except (TypeError, AttributeError): + continue + text = str(value or "").strip() + if text: + return text + return fallback + + +def _optional_event_value(event: Any, *names: str) -> str | None: + value = _event_value(event, *names) + return value or None diff --git a/adapters/astrbot_command_parser.py b/adapters/astrbot_command_parser.py new file mode 100644 index 0000000..bad2e31 --- /dev/null +++ b/adapters/astrbot_command_parser.py @@ -0,0 +1,76 @@ +"""Parsing for ImageStudio AstrBot command arguments.""" + +from __future__ import annotations + +import shlex + +from backend.dto.chat_command_dto import ChatCommandParseError, SIZE_ALIASES, SingleImageCommand + + +OPTION_NAMES = { + "--角色": "profile_ref", "--profile": "profile_ref", + "--模型": "model_route", "--model": "model_route", + "--尺寸": "size", "--size": "size", + "--数量": "count", "--count": "count", + "--负面": "negative_prompt", "--negative": "negative_prompt", +} +FLAG_OPTIONS = {"--转译": "translation_consent", "--translate": "translation_consent"} + + +def parse_single_image_command(raw: str) -> SingleImageCommand: + try: + tokens = shlex.split(str(raw or ""), posix=True) + except ValueError as exc: + raise ChatCommandParseError(f"参数引号不完整:{exc}") from exc + prompt_parts: list[str] = [] + values: dict[str, str] = {} + index = 0 + while index < len(tokens): + token = tokens[index] + flag = FLAG_OPTIONS.get(token.lower()) + if flag: + values[flag] = "true" + index += 1 + continue + key = OPTION_NAMES.get(token.lower()) + if key: + if index + 1 >= len(tokens) or tokens[index + 1].startswith("--"): + raise ChatCommandParseError(f"{token} 缺少参数值") + values[key] = tokens[index + 1] + index += 2 + continue + if token.startswith("--"): + raise ChatCommandParseError(f"未知参数:{token}") + prompt_parts.append(token) + index += 1 + prompt = " ".join(prompt_parts).strip() + if not prompt: + raise ChatCommandParseError("请提供生图提示词") + size_text = values.get("size", "1K").lower() + size = SIZE_ALIASES.get(size_text, values.get("size", "1920x1080")) + if size not in SIZE_ALIASES.values(): + raise ChatCommandParseError("尺寸仅支持 1K、2K、4K") + try: + count = int(values.get("count", "1")) + except ValueError as exc: + raise ChatCommandParseError("数量必须是整数") from exc + if count < 1 or count > 4: + raise ChatCommandParseError("数量必须在 1 到 4 之间") + return SingleImageCommand( + prompt=prompt, + profile_ref=values.get("profile_ref"), + model_route=values.get("model_route"), + size=size, + count=count, + negative_prompt=values.get("negative_prompt"), + translation_consent=values.get("translation_consent") == "true", + ) + + +def single_image_usage(error: str) -> str: + return ( + f"❌ {error}\n\n" + "示例:\n" + "/yp 单图 雨夜街道中的少女 --角色 米卡 --尺寸 1K --数量 1\n\n" + "可用参数:--角色、--模型、--尺寸、--数量、--负面、--转译" + ) diff --git a/adapters/astrbot_media_adapter.py b/adapters/astrbot_media_adapter.py new file mode 100644 index 0000000..66d7684 --- /dev/null +++ b/adapters/astrbot_media_adapter.py @@ -0,0 +1,20 @@ +"""Extract image references from AstrBot message components.""" + +from __future__ import annotations + +from typing import Any + + +def image_references_from_event(event: Any) -> list[str]: + message_obj = getattr(event, "message_obj", None) + chain = getattr(message_obj, "message", None) or getattr(event, "message_chain", None) or [] + references: list[str] = [] + for component in chain if isinstance(chain, (list, tuple)) else []: + if "image" not in component.__class__.__name__.lower(): + continue + for name in ("url", "file", "path", "image_url"): + text = str(getattr(component, name, None) or "").strip() + if text and text not in references: + references.append(text) + break + return references diff --git a/backend/README.md b/backend/README.md index 6408f69..7bc3511 100644 --- a/backend/README.md +++ b/backend/README.md @@ -1,13 +1,16 @@ # Backend -The backend exposes v3 P0 APIs through AstrBot plugin extension routes. +后端由 AstrBot 插件生命周期托管,不启动独立 Web 服务。 -Initial backend tasks: +- `main.py` 中的 `Star` 子类在 `initialize()` 阶段调用 `context.register_web_api`。 +- `terminate()` 用于插件停用或重载时清理资源。 +- route registry 和业务层只维护 `ui/bootstrap`、`profiles`、`nodes/{node_id}/generate` 等相对 endpoint。 +- AstrBot v4.25 插件页面 bridge 兼容 `/api/plug/astrbot_plugin_imagestudio/`。 +- 当前直接调用兼容 `/api/v1/plugins/extensions/astrbot_plugin_imagestudio/`。 +- 前端 bridge 必须传相对 endpoint,不能传完整宿主 URL。 -- Common response DTO and error code handling. -- Request context, permission checks, and request id creation. -- Audit logging foundation. -- Database helper and migration runner. -- Route modules for `ui/bootstrap`, `dashboard/summary`, profiles, workflows, nodes, images, jobs, settings, storage, media, and web hosting. +插件 UI 由 `pages/imagestudio/index.html` 自动发现。未注册的 `/web` 目录不是生产入口。 -Do not start a separate web server. Register routes through AstrBot plugin extension APIs. +SQLite、媒体、日志和缓存等运行数据应写入 `data/plugin_data/astrbot_plugin_imagestudio/`,不要写入插件源码目录。 + +当前 API 的产品边界是可信 AstrBot WebUI 管理员工作台。角色档案已按调用身份过滤,但其他资源尚未完成统一祖先链 owner 授权;在补齐该授权与跨用户回归测试前,不应把本插件作为互不信任用户共享的多租户服务发布。 diff --git a/backend/dto/chat_command_dto.py b/backend/dto/chat_command_dto.py new file mode 100644 index 0000000..26407de --- /dev/null +++ b/backend/dto/chat_command_dto.py @@ -0,0 +1,34 @@ +"""DTOs for platform-neutral ImageStudio chat commands.""" + +from __future__ import annotations + +from dataclasses import dataclass + + +SIZE_ALIASES = {"1k": "1920x1080", "2k": "2560x1440", "4k": "3840x2160"} + + +@dataclass(frozen=True) +class SingleImageCommand: + prompt: str + profile_ref: str | None = None + model_route: str | None = None + size: str = "1920x1080" + count: int = 1 + negative_prompt: str | None = None + translation_consent: bool = False + + def to_payload(self) -> dict: + return { + "prompt": self.prompt, + "profile_ref": self.profile_ref, + "model_route": self.model_route, + "size": self.size, + "count": self.count, + "negative_prompt": self.negative_prompt, + "translation_consent": self.translation_consent, + } + + +class ChatCommandParseError(ValueError): + pass diff --git a/backend/dto/common.py b/backend/dto/common.py index d602f4a..2ddc536 100644 --- a/backend/dto/common.py +++ b/backend/dto/common.py @@ -17,6 +17,7 @@ class ErrorCode(StrEnum): TASK_NOT_CANCELABLE = "TASK_NOT_CANCELABLE" MODEL_CAPABILITY_UNCONFIRMED = "MODEL_CAPABILITY_UNCONFIRMED" MODEL_CAPABILITY_UNSUPPORTED = "MODEL_CAPABILITY_UNSUPPORTED" + GENERATION_FAILED = "GENERATION_FAILED" STORAGE_WRITE_FAILED = "STORAGE_WRITE_FAILED" MEDIA_NOT_FOUND = "MEDIA_NOT_FOUND" UPLOAD_TOO_LARGE = "UPLOAD_TOO_LARGE" diff --git a/backend/dto/image_dto.py b/backend/dto/image_dto.py index 7759995..dc7242a 100644 --- a/backend/dto/image_dto.py +++ b/backend/dto/image_dto.py @@ -52,3 +52,19 @@ class GenerationImage: "metadata": self.metadata, "created_at": self.created_at, } + + +@dataclass(frozen=True) +class ImageListResult: + items: list[GenerationImage] + total: int + page: int + page_size: int + + def to_dict(self) -> dict[str, Any]: + return { + "items": [item.to_dict() for item in self.items], + "total": self.total, + "page": self.page, + "page_size": self.page_size, + } diff --git a/backend/dto/material_dto.py b/backend/dto/material_dto.py index 80f8729..7721efc 100644 --- a/backend/dto/material_dto.py +++ b/backend/dto/material_dto.py @@ -6,6 +6,66 @@ from dataclasses import dataclass from typing import Any +@dataclass(frozen=True) +class RawAnalysis: + schema_version: str + prompt_version: str + created_at: str + model_source: dict[str, Any] + attributes: dict[str, Any] + description: str | None = None + source_payload: dict[str, Any] | None = None + + def to_dict(self) -> dict[str, Any]: + return { + "schema_version": self.schema_version, + "prompt_version": self.prompt_version, + "created_at": self.created_at, + "model_source": self.model_source, + "attributes": self.attributes, + "description": self.description, + "source_payload": self.source_payload or {}, + } + + +@dataclass(frozen=True) +class NormalizedAssetProfile: + schema_version: str + prompt_version: str + created_at: str + category: str + role: str + body_slots: list[str] + wear_layer: str | None + attributes: dict[str, Any] + must_preserve: list[str] + forbidden_changes: list[str] + confidence: float + requires_confirmation: bool + confirmed: bool = False + user_locked: bool = False + stackable: bool = False + + def to_dict(self) -> dict[str, Any]: + return dict(self.__dict__) + + +@dataclass(frozen=True) +class GenerationCapsule: + schema_version: str + prompt_version: str + created_at: str + tier: str + prompt_parts: list[str] + negative_parts: list[str] + must_preserve: list[str] + forbidden_changes: list[str] + legacy: bool = False + + def to_dict(self) -> dict[str, Any]: + return dict(self.__dict__) + + @dataclass(frozen=True) class MaterialCategory: category_id: str diff --git a/backend/dto/profile_dto.py b/backend/dto/profile_dto.py index 62a2409..ae3f347 100644 --- a/backend/dto/profile_dto.py +++ b/backend/dto/profile_dto.py @@ -25,6 +25,7 @@ class CharacterProfile: metadata: dict[str, Any] created_at: str updated_at: str + generation_kind: str = "ungenerated" def to_dict(self) -> dict[str, Any]: return { @@ -45,6 +46,7 @@ class CharacterProfile: "metadata": self.metadata, "created_at": self.created_at, "updated_at": self.updated_at, + "generation_kind": self.generation_kind, } diff --git a/backend/route_registry.py b/backend/route_registry.py index 694e4d4..4fb37ef 100644 --- a/backend/route_registry.py +++ b/backend/route_registry.py @@ -19,6 +19,12 @@ PERMISSION_MATRIX: dict[str, dict[str, Any]] = { "required_permission": "can_manage_materials", "description": "material library management", }, + "retry": { + "auth_required": True, + "required_permission": "can_retry_tasks", + "required_permissions": ["can_write", "can_retry_tasks"], + "description": "task retry management (write + retry capabilities)", + }, "public_token": {"auth_required": False, "required_permission": None, "description": "signed public token"}, } @@ -39,6 +45,10 @@ class RouteSpec: "permission": self.permission, "auth_required": matrix["auth_required"], "required_permission": matrix["required_permission"], + "required_permissions": list( + matrix.get("required_permissions") + or ([matrix["required_permission"]] if matrix["required_permission"] else []) + ), } @@ -69,11 +79,12 @@ ROUTES: tuple[RouteSpec, ...] = ( RouteSpec("POST", "workflows/{workflow_id}/branches", "backend.routes.branch_routes.create_workflow_branch", "write"), RouteSpec("POST", "workflows/{workflow_id}/switch-branch", "backend.routes.branch_routes.switch_workflow_branch", "write"), RouteSpec("GET", "workflows/{workflow_id}/history", "backend.routes.history_routes.list_workflow_history", "read"), + RouteSpec("POST", "workflows/{workflow_id}/history", "backend.routes.history_routes.save_workflow_history", "write"), RouteSpec("GET", "nodes/{node_id}", "backend.routes.node_routes.get_node", "read"), RouteSpec("GET", "nodes/{node_id}/dependency-context", "backend.routes.node_routes.get_node_dependency_context", "read"), RouteSpec("PUT", "nodes/{node_id}", "backend.routes.node_routes.update_node", "write"), RouteSpec("POST", "nodes/{node_id}/generate", "backend.routes.node_routes.generate_node", "write"), - RouteSpec("POST", "nodes/{node_id}/retry", "backend.routes.node_routes.retry_node", "write"), + RouteSpec("POST", "nodes/{node_id}/retry", "backend.routes.node_routes.retry_node", "retry"), RouteSpec("POST", "nodes/{node_id}/complete", "backend.routes.node_routes.mark_node_completed", "write"), RouteSpec("POST", "nodes/{node_id}/mark-completed", "backend.routes.node_routes.mark_node_completed", "write"), RouteSpec("POST", "nodes/{node_id}/select-image", "backend.routes.node_routes.select_node_image", "write"), @@ -82,21 +93,24 @@ ROUTES: tuple[RouteSpec, ...] = ( RouteSpec("POST", "review/policy/validate", "backend.routes.review_routes.validate_review_policy", "config"), RouteSpec("POST", "review/policy/save", "backend.routes.review_routes.save_review_policy", "config"), RouteSpec("POST", "prompts/expand", "backend.routes.prompt_routes.expand_prompt", "read"), + RouteSpec("POST", "prompts/governance-preview", "backend.routes.prompt_routes.preview_prompt_governance", "read"), RouteSpec("GET", "nodes/{node_id}/review", "backend.routes.review_routes.review_node", "read"), RouteSpec("GET", "nodes/{node_id}/images", "backend.routes.image_routes.list_node_images", "read"), RouteSpec("GET", "workflows/{workflow_id}/review", "backend.routes.review_routes.review_workflow", "read"), + RouteSpec("GET", "images", "backend.routes.image_routes.list_images", "read"), RouteSpec("GET", "images/{image_id}", "backend.routes.image_routes.get_image", "read"), RouteSpec("POST", "images/{image_id}/select", "backend.routes.image_routes.select_image", "write"), RouteSpec("POST", "images/{image_id}/unselect", "backend.routes.image_routes.unselect_image", "write"), RouteSpec("POST", "images/{image_id}/archive", "backend.routes.image_routes.archive_image", "write"), + RouteSpec("POST", "images/{image_id}/moderation-review", "backend.routes.image_routes.review_moderated_image", "materials"), RouteSpec("POST", "images/{image_id}/safe-delete", "backend.routes.image_routes.safe_delete_image", "write"), RouteSpec("DELETE", "images/{image_id}", "backend.routes.image_routes.delete_image", "write"), RouteSpec("GET", "jobs", "backend.routes.job_routes.list_jobs", "read"), RouteSpec("GET", "jobs/{task_id}", "backend.routes.job_routes.get_job", "read"), RouteSpec("POST", "jobs/{task_id}/cancel", "backend.routes.job_routes.cancel_job", "write"), - RouteSpec("POST", "jobs/{task_id}/retry", "backend.routes.job_routes.retry_job", "write"), - RouteSpec("POST", "jobs/dead-letter/retry", "backend.routes.job_routes.retry_dead_letter_jobs", "write"), - RouteSpec("POST", "jobs/dead-letter/move-failed", "backend.routes.job_routes.move_failed_jobs_to_dead_letter", "write"), + RouteSpec("POST", "jobs/{task_id}/retry", "backend.routes.job_routes.retry_job", "retry"), + RouteSpec("POST", "jobs/dead-letter/retry", "backend.routes.job_routes.retry_dead_letter_jobs", "retry"), + RouteSpec("POST", "jobs/dead-letter/move-failed", "backend.routes.job_routes.move_failed_jobs_to_dead_letter", "retry"), RouteSpec("GET", "costs/usage", "backend.routes.cost_routes.get_cost_usage", "read"), RouteSpec("POST", "costs/guard", "backend.routes.cost_routes.check_cost_guard", "read"), RouteSpec("GET", "settings/schema", "backend.routes.settings_routes.get_settings_schema", "read"), @@ -110,6 +124,10 @@ ROUTES: tuple[RouteSpec, ...] = ( RouteSpec("GET", "settings/snapshots", "backend.routes.settings_routes.list_settings_snapshots", "config"), RouteSpec("GET", "settings/snapshots/{snapshot_id}", "backend.routes.settings_routes.get_settings_snapshot", "config"), RouteSpec("POST", "settings/rollback", "backend.routes.settings_routes.rollback_settings", "config"), + RouteSpec("GET", "backups", "backend.routes.data_backup_routes.list_backups", "config"), + RouteSpec("POST", "backups", "backend.routes.data_backup_routes.create_backup", "config"), + RouteSpec("GET", "backups/{backup_id}", "backend.routes.data_backup_routes.inspect_backup", "config"), + RouteSpec("POST", "backups/{backup_id}/restore", "backend.routes.data_backup_routes.restore_backup", "config"), RouteSpec("GET", "storage/status", "backend.routes.storage_routes.get_storage_status", "read"), RouteSpec("POST", "storage/migrate", "backend.routes.storage_routes.migrate_storage", "config"), RouteSpec("GET", "media/sign", "backend.routes.media_routes.sign_media", "read"), @@ -131,13 +149,16 @@ ROUTES: tuple[RouteSpec, ...] = ( "materials", ), RouteSpec("GET", "materials", "backend.routes.material_routes.list_materials", "read"), + RouteSpec("GET", "materials/analytics", "backend.routes.material_routes.material_analytics", "read"), RouteSpec("POST", "materials", "backend.routes.material_routes.create_material", "materials"), RouteSpec("POST", "materials/upload", "backend.routes.material_routes.upload_material", "materials"), + RouteSpec("POST", "materials/renormalize-batch", "backend.routes.material_routes.renormalize_materials_batch", "materials"), RouteSpec("GET", "materials/{material_id}", "backend.routes.material_routes.get_material", "read"), RouteSpec("PUT", "materials/{material_id}", "backend.routes.material_routes.update_material", "materials"), RouteSpec("DELETE", "materials/{material_id}", "backend.routes.material_routes.delete_material", "materials"), RouteSpec("POST", "materials/{material_id}/recognize", "backend.routes.material_routes.recognize_material", "materials"), RouteSpec("POST", "materials/{material_id}/re-recognize", "backend.routes.material_routes.re_recognize_material", "materials"), + RouteSpec("POST", "materials/{material_id}/renormalize", "backend.routes.material_routes.renormalize_material", "materials"), RouteSpec("POST", "materials/{material_id}/confirm", "backend.routes.material_routes.confirm_material", "materials"), RouteSpec("POST", "assets/import-pack", "backend.routes.asset_routes.import_asset_pack", "materials"), RouteSpec("GET", "profiles/{profile_id}/wardrobe", "backend.routes.wardrobe_routes.list_wardrobe_items", "read"), @@ -156,6 +177,10 @@ ROUTES: tuple[RouteSpec, ...] = ( "materials", ), RouteSpec("POST", "profiles/{profile_id}/wardrobe/match", "backend.routes.wardrobe_routes.match_wardrobe", "read"), + RouteSpec("GET", "profiles/{profile_id}/wardrobe-templates", "backend.routes.wardrobe_routes.list_wardrobe_templates", "read"), + RouteSpec("POST", "profiles/{profile_id}/wardrobe-templates", "backend.routes.wardrobe_routes.save_wardrobe_template", "materials"), + RouteSpec("POST", "profiles/{profile_id}/wardrobe-templates/{template_id}/apply", "backend.routes.wardrobe_routes.apply_wardrobe_template", "read"), + RouteSpec("DELETE", "profiles/{profile_id}/wardrobe-templates/{template_id}", "backend.routes.wardrobe_routes.delete_wardrobe_template", "materials"), RouteSpec("POST", "integrations/private-companion/daily-outfit", "backend.routes.integration_routes.private_companion_daily_outfit", "read"), RouteSpec("GET", "integrations/private-companion/token", "backend.routes.integration_routes.get_private_companion_token_status", "config"), RouteSpec("POST", "integrations/private-companion/token/rotate", "backend.routes.integration_routes.rotate_private_companion_token", "config"), diff --git a/backend/routes/asset_routes.py b/backend/routes/asset_routes.py index 28976cb..3aa3dc9 100644 --- a/backend/routes/asset_routes.py +++ b/backend/routes/asset_routes.py @@ -6,6 +6,7 @@ from typing import Any from backend.dto.common import ApiError, ErrorCode from backend.routes.common import handle_request +from backend.routes.params import as_bool from backend.services.asset_import_service import AssetImportService from backend.services.material_service import MaterialService from core.audit_service import AuditService @@ -29,7 +30,7 @@ def import_asset_pack( raise ApiError(ErrorCode.VALIDATION_FAILED, "package_uri is required", {"field": "package_uri"}) return asset_service.plan_import_pack( package_uri, - dry_run=_as_bool(payload.get("dry_run"), default=True), + dry_run=as_bool(payload.get("dry_run"), default=True), ctx=ctx, ) @@ -44,11 +45,3 @@ def _create_asset_import_service() -> AssetImportService: audit_service=AuditService(database), ) ) - - -def _as_bool(value: Any, *, default: bool = False) -> bool: - if value is None: - return default - if isinstance(value, bool): - return value - return str(value).strip().lower() in {"1", "true", "yes", "on"} diff --git a/backend/routes/cost_routes.py b/backend/routes/cost_routes.py index 542e5d3..edec24f 100644 --- a/backend/routes/cost_routes.py +++ b/backend/routes/cost_routes.py @@ -6,6 +6,7 @@ import json from typing import Any from backend.routes.common import handle_request +from backend.routes.params import int_or_default, optional_float, optional_int, optional_str from backend.services.cost_guard_service import CostGuardService from backend.services.cost_service import CostService from core.identity import RequestContext @@ -24,11 +25,11 @@ def get_cost_usage( def action() -> dict[str, Any]: require_authenticated(ctx) return cost_service.usage_summary( - profile_id=_optional_str(params.get("profile_id")), - workflow_id=_optional_str(params.get("workflow_id")), - node_id=_optional_str(params.get("node_id")), - provider_id=_optional_str(params.get("provider_id")), - days=_optional_int(params.get("days")), + profile_id=optional_str(params.get("profile_id")), + workflow_id=optional_str(params.get("workflow_id")), + node_id=optional_str(params.get("node_id")), + provider_id=optional_str(params.get("provider_id")), + days=optional_int(params.get("days")), unit_prices=_unit_prices(params.get("unit_prices")), ) @@ -46,16 +47,16 @@ def check_cost_guard( def action() -> dict[str, Any]: require_authenticated(ctx) return guard_service.check_generation_budget( - provider_id=_optional_str(params.get("provider_id")), - provider_model=_optional_str(params.get("provider_model")), - model_route=_optional_str(params.get("model_route")), - planned_task_count=_int_or_default(params.get("planned_task_count"), 1), - planned_image_count=_int_or_default(params.get("planned_image_count"), 1), - planned_total_tokens=_int_or_default(params.get("planned_total_tokens"), 0), - max_task_count=_optional_int(params.get("max_task_count")), - max_image_count=_optional_int(params.get("max_image_count")), - max_estimated_cost=_optional_float(params.get("max_estimated_cost")), - days=_optional_int(params.get("days")), + provider_id=optional_str(params.get("provider_id")), + provider_model=optional_str(params.get("provider_model")), + model_route=optional_str(params.get("model_route")), + planned_task_count=int_or_default(params.get("planned_task_count"), 1), + planned_image_count=int_or_default(params.get("planned_image_count"), 1), + planned_total_tokens=int_or_default(params.get("planned_total_tokens"), 0), + max_task_count=optional_int(params.get("max_task_count")), + max_image_count=optional_int(params.get("max_image_count")), + max_estimated_cost=optional_float(params.get("max_estimated_cost")), + days=optional_int(params.get("days")), unit_prices=_unit_prices(params.get("unit_prices")), ) @@ -77,35 +78,3 @@ def _unit_prices(value: Any) -> dict[str, Any]: return {} return parsed if isinstance(parsed, dict) else {} return {} - - -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None - - -def _optional_int(value: Any) -> int | None: - if value is None or value == "": - return None - try: - return int(value) - except (TypeError, ValueError): - return None - - -def _optional_float(value: Any) -> float | None: - if value is None or value == "": - return None - try: - return float(value) - except (TypeError, ValueError): - return None - - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default diff --git a/backend/routes/data_backup_routes.py b/backend/routes/data_backup_routes.py new file mode 100644 index 0000000..901fb24 --- /dev/null +++ b/backend/routes/data_backup_routes.py @@ -0,0 +1,27 @@ +"""Full data backup route handlers.""" +from __future__ import annotations +from backend.routes.common import handle_request +from backend.routes.params import as_bool +from backend.services.data_backup_service import DataBackupService +from core.identity import RequestContext +from core.security import require_config_manager + +def list_backups(ctx:RequestContext,service:DataBackupService|None=None)->dict: + instance=service or DataBackupService() + return handle_request(ctx,lambda:(require_config_manager(ctx),instance.list_backups())[1]) + +def create_backup(ctx:RequestContext,payload:dict|None=None,service:DataBackupService|None=None)->dict: + instance=service or DataBackupService();params=payload or {} + return handle_request(ctx,lambda:(require_config_manager(ctx),instance.create_backup( + include_media=as_bool(params.get("include_media"),default=True), + include_prompt_records=as_bool(params.get("include_prompt_records"),default=True), + include_moderation_records=as_bool(params.get("include_moderation_records"),default=True), + ))[1]) + +def inspect_backup(ctx:RequestContext,backup_id:str,service:DataBackupService|None=None)->dict: + instance=service or DataBackupService() + return handle_request(ctx,lambda:(require_config_manager(ctx),instance.inspect_backup(backup_id))[1]) + +def restore_backup(ctx:RequestContext,backup_id:str,payload:dict|None=None,service:DataBackupService|None=None)->dict: + instance=service or DataBackupService();params=payload or {} + return handle_request(ctx,lambda:(require_config_manager(ctx),instance.restore_backup(backup_id,confirm=as_bool(params.get("confirm"))))[1]) diff --git a/backend/routes/event_routes.py b/backend/routes/event_routes.py index e9ad1d6..f66caeb 100644 --- a/backend/routes/event_routes.py +++ b/backend/routes/event_routes.py @@ -5,6 +5,7 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import int_or_default, optional_str from backend.services.event_stream_service import EventStreamService from core.identity import RequestContext from core.security import require_authenticated @@ -21,22 +22,8 @@ def get_event_stream( def action() -> dict[str, Any]: require_authenticated(ctx) return event_service.recent_events( - event_type=_optional_str(params.get("event_type")), - limit=_int_or_default(params.get("limit"), 50), + event_type=optional_str(params.get("event_type")), + limit=int_or_default(params.get("limit"), 50), ) return handle_request(ctx, action) - - -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None - - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default diff --git a/backend/routes/history_routes.py b/backend/routes/history_routes.py index d423262..7929159 100644 --- a/backend/routes/history_routes.py +++ b/backend/routes/history_routes.py @@ -5,6 +5,7 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import optional_str from backend.services.history_service import HistoryService from backend.services.image_service import ImageService from backend.services.workflow_service import WorkflowService @@ -43,12 +44,27 @@ def list_workflow_history( def action() -> list[dict[str, Any]]: require_authenticated(ctx) - node_id = _optional_str(params.get("node_id")) + node_id = optional_str(params.get("node_id")) return [entry.to_dict() for entry in history_service.list_workflow_history(workflow_id, node_id=node_id)] return handle_request(ctx, action) +def save_workflow_history( + ctx: RequestContext, + workflow_id: str, + payload: dict[str, Any], + service: HistoryService | None = None, +) -> dict: + history_service = service or _create_history_service() + + def action() -> dict[str, Any]: + require_write(ctx) + return history_service.save_workflow_snapshot(workflow_id, payload, ctx).to_dict() + + return handle_request(ctx, action) + + def _create_history_service() -> HistoryService: database = Database(load_database_settings()) audit_service = AuditService(database) @@ -69,10 +85,3 @@ def _create_history_service() -> HistoryService: image_service=image_service, audit_service=audit_service, ) - - -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None diff --git a/backend/routes/image_routes.py b/backend/routes/image_routes.py index 9071dfe..1af528a 100644 --- a/backend/routes/image_routes.py +++ b/backend/routes/image_routes.py @@ -4,9 +4,10 @@ from __future__ import annotations from typing import Any -from backend.dto.common import ApiError -from backend.dto.image_dto import GenerationImage +from backend.dto.common import ApiError, ErrorCode +from backend.dto.image_dto import GenerationImage, ImageListResult from backend.routes.common import handle_request +from backend.routes.params import as_bool, int_or_default, optional_str from backend.services.deletion_safety_service import DeletionSafetyService from backend.services.image_service import ImageService from backend.services.media_service import MediaService @@ -18,6 +19,39 @@ from database.repositories.image_repository import ImageRepository from database.repositories.workflow_repository import WorkflowRepository +def list_images( + ctx: RequestContext, + query: dict[str, Any] | None = None, + service: ImageService | None = None, + media_service: MediaService | None = None, +) -> dict: + image_service = service or _create_image_service() + signer = media_service or MediaService() + params = query or {} + + def action() -> dict[str, Any]: + require_authenticated(ctx) + result = image_service.list_images( + profile_id=optional_str(params.get("profile_id")), + workflow_id=optional_str(params.get("workflow_id")), + include_archived=as_bool(params.get("include_archived"), default=True, non_string_truthiness=True), + page=int_or_default(params.get("page"), 1), + page_size=int_or_default(params.get("page_size"), 60), + ) + include_restricted = as_bool(params.get("include_restricted"), default=False, non_string_truthiness=True) + if include_restricted and not (ctx.has_permission("can_manage_materials") or ctx.has_permission("*")): + include_restricted = False + payload = _serialize_image_list(result, signer) + if not include_restricted: + visible = [item for item in payload["items"] if not _is_restricted_image(item)] + payload["restricted_hidden"] = len(payload["items"]) - len(visible) + payload["items"] = visible + payload["total"] = max(len(visible), int(payload.get("total") or 0) - payload["restricted_hidden"]) + return payload + + return handle_request(ctx, action) + + def list_node_images( ctx: RequestContext, node_id: str, @@ -31,8 +65,12 @@ def list_node_images( def action() -> list[dict[str, Any]]: require_authenticated(ctx) - include_archived = _as_bool(params.get("include_archived")) - return [_serialize_image(image, signer) for image in image_service.list_by_node(node_id, include_archived)] + include_archived = as_bool(params.get("include_archived"), non_string_truthiness=True) + include_restricted = as_bool(params.get("include_restricted"), default=False, non_string_truthiness=True) + if include_restricted and not (ctx.has_permission("can_manage_materials") or ctx.has_permission("*")): + include_restricted = False + images = [_serialize_image(image, signer) for image in image_service.list_by_node(node_id, include_archived)] + return images if include_restricted else [item for item in images if not _is_restricted_image(item)] return handle_request(ctx, action) @@ -48,7 +86,10 @@ def get_image( def action() -> dict[str, Any]: require_authenticated(ctx) - return _serialize_image(image_service.get_image(image_id), signer) + image = _serialize_image(image_service.get_image(image_id), signer) + if _is_restricted_image(image) and not (ctx.has_permission("can_manage_materials") or ctx.has_permission("*")): + raise ApiError(ErrorCode.FORBIDDEN, "受限图片仅管理员可查看", {"image_id": image_id}) + return image return handle_request(ctx, action) @@ -101,6 +142,24 @@ def archive_image( return handle_request(ctx, action) +def review_moderated_image( + ctx: RequestContext, + image_id: str, + payload: dict[str, Any], + service: ImageService | None = None, + media_service: MediaService | None = None, +) -> dict: + image_service = service or _create_image_service() + signer = media_service or MediaService() + + def action() -> dict[str, Any]: + if not (ctx.has_permission("can_manage_materials") or ctx.has_permission("*")): + raise ApiError(ErrorCode.FORBIDDEN, "仅管理员可复核受限图片", {"image_id": image_id}) + return _serialize_image(image_service.review_moderated_image(image_id, payload, ctx), signer) + + return handle_request(ctx, action) + + def delete_image( ctx: RequestContext, image_id: str, @@ -135,8 +194,8 @@ def safe_delete_image( return deletion_service.delete_image( image_id, ctx, - dry_run=_as_bool(params.get("dry_run", True)), - delete_files=_as_bool(params.get("delete_files")), + dry_run=as_bool(params.get("dry_run", True), non_string_truthiness=True), + delete_files=as_bool(params.get("delete_files"), non_string_truthiness=True), ) return handle_request(ctx, action) @@ -151,14 +210,6 @@ def _create_image_service() -> ImageService: ) -def _as_bool(value: Any) -> bool: - if isinstance(value, bool): - return value - if isinstance(value, str): - return value.strip().lower() in {"1", "true", "yes", "on"} - return bool(value) - - def _serialize_image(image: GenerationImage, media_service: MediaService) -> dict[str, Any]: payload = image.to_dict() payload["image_url"] = _signed_url(media_service, image.image_uri) @@ -166,6 +217,12 @@ def _serialize_image(image: GenerationImage, media_service: MediaService) -> dic return payload +def _serialize_image_list(result: ImageListResult, media_service: MediaService) -> dict[str, Any]: + payload = result.to_dict() + payload["items"] = [_serialize_image(image, media_service) for image in result.items] + return payload + + def _signed_url(media_service: MediaService, uri: str | None) -> str | None: if not uri: return None @@ -173,3 +230,8 @@ def _signed_url(media_service: MediaService, uri: str | None) -> str | None: return media_service.sign_media(uri)["url"] except ApiError: return None + + +def _is_restricted_image(image: dict[str, Any]) -> bool: + metadata = image.get("metadata") if isinstance(image.get("metadata"), dict) else {} + return bool(metadata.get("quarantined") or metadata.get("gallery_visibility") == "restricted") diff --git a/backend/routes/integration_routes.py b/backend/routes/integration_routes.py index 03338b7..4cb57a4 100644 --- a/backend/routes/integration_routes.py +++ b/backend/routes/integration_routes.py @@ -81,6 +81,7 @@ def rotate_private_companion_token( def _create_integration_service() -> IntegrationService: database = Database(load_database_settings()) audit_service = AuditService(database) + config_service = ConfigService() wardrobe_service = WardrobeService( wardrobe_repository=WardrobeRepository(database), profile_repository=ProfileRepository(database), @@ -90,7 +91,8 @@ def _create_integration_service() -> IntegrationService: return IntegrationService( wardrobe_service=wardrobe_service, integration_repository=IntegrationRepository(database), - token_service=IntegrationTokenService(audit_service=audit_service), + token_service=IntegrationTokenService(config_service, audit_service=audit_service), + enabled=config_service.load().feature_flags().get("private_companion_integration", False), ) diff --git a/backend/routes/job_routes.py b/backend/routes/job_routes.py index 041ab76..3dc7c38 100644 --- a/backend/routes/job_routes.py +++ b/backend/routes/job_routes.py @@ -5,11 +5,13 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import int_or_default, optional_str from backend.services.event_stream_service import get_default_event_bus +from backend.services.generation_job_retry_service import GenerationJobRetryService from backend.services.queue_service import QueueService from core.audit_service import AuditService from core.identity import RequestContext -from core.security import require_authenticated, require_write +from core.security import require_authenticated, require_task_retry, require_write from database.db_helper import Database, load_database_settings from database.repositories.job_repository import JobRepository @@ -25,11 +27,11 @@ def list_jobs( def action() -> dict[str, Any]: require_authenticated(ctx) return queue_service.list_jobs( - status=_optional_str(params.get("status")), - workflow_id=_optional_str(params.get("workflow_id")), - profile_id=_optional_str(params.get("profile_id")), - page=_int_or_default(params.get("page"), 1), - page_size=_int_or_default(params.get("page_size"), 30), + status=optional_str(params.get("status")), + workflow_id=optional_str(params.get("workflow_id")), + profile_id=optional_str(params.get("profile_id")), + page=int_or_default(params.get("page"), 1), + page_size=int_or_default(params.get("page_size"), 30), ).to_dict() return handle_request(ctx, action) @@ -66,13 +68,14 @@ def cancel_job( def retry_job( ctx: RequestContext, task_id: str, - service: QueueService | None = None, + payload: dict[str, Any] | None = None, + service: GenerationJobRetryService | None = None, ) -> dict: - queue_service = service or _create_queue_service() + retry_service = service or _create_generation_job_retry_service() def action() -> dict[str, Any]: - require_write(ctx) - return queue_service.retry_job(task_id, ctx).to_dict() + require_task_retry(ctx) + return retry_service.retry_job(task_id, ctx, overrides=payload or {}).to_dict() return handle_request(ctx, action) @@ -80,14 +83,14 @@ def retry_job( def retry_dead_letter_jobs( ctx: RequestContext, payload: dict[str, Any] | None = None, - service: QueueService | None = None, + service: GenerationJobRetryService | None = None, ) -> dict: - queue_service = service or _create_queue_service() + retry_service = service or _create_generation_job_retry_service() params = payload or {} def action() -> dict[str, Any]: - require_write(ctx) - return queue_service.retry_dead_letters(ctx, limit=_int_or_default(params.get("limit"), 50)) + require_task_retry(ctx) + return retry_service.retry_dead_letters(ctx, limit=int_or_default(params.get("limit"), 5)) return handle_request(ctx, action) @@ -101,8 +104,8 @@ def move_failed_jobs_to_dead_letter( params = payload or {} def action() -> dict[str, Any]: - require_write(ctx) - return queue_service.move_failed_to_dead_letter(ctx, limit=_int_or_default(params.get("limit"), 50)) + require_task_retry(ctx) + return queue_service.move_failed_to_dead_letter(ctx, limit=int_or_default(params.get("limit"), 50)) return handle_request(ctx, action) @@ -116,15 +119,12 @@ def _create_queue_service() -> QueueService: ) -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None +def _create_generation_job_retry_service() -> GenerationJobRetryService: + # Import lazily so route discovery does not construct provider runtimes. + from backend.routes.node_routes import _create_generation_service - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default + generation_service = _create_generation_service() + return GenerationJobRetryService( + queue_service=generation_service.queue_service, + generation_service=generation_service, + ) diff --git a/backend/routes/log_routes.py b/backend/routes/log_routes.py index 77ae3a8..71f82e7 100644 --- a/backend/routes/log_routes.py +++ b/backend/routes/log_routes.py @@ -5,6 +5,7 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import int_or_default, optional_str from backend.services.log_service import LogService from core.identity import RequestContext from core.security import require_authenticated, require_config_manager @@ -17,10 +18,10 @@ def get_generation_logs(ctx: RequestContext, query: dict[str, Any] | None = None def action(log_service: LogService, limit: int) -> list[dict[str, Any]]: return log_service.generation_logs( limit, - status=_optional_str(params.get("status")), - workflow_id=_optional_str(params.get("workflow_id")), - profile_id=_optional_str(params.get("profile_id")), - node_id=_optional_str(params.get("node_id")), + status=optional_str(params.get("status")), + workflow_id=optional_str(params.get("workflow_id")), + profile_id=optional_str(params.get("profile_id")), + node_id=optional_str(params.get("node_id")), ) return _read_logs(ctx, query, service, action) @@ -31,11 +32,11 @@ def get_workflow_logs(ctx: RequestContext, query: dict[str, Any] | None = None, def action(log_service: LogService, limit: int) -> list[dict[str, Any]]: return log_service.workflow_logs( - _optional_str(params.get("workflow_id")), + optional_str(params.get("workflow_id")), limit, - action=_optional_str(params.get("action")), - node_id=_optional_str(params.get("node_id")), - branch_name=_optional_str(params.get("branch_name")), + action=optional_str(params.get("action")), + node_id=optional_str(params.get("node_id")), + branch_name=optional_str(params.get("branch_name")), ) return _read_logs(ctx, query, service, action) @@ -47,8 +48,8 @@ def get_error_logs(ctx: RequestContext, query: dict[str, Any] | None = None, ser def action(log_service: LogService, limit: int) -> list[dict[str, Any]]: return log_service.error_logs( limit, - source=_optional_str(params.get("source")), - status=_optional_str(params.get("status")), + source=optional_str(params.get("source")), + status=optional_str(params.get("status")), ) return _read_logs(ctx, query, service, action) @@ -59,10 +60,10 @@ def get_integration_logs(ctx: RequestContext, query: dict[str, Any] | None = Non def action(log_service: LogService, limit: int) -> list[dict[str, Any]]: return log_service.integration_logs( - _optional_str(params.get("integration_name")), + optional_str(params.get("integration_name")), limit, - action=_optional_str(params.get("action")), - status=_optional_str(params.get("status")), + action=optional_str(params.get("action")), + status=optional_str(params.get("status")), ) return _read_logs(ctx, query, service, action) @@ -75,10 +76,10 @@ def get_audit_logs(ctx: RequestContext, query: dict[str, Any] | None = None, ser def action() -> list[dict[str, Any]]: require_config_manager(ctx) return log_service.audit_logs( - _int_or_default(params.get("limit"), 50), - action=_optional_str(params.get("action")), - target_type=_optional_str(params.get("target_type")), - actor_id=_optional_str(params.get("actor_id")), + int_or_default(params.get("limit"), 50), + action=optional_str(params.get("action")), + target_type=optional_str(params.get("target_type")), + actor_id=optional_str(params.get("actor_id")), ) return handle_request(ctx, action) @@ -95,20 +96,6 @@ def _read_logs( def action() -> list[dict[str, Any]]: require_authenticated(ctx) - return reader(log_service, _int_or_default(params.get("limit"), 50)) + return reader(log_service, int_or_default(params.get("limit"), 50)) return handle_request(ctx, action) - - -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None - - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default diff --git a/backend/routes/material_routes.py b/backend/routes/material_routes.py index 760914b..714d209 100644 --- a/backend/routes/material_routes.py +++ b/backend/routes/material_routes.py @@ -4,17 +4,20 @@ from __future__ import annotations from typing import Any -from backend.dto.common import ApiError +from backend.dto.common import ApiError, ErrorCode from backend.dto.material_dto import Material, MaterialListResult from backend.routes.common import handle_request +from backend.routes.params import int_or_default, optional_str from backend.services.material_service import MaterialService +from backend.services.material_analytics_service import MaterialAnalyticsService from backend.services.media_service import MediaService +from backend.services.model_provider_bootstrap import sync_vision_provider_from_config from backend.services.model_provider_runtime import ModelProviderRuntime from backend.services.model_provider_service import ModelProviderService from core.audit_service import AuditService from core.identity import RequestContext from core.security import require_authenticated, require_permission -from database.db_helper import Database, load_database_settings +from database.db_helper import Database, load_config_payload, load_database_settings from database.repositories.material_repository import MaterialRepository from database.repositories.model_provider_repository import ModelProviderRepository @@ -88,16 +91,27 @@ def list_materials( def action() -> dict[str, Any]: require_authenticated(ctx) result = material_service.list_materials( - category_id=_optional_str(params.get("category_id")), - keyword=_optional_str(params.get("keyword")), - page=_int_or_default(params.get("page"), 1), - page_size=_int_or_default(params.get("page_size"), 30), + category_id=optional_str(params.get("category_id")), + keyword=optional_str(params.get("keyword")), + page=int_or_default(params.get("page"), 1), + page_size=int_or_default(params.get("page_size"), 30), + color=optional_str(params.get("color")), + material_type=optional_str(params.get("material_type")), + body_slot=optional_str(params.get("body_slot")), + wear_layer=optional_str(params.get("wear_layer")), + min_confidence=_optional_float(params.get("min_confidence")), + confirmation_state=optional_str(params.get("confirmation_state")), ) return _serialize_material_list(result, signer) return handle_request(ctx, action) +def material_analytics(ctx: RequestContext, service: MaterialAnalyticsService | None = None) -> dict: + analytics = service or MaterialAnalyticsService(Database(load_database_settings())) + return handle_request(ctx, lambda: (require_authenticated(ctx), analytics.summarize())[1]) + + def create_material( ctx: RequestContext, payload: dict[str, Any], @@ -211,6 +225,36 @@ def re_recognize_material( return handle_request(ctx, action) +def renormalize_material( + ctx: RequestContext, + material_id: str, + service: MaterialService | None = None, + media_service: MediaService | None = None, +) -> dict: + material_service = service or _create_material_service() + signer = media_service or MediaService() + + def action() -> dict[str, Any]: + require_permission(ctx, "can_manage_materials") + return _serialize_material(material_service.renormalize_material(material_id, ctx), signer) + + return handle_request(ctx, action) + + +def renormalize_materials_batch( + ctx: RequestContext, + payload: dict[str, Any] | None = None, + service: MaterialService | None = None, +) -> dict: + material_service = service or _create_material_service() + + def action() -> dict[str, Any]: + require_permission(ctx, "can_manage_materials") + return material_service.renormalize_batch(payload or {}, ctx) + + return handle_request(ctx, action) + + def confirm_material( ctx: RequestContext, material_id: str, @@ -232,10 +276,19 @@ def _create_material_service() -> MaterialService: database = Database(load_database_settings()) model_provider_service = ModelProviderService(ModelProviderRepository(database)) model_provider_runtime = ModelProviderRuntime(model_provider_service) + config = load_config_payload() + sync_vision_provider_from_config(model_provider_service, config) + vision = config.get("vision") if isinstance(config.get("vision"), dict) else {} + provider = vision.get("provider") if isinstance(vision.get("provider"), dict) else {} + assignments = config.get("model_assignments") if isinstance(config.get("model_assignments"), dict) else {} + recognition_prompts = config.get("recognition_prompts") if isinstance(config.get("recognition_prompts"), dict) else {} + default_route = str(assignments.get("material_recognition") or provider.get("route_name") or vision.get("default_route") or "material_vlm").strip() return MaterialService( repository=MaterialRepository(database), audit_service=AuditService(database), vision_provider_factory=model_provider_runtime.vision_provider_for_route, + default_recognition_route=default_route, + recognition_prompt_config=recognition_prompts, ) @@ -252,6 +305,15 @@ def _serialize_material(material: Material, media_service: MediaService) -> dict return payload +def _optional_float(value: Any) -> float | None: + if value in (None, ""): + return None + try: + return float(value) + except (TypeError, ValueError): + raise ApiError(ErrorCode.VALIDATION_FAILED, "min_confidence must be a number", {"field": "min_confidence"}) from None + + def _signed_url(media_service: MediaService, uri: str | None) -> str | None: if not uri: return None @@ -259,17 +321,3 @@ def _signed_url(media_service: MediaService, uri: str | None) -> str | None: return media_service.sign_media(uri)["url"] except ApiError: return None - - -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None - - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default diff --git a/backend/routes/media_routes.py b/backend/routes/media_routes.py index c1090d6..23d6f86 100644 --- a/backend/routes/media_routes.py +++ b/backend/routes/media_routes.py @@ -6,6 +6,7 @@ from typing import Any from backend.dto.common import ApiError, ErrorCode from backend.routes.common import handle_request +from backend.routes.params import int_or_default, optional_str from backend.services.media_index_service import MediaIndexService from backend.services.media_service import MediaService from backend.services.orphan_media_service import OrphanMediaService @@ -58,8 +59,8 @@ def index_media( def action() -> dict[str, Any]: require_authenticated(ctx) return index_service.index_files( - root=_optional_str(params.get("root")), - limit=_int_or_default(params.get("limit"), 500), + root=optional_str(params.get("root")), + limit=int_or_default(params.get("limit"), 500), ) return handle_request(ctx, action) @@ -79,22 +80,8 @@ def scan_orphan_media( def action() -> dict[str, Any]: require_authenticated(ctx) return orphan_service.scan( - root=_optional_str(params.get("root")), - limit=_int_or_default(params.get("limit"), 500), + root=optional_str(params.get("root")), + limit=int_or_default(params.get("limit"), 500), ) return handle_request(ctx, action) - - -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None - - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default diff --git a/backend/routes/node_routes.py b/backend/routes/node_routes.py index e58a11f..f69dd50 100644 --- a/backend/routes/node_routes.py +++ b/backend/routes/node_routes.py @@ -8,22 +8,40 @@ from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode from backend.routes.common import handle_request from backend.services.event_stream_service import get_default_event_bus +from backend.services.cost_guard_service import CostGuardService +from backend.services.cost_service import CostService from backend.services.generation_service import GenerationService from backend.services.image_service import ImageService from backend.services.model_provider_runtime import ModelProviderRuntime +from backend.services.model_provider_bootstrap import sync_generation_provider_from_config from backend.services.model_provider_service import ModelProviderService from backend.services.node_dependency_context_service import NodeDependencyContextService +from backend.services.prompt_expansion_service import PromptExpansionService +from backend.services.prompt_governance_service import ( + GenerationRouteService, + ImageModerationService, + PromptModerationService, + PromptSemanticNormalizer, + PromptTranslationService, +) from backend.services.queue_service import QueueService -from backend.services.workflow_service import WorkflowService +from backend.services.workflow_service import ( + LOCKED_NODE_STATUSES, + WorkflowService, + is_repeatable_single_image_node, + locked_node_edit_error, +) from core.audit_service import AuditService from core.identity import RequestContext -from core.security import require_authenticated, require_write -from database.db_helper import Database, load_database_settings +from core.security import require_authenticated, require_task_retry, require_write +from database.db_helper import Database, load_config_payload, load_database_settings from database.repositories.image_repository import ImageRepository from database.repositories.job_repository import JobRepository +from database.repositories.material_repository import MaterialRepository from database.repositories.model_provider_repository import ModelProviderRepository from database.repositories.profile_repository import ProfileRepository from database.repositories.workflow_repository import WorkflowRepository +from engines.single_flow import SingleFlowEngine def list_workflow_nodes( @@ -144,14 +162,57 @@ def generate_node( def retry_node( ctx: RequestContext, node_id: str, + payload: dict[str, Any] | None = None, service: WorkflowService | None = None, + generation_service: GenerationService | None = None, ) -> dict: - workflow_service = service or _create_workflow_service() + resolved_generation_service = generation_service + if resolved_generation_service is None and service is None: + resolved_generation_service = _create_generation_service() + workflow_service = ( + service + or getattr(resolved_generation_service, "workflow_service", None) + or _create_workflow_service() + ) + request_overrides = payload or {} def action() -> dict[str, Any]: - require_write(ctx) - node = workflow_service.update_node(node_id, {"status": "queued"}, ctx) - return {"node_id": node.node_id, "node_status": node.status, "queue_placeholder": True} + require_task_retry(ctx) + node = workflow_service.get_node(node_id) + if node.status in LOCKED_NODE_STATUSES and not is_repeatable_single_image_node(node): + raise locked_node_edit_error(node) + if node.status not in _RETRYABLE_NODE_STATUSES and not ( + node.status in LOCKED_NODE_STATUSES and is_repeatable_single_image_node(node) + ): + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "node cannot be retried from its current status", + { + "node_id": node.node_id, + "status": node.status, + "allowed_statuses": sorted(_RETRYABLE_NODE_STATUSES), + }, + ) + metadata = node.metadata if isinstance(node.metadata, dict) else {} + last_request = metadata.get("last_generate_request") + if not isinstance(last_request, dict) or not last_request: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "node has no previous generation request to retry", + { + "node_id": node.node_id, + "status": node.status, + "field": "metadata.last_generate_request", + }, + ) + if resolved_generation_service is None: + raise ApiError( + ErrorCode.UNKNOWN_ERROR, + "generation service is unavailable for node retry", + {"node_id": node.node_id}, + ) + retry_request = _build_retry_generation_request(node.node_id, last_request, request_overrides) + return resolved_generation_service.generate_node(node.node_id, retry_request, ctx) return handle_request(ctx, action) @@ -237,6 +298,34 @@ def _create_generation_service() -> GenerationService: event_bus = get_default_event_bus() model_provider_service = ModelProviderService(ModelProviderRepository(database)) model_provider_runtime = ModelProviderRuntime(model_provider_service) + config_payload = load_config_payload() + sync_generation_provider_from_config(model_provider_service, config_payload) + generation_config = config_payload.get("generation") if isinstance(config_payload.get("generation"), dict) else {} + model_assignments = config_payload.get("model_assignments") if isinstance(config_payload.get("model_assignments"), dict) else {} + governance_config = config_payload.get("prompt_governance") if isinstance(config_payload.get("prompt_governance"), dict) else {} + translation_config = governance_config.get("translation") if isinstance(governance_config.get("translation"), dict) else {} + moderation_config = governance_config.get("moderation") if isinstance(governance_config.get("moderation"), dict) else {} + routing_config = governance_config.get("routing") if isinstance(governance_config.get("routing"), dict) else {} + image_moderation_config = governance_config.get("image_moderation") if isinstance(governance_config.get("image_moderation"), dict) else {} + translation_route = str(translation_config.get("model_route") or model_assignments.get("prompt_planning") or "").strip() + moderation_route = str(moderation_config.get("model_route") or model_assignments.get("cross_review_primary") or "").strip() + translation_callable = model_provider_runtime.structured_text_callable(translation_route) if translation_route else None + moderation_callable = model_provider_runtime.structured_text_callable(moderation_route) if moderation_route else None + single_flow_engine = SingleFlowEngine( + image_provider_factory=model_provider_runtime.image_provider_for_route, + allow_mock_fallback=False, + ) + image_moderation_route = str(image_moderation_config.get("model_route") or "").strip() + image_moderation_callable = model_provider_runtime.structured_vision_callable( + image_moderation_route, + storage=single_flow_engine.storage, + ) if image_moderation_route else None + if translation_callable: + raw_translation_callable = translation_callable + translation_callable = lambda request: raw_translation_callable({**request, "purpose": "Chinese-to-English image prompt translation and structured visual decomposition"}) + if moderation_callable: + raw_moderation_callable = moderation_callable + moderation_callable = lambda request: raw_moderation_callable({**request, "purpose": "image prompt content risk classification"}) return GenerationService( workflow_service=WorkflowService( workflow_repository=workflow_repository, @@ -254,9 +343,20 @@ def _create_generation_service() -> GenerationService: workflow_repository=workflow_repository, audit_service=audit_service, ), - single_flow_engine=SingleFlowEngine(image_provider_factory=model_provider_runtime.image_provider_for_route), + single_flow_engine=single_flow_engine, model_provider_service=model_provider_service, + cost_guard_service=CostGuardService(CostService(database)), + prompt_expansion_service=PromptExpansionService(), event_bus=event_bus, + default_model_route=str(generation_config.get("default_model") or "").strip() or None, + require_resolved_model_route=True, + material_resolver=MaterialRepository(database), + model_assignments=model_assignments, + prompt_translation_service=PromptTranslationService(translation_config, translation_callable), + prompt_semantic_normalizer=PromptSemanticNormalizer(), + prompt_moderation_service=PromptModerationService(moderation_config, moderation_callable), + generation_route_service=GenerationRouteService(routing_config), + image_moderation_service=ImageModerationService(image_moderation_config, image_moderation_callable), ) @@ -264,3 +364,91 @@ def _bool(value: Any) -> bool: if isinstance(value, str): return value.strip().lower() in {"1", "true", "yes", "on"} return bool(value) + + +_RETRYABLE_NODE_STATUSES = frozenset({"failed", "cancelled", "skipped"}) +_RETRY_DERIVED_REQUEST_FIELDS = frozenset( + { + "cost_guard", + "expanded_negative_prompt", + "expanded_prompt", + "prompt_expansion", + "reference_persistence", + "resolved_model_route", + } +) + + +def _build_retry_generation_request( + node_id: str, + last_request: dict[str, Any], + overrides: dict[str, Any], +) -> dict[str, Any]: + omitted_references = _ephemeral_reference_count(last_request) + if omitted_references and not _has_reference_override(overrides): + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "the previous generation used inline reference images; provide replacement references before retrying", + { + "node_id": node_id, + "ephemeral_data_urls_omitted": omitted_references, + "required_fields": ["input_images", "image", "selected_draft_image_ids", "stage_inputs"], + }, + ) + request = { + key: value + for key, value in last_request.items() + if key not in _RETRY_DERIVED_REQUEST_FIELDS and key != "idempotency_key" + } + for key, value in overrides.items(): + if key not in _RETRY_DERIVED_REQUEST_FIELDS and key != "idempotency_key": + request[key] = value + + previous_metadata = last_request.get("metadata") + override_metadata = overrides.get("metadata") + if isinstance(previous_metadata, dict) or isinstance(override_metadata, dict): + request["metadata"] = { + **(previous_metadata if isinstance(previous_metadata, dict) else {}), + **(override_metadata if isinstance(override_metadata, dict) else {}), + } + request["idempotency_key"] = f"node-retry-{node_id}-{uuid4().hex}" + return request + + +def _ephemeral_reference_count(request: dict[str, Any]) -> int: + persistence = request.get("reference_persistence") + if not isinstance(persistence, dict): + return 0 + try: + return max(int(persistence.get("ephemeral_data_urls_omitted") or 0), 0) + except (TypeError, ValueError): + return 0 + + +def _has_reference_override(payload: dict[str, Any]) -> bool: + for key in ("input_images", "image", "selected_draft_image_ids", "stage_inputs"): + if _has_nonempty_value(payload.get(key)): + return True + metadata = payload.get("metadata") if isinstance(payload.get("metadata"), dict) else {} + dependency = payload.get("dependency_context") if isinstance(payload.get("dependency_context"), dict) else {} + return any( + _has_nonempty_value(value) + for value in ( + metadata.get("selected_draft_image_ids"), + metadata.get("stage_inputs"), + dependency.get("selected_image_ids"), + dependency.get("selected_draft_image_ids"), + dependency.get("stage_inputs"), + dependency.get("material_bindings"), + ) + ) + + +def _has_nonempty_value(value: Any) -> bool: + if value is None: + return False + if isinstance(value, str): + return bool(value.strip()) + if isinstance(value, (list, tuple, dict, set)): + return bool(value) + return True diff --git a/backend/routes/params.py b/backend/routes/params.py new file mode 100644 index 0000000..a375c6f --- /dev/null +++ b/backend/routes/params.py @@ -0,0 +1,82 @@ +"""Side-effect-free coercion helpers shared by route handlers.""" + +from __future__ import annotations + +from typing import Any + + +_TRUE_TEXT_VALUES = frozenset({"1", "true", "yes", "on"}) + + +def optional_str(value: Any) -> str | None: + """Return a stripped string, or ``None`` for missing/blank values.""" + + if value is None: + return None + text = str(value).strip() + return text or None + + +# Both names are used for this coercion across the backend. Keep the alias so +# route and service code can use the vocabulary that best fits the field. +optional_text = optional_str + + +def optional_int(value: Any) -> int | None: + """Return an integer when coercion succeeds, otherwise ``None``.""" + + if value is None or value == "": + return None + try: + return int(value) + except (TypeError, ValueError): + return None + + +def optional_float(value: Any) -> float | None: + """Return a float when coercion succeeds, otherwise ``None``.""" + + if value is None or value == "": + return None + try: + return float(value) + except (TypeError, ValueError): + return None + + +def int_or_default(value: Any, default: int) -> int: + """Return an integer when coercion succeeds, otherwise ``default``. + + Range normalization intentionally remains the responsibility of the + receiving service; route parsing only handles type coercion. + """ + + try: + return int(value) + except (TypeError, ValueError): + return default + + +def as_bool( + value: Any, + *, + default: bool = False, + non_string_truthiness: bool = False, +) -> bool: + """Coerce a route parameter to bool while preserving legacy semantics. + + Strings accept the route-level true tokens ``1``, ``true``, ``yes`` and + ``on``. Some existing routes historically applied Python truthiness to + non-string values; they opt into that behavior with + ``non_string_truthiness=True``. + """ + + if value is None: + return default + if isinstance(value, bool): + return value + if isinstance(value, str): + return value.strip().lower() in _TRUE_TEXT_VALUES + if non_string_truthiness: + return bool(value) + return str(value).strip().lower() in _TRUE_TEXT_VALUES diff --git a/backend/routes/profile_routes.py b/backend/routes/profile_routes.py index f2a5206..aae6669 100644 --- a/backend/routes/profile_routes.py +++ b/backend/routes/profile_routes.py @@ -5,13 +5,17 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import int_or_default, optional_str +from backend.services.event_stream_service import get_default_event_bus from backend.services.profile_service import ProfileService from backend.services.quick_profile_service import QuickProfileService +from backend.services.workflow_service import WorkflowService from core.audit_service import AuditService from core.identity import RequestContext from core.security import require_authenticated, require_write from database.db_helper import Database, load_database_settings from database.repositories.profile_repository import ProfileRepository +from database.repositories.workflow_repository import WorkflowRepository def list_profiles( @@ -26,10 +30,10 @@ def list_profiles( require_authenticated(ctx) result = profile_service.list_profiles( ctx=ctx, - keyword=_optional_str(params.get("keyword")), - status=_optional_str(params.get("status")) or "active", - page=_int_or_default(params.get("page"), 1), - page_size=_int_or_default(params.get("page_size"), 30), + keyword=optional_str(params.get("keyword")), + status=optional_str(params.get("status")) or "active", + page=int_or_default(params.get("page"), 1), + page_size=int_or_default(params.get("page_size"), 30), ) return result.to_dict() @@ -59,7 +63,7 @@ def get_profile( def action() -> dict: require_authenticated(ctx) - return profile_service.get_profile(profile_id).to_dict() + return profile_service.get_profile(profile_id, ctx).to_dict() return handle_request(ctx, action) @@ -120,34 +124,34 @@ def save_quick_profile_draft( payload: dict[str, Any], service: QuickProfileService | None = None, profile_service: ProfileService | None = None, + workflow_service: WorkflowService | None = None, ) -> dict: quick_service = service or QuickProfileService() resolved_profile_service = profile_service or _create_profile_service() + resolved_workflow_service = workflow_service or _create_workflow_service( + resolved_profile_service.repository.database + ) def action() -> dict: require_write(ctx) - return quick_service.save_draft(payload, resolved_profile_service, ctx) + return quick_service.save_draft(payload, resolved_profile_service, ctx, resolved_workflow_service) return handle_request(ctx, action) -def _create_profile_service() -> ProfileService: - database = Database(load_database_settings()) +def _create_profile_service(database: Database | None = None) -> ProfileService: + database = database or Database(load_database_settings()) return ProfileService( repository=ProfileRepository(database), audit_service=AuditService(database), ) -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None - - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default +def _create_workflow_service(database: Database | None = None) -> WorkflowService: + database = database or Database(load_database_settings()) + return WorkflowService( + workflow_repository=WorkflowRepository(database), + profile_repository=ProfileRepository(database), + audit_service=AuditService(database), + event_bus=get_default_event_bus(), + ) diff --git a/backend/routes/prompt_routes.py b/backend/routes/prompt_routes.py index 7a6049b..5acaaa0 100644 --- a/backend/routes/prompt_routes.py +++ b/backend/routes/prompt_routes.py @@ -6,8 +6,13 @@ from typing import Any from backend.routes.common import handle_request from backend.services.prompt_expansion_service import PromptExpansionService +from backend.services.model_provider_runtime import ModelProviderRuntime +from backend.services.model_provider_service import ModelProviderService +from backend.services.prompt_governance_service import GenerationRouteService, PromptModerationService, PromptSemanticNormalizer, PromptTranslationService from core.identity import RequestContext from core.security import require_authenticated +from database.db_helper import Database, load_config_payload, load_database_settings +from database.repositories.model_provider_repository import ModelProviderRepository def expand_prompt( @@ -23,3 +28,46 @@ def expand_prompt( return prompt_service.expand_payload(request_payload) return handle_request(ctx, action) + + +def preview_prompt_governance(ctx: RequestContext, payload: dict[str, Any] | None = None) -> dict: + request = payload or {} + + def action() -> dict[str, Any]: + require_authenticated(ctx) + config = load_config_payload() + governance = config.get("prompt_governance") if isinstance(config.get("prompt_governance"), dict) else {} + assignments = config.get("model_assignments") if isinstance(config.get("model_assignments"), dict) else {} + translation_config = governance.get("translation") if isinstance(governance.get("translation"), dict) else {} + moderation_config = governance.get("moderation") if isinstance(governance.get("moderation"), dict) else {} + routing_config = governance.get("routing") if isinstance(governance.get("routing"), dict) else {} + model_service = ModelProviderService(ModelProviderRepository(Database(load_database_settings()))) + runtime = ModelProviderRuntime(model_service) + translation_route = str(translation_config.get("model_route") or assignments.get("prompt_planning") or "").strip() + moderation_route = str(moderation_config.get("model_route") or assignments.get("cross_review_primary") or "").strip() + translator = runtime.structured_text_callable(translation_route) if translation_route else None + reviewer = runtime.structured_text_callable(moderation_route) if moderation_route else None + original = str(request.get("prompt") or "").strip() + translation = PromptTranslationService(translation_config, translator).translate( + original, + entrypoint=str(request.get("entrypoint") or "web"), + user_allowed=bool(request.get("translation_consent", False)), + model_family=str(request.get("model_family") or "natural_language"), + locked_terms=[str(item) for item in request.get("locked_terms", [])] if isinstance(request.get("locked_terms"), list) else [], + budget=int(request.get("prompt_budget")) if str(request.get("prompt_budget") or "").isdigit() else None, + ) + selection = str(request.get("selection") or "translated") + selected = original if selection == "original" else str(request.get("manual_prompt") or "").strip() if selection == "manual" else translation.translated_text + normalized = PromptSemanticNormalizer().normalize(selected, locked_terms=[str(item) for item in request.get("locked_terms", [])] if isinstance(request.get("locked_terms"), list) else []) + moderation = PromptModerationService(moderation_config, reviewer).moderate(original_prompt=original, translated_prompt=translation.translated_text, final_prompt=normalized["normalized_text"], negative_prompt=str(request.get("negative_prompt") or "")) + route = GenerationRouteService(routing_config).decide( + str(request.get("model_route") or routing_config.get("safe_route") or "image_low"), + moderation, + entrypoint=str(request.get("entrypoint") or "web"), + user_id=ctx.actor_id, + group_id=str(request.get("group_id") or "").strip() or None, + permission_context=request.get("content_permissions") if isinstance(request.get("content_permissions"), dict) else {}, + ) + return {"original_prompt": original, "translation": translation.to_dict(), "selection": selection, "selected_prompt": selected, "normalization": normalized, "moderation": moderation.to_dict(), "route_decision": route.to_dict()} + + return handle_request(ctx, action) diff --git a/backend/routes/settings_routes.py b/backend/routes/settings_routes.py index e7e3a3d..1574ed3 100644 --- a/backend/routes/settings_routes.py +++ b/backend/routes/settings_routes.py @@ -5,6 +5,7 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import int_or_default from backend.services.settings_service import SettingsService from core.audit_service import AuditService from core.identity import RequestContext @@ -130,7 +131,7 @@ def list_settings_snapshots( def action() -> list[dict[str, Any]]: require_config_manager(ctx) - return settings_service.list_snapshots(_int_or_default(params.get("limit"), 20)) + return settings_service.list_snapshots(int_or_default(params.get("limit"), 20)) return handle_request(ctx, action) @@ -166,10 +167,3 @@ def rollback_settings( def _create_settings_service() -> SettingsService: database = Database(load_database_settings()) return SettingsService(audit_service=AuditService(database)) - - -def _int_or_default(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default diff --git a/backend/routes/storage_routes.py b/backend/routes/storage_routes.py index 0ff3e6a..bbce4c1 100644 --- a/backend/routes/storage_routes.py +++ b/backend/routes/storage_routes.py @@ -5,6 +5,7 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import as_bool from backend.services.media_service import MediaService from backend.dto.common import ApiError, ErrorCode from core.identity import RequestContext @@ -42,15 +43,7 @@ def migrate_storage( return media_service.plan_storage_migration( source_uri, target_uri, - dry_run=_as_bool(payload.get("dry_run"), default=True), + dry_run=as_bool(payload.get("dry_run"), default=True), ) return handle_request(ctx, action) - - -def _as_bool(value: Any, *, default: bool = False) -> bool: - if value is None: - return default - if isinstance(value, bool): - return value - return str(value).strip().lower() in {"1", "true", "yes", "on"} diff --git a/backend/routes/wardrobe_routes.py b/backend/routes/wardrobe_routes.py index 3c43eb9..e324f02 100644 --- a/backend/routes/wardrobe_routes.py +++ b/backend/routes/wardrobe_routes.py @@ -5,6 +5,7 @@ from __future__ import annotations from typing import Any from backend.routes.common import handle_request +from backend.routes.params import optional_str from backend.services.wardrobe_service import WardrobeService from core.audit_service import AuditService from core.identity import RequestContext @@ -28,7 +29,11 @@ def list_wardrobe_items( require_authenticated(ctx) return [ item.to_dict() - for item in wardrobe_service.list_items(profile_id, _optional_str(params.get("wardrobe_category"))) + for item in wardrobe_service.list_items( + profile_id, + optional_str(params.get("wardrobe_category")), + ctx, + ) ] return handle_request(ctx, action) @@ -79,6 +84,26 @@ def match_wardrobe( return handle_request(ctx, action) +def list_wardrobe_templates(ctx: RequestContext, profile_id: str, service: WardrobeService | None = None) -> dict: + wardrobe_service = service or _create_wardrobe_service() + return handle_request(ctx, lambda: (require_authenticated(ctx), wardrobe_service.list_templates(profile_id, ctx))[1]) + + +def save_wardrobe_template(ctx: RequestContext, profile_id: str, payload: dict[str, Any], service: WardrobeService | None = None) -> dict: + wardrobe_service = service or _create_wardrobe_service() + return handle_request(ctx, lambda: (require_permission(ctx, "can_manage_materials"), wardrobe_service.save_template(profile_id, payload, ctx))[1]) + + +def delete_wardrobe_template(ctx: RequestContext, profile_id: str, template_id: str, service: WardrobeService | None = None) -> dict: + wardrobe_service = service or _create_wardrobe_service() + return handle_request(ctx, lambda: (require_permission(ctx, "can_manage_materials"), wardrobe_service.delete_template(profile_id, template_id, ctx))[1]) + + +def apply_wardrobe_template(ctx: RequestContext, profile_id: str, template_id: str, payload: dict[str, Any] | None = None, service: WardrobeService | None = None) -> dict: + wardrobe_service = service or _create_wardrobe_service() + return handle_request(ctx, lambda: (require_authenticated(ctx), wardrobe_service.apply_template(profile_id, template_id, payload or {}, ctx))[1]) + + def _create_wardrobe_service() -> WardrobeService: database = Database(load_database_settings()) return WardrobeService( @@ -87,10 +112,3 @@ def _create_wardrobe_service() -> WardrobeService: material_repository=MaterialRepository(database), audit_service=AuditService(database), ) - - -def _optional_str(value: Any) -> str | None: - if value is None: - return None - text = str(value).strip() - return text or None diff --git a/backend/routes/workflow_routes.py b/backend/routes/workflow_routes.py index 1c5cebb..8379f33 100644 --- a/backend/routes/workflow_routes.py +++ b/backend/routes/workflow_routes.py @@ -7,11 +7,13 @@ from typing import Any from backend.routes.common import handle_request from backend.services.character_workflow_service import CharacterWorkflowService from backend.services.event_stream_service import get_default_event_bus +from backend.services.queue_service import QueueService from backend.services.workflow_service import WorkflowService from core.audit_service import AuditService from core.identity import RequestContext from core.security import require_authenticated, require_write from database.db_helper import Database, load_database_settings +from database.repositories.job_repository import JobRepository from database.repositories.profile_repository import ProfileRepository from database.repositories.workflow_repository import WorkflowRepository @@ -87,7 +89,18 @@ def cancel_workflow( workflow_id: str, service: WorkflowService | None = None, ) -> dict: - return _set_workflow_status(ctx, workflow_id, "cancelled", service) + workflow_service = service or _create_workflow_service() + queue_service = QueueService( + JobRepository(workflow_service.workflow_repository.database), + workflow_service.audit_service, + workflow_service.event_bus, + ) + + def action() -> dict[str, Any]: + require_write(ctx) + return workflow_service.cancel_workflow(workflow_id, ctx, queue_service).to_dict() + + return handle_request(ctx, action) def archive_workflow( diff --git a/backend/services/api_discovery_service.py b/backend/services/api_discovery_service.py index e4cd0ec..7f2f066 100644 --- a/backend/services/api_discovery_service.py +++ b/backend/services/api_discovery_service.py @@ -2,6 +2,10 @@ from __future__ import annotations +import json +import os +import urllib.error +import urllib.request from typing import Any from uuid import uuid4 @@ -44,18 +48,25 @@ class ApiDiscoveryService: if not model_id: raise ApiError(ErrorCode.VALIDATION_FAILED, "model_id is required", {"field": "model_id"}) dialect = str(payload.get("dialect") or "unknown") + live_result = _live_probe(payload) if payload.get("live_probe") else { + "mode": "safe", + "message": "safe probe recorded metadata only; no external API call was made", + } + capabilities = dict(payload.get("capabilities") or {}) + if live_result.get("models"): + capabilities["discovered_models"] = live_result["models"] + if live_result.get("supported_sizes"): + capabilities["supported_sizes"] = live_result["supported_sizes"] + capabilities["max_tested_size"] = live_result["supported_sizes"][-1] profile = self.repository.upsert_profile( { "capability_profile_id": payload.get("capability_profile_id") or new_capability_profile_id(), "provider_id": provider_id, "model_id": model_id, "dialect": dialect, - "status": "unconfirmed", - "capabilities": payload.get("capabilities") or {}, - "probe_result": { - "mode": "safe", - "message": "safe probe recorded metadata only; no external API call was made", - }, + "status": "confirmed_supported" if live_result.get("connected") else "unconfirmed", + "capabilities": capabilities, + "probe_result": live_result, } ) return profile @@ -163,3 +174,48 @@ class ApiDiscoveryService: def new_capability_profile_id() -> str: return f"cap_{uuid4().hex}" + + +def _live_probe(payload: dict[str, Any]) -> dict[str, Any]: + base_url = str(payload.get("base_url") or "").strip().rstrip("/") + if not base_url.startswith(("http://", "https://")): + raise ApiError(ErrorCode.VALIDATION_FAILED, "valid base_url is required", {"field": "base_url"}) + timeout = min(max(int(payload.get("timeout_seconds") or 20), 3), 60) + env_name = str(payload.get("api_key_env_var") or "").strip() + api_key = os.getenv(env_name, "") if env_name else "" + headers = {"Accept": "application/json", **({"Authorization": f"Bearer {api_key}"} if api_key else {})} + result: dict[str, Any] = {"mode": "live", "connected": False, "models": [], "supported_sizes": [], "warnings": []} + api_prefix = base_url if base_url.lower().endswith("/v1") else f"{base_url}/v1" + models_url = f"{api_prefix}/models" + result["resolved_api_base"] = api_prefix + result["image_endpoint"] = "/images/generations" if base_url.lower().endswith("/v1") else "/v1/images/generations" + try: + request = urllib.request.Request(models_url, headers=headers, method="GET") + with urllib.request.urlopen(request, timeout=timeout) as response: + body = response.read(2 * 1024 * 1024).decode("utf-8-sig") + parsed = json.loads(body) + items = parsed.get("data", []) if isinstance(parsed, dict) else [] + result["models"] = [str(item.get("id")) for item in items if isinstance(item, dict) and item.get("id")][:200] + result["connected"] = True + result["http_status"] = response.status + except urllib.error.HTTPError as exc: + result.update({"http_status": exc.code, "error": f"models endpoint returned HTTP {exc.code}"}) + except (urllib.error.URLError, TimeoutError, json.JSONDecodeError, UnicodeDecodeError) as exc: + result["error"] = str(exc) + + if payload.get("allow_lightweight_test") and result["connected"]: + endpoint = str(payload.get("image_endpoint") or result["image_endpoint"]) + if base_url.lower().endswith("/v1") and endpoint.lower().startswith("/v1/"): + endpoint = endpoint[3:] + endpoint_url = f"{base_url}{endpoint if endpoint.startswith('/') else '/' + endpoint}" + model_id = str(payload.get("model_id") or "").strip() + for size in ("1920x1080", "2560x1440", "3840x2160"): + data = json.dumps({"model": model_id, "prompt": "A plain gray circle on white background, capability test", "size": size, "n": 1, "response_format": "b64_json"}).encode("utf-8") + request = urllib.request.Request(endpoint_url, data=data, headers={**headers, "Content-Type": "application/json"}, method="POST") + try: + with urllib.request.urlopen(request, timeout=timeout) as response: + response.read(1024) + result["supported_sizes"].append(size) + except Exception as exc: # Provider dialects expose inconsistent errors; retain a short diagnostic only. + result["warnings"].append({"size": size, "message": str(exc)[:180]}) + return result diff --git a/backend/services/asset_import_service.py b/backend/services/asset_import_service.py index bfa1b67..0e08644 100644 --- a/backend/services/asset_import_service.py +++ b/backend/services/asset_import_service.py @@ -47,12 +47,27 @@ class AssetImportService: manifest_candidates = _manifest_candidates(root, relative, package_path) manifest_summary = _manifest_summary(package_path, manifest_candidates) applied = {"materials": 0, "styles": 0, "material_ids": [], "created": 0, "updated": 0} + unsupported_features = [] + if manifest_summary["styles"]: + unsupported_features.append( + { + "code": "STYLE_IMPORT_UNSUPPORTED", + "message": "style entries are not imported by the current asset-pack service", + "count": manifest_summary["styles"], + } + ) if not dry_run and not manifest_summary["valid"]: raise ApiError( ErrorCode.VALIDATION_FAILED, "asset pack manifest is invalid", {"field": "manifest", "errors": manifest_summary["errors"]}, ) + if not dry_run and unsupported_features: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "asset pack contains unsupported entries", + {"field": "styles", "unsupported_features": unsupported_features}, + ) if not dry_run: if self.material_service is None or ctx is None: raise ApiError( @@ -69,8 +84,9 @@ class AssetImportService: "bytes": total_bytes, "manifest_candidates": [item["uri"] for item in manifest_candidates], "manifest": manifest_summary, - "import_enabled": manifest_summary["valid"], - "execution_enabled": True, + "import_enabled": manifest_summary["valid"] and not unsupported_features, + "execution_enabled": manifest_summary["valid"] and not unsupported_features, + "unsupported_features": unsupported_features, "applied": applied, } @@ -84,10 +100,18 @@ class AssetImportService: ctx: RequestContext, ) -> dict[str, Any]: payload = _load_manifest_payload(manifest["path"]) + if _list_count(payload.get("styles")): + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "style entries are not supported by asset pack import", + {"field": "styles"}, + ) material_ids: list[str] = [] created_count = 0 updated_count = 0 base_relative = relative if package_path.is_dir() else relative.parent + planned_materials: list[tuple[dict[str, Any], bool]] = [] + planned_material_ids: set[str] = set() for index, item in enumerate(payload.get("materials") or []): if not isinstance(item, dict): raise ApiError( @@ -103,11 +127,25 @@ class AssetImportService: manifest_uri=manifest["uri"], index=index, ) + material_id = str(material_payload["material_id"]) + if material_id in planned_material_ids: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "asset pack contains duplicate material IDs", + {"field": f"materials[{index}].material_id", "material_id": material_id}, + ) + planned_material_ids.add(material_id) self.storage.ensure_file(material_payload["image_uri"]) if material_payload.get("thumbnail_uri"): self.storage.ensure_file(material_payload["thumbnail_uri"]) existing = self.material_service.repository.get_material(material_payload["material_id"]) - if existing is None: + if existing is not None: + self.material_service.get_material(material_payload["material_id"]) + self.material_service.validate_material_payload(material_payload, creating=existing is None) + planned_materials.append((material_payload, existing is None)) + + for material_payload, creating in planned_materials: + if creating: material = self.material_service.create_material(material_payload, ctx) created_count += 1 else: @@ -116,7 +154,7 @@ class AssetImportService: material_ids.append(material.material_id) return { "materials": len(material_ids), - "styles": _list_count(payload.get("styles")), + "styles": 0, "material_ids": material_ids, "created": created_count, "updated": updated_count, diff --git a/backend/services/character_workflow_service.py b/backend/services/character_workflow_service.py index 0f0ce5b..5f28825 100644 --- a/backend/services/character_workflow_service.py +++ b/backend/services/character_workflow_service.py @@ -11,7 +11,9 @@ from backend.services.workflow_service import WorkflowService from core.identity import RequestContext -CHARACTER_WORKFLOW_TYPES = {"character_fast", "character_gradual"} +CHARACTER_WORKFLOW_TYPES = {"character_fast", "character_gradual", "quick_profile"} +GENERATION_NODE_TYPES = {"generation", "single_image"} +EDITABLE_PREPARATION_NODE_TYPES = {"planning", "reference_binding"} class CharacterWorkflowService: @@ -37,18 +39,52 @@ class CharacterWorkflowService: ] if not active_nodes: raise ApiError(ErrorCode.VALIDATION_FAILED, "workflow has no executable nodes", {"workflow_id": workflow_id}) + if workflow.status == "completed": + return { + "workflow": workflow.to_dict(), + "nodes": [node.to_dict() for node in active_nodes], + "completed_node_ids": [ + node.node_id for node in active_nodes if node.status in {"completed", "selected"} + ], + "queued_node_ids": [node.node_id for node in active_nodes if node.status == "queued"], + "prepared_node_ids": [], + "idempotent_replay": True, + } + previous_execution = workflow.metadata.get("character_execution") + if workflow.status == "running" and isinstance(previous_execution, dict): + completed_node_ids = [node.node_id for node in active_nodes if node.status == "completed"] + queued_node_ids = [node.node_id for node in active_nodes if node.status == "queued"] + prepared_node_ids = [ + node.node_id + for node in active_nodes + if node.node_type in EDITABLE_PREPARATION_NODE_TYPES and node.status == "draft" + ] + return { + "workflow": workflow.to_dict(), + "nodes": [node.to_dict() for node in active_nodes], + "completed_node_ids": completed_node_ids, + "queued_node_ids": queued_node_ids, + "prepared_node_ids": prepared_node_ids, + "idempotent_replay": True, + } running_workflow = workflow - if workflow.status != "running": + if workflow.status not in {"running", "completed"}: running_workflow = self.workflow_service.update_workflow(workflow_id, {"status": "running"}, ctx) - completed_nodes: list[WorkflowNode] = [] + prepared_nodes: list[WorkflowNode] = [] prompt_context: list[str] = [] for node in active_nodes: prompt = _node_prompt(profile, node, running_workflow, prompt_context) negative_prompt = _negative_prompt(profile, node) - completed = self.workflow_service.update_node( + if node.node_type in GENERATION_NODE_TYPES: + target_status = "queued" + elif node.node_type in EDITABLE_PREPARATION_NODE_TYPES: + target_status = node.status + else: + target_status = "completed" + prepared = self.workflow_service.update_node( node.node_id, { - "status": "completed", + "status": target_status, "prompt": prompt, "negative_prompt": negative_prompt, "metadata": { @@ -57,33 +93,46 @@ class CharacterWorkflowService: "executor": "character_workflow", "workflow_type": workflow.workflow_type, "context_prompts": len(prompt_context), + "prepared_only": target_status in {"queued", "draft"}, }, }, }, ctx, ) - completed_nodes.append(completed) + prepared_nodes.append(prepared) prompt_context.append(prompt) - completed_workflow = self.workflow_service.update_workflow( + queued_nodes = [node for node in prepared_nodes if node.status == "queued"] + completed_nodes = [node for node in prepared_nodes if node.status == "completed"] + editable_nodes = [ + node + for node in prepared_nodes + if node.node_type in EDITABLE_PREPARATION_NODE_TYPES and node.status == "draft" + ] + workflow_status = "running" if queued_nodes or editable_nodes else "completed" + updated_workflow = self.workflow_service.update_workflow( workflow_id, { - "status": "completed", - "current_node_id": completed_nodes[-1].node_id, + "status": workflow_status, + "current_node_id": (queued_nodes[0] if queued_nodes else prepared_nodes[-1]).node_id, "metadata": { **running_workflow.metadata, "character_execution": { "executor": "character_workflow", "workflow_type": workflow.workflow_type, "completed_node_ids": [node.node_id for node in completed_nodes], + "queued_node_ids": [node.node_id for node in queued_nodes], + "prepared_node_ids": [node.node_id for node in editable_nodes], }, }, }, ctx, ) return { - "workflow": completed_workflow.to_dict(), - "nodes": [node.to_dict() for node in completed_nodes], + "workflow": updated_workflow.to_dict(), + "nodes": [node.to_dict() for node in prepared_nodes], "completed_node_ids": [node.node_id for node in completed_nodes], + "queued_node_ids": [node.node_id for node in queued_nodes], + "prepared_node_ids": [node.node_id for node in editable_nodes], } @@ -111,6 +160,10 @@ def _negative_prompt(profile: CharacterProfile, node: WorkflowNode) -> str: def _step_instruction(workflow_type: str, node_key: str) -> str: + quick_profile = { + "questionnaire": "normalized questionnaire and quick profile archive input", + "preview": "saved quick profile prompts ready for structured setting split", + } fast = { "fast_profile": "condensed character design brief", "fast_candidates": "candidate batch, varied composition, keep identity consistent", @@ -118,6 +171,8 @@ def _step_instruction(workflow_type: str, node_key: str) -> str: } gradual = { "profile_brief": "core identity and visual anchors", + "prompt_plan": "prompt matrix for portrait, half body, full body, and design sheet nodes", + "material_bind": "reference material binding rules for style, composition, clothing, shoes, and accessories", "headshot": "headshot, face, hair, eyes, expression", "half_body": "half body, upper outfit, posture and silhouette", "full_body": "full body, complete outfit and proportions", @@ -126,7 +181,12 @@ def _step_instruction(workflow_type: str, node_key: str) -> str: "style_lock": "style consistency lock and final rendering rules", "final_pack": "final character pack, accepted images and reusable metadata", } - table = fast if workflow_type == "character_fast" else gradual + tables = { + "character_fast": fast, + "character_gradual": gradual, + "quick_profile": quick_profile, + } + table = tables.get(workflow_type, {}) return table.get(node_key, node_key.replace("_", " ")) diff --git a/backend/services/chat_command_guard_service.py b/backend/services/chat_command_guard_service.py new file mode 100644 index 0000000..bbd36e8 --- /dev/null +++ b/backend/services/chat_command_guard_service.py @@ -0,0 +1,101 @@ +"""Persistent chat command quota checks and audit records.""" + +from __future__ import annotations + +import json +from datetime import datetime, timezone +from typing import Any + +from adapters.astrbot_command_adapter import ChatCommandContext +from core.audit_service import AuditService +from core.identity import RequestContext +from database.db_helper import Database, load_database_settings + + +class ChatCommandGuardService: + def __init__(self, database: Database | None = None): + self.database = database or Database(load_database_settings()) + self.audit = AuditService(self.database) + + def check_generation(self, config: dict[str, Any], chat: ChatCommandContext, *, size: str, active_jobs: list[dict[str, Any]]) -> None: + access = config.get("chat_access") if isinstance(config.get("chat_access"), dict) else {} + user_limit = _positive_int(access.get("user_concurrency_limit"), 2) + group_limit = _positive_int(access.get("group_concurrency_limit"), 5) + daily_limit = _positive_int(access.get("daily_user_generation_limit"), 30) + daily_group_limit = _positive_int(access.get("daily_group_generation_limit"), 100) + if size == "3840x2160" and access.get("allow_4k") is not True: + raise ValueError("聊天命令尚未授权 4K 生成") + if size == "2560x1440" and access.get("allow_2k", True) is not True: + raise ValueError("聊天命令尚未授权 2K 生成") + user_active = sum(1 for item in active_jobs if _request_metadata(item).get("user_id") == chat.user_id) + group_active = sum(1 for item in active_jobs if chat.group_id and _request_metadata(item).get("group_id") == chat.group_id) + if user_active >= user_limit: + raise ValueError(f"用户并发任务已达到上限 {user_limit}") + if chat.group_id and group_active >= group_limit: + raise ValueError(f"群聊并发任务已达到上限 {group_limit}") + if self._daily_count(chat.user_id) >= daily_limit: + raise ValueError(f"用户今日生成次数已达到上限 {daily_limit}") + if chat.group_id and self._daily_group_count(chat.group_id) >= daily_group_limit: + raise ValueError(f"群聊今日生成次数已达到上限 {daily_group_limit}") + + def record(self, chat: ChatCommandContext, action: str, *, target_id: str | None = None, allowed: bool = True, details: dict[str, Any] | None = None) -> None: + ctx = RequestContext.from_astrbot(username=chat.user_id, user_id=chat.user_id, source=f"astrbot_chat:{chat.platform}", permissions=set()) + self.audit.record( + ctx=ctx, + action="chat_command_allowed" if allowed else "chat_command_denied", + target_type="chat_command", + target_id=target_id, + metadata={ + "command": action, + "platform": chat.platform, + "message_origin": chat.message_origin, + "group_id": chat.group_id, + **(details or {}), + }, + ) + + def _daily_count(self, user_id: str) -> int: + placeholder = self.database._placeholder + start = datetime.now(timezone.utc).date().isoformat() + conn = self.database.connect() + try: + row = conn.execute( + f"SELECT COUNT(*) AS count FROM audit_logs WHERE actor_id = {placeholder} AND action = {placeholder} AND created_at >= {placeholder}", + (user_id, "chat_command_allowed", start), + ).fetchone() + return int(row["count"] if row else 0) + finally: + conn.close() + + def _daily_group_count(self, group_id: str) -> int: + placeholder = self.database._placeholder + start = datetime.now(timezone.utc).date().isoformat() + conn = self.database.connect() + try: + rows = conn.execute( + f"SELECT metadata_json FROM audit_logs WHERE action = {placeholder} AND created_at >= {placeholder}", + ("chat_command_allowed", start), + ).fetchall() + count = 0 + for row in rows: + try: + metadata = json.loads(row["metadata_json"] or "{}") + except (TypeError, ValueError): + continue + if str(metadata.get("group_id") or "") == group_id: + count += 1 + return count + finally: + conn.close() + + +def _request_metadata(job: dict[str, Any]) -> dict[str, Any]: + request = job.get("request") if isinstance(job.get("request"), dict) else {} + return request.get("metadata") if isinstance(request.get("metadata"), dict) else {} + + +def _positive_int(value: Any, default: int) -> int: + try: + return max(int(value), 1) + except (TypeError, ValueError): + return default diff --git a/backend/services/chat_media_import_service.py b/backend/services/chat_media_import_service.py new file mode 100644 index 0000000..326fcfa --- /dev/null +++ b/backend/services/chat_media_import_service.py @@ -0,0 +1,47 @@ +"""Safely persist chat-platform image references into ImageStudio storage.""" + +from __future__ import annotations + +import base64 +import hashlib +from pathlib import Path +from urllib.parse import urlsplit + +from core.remote_fetch import read_http_response_limited +from core.storage import VfsStorage + + +MAX_CHAT_IMAGE_BYTES = 20 * 1024 * 1024 + + +class ChatMediaImportService: + def __init__(self, storage: VfsStorage | None = None): + self.storage = storage or VfsStorage() + + def import_image(self, reference: str) -> tuple[str, str, int]: + text = str(reference or "").strip() + if text.startswith("storage://"): + path = self.storage.ensure_file(text) + return text, path.name, path.stat().st_size + if text.startswith("data:image/"): + header, encoded = text.split(",", 1) + body = base64.b64decode(encoded, validate=True) + extension = header.split("/", 1)[1].split(";", 1)[0].replace("jpeg", "jpg") + elif text.startswith(("http://", "https://")): + body = read_http_response_limited(text, timeout_seconds=30, max_bytes=MAX_CHAT_IMAGE_BYTES) + extension = Path(urlsplit(text).path).suffix.lstrip(".").lower() or "jpg" + else: + source = Path(text).expanduser().resolve() + body = source.read_bytes() + extension = source.suffix.lstrip(".").lower() or "jpg" + if not body or len(body) > MAX_CHAT_IMAGE_BYTES: + raise ValueError("聊天图片为空或超过 20MB") + if extension not in {"png", "jpg", "jpeg", "webp", "gif"}: + extension = "jpg" + digest = hashlib.sha256(body).hexdigest() + uri = f"storage://materials/chat/{digest[:2]}/{digest}.{extension}" + target = self.storage.resolve_uri(uri) + target.parent.mkdir(parents=True, exist_ok=True) + if not target.exists(): + target.write_bytes(body) + return uri, target.name, len(body) diff --git a/backend/services/chat_session_service.py b/backend/services/chat_session_service.py new file mode 100644 index 0000000..c3eff37 --- /dev/null +++ b/backend/services/chat_session_service.py @@ -0,0 +1,86 @@ +"""Bounded, reload-safe chat sessions for confirmations and selection.""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from datetime import datetime, timedelta, timezone +from threading import RLock +from typing import Any +from pathlib import Path +import json + +from core.config_writer import write_text_atomic +from core.runtime_paths import resolve_data_path + + +@dataclass +class ChatSession: + key: str + action: str + payload: dict[str, Any] + expires_at: datetime + created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc)) + + +class ChatSessionStore: + def __init__(self, *, ttl_seconds: int = 600, max_sessions: int = 1000, persistence_path: str | Path | None = None): + self.ttl_seconds = ttl_seconds + self.max_sessions = max_sessions + self._items: dict[str, ChatSession] = {} + self._lock = RLock() + self.persistence_path = Path(persistence_path) if persistence_path is not None else resolve_data_path("data/chat_sessions.json") + self._load() + + def put(self, key: str, action: str, payload: dict[str, Any]) -> ChatSession: + with self._lock: + self._prune() + if len(self._items) >= self.max_sessions: + oldest = min(self._items.values(), key=lambda item: item.created_at) + self._items.pop(oldest.key, None) + session = ChatSession(key, action, dict(payload), datetime.now(timezone.utc) + timedelta(seconds=self.ttl_seconds)) + self._items[key] = session + self._save() + return session + + def get(self, key: str) -> ChatSession | None: + with self._lock: + self._prune() + return self._items.get(key) + + def pop(self, key: str) -> ChatSession | None: + with self._lock: + self._prune() + session = self._items.pop(key, None) + self._save() + return session + + def _prune(self) -> None: + now = datetime.now(timezone.utc) + for key in [key for key, item in self._items.items() if item.expires_at <= now]: + self._items.pop(key, None) + + def _load(self) -> None: + try: + payload = json.loads(self.persistence_path.read_text(encoding="utf-8")) + except (OSError, ValueError, TypeError): + return + for item in payload if isinstance(payload, list) else []: + try: + session = ChatSession( + key=str(item["key"]), action=str(item["action"]), payload=dict(item.get("payload") or {}), + expires_at=datetime.fromisoformat(item["expires_at"]), created_at=datetime.fromisoformat(item["created_at"]), + ) + self._items[session.key] = session + except (KeyError, TypeError, ValueError): + continue + self._prune() + + def _save(self) -> None: + payload = [ + {"key": item.key, "action": item.action, "payload": item.payload, "expires_at": item.expires_at.isoformat(), "created_at": item.created_at.isoformat()} + for item in self._items.values() + ] + try: + write_text_atomic(self.persistence_path, json.dumps(payload, ensure_ascii=False, indent=2)) + except OSError: + return diff --git a/backend/services/chat_workflow_service.py b/backend/services/chat_workflow_service.py new file mode 100644 index 0000000..cfa6910 --- /dev/null +++ b/backend/services/chat_workflow_service.py @@ -0,0 +1,377 @@ +"""Chat-facing orchestration built entirely on the existing ImageStudio API contract.""" + +from __future__ import annotations + +import hashlib +import re +from pathlib import Path +from typing import Any, Callable + +from adapters.astrbot_command_adapter import ChatCommandContext +from backend.dto.chat_command_dto import SingleImageCommand +from backend.services.chat_media_import_service import ChatMediaImportService +from backend.services.chat_command_guard_service import ChatCommandGuardService +from backend.services.cross_review_service import CrossReviewService +from core.identity import RequestContext +from core.chat_access_policy import chat_action_allowed +from database.db_helper import load_config_payload + + +Dispatcher = Callable[..., dict[str, Any]] + + +class ChatWorkflowService: + def __init__(self, dispatcher: Dispatcher, config_loader: Callable[[], dict[str, Any]] = load_config_payload, media_importer: ChatMediaImportService | None = None, guard: ChatCommandGuardService | None = None, cross_reviewer_factory: Callable[[], CrossReviewService] = CrossReviewService): + self.dispatcher = dispatcher + self.config_loader = config_loader + self.media_importer = media_importer or ChatMediaImportService() + self.guard = guard or ChatCommandGuardService() + self.cross_reviewer_factory = cross_reviewer_factory + + def submit_single_image(self, command: SingleImageCommand, chat: ChatCommandContext) -> dict[str, Any]: + ctx = self._request_context(chat, "single_image") + config = self.config_loader() + active = self._data(self.dispatcher(ctx, "GET", "jobs", query={"status": "running", "page": 1, "page_size": 100})) + active_items = active.get("items", []) if isinstance(active, dict) else [] + self.guard.check_generation(config, chat, size=command.size, active_jobs=active_items) + profile = self._resolve_profile(ctx, command.profile_ref, chat.user_id) + workflow = self._data(self.dispatcher( + ctx, + "POST", + f"profiles/{profile['profile_id']}/workflows", + payload={ + "workflow_type": "single_image", + "display_name": f"聊天单图 · {command.prompt[:24]}", + "mode": "quick", + "metadata": {"source": "astrbot_chat", "message_origin": chat.message_origin}, + }, + )) + workflow_data = workflow["workflow"] + node = workflow["nodes"][0] + self._data(self.dispatcher(ctx, "POST", f"workflows/{workflow_data['workflow_id']}/start", payload={})) + idempotency_source = chat.message_id or f"{chat.message_origin}:{chat.user_id}:{command.prompt}" + generation = self._data(self.dispatcher( + ctx, + "POST", + f"nodes/{node['node_id']}/generate", + payload={ + "prompt": command.prompt, + "negative_prompt": command.negative_prompt, + "model_route": command.model_route, + "size": command.size, + "count": command.count, + "entrypoint": _chat_entrypoint(chat.platform), + "platform": chat.platform, + "user_id": chat.user_id, + "group_id": chat.group_id, + "translation_consent": command.translation_consent, + "content_permissions": _content_permissions(config, chat), + "idempotency_key": "chat_" + hashlib.sha256(idempotency_source.encode("utf-8")).hexdigest()[:32], + "metadata": { + "source": "astrbot_chat", + "platform": chat.platform, + "message_origin": chat.message_origin, + "user_id": chat.user_id, + "group_id": chat.group_id, + }, + }, + )) + images = [ + self._data(self.dispatcher(ctx, "GET", f"images/{item['image_id']}", query={})) + for item in generation.get("images", []) + if item.get("image_id") + ] + self.guard.record(chat, "single_image", target_id=generation.get("task_id"), details={"size": command.size, "count": command.count}) + return { + "profile": profile, + "workflow": workflow_data, + "node": node, + "generation": generation, + "images": images, + } + + def list_node_images(self, node_id: str, chat: ChatCommandContext) -> list[dict[str, Any]]: + return self._data( + self.dispatcher( + self._request_context(chat), + "GET", + f"nodes/{node_id}/images", + query={"include_archived": False}, + ) + ) + + def list_jobs(self, chat: ChatCommandContext, *, task_id: str | None = None) -> Any: + ctx = self._request_context(chat, "single_image") + endpoint = f"jobs/{task_id}" if task_id else "jobs" + return self._data(self.dispatcher(ctx, "GET", endpoint, query={"page": 1, "page_size": 10})) + + def cancel_job(self, chat: ChatCommandContext, task_id: str) -> Any: + return self._data(self.dispatcher(self._request_context(chat, "single_image"), "POST", f"jobs/{task_id}/cancel", payload={})) + + def retry_job(self, chat: ChatCommandContext, task_id: str) -> Any: + return self._data(self.dispatcher(self._request_context(chat, "single_image"), "POST", f"jobs/{task_id}/retry", payload={})) + + def create_quick_profile_draft(self, description: str, chat: ChatCommandContext) -> dict[str, Any]: + text = str(description or "").strip() + if not text: + raise ValueError("请提供角色名称和描述,例如:/yp 快速角色 米卡 银发蓝眼的冷静少女") + questionnaire = _quick_questionnaire(text) + return self._data(self.dispatcher(self._request_context(chat), "POST", "profiles/quick-draft", payload={ + "questionnaire": questionnaire, + })) + + def save_quick_profile_draft(self, draft: dict[str, Any], chat: ChatCommandContext) -> dict[str, Any]: + return self._data(self.dispatcher(self._request_context(chat), "POST", "profiles/quick-draft/save", payload={ + **draft, + "draft": draft, + "create_workflow": True, + "workflow_status": "draft", + "metadata": {"source": "astrbot_chat", "message_origin": chat.message_origin}, + })) + + def create_or_resume_character_workflow(self, profile_ref: str, chat: ChatCommandContext) -> dict[str, Any]: + ctx = self._request_context(chat, "single_image") + profile = self._resolve_required_profile(ctx, profile_ref) + workflows = self._data(self.dispatcher(ctx, "GET", f"profiles/{profile['profile_id']}/workflows", query={})) + active = next((item for item in workflows if item.get("workflow_type") == "character_gradual" and item.get("status") not in {"completed", "cancelled", "archived"}), None) + if active: + detail = self._data(self.dispatcher(ctx, "GET", f"workflows/{active['workflow_id']}", query={})) + return {"profile": profile, **detail, "resumed": True} + detail = self._data(self.dispatcher(ctx, "POST", f"profiles/{profile['profile_id']}/workflows", payload={ + "workflow_type": "character_gradual", + "display_name": f"{profile['name']} 详细角色工作流", + "mode": "gradual", + "metadata": {"source": "astrbot_chat", "message_origin": chat.message_origin}, + })) + return {"profile": profile, **detail, "resumed": False} + + def workflow_detail(self, workflow_id: str, chat: ChatCommandContext) -> dict[str, Any]: + return self._data(self.dispatcher(self._request_context(chat), "GET", f"workflows/{workflow_id}", query={})) + + def generate_node(self, node_id: str, chat: ChatCommandContext, *, prompt: str = "") -> dict[str, Any]: + config = self.config_loader() + payload = { + **({"prompt": prompt} if prompt else {}), + "entrypoint": _chat_entrypoint(chat.platform), + "platform": chat.platform, + "user_id": chat.user_id, + "group_id": chat.group_id, + "content_permissions": _content_permissions(config, chat), + } + ctx = self._request_context(chat) + result = self._data(self.dispatcher(ctx, "POST", f"nodes/{node_id}/generate", payload=payload)) + result["images"] = [ + self._data(self.dispatcher(ctx, "GET", f"images/{item['image_id']}", query={})) + for item in result.get("images", []) + if item.get("image_id") + ] + return result + + def retry_node(self, node_id: str, chat: ChatCommandContext) -> dict[str, Any]: + return self._data(self.dispatcher(self._request_context(chat), "POST", f"nodes/{node_id}/retry", payload={})) + + def complete_node(self, node_id: str, chat: ChatCommandContext) -> dict[str, Any]: + return self._data(self.dispatcher(self._request_context(chat), "POST", f"nodes/{node_id}/complete", payload={})) + + def next_workflow_step(self, workflow_id: str, chat: ChatCommandContext) -> dict[str, Any]: + ctx = self._request_context(chat) + detail = self._data(self.dispatcher(ctx, "GET", f"workflows/{workflow_id}", query={})) + workflow = detail["workflow"] + nodes = detail.get("nodes", []) + current = next((item for item in nodes if item.get("node_id") == workflow.get("current_node_id")), None) + candidate = current if current and current.get("status") not in {"completed", "skipped"} else next((item for item in nodes if item.get("status") in {"pending", "draft", "failed"}), None) + if candidate is None: + return {"completed": True, "workflow": workflow} + if candidate.get("node_id") != workflow.get("current_node_id"): + self._data(self.dispatcher(ctx, "PUT", f"workflows/{workflow_id}", payload={"current_node_id": candidate["node_id"]})) + generation = self._data(self.dispatcher(ctx, "POST", f"nodes/{candidate['node_id']}/generate", payload={ + "entrypoint": _chat_entrypoint(chat.platform), + "platform": chat.platform, + "user_id": chat.user_id, + "group_id": chat.group_id, + "content_permissions": _content_permissions(self.config_loader(), chat), + })) + return {"completed": False, "workflow": workflow, "node": candidate, "generation": generation} + + def select_node_image(self, node_id: str, image_id: str, chat: ChatCommandContext) -> dict[str, Any]: + ctx = self._request_context(chat) + selected = self._data(self.dispatcher(ctx, "POST", f"nodes/{node_id}/select-image", payload={"image_id": image_id})) + image = self._data(self.dispatcher(ctx, "GET", f"images/{image_id}", query={})) + if image.get("profile_id") and image.get("image_uri"): + self._data(self.dispatcher(ctx, "PUT", f"profiles/{image['profile_id']}", payload={"avatar_image_uri": image["image_uri"]})) + return selected + + def select_node_image_by_number(self, node_id: str, number: int, chat: ChatCommandContext) -> dict[str, Any]: + images = self.list_node_images(node_id, chat) + if number < 1 or number > len(images): + raise ValueError(f"图片序号必须在 1 到 {len(images)} 之间") + return self.select_node_image(node_id, images[number - 1]["image_id"], chat) + + def review(self, target_id: str, chat: ChatCommandContext) -> dict[str, Any]: + prefix = "workflows" if target_id.startswith(("wf_", "workflow_")) else "nodes" + ctx = self._request_context(chat) + rules = self._data(self.dispatcher(ctx, "GET", f"{prefix}/{target_id}/review", query={})) + node_ids = [target_id] + if prefix == "workflows": + detail = self._data(self.dispatcher(ctx, "GET", f"workflows/{target_id}", query={})) + node_ids = [item["node_id"] for item in detail.get("nodes", []) if item.get("status") == "completed"] + model_reviews = {node_id: self.cross_reviewer_factory().review_node(node_id) for node_id in node_ids} + return {"rules": rules, "model_reviews": model_reviews} + + def hires_fix(self, image_id: str, size: str, chat: ChatCommandContext) -> dict[str, Any]: + ctx = self._request_context(chat) + image = self._data(self.dispatcher(ctx, "GET", f"images/{image_id}", query={})) + workflow = self._data(self.dispatcher( + ctx, + "POST", + f"profiles/{image['profile_id']}/workflows", + payload={ + "workflow_type": "single_image", + "display_name": f"高清修复 · {image_id}", + "mode": "quick", + "metadata": {"source": "astrbot_chat_hires", "source_image_id": image_id}, + }, + )) + workflow_data = workflow["workflow"] + node = workflow["nodes"][0] + self._data(self.dispatcher(ctx, "POST", f"workflows/{workflow_data['workflow_id']}/start", payload={})) + config = load_config_payload() + assignments = config.get("model_assignments") if isinstance(config.get("model_assignments"), dict) else {} + generation = self._data(self.dispatcher(ctx, "POST", f"nodes/{node['node_id']}/generate", payload={ + "prompt": "high resolution restoration, preserve identity, composition, clothing and original style", + "negative_prompt": "identity change, composition change, clothing change, oversharpening, artifacts", + "model_route": assignments.get("hires_fix"), + "size": size, + "count": 1, + "input_images": [{"image_uri": image["image_uri"]}], + "metadata": {"source": "astrbot_chat_hires", "source_image_id": image_id}, + })) + images = [self._data(self.dispatcher(ctx, "GET", f"images/{item['image_id']}", query={})) for item in generation.get("images", [])] + return {"source_image": image, "workflow": workflow_data, "node": node, "generation": generation, "images": images} + + def import_material(self, image_reference: str, category_ref: str, chat: ChatCommandContext) -> dict[str, Any]: + ctx = self._request_context(chat) + categories = self._data(self.dispatcher(ctx, "GET", "material-categories", query={})) + category = next((item for item in categories if item.get("category_id") == category_ref or item.get("name") == category_ref), None) + if category is None: + raise ValueError(f"未找到素材分类:{category_ref}") + uri, filename, size_bytes = self.media_importer.import_image(image_reference) + material = self._data(self.dispatcher(ctx, "POST", "materials/upload", payload={ + "category_id": category["category_id"], "display_name": Path(filename).stem, + "image_uri": uri, "thumbnail_uri": uri, "filename": filename, "size_bytes": size_bytes, + "metadata": {"source": "astrbot_chat", "message_origin": chat.message_origin}, + })) + recognized = self._data(self.dispatcher(ctx, "POST", f"materials/{material['material_id']}/recognize", payload={})) + profiles = self._data(self.dispatcher(ctx, "GET", "profiles", query={"page": 1, "page_size": 20})) + return {"category": category, "material": recognized, "profiles": profiles.get("items", []) if isinstance(profiles, dict) else []} + + def confirm_material(self, material_id: str, chat: ChatCommandContext) -> dict[str, Any]: + return self._data(self.dispatcher(self._request_context(chat), "POST", f"materials/{material_id}/confirm", payload={})) + + def re_recognize_material(self, material_id: str, chat: ChatCommandContext) -> dict[str, Any]: + return self._data(self.dispatcher(self._request_context(chat), "POST", f"materials/{material_id}/re-recognize", payload={})) + + def add_material_to_wardrobe(self, material_id: str, profile_ref: str, wardrobe_category: str, chat: ChatCommandContext) -> dict[str, Any]: + ctx = self._request_context(chat) + profile = self._resolve_required_profile(ctx, profile_ref) + return self._data(self.dispatcher(ctx, "POST", f"profiles/{profile['profile_id']}/wardrobe/import-material", payload={ + "material_id": material_id, "wardrobe_category": wardrobe_category, + })) + + def _resolve_profile(self, ctx: RequestContext, profile_ref: str | None, user_id: str) -> dict[str, Any]: + keyword = profile_ref or f"聊天单图 {user_id}" + result = self._data(self.dispatcher(ctx, "GET", "profiles", query={"keyword": keyword, "page": 1, "page_size": 30})) + items = result.get("items", []) if isinstance(result, dict) else [] + if profile_ref: + exact = next((item for item in items if item.get("profile_id") == profile_ref or item.get("name") == profile_ref), None) + if exact: + return exact + raise ValueError(f"未找到角色:{profile_ref}") + if items: + return items[0] + return self._data(self.dispatcher(ctx, "POST", "profiles", payload={ + "name": f"聊天单图 {user_id}", + "profile_type": "quick", + "description": "AstrBot 聊天单图专用角色容器", + "metadata": {"source": "astrbot_chat", "system_container": True}, + })) + + def _resolve_required_profile(self, ctx: RequestContext, profile_ref: str) -> dict[str, Any]: + text = str(profile_ref or "").strip() + if not text: + raise ValueError("请提供角色名称或角色 ID") + result = self._data(self.dispatcher(ctx, "GET", "profiles", query={"keyword": text, "page": 1, "page_size": 30})) + items = result.get("items", []) if isinstance(result, dict) else [] + profile = next((item for item in items if item.get("profile_id") == text or item.get("name") == text), None) + if profile is None: + raise ValueError(f"未找到角色:{text}") + return profile + + def _request_context(self, chat: ChatCommandContext, action: str = "character_workflow") -> RequestContext: + config = self.config_loader() + if not chat_action_allowed(config, action, group_id=chat.group_id, user_id=chat.user_id): + try: + self.guard.record(chat, action, allowed=False, details={"reason": "allowlist"}) + except Exception: + pass + raise ValueError("当前群聊或用户不满足聊天白名单规则") + return RequestContext.from_astrbot( + username=chat.user_id or "chat-user", + user_id=chat.user_id or "chat-user", + permissions={"can_write", "can_retry_tasks", "can_manage_materials"}, + source=f"astrbot_chat:{chat.platform}", + ) + + @staticmethod + def _data(response: dict[str, Any]) -> Any: + if not response.get("ok"): + raise ValueError(str(response.get("message") or response.get("error_code") or "ImageStudio 请求失败")) + return response.get("data") + + +def _content_permissions(config: dict[str, Any], chat: ChatCommandContext) -> dict[str, Any]: + access = config.get("chat_access") if isinstance(config.get("chat_access"), dict) else {} + adult_users = _id_set(access.get("adult_user_allowlist")) + adult_groups = _id_set(access.get("adult_group_allowlist")) + return { + "adult_user_allowed": bool(chat.user_id and chat.user_id in adult_users), + "adult_group_allowed": bool(not chat.group_id or chat.group_id in adult_groups), + "quota_allowed": True, + } + + +def _chat_entrypoint(platform: str) -> str: + value = str(platform or "").lower() + if "qq" in value or "onebot" in value: + return "qq" + if "telegram" in value: + return "telegram" + if "discord" in value: + return "discord" + return "unknown" + + +def _id_set(value: Any) -> set[str]: + if isinstance(value, str): + value = re.split(r"[,,\n]", value) + if not isinstance(value, (list, tuple, set)): + return set() + return {str(item).strip() for item in value if str(item).strip()} + + +def _quick_questionnaire(text: str) -> dict[str, Any]: + parts = text.split(maxsplit=1) + if len(parts) < 2: + raise ValueError("请同时提供角色名和角色描述,例如:/yp 快速角色 米卡 银发蓝眼的冷静少女") + name, description = parts[0].strip(), parts[1].strip() + colors = re.findall(r"([黑白红橙黄绿青蓝紫粉灰棕金银]{1,2})色?(?:头发|发|眼睛|眼)", description) + hair = next((value for value in colors if f"{value}发" in description or f"{value}色头发" in description), "") + eyes = next((value for value in colors if f"{value}眼" in description or f"{value}色眼睛" in description), "") + personality_keywords = [item for item in ("冷静", "温柔", "活泼", "开朗", "内向", "傲娇", "成熟", "勇敢", "神秘") if item in description] + clothing_keywords = [item for item in ("制服", "礼服", "外套", "裙", "衬衫", "盔甲", "斗篷", "和服", "运动服") if item in description] + hairstyle = next((item for item in ("长发", "短发", "双马尾", "马尾", "卷发", "直发", "齐刘海") if item in description), "") + return { + "name": name, "identity": description, "personality": personality_keywords, + "hair_color": hair, "hairstyle": hairstyle, "eye_color": eyes, + "clothing_keywords": clothing_keywords, + } diff --git a/backend/services/composition_builder.py b/backend/services/composition_builder.py new file mode 100644 index 0000000..15a6c53 --- /dev/null +++ b/backend/services/composition_builder.py @@ -0,0 +1,298 @@ +"""Deterministic, explainable material composition for generation prompts.""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any, Mapping + +from backend.services.material_normalization_service import CAPSULE_TIERS, WEAR_LAYERS + + +BASE_WEIGHTS = { + "identity": 100.0, + "dress": 82.0, + "suit": 80.0, + "outerwear": 72.0, + "top": 70.0, + "bottom": 70.0, + "hair": 68.0, + "footwear": 55.0, + "jewelry": 38.0, + "headwear": 38.0, + "face_accessory": 38.0, + "prop": 34.0, + "pose": 45.0, + "style": 42.0, + "background": 30.0, + "other": 20.0, +} +LAYER_ORDER = {layer: index for index, layer in enumerate(("skin", "inner", "base", "middle", "outer", "overlay", "accessory", "prop"))} +CONTRADICTIONS = ( + ("fitted", "oversized"), ("matte", "glossy"), ("short sleeve", "long sleeve"), + ("transparent", "opaque"), ("minimal", "ornate"), +) + + +@dataclass(frozen=True) +class ReferenceImageBinding: + material_id: str + image_uri: str + role: str + use_for: list[str] + do_not_inherit: list[str] + weight: float + + def to_dict(self) -> dict[str, Any]: + return dict(self.__dict__) + + +@dataclass(frozen=True) +class CompositionPlan: + materials: list[dict[str, Any]] + identity_prompt: str = "" + user_prompt: str = "" + negative_prompt: str = "" + prompt_budget: int = 4000 + max_reference_images: int = 8 + visible_slots: list[str] = field(default_factory=list) + season: str = "" + scene: str = "" + + +@dataclass +class CompositionResult: + prompt: str + negative_prompt: str + selected_materials: list[dict[str, Any]] = field(default_factory=list) + resolution_log: list[dict[str, Any]] = field(default_factory=list) + conflicts: list[dict[str, Any]] = field(default_factory=list) + ignored_materials: list[dict[str, Any]] = field(default_factory=list) + downgraded_materials: list[dict[str, Any]] = field(default_factory=list) + reference_bindings: list[ReferenceImageBinding] = field(default_factory=list) + budget: dict[str, Any] = field(default_factory=dict) + + def to_dict(self) -> dict[str, Any]: + return { + "prompt": self.prompt, + "negative_prompt": self.negative_prompt, + "selected_materials": self.selected_materials, + "resolution_log": self.resolution_log, + "conflicts": self.conflicts, + "ignored_materials": self.ignored_materials, + "downgraded_materials": self.downgraded_materials, + "reference_bindings": [item.to_dict() for item in self.reference_bindings], + "budget": self.budget, + } + + +class CompositionBuilder: + def build(self, plan: CompositionPlan | Mapping[str, Any]) -> CompositionResult: + if not isinstance(plan, CompositionPlan): + plan = CompositionPlan( + materials=list(plan.get("materials") or []), + identity_prompt=_text(plan.get("identity_prompt")), + user_prompt=_text(plan.get("user_prompt")), + negative_prompt=_text(plan.get("negative_prompt")), + prompt_budget=max(int(plan.get("prompt_budget") or 4000), 256), + max_reference_images=max(int(plan.get("max_reference_images") or 8), 0), + visible_slots=_string_list(plan.get("visible_slots")), + season=_text(plan.get("season")), + scene=_text(plan.get("scene")), + ) + prepared = [self._prepare_material(item, plan) for item in plan.materials if isinstance(item, dict)] + prepared.sort(key=lambda item: (-item["effective_weight"], item["material_id"])) + kept, ignored, conflicts, resolution_log = self._resolve_conflicts(prepared) + kept.sort(key=lambda item: (self._semantic_order(item), -item["effective_weight"], item["material_id"])) + prompt_parts = _unique([plan.identity_prompt, *self._capsule_parts(kept), plan.user_prompt]) + negative_parts = _unique([plan.negative_prompt, *self._negative_parts(kept)]) + prompt, budget, downgraded = self._apply_budget(prompt_parts, kept, plan.prompt_budget) + references = self._reference_bindings(kept, plan.max_reference_images, downgraded) + return CompositionResult( + prompt=prompt, + negative_prompt=", ".join(negative_parts), + selected_materials=kept, + resolution_log=resolution_log, + conflicts=conflicts, + ignored_materials=ignored, + downgraded_materials=downgraded, + reference_bindings=references, + budget=budget, + ) + + def _prepare_material(self, item: dict[str, Any], plan: CompositionPlan) -> dict[str, Any]: + metadata = item.get("metadata") if isinstance(item.get("metadata"), dict) else {} + profile = metadata.get("normalized_asset_profile") if isinstance(metadata.get("normalized_asset_profile"), dict) else {} + capsules = metadata.get("generation_capsules") if isinstance(metadata.get("generation_capsules"), dict) else {} + category = _text(profile.get("category") or item.get("category") or item.get("wardrobe_category") or "other") + role = _text(profile.get("role") or item.get("role") or "other") + confidence = _float(profile.get("confidence"), 0.5) + locked = bool(item.get("locked") or item.get("user_locked") or profile.get("user_locked")) + user_adjustment = _float(item.get("user_adjustment"), 0.0) + role_bonus = 20.0 if role == "identity" else 8.0 if category in {"dress", "suit", "top", "bottom", "outerwear"} else 0.0 + lock_bonus = 100.0 if locked else 0.0 + slots = _string_list(profile.get("body_slots")) + automatic_visibility = 10.0 if plan.visible_slots and set(slots) & set(plan.visible_slots) else 0.0 + visibility_bonus = _float(item.get("visibility_bonus"), 0.0) + automatic_visibility + restoration_feedback_bonus = (_float(item.get("restoration_score"), 0.5) - 0.5) * 20.0 + season_penalty = 12.0 if plan.season and _text(item.get("season")) and plan.season.casefold() != _text(item.get("season")).casefold() else 0.0 + low_confidence_penalty = max(0.0, 0.80 - confidence) * 40.0 + base_weight = BASE_WEIGHTS.get(category, BASE_WEIGHTS["other"]) + effective_weight = base_weight + user_adjustment + role_bonus + lock_bonus + visibility_bonus + restoration_feedback_bonus - low_confidence_penalty - season_penalty + tier = _text(item.get("capsule_tier") or "detailed").lower() + if tier not in CAPSULE_TIERS: + tier = "standard" + capsule = capsules.get(tier) if isinstance(capsules.get(tier), dict) else {} + prompt_parts = _string_list(capsule.get("prompt_parts")) or _split_prompt(item.get("prompt")) + negative_parts = _string_list(capsule.get("negative_parts")) or _split_prompt(item.get("negative_prompt")) + return { + **item, + "material_id": _text(item.get("material_id")) or _text(item.get("wardrobe_item_id")) or f"material-{id(item)}", + "category": category, + "role": role, + "body_slots": slots, + "wear_layer": _text(profile.get("wear_layer")) or None, + "stackable": bool(item.get("stackable", profile.get("stackable", False))), + "locked": locked, + "capsule_tier": tier, + "prompt_parts": prompt_parts, + "negative_parts": negative_parts, + "must_preserve": _string_list(capsule.get("must_preserve") or profile.get("must_preserve")), + "forbidden_changes": _string_list(capsule.get("forbidden_changes") or profile.get("forbidden_changes")), + "effective_weight": round(effective_weight, 3), + "weight_breakdown": { + "base_weight": base_weight, + "user_adjustment": user_adjustment, + "lock_bonus": lock_bonus, + "role_bonus": role_bonus, + "visibility_bonus": visibility_bonus, + "restoration_feedback_bonus": round(restoration_feedback_bonus, 3), + "season_penalty": season_penalty, + "low_confidence_penalty": round(low_confidence_penalty, 3), + }, + } + + def _resolve_conflicts(self, items: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], list[dict[str, Any]], list[dict[str, Any]], list[dict[str, Any]]]: + kept: list[dict[str, Any]] = [] + ignored: list[dict[str, Any]] = [] + conflicts: list[dict[str, Any]] = [] + log: list[dict[str, Any]] = [] + for candidate in items: + blocker = next((current for current in kept if self._conflict_reason(current, candidate)), None) + if blocker is None: + kept.append(candidate) + log.append({"material_id": candidate["material_id"], "decision": "kept", "reason": "no_conflict", "effective_weight": candidate["effective_weight"]}) + continue + reason = self._conflict_reason(blocker, candidate) or "conflict" + conflict = {"type": reason, "winner": blocker["material_id"], "loser": candidate["material_id"]} + conflicts.append(conflict) + ignored_item = {**candidate, "ignored_reason": reason, "conflicts_with": blocker["material_id"]} + ignored.append(ignored_item) + log.append({"material_id": candidate["material_id"], "decision": "ignored", "reason": reason, "winner": blocker["material_id"]}) + return kept, ignored, conflicts, log + + def _conflict_reason(self, left: Mapping[str, Any], right: Mapping[str, Any]) -> str | None: + categories = {left.get("category"), right.get("category")} + if "dress" in categories and categories & {"top", "bottom", "suit"}: + return "dress_separate_garment_conflict" + slots = set(left.get("body_slots") or []) & set(right.get("body_slots") or []) + if slots: + if left.get("stackable") and right.get("stackable"): + return None + left_layer, right_layer = left.get("wear_layer"), right.get("wear_layer") + if left_layer and right_layer and left_layer != right_layer: + return None + return "body_slot_conflict" + left_text = " ".join(left.get("prompt_parts") or []).lower() + right_text = " ".join(right.get("prompt_parts") or []).lower() + if any((a in left_text and b in right_text) or (b in left_text and a in right_text) for a, b in CONTRADICTIONS): + return "attribute_contradiction" + return None + + def _semantic_order(self, item: Mapping[str, Any]) -> tuple[int, int]: + role_order = {"identity": 0, "garment": 2, "accessory": 6, "prop": 7, "pose": 8, "background": 9, "style": 10, "other": 11} + return role_order.get(_text(item.get("role")), 11), LAYER_ORDER.get(_text(item.get("wear_layer")), 99) + + def _capsule_parts(self, items: list[dict[str, Any]]) -> list[str]: + result: list[str] = [] + for item in items: + result.extend(item["prompt_parts"]) + result.extend(f"must preserve {value}" for value in item["must_preserve"]) + return result + + def _negative_parts(self, items: list[dict[str, Any]]) -> list[str]: + return [part for item in items for part in [*item["negative_parts"], *item["forbidden_changes"]]] + + def _apply_budget(self, parts: list[str], items: list[dict[str, Any]], budget: int) -> tuple[str, dict[str, Any], list[dict[str, Any]]]: + original = ", ".join(parts) + if len(original) <= budget: + return original, {"limit": budget, "before": len(original), "after": len(original), "removed": []}, [] + locked_terms = {term for item in items if item["locked"] or item["role"] == "identity" for term in [*item["prompt_parts"], *item["must_preserve"]]} + kept = list(parts) + removed: list[str] = [] + for part in reversed(parts): + if len(", ".join(kept)) <= budget: + break + if part in locked_terms or part.startswith("must preserve "): + continue + kept.remove(part) + removed.append(part) + text = ", ".join(kept) + if len(text) > budget: + # Locked and safety-relevant content is never silently truncated. + text = text + downgraded = [{"reason": "prompt_budget", "removed_prompt_parts": removed}] if removed else [] + return text, {"limit": budget, "before": len(original), "after": len(text), "removed": removed}, downgraded + + def _reference_bindings(self, items: list[dict[str, Any]], limit: int, downgraded: list[dict[str, Any]]) -> list[ReferenceImageBinding]: + candidates = [item for item in items if _text(item.get("image_uri"))] + candidates.sort(key=lambda item: (-item["effective_weight"], item["material_id"])) + bindings: list[ReferenceImageBinding] = [] + for item in candidates[:limit]: + role = item["role"] + if role == "identity": + use_for = ["identity", "face", "hair"] + do_not_inherit = ["pose", "background", "style"] + elif role in {"garment", "accessory"}: + use_for = ["clothing", "design", "color", "material"] + do_not_inherit = ["face", "identity", "body_shape", "pose", "background", "style"] + else: + use_for = [role] + do_not_inherit = ["identity", "face"] + bindings.append(ReferenceImageBinding(item["material_id"], item["image_uri"], role, use_for, do_not_inherit, min(max(item["effective_weight"] / 100.0, 0.1), 1.0))) + for item in candidates[limit:]: + downgraded.append({"material_id": item["material_id"], "reason": "reference_image_limit", "fallback": "text_capsule"}) + return bindings + + +def _split_prompt(value: Any) -> list[str]: + return [part.strip() for part in _text(value).replace(";", ",").split(",") if part.strip()] + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return _split_prompt(value) + if not isinstance(value, (list, tuple, set)): + return [] + return _unique([_text(item) for item in value]) + + +def _unique(values: list[str]) -> list[str]: + result: list[str] = [] + seen: set[str] = set() + for item in values: + text = _text(item) + if text and text.casefold() not in seen: + seen.add(text.casefold()) + result.append(text) + return result + + +def _float(value: Any, default: float) -> float: + try: + return float(value) + except (TypeError, ValueError): + return default + + +def _text(value: Any) -> str: + return str(value or "").strip() diff --git a/backend/services/cost_service.py b/backend/services/cost_service.py index e3249b4..8675b3b 100644 --- a/backend/services/cost_service.py +++ b/backend/services/cost_service.py @@ -51,10 +51,28 @@ class CostService: key = (provider, provider_model, model_route) group = groups.setdefault(key, _new_group(provider, provider_model, model_route)) group["image_count"] += 1 - group["_task_ids"].add(str(row["task_id"] or row["image_id"])) - group["total_tokens"] += _usage_number(usage, "total_tokens", "total") - group["input_tokens"] += _usage_number(usage, "input_tokens", "prompt_tokens") - group["output_tokens"] += _usage_number(usage, "output_tokens", "completion_tokens") + task_key = str(row["task_id"] or row["image_id"]) + group["_task_ids"].add(task_key) + # Image providers report usage for the whole request, while one + # request may persist several image rows. Keep the largest value + # observed for each task so the same request is not charged once + # per output image. + task_usage = group["_usage_by_task"].setdefault( + task_key, + {"total_tokens": 0, "input_tokens": 0, "output_tokens": 0}, + ) + task_usage["total_tokens"] = max( + task_usage["total_tokens"], + _usage_number(usage, "total_tokens", "total"), + ) + task_usage["input_tokens"] = max( + task_usage["input_tokens"], + _usage_number(usage, "input_tokens", "prompt_tokens"), + ) + task_usage["output_tokens"] = max( + task_usage["output_tokens"], + _usage_number(usage, "output_tokens", "completion_tokens"), + ) finalized = [_finalize_group(group, prices) for group in groups.values()] totals = { "image_count": sum(group["image_count"] for group in finalized), @@ -153,12 +171,17 @@ def _new_group(provider_id: str, provider_model: str, model_route: str) -> dict[ "output_tokens": 0, "estimated_cost": 0.0, "_task_ids": set(), + "_usage_by_task": {}, } def _finalize_group(group: dict[str, Any], prices: dict[str, PriceSpec]) -> dict[str, Any]: group = dict(group) group["task_count"] = len(group.pop("_task_ids")) + usage_by_task = group.pop("_usage_by_task") + group["total_tokens"] = sum(item["total_tokens"] for item in usage_by_task.values()) + group["input_tokens"] = sum(item["input_tokens"] for item in usage_by_task.values()) + group["output_tokens"] = sum(item["output_tokens"] for item in usage_by_task.values()) group["estimated_cost"] = round(_estimate_cost(group, prices), 6) return group @@ -195,7 +218,7 @@ def _price_spec(group: dict[str, Any], prices: dict[str, PriceSpec]) -> PriceSpe def _provider_id(metadata: dict[str, Any]) -> str: provider_error = metadata.get("provider_error") if isinstance(metadata.get("provider_error"), dict) else {} - return str(metadata.get("provider_id") or provider_error.get("provider_id") or "mock") + return str(metadata.get("provider_id") or provider_error.get("provider_id") or "unknown") def _provider_model(metadata: dict[str, Any]) -> str: diff --git a/backend/services/cross_review_service.py b/backend/services/cross_review_service.py new file mode 100644 index 0000000..3c2e008 --- /dev/null +++ b/backend/services/cross_review_service.py @@ -0,0 +1,95 @@ +"""Multi-route visual review used by AstrBot chat commands.""" + +from __future__ import annotations + +import base64 +import mimetypes +from concurrent.futures import ThreadPoolExecutor, as_completed +from typing import Any + +from backend.services.model_provider_bootstrap import sync_vision_provider_from_config +from backend.services.model_provider_runtime import ModelProviderRuntime +from backend.services.model_provider_service import ModelProviderService +from core.storage import VfsStorage +from database.db_helper import Database, load_config_payload, load_database_settings +from database.repositories.image_repository import ImageRepository +from database.repositories.model_provider_repository import ModelProviderRepository +from engines.vision_provider import VisionRecognitionRequest + + +class CrossReviewService: + def __init__(self): + self.database = Database(load_database_settings()) + self.images = ImageRepository(self.database) + self.storage = VfsStorage() + self.config = load_config_payload() + provider_service = ModelProviderService(ModelProviderRepository(self.database)) + sync_vision_provider_from_config(provider_service, self.config) + self.runtime = ModelProviderRuntime(provider_service) + + def review_node(self, node_id: str) -> dict[str, Any]: + images = self.images.list_by_node(node_id, include_archived=False) + selected = next((item for item in images if item.is_selected), None) or (images[0] if images else None) + if selected is None: + return {"routes": [], "reviews": [], "consensus": "节点没有可审核图片", "disagreements": []} + image_data = self._data_url(selected.image_uri) + routes = self._routes() + reviews: list[dict[str, Any]] = [] + with ThreadPoolExecutor(max_workers=min(len(routes), 4) or 1) as executor: + futures = {executor.submit(self._review_route, route, image_data, selected.image_id): route for route in routes} + for future in as_completed(futures): + route = futures[future] + try: + reviews.append(future.result()) + except Exception as exc: + reviews.append({"route": route, "status": "failed", "error": str(exc)}) + successful = [item for item in reviews if item.get("status") == "succeeded"] + defects = sorted({tag for item in successful for tag in item.get("defects", [])}) + return { + "image_id": selected.image_id, + "routes": routes, + "reviews": reviews, + "consensus": "、".join(defects) if defects else ("未发现明确缺陷" if successful else "没有模型完成审核"), + "disagreements": _disagreements(successful), + "average_confidence": round(sum(float(item.get("confidence") or 0) for item in successful) / len(successful), 3) if successful else 0, + } + + def _review_route(self, route: str, image_data: str, image_id: str) -> dict[str, Any]: + provider = self.runtime.vision_provider_for_route({"route_name": route, "modality": "cross_review", "image_id": image_id}) + if provider is None: + raise ValueError("审核模型路由不可用或凭据缺失") + instruction = ( + "Independently inspect this generated character image. Check identity consistency, anatomy, hands, face, " + "clothing structure, material, composition, text, watermark and rendering artifacts. Return JSON using the " + "required fields. Put defect labels in tags, detailed verdict in description, recommended correction in prompt, " + "and confidence from 0 to 1. Do not agree with another reviewer because no other verdict is provided." + ) + result = provider.recognize_material(VisionRecognitionRequest( + display_name=f"cross review {image_id}", image_uri=image_data, + metadata={"recognition_instruction": instruction}, + )) + return { + "route": route, "provider_id": result.provider_id, "model": result.model, "status": "succeeded", + "defects": result.tags, "confidence": result.confidence, "verdict": result.description, + "recommendation": result.prompt, "negative_prompt": result.negative_prompt, + } + + def _routes(self) -> list[str]: + assignments = self.config.get("model_assignments") if isinstance(self.config.get("model_assignments"), dict) else {} + configured = assignments.get("cross_review_routes") + routes = [str(item).strip() for item in configured] if isinstance(configured, list) else [] + routes.extend([str(assignments.get("cross_review_primary") or "").strip(), str(assignments.get("cross_review_secondary") or "").strip()]) + return list(dict.fromkeys(item for item in routes if item)) + + def _data_url(self, uri: str) -> str: + path = self.storage.ensure_file(uri) + mime = mimetypes.guess_type(path.name)[0] or "image/png" + return f"data:{mime};base64,{base64.b64encode(path.read_bytes()).decode('ascii')}" + + +def _disagreements(reviews: list[dict[str, Any]]) -> list[str]: + if len(reviews) < 2: + return [] + tag_sets = [set(item.get("defects", [])) for item in reviews] + union = set().union(*tag_sets) + return sorted(tag for tag in union if not all(tag in tags for tags in tag_sets)) diff --git a/backend/services/dashboard_service.py b/backend/services/dashboard_service.py index bdbb3e7..9d869be 100644 --- a/backend/services/dashboard_service.py +++ b/backend/services/dashboard_service.py @@ -7,7 +7,8 @@ from typing import Any from backend.dto.common import ApiError from core.config_service import ConfigService, ImageStudioConfig -from database.db_helper import Database, REPO_ROOT +from core.runtime_paths import resolve_data_path +from database.db_helper import Database from database.repositories.job_repository import JobRepository from backend.services.media_service import MediaService @@ -51,7 +52,7 @@ class DashboardService: counts = queue_repository.count_by_status() return { "pending": counts.get("created", 0) + counts.get("queued", 0), - "running": counts.get("running", 0), + "running": counts.get("running", 0) + counts.get("retrying", 0), "failed": counts.get("failed", 0), "dead_letter": counts.get("dead_letter", 0), } @@ -94,10 +95,10 @@ class DashboardService: return None def _storage_summary(self, config: ImageStudioConfig) -> dict[str, int]: - images_bytes = _directory_size(_resolve_data_path(config.storage.images_path)) - thumbnails_bytes = _directory_size(_resolve_data_path(config.storage.thumbnails_path)) - materials_bytes = _directory_size(_resolve_data_path(config.storage.materials_path)) - temp_bytes = _directory_size(_resolve_data_path(config.storage.temp_path)) + images_bytes = _directory_size(resolve_data_path(config.storage.images_path)) + thumbnails_bytes = _directory_size(resolve_data_path(config.storage.thumbnails_path)) + materials_bytes = _directory_size(resolve_data_path(config.storage.materials_path)) + temp_bytes = _directory_size(resolve_data_path(config.storage.temp_path)) return { "total_bytes": images_bytes + thumbnails_bytes + materials_bytes + temp_bytes, "images_bytes": images_bytes + thumbnails_bytes, @@ -106,13 +107,6 @@ class DashboardService: } -def _resolve_data_path(path: str) -> Path: - resolved = Path(path) - if not resolved.is_absolute(): - resolved = REPO_ROOT / resolved - return resolved - - def _directory_size(path: Path) -> int: if not path.exists(): return 0 diff --git a/backend/services/data_backup_service.py b/backend/services/data_backup_service.py new file mode 100644 index 0000000..8462598 --- /dev/null +++ b/backend/services/data_backup_service.py @@ -0,0 +1,293 @@ +"""Server-side ImageStudio data backup and guarded restore.""" + +from __future__ import annotations + +import hashlib +import json +import shutil +import sqlite3 +import tempfile +import zipfile +from datetime import datetime, timezone +from pathlib import Path +from typing import Any +from uuid import uuid4 + +from backend.dto.common import ApiError, ErrorCode +from core.config_service import ConfigService +from core.runtime_paths import resolve_data_path +from database.db_helper import load_config_payload, load_database_settings + +MAX_BACKUP_FILES = 50_000 +MAX_RESTORE_BYTES = 10 * 1024 * 1024 * 1024 + + +class DataBackupService: + def __init__(self, config_service: ConfigService | None = None): + self.config_service = config_service or ConfigService() + + def list_backups(self) -> list[dict[str, Any]]: + root = self._backup_root() + if not root.exists(): + return [] + return [self._summary(path) for path in sorted(root.glob("*.zip"), key=lambda item: item.stat().st_mtime, reverse=True)] + + def create_backup( + self, + *, + include_media: bool = True, + include_prompt_records: bool = True, + include_moderation_records: bool = True, + ) -> dict[str, Any]: + backup_id = f"backup_{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}_{uuid4().hex[:8]}" + root = self._backup_root() + root.mkdir(parents=True, exist_ok=True) + destination = root / f"{backup_id}.zip" + config = load_config_payload(self.config_service.config_path) + entries: list[dict[str, Any]] = [] + with tempfile.TemporaryDirectory(prefix="imagestudio-backup-") as temp_dir: + temp = Path(temp_dir) + config_copy = temp / "config.yaml" + if self.config_service.config_path.exists(): + shutil.copy2(self.config_service.config_path, config_copy) + entries.append(_entry(config_copy, "config/config.yaml")) + database_settings = load_database_settings(self.config_service.config_path) + warnings: list[str] = [] + if database_settings.selected_backend == "sqlite": + source_db = resolve_data_path(database_settings.sqlite.path) + if source_db.exists(): + db_copy = temp / "imagestudio.db" + source = sqlite3.connect(source_db) + target = sqlite3.connect(db_copy) + try: + source.backup(target) + finally: + target.close(); source.close() + if not include_prompt_records or not include_moderation_records: + _sanitize_sqlite_backup( + db_copy, + include_prompt_records=include_prompt_records, + include_moderation_records=include_moderation_records, + ) + entries.append(_entry(db_copy, "database/imagestudio.db")) + else: + warnings.append("PostgreSQL data is not embedded; use pg_dump with the connection recorded in the redacted config.") + storage = config.get("storage") if isinstance(config.get("storage"), dict) else {} + media_roots = { + key: resolve_data_path(storage.get(f"{key}_path") or f"data/{key}") + for key in ("images", "thumbnails", "materials", "styles") + } + if include_media: + for key, media_root in media_roots.items(): + if not media_root.exists(): + continue + for path in media_root.rglob("*"): + if not path.is_file(): + continue + if len(entries) >= MAX_BACKUP_FILES: + raise ApiError(ErrorCode.VALIDATION_FAILED, "backup file count exceeds safety limit", {"limit": MAX_BACKUP_FILES}) + entries.append(_entry(path, f"media/{key}/{path.relative_to(media_root).as_posix()}")) + manifest = { + "format": "imagestudio-full-backup", + "version": 2, + "material_schema_version": "1.0", + "backup_id": backup_id, + "created_at": datetime.now(timezone.utc).isoformat(), + "database_backend": database_settings.selected_backend, + "include_media": include_media, + "include_prompt_records": include_prompt_records, + "include_moderation_records": include_moderation_records, + "entries": entries, + "warnings": warnings, + } + with zipfile.ZipFile(destination, "w", compression=zipfile.ZIP_DEFLATED, compresslevel=6) as archive: + for item in entries: + source_path = config_copy if item["path"] == "config/config.yaml" else db_copy if item["path"] == "database/imagestudio.db" else _source_for_media_entry(item["path"], media_roots) + archive.write(source_path, item["path"]) + archive.writestr("manifest.json", json.dumps(manifest, ensure_ascii=False, indent=2)) + return self._summary(destination, manifest=manifest) + + def inspect_backup(self, backup_id: str) -> dict[str, Any]: + path = self._backup_path(backup_id) + with zipfile.ZipFile(path) as archive: + manifest = _read_manifest(archive) + _validate_archive(archive, manifest) + return {**self._summary(path, manifest=manifest), "manifest": manifest, "valid": True} + + def restore_backup(self, backup_id: str, *, confirm: bool = False) -> dict[str, Any]: + if not confirm: + return {**self.inspect_backup(backup_id), "restored": False, "requires_confirmation": True} + path = self._backup_path(backup_id) + with zipfile.ZipFile(path) as archive: + manifest = _read_manifest(archive) + _validate_archive(archive, manifest) + safety_backup = self.create_backup(include_media=False, include_prompt_records=True, include_moderation_records=True) + with tempfile.TemporaryDirectory(prefix="imagestudio-restore-") as temp_dir: + temp = Path(temp_dir) + archive.extractall(temp) + config_source = temp / "config" / "config.yaml" + if config_source.exists(): + self.config_service.config_path.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(config_source, self.config_service.config_path) + database_settings = load_database_settings(self.config_service.config_path) + db_source = temp / "database" / "imagestudio.db" + if db_source.exists() and database_settings.selected_backend == "sqlite": + db_target = resolve_data_path(database_settings.sqlite.path) + db_target.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(db_source, db_target) + config = load_config_payload(self.config_service.config_path) + storage = config.get("storage") if isinstance(config.get("storage"), dict) else {} + for key in ("images", "thumbnails", "materials", "styles"): + source_root = temp / "media" / key + if not source_root.exists(): + continue + target_root = resolve_data_path(storage.get(f"{key}_path") or f"data/{key}") + target_root.mkdir(parents=True, exist_ok=True) + shutil.copytree(source_root, target_root, dirs_exist_ok=True) + return {"restored": True, "backup_id": backup_id, "safety_backup_id": safety_backup["backup_id"]} + + def _backup_root(self) -> Path: + config = self.config_service.load() + return resolve_data_path(config.storage.archives_path) / "backups" + + def _backup_path(self, backup_id: str) -> Path: + normalized = str(backup_id or "").strip() + if not normalized or any(char not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-" for char in normalized): + raise ApiError(ErrorCode.VALIDATION_FAILED, "invalid backup_id", {"backup_id": backup_id}) + path = self._backup_root() / f"{normalized}.zip" + if not path.exists(): + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "backup not found", {"backup_id": normalized}) + return path + + def _summary(self, path: Path, *, manifest: dict[str, Any] | None = None) -> dict[str, Any]: + loaded = manifest + if loaded is None: + try: + with zipfile.ZipFile(path) as archive: + loaded = _read_manifest(archive) + except Exception: + loaded = {} + stat = path.stat() + return {"backup_id": loaded.get("backup_id") or path.stem, "created_at": loaded.get("created_at") or datetime.fromtimestamp(stat.st_mtime,timezone.utc).isoformat(), "size_bytes": stat.st_size, "file_count": len(loaded.get("entries") or []), "database_backend": loaded.get("database_backend"), "include_media": bool(loaded.get("include_media")), "include_prompt_records": loaded.get("include_prompt_records", True), "include_moderation_records": loaded.get("include_moderation_records", True), "material_schema_version": loaded.get("material_schema_version"), "warnings": loaded.get("warnings") or []} + + +def _entry(path: Path, archive_path: str) -> dict[str, Any]: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return {"path": archive_path, "size": path.stat().st_size, "sha256": digest.hexdigest()} + + +def _source_for_media_entry(archive_path: str, roots: dict[str, Path]) -> Path: + _, key, relative = archive_path.split("/", 2) + return roots[key] / Path(relative) + + +def _read_manifest(archive: zipfile.ZipFile) -> dict[str, Any]: + try: + value = json.loads(archive.read("manifest.json")) + except (KeyError, json.JSONDecodeError) as exc: + raise ApiError(ErrorCode.VALIDATION_FAILED, "invalid backup manifest", {}) from exc + if not isinstance(value, dict) or value.get("format") != "imagestudio-full-backup": + raise ApiError(ErrorCode.VALIDATION_FAILED, "unsupported backup format", {}) + return value + + +def _validate_archive(archive: zipfile.ZipFile, manifest: dict[str, Any]) -> None: + entries = manifest.get("entries") if isinstance(manifest.get("entries"), list) else [] + if len(entries) > MAX_BACKUP_FILES: + raise ApiError(ErrorCode.VALIDATION_FAILED, "backup file count exceeds safety limit", {}) + names = set(archive.namelist()) + declared_names = {str(item.get("path") or "") for item in entries if isinstance(item, dict)} + if names - declared_names != {"manifest.json"}: + raise ApiError(ErrorCode.VALIDATION_FAILED, "backup contains undeclared entries", {}) + total = 0 + for item in entries: + name = str(item.get("path") or "") if isinstance(item, dict) else "" + parts = Path(name).parts + if not name or name not in names or Path(name).is_absolute() or ".." in parts: + raise ApiError(ErrorCode.VALIDATION_FAILED, "unsafe backup entry", {"path": name}) + info = archive.getinfo(name) + total += info.file_size + if total > MAX_RESTORE_BYTES: + raise ApiError(ErrorCode.VALIDATION_FAILED, "backup exceeds restore size limit", {}) + digest = hashlib.sha256(archive.read(name)).hexdigest() + if digest != item.get("sha256"): + raise ApiError(ErrorCode.VALIDATION_FAILED, "backup checksum mismatch", {"path": name}) + + +def _sanitize_sqlite_backup( + path: Path, + *, + include_prompt_records: bool, + include_moderation_records: bool, +) -> None: + conn = sqlite3.connect(path) + conn.row_factory = sqlite3.Row + try: + tables = {str(row["name"]) for row in conn.execute("SELECT name FROM sqlite_master WHERE type = 'table'")} + if not include_prompt_records: + if "materials" in tables: + conn.execute("UPDATE materials SET prompt = NULL, negative_prompt = NULL") + if "generation_images" in tables: + conn.execute("UPDATE generation_images SET positive_prompt = NULL, negative_prompt = NULL") + json_columns = { + "materials": ("material_id", ("metadata_json",)), + "generation_images": ("image_id", ("metadata_json",)), + "generation_tasks": ("task_id", ("request_json", "result_json", "metadata_json")), + "audit_logs": ("audit_id", ("before_snapshot", "after_snapshot", "metadata_json")), + } + for table, (primary_key, columns) in json_columns.items(): + if table not in tables: + continue + existing_columns = {str(row["name"]) for row in conn.execute(f"PRAGMA table_info({table})")} + selected_columns = [column for column in columns if column in existing_columns] + if not selected_columns: + continue + for row in conn.execute(f"SELECT {primary_key}, {', '.join(selected_columns)} FROM {table}").fetchall(): + updates: dict[str, str | None] = {} + for column in selected_columns: + raw = row[column] + if not raw: + continue + try: + parsed = json.loads(raw) + except (TypeError, json.JSONDecodeError): + continue + cleaned = _strip_sensitive_records( + parsed, + include_prompt_records=include_prompt_records, + include_moderation_records=include_moderation_records, + ) + updates[column] = json.dumps(cleaned, ensure_ascii=False, separators=(",", ":")) + if updates: + assignments = ", ".join(f"{column} = ?" for column in updates) + conn.execute(f"UPDATE {table} SET {assignments} WHERE {primary_key} = ?", (*updates.values(), row[primary_key])) + conn.commit() + finally: + conn.close() + + +def _strip_sensitive_records( + value: Any, + *, + include_prompt_records: bool, + include_moderation_records: bool, +) -> Any: + prompt_keys = {"prompt", "positive_prompt", "negative_prompt", "expanded_prompt", "expanded_negative_prompt", "original_text", "translated_text", "normalized_text", "source_payload"} + moderation_keys = {"prompt_governance", "moderation", "moderation_recheck", "image_moderation", "route_fallback", "generation_route_decision"} + if isinstance(value, dict): + result: dict[str, Any] = {} + for key, item in value.items(): + lowered = str(key).lower() + if not include_prompt_records and lowered in prompt_keys: + result[key] = "[EXCLUDED_FROM_BACKUP]" + continue + if not include_moderation_records and lowered in moderation_keys: + continue + result[key] = _strip_sensitive_records(item, include_prompt_records=include_prompt_records, include_moderation_records=include_moderation_records) + return result + if isinstance(value, list): + return [_strip_sensitive_records(item, include_prompt_records=include_prompt_records, include_moderation_records=include_moderation_records) for item in value] + return value diff --git a/backend/services/generation_job_retry_service.py b/backend/services/generation_job_retry_service.py new file mode 100644 index 0000000..c8ec3af --- /dev/null +++ b/backend/services/generation_job_retry_service.py @@ -0,0 +1,433 @@ +"""Synchronous, lineage-preserving retries for persisted generation jobs.""" + +from __future__ import annotations + +from datetime import datetime, timezone +from typing import Any +from uuid import uuid4 + +from backend.dto.common import ApiError, ErrorCode +from backend.dto.job_dto import GenerationTask +from backend.services.generation_service import GenerationService +from backend.services.queue_service import QueueService +from core.identity import RequestContext + + +MAX_SYNCHRONOUS_DEAD_LETTER_RETRIES = 5 + + +class GenerationJobRetryService: + """Create and execute a replacement task from a persisted task request. + + ImageStudio intentionally does not start QueueWorker in the AstrBot plugin + process: no production worker handler or lifecycle owner exists yet. A web + retry therefore has to execute through GenerationService instead of merely + changing the old row back to ``queued``. + """ + + def __init__(self, queue_service: QueueService, generation_service: GenerationService): + self.queue_service = queue_service + self.generation_service = generation_service + + def retry_job( + self, + task_id: str, + ctx: RequestContext, + *, + overrides: dict[str, Any] | None = None, + ) -> GenerationTask: + original = self.queue_service.get_job(task_id) + self._validate_executable(original) + request_overrides = overrides or {} + self._validate_ephemeral_references(original, request_overrides) + operation_id = f"retry_{uuid4().hex}" + retry_key = f"{operation_id}:{original.task_id}" + claimed = self.queue_service.begin_retry(task_id, ctx, operation_id=operation_id) + previous_status = str(claimed.metadata["retry_execution"]["previous_status"]) + request = self._replacement_request( + claimed, + operation_id=operation_id, + retry_key=retry_key, + overrides=request_overrides, + ) + prepared_node = None + + try: + prepared_node = self._prepare_node(claimed, ctx) + response = self.generation_service.generate_node(str(claimed.node_id), request, ctx) + replacement = self.queue_service.get_job(str(response["task_id"])) + except Exception as exc: + replacement = self.queue_service.get_job_by_idempotency_key(retry_key) + node_status_restored = replacement is not None + if replacement is None: + try: + node_status_restored = self._restore_prepared_node(prepared_node, claimed.task_id, ctx) + except Exception: + node_status_restored = False + self._restore_original(claimed, previous_status=previous_status, exc=exc, ctx=ctx) + else: + replacement = self._finalize_lineage(claimed, replacement, operation_id=operation_id, ctx=ctx) + raise self._retry_error( + exc, + original=claimed, + replacement=replacement, + operation_id=operation_id, + original_status_restored=replacement is None, + node_status_restored=node_status_restored, + ) from exc + + return self._finalize_lineage(claimed, replacement, operation_id=operation_id, ctx=ctx) + + def retry_dead_letters(self, ctx: RequestContext, *, limit: int = 5) -> dict[str, Any]: + applied_limit = _synchronous_batch_limit(limit) + jobs = self.queue_service.list_jobs( + status="dead_letter", + page=1, + page_size=applied_limit, + ).items + retried: list[dict[str, Any]] = [] + errors: list[dict[str, Any]] = [] + for job in jobs: + try: + retried.append(self.retry_job(job.task_id, ctx).to_dict()) + except ApiError as exc: + details = dict(exc.details or {}) + errors.append( + { + "task_id": job.task_id, + "error_code": exc.error_code.value, + "message": exc.message, + "replacement_task_id": details.get("replacement_task_id"), + "retry_lineage": details.get("retry_lineage"), + } + ) + return { + "execution_mode": "synchronous", + "requested_limit": _int_or_default(limit, 5), + "applied_limit": applied_limit, + "max_limit": MAX_SYNCHRONOUS_DEAD_LETTER_RETRIES, + "retried": retried, + "errors": errors, + "total": len(retried), + "processed": len(retried) + len(errors), + "remaining": self.queue_service.queue_summary()["dead_letter"], + } + + def _validate_executable(self, task: GenerationTask) -> None: + if task.task_type != "generation" or not task.node_id or not isinstance(task.request, dict): + raise ApiError( + ErrorCode.TASK_NOT_RETRYABLE, + "task has no executable generation request", + { + "task_id": task.task_id, + "task_type": task.task_type, + "node_id": task.node_id, + "request_type": type(task.request).__name__, + }, + ) + + def _validate_ephemeral_references(self, task: GenerationTask, overrides: dict[str, Any]) -> None: + omitted = _ephemeral_reference_count(task.request) + if omitted and not _has_reference_override(overrides): + raise ApiError( + ErrorCode.TASK_NOT_RETRYABLE, + "task used inline reference images that were not persisted; provide replacement references", + { + "task_id": task.task_id, + "ephemeral_data_urls_omitted": omitted, + "required_fields": ["input_images", "image", "selected_draft_image_ids", "stage_inputs"], + }, + ) + + def _prepare_node(self, task: GenerationTask, ctx: RequestContext) -> Any | None: + node = self.generation_service.workflow_service.get_node(str(task.node_id)) + if node.status not in {"queued", "running"}: + return None + last_task_id = _optional_text(node.metadata.get("last_task_id")) + if last_task_id != task.task_id: + raise ApiError( + ErrorCode.TASK_NOT_RETRYABLE, + "node is already owned by another active generation task", + { + "task_id": task.task_id, + "node_id": node.node_id, + "active_task_id": last_task_id, + "node_status": node.status, + }, + ) + self.generation_service.workflow_service.update_node( + node.node_id, + { + "status": "failed", + "metadata": { + **node.metadata, + "retry_preparation": { + "task_id": task.task_id, + "prepared_at": _now(), + "previous_status": node.status, + }, + }, + }, + ctx, + ) + return node + + def _restore_prepared_node(self, original_node: Any | None, task_id: str, ctx: RequestContext) -> bool: + if original_node is None: + return True + workflow_service = self.generation_service.workflow_service + current = workflow_service.get_node(original_node.node_id) + preparation = current.metadata.get("retry_preparation") if isinstance(current.metadata, dict) else None + if ( + current.status != "failed" + or not isinstance(preparation, dict) + or _optional_text(preparation.get("task_id")) != task_id + ): + return False + after = workflow_service.workflow_repository.update_node( + original_node.node_id, + {"status": original_node.status, "metadata": original_node.metadata}, + ) + if after is None: + return False + workflow_service.audit_service.record( + ctx=ctx, + action="rollback_retry_preparation", + target_type="workflow_node", + target_id=original_node.node_id, + before_snapshot=current.to_dict(), + after_snapshot=after.to_dict(), + metadata={"task_id": task_id}, + ) + workflow_service._publish( + "workflow.node_status_changed", + { + "workflow_id": after.workflow_id, + "node_id": after.node_id, + "before_status": current.status, + "after_status": after.status, + }, + ) + workflow_service._publish( + "workflow.node_updated", + {"workflow_id": after.workflow_id, "node_id": after.node_id}, + ) + return True + + def _replacement_request( + self, + task: GenerationTask, + *, + operation_id: str, + retry_key: str, + overrides: dict[str, Any], + ) -> dict[str, Any]: + request = { + key: value + for key, value in dict(task.request or {}).items() + if key not in _DERIVED_REQUEST_FIELDS and key != "idempotency_key" + } + for key, value in overrides.items(): + if key not in _DERIVED_REQUEST_FIELDS and key != "idempotency_key": + request[key] = value + request.update( + { + "idempotency_key": retry_key, + "retry_of_task_id": task.task_id, + "retry_operation_id": operation_id, + } + ) + if task.model_route and not _optional_text(request.get("model_route")): + request["model_route"] = task.model_route + return request + + def _finalize_lineage( + self, + original: GenerationTask, + replacement: GenerationTask, + *, + operation_id: str, + ctx: RequestContext, + ) -> GenerationTask: + previous_lineage = original.metadata.get("retry_lineage") + root_task_id = ( + _optional_text(previous_lineage.get("root_task_id")) + if isinstance(previous_lineage, dict) + else None + ) or original.task_id + lineage = { + "operation_id": operation_id, + "retry_of_task_id": original.task_id, + "root_task_id": root_task_id, + "original_status": original.metadata["retry_execution"]["previous_status"], + "replacement_task_id": replacement.task_id, + "replacement_status": replacement.status, + "completed_at": _now(), + } + updated_replacement = self.queue_service.update_task( + replacement.task_id, + {"metadata": {**replacement.metadata, "retry_lineage": lineage}}, + ctx, + ) + latest_original = self.queue_service.get_job(original.task_id) + retry_execution = dict(latest_original.metadata.get("retry_execution") or {}) + self.queue_service.update_task( + original.task_id, + { + "status": "superseded", + "progress": 100, + "result": { + **latest_original.result, + "replacement_task_id": updated_replacement.task_id, + }, + "metadata": { + **latest_original.metadata, + "retry_execution": { + **retry_execution, + "status": "completed", + "completed_at": lineage["completed_at"], + "replacement_task_id": updated_replacement.task_id, + "replacement_status": updated_replacement.status, + }, + }, + }, + ctx, + ) + return updated_replacement + + def _restore_original( + self, + original: GenerationTask, + *, + previous_status: str, + exc: Exception, + ctx: RequestContext, + ) -> None: + latest = self.queue_service.get_job(original.task_id) + if latest.status != "retrying": + return + retry_execution = dict(latest.metadata.get("retry_execution") or {}) + self.queue_service.update_task( + original.task_id, + { + "status": previous_status, + "metadata": { + **latest.metadata, + "retry_execution": { + **retry_execution, + "status": "rejected", + "rejected_at": _now(), + "error_type": type(exc).__name__, + "error_message": _safe_error_message(exc), + }, + }, + }, + ctx, + ) + + def _retry_error( + self, + exc: Exception, + *, + original: GenerationTask, + replacement: GenerationTask | None, + operation_id: str, + original_status_restored: bool, + node_status_restored: bool, + ) -> ApiError: + replacement_task_id = replacement.task_id if replacement else None + lineage = { + "operation_id": operation_id, + "retry_of_task_id": original.task_id, + "replacement_task_id": replacement_task_id, + "replacement_status": replacement.status if replacement else None, + } + details = { + **(dict(exc.details or {}) if isinstance(exc, ApiError) else {}), + "retry_of_task_id": original.task_id, + "replacement_task_id": replacement_task_id, + "original_status_restored": original_status_restored, + "node_status_restored": node_status_restored, + "retry_lineage": lineage, + } + if isinstance(exc, ApiError): + return ApiError(exc.error_code, exc.message, details) + return ApiError(ErrorCode.GENERATION_FAILED, "task retry execution failed", details) + + +def _synchronous_batch_limit(value: Any) -> int: + return min(max(_int_or_default(value, 5), 1), MAX_SYNCHRONOUS_DEAD_LETTER_RETRIES) + + +def _int_or_default(value: Any, default: int) -> int: + try: + return int(value) + except (TypeError, ValueError): + return default + + +def _optional_text(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + + +def _safe_error_message(exc: Exception) -> str: + text = " ".join(str(exc or type(exc).__name__).split()) + return text[:500] if text else type(exc).__name__ + + +def _now() -> str: + return datetime.now(timezone.utc).isoformat() + + +_DERIVED_REQUEST_FIELDS = frozenset( + { + "cost_guard", + "expanded_negative_prompt", + "expanded_prompt", + "prompt_expansion", + "reference_persistence", + "resolved_model_route", + } +) + + +def _ephemeral_reference_count(request: dict[str, Any]) -> int: + persistence = request.get("reference_persistence") + if not isinstance(persistence, dict): + return 0 + try: + return max(int(persistence.get("ephemeral_data_urls_omitted") or 0), 0) + except (TypeError, ValueError): + return 0 + + +def _has_reference_override(payload: dict[str, Any]) -> bool: + for key in ("input_images", "image", "selected_draft_image_ids", "stage_inputs"): + if _has_nonempty_value(payload.get(key)): + return True + metadata = payload.get("metadata") if isinstance(payload.get("metadata"), dict) else {} + dependency = payload.get("dependency_context") if isinstance(payload.get("dependency_context"), dict) else {} + return any( + _has_nonempty_value(value) + for value in ( + metadata.get("selected_draft_image_ids"), + metadata.get("stage_inputs"), + dependency.get("selected_image_ids"), + dependency.get("selected_draft_image_ids"), + dependency.get("stage_inputs"), + dependency.get("material_bindings"), + ) + ) + + +def _has_nonempty_value(value: Any) -> bool: + if value is None: + return False + if isinstance(value, str): + return bool(value.strip()) + if isinstance(value, (list, tuple, dict, set)): + return bool(value) + return True diff --git a/backend/services/generation_reference_service.py b/backend/services/generation_reference_service.py new file mode 100644 index 0000000..adb3c1c --- /dev/null +++ b/backend/services/generation_reference_service.py @@ -0,0 +1,470 @@ +"""Resolve generation reference images into provider-ready inputs.""" + +from __future__ import annotations + +import base64 +import binascii +import re +from typing import Any, Protocol +from urllib.parse import urlsplit + +from backend.dto.common import ApiError, ErrorCode +from backend.services.image_service import ImageService +from core.storage import VfsStorage + + +MAX_REFERENCE_IMAGE_BYTES = 25 * 1024 * 1024 +MAX_REFERENCE_IMAGE_TOTAL_BYTES = 50 * 1024 * 1024 +MAX_REFERENCE_IMAGES = 16 +_SUPPORTED_IMAGE_MIMES = {"image/png", "image/jpeg", "image/gif", "image/webp"} +_DATA_IMAGE_RE = re.compile(r"^data:([^;,]+)(?:;[^,]*)?;base64,(.*)$", re.IGNORECASE | re.DOTALL) + + +class MaterialResolver(Protocol): + def get_material(self, material_id: str) -> Any | None: ... + + +class GenerationReferenceService: + def __init__( + self, + image_service: ImageService, + storage: VfsStorage, + material_resolver: MaterialResolver | None = None, + ): + self.image_service = image_service + self.storage = storage + self.material_resolver = material_resolver + + def resolve(self, workflow_id: str, payload: dict[str, Any]) -> tuple[str, ...]: + resolved: list[str] = [] + seen: set[str] = set() + image_cache: dict[str, Any] = {} + storage_cache: dict[str, tuple[str, int]] = {} + total_bytes = 0 + for value, source, force_image_id in _reference_candidates(payload): + prepared, canonical_key, byte_count = self._resolve_candidate( + workflow_id, + value, + source=source, + force_image_id=force_image_id, + image_cache=image_cache, + storage_cache=storage_cache, + ) + total_bytes = _append_reference( + resolved, + seen, + prepared, + canonical_key, + byte_count, + total_bytes, + source, + ) + if self.material_resolver is not None: + for material_id, source in _material_reference_candidates(payload): + prepared, canonical_key, byte_count = self._resolve_material( + material_id, + source=source, + storage_cache=storage_cache, + ) + total_bytes = _append_reference( + resolved, + seen, + prepared, + canonical_key, + byte_count, + total_bytes, + source, + ) + return tuple(resolved) + + def _resolve_material( + self, + material_id: str, + *, + source: str, + storage_cache: dict[str, tuple[str, int]], + ) -> tuple[str, str, int]: + material = self.material_resolver.get_material(material_id) if self.material_resolver else None + if material is None: + raise ApiError( + ErrorCode.RESOURCE_NOT_FOUND, + "reference material does not exist", + {"material_id": material_id, "source": source}, + ) + if str(material.recognition_status or "").strip() != "confirmed": + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference material is not confirmed", + { + "material_id": material_id, + "recognition_status": material.recognition_status, + "source": source, + }, + ) + image_uri = _optional_text(material.image_uri) + if not image_uri: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "confirmed reference material has no image", + {"material_id": material_id, "source": source}, + ) + return self._prepare_uri(image_uri, source=source, storage_cache=storage_cache) + + def _resolve_candidate( + self, + workflow_id: str, + value: Any, + *, + source: str, + force_image_id: bool, + image_cache: dict[str, Any], + storage_cache: dict[str, tuple[str, int]], + ) -> tuple[str, str, int]: + if isinstance(value, dict): + image_id = _optional_text(value.get("image_id")) + if image_id: + return self._resolve_image_id( + workflow_id, + image_id, + source=source, + image_cache=image_cache, + storage_cache=storage_cache, + ) + for key in ("image_uri", "url", "data_url", "input_image"): + reference = _optional_text(value.get(key)) + if reference: + return self._prepare_uri(reference, source=source, storage_cache=storage_cache) + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference image object must include image_id or image_uri", + {"source": source}, + ) + + reference = _optional_text(value) + if not reference: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference image cannot be empty", + {"source": source}, + ) + if force_image_id or not _is_reference_uri(reference): + return self._resolve_image_id( + workflow_id, + reference, + source=source, + image_cache=image_cache, + storage_cache=storage_cache, + ) + return self._prepare_uri(reference, source=source, storage_cache=storage_cache) + + def _resolve_image_id( + self, + workflow_id: str, + image_id: str, + *, + source: str, + image_cache: dict[str, Any], + storage_cache: dict[str, tuple[str, int]], + ) -> tuple[str, str, int]: + image = image_cache.get(image_id) + if image is None: + try: + image = self.image_service.get_image(image_id) + except ApiError as exc: + if exc.error_code != ErrorCode.RESOURCE_NOT_FOUND: + raise + raise ApiError( + ErrorCode.RESOURCE_NOT_FOUND, + "reference image does not exist", + {"image_id": image_id, "source": source}, + ) from exc + image_cache[image_id] = image + if image.workflow_id != workflow_id: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference image does not belong to this workflow", + { + "image_id": image_id, + "workflow_id": workflow_id, + "image_workflow_id": image.workflow_id, + "source": source, + }, + ) + return self._prepare_uri(image.image_uri, source=source, storage_cache=storage_cache) + + def _prepare_uri( + self, + reference: str, + *, + source: str, + storage_cache: dict[str, tuple[str, int]], + ) -> tuple[str, str, int]: + if reference.startswith("storage://"): + cached = storage_cache.get(reference) + if cached is None: + try: + path = self.storage.ensure_file(reference) + size = path.stat().st_size + if size > MAX_REFERENCE_IMAGE_BYTES: + raise _image_too_large(reference, size, source) + body = path.read_bytes() + except ApiError as exc: + if exc.error_code != ErrorCode.MEDIA_NOT_FOUND: + raise + raise ApiError( + ErrorCode.MEDIA_NOT_FOUND, + "reference image file does not exist", + {"uri": reference, "source": source}, + ) from exc + except OSError as exc: + raise ApiError( + ErrorCode.MEDIA_NOT_FOUND, + "reference image file could not be read", + {"uri": reference, "source": source}, + ) from exc + if len(body) > MAX_REFERENCE_IMAGE_BYTES: + raise _image_too_large(reference, len(body), source) + mime = _detect_image_mime(body) + if mime is None: + raise ApiError( + ErrorCode.UNSUPPORTED_FILE_TYPE, + "reference image is not a supported PNG, JPEG, GIF, or WebP image", + {"uri": reference, "source": source}, + ) + cached = (f"data:{mime};base64,{base64.b64encode(body).decode('ascii')}", len(body)) + storage_cache[reference] = cached + return cached[0], f"storage:{reference}", cached[1] + + if reference.lower().startswith("data:"): + byte_count = _validate_data_image_url(reference, source=source) + return reference, f"data:{reference}", byte_count + + parsed = urlsplit(reference) + if parsed.scheme.lower() in {"http", "https"} and parsed.hostname: + return reference, f"url:{reference}", 0 + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference image must be an image ID, storage URI, HTTP(S) URL, or image data URL", + {"reference": reference, "source": source}, + ) + + +def _reference_candidates(payload: dict[str, Any]) -> list[tuple[Any, str, bool]]: + candidates: list[tuple[Any, str, bool]] = [] + for key in ("input_images", "image"): + for index, value in enumerate(_ordered_values(payload.get(key))): + candidates.append((value, f"{key}[{index}]", False)) + + dependency_context = payload.get("dependency_context") + if not isinstance(dependency_context, dict): + dependency_context = {} + for index, value in enumerate(_ordered_values(dependency_context.get("selected_image_ids"))): + candidates.append((value, f"dependency_context.selected_image_ids[{index}]", True)) + + metadata = payload.get("metadata") if isinstance(payload.get("metadata"), dict) else {} + for prefix, value in ( + ("selected_draft_image_ids", payload.get("selected_draft_image_ids")), + ("metadata.selected_draft_image_ids", metadata.get("selected_draft_image_ids")), + ("dependency_context.selected_draft_image_ids", dependency_context.get("selected_draft_image_ids")), + ): + for index, image_id in enumerate(_ordered_values(value)): + candidates.append((image_id, f"{prefix}[{index}]", True)) + + injection_template = ( + dependency_context.get("injection_template") + if isinstance(dependency_context.get("injection_template"), dict) + else {} + ) + for index, image_id in enumerate(_ordered_values(injection_template.get("selected_draft_image_ids"))): + candidates.append( + ( + image_id, + f"dependency_context.injection_template.selected_draft_image_ids[{index}]", + True, + ) + ) + + for prefix, value in ( + ("stage_inputs", payload.get("stage_inputs")), + ("metadata.stage_inputs", metadata.get("stage_inputs")), + ("dependency_context.stage_inputs", dependency_context.get("stage_inputs")), + ("dependency_context.injection_template.stage_inputs", injection_template.get("stage_inputs")), + ): + for index, reference in enumerate(_stage_input_references(value)): + candidates.append((reference, f"{prefix}[{index}]", False)) + return candidates + + +def _material_reference_candidates(payload: dict[str, Any]) -> list[tuple[str, str]]: + dependency_context = payload.get("dependency_context") + if not isinstance(dependency_context, dict): + return [] + bindings = dependency_context.get("material_bindings") + if not isinstance(bindings, dict): + return [] + + candidates: list[tuple[str, str]] = [] + generation_params = bindings.get("generation_params") + if isinstance(generation_params, dict): + for index, material_id in enumerate(_ordered_values(generation_params.get("reference_material_ids"))): + text = _optional_text(material_id) + if text: + candidates.append( + (text, f"dependency_context.material_bindings.generation_params.reference_material_ids[{index}]") + ) + active_bindings = bindings.get("active_bindings") + if isinstance(active_bindings, list): + for index, binding in enumerate(active_bindings): + if not isinstance(binding, dict): + continue + material = binding.get("material") if isinstance(binding.get("material"), dict) else {} + material_id = _optional_text(material.get("material_id") or binding.get("material_id")) + if material_id: + candidates.append( + (material_id, f"dependency_context.material_bindings.active_bindings[{index}].material_id") + ) + + result: list[tuple[str, str]] = [] + seen: set[str] = set() + for material_id, source in candidates: + if material_id not in seen: + result.append((material_id, source)) + seen.add(material_id) + return result + + +def _append_reference( + resolved: list[str], + seen: set[str], + prepared: str, + canonical_key: str, + byte_count: int, + total_bytes: int, + source: str, +) -> int: + if canonical_key in seen: + return total_bytes + if len(resolved) >= MAX_REFERENCE_IMAGES: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "too many reference images", + {"max_reference_images": MAX_REFERENCE_IMAGES, "source": source}, + ) + if total_bytes + byte_count > MAX_REFERENCE_IMAGE_TOTAL_BYTES: + raise ApiError( + ErrorCode.UPLOAD_TOO_LARGE, + "reference images exceed the total size limit", + {"max_total_bytes": MAX_REFERENCE_IMAGE_TOTAL_BYTES, "source": source}, + ) + resolved.append(prepared) + seen.add(canonical_key) + return total_bytes + byte_count + + +def _ordered_values(value: Any) -> list[Any]: + values = list(value) if isinstance(value, (list, tuple)) else ([] if value is None else [value]) + return [item for item in values if not isinstance(item, str) or item.strip()] + + +def _stage_input_references(value: Any) -> list[Any]: + if value is None: + return [] + if isinstance(value, str): + return [value] if value.strip() else [] + if isinstance(value, (list, tuple)): + return [reference for item in value for reference in _stage_input_references(item)] + if not isinstance(value, dict): + return [] + if any(_optional_text(value.get(key)) for key in ("image_id", "image_uri", "url", "data_url")): + return [value] + return [ + reference + for item in value.values() + if isinstance(item, (dict, list, tuple)) + for reference in _stage_input_references(item) + ] + + +def _validate_data_image_url(value: str, *, source: str) -> int: + match = _DATA_IMAGE_RE.fullmatch(value) + if match is None: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference data URL must contain a base64-encoded image", + {"source": source}, + ) + declared_mime = match.group(1).strip().lower() + if declared_mime == "image/jpg": + declared_mime = "image/jpeg" + if declared_mime not in _SUPPORTED_IMAGE_MIMES: + raise ApiError( + ErrorCode.UNSUPPORTED_FILE_TYPE, + "reference data URL uses an unsupported image type", + {"mime": declared_mime, "source": source}, + ) + encoded = re.sub(r"\s+", "", match.group(2)) + if len(encoded) > ((MAX_REFERENCE_IMAGE_BYTES + 2) // 3) * 4: + raise ApiError( + ErrorCode.UPLOAD_TOO_LARGE, + "reference data URL exceeds the size limit", + {"max_bytes": MAX_REFERENCE_IMAGE_BYTES, "source": source}, + ) + try: + body = base64.b64decode(encoded, validate=True) + except (binascii.Error, ValueError) as exc: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference data URL contains invalid base64", + {"source": source}, + ) from exc + if len(body) > MAX_REFERENCE_IMAGE_BYTES: + raise ApiError( + ErrorCode.UPLOAD_TOO_LARGE, + "reference data URL exceeds the size limit", + {"bytes": len(body), "max_bytes": MAX_REFERENCE_IMAGE_BYTES, "source": source}, + ) + detected_mime = _detect_image_mime(body) + if detected_mime is None: + raise ApiError( + ErrorCode.UNSUPPORTED_FILE_TYPE, + "reference data URL does not contain a supported image", + {"source": source}, + ) + if detected_mime != declared_mime: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "reference data URL MIME type does not match its image payload", + {"declared_mime": declared_mime, "detected_mime": detected_mime, "source": source}, + ) + return len(body) + + +def _detect_image_mime(body: bytes) -> str | None: + if body.startswith(b"\x89PNG\r\n\x1a\n"): + return "image/png" + if body.startswith(b"\xff\xd8\xff"): + return "image/jpeg" + if body.startswith((b"GIF87a", b"GIF89a")): + return "image/gif" + if len(body) >= 12 and body.startswith(b"RIFF") and body[8:12] == b"WEBP": + return "image/webp" + return None + + +def _image_too_large(uri: str, size: int, source: str) -> ApiError: + return ApiError( + ErrorCode.UPLOAD_TOO_LARGE, + "reference image exceeds the size limit", + {"uri": uri, "bytes": size, "max_bytes": MAX_REFERENCE_IMAGE_BYTES, "source": source}, + ) + + +def _is_reference_uri(value: str) -> bool: + return value.lower().startswith(("storage://", "data:", "http://", "https://")) + + +def _optional_text(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None diff --git a/backend/services/generation_service.py b/backend/services/generation_service.py index 96ab207..62c2569 100644 --- a/backend/services/generation_service.py +++ b/backend/services/generation_service.py @@ -2,21 +2,51 @@ from __future__ import annotations +import threading from typing import Any from backend.dto.common import ApiError, ErrorCode from backend.dto.job_dto import GenerationTask from backend.services.cost_guard_service import CostGuardService +from backend.services.composition_builder import CompositionBuilder +from backend.services.generation_reference_service import GenerationReferenceService, MaterialResolver from backend.services.image_service import ImageService from backend.services.model_provider_service import ModelProviderService +from backend.services.node_dependency_context_service import NodeDependencyContextService from backend.services.prompt_expansion_service import PromptExpansionService +from backend.services.prompt_governance_service import ( + GenerationFailureClassifier, + GenerationRouteService, + ImageModerationService, + PromptModerationService, + PromptSemanticNormalizer, + PromptTranslationService, +) from backend.services.queue_service import QueueService -from backend.services.workflow_service import WorkflowService +from backend.services.reference_binding_mapper import ReferenceBindingMapper +from backend.services.workflow_service import ( + LOCKED_NODE_STATUSES, + WorkflowService, + is_repeatable_single_image_node, + locked_node_edit_error, +) from core.event_bus import EventBus +from core.generation_errors import provider_failure_attribution from core.identity import RequestContext +from engines.image_provider import ImageProviderError from engines.single_flow import SingleFlowEngine +_NODE_GENERATION_LOCKS_GUARD = threading.Lock() +_NODE_GENERATION_LOCKS: dict[str, "_NodeGenerationLock"] = {} + + +class _NodeGenerationLock: + def __init__(self): + self.lock = threading.RLock() + self.users = 0 + + class GenerationService: def __init__( self, @@ -28,38 +58,56 @@ class GenerationService: cost_guard_service: CostGuardService | None = None, prompt_expansion_service: PromptExpansionService | None = None, event_bus: EventBus | None = None, + default_model_route: str | None = None, + require_resolved_model_route: bool = False, + material_resolver: MaterialResolver | None = None, + model_assignments: dict[str, Any] | None = None, + prompt_translation_service: PromptTranslationService | None = None, + prompt_semantic_normalizer: PromptSemanticNormalizer | None = None, + prompt_moderation_service: PromptModerationService | None = None, + generation_route_service: GenerationRouteService | None = None, + generation_failure_classifier: GenerationFailureClassifier | None = None, + image_moderation_service: ImageModerationService | None = None, + reference_binding_mapper: ReferenceBindingMapper | None = None, + composition_builder: CompositionBuilder | None = None, ): self.workflow_service = workflow_service self.queue_service = queue_service self.image_service = image_service self.single_flow_engine = single_flow_engine or SingleFlowEngine() + self.reference_service = GenerationReferenceService( + image_service, + self.single_flow_engine.storage, + material_resolver, + ) self.model_provider_service = model_provider_service self.cost_guard_service = cost_guard_service self.prompt_expansion_service = prompt_expansion_service self.event_bus = event_bus + self.default_model_route = _optional_text(default_model_route) + self.model_assignments = dict(model_assignments or {}) + self.require_resolved_model_route = require_resolved_model_route + self.prompt_translation_service = prompt_translation_service + self.prompt_semantic_normalizer = prompt_semantic_normalizer + self.prompt_moderation_service = prompt_moderation_service + self.generation_route_service = generation_route_service + self.generation_failure_classifier = generation_failure_classifier or GenerationFailureClassifier() + self.image_moderation_service = image_moderation_service + self.reference_binding_mapper = reference_binding_mapper or ReferenceBindingMapper() + self.composition_builder = composition_builder or CompositionBuilder() + self.material_resolver = material_resolver def generate_node(self, node_id: str, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: + lock_entry = _retain_node_generation_lock(node_id) + lock_entry.lock.acquire() + try: + return self._generate_node_locked(node_id, payload, ctx) + finally: + lock_entry.lock.release() + _release_node_generation_lock(node_id, lock_entry) + + def _generate_node_locked(self, node_id: str, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: node = self.workflow_service.get_node(node_id) - workflow = self.workflow_service.get_workflow(node.workflow_id).workflow - model_route = str(payload.get("model_route") or node.model_route or "image_low") - route_resolution = self._resolve_model_route(model_route) - cost_guard = self._check_cost_budget(payload, model_route, route_resolution) - prompt_expansion = self._expand_prompt(workflow.profile_id, node, payload) - generation_request = { - **payload, - "model_route": model_route, - "resolved_model_route": route_resolution, - **({"cost_guard": cost_guard} if cost_guard else {}), - **( - { - "prompt_expansion": prompt_expansion, - "expanded_prompt": prompt_expansion["prompt"], - "expanded_negative_prompt": prompt_expansion["negative_prompt"], - } - if prompt_expansion - else {} - ), - } idempotency_key = _optional_text(payload.get("idempotency_key")) if idempotency_key: existing_task = self.queue_service.get_job_by_idempotency_key(idempotency_key) @@ -75,6 +123,99 @@ class GenerationService: }, ) return self._existing_generation_response(existing_task, node_id) + active_task = self._active_generation_task(node) + if node.status in {"queued", "running"} and active_task is not None: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "generation is already in progress for this node", + { + "node_id": node_id, + "status": node.status, + "last_task_id": active_task.task_id, + "task_status": active_task.status, + }, + ) + if node.status in LOCKED_NODE_STATUSES and not is_repeatable_single_image_node(node): + raise locked_node_edit_error(node) + workflow = self.workflow_service.get_workflow(node.workflow_id).workflow + if workflow.status == "cancelled": + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "generation cannot start for a cancelled workflow", + {"workflow_id": workflow.workflow_id, "status": workflow.status}, + ) + profile = self.workflow_service.profile_repository.get(workflow.profile_id) + profile_model_route = _optional_text(getattr(profile, "default_model_route", None)) + assigned_route = _optional_text(self.model_assignments.get(_assignment_key(node))) + requested_model_route = str( + payload.get("model_route") + or node.model_route + or assigned_route + or profile_model_route + or self.default_model_route + or "image_low" + ) + enriched_payload = self._enrich_generation_payload(node.node_id, payload) + prompt_expansion = self._expand_prompt(workflow.profile_id, node, enriched_payload) + governance = self._govern_prompt( + node=node, + payload=enriched_payload, + prompt_expansion=prompt_expansion, + requested_route=requested_model_route, + ctx=ctx, + ) + model_route = governance["route_decision"]["selected_route"] if governance else requested_model_route + if governance and governance.get("prompt"): + prompt_expansion = { + **(prompt_expansion or {}), + "prompt": governance["prompt"], + "negative_prompt": governance.get("negative_prompt") or (prompt_expansion or {}).get("negative_prompt", ""), + "governance": governance, + } + route_resolution = self._resolve_model_route(model_route) + if governance: + excluded_fallback = _optional_text(governance.get("route_decision", {}).get("fallback_route")) + if excluded_fallback and excluded_fallback != model_route: + route_resolution["excluded_fallback_routes"] = [excluded_fallback] + if self.require_resolved_model_route and route_resolution.get("status") != "resolved": + raise ApiError( + ErrorCode.MODEL_CAPABILITY_UNCONFIRMED, + "未找到可用的生图模型路由,请先在模型配置中完成提供商与路由设置", + {"model_route": model_route, "resolution": route_resolution}, + ) + reference_mapping = self.reference_binding_mapper.map(_reference_binding_summary(enriched_payload), route_resolution) + if reference_mapping["text_constraints"]: + base_prompt = _optional_text((prompt_expansion or {}).get("prompt")) or _optional_text(getattr(node, "prompt", None)) or "" + prompt_expansion = { + **(prompt_expansion or {}), + "prompt": ", ".join([base_prompt, *reference_mapping["text_constraints"]]), + "negative_prompt": _optional_text((prompt_expansion or {}).get("negative_prompt")) or "", + "reference_binding_mapping": reference_mapping, + } + cost_guard = self._check_cost_budget(payload, model_route, route_resolution) + reference_images = self.reference_service.resolve(workflow.workflow_id, enriched_payload) + runtime_generation_request = { + **payload, + "model_route": model_route, + "resolved_model_route": route_resolution, + **({"cost_guard": cost_guard} if cost_guard else {}), + **({"prompt_governance": governance} if governance else {}), + "reference_binding_mapping": reference_mapping, + **( + { + "prompt_expansion": prompt_expansion, + "expanded_prompt": prompt_expansion["prompt"], + "expanded_negative_prompt": prompt_expansion["negative_prompt"], + } + if prompt_expansion + else {} + ), + } + generation_request, ephemeral_reference_count = _strip_ephemeral_image_data_urls(runtime_generation_request) + if ephemeral_reference_count: + generation_request["reference_persistence"] = { + "ephemeral_data_urls_omitted": ephemeral_reference_count, + } task = self.queue_service.create_task( { "profile_id": workflow.profile_id, @@ -87,6 +228,7 @@ class GenerationService: "metadata": { "source": "nodes.generate", "model_route_resolution": route_resolution, + **({"prompt_governance": _governance_audit_summary(governance)} if governance else {}), }, }, ctx, @@ -95,29 +237,140 @@ class GenerationService: "generation.node_queued", {"workflow_id": workflow.workflow_id, "node_id": node_id, "task_id": task.task_id}, ) - self.workflow_service.update_node( - node_id, - { - "status": "queued", - "model_route": model_route, - "metadata": { - **node.metadata, - "last_task_id": task.task_id, - "last_generate_request": generation_request, - }, - }, + queued_metadata = { + **node.metadata, + "last_task_id": task.task_id, + "last_generate_request": generation_request, + } + execution_metadata = queued_metadata.get("execution") + if isinstance(execution_metadata, dict) and execution_metadata.get("prepared_only") is True: + queued_metadata["execution"] = { + **execution_metadata, + "prepared_only": False, + "task_id": task.task_id, + } + queued_node_payload = { + "status": "queued", + "model_route": model_route, + "metadata": queued_metadata, + } + try: + self._queue_generation_node(node, queued_node_payload, ctx) + except Exception: + if self._cancel_generation_if_requested(task.task_id, workflow.workflow_id, ctx): + return self._finalize_cancelled_generation( + task_id=task.task_id, + node_id=node_id, + workflow_id=workflow.workflow_id, + assets=[], + images=[], + ctx=ctx, + ) + try: + self.queue_service.update_task_if_status( + task.task_id, + {"queued"}, + { + "status": "failed", + "progress": 100, + "error_message": "node could not enter the generation queue", + }, + ctx, + ) + except Exception: + pass + raise + running_task = self.queue_service.update_task_if_status( + task.task_id, + {"queued"}, + {"status": "running", "progress": 20}, ctx, + require_workflow_not_cancelled=True, ) - self.queue_service.update_task(task.task_id, {"status": "running", "progress": 20}, ctx) + if running_task is None: + current_task = self.queue_service.get_job(task.task_id) + if self._cancel_generation_if_requested(task.task_id, workflow.workflow_id, ctx): + return self._finalize_cancelled_generation( + task_id=task.task_id, + node_id=node_id, + workflow_id=workflow.workflow_id, + assets=[], + images=[], + ctx=ctx, + ) + raise ApiError( + ErrorCode.GENERATION_FAILED, + "generation task changed before provider execution", + {"task_id": task.task_id, "status": current_task.status}, + ) + task = running_task running_node = self.workflow_service.update_node(node_id, {"status": "running"}, ctx) self._publish( "generation.node_running", {"workflow_id": workflow.workflow_id, "node_id": node_id, "task_id": task.task_id}, ) - assets = self.single_flow_engine.generate(running_node, task, generation_request) - images = [ - self.image_service.create_image( - { + if self._cancel_generation_if_requested(task.task_id, workflow.workflow_id, ctx): + return self._finalize_cancelled_generation( + task_id=task.task_id, + node_id=node_id, + workflow_id=workflow.workflow_id, + assets=[], + images=[], + ctx=ctx, + ) + assets: list[Any] = [] + images: list[Any] = [] + restricted_images: list[Any] = [] + try: + runtime_generation_request = { + **runtime_generation_request, + "input_images": reference_images, + } + runtime_generation_request.pop("image", None) + try: + assets = self.single_flow_engine.generate(running_node, task, runtime_generation_request) + except ImageProviderError as exc: + fallback = self._content_safety_fallback( + exc=exc, + runtime_request=runtime_generation_request, + governance=governance, + requested_route=requested_model_route, + ctx=ctx, + ) + if fallback is None: + raise + runtime_generation_request = fallback["request"] + route_resolution = fallback["route_resolution"] + model_route = fallback["model_route"] + assets = self.single_flow_engine.generate(running_node, task, runtime_generation_request) + if self._cancel_generation_if_requested(task.task_id, workflow.workflow_id, ctx): + return self._finalize_cancelled_generation( + task_id=task.task_id, + node_id=node_id, + workflow_id=workflow.workflow_id, + assets=assets, + images=images, + ctx=ctx, + ) + precomputed_moderation: dict[str, dict[str, Any]] = {} + if self.image_moderation_service: + for asset in assets: + preview_payload = { + "image_uri": asset.image_uri, + "thumbnail_uri": asset.thumbnail_uri, + "positive_prompt": asset.positive_prompt, + "negative_prompt": asset.negative_prompt, + "model_route": asset.model_route, + } + precomputed_moderation[asset.image_uri] = self._moderate_generated_image(preview_payload, governance, model_route) + if any(item.get("action") == "regenerate" for item in precomputed_moderation.values()) and not runtime_generation_request.get("post_moderation_regeneration"): + self._cleanup_generation_outputs(assets=assets, images=[]) + retry_seed = int(runtime_generation_request.get("seed") or 1000) + 1 + runtime_generation_request = {**runtime_generation_request, "seed": retry_seed, "post_moderation_regeneration": {"attempted": True, "reason": "image_moderation"}} + assets = self.single_flow_engine.generate(running_node, task, runtime_generation_request) + precomputed_moderation = {} + for asset in assets: + image_payload = { "profile_id": workflow.profile_id, "workflow_id": workflow.workflow_id, "node_id": node_id, @@ -130,15 +383,67 @@ class GenerationService: "seed": asset.seed, "width": asset.width, "height": asset.height, - "is_selected": index == 0, - "metadata": asset.metadata, - }, - ctx, + # Keep records unselected until the task completion CAS + # wins. A concurrent cancellation can then roll them + # back without disturbing an older selected image. + "is_selected": False, + "metadata": { + **asset.metadata, + **({"prompt_governance": _governance_audit_summary(governance)} if governance else {}), + "reference_binding_mapping": runtime_generation_request.get("reference_binding_mapping") or {}, + }, + } + moderation = precomputed_moderation.get(asset.image_uri) or self._moderate_generated_image(image_payload, governance, model_route) + self._apply_restoration_feedback(moderation) + image_payload["metadata"] = {**image_payload["metadata"], "image_moderation": moderation} + if moderation.get("action") in {"quarantine", "reject", "regenerate"}: + image_payload["metadata"]["quarantined"] = True + image_payload["metadata"]["gallery_visibility"] = "restricted" + created_image = self.image_service.create_image(image_payload, ctx) + if moderation.get("action") in {"quarantine", "reject", "regenerate"}: + restricted_images.append(created_image) + else: + images.append(created_image) + except Exception as exc: + if self._cancel_generation_if_requested(task.task_id, workflow.workflow_id, ctx): + return self._finalize_cancelled_generation( + task_id=task.task_id, + node_id=node_id, + workflow_id=workflow.workflow_id, + assets=assets, + images=images, + ctx=ctx, + ) + self._cleanup_generation_outputs(assets=assets, images=[*images, *restricted_images]) + failure = self._record_generation_failure( + task=task, + node_id=node_id, + workflow_id=workflow.workflow_id, + route_resolution=route_resolution, + exc=exc, + ctx=ctx, ) - for index, asset in enumerate(assets) - ] + if self._cancel_generation_if_requested(task.task_id, workflow.workflow_id, ctx): + return self._finalize_cancelled_generation( + task_id=task.task_id, + node_id=node_id, + workflow_id=workflow.workflow_id, + assets=assets, + images=images, + ctx=ctx, + ) + if isinstance(exc, ApiError): + raise + raise ApiError( + ErrorCode.GENERATION_FAILED, + "图像生成失败", + failure, + ) from exc provider_failure = _provider_failure_from_assets([asset.metadata for asset in assets]) result: dict[str, Any] = {"image_ids": [image.image_id for image in images]} + if restricted_images: + result["restricted_image_ids"] = [image.image_id for image in restricted_images] + result["image_moderation"] = "restricted" update_payload: dict[str, Any] = { "status": "succeeded", "progress": 100, @@ -147,11 +452,32 @@ class GenerationService: if provider_failure: result["provider_failure"] = provider_failure update_payload["metadata"] = {**task.metadata, "provider_failure": provider_failure} - completed_task = self.queue_service.update_task( + completed_task = self.queue_service.update_task_if_status( task.task_id, + {"running"}, update_payload, ctx, + require_workflow_not_cancelled=True, ) + if completed_task is None: + current_task = self.queue_service.get_job(task.task_id) + if self._cancel_generation_if_requested(task.task_id, workflow.workflow_id, ctx): + return self._finalize_cancelled_generation( + task_id=task.task_id, + node_id=node_id, + workflow_id=workflow.workflow_id, + assets=assets, + images=images, + ctx=ctx, + ) + self._cleanup_generation_outputs(assets=assets, images=[*images, *restricted_images]) + raise ApiError( + ErrorCode.GENERATION_FAILED, + "generation task changed before completion", + {"task_id": task.task_id, "status": current_task.status}, + ) + if images: + images[0] = self.image_service.select_image(images[0].image_id, ctx) completed_node = self.workflow_service.update_node(node_id, {"status": "completed"}, ctx) self._publish( "generation.node_completed", @@ -167,6 +493,7 @@ class GenerationService: "task_status": completed_task.status, "node_status": completed_node.status, "images": [image.to_dict() for image in images], + **({"restricted_count": len(restricted_images), "image_moderation": "restricted"} if restricted_images else {}), } def _existing_generation_response(self, task: GenerationTask, node_id: str) -> dict[str, Any]: @@ -186,6 +513,21 @@ class GenerationService: "idempotent_replay": True, } + def _active_generation_task(self, node: Any) -> GenerationTask | None: + metadata = node.metadata if isinstance(node.metadata, dict) else {} + task_id = _optional_text(metadata.get("last_task_id")) + if not task_id: + return None + try: + task = self.queue_service.get_job(task_id) + except ApiError as exc: + if exc.error_code == ErrorCode.RESOURCE_NOT_FOUND: + return None + raise + if task.node_id == node.node_id and task.status in {"queued", "running"}: + return task + return None + def _resolve_model_route(self, route_name: str) -> dict[str, Any]: if not self.model_provider_service: return {"route_name": route_name, "status": "unresolved"} @@ -202,6 +544,8 @@ class GenerationService: "model_id": resolved["model_id"], "modality": resolved["modality"], "priority": resolved["priority"], + "settings": dict(resolved.get("settings") or {}), + "capabilities": dict(provider.get("capabilities") or {}), } def _check_cost_budget( @@ -230,6 +574,56 @@ class GenerationService: raise ApiError(ErrorCode.VALIDATION_FAILED, "generation budget exceeded", guard) return guard + def _queue_generation_node( + self, + original_node: Any, + payload: dict[str, Any], + ctx: RequestContext, + ) -> Any: + current = self.workflow_service.get_node(original_node.node_id) + if current.status not in LOCKED_NODE_STATUSES: + return self.workflow_service.update_node(current.node_id, payload, ctx) + if not is_repeatable_single_image_node(current): + raise locked_node_edit_error(current) + + repeat_payload = dict(payload) + repeat_metadata = repeat_payload.get("metadata") if isinstance(repeat_payload.get("metadata"), dict) else {} + repeat_payload["metadata"] = { + **repeat_metadata, + "repeat_generation": { + "previous_status": current.status, + "previous_selected_image_id": current.selected_image_id, + "task_id": repeat_metadata.get("last_task_id"), + }, + } + after = self.workflow_service.workflow_repository.update_node(current.node_id, repeat_payload) + if after is None: + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "node not found", {"node_id": current.node_id}) + self.workflow_service.audit_service.record( + ctx=ctx, + action="update_node", + target_type="workflow_node", + target_id=current.node_id, + before_snapshot=current.to_dict(), + after_snapshot=after.to_dict(), + metadata={"source": "generation.repeatable_single_image"}, + ) + if self.workflow_service.event_bus: + self.workflow_service.event_bus.publish( + "workflow.node_status_changed", + { + "workflow_id": after.workflow_id, + "node_id": after.node_id, + "before_status": current.status, + "after_status": after.status, + }, + ) + self.workflow_service.event_bus.publish( + "workflow.node_updated", + {"workflow_id": after.workflow_id, "node_id": after.node_id}, + ) + return after + def _expand_prompt( self, profile_id: str, @@ -243,10 +637,493 @@ class GenerationService: return None return self.prompt_expansion_service.expand_for_node(profile, node, payload) + def _govern_prompt( + self, + *, + node: Any, + payload: dict[str, Any], + prompt_expansion: dict[str, Any] | None, + requested_route: str, + ctx: RequestContext, + ) -> dict[str, Any] | None: + if not any((self.prompt_translation_service, self.prompt_semantic_normalizer, self.prompt_moderation_service, self.generation_route_service)): + return None + original = _optional_text(payload.get("user_prompt") or payload.get("prompt") or payload.get("prompt_patch")) or _optional_text(getattr(node, "prompt", None)) or "" + expanded = _optional_text((prompt_expansion or {}).get("prompt")) or original + negative = _optional_text((prompt_expansion or {}).get("negative_prompt")) or _optional_text(getattr(node, "negative_prompt", None)) or "" + entrypoint = _entrypoint(payload) + locked_terms = _string_list(payload.get("locked_terms")) + translation = None + translated = expanded + if self.prompt_translation_service: + translation_result = self.prompt_translation_service.translate( + expanded, + entrypoint=entrypoint, + user_allowed=bool(payload.get("translation_consent", False)), + model_family=str(payload.get("prompt_model_family") or "natural_language"), + locked_terms=locked_terms, + budget=_positive_int(payload.get("prompt_budget")), + ) + translation = translation_result.to_dict() + translated = translation_result.translated_text + normalization = None + final_prompt = translated + if self.prompt_semantic_normalizer: + normalization = self.prompt_semantic_normalizer.normalize(translated, locked_terms=locked_terms) + final_prompt = normalization["normalized_text"] + moderation = ( + self.prompt_moderation_service.moderate( + original_prompt=original, + translated_prompt=translated, + final_prompt=final_prompt, + negative_prompt=negative, + reference_bindings=_reference_binding_summary(payload), + ) + if self.prompt_moderation_service + else None + ) + if moderation is None: + from backend.services.prompt_governance_service import PromptModerationDecision + + moderation = PromptModerationDecision("safe", 1.0, [], ["moderation_not_configured"], "allow", None, "none", None, "") + if self.generation_route_service: + route = self.generation_route_service.decide( + requested_route, + moderation, + entrypoint=entrypoint, + user_id=ctx.actor_id, + group_id=_optional_text(payload.get("group_id") or payload.get("chat_group_id")), + permission_context=_permission_context(payload), + ) + else: + from backend.services.prompt_governance_service import GenerationRouteDecision + + route = GenerationRouteDecision(requested_route, moderation.level, requested_route if moderation.allowed else None, "static_route", None, moderation.allowed, moderation.level == "uncertain", "") + if not route.allowed: + if moderation.level in {"terror_prohibited", "violence_prohibited", "prohibited"}: + try: + self.workflow_service.audit_service.record( + ctx=ctx, + action="generation_content_safety_blocked", + target_type="workflow_node", + target_id=str(getattr(node, "node_id", "")), + metadata={ + "priority": "high" if moderation.level in {"terror_prohibited", "violence_prohibited"} else "normal", + "entrypoint": entrypoint, + "moderation_level": moderation.level, + "reason_codes": moderation.reason_codes, + "rules_version": moderation.rules_version, + "group_id": _optional_text(payload.get("group_id") or payload.get("chat_group_id")), + }, + ) + except Exception: + pass + raise ApiError( + ErrorCode.VALIDATION_FAILED, + moderation.warning or ("内容需要确认后才能生成" if route.requires_confirmation else "内容审核未通过,未创建生成任务"), + { + "moderation_level": moderation.level, + "reason_codes": moderation.reason_codes, + "route_reason": route.reason, + "requires_confirmation": route.requires_confirmation, + "warning": moderation.warning, + }, + ) + return { + "prompt": final_prompt, + "negative_prompt": negative, + "translation": translation, + "normalization": normalization, + "moderation": moderation.to_dict(), + "route_decision": route.to_dict(), + "entrypoint": entrypoint, + } + + def _content_safety_fallback( + self, + *, + exc: ImageProviderError, + runtime_request: dict[str, Any], + governance: dict[str, Any] | None, + requested_route: str, + ctx: RequestContext, + ) -> dict[str, Any] | None: + failure = self.generation_failure_classifier.classify(exc) + if failure["failure_class"] != "content_safety_rejection" or not self.prompt_moderation_service or not self.generation_route_service: + return None + if (runtime_request.get("route_fallback") or {}).get("attempted"): + return None + prompt = _optional_text(runtime_request.get("expanded_prompt")) or "" + negative = _optional_text(runtime_request.get("expanded_negative_prompt")) or "" + translation = (governance or {}).get("translation") + translation = translation if isinstance(translation, dict) else {} + moderation = self.prompt_moderation_service.moderate( + original_prompt=_optional_text(translation.get("original_text")) or prompt, + translated_prompt=_optional_text(translation.get("translated_text")) or prompt, + final_prompt=prompt, + negative_prompt=negative, + reference_bindings=[], + ) + payload = runtime_request + route = self.generation_route_service.decide( + requested_route, + moderation, + entrypoint=_entrypoint(payload), + user_id=ctx.actor_id, + group_id=_optional_text(payload.get("group_id") or payload.get("chat_group_id")), + permission_context=_permission_context(payload), + ) + if not route.allowed or moderation.level != "adult_allowed" or not route.selected_route or route.selected_route == runtime_request.get("model_route"): + return None + resolution = self._resolve_model_route(route.selected_route) + if self.require_resolved_model_route and resolution.get("status") != "resolved": + return None + fallback_cost_guard = self._check_cost_budget(runtime_request, route.selected_route, resolution) + fallback_record = { + "attempted": True, + "initial_route": runtime_request.get("model_route"), + "failure_class": failure, + "moderation_recheck": moderation.to_dict(), + "fallback_route": route.selected_route, + "fallback_reason": route.reason, + } + return { + "model_route": route.selected_route, + "route_resolution": resolution, + "request": { + **runtime_request, + "model_route": route.selected_route, + "resolved_model_route": resolution, + "route_fallback": fallback_record, + "allow_provider_fallback": False, + **({"cost_guard": fallback_cost_guard} if fallback_cost_guard else {}), + }, + } + + def _moderate_generated_image(self, image_payload: dict[str, Any], governance: dict[str, Any] | None, model_route: str) -> dict[str, Any]: + if not self.image_moderation_service: + return {"status": "not_configured", "action": "allow", "version": "none"} + level = _optional_text((governance or {}).get("moderation", {}).get("level")) or "safe" + route_kind = "nsfw" if level == "adult_allowed" else "safe" + return self.image_moderation_service.moderate(image_payload, requested_level=level, route_kind=route_kind) + + def _apply_restoration_feedback(self, moderation: dict[str, Any]) -> None: + if self.material_resolver is None or not hasattr(self.material_resolver, "update_material"): + return + scores = moderation.get("material_restoration_scores") + if not isinstance(scores, dict): + return + for material_id, raw_score in scores.items(): + try: + score = min(max(float(raw_score), 0.0), 1.0) + material = self.material_resolver.get_material(str(material_id)) + if material is None: + continue + old = material.metadata.get("restoration_feedback") if isinstance(material.metadata.get("restoration_feedback"), dict) else {} + count = int(old.get("count") or 0) + average = ((float(old.get("score") or 0.0) * count) + score) / (count + 1) + self.material_resolver.update_material(str(material_id), {"metadata": {**material.metadata, "restoration_feedback": {"score": round(average, 4), "count": count + 1}}}) + except Exception: + continue + + def _enrich_generation_payload(self, node_id: str, payload: dict[str, Any]) -> dict[str, Any]: + enriched_payload = dict(payload) + if not isinstance(enriched_payload.get("dependency_context"), dict): + enriched_payload["dependency_context"] = NodeDependencyContextService( + self.workflow_service, + self.image_service, + ).get_context(node_id) + self._compile_material_bindings(enriched_payload) + return enriched_payload + + def _compile_material_bindings(self, payload: dict[str, Any]) -> None: + if self.material_resolver is None: + return + dependency = payload.get("dependency_context") if isinstance(payload.get("dependency_context"), dict) else {} + bindings = dependency.get("material_bindings") if isinstance(dependency.get("material_bindings"), dict) else {} + active = bindings.get("active_bindings") if isinstance(bindings.get("active_bindings"), list) else [] + materials: list[dict[str, Any]] = [] + for binding in active: + if not isinstance(binding, dict): + continue + material_ref = binding.get("material") if isinstance(binding.get("material"), dict) else {} + material_id = _optional_text(material_ref.get("material_id") or binding.get("material_id")) + material = self.material_resolver.get_material(material_id) if material_id else None + if material is None or str(getattr(material, "recognition_status", "")) != "confirmed": + continue + strength = binding.get("reference_strength") if isinstance(binding.get("reference_strength"), dict) else {} + materials.append({ + "material_id": material.material_id, + "category": binding.get("category"), + "prompt": material.prompt or material_ref.get("prompt"), + "negative_prompt": material.negative_prompt, + "image_uri": material.image_uri, + "metadata": material.metadata, + "user_adjustment": 20.0 if binding.get("required") else 0.0, + "visibility_bonus": 10.0 if binding.get("required") else 0.0, + "reference_strength": strength.get("key"), + "restoration_score": ( + material.metadata.get("restoration_feedback", {}).get("score") + if isinstance(material.metadata.get("restoration_feedback"), dict) + else None + ), + }) + if not materials: + return + composition = self.composition_builder.build({ + "materials": materials, + "prompt_budget": payload.get("prompt_budget") or 4000, + "max_reference_images": payload.get("max_reference_images") or 8, + "visible_slots": payload.get("visible_slots") or [], + "season": payload.get("season") or "", + "scene": payload.get("scene") or "", + }).to_dict() + dependency["material_bindings"] = { + **bindings, + "composition": composition, + "material_prompts": [composition["prompt"]] if composition.get("prompt") else [], + "prompt_context": composition.get("prompt") or "", + "reference_bindings": composition.get("reference_bindings") or [], + "generation_params": { + **(bindings.get("generation_params") if isinstance(bindings.get("generation_params"), dict) else {}), + "reference_material_ids": [item["material_id"] for item in composition.get("selected_materials") or []], + }, + } + payload["dependency_context"] = dependency + def _publish(self, event_type: str, payload: dict[str, Any]) -> None: if self.event_bus: self.event_bus.publish(event_type, payload) + def _cancel_generation_if_requested( + self, + task_id: str, + workflow_id: str, + ctx: RequestContext, + ) -> bool: + task = self.queue_service.get_job(task_id) + if task.status == "cancelled": + return True + workflow = self.workflow_service.get_workflow(workflow_id).workflow + if workflow.status != "cancelled" or task.status not in {"queued", "running"}: + return False + try: + task = self.queue_service.cancel_job( + task_id, + ctx, + reason="owning workflow was cancelled", + ) + except ApiError: + task = self.queue_service.get_job(task_id) + return task.status == "cancelled" + + def _finalize_cancelled_generation( + self, + *, + task_id: str, + node_id: str, + workflow_id: str, + assets: list[Any], + images: list[Any], + ctx: RequestContext, + ) -> dict[str, Any]: + cleanup = self._cleanup_generation_outputs(assets=assets, images=images) + task = self.queue_service.get_job(task_id) + node = self.workflow_service.get_node(node_id) + node_metadata = node.metadata if isinstance(node.metadata, dict) else {} + node_task_id = _optional_text(node_metadata.get("last_task_id")) + if node_task_id == task_id and node.status in {"queued", "running", "cancelled"}: + cancellation = task.metadata.get("cancellation") if isinstance(task.metadata, dict) else None + cancellation_reason = ( + _optional_text(cancellation.get("reason")) + if isinstance(cancellation, dict) + else None + ) + node_update: dict[str, Any] = { + "metadata": { + **node_metadata, + "generation_cancellation": { + "task_id": task_id, + "reason": cancellation_reason or "cancelled", + "cleanup": cleanup, + }, + }, + } + if node.status in {"queued", "running"}: + node_update["status"] = "cancelled" + node = self.workflow_service.update_node( + node_id, + node_update, + ctx, + ) + self._publish( + "generation.node_cancelled", + { + "workflow_id": workflow_id, + "node_id": node_id, + "task_id": task_id, + "node_status": node.status, + "cleanup": cleanup, + }, + ) + return { + "task_id": task.task_id, + "task_status": task.status, + "node_status": node.status, + "images": [], + "cancelled": True, + } + + def _cleanup_generation_outputs(self, *, assets: list[Any], images: list[Any]) -> dict[str, Any]: + removed_image_ids: list[str] = [] + removed_uris: list[str] = [] + errors: list[dict[str, str]] = [] + for image in reversed(images): + image_id = _optional_text(getattr(image, "image_id", None)) + if not image_id: + continue + try: + removed = self.image_service.image_repository.delete_metadata(image_id) + if removed is not None: + removed_image_ids.append(image_id) + except Exception as exc: + errors.append( + { + "target": image_id, + "error_type": exc.__class__.__name__, + "message": str(exc), + } + ) + + seen_uris: set[str] = set() + for asset in assets: + for field in ("image_uri", "thumbnail_uri"): + uri = _optional_text(getattr(asset, field, None)) + if not uri or uri in seen_uris: + continue + seen_uris.add(uri) + try: + path = self.single_flow_engine.storage.resolve_uri(uri) + if path.exists() and path.is_file(): + path.unlink() + removed_uris.append(uri) + except Exception as exc: + errors.append( + { + "target": uri, + "error_type": exc.__class__.__name__, + "message": str(exc), + } + ) + return { + "image_records_removed": removed_image_ids, + "files_removed": removed_uris, + "errors": errors, + } + + def _record_generation_failure( + self, + *, + task: GenerationTask, + node_id: str, + workflow_id: str, + route_resolution: dict[str, Any], + exc: Exception, + ctx: RequestContext, + ) -> dict[str, Any]: + if isinstance(exc, ImageProviderError): + failure = provider_failure_attribution(exc, route_context=route_resolution, fallback_used=False) + else: + failure = { + "source": "generation_pipeline", + "category": "internal_error", + "error_type": exc.__class__.__name__, + "message": "generation pipeline failed", + "retryable": False, + "fallback_used": False, + } + failed_task = None + try: + failed_task = self.queue_service.update_task_if_status( + task.task_id, + {"queued", "running"}, + { + "status": "failed", + "progress": 100, + "error_message": str(failure.get("message") or "generation failed"), + "metadata": {**task.metadata, "generation_failure": failure}, + }, + ctx, + ) + except Exception: + pass + if failed_task is not None: + try: + node = self.workflow_service.get_node(node_id) + node_metadata = node.metadata if isinstance(node.metadata, dict) else {} + if ( + _optional_text(node_metadata.get("last_task_id")) == task.task_id + and node.status in {"queued", "running"} + ): + self.workflow_service.update_node( + node_id, + { + "status": "failed", + "metadata": { + **node_metadata, + "generation_failure": failure, + "last_task_id": task.task_id, + }, + }, + ctx, + ) + except Exception: + pass + self._publish( + "generation.node_failed", + {"workflow_id": workflow_id, "node_id": node_id, "task_id": task.task_id, "failure": failure}, + ) + return failure + + +def _retain_node_generation_lock(node_id: str) -> _NodeGenerationLock: + key = str(node_id) + with _NODE_GENERATION_LOCKS_GUARD: + entry = _NODE_GENERATION_LOCKS.get(key) + if entry is None: + entry = _NodeGenerationLock() + _NODE_GENERATION_LOCKS[key] = entry + entry.users += 1 + return entry + + +def _release_node_generation_lock(node_id: str, entry: _NodeGenerationLock) -> None: + key = str(node_id) + with _NODE_GENERATION_LOCKS_GUARD: + entry.users -= 1 + if entry.users == 0 and _NODE_GENERATION_LOCKS.get(key) is entry: + _NODE_GENERATION_LOCKS.pop(key, None) + + +def _assignment_key(node: Any) -> str: + node_key = str(getattr(node, "node_key", "") or "").strip() + node_type = str(getattr(node, "node_type", "") or "").strip() + mapping = { + "headshot": "portrait_generation", + "half_body": "halfbody_generation", + "full_body": "fullbody_generation", + "expression_sheet": "sheet_generation", + "pose_sheet": "sheet_generation", + "style_lock": "hires_fix", + "questionnaire": "quick_generation", + "preview": "quick_generation", + } + if node_key in mapping: + return mapping[node_key] + if node_type == "virtual_single_image": + return "quick_generation" + return "quick_generation" if node_type in {"profile_draft", "profile_preview"} else node_key + def _optional_text(value: Any) -> str | None: if value is None: @@ -255,6 +1132,37 @@ def _optional_text(value: Any) -> str | None: return text or None +_OMITTED_EPHEMERAL_REFERENCE = object() + + +def _strip_ephemeral_image_data_urls(value: Any) -> tuple[Any, int]: + """Remove inline image payloads before writing a generation request to storage.""" + + if isinstance(value, str): + if value.lstrip().lower().startswith("data:image/"): + return _OMITTED_EPHEMERAL_REFERENCE, 1 + return value, 0 + if isinstance(value, dict): + cleaned: dict[Any, Any] = {} + omitted = 0 + for key, item in value.items(): + prepared, item_omitted = _strip_ephemeral_image_data_urls(item) + omitted += item_omitted + if prepared is not _OMITTED_EPHEMERAL_REFERENCE: + cleaned[key] = prepared + return cleaned, omitted + if isinstance(value, (list, tuple)): + cleaned_items: list[Any] = [] + omitted = 0 + for item in value: + prepared, item_omitted = _strip_ephemeral_image_data_urls(item) + omitted += item_omitted + if prepared is not _OMITTED_EPHEMERAL_REFERENCE: + cleaned_items.append(prepared) + return cleaned_items, omitted + return value, 0 + + def _provider_failure_from_assets(metadata_items: list[dict[str, Any]]) -> dict[str, Any] | None: for metadata in metadata_items: failure = metadata.get("provider_error") @@ -269,3 +1177,86 @@ def _planned_image_count(value: Any) -> int: except (TypeError, ValueError): count = 1 return min(max(count, 1), 8) + + +def _entrypoint(payload: dict[str, Any]) -> str: + value = _optional_text(payload.get("entrypoint") or payload.get("platform") or payload.get("source")) or "web" + lowered = value.lower() + if "qq" in lowered or "onebot" in lowered: + return "qq" + if "telegram" in lowered: + return "telegram" + if "discord" in lowered: + return "discord" + if lowered in {"web", "api"}: + return lowered + return "unknown" + + +def _permission_context(payload: dict[str, Any]) -> dict[str, Any]: + configured = payload.get("content_permissions") if isinstance(payload.get("content_permissions"), dict) else {} + result = { + **configured, + "adult_user_allowed": bool(configured.get("adult_user_allowed", payload.get("adult_user_allowed", False))), + "quota_allowed": bool(configured.get("quota_allowed", payload.get("quota_allowed", True))), + } + if "adult_group_allowed" in configured or "adult_group_allowed" in payload: + result["adult_group_allowed"] = bool(configured.get("adult_group_allowed", payload.get("adult_group_allowed"))) + return result + + +def _reference_binding_summary(payload: dict[str, Any]) -> list[dict[str, Any]]: + composition = payload.get("composition") if isinstance(payload.get("composition"), dict) else {} + bindings = composition.get("reference_bindings") + if not isinstance(bindings, list): + dependency = payload.get("dependency_context") if isinstance(payload.get("dependency_context"), dict) else {} + material_bindings = dependency.get("material_bindings") if isinstance(dependency.get("material_bindings"), dict) else {} + bindings = material_bindings.get("reference_bindings") + return [dict(item) for item in bindings or [] if isinstance(item, dict)] + + +def _governance_audit_summary(governance: dict[str, Any]) -> dict[str, Any]: + translation = governance.get("translation") if isinstance(governance.get("translation"), dict) else {} + moderation = governance.get("moderation") if isinstance(governance.get("moderation"), dict) else {} + route = governance.get("route_decision") if isinstance(governance.get("route_decision"), dict) else {} + normalization = governance.get("normalization") if isinstance(governance.get("normalization"), dict) else {} + return { + "translation": { + "status": translation.get("status"), + "model_route": translation.get("model_route"), + "template_version": translation.get("template_version"), + "accepted": translation.get("accepted"), + }, + "normalization": { + "changed_terms": normalization.get("changed_terms") or [], + "meaning_preserved": normalization.get("meaning_preserved", True), + "version": normalization.get("version"), + }, + "moderation": { + "level": moderation.get("level"), + "confidence": moderation.get("confidence"), + "categories": moderation.get("categories") or [], + "reason_codes": moderation.get("reason_codes") or [], + "rules_version": moderation.get("rules_version"), + }, + "route_decision": route, + "entrypoint": governance.get("entrypoint"), + } + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + values = value.split(",") + elif isinstance(value, (list, tuple, set)): + values = value + else: + return [] + return [str(item).strip() for item in values if str(item).strip()] + + +def _positive_int(value: Any) -> int | None: + try: + number = int(value) + except (TypeError, ValueError): + return None + return number if number > 0 else None diff --git a/backend/services/history_service.py b/backend/services/history_service.py index 77abd6e..f837944 100644 --- a/backend/services/history_service.py +++ b/backend/services/history_service.py @@ -7,6 +7,7 @@ from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode from backend.dto.workflow_dto import WorkflowHistoryEntry +from backend.dto.workflow_dto import ProfileWorkflow, WorkflowNode from backend.services.image_service import ImageService from backend.services.workflow_service import WorkflowService from core.audit_service import AuditService @@ -35,9 +36,11 @@ class HistoryService: ) -> WorkflowHistoryEntry: node = self.workflow_service.get_node(node_id) workflow = self.workflow_service.get_workflow(node.workflow_id).workflow + payload_metadata = _mapping(payload.get("metadata")) image_id = str(payload.get("image_id") or "").strip() if not image_id: raise ApiError(ErrorCode.VALIDATION_FAILED, "image_id is required", {"field": "image_id"}) + next_node = self._resolve_handoff_next_node(workflow, payload_metadata) image = self.image_service.get_image(image_id) if image.node_id != node_id: raise ApiError( @@ -47,12 +50,15 @@ class HistoryService: ) if image.is_archived: raise ApiError(ErrorCode.VALIDATION_FAILED, "archived image cannot be adopted", {"image_id": image_id}) - selected = self.image_service.select_image(image_id, ctx) if not image.is_selected else image updated_node = self.workflow_service.update_node( node_id, - {"status": "selected", "selected_image_id": selected.image_id}, + {"status": "selected", "selected_image_id": image.image_id}, ctx, + allow_locked_adoption=True, ) + selected = self.image_service.image_repository.select_exclusive(image) if not image.is_selected else image + if selected is None: + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "image not found", {"image_id": image_id}) history = self.workflow_repository.create_history( { "history_id": payload.get("history_id") or new_history_id(), @@ -68,10 +74,19 @@ class HistoryService: "image_uri": selected.image_uri, "thumbnail_uri": selected.thumbnail_uri, "source": "node_adoption", - **(payload.get("metadata") or {}), + **payload_metadata, }, } ) + self._persist_stage_handoff( + workflow, + updated_node, + selected, + history, + payload_metadata, + next_node, + ctx, + ) self.audit_service.record( ctx=ctx, action="adopt_image", @@ -81,6 +96,103 @@ class HistoryService: ) return history + def _resolve_handoff_next_node( + self, + workflow: ProfileWorkflow, + metadata: dict[str, Any], + ) -> WorkflowNode | None: + next_node_id = _optional_text(metadata.get("next_backend_node_id")) + if not next_node_id: + return None + next_node = self.workflow_service.get_node(next_node_id) + if next_node.workflow_id != workflow.workflow_id: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "next_backend_node_id does not belong to this workflow", + {"workflow_id": workflow.workflow_id, "next_backend_node_id": next_node_id}, + ) + if next_node.branch_name != workflow.current_branch: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "next_backend_node_id does not belong to the current branch", + { + "workflow_id": workflow.workflow_id, + "current_branch": workflow.current_branch, + "next_backend_node_id": next_node_id, + "next_branch_name": next_node.branch_name, + }, + ) + return next_node + + def _persist_stage_handoff( + self, + workflow: ProfileWorkflow, + source_node: WorkflowNode, + selected, + history: WorkflowHistoryEntry, + metadata: dict[str, Any], + next_node: WorkflowNode | None, + ctx: RequestContext, + ) -> None: + if next_node is None: + return + existing_metadata = dict(workflow.metadata or {}) + existing_stage_inputs = existing_metadata.get("stage_inputs") + stage_inputs = dict(existing_stage_inputs) if isinstance(existing_stage_inputs, dict) else {} + handoff_record = { + "schema_version": STAGE_INPUT_SCHEMA_VERSION, + "image_id": selected.image_id, + "image_uri": selected.image_uri, + "thumbnail_uri": selected.thumbnail_uri, + "from_node_id": source_node.node_id, + "from_node_key": source_node.node_key, + "from_review_node_id": metadata.get("ui_review_node_id"), + "source_ui_node_id": metadata.get("source_ui_node_id"), + "next_ui_node_id": metadata.get("next_ui_node_id"), + "history_id": history.history_id, + "role": metadata.get("handoff_role") or "next_stage_input", + "handoff_bucket": metadata.get("handoff_bucket"), + "branch_name": source_node.branch_name, + "model_route": selected.model_route, + "model_used": selected.model_used, + "positive_prompt": selected.positive_prompt or source_node.prompt, + "negative_prompt": selected.negative_prompt or source_node.negative_prompt, + "locked_source_node": True, + "confirmed_at": history.created_at, + } + stage_inputs[next_node.node_id] = handoff_record + handoff_log = existing_metadata.get("stage_review_handoffs") + if not isinstance(handoff_log, list): + handoff_log = [] + handoff_log = [ + *handoff_log, + { + "history_id": history.history_id, + "from_node_id": source_node.node_id, + "to_node_id": next_node.node_id, + "image_id": selected.image_id, + "role": handoff_record["role"], + "confirmed_at": history.created_at, + }, + ][-20:] + self.workflow_service.update_workflow( + workflow.workflow_id, + { + "current_node_id": next_node.node_id, + "metadata": { + **existing_metadata, + "stage_input_schema_version": STAGE_INPUT_SCHEMA_VERSION, + "stage_inputs": stage_inputs, + "stage_review_handoffs": handoff_log, + "last_stage_review_confirmed_at": history.created_at, + "last_stage_review_node_id": metadata.get("ui_review_node_id"), + "last_stage_review_image_id": selected.image_id, + "current_stage_input_image_id": selected.image_id, + }, + }, + ctx, + ) + def list_workflow_history( self, workflow_id: str, @@ -97,6 +209,98 @@ class HistoryService: ) return self.workflow_repository.list_history(workflow_id, node_id=node_id) + def save_workflow_snapshot( + self, + workflow_id: str, + payload: dict[str, Any], + ctx: RequestContext, + ) -> WorkflowHistoryEntry: + detail = self.workflow_service.get_workflow(workflow_id) + workflow = detail.workflow + node_id = _optional_text(payload.get("node_id")) + node = None + if node_id: + node = self.workflow_service.get_node(node_id) + if node.workflow_id != workflow_id: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "node does not belong to this workflow", + {"workflow_id": workflow_id, "node_id": node_id}, + ) + image_id = _optional_text(payload.get("image_id")) + if image_id: + image = self.image_service.get_image(image_id) + if image.workflow_id != workflow_id: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "image does not belong to this workflow", + { + "workflow_id": workflow_id, + "image_id": image_id, + "image_workflow_id": image.workflow_id, + }, + ) + if image.node_id: + image_node = self.workflow_service.get_node(image.node_id) + if image_node.workflow_id != workflow_id: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "image node does not belong to this workflow", + { + "workflow_id": workflow_id, + "image_id": image_id, + "image_node_id": image.node_id, + "image_node_workflow_id": image_node.workflow_id, + }, + ) + if node_id and image.node_id != node_id: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "image does not belong to this node", + {"workflow_id": workflow_id, "node_id": node_id, "image_id": image_id}, + ) + history = self.workflow_repository.create_history( + { + "history_id": payload.get("history_id") or new_history_id(), + "profile_id": workflow.profile_id, + "workflow_id": workflow.workflow_id, + "node_id": node_id, + "image_id": image_id, + "branch_name": payload.get("branch_name") or workflow.current_branch or "main", + "action": payload.get("action") or "save_version", + "title": payload.get("title") or workflow.display_name, + "prompt": payload.get("prompt") or (node.prompt if node else None), + "metadata": { + "source": "workflow_snapshot", + "workflow": workflow.to_dict(), + "nodes": [item.to_dict() for item in detail.nodes] if payload.get("include_nodes", True) else [], + **(payload.get("metadata") if isinstance(payload.get("metadata"), dict) else {}), + }, + } + ) + self.audit_service.record( + ctx=ctx, + action="save_workflow_snapshot", + target_type="workflow_history", + target_id=history.history_id, + after_snapshot=history.to_dict(), + ) + return history + def new_history_id() -> str: return f"hist_{uuid4().hex}" + + +STAGE_INPUT_SCHEMA_VERSION = "2026-07-09.stage-input-bucket.v1" + + +def _optional_text(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + + +def _mapping(value: Any) -> dict[str, Any]: + return value if isinstance(value, dict) else {} diff --git a/backend/services/image_service.py b/backend/services/image_service.py index 8bc54d4..730e25d 100644 --- a/backend/services/image_service.py +++ b/backend/services/image_service.py @@ -6,7 +6,8 @@ from typing import Any from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode -from backend.dto.image_dto import GenerationImage +from backend.dto.image_dto import GenerationImage, ImageListResult +from backend.services.workflow_service import LOCKED_NODE_STATUSES, locked_node_edit_error from core.audit_service import AuditService from core.identity import RequestContext from database.repositories.image_repository import ImageRepository @@ -79,14 +80,68 @@ class ImageService: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "节点不存在", {"node_id": node_id}) return self.image_repository.list_by_node(node_id, include_archived=include_archived) + def list_images( + self, + *, + profile_id: str | None = None, + workflow_id: str | None = None, + include_archived: bool = True, + page: int = 1, + page_size: int = 60, + ) -> ImageListResult: + normalized_page = max(page, 1) + normalized_page_size = min(max(page_size, 1), 100) + items, total = self.image_repository.list_images( + profile_id=profile_id, + workflow_id=workflow_id, + include_archived=include_archived, + page=normalized_page, + page_size=normalized_page_size, + ) + return ImageListResult( + items=items, + total=total, + page=normalized_page, + page_size=normalized_page_size, + ) + def get_image(self, image_id: str) -> GenerationImage: image = self.image_repository.get(image_id) if image is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "图片不存在", {"image_id": image_id}) return image + def review_moderated_image(self, image_id: str, payload: dict[str, Any], ctx: RequestContext) -> GenerationImage: + before = self.get_image(image_id) + action = str(payload.get("action") or "").strip().lower() + if action not in {"approve", "reject"}: + raise ApiError(ErrorCode.VALIDATION_FAILED, "moderation review action must be approve or reject", {"field": "action"}) + metadata = before.metadata if isinstance(before.metadata, dict) else {} + if not (metadata.get("quarantined") or metadata.get("gallery_visibility") == "restricted"): + raise ApiError(ErrorCode.VALIDATION_FAILED, "image is not awaiting moderation review", {"image_id": image_id}) + review = { + "action": action, + "reviewer_id": ctx.actor_id, + "reason": str(payload.get("reason") or "").strip(), + } + update = { + "metadata": { + **metadata, + "quarantined": False if action == "approve" else True, + "gallery_visibility": "normal" if action == "approve" else "restricted", + "moderation_admin_review": review, + }, + **({"is_archived": True} if action == "reject" else {}), + } + after = self.image_repository.update(image_id, update) + if after is None: + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "图片不存在", {"image_id": image_id}) + self.audit_service.record(ctx=ctx, action="review_moderated_image", target_type="generation_image", target_id=image_id, before_snapshot=before.to_dict(), after_snapshot=after.to_dict(), metadata=review) + return after + def select_image(self, image_id: str, ctx: RequestContext) -> GenerationImage: before = self.get_image(image_id) + self._ensure_image_node_is_editable(before) if before.is_archived: raise ApiError(ErrorCode.VALIDATION_FAILED, "不能选择已归档图片", {"image_id": image_id}) after = self.image_repository.select_exclusive(before) @@ -106,6 +161,7 @@ class ImageService: def unselect_image(self, image_id: str, ctx: RequestContext) -> GenerationImage: before = self.get_image(image_id) + self._ensure_image_node_is_editable(before) after = self.image_repository.update(image_id, {"is_selected": False}) if after is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "图片不存在", {"image_id": image_id}) @@ -125,6 +181,7 @@ class ImageService: def archive_image(self, image_id: str, ctx: RequestContext) -> GenerationImage: before = self.get_image(image_id) + self._ensure_image_node_is_editable(before) after = self.image_repository.update( image_id, { @@ -149,6 +206,8 @@ class ImageService: return after def delete_image_metadata(self, image_id: str, ctx: RequestContext) -> GenerationImage: + existing = self.get_image(image_id) + self._ensure_image_node_is_editable(existing) before = self.image_repository.delete_metadata(image_id) if before is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "图片不存在", {"image_id": image_id}) @@ -166,6 +225,18 @@ class ImageService: ) return before + def _ensure_image_node_is_editable(self, image: GenerationImage) -> None: + if not image.node_id: + return + node = self.workflow_repository.get_node(image.node_id) + # ``completed`` means generation finished and its candidates are ready + # for curation. Prompt/node edits stay locked in WorkflowService, but + # users must still be able to change the tentative selection or + # archive rejected candidates until an image is formally adopted and + # the node enters ``selected``. + if node and node.status in LOCKED_NODE_STATUSES and node.status != "completed": + raise locked_node_edit_error(node) + def new_image_id() -> str: return f"img_{uuid4().hex}" diff --git a/backend/services/integration_service.py b/backend/services/integration_service.py index a3521a4..01b49ac 100644 --- a/backend/services/integration_service.py +++ b/backend/services/integration_service.py @@ -22,13 +22,24 @@ class IntegrationService: wardrobe_service: WardrobeService, integration_repository: IntegrationRepository, token_service: IntegrationTokenService | None = None, + *, + enabled: bool = True, ): self.wardrobe_service = wardrobe_service self.integration_repository = integration_repository self.token_service = token_service + self.enabled = enabled def daily_outfit(self, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: source = str(payload.get("request_source") or PRIVATE_COMPANION) + if not self.enabled: + error = ApiError( + ErrorCode.FORBIDDEN, + "private companion integration is disabled", + {"request_source": source, "feature": "private_companion_integration"}, + ) + self._record_failure(source, payload, error) + raise error if source != PRIVATE_COMPANION: error = ApiError( ErrorCode.INTEGRATION_TOKEN_INVALID, diff --git a/backend/services/integration_token_service.py b/backend/services/integration_token_service.py index e09d333..a40ae32 100644 --- a/backend/services/integration_token_service.py +++ b/backend/services/integration_token_service.py @@ -6,13 +6,12 @@ import hashlib import hmac import secrets from datetime import datetime, timedelta, timezone -from pathlib import Path from typing import Any from backend.dto.common import ApiError, ErrorCode -from backend.services.settings_service import _dump_yaml from core.audit_service import AuditService from core.config_service import ConfigService +from core.config_writer import config_file_lock, dump_yaml, write_text_atomic from core.identity import RequestContext from core.interop import PRIVATE_COMPANION from database.db_helper import _load_yaml_subset @@ -45,27 +44,30 @@ class IntegrationTokenService: def rotate(self, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: token = _token_or_generate(payload.get("token")) grace_seconds = _grace_seconds(payload.get("grace_seconds")) - config = self._load_config() - security = dict(config.get("security") or {}) - now = _now() - current_hash = str(security.get("integration_token_hash") or "").strip() - current_salt = str(security.get("integration_token_salt") or "").strip() - if current_hash and current_salt: - security["previous_integration_token_hash"] = current_hash - security["previous_integration_token_salt"] = current_salt - security["previous_integration_token_expires_at"] = _format_time(now + timedelta(seconds=grace_seconds)) - else: - security.pop("previous_integration_token_hash", None) - security.pop("previous_integration_token_salt", None) - security.pop("previous_integration_token_expires_at", None) - salt = secrets.token_hex(16) - security["integration_token_hash"] = _hash_token(token, salt) - security["integration_token_salt"] = salt - security["integration_token_rotated_at"] = _format_time(now) - security["integration_token_set"] = True - security.pop("integration_token", None) - config["security"] = security - self._write_config(config) + with config_file_lock(self.config_service.config_path): + config = self._load_config() + security = dict(config.get("security") or {}) + now = _now() + current_hash = str(security.get("integration_token_hash") or "").strip() + current_salt = str(security.get("integration_token_salt") or "").strip() + if current_hash and current_salt: + security["previous_integration_token_hash"] = current_hash + security["previous_integration_token_salt"] = current_salt + security["previous_integration_token_expires_at"] = _format_time( + now + timedelta(seconds=grace_seconds) + ) + else: + security.pop("previous_integration_token_hash", None) + security.pop("previous_integration_token_salt", None) + security.pop("previous_integration_token_expires_at", None) + salt = secrets.token_hex(16) + security["integration_token_hash"] = _hash_token(token, salt) + security["integration_token_salt"] = salt + security["integration_token_rotated_at"] = _format_time(now) + security["integration_token_set"] = True + security.pop("integration_token", None) + config["security"] = security + self._write_config(config) public_status = self.status() if self.audit_service: self.audit_service.record( @@ -93,7 +95,11 @@ class IntegrationTokenService: {"request_source": normalized_source}, ) if not _current_token_set(security): - return {"valid": True, "required": False, "token_slot": "not_configured"} + raise ApiError( + ErrorCode.INTEGRATION_TOKEN_INVALID, + "integration token has not been configured", + {"request_source": normalized_source, "token_configured": False}, + ) candidate = str(token or "").strip() if not candidate: raise ApiError(ErrorCode.INTEGRATION_TOKEN_INVALID, "integration token is required", {}) @@ -120,9 +126,7 @@ class IntegrationTokenService: return _load_yaml_subset(path.read_text(encoding="utf-8")) def _write_config(self, config: dict[str, Any]) -> None: - path: Path = self.config_service.config_path - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(_dump_yaml(config), encoding="utf-8") + write_text_atomic(self.config_service.config_path, dump_yaml(config)) def _current_token_set(security: dict[str, Any]) -> bool: diff --git a/backend/services/material_analytics_service.py b/backend/services/material_analytics_service.py new file mode 100644 index 0000000..a645dc8 --- /dev/null +++ b/backend/services/material_analytics_service.py @@ -0,0 +1,97 @@ +"""Aggregate material composition and prompt-governance adoption metrics.""" + +from __future__ import annotations + +import json +from collections import Counter +from typing import Any + +from database.db_helper import Database + + +class MaterialAnalyticsService: + def __init__(self, database: Database): + self.database = database + + def summarize(self) -> dict[str, Any]: + conn = self.database.connect() + try: + materials = conn.execute("SELECT recognition_status, metadata_json FROM materials WHERE recognition_status != 'archived'").fetchall() + audits = conn.execute("SELECT action, after_snapshot, metadata_json FROM audit_logs WHERE action IN ('wardrobe_match','apply_wardrobe_template','recognize_material','renormalize_material')").fetchall() + tasks = conn.execute("SELECT model_route, request_json, metadata_json FROM generation_tasks").fetchall() + finally: + conn.close() + schema_count = 0 + confirmed_count = 0 + category_counts: Counter[str] = Counter() + needs_confirmation = 0 + for row in materials: + metadata = _json(row["metadata_json"]) + profile = metadata.get("normalized_asset_profile") if isinstance(metadata.get("normalized_asset_profile"), dict) else {} + if profile: + schema_count += 1 + category_counts[str(profile.get("category") or "other")] += 1 + needs_confirmation += int(bool(profile.get("requires_confirmation"))) + confirmed_count += int(str(row["recognition_status"]) == "confirmed") + composition_count = conflicts = ignored = downgraded = 0 + for row in audits: + snapshot = _json(row["after_snapshot"]) + composition = snapshot.get("composition") if isinstance(snapshot.get("composition"), dict) else snapshot.get("composition_result") if isinstance(snapshot.get("composition_result"), dict) else {} + if not composition and row["action"] == "wardrobe_match": + composition = snapshot.get("composition") if isinstance(snapshot.get("composition"), dict) else {} + if composition: + composition_count += 1 + conflicts += len(composition.get("conflicts") or []) + ignored += len(composition.get("ignored_materials") or []) + downgraded += len(composition.get("downgraded_materials") or []) + translation_status: Counter[str] = Counter() + moderation_levels: Counter[str] = Counter() + route_reasons: Counter[str] = Counter() + route_counts: Counter[str] = Counter() + for row in tasks: + request = _json(row["request_json"]) + governance = request.get("prompt_governance") if isinstance(request.get("prompt_governance"), dict) else {} + translation = governance.get("translation") if isinstance(governance.get("translation"), dict) else {} + moderation = governance.get("moderation") if isinstance(governance.get("moderation"), dict) else {} + route = governance.get("route_decision") if isinstance(governance.get("route_decision"), dict) else {} + if translation: + translation_status[str(translation.get("status") or "unknown")] += 1 + if moderation: + moderation_levels[str(moderation.get("level") or "unknown")] += 1 + if route: + route_reasons[str(route.get("reason") or "unknown")] += 1 + route_counts[str(row["model_route"] or route.get("selected_route") or "unresolved")] += 1 + total = len(materials) + return { + "materials": { + "total": total, + "confirmed": confirmed_count, + "schema_adoption_count": schema_count, + "schema_adoption_rate": round(schema_count / total, 4) if total else 0.0, + "needs_confirmation": needs_confirmation, + "categories": dict(category_counts), + }, + "composition": { + "runs": composition_count, + "conflicts": conflicts, + "ignored_materials": ignored, + "downgraded_materials": downgraded, + "conflict_rate": round(conflicts / composition_count, 4) if composition_count else 0.0, + }, + "prompt_governance": { + "translation_status": dict(translation_status), + "moderation_levels": dict(moderation_levels), + "route_reasons": dict(route_reasons), + "route_distribution": dict(route_counts), + }, + } + + +def _json(value: Any) -> dict[str, Any]: + if isinstance(value, dict): + return value + try: + parsed = json.loads(value or "{}") + except (TypeError, json.JSONDecodeError): + return {} + return parsed if isinstance(parsed, dict) else {} diff --git a/backend/services/material_binding_schema_service.py b/backend/services/material_binding_schema_service.py new file mode 100644 index 0000000..14ad536 --- /dev/null +++ b/backend/services/material_binding_schema_service.py @@ -0,0 +1,297 @@ +"""Normalize workflow material binding payloads for generation nodes.""" + +from __future__ import annotations + +from copy import deepcopy +from typing import Any + +from backend.dto.workflow_dto import WorkflowNode + + +MATERIAL_BINDING_SCHEMA_VERSION = "2026-07-09.material-binding-schema.v1" + +REFERENCE_POLICY = { + "mode": "explicit_only", + "default_use_reference": False, + "empty_material_means": "do_not_use_reference", +} + +STAGE_NODE_KEYS: dict[str, tuple[str, ...]] = { + "profile-prep": ("profile_brief", "prompt_plan", "material_bind"), + "portrait-draft": ("headshot", "fast_candidates"), + "halfbody-draft": ("half_body",), + "body-sheet": ("full_body", "expression_sheet", "pose_sheet"), + "review-archive": ("style_lock", "final_pack", "fast_final"), +} + +STRENGTHS: dict[str, tuple[str, float]] = { + "强": ("strong", 0.85), + "strong": ("strong", 0.85), + "high": ("strong", 0.85), + "中": ("medium", 0.55), + "medium": ("medium", 0.55), + "mid": ("medium", 0.55), + "弱": ("weak", 0.3), + "weak": ("weak", 0.3), + "low": ("weak", 0.3), +} + + +def normalize_material_bindings( + payload: Any, + workflow_nodes: list[WorkflowNode] | None = None, +) -> dict[str, Any]: + if isinstance(payload, dict) and payload.get("schema_version") == MATERIAL_BINDING_SCHEMA_VERSION: + normalized = deepcopy(payload) + if workflow_nodes: + rules = normalized.get("active_rules") + if not isinstance(rules, list): + rules = [ + rule + for rule in normalized.get("rules", []) + if isinstance(rule, dict) and rule.get("use_reference") + ] + normalized["by_node_id"] = _index_by_node_id(rules, workflow_nodes) + return normalized + + source = _extract_source_payload(payload) + rules = [ + _normalize_rule(rule, index + 1) + for index, rule in enumerate(source.get("rules", []) if isinstance(source.get("rules"), list) else []) + if isinstance(rule, dict) + ] + active_rules = [rule for rule in rules if rule["use_reference"]] + by_stage = _index_by_stage(active_rules) + by_node_key = _index_by_node_key(active_rules) + by_node_id = _index_by_node_id(active_rules, workflow_nodes or []) + selected_material_ids = _string_list(source.get("selected_material_ids")) + if not selected_material_ids: + selected_material_ids = [rule["material"]["material_id"] for rule in active_rules] + return { + "schema_version": MATERIAL_BINDING_SCHEMA_VERSION, + "reference_policy": deepcopy(REFERENCE_POLICY), + "profile_id": _optional_text(source.get("profile_id")), + "profile_name": _optional_text(source.get("profile_name")), + "selected_material_ids": _unique_texts(selected_material_ids), + "rules": rules, + "active_rules": active_rules, + "by_stage": by_stage, + "by_node_key": by_node_key, + "by_node_id": by_node_id, + "node_parameters": by_node_id, + "prompt_context": _prompt_context(active_rules), + "summary": { + "rule_count": len(rules), + "active_rule_count": len(active_rules), + "has_references": bool(active_rules), + "disabled_categories": [rule["category"] for rule in rules if not rule["use_reference"]], + "target_stage_ids": sorted(by_stage), + "target_node_keys": sorted(by_node_key), + }, + } + + +def material_bindings_for_node( + workflow_metadata: dict[str, Any], + node: WorkflowNode, + workflow_nodes: list[WorkflowNode] | None = None, +) -> dict[str, Any]: + normalized = normalize_material_bindings(workflow_metadata.get("material_bindings"), workflow_nodes) + return node_material_binding_view(normalized, node) + + +def node_material_binding_view(normalized: dict[str, Any], node: WorkflowNode) -> dict[str, Any]: + by_node_id = normalized.get("by_node_id") if isinstance(normalized.get("by_node_id"), dict) else {} + by_node_key = normalized.get("by_node_key") if isinstance(normalized.get("by_node_key"), dict) else {} + node_view = by_node_id.get(node.node_id) + if not isinstance(node_view, dict): + node_view = _node_view( + active_rules=by_node_key.get(node.node_key, []), + node_id=node.node_id, + node_key=node.node_key, + title=node.title, + ) + return { + "schema_version": normalized.get("schema_version", MATERIAL_BINDING_SCHEMA_VERSION), + "reference_policy": normalized.get("reference_policy", deepcopy(REFERENCE_POLICY)), + "summary": normalized.get("summary", {}), + "target_node_id": node.node_id, + "target_node_key": node.node_key, + "target_node_title": node.title, + **node_view, + } + + +def _extract_source_payload(payload: Any) -> dict[str, Any]: + if not isinstance(payload, dict): + return {} + material_bindings = payload.get("material_bindings") + if isinstance(material_bindings, dict) and isinstance(material_bindings.get("rules"), list): + return material_bindings + if isinstance(payload.get("rules"), list): + return payload + return {} + + +def _normalize_rule(rule: dict[str, Any], ordinal: int) -> dict[str, Any]: + material_id = _optional_text(rule.get("material_id")) + material_prompt = _optional_text(rule.get("material_prompt")) + category = _optional_text(rule.get("category")) or "other" + target_stages = _target_stages(rule.get("bind_to_stages")) + target_node_keys = _unique_texts( + node_key for stage in target_stages for node_key in STAGE_NODE_KEYS.get(stage["stage_id"], ()) + ) + strength_label = _optional_text(rule.get("reference_strength")) or "medium" + strength_key, strength_weight = STRENGTHS.get(strength_label.lower(), STRENGTHS.get(strength_label, ("medium", 0.55))) + use_reference = bool(material_id or material_prompt) + return { + "id": _optional_text(rule.get("id")) or f"material_binding_{ordinal:02d}", + "category": category, + "use_reference": use_reference, + "required": bool(rule.get("required")), + "status": _optional_text(rule.get("status")) or ("ready" if use_reference else "pending"), + "conflict": _optional_text(rule.get("conflict")), + "material": { + "material_id": material_id, + "label": _optional_text(rule.get("material_label")), + "group": _optional_text(rule.get("material_group")), + "prompt": material_prompt, + "colors": _string_list(rule.get("material_colors")), + }, + "target_stages": target_stages, + "target_node_keys": target_node_keys, + "inheritance_rule": _optional_text(rule.get("inheritance_rule")) or "", + "reference_strength": { + "label": strength_label, + "key": strength_key, + "weight": strength_weight, + }, + "generation_params": { + "reference_policy": "explicit_only" if use_reference else "do_not_use_reference", + "reference_strength": strength_key if use_reference else "none", + "reference_weight": strength_weight if use_reference else 0, + }, + "expand_to_nodes_later": True, + } + + +def _target_stages(value: Any) -> list[dict[str, str]]: + if not isinstance(value, list): + return [] + stages: list[dict[str, str]] = [] + for item in value: + if not isinstance(item, dict): + continue + stage_id = _optional_text(item.get("stage_id") or item.get("id")) + if not stage_id: + continue + stage = { + "stage_id": stage_id, + "stage_title": _optional_text(item.get("stage_title") or item.get("title")) or stage_id, + } + if stage not in stages: + stages.append(stage) + return stages + + +def _index_by_stage(rules: list[dict[str, Any]]) -> dict[str, dict[str, Any]]: + result: dict[str, dict[str, Any]] = {} + for rule in rules: + for stage in rule["target_stages"]: + stage_id = stage["stage_id"] + bucket = result.setdefault( + stage_id, + { + "stage_id": stage_id, + "stage_title": stage["stage_title"], + "active_bindings": [], + "material_prompts": [], + }, + ) + bucket["active_bindings"].append(rule) + if rule["material"]["prompt"]: + bucket["material_prompts"].append(rule["material"]["prompt"]) + return result + + +def _index_by_node_key(rules: list[dict[str, Any]]) -> dict[str, list[dict[str, Any]]]: + result: dict[str, list[dict[str, Any]]] = {} + for rule in rules: + for node_key in rule["target_node_keys"]: + result.setdefault(node_key, []).append(rule) + return result + + +def _index_by_node_id( + rules: list[dict[str, Any]], + workflow_nodes: list[WorkflowNode], +) -> dict[str, dict[str, Any]]: + by_node_key = _index_by_node_key(rules) + return { + node.node_id: _node_view(by_node_key.get(node.node_key, []), node.node_id, node.node_key, node.title) + for node in workflow_nodes + if by_node_key.get(node.node_key) + } + + +def _node_view( + active_rules: list[dict[str, Any]], + node_id: str | None, + node_key: str, + title: str, +) -> dict[str, Any]: + prompts = [rule["material"]["prompt"] for rule in active_rules if rule["material"]["prompt"]] + return { + "target_node_id": node_id, + "target_node_key": node_key, + "target_node_title": title, + "reference_policy": deepcopy(REFERENCE_POLICY), + "has_references": bool(active_rules), + "active_bindings": deepcopy(active_rules), + "material_prompts": prompts, + "prompt_context": ", ".join(prompts), + "generation_params": { + "reference_material_ids": [ + rule["material"]["material_id"] for rule in active_rules if rule["material"]["material_id"] + ], + "reference_prompts": prompts, + "reference_strengths": [rule["reference_strength"] for rule in active_rules], + }, + } + + +def _prompt_context(rules: list[dict[str, Any]]) -> list[dict[str, str]]: + return [ + { + "category": rule["category"], + "material_id": rule["material"]["material_id"] or "", + "prompt": rule["material"]["prompt"] or "", + "target_node_keys": ", ".join(rule["target_node_keys"]), + } + for rule in rules + if rule["material"]["prompt"] + ] + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return [value] if value.strip() else [] + if not isinstance(value, list): + return [] + return _unique_texts(str(item).strip() for item in value if str(item or "").strip()) + + +def _unique_texts(values: Any) -> list[str]: + result: list[str] = [] + for value in values: + text = str(value or "").strip() + if text and text not in result: + result.append(text) + return result + + +def _optional_text(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None diff --git a/backend/services/material_normalization_service.py b/backend/services/material_normalization_service.py new file mode 100644 index 0000000..03e28b3 --- /dev/null +++ b/backend/services/material_normalization_service.py @@ -0,0 +1,404 @@ +"""Versioned material recognition normalization and generation capsules. + +The first rollout intentionally stores these structures inside ``material.metadata``. +That keeps existing database rows and API consumers compatible while giving newer +composition code a stable, explainable representation. +""" + +from __future__ import annotations + +from copy import deepcopy +from datetime import datetime, timezone +from typing import Any, Mapping + + +MATERIAL_SCHEMA_VERSION = "1.0" +MATERIAL_PROMPT_VERSION = "1.0" + +ASSET_CATEGORIES = { + "identity", "hair", "headwear", "face_accessory", "jewelry", "top", "bottom", + "dress", "suit", "outerwear", "underwear", "hosiery", "footwear", "prop", + "pose", "background", "style", "other", +} +ASSET_ROLES = {"identity", "garment", "accessory", "prop", "pose", "background", "style", "other"} +BODY_SLOTS = { + "identity", "hair", "head", "face", "ears", "neck", "upper_body", "arms", "hands", + "waist", "hips", "upper_legs", "lower_legs", "feet", "pose", "background", "global_style", +} +WEAR_LAYERS = {"skin", "inner", "base", "middle", "outer", "overlay", "accessory", "prop"} +SOURCE_STATES = {"observed", "inferred", "unknown", "occluded", "low_visibility"} +CAPSULE_TIERS = ("compact", "standard", "detailed") + +CATEGORY_ALIASES = { + "shirt": "top", "blouse": "top", "t-shirt": "top", "top": "top", "上衣": "top", "衬衫": "top", + "skirt": "bottom", "pants": "bottom", "trousers": "bottom", "shorts": "bottom", "裙子": "bottom", "裤子": "bottom", + "dress": "dress", "gown": "dress", "连衣裙": "dress", + "coat": "outerwear", "jacket": "outerwear", "cloak": "outerwear", "外套": "outerwear", + "shoe": "footwear", "shoes": "footwear", "boot": "footwear", "boots": "footwear", "鞋": "footwear", "靴": "footwear", + "necklace": "jewelry", "earring": "jewelry", "bracelet": "jewelry", "ring": "jewelry", "首饰": "jewelry", + "hat": "headwear", "cap": "headwear", "头饰": "headwear", + "hair": "hair", "发型": "hair", "pose": "pose", "姿势": "pose", + "background": "background", "scene": "background", "背景": "background", + "style": "style", "画风": "style", "character": "identity", "identity": "identity", "角色": "identity", +} + +CATEGORY_SEMANTICS: dict[str, tuple[str, list[str], str | None]] = { + "identity": ("identity", ["identity", "face"], None), + "hair": ("identity", ["hair"], None), + "headwear": ("accessory", ["head"], "accessory"), + "face_accessory": ("accessory", ["face"], "accessory"), + "jewelry": ("accessory", ["ears", "neck", "hands"], "accessory"), + "top": ("garment", ["upper_body", "arms"], "base"), + "bottom": ("garment", ["waist", "hips", "upper_legs", "lower_legs"], "base"), + "dress": ("garment", ["upper_body", "waist", "hips", "upper_legs", "lower_legs"], "base"), + "suit": ("garment", ["upper_body", "waist", "hips", "upper_legs", "lower_legs"], "middle"), + "outerwear": ("garment", ["upper_body", "arms"], "outer"), + "underwear": ("garment", ["upper_body", "waist", "hips"], "inner"), + "hosiery": ("garment", ["upper_legs", "lower_legs"], "inner"), + "footwear": ("garment", ["feet"], "base"), + "prop": ("prop", ["hands"], "prop"), + "pose": ("pose", ["pose"], None), + "background": ("background", ["background"], None), + "style": ("style", ["global_style"], None), + "other": ("other", [], None), +} + + +RAW_ANALYSIS_SCHEMA: dict[str, Any] = { + "type": "object", + "required": ["schema_version", "created_at", "model_source", "attributes"], + "properties": { + "schema_version": {"type": "string"}, + "created_at": {"type": "string"}, + "model_source": {"type": "object"}, + "attributes": {"type": "object"}, + "source_payload": {"type": "object"}, + }, +} +NORMALIZED_ASSET_PROFILE_SCHEMA: dict[str, Any] = { + "type": "object", + "required": ["schema_version", "category", "role", "body_slots", "attributes"], + "properties": { + "category": {"enum": sorted(ASSET_CATEGORIES)}, + "role": {"enum": sorted(ASSET_ROLES)}, + "body_slots": {"type": "array", "items": {"enum": sorted(BODY_SLOTS)}}, + "wear_layer": {"type": ["string", "null"]}, + }, +} +GENERATION_CAPSULE_SCHEMA: dict[str, Any] = { + "type": "object", + "required": ["schema_version", "prompt_version", "tier", "prompt_parts", "negative_parts"], + "properties": {"tier": {"enum": list(CAPSULE_TIERS)}}, +} + + +class MaterialNormalizationError(ValueError): + """Raised when provider data cannot be normalized safely.""" + + +class MaterialNormalizationService: + def normalize_recognition( + self, + structured_data: Mapping[str, Any], + *, + prompt: str, + negative_prompt: str | None, + tags: list[str], + confidence: float | None, + provider_id: str, + model: str, + route_name: str, + category_hint: str | None = None, + previous_metadata: Mapping[str, Any] | None = None, + ) -> dict[str, Any]: + now = _utc_now() + payload = deepcopy(dict(structured_data)) + raw_section = payload.get("raw_analysis") if isinstance(payload.get("raw_analysis"), dict) else payload + raw_attributes = self._normalize_attributes(raw_section.get("attributes"), confidence) + if not raw_attributes: + raw_attributes = self._attributes_from_legacy(prompt, tags, confidence) + raw_analysis = { + "schema_version": MATERIAL_SCHEMA_VERSION, + "prompt_version": MATERIAL_PROMPT_VERSION, + "created_at": now, + "model_source": {"provider_id": provider_id, "model": model, "route_name": route_name}, + "attributes": raw_attributes, + "description": _text(raw_section.get("description") or payload.get("description")), + "source_payload": payload, + } + + profile_input = payload.get("normalized_asset_profile") + if not isinstance(profile_input, dict): + profile_input = payload.get("asset_profile") if isinstance(payload.get("asset_profile"), dict) else {} + normalized = self._build_profile( + profile_input, + raw_attributes=raw_attributes, + prompt=prompt, + negative_prompt=negative_prompt, + tags=tags, + confidence=confidence, + category_hint=category_hint, + created_at=now, + ) + capsules = self._build_capsules(payload.get("generation_capsules"), normalized, prompt, negative_prompt, now) + result = {"raw_analysis": raw_analysis, "normalized_asset_profile": normalized, "generation_capsules": capsules} + validate_material_structures(result) + return self._preserve_history(result, previous_metadata) + + def ensure_legacy_structures( + self, + *, + prompt: str | None, + negative_prompt: str | None, + tags: list[str], + metadata: Mapping[str, Any] | None, + category_hint: str | None = None, + ) -> dict[str, Any]: + existing = deepcopy(dict(metadata or {})) + if all(isinstance(existing.get(key), dict) for key in ("raw_analysis", "normalized_asset_profile", "generation_capsules")): + return existing + normalized = self.normalize_recognition( + {"legacy": True}, + prompt=_text(prompt) or ", ".join(tags) or "material reference", + negative_prompt=negative_prompt, + tags=tags, + confidence=None, + provider_id="legacy", + model="legacy-prompt", + route_name="legacy", + category_hint=category_hint, + previous_metadata=existing, + ) + return {**existing, **normalized, "material_schema_migrated": True} + + def _build_profile( + self, + value: Mapping[str, Any], + *, + raw_attributes: dict[str, Any], + prompt: str, + negative_prompt: str | None, + tags: list[str], + confidence: float | None, + category_hint: str | None, + created_at: str, + ) -> dict[str, Any]: + category = normalize_category(value.get("category") or category_hint or _category_from_text([*tags, prompt])) + role, default_slots, default_layer = CATEGORY_SEMANTICS[category] + body_slots = _enum_list(value.get("body_slots") or value.get("slots"), BODY_SLOTS) or default_slots + wear_layer = _enum(value.get("wear_layer") or value.get("layer"), WEAR_LAYERS) or default_layer + attributes = { + key: item for key, item in raw_attributes.items() + if item.get("source_state") == "observed" and float(item.get("confidence") or 0.0) >= 0.60 + } + requires_confirmation = category == "other" or any( + item.get("source_state") != "observed" or float(item.get("confidence") or 0.0) < 0.80 + for item in raw_attributes.values() + ) + return { + "schema_version": MATERIAL_SCHEMA_VERSION, + "prompt_version": MATERIAL_PROMPT_VERSION, + "created_at": created_at, + "category": category, + "role": _enum(value.get("role"), ASSET_ROLES) or role, + "body_slots": body_slots, + "wear_layer": wear_layer, + "attributes": attributes, + "must_preserve": _string_list(value.get("must_preserve")), + "forbidden_changes": _string_list(value.get("forbidden_changes")) or _split_prompt(negative_prompt), + "tags": sorted(set(tags)), + "confidence": _confidence(value.get("confidence"), confidence), + "requires_confirmation": requires_confirmation, + "confirmed": bool(value.get("confirmed", False)), + "user_locked": bool(value.get("user_locked", False)), + "stackable": bool(value.get("stackable", category in {"jewelry", "headwear"})), + } + + def _build_capsules( + self, + provided: Any, + profile: Mapping[str, Any], + prompt: str, + negative_prompt: str | None, + created_at: str, + ) -> dict[str, Any]: + supplied = provided if isinstance(provided, dict) else {} + observed_parts = _attribute_prompt_parts(profile.get("attributes")) + must_preserve = _string_list(profile.get("must_preserve")) + base_parts = _unique([prompt, *observed_parts, *must_preserve]) + negative_parts = _unique(_split_prompt(negative_prompt) + _string_list(profile.get("forbidden_changes"))) + limits = {"compact": 4, "standard": 10, "detailed": 24} + capsules: dict[str, Any] = {} + for tier in CAPSULE_TIERS: + item = supplied.get(tier) if isinstance(supplied.get(tier), dict) else {} + capsules[tier] = { + "schema_version": MATERIAL_SCHEMA_VERSION, + "prompt_version": MATERIAL_PROMPT_VERSION, + "created_at": created_at, + "tier": tier, + "prompt_parts": _string_list(item.get("prompt_parts")) or base_parts[: limits[tier]], + "negative_parts": _string_list(item.get("negative_parts")) or negative_parts, + "must_preserve": _string_list(item.get("must_preserve")) or must_preserve, + "forbidden_changes": _string_list(item.get("forbidden_changes")) or _string_list(profile.get("forbidden_changes")), + "legacy": not bool(supplied), + } + return capsules + + def _normalize_attributes(self, value: Any, fallback_confidence: float | None) -> dict[str, Any]: + if not isinstance(value, dict): + return {} + result: dict[str, Any] = {} + for key, raw in value.items(): + name = _text(key).lower().replace(" ", "_") + if not name: + continue + item = raw if isinstance(raw, dict) else {"value": raw} + source_state = _enum(item.get("source_state") or item.get("state"), SOURCE_STATES) or "observed" + confidence = _confidence(item.get("confidence"), fallback_confidence) + result[name] = { + "value": deepcopy(item.get("value")), + "confidence": confidence, + "source_state": source_state, + "source": _text(item.get("source")) or "vision_model", + "locked": bool(item.get("locked", False)), + "eligible_for_generation": source_state == "observed" and confidence >= 0.60, + "requires_confirmation": source_state != "observed" or confidence < 0.80, + } + return result + + def _attributes_from_legacy(self, prompt: str, tags: list[str], confidence: float | None) -> dict[str, Any]: + value = _unique([*tags, *_split_prompt(prompt)]) + score = _confidence(confidence, 0.60) + return { + "legacy_description": { + "value": value, + "confidence": score, + "source_state": "observed" if confidence is not None else "inferred", + "source": "legacy_prompt", + "locked": False, + "eligible_for_generation": confidence is not None and score >= 0.60, + "requires_confirmation": confidence is None or score < 0.80, + } + } + + def _preserve_history(self, current: dict[str, Any], previous_metadata: Mapping[str, Any] | None) -> dict[str, Any]: + previous = dict(previous_metadata or {}) + old_profile = previous.get("normalized_asset_profile") + old_raw = previous.get("raw_analysis") + history = list(previous.get("material_schema_history") or []) + if isinstance(old_profile, dict) or isinstance(old_raw, dict): + history.append({"archived_at": _utc_now(), "raw_analysis": old_raw, "normalized_asset_profile": old_profile, "generation_capsules": previous.get("generation_capsules")}) + if history: + current["material_schema_history"] = history[-10:] + if isinstance(old_profile, dict): + profile = current["normalized_asset_profile"] + if old_profile.get("confirmed") or old_profile.get("user_locked"): + for key in ("category", "role", "body_slots", "wear_layer", "must_preserve", "forbidden_changes", "stackable", "confirmed", "user_locked"): + if key in old_profile: + profile[key] = deepcopy(old_profile[key]) + old_attributes = old_profile.get("attributes") + if isinstance(old_attributes, dict): + for key, item in old_attributes.items(): + if isinstance(item, dict) and item.get("locked"): + profile.setdefault("attributes", {})[key] = deepcopy(item) + return current + + +def normalize_category(value: Any) -> str: + text = _text(value).lower().replace("_", " ").strip() + if text in ASSET_CATEGORIES: + return text + for alias, category in CATEGORY_ALIASES.items(): + if alias in text: + return category + return "other" + + +def validate_material_structures(value: Mapping[str, Any]) -> None: + raw = value.get("raw_analysis") + profile = value.get("normalized_asset_profile") + capsules = value.get("generation_capsules") + if not isinstance(raw, dict) or not isinstance(raw.get("attributes"), dict): + raise MaterialNormalizationError("raw_analysis.attributes must be an object") + if not isinstance(profile, dict) or profile.get("category") not in ASSET_CATEGORIES: + raise MaterialNormalizationError("normalized_asset_profile.category is invalid") + if profile.get("role") not in ASSET_ROLES: + raise MaterialNormalizationError("normalized_asset_profile.role is invalid") + if any(slot not in BODY_SLOTS for slot in profile.get("body_slots", [])): + raise MaterialNormalizationError("normalized_asset_profile.body_slots contains an invalid value") + if profile.get("wear_layer") is not None and profile.get("wear_layer") not in WEAR_LAYERS: + raise MaterialNormalizationError("normalized_asset_profile.wear_layer is invalid") + if not isinstance(capsules, dict) or any(not isinstance(capsules.get(tier), dict) for tier in CAPSULE_TIERS): + raise MaterialNormalizationError("generation_capsules must contain compact, standard and detailed") + + +def _category_from_text(values: list[str]) -> str: + return " ".join(_text(item) for item in values) + + +def _attribute_prompt_parts(value: Any) -> list[str]: + if not isinstance(value, dict): + return [] + parts: list[str] = [] + for name, item in value.items(): + if not isinstance(item, dict) or not item.get("eligible_for_generation", True): + continue + raw = item.get("value") + if isinstance(raw, list): + text = ", ".join(_text(part) for part in raw if _text(part)) + else: + text = _text(raw) + if text: + parts.append(f"{name.replace('_', ' ')}: {text}") + return parts + + +def _split_prompt(value: Any) -> list[str]: + text = _text(value) + return [part.strip() for part in text.replace(";", ",").split(",") if part.strip()] + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return _split_prompt(value) + if not isinstance(value, (list, tuple, set)): + return [] + return _unique([_text(item) for item in value]) + + +def _unique(values: list[str]) -> list[str]: + result: list[str] = [] + seen: set[str] = set() + for value in values: + text = _text(value) + key = text.casefold() + if text and key not in seen: + seen.add(key) + result.append(text) + return result + + +def _enum(value: Any, allowed: set[str]) -> str | None: + text = _text(value).lower() + return text if text in allowed else None + + +def _enum_list(value: Any, allowed: set[str]) -> list[str]: + values = value if isinstance(value, (list, tuple, set)) else [value] + return _unique([item for item in (_enum(part, allowed) for part in values) if item]) + + +def _confidence(value: Any, fallback: float | None) -> float: + try: + number = float(value if value is not None else fallback if fallback is not None else 0.0) + except (TypeError, ValueError): + number = 0.0 + return min(max(number, 0.0), 1.0) + + +def _text(value: Any) -> str: + return str(value or "").strip() + + +def _utc_now() -> str: + return datetime.now(timezone.utc).isoformat() diff --git a/backend/services/material_service.py b/backend/services/material_service.py index 2ba89ad..bb27c0e 100644 --- a/backend/services/material_service.py +++ b/backend/services/material_service.py @@ -2,17 +2,29 @@ from __future__ import annotations +import base64 +from dataclasses import replace from pathlib import PurePosixPath from typing import Any, Callable from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode from backend.dto.material_dto import Material, MaterialCategory, MaterialListResult +from backend.services.material_normalization_service import ( + MaterialNormalizationError, + MaterialNormalizationService, + validate_material_structures, +) from core.audit_service import AuditService +from core.image_formats import ( + IMAGE_EXTENSIONS_BY_MIME, + SUPPORTED_IMAGE_EXTENSIONS, + detect_image_format, +) from core.identity import RequestContext +from core.storage import VfsStorage from database.repositories.material_repository import MaterialRepository from engines.vision_provider import ( - MockVisionProvider, VisionProvider, VisionProviderError, VisionRecognitionRequest, @@ -23,11 +35,9 @@ from engines.vision_provider import ( MATERIAL_STATUSES = {"pending_upload", "pending_recognition", "pending_confirm", "confirmed", "archived"} MAX_UPLOAD_BYTES = 20 * 1024 * 1024 UPLOAD_SIZE_FIELDS = ("size_bytes", "content_length", "file_size", "size") -UPLOAD_ALLOWED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".webp"} +UPLOAD_ALLOWED_EXTENSIONS = set(SUPPORTED_IMAGE_EXTENSIONS) UPLOAD_ALLOWED_CONTENT_TYPES = { - "image/jpeg": {".jpg", ".jpeg"}, - "image/png": {".png"}, - "image/webp": {".webp"}, + mime_type: set(extensions) for mime_type, extensions in IMAGE_EXTENSIONS_BY_MIME.items() } @@ -39,11 +49,17 @@ class MaterialService: *, vision_provider_factory: Callable[[dict[str, Any]], VisionProvider | None] | None = None, default_recognition_route: str = "material_vlm", + recognition_prompt_config: dict[str, Any] | None = None, + normalization_service: MaterialNormalizationService | None = None, + storage: VfsStorage | None = None, ): self.repository = repository self.audit_service = audit_service self.vision_provider_factory = vision_provider_factory self.default_recognition_route = default_recognition_route + self.recognition_prompt_config = recognition_prompt_config or {} + self.normalization_service = normalization_service or MaterialNormalizationService() + self.storage = storage or VfsStorage() def create_category(self, payload: dict[str, Any], ctx: RequestContext) -> MaterialCategory: name = str(payload.get("name") or "").strip() @@ -78,6 +94,12 @@ class MaterialService: before = self.get_category(category_id) if "name" in payload and not str(payload["name"]).strip(): raise ApiError(ErrorCode.VALIDATION_FAILED, "category name is required", {"field": "name"}) + parent_id = payload.get("parent_id") if "parent_id" in payload else None + if parent_id: + if parent_id == category_id: + raise ApiError(ErrorCode.VALIDATION_FAILED, "category cannot be its own parent", {"field": "parent_id"}) + if self.repository.get_category(parent_id) is None: + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "parent category not found", {"parent_id": parent_id}) after = self.repository.update_category(category_id, payload) if after is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "category not found", {"category_id": category_id}) @@ -111,14 +133,9 @@ class MaterialService: return category def create_material(self, payload: dict[str, Any], ctx: RequestContext) -> Material: + self.validate_material_payload(payload, creating=True) display_name = str(payload.get("display_name") or "").strip() - if not display_name: - raise ApiError(ErrorCode.VALIDATION_FAILED, "display_name is required", {"field": "display_name"}) - self._validate_category(payload.get("category_id")) - self._validate_storage_uri(payload.get("image_uri"), "image_uri") - self._validate_storage_uri(payload.get("thumbnail_uri"), "thumbnail_uri") status = payload.get("recognition_status", "pending_upload") - self._validate_status(status) material = self.repository.create_material( { "material_id": payload.get("material_id") or new_material_id(), @@ -149,38 +166,77 @@ class MaterialService: keyword: str | None = None, page: int = 1, page_size: int = 30, + color: str | None = None, + material_type: str | None = None, + body_slot: str | None = None, + wear_layer: str | None = None, + min_confidence: float | None = None, + confirmation_state: str | None = None, ) -> MaterialListResult: normalized_page = max(page, 1) normalized_page_size = min(max(page_size, 1), 100) + advanced_filters = any(value not in (None, "") for value in (color, material_type, body_slot, wear_layer, min_confidence, confirmation_state)) + if advanced_filters: + all_items: list[Material] = [] + repository_page = 1 + while True: + batch, total = self.repository.list_materials( + category_id=category_id, + keyword=keyword, + page=repository_page, + page_size=100, + ) + all_items.extend(self._with_material_structures(item) for item in batch) + if len(all_items) >= total or not batch: + break + repository_page += 1 + filtered = [ + item for item in all_items + if _matches_material_filters( + item, + color=color, + material_type=material_type, + body_slot=body_slot, + wear_layer=wear_layer, + min_confidence=min_confidence, + confirmation_state=confirmation_state, + ) + ] + start = (normalized_page - 1) * normalized_page_size + return MaterialListResult(items=filtered[start : start + normalized_page_size], total=len(filtered), page=normalized_page, page_size=normalized_page_size) items, total = self.repository.list_materials( category_id=category_id, keyword=keyword, page=normalized_page, page_size=normalized_page_size, ) - return MaterialListResult(items=items, total=total, page=normalized_page, page_size=normalized_page_size) + return MaterialListResult( + items=[self._with_material_structures(item) for item in items], + total=total, + page=normalized_page, + page_size=normalized_page_size, + ) def get_material(self, material_id: str) -> Material: material = self.repository.get_material(material_id) if material is None or material.recognition_status == "archived": raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "material not found", {"material_id": material_id}) - return material + return self._with_material_structures(material) def update_material(self, material_id: str, payload: dict[str, Any], ctx: RequestContext) -> Material: before = self.get_material(material_id) - if "category_id" in payload: - self._validate_category(payload.get("category_id")) - if "image_uri" in payload: - self._validate_storage_uri(payload.get("image_uri"), "image_uri") - if "thumbnail_uri" in payload: - self._validate_storage_uri(payload.get("thumbnail_uri"), "thumbnail_uri") - if "recognition_status" in payload: - self._validate_status(payload["recognition_status"]) + self.validate_material_payload(payload, creating=False) update_payload = dict(payload) + structured_keys = {"raw_analysis", "normalized_asset_profile", "generation_capsules"} + if structured_keys & update_payload.keys(): + metadata = {**before.metadata} + for key in structured_keys: + if key in update_payload: + metadata[key] = update_payload.pop(key) + validate_material_structures(metadata) + update_payload["metadata"] = metadata if "display_name" in update_payload: update_payload["display_name"] = str(update_payload["display_name"]).strip() - if not update_payload["display_name"]: - raise ApiError(ErrorCode.VALIDATION_FAILED, "display_name is required", {"field": "display_name"}) after = self.repository.update_material(material_id, update_payload) if after is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "material not found", {"material_id": material_id}) @@ -194,6 +250,25 @@ class MaterialService: ) return after + def validate_material_payload(self, payload: dict[str, Any], *, creating: bool) -> None: + if creating or "display_name" in payload: + if not str(payload.get("display_name") or "").strip(): + raise ApiError(ErrorCode.VALIDATION_FAILED, "display_name is required", {"field": "display_name"}) + if creating or "category_id" in payload: + self._validate_category(payload.get("category_id")) + if creating or "image_uri" in payload: + self._validate_storage_uri(payload.get("image_uri"), "image_uri") + if creating or "thumbnail_uri" in payload: + self._validate_storage_uri(payload.get("thumbnail_uri"), "thumbnail_uri") + if creating or "recognition_status" in payload: + self._validate_status(payload.get("recognition_status", "pending_upload")) + metadata = payload.get("metadata") + if isinstance(metadata, dict) and any(key in metadata for key in ("raw_analysis", "normalized_asset_profile", "generation_capsules")): + try: + validate_material_structures(metadata) + except MaterialNormalizationError as exc: + raise ApiError(ErrorCode.VALIDATION_FAILED, str(exc), {"field": "metadata"}) from None + def delete_material(self, material_id: str, ctx: RequestContext) -> Material: return self.update_material(material_id, {"recognition_status": "archived"}, ctx) @@ -226,7 +301,7 @@ class MaterialService: "extension": extension, "size_bytes": declared_size, "max_size_bytes": MAX_UPLOAD_BYTES, - "mock": True, + "storage_reference_only": True, } }, }, @@ -243,25 +318,128 @@ class MaterialService: material = self.get_material(material_id) return self._recognize_material(material, ctx) + def renormalize_material(self, material_id: str, ctx: RequestContext) -> Material: + material = self.get_material(material_id) + raw = material.metadata.get("raw_analysis") if isinstance(material.metadata.get("raw_analysis"), dict) else {} + source_payload = raw.get("source_payload") if isinstance(raw.get("source_payload"), dict) else {"legacy": True} + recognition = material.metadata.get("recognition") if isinstance(material.metadata.get("recognition"), dict) else {} + structures = self.normalization_service.normalize_recognition( + source_payload, + prompt=material.prompt or str(raw.get("description") or "material reference"), + negative_prompt=material.negative_prompt, + tags=material.tags, + confidence=recognition.get("confidence"), + provider_id=str(recognition.get("provider_id") or "stored_analysis"), + model=str(recognition.get("model") or "stored_analysis"), + route_name=str(recognition.get("route_name") or "stored_analysis"), + category_hint=self._material_category_hint(material), + previous_metadata=material.metadata, + ) + after = self.update_material(material_id, {"metadata": {**material.metadata, **structures}}, ctx) + self.audit_service.record( + ctx=ctx, + action="renormalize_material", + target_type="material", + target_id=material_id, + before_snapshot=material.to_dict(), + after_snapshot=after.to_dict(), + metadata={"vision_provider_called": False}, + ) + return after + + def renormalize_batch(self, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: + requested = [str(item).strip() for item in payload.get("material_ids", []) if str(item).strip()] if isinstance(payload.get("material_ids"), list) else [] + if not requested: + items: list[Material] = [] + page = 1 + while True: + batch, total = self.repository.list_materials(page=page, page_size=100) + items.extend(batch) + if len(items) >= total or not batch: + break + page += 1 + requested = [item.material_id for item in items if item.recognition_status in {"pending_confirm", "confirmed"}] + limit = min(max(int(payload.get("limit") or 500), 1), 5000) + succeeded: list[str] = [] + failed: list[dict[str, str]] = [] + for material_id in requested[:limit]: + try: + self.renormalize_material(material_id, ctx) + succeeded.append(material_id) + except Exception as exc: + failed.append({"material_id": material_id, "error_type": exc.__class__.__name__}) + return {"requested": min(len(requested), limit), "succeeded": succeeded, "failed": failed, "vision_provider_called": False} + def _recognize_material(self, material: Material, ctx: RequestContext) -> Material: - recognition, provider_error, fallback_used, route_name = self._recognize_with_adapter(material) + route_name = _recognition_route(material.metadata, self.default_recognition_route) + provider = self._vision_provider(material, route_name) + if provider is None: + raise ApiError( + ErrorCode.MODEL_CAPABILITY_UNCONFIRMED, + "material vision route is not configured or its credentials are unavailable", + { + "material_id": material.material_id, + "route_name": route_name, + "modality": "vision_recognition", + }, + ) + + request = _vision_request(material, self.storage, self._recognition_instruction(material)) + try: + recognition = provider.recognize_material(request) + except VisionProviderError as exc: + failure = _vision_provider_error(exc) + self._record_recognition_failure(material, route_name, failure, ctx) + raise _vision_api_error(material.material_id, route_name, failure) from None + except Exception as exc: + failure = _unexpected_vision_provider_error(provider, exc) + self._record_recognition_failure(material, route_name, failure, ctx) + raise _vision_api_error(material.material_id, route_name, failure) from None + generated_tags = recognition.tags - prompt = material.prompt or recognition.prompt or _mock_prompt(material) + metadata = { + key: value + for key, value in material.metadata.items() + if key not in {"confirmed", "recognition", "recognition_failure"} + } + category_hint = self._material_category_hint(material) + try: + structures = self.normalization_service.normalize_recognition( + recognition.structured_data, + prompt=recognition.prompt, + negative_prompt=recognition.negative_prompt, + tags=generated_tags, + confidence=recognition.confidence, + provider_id=recognition.provider_id, + model=recognition.model, + route_name=route_name, + category_hint=category_hint, + previous_metadata=material.metadata, + ) + except MaterialNormalizationError as exc: + failure = { + "source": "material_normalization", + "provider_id": recognition.provider_id, + "status_code": None, + "error_type": "invalid_structured_analysis", + "reason": str(exc), + } + self._record_recognition_failure(material, route_name, failure, ctx) + raise _vision_api_error(material.material_id, route_name, failure) from None after = self.update_material( material.material_id, { - "prompt": prompt, - "negative_prompt": material.negative_prompt or recognition.negative_prompt or "body, face, background, watermark", + "prompt": material.prompt or recognition.prompt, + "negative_prompt": material.negative_prompt or recognition.negative_prompt, "tags": sorted(set(material.tags + generated_tags)), "recognition_status": "pending_confirm", "metadata": { - **material.metadata, + **metadata, "recognition": _recognition_metadata( recognition, route_name=route_name, - fallback_used=fallback_used, - provider_error=provider_error, ), + **structures, }, }, ctx, @@ -276,39 +454,100 @@ class MaterialService: ) return after - def _recognize_with_adapter( + def _recognition_instruction(self, material: Material) -> str: + base = str(self.recognition_prompt_config.get("default_instruction") or "").strip() + requirements = self.recognition_prompt_config.get("category_requirements") + if not isinstance(requirements, list): + return base + category_name = "" + if material.category_id: + category = self.repository.get_category(material.category_id) + category_name = category.name if category else "" + additions: list[str] = [] + for item in requirements: + if not isinstance(item, dict) or item.get("enabled") is False: + continue + category = str(item.get("category") or "").strip().lower() + requirement = str(item.get("requirement") or "").strip() + if requirement and (not category or category in {str(material.category_id or "").lower(), category_name.lower()}): + additions.append(requirement) + return "\n\n".join([part for part in [base, *additions] if part]) + + def _vision_provider(self, material: Material, route_name: str) -> VisionProvider | None: + if self.vision_provider_factory is None: + return None + return self.vision_provider_factory( + { + "route_name": route_name, + "modality": "vision_recognition", + "material_id": material.material_id, + "image_uri": material.image_uri, + "thumbnail_uri": material.thumbnail_uri, + } + ) + + def _record_recognition_failure( self, material: Material, - ) -> tuple[VisionRecognitionResult, dict[str, Any] | None, bool, str]: - route_name = _recognition_route(material.metadata, self.default_recognition_route) - request = _vision_request(material) - provider = None - if self.vision_provider_factory: - provider = self.vision_provider_factory( - { - "route_name": route_name, - "modality": "vision_recognition", - "material_id": material.material_id, - "image_uri": material.image_uri, - "thumbnail_uri": material.thumbnail_uri, - } - ) - if provider is None: - return _mock_vision_recognition(material), None, False, route_name - try: - return provider.recognize_material(request), None, False, route_name - except VisionProviderError as exc: - return _mock_vision_recognition(material), _vision_provider_error(exc), True, route_name + route_name: str, + failure: dict[str, Any], + ctx: RequestContext, + ) -> Material: + metadata = { + key: value + for key, value in material.metadata.items() + if key not in {"confirmed", "recognition", "recognition_failure"} + } + after = self.update_material( + material.material_id, + { + "recognition_status": "pending_recognition", + "metadata": { + **metadata, + "recognition_failure": {**failure, "route_name": route_name}, + }, + }, + ctx, + ) + self.audit_service.record( + ctx=ctx, + action="recognize_material_failed", + target_type="material", + target_id=material.material_id, + before_snapshot=material.to_dict(), + after_snapshot=after.to_dict(), + metadata={"route_name": route_name, "error_type": failure.get("error_type")}, + ) + return after def confirm_material(self, material_id: str, payload: dict[str, Any], ctx: RequestContext) -> Material: material = self.get_material(material_id) + if material.recognition_status not in {"pending_confirm", "confirmed"}: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "material must have a successful recognition result before confirmation", + { + "material_id": material_id, + "recognition_status": material.recognition_status, + "required_status": "pending_confirm", + }, + ) + metadata_patch = payload.get("metadata") if isinstance(payload.get("metadata"), dict) else {} + metadata = {**material.metadata, **metadata_patch, "confirmed": True} + profile = metadata.get("normalized_asset_profile") + if isinstance(profile, dict): + profile = {**profile, "confirmed": True} + for key in ("category", "role", "body_slots", "wear_layer", "must_preserve", "forbidden_changes", "stackable", "user_locked"): + if key in payload: + profile[key] = payload[key] + metadata["normalized_asset_profile"] = profile update_payload = { "recognition_status": "confirmed", "display_name": payload.get("display_name", material.display_name), "prompt": payload.get("prompt", material.prompt), "negative_prompt": payload.get("negative_prompt", material.negative_prompt), "tags": payload.get("tags", material.tags), - "metadata": {**material.metadata, "confirmed": True}, + "metadata": metadata, } after = self.update_material(material_id, update_payload, ctx) self.audit_service.record( @@ -321,6 +560,24 @@ class MaterialService: ) return after + def _material_category_hint(self, material: Material) -> str | None: + if not material.category_id: + return None + category = self.repository.get_category(material.category_id) + return category.name if category else None + + def _with_material_structures(self, material: Material) -> Material: + if material.recognition_status not in {"pending_confirm", "confirmed"}: + return material + metadata = self.normalization_service.ensure_legacy_structures( + prompt=material.prompt, + negative_prompt=material.negative_prompt, + tags=material.tags, + metadata=material.metadata, + category_hint=self._material_category_hint(material), + ) + return material if metadata == material.metadata else replace(material, metadata=metadata) + def _validate_category(self, category_id: Any) -> None: if category_id and self.repository.get_category(str(category_id)) is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "category not found", {"category_id": category_id}) @@ -348,34 +605,98 @@ def new_material_id() -> str: return f"mat_{uuid4().hex}" -def _mock_prompt(material: Material) -> str: - base = material.display_name.strip().lower().replace("_", " ") - return f"{base}, clean product reference, transparent background" - - -def _mock_tags(material: Material) -> list[str]: - text = f"{material.display_name} {material.prompt or ''}".lower() - tags = ["material"] - for keyword in ("white", "blue", "black", "red", "ribbon", "shirt", "blouse", "dress", "skirt", "shoe"): - if keyword in text: - tags.append(keyword) - return tags - - -def _vision_request(material: Material) -> VisionRecognitionRequest: +def _vision_request(material: Material, storage: VfsStorage, recognition_instruction: str = "") -> VisionRecognitionRequest: + image_data_url, content_type, size_bytes = _storage_image_data_url( + str(material.image_uri or ""), + material.metadata, + storage, + ) return VisionRecognitionRequest( display_name=material.display_name, - image_uri=str(material.image_uri or ""), - thumbnail_uri=material.thumbnail_uri, + image_uri=image_data_url, + thumbnail_uri=None, prompt=material.prompt, negative_prompt=material.negative_prompt, tags=tuple(material.tags), - metadata=material.metadata, + metadata={ + **material.metadata, + "recognition_instruction": recognition_instruction, + "vision_input": { + "source_uri": material.image_uri, + "content_type": content_type, + "size_bytes": size_bytes, + }, + }, ) -def _mock_vision_recognition(material: Material) -> VisionRecognitionResult: - return MockVisionProvider().recognize_material(_vision_request(material)) +def _storage_image_data_url( + image_uri: str, + metadata: dict[str, Any], + storage: VfsStorage, +) -> tuple[str, str, int]: + if not image_uri: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "material image_uri is required for recognition", + {"field": "image_uri"}, + ) + path = storage.ensure_file(image_uri) + try: + reported_size = path.stat().st_size + except OSError: + raise ApiError(ErrorCode.MEDIA_NOT_FOUND, "material image could not be inspected", {"uri": image_uri}) from None + if reported_size > MAX_UPLOAD_BYTES: + raise ApiError( + ErrorCode.UPLOAD_TOO_LARGE, + "material image exceeds the recognition size limit", + {"uri": image_uri, "size_bytes": reported_size, "max_size_bytes": MAX_UPLOAD_BYTES}, + ) + try: + with path.open("rb") as handle: + content = handle.read(MAX_UPLOAD_BYTES + 1) + except OSError: + raise ApiError(ErrorCode.MEDIA_NOT_FOUND, "material image could not be read", {"uri": image_uri}) from None + if len(content) > MAX_UPLOAD_BYTES: + raise ApiError( + ErrorCode.UPLOAD_TOO_LARGE, + "material image exceeds the recognition size limit", + {"uri": image_uri, "size_bytes": len(content), "max_size_bytes": MAX_UPLOAD_BYTES}, + ) + content_type = _image_content_type(content) + if content_type is None: + raise ApiError( + ErrorCode.UNSUPPORTED_FILE_TYPE, + "material image bytes do not match a supported image format", + {"uri": image_uri, "allowed": sorted(UPLOAD_ALLOWED_CONTENT_TYPES)}, + ) + + extension = _suffix(image_uri) + if extension and extension not in UPLOAD_ALLOWED_CONTENT_TYPES[content_type]: + raise ApiError( + ErrorCode.UNSUPPORTED_FILE_TYPE, + "material image extension does not match its file signature", + {"uri": image_uri, "extension": extension, "detected_content_type": content_type}, + ) + upload = metadata.get("upload") if isinstance(metadata.get("upload"), dict) else {} + declared_content_type = _optional_text(upload.get("content_type")) + if declared_content_type and declared_content_type != content_type: + raise ApiError( + ErrorCode.UNSUPPORTED_FILE_TYPE, + "material image content type does not match its file signature", + { + "uri": image_uri, + "declared_content_type": declared_content_type, + "detected_content_type": content_type, + }, + ) + encoded = base64.b64encode(content).decode("ascii") + return f"data:{content_type};base64,{encoded}", content_type, len(content) + + +def _image_content_type(content: bytes) -> str | None: + image_format = detect_image_format(content) + return image_format.mime_type if image_format is not None else None def _recognition_route(metadata: dict[str, Any], default_route: str) -> str: @@ -392,8 +713,6 @@ def _recognition_metadata( recognition: VisionRecognitionResult, *, route_name: str, - fallback_used: bool, - provider_error: dict[str, Any] | None, ) -> dict[str, Any]: metadata = { "engine": "mock_vlm" if recognition.provider_id == "mock" else "vlm_adapter", @@ -401,26 +720,52 @@ def _recognition_metadata( "model": recognition.model, "confidence": recognition.confidence, "route_name": route_name, - "fallback_used": fallback_used, + "fallback_used": False, "description": recognition.description, "raw_metadata": recognition.raw_metadata, } - if provider_error: - metadata["provider_error"] = provider_error return metadata def _vision_provider_error(exc: VisionProviderError) -> dict[str, Any]: + failure = { + "source": "vision_provider", + "provider_id": _optional_text(exc.provider_id) or "unknown", + "status_code": exc.status_code, + "error_type": _optional_text(exc.error_type) or "provider_error", + } + if isinstance(exc.details.get("retryable"), bool): + failure["retryable"] = exc.details["retryable"] + return failure + + +def _unexpected_vision_provider_error(provider: VisionProvider, exc: Exception) -> dict[str, Any]: return { "source": "vision_provider", - "provider_id": exc.provider_id, - "status_code": exc.status_code, - "error_type": exc.error_type, - "message": str(exc), - "details": exc.details, + "provider_id": _optional_text(getattr(provider, "provider_id", None)) or "unknown", + "status_code": None, + "error_type": "unexpected_provider_error", + "exception_type": exc.__class__.__name__, } +def _vision_api_error(material_id: str, route_name: str, failure: dict[str, Any]) -> ApiError: + details = { + "material_id": material_id, + "route_name": route_name, + "provider_id": failure.get("provider_id"), + "status_code": failure.get("status_code"), + "error_type": failure.get("error_type"), + } + if "retryable" in failure: + details["retryable"] = failure["retryable"] + return ApiError( + ErrorCode.GENERATION_FAILED, + "material vision provider request failed", + details, + ) + + def _normalize_upload_filename(value: Any) -> str | None: if value is None: return None @@ -522,3 +867,42 @@ def _display_name_from_filename(filename: str | None) -> str | None: def _optional_text(value: Any) -> str | None: text = str(value or "").strip() return text or None + + +def _matches_material_filters( + material: Material, + *, + color: str | None, + material_type: str | None, + body_slot: str | None, + wear_layer: str | None, + min_confidence: float | None, + confirmation_state: str | None, +) -> bool: + profile = material.metadata.get("normalized_asset_profile") if isinstance(material.metadata.get("normalized_asset_profile"), dict) else {} + attributes = profile.get("attributes") if isinstance(profile.get("attributes"), dict) else {} + searchable = " ".join( + [material.display_name, material.prompt or "", " ".join(material.tags), str(attributes)] + ).casefold() + if color and str(color).casefold() not in searchable: + return False + if material_type and str(material_type).casefold() not in searchable: + return False + if body_slot and str(body_slot) not in (profile.get("body_slots") or []): + return False + if wear_layer and str(profile.get("wear_layer") or "") != str(wear_layer): + return False + if min_confidence is not None: + try: + if float(profile.get("confidence") or 0.0) < float(min_confidence): + return False + except (TypeError, ValueError): + return False + state = str(confirmation_state or "").lower() + if state == "confirmed" and not profile.get("confirmed"): + return False + if state == "needs_confirmation" and not profile.get("requires_confirmation"): + return False + if state == "unconfirmed" and profile.get("confirmed"): + return False + return True diff --git a/backend/services/model_provider_bootstrap.py b/backend/services/model_provider_bootstrap.py new file mode 100644 index 0000000..6e87648 --- /dev/null +++ b/backend/services/model_provider_bootstrap.py @@ -0,0 +1,375 @@ +"""Bootstrap image-generation and material-vision routes from configuration.""" + +from __future__ import annotations + +from collections.abc import Mapping +import re +from typing import Any + +from backend.services.model_provider_service import ModelProviderService + + +def sync_generation_provider_from_config( + service: ModelProviderService, + config: Mapping[str, Any], +) -> dict[str, Any]: + generation = _mapping(config.get("generation")) + default_route = _text(generation.get("default_model")) + configured_adapters = generation.get("adapters") + adapters = [ + _mapping(item) + for item in configured_adapters + if isinstance(item, Mapping) + ] if isinstance(configured_adapters, list) else [] + if not adapters: + legacy_provider = _mapping(generation.get("provider")) + if legacy_provider: + adapters = [legacy_provider] + if not adapters and not default_route: + _disable_config_routes(service, "image_generation") + return {"configured": False, "status": "not_configured"} + + ready: list[dict[str, Any]] = [] + errors: list[dict[str, Any]] = [] + active_route_ids: set[str] = set() + for index, provider in enumerate(adapters): + provider_id = _text(provider.get("provider_id") or provider.get("id")) or f"imagestudio_adapter_{index + 1}" + route_name = _text(provider.get("route_name")) or (default_route if index == 0 else provider_id) + model_id = _text(provider.get("model_id")) or route_name + base_url = _text(provider.get("base_url")) + missing = [ + field + for field, value in ( + ("route_name", route_name), + ("model_id", model_id), + ("base_url", base_url), + ) + if not value + ] + if missing: + errors.append({"adapter_id": provider_id, "index": index, "missing_fields": missing}) + continue + provider_payload: dict[str, Any] = { + "provider_id": provider_id, + "display_name": _text(provider.get("display_name") or provider.get("name")) or provider_id, + "provider_type": _text(provider.get("provider_type")) or "openai_compatible", + "base_url": base_url, + "image_endpoint": _text(provider.get("image_endpoint")) or "/v1/images/generations", + "timeout_seconds": _positive_int(provider.get("timeout_seconds"), 180), + "enabled": _bool(provider.get("enabled", True)), + "metadata": {"source": "config_bootstrap", "adapter_index": index}, + } + api_key_env_var = _text(provider.get("api_key_env_var")) + api_key_secret_ref = _text(provider.get("api_key_secret_ref")) + if api_key_env_var: + provider_payload["api_key_env_var"] = api_key_env_var + if api_key_secret_ref: + provider_payload["api_key_secret_ref"] = api_key_secret_ref + service.upsert_provider(provider_payload) + route = _upsert_config_route( + service, + { + "route_name": route_name, + "provider_id": provider_id, + "model_id": model_id, + "modality": "image_generation", + "priority": _int(provider.get("priority"), 10 + index), + "enabled": _bool(provider.get("enabled", True)), + "metadata": { + "source": "config_bootstrap", + "adapter_index": index, + "is_default": index == 0 or route_name == default_route, + }, + }, + disable_others=False, + ) + active_route_ids.add(str(route["route_id"])) + ready.append( + { + "provider_id": provider_id, + "route_name": route["route_name"], + "model_id": route["model_id"], + "enabled": bool(route["enabled"]), + "api_key_source": "env" if api_key_env_var else "secret_ref" if api_key_secret_ref else "none", + } + ) + _disable_config_routes_except(service, "image_generation", active_route_ids) + if not ready: + _disable_config_routes(service, "image_generation") + missing_fields = [ + f"generation.provider.{field}" + for field in ((errors[0].get("missing_fields") if errors else ["base_url", "model_id"])) + ] + return { + "configured": False, + "status": "incomplete", + "missing_fields": missing_fields, + "adapter_errors": errors, + } + default = next((item for item in ready if item["route_name"] == default_route), ready[0]) + return { + "configured": True, + "status": "ready" if not errors else "partial", + **default, + "adapter_count": len(ready), + "adapters": ready, + "adapter_errors": errors, + } + + +def sync_vision_provider_from_config( + service: ModelProviderService, + config: Mapping[str, Any], +) -> dict[str, Any]: + """Persist the configured material-recognition route. + + Vision settings may omit provider connection fields. In that case they + inherit the generation provider's ID, base URL, key reference, timeout and + compatible image endpoint. The vision model remains explicit because an + image-generation model is not necessarily multimodal. + """ + + generation = _mapping(config.get("generation")) + generation_provider = _mapping(generation.get("provider")) + vision = _mapping(config.get("vision")) + vision_provider = _mapping(vision.get("provider")) + + default_route = _text(vision.get("default_route")) + route_name = _text(vision_provider.get("route_name")) or default_route + model_id = _text(vision_provider.get("model_id")) + base_url = _text(vision_provider.get("base_url")) or _text(generation_provider.get("base_url")) + vision_endpoint = _text(vision_provider.get("vision_endpoint")) or "/v1/chat/completions" + + configured_values = ( + model_id, + _text(vision_provider.get("base_url")), + _text(vision_provider.get("provider_id")), + _text(vision_provider.get("api_key_env_var")), + _text(vision_provider.get("api_key_secret_ref")), + ) + if not any(configured_values): + _disable_config_routes(service, "vision_recognition") + return { + "configured": False, + "status": "not_configured", + "route_name": route_name or None, + } + + missing = [ + field + for field, value in ( + ("vision.default_route", route_name), + ("vision.provider.model_id", model_id), + ("vision.provider.base_url", base_url), + ) + if not value + ] + if missing: + _disable_config_routes(service, "vision_recognition") + return { + "configured": False, + "status": "incomplete", + "route_name": route_name or None, + "missing_fields": missing, + } + + generation_provider_id = _text(generation_provider.get("provider_id")) or "imagestudio_default" + requested_provider_id = _text(vision_provider.get("provider_id")) + generation_api_key_env_var = _text(generation_provider.get("api_key_env_var")) + generation_api_key_secret_ref = _text(generation_provider.get("api_key_secret_ref")) + api_key_env_var = _text(vision_provider.get("api_key_env_var")) or generation_api_key_env_var + api_key_secret_ref = _text(vision_provider.get("api_key_secret_ref")) or generation_api_key_secret_ref + generation_timeout = _positive_int(generation_provider.get("timeout_seconds"), 180) + vision_timeout = _positive_int( + vision_provider.get("timeout_seconds", generation_provider.get("timeout_seconds")), + 180, + ) + connection_differs = any( + ( + requested_provider_id and requested_provider_id != generation_provider_id, + _text(vision_provider.get("base_url")) + and _text(vision_provider.get("base_url")) != _text(generation_provider.get("base_url")), + _text(vision_provider.get("api_key_env_var")) + and api_key_env_var != generation_api_key_env_var, + _text(vision_provider.get("api_key_secret_ref")) + and api_key_secret_ref != generation_api_key_secret_ref, + _text(vision_provider.get("provider_type")) + and _text(vision_provider.get("provider_type")) + != (_text(generation_provider.get("provider_type")) or "openai_compatible"), + _text(vision_provider.get("display_name")) + and _text(vision_provider.get("display_name")) != _text(generation_provider.get("display_name")), + vision_timeout != generation_timeout, + ) + ) + reuses_generation_provider = not connection_differs + provider_id = ( + generation_provider_id + if reuses_generation_provider + else requested_provider_id or f"{generation_provider_id}_vision" + ) + route_enabled = _bool(vision_provider.get("enabled", True)) + provider_enabled = ( + _bool(generation_provider.get("enabled", True)) + if reuses_generation_provider + else route_enabled + ) + provider_payload: dict[str, Any] = { + "provider_id": provider_id, + "display_name": ( + _text(vision_provider.get("display_name")) + or _text(generation_provider.get("display_name")) + or "ImageStudio vision provider" + ), + "provider_type": ( + _text(vision_provider.get("provider_type")) + or _text(generation_provider.get("provider_type")) + or "openai_compatible" + ), + "base_url": base_url, + "image_endpoint": _text(generation_provider.get("image_endpoint")) or "/v1/images/generations", + "timeout_seconds": vision_timeout if not reuses_generation_provider else generation_timeout, + "enabled": provider_enabled, + "metadata": {"source": "config_bootstrap", "vision_enabled": True}, + } + if api_key_env_var: + provider_payload["api_key_env_var"] = api_key_env_var + if api_key_secret_ref: + provider_payload["api_key_secret_ref"] = api_key_secret_ref + service.upsert_provider(provider_payload) + route = _upsert_config_route( + service, + { + "route_name": route_name, + "provider_id": provider_id, + "model_id": model_id, + "modality": "vision_recognition", + "priority": _int(vision_provider.get("priority"), 10), + "enabled": route_enabled, + "settings": {"vision_endpoint": vision_endpoint}, + "metadata": { + "source": "config_bootstrap", + "reuses_generation_provider": reuses_generation_provider, + }, + } + ) + return { + "configured": True, + "status": "ready", + "provider_id": provider_id, + "route_name": route["route_name"], + "model_id": route["model_id"], + "vision_endpoint": vision_endpoint, + "reuses_generation_provider": reuses_generation_provider, + "provider_id_auto_derived": bool(not reuses_generation_provider and not requested_provider_id), + "api_key_source": "env" if api_key_env_var else "secret_ref" if api_key_secret_ref else "none", + } + + +def _upsert_config_route( + service: ModelProviderService, + payload: dict[str, Any], + *, + disable_others: bool = True, +) -> dict[str, Any]: + """Keep a single active config-owned route across provider/model changes.""" + + route_name = _text(payload.get("route_name")) + modality = _text(payload.get("modality")) or "image_generation" + stable_route_id = _config_route_id(route_name, modality) + route = service.upsert_route({**payload, "route_id": stable_route_id}) + if disable_others: + _disable_config_routes(service, modality, except_route_id=stable_route_id) + return route + + +def _disable_config_routes_except( + service: ModelProviderService, + modality: str, + active_route_ids: set[str], +) -> None: + all_routes = service.list_routes() + active_by_name = { + str(route.get("route_name")): str(route.get("route_id")) + for route in all_routes + if route.get("route_id") in active_route_ids + } + for existing in all_routes: + metadata = _mapping(existing.get("metadata")) + if ( + existing.get("route_id") in active_route_ids + or existing.get("modality") != modality + or metadata.get("source") != "config_bootstrap" + or not existing.get("enabled") + ): + continue + service.upsert_route( + { + **existing, + "enabled": False, + "metadata": { + **metadata, + **( + {"superseded_by": active_by_name[str(existing.get("route_name"))]} + if str(existing.get("route_name")) in active_by_name + else {"disabled_by": "config"} + ), + }, + } + ) + + +def _disable_config_routes( + service: ModelProviderService, + modality: str, + *, + except_route_id: str | None = None, +) -> None: + for existing in service.list_routes(): + metadata = _mapping(existing.get("metadata")) + if ( + existing.get("route_id") == except_route_id + or existing.get("modality") != modality + or metadata.get("source") != "config_bootstrap" + or not existing.get("enabled") + ): + continue + service.upsert_route( + { + **existing, + "enabled": False, + "metadata": { + **metadata, + **({"superseded_by": except_route_id} if except_route_id else {"disabled_by": "config"}), + }, + } + ) + + +def _config_route_id(route_name: str, modality: str) -> str: + stem = re.sub(r"[^A-Za-z0-9_]+", "_", f"{modality}_{route_name}").strip("_").lower() + return f"route_config_{stem or 'model'}" + + +def _mapping(value: Any) -> dict[str, Any]: + return dict(value) if isinstance(value, Mapping) else {} + + +def _text(value: Any) -> str: + return str(value or "").strip() + + +def _bool(value: Any) -> bool: + if isinstance(value, str): + return value.strip().lower() in {"1", "true", "yes", "on", "enabled"} + return bool(value) + + +def _int(value: Any, default: int) -> int: + try: + return int(value) + except (TypeError, ValueError): + return default + + +def _positive_int(value: Any, default: int) -> int: + return max(_int(value, default), 1) diff --git a/backend/services/model_provider_runtime.py b/backend/services/model_provider_runtime.py index f22d3de..02f8d34 100644 --- a/backend/services/model_provider_runtime.py +++ b/backend/services/model_provider_runtime.py @@ -2,11 +2,18 @@ from __future__ import annotations +import base64 +import json +import urllib.request from typing import Any +from urllib.parse import urlsplit from backend.services.model_provider_service import ModelProviderService from core.secret_service import SecretStore, resolve_secret_value -from engines.image_provider import ImageProvider, ImageProviderConfig, OpenAIImageProvider +from core.remote_fetch import read_http_response_limited, validate_http_url +from core.image_formats import detect_image_format +from core.storage import VfsStorage +from engines.image_provider import FallbackImageProvider, ImageProvider, ImageProviderConfig, OpenAIImageProvider from engines.vision_provider import OpenAIVisionProvider, VisionProvider, VisionProviderConfig @@ -27,6 +34,29 @@ class ModelProviderRuntime: resolved = self.model_provider_service.resolve_route(route_name, modality="image_generation") if not resolved: return None + candidates = [resolved] + fallback_enabled = bool(route_context.get("allow_provider_fallback", True)) + if fallback_enabled: + seen = {str(resolved.get("route_id"))} + excluded_routes = {str(item).strip() for item in route_context.get("excluded_fallback_routes", []) if str(item).strip()} if isinstance(route_context.get("excluded_fallback_routes"), list) else set() + fallback_routes = sorted( + self.model_provider_service.list_routes(enabled=True), + key=lambda item: (int(item.get("priority") or 100), str(item.get("route_id") or "")), + ) + for route in fallback_routes: + if str(route.get("route_name") or "") in excluded_routes: + continue + if route.get("modality") != "image_generation" or str(route.get("route_id")) in seen: + continue + candidate = self.model_provider_service.resolve_route(str(route.get("route_name")), modality="image_generation") + if candidate: + candidates.append(candidate); seen.add(str(candidate.get("route_id"))) + providers = [provider for candidate in candidates if (provider := self._image_provider_from_resolved(candidate))] + if not providers: + return None + return providers[0] if len(providers) == 1 else FallbackImageProvider(providers) + + def _image_provider_from_resolved(self, resolved: dict[str, Any]) -> ImageProvider | None: provider = resolved.get("provider") or {} if str(provider.get("provider_type") or "openai_compatible") != "openai_compatible": return None @@ -35,18 +65,28 @@ class ModelProviderRuntime: secret_ref=provider.get("api_key_secret_ref"), secret_store=self.secret_store, ) - if not secret.value: + secret_configured = bool(provider.get("api_key_env_var") or provider.get("api_key_secret_ref")) + if secret_configured and not secret.value: return None + settings = dict(resolved.get("settings") or {}) + capabilities = dict(provider.get("capabilities") or {}) + base_url = str(provider.get("base_url") or "") return OpenAIImageProvider( ImageProviderConfig( provider_id=str(resolved["provider_id"]), - base_url=str(provider.get("base_url") or ""), - api_key=secret.value, + base_url=base_url, + api_key=secret.value or "", image_model=str(resolved["model_id"]), image_endpoint=str(provider.get("image_endpoint") or "/v1/images/generations"), provider_type=str(provider.get("provider_type") or "openai_compatible"), timeout_seconds=int(provider.get("timeout_seconds") or 180), extra_headers=dict(provider.get("extra_headers") or {}), + supports_negative_prompt=_configured_bool( + settings.get("supports_negative_prompt"), + capabilities.get("supports_negative_prompt"), + capabilities.get("negative_prompt"), + default=_default_negative_prompt_support(base_url), + ), ) ) @@ -65,15 +105,19 @@ class ModelProviderRuntime: secret_ref=provider.get("api_key_secret_ref"), secret_store=self.secret_store, ) - if not secret.value: + secret_configured = bool(provider.get("api_key_env_var") or provider.get("api_key_secret_ref")) + if secret_configured and not secret.value: + return None + base_url = str(provider.get("base_url") or "").strip() + if not base_url: return None settings = dict(resolved.get("settings") or {}) capabilities = dict(provider.get("capabilities") or {}) return OpenAIVisionProvider( VisionProviderConfig( provider_id=str(resolved["provider_id"]), - base_url=str(provider.get("base_url") or ""), - api_key=secret.value, + base_url=base_url, + api_key=secret.value or "", vision_model=str(resolved["model_id"]), vision_endpoint=str( settings.get("vision_endpoint") @@ -85,3 +129,156 @@ class ModelProviderRuntime: extra_headers=dict(provider.get("extra_headers") or {}), ) ) + + def structured_text_callable(self, route_name: str): + """Return an OpenAI-compatible JSON completion callable for a configured route.""" + resolved = self.model_provider_service.resolve_route(route_name) + if not resolved: + return None + provider = resolved.get("provider") or {} + if str(provider.get("provider_type") or "openai_compatible") != "openai_compatible": + return None + secret = resolve_secret_value( + env_var=provider.get("api_key_env_var"), + secret_ref=provider.get("api_key_secret_ref"), + secret_store=self.secret_store, + ) + if (provider.get("api_key_env_var") or provider.get("api_key_secret_ref")) and not secret.value: + return None + base_url = str(provider.get("base_url") or "").rstrip("/") + if not base_url: + return None + settings = dict(resolved.get("settings") or {}) + capabilities = dict(provider.get("capabilities") or {}) + endpoint = str(settings.get("chat_endpoint") or capabilities.get("chat_endpoint") or "/v1/chat/completions") + url = f"{base_url}{endpoint if endpoint.startswith('/') else '/' + endpoint}" + validate_http_url(url, allow_private=True) + headers = { + "Content-Type": "application/json", + **({"Authorization": f"Bearer {secret.value}"} if secret.value else {}), + **dict(provider.get("extra_headers") or {}), + } + timeout = int(provider.get("timeout_seconds") or 180) + model = str(resolved.get("model_id") or "") + + def complete(request_payload: dict[str, Any]) -> dict[str, Any]: + purpose = str(request_payload.get("purpose") or "structured prompt processing") + body = { + "model": model, + "messages": [ + {"role": "system", "content": f"Perform {purpose}. Return one JSON object only. Preserve safety-relevant meaning and do not invent facts."}, + {"role": "user", "content": json.dumps(request_payload, ensure_ascii=False)}, + ], + "temperature": 0.1, + "response_format": {"type": "json_object"}, + } + raw = read_http_response_limited( + urllib.request.Request(url, data=json.dumps(body, ensure_ascii=False).encode("utf-8"), headers=headers, method="POST"), + timeout_seconds=timeout, + max_bytes=4 * 1024 * 1024, + allow_private=True, + same_origin_redirects_only=True, + ).decode("utf-8-sig") + response = json.loads(raw) + content = _completion_content(response) + parsed = json.loads(content) + if not isinstance(parsed, dict): + raise ValueError("structured completion must return an object") + return parsed + + return complete + + def structured_vision_callable(self, route_name: str, *, storage: VfsStorage | None = None): + resolved = self.model_provider_service.resolve_route(route_name) + if not resolved: + return None + provider = resolved.get("provider") or {} + secret = resolve_secret_value(env_var=provider.get("api_key_env_var"), secret_ref=provider.get("api_key_secret_ref"), secret_store=self.secret_store) + if (provider.get("api_key_env_var") or provider.get("api_key_secret_ref")) and not secret.value: + return None + base_url = str(provider.get("base_url") or "").rstrip("/") + if not base_url: + return None + settings = dict(resolved.get("settings") or {}) + capabilities = dict(provider.get("capabilities") or {}) + endpoint = str(settings.get("vision_endpoint") or capabilities.get("vision_endpoint") or settings.get("chat_endpoint") or "/v1/chat/completions") + url = f"{base_url}{endpoint if endpoint.startswith('/') else '/' + endpoint}" + validate_http_url(url, allow_private=True) + headers = {"Content-Type": "application/json", **({"Authorization": f"Bearer {secret.value}"} if secret.value else {}), **dict(provider.get("extra_headers") or {})} + timeout = int(provider.get("timeout_seconds") or 180) + model = str(resolved.get("model_id") or "") + vfs = storage or VfsStorage() + + def review(request_payload: dict[str, Any]) -> dict[str, Any]: + image = request_payload.get("image") if isinstance(request_payload.get("image"), dict) else {} + uri = str(image.get("image_uri") or image.get("thumbnail_uri") or "").strip() + if uri.startswith("storage://"): + path = vfs.ensure_file(uri) + body = path.read_bytes() + if len(body) > 25 * 1024 * 1024: + raise ValueError("image moderation input exceeds size limit") + image_format = detect_image_format(body) + if image_format is None: + raise ValueError("image moderation input is not a supported image") + uri = f"data:{image_format.mime_type};base64,{base64.b64encode(body).decode('ascii')}" + if not uri.startswith(("data:image/", "http://", "https://")): + raise ValueError("image moderation requires a storage, data, or HTTP image URI") + instruction = { + "requested_level": request_payload.get("requested_level"), + "route_kind": request_payload.get("route_kind"), + "reference_binding_mapping": (image.get("metadata") or {}).get("reference_binding_mapping") if isinstance(image.get("metadata"), dict) else {}, + "actions": ["allow", "warn", "quarantine", "regenerate", "reject"], + "requirements": ["detect age ambiguity", "detect unexpected nudity", "detect terrorism or high-risk violence", "detect content beyond the allowed request", "when material IDs are available, return material_restoration_scores from 0 to 1"], + } + payload = { + "model": model, + "messages": [ + {"role": "system", "content": "Review the generated image for content safety. Return JSON with action, categories, confidence, reason_codes, and version. Terrorism, high-risk violence, sexual minors, and extreme gore must be reject or quarantine."}, + {"role": "user", "content": [{"type": "text", "text": json.dumps(instruction, ensure_ascii=False)}, {"type": "image_url", "image_url": {"url": uri}}]}, + ], + "temperature": 0.0, + "response_format": {"type": "json_object"}, + } + raw = read_http_response_limited(urllib.request.Request(url, data=json.dumps(payload, ensure_ascii=False).encode("utf-8"), headers=headers, method="POST"), timeout_seconds=timeout, max_bytes=4 * 1024 * 1024, allow_private=True, same_origin_redirects_only=True).decode("utf-8-sig") + parsed = json.loads(_completion_content(json.loads(raw))) + if not isinstance(parsed, dict): + raise ValueError("image moderation response must be an object") + return parsed + + return review + + +def _configured_bool(*values: Any, default: bool) -> bool: + for value in values: + if value is None: + continue + if isinstance(value, str): + normalized = value.strip().lower() + if normalized in {"1", "true", "yes", "on", "enabled"}: + return True + if normalized in {"0", "false", "no", "off", "disabled"}: + return False + return bool(value) + return default + + +def _default_negative_prompt_support(base_url: str) -> bool: + try: + hostname = str(urlsplit(base_url).hostname or "").lower() + except ValueError: + return True + return not (hostname == "api.openai.com" or hostname.endswith(".openai.azure.com")) + + +def _completion_content(response: dict[str, Any]) -> str: + if isinstance(response.get("output_text"), str): + return response["output_text"].strip() + choices = response.get("choices") + if isinstance(choices, list) and choices and isinstance(choices[0], dict): + message = choices[0].get("message") if isinstance(choices[0].get("message"), dict) else {} + content = message.get("content") + if isinstance(content, str): + return content.strip() + if isinstance(content, list): + return " ".join(str(item.get("text") or "") for item in content if isinstance(item, dict)).strip() + raise ValueError("structured completion response has no text content") diff --git a/backend/services/node_dependency_context_service.py b/backend/services/node_dependency_context_service.py index 732e88a..d7a745b 100644 --- a/backend/services/node_dependency_context_service.py +++ b/backend/services/node_dependency_context_service.py @@ -5,7 +5,10 @@ from __future__ import annotations from typing import Any from backend.dto.common import ApiError, ErrorCode +from backend.services.material_binding_schema_service import material_bindings_for_node from backend.services.image_service import ImageService +from backend.services.node_injection_template_service import build_node_injection_contract +from backend.services.prompt_plan_schema_service import prompt_plan_for_node from backend.services.workflow_service import WorkflowService @@ -37,7 +40,7 @@ class NodeDependencyContextService: for item in dependencies if isinstance(item.get("selected_image"), dict) ] - return { + result = { "workflow_id": target.workflow_id, "node_id": target.node_id, "branch_name": target.branch_name, @@ -45,7 +48,17 @@ class NodeDependencyContextService: "dependencies": dependencies, "prompt_context": ", ".join(prompt_parts), "selected_image_ids": selected_image_ids, + "material_bindings": material_bindings_for_node(detail.workflow.metadata, target, detail.nodes), + "prompt_plan": prompt_plan_for_node(detail.workflow.metadata, target, detail.nodes), } + result["injection_template"] = build_node_injection_contract( + target, + dependency_context=result, + material_bindings=result["material_bindings"], + prompt_plan=result["prompt_plan"], + stage_inputs=_stage_inputs_for_node(detail.workflow.metadata, target.node_id), + ) + return result def _node_context(self, node) -> dict[str, Any]: selected_image = None @@ -74,3 +87,10 @@ class NodeDependencyContextService: "selected_image": selected_image, "metadata": node.metadata, } + + +def _stage_inputs_for_node(workflow_metadata: dict[str, Any], node_id: str) -> Any: + stage_inputs = workflow_metadata.get("stage_inputs") + if not isinstance(stage_inputs, dict): + return {} + return stage_inputs.get(node_id) or {} diff --git a/backend/services/node_injection_template_service.py b/backend/services/node_injection_template_service.py new file mode 100644 index 0000000..6adb47b --- /dev/null +++ b/backend/services/node_injection_template_service.py @@ -0,0 +1,357 @@ +"""Node-specific workflow chat injection contracts.""" + +from __future__ import annotations + +from copy import deepcopy +from typing import Any + +from backend.dto.workflow_dto import WorkflowNode + + +NODE_INJECTION_TEMPLATE_VERSION = "2026-07-09.node-injection-schema.v1" + +NODE_KEY_TEMPLATE_KINDS: dict[str, str] = { + "profile_brief": "persona_split", + "prompt_plan": "prompt_plan", + "material_bind": "material_bind", + "headshot": "portrait_generation", + "half_body": "halfbody_generation", + "full_body": "fullbody_generation", + "expression_sheet": "sheet_generation", + "pose_sheet": "pose_generation", + "style_lock": "hires_fix", + "final_pack": "archive", + "fast_profile": "persona_split", + "fast_candidates": "portrait_generation", + "fast_final": "archive", + "questionnaire": "profile_import", + "preview": "prompt_plan", + "virtual_single_image": "single_image_generation", +} + +TEMPLATE_INTENTS: dict[str, str] = { + "profile_import": "upsert_profile_fields", + "persona_split": "update_structured_persona", + "prompt_plan": "update_prompt_matrix", + "material_bind": "update_material_bindings", + "portrait_generation": "update_portrait_node", + "halfbody_generation": "update_halfbody_node", + "fullbody_generation": "update_fullbody_node", + "sheet_generation": "update_design_sheet_node", + "pose_generation": "update_pose_sheet_node", + "stage_review": "review_stage_candidate", + "candidate_review": "review_candidates", + "hires_fix": "update_final_refinement", + "archive": "archive_character_workflow", + "single_image_generation": "single_image_generation", + "generic_workflow_node": "update_current_node", +} + +EXPECTED_RESPONSE_SCHEMAS: dict[str, dict[str, Any]] = { + "profile_import": { + "intent": "upsert_profile_fields", + "target_language": "zh-CN", + "profile_patch": { + "name": "string?", + "identity": "string?", + "personality": "string?", + "appearance": "string?", + "outfit": "string?", + "style": "string?", + "negative": "string?", + }, + "missing_fields": ["string"], + "conflicts": ["string"], + "next_action": "refresh_profile_import | ask_user | enter_persona_split", + }, + "persona_split": { + "intent": "update_structured_persona", + "split_fields_patch": { + "identity": "string?", + "appearance": "string?", + "outfit": "string?", + "materials": "string?", + "guardrails": "string?", + }, + "locked_fields": ["string"], + "conflicts": ["string"], + "next_action": "refresh_persona_split | enter_prompt_plan", + }, + "prompt_plan": { + "intent": "update_prompt_matrix", + "schema_version": "2026-07-09.prompt-plan-schema.v1", + "target_language": "en", + "matrix": [ + { + "target_id": "portrait | bust | fullbody | sheet | all", + "target_node_keys": ["headshot | half_body | full_body | expression_sheet | pose_sheet"], + "positive_patch": "string", + "negative_patch": "string?", + "weight_suggestions": [{"label": "string", "value": "number?", "reason": "string"}], + "weight_basis": "string", + "inheritance_fields": ["string"], + "inheritance_changes": ["string"], + "conflicts": ["string"], + "status": "ready | review | pending", + } + ], + "guardrails": { + "negative_prompt": "string", + "checks": [{"key": "string", "label": "string", "value": "string", "status": "ready | review | pending"}], + }, + "conflicts": [{"target_id": "string", "message": "string", "severity": "info | warn | error"}], + "next_action": "refresh_prompt_plan | enter_material_bind", + }, + "material_bind": { + "intent": "update_material_bindings", + "schema_version": "2026-07-09.material-binding-schema.v1", + "reference_policy": { + "mode": "explicit_only", + "default_use_reference": False, + "empty_material_means": "do_not_use_reference", + }, + "bindings": [ + { + "material_category": "style | composition | clothing | socks | shoes | accessory | other", + "bind_to_stages": [{"stage_id": "string", "stage_title": "string"}], + "target_node_keys": ["headshot | half_body | full_body | expression_sheet | pose_sheet | style_lock"], + "inheritance_rule": "string", + "reference_strength": "string?", + "use_reference": "boolean", + "expand_to_nodes_later": True, + } + ], + "conflicts": ["string"], + "next_action": "refresh_material_bind | enter_draft_generation", + }, + "portrait_generation": { + "intent": "update_portrait_node", + "target_language": "en", + "prompt_patch": "string", + "negative_patch": "string?", + "face_locks": ["face shape | hairstyle | hair color | eye color | expression"], + "generation_params": {"count": "number?", "aspect_ratio": "1:1", "reference_strength": "string?"}, + "next_action": "generate_portrait | rerun_portrait | confirm_use | ask_user", + }, + "halfbody_generation": { + "intent": "update_halfbody_node", + "target_language": "en", + "prompt_patch": "string", + "negative_patch": "string?", + "inherited_locks": ["portrait baseline | face | hair | eye direction"], + "outfit_patch": "string?", + "pose_patch": "string?", + "generation_params": {"count": "number?", "aspect_ratio": "3:4", "reference_strength": "string?"}, + "next_action": "generate_halfbody | rerun_halfbody | confirm_use | ask_user", + }, + "fullbody_generation": { + "intent": "update_fullbody_node", + "target_language": "en", + "prompt_patch": "string", + "negative_patch": "string?", + "proportion_locks": ["head ratio | body proportion | outfit silhouette | shoes and socks"], + "accessory_requirements": ["string"], + "generation_params": {"count": "number?", "aspect_ratio": "2:3", "reference_strength": "string?"}, + "next_action": "generate_fullbody | rerun_fullbody | confirm_use | ask_user", + }, + "sheet_generation": { + "intent": "update_design_sheet_node", + "target_language": "en", + "prompt_patch": "string", + "negative_patch": "string?", + "sheet_layout": "front view | detail callouts | expression set | material labels", + "inherited_locks": ["portrait | halfbody | fullbody | detail nodes"], + "next_action": "generate_sheet | rerun_sheet | confirm_use | ask_user", + }, + "pose_generation": { + "intent": "update_pose_sheet_node", + "target_language": "en", + "prompt_patch": "string", + "negative_patch": "string?", + "pose_targets": ["standing | action | hand pose | prop interaction"], + "inherited_locks": ["identity | outfit | style"], + "next_action": "generate_pose_sheet | rerun_pose_sheet | ask_user", + }, + "stage_review": { + "intent": "review_stage_candidate", + "target_language": "en", + "review_stage": "portrait | halfbody | body_sheet", + "same_prompt_regeneration": True, + "primary_image_id": "string?", + "selected_image_ids": ["string"], + "model_route_notes": ["string"], + "defects_to_regenerate": ["string"], + "locked_items": ["string"], + "selection_reason": "string", + "next_action": "regenerate_selected_image | confirm_next_flow | ask_user", + }, + "candidate_review": { + "intent": "review_candidates", + "review_dimensions": [ + "identity consistency", + "outfit inheritance", + "style consistency", + "image defects", + "policy/negative constraints", + ], + "primary_image_id": "string?", + "selected_image_ids": ["string"], + "reject_image_ids": ["string"], + "selection_reason": "string", + "next_round_direction": "string", + "next_action": "select_candidate | regenerate_node | enter_hires_fix", + }, + "hires_fix": { + "intent": "update_final_refinement", + "target_language": "en", + "prompt_patch": "string", + "defects_to_fix": ["string"], + "keep_locked_items": ["string"], + "next_action": "regenerate_hires | enter_archive", + }, + "archive": { + "intent": "archive_character_workflow", + "archive_summary": "string", + "reusable_tags": ["string"], + "version_note": "string", + "next_action": "save_archive | ask_user", + }, + "single_image_generation": { + "intent": "single_image_generation", + "target_language": "en", + "prompt_patch": "string", + "negative_patch": "string?", + "generation_params": {"count": "number?", "aspect_ratio": "string?"}, + "next_action": "generate_image | ask_user", + }, + "generic_workflow_node": { + "intent": "update_current_node", + "target_language": "en", + "prompt_patch": "string", + "metadata_patch": "object", + "next_action": "refresh_current_node | ask_user", + }, +} + +CONTEXT_REQUIREMENTS: dict[str, list[str]] = { + "profile_import": ["user_instruction", "current_profile", "source_language"], + "persona_split": ["user_instruction", "current_profile", "structured_persona_fields"], + "prompt_plan": ["user_instruction", "structured_persona_fields", "material_bindings"], + "material_bind": ["user_instruction", "available_materials", "target_stage_ids"], + "portrait_generation": ["user_instruction", "current_profile", "material_bindings"], + "halfbody_generation": ["user_instruction", "upstream_selected_images", "material_bindings"], + "fullbody_generation": ["user_instruction", "upstream_selected_images", "material_bindings"], + "sheet_generation": ["user_instruction", "upstream_selected_images", "material_bindings"], + "pose_generation": ["user_instruction", "upstream_selected_images", "material_bindings"], + "stage_review": ["user_instruction", "selected_draft_image_ids", "upstream_selected_images"], + "candidate_review": ["user_instruction", "selected_draft_image_ids", "review_policy"], + "hires_fix": ["user_instruction", "stage_inputs", "upstream_selected_images"], + "archive": ["user_instruction", "accepted_images", "workflow_history"], + "single_image_generation": ["user_instruction", "current_profile"], + "generic_workflow_node": ["user_instruction", "node_context"], +} + + +def template_kind_for_node(node: WorkflowNode, requested_kind: str | None = None) -> str: + normalized = str(requested_kind or "").strip() + if normalized in EXPECTED_RESPONSE_SCHEMAS: + return normalized + return NODE_KEY_TEMPLATE_KINDS.get(node.node_key, "generic_workflow_node") + + +def build_node_injection_contract( + node: WorkflowNode, + *, + requested_kind: str | None = None, + user_instruction: str | None = None, + dependency_context: dict[str, Any] | None = None, + material_bindings: Any | None = None, + prompt_plan: Any | None = None, + stage_inputs: Any | None = None, + selected_draft_image_ids: list[str] | None = None, + ui_context: dict[str, Any] | None = None, +) -> dict[str, Any]: + kind = template_kind_for_node(node, requested_kind) + dependency_context = dependency_context if isinstance(dependency_context, dict) else {} + return { + "version": NODE_INJECTION_TEMPLATE_VERSION, + "template_kind": kind, + "node_key": node.node_key, + "node_id": node.node_id, + "intent": TEMPLATE_INTENTS.get(kind, TEMPLATE_INTENTS["generic_workflow_node"]), + "language_policy": { + "source_language": "auto", + "target_prompt_language": "en", + "translation_mode": "backend_llm", + "preserve_character_names": True, + }, + "user_instruction": { + "raw_text": str(user_instruction or "").strip(), + "source_language": "auto", + }, + "node_context": { + "title": node.title, + "step_name": node.step_name, + "node_type": node.node_type, + "status": node.status, + "prompt": node.prompt, + "negative_prompt": node.negative_prompt, + "selected_image_id": node.selected_image_id, + "metadata": node.metadata, + }, + "dependency_context": { + "dependency_count": dependency_context.get("dependency_count", 0), + "prompt_context": dependency_context.get("prompt_context", ""), + "selected_image_ids": dependency_context.get("selected_image_ids", []), + "dependencies": dependency_context.get("dependencies", []), + }, + "material_bindings": material_bindings if material_bindings is not None else [], + "prompt_plan": prompt_plan if prompt_plan is not None else {}, + "stage_inputs": stage_inputs if stage_inputs is not None else {}, + "selected_draft_image_ids": selected_draft_image_ids or [], + "ui_context": ui_context or {}, + "context_requirements": CONTEXT_REQUIREMENTS.get(kind, CONTEXT_REQUIREMENTS["generic_workflow_node"]), + "expected_response_schema": deepcopy( + EXPECTED_RESPONSE_SCHEMAS.get(kind, EXPECTED_RESPONSE_SCHEMAS["generic_workflow_node"]) + ), + "prompt_contract": [ + "Understand the user's source-language instruction.", + "Translate user-facing edits into concise English image-generation prompt language.", + "Preserve locked identity traits, selected upstream images, material bindings, and negative constraints.", + "Return a JSON-compatible object matching expected_response_schema.", + "If the target node is locked, return a branch-required next_action instead of mutating the original chain.", + ], + } + + +def injection_payload_from_generation_request(payload: dict[str, Any]) -> dict[str, Any]: + metadata = payload.get("metadata") if isinstance(payload.get("metadata"), dict) else {} + user_instruction = ( + metadata.get("user_instruction") + or payload.get("user_instruction") + or payload.get("prompt_patch") + or payload.get("extra_prompt") + or "" + ) + return { + "requested_kind": metadata.get("node_injection_kind") or payload.get("node_injection_kind"), + "user_instruction": str(user_instruction or ""), + "material_bindings": metadata.get("material_binding_rules") or payload.get("material_bindings") or [], + "stage_inputs": metadata.get("stage_inputs") or payload.get("stage_inputs") or {}, + "selected_draft_image_ids": _string_list( + metadata.get("selected_draft_image_ids") or payload.get("selected_draft_image_ids") + ), + "ui_context": { + "ui_node_id": metadata.get("ui_node_id"), + "ui_node_title": metadata.get("ui_node_title"), + "ui_node_phase": metadata.get("ui_node_phase"), + "review_source_ui_node_id": metadata.get("review_source_ui_node_id"), + "review_next_ui_node_id": metadata.get("review_next_ui_node_id"), + }, + } + + +def _string_list(value: Any) -> list[str]: + if not isinstance(value, list): + return [] + return [str(item) for item in value if str(item or "").strip()] diff --git a/backend/services/profile_service.py b/backend/services/profile_service.py index f29f4a9..2fe3c7a 100644 --- a/backend/services/profile_service.py +++ b/backend/services/profile_service.py @@ -23,7 +23,9 @@ class ProfileService: def create_profile(self, payload: dict[str, Any], ctx: RequestContext) -> CharacterProfile: self._validate_create_payload(payload) - owner_user_id = payload.get("owner_user_id") or ctx.actor_id + # Ownership comes from the authenticated caller. Accepting an + # arbitrary owner_user_id would allow cross-tenant profile creation. + owner_user_id = ctx.actor_id if not owner_user_id: raise ApiError( ErrorCode.VALIDATION_FAILED, @@ -57,7 +59,7 @@ class ProfileService: ) return profile - def get_profile(self, profile_id: str) -> CharacterProfile: + def get_profile(self, profile_id: str, ctx: RequestContext | None = None) -> CharacterProfile: profile = self.repository.get(profile_id) if profile is None or profile.status == "deleted": raise ApiError( @@ -65,6 +67,8 @@ class ProfileService: "角色档案不存在", {"profile_id": profile_id}, ) + if ctx is not None: + self._require_owner(profile, ctx) return profile def list_profiles( @@ -90,7 +94,7 @@ class ProfileService: payload: dict[str, Any], ctx: RequestContext, ) -> CharacterProfile: - before = self.get_profile(profile_id) + before = self.get_profile(profile_id, ctx) if "name" in payload and not str(payload["name"]).strip(): raise ApiError(ErrorCode.VALIDATION_FAILED, "角色名称不能为空", {"field": "name"}) update_payload = dict(payload) @@ -114,7 +118,7 @@ class ProfileService: return after def delete_profile(self, profile_id: str, ctx: RequestContext) -> CharacterProfile: - before = self.get_profile(profile_id) + before = self.get_profile(profile_id, ctx) after = self.repository.soft_delete(profile_id) if after is None: raise ApiError( @@ -137,6 +141,17 @@ class ProfileService: if not name: raise ApiError(ErrorCode.VALIDATION_FAILED, "角色名称不能为空", {"field": "name"}) + def _require_owner(self, profile: CharacterProfile, ctx: RequestContext) -> None: + if ctx.actor_id and ctx.actor_id == profile.owner_user_id: + return + # Use not-found semantics so guessed resource IDs cannot enumerate + # another user's profiles. + raise ApiError( + ErrorCode.RESOURCE_NOT_FOUND, + "角色档案不存在", + {"profile_id": profile.profile_id}, + ) + def new_profile_id() -> str: return f"profile_{uuid4().hex}" diff --git a/backend/services/prompt_expansion_service.py b/backend/services/prompt_expansion_service.py index 1ff7731..0c957f6 100644 --- a/backend/services/prompt_expansion_service.py +++ b/backend/services/prompt_expansion_service.py @@ -6,6 +6,10 @@ from typing import Any from backend.dto.profile_dto import CharacterProfile from backend.dto.workflow_dto import WorkflowNode +from backend.services.node_injection_template_service import ( + build_node_injection_contract, + injection_payload_from_generation_request, +) from core.prompt_pipeline import merge_prompt_sections @@ -26,12 +30,47 @@ class PromptExpansionService: node: WorkflowNode, payload: dict[str, Any], ) -> dict[str, Any]: + dependency_context = payload.get("dependency_context") if isinstance(payload.get("dependency_context"), dict) else {} + prompt_plan = dependency_context.get("prompt_plan") if isinstance(dependency_context.get("prompt_plan"), dict) else {} + prompt_plan_generation = ( + prompt_plan.get("generation_params") if isinstance(prompt_plan.get("generation_params"), dict) else {} + ) variables = { "profile_name": profile.name, "node_title": node.title, **(profile.metadata.get("prompt_variables") if isinstance(profile.metadata.get("prompt_variables"), dict) else {}), **(payload.get("variables") if isinstance(payload.get("variables"), dict) else {}), } + injection_payload = injection_payload_from_generation_request(payload) + dependency_injection = ( + dependency_context.get("injection_template") + if isinstance(dependency_context.get("injection_template"), dict) + else {} + ) + material_bindings = ( + injection_payload["material_bindings"] + or dependency_context.get("material_bindings") + or dependency_injection.get("material_bindings") + or [] + ) + stage_inputs = ( + injection_payload["stage_inputs"] + or dependency_context.get("stage_inputs") + or dependency_injection.get("stage_inputs") + or {} + ) + selected_draft_image_ids = ( + injection_payload["selected_draft_image_ids"] + or dependency_context.get("selected_draft_image_ids") + or dependency_injection.get("selected_draft_image_ids") + or [] + ) + context_prompt_parts = _unique_prompt_parts( + [ + *_prompt_values(dependency_context.get("prompt_context")), + *_material_prompt_parts(material_bindings), + ] + ) result = merge_prompt_sections( { "parts": [profile.base_prompt, profile.appearance_prompt, profile.style_prompt], @@ -43,17 +82,38 @@ class PromptExpansionService: "negative_parts": [node.negative_prompt], "variables": variables, }, + { + "parts": _as_list(prompt_plan_generation.get("prompt_patch")), + "negative_parts": _as_list(prompt_plan_generation.get("negative_patch")), + "variables": variables, + }, + { + "parts": context_prompt_parts, + "variables": variables, + }, { "parts": _payload_parts(payload), "negative_parts": _payload_negative_parts(payload), "variables": variables, }, ) + node_injection = build_node_injection_contract( + node, + requested_kind=injection_payload["requested_kind"], + user_instruction=injection_payload["user_instruction"], + dependency_context=dependency_context, + material_bindings=material_bindings, + prompt_plan=prompt_plan or dependency_injection.get("prompt_plan") or {}, + stage_inputs=stage_inputs, + selected_draft_image_ids=_string_list(selected_draft_image_ids), + ui_context=injection_payload["ui_context"], + ) return { **result.to_dict(), "source": "profile_node", "profile_id": profile.profile_id, "node_id": node.node_id, + "node_injection": node_injection, } @@ -83,3 +143,121 @@ def _as_list(value: Any) -> list[Any]: if isinstance(value, list): return value return [value] + + +def _material_prompt_parts(material_bindings: Any) -> list[str]: + if not isinstance(material_bindings, (dict, list, tuple)): + return [] + + structured: list[str] = [] + contexts: list[str] = [] + if isinstance(material_bindings, dict): + composition = material_bindings.get("composition") if isinstance(material_bindings.get("composition"), dict) else {} + composition_prompt = str(composition.get("prompt") or "").strip() + if composition_prompt: + return _split_prompt_text(composition_prompt) + structured.extend(_prompt_values(material_bindings.get("material_prompts"))) + structured.extend(_prompt_values(material_bindings.get("reference_prompts"))) + generation_params = material_bindings.get("generation_params") + if isinstance(generation_params, dict): + structured.extend(_prompt_values(generation_params.get("reference_prompts"))) + structured.extend(_raw_material_prompt_values(material_bindings.get("active_bindings"))) + structured.extend(_raw_material_prompt_values(material_bindings.get("rules"))) + contexts.extend(_prompt_values(material_bindings.get("prompt_context"))) + else: + structured.extend(_raw_material_prompt_values(material_bindings)) + + return _unique_prompt_parts([*structured, *contexts]) + + +def _raw_material_prompt_values(value: Any) -> list[str]: + if not isinstance(value, (list, tuple)): + return [] + result: list[str] = [] + for item in value: + if not isinstance(item, dict): + continue + result.extend(_prompt_values(item.get("material_prompt"))) + material = item.get("material") + if isinstance(material, dict): + result.extend(_prompt_values(material.get("prompt"))) + return result + + +def _prompt_values(value: Any) -> list[str]: + if value is None: + return [] + if isinstance(value, str): + text = value.strip() + return _split_prompt_text(text) if text else [] + if isinstance(value, (list, tuple)): + result: list[str] = [] + for item in value: + result.extend(_prompt_values(item)) + return result + if isinstance(value, dict): + for key in ("prompt", "text"): + if key in value: + return _prompt_values(value.get(key)) + return [] + + +def _unique_prompt_parts(values: list[str]) -> list[str]: + result: list[str] = [] + seen: set[str] = set() + for value in values: + text = str(value or "").strip() + key = text.casefold() + if text and key not in seen: + result.append(text) + seen.add(key) + return result + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return [value] if value.strip() else [] + if not isinstance(value, (list, tuple)): + return [] + return _unique_prompt_parts([str(item) for item in value if str(item or "").strip()]) + + +def _split_prompt_text(text: str) -> list[str]: + parts: list[str] = [] + current: list[str] = [] + depth = 0 + quote: str | None = None + escaped = False + for character in text: + if escaped: + current.append(character) + escaped = False + continue + if character == "\\": + current.append(character) + escaped = True + continue + if quote: + current.append(character) + if character == quote: + quote = None + continue + if character in {'"', "'"}: + current.append(character) + quote = character + continue + if character in "([{": + depth += 1 + elif character in ")]}": + depth = max(depth - 1, 0) + if character == "," and depth == 0: + part = "".join(current).strip() + if part: + parts.append(part) + current = [] + continue + current.append(character) + part = "".join(current).strip() + if part: + parts.append(part) + return parts diff --git a/backend/services/prompt_governance_service.py b/backend/services/prompt_governance_service.py new file mode 100644 index 0000000..7e59675 --- /dev/null +++ b/backend/services/prompt_governance_service.py @@ -0,0 +1,404 @@ +"""Prompt translation, semantic normalization, moderation and route governance. + +The services are provider-neutral. Optional callables can connect configured LLM/VLM +routes, while deterministic validation and conservative fallbacks remain available +when an external reviewer is unavailable. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass +from datetime import datetime, timezone +from typing import Any, Callable, Mapping + + +RISK_LEVELS = ( + "safe", "sensitive", "adult_allowed", "uncertain", + "violence_prohibited", "terror_prohibited", "prohibited", +) +RISK_PRIORITY = {level: index for index, level in enumerate(RISK_LEVELS)} +ENTRYPOINTS = {"web", "qq", "telegram", "discord", "api", "unknown"} + +TranslationCallable = Callable[[dict[str, Any]], Mapping[str, Any] | str] +ModerationCallable = Callable[[dict[str, Any]], Mapping[str, Any]] +ImageModerationCallable = Callable[[dict[str, Any]], Mapping[str, Any]] + + +@dataclass(frozen=True) +class PromptTranslationResult: + original_text: str + translated_text: str + status: str + enabled: bool + accepted: bool + model_route: str | None + template_version: str + created_at: str + sections: dict[str, Any] + difference_summary: list[str] + error: str | None = None + + def to_dict(self) -> dict[str, Any]: + return dict(self.__dict__) + + +class PromptTranslationService: + def __init__(self, config: Mapping[str, Any] | None = None, translator: TranslationCallable | None = None): + self.config = dict(config or {}) + self.translator = translator + + def translate(self, text: str, *, entrypoint: str = "web", user_allowed: bool = False, model_family: str = "natural_language", locked_terms: list[str] | None = None, budget: int | None = None) -> PromptTranslationResult: + original = _text(text) + enabled = bool(self.config.get("enabled", False)) + entrypoint = entrypoint if entrypoint in ENTRYPOINTS else "unknown" + entrypoint_allowed = bool((self.config.get("entrypoints") or {}).get(entrypoint, entrypoint == "web")) + allowed = enabled and user_allowed and entrypoint_allowed + now = _utc_now() + route = _optional_text(self.config.get("model_route")) + version = _text(self.config.get("template_version")) or "1.0" + if not original or not allowed: + return PromptTranslationResult(original, original, "bypassed", enabled, False, route, version, now, {}, []) + if _is_english_only(original): + return PromptTranslationResult(original, original, "english_passthrough", True, True, route, version, now, {"prompt": original}, []) + request = { + "text": original, + "target_language": "English", + "model_family": model_family, + "locked_terms": list(locked_terms or []), + "budget": budget, + "requirements": [ + "preserve the user's visual intent", + "separate subject, appearance, clothing, action, composition, scene, style, lighting, camera and negative constraints", + "do not hide or encode safety-relevant meaning", + "do not change age or identity", + ], + } + if self.translator is None: + return PromptTranslationResult(original, original, "unavailable_fallback", True, False, route, version, now, {}, [], "translation model is unavailable") + try: + response = self.translator(request) + if isinstance(response, str): + translated, sections = _text(response), {"prompt": _text(response)} + elif isinstance(response, Mapping): + translated = _text(response.get("translated_text") or response.get("prompt")) + sections = dict(response.get("sections") or {}) + else: + translated, sections = "", {} + if not translated: + raise ValueError("translator returned an empty prompt") + translated = _restore_locked_terms(original, translated, locked_terms or []) + if budget and len(translated) > budget: + translated = _budget_text(translated, budget, locked_terms or []) + return PromptTranslationResult(original, translated, "translated", True, True, route, version, now, sections, _difference_summary(original, translated)) + except Exception as exc: + return PromptTranslationResult(original, original, "failed_fallback", True, False, route, version, now, {}, [], exc.__class__.__name__) + + +class PromptSemanticNormalizer: + VERSION = "1.0" + + def normalize(self, text: str, *, locked_terms: list[str] | None = None) -> dict[str, Any]: + original = _text(text) + normalized = original + changes: list[dict[str, str]] = [] + replacements = { + r"(?i)\b(girl|boy)\b": "adult woman" if re.search(r"(?i)\bgirl\b", normalized) else "adult man", + r"少女": "成年女性", + r"少年": "成年男性", + } + # Age-disambiguation is only added for otherwise lawful adult contexts. + if _contains_any(normalized, ADULT_TERMS) and not _contains_any(normalized, MINOR_TERMS): + if not _contains_any(normalized, ADULT_AGE_TERMS): + normalized = f"adult subject, {normalized}" + changes.append({"from": "implicit age", "to": "adult subject", "reason": "adult_age_disambiguation"}) + for pattern, replacement in replacements.items(): + if re.search(pattern, normalized) and _contains_any(normalized, ADULT_TERMS): + before = normalized + normalized = re.sub(pattern, replacement, normalized) + if normalized != before: + changes.append({"from": pattern, "to": replacement, "reason": "age_disambiguation"}) + normalized = _restore_locked_terms(original, normalized, locked_terms or []) + return { + "original_text": original, + "normalized_text": normalized, + "changed_terms": changes, + "reason": "reduce ambiguity without concealing safety meaning" if changes else "no_change", + "meaning_preserved": True, + "requires_confirmation": False, + "version": self.VERSION, + "created_at": _utc_now(), + } + + +@dataclass(frozen=True) +class PromptModerationDecision: + level: str + confidence: float + categories: list[str] + reason_codes: list[str] + action: str + warning: str | None + rules_version: str + model_route: str | None + created_at: str + + @property + def allowed(self) -> bool: + return self.level in {"safe", "sensitive", "adult_allowed"} + + def to_dict(self) -> dict[str, Any]: + return {**self.__dict__, "allowed": self.allowed} + + +class PromptModerationService: + RULES_VERSION = "1.0" + + def __init__(self, config: Mapping[str, Any] | None = None, reviewer: ModerationCallable | None = None): + self.config = dict(config or {}) + self.reviewer = reviewer + + def moderate(self, *, original_prompt: str, translated_prompt: str = "", final_prompt: str = "", negative_prompt: str = "", reference_bindings: list[dict[str, Any]] | None = None) -> PromptModerationDecision: + text = "\n".join(_text(value) for value in (original_prompt, translated_prompt, final_prompt, negative_prompt) if _text(value)) + rule = self._rule_decision(text) + model = self._model_decision(text, reference_bindings or []) + return _stricter_decision(rule, model) + + def _rule_decision(self, text: str) -> PromptModerationDecision: + lower = text.casefold() + if _contains_any(lower, TERROR_TERMS): + return _decision("terror_prohibited", 0.96, ["terrorism"], ["terrorism_or_extremism"], "reject_warn", "请求涉及恐怖主义或极端主义内容,已禁止生成。") + if _contains_any(lower, VIOLENCE_PROHIBITED_TERMS): + return _decision("violence_prohibited", 0.94, ["high_risk_violence"], ["violent_harm_or_extreme_gore"], "reject_warn", "请求涉及高危暴力、现实伤害或极端血腥内容,已禁止生成。") + adult = _contains_any(lower, ADULT_TERMS) + minor = _contains_any(lower, MINOR_TERMS) + if adult and minor: + return _decision("prohibited", 0.98, ["sexual_minors"], ["adult_content_with_minor_or_ambiguous_age"], "reject_warn", "成人内容中的人物年龄不明确或涉及未成年人,已禁止生成。") + if _contains_any(lower, OTHER_PROHIBITED_TERMS): + return _decision("prohibited", 0.93, ["prohibited"], ["prohibited_content"], "reject", "请求包含禁止生成的内容。") + if adult: + if not _contains_any(lower, ADULT_AGE_TERMS): + return _decision("uncertain", 0.72, ["adult", "ambiguous_age"], ["adult_age_not_explicit"], "confirm_or_reject", "请先明确所有人物均为成年人。") + return _decision("adult_allowed", 0.90, ["adult"], ["lawful_adult_content"], "route_nsfw", None) + if _contains_any(lower, SENSITIVE_TERMS): + return _decision("sensitive", 0.75, ["sensitive"], ["sensitive_but_allowed"], "allow", None) + return _decision("safe", 0.85, [], ["no_risk_signal"], "allow", None) + + def _model_decision(self, text: str, references: list[dict[str, Any]]) -> PromptModerationDecision | None: + if self.reviewer is None or not bool(self.config.get("llm_review_enabled", False)): + return None + try: + response = self.reviewer({"text": text, "reference_bindings": references, "levels": list(RISK_LEVELS), "rules_version": self.RULES_VERSION}) + level = _text(response.get("level")) + if level not in RISK_LEVELS: + raise ValueError("invalid moderation level") + return PromptModerationDecision( + level=level, + confidence=_bounded_float(response.get("confidence"), 0.5), + categories=_string_list(response.get("categories")), + reason_codes=_string_list(response.get("reason_codes")) or ["model_review"], + action=_text(response.get("action")) or _action_for_level(level), + warning=_optional_text(response.get("warning")), + rules_version=_text(response.get("rules_version")) or self.RULES_VERSION, + model_route=_optional_text(self.config.get("model_route")), + created_at=_utc_now(), + ) + except Exception: + fallback = _text(self.config.get("failure_policy") or "fail_closed") + return _decision("uncertain" if fallback == "confirm" else "prohibited", 0.5, ["review_unavailable"], ["moderation_service_failure"], "confirm_or_reject" if fallback == "confirm" else "reject", "内容审核暂时不可用,已采用保守策略。") + + +@dataclass(frozen=True) +class GenerationRouteDecision: + requested_route: str + moderation_level: str + selected_route: str | None + reason: str + fallback_route: str | None + allowed: bool + requires_confirmation: bool + created_at: str + + def to_dict(self) -> dict[str, Any]: + return dict(self.__dict__) + + +class GenerationRouteService: + def __init__(self, config: Mapping[str, Any] | None = None): + self.config = dict(config or {}) + + def decide(self, requested_route: str, moderation: PromptModerationDecision, *, entrypoint: str, user_id: str | None, group_id: str | None, permission_context: Mapping[str, Any] | None = None) -> GenerationRouteDecision: + permissions = dict(permission_context or {}) + level = moderation.level + if level in {"violence_prohibited", "terror_prohibited", "prohibited"}: + return self._decision(requested_route, level, None, "content_prohibited", None, False, False) + if level == "uncertain": + return self._decision(requested_route, level, None, "moderation_confirmation_required", None, False, True) + safe_route = _optional_text(self.config.get("safe_route")) or requested_route + nsfw_route = _optional_text(self.config.get("nsfw_route")) + if level != "adult_allowed": + return self._decision(requested_route, level, safe_route, "safe_or_sensitive_content", nsfw_route, True, False) + enabled = bool(self.config.get("adult_auto_route_enabled", False)) + entry_allowed = bool((self.config.get("entrypoints") or {}).get(entrypoint, False)) + user_allowed = bool(permissions.get("adult_user_allowed", False)) + group_allowed = bool(permissions.get("adult_group_allowed", group_id is None)) + quota_allowed = bool(permissions.get("quota_allowed", True)) + if not all((enabled, entry_allowed, user_allowed, group_allowed, quota_allowed)): + return self._decision(requested_route, level, None, "adult_route_permission_denied", nsfw_route, False, False) + if not nsfw_route: + return self._decision(requested_route, level, None, "nsfw_route_not_configured", None, False, False) + return self._decision(requested_route, level, nsfw_route, "adult_content_auto_routed", None, True, False) + + def _decision(self, requested: str, level: str, selected: str | None, reason: str, fallback: str | None, allowed: bool, confirm: bool) -> GenerationRouteDecision: + return GenerationRouteDecision(requested, level, selected, reason, fallback, allowed, confirm, _utc_now()) + + +class GenerationFailureClassifier: + CONTENT_SIGNALS = ("nsfw", "safety", "content policy", "moderation", "sexual content", "adult content", "blocked prompt", "内容审核", "敏感内容") + + def classify(self, failure: Mapping[str, Any] | BaseException) -> dict[str, Any]: + if isinstance(failure, Mapping): + status = failure.get("status_code") + error_type = _text(failure.get("error_type") or failure.get("category")) + detail = _text(failure.get("message")) + " " + _text(failure.get("details")) + provider_id = _optional_text(failure.get("provider_id")) + else: + status = getattr(failure, "status_code", None) + error_type = _text(getattr(failure, "error_type", failure.__class__.__name__)) + details = getattr(failure, "details", {}) + detail = f"{failure} {details}" + provider_id = _optional_text(getattr(failure, "provider_id", None)) + lower = f"{error_type} {detail}".casefold() + if any(signal in lower for signal in self.CONTENT_SIGNALS): + category = "content_safety_rejection" + elif "quota" in lower or "credit" in lower or "balance" in lower: + category = "quota_exhausted" + elif status == 429 or "rate" in lower: + category = "rate_limit" + elif status in {401, 403} or "auth" in lower or "credential" in lower: + category = "authentication" + elif "timeout" in lower or status in {408, 504}: + category = "timeout" + elif "unsupported" in lower or "capability" in lower: + category = "unsupported_capability" + elif status is not None and 400 <= int(status) < 500: + category = "invalid_request" + elif "network" in lower or (status is not None and int(status) >= 500): + category = "upstream_error" + else: + category = "unknown" + return {"failure_class": category, "provider_id": provider_id, "status_code": status, "error_type": error_type, "summary": _redact(detail)[:500]} + + +class ImageModerationService: + def __init__(self, config: Mapping[str, Any] | None = None, reviewer: ImageModerationCallable | None = None): + self.config = dict(config or {}) + self.reviewer = reviewer + + def moderate(self, image: Mapping[str, Any], *, requested_level: str, route_kind: str) -> dict[str, Any]: + if self.reviewer is None or not bool(self.config.get("enabled", False)): + return {"status": "not_configured", "action": "allow", "requested_level": requested_level, "route_kind": route_kind, "version": "1.0"} + try: + result = dict(self.reviewer({"image": dict(image), "requested_level": requested_level, "route_kind": route_kind})) + except Exception: + return {"status": "review_failed", "action": "quarantine", "requested_level": requested_level, "route_kind": route_kind, "version": "1.0"} + action = _text(result.get("action")) + if action not in {"allow", "warn", "quarantine", "regenerate", "reject"}: + action = "quarantine" + return {**result, "action": action, "status": "reviewed", "version": _text(result.get("version")) or "1.0"} + + +TERROR_TERMS = ("terrorist propaganda", "terror recruitment", "join isis", "join al qaeda", "恐怖组织宣传", "恐怖主义招募", "加入恐怖组织", "极端主义行动") +VIOLENCE_PROHIBITED_TERMS = ("how to kill", "murder a real", "bomb attack", "dismemberment", "torture gore", "beheading gore", "如何杀害", "现实伤害威胁", "炸弹袭击", "肢解", "酷刑血腥", "斩首血腥") +ADULT_TERMS = ("nsfw", "nude", "nudity", "explicit sexual", "erotic", "lingerie", "裸露", "裸体", "裸像", "色情", "成人内容", "情趣内衣") +ADULT_AGE_TERMS = ("adult", "18+", "over 18", "成年", "成年人") +MINOR_TERMS = ("minor", "underage", "child", "kid", "schoolgirl", "schoolboy", "幼女", "幼童", "未成年", "儿童", "小学生", "初中生") +OTHER_PROHIBITED_TERMS = ("sexual assault", "rape scene", "non-consensual explicit", "性侵", "强奸场景", "非自愿色情") +SENSITIVE_TERMS = ("blood", "weapon", "swimsuit", "medical", "内衣写真", "泳装", "武器", "少量血迹") + + +def _stricter_decision(left: PromptModerationDecision, right: PromptModerationDecision | None) -> PromptModerationDecision: + if right is None: + return left + return right if RISK_PRIORITY[right.level] > RISK_PRIORITY[left.level] else left + + +def _decision(level: str, confidence: float, categories: list[str], reasons: list[str], action: str, warning: str | None) -> PromptModerationDecision: + return PromptModerationDecision(level, confidence, categories, reasons, action, warning, PromptModerationService.RULES_VERSION, None, _utc_now()) + + +def _action_for_level(level: str) -> str: + return {"safe": "allow", "sensitive": "allow", "adult_allowed": "route_nsfw", "uncertain": "confirm_or_reject", "violence_prohibited": "reject_warn", "terror_prohibited": "reject_warn", "prohibited": "reject"}[level] + + +def _contains_any(text: str, terms: tuple[str, ...]) -> bool: + lowered = text.casefold() + return any(term.casefold() in lowered for term in terms) + + +def _is_english_only(value: str) -> bool: + return not bool(re.search(r"[\u3400-\u9fff]", value)) + + +def _restore_locked_terms(original: str, translated: str, locked_terms: list[str]) -> str: + result = translated + for term in locked_terms: + text = _text(term) + if text and text in original and text not in result: + result = f"{result}, {text}" + return result + + +def _budget_text(value: str, budget: int, locked_terms: list[str]) -> str: + parts = _split_prompt(value) + locked = {_text(item).casefold() for item in locked_terms} + while len(", ".join(parts)) > budget and parts: + removable = next((index for index in range(len(parts) - 1, -1, -1) if parts[index].casefold() not in locked), None) + if removable is None: + break + parts.pop(removable) + return ", ".join(parts) + + +def _difference_summary(original: str, translated: str) -> list[str]: + summary = ["language: source -> English"] if not _is_english_only(original) and _is_english_only(translated) else [] + if len(translated) > len(original) * 1.5: + summary.append("structured visual details expanded") + return summary + + +def _split_prompt(value: Any) -> list[str]: + return [part.strip() for part in _text(value).replace(";", ",").split(",") if part.strip()] + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return _split_prompt(value) + if not isinstance(value, (list, tuple, set)): + return [] + return [_text(item) for item in value if _text(item)] + + +def _bounded_float(value: Any, default: float) -> float: + try: + return min(max(float(value), 0.0), 1.0) + except (TypeError, ValueError): + return default + + +def _redact(value: str) -> str: + text = re.sub(r"(?i)bearer\s+\S+", "Bearer [REDACTED]", value) + return re.sub(r"\bsk-[A-Za-z0-9_-]{8,}\b", "[REDACTED]", text) + + +def _optional_text(value: Any) -> str | None: + text = _text(value) + return text or None + + +def _text(value: Any) -> str: + return str(value or "").strip() + + +def _utc_now() -> str: + return datetime.now(timezone.utc).isoformat() diff --git a/backend/services/prompt_plan_schema_service.py b/backend/services/prompt_plan_schema_service.py new file mode 100644 index 0000000..1b56839 --- /dev/null +++ b/backend/services/prompt_plan_schema_service.py @@ -0,0 +1,366 @@ +"""Normalize prompt planning output for workflow generation nodes.""" + +from __future__ import annotations + +from copy import deepcopy +from typing import Any + +from backend.dto.workflow_dto import WorkflowNode + + +PROMPT_PLAN_SCHEMA_VERSION = "2026-07-09.prompt-plan-schema.v1" + +TARGET_NODE_KEYS: dict[str, tuple[str, ...]] = { + "portrait": ("headshot", "fast_candidates"), + "headshot": ("headshot", "fast_candidates"), + "face": ("headshot", "fast_candidates"), + "bust": ("half_body",), + "halfbody": ("half_body",), + "half_body": ("half_body",), + "fullbody": ("full_body",), + "full_body": ("full_body",), + "body": ("full_body",), + "sheet": ("expression_sheet", "pose_sheet"), + "design_sheet": ("expression_sheet", "pose_sheet"), + "expression_sheet": ("expression_sheet",), + "pose_sheet": ("pose_sheet",), + "all": ("headshot", "fast_candidates", "half_body", "full_body", "expression_sheet", "pose_sheet"), +} + +TARGET_TITLES: dict[str, str] = { + "portrait": "Portrait/headshot", + "bust": "Half body", + "fullbody": "Full body", + "sheet": "Design sheet", + "all": "All generation nodes", +} + + +def normalize_prompt_plan( + payload: Any, + workflow_nodes: list[WorkflowNode] | None = None, +) -> dict[str, Any]: + if isinstance(payload, dict) and payload.get("schema_version") == PROMPT_PLAN_SCHEMA_VERSION: + normalized = deepcopy(payload) + if workflow_nodes: + matrix = normalized.get("matrix") if isinstance(normalized.get("matrix"), list) else [] + normalized["by_node_id"] = _index_by_node_id(matrix, workflow_nodes) + normalized["node_parameters"] = normalized["by_node_id"] + return normalized + + source = _extract_source_payload(payload) + matrix = [ + _normalize_matrix_item(item, index + 1) + for index, item in enumerate(_matrix_items(source)) + if isinstance(item, dict) + ] + guardrails = _normalize_guardrails(source, matrix) + conflicts = _collect_conflicts(matrix, guardrails) + by_target = {item["target_id"]: item for item in matrix} + by_node_key = _index_by_node_key(matrix) + by_node_id = _index_by_node_id(matrix, workflow_nodes or []) + return { + "schema_version": PROMPT_PLAN_SCHEMA_VERSION, + "target_prompt_language": "en", + "profile_id": _optional_text(source.get("profile_id")), + "profile_name": _optional_text(source.get("profile_name")), + "matrix": matrix, + "guardrails": guardrails, + "conflicts": conflicts, + "by_target": by_target, + "by_node_key": by_node_key, + "by_node_id": by_node_id, + "node_parameters": by_node_id, + "prompt_context": _prompt_context(matrix), + "summary": { + "target_count": len(matrix), + "ready_count": len([item for item in matrix if item["status"] == "ready"]), + "review_count": len([item for item in matrix if item["status"] == "review"]), + "conflict_count": len(conflicts), + "negative_prompt": guardrails["negative_prompt"], + "target_node_keys": sorted(by_node_key), + }, + } + + +def prompt_plan_for_node( + workflow_metadata: dict[str, Any], + node: WorkflowNode, + workflow_nodes: list[WorkflowNode] | None = None, +) -> dict[str, Any]: + normalized = normalize_prompt_plan(workflow_metadata.get("prompt_plan"), workflow_nodes) + return node_prompt_plan_view(normalized, node) + + +def node_prompt_plan_view(normalized: dict[str, Any], node: WorkflowNode) -> dict[str, Any]: + by_node_id = normalized.get("by_node_id") if isinstance(normalized.get("by_node_id"), dict) else {} + by_node_key = normalized.get("by_node_key") if isinstance(normalized.get("by_node_key"), dict) else {} + node_view = by_node_id.get(node.node_id) + if not isinstance(node_view, dict): + node_view = _node_view(by_node_key.get(node.node_key, []), node.node_id, node.node_key, node.title) + return { + "schema_version": normalized.get("schema_version", PROMPT_PLAN_SCHEMA_VERSION), + "target_prompt_language": normalized.get("target_prompt_language", "en"), + "summary": normalized.get("summary", {}), + "guardrails": normalized.get("guardrails", _normalize_guardrails({}, [])), + "conflicts": normalized.get("conflicts", []), + "target_node_id": node.node_id, + "target_node_key": node.node_key, + "target_node_title": node.title, + **node_view, + } + + +def _extract_source_payload(payload: Any) -> dict[str, Any]: + if not isinstance(payload, dict): + return {} + for key in ("prompt_plan", "prompt_matrix"): + value = payload.get(key) + if isinstance(value, dict): + return value + return payload + + +def _matrix_items(source: dict[str, Any]) -> list[Any]: + for key in ("matrix", "items", "targets", "updates"): + value = source.get(key) + if isinstance(value, list): + return value + return [] + + +def _normalize_matrix_item(item: dict[str, Any], ordinal: int) -> dict[str, Any]: + raw_target = _optional_text(item.get("target") or item.get("target_id") or item.get("id")) or f"target_{ordinal:02d}" + target_id = _canonical_target(raw_target) + target_node_keys = _target_node_keys(item, target_id) + positive_prompt = _optional_text( + item.get("positive_prompt") + or item.get("prompt") + or item.get("positive_patch") + or item.get("prompt_patch") + ) + negative_prompt = _optional_text(item.get("negative_prompt") or item.get("negative") or item.get("negative_patch")) + conflicts = _string_list(item.get("conflicts") or item.get("conflict")) + status = _status(item.get("status"), conflicts) + return { + "id": _optional_text(item.get("id")) or target_id, + "target_id": target_id, + "target_title": _optional_text(item.get("title") or item.get("target_title")) or TARGET_TITLES.get(target_id, raw_target), + "target_label": _optional_text(item.get("target_label") or item.get("target")) or raw_target, + "target_node_keys": target_node_keys, + "positive_prompt": positive_prompt or "", + "positive_patch": _optional_text(item.get("positive_patch") or item.get("prompt_patch")) or positive_prompt or "", + "negative_prompt": negative_prompt or "", + "negative_patch": _optional_text(item.get("negative_patch")) or negative_prompt or "", + "weight_suggestions": _weight_suggestions(item), + "weight_basis": _optional_text(item.get("weight_basis") or item.get("weightBasis")) or "", + "inheritance_fields": _string_list(item.get("inheritance_fields") or item.get("inherits")), + "inheritance_changes": _string_list(item.get("inheritance_changes")), + "conflicts": conflicts, + "status": status, + "next_action": _optional_text(item.get("next_action")) or ("review_conflicts" if conflicts else "apply_prompt_plan"), + } + + +def _normalize_guardrails(source: dict[str, Any], matrix: list[dict[str, Any]]) -> dict[str, Any]: + raw_guards = source.get("guardrails") or source.get("guards") or source.get("guard_checks") or [] + guard_items = raw_guards if isinstance(raw_guards, list) else [] + negative_prompt = _optional_text(source.get("negative_prompt") or source.get("negative")) + if not negative_prompt: + negative_prompt = next((item["negative_prompt"] for item in matrix if item.get("negative_prompt")), "") + checks = [_normalize_guard_item(item) for item in guard_items if isinstance(item, dict)] + if negative_prompt and not any(item["key"] == "negative_prompt" for item in checks): + checks.insert( + 0, + { + "key": "negative_prompt", + "label": "Negative prompt", + "value": negative_prompt, + "status": "ready", + "severity": "info", + }, + ) + return { + "negative_prompt": negative_prompt or "", + "checks": checks, + "language_policy": { + "source_language": "auto", + "target_prompt_language": "en", + "translation_mode": "backend_llm", + }, + } + + +def _normalize_guard_item(item: dict[str, Any]) -> dict[str, str]: + label = _optional_text(item.get("label") or item.get("key")) or "Guard" + status = _status(item.get("status"), _string_list(item.get("conflicts"))) + return { + "key": _key(label), + "label": label, + "value": _optional_text(item.get("value")) or "", + "status": status, + "severity": _optional_text(item.get("severity")) or ("warn" if status == "review" else "info"), + } + + +def _index_by_node_key(matrix: list[dict[str, Any]]) -> dict[str, list[dict[str, Any]]]: + result: dict[str, list[dict[str, Any]]] = {} + for item in matrix: + for node_key in item["target_node_keys"]: + result.setdefault(node_key, []).append(item) + return result + + +def _index_by_node_id( + matrix: list[dict[str, Any]], + workflow_nodes: list[WorkflowNode], +) -> dict[str, dict[str, Any]]: + by_node_key = _index_by_node_key(matrix) + return { + node.node_id: _node_view(by_node_key.get(node.node_key, []), node.node_id, node.node_key, node.title) + for node in workflow_nodes + if by_node_key.get(node.node_key) + } + + +def _node_view( + items: list[dict[str, Any]], + node_id: str | None, + node_key: str, + title: str, +) -> dict[str, Any]: + positive_parts = [item["positive_patch"] for item in items if item.get("positive_patch")] + negative_parts = [item["negative_patch"] for item in items if item.get("negative_patch")] + return { + "target_node_id": node_id, + "target_node_key": node_key, + "target_node_title": title, + "has_prompt_plan": bool(items), + "matrix_items": deepcopy(items), + "positive_prompt_patches": positive_parts, + "negative_prompt_patches": negative_parts, + "prompt_context": ", ".join(positive_parts), + "generation_params": { + "prompt_patch": ", ".join(positive_parts), + "negative_patch": ", ".join(negative_parts), + "weight_suggestions": [suggestion for item in items for suggestion in item["weight_suggestions"]], + "inheritance_fields": _unique_texts(field for item in items for field in item["inheritance_fields"]), + }, + } + + +def _target_node_keys(item: dict[str, Any], target_id: str) -> list[str]: + explicit = _string_list(item.get("target_node_keys") or item.get("node_keys")) + if explicit: + return explicit + return list(TARGET_NODE_KEYS.get(target_id, TARGET_NODE_KEYS.get("all", ()))) + + +def _canonical_target(value: str) -> str: + normalized = value.strip().lower().replace("-", "_").replace(" ", "_") + aliases = { + "头像": "portrait", + "头像节点": "portrait", + "portrait": "portrait", + "headshot": "portrait", + "face": "portrait", + "半身": "bust", + "半身节点": "bust", + "bust": "bust", + "halfbody": "bust", + "half_body": "bust", + "全身": "fullbody", + "全身节点": "fullbody", + "body": "fullbody", + "fullbody": "fullbody", + "full_body": "fullbody", + "设定图": "sheet", + "设定图节点": "sheet", + "sheet": "sheet", + "design_sheet": "sheet", + "all": "all", + } + return aliases.get(normalized, normalized) + + +def _weight_suggestions(item: dict[str, Any]) -> list[dict[str, Any]]: + raw = item.get("weight_suggestions") or item.get("weight_changes") or item.get("weight") + if isinstance(raw, str): + return [{"label": raw, "value": None, "reason": _optional_text(item.get("weight_basis") or item.get("weightBasis")) or ""}] + if not isinstance(raw, list): + return [] + suggestions = [] + for entry in raw: + if isinstance(entry, dict): + suggestions.append( + { + "label": _optional_text(entry.get("label") or entry.get("target")) or "weight", + "value": entry.get("value"), + "reason": _optional_text(entry.get("reason") or entry.get("basis")) or "", + } + ) + else: + text = _optional_text(entry) + if text: + suggestions.append({"label": text, "value": None, "reason": ""}) + return suggestions + + +def _collect_conflicts(matrix: list[dict[str, Any]], guardrails: dict[str, Any]) -> list[dict[str, str]]: + conflicts = [ + {"target_id": item["target_id"], "message": conflict, "severity": "warn"} + for item in matrix + for conflict in item["conflicts"] + ] + for guard in guardrails.get("checks", []): + if guard.get("status") == "review": + conflicts.append({"target_id": "guardrails", "message": guard.get("value", ""), "severity": "warn"}) + return conflicts + + +def _prompt_context(matrix: list[dict[str, Any]]) -> list[dict[str, str]]: + return [ + { + "target_id": item["target_id"], + "target_node_keys": ", ".join(item["target_node_keys"]), + "positive_patch": item["positive_patch"], + "negative_patch": item["negative_patch"], + } + for item in matrix + if item.get("positive_patch") or item.get("negative_patch") + ] + + +def _status(value: Any, conflicts: list[str]) -> str: + status = str(value or "").strip().lower() + if status in {"ready", "review", "pending"}: + return status + return "review" if conflicts else "ready" + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return [value] if value.strip() else [] + if not isinstance(value, list): + return [] + return _unique_texts(str(item).strip() for item in value if str(item or "").strip()) + + +def _unique_texts(values: Any) -> list[str]: + result: list[str] = [] + for value in values: + text = str(value or "").strip() + if text and text not in result: + result.append(text) + return result + + +def _optional_text(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + + +def _key(value: str) -> str: + return "_".join(value.lower().replace("/", " ").replace("-", " ").split()) or "guard" diff --git a/backend/services/queue_service.py b/backend/services/queue_service.py index 7bc4d39..a600d89 100644 --- a/backend/services/queue_service.py +++ b/backend/services/queue_service.py @@ -24,8 +24,12 @@ TASK_STATUSES = { "failed", "cancelled", "dead_letter", + "retrying", + "superseded", } +RETRYABLE_TASK_STATUSES = {"failed", "dead_letter", "cancelled"} + class QueueService: def __init__(self, repository: JobRepository, audit_service: AuditService, event_bus: EventBus | None = None): @@ -125,34 +129,44 @@ class QueueService: return self.repository.get_by_idempotency_key(normalized_key) def cancel_job(self, task_id: str, ctx: RequestContext, *, reason: str | None = None) -> GenerationTask: - before = self.get_job(task_id) - if before.status == "cancelled": - return before - if before.status in {"completed", "succeeded"}: - raise ApiError( - ErrorCode.TASK_NOT_CANCELABLE, - "task cannot be cancelled from current status", - {"task_id": task_id, "status": before.status}, - ) - self._validate_task_status_transition(before.status, "cancelled") - after = self.repository.update( - task_id, - { - "status": "cancelled", - "error_message": _optional_text(reason) or before.error_message, - "metadata": { - **before.metadata, - "cancellation": { - "requested_by": ctx.actor_id or "system", - "requested_at": _now(), - "reason": _optional_text(reason) or "cancelled", - "previous_status": before.status, + while True: + before = self.get_job(task_id) + if before.status == "cancelled": + return before + if before.status == "retrying": + raise ApiError( + ErrorCode.TASK_NOT_CANCELABLE, + "task retry execution is already in progress", + {"task_id": task_id, "status": before.status}, + ) + if before.status in {"completed", "succeeded", "superseded"}: + raise ApiError( + ErrorCode.TASK_NOT_CANCELABLE, + "task cannot be cancelled from current status", + {"task_id": task_id, "status": before.status}, + ) + self._validate_task_status_transition(before.status, "cancelled") + after = self.repository.update_if_status( + task_id, + {before.status}, + { + "status": "cancelled", + "error_message": _optional_text(reason) or before.error_message, + "metadata": { + **before.metadata, + "cancellation": { + "requested_by": ctx.actor_id or "system", + "requested_at": _now(), + "reason": _optional_text(reason) or "cancelled", + "previous_status": before.status, + }, }, }, - }, - ) - if after is None: - raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "task not found", {"task_id": task_id}) + ) + if after is not None: + break + # The task changed after it was read. Re-evaluate the current + # state instead of applying cancellation from a stale snapshot. self.audit_service.record( ctx=ctx, action="cancel_task", @@ -164,12 +178,155 @@ class QueueService: self._publish_status_change(before, after) return after + def update_task_if_status( + self, + task_id: str, + expected_statuses: set[str] | tuple[str, ...] | list[str], + payload: dict[str, Any], + ctx: RequestContext, + *, + require_workflow_not_cancelled: bool = False, + ) -> GenerationTask | None: + """Update a task only if the status observed by this caller still owns it.""" + + normalized_statuses = { + str(status).strip() + for status in expected_statuses + if str(status).strip() + } + before = self.get_job(task_id) + if before.status not in normalized_statuses: + return None + status = payload.get("status") + if status is not None: + self._validate_task_status_transition(before.status, str(status)) + if require_workflow_not_cancelled: + after = self.repository.update_if_status_unless_workflow_cancelled( + task_id, + {before.status}, + payload, + ) + else: + after = self.repository.update_if_status(task_id, {before.status}, payload) + if after is None: + return None + self.audit_service.record( + ctx=ctx, + action="update_task", + target_type="generation_task", + target_id=task_id, + before_snapshot=before.to_dict(), + after_snapshot=after.to_dict(), + metadata={"compare_and_set": True}, + ) + if before.status != after.status: + self._publish_status_change(before, after) + self._publish("generation.task_updated", {"task_id": task_id, "status": after.status}) + return after + + def cancel_workflow_jobs( + self, + workflow_id: str, + ctx: RequestContext, + *, + reason: str = "workflow cancelled", + ) -> dict[str, Any]: + """CAS-cancel every queued/running task currently owned by a workflow.""" + + cancelled_ids: list[str] = [] + skipped: list[dict[str, str]] = [] + for status in ("queued", "running"): + while True: + candidates, _ = self.repository.list( + status=status, + workflow_id=workflow_id, + page=1, + page_size=100, + ) + if not candidates: + break + changed = 0 + for candidate in candidates: + try: + cancelled = self.cancel_job(candidate.task_id, ctx, reason=reason) + if cancelled.status == "cancelled": + cancelled_ids.append(cancelled.task_id) + except ApiError as exc: + current = self.get_job(candidate.task_id) + skipped.append( + { + "task_id": candidate.task_id, + "status": current.status, + "error_code": exc.error_code.value, + } + ) + current = self.get_job(candidate.task_id) + if current.status != status: + changed += 1 + if changed == 0: + break + return { + "workflow_id": workflow_id, + "cancelled_task_ids": cancelled_ids, + "skipped": skipped, + } + def is_cancelled(self, task_id: str) -> bool: return self.get_job(task_id).status == "cancelled" - def retry_job(self, task_id: str, ctx: RequestContext) -> GenerationTask: + def claim_queued_task( + self, + task_id: str, + ctx: RequestContext, + *, + worker_id: str, + ) -> GenerationTask | None: + """Atomically claim one queued task for a worker.""" + before = self.get_job(task_id) - if before.status not in {"failed", "dead_letter", "cancelled"}: + if before.status != "queued": + return None + self._validate_task_status_transition(before.status, "running") + attempt = _int_or_default(before.metadata.get("worker_attempt"), 0) + 1 + after = self.repository.update_if_status( + task_id, + {"queued"}, + { + "status": "running", + "progress": max(before.progress, 1), + "metadata": { + **before.metadata, + "worker_attempt": attempt, + "worker": { + "worker_id": worker_id, + "status": "running", + "claimed_at": _now(), + }, + }, + }, + ) + if after is None: + return None + self.audit_service.record( + ctx=ctx, + action="claim_task", + target_type="generation_task", + target_id=task_id, + before_snapshot=before.to_dict(), + after_snapshot=after.to_dict(), + ) + self._publish_status_change(before, after) + return after + + def requeue_job(self, task_id: str, ctx: RequestContext) -> GenerationTask: + """Requeue a task for an explicitly managed QueueWorker. + + Web retry routes use GenerationJobRetryService instead because the + AstrBot plugin does not start a persistent QueueWorker by default. + """ + + before = self.get_job(task_id) + if before.status not in RETRYABLE_TASK_STATUSES: raise ApiError( ErrorCode.TASK_NOT_RETRYABLE, "task cannot be retried from current status", @@ -182,14 +339,59 @@ class QueueService: "status": "queued", "progress": 0, "error_message": None, - "metadata": {**before.metadata, "retry_requested": True}, + "metadata": {**before.metadata, "requeue_requested": True}, }, ) if after is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "task not found", {"task_id": task_id}) self.audit_service.record( ctx=ctx, - action="retry_task", + action="requeue_task", + target_type="generation_task", + target_id=task_id, + before_snapshot=before.to_dict(), + after_snapshot=after.to_dict(), + ) + self._publish_status_change(before, after) + return after + + def begin_retry(self, task_id: str, ctx: RequestContext, *, operation_id: str) -> GenerationTask: + """Atomically reserve a failed task for synchronous retry execution.""" + + before = self.get_job(task_id) + if before.status not in RETRYABLE_TASK_STATUSES: + raise ApiError( + ErrorCode.TASK_NOT_RETRYABLE, + "task cannot be retried from current status", + {"task_id": task_id, "status": before.status}, + ) + self._validate_task_status_transition(before.status, "retrying") + retry_metadata = { + "operation_id": operation_id, + "status": "running", + "requested_by": ctx.actor_id or "system", + "requested_at": _now(), + "previous_status": before.status, + } + after = self.repository.update_if_status( + task_id, + {before.status}, + { + "status": "retrying", + "progress": 0, + "metadata": {**before.metadata, "retry_execution": retry_metadata}, + }, + ) + if after is None: + current = self.get_job(task_id) + raise ApiError( + ErrorCode.TASK_NOT_RETRYABLE, + "task retry was already claimed or its status changed", + {"task_id": task_id, "status": current.status}, + ) + self.audit_service.record( + ctx=ctx, + action="begin_retry_task", target_type="generation_task", target_id=task_id, before_snapshot=before.to_dict(), @@ -223,7 +425,7 @@ class QueueService: counts = self.repository.count_by_status() return { "pending": counts.get("created", 0) + counts.get("queued", 0), - "running": counts.get("running", 0), + "running": counts.get("running", 0) + counts.get("retrying", 0), "failed": counts.get("failed", 0), "dead_letter": counts.get("dead_letter", 0), } @@ -231,16 +433,16 @@ class QueueService: def recent_jobs(self, limit: int = 10) -> list[dict[str, Any]]: return [task.to_dict() for task in self.repository.recent(limit=limit)] - def retry_dead_letters(self, ctx: RequestContext, *, limit: int = 50) -> dict[str, Any]: + def requeue_dead_letters(self, ctx: RequestContext, *, limit: int = 50) -> dict[str, Any]: jobs = self.list_jobs(status="dead_letter", page=1, page_size=_batch_limit(limit)).items - retried = [] + requeued = [] errors = [] for job in jobs: try: - retried.append(self.retry_job(job.task_id, ctx).to_dict()) + requeued.append(self.requeue_job(job.task_id, ctx).to_dict()) except ApiError as exc: errors.append({"task_id": job.task_id, "error_code": exc.error_code.value, "message": exc.message}) - return {"retried": retried, "errors": errors, "total": len(retried)} + return {"requeued": requeued, "errors": errors, "total": len(requeued)} def move_failed_to_dead_letter(self, ctx: RequestContext, *, limit: int = 50) -> dict[str, Any]: jobs = self.list_jobs(status="failed", page=1, page_size=_batch_limit(limit)).items @@ -311,3 +513,10 @@ def _batch_limit(value: int) -> int: except (TypeError, ValueError): limit = 50 return min(max(limit, 1), 100) + + +def _int_or_default(value: Any, default: int) -> int: + try: + return int(value) + except (TypeError, ValueError): + return default diff --git a/backend/services/quick_profile_service.py b/backend/services/quick_profile_service.py index a2d4b3b..64d87aa 100644 --- a/backend/services/quick_profile_service.py +++ b/backend/services/quick_profile_service.py @@ -6,13 +6,16 @@ the prompt expansion internals without changing the API contract. from __future__ import annotations -from typing import Any +from typing import TYPE_CHECKING, Any from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode from backend.services.profile_service import ProfileService from core.identity import RequestContext +if TYPE_CHECKING: + from backend.services.workflow_service import WorkflowService + class QuickProfileService: def create_draft(self, payload: dict[str, Any]) -> dict[str, Any]: @@ -71,6 +74,7 @@ class QuickProfileService: payload: dict[str, Any], profile_service: ProfileService, ctx: RequestContext, + workflow_service: "WorkflowService | None" = None, ) -> dict[str, Any]: draft = payload.get("draft") if isinstance(payload.get("draft"), dict) else {} preview = payload.get("profile_preview") if isinstance(payload.get("profile_preview"), dict) else None @@ -103,10 +107,42 @@ class QuickProfileService: }, ctx, ) - return { + result = { "draft_id": draft_id or None, "profile": profile.to_dict(), } + should_create_workflow = bool(payload.get("create_workflow", workflow_service is not None)) + if should_create_workflow: + if workflow_service is None: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "\u7f3a\u5c11\u5feb\u901f\u89d2\u8272\u5de5\u4f5c\u6d41\u670d\u52a1", + {"field": "workflow_service"}, + ) + workflow_detail = workflow_service.create_workflow( + profile.profile_id, + { + "workflow_type": "quick_profile", + "display_name": _text(payload.get("workflow_display_name")) + or f"{name} \u5feb\u901f\u89d2\u8272\u5de5\u4f5c\u6d41", + "status": _text(payload.get("workflow_status")) or "draft", + "metadata": { + "source": "quick_profile", + "profile_id": profile.profile_id, + **({"draft_id": draft_id} if draft_id else {}), + "profile_preview": preview, + **( + {"questionnaire": payload["questionnaire"]} + if isinstance(payload.get("questionnaire"), dict) + else {} + ), + }, + }, + ctx, + ) + result["workflow"] = workflow_detail.workflow.to_dict() + result["nodes"] = [node.to_dict() for node in workflow_detail.nodes] + return result def _text(value: Any) -> str: diff --git a/backend/services/reference_binding_mapper.py b/backend/services/reference_binding_mapper.py new file mode 100644 index 0000000..037b646 --- /dev/null +++ b/backend/services/reference_binding_mapper.py @@ -0,0 +1,69 @@ +"""Map semantic reference-image responsibilities to adapter-native inputs.""" + +from __future__ import annotations + +from typing import Any, Mapping + + +class ReferenceBindingMapper: + def map(self, bindings: list[dict[str, Any]], route_resolution: Mapping[str, Any]) -> dict[str, Any]: + settings = route_resolution.get("settings") if isinstance(route_resolution.get("settings"), dict) else {} + capabilities = route_resolution.get("capabilities") if isinstance(route_resolution.get("capabilities"), dict) else {} + max_references = _positive_int(settings.get("max_reference_images") or capabilities.get("max_reference_images"), 8) + ordered = sorted( + [dict(item) for item in bindings if isinstance(item, dict)], + key=lambda item: (-_float(item.get("weight"), 0.5), str(item.get("material_id") or item.get("image_uri") or "")), + ) + active, downgraded = ordered[:max_references], ordered[max_references:] + supports_roles = bool(settings.get("supports_reference_roles", capabilities.get("supports_reference_roles", False))) + dialect = str(settings.get("reference_dialect") or capabilities.get("reference_dialect") or "text_constraints") + native: list[dict[str, Any]] = [] + constraints: list[str] = [] + for index, binding in enumerate(active): + role = str(binding.get("role") or "reference") + use_for = _strings(binding.get("use_for")) + do_not = _strings(binding.get("do_not_inherit")) + if supports_roles: + native.append({ + "index": index, + "role": role, + "use_for": use_for, + "exclude": do_not, + "weight": min(max(_float(binding.get("weight"), 0.5), 0.0), 1.0), + "dialect": dialect, + }) + else: + constraint = f"Reference image {index + 1} is for {', '.join(use_for) or role} only" + if do_not: + constraint += f"; do not inherit {', '.join(do_not)}" + constraints.append(constraint) + return { + "dialect": dialect, + "supports_native_roles": supports_roles, + "native_bindings": native, + "text_constraints": constraints, + "active_count": len(active), + "downgraded": [{"material_id": item.get("material_id"), "reason": "adapter_reference_limit", "fallback": "text_capsule"} for item in downgraded], + } + + +def _strings(value: Any) -> list[str]: + if isinstance(value, str): + value = value.split(",") + if not isinstance(value, (list, tuple, set)): + return [] + return [str(item).strip() for item in value if str(item).strip()] + + +def _float(value: Any, default: float) -> float: + try: + return float(value) + except (TypeError, ValueError): + return default + + +def _positive_int(value: Any, default: int) -> int: + try: + return max(int(value), 0) + except (TypeError, ValueError): + return default diff --git a/backend/services/review_service.py b/backend/services/review_service.py index 84f48a9..b91bfdc 100644 --- a/backend/services/review_service.py +++ b/backend/services/review_service.py @@ -6,8 +6,8 @@ from typing import Any from backend.dto.common import ApiError, ErrorCode from backend.services.workflow_service import WorkflowService -from backend.services.settings_service import _dump_yaml from core.config_service import ConfigService +from core.config_writer import config_file_lock, dump_yaml, write_text_atomic from database.db_helper import _load_yaml_subset from database.repositories.image_repository import ImageRepository from engines.review_flow import ReviewFlowEngine, ReviewPolicy @@ -36,17 +36,18 @@ class ReviewService: validation = self.validate_policy(payload) if not validation["valid"]: raise ApiError(ErrorCode.VALIDATION_FAILED, "review policy validation failed", validation) - config = self._load_config_mapping() - config["review_policy"] = validation["policy"] - self.config_service.config_path.parent.mkdir(parents=True, exist_ok=True) - self.config_service.config_path.write_text(_dump_yaml(config), encoding="utf-8") + with config_file_lock(self.config_service.config_path): + config = self._load_config_mapping() + config["review_policy"] = validation["policy"] + write_text_atomic(self.config_service.config_path, dump_yaml(config)) self.review_engine = ReviewFlowEngine(ReviewPolicy.from_mapping(validation["policy"])[0]) return {"saved": True, "policy": self.get_policy()} def review_node(self, node_id: str) -> dict[str, Any]: node = self.workflow_service.get_node(node_id) + workflow = self.workflow_service.get_workflow(node.workflow_id).workflow images = self.image_repository.list_by_node(node_id, include_archived=True) - return self.review_engine.review_node(node, images) + return self.review_engine.review_node(node, images, workflow.metadata) def review_workflow(self, workflow_id: str) -> dict[str, Any]: detail = self.workflow_service.get_workflow(workflow_id) diff --git a/backend/services/settings_service.py b/backend/services/settings_service.py index 470f25a..355285e 100644 --- a/backend/services/settings_service.py +++ b/backend/services/settings_service.py @@ -9,9 +9,17 @@ from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode from core.config_service import ConfigService +from core.config_writer import config_file_lock, dump_yaml, write_text_atomic from core.audit_service import AuditService from core.identity import RequestContext -from database.db_helper import REPO_ROOT, _load_yaml_subset +from core.runtime_paths import is_default_config_path, resolve_plugin_path +from database.db_helper import ( + Database, + DatabaseSettings, + get_runtime_config_overrides, + load_config_file_payload, + load_config_payload, +) SENSITIVE_KEY_PARTS = ("password", "token", "secret", "api_key", "apikey", "key", "dsn") @@ -55,6 +63,111 @@ class SettingsService: _field("private_companion_integration", "bool", label="私伴插件集成"), ], }, + { + "key": "generation", + "title": "生图提供方", + "fields": [ + _field("default_model", "string", label="默认路由名"), + _field("provider.provider_id", "string", label="提供方 ID"), + _field("provider.base_url", "string", label="API 根地址"), + _field("provider.image_endpoint", "string", label="生图 endpoint"), + _field("provider.model_id", "string", label="模型 ID"), + _field("provider.route_name", "string", label="路由名"), + _field("provider.api_key_env_var", "string", label="API Key 环境变量"), + _field("provider.timeout_seconds", "number", label="超时秒数"), + ], + }, + { + "key": "vision", + "title": "素材视觉识别", + "fields": [ + _field("default_route", "string", label="默认 Vision 路由名"), + _field("provider.provider_id", "string", label="Vision 提供方 ID(留空复用生图)"), + _field("provider.base_url", "string", label="Vision API 根地址(留空复用生图)"), + _field("provider.vision_endpoint", "string", label="Vision endpoint"), + _field("provider.model_id", "string", label="Vision 模型 ID"), + _field("provider.route_name", "string", label="Vision 路由名"), + _field( + "provider.api_key_env_var", + "string", + label="Vision API Key 环境变量(留空复用生图)", + ), + _field("provider.timeout_seconds", "number", label="Vision 超时秒数"), + _field("provider.enabled", "bool", label="启用 Vision 路由"), + ], + }, + { + "key": "model_assignments", + "title": "模型分工", + "fields": [ + _field("material_recognition", "string", label="素材解析路由"), + _field("profile_parse", "string", label="角色导入解析路由"), + _field("persona_split", "string", label="人设拆解路由"), + _field("prompt_planning", "string", label="提示词规划路由"), + _field("cross_review_primary", "string", label="交叉审核主模型路由"), + _field("cross_review_secondary", "string", label="交叉审核复核模型路由"), + _field("cross_review_routes", "list", label="交叉审核模型路由列表"), + _field("quick_generation", "string", label="快速生图路由"), + _field("portrait_generation", "string", label="头像生成路由"), + _field("halfbody_generation", "string", label="半身生成路由"), + _field("fullbody_generation", "string", label="全身生成路由"), + _field("sheet_generation", "string", label="设定图生成路由"), + _field("hires_fix", "string", label="高清修复路由"), + ], + }, + { + "key": "recognition_prompts", + "title": "识别提示词", + "fields": [ + _field("default_instruction", "text", label="默认素材识别指令"), + _field("category_requirements", "list", label="分类附加要求"), + ], + }, + { + "key": "prompt_governance", + "title": "提示词与内容分流", + "fields": [ + _field("translation.enabled", "bool", label="启用 LLM 提示词转译"), + _field("translation.model_route", "string", label="提示词转译模型路由"), + _field("translation.template_version", "string", label="转译模板版本"), + _field("translation.entrypoints.web", "bool", label="Web 允许转译"), + _field("translation.entrypoints.qq", "bool", label="QQ 允许转译"), + _field("translation.entrypoints.telegram", "bool", label="Telegram 允许转译"), + _field("translation.entrypoints.discord", "bool", label="Discord 允许转译"), + _field("moderation.llm_review_enabled", "bool", label="启用 LLM 内容复核"), + _field("moderation.model_route", "string", label="内容审核模型路由"), + _field("moderation.failure_policy", "select", label="审核失败策略", options=["fail_closed", "confirm"]), + _field("routing.safe_route", "string", label="普通生图路由"), + _field("routing.nsfw_route", "string", label="NSFW 生图路由"), + _field("routing.adult_auto_route_enabled", "bool", label="允许成人内容自动分流"), + _field("routing.entrypoints.web", "bool", label="Web 允许成人路由"), + _field("routing.entrypoints.qq", "bool", label="QQ 允许成人路由"), + _field("routing.entrypoints.telegram", "bool", label="Telegram 允许成人路由"), + _field("routing.entrypoints.discord", "bool", label="Discord 允许成人路由"), + _field("image_moderation.enabled", "bool", label="启用生成后图片审核"), + _field("image_moderation.model_route", "string", label="图片审核模型路由"), + _field("retention.raw_prompt_days", "number", label="原始提示词保留天数"), + _field("retention.translated_prompt_days", "number", label="转译提示词保留天数"), + _field("retention.moderation_record_days", "number", label="审核记录保留天数"), + _field("retention.redact_sensitive_logs", "bool", label="日志默认脱敏"), + ], + }, + { + "key": "chat_access", + "title": "聊天权限", + "fields": [ + _field("group_allowlist", "list", label="群聊白名单"), + _field("user_allowlist", "list", label="用户白名单"), + _field("adult_group_allowlist", "list", label="成人内容群聊白名单"), + _field("adult_user_allowlist", "list", label="成人内容用户白名单"), + _field("user_concurrency_limit", "number", label="用户并发任务上限"), + _field("group_concurrency_limit", "number", label="群聊并发任务上限"), + _field("daily_user_generation_limit", "number", label="用户每日生成上限"), + _field("daily_group_generation_limit", "number", label="群聊每日生成上限"), + _field("allow_2k", "bool", label="允许聊天命令使用 2K"), + _field("allow_4k", "bool", label="允许聊天命令使用 4K"), + ], + }, { "key": "storage", "title": "存储", @@ -106,6 +219,48 @@ class SettingsService: raise ApiError(ErrorCode.VALIDATION_FAILED, "config must be an object", {"field": "config"}) errors: list[dict[str, Any]] = [] warnings: list[dict[str, Any]] = [] + generation = config.get("generation", {}) + generation_provider: Mapping[str, Any] = {} + if isinstance(generation, Mapping): + provider = generation.get("provider", {}) + provider = provider if isinstance(provider, Mapping) else {} + generation_provider = provider + route_name = str(provider.get("route_name") or generation.get("default_model") or "").strip() + model_id = str(provider.get("model_id") or generation.get("default_model") or "").strip() + base_url = str(provider.get("base_url") or "").strip() + if any((route_name, model_id, base_url)): + for field, value in ( + ("generation.default_model", route_name), + ("generation.provider.model_id", model_id), + ("generation.provider.base_url", base_url), + ): + if not value: + errors.append({"field": field, "message": "configured generation provider field is required"}) + vision = config.get("vision", {}) + if isinstance(vision, Mapping): + provider = vision.get("provider", {}) + provider = provider if isinstance(provider, Mapping) else {} + route_name = str(provider.get("route_name") or vision.get("default_route") or "").strip() + model_id = str(provider.get("model_id") or "").strip() + base_url = str(provider.get("base_url") or generation_provider.get("base_url") or "").strip() + explicitly_configured = any( + str(provider.get(key) or "").strip() + for key in ( + "provider_id", + "base_url", + "model_id", + "api_key_env_var", + "api_key_secret_ref", + ) + ) + if explicitly_configured: + for field, value in ( + ("vision.default_route", route_name), + ("vision.provider.model_id", model_id), + ("vision.provider.base_url", base_url), + ): + if not value: + errors.append({"field": field, "message": "configured vision provider field is required"}) database = config.get("database", {}) if isinstance(database, Mapping): backend = str(database.get("backend", "auto")).lower() @@ -128,6 +283,28 @@ class SettingsService: "message": "postgresql backend is selected but connection fields are incomplete", } ) + governance = config.get("prompt_governance", {}) + if isinstance(governance, Mapping): + routing = governance.get("routing", {}) + if isinstance(routing, Mapping): + safe_route = str(routing.get("safe_route") or "").strip() + nsfw_route = str(routing.get("nsfw_route") or "").strip() + if routing.get("adult_auto_route_enabled") and not nsfw_route: + errors.append({"field": "prompt_governance.routing.nsfw_route", "message": "NSFW route is required when adult auto routing is enabled"}) + if safe_route and nsfw_route and safe_route == nsfw_route: + errors.append({"field": "prompt_governance.routing", "message": "safe and NSFW routes must use different configured routes"}) + translation = governance.get("translation", {}) + if isinstance(translation, Mapping) and translation.get("enabled") and not str(translation.get("model_route") or "").strip(): + warnings.append({"field": "prompt_governance.translation.model_route", "message": "translation is enabled without a dedicated model route; runtime will safely use the original prompt"}) + retention = governance.get("retention", {}) + if isinstance(retention, Mapping): + for key in ("raw_prompt_days", "translated_prompt_days", "moderation_record_days"): + try: + days = int(retention.get(key, 0)) + except (TypeError, ValueError): + days = -1 + if days < 0: + errors.append({"field": f"prompt_governance.retention.{key}", "message": "retention days must be zero or greater"}) return {"valid": not errors, "errors": errors, "warnings": warnings} def diff(self, candidate: Mapping[str, Any]) -> dict[str, Any]: @@ -136,7 +313,7 @@ class SettingsService: if not isinstance(incoming, Mapping): raise ApiError(ErrorCode.VALIDATION_FAILED, "config must be an object", {"field": "config"}) before = _flatten(current) - after = _flatten(dict(incoming)) + after = _flatten(_merge_masked(current, incoming)) keys = sorted(set(before) | set(after)) changes = [] for key in keys: @@ -177,15 +354,29 @@ class SettingsService: validation = self.validate(candidate) if not validation["valid"]: raise ApiError(ErrorCode.VALIDATION_FAILED, "config validation failed", validation) - before = self._load_current_mapping() - self._record_snapshot(before, ctx, "before_save") incoming = candidate.get("config", candidate) if not isinstance(incoming, Mapping): raise ApiError(ErrorCode.VALIDATION_FAILED, "config must be an object", {"field": "config"}) - merged = _merge_masked(before, incoming) - self.config_service.config_path.parent.mkdir(parents=True, exist_ok=True) - self.config_service.config_path.write_text(_dump_yaml(merged), encoding="utf-8") - after = self._load_current_mapping() + with config_file_lock(self.config_service.config_path): + before = self._load_current_mapping() + before_file = load_config_file_payload(self.config_service.config_path) + merged_effective = _merge_masked(before, incoming) + conflicts = _runtime_override_conflicts(merged_effective, self.config_service.config_path) + if conflicts: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "these settings are managed by AstrBot; update them in the AstrBot plugin settings", + {"runtime_managed_fields": conflicts}, + ) + merged = _restore_runtime_managed_fields( + merged_effective, + before_file, + get_runtime_config_overrides() if is_default_config_path(self.config_service.config_path) else {}, + ) + _initialize_database_if_changed(before_file, merged) + self._record_snapshot(before_file, ctx, "before_save") + write_text_atomic(self.config_service.config_path, dump_yaml(merged)) + after = self._load_current_mapping() if self.audit_service: self.audit_service.record( ctx=ctx, @@ -279,11 +470,25 @@ class SettingsService: validation = self.validate({"config": config}) if not validation["valid"]: raise ApiError(ErrorCode.VALIDATION_FAILED, "snapshot config validation failed", validation) - before = self._load_current_mapping() - self._record_snapshot(before, ctx, "before_rollback") - self.config_service.config_path.parent.mkdir(parents=True, exist_ok=True) - self.config_service.config_path.write_text(_dump_yaml(config), encoding="utf-8") - after = self._load_current_mapping() + with config_file_lock(self.config_service.config_path): + before = self._load_current_mapping() + before_file = load_config_file_payload(self.config_service.config_path) + conflicts = _runtime_override_conflicts(config, self.config_service.config_path) + if conflicts: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "snapshot changes settings managed by AstrBot", + {"runtime_managed_fields": conflicts}, + ) + rollback_config = _restore_runtime_managed_fields( + config, + before_file, + get_runtime_config_overrides() if is_default_config_path(self.config_service.config_path) else {}, + ) + _initialize_database_if_changed(before_file, rollback_config) + self._record_snapshot(before_file, ctx, "before_rollback") + write_text_atomic(self.config_service.config_path, dump_yaml(rollback_config)) + after = self._load_current_mapping() self.audit_service.record( ctx=ctx, action="rollback_settings", @@ -318,10 +523,7 @@ class SettingsService: return json.loads(self.schema_path.read_text(encoding="utf-8")) def _load_current_mapping(self) -> dict[str, Any]: - path = self.config_service.config_path - if not path.exists(): - return {} - return _load_yaml_subset(path.read_text(encoding="utf-8")) + return load_config_payload(self.config_service.config_path) def _record_snapshot(self, config: Mapping[str, Any], ctx: RequestContext, reason: str) -> str | None: if not self.audit_service: @@ -427,10 +629,7 @@ def _mask_path_value(value: Any) -> Any: def _resolve_path(path: str | Path) -> Path: - resolved = Path(path) - if not resolved.is_absolute(): - resolved = REPO_ROOT / resolved - return resolved + return resolve_plugin_path(path) def _merge_masked(current: Mapping[str, Any], incoming: Mapping[str, Any]) -> dict[str, Any]: @@ -445,30 +644,86 @@ def _merge_masked(current: Mapping[str, Any], incoming: Mapping[str, Any]) -> di return result -def _dump_yaml(value: Mapping[str, Any], indent: int = 0) -> str: - lines: list[str] = [] - prefix = " " * indent +def _runtime_override_conflicts(incoming: Mapping[str, Any], config_path: Path) -> list[str]: + if not is_default_config_path(config_path): + return [] + overrides = _flatten(get_runtime_config_overrides()) + if not overrides: + return [] + candidate = _flatten(incoming) + return sorted(path for path, value in overrides.items() if path in candidate and candidate[path] != value) + + +def _restore_runtime_managed_fields( + candidate: Mapping[str, Any], + file_config: Mapping[str, Any], + runtime_overrides: Mapping[str, Any], +) -> dict[str, Any]: + result = _copy_mapping(candidate) + file_values = _flatten(file_config) + for path in _flatten(runtime_overrides): + if path in file_values: + _set_nested_value(result, path.split("."), file_values[path]) + else: + _delete_nested_value(result, path.split(".")) + return result + + +def _copy_mapping(value: Mapping[str, Any]) -> dict[str, Any]: + copied: dict[str, Any] = {} for key, child in value.items(): if isinstance(child, Mapping): - lines.append(f"{prefix}{key}:") - lines.append(_dump_yaml(child, indent + 2).rstrip()) + copied[str(key)] = _copy_mapping(child) elif isinstance(child, list): - lines.append(f"{prefix}{key}:") - for item in child: - lines.append(f"{prefix} - {_format_scalar(item)}") + copied[str(key)] = [ + _copy_mapping(item) if isinstance(item, Mapping) else item + for item in child + ] else: - lines.append(f"{prefix}{key}: {_format_scalar(child)}") - return "\n".join(lines) + "\n" + copied[str(key)] = child + return copied -def _format_scalar(value: Any) -> str: - if isinstance(value, bool): - return "true" if value else "false" - if value is None: - return "null" - if isinstance(value, (int, float)): - return str(value) - text = str(value) - if text == "" or any(char in text for char in [":", "#", "{", "}", "[", "]", ","]) or text.strip() != text: - return '"' + text.replace("\\", "\\\\").replace('"', '\\"') + '"' - return text +def _set_nested_value(target: dict[str, Any], parts: list[str], value: Any) -> None: + cursor = target + for part in parts[:-1]: + child = cursor.get(part) + if not isinstance(child, dict): + child = {} + cursor[part] = child + cursor = child + if parts: + cursor[parts[-1]] = value + + +def _delete_nested_value(target: dict[str, Any], parts: list[str]) -> None: + if not parts: + return + cursor: dict[str, Any] = target + parents: list[tuple[dict[str, Any], str]] = [] + for part in parts[:-1]: + child = cursor.get(part) + if not isinstance(child, dict): + return + parents.append((cursor, part)) + cursor = child + cursor.pop(parts[-1], None) + for parent, key in reversed(parents): + child = parent.get(key) + if isinstance(child, dict) and not child: + parent.pop(key, None) + else: + break + + +def _initialize_database_if_changed(before: Mapping[str, Any], after: Mapping[str, Any]) -> None: + before_database = before.get("database") if isinstance(before.get("database"), Mapping) else {} + after_database = after.get("database") if isinstance(after.get("database"), Mapping) else {} + if dict(before_database) == dict(after_database): + return + Database(DatabaseSettings.from_mapping(after)).initialize() + + +# Compatibility alias for existing tests and services that imported the +# previous private serializer. New code should import core.config_writer. +_dump_yaml = dump_yaml diff --git a/backend/services/wardrobe_service.py b/backend/services/wardrobe_service.py index 11347f7..26fa790 100644 --- a/backend/services/wardrobe_service.py +++ b/backend/services/wardrobe_service.py @@ -2,11 +2,14 @@ from __future__ import annotations +import sqlite3 from typing import Any from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode from backend.dto.wardrobe_dto import WardrobeItem +from backend.services.composition_builder import CompositionBuilder, CompositionPlan +from backend.services.material_normalization_service import MaterialNormalizationService from core.audit_service import AuditService from core.identity import RequestContext from database.repositories.material_repository import MaterialRepository @@ -24,14 +27,18 @@ class WardrobeService: profile_repository: ProfileRepository, material_repository: MaterialRepository, audit_service: AuditService, + composition_builder: CompositionBuilder | None = None, + normalization_service: MaterialNormalizationService | None = None, ): self.wardrobe_repository = wardrobe_repository self.profile_repository = profile_repository self.material_repository = material_repository self.audit_service = audit_service + self.composition_builder = composition_builder or CompositionBuilder() + self.normalization_service = normalization_service or MaterialNormalizationService() def import_material(self, profile_id: str, payload: dict[str, Any], ctx: RequestContext) -> WardrobeItem: - self._ensure_profile(profile_id) + self._ensure_profile(profile_id, ctx) material_id = str(payload.get("material_id") or "").strip() wardrobe_category = str(payload.get("wardrobe_category") or "").strip() if not material_id: @@ -45,18 +52,33 @@ class WardrobeService: material = self.material_repository.get_material(material_id) if material is None or material.recognition_status != "confirmed": raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "confirmed material not found", {"material_id": material_id}) - item = self.wardrobe_repository.create( - { - "wardrobe_item_id": payload.get("wardrobe_item_id") or new_wardrobe_item_id(), - "profile_id": profile_id, - "material_id": material_id, - "wardrobe_category": wardrobe_category, - "tags": payload.get("tags") or material.tags, - "season": payload.get("season"), - "style": payload.get("style"), - "is_favorite": payload.get("is_favorite", False), - } + duplicate = next( + ( + existing + for existing in self.wardrobe_repository.list_by_profile(profile_id, wardrobe_category) + if existing.material_id == material_id + ), + None, ) + if duplicate is not None: + raise _duplicate_wardrobe_item_error(profile_id, material_id, wardrobe_category, duplicate.wardrobe_item_id) + try: + item = self.wardrobe_repository.create( + { + "wardrobe_item_id": payload.get("wardrobe_item_id") or new_wardrobe_item_id(), + "profile_id": profile_id, + "material_id": material_id, + "wardrobe_category": wardrobe_category, + "tags": payload.get("tags") or material.tags, + "season": payload.get("season"), + "style": payload.get("style"), + "is_favorite": payload.get("is_favorite", False), + } + ) + except Exception as exc: + if not _is_unique_constraint_error(exc): + raise + raise _duplicate_wardrobe_item_error(profile_id, material_id, wardrobe_category) from None self.audit_service.record( ctx=ctx, action="add_wardrobe_item", @@ -66,12 +88,17 @@ class WardrobeService: ) return item - def list_items(self, profile_id: str, wardrobe_category: str | None = None) -> list[WardrobeItem]: - self._ensure_profile(profile_id) + def list_items( + self, + profile_id: str, + wardrobe_category: str | None, + ctx: RequestContext, + ) -> list[WardrobeItem]: + self._ensure_profile(profile_id, ctx) return self.wardrobe_repository.list_by_profile(profile_id, wardrobe_category) def delete_item(self, profile_id: str, wardrobe_item_id: str, ctx: RequestContext) -> WardrobeItem: - self._ensure_profile(profile_id) + self._ensure_profile(profile_id, ctx) before = self.wardrobe_repository.get(wardrobe_item_id) if before is None or before.profile_id != profile_id: raise ApiError( @@ -90,15 +117,92 @@ class WardrobeService: ) return deleted + def list_templates(self, profile_id: str, ctx: RequestContext) -> list[dict[str, Any]]: + profile = self._ensure_profile(profile_id, ctx) + templates = profile.metadata.get("wardrobe_templates") if isinstance(profile.metadata.get("wardrobe_templates"), list) else [] + return [dict(item) for item in templates if isinstance(item, dict)] + + def save_template(self, profile_id: str, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: + profile = self._ensure_profile(profile_id, ctx) + name = str(payload.get("name") or "").strip() + material_ids = [str(item).strip() for item in payload.get("material_ids", []) if str(item).strip()] if isinstance(payload.get("material_ids"), list) else [] + if not name or not material_ids: + raise ApiError(ErrorCode.VALIDATION_FAILED, "template name and material_ids are required", {"fields": ["name", "material_ids"]}) + owned_material_ids = {item.material_id for item in self.wardrobe_repository.list_by_profile(profile_id)} + missing = [material_id for material_id in material_ids if material_id not in owned_material_ids] + if missing: + raise ApiError(ErrorCode.VALIDATION_FAILED, "template contains materials outside this wardrobe", {"material_ids": missing}) + template_id = str(payload.get("template_id") or f"outfit_{uuid4().hex}") + templates = self.list_templates(profile_id, ctx) + template = { + "template_id": template_id, + "name": name, + "material_ids": list(dict.fromkeys(material_ids)), + "description": str(payload.get("description") or "").strip(), + "scene": str(payload.get("scene") or "").strip(), + "season": str(payload.get("season") or "").strip(), + "style": str(payload.get("style") or "").strip(), + "material_options": payload.get("material_options") if isinstance(payload.get("material_options"), dict) else {}, + } + templates = [item for item in templates if item.get("template_id") != template_id] + [template] + updated = self.profile_repository.update(profile_id, {"metadata": {**profile.metadata, "wardrobe_templates": templates}}) + if updated is None: + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "profile not found", {"profile_id": profile_id}) + self.audit_service.record(ctx=ctx, action="save_wardrobe_template", target_type="character_profile", target_id=profile_id, after_snapshot=template) + return template + + def delete_template(self, profile_id: str, template_id: str, ctx: RequestContext) -> dict[str, Any]: + profile = self._ensure_profile(profile_id, ctx) + templates = self.list_templates(profile_id, ctx) + removed = next((item for item in templates if item.get("template_id") == template_id), None) + if removed is None: + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "wardrobe template not found", {"template_id": template_id}) + self.profile_repository.update(profile_id, {"metadata": {**profile.metadata, "wardrobe_templates": [item for item in templates if item.get("template_id") != template_id]}}) + self.audit_service.record(ctx=ctx, action="delete_wardrobe_template", target_type="character_profile", target_id=profile_id, before_snapshot=removed) + return removed + + def apply_template(self, profile_id: str, template_id: str, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: + template = next((item for item in self.list_templates(profile_id, ctx) if item.get("template_id") == template_id), None) + if template is None: + raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "wardrobe template not found", {"template_id": template_id}) + wardrobe = self.wardrobe_repository.list_by_profile(profile_id) + item_by_material = {item.material_id: item for item in wardrobe} + selected: list[dict[str, Any]] = [] + options = template.get("material_options") if isinstance(template.get("material_options"), dict) else {} + for material_id in template.get("material_ids") or []: + item = item_by_material.get(str(material_id)) + material = self.material_repository.get_material(str(material_id)) + if item is None or material is None or material.recognition_status != "confirmed": + continue + selected.append({ + "wardrobe_item_id": item.wardrobe_item_id, + "material_id": material.material_id, + "wardrobe_category": item.wardrobe_category, + "display_name": material.display_name, + "prompt": material.prompt, + "negative_prompt": material.negative_prompt, + "image_uri": material.image_uri, + "metadata": self.normalization_service.ensure_legacy_structures(prompt=material.prompt, negative_prompt=material.negative_prompt, tags=material.tags, metadata=material.metadata, category_hint=item.wardrobe_category), + **(options.get(material.material_id) if isinstance(options.get(material.material_id), dict) else {}), + }) + result = self.composition_builder.build({ + "materials": selected, + "user_prompt": str(payload.get("prompt") or template.get("scene") or "").strip(), + "prompt_budget": payload.get("prompt_budget") or 4000, + "max_reference_images": payload.get("max_reference_images") or 8, + }).to_dict() + self.audit_service.record(ctx=ctx, action="apply_wardrobe_template", target_type="character_profile", target_id=profile_id, after_snapshot={"template_id": template_id, "composition": result}) + return {"profile_id": profile_id, "template": template, "composition": result} + def match(self, profile_id: str, payload: dict[str, Any], ctx: RequestContext) -> dict[str, Any]: - self._ensure_profile(profile_id) + self._ensure_profile(profile_id, ctx) categories = _normalize_categories(payload.get("categories")) if not categories: categories = list(DEFAULT_WARDROBE_CATEGORIES) context = _match_context(payload) selected: list[dict[str, Any]] = [] - prompt_parts: list[str] = [] missing_categories: list[str] = [] + material_options = payload.get("material_options") if isinstance(payload.get("material_options"), dict) else {} for category in categories: candidates = self.wardrobe_repository.list_by_profile(profile_id, str(category)) if not candidates: @@ -107,7 +211,7 @@ class WardrobeService: scored_candidates = [] for index, item in enumerate(candidates): material = self.material_repository.get_material(item.material_id) - if material is None: + if material is None or material.recognition_status != "confirmed": continue score, breakdown = _score_candidate(item, material, context) scored_candidates.append((score, -index, item, material, breakdown)) @@ -123,6 +227,15 @@ class WardrobeService: "wardrobe_category": item.wardrobe_category, "display_name": material.display_name, "prompt": material.prompt, + "negative_prompt": material.negative_prompt, + "image_uri": material.image_uri, + "metadata": self.normalization_service.ensure_legacy_structures( + prompt=material.prompt, + negative_prompt=material.negative_prompt, + tags=material.tags, + metadata=material.metadata, + category_hint=item.wardrobe_category, + ), "tags": item.tags, "season": item.season, "style": item.style, @@ -130,14 +243,28 @@ class WardrobeService: "usage_count": item.usage_count, "score": score, "score_breakdown": breakdown, + **( + material_options.get(material.material_id) + if isinstance(material_options.get(material.material_id), dict) + else {} + ), } ) - if material.prompt: - prompt_parts.append(material.prompt) fallback_strategy = _fallback_outfit_strategy(payload, categories, missing_categories, selected, context) - outfit_prompt = ", ".join(prompt_parts) - if not outfit_prompt: - outfit_prompt = fallback_strategy["prompt"] + composition = self.composition_builder.build( + CompositionPlan( + materials=selected, + identity_prompt=str(payload.get("identity_prompt") or "").strip(), + user_prompt=fallback_strategy["prompt"] if fallback_strategy["reason"] else "", + negative_prompt=str(payload.get("negative_prompt") or "").strip(), + prompt_budget=max(int(payload.get("prompt_budget") or 4000), 256), + max_reference_images=max(int(payload.get("max_reference_images") or 8), 0), + visible_slots=[str(item).strip() for item in payload.get("visible_slots", []) if str(item).strip()] if isinstance(payload.get("visible_slots"), list) else [], + season=str(payload.get("season") or "").strip(), + scene=str(payload.get("scene") or "").strip(), + ) + ) + outfit_prompt = composition.prompt or fallback_strategy["prompt"] result = { "profile_id": profile_id, "selected_materials": selected, @@ -147,6 +274,7 @@ class WardrobeService: "fallback_reason": fallback_strategy["reason"], "fallback_prompt": fallback_strategy["prompt"] if fallback_strategy["reason"] else None, "fallback_strategy": fallback_strategy, + "composition": composition.to_dict(), "match_details": { "requested_categories": categories, "missing_categories": missing_categories, @@ -166,16 +294,39 @@ class WardrobeService: ) return result - def _ensure_profile(self, profile_id: str) -> None: + def _ensure_profile(self, profile_id: str, ctx: RequestContext): profile = self.profile_repository.get(profile_id) - if profile is None or profile.status == "deleted": + if profile is None or profile.status == "deleted" or profile.owner_user_id != ctx.actor_id: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "profile not found", {"profile_id": profile_id}) + return profile def new_wardrobe_item_id() -> str: return f"wardrobe_{uuid4().hex}" +def _duplicate_wardrobe_item_error( + profile_id: str, + material_id: str, + wardrobe_category: str, + wardrobe_item_id: str | None = None, +) -> ApiError: + details: dict[str, Any] = { + "profile_id": profile_id, + "material_id": material_id, + "wardrobe_category": wardrobe_category, + } + if wardrobe_item_id: + details["wardrobe_item_id"] = wardrobe_item_id + return ApiError(ErrorCode.VALIDATION_FAILED, "material already exists in wardrobe category", details) + + +def _is_unique_constraint_error(exc: Exception) -> bool: + if isinstance(exc, sqlite3.IntegrityError): + return True + return getattr(exc, "sqlstate", None) == "23505" or getattr(exc, "pgcode", None) == "23505" + + def _normalize_categories(value: Any) -> list[str]: if not isinstance(value, list): return [] diff --git a/backend/services/worker_service.py b/backend/services/worker_service.py index ad77943..fe11af8 100644 --- a/backend/services/worker_service.py +++ b/backend/services/worker_service.py @@ -8,7 +8,6 @@ from dataclasses import dataclass, field from datetime import datetime, timedelta, timezone from typing import Any -from backend.dto.common import ApiError from backend.dto.job_dto import GenerationTask from backend.services.queue_service import QueueService from core.identity import RequestContext @@ -186,27 +185,13 @@ class QueueWorker: for candidate in queued: if _retry_deferred(candidate): continue - try: - attempt = _int(candidate.metadata.get("worker_attempt"), 0) + 1 - return self.queue_service.update_task( - candidate.task_id, - { - "status": "running", - "progress": max(candidate.progress, 1), - "metadata": { - **candidate.metadata, - "worker_attempt": attempt, - "worker": { - "worker_id": self.worker_id, - "status": "running", - "claimed_at": _now(), - }, - }, - }, - ctx, - ) - except ApiError: - continue + claimed = self.queue_service.claim_queued_task( + candidate.task_id, + ctx, + worker_id=self.worker_id, + ) + if claimed is not None: + return claimed return None def _handle_failure(self, task: GenerationTask, exc: Exception, ctx: RequestContext) -> WorkerRunResult: diff --git a/backend/services/workflow_service.py b/backend/services/workflow_service.py index 11efce4..fefb65e 100644 --- a/backend/services/workflow_service.py +++ b/backend/services/workflow_service.py @@ -7,6 +7,15 @@ from uuid import uuid4 from backend.dto.common import ApiError, ErrorCode from backend.dto.workflow_dto import ProfileWorkflow, WorkflowBranch, WorkflowDetail, WorkflowNode +from backend.services.material_binding_schema_service import ( + MATERIAL_BINDING_SCHEMA_VERSION, + normalize_material_bindings, +) +from backend.services.prompt_plan_schema_service import ( + PROMPT_PLAN_SCHEMA_VERSION, + normalize_prompt_plan, +) +from backend.services.queue_service import QueueService from core.audit_service import AuditService from core.event_bus import EventBus from core.fsm import node_transition, workflow_transition @@ -28,9 +37,11 @@ NODE_STATUSES = { "completed", "selected", "failed", + "cancelled", "skipped", "discarded", } +LOCKED_NODE_STATUSES = {"completed", "selected"} class WorkflowService: @@ -67,26 +78,25 @@ class WorkflowService: "workflow display_name is required", {"field": "display_name"}, ) - workflow = self.workflow_repository.create_workflow( + status = payload.get("status", "draft") + self._validate_workflow_status(status) + workflow_id = payload.get("workflow_id") or new_workflow_id() + node_payloads = self._initial_node_payloads(workflow_id, workflow_type, payload) + workflow, nodes = self.workflow_repository.create_workflow_with_nodes( { - "workflow_id": payload.get("workflow_id") or new_workflow_id(), + "workflow_id": workflow_id, "profile_id": profile_id, "workflow_type": workflow_type, "display_name": display_name, "mode": payload.get("mode"), - "status": payload.get("status", "draft"), + "status": status, + "current_node_id": node_payloads[0]["node_id"] if node_payloads else None, "current_branch": payload.get("current_branch", "main"), "session_key": payload.get("session_key") or ctx.session_key, "metadata": payload.get("metadata") or {}, - } + }, + node_payloads, ) - nodes = self._create_initial_nodes(workflow, payload, ctx) - if nodes: - self.workflow_repository.update_workflow( - workflow.workflow_id, - {"current_node_id": nodes[0].node_id}, - ) - workflow = self.workflow_repository.get_workflow(workflow.workflow_id) or workflow self.audit_service.record( ctx=ctx, action="create_workflow", @@ -140,6 +150,8 @@ class WorkflowService: update_payload = dict(payload) if "display_name" in update_payload: update_payload["display_name"] = str(update_payload["display_name"]).strip() + if "metadata" in update_payload and isinstance(update_payload["metadata"], dict): + update_payload["metadata"] = self._normalize_workflow_metadata(before, update_payload["metadata"]) after = self.workflow_repository.update_workflow(workflow_id, update_payload) if after is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "workflow not found", {"workflow_id": workflow_id}) @@ -164,6 +176,101 @@ class WorkflowService: self._publish("workflow.updated", {"workflow_id": workflow_id, "profile_id": after.profile_id}) return after + def cancel_workflow( + self, + workflow_id: str, + ctx: RequestContext, + queue_service: QueueService, + ) -> ProfileWorkflow: + """Cancel a workflow and CAS-cancel its active tasks and nodes.""" + + while True: + before = self.get_workflow(workflow_id).workflow + if before.status == "cancelled": + after = before + break + self._validate_workflow_status_transition(before.status, "cancelled") + after = self.workflow_repository.update_workflow_if_status( + workflow_id, + {before.status}, + {"status": "cancelled"}, + ) + if after is not None: + self.audit_service.record( + ctx=ctx, + action="cancel_workflow", + target_type="profile_workflow", + target_id=workflow_id, + before_snapshot=before.to_dict(), + after_snapshot=after.to_dict(), + ) + self._publish( + "workflow.status_changed", + { + "workflow_id": workflow_id, + "profile_id": after.profile_id, + "before_status": before.status, + "after_status": after.status, + }, + ) + self._publish("workflow.updated", {"workflow_id": workflow_id, "profile_id": after.profile_id}) + break + + queue_service.cancel_workflow_jobs(workflow_id, ctx, reason="workflow cancelled") + for node in self.workflow_repository.list_nodes(workflow_id): + if node.status not in {"queued", "running"}: + continue + node_metadata = node.metadata if isinstance(node.metadata, dict) else {} + last_task_id = str(node_metadata.get("last_task_id") or "").strip() + if last_task_id: + try: + last_task = queue_service.get_job(last_task_id) + except ApiError as exc: + if exc.error_code != ErrorCode.RESOURCE_NOT_FOUND: + raise + else: + # A task that committed before workflow cancellation owns + # its terminal node update. Tasks still active were CAS- + # cancelled above and their nodes are cancelled here. + if last_task.status in {"completed", "succeeded"}: + continue + cancelled_node = self.workflow_repository.update_node_if_status( + node.node_id, + {node.status}, + { + "status": "cancelled", + "metadata": { + **node_metadata, + "workflow_cancellation": { + "workflow_id": workflow_id, + "requested_by": ctx.actor_id or "system", + }, + }, + }, + ) + if cancelled_node is None: + continue + self.audit_service.record( + ctx=ctx, + action="cancel_node", + target_type="workflow_node", + target_id=node.node_id, + before_snapshot=node.to_dict(), + after_snapshot=cancelled_node.to_dict(), + metadata={"source": "workflow.cancel"}, + ) + self._publish( + "workflow.node_status_changed", + { + "workflow_id": workflow_id, + "node_id": node.node_id, + "before_status": node.status, + "after_status": cancelled_node.status, + }, + ) + self._publish("workflow.node_updated", {"workflow_id": workflow_id, "node_id": node.node_id}) + return after + def create_branch( self, workflow_id: str, @@ -180,8 +287,11 @@ class WorkflowService: "branch already exists", {"workflow_id": workflow_id, "branch_name": branch_name}, ) - base_branch_name = str(payload.get("base_branch_name") or workflow.current_branch or "main") + base_branch_name = str(payload.get("base_branch_name") or workflow.current_branch or "main").strip() base_node_id = payload.get("base_node_id") or workflow.current_node_id + copy_nodes = payload.get("copy_nodes", True) + if copy_nodes or base_node_id: + self._resolve_branch_copy_base(workflow_id, base_branch_name, base_node_id) branch = self.workflow_repository.create_branch( { "branch_id": payload.get("branch_id") or new_branch_id(), @@ -194,8 +304,8 @@ class WorkflowService: } ) copied_nodes = [] - if payload.get("copy_nodes", True): - copied_nodes = self._copy_branch_nodes(workflow_id, base_branch_name, branch_name, ctx) + if copy_nodes: + copied_nodes = self._copy_branch_nodes(workflow_id, base_branch_name, branch_name, ctx, base_node_id) self.audit_service.record( ctx=ctx, action="create_branch", @@ -324,8 +434,18 @@ class WorkflowService: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "node not found", {"node_id": node_id}) return node - def update_node(self, node_id: str, payload: dict[str, Any], ctx: RequestContext) -> WorkflowNode: + def update_node( + self, + node_id: str, + payload: dict[str, Any], + ctx: RequestContext, + *, + allow_locked_adoption: bool = False, + ) -> WorkflowNode: before = self.get_node(node_id) + if before.status in LOCKED_NODE_STATUSES and _node_payload_changes(before, payload): + if not (allow_locked_adoption and _is_locked_image_adoption(before, payload)): + raise locked_node_edit_error(before) if "status" in payload: self._validate_node_status_transition(before.status, str(payload["status"])) if "title" in payload and not str(payload["title"]).strip(): @@ -336,6 +456,8 @@ class WorkflowService: after = self.workflow_repository.update_node(node_id, update_payload) if after is None: raise ApiError(ErrorCode.RESOURCE_NOT_FOUND, "node not found", {"node_id": node_id}) + self._sync_material_bindings_from_node(after) + self._sync_prompt_plan_from_node(after) self.audit_service.record( ctx=ctx, action="update_node", @@ -357,23 +479,41 @@ class WorkflowService: self._publish("workflow.node_updated", {"workflow_id": after.workflow_id, "node_id": node_id}) return after - def _create_initial_nodes( + def _initial_node_payloads( self, - workflow: ProfileWorkflow, + workflow_id: str, + workflow_type: str, payload: dict[str, Any], - ctx: RequestContext, - ) -> list[WorkflowNode]: + ) -> list[dict[str, Any]]: if payload.get("auto_create_nodes", True) is False: return [] - return [ - self.create_node( - workflow.workflow_id, - template, - ctx, - audit=False, + nodes: list[dict[str, Any]] = [] + for template in _initial_node_templates(workflow_type): + node_key = str(template.get("node_key") or "").strip() + title = str(template.get("title") or "").strip() + status = str(template.get("status") or "draft") + if not node_key or not title or status not in NODE_STATUSES: + raise RuntimeError(f"invalid initial node template for {workflow_type}") + nodes.append( + { + "node_id": template.get("node_id") or new_node_id(), + "workflow_id": workflow_id, + "node_key": node_key, + "title": title, + "description": template.get("description"), + "step_name": template.get("step_name"), + "node_type": template.get("node_type", "generation"), + "status": status, + "branch_name": template.get("branch_name", "main"), + "prompt": template.get("prompt"), + "negative_prompt": template.get("negative_prompt"), + "model_route": template.get("model_route"), + "selected_image_id": template.get("selected_image_id"), + "sort_order": template.get("sort_order", 0), + "metadata": template.get("metadata") or {}, + } ) - for template in _initial_node_templates(workflow.workflow_type) - ] + return nodes def _copy_branch_nodes( self, @@ -381,10 +521,9 @@ class WorkflowService: from_branch: str, to_branch: str, ctx: RequestContext, + base_node_id: str | None = None, ) -> list[WorkflowNode]: - source_nodes = self.workflow_repository.list_nodes(workflow_id, branch_name=from_branch) - if not source_nodes and from_branch != "main": - source_nodes = self.workflow_repository.list_nodes(workflow_id, branch_name="main") + source_nodes, base_sort_order = self._resolve_branch_copy_base(workflow_id, from_branch, base_node_id) return [ self.create_node( workflow_id, @@ -394,13 +533,14 @@ class WorkflowService: "description": node.description, "step_name": node.step_name, "node_type": node.node_type, - "status": "draft", + "status": _branch_copy_status(node, base_sort_order), "branch_name": to_branch, "prompt": node.prompt, "negative_prompt": node.negative_prompt, "model_route": node.model_route, + "selected_image_id": _branch_copy_selected_image_id(node, base_sort_order), "sort_order": node.sort_order, - "metadata": {**node.metadata, "copied_from_node_id": node.node_id}, + "metadata": _branch_copy_metadata(node, base_sort_order), }, ctx, audit=False, @@ -408,6 +548,26 @@ class WorkflowService: for node in source_nodes ] + def _resolve_branch_copy_base( + self, + workflow_id: str, + from_branch: str, + base_node_id: str | None = None, + ) -> tuple[list[WorkflowNode], int | None]: + source_nodes = self.workflow_repository.list_nodes(workflow_id, branch_name=from_branch) + if not source_nodes and from_branch != "main": + source_nodes = self.workflow_repository.list_nodes(workflow_id, branch_name="main") + base_node = None + if base_node_id: + base_node = next((node for node in source_nodes if node.node_id == base_node_id), None) + if base_node is None: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "base_node_id does not belong to the source branch", + {"workflow_id": workflow_id, "branch_name": from_branch, "base_node_id": base_node_id}, + ) + return source_nodes, base_node.sort_order if base_node else None + def _ensure_profile_exists(self, profile_id: str) -> None: profile = self.profile_repository.get(profile_id) if profile is None or profile.status == "deleted": @@ -428,12 +588,7 @@ class WorkflowService: return workflow def _validate_workflow_status_transition(self, current: str, target: str) -> None: - if target not in WORKFLOW_STATUSES: - raise ApiError( - ErrorCode.VALIDATION_FAILED, - "unsupported workflow status", - {"field": "status", "allowed": sorted(WORKFLOW_STATUSES)}, - ) + self._validate_workflow_status(target) transition = workflow_transition(current, target) if not transition.allowed: raise ApiError( @@ -442,6 +597,14 @@ class WorkflowService: {"current": current, "target": target, "reason": transition.reason}, ) + def _validate_workflow_status(self, status: Any) -> None: + if not isinstance(status, str) or status not in WORKFLOW_STATUSES: + raise ApiError( + ErrorCode.VALIDATION_FAILED, + "unsupported workflow status", + {"field": "status", "allowed": sorted(WORKFLOW_STATUSES)}, + ) + def _validate_node_status_transition(self, current: str, target: str) -> None: if target not in NODE_STATUSES: raise ApiError( @@ -461,6 +624,74 @@ class WorkflowService: if self.event_bus: self.event_bus.publish(event_type, payload) + def _normalize_workflow_metadata( + self, + workflow: ProfileWorkflow, + metadata: dict[str, Any], + ) -> dict[str, Any]: + material_bindings = metadata.get("material_bindings") + prompt_plan = metadata.get("prompt_plan") or metadata.get("prompt_matrix") + if material_bindings is None and prompt_plan is None: + return metadata + nodes = self.workflow_repository.list_nodes(workflow.workflow_id, branch_name=workflow.current_branch) + normalized = dict(metadata) + if material_bindings is not None: + normalized["material_binding_schema_version"] = MATERIAL_BINDING_SCHEMA_VERSION + normalized["material_bindings"] = normalize_material_bindings(material_bindings, nodes) + if prompt_plan is not None: + normalized["prompt_plan_schema_version"] = PROMPT_PLAN_SCHEMA_VERSION + normalized["prompt_plan"] = normalize_prompt_plan(prompt_plan, nodes) + normalized.pop("prompt_matrix", None) + return normalized + + def _sync_material_bindings_from_node(self, node: WorkflowNode) -> None: + if node.node_key != "material_bind" or not isinstance(node.metadata, dict): + return + raw_bindings = node.metadata.get("material_bindings") + if raw_bindings is None: + return + workflow = self.workflow_repository.get_workflow(node.workflow_id) + if workflow is None: + return + nodes = self.workflow_repository.list_nodes(node.workflow_id, branch_name=node.branch_name) + normalized = normalize_material_bindings(raw_bindings, nodes) + self.workflow_repository.update_workflow( + workflow.workflow_id, + { + "metadata": { + **(workflow.metadata or {}), + "material_binding_schema_version": MATERIAL_BINDING_SCHEMA_VERSION, + "material_binding_source_node_id": node.node_id, + "material_binding_updated_at": node.updated_at, + "material_bindings": normalized, + } + }, + ) + + def _sync_prompt_plan_from_node(self, node: WorkflowNode) -> None: + if node.node_key != "prompt_plan" or not isinstance(node.metadata, dict): + return + raw_plan = node.metadata.get("prompt_plan") or node.metadata.get("prompt_matrix") + if raw_plan is None: + return + workflow = self.workflow_repository.get_workflow(node.workflow_id) + if workflow is None: + return + nodes = self.workflow_repository.list_nodes(node.workflow_id, branch_name=node.branch_name) + normalized = normalize_prompt_plan(raw_plan, nodes) + self.workflow_repository.update_workflow( + workflow.workflow_id, + { + "metadata": { + **(workflow.metadata or {}), + "prompt_plan_schema_version": PROMPT_PLAN_SCHEMA_VERSION, + "prompt_plan_source_node_id": node.node_id, + "prompt_plan_updated_at": node.updated_at, + "prompt_plan": normalized, + } + }, + ) + def new_workflow_id() -> str: return f"workflow_{uuid4().hex}" @@ -474,6 +705,108 @@ def new_branch_id() -> str: return f"branch_{uuid4().hex}" +def locked_node_edit_error(node: WorkflowNode) -> ApiError: + return ApiError( + ErrorCode.VALIDATION_FAILED, + "completed node is locked; create a workflow branch before editing", + { + "node_id": node.node_id, + "workflow_id": node.workflow_id, + "branch_name": node.branch_name, + "status": node.status, + "suggested_action": "create_branch", + "base_node_id": node.node_id, + }, + ) + + +def is_repeatable_single_image_node(node: WorkflowNode) -> bool: + return bool( + node.node_key == "virtual_single_image" + or node.node_type == "virtual_single_image" + or (isinstance(node.metadata, dict) and node.metadata.get("virtual") is True) + ) + + +def _node_payload_changes(node: WorkflowNode, payload: dict[str, Any]) -> bool: + comparable = { + "title": node.title, + "description": node.description, + "step_name": node.step_name, + "node_type": node.node_type, + "status": node.status, + "branch_name": node.branch_name, + "prompt": node.prompt, + "negative_prompt": node.negative_prompt, + "model_route": node.model_route, + "selected_image_id": node.selected_image_id, + "sort_order": node.sort_order, + "metadata": node.metadata, + } + for key, value in payload.items(): + if key not in comparable: + continue + current = comparable[key] + if key == "sort_order": + try: + value = int(value) + except (TypeError, ValueError): + return True + if key == "title" and value is not None: + value = str(value).strip() + if value != current: + return True + return False + + +def _is_locked_image_adoption(node: WorkflowNode, payload: dict[str, Any]) -> bool: + allowed_keys = {"status", "selected_image_id"} + if any(key not in allowed_keys for key in payload): + return False + selected_image_id = payload.get("selected_image_id") + if not selected_image_id: + return False + target_status = payload.get("status", node.status) + if target_status != "selected": + return False + if node.status == "completed": + return True + return node.status == "selected" and node.selected_image_id == selected_image_id + + +def _branch_copy_status(node: WorkflowNode, base_sort_order: int | None) -> str: + if base_sort_order is None: + return "draft" + if node.sort_order < base_sort_order: + return node.status + return "draft" + + +def _branch_copy_selected_image_id(node: WorkflowNode, base_sort_order: int | None) -> str | None: + if base_sort_order is not None and node.sort_order < base_sort_order: + return node.selected_image_id + return None + + +def _branch_copy_metadata(node: WorkflowNode, base_sort_order: int | None) -> dict[str, Any]: + metadata = { + **node.metadata, + "copied_from_node_id": node.node_id, + } + if base_sort_order is None: + return metadata + locked_ancestor = node.sort_order < base_sort_order and node.status in LOCKED_NODE_STATUSES + metadata.update( + { + "branch_cut_base_sort_order": base_sort_order, + "branch_reset_from_status": node.status, + "branch_locked_ancestor": locked_ancestor, + "branch_editable_from_cut": node.sort_order >= base_sort_order, + } + ) + return metadata + + def _initial_node_templates(workflow_type: str) -> list[dict[str, Any]]: if workflow_type == "single_image": return [ diff --git a/config.yaml b/config.yaml index 1bc664c..0d59d25 100644 --- a/config.yaml +++ b/config.yaml @@ -1,7 +1,7 @@ plugin: name: astrbot_plugin_imagestudio display_name: ImageStudio - version: 3.0.0 + version: 1.0.0 features: quick_profile: true @@ -11,7 +11,112 @@ features: api_discovery: true private_companion_integration: false +generation: + # default_model 是 ImageStudio 路由名;route_name 留空时复用它。 + default_model: "" + provider: + provider_id: imagestudio_default + display_name: ImageStudio default provider + base_url: "" + image_endpoint: /v1/images/generations + model_id: "" + route_name: "" + api_key_env_var: "" + timeout_seconds: 180 + priority: 10 + enabled: true + +vision: + # 素材识别路由;provider 中留空的连接字段会复用 generation.provider。 + # 若地址、密钥或超时不同且 provider_id 留空,会自动派生独立 Vision provider ID,避免覆盖生图配置。 + # Vision 模型必须单独填写,不能假定生图模型具备视觉理解能力。 + default_route: material_vlm + provider: + provider_id: "" + display_name: "" + base_url: "" + vision_endpoint: /v1/chat/completions + model_id: "" + route_name: material_vlm + api_key_env_var: "" + timeout_seconds: 180 + priority: 10 + enabled: true + +# 按功能分配模型路由。留空时回退到 generation.default_model 或 vision.default_route。 +model_assignments: + material_recognition: material_vlm + profile_parse: "" + persona_split: "" + prompt_planning: "" + cross_review_primary: material_vlm + cross_review_secondary: material_vlm + cross_review_routes: + - material_vlm + - material_vlm + quick_generation: "" + portrait_generation: "" + halfbody_generation: "" + fullbody_generation: "" + sheet_generation: "" + hires_fix: "" + +recognition_prompts: + default_instruction: >- + 请先判断图片中素材的准确分类,再尽可能完整地提取所有可见事实,包括物品类型、主色与辅色、材质、纹理、结构、轮廓、图案、裁剪、缝线、褶皱、装饰、配件、状态、视角、光照及可复用的设计特征。不要把无法确认的推测写成事实,并明确区分确定信息与低置信度判断。根据识别结果生成一段细节充分、可直接用于生图且能够保留原素材设计的正向提示词,同时生成用于排除错误类别、错误材质、结构畸变、文字、水印和无关背景的负向提示词。仅返回 JSON,字段必须包含 prompt、negative_prompt、tags、confidence、description;tags 应简短、可检索,并包含分类、颜色、材质和关键设计元素。 + category_requirements: [] + +# 提示词转译、内容审核和普通/NSFW 模型动态分流。 +# 默认只启用本地规则审核;LLM 转译与成人自动分流必须由管理员明确开启。 +prompt_governance: + translation: + enabled: false + model_route: "" + template_version: "1.0" + entrypoints: + web: true + qq: false + telegram: false + discord: false + api: false + moderation: + llm_review_enabled: false + model_route: "" + failure_policy: fail_closed + routing: + safe_route: "" + nsfw_route: "" + adult_auto_route_enabled: false + entrypoints: + web: false + qq: false + telegram: false + discord: false + api: false + image_moderation: + enabled: false + model_route: "" + retention: + raw_prompt_days: 30 + translated_prompt_days: 90 + moderation_record_days: 180 + redact_sensitive_logs: true + +chat_access: + group_allowlist: [] + user_allowlist: [] + adult_group_allowlist: [] + adult_user_allowlist: [] + user_concurrency_limit: 2 + group_concurrency_limit: 5 + daily_user_generation_limit: 30 + daily_group_generation_limit: 100 + allow_2k: true + allow_4k: false + storage: + # 这里使用逻辑 data/... 路径;AstrBot 运行时映射到 + # data/plugin_data/astrbot_plugin_imagestudio/...,本地开发仍映射到仓库 data/。 db_path: data/db/imagestudio.db images_path: data/images thumbnails_path: data/thumbnails diff --git a/core/chat_access_policy.py b/core/chat_access_policy.py new file mode 100644 index 0000000..8f31f6b --- /dev/null +++ b/core/chat_access_policy.py @@ -0,0 +1,47 @@ +"""Chat command allowlist policy shared by AstrBot adapters.""" + +from __future__ import annotations + +from collections.abc import Mapping +from typing import Any + + +STRICT_ACTIONS = frozenset({"quick_profile", "character_workflow"}) +ANY_ALLOWLIST_ACTIONS = frozenset({"single_image"}) + + +def chat_action_allowed( + config: Mapping[str, Any] | None, + action: str, + *, + group_id: str | None, + user_id: str | None, +) -> bool: + """Return whether a chat action passes the configured allowlists. + + Quick profile and character workflow require both group and user matches. + Single-image generation accepts either match. Private chats have no group + match, so strict actions are intentionally unavailable there unless the + caller supplies a platform conversation/group identifier. + """ + + access = config.get("chat_access") if isinstance(config, Mapping) else None + access = access if isinstance(access, Mapping) else {} + groups = _normalized_ids(access.get("group_allowlist")) + users = _normalized_ids(access.get("user_allowlist")) + group_match = bool(group_id and str(group_id).strip() in groups) + user_match = bool(user_id and str(user_id).strip() in users) + normalized_action = str(action or "").strip().lower() + if normalized_action in STRICT_ACTIONS: + return group_match and user_match + if normalized_action in ANY_ALLOWLIST_ACTIONS: + return group_match or user_match + return False + + +def _normalized_ids(value: Any) -> set[str]: + if isinstance(value, str): + value = value.replace(",", "\n").splitlines() + if not isinstance(value, (list, tuple, set, frozenset)): + return set() + return {str(item).strip() for item in value if str(item).strip()} diff --git a/core/config_service.py b/core/config_service.py index d030c36..6d4217f 100644 --- a/core/config_service.py +++ b/core/config_service.py @@ -6,7 +6,8 @@ from dataclasses import dataclass from pathlib import Path from typing import Any -from database.db_helper import REPO_ROOT, load_config_payload +from core.runtime_paths import resolve_config_path +from database.db_helper import load_config_payload @dataclass(frozen=True) @@ -75,7 +76,4 @@ class ConfigService: def _resolve_path(path: str | Path) -> Path: - resolved = Path(path) - if not resolved.is_absolute(): - resolved = REPO_ROOT / resolved - return resolved + return resolve_config_path(path) diff --git a/core/config_writer.py b/core/config_writer.py new file mode 100644 index 0000000..cd73659 --- /dev/null +++ b/core/config_writer.py @@ -0,0 +1,79 @@ +"""Shared serialization and atomic-write helpers for the runtime config.""" + +from __future__ import annotations + +import json +import threading +from collections.abc import Mapping +from pathlib import Path +from typing import Any +from uuid import uuid4 + + +_LOCKS_GUARD = threading.Lock() +_PATH_LOCKS: dict[str, threading.RLock] = {} + + +def config_file_lock(path: str | Path) -> threading.RLock: + """Serialize read-modify-write operations for one config path.""" + + # Return the native lock context manager directly. A generator-based + # contextmanager mutates an escaping exception's ``__traceback__``; + # ImageStudio's frozen ApiError intentionally rejects that mutation. + return _lock_for_path(Path(path)) + + +def write_text_atomic(path: str | Path, content: str) -> None: + """Replace a UTF-8 text file atomically in its destination directory.""" + + destination = Path(path) + with config_file_lock(destination): + destination.parent.mkdir(parents=True, exist_ok=True) + temporary = destination.with_name(f".{destination.name}.{uuid4().hex}.tmp") + try: + temporary.write_text(content, encoding="utf-8") + temporary.replace(destination) + finally: + if temporary.exists(): + temporary.unlink() + + +def dump_yaml(value: Mapping[str, Any], indent: int = 0) -> str: + """Serialize the simple mapping/list/scalar YAML subset used by ImageStudio.""" + + lines: list[str] = [] + prefix = " " * indent + for key, child in value.items(): + if isinstance(child, Mapping): + lines.append(f"{prefix}{key}:") + lines.append(dump_yaml(child, indent + 2).rstrip()) + elif isinstance(child, list): + lines.append(f"{prefix}{key}:") + for item in child: + lines.append(f"{prefix} - {_format_scalar(item)}") + else: + lines.append(f"{prefix}{key}: {_format_scalar(child)}") + return "\n".join(lines) + "\n" + + +def _lock_for_path(path: Path) -> threading.RLock: + try: + key = str(path.resolve()) + except OSError: + key = str(path.absolute()) + with _LOCKS_GUARD: + lock = _PATH_LOCKS.get(key) + if lock is None: + lock = threading.RLock() + _PATH_LOCKS[key] = lock + return lock + + +def _format_scalar(value: Any) -> str: + if isinstance(value, bool): + return "true" if value else "false" + if value is None: + return "null" + if isinstance(value, (int, float)): + return str(value) + return json.dumps(str(value), ensure_ascii=False) diff --git a/core/fsm.py b/core/fsm.py index 7fce626..0eb6bc5 100644 --- a/core/fsm.py +++ b/core/fsm.py @@ -34,9 +34,11 @@ TASK_TRANSITIONS: dict[str, set[str]] = { "running": {"completed", "succeeded", "failed", "cancelled"}, "completed": set(), "succeeded": set(), - "failed": {"queued", "cancelled", "dead_letter"}, - "cancelled": {"queued"}, - "dead_letter": {"queued"}, + "failed": {"queued", "cancelled", "dead_letter", "retrying"}, + "cancelled": {"queued", "retrying"}, + "dead_letter": {"queued", "retrying"}, + "retrying": {"failed", "cancelled", "dead_letter", "superseded"}, + "superseded": set(), } diff --git a/core/generation_errors.py b/core/generation_errors.py index 9d29844..72ce014 100644 --- a/core/generation_errors.py +++ b/core/generation_errors.py @@ -2,6 +2,7 @@ from __future__ import annotations +import re from typing import Any from engines.image_provider import ImageProviderError @@ -83,6 +84,13 @@ def _safe_details(details: dict[str, Any]) -> dict[str, Any]: def _safe_message(value: str, *, limit: int = 240) -> str: text = " ".join(str(value or "").split()) + text = re.sub(r"(?i)\bBearer\s+[A-Za-z0-9._~+/=-]+", "Bearer [REDACTED]", text) + text = re.sub( + r'''(?ix)(["']?(?:api[_-]?key|apikey|authorization|token|password|secret)["']?\s*[:=]\s*["']?)(?:Bearer\s+)?[^"'\s,;}]+''', + r"\1[REDACTED]", + text, + ) + text = re.sub(r"\bsk-[A-Za-z0-9_-]{8,}\b", "[REDACTED]", text) return text[:limit] if text else "image provider failed" diff --git a/core/image_formats.py b/core/image_formats.py new file mode 100644 index 0000000..05349a3 --- /dev/null +++ b/core/image_formats.py @@ -0,0 +1,52 @@ +"""Supported raster image signature, MIME, and extension mappings.""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class ImageFormat: + mime_type: str + extension: str + accepted_extensions: frozenset[str] + + +PNG = ImageFormat("image/png", ".png", frozenset({".png"})) +JPEG = ImageFormat("image/jpeg", ".jpg", frozenset({".jpg", ".jpeg"})) +GIF = ImageFormat("image/gif", ".gif", frozenset({".gif"})) +WEBP = ImageFormat("image/webp", ".webp", frozenset({".webp"})) + +SUPPORTED_IMAGE_FORMATS = (PNG, JPEG, GIF, WEBP) +SUPPORTED_IMAGE_MIME_TYPES = frozenset(item.mime_type for item in SUPPORTED_IMAGE_FORMATS) +SUPPORTED_IMAGE_EXTENSIONS = frozenset( + extension for item in SUPPORTED_IMAGE_FORMATS for extension in item.accepted_extensions +) +IMAGE_EXTENSIONS_BY_MIME = { + item.mime_type: item.accepted_extensions for item in SUPPORTED_IMAGE_FORMATS +} + + +def detect_image_format(content: bytes) -> ImageFormat | None: + """Return the supported image format identified by its file signature.""" + + if content.startswith(b"\x89PNG\r\n\x1a\n"): + return PNG + if content.startswith(b"\xff\xd8\xff"): + return JPEG + if content.startswith((b"GIF87a", b"GIF89a")): + return GIF + if len(content) >= 12 and content.startswith(b"RIFF") and content[8:12] == b"WEBP": + return WEBP + return None + + +def detect_file_image_format(path: Path) -> ImageFormat | None: + """Inspect only the bytes required to identify a supported image file.""" + + try: + with path.open("rb") as handle: + return detect_image_format(handle.read(12)) + except OSError: + return None diff --git a/core/media_proxy.py b/core/media_proxy.py index 6da15b6..9c5eb0a 100644 --- a/core/media_proxy.py +++ b/core/media_proxy.py @@ -9,16 +9,19 @@ import hmac import json import mimetypes import os +import secrets import time from pathlib import Path from typing import Any from backend.dto.common import ApiError, ErrorCode +from core.image_formats import detect_file_image_format from core.storage import VfsStorage DEFAULT_MEDIA_TTL_SECONDS = 300 -MEDIA_FETCH_PATH = "/api/v1/plugins/extensions/astrbot_plugin_imagestudio/media/fetch" +MEDIA_FETCH_PATH = "/api/plug/astrbot_plugin_imagestudio/media/fetch" +_PROCESS_MEDIA_SIGNING_SECRET = os.environ.get("IMAGESTUDIO_MEDIA_SIGNING_SECRET") or secrets.token_urlsafe(48) class MediaProxy: @@ -30,10 +33,10 @@ class MediaProxy: ttl_seconds: int = DEFAULT_MEDIA_TTL_SECONDS, ): self.storage = storage or VfsStorage() - self.signing_secret = signing_secret or os.environ.get( - "IMAGESTUDIO_MEDIA_SIGNING_SECRET", - "imagestudio-dev-media-signing-secret", - ) + # A process-local random key is safer than a public source-code default. + # Operators can set IMAGESTUDIO_MEDIA_SIGNING_SECRET when tokens must + # remain valid across AstrBot process restarts. + self.signing_secret = signing_secret or _PROCESS_MEDIA_SIGNING_SECRET self.ttl_seconds = ttl_seconds def sign(self, uri: str) -> dict[str, Any]: @@ -63,6 +66,11 @@ class MediaProxy: "body_base64": _b64encode(path.read_bytes()), } + def resolve_file(self, token: str) -> tuple[Path, dict[str, Any]]: + """Resolve a signed token for host-native streaming/file responses.""" + + return self._resolve_token(token) + def _resolve_token(self, token: str) -> tuple[Path, dict[str, Any]]: payload = self._decode(token) uri = str(payload.get("uri") or "") @@ -110,6 +118,9 @@ class MediaProxy: def _guess_content_type(path: Path) -> str: + image_format = detect_file_image_format(path) + if image_format is not None: + return image_format.mime_type return mimetypes.guess_type(path.name)[0] or "application/octet-stream" diff --git a/core/remote_fetch.py b/core/remote_fetch.py new file mode 100644 index 0000000..51f4b84 --- /dev/null +++ b/core/remote_fetch.py @@ -0,0 +1,179 @@ +"""Bounded HTTP helpers for provider-controlled remote responses.""" + +from __future__ import annotations + +import ipaddress +import socket +import urllib.request +from collections.abc import Iterable +from typing import Any +from urllib.parse import urlsplit + + +class RemoteFetchError(ValueError): + """Raised when a remote URL or response violates the fetch policy.""" + + +def read_http_response_limited( + request: str | urllib.request.Request, + *, + timeout_seconds: int, + max_bytes: int, + allow_private: bool = False, + allowed_private_hosts: Iterable[str] = (), + same_origin_redirects_only: bool = False, +) -> bytes: + """Fetch an HTTP(S) response while validating redirects and size.""" + + if max_bytes <= 0: + raise ValueError("max_bytes must be positive") + url = request.full_url if isinstance(request, urllib.request.Request) else str(request) + allowed_hosts = frozenset(_normalize_host(host) for host in allowed_private_hosts if _normalize_host(host)) + validate_http_url(url, allow_private=allow_private, allowed_private_hosts=allowed_hosts) + opener = urllib.request.build_opener( + _ValidatedRedirectHandler( + allow_private=allow_private, + allowed_private_hosts=allowed_hosts, + same_origin_only=same_origin_redirects_only, + ) + ) + with opener.open(request, timeout=timeout_seconds) as response: + declared_size = _content_length(response) + if declared_size is not None and declared_size > max_bytes: + raise RemoteFetchError(f"remote response exceeds {max_bytes} bytes") + body = response.read(max_bytes + 1) + if len(body) > max_bytes: + raise RemoteFetchError(f"remote response exceeds {max_bytes} bytes") + return body + + +def validate_http_url( + url: str, + *, + allow_private: bool = False, + allowed_private_hosts: Iterable[str] = (), +) -> None: + """Reject non-HTTP, credential-bearing and unsafe private-network URLs.""" + + try: + parsed = urlsplit(str(url or "").strip()) + host = _normalize_host(parsed.hostname) + port = parsed.port or (443 if parsed.scheme.lower() == "https" else 80) + except ValueError as exc: + raise RemoteFetchError("remote URL is malformed") from exc + if parsed.scheme.lower() not in {"http", "https"}: + raise RemoteFetchError("remote URL must use http or https") + if parsed.username is not None or parsed.password is not None: + raise RemoteFetchError("remote URL must not include credentials") + if not host: + raise RemoteFetchError("remote URL must include a hostname") + allowed_hosts = {_normalize_host(item) for item in allowed_private_hosts} + if allow_private or host in allowed_hosts: + return + for address in _resolve_addresses(host, port): + if _is_unsafe_address(address): + raise RemoteFetchError("remote URL resolves to a private or local network address") + + +def url_hostname(url: str | None) -> str | None: + """Return a normalized hostname from an HTTP(S) URL when present.""" + + try: + parsed = urlsplit(str(url or "").strip()) + except ValueError: + return None + if parsed.scheme.lower() not in {"http", "https"}: + return None + return _normalize_host(parsed.hostname) or None + + +class _ValidatedRedirectHandler(urllib.request.HTTPRedirectHandler): + def __init__( + self, + *, + allow_private: bool, + allowed_private_hosts: frozenset[str], + same_origin_only: bool, + ): + super().__init__() + self.allow_private = allow_private + self.allowed_private_hosts = allowed_private_hosts + self.same_origin_only = same_origin_only + + def redirect_request( + self, + req: urllib.request.Request, + fp: Any, + code: int, + msg: str, + headers: Any, + newurl: str, + ) -> urllib.request.Request | None: + validate_http_url( + newurl, + allow_private=self.allow_private, + allowed_private_hosts=self.allowed_private_hosts, + ) + old_origin = _origin(req.full_url) + new_origin = _origin(newurl) + if old_origin[0] == "https" and new_origin[0] != "https": + raise RemoteFetchError("HTTPS redirects must not downgrade to HTTP") + if self.same_origin_only and old_origin != new_origin: + raise RemoteFetchError("provider endpoint redirected to a different origin") + redirected = super().redirect_request(req, fp, code, msg, headers, newurl) + if redirected is not None and old_origin != new_origin: + redirected.remove_header("Authorization") + return redirected + + +def _resolve_addresses(host: str, port: int) -> set[ipaddress.IPv4Address | ipaddress.IPv6Address]: + try: + literal = ipaddress.ip_address(host) + except ValueError: + literal = None + if literal is not None: + return {literal} + try: + rows = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM) + except OSError as exc: + raise RemoteFetchError("remote hostname could not be resolved") from exc + addresses: set[ipaddress.IPv4Address | ipaddress.IPv6Address] = set() + for row in rows: + try: + addresses.add(ipaddress.ip_address(row[4][0].split("%", 1)[0])) + except (ValueError, IndexError): + continue + if not addresses: + raise RemoteFetchError("remote hostname did not resolve to an IP address") + return addresses + + +def _is_unsafe_address(address: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool: + # is_global also excludes shared/CGNAT (100.64.0.0/10), documentation, + # loopback, link-local, reserved and unspecified ranges. + return not address.is_global or address.is_multicast + + +def _content_length(response: Any) -> int | None: + value = response.headers.get("Content-Length") if getattr(response, "headers", None) else None + if value in (None, ""): + return None + try: + return int(value) + except (TypeError, ValueError): + return None + + +def _normalize_host(value: str | None) -> str: + return str(value or "").strip().rstrip(".").lower() + + +def _origin(url: str) -> tuple[str, str, int]: + try: + parsed = urlsplit(url) + scheme = parsed.scheme.lower() + host = _normalize_host(parsed.hostname) + port = parsed.port or (443 if scheme == "https" else 80) + except ValueError as exc: + raise RemoteFetchError("redirect URL is malformed") from exc + return scheme, host, port diff --git a/core/runtime_paths.py b/core/runtime_paths.py new file mode 100644 index 0000000..c106fbc --- /dev/null +++ b/core/runtime_paths.py @@ -0,0 +1,112 @@ +"""Resolve writable ImageStudio paths for local and AstrBot runtimes.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Callable + + +PLUGIN_NAME = "astrbot_plugin_imagestudio" +REPO_ROOT = Path(__file__).resolve().parents[1] + + +def get_astrbot_runtime_data_root() -> Path | None: + """Return this plugin's AstrBot-managed data directory when available.""" + + getter = _astrbot_plugin_data_path_getter() + if getter is None: + return None + try: + root = getter() + except Exception: + return None + if root is None or not str(root).strip(): + return None + return Path(root).expanduser() / PLUGIN_NAME + + +def get_runtime_data_root() -> Path: + """Return the root used for mutable data in the current runtime.""" + + return get_astrbot_runtime_data_root() or (REPO_ROOT / "data") + + +def resolve_data_path(path: str | Path) -> Path: + """Resolve a configured mutable-data path without changing absolute paths. + + The repository configuration historically uses ``data/...``. Under + AstrBot, the leading ``data`` component is replaced by AstrBot's + plugin-specific writable directory. Local development keeps the existing + ``REPO_ROOT/data/...`` layout. + """ + + resolved = Path(path).expanduser() + if resolved.is_absolute(): + return resolved + + astrbot_root = get_astrbot_runtime_data_root() + if astrbot_root is None: + return REPO_ROOT / resolved + + parts = resolved.parts + if parts and parts[0].lower() == "data": + resolved = Path(*parts[1:]) if len(parts) > 1 else Path() + return astrbot_root / resolved + + +def get_default_config_path() -> Path: + """Return the writable default configuration path for this runtime.""" + + astrbot_root = get_astrbot_runtime_data_root() + if astrbot_root is not None: + return astrbot_root / "config.yaml" + return REPO_ROOT / "config.yaml" + + +def resolve_config_path(path: str | Path = "config.yaml") -> Path: + """Resolve default config to writable storage while preserving explicit paths.""" + + resolved = Path(path).expanduser() + if resolved.is_absolute(): + return resolved + if _is_default_config_reference(resolved): + return get_default_config_path() + return REPO_ROOT / resolved + + +def is_default_config_path(path: str | Path) -> bool: + """Whether a resolved path is the runtime's default writable config.""" + + return _same_path(Path(path), get_default_config_path()) + + +def get_packaged_config_path() -> Path: + """Return the read-only configuration defaults shipped with the plugin.""" + + return REPO_ROOT / "config.yaml" + + +def resolve_plugin_path(path: str | Path) -> Path: + """Resolve a plugin resource path relative to the installed package.""" + + resolved = Path(path).expanduser() + return resolved if resolved.is_absolute() else REPO_ROOT / resolved + + +def _astrbot_plugin_data_path_getter() -> Callable[[], str | Path] | None: + try: + from astrbot.core.utils.astrbot_path import get_astrbot_plugin_data_path + except Exception: + return None + return get_astrbot_plugin_data_path + + +def _is_default_config_reference(path: Path) -> bool: + return path == Path("config.yaml") + + +def _same_path(left: Path, right: Path) -> bool: + try: + return left.resolve() == right.resolve() + except OSError: + return left.absolute() == right.absolute() diff --git a/core/security.py b/core/security.py index 3e67984..4750dbd 100644 --- a/core/security.py +++ b/core/security.py @@ -28,5 +28,10 @@ def require_write(ctx: RequestContext) -> None: require_permission(ctx, "can_write") +def require_task_retry(ctx: RequestContext) -> None: + require_write(ctx) + require_permission(ctx, "can_retry_tasks") + + def require_config_manager(ctx: RequestContext) -> None: require_permission(ctx, "can_manage_config") diff --git a/core/storage.py b/core/storage.py index 0381f90..bbd91a0 100644 --- a/core/storage.py +++ b/core/storage.py @@ -10,7 +10,7 @@ from typing import Any from backend.dto.common import ApiError, ErrorCode from core.config_service import ConfigService, ImageStudioConfig -from database.db_helper import REPO_ROOT +from core.runtime_paths import resolve_data_path @dataclass(frozen=True) @@ -71,7 +71,7 @@ class VfsStorage: items: list[dict[str, Any]] = [] total = 0 for root_name in selected_roots: - root_path = _resolve_data_path(roots[root_name]).resolve() + root_path = resolve_data_path(roots[root_name]).resolve() if not root_path.exists(): continue for path in sorted(root_path.rglob("*")): @@ -102,7 +102,7 @@ class VfsStorage: root_path = self._root_paths().get(root) if root_path is None: raise ApiError(ErrorCode.MEDIA_NOT_FOUND, "unknown storage root", {"uri": uri, "root": root}) - resolved_root = _resolve_data_path(root_path).resolve() + resolved_root = resolve_data_path(root_path).resolve() resolved_path = (resolved_root / relative).resolve() if resolved_path != resolved_root and resolved_root not in resolved_path.parents: raise ApiError(ErrorCode.FORBIDDEN, "storage uri escapes its root", {"uri": uri}) @@ -129,7 +129,7 @@ class VfsStorage: return root, relative_path def _entry(self, root: str, path: str) -> VfsEntry: - resolved = _resolve_data_path(path) + resolved = resolve_data_path(path) return VfsEntry( root=root, uri_prefix=f"storage://{root}/", @@ -154,13 +154,6 @@ def _storage_roots(config: ImageStudioConfig) -> dict[str, str]: } -def _resolve_data_path(path: str) -> Path: - resolved = Path(path) - if not resolved.is_absolute(): - resolved = REPO_ROOT / resolved - return resolved - - def _directory_size(path: Path) -> int: if not path.exists(): return 0 diff --git a/database/db_helper.py b/database/db_helper.py index 9b67a1f..22099d7 100644 --- a/database/db_helper.py +++ b/database/db_helper.py @@ -6,14 +6,21 @@ configuration explicitly enables it, or when `backend: postgresql` is set. from __future__ import annotations +import json import os import sqlite3 from dataclasses import dataclass from pathlib import Path from typing import Any, Iterable, Mapping +from core.runtime_paths import ( + REPO_ROOT, + get_packaged_config_path, + is_default_config_path, + resolve_config_path, + resolve_data_path, +) -REPO_ROOT = Path(__file__).resolve().parents[1] _RUNTIME_CONFIG_OVERRIDES: dict[str, Any] = {} @@ -28,15 +35,27 @@ def get_runtime_config_overrides() -> dict[str, Any]: def load_config_payload(config_path: str | Path = "config.yaml") -> dict[str, Any]: - path = Path(config_path) - if not path.is_absolute(): - path = REPO_ROOT / path - payload = _load_yaml_subset(path.read_text(encoding="utf-8")) if path.exists() else {} + path = resolve_config_path(config_path) + payload = load_config_file_payload(path) if _should_apply_runtime_overrides(path): return _merge_mappings(payload, _RUNTIME_CONFIG_OVERRIDES) return payload +def load_config_file_payload(config_path: str | Path = "config.yaml") -> dict[str, Any]: + """Load packaged and writable YAML without applying in-process overrides.""" + + path = resolve_config_path(config_path) + payload: dict[str, Any] = {} + if _should_apply_runtime_overrides(path): + packaged_path = get_packaged_config_path() + if not _same_path(path, packaged_path) and packaged_path.exists(): + payload = _load_yaml_subset(packaged_path.read_text(encoding="utf-8")) + if path.exists(): + payload = _merge_mappings(payload, _load_yaml_subset(path.read_text(encoding="utf-8"))) + return payload + + @dataclass(frozen=True) class SqliteSettings: path: str = "data/db/imagestudio.db" @@ -116,6 +135,7 @@ class Database: db_path.parent.mkdir(parents=True, exist_ok=True) conn = sqlite3.connect(db_path) conn.row_factory = sqlite3.Row + conn.execute("PRAGMA foreign_keys = ON") return conn return self._connect_postgresql() @@ -137,10 +157,7 @@ class Database: conn.close() def _resolve_sqlite_path(self) -> Path: - db_path = Path(self.settings.sqlite.path) - if not db_path.is_absolute(): - db_path = REPO_ROOT / db_path - return db_path + return resolve_data_path(self.settings.sqlite.path) def _connect_postgresql(self): try: @@ -255,10 +272,14 @@ def _as_bool(value: Any) -> bool: def _should_apply_runtime_overrides(path: Path) -> bool: + return is_default_config_path(path) + + +def _same_path(left: Path, right: Path) -> bool: try: - return path.resolve() == (REPO_ROOT / "config.yaml").resolve() + return left.resolve() == right.resolve() except OSError: - return path.absolute() == (REPO_ROOT / "config.yaml").absolute() + return left.absolute() == right.absolute() def _copy_plain_mapping(value: Mapping[str, Any]) -> dict[str, Any]: @@ -300,7 +321,7 @@ def _load_yaml_subset(text: str) -> dict[str, Any]: last_key_by_indent: dict[int, str] = {} for raw_line in text.splitlines(): - line = raw_line.split("#", 1)[0].rstrip() + line = _strip_yaml_comment(raw_line).rstrip() if not line.strip(): continue indent = len(line) - len(line.lstrip(" ")) @@ -337,8 +358,13 @@ def _load_yaml_subset(text: str) -> dict[str, Any]: def _parse_scalar(value: str) -> Any: - if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}: - return value[1:-1] + if len(value) >= 2 and value[0] == value[-1] == '"': + try: + return json.loads(value) + except json.JSONDecodeError: + return value[1:-1] + if len(value) >= 2 and value[0] == value[-1] == "'": + return value[1:-1].replace("''", "'") lowered = value.lower() if lowered == "true": return True @@ -350,3 +376,30 @@ def _parse_scalar(value: str) -> Any: return int(value) except ValueError: return os.path.expandvars(value) + + +def _strip_yaml_comment(line: str) -> str: + quote: str | None = None + escaped = False + index = 0 + while index < len(line): + char = line[index] + if quote == '"': + if escaped: + escaped = False + elif char == "\\": + escaped = True + elif char == quote: + quote = None + elif quote == "'": + if char == quote: + if index + 1 < len(line) and line[index + 1] == quote: + index += 1 + else: + quote = None + elif char in {"'", '"'}: + quote = char + elif char == "#" and (index == 0 or line[index - 1].isspace()): + return line[:index] + index += 1 + return line diff --git a/database/repositories/image_repository.py b/database/repositories/image_repository.py index 2fcf631..e098871 100644 --- a/database/repositories/image_repository.py +++ b/database/repositories/image_repository.py @@ -138,6 +138,50 @@ class ImageRepository: conn.close() return [_row_to_image(row) for row in rows] + def list_images( + self, + *, + profile_id: str | None = None, + workflow_id: str | None = None, + include_archived: bool = True, + page: int = 1, + page_size: int = 60, + ) -> tuple[list[GenerationImage], int]: + placeholder = self.placeholder + clauses: list[str] = [] + params: list[Any] = [] + if profile_id: + clauses.append(f"profile_id = {placeholder}") + params.append(profile_id) + if workflow_id: + clauses.append(f"workflow_id = {placeholder}") + params.append(workflow_id) + if not include_archived: + clauses.append("is_archived = 0") + where_clause = f"WHERE {' AND '.join(clauses)}" if clauses else "" + normalized_page = max(page, 1) + normalized_page_size = min(max(page_size, 1), 100) + offset = (normalized_page - 1) * normalized_page_size + conn = self.database.connect() + try: + total_row = conn.execute( + f"SELECT COUNT(*) AS total FROM generation_images {where_clause}", + tuple(params), + ).fetchone() + rows = conn.execute( + f""" + SELECT {IMAGE_COLUMNS} + FROM generation_images + {where_clause} + ORDER BY is_archived ASC, created_at DESC, image_id DESC + LIMIT {placeholder} OFFSET {placeholder} + """, + tuple(params + [normalized_page_size, offset]), + ).fetchall() + finally: + conn.close() + return [_row_to_image(row) for row in rows], int(total_row["total"] if total_row else 0) + def update(self, image_id: str, payload: dict[str, Any]) -> GenerationImage | None: allowed = { "thumbnail_uri", diff --git a/database/repositories/job_repository.py b/database/repositories/job_repository.py index 0b1495f..83c6d20 100644 --- a/database/repositories/job_repository.py +++ b/database/repositories/job_repository.py @@ -157,36 +157,8 @@ class JobRepository: return [_row_to_task(row) for row in rows], int(total_row["total"] if total_row else 0) def update(self, task_id: str, payload: dict[str, Any]) -> GenerationTask | None: - allowed = { - "status", - "model_route", - "progress", - "request", - "result", - "error_message", - "metadata", - } - assignments: list[str] = [] - params: list[Any] = [] placeholder = self.placeholder - for key in allowed: - if key not in payload: - continue - column = key - value = payload[key] - if key == "request": - column = "request_json" - value = _json_or_none(value) - elif key == "result": - column = "result_json" - value = _json_or_none(value) - elif key == "metadata": - column = "metadata_json" - value = _json_or_none(value) - elif key == "progress": - value = int(value) - assignments.append(f"{column} = {placeholder}") - params.append(value) + assignments, params = _update_assignments(payload, placeholder) if not assignments: return self.get(task_id) assignments.append("updated_at = CURRENT_TIMESTAMP") @@ -206,6 +178,109 @@ class JobRepository: conn.close() return self.get(task_id) + def update_if_status( + self, + task_id: str, + expected_statuses: set[str] | tuple[str, ...] | list[str], + payload: dict[str, Any], + ) -> GenerationTask | None: + """Update a task only while it is still in an expected state. + + The status predicate and mutation run in one SQL statement. This is the + compare-and-set primitive used by retry orchestration and queue workers + to prevent two callers from claiming the same task. + """ + + normalized_statuses = tuple( + sorted({str(status).strip() for status in expected_statuses if str(status).strip()}) + ) + if not normalized_statuses: + return None + placeholder = self.placeholder + assignments, params = _update_assignments(payload, placeholder) + if not assignments: + task = self.get(task_id) + return task if task and task.status in normalized_statuses else None + assignments.append("updated_at = CURRENT_TIMESTAMP") + status_placeholders = ", ".join(placeholder for _ in normalized_statuses) + params.extend([task_id, *normalized_statuses]) + conn = self.database.connect() + try: + cursor = conn.execute( + f""" + UPDATE generation_tasks + SET {", ".join(assignments)} + WHERE task_id = {placeholder} + AND status IN ({status_placeholders}) + """, + tuple(params), + ) + changed = int(cursor.rowcount or 0) > 0 + conn.commit() + finally: + conn.close() + return self.get(task_id) if changed else None + + def update_if_status_unless_workflow_cancelled( + self, + task_id: str, + expected_statuses: set[str] | tuple[str, ...] | list[str], + payload: dict[str, Any], + ) -> GenerationTask | None: + """CAS-update a task while its owning workflow is not cancelled.""" + + normalized_statuses = tuple( + sorted({str(status).strip() for status in expected_statuses if str(status).strip()}) + ) + if not normalized_statuses: + return None + placeholder = self.placeholder + assignments, params = _update_assignments(payload, placeholder) + if not assignments: + task = self.get(task_id) + if task is None or task.status not in normalized_statuses: + return None + if task.workflow_id: + conn = self.database.connect() + try: + row = conn.execute( + f"SELECT status FROM profile_workflows WHERE workflow_id = {placeholder}", + (task.workflow_id,), + ).fetchone() + finally: + conn.close() + if row is None or str(row["status"]) == "cancelled": + return None + return task + assignments.append("updated_at = CURRENT_TIMESTAMP") + status_placeholders = ", ".join(placeholder for _ in normalized_statuses) + params.extend([task_id, *normalized_statuses, "cancelled"]) + conn = self.database.connect() + try: + cursor = conn.execute( + f""" + UPDATE generation_tasks + SET {", ".join(assignments)} + WHERE task_id = {placeholder} + AND status IN ({status_placeholders}) + AND ( + workflow_id IS NULL + OR EXISTS ( + SELECT 1 + FROM profile_workflows + WHERE profile_workflows.workflow_id = generation_tasks.workflow_id + AND profile_workflows.status <> {placeholder} + ) + ) + """, + tuple(params), + ) + changed = int(cursor.rowcount or 0) > 0 + conn.commit() + finally: + conn.close() + return self.get(task_id) if changed else None + def count_by_status(self) -> dict[str, int]: conn = self.database.connect() try: @@ -264,6 +339,39 @@ def _json_or_none(value: Any | None) -> str | None: return json.dumps(value, ensure_ascii=False, sort_keys=True) +def _update_assignments(payload: dict[str, Any], placeholder: str) -> tuple[list[str], list[Any]]: + allowed = { + "status", + "model_route", + "progress", + "request", + "result", + "error_message", + "metadata", + } + assignments: list[str] = [] + params: list[Any] = [] + for key in allowed: + if key not in payload: + continue + column = key + value = payload[key] + if key == "request": + column = "request_json" + value = _json_or_none(value) + elif key == "result": + column = "result_json" + value = _json_or_none(value) + elif key == "metadata": + column = "metadata_json" + value = _json_or_none(value) + elif key == "progress": + value = int(value) + assignments.append(f"{column} = {placeholder}") + params.append(value) + return assignments, params + + def _limit(value: int) -> int: try: limit = int(value) diff --git a/database/repositories/profile_repository.py b/database/repositories/profile_repository.py index 5650250..d1fdfeb 100644 --- a/database/repositories/profile_repository.py +++ b/database/repositories/profile_repository.py @@ -29,6 +29,26 @@ PROFILE_COLUMNS = """ updated_at """ +PROFILE_GENERATION_KIND_COLUMN = """ + CASE + WHEN EXISTS ( + SELECT 1 + FROM profile_workflows AS detailed_workflow + WHERE detailed_workflow.profile_id = character_profiles.profile_id + AND detailed_workflow.workflow_type = 'character_gradual' + AND LOWER(COALESCE(detailed_workflow.status, '')) <> 'cancelled' + ) THEN 'detailed' + WHEN EXISTS ( + SELECT 1 + FROM profile_workflows AS quick_workflow + WHERE quick_workflow.profile_id = character_profiles.profile_id + AND quick_workflow.workflow_type = 'character_fast' + AND LOWER(COALESCE(quick_workflow.status, '')) <> 'cancelled' + ) THEN 'quick' + ELSE 'ungenerated' + END AS generation_kind +""" + class ProfileRepository: def __init__(self, database: Database): @@ -97,7 +117,11 @@ class ProfileRepository: conn = self.database.connect() try: row = conn.execute( - f"SELECT {PROFILE_COLUMNS} FROM character_profiles WHERE profile_id = {placeholder}", + f""" + SELECT {PROFILE_COLUMNS}, {PROFILE_GENERATION_KIND_COLUMN} + FROM character_profiles + WHERE profile_id = {placeholder} + """, (profile_id,), ).fetchone() finally: @@ -129,7 +153,7 @@ class ProfileRepository: ).fetchone() rows = conn.execute( f""" - SELECT {PROFILE_COLUMNS} + SELECT {PROFILE_COLUMNS}, {PROFILE_GENERATION_KIND_COLUMN} FROM character_profiles {where} ORDER BY updated_at DESC, created_at DESC @@ -242,6 +266,7 @@ def _row_to_profile(row) -> CharacterProfile: metadata=metadata, created_at=str(row["created_at"]), updated_at=str(row["updated_at"]), + generation_kind=str(row["generation_kind"] or "ungenerated"), ) diff --git a/database/repositories/workflow_repository.py b/database/repositories/workflow_repository.py index cbdd092..e33ec04 100644 --- a/database/repositories/workflow_repository.py +++ b/database/repositories/workflow_repository.py @@ -80,48 +80,30 @@ class WorkflowRepository: return self.database._placeholder def create_workflow(self, payload: dict[str, Any]) -> ProfileWorkflow: + workflow, _ = self.create_workflow_with_nodes(payload, []) + return workflow + + def create_workflow_with_nodes( + self, + payload: dict[str, Any], + node_payloads: list[dict[str, Any]], + ) -> tuple[ProfileWorkflow, list[WorkflowNode]]: placeholder = self.placeholder conn = self.database.connect() try: - conn.execute( - f""" - INSERT INTO profile_workflows ( - workflow_id, - profile_id, - workflow_type, - display_name, - mode, - status, - current_node_id, - current_branch, - session_key, - metadata_json - ) VALUES ( - {placeholder}, {placeholder}, {placeholder}, {placeholder}, - {placeholder}, {placeholder}, {placeholder}, {placeholder}, - {placeholder}, {placeholder} - ) - """, - ( - payload["workflow_id"], - payload["profile_id"], - payload["workflow_type"], - payload["display_name"], - payload.get("mode"), - payload.get("status", "draft"), - payload.get("current_node_id"), - payload.get("current_branch", "main"), - payload.get("session_key"), - _json_or_none(payload.get("metadata")), - ), - ) + _insert_workflow(conn, placeholder, payload) + for node_payload in node_payloads: + _insert_node(conn, placeholder, node_payload) conn.commit() + except Exception: + conn.rollback() + raise finally: conn.close() workflow = self.get_workflow(payload["workflow_id"]) if workflow is None: raise RuntimeError("workflow was inserted but could not be read back") - return workflow + return workflow, self.list_nodes(workflow.workflow_id) def get_workflow(self, workflow_id: str) -> ProfileWorkflow | None: placeholder = self.placeholder @@ -195,53 +177,64 @@ class WorkflowRepository: conn.close() return self.get_workflow(workflow_id) + def update_workflow_if_status( + self, + workflow_id: str, + expected_statuses: set[str] | tuple[str, ...] | list[str], + payload: dict[str, Any], + ) -> ProfileWorkflow | None: + normalized_statuses = tuple( + sorted({str(status).strip() for status in expected_statuses if str(status).strip()}) + ) + if not normalized_statuses: + return None + allowed = { + "display_name", + "mode", + "status", + "current_node_id", + "current_branch", + "session_key", + "metadata", + } + assignments: list[str] = [] + params: list[Any] = [] + placeholder = self.placeholder + for key in allowed: + if key not in payload: + continue + column = "metadata_json" if key == "metadata" else key + value = _json_or_none(payload[key]) if key == "metadata" else payload[key] + assignments.append(f"{column} = {placeholder}") + params.append(value) + if not assignments: + workflow = self.get_workflow(workflow_id) + return workflow if workflow and workflow.status in normalized_statuses else None + assignments.append("updated_at = CURRENT_TIMESTAMP") + status_placeholders = ", ".join(placeholder for _ in normalized_statuses) + params.extend([workflow_id, *normalized_statuses]) + conn = self.database.connect() + try: + cursor = conn.execute( + f""" + UPDATE profile_workflows + SET {", ".join(assignments)} + WHERE workflow_id = {placeholder} + AND status IN ({status_placeholders}) + """, + tuple(params), + ) + changed = int(cursor.rowcount or 0) > 0 + conn.commit() + finally: + conn.close() + return self.get_workflow(workflow_id) if changed else None + def create_node(self, payload: dict[str, Any]) -> WorkflowNode: placeholder = self.placeholder conn = self.database.connect() try: - conn.execute( - f""" - INSERT INTO workflow_nodes ( - node_id, - workflow_id, - node_key, - title, - description, - step_name, - node_type, - status, - branch_name, - prompt, - negative_prompt, - model_route, - selected_image_id, - sort_order, - metadata_json - ) VALUES ( - {placeholder}, {placeholder}, {placeholder}, {placeholder}, - {placeholder}, {placeholder}, {placeholder}, {placeholder}, - {placeholder}, {placeholder}, {placeholder}, {placeholder}, - {placeholder}, {placeholder}, {placeholder} - ) - """, - ( - payload["node_id"], - payload["workflow_id"], - payload["node_key"], - payload["title"], - payload.get("description"), - payload.get("step_name"), - payload.get("node_type", "generation"), - payload.get("status", "draft"), - payload.get("branch_name", "main"), - payload.get("prompt"), - payload.get("negative_prompt"), - payload.get("model_route"), - payload.get("selected_image_id"), - int(payload.get("sort_order", 0)), - _json_or_none(payload.get("metadata")), - ), - ) + _insert_node(conn, placeholder, payload) conn.commit() finally: conn.close() @@ -328,6 +321,64 @@ class WorkflowRepository: conn.close() return self.get_node(node_id) + def update_node_if_status( + self, + node_id: str, + expected_statuses: set[str] | tuple[str, ...] | list[str], + payload: dict[str, Any], + ) -> WorkflowNode | None: + normalized_statuses = tuple( + sorted({str(status).strip() for status in expected_statuses if str(status).strip()}) + ) + if not normalized_statuses: + return None + allowed = { + "title", + "description", + "step_name", + "node_type", + "status", + "branch_name", + "prompt", + "negative_prompt", + "model_route", + "selected_image_id", + "sort_order", + "metadata", + } + assignments: list[str] = [] + params: list[Any] = [] + placeholder = self.placeholder + for key in allowed: + if key not in payload: + continue + column = "metadata_json" if key == "metadata" else key + value = _json_or_none(payload[key]) if key == "metadata" else payload[key] + assignments.append(f"{column} = {placeholder}") + params.append(value) + if not assignments: + node = self.get_node(node_id) + return node if node and node.status in normalized_statuses else None + assignments.append("updated_at = CURRENT_TIMESTAMP") + status_placeholders = ", ".join(placeholder for _ in normalized_statuses) + params.extend([node_id, *normalized_statuses]) + conn = self.database.connect() + try: + cursor = conn.execute( + f""" + UPDATE workflow_nodes + SET {", ".join(assignments)} + WHERE node_id = {placeholder} + AND status IN ({status_placeholders}) + """, + tuple(params), + ) + changed = int(cursor.rowcount or 0) > 0 + conn.commit() + finally: + conn.close() + return self.get_node(node_id) if changed else None + def create_history(self, payload: dict[str, Any]) -> WorkflowHistoryEntry: placeholder = self.placeholder conn = self.database.connect() @@ -477,6 +528,87 @@ class WorkflowRepository: return [_row_to_branch(row) for row in rows] +def _insert_workflow(conn, placeholder: str, payload: dict[str, Any]) -> None: + conn.execute( + f""" + INSERT INTO profile_workflows ( + workflow_id, + profile_id, + workflow_type, + display_name, + mode, + status, + current_node_id, + current_branch, + session_key, + metadata_json + ) VALUES ( + {placeholder}, {placeholder}, {placeholder}, {placeholder}, + {placeholder}, {placeholder}, {placeholder}, {placeholder}, + {placeholder}, {placeholder} + ) + """, + ( + payload["workflow_id"], + payload["profile_id"], + payload["workflow_type"], + payload["display_name"], + payload.get("mode"), + payload.get("status", "draft"), + payload.get("current_node_id"), + payload.get("current_branch", "main"), + payload.get("session_key"), + _json_or_none(payload.get("metadata")), + ), + ) + + +def _insert_node(conn, placeholder: str, payload: dict[str, Any]) -> None: + conn.execute( + f""" + INSERT INTO workflow_nodes ( + node_id, + workflow_id, + node_key, + title, + description, + step_name, + node_type, + status, + branch_name, + prompt, + negative_prompt, + model_route, + selected_image_id, + sort_order, + metadata_json + ) VALUES ( + {placeholder}, {placeholder}, {placeholder}, {placeholder}, + {placeholder}, {placeholder}, {placeholder}, {placeholder}, + {placeholder}, {placeholder}, {placeholder}, {placeholder}, + {placeholder}, {placeholder}, {placeholder} + ) + """, + ( + payload["node_id"], + payload["workflow_id"], + payload["node_key"], + payload["title"], + payload.get("description"), + payload.get("step_name"), + payload.get("node_type", "generation"), + payload.get("status", "draft"), + payload.get("branch_name", "main"), + payload.get("prompt"), + payload.get("negative_prompt"), + payload.get("model_route"), + payload.get("selected_image_id"), + int(payload.get("sort_order", 0)), + _json_or_none(payload.get("metadata")), + ), + ) + + def _row_to_workflow(row) -> ProfileWorkflow: return ProfileWorkflow( workflow_id=row["workflow_id"], diff --git a/database/seed.py b/database/seed.py index e78f177..e9c0444 100644 --- a/database/seed.py +++ b/database/seed.py @@ -13,6 +13,7 @@ from database.db_helper import Database class SeedCategory: category_id: str name: str + parent_id: str | None = None category_type: str = "material" sort_order: int = 0 metadata: dict[str, Any] = field(default_factory=dict) @@ -29,6 +30,100 @@ DEFAULT_MATERIAL_CATEGORIES: tuple[SeedCategory, ...] = ( sort_order=50, metadata={"seed": True, "wardrobe_category": "accessory"}, ), + SeedCategory( + "cat_imagestudio_构图", + "构图", + sort_order=110, + metadata={"seed": True, "source": "imagestudio_backend", "frontend_group": "构图"}, + ), + SeedCategory( + "cat_imagestudio_服装", + "服装", + sort_order=120, + metadata={"seed": True, "source": "imagestudio_backend", "frontend_group": "服装"}, + ), + SeedCategory( + "cat_imagestudio_风格", + "风格", + sort_order=130, + metadata={"seed": True, "source": "imagestudio_backend", "frontend_group": "风格"}, + ), + SeedCategory( + "cat_imagestudio_配饰", + "配饰", + sort_order=140, + metadata={"seed": True, "source": "imagestudio_backend", "frontend_group": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_鞋袜", + "鞋袜", + sort_order=150, + metadata={"seed": True, "source": "imagestudio_backend", "frontend_group": "鞋袜"}, + ), + SeedCategory( + "cat_imagestudio_accessory_眼部", + "眼部", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=210, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_accessory_头部", + "头部", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=220, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_accessory_手部", + "手部", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=230, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_accessory_胸前", + "胸前", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=240, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_accessory_颈部", + "颈部", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=250, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_accessory_腰部", + "腰部", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=260, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_accessory_脸部", + "脸部", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=270, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), + SeedCategory( + "cat_imagestudio_accessory_臂部", + "臂部", + parent_id="cat_imagestudio_配饰", + category_type="accessory_group", + sort_order=280, + metadata={"seed": True, "source": "imagestudio_backend", "parent_category": "配饰"}, + ), ) @@ -51,12 +146,13 @@ def seed_default_material_categories(database: Database) -> dict[str, Any]: INSERT INTO material_categories ( category_id, name, parent_id, category_type, sort_order, metadata_json ) VALUES ( - {placeholder}, {placeholder}, NULL, {placeholder}, {placeholder}, {placeholder} + {placeholder}, {placeholder}, {placeholder}, {placeholder}, {placeholder}, {placeholder} ) """, ( category.category_id, category.name, + category.parent_id, category.category_type, category.sort_order, json.dumps(category.metadata, ensure_ascii=False, sort_keys=True), diff --git a/docs/astrbot-chat-command-queue.md b/docs/astrbot-chat-command-queue.md new file mode 100644 index 0000000..2894016 --- /dev/null +++ b/docs/astrbot-chat-command-queue.md @@ -0,0 +1,226 @@ +# AstrBot 聊天命令开发队列 + +更新日期:2026-07-12 + +目标:在 QQ、Telegram、Discord 等 AstrBot 消息平台中,通过 `/影棚` 或 `/yp` 操作 ImageStudio 的单图生成、快速角色和详细角色工作流,并获得异步任务、图片、选图和审核回执。 + +## 当前完成基线 + +- [x] 注册 `/影棚` 命令组。 +- [x] 注册 `/yp` 短命令组。 +- [x] 提供帮助、状态、快速角色、工作流、单图子命令框架。 +- [x] 提取平台、统一消息来源、群聊 ID 和用户 ID。 +- [x] 配置群聊白名单与用户白名单。 +- [x] 快速角色和角色工作流要求群聊、用户同时命中。 +- [x] 单图生成允许群聊或用户任一白名单命中。 +- [x] 帮助和状态支持 PNG 卡片,异常时回退文字。 +- [x] AstrBot 不存在时保持本地测试可导入。 + +## P0:单图生成最小闭环 + +### P0-01 命令参数合同 + +- [x] 支持 `/yp 单图 <提示词>`。 +- [x] 支持角色、模型路由、尺寸、数量、负面提示词等显式参数。 +- [x] 同时支持自然文本和 `--参数 值` 两种输入。 +- [x] 将命令统一转换为 `SingleImageCommand` DTO。 +- [x] 参数错误返回示例和可复制的正确命令。 + +建议参数: + +```text +/yp 单图 雨夜街道中的少女 --角色 米卡 --尺寸 1K --数量 1 +``` + +### P0-02 单图任务提交 + +- [x] 将单图命令接入现有生成服务和模型分工中的 `quick_generation`。 +- [x] 执行角色归属、模型能力、预算和白名单预检。 +- [x] 创建队列任务并立即返回 `task_id`,不阻塞 AstrBot 消息协程。 +- [x] 保存平台、统一消息来源、用户、群聊和原消息 ID。 +- [x] 防止同一消息重复提交任务。 + +### P0-03 任务查询与取消 + +- [x] `/yp 任务`:列出当前用户最近任务。 +- [x] `/yp 任务 `:显示任务详情。 +- [x] `/yp 取消 `:取消等待中或运行中的任务。 +- [x] `/yp 重试 `:按权限重试失败任务。 +- [x] 任务卡片显示排队位置、模型、尺寸、数量和耗时。 + +### P0-04 异步图片回执 + +- [x] Worker 完成任务后向原 `unified_msg_origin` 主动发送结果。 +- [x] 使用 AstrBot 图片消息组件发送图片,不泄露服务器本地路径。 +- [x] 同时返回图片编号、模型、尺寸、种子、耗时和任务 ID。 +- [x] 图片发送失败时回退为签名媒体 URL 或明确错误消息。 +- [x] 失败、取消、超时分别使用不同回执。 + +## P1:快速角色闭环 + +### P1-01 快速角色创建 + +- [x] `/yp 快速角色 <角色描述>` 接入快速角色草稿服务。 +- [x] 支持从聊天文本提取角色名、外观、性格、服装和禁用项。 +- [x] 缺少角色名时进入短期会话询问,不直接创建匿名角色。 +- [x] 创建前检查群聊与用户双白名单。 +- [x] 创建的角色 owner 绑定发起用户。 + +### P1-02 快速角色确认 + +- [x] 返回结构化角色摘要卡片。 +- [x] 支持“确认”“修改”“取消”。 +- [x] 修改时允许补充一段自然语言并重新解析。 +- [x] 会话绑定群聊、用户和角色草稿,避免其他群员误操作。 +- [x] 会话 5–10 分钟自动过期。 + +### P1-03 快速角色生图 + +- [x] 确认角色后可直接回复“生成”。 +- [x] 使用 `quick_generation` 模型分工路由。 +- [x] 返回候选图并建立选图会话。 +- [x] 支持回复数字采用候选图。 +- [x] 选中结果写回角色封面和图库。 + +## P1:详细角色工作流 + +### P1-04 工作流创建与恢复 + +- [x] `/yp 工作流 <角色名或 ID>` 创建或恢复工作流。 +- [x] 检查角色 owner、归档状态和双白名单。 +- [x] 同一角色存在未完成工作流时默认恢复,避免重复创建。 +- [x] 返回工作流 ID、节点进度和当前节点。 + +### P1-05 节点操作 + +- [x] `/yp 节点 ` 查看节点列表。 +- [x] `/yp 生成 ` 执行当前节点。 +- [x] `/yp 重试节点 ` 重试节点。 +- [x] `/yp 完成节点 ` 确认节点。 +- [x] `/yp 下一步 ` 进入下一个可执行节点。 +- [x] 拒绝跳过未满足依赖的节点。 + +### P1-06 候选图选择 + +- [x] 生成完成后返回带编号候选图。 +- [x] 支持回复 `1`、`2`、`3` 选图。 +- [x] 支持 `/yp 选图 <序号>` 无状态操作。 +- [x] 选图前校验图片属于节点且未归档。 +- [x] 选图后提示继续、重试或审核。 + +## P2:审核和高清修复 + +### P2-01 交叉审核 + +- [x] `/yp 审核 ` 提交审核。 +- [x] 使用设置页配置的多个 `cross_review_routes`。 +- [x] 主审、复核和附加审核并行执行。 +- [x] 聚合一致结论、分歧和置信度。 +- [x] 返回缺陷、评分和建议操作卡片。 + +### P2-02 高清修复 + +- [x] `/yp 高清 --尺寸 2K|4K`。 +- [x] 使用 `hires_fix` 模型分工路由。 +- [x] 执行模型能力、最大分辨率和预算预检。 +- [x] 保留原图、修复图和生成关系。 +- [x] 完成后返回原图与修复图对照。 + +## P2:素材聊天操作 + +### P2-03 图片素材导入 + +- [x] 支持命令消息附带一张或多张图片。 +- [x] `/yp 素材导入 --分类 <分类>`。 +- [x] 分类只能从现有素材分类中解析。 +- [x] 使用 `material_recognition` 模型路由和分类识别提示词。 +- [x] 返回识别结果并等待确认。 + +### P2-04 素材确认与角色衣橱 + +- [x] 支持确认、重新识别和取消素材。 +- [x] 支持将确认素材加入指定角色衣橱。 +- [x] 校验素材权限、角色 owner 和衣橱分类。 +- [x] 返回素材卡片和可用角色列表。 + +## P2:会话状态层 + +### P2-05 会话存储 + +- [x] 新建平台无关的 `ChatSessionStore`。 +- [x] 会话键包含平台、消息来源和用户 ID。 +- [x] 支持角色确认、选图、素材确认和危险操作确认。 +- [x] 设置过期时间和最大会话数量。 +- [x] 插件重载后可选择恢复关键会话。 + +### P2-06 非命令回复捕获 + +- [x] 仅在存在活动会话时处理“1”“确认”“继续”等普通消息。 +- [x] 不抢占其他 AstrBot 插件的普通聊天消息。 +- [x] 群聊中只接受原发起用户的回复。 +- [x] 提供“退出”“取消”结束会话。 + +## P3:跨平台和运营保护 + +### P3-01 跨平台消息适配 + +- [x] QQ、Telegram、Discord 至少各完成一次文本和图片回执测试。 +- [x] 统一处理平台图片 URL、本地缓存和消息组件差异。 +- [x] 不依赖 QQ 专属字段作为业务主键。 +- [x] 优先使用 AstrBot `unified_msg_origin` 主动回执。 + +### P3-02 限流和成本保护 + +- [x] 用户并发任务限制。 +- [x] 群聊并发任务限制。 +- [x] 用户、群聊每日生成额度。 +- [x] 2K、4K 和高成本模型独立授权。 +- [x] 命令提交前调用现有成本保护服务。 + +### P3-03 管理与审计 + +- [x] 记录命令、调用用户、群聊、任务 ID 和结果。 +- [x] 日志中不保存完整敏感提示词和凭据。 +- [x] Web 日志页面可以筛选聊天命令来源。 +- [x] 管理员可以查询被拒绝的白名单请求。 + +## P3:测试与验收 + +### P3-04 单元测试 + +- [x] 命令参数解析覆盖中文空格、引号和显式参数。 +- [x] 白名单组合覆盖群聊、私聊和空名单。 +- [x] 事件字段兼容不同 AstrBot 平台适配器。 +- [x] 图片卡片失败时验证文字降级。 +- [x] 会话过期、越权回复和重复提交测试。 + +### P3-05 AstrBot 集成测试 + +- [x] 插件加载后命令组可被 AstrBot 正确发现。 +- [x] `/影棚` 与 `/yp` 行为一致。 +- [x] 命令协程不等待长时间生图。 +- [x] Worker 完成后能够主动发送到原会话。 +- [x] 插件重载和 AstrBot 重启后任务状态不丢失。 + +### P3-06 发布验收 + +- [x] 更新 README 命令说明。 +- [x] 更新配置迁移和白名单说明。 +- [x] 完整后端测试通过。 +- [x] 前端类型检查和生产构建通过。 +- [x] Windows、Linux/Docker 路径与图片发送测试通过。 +- [x] Forgejo 提交、推送并记录版本号。 + +## 推荐实施顺序 + +```text +P0-01 参数合同 + → P0-02 单图任务提交 + → P0-03 任务查询 + → P0-04 异步图片回执 + → P1-01~P1-03 快速角色 + → P1-04~P1-06 详细工作流与选图 + → P2-05~P2-06 会话状态层 + → P2-01~P2-04 审核、高清和素材 + → P3 跨平台、限流、审计和发布验收 +``` diff --git a/docs/code-audit-2026-07-10.md b/docs/code-audit-2026-07-10.md new file mode 100644 index 0000000..a4c6e45 --- /dev/null +++ b/docs/code-audit-2026-07-10.md @@ -0,0 +1,146 @@ +# ImageStudio 代码与功能审计(2026-07-10) + +## 审计范围 + +本轮审计覆盖仓库中的 Python、TypeScript、Vue、HTML、YAML、JSON、SQL 迁移、脚本、测试和全部 Markdown 文档,重点核对: + +- AstrBot 插件生命周期、Plugin Page、Web API、Bridge 与持久化目录兼容性。 +- 后端路由、服务、数据库外键、配置写入、Provider 调用、任务状态机和错误传播。 +- 前端功能入口、API 契约、真实数据与示例数据边界、按钮行为和生产构建产物。 +- 重复代码、可复用解析逻辑、生产 mock/fallback、SSRF、身份伪造和 owner 边界。 + +## AstrBot 官方运行模式核对 + +官方资料: + +- [插件开发指南](https://docs.astrbot.app/dev/star/plugin-new.html) +- [插件 Pages](https://docs.astrbot.app/dev/star/guides/plugin-pages.html) +- [插件配置](https://docs.astrbot.app/dev/star/guides/plugin-config.html) +- [插件存储](https://docs.astrbot.app/dev/star/guides/storage.html) +- [AstrBot 官方仓库](https://github.com/AstrBotDevs/AstrBot) + +核对结论: + +1. 插件继承 `Star`;AstrBot 在启用时调用 `initialize()`,在停用或重载时调用 `terminate()`。 +2. `pages//index.html` 会被自动发现,并在受限 iframe 中运行;页面通过 `window.AstrBotPluginPage` 调用后端。 +3. 后端使用 `context.register_web_api(route, handler, methods, desc)` 注册,动态路径语法为 `` 和 ``。ImageStudio 的 `//` catch-all 与官方语法一致。 +4. Page 端传相对 endpoint;Dashboard 转发到 `/api/v1/plugins/extensions//`。v4.25 legacy 前缀仍兼容 `/api/plug//`。 +5. 当前官方 Bridge 的 JSON 操作以 GET/POST 为主;ImageStudio 对旧 Bridge 的 PUT/PATCH/DELETE 使用 POST method-override 信封兼容。 +6. 官方 `PluginRequest` 提供 `username`、`client_host`、`query`、`json()` 和 `path_params` 等字段,不提供可用于细粒度角色授权的完整用户角色模型。 +7. `get_astrbot_plugin_data_path()` 返回 AstrBot 的 `data/plugin_data` 根目录,插件仍需追加自己的插件名。因此运行数据正确位置是 `data/plugin_data/astrbot_plugin_imagestudio/`。 + +## 已修复问题 + +### AstrBot 接入与安全 + +- Bridge 不再信任 `x-username`、`x-user-id` 或 `x-forwarded-for` 作为身份来源。 +- 用显式能力替代 wildcard 权限;任务重试同时要求 `can_write` 与 `can_retry_tasks`。 +- Bridge 同步业务分发移到工作线程,长时间 Provider 请求不再阻塞 AstrBot 事件循环。 +- Bridge 路由按方法只注册一个 catch-all,兼容官方 GET/POST 调用、POST method-override 信封和旧版 Quart 请求。 +- 媒体下载优先使用 AstrBot `file_response`,避免生产请求整文件 base64 复制。 +- Provider 限制 HTTP(S)、响应大小、图片魔数和同源重定向,并拒绝非显式允许的私网目标。 +- Provider 错误正文中的 Bearer token、API key 与常见密钥字段会在记录和返回前脱敏。 + +### 持久化与配置 + +- 运行数据库、媒体和可写配置映射到 AstrBot 插件数据目录。 +- SQLite 强制启用外键。 +- 配置、审查策略和集成 token 使用同目录临时文件、原子替换和进程内路径锁。 +- YAML 标量解析修复引号内 `#` 和字符串类型漂移。 +- 数据库配置切换会先初始化并验证目标数据库,验证失败时不会切换运行连接。 +- AstrBot runtime override 管理的 DSN、数据库密码与 Provider API key 不会因设置保存、回滚或快照操作写入 YAML;掩码后的 diff 也不会误报变化。 +- 生图和 Vision Provider 配置均可从 AstrBot 配置自动 bootstrap 到运行数据库,fresh install 不再缺少模型路由。 +- 配置路由使用稳定 ID;模型变化或配置清空会停用旧路由,Vision 配置也不会覆盖生图 Provider。 + +### Provider、媒体与生成上下文 + +- Provider 未配置或调用失败时,任务和节点进入失败状态;生产模式不再把 mock 图片当成功结果。 +- 默认模型、档案默认模型、Prompt Expansion 和 Cost Guard 已接入生成请求。 +- 上游节点图片、草稿参考图、stage input、显式参考图以及已确认素材都会真正传入 Provider,不再只保留 URI 元数据。 +- `storage://` 图片会校验归属、数量、单图/总大小和文件魔数,再转换为带正确 MIME 的 data URL。 +- 大型 inline data URL 不再持久化到任务数据库;若旧任务使用 inline 参考图,重试时必须提供替代参考图,不能静默丢失生成条件。 +- `negative_prompt` 已贯通到兼容 Provider。官方 OpenAI/Azure OpenAI 默认关闭该未知字段,其他路由可通过 capabilities/settings 显式开关。 +- PNG、JPEG、GIF 与 WebP 按真实魔数保存,并保持文件后缀、元数据 MIME、媒体响应与实际字节一致。 +- Provider JSON 响应上限按请求图片数量和 Base64 开销动态计算,总硬上限为 128 MiB。 + +### 任务、重试、取消与工作流 + +- 工作流初始状态执行枚举校验;非法状态不会落库。 +- 未完成的生成节点会让工作流审查失败。 +- 历史快照在写入前校验图片、工作流和节点归属,避免外键异常泄漏为未知错误。 +- 虚拟单图节点允许连续发起生成,不会在第一次完成后永久锁死。 +- node retry 与 job retry 都会执行真实、受控的同步重试,并生成新的幂等键、lineage、替代任务记录与有上限的 DLQ 记录。 +- job retry 预检失败会恢复原任务和节点状态;并发重试使用 CAS,只有一个请求能取得执行权。 +- task 运行、成功、失败和取消均使用比较并交换更新,陈旧请求不能覆盖终态。 +- workflow 取消会级联取消其活跃 task/node;若 Provider 已在远程执行,返回结果会被拒绝提交,暂存图片元数据、原图和缩略图会清理。 +- 同一 Provider 请求生成多张图时,usage 按 task 聚合,不再按每张输出图重复累计。 + +### 素材、素材包与 Vision + +- Vision 识别会读取真实 `storage://` 图片,并校验文件大小、扩展名、MIME 和魔数后再发送给 Provider。 +- Vision 未配置、调用失败或返回无效时不会回退 mock;素材保持 `pending_recognition`,且未成功识别的素材不能 confirm。 +- 素材包在首条记录写入前对清单、文件、类型、重复 material ID 和所有条目执行确定性预检。 +- 素材包 `styles` 当前明确报告为不支持,不再假报成功导入。 +- confirmed 素材可由前端导入角色衣橱;未确认素材会被后端拒绝。 + +### 前端与代码精简 + +- Plugin Page 删除重复轻量导航,直接复用完整 Workbench。 +- API client 统一解包 AstrBot 与 ImageStudio 响应,并兼容旧 Bridge method override。 +- 修复工作流详情解构、角色工作流后端 ID 同步、并发创建去重和素材选择假 ID。 +- mock、provider error、fallback 和 degraded 图片不能被采纳、归档或作为后续参考图。 +- 快速角色素材选择只提交真实、已确认的后端素材;示例素材只在预览/错误模式显示。 +- completed 节点可以持久化切换候选图,selected 节点仍保持锁定;成本面板使用真实 Provider usage。 +- 新增 `MaterialOperationDialog.vue`,提供已有 `storage://` 素材登记、素材包 dry-run/apply 和 confirmed 素材导入衣橱入口。 +- 导航、指标卡、状态卡、空态、图片预览、素材分类和素材详情已拆为独立组件;外部角色卡解析已拆为纯函数 feature 模块。 +- 状态、时间、容量、成本、集合去重和 `storage://` 校验已迁入共享工具,素材/队列/性能页的重复指标模板已合并。 +- 搜索支持 Ctrl/Cmd+K、方向键和 Enter;移动导航显示标签,弹窗补齐焦点管理、Escape、safe-area 与 `100dvh`。 +- 真实后端无模型或空素材库时不再伪装成示例数据;示例模型/素材只在明确的预览或错误模式显示。 +- node retry 前端只发送一次请求;模型 discovery 文案明确为 metadata-only。 +- 13 个 route 文件的重复字符串、数字和布尔参数解析已合并到 `backend/routes/params.py`。 +- 生成参考图收集、任务重试、Provider bootstrap、配置安全写入、图片格式识别与远程抓取分别拆入独立服务/核心模块,减少路由与主流程中的重复实现。 + +## 当前发布边界与延期项 + +以下项目不应被误解为已完成: + +1. **完整多租户 owner 链**:角色档案已按身份隔离,但 workflow、node、image、job、wardrobe、Dashboard 和媒体仍未完成统一祖先链授权。当前产品边界是可信 AstrBot WebUI 管理员工作台。 +2. **可信成本上限**:成本接口可以聚合 Provider usage,但服务端尚无可信价格表;调用方提供的 `unit_prices` 只能用于估算,不能作为强财务控制。 +3. **持久后台 Worker**:生产重试采用受控同步重试闭环;尚未引入跨进程持久 worker、租约和原子 claim 调度器。进程被强制终止时,极端情况下可能遗留 `retrying` 状态,需要恢复任务或后续巡检。 +4. **Provider 物理中止**:远程 HTTP 请求一旦发出无法可靠撤回;取消操作保证结果不提交并清理本地暂存结果,但不能保证上游 Provider 停止计费或计算。 +5. **前端进一步拆分**:已抽出导航、指标/状态原语、图片预览、素材弹窗、角色卡解析及共享工具,但完整 `App.vue` Workbench 仍有约 18,138 行,后续应继续拆分工作流 store/actions、通知和异步资源 composable。 +6. **模型 discovery 语义**:`api-discovery/probe-*` 当前只登记并校验能力元数据,不执行外部网络探测。真实可用性由配置后的 Provider 请求验证;UI 和文档不得把 metadata-only 操作描述为在线探测成功。 +7. **素材包事务边界**:导入前已有完整、确定性的全包预检,但多条素材记录仍不是单一数据库事务;若底层数据库在写入阶段发生异常,可能需要人工处理已写入的前缀记录。 + +## 最终验证 + +最终交付前已执行: + +```powershell +python -m compileall -q adapters backend core database engines scripts tests main.py + +cd frontend +npm.cmd run typecheck +npm.cmd run build:plugin +npm.cmd run build:web +``` + +结果: + +- 全量 Python:共 359 项,结果 OK,其中 1 项按环境条件跳过。 +- 高风险组合回归:170 项全部通过。 +- `compileall`:通过。 +- 非 UI smoke:通过,111 条路由/handler 注册及调用健康。 +- PostgreSQL smoke `--dry-run`:通过。 +- `git diff --check`:通过;仅报告工作区 LF 将转 CRLF 的提示,无空白错误。 +- 前端 `typecheck`、`build:plugin`、`build:web`:全部通过。 +- 浏览器 UI smoke:桌面与 390×844 移动端渲染正常;导航固定、移动标签、Ctrl/Cmd+K 搜索、队列 tab、明暗主题和图片缩放弹窗通过,控制台无应用 error/warn。 + +最终构建产物: + +- `pages/imagestudio/assets/index-CF6dVfCE.js` +- `pages/imagestudio/assets/index-D4NekmM3.css` +- `web/assets/index-djgvYVE7.js` +- `web/assets/index-VBJk7aNa.css` + +旧 hash 产物已由构建流程移除。 diff --git a/docs/frontend-integration-queue.md b/docs/frontend-integration-queue.md new file mode 100644 index 0000000..cb764c2 --- /dev/null +++ b/docs/frontend-integration-queue.md @@ -0,0 +1,113 @@ +# ImageStudio 前端接入队列 + +更新时间:2026-07-11 + +本文记录后端已经具备、但前端尚未完整接入的功能,以及当前前后端仍需闭环的能力。实施时按优先级从上到下推进。 + +## P0:核心功能闭环 + +### 1. 多适配器真实路由(已完成,含默认切换与运行时故障回退) + +- 后端加载并管理全部生成适配器,而非只使用 `generation.provider`。 +- 支持默认适配器切换、任务指定适配器和禁用状态。 +- 支持故障切换、路由优先级和不可用适配器跳过。 +- 将模型能力档案绑定到具体适配器与模型。 +- 删除适配器前检查是否仍被任务、路由或配置引用。 + +### 2. 配置快照管理(已接入) + +- 展示快照列表、详情、操作人、时间和变更原因。 +- 调用 `settings/diff` 展示当前配置与快照的差异。 +- 接入 `settings/snapshots/{snapshot_id}` 和 `settings/rollback`。 +- 回滚前展示影响范围并要求二次确认。 + +### 3. 存储维护(已接入) + +- 接入 `storage/status`,展示目录可写性、空间与存储状态。 +- 接入 `storage/migrate`,展示迁移计划、进度和结果。 +- 接入 `media/index`,支持媒体索引重建。 +- 接入 `media/orphans`,支持孤儿文件扫描和安全处理。 + +### 4. 图库完整管理(已接入) + +- 接入选中、取消选中、归档和批量操作。 +- 接入安全删除与永久删除,并提供二次确认。 +- 展示批量操作进度、失败项和可恢复状态。 + +## P1:工作流与模型能力 + +### 5. 工作流生命周期管理(已接入) + +- 接入启动、取消和归档工作流。 +- 增加工作流状态操作栏。 +- 展示节点依赖上下文。 +- 接入工作流整体审查与历史版本恢复。 + +### 6. 审查策略页面(已接入) + +- 接入审查策略读取、校验和保存。 +- 配置角色一致性、候选图规则、错误与警告阈值。 +- 展示规则校验错误及影响范围。 + +### 7. 模型能力档案(已接入) + +- 展示已探测的适配器、模型和能力档案。 +- 区分生图、文本和视觉模型。 +- 支持重新探测、手动修正和能力状态管理。 +- 接入渲染计划兼容性校验。 + +### 8. 实时事件流(已接入增量轮询) + +- 接入 `events/stream`。 +- 实时更新生成进度、队列状态和日志。 +- 支持断线重连,并保留轮询降级方案。 + +## P2:系统管理能力 + +### 9. 私伴插件集成管理(已接入) + +- 展示集成 Token 状态。 +- 支持 Token 轮换、集成启停和最近调用记录。 +- 提供每日穿搭联调测试入口。 + +### 10. 成本保护(已接入) + +- 接入 `costs/guard`。 +- 配置单次、每日和每月预算。 +- 支持超限警告或阻止策略。 +- 按适配器和模型展示费用异常。 + +### 11. 完整数据备份(SQLite 与媒体已接入;PostgreSQL 提供 pg_dump 提示) + +- 当前页面仅支持脱敏配置导入导出。 +- 后续增加 SQLite 数据库备份和恢复。 +- 设计 PostgreSQL 导出、恢复和版本兼容方案。 +- 支持图片、素材与清单文件打包,并生成校验值。 + +### 12. 系统诊断页面(已接入) + +- 展示 API 合同、前后端版本、Web Manifest 和健康状态。 +- 展示已挂载路由及当前用户权限矩阵。 +- 提供诊断信息复制与脱敏导出。 + +## P3:补充与体验优化 + +### 13. 提示词扩写(已接入) + +- 接入 `prompts/expand`。 +- 支持扩写前后对比、一键采用和历史记录。 + +### 14. 素材包导入(已有预检与执行流程) + +- 完善导入预检、内容清单和冲突处理。 +- 展示导入进度、失败项和最终报告。 + +### 15. 页面权限统一(已完成主要入口) + +- 根据后端权限隐藏或禁用操作。 +- 统一展示无权限原因和只读状态。 +- 权限变化后自动刷新页面能力。 + +## 推荐实施顺序 + +`多适配器真实路由 → 配置快照 → 存储维护 → 图库管理 → 工作流生命周期 → 审查策略 → 能力档案 → 实时事件流 → 系统管理能力` diff --git a/docs/material-composition-prompt-routing-development-queue.md b/docs/material-composition-prompt-routing-development-queue.md new file mode 100644 index 0000000..db905e9 --- /dev/null +++ b/docs/material-composition-prompt-routing-development-queue.md @@ -0,0 +1,372 @@ +# ImageStudio 素材编译、提示词转译与模型分流开发队列 + +> 状态:已完成(2026-07-12) +> 建立日期:2026-07-12 +> 依据:`astrbot_imagestudio_asset_materials_focus_doc_v1.md` 与当前 ImageStudio 后端实现审计 +> 目标:升级“素材识别 → 素材库 → 角色拼装 → 提示词转译 → 内容审核 → 模型路由 → 生图”完整链路。 + +## 1. 当前实现审计结论 + +当前系统已经具备素材上传、视觉识别、人工确认、素材分类、角色衣橱、素材匹配、工作流节点绑定、参考图注入、静态模型路由、提示词合并和生成结果审核基础能力。 + +当前主要缺口: + +- 素材识别结果仍以单份 `prompt / negative_prompt / tags` 为主,没有 RawAnalysis、NormalizedAssetProfile、GenerationCapsule 三层结构。 +- 视觉模型完整结构化返回没有被持久保存,无法可靠地重新标准化或逐项人工修正。 +- 衣橱匹配后仍通过字符串连接形成 `outfit_prompt`,没有身体槽位、穿着层级、冲突解决和提示词预算。 +- 现有素材 reference strength 只表示参考图强度,不等于素材语义优先级。 +- 多张参考图没有 `role / use_for / do_not_inherit` 职责约束。 +- `PromptExpansionService` 只负责合并提示词,没有中文到英文的语义转译层。 +- 现有模型路由主要按节点、角色或设置静态选择,不能根据提示词审核结果动态分流。 +- 现有 review 主要用于工作流、候选图和结果质量审核,不是生成前提示词内容审核。 + +## 2. 总体目标链路 + +```text +用户中文需求 + → 原始需求保存 + → 用户授权检查 + → 语义转译与提示词结构化 + → 合规语义规范化(降低歧义和误报,不绕过安全策略) + → 素材 CompositionBuilder 编译 + → 生成前内容审核 + → 风险分级与路由决策 + ├─ 普通内容 → 默认生图模型 + ├─ 允许的成人内容 → 自动分流到已配置 NSFW 模型 + ├─ 不确定内容 → 人工确认或保守路由 + ├─ 恐怖主义 / 暴力宣扬 / 极端血腥 → 拒绝生成并警告 + └─ 其他禁止内容 → 拒绝生成 + → 生图 + → 生成后图片审核 + → 返回、隔离、重试或拒绝 +``` + +## 3. P0:素材三层数据结构 + +- [ ] 定义 `RawAnalysis` DTO 与 JSON Schema。 +- [ ] 定义 `NormalizedAssetProfile` DTO 与 JSON Schema。 +- [ ] 定义 `GenerationCapsule` DTO 与 JSON Schema。 +- [ ] 为三层数据增加 `schema_version`、`prompt_version`、`created_at` 和模型来源。 +- [ ] 扩展 `VisionRecognitionResult`,保留视觉模型完整结构化结果。 +- [ ] 将完整原始识别结果写入素材 metadata,而不是只保存响应键名。 +- [ ] 增加视觉模型返回 JSON Schema 校验。 +- [ ] 增加可修复 JSON 的有限修复流程;无法修复时明确失败,不生成伪结构。 +- [ ] 区分 `observed / inferred / unknown / occluded / low_visibility`。 +- [ ] 实现 0.60 与 0.80 置信度分层规则。 +- [ ] 低置信度和推测属性默认不得进入生图输入。 +- [ ] 实现 `MaterialNormalizationService`。 +- [ ] 建立类别、服装类型、颜色、长度、版型、材质等同义词归一表。 +- [ ] 自动生成 Compact、Standard、Detailed 三档 Generation Capsule。 +- [ ] 旧素材根据现有 prompt 自动生成 Legacy Capsule,保证无破坏迁移。 +- [ ] 重新识别时保留历史版本和人工修改,不直接覆盖已确认结构。 + +## 4. P0:素材语义、身体槽位与穿着层级 + +- [ ] 定义稳定的素材 `category` 枚举。 +- [ ] 定义素材 `role` 枚举。 +- [ ] 定义身体槽位:identity、hair、head、face、ears、neck、upper_body、arms、hands、waist、hips、upper_legs、lower_legs、feet、pose、background、global_style。 +- [ ] 定义穿着层级:skin、inner、base、middle、outer、overlay、accessory、prop。 +- [ ] 建立现有素材分类到标准类别、角色、槽位和层级的映射。 +- [ ] 建立衣橱分类到标准语义的映射。 +- [ ] 支持用户在确认素材时修正类别、角色、槽位和层级。 +- [ ] 为连衣裙、套装、外套、内搭、首饰、鞋靴和道具增加专门规则。 +- [ ] 对未知分类使用 `other`,并要求确认后再参加自动拼装。 + +## 5. P0:CompositionBuilder 素材拼接器 + +- [ ] 新建 `CompositionPlan` DTO。 +- [ ] 新建 `CompositionResult` DTO。 +- [ ] 新建 `CompositionBuilder` 或 `OutfitPromptCompiler` 服务。 +- [ ] 将角色身份作为最高优先级输入。 +- [ ] 从衣橱匹配结果读取素材,而不是直接连接素材 prompt。 +- [ ] 从素材读取 Generation Capsule。 +- [ ] 按身体槽位对素材分组。 +- [ ] 按穿着层级排序。 +- [ ] 实现连衣裙与独立上衣、下装冲突检测。 +- [ ] 实现同身体槽位多素材冲突检测。 +- [ ] 支持配饰可叠戴和不可叠戴规则。 +- [ ] 实现同一属性值矛盾检测,例如 fitted/oversized、matte/glossy。 +- [ ] 冲突解决遵循:用户锁定 > 人工修正 > 高置信观察 > 中置信观察 > 推测。 +- [ ] 记录保留、降级、忽略素材及具体原因。 +- [ ] 按统一顺序编译身份、发型、体型、内搭、外套、下装、鞋靴、配饰、姿势、场景、风格和光线。 +- [ ] 将 `must_preserve` 编译为明确约束段。 +- [ ] 将 `forbidden_changes` 编译到负向提示词或禁止变化段。 +- [ ] 输出 `resolution_log`、`conflicts`、`ignored_materials` 和 `downgraded_materials`。 +- [ ] 接入快速角色、详细角色和单图生成流程。 + +## 6. P0:素材权重系统 + +- [ ] 定义角色身份、主服装、连衣裙、下装、发型、鞋靴、配饰、姿势、画风和背景的默认基础权重。 +- [ ] 将衣橱“匹配得分”和生成“语义权重”拆成两个独立概念。 +- [ ] 增加 `user_adjustment`。 +- [ ] 增加素材锁定状态和 `lock_bonus`。 +- [ ] 增加主服装、身份素材等 `role_bonus`。 +- [ ] 根据当前构图和节点增加 `visibility_bonus`。 +- [ ] 根据冲突结果计算 `conflict_penalty`。 +- [ ] 根据置信度和推测状态计算 `low_confidence_penalty`。 +- [ ] 计算 `effective_weight` 并保留计算明细。 +- [ ] 根据最终权重决定胶囊档位、提示词顺序和参考图保留状态。 +- [ ] 保留现有 strong/medium/weak 作为模型参考图强度,不与素材语义权重混用。 + +## 7. P0:用户授权与提示词转译层 + +- [ ] 新建 `PromptTranslationService`,放在原始用户输入与 CompositionBuilder/PromptExpansionService 之间。 +- [ ] 设置页面增加“启用 LLM 提示词转译”总开关,默认关闭或要求用户明确启用。 +- [ ] 分别配置 Web、QQ、Telegram、Discord 等入口是否允许自动转译。 +- [ ] 保存用户原始输入、转译结果、模型、模板版本和时间,便于追踪与回滚。 +- [ ] 原始输入始终保留,不允许转译层静默改变用户核心意图。 +- [ ] 支持中文需求转为面向目标生图模型的英文提示词。 +- [ ] 支持自动拆分主体、外观、服装、动作、构图、场景、画风、光线、镜头和负向约束。 +- [ ] 支持对 CompositionBuilder 的结构化结果进行英文编译,而不是对最终长字符串盲目翻译。 +- [ ] 支持模型专用转译模板,例如自然语言型、标签型、SD 权重型。 +- [ ] 支持专有名词、角色名、品牌名和用户锁定词“不翻译/固定翻译”词典。 +- [ ] 支持中英文混合输入和仅英文输入直通。 +- [ ] 支持转译前后差异摘要。 +- [ ] 用户可选择“直接使用原文 / 使用转译 / 手动编辑后使用”。 +- [ ] 转译失败时安全回退到原始输入,不得生成空提示词。 +- [ ] 对转译服务设置超时、重试、token 预算和成本审计。 + +## 8. P0:合规语义规范化层 + +> 本层用于降低歧义、减少无意误报并保持需求效果,不用于隐藏禁止内容或绕过模型、平台及服务商的安全审核。 + +- [ ] 新建 `PromptSemanticNormalizer`,与翻译服务解耦。 +- [ ] 将容易产生歧义的口语转换为准确、中性的视觉描述。 +- [ ] 将年龄、人物身份、身体状态等高风险歧义显式化,避免模型误判。 +- [ ] 对合法成人题材使用明确的“成年角色”约束。 +- [ ] 对摄影、医学、艺术、泳装、舞台服装等合法上下文补充必要语境。 +- [ ] 不允许删除或伪装会改变安全分类的核心语义。 +- [ ] 不允许将被禁止内容改写成隐语、编码词或规避词。 +- [ ] 保留改写前后文本及改写理由。 +- [ ] 用户锁定的核心视觉效果不可被无提示删除。 +- [ ] 如果规范化会显著改变需求,必须要求用户确认。 +- [ ] 为每次规范化输出 `changed_terms`、`reason`、`meaning_preserved`。 + +## 9. P0:生成前提示词审核层 + +- [ ] 新建 `PromptModerationService`,在最终模型路由确定前执行。 +- [ ] 审核对象包含原始输入、转译输入、CompositionResult、负向提示词和参考图职责。 +- [ ] 审核返回结构化类别、置信度、风险等级、原因和处置建议。 +- [ ] 风险等级至少区分 `safe / sensitive / adult_allowed / uncertain / violence_prohibited / terror_prohibited / prohibited`。 +- [ ] 区分合法成人内容与禁止内容,禁止内容不得通过 NSFW 路由继续生成。 +- [ ] 单独识别恐怖主义宣传、招募、赞美、威胁和现实行动引导。 +- [ ] 单独识别暴力宣扬、针对现实目标的伤害威胁和高危暴力行动请求。 +- [ ] 单独识别以极端血腥、肢解、酷刑为核心表现目的的内容。 +- [ ] `terror_prohibited` 与 `violence_prohibited` 必须直接拒绝,不允许进入普通或 NSFW 生图模型。 +- [ ] 恐暴拒绝响应必须返回明确警告,同时避免复述可操作的危险细节。 +- [ ] 恐暴命中写入高优先级安全审计事件,包含入口、用户、群聊、分类和规则版本。 +- [ ] 对涉及未成年人或年龄不明确的成人内容采用拒绝或人工确认策略。 +- [ ] 对高风险但不确定内容进入保守策略,不自动选择宽松模型。 +- [ ] 支持规则审核与 LLM/VLM 审核组合。 +- [ ] 规则结果与模型审核冲突时采用更严格结果,或按配置进入人工确认。 +- [ ] 审核服务失败时采用 fail-closed 或由管理员明确配置的保守回退。 +- [ ] 将审核决策写入任务 metadata 和审计日志,但敏感原文按日志脱敏策略保存。 +- [ ] 增加审核模板和分类版本号。 + +## 10. P0:普通与 NSFW 生图模型动态分流 + +- [ ] 设置页面增加“内容审核与模型分流”配置栏。 +- [ ] 增加默认安全生图路由选择。 +- [ ] 增加允许成人内容的 NSFW 生图路由选择。 +- [ ] NSFW 路由只能选择管理员明确标记为允许成人内容的已配置适配器和模型。 +- [ ] 增加“是否允许成人内容自动分流”开关,默认关闭;管理员开启后,对通过审核与权限检查的 `adult_allowed` 内容自动执行分流。 +- [ ] 增加不同入口、群聊、用户和角色工作流的成人内容权限开关。 +- [ ] 成人内容分流同时受现有用户白名单、群聊白名单和额度策略约束。 +- [ ] 实现 `GenerationRouteDecision`:requested_route、moderation_level、selected_route、reason、fallback_route。 +- [ ] safe/sensitive 默认进入普通模型。 +- [ ] `adult_allowed` 且用户、入口、管理员策略全部允许时自动进入 NSFW 模型,无需再次手动选择模型。 +- [ ] uncertain 默认要求确认或拒绝,不自动分流到 NSFW 模型。 +- [ ] `violence_prohibited`、`terror_prohibited` 和 `prohibited` 始终拒绝,不进入任何生图模型。 +- [ ] NSFW 模型只承接允许的成人内容,不得作为恐怖、暴力或其他禁止内容的兜底模型。 +- [ ] 未配置 NSFW 路由时返回明确错误,不偷偷回退到普通模型。 +- [ ] 增加 `GenerationFailureClassifier`,将普通模型失败至少拆分为 `content_safety_rejection / rate_limit / quota_exhausted / authentication / invalid_request / unsupported_capability / timeout / upstream_error / unknown`,保留供应商状态码、错误码和脱敏错误摘要。 +- [ ] 普通模型返回失败时先判断失败类型;网络、超时、额度、鉴权、参数、能力不支持和未知错误不得触发 NSFW 路由。 +- [ ] 当供应商明确返回成人内容或内容安全拒绝信号时,重新调用 `PromptModerationService` 复核原始提示词、转译提示词和实际发送提示词,不能仅凭模糊的供应商报错改投 NSFW。 +- [ ] 二次审核结果为 `adult_allowed`,且管理员开关、入口权限、用户/群聊白名单、额度策略和 NSFW 路由配置全部通过时,允许从普通模型单次回退到 NSFW 模型尝试。 +- [ ] 二次审核为 `uncertain` 时要求确认或拒绝;为 `violence_prohibited / terror_prohibited / prohibited` 时直接禁止并警告,不得回退。 +- [ ] 普通模型到 NSFW 模型的失败回退最多执行一次,记录 `initial_route / failure_class / moderation_recheck / fallback_route / fallback_reason`,禁止循环切换模型。 +- [ ] NSFW 回退也失败时返回明确的最终失败原因,不再自动投递其他成人模型;是否允许人工重试由管理员策略决定。 +- [ ] 动态路由决策必须发生在成本检查、任务入队和模型解析前。 +- [ ] 将最终路由原因写入任务、图片和聊天审计记录。 + +## 11. P1:生成后图片审核 + +- [ ] 新建独立的 `ImageModerationService`,不要复用工作流质量审核结果冒充内容审核。 +- [ ] 对普通模型和 NSFW 模型生成结果采用不同但明确的审核策略。 +- [ ] 审核生成图片是否超出请求允许范围。 +- [ ] 审核生成结果是否出现年龄不明、禁止内容或意外裸露。 +- [ ] 结果分为允许返回、警告返回、隔离、重新生成和拒绝返回。 +- [ ] 聊天端返回前执行图片审核。 +- [ ] Web 图库对受限结果增加权限控制和遮罩。 +- [ ] 隔离图片不进入普通图库、角色封面和素材库。 +- [ ] 为误判提供管理员复核流程。 +- [ ] 保存审核模型、分类、置信度和版本,但避免不必要地复制敏感图片。 + +## 12. P1:提示词预算与压缩 + +- [ ] 为不同模型路由配置提示词/token 预算。 +- [ ] 根据预算自动选择 Detailed、Standard 或 Compact 胶囊。 +- [ ] 超出预算时依次删除推测项、可选特征和低优先级细节。 +- [ ] 用户锁定项、身份级特征和安全必要语境不得自动删除。 +- [ ] 转译层必须接收预算信息,避免先生成超长文本再粗暴截断。 +- [ ] 输出预算前后长度、删除内容和保留原因。 +- [ ] 预算压缩后重新进行一次轻量审核,防止上下文删除造成语义变化。 + +## 13. P1:参考图职责绑定 + +- [ ] 定义 `ReferenceImageBinding`。 +- [ ] 每张参考图保存 `role`、`use_for`、`do_not_inherit` 和权重。 +- [ ] 角色参考图默认只继承身份、面部和发型。 +- [ ] 服装参考图默认禁止继承面部、体型、姿势、背景和画风。 +- [ ] 根据素材权重决定图片参考保留或降级为文字胶囊。 +- [ ] 超过模型参考图上限时给出确定性的降级顺序。 +- [ ] 为不同适配器实现职责与原生参数映射。 +- [ ] 不支持职责参数的模型自动追加明确的英文文本约束。 + +## 14. P1:API 与前端 + +- [ ] 素材详情接口返回 RawAnalysis、NormalizedAssetProfile 和三档 Generation Capsule。 +- [ ] 素材详情增加“原始识别 / 标准属性 / 生图胶囊”页签。 +- [ ] 支持逐项修改属性、置信度、来源类型、槽位、层级和锁定状态。 +- [ ] 素材卡展示类别、槽位、层级和确认状态。 +- [ ] 增加按颜色、材质、槽位、层级、置信度筛选。 +- [ ] 角色衣橱支持设置优先级、锁定和可叠戴。 +- [ ] 角色拼装页增加槽位与层级视图。 +- [ ] 生图前显示冲突、自动解决方案和忽略项。 +- [ ] 提供最终提示词、负向提示词、参考图职责和路由决策预览。 +- [ ] 设置页增加提示词转译模型卡片和模板配置。 +- [ ] 设置页增加用户授权、入口授权和转译总开关。 +- [ ] 设置页增加普通生图路由、NSFW 生图路由和审核模型路由。 +- [ ] 设置页明确显示“成人内容可自动路由;恐怖与高危暴力内容始终禁止”的固定策略。 +- [ ] 设置页明确展示禁止内容不会被分流生成。 +- [ ] 聊天命令支持查看本次转译、审核和路由摘要,但默认不泄露内部规则细节。 +- [ ] 高风险或不确定内容提供用户确认交互。 + +## 15. P1:数据库、迁移与隐私 + +- [ ] 第一阶段继续使用 material metadata 保存三层素材结构,避免立即大改表结构。 +- [ ] 为旧素材增加惰性迁移器。 +- [ ] 为提示词转译、审核和路由决策定义版本化记录结构。 +- [ ] 明确原始提示词、转译提示词和审核结果的保留时间。 +- [ ] 日志界面默认脱敏高风险原文。 +- [ ] 数据导出允许用户选择是否包含原始提示词和审核记录。 +- [ ] 数据备份包含素材 Schema 与胶囊版本。 +- [ ] 统计查询需求稳定后再评估独立 JSON 字段或子表。 + +## 16. P1:测试队列 + +- [ ] 视觉模型合法三层结构返回测试。 +- [ ] 缺字段、错误字段类型、非 JSON 和超长响应测试。 +- [ ] 置信度 0.60/0.80 边界测试。 +- [ ] observed/inferred 分流测试。 +- [ ] 三档胶囊生成和旧素材 Legacy Capsule 测试。 +- [ ] 同义词标准化测试。 +- [ ] 连衣裙与上下装冲突测试。 +- [ ] 外套与内搭合法叠穿测试。 +- [ ] 同槽位配饰冲突与允许叠戴测试。 +- [ ] 用户锁定覆盖自动权重测试。 +- [ ] 中文转英文语义保持测试。 +- [ ] 中英文混合输入和专有名词锁定测试。 +- [ ] 转译模型超时、失败、空返回和格式错误测试。 +- [ ] 合规语义规范化不改变核心需求测试。 +- [ ] 禁止内容不能通过改写进入生成路由测试。 +- [ ] safe、sensitive、adult_allowed、uncertain、violence_prohibited、terror_prohibited、prohibited 分级审核测试。 +- [ ] 成人内容权限、用户白名单和群白名单组合测试。 +- [ ] 允许的成人内容在全部权限命中后自动选择 NSFW 路由测试。 +- [ ] 恐怖主义内容直接拒绝、返回警告且不创建任务测试。 +- [ ] 高危暴力与极端血腥内容直接拒绝、返回警告且不创建任务测试。 +- [ ] 恐暴内容不能通过手动指定 NSFW 路由绕过审核测试。 +- [ ] NSFW 路由未配置时明确失败测试。 +- [ ] 普通供应商的网络、超时、额度、鉴权、参数和未知失败不会触发 NSFW 路由测试。 +- [ ] 普通供应商明确内容安全拒绝后会触发二次审核测试。 +- [ ] 二次审核为 `adult_allowed` 且权限全部通过时,普通模型失败后单次回退 NSFW 模型测试。 +- [ ] 二次审核为 uncertain、恐暴或其他禁止内容时不回退并按策略确认、拒绝或警告测试。 +- [ ] NSFW 回退失败后不循环重试、不继续切换其他模型测试。 +- [ ] 路由决策发生在成本检查和任务入队前测试。 +- [ ] 提示词预算压缩与压缩后二次审核测试。 +- [ ] 多参考图职责和数量上限测试。 +- [ ] 生成后图片审核、隔离和图库权限测试。 +- [ ] QQ、Telegram、Discord 和 Web 入口回归测试。 +- [ ] 快速角色、详细角色、单图和高清修复全链路回归测试。 + +## 17. P2:后续增强 + +- [ ] 素材识别和标准化版本对比。 +- [ ] 批量重新标准化,不重新调用视觉模型。 +- [ ] 素材组合模板和角色常用套装。 +- [ ] 根据角色体型、季节、场景自动调整可见性权重。 +- [ ] 基于生成结果审核素材还原度。 +- [ ] 将还原度反馈到素材权重和胶囊档位选择。 +- [ ] 统计素材采用率、冲突率、降级率、转译采用率、审核命中率和路由分布。 +- [ ] 为支持 ControlNet、IP-Adapter、区域提示词和多参考权重的适配器增加高级映射。 +- [ ] 对不同模型建立提示词效果基准集和离线评测。 + +## 18. 推荐实施批次 + +### 批次 A:数据基础 + +- [ ] 三层素材 DTO、Schema、识别保存、标准化和 Legacy Capsule。 +- [ ] 标准类别、身体槽位、穿着层级。 + +### 批次 B:素材编译核心 + +- [ ] CompositionBuilder、权重计算、冲突解决和编译日志。 +- [ ] 接入衣橱、快速角色和详细角色流程。 + +### 批次 C:转译与审核 + +- [ ] PromptTranslationService、用户授权和语义规范化。 +- [ ] PromptModerationService 与成人、恐怖、暴力及其他禁止内容分级结果。 + +### 批次 D:模型动态分流 + +- [ ] 普通/NSFW 路由配置、权限检查和 GenerationRouteDecision。 +- [ ] 在成本检查、任务入队前完成路由决策。 + +### 批次 E:前端闭环 + +- [ ] 素材三层详情、拼装预览、冲突操作、转译预览、审核提示和路由配置。 + +### 批次 F:生成后审核与全量回归 + +- [ ] 图片审核、隔离、图库权限和跨聊天平台测试。 + +## 19. 完成定义 + +- [x] 任意已确认素材均具有可追溯 RawAnalysis、标准化档案和至少一份可用胶囊。 +- [x] 多素材组合不再通过无规则字符串拼接完成。 +- [x] 每次拼装都能解释素材保留、降级和忽略原因。 +- [x] 用户授权后,中文需求可稳定转为保持原意的英文模型提示词。 +- [x] 规范化层不会隐藏禁止内容或绕过安全审核。 +- [x] 每次生成前都有可审计的内容风险结果和模型路由决策。 +- [x] 只有管理员配置、用户权限和审核结果同时允许时,成人内容才会自动进入 NSFW 路由。 +- [x] 恐怖主义、高危暴力和极端血腥内容直接拒绝并警告,且不会创建生成任务。 +- [x] 禁止内容不会进入普通或 NSFW 生图模型。 +- [x] 生成后图片在返回聊天端和进入图库前完成相应审核。 +- [x] 所有旧素材、旧工作流和现有前端功能保持兼容。 +- [x] 后端、前端、数据库迁移及聊天平台测试全部通过。 + +## 20. 实施完成记录 + +- [x] 批次 A:三层素材 DTO/Schema、完整视觉结构保存、置信度规则、Legacy Capsule、标准类别、身体槽位和穿着层级。 +- [x] 批次 B:CompositionBuilder、语义权重、用户锁定、冲突解决、预算压缩、参考图职责和详细工作流素材编译。 +- [x] 批次 C:PromptTranslationService、用户授权、原文回退、PromptSemanticNormalizer、PromptModerationService 和恐暴禁止策略。 +- [x] 批次 D:GenerationRouteDecision、普通/NSFW 动态路由、供应商失败分类、内容安全拒绝二次审核和单次 NSFW 回退。 +- [x] 批次 E:设置页提示词治理卡片、发送前预览、素材三层详情/编辑/历史、语义筛选、套装模板与拼装预览。 +- [x] 批次 F:生成后图片审核、单次自动再生成、隔离图库、管理员复核、聊天入口权限和成人独立白名单。 +- [x] 数据与隐私:惰性迁移、批量重新标准化、Schema 版本备份、提示词/审核记录可选导出和日志脱敏记录。 +- [x] 后续增强:套装模板、场景/季节/可见槽位权重、素材还原度反馈、统计 API、参考图方言映射和离线基准工具。 +- [x] 后端回归:420 passed、1 skipped、31 subtests passed。 +- [x] 前端验证:plugin-page 与 standalone-web 两套 Vite 生产构建通过。 + +主要新增入口: + +- `POST prompts/governance-preview` +- `GET materials/analytics` +- `POST materials/renormalize-batch` +- `POST materials/{material_id}/renormalize` +- `GET/POST profiles/{profile_id}/wardrobe-templates` +- `POST profiles/{profile_id}/wardrobe-templates/{template_id}/apply` +- `POST images/{image_id}/moderation-review` + +说明:上方原始清单保留为需求追溯底稿;本节为最终实施状态,以自动化测试和生产构建结果为验收依据。 diff --git a/docs/ui-function-entry-index.md b/docs/ui-function-entry-index.md new file mode 100644 index 0000000..5a86d9e --- /dev/null +++ b/docs/ui-function-entry-index.md @@ -0,0 +1,297 @@ +# ImageStudio UI 功能入口与接口索引 + +日期:2026-07-07 +维护规则:每新增一个可点击的功能入口、浮层、工作台入口、页面入口或后端 API 占位,都必须更新本文档。 + +> AstrBot 运行约定(2026-07-10 校正):生产 UI 由 `pages/imagestudio/index.html` 自动发现;插件在 `initialize()` 中通过 `context.register_web_api` 注册接口。`window.AstrBotPluginPage` 只传 `ui/bootstrap` 这类相对 endpoint,并兼容 v4.25 legacy `/api/plug/astrbot_plugin_imagestudio/` 与当前 `/api/v1/plugins/extensions/astrbot_plugin_imagestudio/`。文中独立 `web/` 或 Vite 地址只代表开发预览,不是 AstrBot 保证注册的生产入口。 + +## 2026-07-08 已完成闭环总结 + +- 快速角色:创建角色档案问卷已扩展为简单/复杂模式,复杂字段支持世界观、背景、身高/身材、身体特征、喜好、关系、故事钩子、画风和镜头偏好;字段可留空,不阻断生成档案。 +- 快速角色落库:生成档案会调用 `profiles/quick-draft/save`,保存角色档案并创建对应 `quick_profile`/角色工作流关联,前端不会在预览模式伪造落库成功。 +- 外部角色卡导入:工作流导入入口已提供粘贴导入弹窗,可识别外部人设文本并回填角色档案问卷。 +- 角色工作流:前期人设准备节点已接入后端角色工作流执行入口 `workflows/:workflowId/execute-character`,成功后进入 LLM 设定拆解节点。 +- LLM 设定拆解:中间大卡片固定为拆解工作台,身份性格、外观锚点、服装素材、约束负面项按组顺序排列,超出内容滚动展示;右侧详情显示 LLM 拆解摘要。 +- 节点对话注入:底部对话框会打包当前节点、角色档案、素材、拆解字段和用户原文,标记 `source_language:auto`、`target_prompt_language:en`、`translation_mode:backend_llm` 后提交 `nodes/:nodeId/generate`。 +- 工作流锁定与分支:后端将 `completed`/`selected` 节点作为锁定节点,禁止原链路继续修改、重跑、选图、归档或删除绑定图片;如需修改,前端提供“从此节点创建分支”,调用 `workflows/:workflowId/branches` 和 `workflows/:workflowId/switch-branch`,从选中节点切分支并保证旧链路干净。 +- 分支复制规则:从 `base_node_id` 创建分支时,切点之前的节点保留完成状态和选中图;切点及之后重置为 `draft` 并清空选中图,metadata 写入 `copied_from_node_id` 和分支切点信息。 +- 验证结果:后端锁定/分支/生成/图像/历史相关测试 39 条通过;前端 `vue-tsc --noEmit` 和 `build:web` 通过;浏览器预览工作流三栏布局无控制台错误。 + +## 维护方式 + +- `入口名称` 写用户在界面上能理解的名称。 +- `位置` 写清楚入口在哪个页面、哪个卡片或哪个浮层。 +- `当前动作` 写现在点击会发生什么。 +- `关联接口` 写已规划或已接入的 endpoint;纯前端占位写 `local/*`。 +- `状态` 使用:`已接 UI`、`模拟数据`、`占位入口`、`待后端接入`、`已接后端`。 + +## 前端入口总览 + +### 左侧导航 + +| 入口名称 | 位置 | 当前动作 | 关联模块 | 状态 | +| --- | --- | --- | --- | --- | +| 概览 | 左侧星图导航 | 切换到概览页 | `overview` | 已接 UI | +| 生成 | 左侧星图导航 | 切换到生成页 | `generate` | 已接 UI | +| 队列 | 左侧星图导航 | 切换到队列模块 | `queue` | 已接 UI | +| 性能 | 左侧星图导航 | 切换到性能详情页 | `performance` | 占位入口 | +| 图库 | 左侧星图导航 | 切换到图库模块 | `images` | 占位入口 | +| 角色 | 左侧星图导航 | 切换到角色模块 | `suggestions` | 占位入口 | +| 素材 | 左侧星图导航 | 切换到素材模块 | `images` | 占位入口 | +| 设置 | 左侧星图导航 | 切换到设置模块 | `settings` | 已接 UI | +| 日志 | 左侧星图导航 | 切换到日志模块 | `logs` | 已接 UI | + +### 顶部工具区 + +| 入口名称 | 位置 | 当前动作 | 关联接口 | 状态 | +| --- | --- | --- | --- | --- | +| 主题切换 | 顶部工具区 | 日间/夜间模式切换 | `local/preferences` | 已接 UI | +| 消息中心 | 顶部工具区铃铛 | 打开消息浮层,外部点击关闭 | `local/notifications` | 模拟数据 | +| 日志浮层 | 顶部工具区文件图标 | 悬停或点击打开日志浮层 | `logs/generation`, `logs/errors` | 已接 UI | +| 后端连接诊断 | 顶部工具区刷新按钮旁 | 显示插件桥/同源 API 连接状态,预览或异常时展开诊断横幅 | `ui/bootstrap` | 已接 UI | +| 调色盘 | 顶部工具区调色盘 | 打开主题色选择浮层 | `local/preferences` | 已接 UI | +| 搜索 | 顶部搜索框 | 搜索功能入口和模块 | `local/search-index` | 已接 UI | + +## 生成页入口 + +### 生成页模式切换 + +| 入口名称 | 位置 | 当前动作 | 关联接口 | 状态 | +| --- | --- | --- | --- | --- | +| 图片生成 | 生成页顶部模式按钮 | 切换到图片生成工作台 | `nodes/:nodeId/generate` | 已接 UI / 后端落库 | +| 快速角色 | 生成页顶部模式按钮 | 切换到快速角色工作台 | `profiles/quick-draft`, `profiles/quick-draft/save` | 已接 UI / 后端落库 | +| 角色工作流 | 生成页顶部模式按钮 | 切换到角色工作流工作台 | `profiles/:profileId/workflows`, `workflows/:workflowId/nodes` | 已接 UI / 后端数据 | + +### 图片生成工作台 + +| 入口名称 | 位置 | 当前动作 | 关联接口 | 状态 | +| --- | --- | --- | --- | --- | +| 生图模型选择 | 图片生成底部输入框 | 选择当前生图模型 | `api-discovery/capabilities` | 模拟数据 | +| 自然语言输入框 | 图片生成底部 | 输入自然语言生图需求 | `local/prompt-composer` | 占位入口 | +| 提交生成 | 图片生成输入框按钮 | 自动准备单图工作流并提交后端生成,返回图片写入预览 | `profiles`, `profiles/:profileId/workflows`, `nodes/:nodeId/generate` | 已接 UI / 后端落库 | +| 最近请求 | 图片生成输入框内 | 点击填入历史请求 | `local/recent-prompts` | 模拟数据 | +| 图片队列 | 图片生成左侧队列 | 优先展示后端返回图片和最近图库结果,选择后在主图区放大显示 | `images/:imageId` | 已接 UI / 后端数据 | +| 图片主预览 | 图片生成中间主图 | 点击打开图片详情卡 | `images/:imageId` | 已接 UI | +| 图片详情卡 | 图片预览浮层 | 滚轮缩放、双击缩放、拖拽查看 | `images/:imageId` | 已接 UI | + +## 快速角色入口 + +| 入口名称 | 位置 | 当前动作 | 关联接口 | 状态 | +| --- | --- | --- | --- | --- | +| 全部角色档案列表 | 快速角色左侧 | 选择当前角色卡 | `profiles` | 模拟数据 | +| 添加档案 | 全部角色档案列表顶部 | 打开创建角色档案问卷,可在生成档案旁切换简单/复杂问卷 | `profiles/quick-draft` | 已接 UI | +| 角色预览折角 | 角色预览卡右上角 | 打开角色设定详情大卡片 | `profiles/:profileId` | 已接 UI | +| 角色设定详情大卡片 | 角色预览折角打开 | 查看当前角色基础、外观、风格、负面词 | `profiles/:profileId` | 模拟数据 | +| 半身像/头像图片 | 角色预览图 | 打开图片详情卡,可缩放拖拽 | `images/:imageId` | 已接 UI | +| 角色图片类型卡 | 角色图片区域右侧 | 切换头像、半身、立绘、设定图 | `images/:imageId` | 模拟数据 | +| 参考素材卡 | 快速角色参考素材区 | 打开素材选择浮层 | `materials` | 已接 UI | +| 不使用参考 | 素材选择浮层顶部 | 清空对应分类素材 | `local/material-selection` | 已接 UI | +| 添加素材占位卡 | 素材选择浮层底部 | 记录添加意图,后续跳转素材库新增 | `materials/upload` | 占位入口 | +| 配饰细分类 | 配饰素材浮层 | 切换配饰子类并单列展示 | `material-categories` | 模拟数据 | +| 生成人设 | 快速角色功能按钮 | 打开问卷大卡片 | `profiles/quick-draft` | 已接 UI | +| 保存角色 | 快速角色功能按钮 | 调用后端保存快速角色档案,并创建 `quick_profile` 工作流 | `profiles/quick-draft/save` | 已接 UI / 后端落库 | +| 整理素材集 | 快速角色功能按钮 | 后续整理角色专属素材包 | `materials` | 占位入口 | +| 衣橱匹配 | 快速角色功能按钮 | 后续按当前人设匹配服装与配饰 | `profiles/:profileId/wardrobe/match` | 占位入口 | +| 每日穿搭 | 快速角色功能按钮 | 后续生成长期穿搭方案 | `integrations/private-companion/daily-outfit` | 占位入口 | +| 进入工作流 | 快速角色功能按钮 | 切到角色工作流并使用当前人设继续细化 | `profiles/:profileId/workflows` | 已接 UI | + +## 角色工作流入口 + +### 节点流程 + +| 入口名称 | 位置 | 当前动作 | 关联接口 | 状态 | +| --- | --- | --- | --- | --- | +| 大节点卡 | 角色工作流左侧节点流程 | 优先读取已选角色的后端角色工作流节点,缺数据时回退演示节点 | `workflows/:workflowId/nodes` | 已接 UI / 后端数据 | +| 小节点卡 | 大节点内部 | 选择具体节点并联动中间工作幕布 | `nodes/:nodeId` | 已接 UI / 后端数据 | +| 前期人设准备 | 大节点 | 展示人设导入、拆解、提示词和素材绑定节点 | `workflows/:workflowId/nodes` | 模拟数据 | +| 头像草稿 | 大节点 | 展示头像草稿多轮节点 | `nodes/:nodeId/images` | 模拟数据 | +| 全身与设定图 | 大节点 | 展示全身、手部、配饰和设定图拆分节点 | `nodes/:nodeId/images` | 模拟数据 | +| 审核修复并入库 | 大节点 | 展示审核、高清修复和归档节点 | `workflows/:workflowId/review` | 模拟数据 | + +### 工作幕布联动 + +| 入口名称 | 位置 | 当前动作 | 关联接口 | 状态 | +| --- | --- | --- | --- | --- | +| 导入粗略人设工作台 | 选中 `导入粗略人设` 节点时的中间幕布 | 显示新建人设和选择已有角色卡 | `profiles/quick-draft`, `profiles` | 已接 UI | +| 新建粗略人设 | 导入粗略人设工作台左侧右下角 `+` 按钮 | 打开创建角色档案问卷,可选简单/复杂字段 | `profiles/quick-draft` | 已接 UI | +| 选择已有角色卡 | 导入粗略人设工作台右侧 | 选择当前角色卡作为工作流输入 | `profiles/:profileId` | 模拟数据 | +| 当前导入摘要 | 导入粗略人设工作台底部 | 展示当前导入角色卡、风格、素材和状态 | `profiles/:profileId` | 模拟数据 | +| 节点草稿池 | 图片类节点下方 | 按当前后端节点图片显示候选图,缺数据时回退演示图 | `nodes/:nodeId/images` | 已接 UI / 后端数据 | +| 多选草稿 | 节点草稿池 | 多选后同步到主预览,并把当前采用图写回后端节点 | `nodes/:nodeId/select-image` | 已接 UI / 后端落库 | +| 平铺对比 | 主预览右上角 | 打开多图对比浮层 | `local/workflow-compare` | 已接 UI | +| 工作流图片详情 | 主预览图片点击 | 打开图片详情卡,可缩放拖拽 | `images/:imageId` | 已接 UI | +| 节点详情 | 右侧节点详情卡 | 展示后端节点阶段、状态、队列任务和产物摘要,缺字段时使用 UI 摘要 | `nodes/:nodeId` | 已接 UI / 后端数据 | + +### 工作流操作按钮 + +| 入口名称 | 位置 | 当前动作 | 关联接口 | 状态 | +| --- | --- | --- | --- | --- | +| 创建角色档案 | 前期人设准备 / 导入粗略人设 | 打开创建角色档案问卷,简单字段可直接生成,复杂字段作为可选结构传后端并自动选中角色卡 | `profiles/quick-draft/save` | 已接 UI / 后端落库 | +| 导入外部角色卡 | 前期人设准备 / 导入粗略人设 | 打开外部角色卡导入浮层,识别文本后填入创建角色档案问卷 | `local/profile-import-parser`, `profiles/quick-draft` | 已接 UI | +| 导入当前角色卡 | 前期人设准备 / 导入粗略人设 | 选中已落库角色卡作为工作流输入,自动加载或创建角色工作流 | `profiles/:profileId`, `profiles/:profileId/workflows` | 已接 UI / 后端落库 | +| 进入设定拆解 | 前期人设准备 / 导入粗略人设 | 使用当前已落库角色卡调用后端工作流执行,成功后切到 LLM 拆解节点 | `workflows/:workflowId/nodes`, `workflows/:workflowId/execute-character` | 已接 UI / 后端已接 | +| LLM 拆解设定工作台 | 前期人设准备 / LLM 拆解设定 | 展示身份性格、外观锚点、服装素材、约束负面项四组结构化字段,并显示字段完成度 | `workflows/:workflowId/nodes` | 已接 UI / 后端数据 | +| 重新拆解设定 | 前期人设准备 / LLM 拆解设定 | 用当前角色档案重新生成结构化字段,后端连通时写回当前节点 | `nodes/:nodeId/generate` | 已接 UI / 后端预留 | +| 确认字段 | 前期人设准备 / LLM 拆解设定 | 确认身份、外观、服装、配饰和禁用项,成功后进入提示词规划 | `nodes/:nodeId/complete` | 已接 UI / 后端已接 | +| 节点对话注入 | 角色工作流 / 底部对话框 | 自动识别当前节点、角色档案、素材和拆解字段,打包用户原文并标记后端 LLM 翻译为英文提示词后注入节点 | `nodes/:nodeId/generate` | 已接 UI / 后端注入预留 | +| 生成提示词矩阵 | 前期人设准备 / 提示词规划 | 后续生成头像、半身、全身和设定图提示词 | `nodes/:nodeId/generate` | 待后端接入 | +| 校验负面词 | 前期人设准备 / 提示词规划 | 后续检查负面词、权重和冲突项 | `review/policy/validate` | 待后端接入 | +| 绑定参考素材 | 前期人设准备 / 参考素材绑定 | 后续把素材写入节点参数 | `materials`, `nodes/:nodeId` | 待后端接入 | +| 打开素材库 | 前期人设准备 / 参考素材绑定 | 跳转素材模块补充素材 | `materials` | 已接 UI | +| 生成头像草稿 | 头像草稿阶段 | 按当前后端节点生成头像和半身候选,结果写入节点图片 | `nodes/:nodeId/generate`, `nodes/:nodeId/images` | 已接 UI / 后端落库 | +| 对比候选 | 头像草稿阶段 | 打开已选草稿平铺对比 | `local/workflow-compare` | 已接 UI | +| 再跑一轮 | 头像草稿阶段 | 将当前后端节点标记重试并再次生成候选 | `nodes/:nodeId/retry`, `nodes/:nodeId/generate` | 已接 UI / 后端落库 | +| 生成设定图 | 全身与设定图阶段 | 生成全身、手部、配饰或拆分设定图并写入节点图片 | `nodes/:nodeId/generate`, `nodes/:nodeId/images` | 已接 UI / 后端落库 | +| 细节补图 | 全身与设定图阶段 | 后续针对手部、鞋袜、配饰等局部补图 | `nodes/:nodeId/generate` | 待后端接入 | +| 重跑当前设定 | 全身与设定图阶段 | 后续保留人设锁定项重跑当前节点 | `nodes/:nodeId/retry` | 待后端接入 | +| 审核选图 | 审核修复并入库阶段 | 后续标记采用、备选或废弃版本 | `nodes/:nodeId/review` | 待后端接入 | +| 高清修复 | 审核修复并入库阶段 | 对当前后端节点发起高清修复生成,结果写入节点图片 | `nodes/:nodeId/generate`, `nodes/:nodeId/images` | 已接 UI / 后端落库 | +| 保存快照 | 各工作流阶段 | 保存当前流程快照、节点参数和选中角色卡摘要到 `workflow_history` | `POST workflows/:workflowId/history` | 已接 UI / 后端已接 | +| 从此节点创建分支 | 选中已完成/已采用节点时的右侧操作区 | 已完成节点锁定后,点击会以当前节点为切点复制工作流分支并切换到新分支;切点之前保持锁定,切点及之后重置为草稿 | `POST workflows/:workflowId/branches`, `POST workflows/:workflowId/switch-branch` | 已接 UI / 后端已接 | +| 归档图库 | 审核修复并入库阶段 | 后续把最终图放入角色图库 | `images/:imageId/archive` | 待后端接入 | + +## 当前已定义 API endpoint + +源码位置:`frontend/shared/api/endpoints.ts` + +### UI 与概览 + +- `ui/bootstrap` +- `ui/actions` +- `dashboard/summary` + +### 角色与快速角色 + +- `profiles` +- `profiles/:profileId` +- `profiles/:profileId/archive` +- `profiles/quick-draft` +- `profiles/quick-draft/save` +- `profiles/:profileId/workflows` + +### 工作流 + +- `workflows/:workflowId` +- `workflows/:workflowId/start` +- `workflows/:workflowId/cancel` +- `workflows/:workflowId/archive` +- `workflows/:workflowId/execute-character` +- `workflows/:workflowId/nodes` +- `workflows/:workflowId/branches` +- `workflows/:workflowId/switch-branch` +- `workflows/:workflowId/history` +- `workflows/:workflowId/review` + +### 节点 + +- `nodes/:nodeId` +- `nodes/:nodeId/generate` +- `nodes/:nodeId/retry` +- `nodes/:nodeId/complete` +- `nodes/:nodeId/mark-completed` +- `nodes/:nodeId/select-image` +- `nodes/:nodeId/adopt-image` +- `nodes/:nodeId/review` +- `nodes/:nodeId/images` + +### 图片 + +- `images/:imageId` +- `images/:imageId/select` +- `images/:imageId/unselect` +- `images/:imageId/archive` + +### 队列 + +- `jobs` +- `jobs/:taskId` +- `jobs/:taskId/cancel` +- `jobs/:taskId/retry` +- `jobs/dead-letter/retry` +- `jobs/dead-letter/move-failed` + +### 设置与存储 + +- `settings/schema` +- `settings/current` +- `settings/validate` +- `settings/diff` +- `settings/save` +- `settings/reload` +- `settings/export` +- `settings/import` +- `settings/snapshots` +- `settings/snapshots/:snapshotId` +- `settings/rollback` +- `storage/status` +- `storage/migrate` + +### 素材库 + +- `material-categories` +- `material-categories/:categoryId` +- `materials` +- `materials/upload` +- `materials/:materialId` +- `materials/:materialId/recognize` +- `materials/:materialId/re-recognize` +- `materials/:materialId/confirm` +- `assets/import-pack` + +### 衣橱与穿搭 + +- `profiles/:profileId/wardrobe` +- `profiles/:profileId/wardrobe/import-material` +- `profiles/:profileId/wardrobe/items` +- `profiles/:profileId/wardrobe/items/:wardrobeItemId` +- `profiles/:profileId/wardrobe/:wardrobeItemId` +- `profiles/:profileId/wardrobe/match` +- `integrations/private-companion/daily-outfit` + +### 模型能力档案与安全元数据检查 + +`probe-provider` 和 `probe-model` 当前只登记/检查元数据,不会主动向填写的地址发起网络请求;真实 Provider 可用性由配置后的生成调用验证。 + +- `api-discovery/dialects` +- `api-discovery/probe-provider` +- `api-discovery/probe-model` +- `api-discovery/capabilities` +- `api-discovery/manual-override` +- `api-discovery/validate-render-plan` + +### 日志、事件与安全 + +- `logs/generation` +- `logs/workflows` +- `logs/errors` +- `logs/integrations` +- `integrations/logs` +- `security/audit-logs` +- `events/stream` +- `web/manifest` + +## 浮层与弹出卡片 + +| 浮层名称 | 触发入口 | 当前内容 | 关闭方式 | 状态 | +| --- | --- | --- | --- | --- | +| 消息中心 | 顶部铃铛 | 通知列表 | 外部点击 / 再点按钮 / ESC | 已接 UI | +| 日志浮层 | 顶部日志图标 | 生成日志、错误日志 | 外部点击 / ESC | 已接 UI | +| 调色盘 | 顶部调色盘 | 主题色选择 | 外部点击 / ESC | 已接 UI | +| 图片详情卡 | 图片生成/工作流/快速角色图片 | 大图缩放、双击、拖拽 | 外部点击 / 关闭按钮 / ESC | 已接 UI | +| 角色设定详情卡 | 快速角色折角 | 当前角色设定详情 | 外部点击 / 关闭按钮 / ESC | 已接 UI | +| 创建角色档案问卷 | 快速角色生成人设/添加档案/工作流新建人设 | 简单/复杂问卷切换;复杂问卷包含世界观、背景、身高身材、喜好、关系和生成偏好,字段可留空;保存时调用后端生成角色档案与 `quick_profile` 工作流 | 外部点击 / 关闭按钮 / ESC | 已接 UI / 后端落库 | +| 外部角色卡导入 | 工作流导入粗略人设 / 粘贴导入 | 粘贴外部角色卡、人设文本或设定片段,并识别填入创建角色档案问卷 | 外部点击 / 取消 / 关闭按钮 / ESC | 已接 UI | +| 素材选择卡 | 快速角色参考素材 | 不使用、素材候选、添加素材 | 外部点击 / ESC | 已接 UI | +| 工作流多图对比 | 角色工作流平铺对比 | 多草稿大图对比 | 关闭按钮 / ESC | 已接 UI | + +## 待补入口记录 + +新增功能时先写这里,再实现 UI 或接口。 + +| 待补入口 | 预期位置 | 预期接口 | 备注 | +| --- | --- | --- | --- | +| 素材库添加对应分类素材 | 素材选择浮层底部添加卡 | `materials/upload` | 需要带分类参数,如构图、服装、配饰子类 | +| 工作流草稿轮次自动创建 | 角色工作流节点流程 | `workflows/:workflowId/nodes` | 不满意重跑时自动创建草稿 02/03 等小节点 | +| 节点锁定元素写入 | 工作流多选草稿/节点详情 | `nodes/:nodeId/select-image` | 选中五官、发色、服装等锁定项 | +| 角色编辑界面 | 角色设定详情卡或角色模块 | `profiles/:profileId` | 当前只有详情卡和问卷占位 | +| 性能详情页 | 左侧性能入口 | 待定 | 当前只有入口和概览精简卡 | diff --git a/docs/ui-progress-2026-07-07.md b/docs/ui-progress-2026-07-07.md new file mode 100644 index 0000000..ec43d29 --- /dev/null +++ b/docs/ui-progress-2026-07-07.md @@ -0,0 +1,97 @@ +# ImageStudio UI 开发进度交接 + +日期:2026-07-07 +分支:`codex/ui-navigation-s130-s145` +口令:星图导航-S130 + +> 历史记录说明:本文保留当日 UI 开发状态。当前 AstrBot 生产入口以自动发现的 `pages/imagestudio/index.html` 为准;API 在 `initialize()` 中通过 `context.register_web_api` 注册,bridge 传相对 endpoint。独立 `web/` 或 Vite 路径仅用于开发预览。 + +## 已完成进度 + +### 导航与概览页 + +- 左侧星图导航栏已完成当前视觉稿方向:贴边凸出、柔和圆角、滚动容纳、多图标显示、自动隐藏与鼠标悬停展开。 +- 顶部工具区已统一为图标按钮:主题切换、消息中心、日志、调色盘等,并补充了展开/关闭动画。 +- 消息中心、日志、图片预览等浮层已支持点击外部区域关闭,后续新增浮层可复用同一套关闭逻辑。 +- 日间/夜间主题做过统一调整,修复过夜间中间区域发白、日间卡片色调不一致的问题。 +- 概览页重新整理为性能监控、最近图片、当前队列、运行/新建生成等区域,并调整了最近图片卡片比例和底部对齐。 +- 性能监控已精简主卡片展示,并预留性能详情页入口。 + +### 生成页面总入口 + +- 生成页面已建立三段式入口:`图片生成`、`快速角色`、`角色工作流`。 +- 三个入口按钮可切换下方内容,左侧介绍会随当前模块变化。 +- 生成页面整体视觉风格已经与导航页统一。 + +### 图片生成 + +- 已搭建图片生成工作台:左侧图片队列、中间主图预览、右侧图片信息、底部对话框。 +- 底部对话框已改为类似 Codex 输入框的样式,并加入模型选择、最近请求、输入说明。 +- 已生成 10 张测试占位图,模拟已生成结果,并填充不同标题、提示词、模型、尺寸、种子、耗时、来源和状态。 +- 图片队列不会再拉长整个工作区,工作区高度会跟随浏览器自适应。 +- 主图支持完整显示,不再被工作区裁剪。 +- 图片可点击打开大卡片预览,支持滚轮缩放、双击放大/还原、拖拽查看细节。 + +### 快速角色 + +- 已完成快速角色页面的主体卡片布局,并按当前认可比例固定外部排布。 +- 角色档案改为展示型预览卡,不在此处直接编辑;右上角折角效果作为进入编辑界面的入口占位。 +- 角色图片区域支持四张图自动轮播,也支持点击右侧小卡手动切换。 +- 角色大图点击后可打开详情卡片:左侧大图,右侧展示介绍、提示词、生成时间等信息。 +- 半身照上下留白已用卡片元素进行视觉处理。 +- 测试图片已按实际比例重新调整,并通过缩放适配容器。 +- 参考素材已支持点击弹出选择卡片。 +- 参考素材排序已调整为:风格、构图、服装、鞋袜、配饰。 +- 每类参考素材弹窗包含“不使用”、素材候选、添加素材占位卡。 +- “不使用参考”已清空对应素材图片状态。 +- 配饰弹窗已支持细分类展示,细分类切换有平滑缩放过渡;当前细分类保持单列展示。 +- 配饰分类后续需要对接素材库真实分类文件。 +- 功能按钮中“生成草稿”已改为“生成人设”,点击后弹出问卷大卡片占位。 +- “进入工作流”语义已调整为将粗略人设转入角色工作流继续细化。 + +### 角色工作流 + +- 已建立角色工作流 UI 大卡片:节点流程、图片预览、节点详情、对话框和动作按钮。 +- 对话框样式参考图片生成输入框。 +- 节点流程已细化为大节点 + 小节点结构。 +- 当前模拟大节点包括: + - 前期人设准备 + - 头像草稿 + - 全身与设定图 + - 审核修复并入库 +- 小节点已模拟草稿轮次和细节补图流程,例如头像草稿 01/02/03、全身草稿、手部补图、配饰补图、拆分设定图等。 +- `前期人设准备` 已由原 01/02 合并。 +- 节点详情已补充轮次、锁定项、本轮结论等模拟字段。 +- 草稿阶段已支持一次选择多张图,选中的多张图会一起显示在主预览区域。 +- 已增加平铺对比入口,用于大图对比多张草稿。 +- 已实现节点草稿池联动:切换不同节点时,底部草稿池、主预览和已选草稿会按当前节点的 `imageIds` 变化。 + +### 构建与产物 + +- `frontend/shared/api/endpoints.ts` 已补充 `quickProfileDraftSave` 占位接口。 +- 已运行 `npm.cmd run typecheck`。 +- 已运行 `npm.cmd run build:web`,并更新 `web/` 静态产物。 + +## 未完成进度 + +- 图片生成尚未接入真实生图 API、真实任务提交、真实模型配置和真实生成结果回写。 +- 最近请求目前仍是前端模拟数据,需要接入真实请求历史。 +- 图片队列、图片信息、图片详情卡仍以模拟数据为主,需要接后端图片记录。 +- 快速角色的“生成人设”问卷目前是大卡片占位,字段、校验、LLM 生成逻辑和保存逻辑待细化。 +- 角色档案右上角折角入口目前只是 UI 入口,需要真正跳转或打开角色编辑界面。 +- 参考素材弹窗目前使用占位素材;后续素材库开发时,需要读取真实素材库文件和配饰细分类文件。 +- 添加素材占位卡已经预留入口,但素材库新增素材页面和跳转参数尚未实现。 +- 衣橱匹配、每日穿搭、保存角色参考素材等功能语义仍需最终确认后再落地。 +- 角色工作流目前是模拟流程,尚未接入真实工作流状态机、节点执行、队列、失败重试和持久化。 +- 草稿轮次自动创建小节点的真实逻辑尚未实现;当前只是模拟了多轮草稿数据。 +- 多图对比、节点选图、锁定元素、进入下一轮草稿等操作还没有写入真实工作流数据。 +- 性能详情页入口已预留,但详情页数据与实际性能采集仍需后端对接。 +- 当前 UI 主要通过类型检查和构建验证,浏览器自动化读取页面时曾出现超时,后续需要补一次完整可视化回归。 + +## 下次建议优先级 + +1. 先把角色工作流的草稿轮次机制设计成真实数据结构:一次不满意生成新草稿小节点,选定元素后进入下一轮。 +2. 接入素材库真实文件读取,尤其是配饰分类文件,并让快速角色的参考素材弹窗读取真实分类。 +3. 细化“生成人设”问卷卡片,明确字段、LLM 提示词、保存接口和转入工作流的数据格式。 +4. 将图片生成从模拟数据切到真实模型配置、真实任务提交和真实结果展示。 +5. 做一次完整浏览器视觉回归,确认导航、概览、图片生成、快速角色、角色工作流在常见窗口尺寸下没有错位。 diff --git a/docs/ui-progress-2026-07-09.md b/docs/ui-progress-2026-07-09.md new file mode 100644 index 0000000..1dd95f2 --- /dev/null +++ b/docs/ui-progress-2026-07-09.md @@ -0,0 +1,274 @@ +# ImageStudio UI / Workflow Progress Handoff + +日期:2026-07-09 +范围:生成页、快速角色、角色工作流、阶段审查、锁定与分支、后端接口闭环 + +> 历史记录说明:本文保留当日开发与验证结果。当前 AstrBot 生产页面由 `pages/imagestudio/index.html` 自动发现;bridge 调用传相对 endpoint,兼容 v4.25 legacy `/api/plug/astrbot_plugin_imagestudio/` 与当前 `/api/v1/plugins/extensions/astrbot_plugin_imagestudio/`。下文 `/web` URL 是 Vite 预览记录,不表示 AstrBot 注册了该生产入口。 + +## 当前总体状态 + +- 生成页已经形成三条主入口:`图片生成`、`快速角色`、`角色工作流`。 +- 三条入口都已经有前端工作台,并已接入或预留对应后端接口: + - 图片生成:走单图工作流与 `nodes/:nodeId/generate`。 + - 快速角色:走 `profiles/quick-draft/save` 保存档案并创建工作流关联。 + - 角色工作流:走 `profiles/:profileId/workflows`、`workflows/:workflowId/nodes` 和节点级生成/审查/历史接口。 +- 当前重点已经从纯 UI 进入“角色工作流闭环”:档案导入、设定拆解、提示词规划、素材绑定、头像/半身/设定图、审查、高清修复、入库。 + +## 已完成内容 + +### 快速角色与角色档案 + +- 创建角色档案问卷已支持简单/复杂模式。 +- 复杂问卷字段包含世界观、背景、身高/身材、身体特征、喜好、关系、故事钩子、生成风格、镜头偏好等。 +- 问卷字段可留空,不要求全部填写,避免破坏创建体验。 +- 生成档案按钮语义已从“生成预览”改为“生成档案”。 +- 生成档案后会在右侧角色卡区域生成/选中新角色卡。 +- 外部角色卡导入已提供弹窗入口,可粘贴外部人设文本并回填问卷字段。 +- 快速角色参考素材选择沿用分类素材卡,默认生产语义应为“不使用参考”,避免无意引入参考图。 + +### 前期人设准备 + +- `前期人设准备` 页面布局已调整:导入入口放到侧面,上方主卡片下移,侧边空间更有效。 +- 该阶段包含: + - 导入粗略人设 + - LLM 设定拆解 + - 提示词规划 + - 参考素材绑定 +- LLM 设定拆解卡片按组展示身份性格、外观锚点、服装素材、约束负面项。 +- 大卡片尺寸保持稳定,内部小项目按顺序排列,内容超出时滚动。 +- 底部对话框会根据当前节点注入上下文,包括角色档案、节点信息、素材、拆解字段和用户原文,并标记后端 LLM 翻译为英文提示词。 + +### 提示词规划 + +- 提示词规划页面已搭建矩阵化展示。 +- 支持头像、半身、全身、设定图等目标节点的提示词条目。 +- 已加入继承字段、权重依据、负面词和冲突项校验区域。 +- 修复过卡片重叠、内容被遮挡、胶囊超出卡片下沿等布局问题。 +- 提示词规划 LLM 输出结构已补齐后端 schema,版本为 `2026-07-09.prompt-plan-schema.v1`。 +- 后端提示词规划 schema 包含: + - `matrix`:头像、半身、全身、设定图等目标提示词矩阵。 + - `weight_suggestions`:权重建议、数值和依据。 + - `inheritance_fields` / `inheritance_changes`:继承字段与变更建议。 + - `guardrails`:负面提示词、语言策略和检查项。 + - `conflicts`:按目标节点归属的冲突项。 + - `by_target` / `by_node_key` / `by_node_id`:按目标、后端节点类型和真实节点索引。 +- 保存 `prompt_plan` 节点 metadata 或 workflow metadata 时,后端会自动规范化为正式 prompt plan schema。 +- 节点依赖上下文会按当前目标节点过滤提示词规划,例如头像节点只拿头像 prompt patch,全身节点只拿全身 prompt patch。 +- `PromptExpansionService` 会从 dependency context 自动继承 prompt plan 的 positive/negative patch,并注入生成链路与 `node_injection.prompt_plan`。 + +### 参考素材绑定 + +- 参考素材绑定页面已建立。 +- 绑定节点已翻译为中文,并改为直接绑定大节点,不再绑定后期生成才会出现的小节点。 +- 已按“快速角色参考素材”的交互方式提供素材选择。 +- 参考素材选择弹窗已改为悬浮在上方,而不是向下撑开布局。 +- 默认生产语义应为“不使用参考”,只有用户显式选择后才注入素材。 +- 素材绑定规则已接入后端 metadata 写入与 workflow 级规范化落库。 +- 后端素材绑定 schema 版本为 `2026-07-09.material-binding-schema.v1`。 +- 保存 `material_bind` 节点时,后端会自动生成: + - `rules`:完整规则,包含未选择素材的“不使用参考”分类。 + - `active_rules`:用户显式选择素材后才启用的规则。 + - `by_stage`:按前端大阶段索引。 + - `by_node_key`:按后端节点类型索引。 + - `by_node_id` / `node_parameters`:按真实 workflow node 索引。 +- 后端默认参考策略为 `explicit_only`,`default_use_reference=false`,空素材分类不会进入生成参考。 +- 节点依赖上下文会按当前目标节点过滤素材,例如头像只拿头像阶段绑定,半身只拿半身阶段绑定。 +- `PromptExpansionService` 会从 dependency context 自动继承规范化素材绑定,供后端 LLM 和生成链路注入。 + +### 角色工作流大节点 + +- 前端大节点已对齐后端设计,当前角色工作流大节点包括: + - 前期人设准备 + - 头像 + - 半身像 + - 全身与设定图 + - 高清修复并入库 +- 已补齐半身像节点。 +- 前期人设准备暂时不再调整,后续重点转入生成链路与审查链路。 + +### 头像 / 半身 / 设定图生成逻辑 + +- 头像节点: + - 第一个节点按钮为“生成头像”。 + - 后续轮次按钮为“再跑一轮”。 + - 原“再跑一轮”语义下的确认按钮已改为“确认使用”。 +- 半身像和全身/设定图已按同样逻辑整理。 +- 选中候选后可进入对应阶段审查节点。 +- 下方对话框已按当前节点分别注入节点上下文。 + +### 阶段审查节点 + +- 已加入 UI 层阶段审查节点: + - `portrait-review`:头像审查 + - `halfbody-review`:半身像审查 + - `body-sheet-review`:设定图审查 +- 阶段审查映射到真实后端节点: + - 头像审查映射到 `headshot` + - 半身像审查映射到 `half_body` + - 设定图审查映射到 `expression_sheet` / 设定图相关节点 +- 审查阶段按钮已调整为: + - `选中的图片重新生成` + - `对比候选` + - `确认` +- `选中的图片重新生成` 用于同提示词不同模型复跑,解决某个模型偶尔异常的问题。 +- `确认` 用于通过审查并推进到下一阶段。 + +### 审查确认闭环 + +- 审查阶段点击 `确认` 已接入真实闭环: + - 调用 `nodes/:nodeId/adopt-image`。 + - 将选中图片写入后端采用历史。 + - 将上一真实生成节点状态收敛为 `selected`。 + - 上一节点进入锁定状态,后续修改需要从该点创建分支。 + - 推进 `workflow.current_node_id` 到下一真实节点。 + - 在 workflow metadata 的 `stage_inputs` 中记录阶段输入交接。 +- 设定图审查确认后,选中的成品稿会放入 `高清修复` 输入桶。 +- 前端维护 `workflowInputImagesByNodeId`,用于展示“下一节点输入图”,不伪造图片真实归属。 +- 如果下一节点已经锁定,前端不会再尝试修改下一节点 metadata,只记录 workflow 级交接信息。 +- 后端 `adopt_node_image` 已消费前端传入的 `next_backend_node_id`、`handoff_bucket`、`handoff_role` 等 metadata。 +- 后端会将阶段交接正式写入 `workflow.metadata.stage_inputs`,schema 版本为 `2026-07-09.stage-input-bucket.v1`。 +- 后端会同步推进 `workflow.current_node_id` 到下一真实节点,并保留最近阶段审查交接日志 `stage_review_handoffs`。 +- 前端已补齐 `workflowReviewAdopt` 操作: + - 头像/半身/设定图阶段显示 `确认采用`。 + - 高清修复阶段显示 `确认修复结果`。 + - 采用前要求选中的图片是真实后端图片,且必须属于当前后端节点。 + - `completed` 节点允许本地切换候选图并执行一次确认采用,但不允许重新生成、重跑或修改提示词。 + - `selected` 节点视为最终锁定,只显示创建分支和保存快照。 + - 头像/半身采用后优先按后端 `sort_order` 推进到下一真实节点。 + - 设定图采用后固定交接到 `高清修复与统一风格`。 + - 高清修复采用后交接到 `入库`。 + - 采用成功后前端会刷新 workflow、nodes 和 node images,并跳转到下一 UI 节点。 + +### 后端注入与审查结构 + +- 已新增 `backend/services/node_injection_template_service.py`,统一定义每个节点的对话注入契约。 +- 注入契约包含用户原文、节点上下文、上游选图、素材绑定、阶段输入、语言翻译策略和预期 JSON 返回结构。 +- 已补齐 `stage_review` 与 `candidate_review` 注入模板,支持审查节点下方对话框按节点独立注入。 +- `prompt_plan` 注入模板已补入正式提示词规划 schema、权重建议、继承字段、负面词和冲突返回结构。 +- `material_bind` 注入模板已补入正式素材绑定 schema、默认参考策略和目标后端节点字段。 +- `NodeDependencyContextService` 已返回 `injection_template`,供前端或后续 LLM 编排读取。 +- `PromptExpansionService` 已返回 `node_injection`,生成链路可以把节点注入契约一并传给后端 LLM。 +- `ReviewFlowEngine.review_node` 已返回正式 `stage_review` 结构,schema 版本为 `2026-07-09.stage-review-result.v1`。 +- 阶段审查结果结构包含: + - 同提示词多模型候选 `candidates` + - 候选评分 `score` + - 画面缺陷与警告 `defects` / `warnings` + - 选中候选 `selected_candidate` + - 重生成记录 `regeneration_records` + - 人工确认动作 `manual_confirmation` + - 下一阶段输入桶预期结构 `handoff` + +### 锁定与分支 + +- 后端锁定状态包括 `completed` 和 `selected`。 +- 普通编辑、重跑、选图、归档或删除锁定节点绑定图片会被阻止。 +- 前端在锁定节点显示“从此节点创建分支”入口。 +- 创建分支调用: + - `POST workflows/:workflowId/branches` + - `POST workflows/:workflowId/switch-branch` +- 分支规则: + - 切点之前的节点保留完成状态和选中图。 + - 切点及之后重置为 `draft` 并清空选中图。 + - metadata 记录 `copied_from_node_id`、切点和分支继承信息。 +- 为审查确认补充了后端受控例外: + - 允许 `completed -> selected` 的采用收敛。 + - 已经 `selected` 后切换到另一张图仍会拒绝,继续要求新建分支。 + +## 后端接口状态 + +### 已接入或已使用 + +- `profiles` +- `profiles/quick-draft` +- `profiles/quick-draft/save` +- `profiles/:profileId/workflows` +- `workflows/:workflowId` +- `workflows/:workflowId/nodes` +- `workflows/:workflowId/execute-character` +- `workflows/:workflowId/branches` +- `workflows/:workflowId/switch-branch` +- `workflows/:workflowId/history` +- `workflows/:workflowId/review` +- `nodes/:nodeId` +- `nodes/:nodeId/generate` +- `nodes/:nodeId/retry` +- `nodes/:nodeId/complete` +- `nodes/:nodeId/select-image` +- `nodes/:nodeId/adopt-image` +- `nodes/:nodeId/review` +- `nodes/:nodeId/images` +- `images/:imageId` + +### 已预留但仍需后端深化 + +- 最终角色版本入库 schema。 +- 角色图库与素材库联动。 + +## 关键代码位置 + +- 前端主页面:`frontend/standalone-web/src/App.vue` +- 前端接口定义:`frontend/shared/api/endpoints.ts` +- 后端 workflow 服务:`backend/services/workflow_service.py` +- 后端 history/adopt 服务:`backend/services/history_service.py` +- 后端提示词规划 schema:`backend/services/prompt_plan_schema_service.py` +- 后端素材绑定 schema:`backend/services/material_binding_schema_service.py` +- 后端节点注入模板:`backend/services/node_injection_template_service.py` +- 后端节点依赖上下文:`backend/services/node_dependency_context_service.py` +- 后端审查引擎:`engines/review_flow.py` +- 后端角色工作流服务:`backend/services/character_workflow_service.py` +- 角色工作流节点模板:`engines/character_flow.py` +- 相关测试: + - `tests/test_history_service.py` + - `tests/test_review_service.py` + - `tests/test_prompt_plan_schema_service.py` + - `tests/test_material_binding_schema_service.py` + - `tests/test_prompt_expansion_service.py` + - `tests/test_node_dependency_context_service.py` + - `tests/test_workflow_service.py` + - `tests/test_character_workflow_service.py` + +## 已验证 + +- 前端类型检查: + - `npm.cmd run typecheck` + - 结果:通过。 +- 后端 history 测试: + - 使用 Codex bundled Python 运行 `python -m unittest tests.test_history_service` + - 结果:4 tests OK。 +- 后端审查/历史/节点注入/单图生成相关测试: + - 使用 Codex bundled Python 运行 `python -m unittest tests.test_review_service tests.test_history_service tests.test_prompt_expansion_service tests.test_node_dependency_context_service tests.test_single_flow_generation` + - 结果:28 tests OK。 +- 后端素材绑定 schema / workflow 同步 / 上下文注入测试: + - 使用 Codex bundled Python 运行 `python -m unittest tests.test_material_binding_schema_service tests.test_workflow_service tests.test_node_dependency_context_service tests.test_prompt_expansion_service tests.test_review_service tests.test_history_service` + - 结果:37 tests OK。 +- 后端生成链路补充测试: + - 使用 Codex bundled Python 运行 `python -m unittest tests.test_single_flow_generation tests.test_character_workflow_service` + - 结果:15 tests OK。 +- 后端提示词规划 schema / workflow 同步 / 上下文注入测试: + - 使用 Codex bundled Python 运行 `python -m unittest tests.test_prompt_plan_schema_service tests.test_workflow_service tests.test_node_dependency_context_service tests.test_prompt_expansion_service` + - 结果:26 tests OK。 +- 后端角色工作流核心回归: + - 使用 Codex bundled Python 运行 `python -m unittest tests.test_prompt_plan_schema_service tests.test_material_binding_schema_service tests.test_workflow_service tests.test_node_dependency_context_service tests.test_prompt_expansion_service tests.test_review_service tests.test_history_service tests.test_single_flow_generation tests.test_character_workflow_service` + - 结果:57 tests OK。 +- 审查确认采用前端闭环补充验证: + - `npm.cmd run typecheck` + - 结果:通过。 + - 使用 Codex bundled Python 运行 `python -m unittest tests.test_prompt_plan_schema_service tests.test_material_binding_schema_service tests.test_review_service tests.test_quick_profile_service tests.test_workflow_service tests.test_node_dependency_context_service tests.test_prompt_expansion_service tests.test_history_service` + - 结果:44 tests OK。 +- 本地 Vite dev server(仅历史预览路径,不是 AstrBot 生产入口): + - `http://127.0.0.1:5173/api/v1/plugins/extensions/astrbot_plugin_imagestudio/web/?view=generate` + - HTTP 返回 200。 + +## 当前限制与风险 + +- 浏览器插件在一次 reload 验证时超时重置,因此没有执行页面内真实点击确认验证,避免误改本地工作流数据。 +- `web/` 静态产物当前有旧资产删除和新资产生成,属于之前构建结果,提交前需要统一确认是否纳入。 +- 工作区存在多处既有改动,提交前需要按功能拆分检查,不要把无关改动混进同一版。 +- 阶段审查与高清修复输入桶已有后端结构,但还没有做完整页面点击回归。 +- 最终入库节点仍需补齐角色版本、图片、提示词、素材引用、审查记录和分支信息的正式 schema。 + +## 建议下一步 + +1. 完成最终入库节点:保存角色版本、图片、提示词、素材引用、审查记录和分支信息。 +2. 做一次完整浏览器手动回归:头像确认、半身确认、设定图确认、进入高清修复、锁定后创建分支。 +3. 视提交策略决定是否纳入当前 `web/` 静态构建产物,避免把无关构建文件混入功能提交。 diff --git a/docs/ui-progress-2026-07-10.md b/docs/ui-progress-2026-07-10.md new file mode 100644 index 0000000..8530f07 --- /dev/null +++ b/docs/ui-progress-2026-07-10.md @@ -0,0 +1,116 @@ +# ImageStudio UI / Workflow Progress Handoff + +日期:2026-07-10 +分支:`codex/ui-navigation-s130-s145` +范围:未提交变更检查、功能验证、静态构建、推送前总结。 + +> 历史记录说明:本文保留当日开发与构建状态。当前 AstrBot 生产页面由 `pages/imagestudio/index.html` 自动发现;插件通过 `context.register_web_api` 提供 API,页面 bridge 传相对 endpoint。下文 `/web` URL 是 Vite 预览记录,不表示 AstrBot 注册了该生产入口。 + +## 后续前端重构更新 + +本文件后续章节保留当时的阶段记录,当前前端状态以 [前端拆分与体验优化报告](ui-refactor-2026-07-10.md) 为准。本轮已拆出导航、指标/状态原语、图片预览、素材弹窗、外部角色卡解析和共享工具,并完成桌面与 390×844 移动端浏览器回归。 + +当前构建产物: + +- `pages/imagestudio/assets/index-CF6dVfCE.js` +- `pages/imagestudio/assets/index-D4NekmM3.css` +- `web/assets/index-djgvYVE7.js` +- `web/assets/index-VBJk7aNa.css` + +当前前端定向验证:`typecheck`、Plugin Page/Web 生产构建、18 项前端契约测试及 7 项文档打包测试均通过。 + +## 推送前状态 + +- 已执行 `git fetch origin`。 +- 检查结果:本地分支相对 `origin/codex/ui-navigation-s130-s145` 为 `0 ahead / 0 behind`,也就是没有“已提交但未推送”的 commit。 +- 当前待发布内容来自工作区未提交改动,主要集中在完整工作台 UI、素材库工作台、队列工作台、角色工作流闭环、后端 schema/注入服务和对应测试。 + +## 已完成内容 + +### 前端工作台 + +- `frontend/standalone-web/src/App.vue` 扩展为更完整的 ImageStudio 工作台。 +- 顶部导航现在聚焦:概览、生成、素材、角色、图库、设置、日志。 +- 新增素材库工作台: + - 素材分类筛选。 + - 示例素材卡和后端素材卡统一展示。 + - 素材详情弹层。 + - 添加分类弹层。 + - 刷新素材、上传素材、识别、重识别、确认入库、导入素材包、并入衣橱、进入素材绑定等操作入口。 +- 新增队列工作台: + - 活跃、失败、完成、全部筛选。 + - 任务详情和批量操作入口。 +- 角色工作流继续补齐: + - 提示词规划节点页面。 + - 参考素材绑定节点页面。 + - 阶段审查和候选图采用动作。 + - 锁定节点创建分支入口。 + - 最终归档入口的 UI 语义。 +- 前端 API 客户端补充 `PUT` 请求方法。 + +### 后端闭环 + +- 新增提示词规划 schema 服务: + - `backend/services/prompt_plan_schema_service.py` + - 支持 prompt matrix、target/node 索引、继承字段、权重建议、guardrails 和 conflicts。 +- 新增素材绑定 schema 服务: + - `backend/services/material_binding_schema_service.py` + - 支持 explicit-only 默认策略、rules、active rules、by stage、by node key、by node id 和 node parameters。 +- 新增节点注入模板服务: + - `backend/services/node_injection_template_service.py` + - 统一生成节点对话注入契约,包含用户原文、节点上下文、素材绑定、提示词规划、阶段输入、草稿图选择和预期返回结构。 +- `WorkflowService`: + - 保存 workflow/node metadata 时同步规范化 prompt plan 和 material bindings。 + - 允许 completed 节点在采用图片时受控进入 selected。 +- `NodeDependencyContextService`: + - 返回 prompt plan、material bindings 和 injection template。 + - 按目标节点过滤上游上下文。 +- `PromptExpansionService`: + - 注入 prompt plan patch、negative patch 和 node injection。 +- `MaterialService`: + - 更新分类时校验 parent_id,避免自引用和无效父分类。 +- 审查和历史相关服务继续补齐阶段审查、图片采用、分支锁定和 stage input handoff。 + +### 静态产物 + +- 已重新执行 `npm.cmd run build:web`。 +- `web/index.html` 已指向新构建产物。 +- 新产物: + - `web/assets/index-C91GfaJN.css` + - `web/assets/index-uM7AKRxw.js` +- 旧产物已被构建替换: + - `web/assets/index-B2NiW3rX.js` + - `web/assets/index-CPo7NaFF.css` + +## 已验证 + +- `npm.cmd run typecheck` + - 结果:通过。 +- `python -m unittest tests.test_prompt_plan_schema_service tests.test_material_binding_schema_service tests.test_workflow_service tests.test_node_dependency_context_service tests.test_prompt_expansion_service tests.test_review_service tests.test_history_service tests.test_material_service tests.test_character_workflow_service tests.test_plugin_entrypoint tests.test_s0_infrastructure` + - 使用 Codex bundled Python。 + - 结果:81 tests OK。 +- `npm.cmd run build:web` + - 结果:通过,生成最新 `web/` 静态产物。 +- 内置浏览器打开本地 Vite 预览(不是 AstrBot 生产入口): + - `http://127.0.0.1:5173/api/v1/plugins/extensions/astrbot_plugin_imagestudio/web/?view=materials` + - 结果:素材页标题、预览模式提示、示例素材卡、素材操作卡正常渲染。 + - 控制台:未发现 error/warn。 + - 素材详情弹层:点击示例素材卡后可打开。 + +## 未完成或仍需补齐 + +- 当前浏览器验证仍在前端预览模式下完成,没有连接真实 AstrBot 插件后端,因此未覆盖真实落库、文件上传、素材 VLM 识别和衣橱写入。 +- 添加分类弹层需要再做一次真实浏览器交互验收;本次自动点击使用的选择器不够稳定,未作为通过项记录。 +- 最终入库节点仍需要补齐正式 schema:角色版本、最终图片、提示词、素材引用、审查记录、分支信息和归档元数据。 +- 角色图库与素材库的真实联动仍需继续深化。 +- 素材上传入口、素材包导入入口目前以 UI/接口入口为主,文件选择、dry-run 预览和导入执行流程还需要后续完整实现。 +- 需要在真实后端环境做一次端到端验收:快速角色创建、工作流生成、提示词规划保存、素材绑定保存、阶段审查采用、锁定后分支、最终归档。 + +## 备注 + +- 本次检查开始时没有未推送 commit;推送动作应在提交当前工作区改动后执行。 +- 已创建本地提交:`feat: extend workflow and materials workbench`。 +- 已尝试执行 `git push origin codex/ui-navigation-s130-s145`,但远端 Git HTTP 返回凭据不正确或已过期。 +- Forgejo API 认证仍可用,并确认远端 `codex/ui-navigation-s130-s145` 当前仍停在 `b61c6274e5a7ddbcbad1d67f0f0fcfe7da7f3264`。 +- 因此当前状态为:本地分支 ahead 1,等待更新本机 Git 凭据后再次推送。 +- Windows shell 提示多处 LF/CRLF 警告,未发现影响构建或测试的行为问题。 diff --git a/docs/ui-refactor-2026-07-10.md b/docs/ui-refactor-2026-07-10.md new file mode 100644 index 0000000..a0333bf --- /dev/null +++ b/docs/ui-refactor-2026-07-10.md @@ -0,0 +1,67 @@ +# ImageStudio 前端拆分与体验优化(2026-07-10) + +## 本轮目标 + +在不改变后端 API 契约的前提下,拆分大型 Workbench、合并重复代码,并修复真实数据与预览数据边界、键盘操作、移动端导航和弹窗体验问题。 + +## 已完成拆分 + +- `AppNavigationRail.vue`:导航渲染、活动页语义和固定/自动收起控制。 +- `MetricCardGrid.vue`:素材、队列、首页和性能页统一指标卡渲染。 +- `StateCard.vue`:统一成功、错误、警告状态反馈及 `aria-live`。 +- `EmptyState.vue`:统一空态和可选操作入口。 +- `ImagePreviewDialog.vue`:统一图片预览、0.5–4 倍缩放、双击缩放、拖拽和 Escape/遮罩关闭。 +- `MaterialOperationDialog.vue`:素材 URI、素材包和衣橱操作;补齐焦点管理、焦点循环和移动端布局。 +- `MaterialCategoryCreateDialog.vue`:素材分类创建表单与焦点管理。 +- `MaterialDetailDialog.vue`:素材详情展示与键盘关闭。 +- `features/quick-profile/profile-import.ts`:外部角色卡解析、默认问卷和空问卷纯函数。 +- `types/navigation.ts`:导航与模块契约。 + +## 已合并的重复代码 + +- `shared/utils/presentation.ts`:状态标准化、状态文案/颜色、时间、容量、成本、任务等待时间和短 ID。 +- `shared/utils/storage-uri.ts`:`storage://` 校验、文件名提取和 PNG/JPEG/GIF/WebP 素材 URI 校验。 +- `shared/utils/collections.ts`:`uniqueBy` 与字符串去重。 +- `shared/types/ui.ts`:通用指标卡、详情行和动作卡类型。 +- 图片预览重复模板、素材分类/详情弹层以及四处指标卡模板已合并为组件。 + +## UI/UX 修复 + +- 桌面导航增加固定/自动收起开关;队列和性能入口重新进入主导航。 +- 390px 移动端底部导航显示文字标签,不再只依赖抽象符号和 hover 提示。 +- 全局搜索升级为标准 combobox/listbox:支持 Ctrl/Cmd+K、`/`、上下方向键和 Enter。 +- 生成与工作流输入支持 Ctrl/Cmd+Enter 提交。 +- 原“生成设置”“工作流设置”“附加素材”假按钮已接入真实设置/素材导航。 +- 首页队列“查看详情”“重试”按钮已连接真实队列选择与重试动作,并按权限/状态禁用。 +- 真实后端已连接但未配置模型时不再伪装成示例/本地模型;提交会被禁用并引导到设置。 +- 真实后端空素材库不再自动展示示例素材;预览/错误模式才显示明确标记的样例。 +- 素材 URI 前端校验与后端一致,支持 GIF。 +- 弹窗补齐遮罩关闭、Escape、自动聚焦、busy 防误关、焦点循环、`100dvh` 与 safe-area。 +- 队列 tab、通知筛选和素材分类筛选补齐 `aria-selected`/`aria-pressed`。 +- 状态卡统一使用 `role="alert"` 或 `role="status"`,错误不再挤在模型选择器的小区域中。 + +## 验证 + +- `npm.cmd run typecheck`:通过。 +- `python -m unittest tests.test_frontend_api_contract tests.test_docs_packaging`:25 项通过。 +- 全量 `python -m unittest discover -s tests`:359 项 OK,其中 1 项按环境条件跳过。 +- `npm.cmd run build:plugin`:通过。 +- `npm.cmd run build:web`:通过。 +- 浏览器桌面与 390×844 移动端回归:通过。 + - 导航固定、移动标签、明暗主题正常。 + - Ctrl/Cmd+K 搜索、方向键和 Enter 跳转正常。 + - 队列筛选具备 tab 语义。 + - 图片预览缩放、拖拽和键盘关闭正常。 + - 页面控制台无应用 error/warn。 + +最终构建产物: + +- `pages/imagestudio/assets/index-CF6dVfCE.js` +- `pages/imagestudio/assets/index-D4NekmM3.css` +- `web/assets/index-djgvYVE7.js` +- `web/assets/index-VBJk7aNa.css` + +## 剩余边界 + +- `App.vue` 已拆出组件、纯函数、共享类型和工具,但仍约 18,138 行,主要体积来自生成/角色工作流状态编排和集中样式;后续应继续拆 `useWorkflowStore`、`useWorkflowActions`、通知和异步资源 composable。 +- 浏览器回归运行在 Vite 预览模式,未连接真实 AstrBot 后端;真实数据库写入、素材 Vision 与 Provider 调用仍由后端测试和 AstrBot 环境验收覆盖。 diff --git a/engines/character_flow.py b/engines/character_flow.py index 4b17ba8..c160cc4 100644 --- a/engines/character_flow.py +++ b/engines/character_flow.py @@ -13,54 +13,70 @@ CHARACTER_GRADUAL_STEPS: tuple[dict[str, Any], ...] = ( "description": "Core identity, personality, and visual anchors.", "sort_order": 10, }, + { + "node_key": "prompt_plan", + "title": "Prompt plan", + "step_name": "prompt_plan", + "description": "Prompt matrix for portrait, half body, full body, and design sheet nodes.", + "node_type": "planning", + "sort_order": 20, + }, + { + "node_key": "material_bind", + "title": "Material binding", + "step_name": "material_bind", + "description": "Bind style, composition, clothing, shoes, and accessory references to downstream nodes.", + "node_type": "reference_binding", + "sort_order": 30, + }, { "node_key": "headshot", "title": "Headshot", "step_name": "headshot", "description": "Face, hair, eyes, and expression direction.", - "sort_order": 20, + "sort_order": 40, }, { "node_key": "half_body", "title": "Half body", "step_name": "half_body", "description": "Upper body costume, posture, and silhouette.", - "sort_order": 30, + "sort_order": 50, }, { "node_key": "full_body", "title": "Full body", "step_name": "full_body", "description": "Full outfit, proportions, and full-body design.", - "sort_order": 40, + "sort_order": 60, }, { "node_key": "expression_sheet", "title": "Expression sheet", "step_name": "expression_sheet", "description": "Reusable expression variants.", - "sort_order": 50, + "sort_order": 70, }, { "node_key": "pose_sheet", "title": "Pose sheet", "step_name": "pose_sheet", "description": "Reusable pose and action variants.", - "sort_order": 60, + "sort_order": 80, }, { "node_key": "style_lock", "title": "Style lock", "step_name": "style_lock", "description": "Final style pass and prompt consistency lock.", - "sort_order": 70, + "sort_order": 90, }, { "node_key": "final_pack", "title": "Final pack", "step_name": "final_pack", "description": "Final accepted images and production metadata.", - "sort_order": 80, + "sort_order": 100, }, ) @@ -69,7 +85,7 @@ def gradual_node_templates() -> list[dict[str, Any]]: return [ { **step, - "node_type": "generation", + "node_type": step.get("node_type", "generation"), "status": "draft", "branch_name": "main", "metadata": {"template": "character_gradual_v1"}, diff --git a/engines/image_provider.py b/engines/image_provider.py index 9d2c47d..e91cc1f 100644 --- a/engines/image_provider.py +++ b/engines/image_provider.py @@ -13,9 +13,20 @@ import urllib.request from dataclasses import dataclass, field from typing import Any, Callable, Protocol +from core.remote_fetch import RemoteFetchError, read_http_response_limited, validate_http_url + JsonDict = dict[str, Any] HttpTransport = Callable[[str, JsonDict, dict[str, str], int], JsonDict] +MAX_PROVIDER_IMAGE_BYTES = 25 * 1024 * 1024 +MAX_PROVIDER_IMAGE_COUNT = 8 +MAX_PROVIDER_JSON_BASE_OVERHEAD_BYTES = 1024 * 1024 +MAX_PROVIDER_JSON_PER_IMAGE_OVERHEAD_BYTES = 128 * 1024 +# Decoding JSON temporarily retains the response bytes, decoded text, and +# parsed base64 strings. Keep the aggregate response bounded even when eight +# images are requested; individual images still receive their full 25 MiB cap. +MAX_PROVIDER_JSON_HARD_BYTES = 128 * 1024 * 1024 +MAX_PROVIDER_ERROR_BYTES = 64 * 1024 class ImageProvider(Protocol): @@ -35,6 +46,7 @@ class ImageProviderConfig: provider_type: str = "openai_compatible" timeout_seconds: int = 180 extra_headers: dict[str, str] = field(default_factory=dict) + supports_negative_prompt: bool = True def endpoint_url(self) -> str: base = self.base_url.rstrip("/") @@ -45,6 +57,7 @@ class ImageProviderConfig: @dataclass(frozen=True) class ImageGenerationRequest: prompt: str + negative_prompt: str | None = None model: str | None = None size: str = "1024x1024" count: int = 1 @@ -113,12 +126,15 @@ class OpenAIImageProvider: "prompt": request.prompt, "size": request.size, "n": _bounded_count(request.count), + "response_format": "b64_json", } if request.input_images: payload["image"] = list(request.input_images) + if self.config.supports_negative_prompt and (negative_prompt := _optional_text(request.negative_prompt)): + payload["negative_prompt"] = negative_prompt headers = { - "Authorization": f"Bearer {self.config.api_key}", "Content-Type": "application/json", + **({"Authorization": f"Bearer {self.config.api_key}"} if self.config.api_key else {}), **self.config.extra_headers, } try: @@ -133,7 +149,43 @@ class OpenAIImageProvider: error_type=exc.error_type, details=exc.details, ) from exc - return _parse_image_response(self.provider_id, model, response) + return _parse_image_response( + self.provider_id, + model, + response, + provider_base_url=self.config.base_url, + ) + + +class FallbackImageProvider: + """Try configured image providers in priority order on provider failures.""" + + def __init__(self, providers: list[ImageProvider]): + if not providers: + raise ValueError("at least one image provider is required") + self.providers = providers + self.provider_id = providers[0].provider_id + + def generate_image(self, request: ImageGenerationRequest) -> ImageGenerationResult: + failures: list[dict[str, Any]] = [] + for provider in self.providers: + try: + result = provider.generate_image(request) + return ImageGenerationResult( + provider_id=result.provider_id, + model=result.model, + images=result.images, + usage=result.usage, + raw_metadata={**result.raw_metadata, "fallback_attempts": failures}, + ) + except ImageProviderError as exc: + failures.append({"provider_id": provider.provider_id, "error_type": exc.error_type, "status_code": exc.status_code}) + raise ImageProviderError( + "all configured image providers failed", + provider_id=self.provider_id, + error_type="provider_pool_exhausted", + details={"attempts": failures}, + ) class MockImageProvider: @@ -155,19 +207,33 @@ class MockImageProvider: def _post_json(url: str, payload: JsonDict, headers: dict[str, str], timeout_seconds: int) -> JsonDict: + try: + validate_http_url(url, allow_private=True) + except RemoteFetchError as exc: + raise ImageProviderError( + "image provider endpoint is invalid", + provider_id="unknown", + error_type="configuration_error", + details={"reason": str(exc)}, + ) from exc data = json.dumps(payload, ensure_ascii=False).encode("utf-8") request = urllib.request.Request(url, data=data, headers=headers, method="POST") try: - with urllib.request.urlopen(request, timeout=timeout_seconds) as response: - body = response.read().decode("utf-8-sig") + body = read_http_response_limited( + request, + timeout_seconds=timeout_seconds, + max_bytes=_provider_json_limit(payload.get("n", 1)), + allow_private=True, + same_origin_redirects_only=True, + ).decode("utf-8-sig") except urllib.error.HTTPError as exc: - body = exc.read().decode("utf-8", errors="replace") + body = exc.read(MAX_PROVIDER_ERROR_BYTES + 1).decode("utf-8", errors="replace") raise ImageProviderError( "image provider returned an HTTP error", provider_id="unknown", status_code=exc.code, error_type="http_error", - details={"body_preview": body[:500]}, + details={"body_preview": body[:500], "body_truncated": len(body) > MAX_PROVIDER_ERROR_BYTES}, ) from exc except urllib.error.URLError as exc: raise ImageProviderError( @@ -176,6 +242,13 @@ def _post_json(url: str, payload: JsonDict, headers: dict[str, str], timeout_sec error_type="network_error", details={"reason": str(exc.reason)}, ) from exc + except (RemoteFetchError, UnicodeDecodeError) as exc: + raise ImageProviderError( + "image provider response exceeded safety limits", + provider_id="unknown", + error_type="invalid_response", + details={"reason": str(exc)}, + ) from exc try: parsed = json.loads(body) except json.JSONDecodeError as exc: @@ -190,7 +263,13 @@ def _post_json(url: str, payload: JsonDict, headers: dict[str, str], timeout_sec return parsed -def _parse_image_response(provider_id: str, model: str, response: JsonDict) -> ImageGenerationResult: +def _parse_image_response( + provider_id: str, + model: str, + response: JsonDict, + *, + provider_base_url: str | None = None, +) -> ImageGenerationResult: data = response.get("data", []) if not isinstance(data, list): raise ImageProviderError( @@ -207,6 +286,7 @@ def _parse_image_response(provider_id: str, model: str, response: JsonDict) -> I raw_metadata={ "created": response.get("created"), "response_keys": sorted(str(key) for key in response.keys()), + "provider_base_url": provider_base_url, }, ) @@ -225,7 +305,19 @@ def _bounded_count(value: int) -> int: count = int(value) except (TypeError, ValueError): count = 1 - return min(max(count, 1), 8) + return min(max(count, 1), MAX_PROVIDER_IMAGE_COUNT) + + +def _provider_json_limit(requested_count: Any) -> int: + """Bound a base64 JSON response according to the requested image count.""" + + count = _bounded_count(requested_count) + base64_bytes_per_image = ((MAX_PROVIDER_IMAGE_BYTES + 2) // 3) * 4 + calculated = ( + MAX_PROVIDER_JSON_BASE_OVERHEAD_BYTES + + count * (base64_bytes_per_image + MAX_PROVIDER_JSON_PER_IMAGE_OVERHEAD_BYTES) + ) + return min(calculated, MAX_PROVIDER_JSON_HARD_BYTES) def _optional_text(value: Any) -> str | None: diff --git a/engines/review_flow.py b/engines/review_flow.py index 0f38edc..d3b63ae 100644 --- a/engines/review_flow.py +++ b/engines/review_flow.py @@ -8,6 +8,8 @@ from typing import Any from backend.dto.image_dto import GenerationImage from backend.dto.workflow_dto import ProfileWorkflow, WorkflowNode +STAGE_REVIEW_RESULT_VERSION = "2026-07-09.stage-review-result.v1" + _KNOWN_NODE_STATUSES = { "draft", @@ -22,6 +24,8 @@ _KNOWN_NODE_STATUSES = { "discarded", } +_GENERATION_NODE_TYPES = {"generation", "single_image", "virtual_single_image"} + @dataclass(frozen=True) class ReviewPolicy: @@ -107,13 +111,22 @@ class ReviewFlowEngine: policy, errors = ReviewPolicy.from_mapping(payload) return {"valid": not errors, "policy": policy.to_dict(), "errors": errors} - def review_node(self, node: WorkflowNode, images: list[GenerationImage]) -> dict[str, Any]: + def review_node( + self, + node: WorkflowNode, + images: list[GenerationImage], + workflow_metadata: dict[str, Any] | None = None, + ) -> dict[str, Any]: warnings: list[dict[str, Any]] = [] errors: list[dict[str, Any]] = [] selected_image = next((image for image in images if image.image_id == node.selected_image_id), None) if self.policy.warn_without_images and not images: warnings.append({"code": "NO_IMAGES", "message": "node has no generated images"}) - if node.status in self.policy.require_selected_image_for_statuses and not node.selected_image_id: + if ( + node.node_type in _GENERATION_NODE_TYPES + and node.status in self.policy.require_selected_image_for_statuses + and not node.selected_image_id + ): errors.append({"code": "MISSING_SELECTED_IMAGE", "message": "completed node has no selected image"}) if ( self.policy.fail_on_selected_image_not_found @@ -158,6 +171,7 @@ class ReviewFlowEngine: "node_model_route": node.model_route, "policy_version": self.policy.version, }, + "stage_review": build_stage_review_result(node, images, workflow_metadata or {}), } def review_workflow( @@ -177,6 +191,20 @@ class ReviewFlowEngine: "node_ids": incomplete, } ) + incomplete_generation = [ + node.node_id + for node in nodes + if node.node_type in _GENERATION_NODE_TYPES + and node.status not in self.policy.workflow_terminal_statuses + ] + if incomplete_generation: + errors.append( + { + "code": "INCOMPLETE_GENERATION_NODES", + "message": "workflow has unfinished generation nodes", + "node_ids": incomplete_generation, + } + ) selected_model_routes = sorted( { str(result["summary"].get("selected_image_model_route")) @@ -227,3 +255,197 @@ def _normalize_status_list(value: Any) -> tuple[str, ...]: def _normalize_prompt(value: str) -> str: return " ".join(str(value or "").lower().replace(",", " ").split()) + + +def build_stage_review_result( + node: WorkflowNode, + images: list[GenerationImage], + workflow_metadata: dict[str, Any] | None = None, +) -> dict[str, Any]: + candidates = [_review_candidate(node, image, index + 1) for index, image in enumerate(images)] + selected_candidate = next( + ( + { + "candidate_id": candidate["candidate_id"], + "image_id": candidate["image_id"], + "model_route": candidate["model_route"], + "score": candidate["score"], + "status": candidate["status"], + } + for candidate in candidates + if candidate["image_id"] == node.selected_image_id or candidate["is_selected"] + ), + None, + ) + return { + "schema_version": STAGE_REVIEW_RESULT_VERSION, + "node_id": node.node_id, + "node_key": node.node_key, + "review_mode": "same_prompt_multi_model", + "prompt_contract": { + "positive_prompt": node.prompt, + "negative_prompt": node.negative_prompt, + "model_route": node.model_route, + "locked_after_confirm": True, + }, + "scoring_dimensions": [ + "identity_consistency", + "prompt_consistency", + "model_route_consistency", + "visual_defects", + "archive_state", + ], + "candidates": candidates, + "selected_candidate": selected_candidate, + "regeneration_records": [ + candidate["regeneration"] + for candidate in candidates + if candidate.get("regeneration", {}).get("is_regeneration") + ], + "manual_confirmation": { + "action": "adopt_image", + "endpoint": f"nodes/{node.node_id}/adopt-image", + "requires_image_id": True, + "changes_node_status_to": "selected", + "locks_source_node": True, + "branch_required_after_confirm": True, + }, + "handoff": { + "current_node_input": _stage_input_for_node(workflow_metadata or {}, node.node_id), + "metadata_key": "stage_inputs", + "expected_next_input_shape": { + "image_id": "string", + "from_node_id": node.node_id, + "history_id": "string", + "role": "next_stage_input | hires_fix_input", + }, + }, + } + + +def _review_candidate(node: WorkflowNode, image: GenerationImage, rank: int) -> dict[str, Any]: + metadata = image.metadata if isinstance(image.metadata, dict) else {} + warnings = _candidate_warnings(node, image) + defects = _string_list(metadata.get("review_defects") or metadata.get("defects")) + score_value, score_basis = _candidate_score(metadata, warnings, defects, image.is_archived) + status = "selected" if image.is_selected or image.image_id == node.selected_image_id else "candidate" + if image.is_archived: + status = "archived" + return { + "candidate_id": f"{node.node_id}:{image.image_id}", + "rank": rank, + "image_id": image.image_id, + "image_uri": image.image_uri, + "thumbnail_uri": image.thumbnail_uri, + "model_route": image.model_route, + "model_used": image.model_used, + "seed": image.seed, + "width": image.width, + "height": image.height, + "positive_prompt": image.positive_prompt, + "negative_prompt": image.negative_prompt, + "is_selected": image.is_selected or image.image_id == node.selected_image_id, + "is_archived": image.is_archived, + "status": status, + "warnings": warnings, + "defects": defects, + "score": { + "value": score_value, + "basis": score_basis, + "warnings_count": len(warnings), + "defects_count": len(defects), + }, + "regeneration": _regeneration_record(node, image, metadata), + } + + +def _candidate_warnings(node: WorkflowNode, image: GenerationImage) -> list[dict[str, Any]]: + warnings: list[dict[str, Any]] = [] + if image.is_archived: + warnings.append({"code": "ARCHIVED", "message": "candidate image is archived"}) + if node.model_route and image.model_route and node.model_route != image.model_route: + warnings.append( + { + "code": "MODEL_ROUTE_MISMATCH", + "message": "candidate model_route differs from node model_route", + "node_model_route": node.model_route, + "image_model_route": image.model_route, + } + ) + if node.prompt and image.positive_prompt: + node_prompt = _normalize_prompt(node.prompt) + image_prompt = _normalize_prompt(image.positive_prompt) + if node_prompt and image_prompt and node_prompt not in image_prompt and image_prompt not in node_prompt: + warnings.append({"code": "PROMPT_MISMATCH", "message": "candidate prompt differs from node prompt"}) + return warnings + + +def _candidate_score( + metadata: dict[str, Any], + warnings: list[dict[str, Any]], + defects: list[str], + is_archived: bool, +) -> tuple[int, str]: + explicit = metadata.get("review_score") + if isinstance(explicit, (int, float)): + return max(0, min(100, int(explicit))), "metadata.review_score" + score = 100 - len(warnings) * 10 - len(defects) * 8 + if is_archived: + score -= 40 + return max(0, min(100, score)), "heuristic" + + +def _regeneration_record(node: WorkflowNode, image: GenerationImage, metadata: dict[str, Any]) -> dict[str, Any]: + source_image_id = _optional_text( + metadata.get("regenerated_from_image_id") + or metadata.get("source_image_id") + or metadata.get("review_source_image_id") + ) + is_regeneration = bool( + source_image_id + or metadata.get("same_prompt_regeneration") + or metadata.get("review_regeneration") + or metadata.get("regeneration_attempt") + ) + return { + "is_regeneration": is_regeneration, + "image_id": image.image_id, + "source_image_id": source_image_id, + "node_id": node.node_id, + "same_prompt": bool(metadata.get("same_prompt_regeneration") or metadata.get("review_regeneration")), + "attempt": _optional_int(metadata.get("regeneration_attempt") or metadata.get("review_attempt")), + "model_route": image.model_route, + "created_at": image.created_at, + } + + +def _stage_input_for_node(workflow_metadata: dict[str, Any], node_id: str) -> dict[str, Any]: + stage_inputs = workflow_metadata.get("stage_inputs") + if not isinstance(stage_inputs, dict): + return {} + value = stage_inputs.get(node_id) + return value if isinstance(value, dict) else {} + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return [value] if value.strip() else [] + if not isinstance(value, list): + return [] + return [str(item).strip() for item in value if str(item or "").strip()] + + +def _optional_text(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + + +def _optional_int(value: Any) -> int | None: + if value is None: + return None + try: + return int(value) + except (TypeError, ValueError): + return None diff --git a/engines/single_flow.py b/engines/single_flow.py index 7061244..13244f6 100644 --- a/engines/single_flow.py +++ b/engines/single_flow.py @@ -1,10 +1,10 @@ -"""Single image workflow mock engine.""" +"""Single-image workflow engine with an explicit development mock mode.""" from __future__ import annotations import base64 +import binascii import re -import urllib.request from dataclasses import dataclass from pathlib import Path from typing import Any, Callable @@ -12,8 +12,15 @@ from typing import Any, Callable from backend.dto.job_dto import GenerationTask from backend.dto.workflow_dto import WorkflowNode from core.generation_errors import provider_failure_attribution +from core.image_formats import ImageFormat, detect_image_format +from core.remote_fetch import RemoteFetchError, read_http_response_limited, url_hostname from core.storage import VfsStorage -from engines.image_provider import ImageGenerationRequest, ImageProvider, ImageProviderError +from engines.image_provider import ( + MAX_PROVIDER_IMAGE_BYTES, + ImageGenerationRequest, + ImageProvider, + ImageProviderError, +) ONE_PIXEL_PNG = base64.b64decode( @@ -39,9 +46,12 @@ class SingleFlowEngine: self, storage: VfsStorage | None = None, image_provider_factory: Callable[[dict[str, Any]], ImageProvider | None] | None = None, + *, + allow_mock_fallback: bool = False, ): self.storage = storage or VfsStorage() self.image_provider_factory = image_provider_factory + self.allow_mock_fallback = allow_mock_fallback def generate(self, node: WorkflowNode, task: GenerationTask, request: dict[str, Any]) -> list[MockGeneratedAsset]: count = _bounded_count(request.get("count", 1)) @@ -64,6 +74,8 @@ class SingleFlowEngine: count, ) except ImageProviderError as exc: + if not self.allow_mock_fallback: + raise route = request.get("resolved_model_route") if isinstance(request.get("resolved_model_route"), dict) else {} return self._generate_mock( task, @@ -76,7 +88,15 @@ class SingleFlowEngine: count, provider_error=provider_failure_attribution(exc, route_context=route, fallback_used=True), ) - return self._generate_mock(task, request, prompt, negative_prompt, model_route, width, height, count) + if self.allow_mock_fallback: + return self._generate_mock(task, request, prompt, negative_prompt, model_route, width, height, count) + route = request.get("resolved_model_route") if isinstance(request.get("resolved_model_route"), dict) else {} + raise ImageProviderError( + "no usable image provider is configured for this model route", + provider_id=str(route.get("provider_id") or "unresolved"), + error_type="configuration_error", + details={"model_route": model_route}, + ) def _generate_mock( self, @@ -135,10 +155,11 @@ class SingleFlowEngine: result = provider.generate_image( ImageGenerationRequest( prompt=prompt, + negative_prompt=negative_prompt, model=str(route.get("model_id") or model_route), size=f"{width}x{height}", count=count, - input_images=tuple(str(item) for item in request.get("input_images") or request.get("image") or []), + input_images=_normalize_input_images(request.get("input_images") or request.get("image")), metadata={"task_id": task.task_id, "model_route": model_route}, ) ) @@ -147,11 +168,17 @@ class SingleFlowEngine: for index, image in enumerate(result.images): if not image.has_image_data: continue - image_bytes = _image_bytes(image.b64_json, image.url, provider_id=result.provider_id) - image_uri = f"storage://images/single/{task_stem}_{index + 1}.png" - thumbnail_uri = f"storage://thumbnails/single/{task_stem}_{index + 1}.png" - self._write_png(image_uri, image_bytes) - self._write_png(thumbnail_uri, image_bytes) + image_bytes = _image_bytes( + image.b64_json, + image.url, + provider_id=result.provider_id, + provider_base_url=str(result.raw_metadata.get("provider_base_url") or ""), + ) + image_format = _require_image_format(image_bytes, provider_id=result.provider_id) + image_uri = f"storage://images/single/{task_stem}_{index + 1}{image_format.extension}" + thumbnail_uri = f"storage://thumbnails/single/{task_stem}_{index + 1}{image_format.extension}" + self._write_image(image_uri, image_bytes) + self._write_image(thumbnail_uri, image_bytes) assets.append( MockGeneratedAsset( image_uri=image_uri, @@ -169,6 +196,8 @@ class SingleFlowEngine: "provider_model": result.model, "usage": result.usage, "source_url": image.url, + "content_type": image_format.mime_type, + "file_extension": image_format.extension, }, ) ) @@ -181,6 +210,9 @@ class SingleFlowEngine: return assets def _write_png(self, uri: str, content: bytes = ONE_PIXEL_PNG) -> None: + self._write_image(uri, content) + + def _write_image(self, uri: str, content: bytes) -> None: path = self.storage.resolve_uri(uri) Path(path).parent.mkdir(parents=True, exist_ok=True) Path(path).write_bytes(content) @@ -229,14 +261,53 @@ def _safe_file_stem(value: str) -> str: return stem or "task" -def _image_bytes(b64_json: str | None, url: str | None, *, provider_id: str = "unknown") -> bytes: +def _normalize_input_images(value: Any) -> tuple[str, ...]: + if value is None: + return () + if isinstance(value, str): + text = value.strip() + return (text,) if text else () + if isinstance(value, (list, tuple, set)): + return tuple(text for item in value if (text := str(item).strip())) + text = str(value).strip() + return (text,) if text else () + + +def _image_bytes( + b64_json: str | None, + url: str | None, + *, + provider_id: str = "unknown", + provider_base_url: str | None = None, +) -> bytes: if b64_json: - return base64.b64decode(re.sub(r"\s+", "", b64_json)) - if url: + encoded = re.sub(r"\s+", "", b64_json) + if len(encoded) > ((MAX_PROVIDER_IMAGE_BYTES + 2) // 3) * 4: + raise ImageProviderError( + "image provider base64 payload exceeds the size limit", + provider_id=provider_id, + error_type="invalid_response", + ) try: - with urllib.request.urlopen(url, timeout=120) as response: - return response.read() - except Exception as exc: + body = base64.b64decode(encoded, validate=True) + except (binascii.Error, ValueError) as exc: + raise ImageProviderError( + "image provider returned invalid base64 image data", + provider_id=provider_id, + error_type="invalid_response", + ) from exc + return _validate_image_bytes(body, provider_id=provider_id) + if url: + provider_host = url_hostname(provider_base_url) + try: + body = read_http_response_limited( + url, + timeout_seconds=120, + max_bytes=MAX_PROVIDER_IMAGE_BYTES, + allowed_private_hosts=(provider_host,) if provider_host else (), + ) + return _validate_image_bytes(body, provider_id=provider_id) + except (OSError, RemoteFetchError) as exc: raise ImageProviderError( "image provider image URL could not be downloaded", provider_id=provider_id, @@ -244,3 +315,27 @@ def _image_bytes(b64_json: str | None, url: str | None, *, provider_id: str = "u details={"reason": str(exc)}, ) from exc raise ImageProviderError("image payload is empty", provider_id=provider_id, error_type="invalid_response") + + +def _validate_image_bytes(body: bytes, *, provider_id: str) -> bytes: + if not body: + raise ImageProviderError("image payload is empty", provider_id=provider_id, error_type="invalid_response") + if len(body) > MAX_PROVIDER_IMAGE_BYTES: + raise ImageProviderError( + "image provider payload exceeds the size limit", + provider_id=provider_id, + error_type="invalid_response", + ) + _require_image_format(body, provider_id=provider_id) + return body + + +def _require_image_format(body: bytes, *, provider_id: str) -> ImageFormat: + image_format = detect_image_format(body) + if image_format is None: + raise ImageProviderError( + "image provider payload is not a supported image", + provider_id=provider_id, + error_type="invalid_response", + ) + return image_format diff --git a/engines/vision_provider.py b/engines/vision_provider.py index c2c2df7..5346935 100644 --- a/engines/vision_provider.py +++ b/engines/vision_provider.py @@ -3,14 +3,19 @@ from __future__ import annotations import json +import re import urllib.error import urllib.request from dataclasses import dataclass, field from typing import Any, Callable, Protocol +from core.remote_fetch import RemoteFetchError, read_http_response_limited, validate_http_url + JsonDict = dict[str, Any] HttpTransport = Callable[[str, JsonDict, dict[str, str], int], JsonDict] +MAX_PROVIDER_JSON_BYTES = 4 * 1024 * 1024 +MAX_PROVIDER_ERROR_BYTES = 64 * 1024 class VisionProvider(Protocol): @@ -58,6 +63,7 @@ class VisionRecognitionResult: tags: list[str] = field(default_factory=list) confidence: float | None = None description: str | None = None + structured_data: dict[str, Any] = field(default_factory=dict) raw_metadata: dict[str, Any] = field(default_factory=dict) @@ -92,15 +98,18 @@ class OpenAIVisionProvider: provider_id=self.provider_id, error_type="configuration_error", ) + if not (_remote_image_url(request.image_uri) or _remote_image_url(request.thumbnail_uri)): + raise VisionProviderError( + "vision image must be supplied as an HTTP(S) URL or data:image URL", + provider_id=self.provider_id, + error_type="invalid_request", + ) payload = { "model": model, "messages": [ { "role": "system", - "content": ( - "Return compact JSON for a reusable outfit material with keys: " - "prompt, negative_prompt, tags, confidence, description." - ), + "content": _system_instruction(request), }, {"role": "user", "content": _user_content(request)}, ], @@ -108,8 +117,8 @@ class OpenAIVisionProvider: "response_format": {"type": "json_object"}, } headers = { - "Authorization": f"Bearer {self.config.api_key}", "Content-Type": "application/json", + **({"Authorization": f"Bearer {self.config.api_key}"} if self.config.api_key else {}), **self.config.extra_headers, } try: @@ -144,6 +153,17 @@ class MockVisionProvider: negative_prompt="body, face, background, watermark", tags=tags, confidence=0.72, + structured_data={ + "raw_analysis": { + "attributes": { + "design": { + "value": base, + "confidence": 0.72, + "source_state": "observed", + } + } + } + }, raw_metadata={"mock": True}, ) @@ -160,14 +180,28 @@ def _user_content(request: VisionRecognitionRequest) -> list[dict[str, Any]]: ), } ] - image_url = _remote_image_url(request.thumbnail_uri) or _remote_image_url(request.image_uri) + image_url = _remote_image_url(request.image_uri) or _remote_image_url(request.thumbnail_uri) if image_url: content.append({"type": "image_url", "image_url": {"url": image_url}}) - else: - content[0]["text"] += f"\nStorage image URI: {request.image_uri}" return content +def _system_instruction(request: VisionRecognitionRequest) -> str: + configured = str(request.metadata.get("recognition_instruction") or "").strip() + fallback = ( + "Analyze the visible material as exhaustively as possible. First identify its category, then extract only " + "visible facts including object type, colors, materials, texture, construction, silhouette, patterns, trims, " + "accessories, condition, viewpoint, lighting and reusable design features. Distinguish certainty from guesses. " + "Produce a detailed image-generation prompt that preserves the recognized design, plus a useful negative prompt. " + "Return one JSON object. Keep compatibility keys prompt, negative_prompt, tags, confidence and description. " + "Also include raw_analysis.attributes where every attribute has value, confidence (0-1), and source_state " + "(observed, inferred, unknown, occluded, or low_visibility); normalized_asset_profile with category, role, " + "body_slots, wear_layer, must_preserve and forbidden_changes; and optional generation_capsules. " + "Never present guesses as observed facts. Tags must be concise and searchable." + ) + return configured or fallback + + def _remote_image_url(value: str | None) -> str | None: text = str(value or "").strip() if text.startswith(("http://", "https://", "data:image/")): @@ -176,19 +210,33 @@ def _remote_image_url(value: str | None) -> str | None: def _post_json(url: str, payload: JsonDict, headers: dict[str, str], timeout_seconds: int) -> JsonDict: + try: + validate_http_url(url, allow_private=True) + except RemoteFetchError as exc: + raise VisionProviderError( + "vision provider endpoint is invalid", + provider_id="unknown", + error_type="configuration_error", + details={"reason": str(exc)}, + ) from exc data = json.dumps(payload, ensure_ascii=False).encode("utf-8") request = urllib.request.Request(url, data=data, headers=headers, method="POST") try: - with urllib.request.urlopen(request, timeout=timeout_seconds) as response: - body = response.read().decode("utf-8-sig") + body = read_http_response_limited( + request, + timeout_seconds=timeout_seconds, + max_bytes=MAX_PROVIDER_JSON_BYTES, + allow_private=True, + same_origin_redirects_only=True, + ).decode("utf-8-sig") except urllib.error.HTTPError as exc: - body = exc.read().decode("utf-8", errors="replace") + body = exc.read(MAX_PROVIDER_ERROR_BYTES + 1).decode("utf-8", errors="replace") raise VisionProviderError( "vision provider returned an HTTP error", provider_id="unknown", status_code=exc.code, error_type="http_error", - details={"body_preview": body[:500]}, + details={"body_preview": body[:500], "body_truncated": len(body) > MAX_PROVIDER_ERROR_BYTES}, ) from exc except urllib.error.URLError as exc: raise VisionProviderError( @@ -197,6 +245,13 @@ def _post_json(url: str, payload: JsonDict, headers: dict[str, str], timeout_sec error_type="network_error", details={"reason": str(exc.reason)}, ) from exc + except (RemoteFetchError, UnicodeDecodeError) as exc: + raise VisionProviderError( + "vision provider response exceeded safety limits", + provider_id="unknown", + error_type="invalid_response", + details={"reason": str(exc)}, + ) from exc try: parsed = json.loads(body) except json.JSONDecodeError as exc: @@ -230,7 +285,11 @@ def _parse_vision_response(provider_id: str, model: str, response: JsonDict) -> tags=_string_list(parsed.get("tags")), confidence=_float_or_none(parsed.get("confidence")), description=_optional_text(parsed.get("description")), - raw_metadata={"response_keys": sorted(str(key) for key in response.keys())}, + structured_data=parsed, + raw_metadata={ + "response_keys": sorted(str(key) for key in response.keys()), + "structured_keys": sorted(str(key) for key in parsed.keys()), + }, ) @@ -265,22 +324,34 @@ def _content_to_text(value: Any) -> str: def _json_from_text(value: str) -> JsonDict: - text = value.strip() + text = value.strip().lstrip("\ufeff") if not text: raise VisionProviderError("vision provider response was empty", provider_id="unknown", error_type="invalid_response") - try: - parsed = json.loads(text) - except json.JSONDecodeError: - start = text.find("{") - end = text.rfind("}") - if start < 0 or end <= start: - raise VisionProviderError( - "vision provider response was not JSON", - provider_id="unknown", - error_type="invalid_response", - details={"content_preview": text[:500]}, - ) from None - parsed = json.loads(text[start : end + 1]) + candidates = [text] + start = text.find("{") + end = text.rfind("}") + if start >= 0 and end > start: + extracted = text[start : end + 1] + if extracted != text: + candidates.append(extracted) + # A deliberately small repair set: fenced JSON and trailing commas only. + # It does not invent fields or values when a provider response is malformed. + candidates.extend(re.sub(r",\s*([}\]])", r"\1", item) for item in list(candidates)) + parsed: Any = None + last_error: json.JSONDecodeError | None = None + for candidate in candidates[:4]: + try: + parsed = json.loads(candidate) + break + except json.JSONDecodeError as exc: + last_error = exc + if parsed is None: + raise VisionProviderError( + "vision provider response was not valid JSON", + provider_id="unknown", + error_type="invalid_response", + details={"content_preview": text[:500], "parse_error": str(last_error or "unknown")}, + ) from None if not isinstance(parsed, dict): raise VisionProviderError("vision provider JSON must be an object", provider_id="unknown", error_type="invalid_response") return parsed diff --git a/frontend/README.md b/frontend/README.md index 4306d1c..f6d14a8 100644 --- a/frontend/README.md +++ b/frontend/README.md @@ -7,11 +7,21 @@ The frontend contains two Vite applications: Shared API clients, types, stores, components, constants, schemas, and utilities live in `shared/`. -Dependencies are declared in `package.json` but not installed yet. +The Workbench is split by responsibility: -Both UIs must consume the same v3 API contract: +- `standalone-web/src/App.vue` owns page orchestration, backend state, and cross-feature coordination. +- `standalone-web/src/components/` contains reusable navigation, metric, state, empty-state, image-preview, and material-dialog components. +- `standalone-web/src/features/` contains feature-level pure logic such as external character-profile parsing. +- `standalone-web/src/types/` contains shell/navigation contracts shared by the Workbench components. +- `shared/utils/` contains presentation and `storage://` helpers reused by both the Workbench and dialogs. -- Plugin Page UI uses `window.AstrBotPluginPage`. +Keep new modal, status-card, metric-card, parsing, and formatting logic out of `App.vue` when an existing component or utility can own it. The Workbench stylesheet is intentionally app-global inside the standalone page/plugin iframe so extracted components can reuse the same design tokens and class primitives. + +Install dependencies with `npm.cmd install` on Windows or `npm install` on other platforms. Validate with `npm.cmd run typecheck`, then build both targets with `npm.cmd run build`. + +Both targets render the same Workbench implementation and consume the same v3 API contract: + +- Plugin Page UI prefers `window.AstrBotPluginPage`; the shared client unwraps AstrBot and ImageStudio response envelopes and provides a POST method-override envelope for legacy bridges that only expose GET/POST. - Standalone Web UI uses `/api/v1/plugins/extensions/astrbot_plugin_imagestudio`. Do not store API keys, media paths, or selected image state in frontend-only storage. diff --git a/frontend/env.d.ts b/frontend/env.d.ts index 7d9c55e..5a7d699 100644 --- a/frontend/env.d.ts +++ b/frontend/env.d.ts @@ -10,10 +10,10 @@ declare module "*.vue" { interface Window { AstrBotPluginPage?: { ready?(): Promise - apiGet(path: string, params?: Record): Promise - apiPost(path: string, body?: unknown): Promise - apiPut(path: string, body?: unknown): Promise - apiPatch(path: string, body?: unknown): Promise - apiDelete(path: string): Promise + apiGet?(path: string, params?: Record): Promise + apiPost?(path: string, body?: unknown): Promise + apiPut?(path: string, body?: unknown): Promise + apiPatch?(path: string, body?: unknown): Promise + apiDelete?(path: string): Promise } } diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 7ff4d2b..fc75723 100644 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -1,12 +1,12 @@ { "name": "astrbot-plugin-imagestudio-frontend", - "version": "3.0.0", + "version": "1.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "astrbot-plugin-imagestudio-frontend", - "version": "3.0.0", + "version": "1.0.0", "dependencies": { "@vitejs/plugin-vue": "^5.2.0", "pinia": "^2.3.0", diff --git a/frontend/package.json b/frontend/package.json index a5a00a4..ec317e3 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,6 +1,6 @@ { "name": "astrbot-plugin-imagestudio-frontend", - "version": "3.0.0", + "version": "1.0.0", "private": true, "type": "module", "scripts": { diff --git a/frontend/plugin-page/index.html b/frontend/plugin-page/index.html index fa207c8..870ea73 100644 --- a/frontend/plugin-page/index.html +++ b/frontend/plugin-page/index.html @@ -1,2 +1,12 @@ -
- + + + + + + AstrBot ImageStudio + + +
+ + + diff --git a/frontend/plugin-page/src/App.vue b/frontend/plugin-page/src/App.vue index e7d79aa..1f66b66 100644 --- a/frontend/plugin-page/src/App.vue +++ b/frontend/plugin-page/src/App.vue @@ -1,453 +1,7 @@ - - diff --git a/frontend/plugin-page/vite.config.ts b/frontend/plugin-page/vite.config.ts index 3a0e328..257aa0f 100644 --- a/frontend/plugin-page/vite.config.ts +++ b/frontend/plugin-page/vite.config.ts @@ -1,13 +1,16 @@ import { resolve } from "node:path" +import { fileURLToPath } from "node:url" import vue from "@vitejs/plugin-vue" import { defineConfig } from "vite" +const rootDir = fileURLToPath(new URL(".", import.meta.url)) + export default defineConfig({ plugins: [vue()], - root: __dirname, + root: rootDir, base: "./", build: { - outDir: resolve(__dirname, "../../pages/imagestudio"), + outDir: resolve(rootDir, "../../pages/imagestudio"), emptyOutDir: true, }, }) diff --git a/frontend/shared/api/client.ts b/frontend/shared/api/client.ts index b0f1a40..276b3fa 100644 --- a/frontend/shared/api/client.ts +++ b/frontend/shared/api/client.ts @@ -1,5 +1,11 @@ import { API_BASE } from "./endpoints" -import type { ApiResponse } from "../types/api" +import type { ApiResponse, AstrBotApiResponse } from "../types/api" + +export type ApiTransport = "plugin-bridge" | "direct-fetch" +export type ApiMethod = "GET" | "POST" | "PUT" | "PATCH" | "DELETE" + +const METHOD_OVERRIDE_KEY = "__imagestudio_method" +const METHOD_OVERRIDE_BODY_KEY = "__imagestudio_body" export class ApiClientError extends Error { readonly code: string @@ -21,36 +27,84 @@ export async function postJson(endpoint: string, body?: unknown) { return request(endpoint, { method: "POST", body }) } +export async function putJson(endpoint: string, body?: unknown) { + return request(endpoint, { method: "PUT", body }) +} + +export async function patchJson(endpoint: string, body?: unknown) { + return request(endpoint, { method: "PATCH", body }) +} + +export async function deleteJson(endpoint: string, body?: unknown) { + return request(endpoint, { method: "DELETE", body }) +} + +export function getApiTransport(): ApiTransport { + const bridge = typeof window === "undefined" ? undefined : window.AstrBotPluginPage + return bridge && (typeof bridge.apiGet === "function" || typeof bridge.apiPost === "function") + ? "plugin-bridge" + : "direct-fetch" +} + export async function request( endpoint: string, options: { - method?: "GET" | "POST" | "PUT" | "DELETE" + method?: ApiMethod query?: Record body?: unknown } = {}, ): Promise { + const hasPluginBridge = typeof window !== "undefined" && Boolean(window.AstrBotPluginPage) const bridgePayload = await requestViaPluginBridge(endpoint, options) - if (bridgePayload !== undefined) { + if (hasPluginBridge) { + if (bridgePayload === undefined) { + throw new ApiClientError("invalid_api_response", "AstrBot Plugin Page bridge returned no response", { + endpoint, + method: options.method ?? "GET", + transport: "plugin-bridge", + }) + } return unwrapApiResponse(bridgePayload) } - const response = await fetch(buildUrl(endpoint, options.query), { - method: options.method ?? "GET", - headers: options.body === undefined ? undefined : { "Content-Type": "application/json" }, - body: options.body === undefined ? undefined : JSON.stringify(options.body), - }) - const payload = (await response.json()) as ApiResponse + const url = buildUrl(endpoint, options.query) + let response: Response + try { + response = await fetch(url, { + method: options.method ?? "GET", + headers: options.body === undefined ? undefined : { "Content-Type": "application/json" }, + body: options.body === undefined ? undefined : JSON.stringify(options.body), + }) + } catch (err) { + throw new ApiClientError("network_unreachable", "无法连接 ImageStudio 后端接口", { + endpoint, + transport: "direct-fetch", + url, + reason: err instanceof Error ? err.message : String(err), + }) + } + + const payload = await parseResponsePayload(response, endpoint, url) + if (!response.ok && !isApiResponse(payload) && !isAstrBotApiResponse(payload)) { + throw new ApiClientError("http_error", `接口请求失败(HTTP ${response.status})`, { + endpoint, + transport: "direct-fetch", + url, + status: response.status, + status_text: response.statusText, + }) + } return unwrapApiResponse(payload) } async function requestViaPluginBridge( endpoint: string, options: { - method?: "GET" | "POST" | "PUT" | "DELETE" + method?: ApiMethod query?: Record body?: unknown }, -): Promise | undefined> { +): Promise | AstrBotApiResponse | undefined> { const bridge = window.AstrBotPluginPage if (!bridge) { return undefined @@ -60,26 +114,113 @@ async function requestViaPluginBridge( await bridge.ready() } if (method === "GET" && typeof bridge.apiGet === "function") { - return bridge.apiGet>(endpoint, compactQuery(options.query)) + return bridge.apiGet | AstrBotApiResponse>(endpoint, compactQuery(options.query)) } if (method === "POST" && typeof bridge.apiPost === "function") { - return bridge.apiPost>(endpoint, options.body) + return bridge.apiPost | AstrBotApiResponse>(endpoint, options.body) } - return undefined + if (method === "PUT" && typeof bridge.apiPut === "function") { + return bridge.apiPut | AstrBotApiResponse>(endpoint, options.body) + } + if (method === "PATCH" && typeof bridge.apiPatch === "function") { + return bridge.apiPatch | AstrBotApiResponse>(endpoint, options.body) + } + if (method === "DELETE" && typeof bridge.apiDelete === "function") { + return bridge.apiDelete | AstrBotApiResponse>(endpoint) + } + if (["PUT", "PATCH", "DELETE"].includes(method) && typeof bridge.apiPost === "function") { + return bridge.apiPost | AstrBotApiResponse>(endpoint, { + [METHOD_OVERRIDE_KEY]: method, + [METHOD_OVERRIDE_BODY_KEY]: method === "DELETE" ? {} : options.body === undefined ? {} : options.body, + }) + } + throw new ApiClientError("bridge_method_unavailable", `AstrBot Plugin Page bridge does not support ${method}`, { + endpoint, + method, + transport: "plugin-bridge", + }) } -function unwrapApiResponse(payload: T | ApiResponse): T { - if (!isApiResponse(payload)) { - return payload +function unwrapApiResponse(payload: T | ApiResponse | AstrBotApiResponse): T { + let current: unknown = payload + for (let depth = 0; depth < 4; depth += 1) { + if (isApiResponse(current)) { + if (!current.ok) { + throw new ApiClientError(current.error_code, current.message, current.details ?? {}) + } + current = current.data + continue + } + if (isAstrBotApiResponse(current)) { + if (current.status === "error") { + if (isApiResponse(current.data)) { + current = current.data + continue + } + throw new ApiClientError( + current.error_code || "astrbot_bridge_error", + current.message || current.error || "AstrBot Plugin Page bridge request failed", + current.details ?? bridgeErrorDetails(current.data), + ) + } + current = current.data + continue + } + return current as T } - if (!payload.ok) { - throw new ApiClientError(payload.error_code, payload.message, payload.details) - } - return payload.data + throw new ApiClientError("invalid_api_response", "API response wrapper nesting is too deep") } -function isApiResponse(payload: T | ApiResponse): payload is ApiResponse { - return Boolean(payload && typeof payload === "object" && "ok" in payload) +async function parseResponsePayload( + response: Response, + endpoint: string, + url: string, +): Promise | AstrBotApiResponse> { + const contentType = response.headers.get("content-type") ?? "" + const isJson = contentType.includes("application/json") || contentType.includes("+json") + if (!isJson) { + const preview = (await response.text()).slice(0, 240) + throw new ApiClientError("invalid_api_response", "接口返回了非 JSON 响应", { + endpoint, + transport: "direct-fetch", + url, + status: response.status, + status_text: response.statusText, + content_type: contentType, + preview, + }) + } + try { + return (await response.json()) as T | ApiResponse | AstrBotApiResponse + } catch (err) { + throw new ApiClientError("invalid_api_response", "接口返回格式不正确", { + endpoint, + transport: "direct-fetch", + url, + status: response.status, + status_text: response.statusText, + content_type: contentType, + reason: err instanceof Error ? err.message : String(err), + }) + } +} + +function isApiResponse(payload: unknown): payload is ApiResponse { + if (!payload || typeof payload !== "object" || !("ok" in payload) || typeof payload.ok !== "boolean") { + return false + } + return payload.ok ? "data" in payload : "error_code" in payload && "message" in payload +} + +function isAstrBotApiResponse(payload: unknown): payload is AstrBotApiResponse { + if (!payload || typeof payload !== "object" || !("status" in payload)) { + return false + } + return payload.status === "error" || (payload.status === "ok" && "data" in payload) +} + +function bridgeErrorDetails(data: unknown): Record { + return data && typeof data === "object" ? { bridge_data: data } : {} } function compactQuery(query?: Record): Record | undefined { diff --git a/frontend/shared/api/endpoints.ts b/frontend/shared/api/endpoints.ts index 05f9f8a..9a8efea 100644 --- a/frontend/shared/api/endpoints.ts +++ b/frontend/shared/api/endpoints.ts @@ -4,20 +4,24 @@ export const endpoints = { uiBootstrap: "ui/bootstrap", uiActions: "ui/actions", dashboardSummary: "dashboard/summary", + contract: "contract", profiles: "profiles", profile: (profileId: string) => `profiles/${profileId}`, profileArchive: (profileId: string) => `profiles/${profileId}/archive`, quickProfileDraft: "profiles/quick-draft", + quickProfileDraftSave: "profiles/quick-draft/save", profileWorkflows: (profileId: string) => `profiles/${profileId}/workflows`, workflow: (workflowId: string) => `workflows/${workflowId}`, workflowStart: (workflowId: string) => `workflows/${workflowId}/start`, workflowCancel: (workflowId: string) => `workflows/${workflowId}/cancel`, workflowArchive: (workflowId: string) => `workflows/${workflowId}/archive`, + workflowExecuteCharacter: (workflowId: string) => `workflows/${workflowId}/execute-character`, workflowNodes: (workflowId: string) => `workflows/${workflowId}/nodes`, workflowBranches: (workflowId: string) => `workflows/${workflowId}/branches`, workflowSwitchBranch: (workflowId: string) => `workflows/${workflowId}/switch-branch`, workflowHistory: (workflowId: string) => `workflows/${workflowId}/history`, node: (nodeId: string) => `nodes/${nodeId}`, + nodeDependencyContext: (nodeId: string) => `nodes/${nodeId}/dependency-context`, nodeGenerate: (nodeId: string) => `nodes/${nodeId}/generate`, nodeRetry: (nodeId: string) => `nodes/${nodeId}/retry`, nodeComplete: (nodeId: string) => `nodes/${nodeId}/complete`, @@ -27,19 +31,26 @@ export const endpoints = { reviewPolicy: "review/policy", reviewPolicyValidate: "review/policy/validate", reviewPolicySave: "review/policy/save", + promptsExpand: "prompts/expand", + promptGovernancePreview: "prompts/governance-preview", nodeReview: (nodeId: string) => `nodes/${nodeId}/review`, nodeImages: (nodeId: string) => `nodes/${nodeId}/images`, workflowReview: (workflowId: string) => `workflows/${workflowId}/review`, + images: "images", image: (imageId: string) => `images/${imageId}`, imageSelect: (imageId: string) => `images/${imageId}/select`, imageUnselect: (imageId: string) => `images/${imageId}/unselect`, imageArchive: (imageId: string) => `images/${imageId}/archive`, + imageSafeDelete: (imageId: string) => `images/${imageId}/safe-delete`, + imageDelete: (imageId: string) => `images/${imageId}`, jobs: "jobs", job: (taskId: string) => `jobs/${taskId}`, jobCancel: (taskId: string) => `jobs/${taskId}/cancel`, jobRetry: (taskId: string) => `jobs/${taskId}/retry`, jobDeadLetterRetry: "jobs/dead-letter/retry", jobDeadLetterMoveFailed: "jobs/dead-letter/move-failed", + costUsage: "costs/usage", + costGuard: "costs/guard", settingsSchema: "settings/schema", settingsCurrent: "settings/current", settingsValidate: "settings/validate", @@ -51,9 +62,14 @@ export const endpoints = { settingsSnapshots: "settings/snapshots", settingsSnapshot: (snapshotId: string) => `settings/snapshots/${snapshotId}`, settingsRollback: "settings/rollback", + backups: "backups", + backup: (backupId: string) => `backups/${backupId}`, + backupRestore: (backupId: string) => `backups/${backupId}/restore`, storageStatus: "storage/status", storageMigrate: "storage/migrate", mediaSign: "media/sign", + mediaIndex: "media/index", + mediaOrphans: "media/orphans", materialCategories: "material-categories", materialCategory: (categoryId: string) => `material-categories/${categoryId}`, materials: "materials", @@ -69,7 +85,15 @@ export const endpoints = { wardrobeItem: (profileId: string, wardrobeItemId: string) => `profiles/${profileId}/wardrobe/items/${wardrobeItemId}`, wardrobeLegacyItem: (profileId: string, wardrobeItemId: string) => `profiles/${profileId}/wardrobe/${wardrobeItemId}`, wardrobeMatch: (profileId: string) => `profiles/${profileId}/wardrobe/match`, + wardrobeTemplates: (profileId: string) => `profiles/${profileId}/wardrobe-templates`, + wardrobeTemplate: (profileId: string, templateId: string) => `profiles/${profileId}/wardrobe-templates/${templateId}`, + wardrobeTemplateApply: (profileId: string, templateId: string) => `profiles/${profileId}/wardrobe-templates/${templateId}/apply`, + materialAnalytics: "materials/analytics", + materialRenormalizeBatch: "materials/renormalize-batch", + materialRenormalize: (materialId: string) => `materials/${materialId}/renormalize`, privateCompanionDailyOutfit: "integrations/private-companion/daily-outfit", + privateCompanionToken: "integrations/private-companion/token", + privateCompanionTokenRotate: "integrations/private-companion/token/rotate", integrationEventLogs: "integrations/logs", apiDiscoveryDialects: "api-discovery/dialects", apiDiscoveryProbeProvider: "api-discovery/probe-provider", diff --git a/frontend/shared/types/api.ts b/frontend/shared/types/api.ts index 594d9c2..0f98869 100644 --- a/frontend/shared/types/api.ts +++ b/frontend/shared/types/api.ts @@ -14,3 +14,20 @@ export interface ApiFailure { } export type ApiResponse = ApiSuccess | ApiFailure + +export interface AstrBotApiSuccess { + status: "ok" + data: T + message?: string +} + +export interface AstrBotApiFailure { + status: "error" + data?: unknown + error?: string + error_code?: string + message?: string + details?: Record +} + +export type AstrBotApiResponse = AstrBotApiSuccess | AstrBotApiFailure diff --git a/frontend/shared/types/assets.ts b/frontend/shared/types/assets.ts index 5289aaa..460e9dd 100644 --- a/frontend/shared/types/assets.ts +++ b/frontend/shared/types/assets.ts @@ -23,6 +23,12 @@ export interface AssetImportPlan { manifest: AssetManifestSummary import_enabled: boolean execution_enabled: boolean + unsupported_features?: Array<{ + code: string + message: string + count?: number + [key: string]: unknown + }> applied: { materials: number styles: number diff --git a/frontend/shared/types/cost.ts b/frontend/shared/types/cost.ts new file mode 100644 index 0000000..bc45431 --- /dev/null +++ b/frontend/shared/types/cost.ts @@ -0,0 +1,27 @@ +export interface CostUsageTotals { + image_count: number + task_count: number + total_tokens: number + input_tokens: number + output_tokens: number + estimated_cost: number +} + +export interface CostUsageGroup extends CostUsageTotals { + provider_id: string + provider_model: string + model_route: string +} + +export interface CostUsageSummary { + currency: string + filters: { + profile_id?: string | null + workflow_id?: string | null + node_id?: string | null + provider_id?: string | null + days?: number | null + } + totals: CostUsageTotals + groups: CostUsageGroup[] +} diff --git a/frontend/shared/types/image.ts b/frontend/shared/types/image.ts index 8865235..1e0cbb8 100644 --- a/frontend/shared/types/image.ts +++ b/frontend/shared/types/image.ts @@ -23,6 +23,13 @@ export interface GenerationImage { created_at: string } +export interface ImageListResult { + items: GenerationImage[] + total: number + page: number + page_size: number +} + export interface NodeGenerateResult { task_id?: string | null request_id?: string | null diff --git a/frontend/shared/types/material-operations.ts b/frontend/shared/types/material-operations.ts new file mode 100644 index 0000000..5ef2174 --- /dev/null +++ b/frontend/shared/types/material-operations.ts @@ -0,0 +1,16 @@ +export type MaterialOperationMode = "register-uri" | "asset-pack" | "wardrobe" + +export interface MaterialRegistrationRequest { + image_uri: string + display_name: string + category_id: string + filename: string +} + +export interface AssetPackRequest { + package_uri: string +} + +export interface WardrobeImportRequest { + wardrobe_category: string +} diff --git a/frontend/shared/types/profile.ts b/frontend/shared/types/profile.ts index 4c36aa8..a302bad 100644 --- a/frontend/shared/types/profile.ts +++ b/frontend/shared/types/profile.ts @@ -1,3 +1,5 @@ +import type { ProfileWorkflow, WorkflowNode } from "./workflow" + export interface CharacterProfile { profile_id: string name: string @@ -9,8 +11,10 @@ export interface CharacterProfile { style_prompt?: string | null negative_prompt?: string | null default_model_route?: string | null + metadata?: Record created_at: string updated_at: string + generation_kind?: "ungenerated" | "quick" | "detailed" } export interface ProfileListResult { @@ -30,3 +34,10 @@ export interface QuickProfileDraftResult { negative_prompt: string } } + +export interface QuickProfileDraftSaveResult { + draft_id?: string | null + profile: CharacterProfile + workflow?: ProfileWorkflow + nodes?: WorkflowNode[] +} diff --git a/frontend/shared/types/ui.ts b/frontend/shared/types/ui.ts index bd47ded..f9bc91d 100644 --- a/frontend/shared/types/ui.ts +++ b/frontend/shared/types/ui.ts @@ -6,3 +6,26 @@ export interface UiAction { requires_permission: string enabled: boolean } + +export interface UiMetricCard { + label: string + value: string + hint: string + state: TState +} + +export interface UiDetailRow { + label: string + value: string +} + +export interface UiActionCard { + id: TId + title: string + detail: string + badge: string + endpoint?: string + action: string + tone: TTone + enabled: boolean +} diff --git a/frontend/shared/types/wardrobe.ts b/frontend/shared/types/wardrobe.ts index 9c5ba5a..360887d 100644 --- a/frontend/shared/types/wardrobe.ts +++ b/frontend/shared/types/wardrobe.ts @@ -14,9 +14,39 @@ export interface WardrobeItem { } export interface WardrobeMatchResult { - profile_id?: string - items?: WardrobeItem[] - matched_items?: WardrobeItem[] - selected?: WardrobeItem | null - [key: string]: unknown + profile_id: string + selected_materials: Array<{ + wardrobe_item_id: string + material_id: string + wardrobe_category: string + display_name: string + prompt?: string | null + tags: string[] + season?: string | null + style?: string | null + is_favorite: boolean + usage_count: number + score: number + score_breakdown: Record + }> + outfit_prompt: string + fallback_used: boolean + fallback_supplement_used: boolean + fallback_reason: "missing_categories" | "no_matching_materials" | null + fallback_prompt?: string | null + fallback_strategy: { + name?: string | null + reason?: string | null + missing_categories: string[] + prompt: string + } + match_details: { + requested_categories: string[] + missing_categories: string[] + matched_count: number + keywords: string[] + requested_tags: string[] + season?: string | null + style?: string | null + } } diff --git a/frontend/shared/utils/collections.ts b/frontend/shared/utils/collections.ts new file mode 100644 index 0000000..d050921 --- /dev/null +++ b/frontend/shared/utils/collections.ts @@ -0,0 +1,13 @@ +export function uniqueBy(items: readonly T[], keyOf: (item: T) => K): T[] { + const seen = new Set() + return items.filter((item) => { + const key = keyOf(item) + if (seen.has(key)) return false + seen.add(key) + return true + }) +} + +export function uniqueStrings(items: readonly string[]): string[] { + return uniqueBy(items, (item) => item) +} diff --git a/frontend/shared/utils/presentation.ts b/frontend/shared/utils/presentation.ts new file mode 100644 index 0000000..5cebbb6 --- /dev/null +++ b/frontend/shared/utils/presentation.ts @@ -0,0 +1,93 @@ +import type { CostUsageSummary } from "../types/cost" +import type { GenerationJob } from "../types/job" + +export function normalizeStatus(status: string): string { + return String(status || "").trim().toLowerCase() +} + +export function statusLabel(status: string): string { + const labels: Record = { + running: "运行中", + degraded: "降级", + stopped: "已停止", + loading: "加载中", + created: "已创建", + pending: "等待中", + queued: "排队中", + completed: "已完成", + succeeded: "已成功", + failed: "失败", + cancelled: "已取消", + dead_letter: "死信", + retrying: "重试执行中", + superseded: "已被重试替代", + } + return labels[normalizeStatus(status)] ?? status +} + +export function statusClass(status: string): "is-bad" | "is-warn" | "is-good" | "is-idle" { + const normalized = normalizeStatus(status) + if (["failed", "dead_letter", "cancelled"].includes(normalized)) return "is-bad" + if (["running", "queued", "created", "pending", "retrying"].includes(normalized)) return "is-warn" + if (["completed", "succeeded"].includes(normalized)) return "is-good" + return "is-idle" +} + +export function formatBytes(value: number): string { + const bytes = Number.isFinite(value) && value > 0 ? value : 0 + if (bytes < 1024) return `${bytes} B` + if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB` + if (bytes < 1024 * 1024 * 1024) return `${(bytes / 1024 / 1024).toFixed(1)} MB` + return `${(bytes / 1024 / 1024 / 1024).toFixed(1)} GB` +} + +export function formatCostEstimate(summary: CostUsageSummary): string { + const amount = Number(summary.totals.estimated_cost) + const currency = summary.currency || "USD" + try { + return `${new Intl.NumberFormat("zh-CN", { + style: "currency", + currency, + minimumFractionDigits: amount < 1 ? 4 : 2, + maximumFractionDigits: amount < 1 ? 6 : 2, + }).format(amount)} 估算` + } catch { + return `${currency} ${amount.toFixed(amount < 1 ? 6 : 2)} 估算` + } +} + +export function formatTime(value?: string): string { + if (!value) return "-" + const date = new Date(value) + if (Number.isNaN(date.getTime())) return value + return new Intl.DateTimeFormat("zh-CN", { + month: "2-digit", + day: "2-digit", + hour: "2-digit", + minute: "2-digit", + }).format(date) +} + +export function timestamp(value: string): number { + const parsed = Date.parse(value) + return Number.isFinite(parsed) ? parsed : 0 +} + +export function waitTime(job: GenerationJob): string { + const start = timestamp(job.created_at) + if (!start) return "-" + const minutes = Math.max(0, Math.round((Date.now() - start) / 60000)) + if (minutes < 1) return "刚刚" + if (minutes < 60) return `${minutes} 分钟` + return `${Math.round(minutes / 60)} 小时` +} + +export function shortId(value: string): string { + return value.length > 10 ? `${value.slice(0, 6)}...${value.slice(-4)}` : value +} + +export function progressValue(job: GenerationJob): number { + const value = Number(job.progress) + if (!Number.isFinite(value)) return 0 + return Math.max(0, Math.min(100, value)) +} diff --git a/frontend/shared/utils/storage-uri.ts b/frontend/shared/utils/storage-uri.ts new file mode 100644 index 0000000..aefb3dc --- /dev/null +++ b/frontend/shared/utils/storage-uri.ts @@ -0,0 +1,12 @@ +export function isStorageUri(value: string): boolean { + return /^storage:\/\/[a-z0-9_-]+(?:\/[^?#]*)?$/i.test(value.trim()) +} + +export function storageFilename(value: string): string { + const normalized = value.trim().split(/[?#]/, 1)[0].replace(/\/+$/, "") + return normalized.split("/").pop() ?? "" +} + +export function isSupportedImageStorageUri(value: string): boolean { + return isStorageUri(value) && /\.(png|jpe?g|gif|webp)$/i.test(storageFilename(value)) +} diff --git a/frontend/standalone-web/index.html b/frontend/standalone-web/index.html index fa207c8..870ea73 100644 --- a/frontend/standalone-web/index.html +++ b/frontend/standalone-web/index.html @@ -1,2 +1,12 @@ -
- + + + + + + AstrBot ImageStudio + + +
+ + + diff --git a/frontend/standalone-web/src/App.vue b/frontend/standalone-web/src/App.vue index 289e151..1f336fa 100644 --- a/frontend/standalone-web/src/App.vue +++ b/frontend/standalone-web/src/App.vue @@ -1,32 +1,65 @@