AstrBot-Desktop/docs/progress.md
YokiiroBW 92ad44d327
一些检查失败了
ci / Portable quality checks (push) Waiting to run
ci / Windows Tauri build (push) Waiting to run
ci / Portable quality checks (pull_request) Has been cancelled
ci / Windows Tauri build (pull_request) Has been cancelled
fix(pet): add stable desktop rendering and context menu
2026-07-20 20:50:27 +08:00

16 KiB

Development Progress

Current phase

P02 dual connections remains active on feat/p02-dual-connections. The foundation has been adjusted for real AstrBot OpenAPI transport after live container probe. Credential enrollment, dual-slot runtime skeleton, and AstrBot SSE mapping are implemented. Full product chat UI activation and production identity enrollment remain open for acceptance.

Completed

Prior phases

  • P00 bootstrap merged.
  • P01 foundation merged through Forgejo PR #2.
  • P02 first increment: persistent dual slot assignment (bcc08cf).
  • P02 second increment: auto-connect validation and operations switch guard (34a563c).

Real-protocol foundation (this session)

  • Reviewed local AstrBot clone and live probe target http://192.168.31.210:26285.
  • Confirmed API Key chat SSE works; dual usernames/session ids isolate cleanly.
  • Added internal mapping module desktop-protocol::astrbot for OpenAPI SSE → Desktop events.
  • Added TransportKind (desktop-v1 | astrbot-openapi) plus remote_username / sticky_session_id profile fields.
  • Migrated SQLite schema to v4.
  • Added Rust AstrBotClient for probe/list/send chat stream.
  • Added DualSlotRuntime with activate/probe/send/deactivate skeleton.
  • Added bounded credential enrollment IPC (enroll_profile_credential / clear_profile_credential) using Windows Credential Manager; secrets never returned to React.
  • Added frontend transport fields and credential enrollment card.
  • Recorded ADR-0005 and live probe report.

In progress

  • User acceptance of real-protocol foundation.
  • Next product increments: chat panel rendering, history persistence, JWT login path, stronger server identity UX.

Remaining

  • Wire auto-connect startup to DualSlotRuntime without weakening secret boundary.
  • Dual chat UI panels consuming normalized runtime events.
  • Message persistence for AstrBot mapped events.
  • Dashboard JWT auth strategy.
  • File attachment OpenAPI path.
  • Certificate fingerprint binding.
  • Full stage gate (fmt/clippy/typecheck/build/installer) before merge.

Blockers / reserved for user

  • Production API Key rotation after shared test key exposure in chat.
  • Optional second AstrBot container if true dual-server soak is required.
  • Product decision on companion/operations remote username naming convention.
  • TLS-terminated HTTPS endpoint if pinning must be validated before release.

Test status

  • desktop-protocol lib tests: 4 passed (AstrBot mapper).
  • astrbot-desktop lib tests: 58 passed.
  • Vitest: 6 passed.
  • Live non-destructive probe artifacts retained under docs/reports/ (secrets redacted).
  • Full release build / Clippy gate / Playwright not re-run in this autonomous stretch.

Current branch

  • feat/p02-dual-connections

Next step after acceptance

  1. Chat panel MVP against AstrBot adapter.
  2. Persist sticky session + message history under profile/server/session keys.
  3. Optional auto-connect activation using enrolled credential references only.

2026-07-20: remote identity and runnable paper pet increment

  • Added remote_display_name as a separate user-facing nickname; remote_username remains the validated ASCII protocol ID used for AstrBot session isolation.
  • Migrated SQLite profiles to schema v9 without changing existing IDs or credentials.
  • Passed the nickname through the Rust-owned AstrBot request boundary as an optional nickname field; secrets remain Rust-only.
  • Added a transparent, borderless, skip-taskbar pet window with tray-resident hide-on-close behavior and a small CSS 2D paper-character state loop.
  • Frontend checks pass; Rust test/build gate remains to be rerun after the final patch.

Queue decision

  • Q1-Q3 complete the real AstrBot text-chat path and main UI before visual polish.
  • Q4 adds the draggable/resizable quick window using the same chat state.
  • Q5 adds the status bubble and configurable click routing.
  • Q6-Q7 reserve pet assets, multi-monitor geometry, and game-mode policy without adding arbitrary desktop control.

Autonomous queue increment

Completed in this increment:

  • Q1: Rust-owned bounded runtime event drain IPC and normalized event identity fields.
  • Q2: SQLite schema v5 chat message persistence keyed by account/profile/slot/server/session/message.
  • Q3: main chat panel MVP with Companion/Operations switching, streaming event merge, send error state, and session history restore.
  • Q4: dynamic Tauri quick-chat window sharing the Rust runtime.
  • Q5: non-default-topmost status bubble window with runtime status display.
  • Q6: pet state, safe PNG/WebP manifest schema, monitor geometry, and safe-edge placement model.

Validation: Rust workspace tests passed (59 tests), Rust Clippy with warnings denied passed, Rust format check passed, TypeScript check passed, Vitest passed (6 tests), and Vite production build passed.

Still not claimed complete: automatic startup activation, real Tauri multi-window manual UX, long-duration reconnect soak, JWT login, attachment protocol, certificate pinning, actual Windows multi-monitor/game-mode integration, voice, and release installer validation.

2026-07-20:Q1 启动自动连接增量

  • 已完成:在 Tauri 启动阶段读取已保存的 auto_connect Profile,并按 Companion / Operations 槽位独立激活。
  • 已完成:启动失败保留在对应 Runtime 状态,不阻断另一槽位;凭据仍由 Rust 读取,React 不接触秘密值。
  • 已完成:启动自动连接复用现有 AstrBot OpenAPI 与 Desktop Gateway 激活路径。
  • 验证:cargo check --workspace --all-targets、cargo test --workspace(59 项通过)、cargo clippy --workspace --all-targets -- -D warnings、pnpm.cmd run typecheck 通过。
  • 未完成:前端启动连接状态提示、自动重连退避、JWT 登录、聊天布局和快速窗口共享状态完善。
  • 当前工作区:本地未提交,等待下一项 Q1/Q2 增量完成后按用户指令提交。

2026-07-20??? 1-6 ??????

  • Q1????????????????????????????
  • Q2??? SQLite schema v6 ???????????????????????60 ? Rust ?????
  • Q3?????????????/????/?????????????? Hook ?????
  • Q4???????????????? URL ???
  • Q5????????????/????????????????????????????
  • Q6????? JSON manifest ???? UI?????????????PNG/WebP ??????????????
  • ?????2e84ea5?5dfac59?e0b48c0?9f235a3?4f13581?1828c61 ???? Forgejo?

2026-07-20??????????

  • ?????????????????????????????????????
  • ????????????????/????/????????????
  • ???????????????????????????????????????????
  • ??????????????/?????????????????? Windows ?????
  • ?????TypeScript?ESLint?Vitest 6/6?Vite production build ???

2026-07-20: chat extensibility contract

  • Added Rust-owned AstrBot capability discovery, model configuration options, and slot-scoped IPC.
  • Added a bounded 16 MiB multipart upload boundary and normalized attachment metadata.
  • Extended chat send requests with optional configId and attachments; the existing dual-slot isolation remains the authority.
  • Updated ChatPanel with model selection, attachment staging, upload state, and clean WebChat-style text labels.
  • Validation: Rust workspace tests 60 passed; Vitest 6/6 passed; TypeScript, ESLint, cargo check, and Vite build passed.

2026-07-20: live chat extensibility probe

  • Both supplied containers returned status=ok for chat sessions and config discovery.
  • Container 1 accepted a minimal multipart upload and returned attachment_id, filename, and type.
  • No chat message or configuration mutation was performed. Full multimodal rendering remains reserved for an explicit send-message acceptance test.

2026-07-20: live end-to-end chat acceptance

  • First test container accepted text plus config selection and streamed a complete response with no error frame.
  • The same session accepted text plus config selection plus the uploaded attachment and streamed a complete response with no error frame.
  • This validates the current model/config and attachment request contract against a real AstrBot instance.

2026-07-20: remote session navigation

  • Added slot-scoped remote session discovery and sticky session switching IPC.
  • ChatPanel now displays remote sessions and reloads history after switching; message filtering includes sessionId.
  • Validation: Rust workspace tests, Clippy, TypeScript, ESLint, Vitest, and Vite build passed.

2026-07-20: Q4 quick-chat window geometry persistence

  • Completed: added the window_states SQLite table (schema v7) for quick-chat position, size, and monitor identity.
  • Completed: restore geometry on open; fall back to the current primary monitor and clamp to its work area when a monitor is unavailable or the saved position is invalid.
  • Completed: save geometry on move, resize, and close-to-tray; keep the quick window single-instance without destroying the shared chat runtime.
  • Completed: added window bounds, monitor clamping, and database round-trip tests.
  • Validation: Rust format, Clippy with warnings denied, and astrbot-desktop unit tests pass.
  • Next: finish the remaining quick-window/status-bubble acceptance work, then continue with the pet asset and multi-monitor controller foundation.

2026-07-20: status bubble text cleanup

  • Fixed the remaining mojibake in status-bubble state labels and accessibility text.
  • Validation: TypeScript, ESLint, and Vitest (6 tests) pass.

2026-07-20: quick-chat close behavior fix

  • Fixed the global close handler so the quick-chat window can be closed normally instead of being intercepted and only hidden.
  • The main window keeps the existing tray behavior; quick-chat geometry is saved before destruction and restored when reopened.
  • Validation: Rust Clippy and astrbot-desktop unit tests (64) pass.

2026-07-20: JWT profile and 2D paper pet foundation

  • Added JWT as a persisted connection authentication type alongside API Key and device token; SQLite schema is now v8 and migrates existing profiles without losing slot assignments.
  • Added remote ID validation: 3-32 ASCII letters or digits; the ID remains separate from the user-facing nickname field planned for the profile UI.
  • Extended the pet-pack manifest with optional animation IDs, loop flags, and bounded movement modes for 2D paper-character actions.
  • Updated the pet importer copy and JWT selection UI; full Dashboard login, JWT expiry handling, and background pet window controller remain next-stage work.
  • Validation: Rust workspace tests, TypeScript, ESLint, Vitest (6), and Vite production build pass.

2026-07-20: JWT login IPC

  • Added a Rust-owned login_profile_jwt command for AstrBot Dashboard login at /api/v1/auth/login.
  • Dashboard username, password, and optional TOTP stay inside Rust; successful JWTs are written to Windows Credential Manager and the Profile is switched to JWT authentication.
  • Added Profile UI fields for JWT login and retained API Key/device-token enrollment for non-JWT profiles.
  • Full JWT expiry/401 re-login UX and nickname/remote-ID separation remain next increments.

2026-07-20: JWT runtime expiry handling

  • AstrBot JWT profiles now pass their authentication type through the Rust runtime instead of being treated as API Key-only profiles.
  • Rust checks the JWT exp claim before activation and maps expired credentials to AuthExpired without logging or returning the token.
  • AstrBot 401/403 responses during probe or chat now preserve the affected slot and expose an authExpired runtime error.
  • Chat UI shows a re-login warning for the affected slot; credentials remain managed by Windows Credential Manager.
  • The floating 2D pet now polls the dual-slot runtime and switches between idle, work, reply, rest, and error states instead of running an isolated demo loop.

2026-07-20: Aemeath pixel sprite integration

  • Reviewed aemeath_withclaude-2.0.8; the supplied repository and referenced pixel asset are MIT-licensed, not CC-licensed.
  • Added the original 8-column, 192x208-frame WebP spritesheet as a project-local pet asset with preserved attribution.
  • Added a validated pet manifest and mapped the source animation rows into idle, thinking, speaking, notification, error, and offline states.
  • Replaced the CSS placeholder character in the floating pet view with the real pixel spritesheet renderer.
  • Validation: Rust workspace tests (65), Clippy, TypeScript, ESLint, Vitest (6), and Vite build pass.

2026-07-20: production pet renderer white-window fix

  • Replaced the floating pet CSS background-image crop with a clipped <img> spritesheet renderer for more reliable WebView2/asset-protocol loading.
  • Added explicit pet-view root sizing and transparent html/body/#root backgrounds so Tauri transparent windows do not fall back to an opaque blank surface.
  • Validation: TypeScript, ESLint, Vitest (6), Vite build, Rust format, Clippy with warnings denied, Rust workspace tests (66), and NSIS bundle build pass.
  • The already-running executable must be fully closed and relaunched; the generated installer is at target/release/bundle/nsis/AstrBot Desktop_0.1.0-alpha.1_x64-setup.exe.

2026-07-20: tray quit reliability fix

  • Fixed the tray Quit action so it destroys the main, quick-chat, status-bubble, and pet windows before requesting process exit.
  • This avoids the global close-to-tray handler intercepting shutdown and leaving the process resident.
  • Validation: Rust format, Clippy with warnings denied, Rust workspace tests (66), TypeScript, ESLint, Vitest (6), and NSIS bundle build pass.

2026-07-20: desktop pet rendering and tray exit follow-up

  • Fixed the pixel spritesheet crop by setting the sprite image transform origin to the top-left; the previous default center origin moved the selected frame outside the clipped viewport and produced a blank transparent pet window.
  • Added an application exiting state so tray Quit allows all windows to close instead of the global close-to-tray handler hiding them and leaving an empty, non-interactive placeholder.
  • Validation: frontend typecheck, Vitest (6), Vite production build, Rust format, Rust check, and Rust workspace tests (66) pass.

2026-07-20: transparent pet and tray process exit hardening

  • Replaced the transparent-window spritesheet crop with a PNG-backed Canvas renderer; this avoids WebView2 transparent-window failures with the large WebP image and CSS transform cropping.
  • Changed tray Quit to call Tauri cleanup and terminate the process immediately, ensuring the tray icon, hidden main window, and pet window cannot remain as orphaned UI surfaces.
  • Rebuilt and launched the Release executable at 18:24:20 for manual verification.

2026-07-20: wire pet display mode and render diagnostics

  • Connected the pet display-mode setting to persisted local UI state; app-ui now embeds the pet in the Pet page, while desktop-floating explicitly opens the transparent Tauri window.
  • Added sprite loading and Canvas failure feedback instead of leaving a silent blank/white surface.
  • Validation: TypeScript, Vitest (6), Vite build, Rust format/check, and Release bundle build pass. The rebuilt executable started at 18:35:29.

2026-07-20: in-app draggable pet overlay

  • Promoted the app-ui pet from a static page preview to an application-level floating overlay, matching the Codex/Hermes-style in-window companion behavior.
  • Added pointer dragging with clamped coordinates and local persistence; the overlay can be hidden without creating a native desktop window.
  • Disabled the broken desktop-floating launch path in the Pet settings and provided a direct switch back to the in-app mode.
  • Validation: TypeScript, Vitest (6), Vite build, Rust format/check, and Release bundle build pass. The rebuilt executable started at 18:44:43.

2026-07-20: restore desktop floating pet transparency

  • Restored the desktop-floating display option and its open_pet_window action; it is no longer disabled.
  • Fixed the Windows transparent WebView configuration by explicitly setting an RGBA-zero background and disabling undecorated window shadow, which could otherwise render a white rectangle around the pet.
  • Preserved the app-ui draggable overlay implementation.
  • Validation: TypeScript, Vitest (6), Rust format/check, Vite build, and Release bundle build pass. The rebuilt executable started at 18:50:25.