296 行
20 KiB
Markdown
296 行
20 KiB
Markdown
# Development Progress
|
||
|
||
## Current phase
|
||
|
||
P02 dual connections remains active on `feat/p02-dual-connections`; Q1-Q5 implementation is now complete for the current MVP scope.
|
||
The foundation has been adjusted for real AstrBot OpenAPI transport after live container probe.
|
||
Credential enrollment, dual-slot runtime skeleton, and AstrBot SSE mapping are implemented.
|
||
Full product chat UI activation and production identity enrollment remain open for acceptance.
|
||
|
||
## Completed
|
||
|
||
### Prior phases
|
||
|
||
- P00 bootstrap merged.
|
||
- P01 foundation merged through Forgejo PR #2.
|
||
- P02 first increment: persistent dual slot assignment (`bcc08cf`).
|
||
- P02 second increment: auto-connect validation and operations switch guard (`34a563c`).
|
||
|
||
### Real-protocol foundation (this session)
|
||
|
||
- Reviewed local AstrBot clone and live probe target `http://192.168.31.210:26285`.
|
||
- Confirmed API Key chat SSE works; dual usernames/session ids isolate cleanly.
|
||
- Added internal mapping module `desktop-protocol::astrbot` for OpenAPI SSE → Desktop events.
|
||
- Added `TransportKind` (`desktop-v1` | `astrbot-openapi`) plus `remote_username` / `sticky_session_id` profile fields.
|
||
- Migrated SQLite schema to v4.
|
||
- Added Rust `AstrBotClient` for probe/list/send chat stream.
|
||
- Added `DualSlotRuntime` with activate/probe/send/deactivate skeleton.
|
||
- Added bounded credential enrollment IPC (`enroll_profile_credential` / `clear_profile_credential`) using Windows Credential Manager; secrets never returned to React.
|
||
- Added frontend transport fields and credential enrollment card.
|
||
- Recorded ADR-0005 and live probe report.
|
||
|
||
## In progress
|
||
|
||
- User acceptance of real-protocol foundation.
|
||
- Next product increments: chat panel rendering, history persistence, JWT login path, stronger server identity UX.
|
||
|
||
## Remaining
|
||
|
||
- Wire auto-connect startup to DualSlotRuntime without weakening secret boundary.
|
||
- Dual chat UI panels consuming normalized runtime events.
|
||
- Message persistence for AstrBot mapped events.
|
||
- Dashboard JWT auth strategy.
|
||
- File attachment OpenAPI path.
|
||
- Certificate fingerprint binding.
|
||
- Full stage gate (fmt/clippy/typecheck/build/installer) before merge.
|
||
|
||
## Blockers / reserved for user
|
||
|
||
- Production API Key rotation after shared test key exposure in chat.
|
||
- Optional second AstrBot container if true dual-server soak is required.
|
||
- Product decision on companion/operations remote username naming convention.
|
||
- TLS-terminated HTTPS endpoint if pinning must be validated before release.
|
||
|
||
## Test status
|
||
|
||
- `desktop-protocol` lib tests: 4 passed (AstrBot mapper).
|
||
- `astrbot-desktop` lib tests: 58 passed.
|
||
- Vitest: 6 passed.
|
||
- Live non-destructive probe artifacts retained under `docs/reports/` (secrets redacted).
|
||
- Full release build / Clippy gate / Playwright not re-run in this autonomous stretch.
|
||
|
||
## Current branch
|
||
|
||
- `feat/p02-dual-connections`
|
||
|
||
## Next step after acceptance
|
||
|
||
1. Chat panel MVP against AstrBot adapter.
|
||
2. Persist sticky session + message history under profile/server/session keys.
|
||
3. Optional auto-connect activation using enrolled credential references only.
|
||
|
||
## 2026-07-20: remote identity and runnable paper pet increment
|
||
|
||
- Added `remote_display_name` as a separate user-facing nickname; `remote_username` remains the validated ASCII protocol ID used for AstrBot session isolation.
|
||
- Migrated SQLite profiles to schema v9 without changing existing IDs or credentials.
|
||
- Passed the nickname through the Rust-owned AstrBot request boundary as an optional `nickname` field; secrets remain Rust-only.
|
||
- Added a transparent, borderless, skip-taskbar pet window with tray-resident hide-on-close behavior and a small CSS 2D paper-character state loop.
|
||
- Frontend checks pass; Rust test/build gate remains to be rerun after the final patch.
|
||
|
||
## Queue decision
|
||
|
||
- Q1-Q3 complete the real AstrBot text-chat path and main UI before visual polish.
|
||
- Q4 adds the draggable/resizable quick window using the same chat state.
|
||
- Q5 adds the status bubble and configurable click routing.
|
||
- Q6-Q7 reserve pet assets, multi-monitor geometry, and game-mode policy without adding arbitrary desktop control.
|
||
|
||
## Autonomous queue increment
|
||
|
||
Completed in this increment:
|
||
|
||
- Q1: Rust-owned bounded runtime event drain IPC and normalized event identity fields.
|
||
- Q2: SQLite schema v5 chat message persistence keyed by account/profile/slot/server/session/message.
|
||
- Q3: main chat panel MVP with Companion/Operations switching, streaming event merge, send error state, and session history restore.
|
||
- Q4: dynamic Tauri quick-chat window sharing the Rust runtime.
|
||
- Q5: non-default-topmost status bubble window with runtime status display.
|
||
- Q6: pet state, safe PNG/WebP manifest schema, monitor geometry, and safe-edge placement model.
|
||
|
||
Validation: Rust workspace tests passed (59 tests), Rust Clippy with warnings denied passed, Rust format check passed, TypeScript check passed, Vitest passed (6 tests), and Vite production build passed.
|
||
|
||
Still not claimed complete: automatic startup activation, real Tauri multi-window manual UX, long-duration reconnect soak, JWT login, attachment protocol, certificate pinning, actual Windows multi-monitor/game-mode integration, voice, and release installer validation.
|
||
|
||
## 2026-07-20:Q1 启动自动连接增量
|
||
|
||
- 已完成:在 Tauri 启动阶段读取已保存的 `auto_connect` Profile,并按 Companion / Operations 槽位独立激活。
|
||
- 已完成:启动失败保留在对应 Runtime 状态,不阻断另一槽位;凭据仍由 Rust 读取,React 不接触秘密值。
|
||
- 已完成:启动自动连接复用现有 AstrBot OpenAPI 与 Desktop Gateway 激活路径。
|
||
- 验证:`cargo check --workspace --all-targets`、`cargo test --workspace`(59 项通过)、`cargo clippy --workspace --all-targets -- -D warnings`、`pnpm.cmd run typecheck` 通过。
|
||
- 未完成:前端启动连接状态提示、自动重连退避、JWT 登录、聊天布局和快速窗口共享状态完善。
|
||
- 当前工作区:本地未提交,等待下一项 Q1/Q2 增量完成后按用户指令提交。
|
||
|
||
## 2026-07-20??? 1-6 ??????
|
||
|
||
- Q1????????????????????????????
|
||
- Q2??? SQLite schema v6 ???????????????????????60 ? Rust ?????
|
||
- Q3?????????????/????/?????????????? Hook ?????
|
||
- Q4???????????????? URL ???
|
||
- Q5????????????/????????????????????????????
|
||
- Q6????? JSON manifest ???? UI?????????????PNG/WebP ??????????????
|
||
- ?????`2e84ea5`?`5dfac59`?`e0b48c0`?`9f235a3`?`4f13581`?`1828c61` ???? Forgejo?
|
||
|
||
## 2026-07-20??????????
|
||
|
||
- ?????????????????????????????????????
|
||
- ????????????????/????/????????????
|
||
- ???????????????????????????????????????????
|
||
- ??????????????/?????????????????? Windows ?????
|
||
- ?????TypeScript?ESLint?Vitest 6/6?Vite production build ???
|
||
|
||
## 2026-07-20: chat extensibility contract
|
||
|
||
- Added Rust-owned AstrBot capability discovery, model configuration options, and slot-scoped IPC.
|
||
- Added a bounded 16 MiB multipart upload boundary and normalized attachment metadata.
|
||
- Extended chat send requests with optional configId and attachments; the existing dual-slot isolation remains the authority.
|
||
- Updated ChatPanel with model selection, attachment staging, upload state, and clean WebChat-style text labels.
|
||
- Validation: Rust workspace tests 60 passed; Vitest 6/6 passed; TypeScript, ESLint, cargo check, and Vite build passed.
|
||
|
||
## 2026-07-20: live chat extensibility probe
|
||
|
||
- Both supplied containers returned status=ok for chat sessions and config discovery.
|
||
- Container 1 accepted a minimal multipart upload and returned attachment_id, filename, and type.
|
||
- No chat message or configuration mutation was performed. Full multimodal rendering remains reserved for an explicit send-message acceptance test.
|
||
|
||
## 2026-07-20: live end-to-end chat acceptance
|
||
|
||
- First test container accepted text plus config selection and streamed a complete response with no error frame.
|
||
- The same session accepted text plus config selection plus the uploaded attachment and streamed a complete response with no error frame.
|
||
- This validates the current model/config and attachment request contract against a real AstrBot instance.
|
||
|
||
## 2026-07-20: remote session navigation
|
||
|
||
- Added slot-scoped remote session discovery and sticky session switching IPC.
|
||
- ChatPanel now displays remote sessions and reloads history after switching; message filtering includes sessionId.
|
||
- Validation: Rust workspace tests, Clippy, TypeScript, ESLint, Vitest, and Vite build passed.
|
||
|
||
## 2026-07-20: Q1-Q5 shared chat runtime completion
|
||
|
||
- Q1: normalized runtime events are now broadcast at the application level, so the main window and quick-chat window receive the same stream without competing for a single drain queue.
|
||
- Q2: chat schema v10 includes `slot` in the message key and all history/read IPC queries; streaming deltas append to the stored draft and completed events replace it with the authoritative content.
|
||
- Q3: main chat sends user messages into durable history after the runtime assigns a session, keeps slot/session filtering isolated, and preserves live assistant updates.
|
||
- Q4: quick chat continues to reuse the Rust runtime and SQLite history, with persisted geometry and shared live events across windows.
|
||
- Q5: the status bubble shows connection state and unread totals; the tray menu shows live Companion/Operations states and provides main/quick chat entry points.
|
||
- Validation: Rust workspace tests pass (67 tests plus protocol/integration suites), frontend typecheck/lint/Vitest/build pass, and touched frontend files are formatted.
|
||
|
||
## 2026-07-20: Q6-Q8 companion, diagnostics, and interaction MVP
|
||
|
||
- Q6: removed the desktop pet's timer-driven idle/resting placeholder; runtime connection and normalized chat events now drive working, speaking, resting, and error states.
|
||
- Q6-Q7: added monitor geometry discovery, safe-edge placement, remembered monitor selection, always-on-top control, and game-mode safe-edge fallback without input injection or arbitrary desktop control.
|
||
- Q7: added bounded read-only diagnostics for the foreground window, a user-selected process ID, and one explicitly requested network host/port.
|
||
- Q8: added browser Web Speech input and speech synthesis playback on top of the shared chat send/message path, plus proactive-message pet updates and configurable quiet hours for bubble reminders.
|
||
- Validation: Rust format, Clippy, workspace tests, TypeScript, ESLint, Vitest, and Vite build pass.
|
||
|
||
## 2026-07-20: Q9 local stability and unsigned release
|
||
|
||
- Added a session marker in the application data directory; an unexpected previous session is reported on the next startup while local profiles and chat history remain recoverable.
|
||
- Bumped the local release line to `0.1.0-alpha.2` and added the Q9 local acceptance checklist.
|
||
- Updated the changelog to distinguish the unsigned local installer from future signed/auto-update releases.
|
||
- Rust workspace tests pass after the version and recovery changes; frontend checks remain green.
|
||
- Pending manual acceptance: install the unsigned NSIS package, exercise tray/second-launch/network-reconnect/window-restore paths, and verify the recovery marker after a disposable forced termination.
|
||
|
||
## 2026-07-20: Q4 quick-chat window geometry persistence
|
||
|
||
- Completed: added the `window_states` SQLite table (schema v7) for quick-chat position, size, and monitor identity.
|
||
- Completed: restore geometry on open; fall back to the current primary monitor and clamp to its work area when a monitor is unavailable or the saved position is invalid.
|
||
- Completed: save geometry on move, resize, and close-to-tray; keep the quick window single-instance without destroying the shared chat runtime.
|
||
- Completed: added window bounds, monitor clamping, and database round-trip tests.
|
||
- Validation: Rust format, Clippy with warnings denied, and `astrbot-desktop` unit tests pass.
|
||
- Next: finish the remaining quick-window/status-bubble acceptance work, then continue with the pet asset and multi-monitor controller foundation.
|
||
|
||
## 2026-07-20: status bubble text cleanup
|
||
|
||
- Fixed the remaining mojibake in status-bubble state labels and accessibility text.
|
||
- Validation: TypeScript, ESLint, and Vitest (6 tests) pass.
|
||
|
||
## 2026-07-20: quick-chat close behavior fix
|
||
|
||
- Fixed the global close handler so the quick-chat window can be closed normally instead of being intercepted and only hidden.
|
||
- The main window keeps the existing tray behavior; quick-chat geometry is saved before destruction and restored when reopened.
|
||
- Validation: Rust Clippy and `astrbot-desktop` unit tests (64) pass.
|
||
|
||
## 2026-07-20: JWT profile and 2D paper pet foundation
|
||
|
||
- Added JWT as a persisted connection authentication type alongside API Key and device token; SQLite schema is now v8 and migrates existing profiles without losing slot assignments.
|
||
- Added remote ID validation: 3-32 ASCII letters or digits; the ID remains separate from the user-facing nickname field planned for the profile UI.
|
||
- Extended the pet-pack manifest with optional animation IDs, loop flags, and bounded movement modes for 2D paper-character actions.
|
||
- Updated the pet importer copy and JWT selection UI; full Dashboard login, JWT expiry handling, and background pet window controller remain next-stage work.
|
||
- Validation: Rust workspace tests, TypeScript, ESLint, Vitest (6), and Vite production build pass.
|
||
|
||
## 2026-07-20: JWT login IPC
|
||
|
||
- Added a Rust-owned login_profile_jwt command for AstrBot Dashboard login at /api/v1/auth/login.
|
||
- Dashboard username, password, and optional TOTP stay inside Rust; successful JWTs are written to Windows Credential Manager and the Profile is switched to JWT authentication.
|
||
- Added Profile UI fields for JWT login and retained API Key/device-token enrollment for non-JWT profiles.
|
||
- Full JWT expiry/401 re-login UX and nickname/remote-ID separation remain next increments.
|
||
|
||
## 2026-07-20: JWT runtime expiry handling
|
||
|
||
- AstrBot JWT profiles now pass their authentication type through the Rust runtime instead of being treated as API Key-only profiles.
|
||
- Rust checks the JWT `exp` claim before activation and maps expired credentials to `AuthExpired` without logging or returning the token.
|
||
- AstrBot 401/403 responses during probe or chat now preserve the affected slot and expose an `authExpired` runtime error.
|
||
- Chat UI shows a re-login warning for the affected slot; credentials remain managed by Windows Credential Manager.
|
||
- The floating 2D pet now polls the dual-slot runtime and switches between idle, work, reply, rest, and error states instead of running an isolated demo loop.
|
||
|
||
## 2026-07-20: Aemeath pixel sprite integration
|
||
|
||
- Reviewed `aemeath_withclaude-2.0.8`; the supplied repository and referenced pixel asset are MIT-licensed, not CC-licensed.
|
||
- Added the original 8-column, 192x208-frame WebP spritesheet as a project-local pet asset with preserved attribution.
|
||
- Added a validated pet manifest and mapped the source animation rows into idle, thinking, speaking, notification, error, and offline states.
|
||
- Replaced the CSS placeholder character in the floating pet view with the real pixel spritesheet renderer.
|
||
- Validation: Rust workspace tests (65), Clippy, TypeScript, ESLint, Vitest (6), and Vite build pass.
|
||
|
||
## 2026-07-20: production pet renderer white-window fix
|
||
|
||
- Replaced the floating pet CSS background-image crop with a clipped `<img>` spritesheet renderer for more reliable WebView2/asset-protocol loading.
|
||
- Added explicit pet-view root sizing and transparent `html`/`body`/`#root` backgrounds so Tauri transparent windows do not fall back to an opaque blank surface.
|
||
- Validation: TypeScript, ESLint, Vitest (6), Vite build, Rust format, Clippy with warnings denied, Rust workspace tests (66), and NSIS bundle build pass.
|
||
- The already-running executable must be fully closed and relaunched; the generated installer is at `target/release/bundle/nsis/AstrBot Desktop_0.1.0-alpha.1_x64-setup.exe`.
|
||
|
||
## 2026-07-20: tray quit reliability fix
|
||
|
||
- Fixed the tray Quit action so it destroys the main, quick-chat, status-bubble, and pet windows before requesting process exit.
|
||
- This avoids the global close-to-tray handler intercepting shutdown and leaving the process resident.
|
||
- Validation: Rust format, Clippy with warnings denied, Rust workspace tests (66), TypeScript, ESLint, Vitest (6), and NSIS bundle build pass.
|
||
|
||
## 2026-07-20: desktop pet rendering and tray exit follow-up
|
||
|
||
- Fixed the pixel spritesheet crop by setting the sprite image transform origin to the top-left; the previous default center origin moved the selected frame outside the clipped viewport and produced a blank transparent pet window.
|
||
- Added an application exiting state so tray Quit allows all windows to close instead of the global close-to-tray handler hiding them and leaving an empty, non-interactive placeholder.
|
||
- Validation: frontend typecheck, Vitest (6), Vite production build, Rust format, Rust check, and Rust workspace tests (66) pass.
|
||
|
||
## 2026-07-20: transparent pet and tray process exit hardening
|
||
|
||
- Replaced the transparent-window spritesheet crop with a PNG-backed Canvas renderer; this avoids WebView2 transparent-window failures with the large WebP image and CSS transform cropping.
|
||
- Changed tray Quit to call Tauri cleanup and terminate the process immediately, ensuring the tray icon, hidden main window, and pet window cannot remain as orphaned UI surfaces.
|
||
- Rebuilt and launched the Release executable at 18:24:20 for manual verification.
|
||
|
||
## 2026-07-20: wire pet display mode and render diagnostics
|
||
|
||
- Connected the pet display-mode setting to persisted local UI state; `app-ui` now embeds the pet in the Pet page, while `desktop-floating` explicitly opens the transparent Tauri window.
|
||
- Added sprite loading and Canvas failure feedback instead of leaving a silent blank/white surface.
|
||
- Validation: TypeScript, Vitest (6), Vite build, Rust format/check, and Release bundle build pass. The rebuilt executable started at 18:35:29.
|
||
|
||
## 2026-07-20: in-app draggable pet overlay
|
||
|
||
- Promoted the app-ui pet from a static page preview to an application-level floating overlay, matching the Codex/Hermes-style in-window companion behavior.
|
||
- Added pointer dragging with clamped coordinates and local persistence; the overlay can be hidden without creating a native desktop window.
|
||
- Disabled the broken desktop-floating launch path in the Pet settings and provided a direct switch back to the in-app mode.
|
||
- Validation: TypeScript, Vitest (6), Vite build, Rust format/check, and Release bundle build pass. The rebuilt executable started at 18:44:43.
|
||
|
||
## 2026-07-20: restore desktop floating pet transparency
|
||
|
||
- Restored the desktop-floating display option and its `open_pet_window` action; it is no longer disabled.
|
||
- Fixed the Windows transparent WebView configuration by explicitly setting an RGBA-zero background and disabling undecorated window shadow, which could otherwise render a white rectangle around the pet.
|
||
- Preserved the app-ui draggable overlay implementation.
|
||
- Validation: TypeScript, Vitest (6), Rust format/check, Vite build, and Release bundle build pass. The rebuilt executable started at 18:50:25.
|
||
|
||
## 2026-07-20: unified slot runtime lifecycle hardening
|
||
|
||
- Added independent connection cancellation and message cancellation tokens for each Companion / Operations slot.
|
||
- Connected the desktop gateway SSE reconnect loop to the active slot lifecycle, including state synchronization and terminal error mapping.
|
||
- Added generation guards so a late response from a cancelled or replaced slot cannot restore a stale client into the new slot state.
|
||
- Preserved AstrBot OpenAPI request-stream handling and bounded retry behavior; AstrBot does not expose a separate unsolicited SSE subscription endpoint in the current adapter.
|
||
- Validation: Rust fmt, workspace check, Rust library tests (66), TypeScript typecheck, and Vitest (6) pass.
|
||
|
||
## 2026-07-20: compact chat-first main window
|
||
|
||
- Fixed runtime status parsing so Rust null option fields are accepted as an unconnected slot state instead of showing raw Zod validation output.
|
||
- Made the chat page chat-first: removed the workspace layout selector and model dropdown from the primary page; secondary settings remain in their dedicated navigation pages.
|
||
- Reduced main-window spacing, added rounded application corners, and preserved the frameless custom title bar.
|
||
- Validation: TypeScript, ESLint, Vitest (6), Vite build, and Rust workspace check pass.
|
||
|
||
## 2026-07-21: AstrBot chat complete-frame handling
|
||
|
||
- Fixed the AstrBot SSE mapper to consume `type: "complete"` frames, which carry the authoritative final response text and were previously ignored.
|
||
- Reconciled final text with partial `plain` chunks and added support for text nested under `text`, `content`, or `message` fields.
|
||
- Added protocol tests for complete-only replies and partial-stream reconciliation.
|
||
- Validation: Rust format, Clippy with warnings denied, Rust workspace tests (67), TypeScript, ESLint, and NSIS bundle build pass.
|