# Development Progress ## Current phase P02 dual connections remains active on `feat/p02-dual-connections`; Q1-Q5 implementation is now complete for the current MVP scope. The foundation has been adjusted for real AstrBot OpenAPI transport after live container probe. Credential enrollment, dual-slot runtime skeleton, and AstrBot SSE mapping are implemented. Full product chat UI activation and production identity enrollment remain open for acceptance. ## Completed ### Prior phases - P00 bootstrap merged. - P01 foundation merged through Forgejo PR #2. - P02 first increment: persistent dual slot assignment (`bcc08cf`). - P02 second increment: auto-connect validation and operations switch guard (`34a563c`). ### Real-protocol foundation (this session) - Reviewed local AstrBot clone and live probe target `http://192.168.31.210:26285`. - Confirmed API Key chat SSE works; dual usernames/session ids isolate cleanly. - Added internal mapping module `desktop-protocol::astrbot` for OpenAPI SSE → Desktop events. - Added `TransportKind` (`desktop-v1` | `astrbot-openapi`) plus `remote_username` / `sticky_session_id` profile fields. - Migrated SQLite schema to v4. - Added Rust `AstrBotClient` for probe/list/send chat stream. - Added `DualSlotRuntime` with activate/probe/send/deactivate skeleton. - Added bounded credential enrollment IPC (`enroll_profile_credential` / `clear_profile_credential`) using Windows Credential Manager; secrets never returned to React. - Added frontend transport fields and credential enrollment card. - Recorded ADR-0005 and live probe report. ## In progress - User acceptance of real-protocol foundation. - Next product increments: chat panel rendering, history persistence, JWT login path, stronger server identity UX. ## Remaining - Wire auto-connect startup to DualSlotRuntime without weakening secret boundary. - Dual chat UI panels consuming normalized runtime events. - Message persistence for AstrBot mapped events. - Dashboard JWT auth strategy. - File attachment OpenAPI path. - Certificate fingerprint binding. - Full stage gate (fmt/clippy/typecheck/build/installer) before merge. ## Blockers / reserved for user - Production API Key rotation after shared test key exposure in chat. - Optional second AstrBot container if true dual-server soak is required. - Product decision on companion/operations remote username naming convention. - TLS-terminated HTTPS endpoint if pinning must be validated before release. ## Test status - `desktop-protocol` lib tests: 4 passed (AstrBot mapper). - `astrbot-desktop` lib tests: 58 passed. - Vitest: 6 passed. - Live non-destructive probe artifacts retained under `docs/reports/` (secrets redacted). - Full release build / Clippy gate / Playwright not re-run in this autonomous stretch. ## Current branch - `feat/p02-dual-connections` ## Next step after acceptance 1. Chat panel MVP against AstrBot adapter. 2. Persist sticky session + message history under profile/server/session keys. 3. Optional auto-connect activation using enrolled credential references only. ## 2026-07-20: remote identity and runnable paper pet increment - Added `remote_display_name` as a separate user-facing nickname; `remote_username` remains the validated ASCII protocol ID used for AstrBot session isolation. - Migrated SQLite profiles to schema v9 without changing existing IDs or credentials. - Passed the nickname through the Rust-owned AstrBot request boundary as an optional `nickname` field; secrets remain Rust-only. - Added a transparent, borderless, skip-taskbar pet window with tray-resident hide-on-close behavior and a small CSS 2D paper-character state loop. - Frontend checks pass; Rust test/build gate remains to be rerun after the final patch. ## Queue decision - Q1-Q3 complete the real AstrBot text-chat path and main UI before visual polish. - Q4 adds the draggable/resizable quick window using the same chat state. - Q5 adds the status bubble and configurable click routing. - Q6-Q7 reserve pet assets, multi-monitor geometry, and game-mode policy without adding arbitrary desktop control. ## Autonomous queue increment Completed in this increment: - Q1: Rust-owned bounded runtime event drain IPC and normalized event identity fields. - Q2: SQLite schema v5 chat message persistence keyed by account/profile/slot/server/session/message. - Q3: main chat panel MVP with Companion/Operations switching, streaming event merge, send error state, and session history restore. - Q4: dynamic Tauri quick-chat window sharing the Rust runtime. - Q5: non-default-topmost status bubble window with runtime status display. - Q6: pet state, safe PNG/WebP manifest schema, monitor geometry, and safe-edge placement model. Validation: Rust workspace tests passed (59 tests), Rust Clippy with warnings denied passed, Rust format check passed, TypeScript check passed, Vitest passed (6 tests), and Vite production build passed. Still not claimed complete: automatic startup activation, real Tauri multi-window manual UX, long-duration reconnect soak, JWT login, attachment protocol, certificate pinning, actual Windows multi-monitor/game-mode integration, voice, and release installer validation. ## 2026-07-20:Q1 启动自动连接增量 - 已完成:在 Tauri 启动阶段读取已保存的 `auto_connect` Profile,并按 Companion / Operations 槽位独立激活。 - 已完成:启动失败保留在对应 Runtime 状态,不阻断另一槽位;凭据仍由 Rust 读取,React 不接触秘密值。 - 已完成:启动自动连接复用现有 AstrBot OpenAPI 与 Desktop Gateway 激活路径。 - 验证:`cargo check --workspace --all-targets`、`cargo test --workspace`(59 项通过)、`cargo clippy --workspace --all-targets -- -D warnings`、`pnpm.cmd run typecheck` 通过。 - 未完成:前端启动连接状态提示、自动重连退避、JWT 登录、聊天布局和快速窗口共享状态完善。 - 当前工作区:本地未提交,等待下一项 Q1/Q2 增量完成后按用户指令提交。 ## 2026-07-20??? 1-6 ?????? - Q1???????????????????????????? - Q2??? SQLite schema v6 ???????????????????????60 ? Rust ????? - Q3?????????????/????/?????????????? Hook ????? - Q4???????????????? URL ??? - Q5????????????/???????????????????????????? - Q6????? JSON manifest ???? UI?????????????PNG/WebP ?????????????? - ?????`2e84ea5`?`5dfac59`?`e0b48c0`?`9f235a3`?`4f13581`?`1828c61` ???? Forgejo? ## 2026-07-20?????????? - ????????????????????????????????????? - ????????????????/????/???????????? - ??????????????????????????????????????????? - ??????????????/?????????????????? Windows ????? - ?????TypeScript?ESLint?Vitest 6/6?Vite production build ??? ## 2026-07-20: chat extensibility contract - Added Rust-owned AstrBot capability discovery, model configuration options, and slot-scoped IPC. - Added a bounded 16 MiB multipart upload boundary and normalized attachment metadata. - Extended chat send requests with optional configId and attachments; the existing dual-slot isolation remains the authority. - Updated ChatPanel with model selection, attachment staging, upload state, and clean WebChat-style text labels. - Validation: Rust workspace tests 60 passed; Vitest 6/6 passed; TypeScript, ESLint, cargo check, and Vite build passed. ## 2026-07-20: live chat extensibility probe - Both supplied containers returned status=ok for chat sessions and config discovery. - Container 1 accepted a minimal multipart upload and returned attachment_id, filename, and type. - No chat message or configuration mutation was performed. Full multimodal rendering remains reserved for an explicit send-message acceptance test. ## 2026-07-20: live end-to-end chat acceptance - First test container accepted text plus config selection and streamed a complete response with no error frame. - The same session accepted text plus config selection plus the uploaded attachment and streamed a complete response with no error frame. - This validates the current model/config and attachment request contract against a real AstrBot instance. ## 2026-07-20: remote session navigation - Added slot-scoped remote session discovery and sticky session switching IPC. - ChatPanel now displays remote sessions and reloads history after switching; message filtering includes sessionId. - Validation: Rust workspace tests, Clippy, TypeScript, ESLint, Vitest, and Vite build passed. ## 2026-07-20: Q1-Q5 shared chat runtime completion - Q1: normalized runtime events are now broadcast at the application level, so the main window and quick-chat window receive the same stream without competing for a single drain queue. - Q2: chat schema v10 includes `slot` in the message key and all history/read IPC queries; streaming deltas append to the stored draft and completed events replace it with the authoritative content. - Q3: main chat sends user messages into durable history after the runtime assigns a session, keeps slot/session filtering isolated, and preserves live assistant updates. - Q4: quick chat continues to reuse the Rust runtime and SQLite history, with persisted geometry and shared live events across windows. - Q5: the status bubble shows connection state and unread totals; the tray menu shows live Companion/Operations states and provides main/quick chat entry points. - Validation: Rust workspace tests pass (67 tests plus protocol/integration suites), frontend typecheck/lint/Vitest/build pass, and touched frontend files are formatted. ## 2026-07-20: Q6-Q8 companion, diagnostics, and interaction MVP - Q6: removed the desktop pet's timer-driven idle/resting placeholder; runtime connection and normalized chat events now drive working, speaking, resting, and error states. - Q6-Q7: added monitor geometry discovery, safe-edge placement, remembered monitor selection, always-on-top control, and game-mode safe-edge fallback without input injection or arbitrary desktop control. - Q7: added bounded read-only diagnostics for the foreground window, a user-selected process ID, and one explicitly requested network host/port. - Q8: added browser Web Speech input and speech synthesis playback on top of the shared chat send/message path, plus proactive-message pet updates and configurable quiet hours for bubble reminders. - Validation: Rust format, Clippy, workspace tests, TypeScript, ESLint, Vitest, and Vite build pass. ## 2026-07-20: Q9 local stability and unsigned release - Added a session marker in the application data directory; an unexpected previous session is reported on the next startup while local profiles and chat history remain recoverable. - Bumped the local release line to `0.1.0-alpha.2` and added the Q9 local acceptance checklist. - Updated the changelog to distinguish the unsigned local installer from future signed/auto-update releases. - Rust workspace tests pass after the version and recovery changes; frontend checks remain green. - Pending manual acceptance: install the unsigned NSIS package, exercise tray/second-launch/network-reconnect/window-restore paths, and verify the recovery marker after a disposable forced termination. ## 2026-07-20: Q4 quick-chat window geometry persistence - Completed: added the `window_states` SQLite table (schema v7) for quick-chat position, size, and monitor identity. - Completed: restore geometry on open; fall back to the current primary monitor and clamp to its work area when a monitor is unavailable or the saved position is invalid. - Completed: save geometry on move, resize, and close-to-tray; keep the quick window single-instance without destroying the shared chat runtime. - Completed: added window bounds, monitor clamping, and database round-trip tests. - Validation: Rust format, Clippy with warnings denied, and `astrbot-desktop` unit tests pass. - Next: finish the remaining quick-window/status-bubble acceptance work, then continue with the pet asset and multi-monitor controller foundation. ## 2026-07-20: status bubble text cleanup - Fixed the remaining mojibake in status-bubble state labels and accessibility text. - Validation: TypeScript, ESLint, and Vitest (6 tests) pass. ## 2026-07-20: quick-chat close behavior fix - Fixed the global close handler so the quick-chat window can be closed normally instead of being intercepted and only hidden. - The main window keeps the existing tray behavior; quick-chat geometry is saved before destruction and restored when reopened. - Validation: Rust Clippy and `astrbot-desktop` unit tests (64) pass. ## 2026-07-20: JWT profile and 2D paper pet foundation - Added JWT as a persisted connection authentication type alongside API Key and device token; SQLite schema is now v8 and migrates existing profiles without losing slot assignments. - Added remote ID validation: 3-32 ASCII letters or digits; the ID remains separate from the user-facing nickname field planned for the profile UI. - Extended the pet-pack manifest with optional animation IDs, loop flags, and bounded movement modes for 2D paper-character actions. - Updated the pet importer copy and JWT selection UI; full Dashboard login, JWT expiry handling, and background pet window controller remain next-stage work. - Validation: Rust workspace tests, TypeScript, ESLint, Vitest (6), and Vite production build pass. ## 2026-07-20: JWT login IPC - Added a Rust-owned login_profile_jwt command for AstrBot Dashboard login at /api/v1/auth/login. - Dashboard username, password, and optional TOTP stay inside Rust; successful JWTs are written to Windows Credential Manager and the Profile is switched to JWT authentication. - Added Profile UI fields for JWT login and retained API Key/device-token enrollment for non-JWT profiles. - Full JWT expiry/401 re-login UX and nickname/remote-ID separation remain next increments. ## 2026-07-20: JWT runtime expiry handling - AstrBot JWT profiles now pass their authentication type through the Rust runtime instead of being treated as API Key-only profiles. - Rust checks the JWT `exp` claim before activation and maps expired credentials to `AuthExpired` without logging or returning the token. - AstrBot 401/403 responses during probe or chat now preserve the affected slot and expose an `authExpired` runtime error. - Chat UI shows a re-login warning for the affected slot; credentials remain managed by Windows Credential Manager. - The floating 2D pet now polls the dual-slot runtime and switches between idle, work, reply, rest, and error states instead of running an isolated demo loop. ## 2026-07-20: Aemeath pixel sprite integration - Reviewed `aemeath_withclaude-2.0.8`; the supplied repository and referenced pixel asset are MIT-licensed, not CC-licensed. - Added the original 8-column, 192x208-frame WebP spritesheet as a project-local pet asset with preserved attribution. - Added a validated pet manifest and mapped the source animation rows into idle, thinking, speaking, notification, error, and offline states. - Replaced the CSS placeholder character in the floating pet view with the real pixel spritesheet renderer. - Validation: Rust workspace tests (65), Clippy, TypeScript, ESLint, Vitest (6), and Vite build pass. ## 2026-07-20: production pet renderer white-window fix - Replaced the floating pet CSS background-image crop with a clipped `` spritesheet renderer for more reliable WebView2/asset-protocol loading. - Added explicit pet-view root sizing and transparent `html`/`body`/`#root` backgrounds so Tauri transparent windows do not fall back to an opaque blank surface. - Validation: TypeScript, ESLint, Vitest (6), Vite build, Rust format, Clippy with warnings denied, Rust workspace tests (66), and NSIS bundle build pass. - The already-running executable must be fully closed and relaunched; the generated installer is at `target/release/bundle/nsis/AstrBot Desktop_0.1.0-alpha.1_x64-setup.exe`. ## 2026-07-20: tray quit reliability fix - Fixed the tray Quit action so it destroys the main, quick-chat, status-bubble, and pet windows before requesting process exit. - This avoids the global close-to-tray handler intercepting shutdown and leaving the process resident. - Validation: Rust format, Clippy with warnings denied, Rust workspace tests (66), TypeScript, ESLint, Vitest (6), and NSIS bundle build pass. ## 2026-07-20: desktop pet rendering and tray exit follow-up - Fixed the pixel spritesheet crop by setting the sprite image transform origin to the top-left; the previous default center origin moved the selected frame outside the clipped viewport and produced a blank transparent pet window. - Added an application exiting state so tray Quit allows all windows to close instead of the global close-to-tray handler hiding them and leaving an empty, non-interactive placeholder. - Validation: frontend typecheck, Vitest (6), Vite production build, Rust format, Rust check, and Rust workspace tests (66) pass. ## 2026-07-20: transparent pet and tray process exit hardening - Replaced the transparent-window spritesheet crop with a PNG-backed Canvas renderer; this avoids WebView2 transparent-window failures with the large WebP image and CSS transform cropping. - Changed tray Quit to call Tauri cleanup and terminate the process immediately, ensuring the tray icon, hidden main window, and pet window cannot remain as orphaned UI surfaces. - Rebuilt and launched the Release executable at 18:24:20 for manual verification. ## 2026-07-20: wire pet display mode and render diagnostics - Connected the pet display-mode setting to persisted local UI state; `app-ui` now embeds the pet in the Pet page, while `desktop-floating` explicitly opens the transparent Tauri window. - Added sprite loading and Canvas failure feedback instead of leaving a silent blank/white surface. - Validation: TypeScript, Vitest (6), Vite build, Rust format/check, and Release bundle build pass. The rebuilt executable started at 18:35:29. ## 2026-07-20: in-app draggable pet overlay - Promoted the app-ui pet from a static page preview to an application-level floating overlay, matching the Codex/Hermes-style in-window companion behavior. - Added pointer dragging with clamped coordinates and local persistence; the overlay can be hidden without creating a native desktop window. - Disabled the broken desktop-floating launch path in the Pet settings and provided a direct switch back to the in-app mode. - Validation: TypeScript, Vitest (6), Vite build, Rust format/check, and Release bundle build pass. The rebuilt executable started at 18:44:43. ## 2026-07-20: restore desktop floating pet transparency - Restored the desktop-floating display option and its `open_pet_window` action; it is no longer disabled. - Fixed the Windows transparent WebView configuration by explicitly setting an RGBA-zero background and disabling undecorated window shadow, which could otherwise render a white rectangle around the pet. - Preserved the app-ui draggable overlay implementation. - Validation: TypeScript, Vitest (6), Rust format/check, Vite build, and Release bundle build pass. The rebuilt executable started at 18:50:25. ## 2026-07-20: unified slot runtime lifecycle hardening - Added independent connection cancellation and message cancellation tokens for each Companion / Operations slot. - Connected the desktop gateway SSE reconnect loop to the active slot lifecycle, including state synchronization and terminal error mapping. - Added generation guards so a late response from a cancelled or replaced slot cannot restore a stale client into the new slot state. - Preserved AstrBot OpenAPI request-stream handling and bounded retry behavior; AstrBot does not expose a separate unsolicited SSE subscription endpoint in the current adapter. - Validation: Rust fmt, workspace check, Rust library tests (66), TypeScript typecheck, and Vitest (6) pass. ## 2026-07-20: compact chat-first main window - Fixed runtime status parsing so Rust null option fields are accepted as an unconnected slot state instead of showing raw Zod validation output. - Made the chat page chat-first: removed the workspace layout selector and model dropdown from the primary page; secondary settings remain in their dedicated navigation pages. - Reduced main-window spacing, added rounded application corners, and preserved the frameless custom title bar. - Validation: TypeScript, ESLint, Vitest (6), Vite build, and Rust workspace check pass. ## 2026-07-21: AstrBot chat complete-frame handling - Fixed the AstrBot SSE mapper to consume `type: "complete"` frames, which carry the authoritative final response text and were previously ignored. - Reconciled final text with partial `plain` chunks and added support for text nested under `text`, `content`, or `message` fields. - Added protocol tests for complete-only replies and partial-stream reconciliation. - Validation: Rust format, Clippy with warnings denied, Rust workspace tests (67), TypeScript, ESLint, and NSIS bundle build pass. ## 2026-07-21: normalize runtime event payloads - Fixed AstrBot runtime event dispatch to expose the inner payload directly instead of nesting it under the protocol envelope. - This restores frontend and database access to `delta`, `content`, and error `message` fields. - Validation: Rust workspace tests, Clippy with warnings denied, TypeScript, ESLint, and NSIS bundle build pass. ## 2026-07-21: Q10-Q15 chat experience increment - Added safe lightweight Markdown rendering for headings, lists, links, inline code, and fenced code blocks. - Added copy, retry, read-aloud, stop-generation, new-chat, refresh, reconnect, disconnect, and runtime-error actions. - Added responsive runtime metadata and chat layout refinements; preserved the existing pet event bus for Q13 state synchronization. - Updated frontend test mocks for runtime status schema changes. - Validation: Vitest (6), TypeScript, ESLint, and targeted Prettier checks pass. ## 2026-07-21: frontend UI audit and chat-page visual increment - Completed the Impeccable audit and recorded the P0-P3 findings in `docs/reports/ui-audit-2026-07-21.md`. - Recorded the P0-P2 redesign queue and preserved contracts in `docs/ui-redesign-plan.md`. - Refined the chat page visual layer in `src/styles.css` only: added shared UI tokens, visible focus states, clearer slot/runtime hierarchy, calmer surfaces, responsive composer layout, and reduced-motion fallback. - No IPC command, event name, route key, profile/slot/session/message schema, or security boundary changed. - Frontend validation after the increment: typecheck passed, lint passed, Vitest 6 passed, Vite build passed; build still reports the existing large JS chunk warning. - Browser preview at 1280x720 confirmed no horizontal overflow on the chat page. In-browser IPC fallback remains expected outside Tauri. ## 2026-07-21: chat page accessibility and polish increment - Added chat slot tab relationships, keyboard left/right switching, selected-tab focus order, message tabpanel semantics, busy state, and alert semantics for send/runtime errors. - Kept the existing chat state, invoke/listen calls, session switching, attachment upload, voice input, retry, copy, read-aloud, and stop-generation interactions unchanged. - Refined chat-only typography, scroll behavior, focus treatment, disabled states, voice active state, file input focusability, touch targets, and narrow-window layout without adding motion. - Frontend validation passed: Prettier on `src/features/chat/ChatPanel.tsx`, typecheck, lint, Vitest 6 tests, and Vite production build. - Browser preview passed at 1280x720 with no horizontal overflow; slot arrow-key switching selected the adjacent tab. Existing pet overlay/navigation overlap remains outside this batch. ## 2026-07-21: P1-P3 frontend hardening increment - Added lazy-loaded chunks for ProfileManager and PetPackImporter, with a status fallback and async test waiting. - Added visible diagnostic field labels, global IPC alert semantics, and a safe desktop-pet left boundary that keeps the main navigation clear on desktop widths. - Consolidated the visual layer through shared UI tokens, Windows-native typography, reduced card shadows, calmer surfaces, consistent focus/disabled/error states, and narrow-window session layout. - Preserved existing routes, page keys, IPC command/event names, connection slots, profile/session/message data, and security boundaries. - Validation: typecheck, lint, Vitest 6 tests, targeted Prettier checks, Vite build, and 1280x720 browser preview passed. Build now emits ProfileManager and PetPackImporter async chunks; the existing large main JS chunk warning remains. - Normalized persisted in-app pet coordinates on mount and window resize so stale positions cannot remain over the sidebar after an upgrade or window-size change. - Final frontend validation after the normalization: targeted Prettier, typecheck, lint, Vitest 6, `git diff --check`, and Vite production build passed. Node 24.14.0 still emits the repository's existing engine warning for the `>=24.17.0` requirement. ## 2026-07-22: complete frontend UI redesign queue P1-P3 - Completed P1 core-page work: ProfileManager credential form labels and feedback, StatusBubble semantic status tones, DesktopBehaviorPage grouping, DiagnosticsPage field labels and async feedback. - Completed P2 code splitting: moved DesktopBehaviorPage and DiagnosticsPage to `src/pages/SettingsPages.tsx`; moved DesktopPetView and sprite loading to `src/features/pet/DesktopPetView.tsx`; kept ProfileManager and PetPackImporter lazy-loaded. - Reduced the initial JavaScript chunk from approximately 5.54 MB raw / 4.02 MB gzip to approximately 305.93 kB raw / 91.43 kB gzip. The pet renderer remains an intentional lazy chunk because its inline sprite asset is approximately 5.23 MB. - Completed P2 consistency work: added shared status colors, numeric typography, responsive 1280/1024/680px rules, reduced-motion fallback, async diagnostic status feedback, visible focus and disabled states, and bounded diagnostic result rendering. - Added `tests/e2e/ui-polish.spec.ts` for desktop shell overflow checks at 1280px, 1024px, and 680px plus diagnostic keyboard accessibility. - Completed P3 polish pass for desktop shell hierarchy, status messaging, profile credentials, diagnostics, desktop pet boundary, empty/error/loading states, and dark/light token usage without adding marketing-style animation. - Validation: targeted Prettier, ESLint, TypeScript, Vitest 6/6, Vite production build, Playwright 5/5, Rust fmt, Clippy with warnings denied, Rust workspace tests (67), and Rust workspace check all passed. - Known non-blocking warnings: Node is `24.14.0` while the package requires `>=24.17.0`; Vite reports the intentional lazy `DesktopPetView` chunk above 500 kB.