65 行
3.3 KiB
Markdown
65 行
3.3 KiB
Markdown
|
|
# P00-03: UI Automation and Selected-Window Capture
|
||
|
|
|
||
|
|
- Status: Passed for a controlled non-elevated standard dialog; elevated/custom-drawn manual coverage remains
|
||
|
|
- Date recorded: 2026-07-18
|
||
|
|
- MVP gate: Required for bounded error-window diagnostics
|
||
|
|
|
||
|
|
## Objective
|
||
|
|
|
||
|
|
Verify read-only extraction from a standard Windows dialog and a user-selected, single-window capture fallback without running the application as administrator or broadening access to the desktop.
|
||
|
|
|
||
|
|
## Environment
|
||
|
|
|
||
|
|
- Host: Windows 11 x64
|
||
|
|
- Target class: standard non-elevated Windows dialog
|
||
|
|
- Rust toolchain: rustc/cargo 1.97.1
|
||
|
|
- MSVC C++ build tools: verified by the Windows test and Tauri build
|
||
|
|
- Elevated target testing: negative boundary only
|
||
|
|
|
||
|
|
## Minimal validation implementation
|
||
|
|
|
||
|
|
The Spike is expected to:
|
||
|
|
|
||
|
|
- Identify the current foreground window.
|
||
|
|
- Return bounded title, process, and UI Automation text fields.
|
||
|
|
- Report unsupported, inaccessible, closed, or elevated targets safely.
|
||
|
|
- Capture only the window explicitly selected by the user.
|
||
|
|
- Keep image bytes in Rust until a display-safe preview is produced.
|
||
|
|
- Delete transient capture data after the diagnostic flow unless the user explicitly saves it.
|
||
|
|
|
||
|
|
## Required execution
|
||
|
|
|
||
|
|
1. Open a standard non-elevated dialog with known text.
|
||
|
|
2. Read its title and UI Automation text.
|
||
|
|
3. Verify an unsupported or custom-drawn window produces a structured fallback result.
|
||
|
|
4. Select one window and capture only its client/window bounds.
|
||
|
|
5. Verify a closed or invalid handle does not capture another window.
|
||
|
|
6. Attempt an elevated target and confirm a safe access-denied result without elevation.
|
||
|
|
7. Confirm transient image data is removed after the test.
|
||
|
|
|
||
|
|
## Actual execution result
|
||
|
|
|
||
|
|
The ignored Windows smoke test opened a test-owned standard MessageBox containing fixed, non-sensitive text. It selected only that foreground HWND, verified title and process identity, read the expected title/body through UI Automation, captured the same selected window through PrintWindow, and validated an in-memory PNG without writing image bytes to disk.
|
||
|
|
|
||
|
|
The test then closed the controlled dialog and verified that the expired HWND identity returned `WindowChanged` and could not capture a replacement window. The test passed in 0.12 seconds.
|
||
|
|
|
||
|
|
An elevated target was not created because that would require a UAC/elevation action outside this bounded Spike. The implementation maps access denied and element-unavailable HRESULTs to structured errors, but a real elevated/custom-drawn target remains an explicit manual boundary test.
|
||
|
|
|
||
|
|
## Log summary
|
||
|
|
|
||
|
|
Command: `cargo test -p astrbot-desktop --test windows_dialog_smoke -- --ignored --nocapture`. Result: 1 passed. Output contained test status only; no extracted text or pixel bytes were logged or persisted.
|
||
|
|
|
||
|
|
## Known limitations
|
||
|
|
|
||
|
|
- UI Automation does not expose all custom-rendered content.
|
||
|
|
- Protected, elevated, DRM, or hardware-accelerated windows may reject inspection or capture.
|
||
|
|
- DPI scaling and multi-monitor coordinates require explicit tests.
|
||
|
|
|
||
|
|
## Security impact
|
||
|
|
|
||
|
|
Window content can contain personal or secret data. Capture must be user-selected, bounded to one target, previewable, and excluded from durable logs.
|
||
|
|
|
||
|
|
## Recommendation
|
||
|
|
|
||
|
|
Keep UI Automation as the preferred text path and selected-window capture as an explicit fallback. Do not add continuous desktop capture, global hooks, input injection, or privilege escalation.
|