AstrBot-Desktop/docs/reports/wgc-s0-target-binding-2026-07-27.md
YokiiroBW 6634cfe91c
一些检测仍在等待运行
ci / Portable quality checks (push) Waiting to run
ci / Windows Tauri build (push) Waiting to run
feat(awareness): add manual WGC acceptance action
2026-07-27 15:26:55 +08:00

50 行
4 KiB
Markdown

# WGC S0.1 Target-Binding Spike — 2026-07-27
## Purpose
Establish the narrowest possible evidence that the Windows client can bind explicit capture targets through Windows.Graphics.Capture (WGC), without starting capture or processing screen content. This is an S0 sub-step, not a delivered screen-awareness feature.
## Implemented boundary
- `wgc_spike_support_status` reports only whether WGC is supported and the current attached-display count.
- `wgc_spike_probe_authorized_window` reuses the existing `CapturePolicy::inspect` gate and creates a WGC item only for an already authorized window.
- `wgc_spike_probe_displays` creates WGC items for the displays attached at call time.
- Each successful probe returns target kind plus width and height. Invalid or non-positive dimensions are rejected.
No command creates a `GraphicsCaptureSession`, retrieves a frame, reads text, consumes capture budget, writes raw content, encodes an image, schedules an observation, or calls AstrBot.
## Validation
- `cargo fmt --all`: passed.
- With `CARGO_TARGET_DIR=src-tauri/.tmp/wgc-check`, `cargo check -p astrbot-desktop --lib`: passed.
- With the same isolated target, `cargo test -p astrbot-desktop --lib`: 117 passed, 0 failed.
The isolated target is required in this workspace because the default Cargo target resolves generated Tauri permissions through the mapped UNC path and can fail with Windows access denied (`os error 5`).
## Non-evidence and next gate
S0.1 provides no proof that WGC returns useful pixels. It does not validate DirectX/Vulkan games, minimized targets, display topology changes, lock screens, protected content, sensitive-application filtering, quality thresholds, or AstrBot visual understanding.
S0.2 must create a bounded D3D11 WGC frame path, copy one frame to a staging texture, convert BGRA to RGBA, detect black/transparent/empty results, and record manual Windows evidence before any screen frame can progress to a local privacy or network path.
## S0.2 implementation status
Implemented a bounded one-frame diagnostic path for an already authorized window:
- Creates a hardware D3D11 device with BGRA support, wraps it as a WinRT `IDirect3DDevice`, and uses a free-threaded WGC frame pool with one buffer and a bounded `FrameArrived` event wait.
- Waits for one frame for at most two seconds, copies the source texture to a CPU-readable staging texture, strips row padding, and converts BGRA to RGBA.
- Applies a 16 megapixel ceiling, rejects malformed mapped rows, samples alpha/luminance to classify black or transparent frames as `rejected_blank`, and encodes a PNG only in memory to validate the conversion path.
- Returns only width, height, encoded byte count and quality. The frame and PNG bytes are discarded before the Tauri command returns; nothing is persisted, scheduled, or sent to AstrBot.
Validation: isolated `cargo check -p astrbot-desktop --lib`, `cargo test -p astrbot-desktop --lib` (121 passed), and `cargo clippy -p astrbot-desktop --lib -- -D warnings` all passed.
Manual evidence is still pending. No active UI invokes the diagnostic command yet, and no game, minimized window, lock screen, protected content, or multi-display run has been claimed as validated. Those observations remain blocking S0 acceptance.
## S0.3 manual acceptance entry
The existing authorized-window card in Diagnostics now exposes `WGC 本地帧验收(不发送)`. It is intentionally a per-session manual action rather than a setting or scheduled feature. The UI receives only the probe metadata and renders one of two explicit outcomes:
- `usable`: the local frame was nonblank, with dimensions and the temporary in-memory PNG byte count.
- `rejected_blank`: the local frame was black or transparent, so no game-support conclusion and no outgoing action is permitted.
The component regression asserts that this action invokes only `wgc_spike_probe_authorized_window_frame`; it cannot call either the AstrBot upload or message-send commands. Frontend validation: TypeScript check, 61 Vitest assertions, and production build passed.