AstrBot-Desktop/docs/reports/wgc-s0-target-binding-2026-07-27.md

61 行
5.1 KiB
Markdown

# WGC S0.1 Target-Binding Spike — 2026-07-27
## Purpose
Establish the narrowest possible evidence that the Windows client can bind explicit capture targets through Windows.Graphics.Capture (WGC), without starting capture or processing screen content. This is an S0 sub-step, not a delivered screen-awareness feature.
## Implemented boundary
- `wgc_spike_support_status` reports only whether WGC is supported and the current attached-display count.
- `wgc_spike_probe_authorized_window` reuses the existing `CapturePolicy::inspect` gate and creates a WGC item only for an already authorized window.
- `wgc_spike_probe_displays` creates WGC items for the displays attached at call time.
- Each successful probe returns target kind plus width and height. Invalid or non-positive dimensions are rejected.
No command creates a `GraphicsCaptureSession`, retrieves a frame, reads text, consumes capture budget, writes raw content, encodes an image, schedules an observation, or calls AstrBot.
## Validation
- `cargo fmt --all`: passed.
- With `CARGO_TARGET_DIR=src-tauri/.tmp/wgc-check`, `cargo check -p astrbot-desktop --lib`: passed.
- With the same isolated target, `cargo test -p astrbot-desktop --lib`: 117 passed, 0 failed.
The isolated target is required in this workspace because the default Cargo target resolves generated Tauri permissions through the mapped UNC path and can fail with Windows access denied (`os error 5`).
## Non-evidence and next gate
S0.1 provides no proof that WGC returns useful pixels. It does not validate DirectX/Vulkan games, minimized targets, display topology changes, lock screens, protected content, sensitive-application filtering, quality thresholds, or AstrBot visual understanding.
S0.2 must create a bounded D3D11 WGC frame path, copy one frame to a staging texture, convert BGRA to RGBA, detect black/transparent/empty results, and record manual Windows evidence before any screen frame can progress to a local privacy or network path.
## S0.2 implementation status
Implemented a bounded one-frame diagnostic path for an already authorized window:
- Creates a hardware D3D11 device with BGRA support, wraps it as a WinRT `IDirect3DDevice`, and uses a free-threaded WGC frame pool with one buffer and a bounded `FrameArrived` event wait.
- Waits for one frame for at most two seconds, copies the source texture to a CPU-readable staging texture, strips row padding, and converts BGRA to RGBA.
- Applies a 16 megapixel ceiling, rejects malformed mapped rows, samples alpha/luminance to classify black or transparent frames as `rejected_blank`, and encodes a PNG only in memory to validate the conversion path.
- Returns only width, height, encoded byte count and quality. The frame and PNG bytes are discarded before the Tauri command returns; nothing is persisted, scheduled, or sent to AstrBot.
Validation: isolated `cargo check -p astrbot-desktop --lib`, `cargo test -p astrbot-desktop --lib` (121 passed), and `cargo clippy -p astrbot-desktop --lib -- -D warnings` all passed.
Manual evidence is still pending. No active UI invokes the diagnostic command yet, and no game, minimized window, lock screen, protected content, or multi-display run has been claimed as validated. Those observations remain blocking S0 acceptance.
## S0.3 manual acceptance entry
The existing authorized-window card in Diagnostics now exposes `WGC 本地帧验收(不发送)`. It is intentionally a per-session manual action rather than a setting or scheduled feature. The UI receives only the probe metadata and renders one of two explicit outcomes:
- `usable`: the local frame was nonblank, with dimensions and the temporary in-memory PNG byte count.
- `rejected_blank`: the local frame was black or transparent, so no game-support conclusion and no outgoing action is permitted.
The component regression asserts that this action invokes only `wgc_spike_probe_authorized_window_frame`; it cannot call either the AstrBot upload or message-send commands. Frontend validation: TypeScript check, 61 Vitest assertions, and production build passed.
## S0.4 manual ordinary-window evidence
- Date: 2026-07-27.
- Target: an explicitly authorized window titled `ChatGPT` (PID `36516`).
- Result shown by the shipped manual action: `usable` / nonblank WGC frame, `3490 × 1889`, with a temporary in-memory PNG size of `4311 KB`.
- Boundary observed: the UI stated that the PNG was validated only in memory and was neither saved nor sent. The action is structurally limited to the local WGC probe command.
- Evidence source: user-supplied test screenshot in this task. The screenshot is deliberately not copied into the repository so unrelated screen content is not added to version control.
This is positive evidence for an ordinary, explicitly authorized window only. It does **not** establish DirectX/Vulkan game compatibility, minimized-window behavior, lock-screen behavior, protected-content handling, sensitive-application filtering, monitor capture, all-display capture, topology revocation, scheduler behavior, or AstrBot visual acceptance. S0 remains open until those required gates receive their own evidence or an explicit supported/unsupported outcome.