AstrBot-Desktop/docs/reports/wgc-s0-target-binding-2026-07-27.md
YokiiroBW 6634cfe91c
一些检测仍在等待运行
ci / Portable quality checks (push) Waiting to run
ci / Windows Tauri build (push) Waiting to run
feat(awareness): add manual WGC acceptance action
2026-07-27 15:26:55 +08:00

4 KiB

WGC S0.1 Target-Binding Spike — 2026-07-27

Purpose

Establish the narrowest possible evidence that the Windows client can bind explicit capture targets through Windows.Graphics.Capture (WGC), without starting capture or processing screen content. This is an S0 sub-step, not a delivered screen-awareness feature.

Implemented boundary

  • wgc_spike_support_status reports only whether WGC is supported and the current attached-display count.
  • wgc_spike_probe_authorized_window reuses the existing CapturePolicy::inspect gate and creates a WGC item only for an already authorized window.
  • wgc_spike_probe_displays creates WGC items for the displays attached at call time.
  • Each successful probe returns target kind plus width and height. Invalid or non-positive dimensions are rejected.

No command creates a GraphicsCaptureSession, retrieves a frame, reads text, consumes capture budget, writes raw content, encodes an image, schedules an observation, or calls AstrBot.

Validation

  • cargo fmt --all: passed.
  • With CARGO_TARGET_DIR=src-tauri/.tmp/wgc-check, cargo check -p astrbot-desktop --lib: passed.
  • With the same isolated target, cargo test -p astrbot-desktop --lib: 117 passed, 0 failed.

The isolated target is required in this workspace because the default Cargo target resolves generated Tauri permissions through the mapped UNC path and can fail with Windows access denied (os error 5).

Non-evidence and next gate

S0.1 provides no proof that WGC returns useful pixels. It does not validate DirectX/Vulkan games, minimized targets, display topology changes, lock screens, protected content, sensitive-application filtering, quality thresholds, or AstrBot visual understanding.

S0.2 must create a bounded D3D11 WGC frame path, copy one frame to a staging texture, convert BGRA to RGBA, detect black/transparent/empty results, and record manual Windows evidence before any screen frame can progress to a local privacy or network path.

S0.2 implementation status

Implemented a bounded one-frame diagnostic path for an already authorized window:

  • Creates a hardware D3D11 device with BGRA support, wraps it as a WinRT IDirect3DDevice, and uses a free-threaded WGC frame pool with one buffer and a bounded FrameArrived event wait.
  • Waits for one frame for at most two seconds, copies the source texture to a CPU-readable staging texture, strips row padding, and converts BGRA to RGBA.
  • Applies a 16 megapixel ceiling, rejects malformed mapped rows, samples alpha/luminance to classify black or transparent frames as rejected_blank, and encodes a PNG only in memory to validate the conversion path.
  • Returns only width, height, encoded byte count and quality. The frame and PNG bytes are discarded before the Tauri command returns; nothing is persisted, scheduled, or sent to AstrBot.

Validation: isolated cargo check -p astrbot-desktop --lib, cargo test -p astrbot-desktop --lib (121 passed), and cargo clippy -p astrbot-desktop --lib -- -D warnings all passed.

Manual evidence is still pending. No active UI invokes the diagnostic command yet, and no game, minimized window, lock screen, protected content, or multi-display run has been claimed as validated. Those observations remain blocking S0 acceptance.

S0.3 manual acceptance entry

The existing authorized-window card in Diagnostics now exposes WGC 本地帧验收(不发送). It is intentionally a per-session manual action rather than a setting or scheduled feature. The UI receives only the probe metadata and renders one of two explicit outcomes:

  • usable: the local frame was nonblank, with dimensions and the temporary in-memory PNG byte count.
  • rejected_blank: the local frame was black or transparent, so no game-support conclusion and no outgoing action is permitted.

The component regression asserts that this action invokes only wgc_spike_probe_authorized_window_frame; it cannot call either the AstrBot upload or message-send commands. Frontend validation: TypeScript check, 61 Vitest assertions, and production build passed.