AstrBot-Desktop/docs/reports/wgc-s0-target-binding-2026-07-27.md
YokiiroBW 48599db6d4
一些检测仍在等待运行
ci / Portable quality checks (push) Waiting to run
ci / Windows Tauri build (push) Waiting to run
feat(awareness): add WGC frame quality probe
2026-07-27 15:08:28 +08:00

3.2 KiB

WGC S0.1 Target-Binding Spike — 2026-07-27

Purpose

Establish the narrowest possible evidence that the Windows client can bind explicit capture targets through Windows.Graphics.Capture (WGC), without starting capture or processing screen content. This is an S0 sub-step, not a delivered screen-awareness feature.

Implemented boundary

  • wgc_spike_support_status reports only whether WGC is supported and the current attached-display count.
  • wgc_spike_probe_authorized_window reuses the existing CapturePolicy::inspect gate and creates a WGC item only for an already authorized window.
  • wgc_spike_probe_displays creates WGC items for the displays attached at call time.
  • Each successful probe returns target kind plus width and height. Invalid or non-positive dimensions are rejected.

No command creates a GraphicsCaptureSession, retrieves a frame, reads text, consumes capture budget, writes raw content, encodes an image, schedules an observation, or calls AstrBot.

Validation

  • cargo fmt --all: passed.
  • With CARGO_TARGET_DIR=src-tauri/.tmp/wgc-check, cargo check -p astrbot-desktop --lib: passed.
  • With the same isolated target, cargo test -p astrbot-desktop --lib: 117 passed, 0 failed.

The isolated target is required in this workspace because the default Cargo target resolves generated Tauri permissions through the mapped UNC path and can fail with Windows access denied (os error 5).

Non-evidence and next gate

S0.1 provides no proof that WGC returns useful pixels. It does not validate DirectX/Vulkan games, minimized targets, display topology changes, lock screens, protected content, sensitive-application filtering, quality thresholds, or AstrBot visual understanding.

S0.2 must create a bounded D3D11 WGC frame path, copy one frame to a staging texture, convert BGRA to RGBA, detect black/transparent/empty results, and record manual Windows evidence before any screen frame can progress to a local privacy or network path.

S0.2 implementation status

Implemented a bounded one-frame diagnostic path for an already authorized window:

  • Creates a hardware D3D11 device with BGRA support, wraps it as a WinRT IDirect3DDevice, and uses a free-threaded WGC frame pool with one buffer and a bounded FrameArrived event wait.
  • Waits for one frame for at most two seconds, copies the source texture to a CPU-readable staging texture, strips row padding, and converts BGRA to RGBA.
  • Applies a 16 megapixel ceiling, rejects malformed mapped rows, samples alpha/luminance to classify black or transparent frames as rejected_blank, and encodes a PNG only in memory to validate the conversion path.
  • Returns only width, height, encoded byte count and quality. The frame and PNG bytes are discarded before the Tauri command returns; nothing is persisted, scheduled, or sent to AstrBot.

Validation: isolated cargo check -p astrbot-desktop --lib, cargo test -p astrbot-desktop --lib (121 passed), and cargo clippy -p astrbot-desktop --lib -- -D warnings all passed.

Manual evidence is still pending. No active UI invokes the diagnostic command yet, and no game, minimized window, lock screen, protected content, or multi-display run has been claimed as validated. Those observations remain blocking S0 acceptance.