feat(awareness): add manual WGC acceptance action
一些检测仍在等待运行
ci / Portable quality checks (push) Waiting to run
ci / Windows Tauri build (push) Waiting to run

这个提交存在于:
YokiiroBW 2026-07-27 15:26:55 +08:00
父节点 48599db6d4
当前提交 6634cfe91c
共有 9 个文件被更改,包括 191 次插入 和 0 次删除

查看文件

@ -12,6 +12,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
- Screen-awareness implementation framework for scoped display consent, WGC capture, local privacy/quality gates, multimodal AstrBot validation, and low-interruption response control.
- WGC S0.1 target-binding preflight for authorized windows and attached displays. It returns metadata only; no capture session, pixels, disk write, or upload is performed.
- Bounded WGC S0.2 diagnostic readback for an already authorized window: one in-memory D3D11 frame, local blank-frame classification, and discarded PNG conversion metadata only.
- Per-session manual WGC acceptance action that displays only local frame-quality metadata and is structurally unable to upload or send an image.
- Debug-level tracing of AstrBot SSE frame shapes: frame type, which identity fields are present, and the key names inside `data`. No payload text is recorded. Enable with `RUST_LOG`.
- Q10-Q15 chat experience increment: lightweight Markdown rendering, message actions, generation controls, session actions, connection controls, and responsive chat layout.
- AstrBot OpenAPI adapter foundation: SSE mapper, dual-slot runtime skeleton, credential enrollment IPC, transport-aware profiles (ADR-0005).

查看文件

@ -252,6 +252,15 @@ The following are deliberately not marked complete: HTTPS certificate pinning, s
验收:隔离 Rust check、121 个库测试和 Clippy 均通过。仍缺少实际窗口/游戏、最小化、锁屏、受保护内容、显示器变化的人工证据,故 S0 整体未完成。
### S0.3:手动 WGC 验收入口(2026-07-27,完成;等待真实场景证据)
- 在“窗口读取”已授权会话卡片内增加 `WGC 本地帧验收(不发送)`;用户必须先经现有 5 秒前台窗口授权,按钮不创建新授权。
- 结果只显示尺寸、内存中验证的 PNG 大小和 `usable` / `rejected_blank` 状态;不显示缩略图、不返回图像字节、不上传、不发送消息。
- 任何失败继续显示 Rust 的可行动错误;黑帧/透明帧明确表示不支持,不能据此开启游戏陪玩。
验收:前端 typecheck、61 项 Vitest、生产 Vite 构建及本次文件 Prettier 检查通过;Rust S0.2 检查仍为 121 passed / Clippy 通过。人工场景结果仍是 S0 的阻塞项。
### C1:聊天、语音聊天、游戏陪玩三页面壳
- 在现有“聊天”之后增加“语音聊天”“游戏陪玩”两个独立导航入口。

查看文件

@ -784,3 +784,17 @@ This is not a user-facing capture feature and has no scheduler, settings UI, dis
- Isolated Rust check: passed.
- Isolated Rust library tests: 121 passed, 0 failed.
- Isolated Rust Clippy with warnings denied: passed.
## 2026-07-27: WGC S0.3 manual acceptance entry
Placed a `WGC 本地帧验收(不发送)` action beside each existing authorized-window session. It reuses the short-lived Rust authorization and returns only width, height, temporary PNG byte count, and usable/blank quality. No preview, image byte array, upload, send request, scheduler, or display-scope expansion was added.
A component regression proves the action invokes only `wgc_spike_probe_authorized_window_frame`, never the upload or send commands. This makes manual Windows evidence possible without weakening the no-network S0 boundary.
### Test status
- Frontend typecheck: passed.
- Vitest: 61 passed.
- Production Vite build: passed.
- Rust S0.2 regression remains 121 passed with Clippy clean.

查看文件

@ -39,3 +39,12 @@ Implemented a bounded one-frame diagnostic path for an already authorized window
Validation: isolated `cargo check -p astrbot-desktop --lib`, `cargo test -p astrbot-desktop --lib` (121 passed), and `cargo clippy -p astrbot-desktop --lib -- -D warnings` all passed.
Manual evidence is still pending. No active UI invokes the diagnostic command yet, and no game, minimized window, lock screen, protected content, or multi-display run has been claimed as validated. Those observations remain blocking S0 acceptance.
## S0.3 manual acceptance entry
The existing authorized-window card in Diagnostics now exposes `WGC 本地帧验收(不发送)`. It is intentionally a per-session manual action rather than a setting or scheduled feature. The UI receives only the probe metadata and renders one of two explicit outcomes:
- `usable`: the local frame was nonblank, with dimensions and the temporary in-memory PNG byte count.
- `rejected_blank`: the local frame was black or transparent, so no game-support conclusion and no outgoing action is permitted.
The component regression asserts that this action invokes only `wgc_spike_probe_authorized_window_frame`; it cannot call either the AstrBot upload or message-send commands. Frontend validation: TypeScript check, 61 Vitest assertions, and production build passed.

查看文件

@ -102,3 +102,14 @@
- 修复或规避:注册 `FrameArrived` 事件并只等待 2 秒,到期返回明确的 `FrameTimedOut`;每次退出移除事件并关闭 frame/session/pool,避免留存捕获资源。
- 预防规则:所有主动捕获路径必须有帧、尺寸、内存和生命周期上限;超时是可解释拒绝,不得重试成后台观察。
- 验证证据:Rust 121 个库测试通过,Clippy 通过;真实 Windows 场景证据仍待人工验收。
## 2026-07-27 — Metadata result and global notice can intentionally repeat safety copy
- 阶段:S0.3 手动 WGC 验收入口
- 类型:测试 / 功能行为
- 现象:首版组件回归用唯一文本查询 `未保存、未发送`,但结果卡和全局通知都故意说明同一隐私保证,测试失败。
- 根因:断言把可访问的重复安全文案误判为重复渲染;两个位置承担不同作用,前者解释帧结果,后者提供操作反馈。
- 修复或规避:改为断言结果卡独有的尺寸元数据,并继续断言没有调用上传或消息发送命令。
- 预防规则:涉及隐私/安全说明时,允许局部结果和全局反馈重复关键保证;测试应定位结构化状态或唯一业务数据,而不是假设文案唯一。
- 验证证据:前端 61 项测试、类型检查和生产构建通过。

查看文件

@ -0,0 +1,64 @@
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
import { invoke } from "@tauri-apps/api/core";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { ScreenAwarenessPanel } from "./ScreenAwarenessPanel";
vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));
const session = {
sessionToken: "6f1e0d3a-4b5c-4d6e-8f90-1a2b3c4d5e6f",
windowTitle: "DirectX game",
processId: 4242,
slot: "companion",
expired: false,
};
const status = {
enabled: true,
sessions: [session],
pixelIntervalMs: 2000,
textIntervalMs: 750,
sessionTtlSeconds: 300,
};
describe("the WGC Spike action", () => {
beforeEach(() => {
vi.mocked(invoke).mockReset();
vi.mocked(invoke).mockImplementation(async (command) => {
if (command === "screen_reading_status") return status;
if (command === "wgc_spike_probe_authorized_window_frame") {
return {
width: 1920,
height: 1080,
encodedPngBytes: 42_000,
quality: "usable",
};
}
throw new Error(`Unexpected command: ${command}`);
});
});
it("runs only the metadata diagnostic and never sends the frame", async () => {
render(<ScreenAwarenessPanel />);
const action = await screen.findByRole("button", { name: /WGC/ });
fireEvent.click(action);
await waitFor(() =>
expect(invoke).toHaveBeenCalledWith(
"wgc_spike_probe_authorized_window_frame",
{ sessionToken: session.sessionToken },
),
);
expect(await screen.findByText(/1920 × 1080/)).toBeInTheDocument();
expect(invoke).not.toHaveBeenCalledWith(
"send_connection_slot_message",
expect.anything(),
);
expect(invoke).not.toHaveBeenCalledWith(
"upload_connection_slot_file",
expect.anything(),
);
});
});

查看文件

@ -7,7 +7,9 @@ import {
screenReadingStatusSchema,
windowFrameSchema,
windowTextSummarySchema,
wgcFrameProbeSchema,
type ScreenReadingStatus,
type WgcFrameProbe,
} from "./types";
/// The window the user is asked to bring forward before the capture starts.
@ -27,6 +29,10 @@ export function ScreenAwarenessPanel() {
const [notice, setNotice] = useState<string | null>(null);
const [busy, setBusy] = useState(false);
const [countdown, setCountdown] = useState(0);
const [wgcProbe, setWgcProbe] = useState<{
sessionToken: string;
frame: WgcFrameProbe;
} | null>(null);
const countdownRef = useRef<number | null>(null);
const refresh = useCallback(async () => {
@ -149,6 +155,32 @@ export function ScreenAwarenessPanel() {
}
};
/// Runs the Spike-only frame path. It returns metadata, not a preview: the
/// raw WGC frame and its temporary PNG are destroyed inside Rust before this
/// function receives a result.
const testWgcFrame = async (sessionToken: string) => {
setBusy(true);
setError(null);
setWgcProbe(null);
try {
const frame = wgcFrameProbeSchema.parse(
await invoke<unknown>("wgc_spike_probe_authorized_window_frame", {
sessionToken,
}),
);
setWgcProbe({ sessionToken, frame });
setNotice(
frame.quality === "usable"
? "WGC 本地验收得到非空画面;图像未保存、未发送。"
: "WGC 返回黑帧或透明帧,已拒绝;不会保存或发送。",
);
} catch (reason) {
setError(errorMessage(reason));
} finally {
setBusy(false);
}
};
const sendFrame = async (sessionToken: string) => {
setBusy(true);
setError(null);
@ -238,6 +270,13 @@ export function ScreenAwarenessPanel() {
>
发送窗口文本
</button>
<button
type="button"
disabled={busy}
onClick={() => void testWgcFrame(session.sessionToken)}
>
WGC 本地帧验收(不发送)
</button>
<button
type="button"
disabled={busy}
@ -253,6 +292,17 @@ export function ScreenAwarenessPanel() {
结束授权
</button>
</div>
{wgcProbe?.sessionToken === session.sessionToken ? (
<p
className="awareness-spike-result"
data-quality={wgcProbe.frame.quality}
role="status"
>
{wgcProbe.frame.quality === "usable"
? `WGC 已捕获非空帧:${wgcProbe.frame.width} × ${wgcProbe.frame.height},仅在内存中验证了 ${Math.round(wgcProbe.frame.encodedPngBytes / 1024)} KB PNG。未保存、未发送。`
: `WGC 返回黑帧或透明帧(${wgcProbe.frame.width} × ${wgcProbe.frame.height}),已拒绝。游戏陪玩不会据此启用。`}
</p>
) : null}
</li>
))}
</ul>

查看文件

@ -39,6 +39,15 @@ export const windowFrameSchema = z
})
.strict();
export const wgcFrameProbeSchema = z
.object({
width: z.number().int().positive(),
height: z.number().int().positive(),
encodedPngBytes: z.number().int().nonnegative(),
quality: z.enum(["usable", "rejected_blank"]),
})
.strict();
export const windowTextSummarySchema = z
.object({
text: z.string(),
@ -52,4 +61,5 @@ export const windowTextSummarySchema = z
export type CaptureSession = z.infer<typeof captureSessionSchema>;
export type ScreenReadingStatus = z.infer<typeof screenReadingStatusSchema>;
export type WindowFrame = z.infer<typeof windowFrameSchema>;
export type WgcFrameProbe = z.infer<typeof wgcFrameProbeSchema>;
export type WindowTextSummary = z.infer<typeof windowTextSummarySchema>;

查看文件

@ -3565,6 +3565,29 @@ button:disabled {
gap: 0.4rem;
}
.awareness-spike-result {
margin: 0;
padding: 0.55rem 0.65rem;
border: 1px solid var(--ui-line);
border-radius: var(--ui-radius-sm);
background: color-mix(in srgb, var(--ui-surface) 72%, #d9a15a);
color: var(--ui-ink-soft, #9aa8bd);
font-size: 0.82rem;
line-height: 1.45;
}
.awareness-spike-result[data-quality="usable"] {
border-color: rgba(96, 181, 130, 0.55);
background: rgba(96, 181, 130, 0.1);
color: #b6e2c5;
}
.awareness-spike-result[data-quality="rejected_blank"] {
border-color: rgba(217, 161, 90, 0.55);
background: rgba(217, 161, 90, 0.1);
color: #e6bd85;
}
.awareness-indicator {
display: flex;
align-items: center;