feat(awareness): add manual WGC acceptance action
这个提交存在于:
父节点
48599db6d4
当前提交
6634cfe91c
共有 9 个文件被更改,包括 191 次插入 和 0 次删除
|
|
@ -12,6 +12,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
|
|||
- Screen-awareness implementation framework for scoped display consent, WGC capture, local privacy/quality gates, multimodal AstrBot validation, and low-interruption response control.
|
||||
- WGC S0.1 target-binding preflight for authorized windows and attached displays. It returns metadata only; no capture session, pixels, disk write, or upload is performed.
|
||||
- Bounded WGC S0.2 diagnostic readback for an already authorized window: one in-memory D3D11 frame, local blank-frame classification, and discarded PNG conversion metadata only.
|
||||
- Per-session manual WGC acceptance action that displays only local frame-quality metadata and is structurally unable to upload or send an image.
|
||||
- Debug-level tracing of AstrBot SSE frame shapes: frame type, which identity fields are present, and the key names inside `data`. No payload text is recorded. Enable with `RUST_LOG`.
|
||||
- Q10-Q15 chat experience increment: lightweight Markdown rendering, message actions, generation controls, session actions, connection controls, and responsive chat layout.
|
||||
- AstrBot OpenAPI adapter foundation: SSE mapper, dual-slot runtime skeleton, credential enrollment IPC, transport-aware profiles (ADR-0005).
|
||||
|
|
|
|||
|
|
@ -252,6 +252,15 @@ The following are deliberately not marked complete: HTTPS certificate pinning, s
|
|||
|
||||
验收:隔离 Rust check、121 个库测试和 Clippy 均通过。仍缺少实际窗口/游戏、最小化、锁屏、受保护内容、显示器变化的人工证据,故 S0 整体未完成。
|
||||
|
||||
|
||||
### S0.3:手动 WGC 验收入口(2026-07-27,完成;等待真实场景证据)
|
||||
|
||||
- 在“窗口读取”已授权会话卡片内增加 `WGC 本地帧验收(不发送)`;用户必须先经现有 5 秒前台窗口授权,按钮不创建新授权。
|
||||
- 结果只显示尺寸、内存中验证的 PNG 大小和 `usable` / `rejected_blank` 状态;不显示缩略图、不返回图像字节、不上传、不发送消息。
|
||||
- 任何失败继续显示 Rust 的可行动错误;黑帧/透明帧明确表示不支持,不能据此开启游戏陪玩。
|
||||
|
||||
验收:前端 typecheck、61 项 Vitest、生产 Vite 构建及本次文件 Prettier 检查通过;Rust S0.2 检查仍为 121 passed / Clippy 通过。人工场景结果仍是 S0 的阻塞项。
|
||||
|
||||
### C1:聊天、语音聊天、游戏陪玩三页面壳
|
||||
|
||||
- 在现有“聊天”之后增加“语音聊天”“游戏陪玩”两个独立导航入口。
|
||||
|
|
|
|||
|
|
@ -784,3 +784,17 @@ This is not a user-facing capture feature and has no scheduler, settings UI, dis
|
|||
- Isolated Rust check: passed.
|
||||
- Isolated Rust library tests: 121 passed, 0 failed.
|
||||
- Isolated Rust Clippy with warnings denied: passed.
|
||||
|
||||
|
||||
## 2026-07-27: WGC S0.3 manual acceptance entry
|
||||
|
||||
Placed a `WGC 本地帧验收(不发送)` action beside each existing authorized-window session. It reuses the short-lived Rust authorization and returns only width, height, temporary PNG byte count, and usable/blank quality. No preview, image byte array, upload, send request, scheduler, or display-scope expansion was added.
|
||||
|
||||
A component regression proves the action invokes only `wgc_spike_probe_authorized_window_frame`, never the upload or send commands. This makes manual Windows evidence possible without weakening the no-network S0 boundary.
|
||||
|
||||
### Test status
|
||||
|
||||
- Frontend typecheck: passed.
|
||||
- Vitest: 61 passed.
|
||||
- Production Vite build: passed.
|
||||
- Rust S0.2 regression remains 121 passed with Clippy clean.
|
||||
|
|
|
|||
|
|
@ -39,3 +39,12 @@ Implemented a bounded one-frame diagnostic path for an already authorized window
|
|||
Validation: isolated `cargo check -p astrbot-desktop --lib`, `cargo test -p astrbot-desktop --lib` (121 passed), and `cargo clippy -p astrbot-desktop --lib -- -D warnings` all passed.
|
||||
|
||||
Manual evidence is still pending. No active UI invokes the diagnostic command yet, and no game, minimized window, lock screen, protected content, or multi-display run has been claimed as validated. Those observations remain blocking S0 acceptance.
|
||||
|
||||
## S0.3 manual acceptance entry
|
||||
|
||||
The existing authorized-window card in Diagnostics now exposes `WGC 本地帧验收(不发送)`. It is intentionally a per-session manual action rather than a setting or scheduled feature. The UI receives only the probe metadata and renders one of two explicit outcomes:
|
||||
|
||||
- `usable`: the local frame was nonblank, with dimensions and the temporary in-memory PNG byte count.
|
||||
- `rejected_blank`: the local frame was black or transparent, so no game-support conclusion and no outgoing action is permitted.
|
||||
|
||||
The component regression asserts that this action invokes only `wgc_spike_probe_authorized_window_frame`; it cannot call either the AstrBot upload or message-send commands. Frontend validation: TypeScript check, 61 Vitest assertions, and production build passed.
|
||||
|
|
|
|||
11
docs/错题本.md
11
docs/错题本.md
|
|
@ -102,3 +102,14 @@
|
|||
- 修复或规避:注册 `FrameArrived` 事件并只等待 2 秒,到期返回明确的 `FrameTimedOut`;每次退出移除事件并关闭 frame/session/pool,避免留存捕获资源。
|
||||
- 预防规则:所有主动捕获路径必须有帧、尺寸、内存和生命周期上限;超时是可解释拒绝,不得重试成后台观察。
|
||||
- 验证证据:Rust 121 个库测试通过,Clippy 通过;真实 Windows 场景证据仍待人工验收。
|
||||
|
||||
|
||||
## 2026-07-27 — Metadata result and global notice can intentionally repeat safety copy
|
||||
|
||||
- 阶段:S0.3 手动 WGC 验收入口
|
||||
- 类型:测试 / 功能行为
|
||||
- 现象:首版组件回归用唯一文本查询 `未保存、未发送`,但结果卡和全局通知都故意说明同一隐私保证,测试失败。
|
||||
- 根因:断言把可访问的重复安全文案误判为重复渲染;两个位置承担不同作用,前者解释帧结果,后者提供操作反馈。
|
||||
- 修复或规避:改为断言结果卡独有的尺寸元数据,并继续断言没有调用上传或消息发送命令。
|
||||
- 预防规则:涉及隐私/安全说明时,允许局部结果和全局反馈重复关键保证;测试应定位结构化状态或唯一业务数据,而不是假设文案唯一。
|
||||
- 验证证据:前端 61 项测试、类型检查和生产构建通过。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,64 @@
|
|||
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
|
||||
import { invoke } from "@tauri-apps/api/core";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { ScreenAwarenessPanel } from "./ScreenAwarenessPanel";
|
||||
|
||||
vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));
|
||||
|
||||
const session = {
|
||||
sessionToken: "6f1e0d3a-4b5c-4d6e-8f90-1a2b3c4d5e6f",
|
||||
windowTitle: "DirectX game",
|
||||
processId: 4242,
|
||||
slot: "companion",
|
||||
expired: false,
|
||||
};
|
||||
|
||||
const status = {
|
||||
enabled: true,
|
||||
sessions: [session],
|
||||
pixelIntervalMs: 2000,
|
||||
textIntervalMs: 750,
|
||||
sessionTtlSeconds: 300,
|
||||
};
|
||||
|
||||
describe("the WGC Spike action", () => {
|
||||
beforeEach(() => {
|
||||
vi.mocked(invoke).mockReset();
|
||||
vi.mocked(invoke).mockImplementation(async (command) => {
|
||||
if (command === "screen_reading_status") return status;
|
||||
if (command === "wgc_spike_probe_authorized_window_frame") {
|
||||
return {
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
encodedPngBytes: 42_000,
|
||||
quality: "usable",
|
||||
};
|
||||
}
|
||||
throw new Error(`Unexpected command: ${command}`);
|
||||
});
|
||||
});
|
||||
|
||||
it("runs only the metadata diagnostic and never sends the frame", async () => {
|
||||
render(<ScreenAwarenessPanel />);
|
||||
|
||||
const action = await screen.findByRole("button", { name: /WGC/ });
|
||||
fireEvent.click(action);
|
||||
|
||||
await waitFor(() =>
|
||||
expect(invoke).toHaveBeenCalledWith(
|
||||
"wgc_spike_probe_authorized_window_frame",
|
||||
{ sessionToken: session.sessionToken },
|
||||
),
|
||||
);
|
||||
expect(await screen.findByText(/1920 × 1080/)).toBeInTheDocument();
|
||||
expect(invoke).not.toHaveBeenCalledWith(
|
||||
"send_connection_slot_message",
|
||||
expect.anything(),
|
||||
);
|
||||
expect(invoke).not.toHaveBeenCalledWith(
|
||||
"upload_connection_slot_file",
|
||||
expect.anything(),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -7,7 +7,9 @@ import {
|
|||
screenReadingStatusSchema,
|
||||
windowFrameSchema,
|
||||
windowTextSummarySchema,
|
||||
wgcFrameProbeSchema,
|
||||
type ScreenReadingStatus,
|
||||
type WgcFrameProbe,
|
||||
} from "./types";
|
||||
|
||||
/// The window the user is asked to bring forward before the capture starts.
|
||||
|
|
@ -27,6 +29,10 @@ export function ScreenAwarenessPanel() {
|
|||
const [notice, setNotice] = useState<string | null>(null);
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [countdown, setCountdown] = useState(0);
|
||||
const [wgcProbe, setWgcProbe] = useState<{
|
||||
sessionToken: string;
|
||||
frame: WgcFrameProbe;
|
||||
} | null>(null);
|
||||
const countdownRef = useRef<number | null>(null);
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
|
|
@ -149,6 +155,32 @@ export function ScreenAwarenessPanel() {
|
|||
}
|
||||
};
|
||||
|
||||
/// Runs the Spike-only frame path. It returns metadata, not a preview: the
|
||||
/// raw WGC frame and its temporary PNG are destroyed inside Rust before this
|
||||
/// function receives a result.
|
||||
const testWgcFrame = async (sessionToken: string) => {
|
||||
setBusy(true);
|
||||
setError(null);
|
||||
setWgcProbe(null);
|
||||
try {
|
||||
const frame = wgcFrameProbeSchema.parse(
|
||||
await invoke<unknown>("wgc_spike_probe_authorized_window_frame", {
|
||||
sessionToken,
|
||||
}),
|
||||
);
|
||||
setWgcProbe({ sessionToken, frame });
|
||||
setNotice(
|
||||
frame.quality === "usable"
|
||||
? "WGC 本地验收得到非空画面;图像未保存、未发送。"
|
||||
: "WGC 返回黑帧或透明帧,已拒绝;不会保存或发送。",
|
||||
);
|
||||
} catch (reason) {
|
||||
setError(errorMessage(reason));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
const sendFrame = async (sessionToken: string) => {
|
||||
setBusy(true);
|
||||
setError(null);
|
||||
|
|
@ -238,6 +270,13 @@ export function ScreenAwarenessPanel() {
|
|||
>
|
||||
发送窗口文本
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
disabled={busy}
|
||||
onClick={() => void testWgcFrame(session.sessionToken)}
|
||||
>
|
||||
WGC 本地帧验收(不发送)
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
disabled={busy}
|
||||
|
|
@ -253,6 +292,17 @@ export function ScreenAwarenessPanel() {
|
|||
结束授权
|
||||
</button>
|
||||
</div>
|
||||
{wgcProbe?.sessionToken === session.sessionToken ? (
|
||||
<p
|
||||
className="awareness-spike-result"
|
||||
data-quality={wgcProbe.frame.quality}
|
||||
role="status"
|
||||
>
|
||||
{wgcProbe.frame.quality === "usable"
|
||||
? `WGC 已捕获非空帧:${wgcProbe.frame.width} × ${wgcProbe.frame.height},仅在内存中验证了 ${Math.round(wgcProbe.frame.encodedPngBytes / 1024)} KB PNG。未保存、未发送。`
|
||||
: `WGC 返回黑帧或透明帧(${wgcProbe.frame.width} × ${wgcProbe.frame.height}),已拒绝。游戏陪玩不会据此启用。`}
|
||||
</p>
|
||||
) : null}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
|
|
|
|||
|
|
@ -39,6 +39,15 @@ export const windowFrameSchema = z
|
|||
})
|
||||
.strict();
|
||||
|
||||
export const wgcFrameProbeSchema = z
|
||||
.object({
|
||||
width: z.number().int().positive(),
|
||||
height: z.number().int().positive(),
|
||||
encodedPngBytes: z.number().int().nonnegative(),
|
||||
quality: z.enum(["usable", "rejected_blank"]),
|
||||
})
|
||||
.strict();
|
||||
|
||||
export const windowTextSummarySchema = z
|
||||
.object({
|
||||
text: z.string(),
|
||||
|
|
@ -52,4 +61,5 @@ export const windowTextSummarySchema = z
|
|||
export type CaptureSession = z.infer<typeof captureSessionSchema>;
|
||||
export type ScreenReadingStatus = z.infer<typeof screenReadingStatusSchema>;
|
||||
export type WindowFrame = z.infer<typeof windowFrameSchema>;
|
||||
export type WgcFrameProbe = z.infer<typeof wgcFrameProbeSchema>;
|
||||
export type WindowTextSummary = z.infer<typeof windowTextSummarySchema>;
|
||||
|
|
|
|||
|
|
@ -3565,6 +3565,29 @@ button:disabled {
|
|||
gap: 0.4rem;
|
||||
}
|
||||
|
||||
.awareness-spike-result {
|
||||
margin: 0;
|
||||
padding: 0.55rem 0.65rem;
|
||||
border: 1px solid var(--ui-line);
|
||||
border-radius: var(--ui-radius-sm);
|
||||
background: color-mix(in srgb, var(--ui-surface) 72%, #d9a15a);
|
||||
color: var(--ui-ink-soft, #9aa8bd);
|
||||
font-size: 0.82rem;
|
||||
line-height: 1.45;
|
||||
}
|
||||
|
||||
.awareness-spike-result[data-quality="usable"] {
|
||||
border-color: rgba(96, 181, 130, 0.55);
|
||||
background: rgba(96, 181, 130, 0.1);
|
||||
color: #b6e2c5;
|
||||
}
|
||||
|
||||
.awareness-spike-result[data-quality="rejected_blank"] {
|
||||
border-color: rgba(217, 161, 90, 0.55);
|
||||
background: rgba(217, 161, 90, 0.1);
|
||||
color: #e6bd85;
|
||||
}
|
||||
|
||||
.awareness-indicator {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
|
|
|
|||
正在加载…
添加表格
在新工单中引用