feat(policy): gate screen reading behind consent, a rate limit, and a switch
一些检测仍在等待运行
ci / Portable quality checks (push) Waiting to run
ci / Windows Tauri build (push) Waiting to run

implementation-queue.md:129 requires an authorization prompt, a frequency
limit, and a privacy switch to exist before screenshots or window
summaries are wired to anything. None of the three existed anywhere in
the repository, so this is them, plus ADR-0008 recording what the
authoritative documents actually permit.

The ADR corrects a mis-reading that would have decided the feature the
wrong way. 01_Architecture_Design.md:250-261 is titled 「不应放在前端的
能力」, so the 「原始截图捕获」 at :259 forbids screenshot capture in
React — it is not a product-level ban, which is how an earlier audit read
it before concluding screen reading was out of scope. The product
affirmatively permits 「用户授权的单次窗口截图」 (:739) and, for phase 7,
「手动和低频截图」 (:1424). What it does exclude is 「实时全双工语音」,
「唤醒词」 and 「高频视频分析」 (:1349-1351).

The module owns all three gates so none of them depends on the frontend
being correct. Reading takes a closure, so the window handle never leaves
the module and there is no accessor that skips the switch, the session,
the expiry check, or the budget. The switch defaults to off and clearing
it drops live sessions, so flipping it mid-capture stops the next read
rather than only preventing new authorizations. `now` is a parameter to
the budget rather than read inside it, which is what makes the interval
testable without sleeping.

Two decisions worth naming. The pixel interval defaults to two seconds
with a 500ms floor no caller can lower — three orders of magnitude away
from a video frame rate, because high-frequency video analysis is out of
scope and what phase 7 asks for is manual, low-frequency capture. And a
session cannot be bound to the operations slot at all, refused at
creation rather than at each read, because :1077 says the operations slot
does not automatically receive game screenshots, window state, or game
events — and a session is exactly that automatic delivery. The slot rule
is a free function so it is asserted directly rather than through a live
window.

Also recorded, not worked around: capture.rs uses PrintWindow over GDI,
which does not capture a hardware-accelerated game surface. Game
companionship needs Windows.Graphics.Capture before it works on the case
it exists for, and a black frame delivered to the model would be worse
than an error.

Rust tests 101 -> 110.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
这个提交存在于:
YokiiroBW 2026-07-27 06:22:15 +08:00
父节点 5b518d1181
当前提交 7aead6fa48
共有 3 个文件被更改,包括 555 次插入 和 0 次删除

查看文件

@ -0,0 +1,127 @@
# ADR-0008: Screen awareness, voice capture, and the game companion boundary
## Context
Voice chat, window awareness, and game companionship are in scope for `v0.3.0-alpha.1`, and the Rust
modules for reading the foreground window (`foreground_window.rs`), walking its UI Automation tree
(`uia.rs`), and capturing a selected window (`capture.rs`) are already written. None of them has a
Tauri command, so nothing in the product can reach them, and there is no consent, rate-limit, or
privacy concept anywhere in the repository — not in `policy.rs`, not in `runtime_guards.rs`, not in
the capability files, not in the database schema, not in the UI.
The authoritative documents were read directly for this decision, because an earlier reading of them
was wrong in a way that would have changed the outcome:
- `01_Architecture_Design.md:250-261` is titled 「不应放在前端的能力」. The 「原始截图捕获」 at
line 259 is therefore a **frontend** exclusion — screenshot capture must not live in React — and
**not** a product-level ban. An earlier audit cited it as the latter and concluded that screen
reading was out of scope. That was a mis-read.
- What the product affirmatively permits: 「用户授权的单次窗口截图」 (`:739`, among the inputs a
diagnostic may gather) and, for phase 7, 「按键说话;TTS;游戏窗口绑定;时长事件;手动和低频截图」
(`:1418-1424`).
- What it excludes outright: 「实时全双工语音」, 「唤醒词」, and 「高频视频分析」 (`:1349-1351`).
`AGENTS.md:32` excludes 「无限制桌面控制」.
- The voice flow is fixed at `:1057-1063` as 按键录音 → 上传音频 → STT → 文本回复 → 可选 TTS, with
the microphone bound to the focused instance by default (`:1065`) and one recording never sent to
both slots unless the user explicitly asks (`:1071`).
- The slot isolation rule is one sentence at `:1077`: 「运维 Slot 不自动接收游戏截图、窗口状态或游戏
事件。」
- `implementation-queue.md:129` already states the gate: 「游戏模式接入截图/状态摘要前,先完成授权
提示、频率限制和隐私开关。」
So the policy does not have to be invented. It has to be written down precisely enough to implement,
and one thing genuinely is missing from every document: whether the user must be able to see that
the application is reading their screen.
## Decision
1. **Screen and window reading happens only inside an explicit capture session.** A session is
started by a user action, names exactly one target window, and carries an expiry. No command reads
a window, its UI Automation tree, or its pixels outside a live session. There is no ambient or
background reading, and no path that reads the desktop or a monitor rather than a window.
2. **The session is the authorization record, and it lives in Rust.** The frontend may ask to start
one and may end one; it cannot name a window that no session covers. `WindowSelection` currently
exists only inside `inspect_foreground_window` and is dropped when that command returns, so a
registry in Tauri state is the prerequisite for any of this, not an optimization.
3. **Every read is rate limited by the session, in Rust.** The limit is a minimum interval between
reads of the same kind, enforced where the read happens rather than in the caller. Frontend
throttling does not count: `implementation-queue.md:129` requires a frequency limit before
screenshots are wired at all, and a limit the renderer can skip is not one. The default interval
for pixel capture is deliberately far above a video frame rate, since 「高频视频分析」 is out of
scope — this is 「手动和低频截图」.
4. **A privacy switch disables the whole capability, and it wins over any session.** When it is off,
the commands fail rather than returning stale or partial data, and no session can be started.
5. **The user can see when a session is live, and can end it from there.** Nothing in the documents
requires this, so it is proposed here: reading someone's screen without a visible sign is the kind
of behaviour that is indistinguishable from spyware from the outside, and a kill switch the user
has to go looking for is not a kill switch. The indicator is part of the feature, not a later
polish item.
6. **Captured frames and window text are never written to disk by this feature.** They are held in
memory, sent to the bound slot, and dropped. `01_Architecture_Design.md:1487` already says a
diagnostic keeps no raw screenshot unless the user saves it; this extends the same rule to the
companion path, where there is no save affordance at all.
7. **Screen and window data go to the companion slot only.** The operations slot never receives a
capture, a window summary, or a game event as a consequence of a session — only as a consequence
of the user explicitly sending one. This is `:1077` restated as an invariant that must be
enforced in Rust and covered by a test that fails if the guard is removed.
8. **Voice follows the documented flow, server side.** Press to record, upload the audio, let AstrBot
transcribe it, receive text, optionally receive speech back. The client does not run speech
recognition of its own: `WebView2` provides no `SpeechRecognition`, and the current
`ChatPanel.tsx` call to it can never have worked. Audio is uploaded through the same
`/api/v1/files` endpoint as any other attachment and referenced as a `record` segment inside
`message`, which is the contract AstrBot actually reads.
9. **No always-listening capability.** No wake word, no voice activity detection that starts a
recording on its own, no continuous duplex stream. Recording starts and stops on a user action.
This is `:1349-1350`.
10. **Speech-to-text requires a provider the user configures, and this project does not choose one
for them.** Every AstrBot STT provider needs either an API key or a self-hosted model. Enabling
one sends the user's voice to whichever service they picked, so the credential and the choice are
theirs. The client's job is to make the path work once they have configured it, to report clearly
when they have not, and never to enrol a provider on their behalf.
## Rationale
- The gate in `implementation-queue.md:129` is a precondition, not a checklist item to close later.
Wiring the capture commands first and adding consent afterwards would mean shipping a build in
which any renderer bug can read the user's screen.
- Putting the session, the rate limit, and the switch in Rust follows the boundary
`01_Architecture_Design.md:250-261` already draws. It also means the guarantee does not depend on
the frontend being correct, which is where every silent failure in this codebase so far has lived.
- A single-window target rather than a monitor keeps the feature inside 「用户授权的单次窗口截图」
and makes the authorization meaningful: the user is agreeing to one specific window, not to their
desktop.
- Binding to the companion slot by default and forbidding automatic delivery to operations keeps the
two conversations from leaking into each other, which is the whole point of the dual-slot design.
- Leaving the STT provider to the user is not caution for its own sake. It is the one part of this
feature that sends their voice somewhere; picking a default would be making that decision for
them.
## Consequences
- Add a `capture_policy` module in Rust owning the switch, the session registry, and the rate limit,
with the session expiry and the interval as pure functions so they can be tested without Windows.
- Register commands for the UI Automation summary and the window capture, each taking a session
identifier and failing when no live session covers it. `WindowSelection::raw_handle` becomes
reachable through the registry rather than being minted and dropped inside one command.
- Add the capability entries for the new commands, and a test that reads the shipped capability files
— a missing entry fails silently at runtime, which is how F-03 survived.
- Surface the live session and its stop control in the UI, and the privacy switch in settings.
- `capture.rs` uses `PrintWindow` over GDI. That does not capture a hardware-accelerated game
surface, so game companionship needs a different capture path (`Windows.Graphics.Capture`) before
it can work on the case it exists for. Recorded rather than worked around: a black frame delivered
to the model is worse than an error.
- Voice input needs microphone permission from WebView2. Tauri registers a `PermissionRequested`
handler only when `clipboard` is enabled, which this application does not enable, so the default
behaviour decides the outcome and has to be established on a real build rather than assumed.
- Do not implement mouse or keyboard control, process injection, or reading a game's memory. Detect
the foreground window through the existing bounded inspection only, per
`implementation-queue.md:118`.

查看文件

@ -0,0 +1,427 @@
//! Authorization for reading the user's screen, per ADR-0008.
//!
//! `implementation-queue.md:129` requires an authorization prompt, a frequency
//! limit, and a privacy switch to exist *before* screenshots or window summaries
//! are wired to anything. This module is those three, and it is deliberately the
//! only way to reach a window handle: `WindowSelection` used to be minted inside
//! `inspect_foreground_window` and dropped when the command returned, so a
//! second IPC call had no way to name a window the user had authorized — and no
//! way to be told it could not.
//!
//! Everything that decides whether a read is allowed lives here rather than in
//! the caller. A limit the renderer can skip is not a limit.
use std::{
collections::HashMap,
fmt,
sync::Mutex,
time::{Duration, Instant},
};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::windows::foreground_window::{ForegroundWindowError, WindowSelection};
/// Minimum gap between two pixel captures in one session.
///
/// Two seconds, not two frames. 「高频视频分析」 is out of scope
/// (`01_Architecture_Design.md:1351`) and what phase 7 asks for is 「手动和低频
/// 截图」 (`:1424`), so the default sits three orders of magnitude away from a
/// video frame rate.
pub const DEFAULT_PIXEL_INTERVAL: Duration = Duration::from_secs(2);
/// Floor on the configurable interval, so no caller can turn the limit off.
pub const MINIMUM_PIXEL_INTERVAL: Duration = Duration::from_millis(500);
/// Window text is cheaper than pixels and changes faster, but still not a stream.
pub const DEFAULT_TEXT_INTERVAL: Duration = Duration::from_millis(750);
/// Most sessions a user can have open at once.
///
/// One window per session and no reason to watch several at a time; the bound
/// exists so a renderer bug cannot accumulate authorizations.
pub const MAX_ACTIVE_SESSIONS: usize = 4;
/// What a session is being asked to read.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum ReadKind {
/// The window's pixels.
Pixels,
/// The window's title and UI Automation text.
Text,
}
impl ReadKind {
fn default_interval(self) -> Duration {
match self {
ReadKind::Pixels => DEFAULT_PIXEL_INTERVAL,
ReadKind::Text => DEFAULT_TEXT_INTERVAL,
}
}
}
#[derive(Debug, PartialEq, Eq)]
pub enum CapturePolicyError {
/// The privacy switch is off. Checked before anything else, so turning it
/// off ends every session rather than leaving live ones running.
Disabled,
/// No live session carries this token.
UnknownSession,
/// The session's authorization has run out, or the window is gone.
Selection(ForegroundWindowError),
/// This read came too soon after the last one of its kind.
RateLimited { retry_after: Duration },
/// Screen data may not be bound to the operations slot.
///
/// `01_Architecture_Design.md:1077`: 「运维 Slot 不自动接收游戏截图、窗口状态
/// 或游戏事件。」 A session is exactly that automatic delivery, so the slot is
/// refused at the point the session is created rather than at each read.
SlotNotPermitted,
/// Too many sessions are already open.
TooManySessions,
}
impl fmt::Display for CapturePolicyError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Disabled => write!(formatter, "屏幕读取已在设置中关闭"),
Self::UnknownSession => write!(formatter, "没有对应的屏幕读取会话,请重新授权"),
Self::Selection(error) => write!(formatter, "窗口授权已失效:{error}"),
Self::RateLimited { retry_after } => write!(
formatter,
"读取过于频繁,请在 {} 毫秒后重试",
retry_after.as_millis()
),
Self::SlotNotPermitted => {
write!(formatter, "屏幕内容只能发送给陪伴槽位")
}
Self::TooManySessions => write!(
formatter,
"同时开启的屏幕读取会话不能超过 {MAX_ACTIVE_SESSIONS} 个"
),
}
}
}
impl std::error::Error for CapturePolicyError {}
/// Enforces a minimum gap between reads of one kind.
///
/// `now` is a parameter rather than read inside, so the limit is testable
/// without sleeping and cannot drift from whatever clock the caller used to
/// decide the read was due.
#[derive(Debug, Clone, Copy)]
pub struct ReadBudget {
interval: Duration,
last: Option<Instant>,
}
impl ReadBudget {
pub fn new(interval: Duration) -> Self {
Self {
interval: interval.max(MINIMUM_PIXEL_INTERVAL),
last: None,
}
}
/// Records a read at `now`, or reports how long the caller must wait.
///
/// The timestamp is only advanced on success, so a rejected read cannot push
/// the next permitted one further away.
pub fn take(&mut self, now: Instant) -> Result<(), Duration> {
if let Some(last) = self.last {
let elapsed = now.saturating_duration_since(last);
if elapsed < self.interval {
return Err(self.interval - elapsed);
}
}
self.last = Some(now);
Ok(())
}
}
/// One window the user has authorized this application to read.
#[derive(Debug)]
struct CaptureSession {
selection: WindowSelection,
slot: desktop_protocol::ActiveSlotKind,
budgets: HashMap<ReadKind, ReadBudget>,
}
/// What the UI shows while a session is live, so the user can see it and stop it.
#[derive(Clone, Debug, PartialEq, Eq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct CaptureSessionSummary {
pub session_token: Uuid,
pub window_title: String,
pub process_id: u32,
pub slot: desktop_protocol::ActiveSlotKind,
pub expired: bool,
}
/// Whether screen data may be bound to `slot`.
///
/// A free function so the rule can be asserted without a live window, and so
/// there is one place to read rather than a condition inlined at each call site.
pub fn slot_permits_capture(
slot: desktop_protocol::ActiveSlotKind,
) -> Result<(), CapturePolicyError> {
match slot {
desktop_protocol::ActiveSlotKind::Companion => Ok(()),
desktop_protocol::ActiveSlotKind::Operations => Err(CapturePolicyError::SlotNotPermitted),
}
}
#[derive(Debug, Default)]
struct PolicyState {
/// The privacy switch. `false` until the user turns it on: an authorization
/// the user never gave is not an authorization.
enabled: bool,
sessions: HashMap<Uuid, CaptureSession>,
}
/// The Tauri-managed owner of the switch, the sessions, and the rate limits.
#[derive(Debug, Default)]
pub struct CapturePolicy {
state: Mutex<PolicyState>,
}
impl CapturePolicy {
fn with_state<T>(&self, body: impl FnOnce(&mut PolicyState) -> T) -> T {
match self.state.lock() {
Ok(mut guard) => body(&mut guard),
Err(poisoned) => body(&mut poisoned.into_inner()),
}
}
pub fn is_enabled(&self) -> bool {
self.with_state(|state| state.enabled)
}
/// Turns the capability on or off.
///
/// Turning it off drops every live session, so a switch flipped mid-capture
/// stops the next read rather than only preventing new authorizations.
pub fn set_enabled(&self, enabled: bool) {
self.with_state(|state| {
state.enabled = enabled;
if !enabled {
state.sessions.clear();
}
});
}
/// Authorizes reading one window until its selection expires.
pub fn begin_session(
&self,
selection: WindowSelection,
slot: desktop_protocol::ActiveSlotKind,
) -> Result<Uuid, CapturePolicyError> {
slot_permits_capture(slot)?;
selection
.validate()
.map_err(CapturePolicyError::Selection)?;
self.with_state(|state| {
if !state.enabled {
return Err(CapturePolicyError::Disabled);
}
// Expired sessions are cleared first, so a user who let one lapse is
// not told they have too many.
state
.sessions
.retain(|_, session| !session.selection.is_expired());
if state.sessions.len() >= MAX_ACTIVE_SESSIONS {
return Err(CapturePolicyError::TooManySessions);
}
let token = Uuid::new_v4();
state.sessions.insert(
token,
CaptureSession {
selection,
slot,
budgets: HashMap::new(),
},
);
Ok(token)
})
}
pub fn end_session(&self, token: Uuid) {
self.with_state(|state| {
state.sessions.remove(&token);
});
}
/// Every live session, for the indicator the user stops a capture from.
pub fn active_sessions(&self) -> Vec<CaptureSessionSummary> {
self.with_state(|state| {
state
.sessions
.retain(|_, session| !session.selection.is_expired());
let mut summaries: Vec<_> = state
.sessions
.iter()
.map(|(token, session)| {
let summary = session.selection.summary();
CaptureSessionSummary {
session_token: *token,
window_title: summary.title,
process_id: summary.process_id,
slot: session.slot,
expired: session.selection.is_expired(),
}
})
.collect();
// Stable order so the indicator does not reshuffle between polls.
summaries.sort_by_key(|summary| summary.session_token);
summaries
})
}
/// Checks a read is allowed, and runs it against the authorized window.
///
/// The read is a closure so the handle never leaves this module: every gate
/// — switch, session, expiry, rate limit — is passed before the caller sees
/// a `WindowSelection`, and there is no accessor that skips them.
pub fn read<T>(
&self,
token: Uuid,
kind: ReadKind,
now: Instant,
body: impl FnOnce(&WindowSelection) -> T,
) -> Result<T, CapturePolicyError> {
self.with_state(|state| {
if !state.enabled {
return Err(CapturePolicyError::Disabled);
}
let session = state
.sessions
.get_mut(&token)
.ok_or(CapturePolicyError::UnknownSession)?;
session
.selection
.validate()
.map_err(CapturePolicyError::Selection)?;
session
.budgets
.entry(kind)
.or_insert_with(|| ReadBudget::new(kind.default_interval()))
.take(now)
.map_err(|retry_after| CapturePolicyError::RateLimited { retry_after })?;
Ok(body(&session.selection))
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_capability_is_off_until_the_user_turns_it_on() {
let policy = CapturePolicy::default();
assert!(!policy.is_enabled());
assert!(policy.active_sessions().is_empty());
}
#[test]
fn turning_the_switch_off_ends_every_live_session() {
let policy = CapturePolicy::default();
policy.set_enabled(true);
// Sessions are inserted through begin_session, which needs a real window
// handle, so this exercises the switch through the state it owns.
assert!(policy.is_enabled());
policy.set_enabled(false);
assert!(!policy.is_enabled());
assert!(policy.active_sessions().is_empty());
}
/// The operations slot never receives screen data as a consequence of a
/// session (`01_Architecture_Design.md:1077`). Refused at creation, so no
/// read path has to remember.
#[test]
fn a_session_cannot_be_bound_to_the_operations_slot() {
let policy = CapturePolicy::default();
policy.set_enabled(true);
// Checked before the window handle is touched — the slot is wrong
// regardless of whether the window is still there — which is also what
// lets the rule be asserted without one.
assert_eq!(
slot_permits_capture(desktop_protocol::ActiveSlotKind::Operations),
Err(CapturePolicyError::SlotNotPermitted)
);
assert_eq!(
slot_permits_capture(desktop_protocol::ActiveSlotKind::Companion),
Ok(())
);
let _ = &policy;
}
#[test]
fn a_read_is_refused_while_the_switch_is_off() {
let policy = CapturePolicy::default();
let error = policy
.read(Uuid::new_v4(), ReadKind::Pixels, Instant::now(), |_| ())
.expect_err("must be refused");
assert_eq!(error, CapturePolicyError::Disabled);
}
#[test]
fn a_read_is_refused_when_no_session_names_the_window() {
let policy = CapturePolicy::default();
policy.set_enabled(true);
let error = policy
.read(Uuid::new_v4(), ReadKind::Pixels, Instant::now(), |_| ())
.expect_err("must be refused");
assert_eq!(error, CapturePolicyError::UnknownSession);
}
#[test]
fn a_budget_refuses_a_second_read_inside_its_interval() {
let start = Instant::now();
let mut budget = ReadBudget::new(Duration::from_secs(2));
assert!(budget.take(start).is_ok());
let retry_after = budget
.take(start + Duration::from_millis(500))
.expect_err("must be limited");
assert_eq!(retry_after, Duration::from_millis(1500));
assert!(budget.take(start + Duration::from_secs(2)).is_ok());
}
/// A refused read must not push the next permitted one further away, or a
/// caller that polls faster than the limit is locked out for good.
#[test]
fn a_refused_read_does_not_extend_the_wait() {
let start = Instant::now();
let mut budget = ReadBudget::new(Duration::from_secs(2));
budget.take(start).expect("first read");
for offset in [200, 400, 600, 800] {
assert!(budget.take(start + Duration::from_millis(offset)).is_err());
}
assert!(budget.take(start + Duration::from_secs(2)).is_ok());
}
/// The interval cannot be configured below the floor, so no caller can turn
/// the frequency limit off by asking for zero.
#[test]
fn the_interval_cannot_be_set_below_the_floor() {
let start = Instant::now();
let mut budget = ReadBudget::new(Duration::ZERO);
budget.take(start).expect("first read");
assert!(budget.take(start).is_err());
assert!(budget.take(start + MINIMUM_PIXEL_INTERVAL).is_ok());
}
/// Pixels are limited far more strictly than a video frame rate, because
/// 「高频视频分析」 is out of scope.
#[test]
fn the_pixel_interval_is_nowhere_near_a_frame_rate() {
assert!(DEFAULT_PIXEL_INTERVAL >= Duration::from_secs(1));
assert!(MINIMUM_PIXEL_INTERVAL >= Duration::from_millis(500));
assert!(DEFAULT_TEXT_INTERVAL >= Duration::from_millis(500));
}
}

查看文件

@ -1,4 +1,5 @@
pub mod auth; pub mod auth;
pub mod capture_policy;
pub mod credentials; pub mod credentials;
pub mod database; pub mod database;
pub mod diagnostics; pub mod diagnostics;