fix(astrbot): scope stream event ids and fix the transport layer
Combines the P0 event-id fix with the three transport fixes that rewrite the same stream loop; they are not separable in these two files. fix(astrbot): scope stream event ids per request (P0-1 / F-01) `AstrBotStreamMapper` restarts its sequence at zero for every stream, but `seen_event_ids` lives on the client and outlives a request. Event ids were built from `server_instance_id + sequence` alone, so the second message on a connection replayed the first message's ids and every one of its events was discarded as a duplicate: the assistant's reply never reached the UI or the database. Ids now carry a per-request `stream_id`. fix(protocol): serialize model options in camelCase (P1-7b / F-07) `AstrBotModelOption` had no `rename_all`, so the frontend's `isDefault` was always undefined and no model was preselected. fix(net): stop total timeouts from killing streams (P1-1 / F-10) `Client::timeout` runs from the start of the request until the body is finished and keeps counting while `bytes_stream()` is consumed. The 30s desktop-v1 budget therefore tore the long-lived SSE connection down every 30 seconds no matter how healthy it was — heartbeats could not keep it alive — and the 120s AstrBot budget cut off any generation longer than two minutes. Streaming now uses a separate client with no total deadline, detecting a stalled peer with `read_timeout` (which resets on every successful read) plus a TCP keepalive. REST endpoints keep their total deadline. fix(astrbot): decode sse bytes on line boundaries (P1-3 / F-11) Decoding each network chunk with `String::from_utf8_lossy` corrupted any character straddling a chunk boundary. For CJK text that is one replacement character per unlucky chunk, persisted into the message. The stream is now split into lines at the byte level and only complete lines are decoded. fix(astrbot): finalize streams on cancel and error (P2-2 / F-13) Cancellation and every error path returned without a `MessageCompleted` event, leaving the row `streaming = 1` so it rendered as "generating" again after a restart. All exit paths now finalize with `Cancelled` or `Error` and keep the text received so far. Cancelling also no longer flips the connection to Disconnected and back, which was a visible flicker (P3-4 / F-56). Hardening found while reviewing the above: a 64 MiB per-response ceiling (removing the total timeout removed the only bound on a response, and `read_timeout` cannot replace it), and a cursor-based line buffer instead of draining from the front, which was quadratic in the chunk size. Tests drive a real `AstrBotClient` against a hand-rolled HTTP/1.1 chunked-SSE server on a raw `TcpListener`: byte-exact control over chunk boundaries and dropping a socket mid-body without the terminal chunk are not expressible through an HTTP framework, and both are needed here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
这个提交存在于:
父节点
41592e2504
当前提交
6e81a3a7ae
共有 3 个文件被更改,包括 688 次插入 和 36 次删除
|
|
@ -81,7 +81,11 @@ pub struct AstrBotChatRequest {
|
|||
}
|
||||
|
||||
/// A model/provider configuration exposed by AstrBot.
|
||||
///
|
||||
/// Serialized towards React, so the field names must match the frontend
|
||||
/// contract in `src/features/chat/types.ts`.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct AstrBotModelOption {
|
||||
pub id: String,
|
||||
#[serde(default)]
|
||||
|
|
@ -221,6 +225,7 @@ struct PendingMedia {
|
|||
#[derive(Debug, Default)]
|
||||
pub struct AstrBotStreamMapper {
|
||||
server_instance_id: String,
|
||||
stream_id: String,
|
||||
session_id: Option<String>,
|
||||
active_message_id: Option<String>,
|
||||
accumulated: String,
|
||||
|
|
@ -231,9 +236,17 @@ pub struct AstrBotStreamMapper {
|
|||
}
|
||||
|
||||
impl AstrBotStreamMapper {
|
||||
pub fn new(server_instance_id: impl Into<String>) -> Self {
|
||||
/// Creates a mapper for a single AstrBot chat request stream.
|
||||
///
|
||||
/// `stream_id` must be unique per request for the lifetime of the client
|
||||
/// that consumes the mapped events. The mapper restarts its sequence
|
||||
/// counter at zero for every stream, so without a per-stream component the
|
||||
/// generated event ids of a second request would collide with the first
|
||||
/// request's ids and be discarded by the client-side deduplicator.
|
||||
pub fn new(server_instance_id: impl Into<String>, stream_id: impl Into<String>) -> Self {
|
||||
Self {
|
||||
server_instance_id: server_instance_id.into(),
|
||||
stream_id: stream_id.into(),
|
||||
..Self::default()
|
||||
}
|
||||
}
|
||||
|
|
@ -252,7 +265,18 @@ impl AstrBotStreamMapper {
|
|||
|
||||
/// Call when the HTTP stream ends without a terminal server frame.
|
||||
pub fn finish_stream(&mut self) -> Vec<GatewayEvent> {
|
||||
self.complete_if_needed(FinishReason::Stop)
|
||||
self.finish_stream_with(FinishReason::Stop)
|
||||
}
|
||||
|
||||
/// Terminates the in-flight message with an explicit reason.
|
||||
///
|
||||
/// Cancellation and transport errors must go through this instead of
|
||||
/// returning early: without a `MessageCompleted` event the message stays
|
||||
/// `streaming = 1` in the database and reappears as "generating" after a
|
||||
/// restart. Returns an empty vector when no assistant text is pending, so
|
||||
/// it is safe to call unconditionally on every exit path.
|
||||
pub fn finish_stream_with(&mut self, reason: FinishReason) -> Vec<GatewayEvent> {
|
||||
self.complete_if_needed(reason)
|
||||
}
|
||||
|
||||
fn map_frame(&mut self, frame: AstrBotStreamFrame) -> Vec<GatewayEvent> {
|
||||
|
|
@ -452,7 +476,10 @@ impl AstrBotStreamMapper {
|
|||
|
||||
fn event(&mut self, message_id: Option<String>, body: GatewayEventBody) -> GatewayEvent {
|
||||
self.sequence = self.sequence.saturating_add(1);
|
||||
let event_id = format!("astrbot-{}-{}", self.server_instance_id, self.sequence);
|
||||
let event_id = format!(
|
||||
"astrbot-{}-{}-{}",
|
||||
self.server_instance_id, self.stream_id, self.sequence
|
||||
);
|
||||
let mut metadata = EventMetadata::new(
|
||||
event_id,
|
||||
self.server_instance_id.clone(),
|
||||
|
|
@ -513,7 +540,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn maps_live_probe_style_plain_stream() {
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test");
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test", "stream-a");
|
||||
let lines = [
|
||||
r#"data: {"type": "session_id", "data": null, "session_id": "desktop-companion-s1"}"#,
|
||||
r#"data: {"type": "user_message_saved", "data": {"id": 1}}"#,
|
||||
|
|
@ -558,7 +585,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn maps_complete_frame_when_no_plain_chunks_are_emitted() {
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test");
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test", "stream-a");
|
||||
let events = mapper
|
||||
.push_line(r#"data: {"type":"complete","data":"A complete reply","streaming":true,"message_id":"m1"}"#)
|
||||
.expect("parse");
|
||||
|
|
@ -575,7 +602,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn complete_frame_reconciles_partial_plain_chunks_without_duplication() {
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test");
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test", "stream-a");
|
||||
mapper
|
||||
.push_line(
|
||||
r#"data: {"type":"plain","data":"Hello","streaming":true,"message_id":"m1"}"#,
|
||||
|
|
@ -597,7 +624,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn maps_error_frames() {
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test");
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test", "stream-a");
|
||||
let events = mapper
|
||||
.push_line(r#"data: {"type":"error","data":"provider missing"}"#)
|
||||
.expect("parse");
|
||||
|
|
@ -613,7 +640,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn maps_image_frames_to_persistable_attachment_markers() {
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test");
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test", "stream-a");
|
||||
assert!(
|
||||
mapper
|
||||
.push_line(r#"data: {"type":"image","data":"[IMAGE]happy.png","message_id":"m1"}"#)
|
||||
|
|
@ -633,6 +660,37 @@ mod tests {
|
|||
assert_eq!(delta, "");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn consecutive_streams_do_not_reuse_event_ids() {
|
||||
// Regression: a client keeps one deduplicator for its whole lifetime,
|
||||
// while a mapper is rebuilt per request and restarts its sequence at
|
||||
// zero. Without a per-stream id the second message's events collided
|
||||
// with the first message's ids and were dropped as duplicates.
|
||||
let frame = r#"data: {"type":"plain","data":"Hi","streaming":true,"message_id":"m1"}"#;
|
||||
|
||||
let collect = |stream_id: &str| {
|
||||
let mut mapper = AstrBotStreamMapper::new("astrbot-test", stream_id);
|
||||
let mut events = mapper.push_line(frame).expect("parse");
|
||||
events.extend(mapper.finish_stream());
|
||||
events
|
||||
.into_iter()
|
||||
.map(|event| event.metadata.event_id)
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
|
||||
let first = collect("stream-a");
|
||||
let second = collect("stream-b");
|
||||
|
||||
assert!(!first.is_empty());
|
||||
assert_eq!(first.len(), second.len());
|
||||
for event_id in &second {
|
||||
assert!(
|
||||
!first.contains(event_id),
|
||||
"event id {event_id} was reused across streams"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn derives_stable_instance_ids() {
|
||||
let a = derive_astrbot_server_instance_id("http://192.168.31.210:26285/");
|
||||
|
|
|
|||
|
|
@ -3,8 +3,8 @@ use std::{collections::VecDeque, sync::Arc, time::Duration};
|
|||
use desktop_protocol::{
|
||||
AstrBotApiEnvelope, AstrBotAttachment, AstrBotCapabilitySnapshot, AstrBotChatRequest,
|
||||
AstrBotConfigListData, AstrBotModelOption, AstrBotSessionListData, AstrBotStreamMapper,
|
||||
AuthenticationType, BoundGatewayEvent, ConnectionState, GatewayEvent, PROTOCOL_VERSION,
|
||||
derive_astrbot_server_instance_id,
|
||||
AuthenticationType, BoundGatewayEvent, ConnectionState, FinishReason, GatewayEvent,
|
||||
PROTOCOL_VERSION, derive_astrbot_server_instance_id,
|
||||
};
|
||||
use futures_util::StreamExt;
|
||||
use reqwest::header::{ACCEPT, AUTHORIZATION, CONTENT_TYPE, HeaderMap, HeaderValue};
|
||||
|
|
@ -44,6 +44,14 @@ pub enum AstrBotClientError {
|
|||
Json(#[from] serde_json::Error),
|
||||
#[error("astrbot stream mapping failed: {0}")]
|
||||
Map(String),
|
||||
/// The response body broke after the server had already started answering.
|
||||
///
|
||||
/// Distinct from [`AstrBotClientError::Request`] on purpose: `/api/v1/chat`
|
||||
/// has no idempotency key, so re-POSTing a message whose response had
|
||||
/// already begun duplicates it in the remote conversation. Only the caller
|
||||
/// (i.e. the user) may retry this.
|
||||
#[error("astrbot stream interrupted after the response started")]
|
||||
StreamInterrupted,
|
||||
#[error("server identity changed")]
|
||||
ServerIdentityChanged,
|
||||
#[error("astrbot event channel closed")]
|
||||
|
|
@ -64,6 +72,9 @@ impl From<AstrBotClientError> for GatewayError {
|
|||
AstrBotClientError::ChannelClosed => GatewayError::ChannelClosed,
|
||||
AstrBotClientError::ApiStatus(message) => GatewayError::Adapter(message),
|
||||
AstrBotClientError::Map(message) => GatewayError::Adapter(message),
|
||||
AstrBotClientError::StreamInterrupted => GatewayError::Adapter(
|
||||
"AstrBot stream interrupted after the response started".into(),
|
||||
),
|
||||
AstrBotClientError::InvalidConfig(message) => GatewayError::Adapter(message.to_owned()),
|
||||
AstrBotClientError::CredentialExpired => {
|
||||
GatewayError::Adapter("AstrBot authentication has expired".into())
|
||||
|
|
@ -74,6 +85,88 @@ impl From<AstrBotClientError> for GatewayError {
|
|||
|
||||
const MAX_DEDUPLICATED_EVENTS: usize = 4_096;
|
||||
|
||||
/// Guards against a hostile or broken server streaming an unbounded single line.
|
||||
const MAX_SSE_LINE_BYTES: usize = 1024 * 1024;
|
||||
|
||||
/// Ceiling on the total size of one chat response.
|
||||
///
|
||||
/// Removing `Client::timeout` from the streaming client (so a slow reply is no
|
||||
/// longer cut off) also removed the only bound on how much a server could send
|
||||
/// in one response. `read_timeout` cannot replace it: it measures the gap
|
||||
/// between successful reads, so a server that keeps trickling bytes keeps the
|
||||
/// stream alive forever, and both this buffer and the mapper's accumulated
|
||||
/// message text would grow without limit. A real reply is far below this.
|
||||
const MAX_STREAM_BYTES: u64 = 64 * 1024 * 1024;
|
||||
|
||||
/// Splits an SSE byte stream into lines at the byte level.
|
||||
///
|
||||
/// TCP chunk boundaries fall on arbitrary byte offsets, so decoding each chunk
|
||||
/// with `String::from_utf8_lossy` corrupts any multi-byte character that
|
||||
/// straddles two chunks — for CJK text (3 bytes per character) that is a
|
||||
/// frequent, user-visible `U+FFFD`. Buffering raw bytes and only decoding
|
||||
/// complete `\n`-delimited lines keeps every character intact, because SSE
|
||||
/// guarantees `\n` (0x0A) never appears inside a UTF-8 multi-byte sequence.
|
||||
#[derive(Debug, Default)]
|
||||
struct SseLineBuffer {
|
||||
bytes: Vec<u8>,
|
||||
/// Offset of the first unconsumed byte.
|
||||
///
|
||||
/// Consuming a line by draining from the front would memmove the entire
|
||||
/// remainder once per line, which is quadratic in the chunk size — and a
|
||||
/// single `bytes_stream()` item from hyper can be several hundred KiB of
|
||||
/// short delta frames. Advancing a cursor instead makes reading a line O(1)
|
||||
/// and leaves one memmove per chunk, in `push_chunk`.
|
||||
cursor: usize,
|
||||
total: u64,
|
||||
}
|
||||
|
||||
impl SseLineBuffer {
|
||||
fn push_chunk(&mut self, chunk: &[u8]) {
|
||||
if self.cursor > 0 {
|
||||
self.bytes.drain(..self.cursor);
|
||||
self.cursor = 0;
|
||||
}
|
||||
self.total = self.total.saturating_add(chunk.len() as u64);
|
||||
self.bytes.extend_from_slice(chunk);
|
||||
}
|
||||
|
||||
/// Pops the next complete line, or `None` when the buffer holds no `\n` yet.
|
||||
///
|
||||
/// Invalid UTF-8 within a single complete line is replaced rather than
|
||||
/// rejected: a lossy line is still parseable SSE, and dropping the whole
|
||||
/// stream over one bad byte would lose the surrounding reply.
|
||||
fn next_line(&mut self) -> Option<String> {
|
||||
let pending = &self.bytes[self.cursor..];
|
||||
let index = pending.iter().position(|byte| *byte == b'\n')?;
|
||||
let mut line = &pending[..index];
|
||||
if line.last() == Some(&b'\r') {
|
||||
line = &line[..line.len() - 1];
|
||||
}
|
||||
let text = String::from_utf8_lossy(line).into_owned();
|
||||
self.cursor += index + 1;
|
||||
Some(text)
|
||||
}
|
||||
|
||||
/// Consumes whatever is left once the stream ends without a trailing `\n`.
|
||||
fn take_remainder(&mut self) -> Option<String> {
|
||||
let mut line = &self.bytes[self.cursor..];
|
||||
if line.is_empty() {
|
||||
return None;
|
||||
}
|
||||
if line.last() == Some(&b'\r') {
|
||||
line = &line[..line.len() - 1];
|
||||
}
|
||||
let text = String::from_utf8_lossy(line).into_owned();
|
||||
self.cursor = self.bytes.len();
|
||||
(!text.trim().is_empty()).then_some(text)
|
||||
}
|
||||
|
||||
/// True when one line, or the whole response, has outgrown its limit.
|
||||
fn is_overflowing(&self) -> bool {
|
||||
self.bytes.len() - self.cursor > MAX_SSE_LINE_BYTES || self.total > MAX_STREAM_BYTES
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct EventDeduplicator {
|
||||
ids: std::collections::HashSet<String>,
|
||||
|
|
@ -95,10 +188,27 @@ impl EventDeduplicator {
|
|||
}
|
||||
}
|
||||
|
||||
/// Total budget for a short request/response exchange (probe, sessions, upload).
|
||||
const REST_TOTAL_TIMEOUT: Duration = Duration::from_secs(120);
|
||||
|
||||
/// How long the chat stream may stay completely silent before it is abandoned.
|
||||
///
|
||||
/// This is deliberately *not* a total timeout. `Client::timeout` counts from the
|
||||
/// start of the request until the body has finished, and it keeps running while
|
||||
/// `bytes_stream()` is being consumed, so any generation longer than the budget
|
||||
/// was cut off mid-reply. `read_timeout` resets after every successful read, so
|
||||
/// a reply may take arbitrarily long as long as it keeps producing bytes.
|
||||
const STREAM_IDLE_TIMEOUT: Duration = Duration::from_secs(120);
|
||||
|
||||
const CONNECT_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct AstrBotClient {
|
||||
config: AstrBotClientConfig,
|
||||
/// Short request/response endpoints; keeps a total deadline.
|
||||
http: reqwest::Client,
|
||||
/// `/api/v1/chat` only; no total deadline, idle detection via `read_timeout`.
|
||||
stream_http: reqwest::Client,
|
||||
seen_event_ids: Arc<Mutex<EventDeduplicator>>,
|
||||
session_id: Arc<Mutex<Option<String>>>,
|
||||
server_instance_id: Arc<Mutex<String>>,
|
||||
|
|
@ -142,10 +252,18 @@ impl AstrBotClient {
|
|||
);
|
||||
|
||||
let http = reqwest::Client::builder()
|
||||
.default_headers(headers.clone())
|
||||
.redirect(reqwest::redirect::Policy::none())
|
||||
.connect_timeout(CONNECT_TIMEOUT)
|
||||
.timeout(REST_TOTAL_TIMEOUT)
|
||||
.build()?;
|
||||
|
||||
let stream_http = reqwest::Client::builder()
|
||||
.default_headers(headers)
|
||||
.redirect(reqwest::redirect::Policy::none())
|
||||
.connect_timeout(Duration::from_secs(5))
|
||||
.timeout(Duration::from_secs(120))
|
||||
.connect_timeout(CONNECT_TIMEOUT)
|
||||
.read_timeout(STREAM_IDLE_TIMEOUT)
|
||||
.tcp_keepalive(Duration::from_secs(60))
|
||||
.build()?;
|
||||
|
||||
let (state_tx, _) = watch::channel(ConnectionState::Disconnected);
|
||||
|
|
@ -156,6 +274,7 @@ impl AstrBotClient {
|
|||
..config
|
||||
},
|
||||
http,
|
||||
stream_http,
|
||||
seen_event_ids: Arc::new(Mutex::new(EventDeduplicator::default())),
|
||||
session_id: Arc::new(Mutex::new(initial_session)),
|
||||
server_instance_id: Arc::new(Mutex::new(initial_identity)),
|
||||
|
|
@ -412,7 +531,7 @@ impl AstrBotClient {
|
|||
};
|
||||
|
||||
let response = self
|
||||
.http
|
||||
.stream_http
|
||||
.post(self.url("/api/v1/chat"))
|
||||
.header(ACCEPT, HeaderValue::from_static("text/event-stream"))
|
||||
.json(&request)
|
||||
|
|
@ -437,7 +556,12 @@ impl AstrBotClient {
|
|||
return Err(AstrBotClientError::ServerIdentityChanged);
|
||||
}
|
||||
|
||||
let mut mapper = AstrBotStreamMapper::new(server_instance_id.clone());
|
||||
// Every request gets its own stream id: the mapper restarts its
|
||||
// sequence counter per stream, and `seen_event_ids` outlives a single
|
||||
// request, so reusing ids would make the deduplicator discard every
|
||||
// event of the second and later messages on this connection.
|
||||
let stream_id = uuid::Uuid::new_v4().to_string();
|
||||
let mut mapper = AstrBotStreamMapper::new(server_instance_id.clone(), stream_id);
|
||||
if let Some(existing) = session_id {
|
||||
let seed = serde_json::json!({
|
||||
"type": "session_id",
|
||||
|
|
@ -448,39 +572,78 @@ impl AstrBotClient {
|
|||
}
|
||||
|
||||
let mut stream = response.bytes_stream();
|
||||
let mut buffer = String::new();
|
||||
let mut buffer = SseLineBuffer::default();
|
||||
self.set_state(ConnectionState::Connected);
|
||||
|
||||
// From here on the server has accepted and is answering this POST, so a
|
||||
// failure must never be retried automatically — see `finish_and_fail`.
|
||||
loop {
|
||||
tokio::select! {
|
||||
_ = cancellation.cancelled() => {
|
||||
self.set_state(ConnectionState::Disconnected);
|
||||
// Leave the connection state alone: the socket is still
|
||||
// healthy and flipping to Disconnected only makes the tray
|
||||
// and the slot badge flicker.
|
||||
let events = mapper.finish_stream_with(FinishReason::Cancelled);
|
||||
self.dispatch_events(events, sender).await?;
|
||||
if let Some(session) = mapper.session_id() {
|
||||
*self.session_id.lock().await = Some(session.to_owned());
|
||||
}
|
||||
self.set_state(ConnectionState::Connected);
|
||||
return Ok(mapper.session_id().map(str::to_owned));
|
||||
}
|
||||
next = stream.next() => {
|
||||
let Some(next) = next else { break; };
|
||||
let chunk = next?;
|
||||
buffer.push_str(&String::from_utf8_lossy(&chunk));
|
||||
while let Some(index) = buffer.find('\n') {
|
||||
let mut line = buffer[..index].to_owned();
|
||||
buffer.drain(..=index);
|
||||
if line.ends_with('\r') {
|
||||
line.pop();
|
||||
let chunk = match next {
|
||||
Ok(chunk) => chunk,
|
||||
Err(_) => {
|
||||
return self
|
||||
.finish_and_fail(&mut mapper, sender, AstrBotClientError::StreamInterrupted)
|
||||
.await;
|
||||
}
|
||||
};
|
||||
buffer.push_chunk(&chunk);
|
||||
if buffer.is_overflowing() {
|
||||
return self
|
||||
.finish_and_fail(
|
||||
&mut mapper,
|
||||
sender,
|
||||
AstrBotClientError::Map(
|
||||
"astrbot response exceeded the supported size".into(),
|
||||
),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
while let Some(line) = buffer.next_line() {
|
||||
match mapper.push_line(&line) {
|
||||
Ok(events) => self.dispatch_events(events, sender).await?,
|
||||
Err(error) => {
|
||||
return self
|
||||
.finish_and_fail(
|
||||
&mut mapper,
|
||||
sender,
|
||||
AstrBotClientError::Map(error.to_string()),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
let events = mapper
|
||||
.push_line(&line)
|
||||
.map_err(|error| AstrBotClientError::Map(error.to_string()))?;
|
||||
self.dispatch_events(events, sender).await?;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if !buffer.trim().is_empty() {
|
||||
let events = mapper
|
||||
.push_line(buffer.trim_end_matches('\r'))
|
||||
.map_err(|error| AstrBotClientError::Map(error.to_string()))?;
|
||||
self.dispatch_events(events, sender).await?;
|
||||
if let Some(remainder) = buffer.take_remainder() {
|
||||
match mapper.push_line(&remainder) {
|
||||
Ok(events) => self.dispatch_events(events, sender).await?,
|
||||
Err(error) => {
|
||||
return self
|
||||
.finish_and_fail(
|
||||
&mut mapper,
|
||||
sender,
|
||||
AstrBotClientError::Map(error.to_string()),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
}
|
||||
let events = mapper.finish_stream();
|
||||
self.dispatch_events(events, sender).await?;
|
||||
|
|
@ -492,6 +655,29 @@ impl AstrBotClient {
|
|||
Ok(self.session_id.lock().await.clone())
|
||||
}
|
||||
|
||||
/// Closes out a half-streamed message before surfacing a transport failure.
|
||||
///
|
||||
/// Every error path after the response headers arrive has to emit a
|
||||
/// `MessageCompleted { finishReason: error }`; otherwise the partial reply
|
||||
/// stays `streaming = 1` in the database and renders as "generating"
|
||||
/// forever, including after a restart.
|
||||
async fn finish_and_fail(
|
||||
&self,
|
||||
mapper: &mut AstrBotStreamMapper,
|
||||
sender: &mpsc::Sender<super::gateway_client::BoundGatewayEvent>,
|
||||
error: AstrBotClientError,
|
||||
) -> Result<Option<String>, AstrBotClientError> {
|
||||
let events = mapper.finish_stream_with(FinishReason::Error);
|
||||
// Best effort: if the channel is already gone there is nothing left to
|
||||
// finalize, and the original transport error is the more useful one.
|
||||
let _ = self.dispatch_events(events, sender).await;
|
||||
if let Some(session) = mapper.session_id() {
|
||||
*self.session_id.lock().await = Some(session.to_owned());
|
||||
}
|
||||
self.set_state(ConnectionState::ServerUnavailable);
|
||||
Err(error)
|
||||
}
|
||||
|
||||
async fn dispatch_events(
|
||||
&self,
|
||||
events: Vec<GatewayEvent>,
|
||||
|
|
@ -631,4 +817,386 @@ mod tests {
|
|||
assert!(jwt_is_expired("e30.eyJleHAiOjF9.signature"));
|
||||
assert!(!jwt_is_expired("not-a-jwt"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn line_buffer_keeps_multibyte_characters_split_across_chunks_intact() {
|
||||
// "你好世界" is 12 UTF-8 bytes; cut in the middle of the second char.
|
||||
let payload = "data: {\"type\":\"plain\",\"data\":\"你好世界\"}\n";
|
||||
let bytes = payload.as_bytes();
|
||||
let split = bytes
|
||||
.iter()
|
||||
.position(|byte| *byte == 0xE4)
|
||||
.expect("payload contains a CJK lead byte")
|
||||
+ 4;
|
||||
assert!(
|
||||
!payload.is_char_boundary(split),
|
||||
"split must cut a character"
|
||||
);
|
||||
|
||||
let mut buffer = SseLineBuffer::default();
|
||||
buffer.push_chunk(&bytes[..split]);
|
||||
assert!(
|
||||
buffer.next_line().is_none(),
|
||||
"an incomplete line must not be emitted"
|
||||
);
|
||||
buffer.push_chunk(&bytes[split..]);
|
||||
|
||||
let line = buffer
|
||||
.next_line()
|
||||
.expect("line completes on the second chunk");
|
||||
assert!(!line.contains('\u{FFFD}'), "decoded line: {line}");
|
||||
assert!(line.contains("你好世界"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn line_buffer_strips_crlf_and_returns_the_trailing_line() {
|
||||
let mut buffer = SseLineBuffer::default();
|
||||
buffer.push_chunk(b"data: one\r\ndata: two\r\n");
|
||||
assert_eq!(buffer.next_line().as_deref(), Some("data: one"));
|
||||
assert_eq!(buffer.next_line().as_deref(), Some("data: two"));
|
||||
assert_eq!(buffer.next_line(), None);
|
||||
|
||||
buffer.push_chunk(b"data: tail");
|
||||
assert_eq!(buffer.next_line(), None);
|
||||
assert_eq!(buffer.take_remainder().as_deref(), Some("data: tail"));
|
||||
assert_eq!(buffer.take_remainder(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn line_buffer_cursor_survives_interleaved_reads_and_writes() {
|
||||
// The cursor is only compacted on push_chunk, so a partially drained
|
||||
// buffer must still read correctly across the next chunk.
|
||||
let mut buffer = SseLineBuffer::default();
|
||||
buffer.push_chunk(b"a\nb\nc");
|
||||
assert_eq!(buffer.next_line().as_deref(), Some("a"));
|
||||
buffer.push_chunk(b"d\ne\n");
|
||||
assert_eq!(buffer.next_line().as_deref(), Some("b"));
|
||||
assert_eq!(buffer.next_line().as_deref(), Some("cd"));
|
||||
assert_eq!(buffer.next_line().as_deref(), Some("e"));
|
||||
assert_eq!(buffer.next_line(), None);
|
||||
assert_eq!(buffer.take_remainder(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn line_buffer_bounds_a_single_line_and_the_whole_response() {
|
||||
let mut oversized_line = SseLineBuffer::default();
|
||||
oversized_line.push_chunk(&vec![b'x'; MAX_SSE_LINE_BYTES + 1]);
|
||||
assert!(oversized_line.is_overflowing());
|
||||
|
||||
// Consumed bytes still count towards the response ceiling, otherwise a
|
||||
// server could stream forever in small well-formed lines: read_timeout
|
||||
// never fires while bytes keep arriving, and there is no total timeout.
|
||||
let mut long_response = SseLineBuffer::default();
|
||||
let line = b"data: {}\n";
|
||||
let repeats = (MAX_STREAM_BYTES as usize / line.len()) + 1;
|
||||
for _ in 0..repeats {
|
||||
long_response.push_chunk(line);
|
||||
while long_response.next_line().is_some() {}
|
||||
}
|
||||
assert!(long_response.is_overflowing());
|
||||
assert!(
|
||||
long_response.bytes.len() < 1024,
|
||||
"the buffer itself must stay small; only the counter grows"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// End-to-end transport tests against a hand-rolled HTTP/1.1 SSE server.
|
||||
///
|
||||
/// A raw `TcpListener` is used instead of an HTTP framework on purpose: these
|
||||
/// tests need byte-exact control over where the response body is split across
|
||||
/// TCP writes, and the ability to drop the socket mid-body without sending the
|
||||
/// terminal chunk. Neither is expressible through a normal server API.
|
||||
#[cfg(test)]
|
||||
mod stream_transport_tests {
|
||||
use super::*;
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
use tokio::net::TcpListener;
|
||||
|
||||
/// What the fake server does after it has written the scripted frames.
|
||||
#[derive(Clone, Copy, PartialEq)]
|
||||
enum Ending {
|
||||
/// Send the terminal `0\r\n\r\n` chunk: a well-formed, complete body.
|
||||
Complete,
|
||||
/// Drop the socket without the terminal chunk, so the client sees a
|
||||
/// truncated body rather than a clean end of stream.
|
||||
Truncate,
|
||||
/// Keep the connection open and silent, so only cancellation ends it.
|
||||
HoldOpen,
|
||||
}
|
||||
|
||||
struct FakeAstrBot {
|
||||
base_url: String,
|
||||
requests: Arc<AtomicUsize>,
|
||||
}
|
||||
|
||||
/// Serves `chunks` as separate chunked-encoding frames with a pause between
|
||||
/// them, which forces hyper to surface them as distinct stream items.
|
||||
async fn spawn_fake(chunks: Vec<Vec<u8>>, ending: Ending) -> FakeAstrBot {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind");
|
||||
let port = listener.local_addr().expect("addr").port();
|
||||
let requests = Arc::new(AtomicUsize::new(0));
|
||||
let counter = Arc::clone(&requests);
|
||||
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
let Ok((mut socket, _)) = listener.accept().await else {
|
||||
return;
|
||||
};
|
||||
let chunks = chunks.clone();
|
||||
let counter = Arc::clone(&counter);
|
||||
tokio::spawn(async move {
|
||||
if read_request(&mut socket).await.is_none() {
|
||||
return;
|
||||
}
|
||||
counter.fetch_add(1, Ordering::SeqCst);
|
||||
|
||||
let header = b"HTTP/1.1 200 OK\r\n\
|
||||
Content-Type: text/event-stream\r\n\
|
||||
Transfer-Encoding: chunked\r\n\r\n";
|
||||
if socket.write_all(header).await.is_err() {
|
||||
return;
|
||||
}
|
||||
for chunk in chunks {
|
||||
let framed = format!("{:X}\r\n", chunk.len());
|
||||
if socket.write_all(framed.as_bytes()).await.is_err()
|
||||
|| socket.write_all(&chunk).await.is_err()
|
||||
|| socket.write_all(b"\r\n").await.is_err()
|
||||
|| socket.flush().await.is_err()
|
||||
{
|
||||
return;
|
||||
}
|
||||
// Guarantees the next frame lands in its own TCP read.
|
||||
tokio::time::sleep(Duration::from_millis(60)).await;
|
||||
}
|
||||
match ending {
|
||||
Ending::Complete => {
|
||||
let _ = socket.write_all(b"0\r\n\r\n").await;
|
||||
let _ = socket.flush().await;
|
||||
}
|
||||
Ending::Truncate => {
|
||||
// Dropping without the terminal chunk makes hyper
|
||||
// report an incomplete message.
|
||||
let _ = socket.shutdown().await;
|
||||
}
|
||||
Ending::HoldOpen => {
|
||||
tokio::time::sleep(Duration::from_secs(30)).await;
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
FakeAstrBot {
|
||||
base_url: format!("http://127.0.0.1:{port}"),
|
||||
requests,
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads request headers plus any declared body so the client's write side
|
||||
/// completes before the response is produced.
|
||||
async fn read_request(socket: &mut tokio::net::TcpStream) -> Option<()> {
|
||||
let mut raw = Vec::new();
|
||||
let mut scratch = [0u8; 1024];
|
||||
loop {
|
||||
let read = socket.read(&mut scratch).await.ok()?;
|
||||
if read == 0 {
|
||||
return None;
|
||||
}
|
||||
raw.extend_from_slice(&scratch[..read]);
|
||||
if let Some(index) = raw
|
||||
.windows(4)
|
||||
.position(|window| window == b"\r\n\r\n")
|
||||
.map(|index| index + 4)
|
||||
{
|
||||
let headers = String::from_utf8_lossy(&raw[..index]).to_ascii_lowercase();
|
||||
let length = headers
|
||||
.lines()
|
||||
.find_map(|line| line.strip_prefix("content-length:"))
|
||||
.and_then(|value| value.trim().parse::<usize>().ok())
|
||||
.unwrap_or(0);
|
||||
let mut body = raw.len() - index;
|
||||
while body < length {
|
||||
let read = socket.read(&mut scratch).await.ok()?;
|
||||
if read == 0 {
|
||||
return None;
|
||||
}
|
||||
body += read;
|
||||
}
|
||||
return Some(());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn client_for(base_url: &str) -> AstrBotClient {
|
||||
AstrBotClient::new(AstrBotClientConfig {
|
||||
profile_id: "profile-a".into(),
|
||||
slot: SlotKind::Companion,
|
||||
base_url: base_url.to_owned(),
|
||||
api_key: SecretString::from("abk_test".to_owned()),
|
||||
remote_username: "desktop-companion".into(),
|
||||
remote_display_name: None,
|
||||
authentication_type: AuthenticationType::ApiKey,
|
||||
session_id: None,
|
||||
expected_server_instance_id: None,
|
||||
})
|
||||
.expect("client")
|
||||
}
|
||||
|
||||
fn drain(
|
||||
receiver: &mut mpsc::Receiver<super::super::gateway_client::BoundGatewayEvent>,
|
||||
) -> Vec<super::super::gateway_client::BoundGatewayEvent> {
|
||||
let mut events = Vec::new();
|
||||
while let Ok(event) = receiver.try_recv() {
|
||||
events.push(event);
|
||||
}
|
||||
events
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn decodes_multibyte_text_split_across_tcp_chunks() {
|
||||
// One SSE frame carrying CJK text, cut in the middle of a character.
|
||||
let frame = "data: {\"type\":\"plain\",\"data\":\"你好,世界\"}\n\n";
|
||||
let bytes = frame.as_bytes().to_vec();
|
||||
let split = bytes
|
||||
.iter()
|
||||
.position(|byte| *byte == 0xE4)
|
||||
.expect("CJK lead byte")
|
||||
+ 1;
|
||||
assert!(!frame.is_char_boundary(split), "split must cut a character");
|
||||
|
||||
let server = spawn_fake(
|
||||
vec![bytes[..split].to_vec(), bytes[split..].to_vec()],
|
||||
Ending::Complete,
|
||||
)
|
||||
.await;
|
||||
let client = client_for(&server.base_url);
|
||||
let (tx, mut rx) = mpsc::channel(64);
|
||||
let cancellation = CancellationToken::new();
|
||||
|
||||
client
|
||||
.send_chat_stream("hi", None, Vec::new(), &tx, &cancellation)
|
||||
.await
|
||||
.expect("stream completes");
|
||||
|
||||
let text = drain(&mut rx)
|
||||
.iter()
|
||||
.filter(|event| event.event_type == "message.completed")
|
||||
.filter_map(|event| {
|
||||
event
|
||||
.payload
|
||||
.get("content")
|
||||
.and_then(Value::as_str)
|
||||
.map(ToOwned::to_owned)
|
||||
})
|
||||
.collect::<String>();
|
||||
assert!(!text.contains('\u{FFFD}'), "decoded content: {text}");
|
||||
assert!(text.contains("你好,世界"), "decoded content: {text}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn truncated_body_reports_stream_interrupted_and_finalizes_the_message() {
|
||||
let server = spawn_fake(
|
||||
vec![b"data: {\"type\":\"plain\",\"data\":\"partial\"}\n\n".to_vec()],
|
||||
Ending::Truncate,
|
||||
)
|
||||
.await;
|
||||
let client = client_for(&server.base_url);
|
||||
let (tx, mut rx) = mpsc::channel(64);
|
||||
let cancellation = CancellationToken::new();
|
||||
|
||||
let error = client
|
||||
.send_chat_stream("hi", None, Vec::new(), &tx, &cancellation)
|
||||
.await
|
||||
.expect_err("a truncated body must not look like success");
|
||||
assert!(
|
||||
matches!(error, AstrBotClientError::StreamInterrupted),
|
||||
"the response had already started, so this must not be auto-retried: {error:?}"
|
||||
);
|
||||
|
||||
let completed = drain(&mut rx)
|
||||
.into_iter()
|
||||
.find(|event| event.event_type == "message.completed")
|
||||
.expect("the half-streamed message must be finalized, not left streaming");
|
||||
assert_eq!(
|
||||
completed
|
||||
.payload
|
||||
.get("finishReason")
|
||||
.and_then(Value::as_str),
|
||||
Some("error")
|
||||
);
|
||||
assert_eq!(
|
||||
completed.payload.get("content").and_then(Value::as_str),
|
||||
Some("partial"),
|
||||
"text received before the break must be preserved"
|
||||
);
|
||||
}
|
||||
|
||||
/// A message whose reply had already started must never be re-sent:
|
||||
/// `POST /api/v1/chat` carries no idempotency key, so a second attempt
|
||||
/// appends the user's message to the remote conversation again.
|
||||
#[tokio::test]
|
||||
async fn a_broken_response_body_does_not_resend_the_chat_post() {
|
||||
let server = spawn_fake(
|
||||
vec![
|
||||
b"data: {\"type\":\"plain\",\"data\":\"partial\"}
|
||||
|
||||
"
|
||||
.to_vec(),
|
||||
],
|
||||
Ending::Truncate,
|
||||
)
|
||||
.await;
|
||||
let client = client_for(&server.base_url);
|
||||
let (tx, _rx) = mpsc::channel(64);
|
||||
let cancellation = CancellationToken::new();
|
||||
|
||||
let _ = client
|
||||
.send_chat_stream("hi", None, Vec::new(), &tx, &cancellation)
|
||||
.await;
|
||||
// Give any resend time to land before sampling.
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
|
||||
assert_eq!(server.requests.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cancellation_finalizes_the_message_as_cancelled() {
|
||||
let server = spawn_fake(
|
||||
vec![b"data: {\"type\":\"plain\",\"data\":\"half a rep\"}\n\n".to_vec()],
|
||||
Ending::HoldOpen,
|
||||
)
|
||||
.await;
|
||||
let client = client_for(&server.base_url);
|
||||
let (tx, mut rx) = mpsc::channel(64);
|
||||
let cancellation = CancellationToken::new();
|
||||
let trigger = cancellation.clone();
|
||||
tokio::spawn(async move {
|
||||
tokio::time::sleep(Duration::from_millis(400)).await;
|
||||
trigger.cancel();
|
||||
});
|
||||
|
||||
client
|
||||
.send_chat_stream("hi", None, Vec::new(), &tx, &cancellation)
|
||||
.await
|
||||
.expect("cancellation is not a failure");
|
||||
|
||||
let completed = drain(&mut rx)
|
||||
.into_iter()
|
||||
.find(|event| event.event_type == "message.completed")
|
||||
.expect("stopping generation must still finalize the message");
|
||||
assert_eq!(
|
||||
completed
|
||||
.payload
|
||||
.get("finishReason")
|
||||
.and_then(Value::as_str),
|
||||
Some("cancelled")
|
||||
);
|
||||
assert_eq!(
|
||||
completed.payload.get("content").and_then(Value::as_str),
|
||||
Some("half a rep"),
|
||||
"content received before the stop must be kept"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -83,10 +83,28 @@ impl EventDeduplicator {
|
|||
}
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
/// Total budget for `/desktop/v1/health` and any other short exchange.
|
||||
const REST_TOTAL_TIMEOUT: Duration = Duration::from_secs(30);
|
||||
|
||||
/// How long the event stream may stay silent before it is torn down and resumed.
|
||||
///
|
||||
/// A total `Client::timeout` keeps counting while `bytes_stream()` is consumed,
|
||||
/// so a 30s total deadline meant the long-lived desktop-v1 connection was
|
||||
/// killed and reconnected every 30 seconds no matter how healthy it was, and
|
||||
/// heartbeats could not keep it alive. `read_timeout` resets on every received
|
||||
/// byte, so heartbeats now do exactly that. Reconnects still resume from
|
||||
/// `Last-Event-ID`, so an idle teardown is lossless.
|
||||
const STREAM_IDLE_TIMEOUT: Duration = Duration::from_secs(90);
|
||||
|
||||
const CONNECT_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct GatewayClient {
|
||||
config: GatewayClientConfig,
|
||||
/// Short request/response endpoints; keeps a total deadline.
|
||||
http: reqwest::Client,
|
||||
/// `/desktop/v1/events/stream` only; no total deadline.
|
||||
stream_http: reqwest::Client,
|
||||
seen_event_ids: Arc<Mutex<EventDeduplicator>>,
|
||||
last_event_id: Arc<Mutex<Option<String>>>,
|
||||
state_tx: watch::Sender<ConnectionState>,
|
||||
|
|
@ -97,13 +115,21 @@ impl GatewayClient {
|
|||
let (state_tx, _) = watch::channel(ConnectionState::Disconnected);
|
||||
let http = reqwest::Client::builder()
|
||||
.redirect(reqwest::redirect::Policy::none())
|
||||
.connect_timeout(Duration::from_secs(5))
|
||||
.timeout(Duration::from_secs(30))
|
||||
.connect_timeout(CONNECT_TIMEOUT)
|
||||
.timeout(REST_TOTAL_TIMEOUT)
|
||||
.build()?;
|
||||
|
||||
let stream_http = reqwest::Client::builder()
|
||||
.redirect(reqwest::redirect::Policy::none())
|
||||
.connect_timeout(CONNECT_TIMEOUT)
|
||||
.read_timeout(STREAM_IDLE_TIMEOUT)
|
||||
.tcp_keepalive(Duration::from_secs(60))
|
||||
.build()?;
|
||||
|
||||
Ok(Self {
|
||||
config,
|
||||
http,
|
||||
stream_http,
|
||||
seen_event_ids: Arc::new(Mutex::new(EventDeduplicator::default())),
|
||||
last_event_id: Arc::new(Mutex::new(None)),
|
||||
state_tx,
|
||||
|
|
@ -140,7 +166,7 @@ impl GatewayClient {
|
|||
) -> Result<(), GatewayError> {
|
||||
self.set_state(ConnectionState::Connecting);
|
||||
let mut request = self
|
||||
.http
|
||||
.stream_http
|
||||
.get(format!("{}/desktop/v1/events/stream", self.config.base_url))
|
||||
.header(ACCEPT, HeaderValue::from_static("text/event-stream"));
|
||||
if let Some(token) = &self.config.bearer_token {
|
||||
|
|
|
|||
正在加载…
添加表格
在新工单中引用