[chat/streaming] _active_streams dict has no eviction, TTL, or size limit #736

New issue

Closed

opened 2026-06-03 00:10:52 +02:00 by sleepy · 0 comments

sleepy commented

2026-06-03 00:10:52 +02:00

Owner

Problem

routes/chat_routes.py line 43 defines _active_streams: Dict[str, dict] = {} — a module-level dictionary that tracks active streaming sessions.

Entries are added when streaming starts and removed in finally blocks. However:

If the finally block fails (e.g., exception during partial-save), the entry persists forever
No TTL or expiry on stale entries
No size limit — a burst of concurrent streams could grow this without bound
No periodic cleanup

This is the same pattern flagged in #682 for _RUNS in agent_runs.py.

Impact

Memory leak in long-running processes if finally cleanup is skipped
Stale entries make /api/chat/stream_status return stale data

Suggested fix

Add a TTL to entries and reject stale lookups (e.g., entries older than 30 minutes are invalid)
Add a periodic cleanup coroutine or a max-size eviction policy
Or merge this with the agent_runs tracking into a unified stream registry

## Problem `routes/chat_routes.py` line 43 defines `_active_streams: Dict[str, dict] = {}` — a module-level dictionary that tracks active streaming sessions. Entries are added when streaming starts and removed in `finally` blocks. However: 1. If the `finally` block fails (e.g., exception during partial-save), the entry persists forever 2. No TTL or expiry on stale entries 3. No size limit — a burst of concurrent streams could grow this without bound 4. No periodic cleanup This is the same pattern flagged in #682 for `_RUNS` in agent_runs.py. ## Impact - Memory leak in long-running processes if `finally` cleanup is skipped - Stale entries make `/api/chat/stream_status` return stale data ## Suggested fix 1. Add a TTL to entries and reject stale lookups (e.g., entries older than 30 minutes are invalid) 2. Add a periodic cleanup coroutine or a max-size eviction policy 3. Or merge this with the `agent_runs` tracking into a unified stream registry

sleepy referenced this issue from a commit

2026-06-03 20:12:01 +02:00

[chat] Add eviction and TTL to _active_streams (#736)

sleepy referenced this issue from a pull request that will close it,

2026-06-03 20:12:16 +02:00

[chat] Add eviction and TTL to _active_streams (#736) #834

sleepy closed this issue

2026-06-03 20:12:21 +02:00