[testing] Zero test coverage for agent loop and entire tool system #679

New issue

Closed

opened 2026-06-02 23:42:40 +02:00 by sleepy · 1 comment

sleepy commented

2026-06-02 23:42:40 +02:00

Owner

No tests found for agent loop or tool system

Affected files (all untested)

src/agent_loop.py — 2158 lines, core streaming agent loop
src/agent_tools.py — 134 lines, facade module
src/agent_runs.py — 201 lines, detached run manager
src/tool_parsing.py — 442 lines, tool call parsing
src/tool_execution.py — 863 lines, tool dispatcher
src/tool_implementations.py — 4043 lines, tool implementations
src/tool_security.py — 74 lines, security policy
src/tool_schemas.py — 1185 lines, JSON schemas
src/tool_index.py — 475 lines, RAG-based tool selection

Evidence

Search for test_*agent* and test_*tool* found zero results in the entire repository
No tests/ directory or test files adjacent to these modules

Critical gaps

Tool parsing — regex-based parsing of 5 different formats is extremely fragile; changes to any regex could silently break tool execution
Security policy — is_public_blocked_tool() and owner_is_admin_or_single_user() are security-critical with zero test coverage
Agent loop — the 20-round loop with budget enforcement, loop-breaker, verifier, and tool result formatting is entirely untested
Tool dispatch — the if/elif chain in execute_tool_block is brittle; adding a new tool without updating it silently drops the tool
Run manager — _RUNS dict, eviction timer, subscriber management — all untested

Suggested priority tests

test_tool_parsing.py — cover all 5 parse formats, edge cases (empty, malformed, mixed)
test_tool_security.py — cover admin/public blocking, auth failure handling
test_agent_runs.py — cover start/subscribe/stop/eviction timer
test_tool_execution.py — cover dispatcher routing, disabled tools, admin checks

## No tests found for agent loop or tool system ### Affected files (all untested) - `src/agent_loop.py` — 2158 lines, core streaming agent loop - `src/agent_tools.py` — 134 lines, facade module - `src/agent_runs.py` — 201 lines, detached run manager - `src/tool_parsing.py` — 442 lines, tool call parsing - `src/tool_execution.py` — 863 lines, tool dispatcher - `src/tool_implementations.py` — 4043 lines, tool implementations - `src/tool_security.py` — 74 lines, security policy - `src/tool_schemas.py` — 1185 lines, JSON schemas - `src/tool_index.py` — 475 lines, RAG-based tool selection ### Evidence - Search for `test_*agent*` and `test_*tool*` found **zero results** in the entire repository - No `tests/` directory or test files adjacent to these modules ### Critical gaps 1. **Tool parsing** — regex-based parsing of 5 different formats is extremely fragile; changes to any regex could silently break tool execution 2. **Security policy** — `is_public_blocked_tool()` and `owner_is_admin_or_single_user()` are security-critical with zero test coverage 3. **Agent loop** — the 20-round loop with budget enforcement, loop-breaker, verifier, and tool result formatting is entirely untested 4. **Tool dispatch** — the if/elif chain in `execute_tool_block` is brittle; adding a new tool without updating it silently drops the tool 5. **Run manager** — `_RUNS` dict, eviction timer, subscriber management — all untested ### Suggested priority tests 1. `test_tool_parsing.py` — cover all 5 parse formats, edge cases (empty, malformed, mixed) 2. `test_tool_security.py` — cover admin/public blocking, auth failure handling 3. `test_agent_runs.py` — cover start/subscribe/stop/eviction timer 4. `test_tool_execution.py` — cover dispatcher routing, disabled tools, admin checks

sleepy added the

bug

enhancement

labels

2026-06-02 23:42:40 +02:00

sleepy closed this issue

2026-06-04 06:40:23 +02:00

sleepy commented

2026-06-04 06:40:27 +02:00

Author

Owner

Fixed via PR #914 — added 46 agent/tool tests covering tool types, name map, parsing, security, selection, system prompt.