[agent] Fix Gemma tool call parsing — add to supported models + raw token fallback #932

Closed

sleepy wants to merge 0 commits from fix/gemma-tool-calls into dev

sleepy commented

2026-06-04 14:16:10 +02:00

Owner

Problem

Gemma models served via llama.cpp were treated as non-API models because "gemma" was missing from the _model_supports_tools keyword list (only "gemini" was present). This meant:

No tool schemas were sent in the LLM request payload
Gemma fell back to raw <|tool_call|> token format
The tool parser had no pattern to handle this format

Changes

1. `src/agent/core.py` — PRIMARY FIX

Added "gemma" to the _model_supports_tools keyword list (line 177). When tool schemas are sent, llama.cpp Gemma emits proper native tool_calls JSON which already works with the existing converter.

2. `src/tool_parsing.py` — FALLBACK

Added Pattern 6: Gemma <|tool_call|> token format parsing as a fallback for when schemas aren't sent (e.g., older llama.cpp versions, manual endpoint_supports=False).

_GEMMA_TOOL_CALL_RE regex handles: <|tool_call|>call:shell:run_shell_command{command: "ls -la"}<|tool_call|>
_parse_gemma_tool_call() helper tries function name then category name via TOOL_NAME_MAP
Handles quasi-JSON with unquoted keys
Added to strip_tool_blocks() to prevent token leakage

Testing

python -m pytest tests/ -q --tb=no → 1152 passed, 9 failed (same baseline)
Zero regressions

## Problem Gemma models served via llama.cpp were treated as non-API models because "gemma" was missing from the `_model_supports_tools` keyword list (only "gemini" was present). This meant: 1. No tool schemas were sent in the LLM request payload 2. Gemma fell back to raw `<|tool_call|>` token format 3. The tool parser had no pattern to handle this format ## Changes ### 1. `src/agent/core.py` — PRIMARY FIX Added `"gemma"` to the `_model_supports_tools` keyword list (line 177). When tool schemas are sent, llama.cpp Gemma emits proper native `tool_calls` JSON which already works with the existing converter. ### 2. `src/tool_parsing.py` — FALLBACK Added Pattern 6: Gemma `<|tool_call|>` token format parsing as a fallback for when schemas aren't sent (e.g., older llama.cpp versions, manual `endpoint_supports=False`). - `_GEMMA_TOOL_CALL_RE` regex handles: `<|tool_call|>call:shell:run_shell_command{command: "ls -la"}<|tool_call|>` - `_parse_gemma_tool_call()` helper tries function name then category name via TOOL_NAME_MAP - Handles quasi-JSON with unquoted keys - Added to `strip_tool_blocks()` to prevent token leakage ## Testing - `python -m pytest tests/ -q --tb=no` → 1152 passed, 9 failed (same baseline) - Zero regressions

sleepy added 1 commit

2026-06-04 14:16:10 +02:00

[agent] Fix Gemma tool call parsing — add to supported models + raw token fallback bf7f45a7e8

- Add 'gemma' to _model_supports_tools keyword list in core.py so tool
  schemas are sent to llama.cpp Gemma endpoints, enabling native tool_calls
- Add Pattern 6 in tool_parsing.py to parse Gemma's raw <|tool_call|>
  token format as fallback when schemas aren't sent
- Add _parse_gemma_tool_call helper with quasi-JSON key quoting
- Add Gemma pattern to strip_tool_blocks() to prevent token leakage

sleepy referenced this pull request from a commit

2026-06-04 14:18:02 +02:00

Merge branch 'fix/gemma-tool-calls' into dev

sleepy closed this pull request

2026-06-04 14:18:12 +02:00