[mtp] Integrate output parser into MTP step to prevent token leakage (#56) #63

Merged

sleepy merged 1 commit from fix/56-mtp-output-parser into main

2026-05-15 18:11:02 +02:00

sleepy commented

2026-05-15 17:58:03 +02:00

Owner

Summary

Fixes #56. The MTP step (_mtp_step) was bypassing the output parser session, causing structural tokens like user and <|im_end|> to leak into the output stream for Harmony models (GPT-4o etc.).

Changes

Route all MTP token emission through _get_output_parser_session() when available
Parser handles token filtering, stop detection, and tool call extraction
Falls back to raw detokenization when no parser session exists (unchanged behavior)
Calls parser_session.finalize() on request finish to flush text and extract tool calls

Test Results

81/82 scheduler tests pass. 1 failure is pre-existing on main (boundary snapshot token tracking).

Acceptance Criteria

MTP path uses output parser when available
No regression when no parser session (raw detokenize path preserved)
Parser is_stop triggers early stopping
Parser record_token controls output_token_ids
Tool calls extracted via finalize()

## Summary Fixes #56. The MTP step (`_mtp_step`) was bypassing the output parser session, causing structural tokens like `user` and `<|im_end|>` to leak into the output stream for Harmony models (GPT-4o etc.). ## Changes - Route all MTP token emission through `_get_output_parser_session()` when available - Parser handles token filtering, stop detection, and tool call extraction - Falls back to raw detokenization when no parser session exists (unchanged behavior) - Calls `parser_session.finalize()` on request finish to flush text and extract tool calls ## Test Results 81/82 scheduler tests pass. 1 failure is pre-existing on main (boundary snapshot token tracking). ## Acceptance Criteria - [x] MTP path uses output parser when available - [x] No regression when no parser session (raw detokenize path preserved) - [x] Parser `is_stop` triggers early stopping - [x] Parser `record_token` controls output_token_ids - [x] Tool calls extracted via `finalize()`

sleepy added 1 commit

2026-05-15 17:58:04 +02:00