[mtp] Fix EOS token leak in output_token_ids (#54) #60

Merged

sleepy merged 1 commit from fix/54-mtp-eos-output into stable/pre-q4kv

2026-05-15 01:01:43 +02:00

sleepy commented

2026-05-15 00:59:10 +02:00

Owner

Summary

Trim stop token from request.output_token_ids on 3 early-return finish paths in _mtp_step()
Without this fix, <|im_end|> appears in output_text on finished requests
The fall-through safety net already handled this; the fix extends the same trimming to early returns

Root cause

request.append_output_token() was called before the stop-token check. On early-return paths, the appended stop token was never removed from output_token_ids.

Test results

5/5 MTP generation prompts: no EOS leak (was 4/5 before fix)
47 existing scheduler tests pass (1 pre-existing failure unrelated to this change)
Model: Qwen3.6-27B-MXFP4-MTP with MTP speculative decoding

Closes #54

## Summary - Trim stop token from `request.output_token_ids` on 3 early-return finish paths in `_mtp_step()` - Without this fix, `<|im_end|>` appears in `output_text` on finished requests - The fall-through safety net already handled this; the fix extends the same trimming to early returns ## Root cause `request.append_output_token()` was called before the stop-token check. On early-return paths, the appended stop token was never removed from `output_token_ids`. ## Test results - 5/5 MTP generation prompts: no EOS leak (was 4/5 before fix) - 47 existing scheduler tests pass (1 pre-existing failure unrelated to this change) - Model: Qwen3.6-27B-MXFP4-MTP with MTP speculative decoding Closes #54

sleepy added 1 commit

2026-05-15 00:59:10 +02:00

fix(mtp): trim stop token from output_token_ids on early-return paths 9ddc4c19e4

Three early-return paths in _mtp_step() appended the stop token to
output_token_ids before the stop check, then returned finished without
trimming. This caused <|im_end|> to leak into output_text.

Trim on: idle phase confirmed stop, has_draft accepted draft stop,
has_draft rejected verify stop.

sleepy merged commit 7c7f30f3b7 into stable/pre-q4kv

2026-05-15 01:01:43 +02:00

sleepy referenced this pull request from a commit

2026-05-15 01:01:43 +02:00

[mtp] Fix EOS token leak in output_token_ids (#54) (#60)

sleepy referenced this pull request

2026-05-15 01:01:52 +02:00

[bug] MTP continues generating past EOS token #54

sleepy referenced this pull request

2026-05-15 01:50:13 +02:00

Tool response token leakage #17

sleepy referenced this pull request