sleepy/llama.cpp
1
0
Fork 0
Code Issues 12 Pull Requests Actions 114 Packages Projects Releases Wiki Activity

common : inhibit lazy grammar sampler while reasoning is active (#20970)

Browse Source 
* common : inhibit grammar while reasoning budget is active

* cont : update force_pos in accept

* cont : fix tests

* cont : tweak should apply logic

* cont : return early not using grammar sampler

* Add tests

* cont : prevent backend sampling when reasoning budget enabled

* cont : fix typo

---------

Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com>
This commit is contained in:
Aldehir Rojas
2026-03-27 12:30:40 -05:00
committed by GitHub
parent ff934e29bc
commit 59d840209a
8 changed files with 295 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/server-common.cpp
+1 -1
View File
@@ -1110,7 +1110,7 @@ json oaicompat_chat_params_parse(
reasoning_budget = json_value(body, "thinking_budget_tokens", -1);
}
if (reasoning_budget >= 0 && !chat_params.thinking_end_tag.empty()) {
if (!chat_params.thinking_end_tag.empty()) {
llama_params["reasoning_budget_tokens"] = reasoning_budget;
llama_params["reasoning_budget_start_tag"] = chat_params.thinking_start_tag;
llama_params["reasoning_budget_end_tag"] = chat_params.thinking_end_tag;