sleepy/llama.cpp
1
0
Fork 0
Code Issues 9 Pull Requests 1 Actions 102 Packages Projects Releases Wiki Activity

server: support load model on startup, support preset-only options (#18206)

Browse Source 
* server: support autoload model, support preset-only options

* add docs

* load-on-startup

* fix

* Update common/arg.cpp

Co-authored-by: Pascal <admin@serveurperso.com>

---------

Co-authored-by: Pascal <admin@serveurperso.com>
This commit is contained in:
Xuan-Son Nguyen
2025-12-20 09:25:27 +01:00
committed by GitHub
parent 74e05131e9
commit 9e39a1e6a9
7 changed files with 80 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/README.md
+3
View File
@@ -1480,6 +1480,9 @@ The precedence rule for preset options is as follows:
2. **Model-specific options** defined in the preset file (e.g. `[ggml-org/MY-MODEL...]`)
3. **Global options** defined in the preset file (`[*]`)
We also offer additional options that are exclusive to presets (these aren't treated as command-line arguments):
- `load-on-startup` (boolean): Controls whether the model loads automatically when the server starts
### Routing requests
Requests are routed according to the requested model name.