mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-01-05 10:10:08 -06:00
Code Issues Packages Projects Releases Wiki Activity

fix(llama.cpp): do not default to linear rope (#5982)

Browse Source 
This seems to somehow sneaked in during the initial pass to gRPC server,
instead of setting linear rope when required, we did default to it if
not specified.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
2025-08-06 23:20:28 +02:00
committed by GitHub
parent 61ba98d43d
commit ec1276e5a9
1 changed files with 3 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
backend/cpp/llama-cpp/grpc-server.cpp
View File

@@ -313,9 +313,11 @@ static void params_parse(const backend::ModelOptions* request,
params.pooling_type = LLAMA_POOLING_TYPE_RANK;
}
if (request->ropescaling() == "none") { params.rope_scaling_type = LLAMA_ROPE_SCALING_TYPE_NONE; }
else if (request->ropescaling() == "yarn") { params.rope_scaling_type = LLAMA_ROPE_SCALING_TYPE_YARN; }
else { params.rope_scaling_type = LLAMA_ROPE_SCALING_TYPE_LINEAR; }
else if (request->ropescaling() == "linear") { params.rope_scaling_type = LLAMA_ROPE_SCALING_TYPE_LINEAR; }
if ( request->yarnextfactor() != 0.0f ) {
params.yarn_ext_factor = request->yarnextfactor();
}