mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-12-30 22:20:20 -06:00
Code Issues Packages Projects Releases Wiki Activity

Add NVCC_THREADS and MAX_JOB environment variables

Browse Source 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
This commit is contained in:
Ettore Di Giacinto
2025-11-21 16:14:13 +01:00
committed by GitHub
parent fa00aa0085
commit 6261c87b1b
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
backend/python/vllm/install.sh
View File

@@ -7,7 +7,8 @@ EXTRA_PIP_INSTALL_FLAGS="--no-build-isolation"
# https://github.com/vllm-project/vllm/issues/20079
# https://docs.vllm.ai/en/v0.8.3/serving/env_vars.html
# https://docs.redhat.com/it/documentation/red_hat_ai_inference_server/3.0/html/vllm_server_arguments/environment_variables-server-arguments
export NUM_JOBS=2
export NVCC_THREADS=2
export MAX_JOB=1
backend_dir=$(dirname $0)