mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-12-30 22:20:20 -06:00
Code Issues Packages Projects Releases Wiki Activity

feat: add cuda13 images (#7404)

Browse Source 
* chore(ci): add cuda13 jobs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add to pipelines and to capabilities. Start to work on the gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* capabilities: try to detect by looking at /usr/local

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* neutts

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backends.yaml

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* add cuda13 l4t requirements.txt

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* add cuda13 requirements.txt

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Pin vllm

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Not all backends are compatible

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* add vllm to requirements

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* vllm is not pre-compiled for cuda 13

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
2025-12-02 14:24:35 +01:00
committed by GitHub
parent 9872bdf455
commit cfd95745ed
22 changed files with 631 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
backend/python/chatterbox/requirements-cublas13.txt Normal file
View File

@@ -0,0 +1,8 @@
--extra-index-url https://download.pytorch.org/whl/cu130
torch
torchaudio
transformers
numpy>=1.24.0,<1.26.0
# https://github.com/mudler/LocalAI/pull/6240#issuecomment-3329518289
chatterbox-tts@git+https://git@github.com/mudler/chatterbox.git@faster
accelerate

0
backend/python/chatterbox/requirements-l4t.txt → backend/python/chatterbox/requirements-l4t12.txt
View File

7
backend/python/chatterbox/requirements-l4t13.txt Normal file
View File

@@ -0,0 +1,7 @@
--extra-index-url https://pypi.jetson-ai-lab.io/sbsa/cu130
torch
torchaudio
transformers
numpy>=1.24.0,<1.26.0
chatterbox-tts@git+https://git@github.com/mudler/chatterbox.git@faster
accelerate

2
backend/python/common/libbackend.sh
View File

@@ -211,7 +211,7 @@ function init() {
# - hipblas
# - intel
function getBuildProfile() {
if [ x"${BUILD_TYPE:-}" == "xcublas" ]; then
if [ x"${BUILD_TYPE:-}" == "xcublas" ] || [ x"${BUILD_TYPE:-}" == "xl4t" ]; then
if [ ! -z "${CUDA_MAJOR_VERSION:-}" ]; then
echo ${BUILD_TYPE}${CUDA_MAJOR_VERSION}
else

7
backend/python/diffusers/install.sh
View File

@@ -16,4 +16,11 @@ if [ "x${BUILD_PROFILE}" == "xintel" ]; then
EXTRA_PIP_INSTALL_FLAGS+=" --upgrade --index-strategy=unsafe-first-match"
fi
# Use python 3.12 for l4t
if [ "x${BUILD_PROFILE}" == "xl4t12" ] || [ "x${BUILD_PROFILE}" == "xl4t13" ]; then
PYTHON_VERSION="3.12"
PYTHON_PATCH="12"
PY_STANDALONE_TAG="20251120"
fi
installRequirements

12
backend/python/diffusers/requirements-cublas13.txt Normal file
View File

@@ -0,0 +1,12 @@
--extra-index-url https://download.pytorch.org/whl/cu130
git+https://github.com/huggingface/diffusers
opencv-python
transformers
torchvision
accelerate
compel
peft
sentencepiece
torch
ftfy
optimum-quanto

0
backend/python/diffusers/requirements-l4t.txt → backend/python/diffusers/requirements-l4t12.txt
View File

12
backend/python/diffusers/requirements-l4t13.txt Normal file
View File

@@ -0,0 +1,12 @@
--extra-index-url https://pypi.jetson-ai-lab.io/sbsa/cu130
torch
git+https://github.com/huggingface/diffusers
transformers
accelerate
compel
peft
optimum-quanto
numpy<2
sentencepiece
torchvision
ftfy

9
backend/python/faster-whisper/requirements-cublas13.txt Normal file
View File

@@ -0,0 +1,9 @@
--extra-index-url https://download.pytorch.org/whl/cu130
torch==2.9.1
faster-whisper
opencv-python
accelerate
compel
peft
sentencepiece
optimum-quanto

7
backend/python/kokoro/requirements-cublas13.txt Normal file
View File

@@ -0,0 +1,7 @@
--extra-index-url https://download.pytorch.org/whl/cu130
torch==2.9.1
torchaudio==2.9.1
transformers
accelerate
kokoro
soundfile

0
backend/python/kokoro/requirements-l4t.txt → backend/python/kokoro/requirements-l4t12.txt
View File

0
backend/python/neutts/requirements-l4t.txt → backend/python/neutts/requirements-l4t12.txt
View File

5
backend/python/rerankers/requirements-cublas13.txt Normal file
View File

@@ -0,0 +1,5 @@
--extra-index-url https://download.pytorch.org/whl/cu130
transformers
accelerate
torch==2.9.1
rerankers[transformers]

8
backend/python/rfdetr/requirements-cublas13.txt Normal file
View File

@@ -0,0 +1,8 @@
--extra-index-url https://download.pytorch.org/whl/cu130
torch==2.9.1
rfdetr
opencv-python
accelerate
inference
peft
optimum-quanto

9
backend/python/transformers/requirements-cublas13.txt Normal file
View File

@@ -0,0 +1,9 @@
--extra-index-url https://download.pytorch.org/whl/cu130
torch==2.9.0
llvmlite==0.43.0
numba==0.60.0
transformers
bitsandbytes
outetts
sentence-transformers==5.1.0
protobuf==6.33.1