mirror of https://github.com/mudler/LocalAI.git synced 2026-01-01 07:01:09 -06:00

Files

Ludovic Leroux 939411300a Bump vLLM version + more options when loading models in vLLM (#1782 )

* Bump vLLM version to 0.3.2

* Add vLLM model loading options

* Remove transformers-exllama

* Fix install exllama

2024-03-01 22:48:53 +01:00

backend_pb2_grpc.py

2023-11-13 22:40:16 +01:00

backend_pb2.py

2024-03-01 22:48:53 +01:00

backend_vllm.py

2024-03-01 22:48:53 +01:00

Makefile

2024-01-06 13:32:28 +01:00

README.md

2023-11-13 22:40:16 +01:00

run.sh

2024-01-06 13:32:28 +01:00

test_backend_vllm.py

2023-12-21 08:35:15 +01:00

test.sh

2024-01-06 13:32:28 +01:00

Creating a separate environment for the vllm project

make vllm