mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-01-04 09:40:32 -06:00
Code Issues Packages Projects Releases Wiki Activity
4,673 Commits 67 Branches 133 Tags
2ebe37b671b77313859f190fee205aceece04444
Commit Graph

40 Commits

Author SHA1 Message Date
Richard Palethorpe
37f5e4f5c1 feat(whisper): Add diarization (tinydiarize) (#6184) 
Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2025-09-10 19:09:28 +02:00
Ettore Di Giacinto
739573e41b feat(flash_attention): set auto for flash_attention in llama.cpp (#6168) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-31 17:59:09 +02:00
Ettore Di Giacinto
9621edb4c5 feat(diffusers): add support for wan2.2 (#6153) 
* feat(diffusers): add support for wan2.2

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(ci): use ttl.sh for PRs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add ftfy deps

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "chore(ci): use ttl.sh for PRs"

This reverts commit c9fc3ecf28.

* Simplify

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: do not pin torch/torchvision on cuda12

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-08-28 10:26:42 +02:00
Ettore Di Giacinto
3d22bfc27c feat(stablediffusion-ggml): add support to ref images (flux Kontext) (#5935) 
* feat(stablediffusion-ggml): add support to ref images

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add it to the model gallery

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-07-30 22:42:34 +02:00
Ettore Di Giacinto
949e5b9be8 feat(rfdetr): add object detection API (#5923) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-07-27 22:02:51 +02:00
Ettore Di Giacinto
b29544d747 feat: split piper from main binary (#5858) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-07-19 08:31:33 +02:00
Ettore Di Giacinto
dfadc3696e feat(llama.cpp): allow to set kv-overrides (#5745) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-06-28 21:26:07 +02:00
Richard Palethorpe
bf6426aef2 feat: Realtime API support reboot (#5392) 
* feat(realtime): Initial Realtime API implementation

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: go mod tidy

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat: Implement transcription only mode for realtime API

Reduce the scope of the real time API for the initial realease and make
transcription only mode functional.

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(build): Build backends on a separate layer to speed up core only changes

Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-25 22:25:05 +02:00
Ettore Di Giacinto
3b0cf52f6a feat(llama.cpp): add reranking (#5396) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-22 21:49:30 +02:00
Ettore Di Giacinto
2c9279a542 feat(video-gen): add endpoint for video generation (#5247) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-26 18:05:01 +02:00
Ettore Di Giacinto
61cc76c455 chore(autogptq): drop archived backend (#5214) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-19 15:52:29 +02:00
Ettore Di Giacinto
67f7bffd18 chore(deps): update llama.cpp and sync with upstream changes (#4950) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-03-06 00:40:58 +01:00
Brandon Beiler
6a6e1a0ea9 feat(vllm): Additional vLLM config options (Disable logging, dtype, and Per-Prompt media limits) (#4855) 
* Adding the following vLLM config options: disable_log_status, dtype, limit_mm_per_prompt

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>

* using " marks in the config.yaml file

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>

* adding in missing colon

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>

---------

Signed-off-by: TheDropZone <brandonbeiler@gmail.com>
 2025-02-18 19:27:58 +01:00
Ettore Di Giacinto
1d6afbd65d feat(llama.cpp): Add support to grammar triggers (#4733) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-02-02 13:25:03 +01:00
mintyleaf
96f8ec0402 feat: add machine tag and inference timings (#4577) 
* Add machine tag option, add extraUsage option, grpc-server -> proto -> endpoint extraUsage data is broken for now

Signed-off-by: mintyleaf <mintyleafdev@gmail.com>

* remove redurant timing fields, fix not working timings output

Signed-off-by: mintyleaf <mintyleafdev@gmail.com>

* use middleware for Machine-Tag only if tag is specified

Signed-off-by: mintyleaf <mintyleafdev@gmail.com>

---------

Signed-off-by: mintyleaf <mintyleafdev@gmail.com>
 2025-01-17 17:05:58 +01:00
Ettore Di Giacinto
d4c1746c7d feat(llama.cpp): expose cache_type_k and cache_type_v for quant of kv cache (#4329) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-12-06 10:23:59 +01:00
Ettore Di Giacinto
44a5dac312 feat(backend): add stablediffusion-ggml (#4289) 
* feat(backend): add stablediffusion-ggml

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(ci): track stablediffusion-ggml

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Use default scheduler and sampler if not specified

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Move cfg scale out of diffusers block

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Make it working

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: set free_params_immediately to false to call the model in sequence

https://github.com/leejet/stable-diffusion.cpp/issues/366

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-12-03 22:41:22 +01:00
Ettore Di Giacinto
b1ea9318e6 feat(silero): add Silero-vad backend (#4204) 
* feat(vad): add silero-vad backend (WIP)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vad): add API endpoint

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(vad): correctly place the onnxruntime libs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(vad): hook silero-vad to binary and container builds

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gRPC): register VAD Server

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(Makefile): consume ONNX_OS consistently

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(Makefile): handle macOS

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2024-11-20 14:48:40 +01:00
Ettore Di Giacinto
947224b952 feat(diffusers): allow multiple lora adapters (#4081) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-11-05 15:14:33 +01:00
Ettore Di Giacinto
61c964dce7 fix(grpc): pass by modelpath (#4023) 
Instead of trying to derive it from the model file. In backends that
specify HF url this results in a fragile logic.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-10-31 12:12:22 +01:00
Ettore Di Giacinto
835932e95e feat: update proto file 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-10-23 15:46:06 +02:00
siddimore
f84b55d1ef feat: Add Get Token Metrics to GRPC server (#3687) 
* Add Get Token Metrics to GRPC server

Signed-off-by: Siddharth More <siddimore@gmail.com>

* Expose LocalAI endpoint

Signed-off-by: Siddharth More <siddimore@gmail.com>

---------

Signed-off-by: Siddharth More <siddimore@gmail.com>
 2024-10-01 14:41:20 +02:00
siddimore
50a3b54e34 feat(api): add correlationID to Track Chat requests (#3668) 
* Add CorrelationID to chat request

Signed-off-by: Siddharth More <siddimore@gmail.com>

* remove get_token_metrics

Signed-off-by: Siddharth More <siddimore@gmail.com>

* Add CorrelationID to proto

Signed-off-by: Siddharth More <siddimore@gmail.com>

* fix correlation method name

Signed-off-by: Siddharth More <siddimore@gmail.com>

* Update core/http/endpoints/openai/chat.go

Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: Siddharth More <siddimore@gmail.com>

* Update core/http/endpoints/openai/chat.go

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: Siddharth More <siddimore@gmail.com>

---------

Signed-off-by: Siddharth More <siddimore@gmail.com>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2024-09-28 17:23:56 +02:00
Ettore Di Giacinto
191bc2e50a feat(api): allow to pass audios to backends (#3603) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-09-19 12:26:53 +02:00
Ettore Di Giacinto
fbb9facda4 feat(api): allow to pass videos to backends (#3601) 
This prepares the API to receive videos as well for video understanding.

It works similarly to images, where the request should be in the form:

{
 "type": "video_url",
 "video_url": { "url": "url or base64 data" }
}

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-09-19 11:21:59 +02:00
Dave
81ae92f017 feat: elevenlabs sound-generation api (#3355) 
* initial version of elevenlabs compatible soundgeneration api and cli command

Signed-off-by: Dave Lee <dave@gray101.com>

* minor cleanup

Signed-off-by: Dave Lee <dave@gray101.com>

* restore TTS, add test

Signed-off-by: Dave Lee <dave@gray101.com>

* remove stray s

Signed-off-by: Dave Lee <dave@gray101.com>

* fix

Signed-off-by: Dave Lee <dave@gray101.com>

---------

Signed-off-by: Dave Lee <dave@gray101.com>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2024-08-24 00:20:28 +00:00
Ettore Di Giacinto
03b1cf51fd feat(whisper): add translate option (#2649) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-06-24 19:21:22 +02:00
Chakib Benziane
b99182c8d4 TTS API improvements (#2308) 
* update doc on COQUI_LANGUAGE env variable

Signed-off-by: blob42 <contact@blob42.xyz>

* return errors from tts gRPC backend

Signed-off-by: blob42 <contact@blob42.xyz>

* handle speaker_id and language in coqui TTS backend

Signed-off-by: blob42 <contact@blob42.xyz>

* TTS endpoint: add optional language paramter

Signed-off-by: blob42 <contact@blob42.xyz>

* tts fix: empty language string breaks non-multilingual models

Signed-off-by: blob42 <contact@blob42.xyz>

* allow tts param definition in config file

- consolidate TTS options under `tts` config entry

Signed-off-by: blob42 <contact@blob42.xyz>

* tts: update doc

Signed-off-by: blob42 <contact@blob42.xyz>

---------

Signed-off-by: blob42 <contact@blob42.xyz>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2024-06-01 18:26:27 +00:00
Ettore Di Giacinto
e49ea0123b feat(llama.cpp): add flash_attention and no_kv_offloading (#2310) 
feat(llama.cpp): add flash_attn and no_kv_offload

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-05-13 19:07:51 +02:00
Ettore Di Giacinto
b664edde29 feat(rerankers): Add new backend, support jina rerankers API (#2121) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-25 00:19:02 +02:00
Taikono-Himazin
03adc1f60d Add tensor_parallel_size setting to vllm setting items (#2085) 
Signed-off-by: Taikono-Himazin <kazu@po.harenet.ne.jp>
 2024-04-20 14:37:02 +00:00
Ettore Di Giacinto
e843d7df0e feat(grpc): return consumed token count and update response accordingly (#2035) 
Fixes: #1920
 2024-04-15 19:47:11 +02:00
Ludovic Leroux
12c0d9443e feat: use tokenizer.apply_chat_template() in vLLM (#1990) 
Use tokenizer.apply_chat_template() in vLLM

Signed-off-by: Ludovic LEROUX <ludovic@inpher.io>
 2024-04-11 19:20:22 +02:00
Richard Palethorpe
643d85d2cc feat(stores): Vector store backend (#1795) 
Add simple vector store backend

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2024-03-22 21:14:04 +01:00
Ettore Di Giacinto
20136ca8b7 feat(tts): add Elevenlabs and OpenAI TTS compatibility layer (#1834) 
* feat(elevenlabs): map elevenlabs API support to TTS

This allows elevenlabs Clients to work automatically with LocalAI by
supporting the elevenlabs API.

The elevenlabs server endpoint is implemented such as it is wired to the
TTS endpoints.

Fixes: https://github.com/mudler/LocalAI/issues/1809

* feat(openai/tts): compat layer with openai tts

Fixes: #1276

* fix: adapt tts CLI
 2024-03-14 23:08:34 +01:00
Ludovic Leroux
939411300a Bump vLLM version + more options when loading models in vLLM (#1782) 
* Bump vLLM version to 0.3.2

* Add vLLM model loading options

* Remove transformers-exllama

* Fix install exllama
 2024-03-01 22:48:53 +01:00
Ettore Di Giacinto
cb7512734d transformers: correctly load automodels (#1643) 
* backends(transformers): use AutoModel with LLM types

* examples: animagine-xl

* Add codellama examples
 2024-01-26 00:13:21 +01:00
Ettore Di Giacinto
9e653d6abe feat: 🐍 add mamba support (#1589) 
feat(mamba): Initial import

This is a first iteration of the mamba backend, loosely based on
mamba-chat(https://github.com/havenhq/mamba-chat).
 2024-01-19 23:42:50 +01:00
Ettore Di Giacinto
7641f92cde feat(diffusers): update, add autopipeline, controlnet (#1432) 
* feat(diffusers): update, add autopipeline, controlenet

* tests with AutoPipeline

* simplify logic
 2023-12-13 19:20:22 +01:00
Ettore Di Giacinto
ad0e30bca5 refactor: move backends into the backends directory (#1279) 
* refactor: move backends into the backends directory

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor: move main close to implementation for every backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2023-11-13 22:40:16 +01:00