mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-01-06 02:29:54 -06:00
Code Issues Packages Projects Releases Wiki Activity
1,673 Commits 67 Branches 133 Tags
a10a95208564bd9fd41b13fd7c8b7f6c48c7c36a
Commit Graph

20 Commits

Author SHA1 Message Date
Ettore Di Giacinto
c89271b2e4 feat(llama.cpp): add distributed llama.cpp inferencing (#2324) 
* feat(llama.cpp): support distributed llama.cpp

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat: let tweak how chat messages are merged together

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Makefile: register to ALL_GRPC_BACKENDS

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactoring, allow disable auto-detection of backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* minor fixups

Signed-off-by: mudler <mudler@localai.io>

* feat: add cmd to start rpc-server from llama.cpp

Signed-off-by: mudler <mudler@localai.io>

* ci: add ccache

Signed-off-by: mudler <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: mudler <mudler@localai.io>
 2024-05-15 01:17:02 +02:00
Ettore Di Giacinto
0674893649 Update .env 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2024-04-28 23:56:10 +02:00
cryptk
24d7dadfed feat: kong cli refactor fixes #1955 (#1974) 
* feat: migrate to alecthomas/kong for CLI

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: bring in new flag for granular log levels

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* chore: go mod tidy

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: allow loading cli flag values from ["./localai.yaml", "~/.config/localai.yaml", "/etc/localai.yaml"] in that order

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: load from .env file instead of a yaml file

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: better loading for environment files

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat(doc): add initial documentation about configuration

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: remove test log lines

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: integrate new documentation into existing pages

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: add documentation on .env files

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: cleanup some documentation table errors

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: refactor CLI logic out to it's own package under core/cli

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

---------

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
 2024-04-11 09:19:24 +02:00
Ettore Di Giacinto
57222497ec fix(docker-compose): update docker compose file (#1824) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-03-13 17:57:45 +01:00
Ettore Di Giacinto
824612f1b4 feat: initial watchdog implementation (#1341) 
* feat: initial watchdog implementation

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

* fiuxups

* Add more output

* wip: idletime checker

* wire idle watchdog checks

* enlarge watchdog time window

* small fixes

* Use stopmodel

* Always delete process

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2023-11-26 18:36:23 +01:00
Ettore Di Giacinto
fdd95d1d86 feat: allow to run parallel requests (#1290) 
* feat: allow to run parallel requests

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixup

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2023-11-16 08:20:05 +01:00
Ettore Di Giacinto
0eae727366 🔥 add LaVA support and GPT vision API, Multiple requests for llama.cpp, return JSON types (#1254) 
* wip

* wip

* Make it functional

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* wip

* Small fixups

* do not inject space on role encoding, encode img at beginning of messages

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Add examples/config defaults

* Add include dir of current source dir

* cleanup

* fixes

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixups

* Revert "fixups"

This reverts commit f1a4731cca.

* fixes

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2023-11-11 13:14:59 +01:00
Ettore Di Giacinto
bdf3f95346 feat(python-grpc): allow to set max workers with PYTHON_GRPC_MAX_WORKERS (#1081) 
**Description**

this allows to customize the maximum number of grpc workers for python
backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2023-09-19 21:30:39 +02:00
Ettore Di Giacinto
cc74fc93b4 feat(llama.cpp): update (#1024) 
**Description**

This PR fixes #

**Notes for Reviewers**


**[Signed
commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)**
- [ ] Yes, I signed my commits.
 

<!--
Thank you for contributing to LocalAI! 

Contributing Conventions:

1. Include descriptive PR titles with [<component-name>] prepended.
2. Build and test your changes before submitting a PR. 
3. Sign your commits

By following the community's contribution conventions upfront, the
review process will
be accelerated and your PR merged more quickly.
-->

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2023-09-08 18:38:22 +02:00
scott4290
71611d2dec docs: base-Update comments in .env for cublas, openblas, clblas (#867) 2023-08-07 08:22:42 +00:00
mudler
4de7f55f2f Make REBUILD=false default behavior 
Add notice to documentation

Signed-off-by: mudler <mudler@localai.io>
 2023-07-07 00:29:14 +02:00
mudler
3dd632fd5a Update .env 2023-06-28 18:28:53 +02:00
Ettore Di Giacinto
05a3d569b0 feat: allow to override model config (#323) 2023-05-20 17:03:53 +02:00
Sébastien Prud'homme
1a7587ee48 feat: add an environment variable to manage rebuild in Docker image (#290) 
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2023-05-18 19:18:32 +02:00
Marc R Kellerman
05f35b182c fix(makefile): fix go-gpt2 folder and add verification before git clone (#51) 
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2023-04-22 00:29:32 +02:00
Ettore Di Giacinto
4b7e83056d Update .env 2023-04-21 01:47:35 +02:00
Ettore Di Giacinto
d517a54e28 Major API enhancements (#44) 2023-04-20 18:33:02 +02:00
Ettore Di Giacinto
db5ac715f3 Use a reasonable default context size (#31) 2023-04-17 18:45:42 +02:00
Marc R Kellerman
c37175271f feature: makefile & updates (#23) 
Co-authored-by: mudler <mudler@c3os.io>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2023-04-15 16:39:07 -07:00
mudler
8042e9a2d6 Add docker-compose 
Fixes #14

Signed-off-by: mudler <mudler@c3os.io>
 2023-04-13 01:13:14 +02:00