mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-01-05 10:10:08 -06:00
Code Issues Packages Projects Releases Wiki Activity
1,581 Commits 68 Branches 133 Tags
f3bcc648e7040fc6803c3cd4fd67970837a52b9d
Commit Graph

74 Commits

Author SHA1 Message Date
Ettore Di Giacinto
b69ff46c7e feat(startup): show CPU/GPU information with --debug (#2241) 
Signed-off-by: mudler <mudler@localai.io>
 2024-05-05 09:10:23 +02:00
Ettore Di Giacinto
117c9873e1 fix(webui): display small navbar with smaller screens (#2240) 
Signed-off-by: mudler <mudler@localai.io>
 2024-05-04 23:38:39 +02:00
Ettore Di Giacinto
530bec9c64 feat(llama.cpp): do not specify backends to autoload and add llama.cpp variants (#2232) 
* feat(initializer): do not specify backends to autoload

We can simply try to autoload the backends extracted in the asset dir.
This will allow to build variants of the same backend (for e.g. with different instructions sets),
so to have a single binary for all the variants.

Signed-off-by: mudler <mudler@localai.io>

* refactor(prepare): refactor out llama.cpp prepare steps

Make it so are idempotent and that we can re-build

Signed-off-by: mudler <mudler@localai.io>

* [TEST] feat(build): build noavx version along

Signed-off-by: mudler <mudler@localai.io>

* build: make build parallel

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* build: do not override CMAKE_ARGS

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* build: add fallback variant

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(huggingface-langchain): fail if no token is set

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(huggingface-langchain): rename

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: do not autoload local-store

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: give priority between the listed backends

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: mudler <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-05-04 17:56:12 +02:00
Ettore Di Giacinto
54faaa87ea fix(webui): correct documentation URL for text2img (#2233) 
Signed-off-by: mudler <mudler@localai.io>
Co-authored-by: Dave <dave@gray101.com>
 2024-05-04 00:25:13 +00:00
Ettore Di Giacinto
b58274b8a2 feat(ui): support multilineand style ul (#2226) 
* feat(ui/chat): handle multiline in the input field

Signed-off-by: mudler <mudler@localai.io>

* feat(ui/chat): correctly display multiline messages

Signed-off-by: mudler <mudler@localai.io>

* feat(ui/chat): add list style

Signed-off-by: mudler <mudler@localai.io>

---------

Signed-off-by: mudler <mudler@localai.io>
 2024-05-03 00:43:02 +02:00
Ettore Di Giacinto
2c5a46bc34 feat(ux): Add chat, tts, and image-gen pages to the WebUI (#2222) 
* feat(webui): Add chat page

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(webui): Add image-gen page

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(webui): Add tts page

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-05-02 21:14:10 +02:00
Dave
c4f958e11b refactor(application): introduce application global state (#2072) 
* start breaking up the giant channel refactor now that it's better understood - easier to merge bites

Signed-off-by: Dave Lee <dave@gray101.com>

* add concurrency and base64 back in, along with new base64 tests.

Signed-off-by: Dave Lee <dave@gray101.com>

* Automatic rename of whisper.go's Result to TranscriptResult

Signed-off-by: Dave Lee <dave@gray101.com>

* remove pkg/concurrency - significant changes coming in split 2

Signed-off-by: Dave Lee <dave@gray101.com>

* fix comments

Signed-off-by: Dave Lee <dave@gray101.com>

* add list_model service as another low-risk service to get it out of the way

Signed-off-by: Dave Lee <dave@gray101.com>

* split backend config loader into seperate file from the actual config struct. No changes yet, just reduce cognative load with smaller files of logical blocks

Signed-off-by: Dave Lee <dave@gray101.com>

* rename state.go ==> application.go

Signed-off-by: Dave Lee <dave@gray101.com>

* fix lost import?

Signed-off-by: Dave Lee <dave@gray101.com>

---------

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-29 17:42:37 +00:00
Dave
11c48a0004 fix: security scanner warning noise: error handlers part 2 (#2145) 
check off a few more error handlers

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-29 15:11:42 +02:00
Ettore Di Giacinto
e8d44447ad feat(gallery): support model deletion (#2173) 
* feat(gallery): op now supports deletion of models

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Wire things with WebUI(WIP)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* minor improvements

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-28 23:42:46 +02:00
Ettore Di Giacinto
fb2a05ff43 feat(gallery): display job status also during navigation (#2151) 
* feat(gallery): keep showing progress also when refreshing

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(intel-gpu): better defaults

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat: make it thread-safe

Signed-off-by: mudler <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: mudler <mudler@localai.io>
 2024-04-27 09:08:33 +02:00
Dave
2dc1fa2474 fix: config_file_watcher.go - root all file reads for safety (#2144) 
callHandler() now has all file access rooted within DynamicConfigDir

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-26 16:46:35 +00:00
Dave
006306b183 fix: use bluemonday as recommended by blackfriday (#2142) 
use bluemonday as recommended by blackfriday

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-26 10:34:50 +02:00
Dave
2cd4936c99 fix: security scanner warning noise: error handlers part 1 (#2141) 
first group of error handlers to reduce security scanner warning noise level

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-26 10:34:31 +02:00
Dave
c8dd8e5ef4 fix: reduce chmod permissions for created files and directories (#2137) 
quiet more security scanner issues: pass one of chmod restriction to remove group and other permissions

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-26 00:47:06 +02:00
cryptk
d98063e80e fix: api key polling was not using correct filepath (#2132) 
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
 2024-04-25 20:06:22 +00:00
Ettore Di Giacinto
b664edde29 feat(rerankers): Add new backend, support jina rerankers API (#2121) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-25 00:19:02 +02:00
Ettore Di Giacinto
4fffc47e77 deps(llama.cpp): update, use better model for function call tests (#2119) 
deps(llama.cpp): update

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-24 18:44:04 +02:00
Ettore Di Giacinto
ac56ac2b2d fix(gallery): show a fake image if no there is no icon (#2111) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-23 20:10:58 +02:00
Ettore Di Giacinto
a09fe1b9ba fix(gallery): set margin for images 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-23 20:00:20 +02:00
Ettore Di Giacinto
d344daf129 feat(models-ui): minor visual enhancements (#2109) 
Show image if present, URL, tags, and better display buttons

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-23 18:43:25 +02:00
cryptk
3411e072ca Fix cleanup sonarqube findings (#2106) 
* fix: update dockerignore and gitignore to exclude sonarqube work dir

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: remove useless equality check

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: use sonarqube Dockerfile recommendations

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

---------

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
 2024-04-23 18:43:00 +02:00
Ettore Di Giacinto
0d8bf91699 feat: Galleries UI (#2104) 
* WIP: add models to webui

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Register routes

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: don't cache models

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* small fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: fixup multiple installs (strings.Clone)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-23 09:22:58 +02:00
Ikko Eltociear Ashimine
220958a87c fix: typo in models.go (#2099) 2024-04-22 04:34:59 +00:00
Ettore Di Giacinto
f3f6535aad fix: rename fiber entrypoint from http/api to http/app (#2096) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Dave <dave@gray101.com>
 2024-04-21 22:39:28 +02:00
Ettore Di Giacinto
180cd4ccda fix(llama.cpp-ggml): fixup max_tokens for old backend (#2094) 
fix(llama.cpp-ggml): set 0 as default for `max_tokens`

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-21 16:34:00 +02:00
Ettore Di Giacinto
284ad026b1 refactor(routes): split routes registration (#2077) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-21 01:19:57 +02:00
Ettore Di Giacinto
afa1bca1e3 fix(llama.cpp): set -1 as default for max tokens (#2087) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-20 20:20:10 +02:00
Taikono-Himazin
03adc1f60d Add tensor_parallel_size setting to vllm setting items (#2085) 
Signed-off-by: Taikono-Himazin <kazu@po.harenet.ne.jp>
 2024-04-20 14:37:02 +00:00
Ettore Di Giacinto
b319ed58b0 models(gallery): add gallery (#2078) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-20 15:22:54 +02:00
cryptk
8d30b39811 feat: fiber logs with zerlog and add trace level (#2082) 
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
 2024-04-20 10:43:37 +02:00
cryptk
b9e7708643 feat: enable polling configs for systems with broken fsnotify (docker volumes on windows) (#2081) 
* feat: enable polling configs for systems with broken fsnotify (docker volumes on windows)

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: update logging to make it clear that the config file is being polled

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

---------

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
 2024-04-19 19:31:15 -05:00
Ettore Di Giacinto
bbea62b907 feat(functions): support models with no grammar, add tests (#2068) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-18 22:43:12 +02:00
cryptk
502c1eedaa feat: refactor the dynamic json configs for api_keys and external_backends (#2055) 
* feat: refactor the dynamic json configs for api_keys and external_backends

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: remove commented code

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

---------

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2024-04-18 03:21:55 +00:00
Ettore Di Giacinto
af9e5a2d05 Revert #1963 (#2056) 
* Revert "fix(fncall): fix regression introduced in #1963 (#2048)"

This reverts commit 6b06d4e0af.

* Revert "fix: action-tmate back to upstream, dead code removal (#2038)"

This reverts commit fdec8a9d00.

* Revert "feat(grpc): return consumed token count and update response accordingly (#2035)"

This reverts commit e843d7df0e.

* Revert "refactor: backend/service split, channel-based llm flow (#1963)"

This reverts commit eed5706994.

* feat(grpc): return consumed token count and update response accordingly

Fixes: #1920

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-17 23:33:49 +02:00
Ettore Di Giacinto
6b06d4e0af fix(fncall): fix regression introduced in #1963 (#2048) 
Signed-off-by: Dave <dave@gray101.com>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Dave <dave@gray101.com>
Co-authored-by: Dave <dave@gray101.com>
 2024-04-16 21:20:11 +00:00
Ettore Di Giacinto
33c78d2228 feat(store): add Golang client (#1977) 
This adds a basic store client for Go

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-16 15:54:14 +02:00
Dave
fdec8a9d00 fix: action-tmate back to upstream, dead code removal (#2038) 
cleanup: upstream action-tmate has taken my PR, drop master reference. Also remove dead code from api.go

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-16 01:46:36 +00:00
cryptk
538a086309 fix: previous CLI rework broke debug logging (#2036) 
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
Co-authored-by: Dave <dave@gray101.com>
 2024-04-15 22:13:59 +02:00
Ettore Di Giacinto
e843d7df0e feat(grpc): return consumed token count and update response accordingly (#2035) 
Fixes: #1920
 2024-04-15 19:47:11 +02:00
Dave
eed5706994 refactor: backend/service split, channel-based llm flow (#1963) 
Refactor: channel based llm flow and services split

---------

Signed-off-by: Dave Lee <dave@gray101.com>
 2024-04-13 09:45:34 +02:00
Ettore Di Giacinto
b2785ff06e feat(gallery): support ConfigURLs (#2012) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-12 00:49:23 +02:00
Ludovic Leroux
12c0d9443e feat: use tokenizer.apply_chat_template() in vLLM (#1990) 
Use tokenizer.apply_chat_template() in vLLM

Signed-off-by: Ludovic LEROUX <ludovic@inpher.io>
 2024-04-11 19:20:22 +02:00
cryptk
24d7dadfed feat: kong cli refactor fixes #1955 (#1974) 
* feat: migrate to alecthomas/kong for CLI

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: bring in new flag for granular log levels

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* chore: go mod tidy

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: allow loading cli flag values from ["./localai.yaml", "~/.config/localai.yaml", "/etc/localai.yaml"] in that order

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: load from .env file instead of a yaml file

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: better loading for environment files

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat(doc): add initial documentation about configuration

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: remove test log lines

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: integrate new documentation into existing pages

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: add documentation on .env files

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* fix: cleanup some documentation table errors

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

* feat: refactor CLI logic out to it's own package under core/cli

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>

---------

Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
 2024-04-11 09:19:24 +02:00
Ettore Di Giacinto
8342553214 fix(llama.cpp): set better defaults for llama.cpp (#1961) 
fix(defaults): set better defaults for llama.cpp

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-06 22:56:45 +02:00
cryptk
b85dad0286 feat: first pass at improving logging (#1956) 
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>
 2024-04-04 09:24:22 +02:00
Ettore Di Giacinto
ff77d3bc22 fix(seed): generate random seed per-request if -1 is set (#1952) 
* fix(seed): generate random seed per-request if -1 is set

Also update ci with new workflows and allow the aio tests to run with an
api key

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(openvino): Add OpenVINO example

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-04-03 22:25:47 +02:00
Ettore Di Giacinto
9bc209ba73 fix(welcome): stable model list (#1949) 2024-04-02 19:25:32 +02:00
Ettore Di Giacinto
e8f02c083f fix(functions): respect when selected from string (#1940) 
* fix(functions): respect when selected from string

* fix(toolschoice): decode both string and objects
 2024-04-01 19:39:54 +02:00
Ettore Di Giacinto
35290e146b fix(grammar): respect JSONmode and grammar from user input (#1935) 
* fix(grammar): Fix JSON mode and custom grammar

* tests(aio): add jsonmode test

* tests(aio): add functioncall test

* fix(aio): use hermes-2-pro-mistral as llm for CPU profile

* add phi-2-orange
 2024-03-31 13:04:09 +02:00
Ettore Di Giacinto
957f428fd5 fix(tools): correctly render tools response in templates (#1932) 
* fix(tools): allow to correctly display both Functions and Tools

* models(hermes-2-pro): correctly display function results
 2024-03-30 19:02:07 +01:00