feat: ⚠️ reduce images size and stop bundling sources (#5721)

feat: reduce images size and stop bundling sources

Do not copy sources anymore, and reduce packages of the base images by
not using builder images.

If needed to rebuild, just build the container image from scratch by
following the docs. We will slowly try to migrate all backends to the
gallery to keep the core small.

This PR is a breaking change, it also sets the base folders to /models
and /backends instead of /build/models and /build/backends.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
This commit is contained in:
Ettore Di Giacinto
2025-06-26 18:41:38 +02:00
committed by GitHub
parent 7c4a2e9b85
commit 6644af10c6
7 changed files with 123 additions and 157 deletions

View File

@@ -268,7 +268,7 @@ The image list is on [quay](https://quay.io/repository/go-skynet/local-ai?tab=ta
To run LocalAI with Docker and sycl starting `phi-2`, you can use the following command as an example:
```bash
docker run -e DEBUG=true --privileged -ti -v $PWD/models:/build/models -p 8080:8080 -v /dev/dri:/dev/dri --rm quay.io/go-skynet/local-ai:master-gpu-intel-f32-ffmpeg-core phi-2
docker run -e DEBUG=true --privileged -ti -v $PWD/models:/models -p 8080:8080 -v /dev/dri:/dev/dri --rm quay.io/go-skynet/local-ai:master-gpu-intel-f32-ffmpeg-core phi-2
```
### Notes
@@ -296,7 +296,7 @@ To use Vulkan, use the images with the `vulkan` tag, for example `{{< version >}
To run LocalAI with Docker and Vulkan, you can use the following command as an example:
```bash
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/build/models localai/localai:latest-vulkan-ffmpeg-core
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/models localai/localai:latest-vulkan-ffmpeg-core
```
### Notes
@@ -308,7 +308,7 @@ These flags are the same as the sections above, depending on the hardware, for [
If you have mixed hardware, you can pass flags for multiple GPUs, for example:
```bash
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/build/models \
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/models \
--gpus=all \ # nvidia passthrough
--device /dev/dri --device /dev/kfd \ # AMD/Intel passthrough
localai/localai:latest-vulkan-ffmpeg-core

View File

@@ -92,7 +92,7 @@ services:
- DEBUG=true
# ...
volumes:
- ./models:/build/models:cached
- ./models:/models:cached
# decomment the following piece if running with Nvidia GPUs
# deploy:
# resources:
@@ -105,21 +105,21 @@ services:
{{% alert icon="💡" %}}
**Models caching**: The **AIO** image will download the needed models on the first run if not already present and store those in `/build/models` inside the container. The AIO models will be automatically updated with new versions of AIO images.
**Models caching**: The **AIO** image will download the needed models on the first run if not already present and store those in `/models` inside the container. The AIO models will be automatically updated with new versions of AIO images.
You can change the directory inside the container by specifying a `MODELS_PATH` environment variable (or `--models-path`).
If you want to use a named model or a local directory, you can mount it as a volume to `/build/models`:
If you want to use a named model or a local directory, you can mount it as a volume to `/models`:
```bash
docker run -p 8080:8080 --name local-ai -ti -v $PWD/models:/build/models localai/localai:latest-aio-cpu
docker run -p 8080:8080 --name local-ai -ti -v $PWD/models:/models localai/localai:latest-aio-cpu
```
or associate a volume:
```bash
docker volume create localai-models
docker run -p 8080:8080 --name local-ai -ti -v localai-models:/build/models localai/localai:latest-aio-cpu
docker run -p 8080:8080 --name local-ai -ti -v localai-models:/models localai/localai:latest-aio-cpu
```
{{% /alert %}}

View File

@@ -35,7 +35,7 @@ docker pull quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
Run the LocalAI container on Nvidia ARM64 devices using the following command, where `/data/models` is the directory containing the models:
```bash
docker run -e DEBUG=true -p 8080:8080 -v /data/models:/build/models -ti --restart=always --name local-ai --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
docker run -e DEBUG=true -p 8080:8080 -v /data/models:/models -ti --restart=always --name local-ai --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
```
Note: `/data/models` is the directory containing the models. You can replace it with the directory containing your models.

View File

@@ -663,7 +663,7 @@ install_docker() {
IMAGE_TAG=${LOCALAI_VERSION}-vulkan
info "Starting LocalAI Docker container..."
$SUDO docker run -v local-ai-data:/build/models \
$SUDO docker run -v local-ai-data:/models \
--device /dev/dri \
--restart=always \
-e API_KEY=$API_KEY \
@@ -690,7 +690,7 @@ install_docker() {
fi
info "Starting LocalAI Docker container..."
$SUDO docker run -v local-ai-data:/build/models \
$SUDO docker run -v local-ai-data:/models \
--gpus all \
--restart=always \
-e API_KEY=$API_KEY \
@@ -705,7 +705,7 @@ install_docker() {
fi
info "Starting LocalAI Docker container..."
$SUDO docker run -v local-ai-data:/build/models \
$SUDO docker run -v local-ai-data:/models \
--device /dev/dri \
--device /dev/kfd \
--group-add=video \
@@ -723,7 +723,7 @@ install_docker() {
fi
info "Starting LocalAI Docker container..."
$SUDO docker run -v local-ai-data:/build/models \
$SUDO docker run -v local-ai-data:/models \
--device /dev/dri \
--restart=always \
-e API_KEY=$API_KEY \