mirror of
https://github.com/mudler/LocalAI.git
synced 2026-01-06 02:29:54 -06:00
feat: ⚠️ reduce images size and stop bundling sources (#5721)
feat: reduce images size and stop bundling sources Do not copy sources anymore, and reduce packages of the base images by not using builder images. If needed to rebuild, just build the container image from scratch by following the docs. We will slowly try to migrate all backends to the gallery to keep the core small. This PR is a breaking change, it also sets the base folders to /models and /backends instead of /build/models and /build/backends. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
This commit is contained in:
committed by
GitHub
parent
7c4a2e9b85
commit
6644af10c6
@@ -268,7 +268,7 @@ The image list is on [quay](https://quay.io/repository/go-skynet/local-ai?tab=ta
|
||||
To run LocalAI with Docker and sycl starting `phi-2`, you can use the following command as an example:
|
||||
|
||||
```bash
|
||||
docker run -e DEBUG=true --privileged -ti -v $PWD/models:/build/models -p 8080:8080 -v /dev/dri:/dev/dri --rm quay.io/go-skynet/local-ai:master-gpu-intel-f32-ffmpeg-core phi-2
|
||||
docker run -e DEBUG=true --privileged -ti -v $PWD/models:/models -p 8080:8080 -v /dev/dri:/dev/dri --rm quay.io/go-skynet/local-ai:master-gpu-intel-f32-ffmpeg-core phi-2
|
||||
```
|
||||
|
||||
### Notes
|
||||
@@ -296,7 +296,7 @@ To use Vulkan, use the images with the `vulkan` tag, for example `{{< version >}
|
||||
To run LocalAI with Docker and Vulkan, you can use the following command as an example:
|
||||
|
||||
```bash
|
||||
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/build/models localai/localai:latest-vulkan-ffmpeg-core
|
||||
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/models localai/localai:latest-vulkan-ffmpeg-core
|
||||
```
|
||||
|
||||
### Notes
|
||||
@@ -308,7 +308,7 @@ These flags are the same as the sections above, depending on the hardware, for [
|
||||
If you have mixed hardware, you can pass flags for multiple GPUs, for example:
|
||||
|
||||
```bash
|
||||
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/build/models \
|
||||
docker run -p 8080:8080 -e DEBUG=true -v $PWD/models:/models \
|
||||
--gpus=all \ # nvidia passthrough
|
||||
--device /dev/dri --device /dev/kfd \ # AMD/Intel passthrough
|
||||
localai/localai:latest-vulkan-ffmpeg-core
|
||||
|
||||
@@ -92,7 +92,7 @@ services:
|
||||
- DEBUG=true
|
||||
# ...
|
||||
volumes:
|
||||
- ./models:/build/models:cached
|
||||
- ./models:/models:cached
|
||||
# decomment the following piece if running with Nvidia GPUs
|
||||
# deploy:
|
||||
# resources:
|
||||
@@ -105,21 +105,21 @@ services:
|
||||
|
||||
{{% alert icon="💡" %}}
|
||||
|
||||
**Models caching**: The **AIO** image will download the needed models on the first run if not already present and store those in `/build/models` inside the container. The AIO models will be automatically updated with new versions of AIO images.
|
||||
**Models caching**: The **AIO** image will download the needed models on the first run if not already present and store those in `/models` inside the container. The AIO models will be automatically updated with new versions of AIO images.
|
||||
|
||||
You can change the directory inside the container by specifying a `MODELS_PATH` environment variable (or `--models-path`).
|
||||
|
||||
If you want to use a named model or a local directory, you can mount it as a volume to `/build/models`:
|
||||
If you want to use a named model or a local directory, you can mount it as a volume to `/models`:
|
||||
|
||||
```bash
|
||||
docker run -p 8080:8080 --name local-ai -ti -v $PWD/models:/build/models localai/localai:latest-aio-cpu
|
||||
docker run -p 8080:8080 --name local-ai -ti -v $PWD/models:/models localai/localai:latest-aio-cpu
|
||||
```
|
||||
|
||||
or associate a volume:
|
||||
|
||||
```bash
|
||||
docker volume create localai-models
|
||||
docker run -p 8080:8080 --name local-ai -ti -v localai-models:/build/models localai/localai:latest-aio-cpu
|
||||
docker run -p 8080:8080 --name local-ai -ti -v localai-models:/models localai/localai:latest-aio-cpu
|
||||
```
|
||||
|
||||
{{% /alert %}}
|
||||
|
||||
@@ -35,7 +35,7 @@ docker pull quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
|
||||
Run the LocalAI container on Nvidia ARM64 devices using the following command, where `/data/models` is the directory containing the models:
|
||||
|
||||
```bash
|
||||
docker run -e DEBUG=true -p 8080:8080 -v /data/models:/build/models -ti --restart=always --name local-ai --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
|
||||
docker run -e DEBUG=true -p 8080:8080 -v /data/models:/models -ti --restart=always --name local-ai --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
|
||||
```
|
||||
|
||||
Note: `/data/models` is the directory containing the models. You can replace it with the directory containing your models.
|
||||
|
||||
8
docs/static/install.sh
vendored
8
docs/static/install.sh
vendored
@@ -663,7 +663,7 @@ install_docker() {
|
||||
IMAGE_TAG=${LOCALAI_VERSION}-vulkan
|
||||
|
||||
info "Starting LocalAI Docker container..."
|
||||
$SUDO docker run -v local-ai-data:/build/models \
|
||||
$SUDO docker run -v local-ai-data:/models \
|
||||
--device /dev/dri \
|
||||
--restart=always \
|
||||
-e API_KEY=$API_KEY \
|
||||
@@ -690,7 +690,7 @@ install_docker() {
|
||||
fi
|
||||
|
||||
info "Starting LocalAI Docker container..."
|
||||
$SUDO docker run -v local-ai-data:/build/models \
|
||||
$SUDO docker run -v local-ai-data:/models \
|
||||
--gpus all \
|
||||
--restart=always \
|
||||
-e API_KEY=$API_KEY \
|
||||
@@ -705,7 +705,7 @@ install_docker() {
|
||||
fi
|
||||
|
||||
info "Starting LocalAI Docker container..."
|
||||
$SUDO docker run -v local-ai-data:/build/models \
|
||||
$SUDO docker run -v local-ai-data:/models \
|
||||
--device /dev/dri \
|
||||
--device /dev/kfd \
|
||||
--group-add=video \
|
||||
@@ -723,7 +723,7 @@ install_docker() {
|
||||
fi
|
||||
|
||||
info "Starting LocalAI Docker container..."
|
||||
$SUDO docker run -v local-ai-data:/build/models \
|
||||
$SUDO docker run -v local-ai-data:/models \
|
||||
--device /dev/dri \
|
||||
--restart=always \
|
||||
-e API_KEY=$API_KEY \
|
||||
|
||||
Reference in New Issue
Block a user