Quickstart

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families and architectures.

Installation Methods link

LocalAI is available as a container image and binary, compatible with various container engines like Docker, Podman, and Kubernetes. Container images are published on quay.io and Docker Hub. Binaries can be downloaded from GitHub.

Prerequisites link

Before you begin, ensure you have a container engine installed if you are not using the binaries. Suitable options include Docker or Podman. For installation instructions, refer to the following guides:

• Install Docker Desktop (Mac, Windows, Linux)
• Install Podman (Linux)
• Install Docker engine (Servers)

Running Models link

Do you have already a model file? Skip to Run models manually.

LocalAI allows one-click runs with popular models. It downloads the model and starts the API with the model loaded.

There are different categories of models: LLMs, Multimodal LLM , Embeddings, Audio to Text, and Text to Audio depending on the backend being used and the model architecture.

  docker run -ti -p 8080:8080 localai/localai:v2.7.0-ffmpeg-core llava phi-2
  

Container images link

LocalAI provides a variety of images to support different environments. These images are available on quay.io and Docker Hub.

For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA images, if you don’t have a GPU, use the CPU images. If you have AMD or Mac Silicon, see the build section.

What’s next? link

Explore further resources and community contributions:

• Community How to’s
• Examples