How to Install and Run Ollama on Raspberry Pi

Pandey's Blog May 30, 2024

How to Install and Run Ollama on Raspberry Pi

How to Install and Run Ollama on Raspberry Pi

The Raspberry Pi, renowned for its versatility and affordability, continues to be a beloved tool for tech enthusiasts and developers. One exciting way to maximize its potential is by running large language models (LLMs) using Ollama. In this guide, we'll explore how to install Ollama on your Raspberry Pi and run various LLMs, opening up a world of possibilities for AI-driven projects.

Why Use Ollama on Raspberry Pi?

Ollama is a lightweight, extensible framework designed to run LLMs locally on your machine. By using Ollama on a Raspberry Pi, you can experiment with powerful language models without relying on cloud services. This setup is ideal for developers, researchers, and hobbyists eager to explore AI on a budget.

Preparing Your Raspberry Pi

Before diving into the installation process, ensure that your Raspberry Pi is up to date and equipped with the necessary software.

  1. Update your system packages: 
    sudo apt update
sudo apt upgrade
  2. Install the curl package (if not already installed): 
    sudo apt install curl

Installing Ollama

With your Raspberry Pi prepared, you can now install Ollama. This process is straightforward thanks to the provided installation script.

curl -fsSL https://ollama.ai/install.sh | sh

It's always a good practice to review scripts before running them. You can view the script content by visiting the provided URL in your browser.

Verifying the Installation

After installing Ollama, verify the installation by checking its version:

ollama --version

If the installation was successful, you should see the version number of Ollama displayed in your terminal.

Running Different LLMs on Raspberry Pi

Ollama supports various LLMs, each suited for different applications. Here are some popular models you can run on your Raspberry Pi:

1. TinyLlama

TinyLlama is a lightweight model based on 1.1 billion parameters. It's ideal for initial experiments due to its small size and relatively fast performance on a Raspberry Pi.

ollama run tinyllama

Once the model is up and running, you can start interacting with it by posing questions and receiving responses.

2. Phi3

Phi3, developed by Microsoft, is a 2.7 billion parameter model that offers a balance between performance and quality. It's slightly more demanding than TinyLlama but still manageable on a Raspberry Pi.

ollama run phi3

Phi3 is capable of handling more complex queries, albeit with longer response times compared to TinyLlama.

3. Llama3

Llama3 is a heavyweight model with exceptional capabilities. Running this model on a Raspberry Pi requires patience and ample free space (at least 4.7GB).

ollama run llama3

Despite its size, Llama3 produces high-quality results, making it suitable for more demanding tasks.

Using Ollama with Docker

For those who prefer containerized environments, Ollama can also be run using Docker. This method is particularly useful for managing dependencies and maintaining a clean setup.

  1. Install Docker on your Raspberry Pi: 
    curl -fsSL https://get.docker.com | sh
  2. Run the Ollama container: 
    docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
  3. Execute a model within the container: 
    docker exec -it ollama ollama run llama2

Customizing Models with Modelfiles

Ollama allows for the customization of models through Modelfiles. This feature enables you to tailor models for specific tasks by adjusting parameters or modifying system messages.

  1. Pull the model: 
    ollama pull llama2
  2. Create a Modelfile to specify custom settings: 
    {
    "temperature": 0.7,
    "max_tokens": 150,
    "system_message": "You are an assistant who provides concise and accurate answers."
}
  3. Create and run the customized model: 
    ollama create mycustommodel -f ./Modelfile
ollama run mycustommodel

Exploring Further

Ollama offers advanced features like importing models from other formats, customizing prompts, and integrating with a REST API for web applications. These capabilities make it a versatile tool for Raspberry Pi enthusiasts to explore AI without significant overheads.

Conclusion

Running open LLM models on Raspberry Pi with Ollama provides a unique opportunity to delve into AI from the comfort of your home or workspace. Whether you're a developer, a student, or an AI enthusiast, Ollama equips you with the tools needed to explore and innovate with large language models. Start your journey today and unlock the potential of AI on your Raspberry Pi.

References

0 Comments