Hour 2 - Exploring File Structure of Ollama

Lecture Notes: 

---

1. Concepts

Understanding the file structure of Ollama is critical for efficient usage and customization of models.

• Why File Structure Matters:
  • Helps manage models, configurations, logs, and cache efficiently.
  • Simplifies debugging and model fine-tuning.

Key Components of Ollama's File System:

1. Model Files: Contain model weights, templates, and prompts.
2. Configuration Files: Store settings and environment variables.
3. Logs: Track operations and errors for debugging.
4. Cache: Stores downloaded models and processed data for quick access.

---

2. Key Aspects

Key Directories and Their Purpose:

• ~/ollama/models/:
  Stores all downloaded and custom-created models.
  Example: Models like llama3.1, swede, etc.

• ~/ollama/config/:
  Contains configuration files for environment variables and settings.
  Example: config.yaml for default settings.

• ~/ollama/logs/:
  Tracks logs for operations performed using Ollama commands.

• ~/ollama/cache/:
  Temporarily stores data to enhance model loading and generation speed.

---

3. Implementation

Step-by-Step: Navigating Ollama's File Structure

1. Locate the File Structure:
   By default, Ollama's files are stored in the user's home directory.

   cd ~/ollama
   

2. Explore Models Directory:
   List all models stored on your system.

   ls ~/ollama/models
   

3. Inspect a Model File:
   Open a specific model's configuration to see the prompt and settings.

   cat ~/ollama/models/llama3.1/modelfile
   

4. Access Logs:
   Review the logs to debug any issues.

   tail -n 20 ~/ollama/logs/ollama.log
   

---

4. CLI Commands for File Structure Exploration

Command	Description	Example
ollama list	Lists all downloaded models.	ollama list
ollama show	Displays details about a specific model.	ollama show llama3.1
ollama cp	Copies a model to create a new reference.	ollama cp llama3.1 custom_model
ollama rm	Removes a model from the system.	ollama rm custom_model
ollama pull	Downloads a model to the models directory.	ollama pull llama3.1
ollama serve	Runs Ollama manually (affects logging and cache).	ollama serve

---

   

C:\Users\AURMC>ollama list 

NAME              ID           SIZE      MODIFIED

llama3.1:latest    46e0c10c039e    4.9 GB    10 minutes ago

mistral:latest       f974a74358d6    4.1 GB    40 hours ago

C:\Users\AURMC>ollama show llama3.1

  Model

    architecture        llama

    parameters          8.0B

    context length      131072

    embedding length    4096

    quantization        Q4_K_M

  Parameters

    stop    "<|start_header_id|>"

    stop    "<|end_header_id|>"

    stop    "<|eot_id|>"

  License

    LLAMA 3.1 COMMUNITY LICENSE AGREEMENT

    Llama 3.1 Version Release Date: July 23, 2024 

5. Real-Life Example

Scenario: Checking Model Details and Debugging Generation Issues

Suppose a student is using the llama3.1 model and encounters an issue with generation output. They need to inspect the model's settings and logs to debug.

1. Navigate to the Models Directory:

   cd ~/ollama/models
   ls
   

2. Inspect the Model's Configurations:

   cat llama3.1/modelfile
   

3. Review Logs for Errors:

   tail -n 20 ~/ollama/logs/ollama.log
   

4. Test the Model Again:

   ollama run llama3.1 --prompt "Explain the lifecycle of a star."
   

---

6. Code Example

Accessing File Structure and Inspecting Models

# List all models installed on your system
ollama list

# Show details about a specific model
ollama show llama3.1

# Navigate to the directory containing models
cd ~/ollama/models

# List files in the llama3.1 model directory
ls llama3.1

# View the modelfile for configuration details
cat llama3.1/modelfile

# Debug issues by reviewing logs
tail -n 20 ~/ollama/logs/ollama.log

Creating a Custom Model

# Create a new model file with custom configurations
echo -e "FROM llama3.1\nSYSTEM \"\"\"\nYou are a helpful tutor for undergraduate students.\n\"\"\"" > modelfile

# Create the model using the modelfile
ollama create tutor_model -f modelfile

# Verify the new model is added
ollama list

---

7. Summary

• Concepts Covered: Importance of file structure and its components.
• Key Aspects: Explored directories like models/, config/, logs/, and cache/.
• CLI Commands: Commands to interact with models, logs, and settings.
• Real-Life Example: Debugging issues by accessing configuration files and logs.
• Code Examples: Commands to explore file structure and create custom models.

---

8. Homework/Practice

1. Use the ollama list and ollama show commands to explore models on your system.
2. Inspect the configuration of a model using cat or a text editor.
3. Create and run a simple custom model based on an existing one.
4. Check the logs for any errors or important messages after using ollama run.

---

These lecture notes provide a structured, practical approach to understanding and using the file structure of Ollama.