Downloading and Running Phi-3 Locally with Ollama

 In the previous article, we installed Ollama and prepared our system for local AI development. At this point, we have the runtime environment ready, but we still need an actual language model.

This is where Phi-3 enters the picture.

Phi-3 is a compact yet capable language model that can run entirely on your local machine. It offers strong reasoning, coding assistance, summarization, and automation capabilities while requiring significantly fewer resources than many larger models.

In this guide, we'll download Phi-3, run it locally, and explore how it can serve as the intelligence layer for our future AI agents.

---

Why Phi-3?

Before installing the model, it's worth understanding why we're choosing Phi-3 for this series.

Phi-3 offers several advantages:

• Fast response times
• Modest hardware requirements
• Strong performance for automation tasks
• Easy deployment through Ollama
• Suitable for local AI agents

For many workflow automation scenarios, Phi-3 provides an excellent balance between performance and efficiency.

Rather than chasing the largest possible model, our goal is to build practical AI systems that can operate reliably on everyday hardware.

---

Step 1: Open Command Prompt

Open a new Command Prompt window.

You can do this by:

1. Pressing:

Windows + R

2. Typing:

cmd

3. Pressing Enter.

Verify that Ollama is available:

ollama --version

If you see version information, you're ready to proceed.

---

Step 2: Download Phi-3

To download the model, run:

ollama pull phi3

Ollama will begin downloading the model files automatically.

Depending on your internet connection, this may take several minutes.

During the download you'll see progress information similar to:

pulling manifest
pulling layers
verifying download
success

Once completed, the model is stored locally and can be used without an internet connection.

---

Step 3: Verify the Installation

To see all installed models, run:

ollama list

You should see output similar to:

NAME      SIZE
phi3      ...

This confirms that the model is installed and ready for use.

---

Step 4: Launch Phi-3

Now we can start the model.

Execute:

ollama run phi3

After a few moments, you'll be presented with a prompt.

For example:

>>> 

You can now begin chatting directly with Phi-3.

Try asking:

Explain what workflow automation is.

Or:

Write a Python function that calculates compound interest.

The responses are generated entirely on your machine.

No cloud API calls are required.

---

Understanding What Happens Behind the Scenes

When you run:

ollama run phi3

Ollama:

1. Loads the model into memory
2. Starts an inference session
3. Processes your prompts
4. Generates responses locally

Everything remains on your computer.

This is one of the major advantages of local AI systems:

• Privacy
• Low latency
• Offline operation
• No API costs

---

Step 5: Exiting the Chat Session

When you're finished chatting, type:

/bye

or simply press:

Ctrl + C

The model session will stop, but the model remains installed and available for future use.

---

Using Phi-3 Through the Local API

One of Ollama's most important features is its built-in API server.

Earlier we confirmed that Ollama exposes:

http://localhost:11434

This means other applications can communicate with Phi-3 programmatically.

For example:

• Python scripts
• Desktop applications
• Web applications
• Automation workflows
• AI agents

Instead of manually chatting with the model, software can send prompts and receive responses automatically.

This is the foundation of agent-based systems.

---

Simple API Example

Suppose an application sends:

{
  "model": "phi3",
  "prompt": "Summarize this document."
}

Phi-3 processes the request and returns a response.

The calling application can then decide what action to take next.

This capability transforms Phi-3 from a chatbot into a component of a larger automation system.

---

Practical Use Cases

Once installed, Phi-3 can be used for:

Content Summarization

Summarize reports, emails, and notes.

Classification

Categorize files or incoming data.

Code Assistance

Generate snippets and explain programming concepts.

Data Extraction

Extract structured information from text.

Workflow Decision-Making

Determine what action an automation should take next.

These are exactly the kinds of tasks that make AI agents useful.

---

What We've Accomplished

At this point, we now have:

✓ Ollama installed

✓ Phi-3 downloaded

✓ Local AI inference working

✓ API access available

✓ A foundation for building AI-powered workflows

This marks the first major milestone in our local AI journey.