Skip to main content
This example demonstrates how to create and use an Upsonic Agent with Ollama models to run LLMs locally. The example shows how to configure the agent to use local models for privacy and cost-efficiency.

Overview

Upsonic framework provides seamless integration for local models via Ollama. This example showcases:
  1. Local Model Integration β€” Using OllamaModel to connect to local LLMs
  2. Privacy β€” Running inferences entirely on your local machine
  3. Task Execution β€” Running simple QA tasks
  4. FastAPI Server β€” Running the agent as a production-ready API server

Project Structure

Environment Variables

Installation

Managing Dependencies

Sections: api, streamlit, development

Usage

Option 1: Run Directly

Runs the agent with a default test query (β€œHello, how are you?”).

Option 2: Run as API Server

Server starts at http://localhost:8000. API documentation at /docs. Example API call:

How It Works

ComponentDescription
OllamaModelConnects to the local Ollama instance (default port 11434)
AgentUses the local model for inference
TaskEncapsulates the user query
ExecutionRuns the task synchronously or asynchronously

Example Output

Query:
Response:

Complete Implementation

main.py

upsonic_configs.json

Repository

View the complete example: Ollama Agent