NVIDIA Agent - Upsonic AI

This example demonstrates how to create and use an Upsonic Agent with NVIDIA NIM models using the NvidiaModel class. The example shows how to leverage NVIDIA’s powerful AI models through their NIM (NVIDIA Inference Microservice) API, including models like Llama 3.1 Nemotron 70B, GPT-OSS, Mistral, and many others.

Overview

Upsonic framework provides seamless integration with NVIDIA’s AI models through the NIM API. This example showcases:

NvidiaModel Integration — Using NVIDIA NIM API to access various AI models
Agent Configuration — Creating an Upsonic Agent with NVIDIA models
Task Execution — Running tasks with the configured agent
FastAPI Server — Running the agent as a production-ready API server

The NvidiaModel class provides access to NVIDIA’s curated collection of AI models, including:

Llama 3.1 Nemotron 70B — High-performance instruction-tuned model
GPT-OSS models — OpenAI’s open-source models
Mistral models — Mistral AI’s powerful language models
And many more — Access to NVIDIA’s full model catalog

Project Structure

examples/nvidia-agent/
├── main.py                    # Agent with NVIDIA model
├── upsonic_configs.json       # Upsonic CLI configuration
├── .env.example               # Example environment variables
└── README.md                  # Quick start guide

Environment Variables

You can configure the model using environment variables:

# Set API key
export NVIDIA_API_KEY="your-api-key"

# Or use NGC_API_KEY (alternative)
export NGC_API_KEY="your-api-key"

# Optional: Set custom base URL
export NVIDIA_BASE_URL="https://your-custom-endpoint.com/v1"

Getting your NVIDIA API key:

Visit https://build.nvidia.com/
Sign up or log in to your NVIDIA account
Navigate to API Keys section
Create a new API key
Copy the key to your environment variables

Installation

# Install dependencies
upsonic install

Managing Dependencies

# Add a package
upsonic add <package> <section>
upsonic add requests api

# Remove a package
upsonic remove <package> <section>
upsonic remove requests api

Sections: api, streamlit, development

Usage

Option 1: Run Directly

uv run main.py

Runs the agent with a default test query.

Option 2: Run as API Server

upsonic run

Server starts at http://localhost:8000. API documentation at /docs. Example API call:

curl -X POST http://localhost:8000/call \
  -H "Content-Type: application/json" \
  -d '{"user_query": "What is artificial intelligence?"}'

How It Works

Component	Description
NvidiaModel	Wraps NVIDIA NIM API for model access
Agent	Upsonic Agent configured with NvidiaModel
Task	Task object containing user query
Execution	Agent processes task and returns response

Example Output

Query:

"What is artificial intelligence?"

Response:

"Artificial intelligence (AI) is a branch of computer science that aims to create 
systems capable of performing tasks that typically require human intelligence..."

Complete Implementation

main.py

"""
NVIDIA Agent Example

This example demonstrates how to create and use an Agent with NVIDIA models.
The example shows:
1. Creating a NvidiaModel instance
2. Creating an Agent with the NvidiaModel
3. Creating a Task
4. Executing the task with the agent

This file contains:
- async main(inputs): For use with `upsonic run` CLI command (FastAPI server)
"""

from upsonic import Agent, Task
from upsonic.models.nvidia import NvidiaModel


async def main(inputs: dict) -> dict:
    """
    Async main function for FastAPI server (used by `upsonic run` command).
    
    This function is called by the Upsonic CLI when running the agent as a server.
    It receives inputs from the API request and returns a response dictionary.
    
    Args:
        inputs: Dictionary containing input parameters as defined in upsonic_configs.json
                Expected key: "user_query" (string)
    
    Returns:
        Dictionary with output schema as defined in upsonic_configs.json
        Expected key: "bot_response" (string)
    """
    user_query = inputs.get("user_query", "Hi, how are you?")

    model = NvidiaModel(
        model_name="meta/llama-3.1-nemotron-70b-instruct:1.0"
    )

    agent = Agent(
        model=model,
        name="NVIDIA Agent"
    )
    

    answering_task = Task(
        description=f"Answer the user question: {user_query}"
    )
    
    result = await agent.print_do_async(answering_task)
    
    return {
        "bot_response": result
    }


if __name__ == "__main__":
    import asyncio
    asyncio.run(main({"user_query": "Hi, how are you?"}))

upsonic_configs.json

{
    "environment_variables": {
        "UPSONIC_WORKERS_AMOUNT": {
            "type": "number",
            "description": "The number of workers for the Upsonic API",
            "default": 1
        },
        "API_WORKERS": {
            "type": "number",
            "description": "The number of workers for the Upsonic API",
            "default": 1
        },
        "RUNNER_CONCURRENCY": {
            "type": "number",
            "description": "The number of runners for the Upsonic API",
            "default": 1
        },
        "NEW_FEATURE_FLAG": {
            "type": "string",
            "description": "New feature flag added in version 2.0",
            "default": "enabled"
        }
    },
    "machine_spec": {
        "cpu": 2,
        "memory": 4096,
        "storage": 1024
    },
    "agent_name": "NVIDIA Agent",
    "description": "NVIDIA-powered AI Agent using Upsonic framework with NvidiaModel",
    "icon": "book",
    "language": "book",
    "streamlit": false,
    "proxy_agent": false,
    "dependencies": {
        "api": [
            "fastapi>=0.115.12",
            "uvicorn>=0.34.2",
            "upsonic",
            "pip"
        ],
        "streamlit": [
            "streamlit==1.32.2",
            "pandas==2.2.1",
            "numpy==1.26.4"
        ],
        "development": [
            "watchdog",
            "python-dotenv",
            "ipdb",
            "pytest",
            "streamlit-autorefresh"
        ]
    },
    "entrypoints": {
        "api_file": "main.py",
        "streamlit_file": "streamlit_app.py"
    },
    "input_schema": {
        "inputs": {
            "user_query": {
                "type": "string",
                "description": "User's question or query for the NVIDIA agent to answer",
                "required": true,
                "default": null
            }
        }
    },
    "output_schema": {
        "bot_response": {
            "type": "string",
            "description": "NVIDIA agent's generated response to the user query"
        }
    }
}

For more information on NVIDIA NIM:

NVIDIA NIM Platform - Get started with NVIDIA NIM
Supported Models Documentation - Complete list of available models, hardware requirements, and specifications

Repository

View the complete example: NVIDIA Agent Example

​Overview

​Project Structure

​Environment Variables

​Installation

​Managing Dependencies

​Usage

​Option 1: Run Directly

​Option 2: Run as API Server

​How It Works

​Example Output

​Complete Implementation

​main.py

​upsonic_configs.json

​Repository