Tool Safety Policies

# pip install "upsonic[safety-engine]"
uv pip install "upsonic[safety-engine]"

​

What is Tool Safety Policy?

1. Pre-execution validation (tool_policy_pre): Detects and blocks harmful tools during registration, preventing dangerous tools from being added to the agent.
2. Post-execution validation (tool_policy_post): Validates tool calls before execution, blocking malicious arguments when the LLM attempts to invoke a tool with dangerous parameters.

​

Why its Important?

• Prevents harmful tools from being registered: Blocks tools with dangerous functionality like file deletion, system shutdown, or privilege escalation from being added to your agent
• Blocks malicious tool calls before execution: Detects and prevents suspicious tool arguments that could cause system damage or security violations
• Maintains system security and integrity: Ensures your AI agent can only use safe tools with safe parameters, protecting your infrastructure from harm

​

Usage

​

Pre-Execution Validation (tool_policy_pre)

from upsonic import Agent, Task
from upsonic.tools import tool
from upsonic.safety_engine.policies import HarmfulToolBlockPolicy

@tool
def delete_all_files(directory: str) -> str:
    """Delete all files in a directory recursively."""
    return f"Would delete all files in {directory}"

agent = Agent(
    "anthropic/claude-sonnet-4-6",
    tool_policy_pre=HarmfulToolBlockPolicy,
    debug=True
)

task = Task(
    description="What are your tools?",
    tools=[delete_all_files]
)

result = agent.print_do(task)
print(result)

​

Post-Execution Validation (tool_policy_post)

from upsonic import Agent, Task
from upsonic.tools import tool
from upsonic.safety_engine.policies import MaliciousToolCallBlockPolicy

@tool
def run_command(command: str) -> str:
    """Execute a system command."""
    return f"Executed: {command}"

agent = Agent(
    "anthropic/claude-sonnet-4-6",
    tool_policy_post=MaliciousToolCallBlockPolicy,
    debug=True
)

task = Task(
    description="Use the run_command tool to execute: rm -rf /tmp/test",
    tools=[run_command]
)

result = agent.print_do(task)
print(result)

​

Combined Pre + Post Validation

from upsonic import Agent, Task
from upsonic.safety_engine.policies import (
    HarmfulToolBlockPolicy,
    MaliciousToolCallBlockPolicy
)

def safe_calculator(a: int, b: int) -> int:
    """A safe calculator tool that adds two numbers."""
    return a + b

agent = Agent(
    "anthropic/claude-sonnet-4-6",
    tools=[safe_calculator],
    tool_policy_pre=HarmfulToolBlockPolicy,
    tool_policy_post=MaliciousToolCallBlockPolicy,
    debug=True,
)

task = Task("What is 2 + 3? Use the safe_calculator tool.", tools=[safe_calculator])
result = agent.print_do(task)

​

Available Variants

​

Harmful Tool Detection Policies

• HarmfulToolBlockPolicy: LLM-powered detection with blocking during tool registration
• HarmfulToolBlockPolicy_LLM: LLM-powered block messages for harmful tools
• HarmfulToolRaiseExceptionPolicy: Raises DisallowedOperation exception for harmful tools
• HarmfulToolRaiseExceptionPolicy_LLM: LLM-generated exception messages for harmful tools

​

Malicious Tool Call Detection Policies

• MaliciousToolCallBlockPolicy: LLM-powered detection with blocking before tool execution
• MaliciousToolCallBlockPolicy_LLM: LLM-powered block messages for malicious tool calls
• MaliciousToolCallRaiseExceptionPolicy: Raises DisallowedOperation exception for malicious tool calls
• MaliciousToolCallRaiseExceptionPolicy_LLM: LLM-generated exception messages for malicious tool calls