Generate Inference Result

Overview

The generate() method is the recommended way to execute your agent when you want automatic trace collection, timing, and inference result creation. It handles the complete lifecycle:

Initializes trace collection
Executes your agent with timing measurement
Creates/updates the inference result with all metadata
Saves collected traces to the platform
Cleans up trace context (prevents memory leaks)

This is the recommended method for production use as it handles all trace lifecycle management automatically.

Agent Options

Simple
Chat History
Structured

The quickest way to get started. Your function receives just the latest user message as a string.

def my_agent(user_message: str) -> str:
    # In a real scenario, call your model here
    return f"Your model output to: {user_message}"

Use this when your agent needs the full conversation context. Your function receives the message list in the OpenAI format ({"role": "...", "content": "..."}).

def my_agent(messages: list[dict]) -> str:
    # messages follows the standard chat format:
    # [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}, ...]
    user_message = messages[-1]["content"]
    return f"Your model output to: {user_message}"

For full control over input and output — including optional usage tracking, cost tracking, and retrieval context for RAG evaluations.

def my_agent(input_data: AgentInput) -> AgentResponse:
    user_message = input_data.last_user_message_str()
    # In a real scenario, call your model here
    model_output = f"Your model output to: {user_message}"
    # Return AgentResponse with optional usage/cost tracking
    return AgentResponse(
        content=model_output,
        usage_info={"input_tokens": 100, "output_tokens": 50},
    )

All three signatures work with generate() and simulate(). Both sync and async functions are supported. The SDK auto-detects which signature you’re using from the type hint on the first parameter.

Parameters

agent

AgentType

required

The agent function to execute. Supported signatures:

Simple: (user_message: str) -> str — receives last user message
Chat History: (messages: list[dict]) -> str — receives full chat history (OpenAI format)
Structured: (input_data: AgentInput) -> AgentResponse — structured input/output

Both sync and async functions are supported. Use the structured signature when you need usage_info, cost_info, or retrieval_context.

session

Session

required

The session object to associate the inference result with. Obtain via galtea.sessions.create() or galtea.sessions.get().

user_input

string

The user’s input message to send to the agent. If not provided and the session is test-based, the test case input will be used automatically. Required for production/monitoring sessions.

throw_if_empty_agent_response

bool

If True, raises a ValueError when the agent returns an empty response. Defaults to False.

Returns

Returns an InferenceResult object with all captured data.

Example

# Define your agent function
def my_agent(user_message: str) -> str:
    return f"Response to: {user_message}"


inference_result = galtea.inference_results.generate(
    agent=my_agent, session=session, user_input="Generate something"
)

What Gets Captured

The generate() method automatically captures and saves:

Data	Source	Description
Input	`user_input` parameter	The user’s message
Output	`AgentResponse.content`	The agent’s response
Retrieval Context	`AgentResponse.retrieval_context`	Context used (for RAG)
Latency	Measured	End-to-end execution time in ms
Usage Info	`AgentResponse.usage_info`	Token counts (if provided)
Cost Info	`AgentResponse.cost_info`	Cost data (if provided)
Traces	`@trace` decorators	All traced operations

Providing Usage and Cost Information

Your agent can return usage and cost information in the AgentResponse:

@galtea.trace(name="main", type=TraceType.AGENT)
def my_agent_with_usage(input_data: AgentInput) -> AgentResponse:
    # Your agent logic...

    return AgentResponse(
        content="Response content",
        usage_info={
            "input_tokens": 150,
            "output_tokens": 75,
            "cache_read_input_tokens": 50,
        },
        cost_info={
            "cost_per_input_token": 0.00001,
            "cost_per_output_token": 0.00003,
            "cost_per_cache_read_input_token": 0.000001,
        },
    )

Comparison with Manual Approach

Feature	`generate()`	Manual with `set_context()`
Context initialization	Automatic	`set_context()`
Agent execution	Automatic	Manual call
Timing measurement	Automatic	Manual
Inference result	Auto-created	`inference_results.create()`
Trace export	Automatic	Automatic on `clear_context()`
Memory cleanup	Automatic	`clear_context()`
Error handling	Built-in	Manual try/finally

Use generate() for most use cases. Use manual set_context()/clear_context() only when you need to manage inference result creation separately or have complex multi-step workflows.

Error Handling

If the agent raises an exception, generate() ensures trace context is cleaned up:

def failing_agent(input_data: AgentInput) -> AgentResponse:
    raise ValueError("Agent failed intentionally")


try:
    result = galtea_client.inference_results.generate(
        agent=failing_agent, session=session_error, user_input="test"
    )
except Exception as e:
    # Trace context is automatically cleaned up
    print(f"Agent failed: {e}")

Create Inference Result - Manual creation
Trace Service - Manual trace management
Simulating Conversations - Multi-turn simulations

SDK

Concepts

​Overview

​Agent Options

​Parameters

​Returns

​Example

​What Gets Captured

​Providing Usage and Cost Information

​Comparison with Manual Approach

​Error Handling

​Related Methods

Overview

Agent Options

Parameters

Returns

Example

What Gets Captured

Providing Usage and Cost Information

Comparison with Manual Approach

Error Handling

Related Methods