Skip to main content

​
Returns

It returns a result object from the Conversation Simulator containing the complete conversation history, status, and metadata. This is used to analyze how your agent performed in the test scenario.

​
Agent Options

The quickest way to get started. Your function receives just the latest user message as a string.
def my_agent(user_message: str) -> str:
    # In a real scenario, call your model here
    return f"Your model output to: {user_message}"
All three signatures work with evaluations.run(), inference_results.generate(), and simulator.simulate(). Both sync and async functions are supported. The SDK auto-detects which signature you’re using from the type hint on the first parameter.

​
AgentInput Fields

When using the Structured signature ((input_data: AgentInput) -> AgentResponse), your function receives an AgentInput object with the following fields:
​
messages
List[ConversationMessage]
The full conversation history up to this turn. Each message has role ("user" or "assistant"), content, and optional retrieval_context and metadata. When the test case has structured JSON input, non-message fields (e.g. chat_type, reasoning_level) are placed in the first user message’s metadata.
​
session_id
str
The current session identifier. Use this to maintain state in stateful agents.
​
context_data
Optional[dict[str, Any]]
Structured context data from the test case or session. Use this to access context fields (e.g. context_data["customer_tier"]).
​
inference_result_id
Optional[str]
The inference result ID for this execution. Forward this to remote agents so they can attach traces to the correct session. See Remote Agent Tracing.
​
metadata
Optional[dict[str, Any]]
Additional metadata for the current execution.

​
Helper Methods

​
last_user_message_str()
Optional[str]
Returns the content of the last user message. Returns None if no user message is found in messages.
​
last_user_message()
Optional[ConversationMessage]
Returns the last ConversationMessage with role == "user", or None if not found.

​
Example

# Define your agent function
def my_agent(user_message: str) -> str:
    return f"Response to: {user_message}"


simulation_result = galtea.simulator.simulate(
    session_id=behavior_session.id,
    agent=my_agent,
    max_turns=3,

)

​
Parameters

​
session_id
str
required
The session identifier for this simulation.
​
agent
AgentType
required
The agent function that generates responses. Supported signatures:
  • Simple: (user_message: str) -> str — receives last user message
  • Chat History: (messages: list[dict]) -> str — receives full chat history (OpenAI format)
  • Structured: (input_data: AgentInput) -> AgentResponse — structured input/output
Both sync and async functions are supported. Use the structured signature when you need usage_info, cost_info, or retrieval_context.
​
max_turns
int
Maximum number of conversation turns. Defaults to the Test Case’s max turns if not provided.
​
log_inference_results
bool
Deprecated — Inference results are now automatically tracked by the API. This parameter is kept for backward compatibility but no longer has any effect.
​
enable_last_inference
bool
Whether to call your agent one final time after the simulated user sends their last message. Default: True. When enabled, your agent will have the opportunity to respond to the conversation’s final user message, ensuring complete dialogue coverage for evaluation purposes.
​
include_metadata
bool
Whether to include metadata in the simulation result. Default: False
​
agent_goes_first
bool
If True, the agent will generate the first message before any user messages are generated. Default: False
When agent_goes_first is set to True:
  • Any input (first user message) defined on the associated TestCase will be ignored.
  • The first call to your agent will have an empty messages list in the AgentInput. Ensure your agent function can handle cases where input_data.last_user_message_str() returns None (or the messages list is empty).
​
stop_when_agent_returns_empty_response
bool
If True, the simulation will stop when the agent returns an empty ("" / None) response. Default: True
When stop_when_agent_returns_empty_response is set to False:
  • The simulation will continue even if the agent returns a response that is empty ("") or None.
  • This allows for more flexible conversation flows where you want to capture all agent interactions regardless of content.
  • Be careful when using this option, as it may lead to further credit consumption if the agent keeps returning empty responses. To mitigate this, you can also make use of the max_turns parameter to set an upper limit on the number of turns.