> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Simulating User Conversations
> Learn how to use Galtea's Conversation Simulator to test your AI with simulated multi-turn conversations.
Galtea's [Conversation Simulator](/concepts/product/test/case/conversation-simulator) allows you to test your AI products — chatbots, assistants, and agents — by simulating realistic multi-turn user interactions. This guide walks you through integrating your agent and running simulations using [Behavior Tests](/concepts/product/test/behavior-tests).
**Using the specification-driven workflow?** If you've defined [Specifications](/concepts/product/specification) with linked metrics and tests, the evaluation pipeline handles simulation automatically — no manual session creation needed. See [Evaluations from Specs](/sdk/tutorials/specification-driven-evaluations).
## Agent Integration Options
The quickest way to get started. Your function receives just the latest user message as a string.
```python theme={"system"}
def my_agent(user_message: str) -> str:
# In a real scenario, call your model here
return f"Your model output to: {user_message}"
```
Use this when your agent needs the full conversation context. Your function receives the message list in the OpenAI format (`{"role": "...", "content": "..."}`).
```python theme={"system"}
def my_agent(messages: list[dict]) -> str:
# messages follows the standard chat format:
# [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}, ...]
user_message = messages[-1]["content"]
return f"Your model output to: {user_message}"
```
For full control over input and output — including optional usage tracking, cost tracking, and retrieval context for RAG evaluations.
```python theme={"system"}
def my_agent(input_data: AgentInput) -> AgentResponse:
user_message = input_data.last_user_message_str()
# Access structured input fields from the first user message's metadata
# (e.g. when test case input is {"user_message": "hello", "chat_type": "support"})
first_msg = input_data.messages[0] if input_data.messages else None
chat_type = first_msg.metadata.get("chat_type") if first_msg and first_msg.metadata else None
# In a real scenario, call your model here
model_output = f"Your model output to: {user_message}"
# Return AgentResponse with optional usage/cost tracking
return AgentResponse(
content=model_output,
usage_info={"input_tokens": 100, "output_tokens": 50},
)
```
All three signatures work with `evaluations.run()`, `inference_results.generate()`, and `simulator.simulate()`. Both sync and async functions are supported. The SDK auto-detects which signature you're using from the type hint on the first parameter.
For the full list of fields available on `AgentInput` (including structured input access via message metadata), see the [AgentInput reference](/sdk/api/agent/input).
## Conversation Simulation Workflow
Define an agent function with one of the supported signatures above.
Create a CSV file with scenario data, or generate test cases from your [specifications](/concepts/product/specification). Each row describes a user goal, persona, scenario, and the first user message.
Upload your scenario CSV to create a test, or use AI-generated tests from the dashboard. The platform generates a session for each test case.
Use `simulator.simulate()` to execute the conversation between your agent and the simulated user for each session.
After simulation, trigger evaluations via `evaluations.create()` or review results in the dashboard's **Analytics** tab.
### Step-by-Step Guide
#### 1. Create a Test and Sessions
First, create behavior test cases with user personas and goals. You can generate these from your product description or upload a CSV:
```python theme={"system"}
# Create a test suite using the behavior test options
# This can be done via the Dashboard or programmatically as shown here
test = galtea_client.tests.create(
product_id=product.id,
name=test_name,
type="BEHAVIOR",
# This time we provide a path to a CSV file with behavior tests, but you can also have Galtea generate them if you do not provide a CSV file
test_file_path="path/to/behavior_test.csv",
)
# Get your test cases
# If Galtea is generating the test for you, it might take a few moments to be ready
test_cases = galtea_client.test_cases.list(test_id=test.id)
```
Once generation completes, you'll see the resulting test cases in your dashboard. For the CSV upload format, see [Behavior Tests](/concepts/product/test/behavior-tests).
#### 2. Run the Conversation Simulator
For each test case/session, use the simulator to run the full simulation with your agent function:
```python theme={"system"}
# Define your agent function (see Agent Integration Options for all signatures)
def my_agent(user_message: str) -> str:
return f"Response to: {user_message}"
# Run simulations with your agent function
for test_case in test_cases:
session = galtea_client.sessions.create(version_id=version.id, test_case_id=test_case.id)
result = galtea_client.simulator.simulate(session_id=session.id, agent=my_agent, max_turns=10)
# Analyze results
print(f"Scenario: {test_case.scenario}")
print(f"Completed {result.total_turns} turns")
print(f"Success: {result.finished}")
if result.stopping_reason:
print(f"Ended because: {result.stopping_reason}")
```
You can optionally use the `@trace` decorator to capture internal operations during simulation. Traces are automatically collected and saved per turn.
See the [Tracing Agent Operations](/sdk/tutorials/tracing-agent-operations) guide for more details.
#### 3. Evaluate the Session
```python theme={"system"}
# After each simulation, you can create an evaluation
evaluations = galtea_client.evaluations.create(
session_id=session.id,
metrics=[{"name": "Role Adherence"}], # Replace with your metrics
)
for evaluation in evaluations:
print(f"Evaluation created: {evaluation.id}")
```
When using structured JSON inputs in test cases (e.g. `{"user_message": "hello", "chat_type": "support"}`), the extra fields are available in `messages[0].metadata` on the `AgentInput`. See [AgentInput reference](/sdk/api/agent/input#accessing-structured-input-fields) for details.
## Advanced Usage: RAG Agents with Retrieval Context
For Retrieval-Augmented Generation (RAG) agents, you can return the context that was retrieved and used to generate the response. This context will be logged with the inference result, enabling evaluations with metrics like `Faithfulness` and `Contextual Relevancy`.
```python theme={"system"}
def my_rag_agent(input_data: galtea.AgentInput) -> galtea.AgentResponse:
user_message = input_data.last_user_message_str()
# Your RAG logic to retrieve context and generate a response
retrieved_docs = vector_store.search(user_message)
response_content = llm.generate(prompt=user_message, context=retrieved_docs)
return galtea.AgentResponse(
content=response_content,
retrieval_context=retrieved_docs,
metadata={"docs_retrieved": len(retrieved_docs)},
)
```
The `retrieval_context` field is optional and can contain:
* Retrieved document snippets or full documents
* Formatted context strings
* JSON-serializable data structures
By providing retrieval context, you enable Galtea to evaluate the faithfulness of your model's responses relative to the retrieved information.
## Next Steps
Automate simulation + evaluation from specifications with `evaluations.run()`.
Capture internal operations of your AI agent during simulations.