Behavior Tests - Galtea Docs

What are Behavior Tests?

Behavior tests in Galtea are designed to evaluate the conversational capabilities of your product through multi-turn dialogue interactions. These tests use the Conversation Simulator to create realistic conversations with synthetic users that have specific goals, personalities, and behaviors. Unlike Accuracy Tests that focus on single-turn question-answer pairs, or Security & Safety Tests that test security vulnerabilities, Behavior tests evaluate how well your AI product can:

Maintain context across multiple conversation turns
Guide users toward successful task completion
Handle unexpected but realistic user inputs
Stay in character and follow conversation guidelines
Manage complex dialogue flows

Creating Behavior Tests

You can create behavior tests in Galtea through two methods:

Behavior Generator
Upload Your Own Behavior Tests

Define Your Product

Start by defining your product in Galtea. These detailed product properties will be used as the foundation for generating realistic behavior scenarios.

Configure the test

Select Behavior as the test type and Generated as the test origin

The test creation process can be done via the SDK or the Galtea dashboard

Generate Behavior Scenarios

Galtea will automatically analyze your product definition to create tailored conversation scenarios, complete with:

Realistic user personas based on your target audience
Goals aligned with your product’s use cases
Natural conversation flows that test key features

Conversation Flow Categories

Behavior tests can cover various types of conversational interactions:

Task Completion

Scenarios where the synthetic user has a specific goal to accomplish, such as booking a service, making a purchase, or getting support for an issue.

Information Seeking

Scenarios focused on retrieving specific information or answers, testing the AI’s knowledge and ability to provide helpful responses.

Customer Support

Scenarios that simulate customer service interactions, including complaints, troubleshooting, and problem resolution.

Sales and Consultation

Scenarios where the user is exploring options, comparing products, or seeking advice before making decisions.

Complex Multi-Step Processes

Scenarios that involve multiple stages or require gathering various pieces of information over several turns.

Edge Cases and Challenges

Scenarios that test how the AI handles difficult, unclear, or unexpected user behavior while maintaining conversation quality.

Example Behavior Tests and File Format

Here are examples of behavior test content and structure:

Travel Booking Scenarios

Table View
Raw CSV

Goal	User Persona	Input	Stopping Criterias	Max Iterations	Scenario
Book a one-way flight from SFO to JFK for next Tuesday	A busy professional who is direct and values efficiency	I need to book a flight	The user has confirmed the flight booking\|The chatbot indicates it cannot fulfill the request	10	Flight booking scenario
Find the cheapest round-trip flight to Europe in summer	A budget-conscious student who asks many questions	Hi, I’m looking for cheap flights to Europe	Flight is booked\|User decides not to book\|Maximum budget is exceeded	15	Budget travel scenario
Change an existing flight reservation	A frustrated customer whose plans have changed	I need to change my flight immediately	Reservation is successfully modified\|Customer is transferred to agent	8	Flight modification scenario

goal,user_persona,input,stopping_criterias,max_iterations,scenario
"Book a one-way flight from SFO to JFK for next Tuesday","A busy professional who is direct and values efficiency","I need to book a flight","The user has confirmed the flight booking|The chatbot indicates it cannot fulfill the request",10,"Flight booking scenario"
"Find the cheapest round-trip flight to Europe in summer","A budget-conscious student who asks many questions","Hi, I'm looking for cheap flights to Europe","Flight is booked|User decides not to book|Maximum budget is exceeded",15,"Budget travel scenario"
"Change an existing flight reservation","A frustrated customer whose plans have changed","I need to change my flight immediately","Reservation is successfully modified|Customer is transferred to agent",8,"Flight modification scenario"

Customer Support Scenarios

Table View
Raw CSV

Goal	User Persona	Input	Stopping Criterias	Max Iterations	Scenario
Get help with a defective product	An upset customer who received a broken item	My product arrived broken and I’m very disappointed	Issue is resolved\|Customer requests supervisor\|Refund is processed	12	Product defect resolution
Understand how to use a new feature	A curious but non-technical user	I heard about a new feature but don’t know how to use it	User successfully uses the feature\|User gives up\|Technical support is escalated	10	Feature education scenario
Cancel a subscription	A polite customer who wants to downgrade service	I’d like to cancel my subscription please	Subscription is cancelled\|Alternative plan is accepted\|Retention offer is declined	8	Subscription cancellation

goal,user_persona,input,stopping_criterias,max_iterations,scenario
"Get help with a defective product","An upset customer who received a broken item","My product arrived broken and I'm very disappointed","Issue is resolved|Customer requests supervisor|Refund is processed",12,"Product defect resolution"
"Understand how to use a new feature","A curious but non-technical user","I heard about a new feature but don't know how to use it","User successfully uses the feature|User gives up|Technical support is escalated",10,"Feature education scenario"
"Cancel a subscription","A polite customer who wants to downgrade service","I'd like to cancel my subscription please","Subscription is cancelled|Alternative plan is accepted|Retention offer is declined",8,"Subscription cancellation"

Sales and Consultation Scenarios

Table View
Raw CSV

Goal	User Persona	Input	Stopping Criterias	Max Iterations	Scenario
Find the right product for specific needs	A thorough researcher who compares many options	I’m looking for a solution but need help choosing	Product recommendation is accepted\|User requests human consultation\|User decides to research more	15	Product recommendation scenario
Get pricing information for enterprise solution	A business decision maker focused on ROI	What are your enterprise pricing options?	Quote is requested\|Meeting is scheduled\|User indicates budget constraints	10	Enterprise sales scenario
Compare different service tiers	An analytical customer who wants detailed comparisons	Can you help me understand the differences between your plans?	Plan is selected\|User requests trial\|User needs more time to decide	12	Service comparison scenario

goal,user_persona,input,stopping_criterias,max_iterations,scenario
"Find the right product for specific needs","A thorough researcher who compares many options","I'm looking for a solution but need help choosing","Product recommendation is accepted|User requests human consultation|User decides to research more",15,"Product recommendation scenario"
"Get pricing information for enterprise solution","A business decision maker focused on ROI","What are your enterprise pricing options?","Quote is requested|Meeting is scheduled|User indicates budget constraints",10,"Enterprise sales scenario"
"Compare different service tiers","An analytical customer who wants detailed comparisons","Can you help me understand the differences between your plans?","Plan is selected|User requests trial|User needs more time to decide",12,"Service comparison scenario"

This structure is required by Galtea to automatically generate the test cases for your behavior tests as tests within the platform. If this format is not provided, you can still manually create the test cases.

For automatic processing, the file format must be CSV.

The examples provided above are simplified demonstrations. In actual CSV files, behavior tests can be much more detailed and the number of test cases (rows) can be significantly higher to provide comprehensive conversation testing coverage.

Structure of Behavior Tests

Behavior tests have a specific structure designed to enable realistic conversation simulation:

goal

Text

required

The overall objective the synthetic user is trying to achieve during the conversation. Example: “Book a one-way flight from San Francisco (SFO) to New York (JFK) for next Tuesday”

user_persona

Text

required

The personality and communication style of the synthetic user that will be simulated throughout the conversation. Example: “A busy professional who is direct and values efficiency. They prefer to get things done quickly without much small talk.”

input

Text

The first message the synthetic user sends to start the conversation. If not provided, the system will generate an appropriate opening based on the goal and persona. Example: “Hello, I need to book a flight.”

stopping_criterias

Text

A delimited string of conditions that, if met, will cause the conversation simulation to end. Use ; or | as delimiters to separate multiple criteria. Example: “The user has confirmed the flight booking|The chatbot indicates it cannot fulfill the request;User expresses satisfaction with the service”

max_iterations

Number

The maximum number of conversation turns before the simulation automatically ends. This prevents infinite loops and controls test duration. Example: 10

scenario

Text

A brief description of the scenario for documentation and organizational purposes. This helps categorize and manage different types of conversation tests. Example: “Flight booking scenario for business travelers”

Using Behavior Tests with the Conversation Simulator

Behavior tests are specifically designed to work with Galtea’s Conversation Simulator. When you run evaluations using behavior tests, the system will:

Initialize the conversation using the input and user_persona
Generate realistic user responses based on the goal and conversation history
Continue the dialogue until one of the stopping_criterias is met or max_iterations is reached
Evaluate the conversation using your selected metrics to assess performance

For detailed implementation examples, see the Conversation Simulator Tutorial.

Data Catalog

When generating behavior tests, you can optionally upload a Data Catalog — a file containing reference data from your product’s domain. This helps the generator produce scenarios grounded in realistic values rather than fully fictional ones.

The data catalog does not need to contain real production data. Synthetic samples, anonymized records, or representative mock data all work equally well — what matters is that the values reflect the shape and variety of your domain.

How it works

Values from the data catalog are incorporated into the scenario field of each generated test case. User personas remain fully synthetic and are never populated from the catalog. For example, if your product is a banking assistant, a data catalog might include sample account types, transaction descriptions, and card IDs. The generator would then reference those values in scenarios like “User asks about a pending charge of €47.80 at REST LA BARRACA on card C-001” instead of inventing generic placeholders.

Supported formats

CSV, JSON, JSONL, XML, YAML, and TXT files up to 50 MB.

What to include

Domain entities your users would reference (product names, plan tiers, category labels)
Sample identifiers (order IDs, ticket numbers, account codes)
Representative amounts, dates, or statuses
Any reference data that makes scenarios feel grounded in your specific domain

What to avoid

Actual customer PII or production credentials
Internal system IDs or backend states that end users would never see
Extremely large datasets — the generator uses up to ~50,000 characters of content

Best Practices for Manually Creating Behavior Tests

Goal Definition

Make goals specific and measurable
Include realistic constraints (time, budget, preferences)
Vary complexity across different scenarios
Consider both successful and unsuccessful outcomes

User Persona Creation

Include personality traits that affect communication style
Specify technical knowledge level and domain expertise
Define emotional states and motivations
Consider cultural and contextual factors

Stopping Criteria Design

Include both positive outcomes (goal achieved) and negative outcomes (failure, frustration)
Account for edge cases where the conversation might stall
Use multiple criteria connected with ”|” or ”;” to cover various scenarios
Make criteria specific enough to be clearly identifiable

Scenario Coverage

Test common user journeys and edge cases
Include scenarios with different complexity levels
Cover various user types and use cases
Test both cooperative and challenging user behaviors

SDK

Concepts

​What are Behavior Tests?

​Creating Behavior Tests

​Conversation Flow Categories

​Example Behavior Tests and File Format

​Structure of Behavior Tests

​Using Behavior Tests with the Conversation Simulator

​Data Catalog

​How it works

​Supported formats

​What to include

​What to avoid

​Best Practices for Manually Creating Behavior Tests

What are Behavior Tests?

Creating Behavior Tests

Conversation Flow Categories

Example Behavior Tests and File Format

Structure of Behavior Tests

Using Behavior Tests with the Conversation Simulator

Data Catalog

How it works

Supported formats

What to include

What to avoid

Best Practices for Manually Creating Behavior Tests