Creating Tests with the SDK
Tests can be generated automatically by Galtea providing a knowledge file via the sdk or the dashboard.
If you have a prepared test file in CSV format, you can upload it directly.
The test file must follow the structure specified for:
- Quality Tests: Standard format with
input
,expected_output
, etc. - Red Teaming Tests: Standard format with adversarial inputs
- Scenarios Tests: Conversation simulator format with
goal
,user_persona
, etc.
New Test Type: Use
type="SCENARIOS"
to create conversation simulation tests that enable multi-turn dialogue evaluation with synthetic users. See the Conversation Simulator Tutorial for complete examples.