​
Returns

Returns a Test object for the given parameters.

​
Examples

Create a quality test to evaluate the quality and correctness of outputs.
test = galtea.tests.create(
    name="example-test-tutorial",
    type="QUALITY",
    product_id="YOUR_PRODUCT_ID",
    ground_truth_file_path="path/to/knowledge_file.pdf",
    language='english',
    variants=["paraphrased", "typos", "incorrect"],
    max_test_cases=100
)

​
Parameters

​
name
string
required
The name of the test.
​
type
string
required
The type of test. Possible values:
  • QUALITY: Tests that evaluate the quality and correctness of outputs
  • RED_TEAMING: Tests that evaluate security, safety, and bias aspects
  • SCENARIOS: Tests that use conversation simulation to evaluate multi-turn dialogue interactions
​
product_id
string
required
The ID of the product you want to evaluate.
​
ground_truth_file_path
string
Path to a local file (e.g., PDF, TXT, JSON, HTML, Markdown) containing the knowledge base. This file is uploaded to Galtea, which then generates test cases based on its content.
​
few_shot_examples
string
Optional few-shot examples to provide more context to our system about how the test cases should be generated. This can help our system better understand the expected format and style wanted for the test cases. Example:
Q: What is the capital of France?
A: The capital of France is Paris.
Q: What is the capital of Germany?
A: The capital of Germany is Berlin.
This field only applies if tests are generated by Galtea and are of type QUALITY.
​
custom_user_focus
string
Narrow down the scope of generated scenarios by describing a specific type of user, context, or situation. This helps ensure test cases align with your most relevant goals and flows. Example: “A medical professional specialized in dementia with more than 15 years on the field.”
This field only applies if tests are generated by Galtea and are of type SCENARIOS.
​
language
string
The language for generating synthetic test cases if ground_truth_file_path is provided. Defaults to the language detected in the ground truth file. If provided, it should be written in English and be a valid language name from the ISO 639 standard (e.g., “english”, “spanish”, “french”). More information can be found at https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes.
​
variants
list[string]
List of variant types to generate for each test case from the ground_truth_file_path. For details on variants for Quality tests, see Quality Test Evolutions. Example: ["paraphrased", "typos", "incorrect", "cognitively_diverse", "linguistic_diverse"]. For Red Teaming tests, the variants parameter can be used to specify which threat categories to generate test cases for.
​
strategies
list[string]
A list of red teaming strategies used when generating test cases for Red Teaming tests. Strategies are techniques for modifying or obfuscating prompts generated for each threat (as specified in variants for Red Teaming tests). This parameter applies only when type is RED_TEAMING. If specified, this list must include the ‘original’ strategy. Example: ["original", "base64", "leetspeak", "zero_width_insertion"].
​
max_test_cases
int
An optional integer to limit the maximum number of test cases generated by Galtea when ground_truth_file_path is used.
​
test_file_path
string
Path to a local CSV file containing predefined test cases. This file is uploaded to Galtea.File Format by Test Type:
  • QUALITY/RED_TEAMING: Standard format with input, expected_output, tag, source columns
  • SCENARIOS: Conversation simulator format with goal, user_persona, initial_prompt, stopping_criterias, max_iterations, scenario columns
See the Conversation Simulator Tutorial for detailed SCENARIOS CSV format examples.