​
Returns

Returns a Test object for the given parameters.

​
Example

test = galtea.tests.create(
    name="example-test-tutorial",
    type="QUALITY",
    product_id="YOUR_PRODUCT_ID",
    ground_truth_file_path="path/to/knowledge_file.pdf",
    language='english',
    variants=["paraphrased", "typos", "incorrect"],
    max_test_cases=100
)

See a complete example of creating custom tests in our Create a Custom Test example.

​
Parameters

​
name
string
required

The name of the test.

​
type
string
required

The type of test (e.g., QUALITY, RED_TEAMING).

​
product_id
string
required

The ID of the product you want to evaluate.

​
ground_truth_file_path
string

Path to a local file (e.g., PDF, TXT, JSON, HTML, Markdown) containing the knowledge base. This file is uploaded to Galtea, which then generates test cases based on its content.

​
language
string

The language for generating synthetic test cases if ground_truth_file_path is provided. Defaults to the language detected in the ground truth file. If provided, it should be written in English and be a valid language name from the ISO 639 standard (e.g., “english”, “spanish”, “french”). More information can be found at https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes.

​
variants
list[string]

List of variant types to generate for each test case from the ground_truth_file_path. For details on variants for Quality tests, see Quality Test Evolutions. Example: ["paraphrased", "typos", "incorrect", "cognitively_diverse", "linguistic_diverse"]. For Red Teaming tests, the variants parameter can be used to specify which threat categories to generate test cases for.

​
strategies
list[string]

A list of red teaming strategies used when generating test cases for Red Teaming tests. Strategies are techniques for modifying or obfuscating prompts generated for each threat (as specified in variants for Red Teaming tests). This parameter applies only when type is RED_TEAMING. If specified, this list must include the ‘original’ strategy. Example: ["original", "base64", "leetspeak", "zero_width_insertion"].

​
max_test_cases
int

An optional integer to limit the maximum number of test cases generated by Galtea when ground_truth_file_path is used.

​
test_file_path
string

Path to a local CSV file containing predefined test cases. This file is uploaded to Galtea.