Returns
Returns a Dataset object for the given parameters.Examples
- Accuracy Dataset
- Security Dataset
- Behavior Dataset
- Accuracy Custom Dataset from CSV
Create an accuracy dataset to evaluate the quality and correctness of outputs.
Parameters
The type of dataset. Required unless
specification_ids is provided (in which case the type is auto-derived from the specifications).ACCURACY: Datasets that evaluate the quality and correctness of outputsSECURITY: Datasets that evaluate security, safety, and bias aspectsBEHAVIOR: Datasets that use conversation simulation to evaluate multi-turn dialogue interactions
DatasetType enum: DatasetType.ACCURACY, DatasetType.SECURITY, DatasetType.BEHAVIOR.The IDs of the Specifications to link the dataset to. They must belong to the same product and share one dataset type. When provided,
type is auto-resolved from them and becomes optional. A dataset can link to several specifications. An evaluation that names several of them runs the dataset once and scores it with the metrics of all of them. Generation reads only the first specification in the list. An API version that links a dataset to one specification only creates the dataset with no link for several ids, and the SDK then raises a RuntimeError that names the dataset.Deprecated: pass
specification_ids=[id] instead. The SDK sends it as a one-item specification_ids. Passing both raises a ValueError unless specification_ids, once its duplicates are removed, is [specification_id].Path to a local file containing the knowledge base. This file is uploaded to Galtea and used to build the knowledge base for your dataset.
Supported formats:
.pdf, .txt, .json, .html, .md, .yaml, .yml,
.xml, .docx, and .zipMaximum file size: 100 MB
Maximum ZIP contents: 1,000 filesOptional few-shot examples to provide more context to our system about how the test cases should be generated. This can help our system better understand the expected format and style wanted for the test cases.
Example:
This field only applies if datasets are generated by Galtea and are of type
ACCURACY.Narrow down the scope of generated scenarios by describing a specific type of user, context, or situation. This helps ensure test cases align with your most relevant goals and flows.
Example: “A medical professional specialized in dementia with more than 15 years on the field.”
This field only applies if datasets are generated by Galtea and are of type
BEHAVIOR.The language for generating synthetic test cases if
ground_truth_file_path is provided. Defaults to the language detected in the ground truth file. Accepts a BCP-47 language tag: a bare ISO 639-1 code ("es") or a code with a region subtag ("es-MX", "en-GB"). The region subtag selects the regional variety in the simulated user’s messages and in voice synthesis (the accent the simulated caller speaks with); generated test text keeps the base language. The value is stored as the full tag. A bare code for ca, de, es, eu, fr, gl, it, nl or pt defaults to that language’s European variety in the simulated user’s messages only (pt → pt-PT); an explicit region always overrides it, and neither voice synthesis nor the stored value is affected.Background noise mixed into the simulated caller’s audio for voice evaluations run over a phone connection. Ignored for datasets that don’t run over a phone connection.
- Clip-backed:
office,street,car - Synthetic:
white,pink
test_cases.create() do not inherit it automatically — pass the same values there to match.Must be set together with
background_noise_level — passing only one of the two is rejected by the API. Leave both unset for a clean recording (the default).How loud the background noise is relative to the caller’s speech, as a fixed signal-to-noise ratio per level. One of
light, medium, or heavy. Must be set together with background_noise_profile.A list of strings that specifies how to generate test cases. The meaning of this parameter depends on the dataset
type:- For Security datasets (
SECURITY): Specifies which threat category to generate test cases for. Exactly one threat must be provided (e.g.,["toxicity"]). Available threats:toxicity,data_leakage,misuse,custom,financial_attacks,illegal_activities.
A list of strings that specifies how to generate test cases related to its style.
- For Security datasets (
SECURITY): Strategies are techniques for modifying or obfuscating prompts generated for each threat. Theoriginalstrategy must always be included. See the full list of security strategies. - For Behavior datasets (
BEHAVIOR): Strategies define the conversation style. Currentlywrittenandspokenare supported, which influence the tone and formality of the generated dialogues. If not specified, the default strategy iswritten.
Description for guiding synthetic data generation for security datasets. This parameter defines the behavior of the
custom threat when used as the variant for Security datasets (SECURITY).Maximum number of test cases to generate from the knowledge base.Default values:
- Accuracy/Security datasets:
50test cases - Behavior datasets:
20test cases
- Accuracy/Security datasets:
1000test cases - Behavior datasets:
100test cases
If you need more than the maximum limit, consider:
- Splitting your knowledge base into multiple datasets
- Creating test cases manually with
test_cases.create() - Uploading a CSV file with predefined test cases using
dataset_file_path
Maximum number of conversation turns per generated scenario for Behavior datasets (
BEHAVIOR). It counts the live turns only: a test case that starts from a predefined conversation runs this many turns on top of it. A 4-turn predefined conversation with max_iterations 1 runs one new turn and the session ends with 5.Only applies when scenarios are generated by Galtea. The value is stamped on every generated test case and consumed by the conversation simulator at evaluation time (read as max_turns).Ignored for Accuracy (ACCURACY) and Security (SECURITY) datasets, and ignored when a custom dataset file is uploaded via dataset_file_path (no generation runs).Constraints: must be >= 1.Default: if omitted, the scenario-generator default is used.Path to a local CSV file containing predefined test cases. This file is uploaded to Galtea.File Format by Dataset Type:
- Accuracy/Security datasets: Standard format with
input,expected_output,tag,sourcecolumns - Behavior datasets: Conversation simulator format with
goal,user_persona,input,stopping_criterias,max_iterations,scenario,conversation_historycolumns.conversation_historyholds a JSON array of turns, each one{"role": "user"|"assistant", "content": "..."}, and the simulation continues that conversation instead of starting empty. A turn carryingretrieval_contextis refused, and a row that sets the column has itsinputignored. Such a row must not attach files to itsinput, because the platform supports file inputs on single-turn runs only and a seeded run is multi-turn. This applies to a text run: a voice run (a phone or WebRTC connection) ignores the column and still opens the conversation frominput.
input_file_paths column attaches files to a row’s input, for a product that reads documents. Galtea sends each attachment to your endpoint when the endpoint connection template names input_files. Separate several entries with ; or |. Each entry is a local path the SDK uploads, or a storage URI from an earlier upload, which is attached without uploading the file again. The SDK rewrites the row before the CSV is sent, so the column itself never reaches Galtea. A row may attach files and leave input empty. A CSV using this column must be UTF-8 encoded. See File inputs.Deprecated alias:
test_file_path.