Quality Tests
Tests that evaluate the quality and correctness of outputs
What are Quality Tests?
Quality tests in Galtea are designed to evaluate the quality, accuracy, and correctness of your product’s outputs. These tests typically consist of questions or prompts paired with ideal answers, allowing Galtea to assess how well your product’s responses match the expected standard.
Creating Quality Tests
You can create quality tests in Galtea through two methods:
Prepare your knowledge base file
Create a file containing all the information that the Product should know and adhere to
Configure the test
Select Quality as the test type and Generated as the test origin
The test creation process acan be done via the SDK or the Galtea platform
Generate the test
Galtea will process the Knowledge Base and generate a Test File and Test Cases for you
Prepare your knowledge base file
Create a file containing all the information that the Product should know and adhere to
Configure the test
Select Quality as the test type and Generated as the test origin
The test creation process acan be done via the SDK or the Galtea platform
Generate the test
Galtea will process the Knowledge Base and generate a Test File and Test Cases for you
Create your test file
Prepare a CSV file following the structure shown above
Configure the test
Select “Quality” as the test type and “Uploaded” as the test origin
The test creation process acan be done via the SDK or the Galtea platform
Upload your test file
Select the file you created and upload it to Galtea
Example Quality Tests and File Format
Here are examples of the content and its creation-document structure:
This structure is the required by Galtea in order to automatically generate the test cases for your test. If that format is not provided, you can still manually create the test cases.
For automatic processing, the file format must be CSV.
The extent of the examples provided above are short extractions. In the actual CSV files they can be much longer and the amount of test cases (rows) can be much higher.
Structure of Quality Tests
Quality tests have a specific structure designed to facilitate effective evaluation:
A unique identifier for the test instance.
The question or prompt that the product is expected to respond to. Example: “What are three key provisions of the GDPR related to data subject rights?”
The expected answer to the question, serving as a benchmark for evaluation. Example: “The GDPR provides several key rights to data subjects, including: the right to access, the right to be forgotten, the right to rectification, the right to restrict processing, the right to data portability, and the right to object to processing.”
A categorization tag for the test instance. Example: “original”
The source text used to generate the test case, which can be a section of a document, a webpage, or any other relevant material. This field is particularly useful for tracing the origin of the information used in the test case.
Example: “GDPR Guidelines Document (2018): The General Data Protection Regulation (GDPR) is a regulation in EU law on data protection and privacy in the European Union and the European Economic Area…”