Skip to main content
POST
/
tests
Create test
curl --request POST \
  --url https://api.galtea.ai/tests \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "productId": "prod_123",
  "name": "Quality Test",
  "type": "QUALITY",
  "specificationId": "spec_123",
  "groundTruthUri": "https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "uri": "https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "fewShot": "Q: What is 2+2? A: 4",
  "languageCode": "en",
  "variants": [
    "rag"
  ],
  "customVariantDescription": "Custom test variant",
  "strategies": [
    "original"
  ],
  "customUserPersona": "Business analyst",
  "maxTestCases": 100,
  "maxIterations": 10,
  "models": [
    "gpt-4"
  ],
  "sourceTestId": "test_123",
  "dataCatalogUri": "https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "metadata": {
    "key": "value"
  }
}
'
{
  "id": "test_123",
  "productId": "prod_123",
  "userId": "user_123",
  "name": "Quality Test",
  "type": "QUALITY",
  "groundTruthUri": "s3://my-bucket/tests/test_123/ground-truth.csv",
  "uri": "s3://my-bucket/tests/test_123/test.csv",
  "error": "<string>",
  "status": "SUCCESS",
  "fewShot": "Example few-shot learning data",
  "languageCode": "en",
  "variants": [
    "rag",
    "summarization"
  ],
  "customVariantDescription": "Custom variant description",
  "strategies": [
    "original"
  ],
  "customUserPersona": "Business analyst",
  "maxTestCases": 100,
  "models": [
    "gpt-4",
    "claude-3"
  ],
  "sourceTestId": "<string>",
  "dataCatalogUri": "s3://my-bucket/tests/test_123/data-catalog.json",
  "metadata": {
    "key": "value"
  },
  "specificationId": "spec_123",
  "createdAt": "2023-11-07T05:31:56Z",
  "deletedAt": "2023-11-07T05:31:56Z"
}

Documentation Index

Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt

Use this file to discover all available pages before exploring further.

Authorizations

​
Authorization
string
header
required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Body

application/json

Test data

​
productId
string
required

Product ID

Example:

"prod_123"

​
name
string
required

Test name

Example:

"Quality Test"

​
type
enum<string>
required
Available options:
QUALITY,
RED_TEAMING,
SCENARIOS
Example:

"QUALITY"

​
specificationId
string

Specification ID (can auto-derive type/variants)

Example:

"spec_123"

​
groundTruthUri
string

Presigned GET URL (S3 X-Amz-* query or Azure Blob SAS) for a ground-truth file previously uploaded via the storage PUT-presign endpoint. The server strips the signing query and persists the underlying storage URI. Cannot be combined with sourceTestId.

Example:

"https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

​
uri
string

Presigned GET URL for a fully prepared custom test file previously uploaded via the storage PUT-presign endpoint. When provided, no test-case generation runs: the file is ingested as-is, the test is created in SUCCESS state, and maxTestCases, generator-related fields, and credit checks are skipped. Cannot be combined with sourceTestId or dataCatalogUri. Must be non-empty.

Example:

"https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

​
fewShot
string

Optional few-shot examples used to guide test-case generation for QUALITY tests.

Example:

"Q: What is 2+2? A: 4"

​
languageCode
string
Example:

"en"

​
variants
string[]

Test variants. QUALITY: rag, entity_extraction, summarization, classification, translation, correction, other. RED_TEAMING: data_leakage, financial_attacks, illegal_activities, misuse, toxicity, custom.

Example:
["rag"]
​
customVariantDescription
string

Custom variant description

Example:

"Custom test variant"

​
strategies
string[]

Generation strategies. "original" is the default for RED_TEAMING. At least one strategy is required for SCENARIOS tests.

Example:
["original"]
​
customUserPersona
string
Example:

"Business analyst"

​
maxTestCases
integer
Example:

100

​
maxIterations
integer

SCENARIOS only. Maximum number of conversation turns per generated scenario when scenario generation is executed. If omitted, the scenario-generator default is used. If uri is provided, no scenario generation runs, so this field is ignored at test-creation time. When generation runs, the value is forwarded to the scenario-generator and persisted on each generated test case. Ignored for QUALITY and RED_TEAMING tests.

Required range: x >= 1
Example:

10

​
models
string[]
Example:
["gpt-4"]
​
sourceTestId
string

Source QUALITY test ID whose approved test cases seed a SCENARIOS test. Only valid when type = SCENARIOS. Cannot be combined with uri or groundTruthUri.

Example:

"test_123"

​
dataCatalogUri
string

Presigned GET URL for a data-catalog file previously uploaded via the storage PUT-presign endpoint. Only valid when type = SCENARIOS. Cannot be combined with uri.

Example:

"https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

​
metadata
object

Arbitrary key-value metadata

Example:
{ "key": "value" }

Response

Test created successfully

​
id
string
Example:

"test_123"

​
productId
string | null
Example:

"prod_123"

​
userId
string | null
Example:

"user_123"

​
name
string
Example:

"Quality Test"

​
type
enum<string>
Available options:
QUALITY,
RED_TEAMING,
SCENARIOS
Example:

"QUALITY"

​
groundTruthUri
string | null

Canonical storage URI of the ground-truth file, derived from the presigned URL supplied at creation. S3: s3://<bucket>/<key>. Azure Blob: blob URL with the SAS query stripped (e.g. https://<account>.blob.core.windows.net/<container>/<path>).

Example:

"s3://my-bucket/tests/test_123/ground-truth.csv"

​
uri
string | null

Canonical storage URI of the uploaded custom test file. Same format rules as groundTruthUri.

Example:

"s3://my-bucket/tests/test_123/test.csv"

​
error
string | null
​
status
enum<string>
Available options:
PENDING,
SUCCESS,
FAILED,
AUGMENTING
Example:

"SUCCESS"

​
fewShot
string | null

Optional few-shot examples (input/output pairs) used to guide test-case generation for QUALITY tests.

Example:

"Example few-shot learning data"

​
languageCode
string | null
Example:

"en"

​
variants
string[]

Test variants. QUALITY: rag, entity_extraction, summarization, classification, translation, correction, other. RED_TEAMING: data_leakage, financial_attacks, illegal_activities, misuse, toxicity, custom.

Example:
["rag", "summarization"]
​
customVariantDescription
string | null
Example:

"Custom variant description"

​
strategies
string[]

Generation strategies. "original" is the default for RED_TEAMING. At least one strategy is required for SCENARIOS tests.

Example:
["original"]
​
customUserPersona
string | null
Example:

"Business analyst"

​
maxTestCases
integer | null
Example:

100

​
models
string[]
Example:
["gpt-4", "claude-3"]
​
sourceTestId
string | null
​
dataCatalogUri
string | null

Canonical storage URI of the data-catalog file (SCENARIOS tests only). Same format rules as groundTruthUri.

Example:

"s3://my-bucket/tests/test_123/data-catalog.json"

​
metadata
object
Example:
{ "key": "value" }
​
specificationId
string | null
Example:

"spec_123"

​
createdAt
string<date-time>
​
deletedAt
string<date-time> | null