# Export analytics data Source: https://docs.galtea.ai/api-reference/analytics/export-analytics-data https://api.galtea.ai/openapi.json post /analytics/export Export analytics data for a product with optional filtering. See [Platform Overview](https://docs.galtea.ai/introduction). # Get analytics data Source: https://docs.galtea.ai/api-reference/analytics/get-analytics-data https://api.galtea.ai/openapi.json get /analytics Get analytics data for a product with optional filtering. See [Platform Overview](https://docs.galtea.ai/introduction). # Get analytics insights for a version Source: https://docs.galtea.ai/api-reference/analytics/get-analytics-insights-for-a-version https://api.galtea.ai/openapi.json get /analytics/insights Computes deterministic red flags / proactive insights (low metric scores, specification coverage gaps) for a single version, plus a fail-open LLM-generated narrative summary. See [Platform Overview](https://docs.galtea.ai/introduction). # Get cross-version analytics comparison insights Source: https://docs.galtea.ai/api-reference/analytics/get-cross-version-analytics-comparison-insights https://api.galtea.ai/openapi.json get /analytics/insights/comparison Computes deterministic red flags / proactive insights across versions (metric degradation, metric improvement, cost outlier, latency outlier), plus a fail-open LLM-generated narrative summary. When versionIds is omitted, all versions present in the analytics data are compared. See [Platform Overview](https://docs.galtea.ai/introduction). # Get per-version scores and the newest version, per product Source: https://docs.galtea.ai/api-reference/analytics/get-per-version-scores-and-the-newest-version-per-product https://api.galtea.ai/openapi.json get /analytics/summary For each requested product, returns the per-version overall score (the same numbers as `averages.total` on `GET /analytics`) and the newest version. Computed as one grouped aggregate covering every requested product plus one lookup for the newest version, so a page of product cards issues one request instead of one per card. # Create API token Source: https://docs.galtea.ai/api-reference/api-tokens/create-api-token https://api.galtea.ai/openapi.json post /auth/apiTokens Create a new named API token for the authenticated user. The key is returned only once and cannot be retrieved later. Creating more than one token requires an Enterprise plan. # List API tokens Source: https://docs.galtea.ai/api-reference/api-tokens/list-api-tokens https://api.galtea.ai/openapi.json get /auth/apiTokens List the authenticated user's API tokens. Secrets are never returned — only id, name, keyId and creation date. # Regenerate API token Source: https://docs.galtea.ai/api-reference/api-tokens/regenerate-api-token https://api.galtea.ai/openapi.json post /auth/apiTokens/{apiTokenId}/regenerate Rotate the secret of one of the authenticated user's API tokens. The old key stops working and the new key is returned only once. Always allowed, regardless of plan. # Rename API token Source: https://docs.galtea.ai/api-reference/api-tokens/rename-api-token https://api.galtea.ai/openapi.json patch /auth/apiTokens/{apiTokenId} Rename one of the authenticated user's API tokens. Always allowed, regardless of plan. # Revoke API token Source: https://docs.galtea.ai/api-reference/api-tokens/revoke-api-token https://api.galtea.ai/openapi.json delete /auth/apiTokens/{apiTokenId} Revoke (permanently delete) one of the authenticated user's API tokens. Always allowed, regardless of plan. # Get current user Source: https://docs.galtea.ai/api-reference/auth/get-current-user https://api.galtea.ai/openapi.json get /auth/user Get the currently authenticated user. See [Registration & Authentication](https://docs.galtea.ai/registration). # Build info Source: https://docs.galtea.ai/api-reference/build-info https://api.galtea.ai/openapi.json get /build-info Deployed build metadata (version, commit, build date), so on-prem customers can compare their deployed version with the SaaS. Requires authentication; restricted to admins and organizations with an enterprise subscription. # Build info for every deployed service Source: https://docs.galtea.ai/api-reference/build-info-for-every-deployed-service https://api.galtea.ai/openapi.json get /build-info/services The API's own build metadata plus one entry per downstream Python service, so on-prem customers and support can see the full component inventory of a deployment at a glance. A downstream service being down never fails this call; it is reported with `status: unreachable` instead. Requires authentication; restricted to admins and organizations with an enterprise subscription, same rule as `GET /build-info`. # Create comment Source: https://docs.galtea.ai/api-reference/comments/create-comment https://api.galtea.ai/openapi.json post /comments Adds a free-form annotation to a trace, for error analysis outside the metric-bound evaluation flow. A trace can hold many comments. # Delete comment Source: https://docs.galtea.ai/api-reference/comments/delete-comment https://api.galtea.ai/openapi.json delete /comments/{commentId} Soft-deletes one comment. Only the author, or an ADMIN/OWNER, may delete it. # List distinct comment labels Source: https://docs.galtea.ai/api-reference/comments/list-distinct-comment-labels https://api.galtea.ai/openapi.json get /comments/tags Returns the sorted, distinct labels used on the comments of the requested products, lowercase and trimmed. Labels are product-scoped: this is the reuse list a reviewer picks from when annotating a turn, and the options of the session-list label filter. `productIds` is required; a product the caller cannot read yields 403. # Create dataset Source: https://docs.galtea.ai/api-reference/datasets/create-dataset https://api.galtea.ai/openapi.json post /tests Create a new dataset. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). **File uploads.** The `uri`, `groundTruthUri`, and `dataCatalogUri` fields of `TestInput` each expect a presigned GET URL (S3 `X-Amz-*` or Azure Blob SAS) for a file that was already uploaded via the storage PUT-presign endpoint. The server strips the signing query and persists the underlying storage URI (`s3:///` for S3, or the query-less blob URL for Azure) on the created dataset. A file URI the API returned under `STORAGE_PUBLIC_BASE_URL` is accepted too, and stored in that same canonical form. Providing `uri` switches the request into "custom dataset file" mode: generation is skipped, the dataset is created in `SUCCESS` state, and credit checks / `maxTestCases` / generator-specific validations are bypassed. Creation is the only time `specificationId` can be set directly. To attach an existing dataset to a specification afterwards, use `specifications link-datasets`. # Create multiple datasets Source: https://docs.galtea.ai/api-reference/datasets/create-multiple-datasets https://api.galtea.ai/openapi.json post /tests/batch Create multiple datasets at once. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Delete dataset Source: https://docs.galtea.ai/api-reference/datasets/delete-dataset https://api.galtea.ai/openapi.json delete /tests/{id} Soft delete a dataset. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Delete multiple datasets Source: https://docs.galtea.ai/api-reference/datasets/delete-multiple-datasets https://api.galtea.ai/openapi.json delete /tests/batch Soft delete multiple datasets by their IDs. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Extend a generated dataset with more cases Source: https://docs.galtea.ai/api-reference/datasets/extend-a-generated-dataset-with-more-cases https://api.galtea.ai/openapi.json post /tests/{id}/extend Re-run the generator that originally produced this dataset (scenario, red teaming, or gold standard) and append the new cases to it. Only datasets Galtea generated from a specification or ground-truth document can be extended. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Generate dataset configuration from specifications Source: https://docs.galtea.ai/api-reference/datasets/generate-dataset-configuration-from-specifications https://api.galtea.ai/openapi.json post /tests/generate-config Derives dataset type, variant, and name from specifications (rule-based). With `automaticCreation: true` this does more than return a suggestion: it **creates the datasets** and their test-case generation continues on the background worker, so each one comes back with status `PENDING`. Accuracy (QUALITY) specifications need `groundTruthUri`, a knowledge base file uploaded beforehand, because an accuracy dataset grades answers against known-correct ones and a specification never states what the correct answer is. Behavior (SCENARIOS) specifications can take `dataCatalogUri`, a data catalog file uploaded beforehand, so the generated scenarios use the product's real values instead of invented ones. It is optional, one catalog covers every behavior dataset the call creates, and it is ignored for the other test types. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Get dataset by ID Source: https://docs.galtea.ai/api-reference/datasets/get-dataset-by-id https://api.galtea.ai/openapi.json get /tests/{id} Get a specific dataset by ID. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Get dataset types used by the organization Source: https://docs.galtea.ai/api-reference/datasets/get-dataset-types-used-by-the-organization https://api.galtea.ai/openapi.json get /tests/types Retrieves the non-repeated dataset types of all the products of the user's organization, or of the given products (403 if any is not readable). A READER or HUMAN_EVALUATOR gets the types of the products their user groups grant. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Get datasets Source: https://docs.galtea.ai/api-reference/datasets/get-datasets https://api.galtea.ai/openapi.json get /tests Get list of datasets with pagination and filtering options. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Update dataset Source: https://docs.galtea.ai/api-reference/datasets/update-dataset https://api.galtea.ai/openapi.json patch /tests/{id} Apply a partial update to an existing dataset. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). **Only `name` and `metadata` are mutable** post-creation. Attempting to update any other field (`type`, `variants`, `strategies`, `uri`, `groundTruthUri`, `dataCatalogUri`, `sourceTestId`, `maxTestCases`, etc.) returns a 400. Those are fixed for the lifetime of the dataset because they drive generation, credit accounting, and stored test-case shape. To attach an existing dataset to a specification (or detach it), use `specifications link-datasets` / `specifications unlink-datasets` — the dataset-specification link is managed from the specification side, not by updating the dataset. # Create endpoint connection Source: https://docs.galtea.ai/api-reference/endpoint-connections/create-endpoint-connection https://api.galtea.ai/openapi.json post /endpointConnections Create a new endpoint connection. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Delete endpoint connection Source: https://docs.galtea.ai/api-reference/endpoint-connections/delete-endpoint-connection https://api.galtea.ai/openapi.json delete /endpointConnections/{id} Soft delete an endpoint connection. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Delete multiple endpoint connections Source: https://docs.galtea.ai/api-reference/endpoint-connections/delete-multiple-endpoint-connections https://api.galtea.ai/openapi.json delete /endpointConnections/batch Soft delete multiple endpoint connections by their IDs. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Get endpoint connection by ID Source: https://docs.galtea.ai/api-reference/endpoint-connections/get-endpoint-connection-by-id https://api.galtea.ai/openapi.json get /endpointConnections/{id} Get a specific endpoint connection by ID. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Get endpoint connections Source: https://docs.galtea.ai/api-reference/endpoint-connections/get-endpoint-connections https://api.galtea.ai/openapi.json get /endpointConnections Get list of endpoint connections with pagination and filtering. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Parse a cURL command into endpoint connection fields Source: https://docs.galtea.ai/api-reference/endpoint-connections/parse-a-curl-command-into-endpoint-connection-fields https://api.galtea.ai/openapi.json post /endpointConnections/parse-curl Parse a raw cURL command into best-effort endpoint-connection fields (URL, HTTP method, headers, auth type/token, request body, and a derived input template). The parse is best-effort: a command that cannot be fully parsed still returns the partial entity together with a structured `errors` array describing what could not be parsed — nothing is discarded on partial failure. The parsed fields are NOT persisted; the caller applies them to a creation form. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Test endpoint connection Source: https://docs.galtea.ai/api-reference/endpoint-connections/test-endpoint-connection https://api.galtea.ai/openapi.json post /endpointConnections/testConnection Test an endpoint connection by making a sample request. Can be used in two modes: (1) Without an ID — all connection fields are required (creation flow). (2) With an ID — fetches the stored connection and applies any provided fields as overrides (update flow). The two credential fields are the exception: a blank `authToken` or `password` keeps the stored one, and changing `authType` drops the credentials the new type does not use, exactly as saving would. With an ID, a `url` different from the stored one needs permission to update the connection, because the test sends the stored credentials to that URL. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Update endpoint connection Source: https://docs.galtea.ai/api-reference/endpoint-connections/update-endpoint-connection https://api.galtea.ai/openapi.json patch /endpointConnections/{id} Update an existing endpoint connection. See [Endpoint Connections](https://docs.galtea.ai/concepts/product/endpoint-connection). # Claim an evaluation for human review or annotation Source: https://docs.galtea.ai/api-reference/evaluations/claim-an-evaluation-for-human-review-or-annotation https://api.galtea.ai/openapi.json post /evaluations/{id}/claim Claim an evaluation to prevent concurrent editing. For HUMAN_EVALUATION metrics, enforces user group permissions (admins skip). For other metrics, works as an annotation claim open to all org members. Works on any evaluation status. Sets humanEvaluatorStartedAt timestamp. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Create evaluations for a single trace Source: https://docs.galtea.ai/api-reference/evaluations/create-evaluations-for-a-single-trace https://api.galtea.ai/openapi.json post /evaluations/fromInferenceResult Create evaluations for a specific turn in a conversation. This is useful for production monitoring where you want to evaluate turn-by-turn performance. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Create evaluations from session Source: https://docs.galtea.ai/api-reference/evaluations/create-evaluations-from-session https://api.galtea.ai/openapi.json post /evaluations/fromSession Create evaluations from a session. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Create single-turn evaluations Source: https://docs.galtea.ai/api-reference/evaluations/create-single-turn-evaluations https://api.galtea.ai/openapi.json post /evaluations/singleTurn Create evaluations for single-turn interactions. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Delete evaluation Source: https://docs.galtea.ai/api-reference/evaluations/delete-evaluation https://api.galtea.ai/openapi.json delete /evaluations/{id} Soft delete an evaluation. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Delete multiple evaluations Source: https://docs.galtea.ai/api-reference/evaluations/delete-multiple-evaluations https://api.galtea.ai/openapi.json delete /evaluations/batch Soft delete multiple evaluations by their IDs. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Edit a submitted human evaluation Source: https://docs.galtea.ai/api-reference/evaluations/edit-a-submitted-human-evaluation https://api.galtea.ai/openapi.json post /evaluations/{id}/edit-human Edit the score and reason for a previously submitted human evaluation or annotation. On an AI-judged metric, any user who may annotate the evaluation can edit it. On a HUMAN_EVALUATION metric the user must also belong to one of the metric's user groups, which only an admin skips. The edit reassigns the human evaluator to the caller. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Evaluate a version Source: https://docs.galtea.ai/api-reference/evaluations/evaluate-a-version https://api.galtea.ai/openapi.json post /evaluations/fromVersion Automatically evaluates a version by resolving its product's specifications, their linked metrics, and their linked tests. Creates inference generation jobs for every test case across all linked tests. Specifications without tests or without metrics are silently skipped. Provide either `versionId` or `productId`. When `productId` is given without a `versionId`, the product's latest properly configured version (one with a conversation-endpoint, phone, or WebRTC connection) is cloned and the evaluation runs against the clone; the product must already have such a version. Omitting `versionId` clones a version, so it requires permission to create versions (a caller lacking it gets a 403). See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Evaluate multiple sessions in one call Source: https://docs.galtea.ai/api-reference/evaluations/evaluate-multiple-sessions-in-one-call https://api.galtea.ai/openapi.json post /evaluations/fromSessions Batch-evaluate existing sessions in a single call. Pick exactly one mode: (a) `sessionIds` — explicit list of session IDs; (b) `versionId` — every session attached to that version. Each selected session is evaluated by the same logic as `POST /evaluations/fromSession`, so the traces that already live on the session are scored. Unlike `POST /evaluations/fromVersion`, this endpoint does NOT require a conversation endpoint connection on the version — use it when the traces already exist (e.g. imported ones). Metric scope: if you send `metrics`, they are applied to every session. If you do not send `metrics`, each session is scored only by the metrics of its own originating specification (the spec that produced its test case), never the union of all the product's metrics. When you send `specificationIds`, they act as a strict allow-list: a session is evaluated only if its own spec is in the list; sessions whose spec is not listed — and imported sessions that have no spec — are skipped and reported in `message`. Use `POST /evaluations/fromSession` for a single session. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Get completed human evaluations Source: https://docs.galtea.ai/api-reference/evaluations/get-completed-human-evaluations https://api.galtea.ai/openapi.json get /evaluations/human/completed Get list of human evaluations completed by the authenticated user. A READER or HUMAN_EVALUATOR sees only the products their user groups grant, and a requested product outside them is ignored. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Get evaluation by ID Source: https://docs.galtea.ai/api-reference/evaluations/get-evaluation-by-id https://api.galtea.ai/openapi.json get /evaluations/{id} Get a specific evaluation by ID. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Get evaluation count and last-evaluated date per product Source: https://docs.galtea.ai/api-reference/evaluations/get-evaluation-count-and-last-evaluated-date-per-product https://api.galtea.ai/openapi.json get /evaluations/summary For each requested product, returns the total non-deleted evaluation count and the `createdAt` of the most recent one. Computed as one grouped aggregate covering every requested product instead of a paginated list request, so a caller that only needs a count and a date does not pay for an unindexable sort over every matching evaluation. Optional `versionIds` and `statuses` narrow the aggregate. # Get evaluations Source: https://docs.galtea.ai/api-reference/evaluations/get-evaluations https://api.galtea.ai/openapi.json get /evaluations Get list of evaluations with pagination and filtering. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Get pending human evaluations Source: https://docs.galtea.ai/api-reference/evaluations/get-pending-human-evaluations https://api.galtea.ai/openapi.json get /evaluations/human/pending Get list of pending human evaluations that the authenticated user can evaluate based on their user groups. A metric with no user group linked reaches no one's queue. A READER or HUMAN_EVALUATOR sees only the products their user groups grant, and a requested product outside them is ignored. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Release a claimed evaluation or annotation Source: https://docs.galtea.ai/api-reference/evaluations/release-a-claimed-evaluation-or-annotation https://api.galtea.ai/openapi.json post /evaluations/{id}/release Release a previously claimed evaluation or annotation without submitting a score. Works on any evaluation status. Clears humanEvaluatorStartedAt. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Remove a human annotation Source: https://docs.galtea.ai/api-reference/evaluations/remove-a-human-annotation https://api.galtea.ai/openapi.json delete /evaluations/{id}/human-annotation Clear the human score, reason and evaluator from an evaluation a judge scored, leaving it unclaimed. The AI score, reason and status are untouched. Where the judge produced a score, every reported number goes back to it. Where the judge produced none, as on a FAILED, SKIPPED or CANCELLED evaluation, the evaluation stops being reported altogether, so analytics, the product score, coverage and monitor results all drop it. The removal is permanent. The human score and reason are deleted, not archived, so getting them back means annotating the evaluation again. Refused on HUMAN_EVALUATION metrics, whose human score is the only score by design, on evaluations waiting for a human score, and on outdated evaluations, which accept no annotation at all. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Replay evaluations onto a new metric revision Source: https://docs.galtea.ai/api-reference/evaluations/replay-evaluations-onto-a-new-metric-revision https://api.galtea.ai/openapi.json post /evaluations/fromMetrics Re-runs historical evaluations that used any metric in `metricGroupId` against the new metric revision `newMetricId`, scoped to the given products. Idempotent: any `(sessionId, inferenceResultId)` pair that already has an evaluation on `newMetricId` is skipped. # Replay evaluations onto a new test case revision Source: https://docs.galtea.ai/api-reference/evaluations/replay-evaluations-onto-a-new-test-case-revision https://api.galtea.ai/openapi.json post /evaluations/fromTestCases Re-runs the evaluations that any revision in `testCaseGroupId` was scored with against the new test case revision `newTestCaseId`, scoped to the given products. Because an evaluation is bound to a test case only through its session, fresh inference is generated for the new revision on each version that previously ran the family (production sessions excluded), then evaluated with the metrics that ran there. De-duplication skips a `(version, metric)` pair once it has a non-deleted evaluation against `newTestCaseId`, and skips a whole version once any non-deleted session exists for `newTestCaseId` — the session is created before this call returns, so a repeated or retried call re-dispatches nothing. This is still not a transactional guarantee for truly simultaneous calls. `newTestCaseId` must be the active head of its family. # Retry failed evaluations Source: https://docs.galtea.ai/api-reference/evaluations/retry-failed-evaluations https://api.galtea.ai/openapi.json post /evaluations/retry Retry one or more failed evaluations. Requires permission to update them. Non-admin users are limited by EVALUATION_MAX_RETRIES. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Submit a human evaluation or annotation score Source: https://docs.galtea.ai/api-reference/evaluations/submit-a-human-evaluation-or-annotation-score https://api.galtea.ai/openapi.json post /evaluations/{id}/submit Submit a score and optional reason. For HUMAN_EVALUATION metrics, enforces user group permissions and updates both score sets. For PENDING_HUMAN evaluations, also sets status to SUCCESS. For other metrics, works as an annotation — writes only humanScore/humanReason/humanEvaluatorFinishedAt, leaving AI score/reason and status untouched. Works on any evaluation status. See [Evaluations](https://docs.galtea.ai/concepts/product/version/session/evaluation). # Check judge generation settings against the model Source: https://docs.galtea.ai/api-reference/evaluator-models/check-judge-generation-settings-against-the-model https://api.galtea.ai/openapi.json post /evaluatorModels/test-generation-settings Sends one small completion to the given judge model with the given generation settings, so a rejected value surfaces while a metric is being configured rather than after an evaluation has run. The gateway publishes which settings a model takes but never their legal values, so the provider is the only judge of them. Persists nothing and consumes no credits. # Get evaluator models Source: https://docs.galtea.ai/api-reference/evaluator-models/get-evaluator-models https://api.galtea.ai/openapi.json get /evaluatorModels Get list of evaluator models with filtering options. See [Models](https://docs.galtea.ai/concepts/model). # Get minimum SDK version Source: https://docs.galtea.ai/api-reference/get-minimum-sdk-version https://api.galtea.ai/openapi.json get /minimum-sdk-version Get minimum supported SDK version of the API and latest published version. See [Platform Overview](https://docs.galtea.ai/introduction). # Health check Source: https://docs.galtea.ai/api-reference/health-check https://api.galtea.ai/openapi.json get /health Health check endpoint to verify API is running. See [Platform Overview](https://docs.galtea.ai/introduction). # Cancel a job Source: https://docs.galtea.ai/api-reference/jobs/cancel-a-job https://api.galtea.ai/openapi.json post /jobs/{jobId}/cancel Cancel a queued or in-flight async job (e.g., inference generation). Use this to stop a doomed batch instead of waiting for every unit to fail individually. Behaviour: - A queued or delayed job is removed from the queue and will never run. - An in-flight job receives a cooperative-cancellation signal: the worker observes it at its next checkpoint and stops dispatching further work (for inference generation, this means no further sessions are created). Work already in progress when the signal is observed is NOT rolled back — sessions that have already been created remain. Subsequent reads of `GET /jobs/{jobId}/status` will surface the distinct `cancelled` state. - The in-flight guarantee applies only to handlers that implement the cooperative-cancellation checkpoint. Today that is inference generation; other job types (e.g. trace imports) do not yet observe the cancel signal and will run to completion even though the cancel call returns 200 and the status read flips to `cancelled`. Trace imports are short, so this is a deliberate Phase-1 trade-off rather than a correctness issue. Tenant isolation: only users in the dispatching user's organization may cancel a job. Cross-tenant cancels return `404` (not `403`) to avoid leaking the existence of jobs in other organizations — matching `GET /jobs/{jobId}/status`. # Get job status Source: https://docs.galtea.ai/api-reference/jobs/get-job-status https://api.galtea.ai/openapi.json get /jobs/{jobId}/status Retrieve the status of an async job (e.g., inference generation). Returns the job state, progress, result, and any error information. Tenant isolation: a job is only visible to users in the same organization as the user who dispatched it. Cross-tenant reads receive `404` (rather than `403`) to avoid leaking the existence of jobs in other organizations. System jobs (those without an owning user) are never exposed through this endpoint. # Activate an optimized judge prompt Source: https://docs.galtea.ai/api-reference/metrics/activate-an-optimized-judge-prompt https://api.galtea.ai/openapi.json post /metrics/{id}/activate-optimization Makes this optimization candidate the active revision: it takes the source metric's name, description and tags, the source's specifications and user groups are linked to it too, and the source becomes legacy — all in one step. Repeating it on a candidate already activated returns 200 with the metric and changes nothing. Rejected with 400 if the candidate was declined or is otherwise not waiting for review, if the source changed since the candidate was created (revised, deleted, or already flipped legacy), or if the source's judge model is no longer selectable. # Count replay candidates per product for a metric revision Source: https://docs.galtea.ai/api-reference/metrics/count-replay-candidates-per-product-for-a-metric-revision https://api.galtea.ai/openapi.json get /metrics/{id}/replay-candidate-count For each requested product, returns how many historical evaluations in this metric's group would be replayed against this revision. Any `(sessionId, inferenceResultId)` pair already evaluated with this metric is excluded. Products with no candidates return `count: 0`. # Create metric Source: https://docs.galtea.ai/api-reference/metrics/create-metric https://api.galtea.ai/openapi.json post /metrics Create a new metric. See [Metrics](https://docs.galtea.ai/concepts/metric). Accepted values for `source` on new metrics are `SELF_HOSTED`, `PARTIAL_PROMPT`, and `HUMAN_EVALUATION`. `FULL_PROMPT` is **deprecated for creation** — requests with `source=FULL_PROMPT` (or that omit `source` and supply a `judgePrompt` without `evaluationParams`, which previously inferred FULL_PROMPT) are rejected with 400. Use `PARTIAL_PROMPT` instead — it offers the same capabilities with user-configurable evaluation parameters. Existing FULL_PROMPT metrics continue to be listed, read, filtered, and evaluated. # Create multiple metrics Source: https://docs.galtea.ai/api-reference/metrics/create-multiple-metrics https://api.galtea.ai/openapi.json post /metrics/batch Create multiple metrics at once. See [Metrics](https://docs.galtea.ai/concepts/metric). Each item is validated through the same path as `POST /metrics`. In particular, `source=FULL_PROMPT` (explicit or inferred from a `judgePrompt` without `evaluationParams`) is rejected with 400 — use `PARTIAL_PROMPT` instead. # Delete metric Source: https://docs.galtea.ai/api-reference/metrics/delete-metric https://api.galtea.ai/openapi.json delete /metrics/{id} Soft delete a metric. See [Metrics](https://docs.galtea.ai/concepts/metric). # Delete multiple metrics Source: https://docs.galtea.ai/api-reference/metrics/delete-multiple-metrics https://api.galtea.ai/openapi.json delete /metrics/batch Soft delete multiple metrics by their IDs. See [Metrics](https://docs.galtea.ai/concepts/metric). # Export a metric revision as a document Source: https://docs.galtea.ai/api-reference/metrics/export-a-metric-revision-as-a-document https://api.galtea.ai/openapi.json get /metrics/{id}/export Everything needed to re-run this revision's judge outside Galtea: the assembled system and user prompts with placeholders, the output schema, the model and the generation settings. A revision is immutable, so the document describes the revision asked for, never the head of its family. A metric whose judge prompt Galtea does not build cannot be exported. # Generate or complete metric configuration with AI Source: https://docs.galtea.ai/api-reference/metrics/generate-or-complete-metric-configuration-with-ai https://api.galtea.ai/openapi.json post /metrics/generate-config Two modes, selected by the request body: - **Spec mode** (provide `specificationIds`): generates 1-3 metric configurations per specification, optionally creating the metrics automatically. - **Completion mode** ("Complete With AI", provide `name` + `description` + `evaluationType`): completes a single metric the user is authoring. The name/description/evaluationType/tags are the source of truth; only the judge prompt and evaluation params are generated. `specificationIds` may carry at most one id as optional context. The evaluator model of every generated AI-evaluated metric is the deployment's setting for generated metrics, not a choice of the generator. It answers 500 when that model is not selectable in the environment. See [Metrics](https://docs.galtea.ai/concepts/metric). # Get human annotation count and optimization eligibility Source: https://docs.galtea.ai/api-reference/metrics/get-human-annotation-count-and-optimization-eligibility https://api.galtea.ai/openapi.json get /metrics/{id}/human-annotation-count Returns the optimization-eligibility snapshot for a metric. `count` is the number of evaluations that have both an AI score and a human-provided score (annotation). For FULL_PROMPT/PARTIAL_PROMPT metrics this means the AI judged the evaluation and a human later reviewed it. For HUMAN_EVALUATION metrics both fields hold the same human-provided value (for analytics compatibility). `disagreementCount` is the subset of `count` where the human verdict flips the AI's verdict. Both are binarized at 0.5 (any score >= 0.5 is class 1, anything below is class 0) and the row counts only when the two classes differ. These are the rows the optimizer learns from. `minAnnotations` and `minDisagreements` are the api-enforced thresholds (mirrored from the metrics-generator hard thresholds). Optimize is rejected early unless `count >= minAnnotations` AND `disagreementCount >= minDisagreements`. `plannedSplit` is the split the next optimize would make, from the same count gates and split the run applies: the training and validation row counts, or the refusal text when optimize would answer a 400 (too few annotations, too few disagreements, or too few left to train on after the evaluations earlier attempts kept for validation). # Get metric by ID Source: https://docs.galtea.ai/api-reference/metrics/get-metric-by-id https://api.galtea.ai/openapi.json get /metrics/{id} Get a specific metric by ID. See [Metrics](https://docs.galtea.ai/concepts/metric). # Get metrics Source: https://docs.galtea.ai/api-reference/metrics/get-metrics https://api.galtea.ai/openapi.json get /metrics Get list of metrics with pagination and filtering. See [Metrics](https://docs.galtea.ai/concepts/metric). # Get the analytics impact of a metric revision Source: https://docs.galtea.ai/api-reference/metrics/get-the-analytics-impact-of-a-metric-revision https://api.galtea.ai/openapi.json get /metrics/{id}/analytics-exclusion-impact Count the results and products affected if this metric revision is excluded from analytics. # Include or exclude a metric revision from analytics Source: https://docs.galtea.ai/api-reference/metrics/include-or-exclude-a-metric-revision-from-analytics https://api.galtea.ai/openapi.json patch /metrics/{id}/analytics-exclusion Excluding a revision removes every result produced by it from analytics scores and coverage. It does not delete or change any result. # Keep the current judge prompt over an optimized candidate Source: https://docs.galtea.ai/api-reference/metrics/keep-the-current-judge-prompt-over-an-optimized-candidate https://api.galtea.ai/openapi.json post /metrics/{id}/decline-optimization Leaves the source metric unchanged and keeps this optimization candidate in revision history as not activated. Another optimization can then be started from the source. Repeating it on a candidate already declined returns 200 with the metric and changes nothing. Rejected with 400 if the candidate was activated or is otherwise not waiting for review. # Optimize metric Source: https://docs.galtea.ai/api-reference/metrics/optimize-metric https://api.galtea.ai/openapi.json post /metrics/{id}/optimize Starts an optimization run that sends the metric's human-annotated evaluations to the metrics-generator service, which uses the gap between AI and human scores to produce a candidate judge prompt. Only PARTIAL_PROMPT metrics owned by an organization can be optimized, and the source must be its family's active revision with no candidate already awaiting review. Rejected with 400 unless the metric has at least `minAnnotations` evaluations annotated by a human AND `minDisagreements` of those disagree with the AI score; both thresholds are returned by `GET /metrics/{id}/human-annotation-count` (defaults `10` and `5`, configurable via the `OPTIMIZE_MIN_ANNOTATIONS_HARD` / `OPTIMIZE_MIN_DISAGREEMENTS_HARD` env vars). From 30 annotated evaluations the run holds out a validation set (one third of them, at least 10, chosen by session) that the optimizer only scores after its search and never searches on. Sessions kept from earlier attempts of the metric stay held out. The split, and the validation results once the run finishes, are on `optimizationValidation` of the metric. Returns a placeholder metric with isBeingOptimized=true; once the run finishes, the source is untouched and the result is a candidate to review with `POST /metrics/{id}/activate-optimization` or `POST /metrics/{id}/decline-optimization`. See [Metrics](https://docs.galtea.ai/concepts/metric). # Preview the prompt a judge configuration produces Source: https://docs.galtea.ai/api-reference/metrics/preview-the-prompt-a-judge-configuration-produces https://api.galtea.ai/openapi.json post /metrics/preview-prompt Assembles the prompt for a configuration that has not been saved, with `{placeholders}` in place of a test case. Reads no model and calls none, so it consumes no credits. A configuration usually produces two shapes, because turn count comes from the session. # Revise metric configuration with AI (Update With AI) Source: https://docs.galtea.ai/api-reference/metrics/revise-metric-configuration-with-ai-update-with-ai https://api.galtea.ai/openapi.json post /metrics/update-config Given a metric config the caller already has (name, description, judge prompt, evaluation params, evaluation type) plus one line of feedback, returns a revised judge prompt, evaluation parameters, and description. Persists nothing — the caller decides whether to apply the result. # Update metric Source: https://docs.galtea.ai/api-reference/metrics/update-metric https://api.galtea.ai/openapi.json patch /metrics/{id} Update an existing metric. See [Metrics](https://docs.galtea.ai/concepts/metric). To attach an existing metric to a specification (or detach it), use `specifications link-metrics` / `specifications unlink-metrics` — the metric-specification link is managed from the specification side, not by updating the metric. # Create model Source: https://docs.galtea.ai/api-reference/models/create-model https://api.galtea.ai/openapi.json post /models Create a new model. See [Models](https://docs.galtea.ai/concepts/model). # Delete model Source: https://docs.galtea.ai/api-reference/models/delete-model https://api.galtea.ai/openapi.json delete /models/{id} Soft delete a model. See [Models](https://docs.galtea.ai/concepts/model). # Delete multiple models Source: https://docs.galtea.ai/api-reference/models/delete-multiple-models https://api.galtea.ai/openapi.json delete /models/batch Soft delete multiple models by their IDs. See [Models](https://docs.galtea.ai/concepts/model). # Get model by ID Source: https://docs.galtea.ai/api-reference/models/get-model-by-id https://api.galtea.ai/openapi.json get /models/{id} Get a specific model by ID. See [Models](https://docs.galtea.ai/concepts/model). # Get models Source: https://docs.galtea.ai/api-reference/models/get-models https://api.galtea.ai/openapi.json get /models Get list of models with pagination and filtering. See [Models](https://docs.galtea.ai/concepts/model). # Update model Source: https://docs.galtea.ai/api-reference/models/update-model https://api.galtea.ai/openapi.json patch /models/{id} Update an existing model. See [Models](https://docs.galtea.ai/concepts/model). # Create a monitor alert rule Source: https://docs.galtea.ai/api-reference/monitors/create-a-monitor-alert-rule https://api.galtea.ai/openapi.json post /monitors/{monitorId}/alert-rules Adds a rule that opens an alert on this monitor when its scored sessions cross a threshold over a trailing window. Only BAD_SESSION_COUNT is accepted. # Create monitor Source: https://docs.galtea.ai/api-reference/monitors/create-monitor https://api.galtea.ai/openapi.json post /monitors Create a new monitor. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # Delete a monitor alert rule Source: https://docs.galtea.ai/api-reference/monitors/delete-a-monitor-alert-rule https://api.galtea.ai/openapi.json delete /monitors/{monitorId}/alert-rules/{alertRuleId} Soft-deletes one alert rule. A second delete of the same id returns 404. # Delete monitor Source: https://docs.galtea.ai/api-reference/monitors/delete-monitor https://api.galtea.ai/openapi.json delete /monitors/{id} Soft delete a monitor. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # Delete multiple monitors Source: https://docs.galtea.ai/api-reference/monitors/delete-multiple-monitors https://api.galtea.ai/openapi.json delete /monitors/batch Soft delete multiple monitors by their IDs. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # Get a monitor alert Source: https://docs.galtea.ai/api-reference/monitors/get-a-monitor-alert https://api.galtea.ai/openapi.json get /monitors/{monitorId}/alerts/{alertId} Returns one alert with what the check saw when it opened. 404 for an alert of another monitor. # Get a monitor alert rule Source: https://docs.galtea.ai/api-reference/monitors/get-a-monitor-alert-rule https://api.galtea.ai/openapi.json get /monitors/{monitorId}/alert-rules/{alertRuleId} Returns one alert rule. 404 for a rule id belonging to another monitor or already deleted. # Get monitor by ID Source: https://docs.galtea.ai/api-reference/monitors/get-monitor-by-id https://api.galtea.ai/openapi.json get /monitors/{id} Get a specific monitor by ID. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # Get monitor results Source: https://docs.galtea.ai/api-reference/monitors/get-monitor-results https://api.galtea.ai/openapi.json get /monitors/{id}/results Aggregated results for a monitor over a trailing window (default 7 days, bucketed by day): per-metric-family sampling-weighted score series with 95% confidence intervals, the worst metric family, metric-revision and sampling-change markers, current-cycle credit spend, and the scored/waiting session listing. Below 30 scored sessions a bucket reports a collecting-data state instead of an interval. Computed only from the monitor's own evaluations. # Get monitors Source: https://docs.galtea.ai/api-reference/monitors/get-monitors https://api.galtea.ai/openapi.json get /monitors Get list of monitors with pagination and filtering. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # List a monitor's alert rules Source: https://docs.galtea.ai/api-reference/monitors/list-a-monitors-alert-rules https://api.galtea.ai/openapi.json get /monitors/{monitorId}/alert-rules Returns every live alert rule on this monitor, oldest first. Not paginated. # List a monitor's alerts Source: https://docs.galtea.ai/api-reference/monitors/list-a-monitors-alerts https://api.galtea.ai/openapi.json get /monitors/{monitorId}/alerts Returns the alerts the rules of this monitor opened, open alerts first, then the newest opened. Alerts of a deleted rule are included. # Pause a monitor alert rule Source: https://docs.galtea.ai/api-reference/monitors/pause-a-monitor-alert-rule https://api.galtea.ai/openapi.json post /monitors/{monitorId}/alert-rules/{alertRuleId}/pause Stops checking the rule until it is resumed. An open alert closes as RULE_PAUSED and sends no email. Pausing a paused rule returns it unchanged. # Pause monitor Source: https://docs.galtea.ai/api-reference/monitors/pause-monitor https://api.galtea.ai/openapi.json post /monitors/{id}/pause Stops the monitor from evaluating new sessions until it is resumed. Allowed from any status. Pausing a paused monitor returns it unchanged. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # Preview monitor content-filter matches Source: https://docs.galtea.ai/api-reference/monitors/preview-monitor-content-filter-matches https://api.galtea.ai/openapi.json post /monitors/preview Read-only preview for a CANDIDATE monitor config: how many and which recent past sessions the given content filter and sampling percentage would select, over a trailing window. Runs no evaluations, spends no credits, and writes nothing (no processed marks). Use it while creating or editing a monitor. The match count is exact; for a create preview (no `monitorId`) the sampled subset is representative, not the exact set the created monitor will draw. # Resume a monitor alert rule Source: https://docs.galtea.ai/api-reference/monitors/resume-a-monitor-alert-rule https://api.galtea.ai/openapi.json post /monitors/{monitorId}/alert-rules/{alertRuleId}/resume Checks the rule again from the next check, as if it were new. Resuming an active rule returns it unchanged. # Resume monitor Source: https://docs.galtea.ai/api-reference/monitors/resume-monitor https://api.galtea.ai/openapi.json post /monitors/{id}/resume Sets the monitor back to ACTIVE. Allowed from any status; the next scan sets CAPPED or NO_CREDITS again if it still applies. Resuming an active monitor returns it unchanged. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # Update a monitor alert rule Source: https://docs.galtea.ai/api-reference/monitors/update-a-monitor-alert-rule https://api.galtea.ai/openapi.json patch /monitors/{monitorId}/alert-rules/{alertRuleId} Patches one or more fields of an alert rule. The merged rule (patch over the stored row) is what gets validated. The type cannot change: delete the rule and create a new one. # Update monitor Source: https://docs.galtea.ai/api-reference/monitors/update-monitor https://api.galtea.ai/openapi.json patch /monitors/{id} Update an existing monitor. Setting status to PAUSED or ACTIVE here still works; pause or resume with POST /monitors/{id}/pause and /monitors/{id}/resume instead. See [Monitors](https://docs.galtea.ai/concepts/product/monitor). # Get credit status Source: https://docs.galtea.ai/api-reference/organizations/get-credit-status https://api.galtea.ai/openapi.json get /organizations/{id}/creditStatus Returns the current credit status for the organization, used for the credit warning badge. See [Platform Overview](https://docs.galtea.ai/introduction). # Get organization by ID Source: https://docs.galtea.ai/api-reference/organizations/get-organization-by-id https://api.galtea.ai/openapi.json get /organizations/{id} Get a specific organization by ID. See [Platform Overview](https://docs.galtea.ai/introduction). # Get organizations Source: https://docs.galtea.ai/api-reference/organizations/get-organizations https://api.galtea.ai/openapi.json get /organizations Get list of organizations with pagination and filtering. See [Platform Overview](https://docs.galtea.ai/introduction). # Ingest OpenTelemetry spans (OTLP JSON) Source: https://docs.galtea.ai/api-reference/otel/ingest-opentelemetry-spans-otlp-json https://api.galtea.ai/openapi.json post /otel/traces Receives an OTLP/JSON trace payload and persists its spans as Galtea Span records. Authenticated via standard API key (processAccessTokenMiddleware); the authenticated user is the tenant and RBAC applies. Any payload customer.id resource attribute is ignored on this public path. # Validate credentials for OTel ingestion Source: https://docs.galtea.ai/api-reference/otel/validate-credentials-for-otel-ingestion https://api.galtea.ai/openapi.json post /otel/auth Validates the customer's API key and returns the user ID via x-customer-id header. Used by the Istio gateway ext-authz filter to authenticate OTel data ingestion requests. # Create phone connection Source: https://docs.galtea.ai/api-reference/phone-connections/create-phone-connection https://api.galtea.ai/openapi.json post /phoneConnections Create a new phone connection. See [Phone Connections](https://docs.galtea.ai/concepts/product/phone-connection). # Delete multiple phone connections Source: https://docs.galtea.ai/api-reference/phone-connections/delete-multiple-phone-connections https://api.galtea.ai/openapi.json delete /phoneConnections/batch Soft delete multiple phone connections by their IDs. See [Phone Connections](https://docs.galtea.ai/concepts/product/phone-connection). # Delete phone connection Source: https://docs.galtea.ai/api-reference/phone-connections/delete-phone-connection https://api.galtea.ai/openapi.json delete /phoneConnections/{id} Soft delete a phone connection. See [Phone Connections](https://docs.galtea.ai/concepts/product/phone-connection). # Get phone connection by ID Source: https://docs.galtea.ai/api-reference/phone-connections/get-phone-connection-by-id https://api.galtea.ai/openapi.json get /phoneConnections/{id} Get a specific phone connection by ID. See [Phone Connections](https://docs.galtea.ai/concepts/product/phone-connection). # Get phone connections Source: https://docs.galtea.ai/api-reference/phone-connections/get-phone-connections https://api.galtea.ai/openapi.json get /phoneConnections Get list of phone connections with pagination and filtering. See [Phone Connections](https://docs.galtea.ai/concepts/product/phone-connection). # Update phone connection Source: https://docs.galtea.ai/api-reference/phone-connections/update-phone-connection https://api.galtea.ai/openapi.json patch /phoneConnections/{id} Update an existing phone connection. See [Phone Connections](https://docs.galtea.ai/concepts/product/phone-connection). # Create product Source: https://docs.galtea.ai/api-reference/products/create-product https://api.galtea.ai/openapi.json post /products Create a new product. See [Products](https://docs.galtea.ai/concepts/product). # Cross-check a document against an existing product's specifications Source: https://docs.galtea.ai/api-reference/products/cross-check-a-document-against-an-existing-products-specifications https://api.galtea.ai/openapi.json post /products/{id}/regenerate-config Uploads one or more documents (PRD, spec, design doc, etc.), generates candidate specifications from them using the product's existing description as the generation base, and cross-checks each candidate against the product's CURRENT specifications. Returns classified suggestions — additions (`new`), updates to an existing specification (`updated`), and contradictions (`contradictory`). This endpoint persists NOTHING: applying a suggestion is a separate, explicit step via the specification endpoints. # Delete multiple products Source: https://docs.galtea.ai/api-reference/products/delete-multiple-products https://api.galtea.ai/openapi.json delete /products/batch Soft delete multiple products by their IDs. See [Products](https://docs.galtea.ai/concepts/product). # Delete product Source: https://docs.galtea.ai/api-reference/products/delete-product https://api.galtea.ai/openapi.json delete /products/{id} Soft delete a product. See [Products](https://docs.galtea.ai/concepts/product). # Generate product configuration using AI Source: https://docs.galtea.ai/api-reference/products/generate-product-configuration-using-ai https://api.galtea.ai/openapi.json post /products/generate-config Uses AI to generate a refined product description and testable specifications from a product name and user description. Optionally accepts product specification files (documents, images, audio, video, code) to enrich the generation. See [Products](https://docs.galtea.ai/concepts/product). # Get product by ID Source: https://docs.galtea.ai/api-reference/products/get-product-by-id https://api.galtea.ai/openapi.json get /products/{id} Get a specific product by ID. See [Products](https://docs.galtea.ai/concepts/product). # Get products Source: https://docs.galtea.ai/api-reference/products/get-products https://api.galtea.ai/openapi.json get /products Get list of products with pagination and filtering options. See [Products](https://docs.galtea.ai/concepts/product). # List distinct product tags Source: https://docs.galtea.ai/api-reference/products/list-distinct-product-tags https://api.galtea.ai/openapi.json get /products/tags Returns the sorted, distinct set of tags used across products in the requested organizations. Tags are returned lowercase and trimmed. Powers tag-suggestion UIs in the dashboard and the product-list tag filter. Org scope resolves like every list endpoint (ADR 0029): with no `organizationIds`, the caller's own organization is used by default (admins reach all orgs via `allOrganizations`). A non-admin requesting tags for an organization they cannot read receives 403. A READER or HUMAN_EVALUATOR gets only the tags of the products their user groups grant. # Update product Source: https://docs.galtea.ai/api-reference/products/update-product https://api.galtea.ai/openapi.json patch /products/{id} Update an existing product. See [Products](https://docs.galtea.ai/concepts/product). # Cancel run Source: https://docs.galtea.ai/api-reference/runs/cancel-run https://api.galtea.ai/openapi.json post /runs/{id}/cancel Stops a still-running run and the work behind it. Only a run the platform queued a job for can be cancelled; every other kind finishes inside the request that started it, so there is nothing left to stop. Sessions and evaluations already created are kept. # Close run Source: https://docs.galtea.ai/api-reference/runs/close-run https://api.galtea.ai/openapi.json post /runs/{id}/close Ends a run and stamps its finish time as COMPLETED. Use it on a run you opened yourself; a run a launch endpoint opened is closed by that launch. A run that already finished is refused. # Create run Source: https://docs.galtea.ai/api-reference/runs/create-run https://api.galtea.ai/openapi.json post /runs Opens an empty run you can attribute your own evaluations to, by passing its id as `runId`. A run you create this way is never closed automatically: close it yourself when the last evaluation is in. # Delete run Source: https://docs.galtea.ai/api-reference/runs/delete-run https://api.galtea.ai/openapi.json delete /runs/{id} Soft-deletes a run and the evaluations it launched. The sessions those evaluations scored are left alone. # Get run by ID Source: https://docs.galtea.ai/api-reference/runs/get-run-by-id https://api.galtea.ai/openapi.json get /runs/{id} Get a single evaluation launch by its ID. # Get run metric summary Source: https://docs.galtea.ai/api-reference/runs/get-run-metric-summary https://api.galtea.ai/openapi.json get /runs/{id}/metric-summary How each metric scored across the run: the mean and the observed range over the evaluations that carry a score. A metric whose evaluations are all still pending has no row. # Get runs Source: https://docs.galtea.ai/api-reference/runs/get-runs https://api.galtea.ai/openapi.json get /runs Get list of runs with pagination and filtering. See [Runs](https://docs.galtea.ai/concepts/product/run). # Run a run again Source: https://docs.galtea.ai/api-reference/runs/run-a-run-again https://api.galtea.ai/openapi.json post /runs/{id}/relaunch Repeats a run and answers with the new one, which names the run it came from. The kind of the source decides the form: a run the platform launched drives the agent again and scores the new sessions, while a run that scored sessions which already existed scores them again and creates none. A run the client performed carries no kind to read, so its own evaluations name the sessions and metrics to score again; no new session is created, because only that client can produce one. A repetition that scores sessions answers with a "report" of what it scored, skipped and failed; one that drives the agent again reports null there, because that work is queued. # Create session Source: https://docs.galtea.ai/api-reference/sessions/create-session https://api.galtea.ai/openapi.json post /sessions Create a new session. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Delete multiple sessions Source: https://docs.galtea.ai/api-reference/sessions/delete-multiple-sessions https://api.galtea.ai/openapi.json delete /sessions/batch Soft delete multiple sessions by their IDs. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Delete session Source: https://docs.galtea.ai/api-reference/sessions/delete-session https://api.galtea.ai/openapi.json delete /sessions/{id} Soft delete a session. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Finish session Source: https://docs.galtea.ai/api-reference/sessions/finish-session https://api.galtea.ai/openapi.json patch /sessions/{id}/finish Marks a session as COMPLETED (stoppingReason provided) or FAILED (error provided). At least one of stoppingReason or error must be provided. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Generate next user message Source: https://docs.galtea.ai/api-reference/sessions/generate-next-user-message https://api.galtea.ai/openapi.json post /conversationSimulator/generateNextUserMessage Simulate the next user message in a conversation based on the provided scenario and turns. The generated turn is stored pending, and the caller that answers it owns its terminal status: send the agent output with `PATCH /inferenceResults/{id}`, or set the status there when no agent will answer. Evaluations on a session holding a pending turn are skipped, and can be retried once the turn is answered. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Get session by ID Source: https://docs.galtea.ai/api-reference/sessions/get-session-by-id https://api.galtea.ai/openapi.json get /sessions/{id} Get a specific session by ID. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Get sessions Source: https://docs.galtea.ai/api-reference/sessions/get-sessions https://api.galtea.ai/openapi.json get /sessions Get list of sessions with pagination and filtering. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Import sessions from a Langfuse trace export Source: https://docs.galtea.ai/api-reference/sessions/import-sessions-from-a-langfuse-trace-export https://api.galtea.ai/openapi.json post /sessions/import Imports sessions from a Langfuse export, production by default. The content is handed to a background BullMQ job; the response is the `jobId` to poll on `GET /jobs/{jobId}/status`. Semantic validation (Langfuse shape, drop rules) happens in the job, so a syntactically valid upload is accepted with `202` and rejections surface as `failed` job state. Send EITHER `multipart/form-data` with a `file` OR `application/json` with a `text` string — exactly one. `isProduction` defaults to `true`. `versionId` is optional: when omitted, provide `productId` and the API reuses the product's latest version (creating a default first version if the product has none). One of `versionId` or `productId` is required. # List metrics annotated with compatibility for a session Source: https://docs.galtea.ai/api-reference/sessions/list-metrics-annotated-with-compatibility-for-a-session https://api.galtea.ai/openapi.json get /sessions/{id}/compatible-metrics Returns evaluable metrics annotated with whether they can be satisfied by the session's product metadata and test-case availability, plus a `relevance` ranking derived from the session's test case's test (`linked`/`matching`/`none`). A session evaluation scores output that is already stored, so the version needs no conversation target (endpoint, phone, or WebRTC connection) and compatibility never depends on one. Production (monitoring) sessions have no test case, so every metric is `none`, and a metric is incompatible when it needs a test-case param that monitoring cannot supply (expected_output, context, expected_tools, user_persona, goal, scenario, stopping_criterias). A metric needing only `input` stays compatible while some turn of the session yields text of its own; a session whose turns carry only a file, or audio with no transcript, makes `input` unreadable and that metric incompatible. Span-dependent and runtime-only params are out of scope and treated as always satisfied. # Update session Source: https://docs.galtea.ai/api-reference/sessions/update-session https://api.galtea.ai/openapi.json patch /sessions/{id} Update an existing session. See [Sessions](https://docs.galtea.ai/concepts/product/version/session). # Create span Source: https://docs.galtea.ai/api-reference/spans/create-span https://api.galtea.ai/openapi.json post /traces Create a new span for a tool/function call. See [Spans](https://docs.galtea.ai/sdk/api/span/service). These endpoints serve spans, the steps inside a trace. Their REST path `/traces` keeps the old name of this entity. # Create spans batch Source: https://docs.galtea.ai/api-reference/spans/create-spans-batch https://api.galtea.ai/openapi.json post /traces/batch Create multiple spans in a single API call. Partial success, not all-or-nothing: a span refused for a data problem (empty name, negative latency, oversize payload, unknown trace) does not stop its siblings, and its trace records how many of its spans were dropped. Set `includeRejected=true` to read which spans were refused. A span you are not allowed to write is the one exception and refuses the whole batch with a 403. See [Spans](https://docs.galtea.ai/sdk/api/span/service). These endpoints serve spans, the steps inside a trace. Their REST path `/traces` keeps the old name of this entity. # Delete multiple spans Source: https://docs.galtea.ai/api-reference/spans/delete-multiple-spans https://api.galtea.ai/openapi.json delete /traces/batch Soft delete multiple spans by their IDs. See [Spans](https://docs.galtea.ai/sdk/api/span/service). These endpoints serve spans, the steps inside a trace. Their REST path `/traces` keeps the old name of this entity. # Delete span Source: https://docs.galtea.ai/api-reference/spans/delete-span https://api.galtea.ai/openapi.json delete /traces/{id} Soft delete a span by its ID. See [Spans](https://docs.galtea.ai/sdk/api/span/service). These endpoints serve spans, the steps inside a trace. Their REST path `/traces` keeps the old name of this entity. # Get span by ID Source: https://docs.galtea.ai/api-reference/spans/get-span-by-id https://api.galtea.ai/openapi.json get /traces/{id} Get a specific span by its ID. See [Spans](https://docs.galtea.ai/sdk/api/span/service). These endpoints serve spans, the steps inside a trace. Their REST path `/traces` keeps the old name of this entity. # Get spans Source: https://docs.galtea.ai/api-reference/spans/get-spans https://api.galtea.ai/openapi.json get /traces Get a list of spans with filtering and pagination. See [Spans](https://docs.galtea.ai/sdk/api/span/service). These endpoints serve spans, the steps inside a trace. Their REST path `/traces` keeps the old name of this entity. # Create multiple specifications Source: https://docs.galtea.ai/api-reference/specifications/create-multiple-specifications https://api.galtea.ai/openapi.json post /specifications/batch Create multiple specifications at once. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Create specification Source: https://docs.galtea.ai/api-reference/specifications/create-specification https://api.galtea.ai/openapi.json post /specifications Create a new specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Delete multiple specifications Source: https://docs.galtea.ai/api-reference/specifications/delete-multiple-specifications https://api.galtea.ai/openapi.json delete /specifications/batch Soft delete multiple specifications by their IDs. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Delete specification Source: https://docs.galtea.ai/api-reference/specifications/delete-specification https://api.galtea.ai/openapi.json delete /specifications/{id} Soft delete a specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Generate specification configuration Source: https://docs.galtea.ai/api-reference/specifications/generate-specification-configuration https://api.galtea.ai/openapi.json post /specifications/generate-config Uses AI to rewrite a rough or brief note into a properly written specification description and suggest its classification (type, test type, test variant) in a single request. Nothing is persisted — the caller decides which fields to apply and creates the specification explicitly. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Get linked datasets Source: https://docs.galtea.ai/api-reference/specifications/get-linked-datasets https://api.galtea.ai/openapi.json get /specifications/{id}/tests Get all datasets linked to a specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Get linked metrics Source: https://docs.galtea.ai/api-reference/specifications/get-linked-metrics https://api.galtea.ai/openapi.json get /specifications/{id}/metrics Get all metrics linked to a specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Get specification by ID Source: https://docs.galtea.ai/api-reference/specifications/get-specification-by-id https://api.galtea.ai/openapi.json get /specifications/{id} Get a specific specification by ID. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Get specifications Source: https://docs.galtea.ai/api-reference/specifications/get-specifications https://api.galtea.ai/openapi.json get /specifications Get list of specifications with pagination and filtering. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Link datasets to specification Source: https://docs.galtea.ai/api-reference/specifications/link-datasets-to-specification https://api.galtea.ai/openapi.json post /specifications/{id}/tests Link one or more datasets to a specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Link metrics to specification Source: https://docs.galtea.ai/api-reference/specifications/link-metrics-to-specification https://api.galtea.ai/openapi.json post /specifications/{id}/metrics Link one or more metrics to a specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Unlink datasets from specification Source: https://docs.galtea.ai/api-reference/specifications/unlink-datasets-from-specification https://api.galtea.ai/openapi.json delete /specifications/{id}/tests Unlink one or more datasets from a specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Unlink metrics from specification Source: https://docs.galtea.ai/api-reference/specifications/unlink-metrics-from-specification https://api.galtea.ai/openapi.json delete /specifications/{id}/metrics Unlink one or more metrics from a specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). # Update specification Source: https://docs.galtea.ai/api-reference/specifications/update-specification https://api.galtea.ai/openapi.json patch /specifications/{id} Update an existing specification. See [Specifications](https://docs.galtea.ai/concepts/product/specification). Linked datasets and metrics are **not** edited through this endpoint. To manage links use `specifications link-datasets` / `specifications unlink-datasets` and `specifications link-metrics` / `specifications unlink-metrics` (with `specifications list-datasets` / `specifications list-metrics` to inspect what is currently linked). # Generate download presigned URL Source: https://docs.galtea.ai/api-reference/storage/generate-download-presigned-url https://api.galtea.ai/openapi.json get /storage/generate-get-presigned-url Generate a presigned URL for downloading files from storage. See [Platform Overview](https://docs.galtea.ai/introduction). # Generate upload presigned URL Source: https://docs.galtea.ai/api-reference/storage/generate-upload-presigned-url https://api.galtea.ai/openapi.json get /storage/generate-put-presigned-url Generate a presigned URL for uploading files to storage. See [Platform Overview](https://docs.galtea.ai/introduction). # Augment test cases Source: https://docs.galtea.ai/api-reference/test-cases/augment-test-cases https://api.galtea.ai/openapi.json post /generateFromFewShot/augment Submit test cases for augmentation using few-shot learning. The API automatically selects up to 5 random test cases from the given dataset as few-shot examples. A dataset whose test cases carry audio or file inputs is refused with a 400, because every generated test case would reuse the same attachment. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Count replay candidates per product for a test case revision Source: https://docs.galtea.ai/api-reference/test-cases/count-replay-candidates-per-product-for-a-test-case-revision https://api.galtea.ai/openapi.json get /testCases/{id}/replay-candidate-count For each requested product, returns how many historical evaluations in this test case's revision family would be replayed against this revision. A candidate is a distinct `(version, metric)` combination that any revision in the family was evaluated with (production sessions excluded) and whose version has not yet been run against this revision. Products with no candidates return `count: 0`. The revision must be the active head of its family, matching the replay call itself. # Create test case Source: https://docs.galtea.ai/api-reference/test-cases/create-test-case https://api.galtea.ai/openapi.json post /testCases Create a new test case. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Delete multiple test cases Source: https://docs.galtea.ai/api-reference/test-cases/delete-multiple-test-cases https://api.galtea.ai/openapi.json delete /testCases/batch Soft delete multiple test cases by their IDs. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Delete test case Source: https://docs.galtea.ai/api-reference/test-cases/delete-test-case https://api.galtea.ai/openapi.json delete /testCases/{id} Soft delete a test case. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Export test cases as CSV Source: https://docs.galtea.ai/api-reference/test-cases/export-test-cases-as-csv https://api.galtea.ai/openapi.json post /testCases/export Generate a CSV from the current DB state of the given test cases, upload to storage, and return a presigned download URL. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Get augmentation task status Source: https://docs.galtea.ai/api-reference/test-cases/get-augmentation-task-status https://api.galtea.ai/openapi.json get /generateFromFewShot/status/{taskId} Retrieve the status of an augmentation task. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Get test case by ID Source: https://docs.galtea.ai/api-reference/test-cases/get-test-case-by-id https://api.galtea.ai/openapi.json get /testCases/{id} Get a specific test case by ID. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Get test case languages Source: https://docs.galtea.ai/api-reference/test-cases/get-test-case-languages https://api.galtea.ai/openapi.json get /testCases/languages Get all available languages for the test cases of a product. Answers 403 or 404 when the caller cannot read the product. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Get test cases Source: https://docs.galtea.ai/api-reference/test-cases/get-test-cases https://api.galtea.ai/openapi.json get /testCases Get list of test cases with pagination and filtering options. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Update test case Source: https://docs.galtea.ai/api-reference/test-cases/update-test-case https://api.galtea.ai/openapi.json patch /testCases/{id} Update an existing test case. See [Datasets](https://docs.galtea.ai/concepts/product/dataset). # Create trace Source: https://docs.galtea.ai/api-reference/traces/create-trace https://api.galtea.ai/openapi.json post /inferenceResults Create a new trace. Identify the session with exactly one of `sessionId` or `sessionCustomId`. With `sessionCustomId` (your own conversation id) an existing session with that custom id is found and appended to: provide either `versionId` or `productId` as the anchor (if both are given, `versionId` wins). The session must already exist — create it first through the sessions API with that `custom_id`. If no matching session exists, the call is rejected with a 400. See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Create traces batch Source: https://docs.galtea.ai/api-reference/traces/create-traces-batch https://api.galtea.ai/openapi.json post /inferenceResults/batch Create multiple traces from conversation turns. Identify the session with exactly one of `sessionId` or `sessionCustomId` (same find-existing semantics as `POST /inferenceResults`). See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Delete multiple traces Source: https://docs.galtea.ai/api-reference/traces/delete-multiple-traces https://api.galtea.ai/openapi.json delete /inferenceResults/batch Soft delete multiple traces by their IDs. See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Delete trace Source: https://docs.galtea.ai/api-reference/traces/delete-trace https://api.galtea.ai/openapi.json delete /inferenceResults/{id} Soft delete a trace. See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Generate traces from test cases Source: https://docs.galtea.ai/api-reference/traces/generate-traces-from-test-cases https://api.galtea.ai/openapi.json post /inferenceResults/generate Triggers async inference generation for specified test cases using the version's endpoint connection. Provide either `versionId` or `productId`. When `productId` is given without a `versionId`, the product's latest properly configured version (one with a conversation-endpoint, phone, or WebRTC connection) is cloned and inferences run against the clone; the product must already have such a version. Omitting `versionId` clones a version, so it requires permission to create versions (a caller lacking it gets a 403). See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Get aggregated performance data Source: https://docs.galtea.ai/api-reference/traces/get-aggregated-performance-data https://api.galtea.ai/openapi.json get /inferenceResults/aggregatedData Get aggregated performance metrics data. Turns that belong to a seed session are always excluded, because a seed holds a predefined conversation and never ran. Answers 403 when any requested organization, product, version, session or trace is not readable. A READER or HUMAN_EVALUATOR aggregates only the products their user groups grant. See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Get trace by ID Source: https://docs.galtea.ai/api-reference/traces/get-trace-by-id https://api.galtea.ai/openapi.json get /inferenceResults/{id} Get a specific trace by ID. See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Get traces Source: https://docs.galtea.ai/api-reference/traces/get-traces https://api.galtea.ai/openapi.json get /inferenceResults Get list of traces with pagination and filtering. See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Update trace Source: https://docs.galtea.ai/api-reference/traces/update-trace https://api.galtea.ai/openapi.json patch /inferenceResults/{id} Update an existing trace. See [Traces](https://docs.galtea.ai/concepts/product/version/session/trace). The REST path `/inferenceResults` and the `inferenceResult*` JSON fields keep the old name of this entity. # Create user group Source: https://docs.galtea.ai/api-reference/user-groups/create-user-group https://api.galtea.ai/openapi.json post /userGroups Create a new user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Delete multiple user groups Source: https://docs.galtea.ai/api-reference/user-groups/delete-multiple-user-groups https://api.galtea.ai/openapi.json delete /userGroups/batch Soft delete multiple user groups by their IDs. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Delete user group Source: https://docs.galtea.ai/api-reference/user-groups/delete-user-group https://api.galtea.ai/openapi.json delete /userGroups/{id} Soft delete a user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Get user group by ID Source: https://docs.galtea.ai/api-reference/user-groups/get-user-group-by-id https://api.galtea.ai/openapi.json get /userGroups/{id} Get a specific user group by ID. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Get user groups Source: https://docs.galtea.ai/api-reference/user-groups/get-user-groups https://api.galtea.ai/openapi.json get /userGroups Get list of user groups with pagination and filtering. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Link metrics to user group Source: https://docs.galtea.ai/api-reference/user-groups/link-metrics-to-user-group https://api.galtea.ai/openapi.json post /userGroups/{id}/metrics Add metrics to a user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Link products to user group Source: https://docs.galtea.ai/api-reference/user-groups/link-products-to-user-group https://api.galtea.ai/openapi.json post /userGroups/{id}/products Add products to a user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Link users to user group Source: https://docs.galtea.ai/api-reference/user-groups/link-users-to-user-group https://api.galtea.ai/openapi.json post /userGroups/{id}/users Add users to a user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Unlink metrics from user group Source: https://docs.galtea.ai/api-reference/user-groups/unlink-metrics-from-user-group https://api.galtea.ai/openapi.json delete /userGroups/{id}/metrics Remove metrics from a user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Unlink products from user group Source: https://docs.galtea.ai/api-reference/user-groups/unlink-products-from-user-group https://api.galtea.ai/openapi.json delete /userGroups/{id}/products Remove products from a user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Unlink users from user group Source: https://docs.galtea.ai/api-reference/user-groups/unlink-users-from-user-group https://api.galtea.ai/openapi.json delete /userGroups/{id}/users Remove users from a user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Update user group Source: https://docs.galtea.ai/api-reference/user-groups/update-user-group https://api.galtea.ai/openapi.json patch /userGroups/{id} Update an existing user group. See [User Groups](https://docs.galtea.ai/concepts/user-group). # Create version Source: https://docs.galtea.ai/api-reference/versions/create-version https://api.galtea.ai/openapi.json post /versions Create a new version. `name` is optional; leave it out and the version is stored under its canonical label (`v` followed by its number). See [Versions](https://docs.galtea.ai/concepts/product/version). # Delete multiple versions Source: https://docs.galtea.ai/api-reference/versions/delete-multiple-versions https://api.galtea.ai/openapi.json delete /versions/batch Soft delete multiple versions by their IDs. See [Versions](https://docs.galtea.ai/concepts/product/version). # Delete version Source: https://docs.galtea.ai/api-reference/versions/delete-version https://api.galtea.ai/openapi.json delete /versions/{id} Soft delete a version. See [Versions](https://docs.galtea.ai/concepts/product/version). # Get or create version by facts Source: https://docs.galtea.ai/api-reference/versions/get-or-create-version-by-facts https://api.galtea.ai/openapi.json post /versions/get-or-create Answers the live version of the product whose facts are the ones you send, and creates it when no version holds them. Send the agent state you are about to test, and callers that run at the same time on that state all receive one version instead of each creating its own. The key order you send does not change the match. The answer is 200 whether the version already existed or was created now, because the caller asks for the version for a state and not for a new row. # Get version by ID Source: https://docs.galtea.ai/api-reference/versions/get-version-by-id https://api.galtea.ai/openapi.json get /versions/{id} Get a specific version by ID. See [Versions](https://docs.galtea.ai/concepts/product/version). # Get versions Source: https://docs.galtea.ai/api-reference/versions/get-versions https://api.galtea.ai/openapi.json get /versions Get list of versions with pagination and filtering options. See [Versions](https://docs.galtea.ai/concepts/product/version). # List metrics annotated with compatibility for a version Source: https://docs.galtea.ai/api-reference/versions/list-metrics-annotated-with-compatibility-for-a-version https://api.galtea.ai/openapi.json get /versions/{id}/compatible-metrics Returns evaluable metrics annotated with whether they can be satisfied by the version's product metadata and conversation endpoint connection output mapping, plus a `relevance` ranking against the tests the evaluation will run against. Span-dependent and runtime-only params are out of scope and treated as always satisfied. A version with no conversation target at all (no endpoint, phone, or WebRTC connection) reports every metric incompatible, since this evaluation has to produce new output before scoring it. Its session counterpart `GET /sessions/{id}/compatible-metrics` scores output that is already recorded, so it assesses each parameter instead. When `testIds` are supplied, each metric is ranked `linked` (linked to a specification of one of those tests), `matching` (linked to a same-product specification whose testType matches one of those tests), or `none`. Without `testIds` every metric is `none`. # Update version Source: https://docs.galtea.ai/api-reference/versions/update-version https://api.galtea.ai/openapi.json patch /versions/{id} Update an existing version. See [Versions](https://docs.galtea.ai/concepts/product/version). # Create WebRTC connection Source: https://docs.galtea.ai/api-reference/web-rtc-connections/create-webrtc-connection https://api.galtea.ai/openapi.json post /webRtcConnections Create a new WebRTC connection. See [WebRTC Connections](https://docs.galtea.ai/concepts/product/webrtc-connection). # Delete multiple WebRTC connections Source: https://docs.galtea.ai/api-reference/web-rtc-connections/delete-multiple-webrtc-connections https://api.galtea.ai/openapi.json delete /webRtcConnections/batch Soft delete multiple WebRTC connections by their IDs. See [WebRTC Connections](https://docs.galtea.ai/concepts/product/webrtc-connection). # Delete WebRTC connection Source: https://docs.galtea.ai/api-reference/web-rtc-connections/delete-webrtc-connection https://api.galtea.ai/openapi.json delete /webRtcConnections/{id} Soft delete a WebRTC connection. See [WebRTC Connections](https://docs.galtea.ai/concepts/product/webrtc-connection). # Get WebRTC connection by ID Source: https://docs.galtea.ai/api-reference/web-rtc-connections/get-webrtc-connection-by-id https://api.galtea.ai/openapi.json get /webRtcConnections/{id} Get a specific WebRTC connection by ID. See [WebRTC Connections](https://docs.galtea.ai/concepts/product/webrtc-connection). # Get WebRTC connections Source: https://docs.galtea.ai/api-reference/web-rtc-connections/get-webrtc-connections https://api.galtea.ai/openapi.json get /webRtcConnections Get list of WebRTC connections with pagination and filtering. See [WebRTC Connections](https://docs.galtea.ai/concepts/product/webrtc-connection). # Update WebRTC connection Source: https://docs.galtea.ai/api-reference/web-rtc-connections/update-webrtc-connection https://api.galtea.ai/openapi.json patch /webRtcConnections/{id} Update an existing WebRTC connection. See [WebRTC Connections](https://docs.galtea.ai/concepts/product/webrtc-connection). # Whether every deployed service can reach its dependencies Source: https://docs.galtea.ai/api-reference/whether-every-deployed-service-can-reach-its-dependencies https://api.galtea.ai/openapi.json get /self-check One entry per downstream service, naming every dependency it resolved and whether exercising it worked, plus an `api` block for the API's own identity provider and mail relay. A readiness probe answers whether a process started; this answers whether the models, storage and queues it was configured with actually respond, which is what an operator needs after an install. A service being down never fails this call; it is reported with `status: unreachable` instead. Requires authentication; restricted to admins and organizations with an enterprise subscription, the same rule as `GET /build-info`. # Installation Source: https://docs.galtea.ai/cli/installation Install the Galtea CLI on your platform The Galtea CLI ships as a single `galtea` binary, repackaged for the channels you already use. Pick the tab that matches your platform. If you have [Homebrew](https://brew.sh) (macOS) or [Linuxbrew](https://docs.brew.sh/Homebrew-on-Linux), the tap auto-installs on first use: ```bash theme={"system"} brew install galtea-ai/tap/galtea ``` Apple Silicon, Intel macOS, and Linux on `x86_64` / `arm64` are all supported. Add the Galtea APT repository (one-time GPG key + `sources.list` bootstrap), then install the CLI: ```bash theme={"system"} sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://pkgs.galtea.ai/public.key \ | sudo tee /etc/apt/keyrings/galtea.asc > /dev/null echo "deb [signed-by=/etc/apt/keyrings/galtea.asc] \ https://pkgs.galtea.ai/apt stable main" \ | sudo tee /etc/apt/sources.list.d/galtea.list sudo apt update sudo apt install galtea ``` Add the Galtea YUM repository (one-time GPG key import + repo file), then install the CLI: ```bash theme={"system"} sudo rpm --import https://pkgs.galtea.ai/public.key sudo tee /etc/yum.repos.d/galtea.repo <<'EOF' [galtea] name=Galtea CLI baseurl=https://pkgs.galtea.ai/yum enabled=1 gpgcheck=1 gpgkey=https://pkgs.galtea.ai/public.key EOF sudo dnf install galtea ``` Rocky Linux and AlmaLinux are supported via the same repository. If you already have Python 3.9+ on your machine, you can install the CLI from PyPI: ```bash theme={"system"} pip install galtea-cli ``` The PyPI package ships per-platform wheels that bundle the same `galtea` binary used by every other channel. The `galtea` console-script entry point lands on your `PATH` automatically. For more information, visit the [galtea-cli PyPI page](https://pypi.org/project/galtea-cli/). ### Verify installation Once installed, confirm the binary is on your `PATH` and prints a version: ```bash theme={"system"} galtea --version ``` ### Update Use the package-manager gesture native to the channel you installed from: ```bash theme={"system"} brew update && brew upgrade galtea ``` ```bash theme={"system"} sudo apt update && sudo apt upgrade galtea ``` ```bash theme={"system"} sudo dnf upgrade galtea ``` ```bash theme={"system"} pip install --upgrade galtea-cli ``` ### Uninstall ```bash theme={"system"} brew uninstall galtea # Optional: drop the tap entirely brew untap galtea-ai/tap ``` ```bash theme={"system"} sudo apt remove galtea ``` ```bash theme={"system"} sudo dnf remove galtea ``` ```bash theme={"system"} pip uninstall galtea-cli ``` ### Troubleshooting If you run into any issues — installing, verifying, updating, or uninstalling — contact our support team at [support@galtea.ai](mailto:support@galtea.ai). # Usage Source: https://docs.galtea.ai/cli/usage Authenticate the Galtea CLI and explore the command surface The `galtea` CLI is a terminal-native client for the Galtea AI testing & evaluation platform. Every [REST API](/api-reference) operation ships as a `galtea ` command, auto-generated from the live OpenAPI spec. ### What it can do Anything the [Galtea API](/api-reference) does — every endpoint is reachable as a CLI command, with auth, retries, and shell completion handled for you. ### Why use it * **Let your coding agent manage Galtea for you.** Claude Code, Cursor, Windsurf, and other agents can use the CLI to create products, run evaluations, pull traces, and more — without leaving your editor. * **Script your workflows.** Automate repetitive tasks like batch-creating tests, kicking off evaluations from CI/CD, or syncing specifications across environments. * **Faster than the dashboard for quick lookups.** List the last 10 evaluations, check a session's status, or pull traces in seconds. ### Authentication The CLI supports both an interactive flow for local use and a non-interactive flow for CI and Docker. ```bash theme={"system"} galtea login # Paste your gsk_* API key when prompted. ``` The key is validated against the server, then stored in your user config directory, typically `~/.config/galtea/apis.json`, with mode `0600` (only your user can read it). You can grab a `gsk_*` key from the [settings page](https://platform.galtea.ai/settings). ```bash theme={"system"} export GALTEA_API_KEY=gsk_... ``` If `GALTEA_API_KEY` is exported in the environment, running `galtea login` becomes **optional** — running it anyway stores the key to disk for future shells. ### Your first command Once you're authenticated, list your products to validate end-to-end auth and connectivity: ```bash theme={"system"} galtea products list ``` If this returns a list (even an empty one) instead of a `401 Unauthorized`, you're ready to go. ### Discovering all commands These are the available OpenAPI commands that were downloaded to your machine. Run `galtea sync` after a new [Galtea API](/api-reference) release to refresh the command tree. ```bash theme={"system"} galtea help ``` For parameter, request body, and response shape details, see the [**REST API**](/api-reference) tab. ### Hand it to your coding agent The CLI is a great fit for AI coding assistants. Install the [Galtea Agent Skill](/sdk/integrations/agent-skill) and Claude Code, Cursor, Windsurf, or any other Agent-Skills-compatible client will reach for `galtea` on your behalf — picking the right command, handling authentication, and following the workflows our team has already validated, so you don't have to brief your agent on Galtea first. # Metric Source: https://docs.galtea.ai/concepts/metric Criteria and methods for evaluating AI product performance ## What are Metrics? Metrics define how Galtea scores your [product's](/concepts/product) outputs during [evaluations](/concepts/product/version/session/evaluation). Each metric captures one specific quality dimension — factual accuracy, tone, security resilience, or a custom criterion you define. Metrics are organization-wide and can be reused across multiple products. You can link metrics to [Specifications](/concepts/product/specification) for structured evaluation workflows. You can **create**, view and manage your metrics on the [Galtea dashboard](https://platform.galtea.ai/) or programmatically using the [Galtea SDK](/sdk/api/metric/service). ## Two Families of Metrics * **Deterministic metrics** apply rule-based logic — string matching, numerical checks, bounding box overlap. Results are consistent and reproducible. Built-in examples: [BLEU](/concepts/metric/bleu), [ROUGE](/concepts/metric/rouge), [Text Similarity](/concepts/metric/text-similarity), [Tool Correctness](/concepts/metric/tool-correctness). You can also define your own using the SDK's `CustomScoreEvaluationMetric` class — see [our tutorial](/sdk/tutorials/evaluate-with-custom-metrics). * **Non-deterministic metrics** use an LLM-as-a-judge to assess open-ended qualities like factual accuracy, misuse resilience, and task completion. Galtea's judges are human-aligned and optimized for each evaluation type. You can also [create your own AI Evaluation metrics](/concepts/metric/evaluation-types#ai-evaluation) with a custom scoring rubric. ## How Metrics Are Evaluated Every metric has an [Evaluation Type](/concepts/metric/evaluation-types) that determines how scoring happens: ```mermaid theme={"system"} %%{init: {"theme": "neutral"}}%% flowchart TD A[What kind of scoring?] --> B{Automated?} B -->|Yes| C{Rule-based logic?} B -->|No| D[Human Evaluation] C -->|Yes| E[Self-Hosted] C -->|No| F[AI Evaluation] E -->|Run locally, upload score| H[Pre-computed or CustomScoreEvaluationMetric] F -->|LLM judges with your rubric| I[judge_prompt + evaluation_params] D -->|Annotators score via dashboard| G[PENDING_HUMAN → score submitted] ```
Your own logic runs locally; only the score is uploaded. An LLM scores responses using a rubric you define. Human annotators score responses using defined criteria. ## Revisions and Analytics Creating a revision marks the previous metric revision as `Legacy`. Legacy and analytics inclusion are separate states. A legacy revision keeps contributing to scores and coverage unless you exclude it. After you save a revision, the dashboard shows how many results and products the earlier revision affects. The default keeps those results in analytics. Select **Exclude from analytics** only when you want both the scores and coverage to stop counting them. You can reverse this decision from the metric's actions menu. Exclusion does not delete or change any evaluation. Each evaluation keeps its score, reason, and metric revision. The metric list hides excluded revisions by default. Use **Show excluded from analytics** to reveal them. Turn on **Include legacy** to inspect earlier revisions. Add the **Analytics** column from the **Columns** menu to see whether each revision is `Included` or `Excluded`. When a product has excluded results, its Analytics page states how many results are outside the reported numbers. The link opens the evaluation list with **Show excluded from analytics** active. Excluded revisions do not appear in the Analytics metric filter because they contribute no scores to that page. The page's **Last Successful** date still counts their evaluations, because it reports when an evaluation last ran. ## Optimization review **Optimize** writes an improved judge prompt into a new revision named ` (Optimized)`. While it runs, that revision's page shows its progress, and you can leave the page. Unlike other revisions, it does not mark the original `Legacy`. When the optimizer returns a different prompt, the revision shows **Ready for review**, and its page shows the changes and both prompts. * **Apply optimization** marks the original revision `Legacy`. The optimized revision takes the original's name, description, and tags, and joins its specifications and user groups. Monitors use it from their next run. * **Keep original** leaves the original unchanged and marks the optimized revision **Not applied**. You can then optimize again. From 30 annotated evaluations, a run keeps some of them as a validation set. The optimizer does not train on the validation set. An evaluation kept for validation stays out of training in later attempts of the same metric. The **Optimize** confirmation shows how many evaluations go to the training set and how many to the validation set. With fewer than 30 annotated evaluations, there is no validation set unless an earlier attempt kept one. Optimize is disabled when the evaluations kept for validation leave too few to train on. Expand **Advanced Options** in the **Optimize** confirmation to narrow a run. **Strategies** sets which rewrites the optimizer tries: **Fix false passes**, **Fix false failures**, **Add a scoring rubric**, **Simplify the prompt** and **Broaden the wording**. Each step asks for one rewrite per selected strategy and keeps the best one. **Maximum steps** sets how many steps the run may take, from 1 to 10, and defaults to 10. A run stops earlier when the prompt stops improving. With every strategy selected and 10 steps, the run is the same as one without options. The review card, the No improvement notice and the page of a declined or applied revision show the steps the run used out of its maximum and the strategies it ran. An attempt that did not report them shows **Not reported**. The review compares the original and the optimized prompt on both sets. For each prompt, it shows how many of the evaluations that both prompts scored agree with your annotation, and the alignment when it is defined. A declined or applied revision keeps these results on its page. The validation result is evidence, not a guarantee: a small validation set gives a rough estimate. Only the active revision of a metric can be optimized. Through the API, call `POST /metrics/{id}/activate-optimization` or `POST /metrics/{id}/decline-optimization` with the optimized revision's id. ## Judge model retirement When a judge model is retiring, you get an email up to 30 days before the retirement date. It names the affected metrics and the judge models you can use instead. The email goes to the organization owners and admins, and to the creator of each affected metric. A metric's judge model cannot change after you create it. For your own metrics, create a new revision with another judge model. Galtea maintains its own metrics, so you cannot revise them. If your organization used one in the last 90 days, its owners and admins get the email too. Galtea may replace one of its metrics with a new revision on another judge model. When it does, the email names the new revision and links to it. Use that revision to keep evaluating. When there is no new revision, create your own metric with one of the listed judge models to keep evaluating after the retirement date. If the judge model becomes unavailable to Galtea on or after the retirement date, and stays unavailable for six hours, the same recipients get one email naming the metrics that stopped scoring. ## Available Metrics The following table lists the default metrics available in the Galtea platform. You can also [create custom metrics](/concepts/metric/evaluation-types#ai-evaluation) or [generate metrics from specifications](/concepts/metric/ai-generation). | Metric | Category | Description | | - | - | - | | **[Factual Accuracy](/concepts/metric/factual-accuracy)** | RAG | Evaluates whether the actual\_output factually aligns with the expected\_output. | | **[Resilience To Noise](/concepts/metric/resilience-to-noise)** | RAG | Evaluates whether the output is resilient to noisy input (typos, OCR/ASR errors). | | **[Answer Relevancy](/concepts/metric/answer-relevancy)** | RAG | Evaluates how relevant the actual\_output is compared to the provided input. | | **[Faithfulness](/concepts/metric/faithfulness)** | RAG | Evaluates whether the actual\_output factually aligns with the retrieval\_context. | | **[Contextual Precision](/concepts/metric/contextual-precision)** | RAG | Evaluates whether relevant retrieval\_context nodes are ranked higher than irrelevant ones. | | **[Contextual Recall](/concepts/metric/contextual-recall)** | RAG | Evaluates whether the retrieval\_context sufficiently covers the expected\_output. | | **[Contextual Relevancy](/concepts/metric/contextual-relevancy)** | RAG | Evaluates the overall relevance of the retrieval\_context for a given input. | | **[BLEU](/concepts/metric/bleu)** | Deterministic | Measures n-gram overlap between actual and expected output. | | **[ROUGE](/concepts/metric/rouge)** | Deterministic | Measures longest common subsequence between actual and expected output. | | **[METEOR](/concepts/metric/meteor)** | Deterministic | Aligns words using exact matches, stems, or synonyms. | | **[Text Similarity](/concepts/metric/text-similarity)** | Deterministic | Quantifies textual resemblance using character-level fuzzy matching. | | **[Text Match](/concepts/metric/text-match)** *(deprecated)* | Deterministic | Binary match using character-level fuzzy matching with a threshold. Use [Text Similarity](/concepts/metric/text-similarity) instead. | | **[IOU](/concepts/metric/iou)** | Deterministic | Measures spatial overlap between predicted and reference bounding boxes. | | **[Spatial Match](/concepts/metric/spatial-match)** | Deterministic | Binary evaluation of spatial alignment using IoU score. | | **[URL Validation](/concepts/metric/url-validation)** | Deterministic | Checks if all URLs in the response are valid and safe. | | **[Tool Correctness](/concepts/metric/tool-correctness)** | Deterministic | Compares tools used by the agent against expected tools. | | **[JSON Field Match](/concepts/metric/json-field-match)** | Deterministic | Compares JSON objects field by field, returning the fraction of expected fields that match. Ignores extra fields in the actual output. | | **[JSON Field Match (Normalized)](/concepts/metric/json-field-match-normalized)** | Deterministic | Compares JSON objects field by field with case-insensitive and accent-insensitive string matching. Ignores extra fields in the actual output. | | **[JSON Field Match (Strict)](/concepts/metric/json-field-match-strict)** | Deterministic | Like JSON Field Match, but an unexpected field in the actual output also lowers the score. | | **[JSON Field Match (Normalized, Strict)](/concepts/metric/json-field-match-normalized-strict)** | Deterministic | Like JSON Field Match (Normalized), but an unexpected field in the actual output also lowers the score. | | **[Role Adherence](/concepts/metric/role-adherence)** | Conversational | Evaluates whether the chatbot adheres to its given role throughout a conversation. | | **[Conversation Completeness](/concepts/metric/conversation-completeness)** | Conversational | Evaluates whether the chatbot satisfies user needs throughout a conversation. | | **[Conversation Relevancy](/concepts/metric/conversation-relevancy)** | Conversational | Evaluates whether the chatbot generates relevant responses throughout a conversation. | | **[Knowledge Retention](/concepts/metric/knowledge-retention)** | Conversational | Assesses factual information retention throughout a conversation. | | **[User Satisfaction](/concepts/metric/user-satisfaction)** | Conversational | Evaluates user satisfaction with the chatbot interaction. | | **[User Objective Accomplished](/concepts/metric/user-objective-accomplished)** | Conversational | Evaluates whether the chatbot fulfilled the user's stated objective. | | **[Non-Toxic](/concepts/metric/non-toxic)** | Security & Safety | Evaluates whether responses are free of toxic language. | | **[Unbiased](/concepts/metric/unbiased)** | Security & Safety | Evaluates whether output is free of gender, racial, or political bias. | | **[Misuse Resilience](/concepts/metric/misuse-resilience)** | Security & Safety | Evaluates resilience to misuse and alignment with the product's defined capabilities and inabilities. | | **[Data Leakage](/concepts/metric/data-leakage)** | Security & Safety | Evaluates whether the LLM returns sensitive information. | | **[Jailbreak Resilience](/concepts/metric/jailbreak-resilience)** | Security & Safety | Evaluates resistance to adversarial prompt manipulation. | ## SDK Integration Manage metrics using the Python SDK ## Metric Properties The name of the metric. **Example**: "Factual Accuracy" A brief description of what the metric evaluates. The model used to score the metric. Does not apply to deterministic metrics. For a [Classifier (Jev)](/concepts/metric/classifier) metric, the platform sets the Jev model. **Example**: "GPT-4.1" Tags for categorization. **Example**: \["RAG", "Conversational"] How outputs are scored: AI Evaluation, Human Evaluation, or Self-Hosted. See [Evaluation Types](/concepts/metric/evaluation-types) for full details. [User groups](/concepts/user-group) for Human Evaluation metrics. Controls which annotators can score evaluations for this metric. The data fields available to the evaluator. See [Evaluation Parameters](/concepts/metric/evaluation-parameters) for the full reference. Not applicable for Self-Hosted metrics. For a Classifier (Jev) metric, the platform derives them from the placeholders in the question. The one Yes/No or Scale question a classifier metric asks. Only for the classifier source. See [Classifier (Jev)](/concepts/metric/classifier). ## Related How Galtea's concepts connect — diagram + per-entity quick reference. The assessment of an evaluation using a specific metric's criteria Write your own judge prompt and scoring rubric. # AI Metric Generation Source: https://docs.galtea.ai/concepts/metric/ai-generation Automatically generate evaluation metrics from your product specifications using AI ## Overview AI Metric Generation lets you automatically create evaluation metrics from your product's [specifications](/concepts/product/specification). Instead of manually crafting judge prompts and configuring evaluation parameters, the AI analyzes your specifications and generates ready-to-use metrics. Evaluation parameters are automatically selected based on each specification's description and test type. The generated judge prompt follows a format optimized for reliable LLM-based evaluation across different evaluator models.