> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> How Galtea's concepts connect

> [!WARNING]
>
> You are reading the docs of Galtea 5.11. [Read this page in the latest version](https://docs.galtea.ai/concepts/overview.md).

## How Everything Connects

```mermaid theme={"system"}
%%{init: {"theme": "neutral"}}%%
flowchart TD
    P[Product]
    T[Datasets]
    Spec[Specifications]
    V[Versions]
    EC[Endpoint Connections]
    TC[Test Cases]
    S[Sessions]
    Sp[Spans]
    Tr[Traces]
    E[Evaluations]
    M[Metrics]
    R[Runs]

    P --> |Defined by| Spec
    Spec -.-> |Challenged by| T
    T -.-> |Groups| TC
    TC -.-> |Simulated as| S
    S --> |Are a set of| Tr
    Tr --> |Hold| Sp
    S --> |Judged in| E
    Spec --> |Evaluated with| M
    E --> |Scored by| M

    P --> |Iterated with| V
    V --> |Linked to| EC
    V --> |Evaluated in| S

    P --> |Launched in| R
    R --> |Groups| E
```

> [!NOTE]
>
> The dotted lines represent relationships that don't exist for production-based evaluations, where there is no Dataset or Test Case and the Session is generated from real user interactions.

> [!NOTE]
>
> The diagram focuses on core concepts. Some related concepts are omitted for clarity — see the [Additional Notes](#additional-notes) section.

## At a Glance

Galtea's concepts split into four lifecycle scopes:

* **What you're testing**: Your [Product](https://docs.galtea.ai/v5.11/concepts/product.md), defined by a set of [Specifications](https://docs.galtea.ai/v5.11/concepts/product/specification.md) and iterated on through new [Versions](https://docs.galtea.ai/v5.11/concepts/product/version.md).
* **What you test with**: [Datasets](https://docs.galtea.ai/v5.11/concepts/product/dataset.md) that group [Test Cases](https://docs.galtea.ai/v5.11/concepts/product/dataset/test-case.md); your Product's performance against them is measured with [Metrics](https://docs.galtea.ai/v5.11/concepts/metric.md).
* **What your Product generates**: [Sessions](https://docs.galtea.ai/v5.11/concepts/product/version/session.md), each a sequence of [Traces](https://docs.galtea.ai/v5.11/concepts/product/version/session/trace.md) along with their [Spans](https://docs.galtea.ai/v5.11/concepts/product/version/session/span.md).
* **How you measure performance**: [Evaluations](https://docs.galtea.ai/v5.11/concepts/product/version/session/evaluation.md) that apply a [Metric](https://docs.galtea.ai/v5.11/concepts/metric.md) to a [Session](https://docs.galtea.ai/v5.11/concepts/product/version/session.md) or to a specific [Trace](https://docs.galtea.ai/v5.11/concepts/product/version/session/trace.md). Each Evaluation you launch by hand belongs to a [Run](https://docs.galtea.ai/v5.11/concepts/product/run.md), the record of that launch.

## Additional Notes

A few relationships are easy to misread, and some are omitted from the diagram for clarity. These notes fill in the gaps:

* **Specifications are the glue.** A [Specification](https://docs.galtea.ai/v5.11/concepts/product/specification.md) links to one or more [Metrics](https://docs.galtea.ai/v5.11/concepts/metric.md) so it can be evaluated, and [Datasets](https://docs.galtea.ai/v5.11/concepts/product/dataset.md) can be generated from a Specification to challenge it.
* **Test Case ↔ Session.** A [Test Case](https://docs.galtea.ai/v5.11/concepts/product/dataset/test-case.md) belongs to a [Dataset](https://docs.galtea.ai/v5.11/concepts/product/dataset.md). When a [Session](https://docs.galtea.ai/v5.11/concepts/product/version/session.md) is generated to simulate a Test Case, it references that Test Case — so the same Test Case can show up across many Sessions (for example, re-running a Dataset across multiple Versions). Sessions can also be generated without a Test Case, for example when [recording real user interactions](https://docs.galtea.ai/v5.11/sdk/tutorials/monitor-production-responses-to-user-queries.md) to evaluate the Product in production.
* **Endpoint Connections shape Dataset generation.** When you create a Dataset with an [Endpoint Connection](https://docs.galtea.ai/v5.11/concepts/product/endpoint-connection.md) selected, the generated Test Cases are produced to match that endpoint's expected input format.
* **Per-turn evaluations.** An [Evaluation](https://docs.galtea.ai/v5.11/concepts/product/version/session/evaluation.md) can target a single [Trace](https://docs.galtea.ai/v5.11/concepts/product/version/session/trace.md) instead of the whole Session — useful when you want to score, for example, how the agent's output reflected the retrieval context the RAG system provided on that turn. Per-turn Evaluations still belong to the Session that contains the turn; they additionally point at the specific Trace being scored.
* **Two types of model.** The standard [Model](https://docs.galtea.ai/v5.11/concepts/model.md) is what your Product runs on — linked to a Version and used to track costs, token usage, and similar. The **evaluator model** is the LLM Galtea uses to perform the judgment for non-deterministic Metrics; it's selected by name on the Metric, not on the Version. See [Models vs. Evaluator Models](https://docs.galtea.ai/v5.11/concepts/model.md#models-vs-evaluator-models) on the Model page for the full distinction.
* **[User Groups](https://docs.galtea.ai/v5.11/concepts/user-group.md) for human evaluation.** A User Group routes [Human Evaluation](https://docs.galtea.ai/v5.11/concepts/metric/evaluation-types.md#human-evaluation) Metrics to a specific set of annotators, so only the right reviewers see those Evaluations on the dashboard.
* **[Runs](https://docs.galtea.ai/v5.11/concepts/product/run.md) group the Evaluations of one launch.** Galtea creates a Run every time you launch Evaluations, and returns its id. The Run records who launched what, against which [Version](https://docs.galtea.ai/v5.11/concepts/product/version.md), and how it ended. An Evaluation carries a Run or a [Monitor](https://docs.galtea.ai/v5.11/concepts/product/monitor.md), never both: a Run is what a Monitor is, on the development side. A Run's Sessions are a union: the [Sessions](https://docs.galtea.ai/v5.11/concepts/product/version/session.md) it created, plus the distinct Sessions of its Evaluations. So a launch that generates inferences and evaluates no Metric still gets a Run, and a launch over Sessions that already exist reaches them through its Evaluations.
* **[Monitors](https://docs.galtea.ai/v5.11/concepts/product/monitor.md) watch production automatically.** A Monitor is an always-on rule on a [Product](https://docs.galtea.ai/v5.11/concepts/product.md) (optionally one [Version](https://docs.galtea.ai/v5.11/concepts/product/version.md)) that scores a sampled fraction of its production [Sessions](https://docs.galtea.ai/v5.11/concepts/product/version/session.md) with metric families, Specifications, or both, with no run to trigger by hand.

## All Concepts

> [card] [Product](https://docs.galtea.ai/v5.11/concepts/product.md)
>
> A functionality or service being evaluated

> [card] [Specification](https://docs.galtea.ai/v5.11/concepts/product/specification.md)
>
> A testable behavioral expectation for a product

> [card] [Version](https://docs.galtea.ai/v5.11/concepts/product/version.md)
>
> A specific iteration of a product

> [card] [Endpoint Connection](https://docs.galtea.ai/v5.11/concepts/product/endpoint-connection.md)
>
> A reusable connection to an external endpoint for AI product evaluation

> [card] [Monitor](https://docs.galtea.ai/v5.11/concepts/product/monitor.md)
>
> An always-on evaluation rule for production sessions

> [card] [Dataset](https://docs.galtea.ai/v5.11/concepts/product/dataset.md)
>
> A dataset of test cases for evaluating product performance

> [card] [Test Case](https://docs.galtea.ai/v5.11/concepts/product/dataset/test-case.md)
>
> Each challenge in a dataset for evaluating product performance

> [card] [Session](https://docs.galtea.ai/v5.11/concepts/product/version/session.md)
>
> A full conversation between a user and an AI system.

> [card] [Trace](https://docs.galtea.ai/v5.11/concepts/product/version/session/trace.md)
>
> A single turn in a conversation between a user and the AI.

> [card] [Span](https://docs.galtea.ai/v5.11/concepts/product/version/session/span.md)
>
> Understand how spans capture the internal operations of your AI agents.

> [card] [Run](https://docs.galtea.ai/v5.11/concepts/product/run.md)
>
> The record of one evaluation launch

> [card] [Evaluation](https://docs.galtea.ai/v5.11/concepts/product/version/session/evaluation.md)
>
> The assessment of an evaluation using a specific metric's criteria

> [card] [Metric](https://docs.galtea.ai/v5.11/concepts/metric.md)
>
> Ways to evaluate and score product performance

> [card] [Model](https://docs.galtea.ai/v5.11/concepts/model.md)
>
> Way to keep track of your models' costs


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.