> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation

> An evaluation that assesses traces from a session using a metric

## What is an Evaluation?

An evaluation in Galtea represents the assessment of traces from a [session](/concepts/product/version/session) using the evaluation criteria of a [metric](/concepts/metric). Evaluations are now directly linked to sessions, allowing for comprehensive evaluation of full sessions containing multiple traces for multi-turn dialogues.

<Note>Evaluations do not call your AI product — they score outputs that have already been generated. Run your product first to produce outputs, then trigger the evaluation to score them.</Note>

You can create evaluations from the [Galtea dashboard](https://platform.galtea.ai/) (using [Endpoint Connections](/concepts/product/endpoint-connection)) or programmatically using the [Galtea SDK](/sdk/api/evaluation/service).

## Evaluation Lifecycle

Evaluations follow a specific lifecycle:

<Steps>
  <Step title="Creation">[Trigger an evaluation](/sdk/tutorials/run-dataset-based-evaluations). It will appear in the session's details page with the status "pending"</Step>
  <Step title="Processing">Galtea's evaluation system processes the evaluation using the evaluation criteria of the selected [metric](/concepts/metric)</Step>
  <Step title="Completion">Once processed, the status changes to "completed" and the [results](/concepts/product/version/session/evaluation#evaluation-properties) are available</Step>
</Steps>

## SDK Integration

The Galtea SDK allows you to create, view, and manage evaluations programmatically. This is particularly useful for organizations that want to automate their evaluation process or integrate it into their CI/CD pipeline.

<CardGroup cols={2}>
  <Card title="Evaluation Service SDK" icon="clipboard-check" iconType="solid" href="/sdk/api/evaluation/service">
    Manage evaluations using the Python SDK
  </Card>

  <Card title="GitHub Actions" icon="code-compare" href="/sdk/integrations/github-actions">
    Learn how to set up GitHub Actions to automatically evaluate new versions
  </Card>
</CardGroup>

## Evaluation Properties

### Result Properties

Once an evaluation completes, the following fields are available:

<ResponseField name="Status" type="Enum">
  The current status of the evaluation.
  Possible values:

  * **Pending**: The evaluation has been created but not yet processed.
  * **Pending Human**: The evaluation is waiting for a human annotator to review and score it. This status is used for metrics with `source: "human_evaluation"`.
  * **Success**: The evaluation was processed successfully.
  * **Failed**: The evaluation encountered an error during processing. Check the `canRetry` field to determine if this evaluation can be retried.
  * **Skipped**: The evaluation could not produce a score. The error message names the cause. Common causes are:

    * Metric validation failed, for example a required parameter is missing.
    * The organization ran out of credits.
    * The metric could not read the actual output, for example plain text where [JSON Field Match](/concepts/metric/json-field-match) expects a JSON object.
    * The judge model was unavailable.

    Check the `canRetry` field to see whether the evaluation is eligible for a retry. A retry re-scores the output already stored on the session. When the cause is that output, send a new inference result instead.
  * **Outdated**: The evaluation was produced by a [Monitor](/concepts/product/monitor), and the session then reopened and gained a new turn. The score describes an older version of the conversation, so it no longer counts as current: charts and averages leave it out, and the monitor scores the session again once it closes. The row keeps its score and its reason, so you can still read what it found.
</ResponseField>

<ResponseField name="Score" type="Number">
  The score assigned to the output by the metric's evaluation criteria. **Example**: 0.85
</ResponseField>

<ResponseField name="Reason" type="Text">
  The explanation of the score assigned to the output by the [metric's](/concepts/metric) evaluation criteria.
</ResponseField>

<ResponseField name="Error" type="Text">
  The error message if the evaluation failed during processing.
</ResponseField>

<ResponseField name="Retries Attempted" type="Number">
  The number of retry attempts made for this evaluation. Starts at 0 and increments each time the evaluation is retried. Used to track retry history and enforce retry limits.
</ResponseField>

<ResponseField name="Can Retry" type="Boolean" default="false">
  Indicates whether the evaluation is eligible for the retry operation. When `false`, it cannot be retried at all. When `true`, it can, but a retry does not always help: `canRetry` is a permission, not a promise. A retry re-scores the output already stored on the session, so it succeeds for a temporary cause (an evaluation processing error, insufficient credits, a judge model that was unavailable) and skips again when the cause is the stored output itself. Defaults to `false`.
</ResponseField>

<ResponseField name="Human Evaluator ID" type="Text" optional>
  The ID of the human evaluator assigned to this evaluation, if applicable.
</ResponseField>

<ResponseField name="Human Score" type="Number" optional>
  The human-provided annotation score, separate from the AI-generated score. When a human annotates an AI-evaluated metric, this field stores their score without overwriting the original AI score.
</ResponseField>

<ResponseField name="Human Reason" type="Text" optional>
  The human-provided annotation reason, separate from the AI-generated reason. When a human annotates an AI-evaluated metric, this field stores their reasoning without overwriting the original AI reason.
</ResponseField>

<ResponseField name="Human Evaluator Started At" type="Text" optional>
  The timestamp when the human evaluator started reviewing this evaluation, if applicable.
</ResponseField>

<ResponseField name="Human Evaluator Finished At" type="Text" optional>
  The timestamp when the human evaluator submitted their evaluation or annotation.
</ResponseField>

## Related

<CardGroup cols={2}>
  <Card title="Concepts overview" icon="diagram-project" iconType="solid" href="/concepts/overview">
    How Galtea's concepts connect — diagram + per-entity quick reference.
  </Card>

  <Card title="Dataset" icon="clipboard-list" iconType="solid" href="/concepts/product/dataset">
    A dataset of test cases for evaluating product performance
  </Card>

  <Card title="Test Case" icon="list" iconType="solid" href="/concepts/product/dataset/test-case">
    Each challenge in a dataset for evaluating product performance
  </Card>

  <Card title="Session" icon="clock-rotate-left" iconType="solid" href="/concepts/product/version/session">
    A full conversation between a user and an AI system.
  </Card>

  <Card title="Metric" icon="gauge" iconType="solid" href="/concepts/metric">
    Ways to evaluate and score product performance
  </Card>
</CardGroup>
