> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation Parameters

> Data fields available to metric evaluators during scoring

> [!WARNING]
>
> You are reading the docs of Galtea 5.13. [Read this page in the latest version](https://docs.galtea.ai/concepts/metric/evaluation-parameters.md).

When creating or configuring a metric, you select which parameters are relevant for your evaluation. These parameters are made available to the evaluator during scoring.

* For **AI Evaluation**, the selected parameters are automatically prepended to your `judge_prompt`.
* For **Human Evaluation**, they determine which data fields are displayed to annotators.
* For **Self-Hosted** metrics, evaluation parameters do not apply.
* For **Classifier (Jev)** metrics, you do not select parameters. They come from the `{{ name }}` placeholders in the question, and only a [short list of names](https://docs.galtea.ai/v5.13/concepts/metric/classifier.md#allowed-placeholders) is allowed.

> [!WARNING]
>
> Only select parameters that your evaluation criteria actually reference. Including unnecessary parameters can reduce evaluation quality by adding noise to the evaluator's context.

## Parameter Reference

<table>
  <thead>
    <tr>
      <th>Parameter</th>
      <th>Description</th>
      <th>Availability</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>**input**</td>
      <td>The prompt or query sent to the model.</td>
      <td>Accuracy, Security & Safety, and Behavior</td>
    </tr>

    <tr>
      <td>**actual\_output**</td>
      <td>The actual output generated by the model.</td>
      <td>Accuracy, Security & Safety, and Behavior</td>
    </tr>

    <tr>
      <td>**expected\_output**</td>
      <td>The ideal answer for the given input. A [field rule](https://docs.galtea.ai/v5.13/concepts/metric/json-field-match.md#field-rules) in a JSON expected output is checked only by the JSON Field Match metrics. Every other metric reads it as plain JSON text.</td>
      <td>Accuracy and Security & Safety</td>
    </tr>

    <tr>
      <td>**context**</td>
      <td>Additional background information provided to the model alongside the input.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**retrieval\_context**</td>
      <td>The context retrieved by your RAG system before sending the user query to your LLM. Derived from your `RETRIEVER` spans; you can also supply it directly and it is stored as a `RETRIEVER` span, not a directly-readable field.</td>
      <td>Accuracy, Security & Safety, and Behavior</td>
    </tr>

    <tr>
      <td>**traces**</td>
      <td>Execution traces from the agent, including tool calls, LLM invocations, and other internal operations.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**expected\_tools**</td>
      <td>List of tools expected to be used by the agent to accomplish the task.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**tools\_used**</td>
      <td>List of tools actually used by the agent during execution (automatically inferred from traces).</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_description**</td>
      <td>The description of the product.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_capabilities**</td>
      <td>The capabilities of the product.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_inabilities**</td>
      <td>The product's known inabilities or restrictions.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_security\_boundaries** *(deprecated)*</td>
      <td>The security boundaries of the product. Deprecated — no longer accepted for new metrics. Use `product_capabilities` and `product_inabilities` instead.</td>
      <td>N/A (rejected by API for new metrics)</td>
    </tr>

    <tr>
      <td>**user\_persona**</td>
      <td>Information about the user interacting with the agent.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**goal**</td>
      <td>The user's objective in the conversation.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**scenario**</td>
      <td>The context or situation for the conversation.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**stopping\_criterias**</td>
      <td>List of criteria that define when a conversation should end.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**conversation\_turns**</td>
      <td>All turns in a conversation, including user and assistant messages.</td>
      <td>Behavior tests (Human Evaluation only)</td>
    </tr>
  </tbody>
</table>

## Troubleshooting skipped evaluations

When an evaluation runs against a session that does not have all the data the metric needs, Galtea marks the evaluation as **SKIPPED** instead of producing a misleading score. The evaluation's error field describes which parameters are missing, grouped by where you provide them.

The categorized error message looks like this:

```
Metric "Contextual Relevancy" requires data that is not available.

Missing from product settings:
  • product_description - Fill in the "Description" field in your product settings.

Missing from test case:
  • expected_output - Provide an "Expected Output" value on your test cases.

Missing span data:
  • retrieval_context - "retrieval_context" is derived from RETRIEVER spans. Configure tracing so your product emits retrieval steps — see the tracing setup docs.
  • traces - This metric needs span data. Configure your product to send spans — see the tracing setup docs.

Learn more: https://docs.galtea.ai/concepts/metric/evaluation-parameters#troubleshooting-skipped-evaluations
```

Each section points to a different place to look:

<table>
  <thead>
    <tr>
      <th>Section</th>
      <th>Where to fix it</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>**Cannot be read by the judge**</td>
      <td>The value is there, but the judge cannot use it. The judge is sent the conversation turns, so the turns decide this: a metric that reads `input` is skipped when no turn is left with text of its own, counting neither a prompt sent beside a file nor an audio part with no transcript. The same rule applies to `actual_output`: a metric that reads it is skipped when no turn answered with text of its own, counting neither a file nor a note sent beside one. One turn carrying text of its own is enough on either side, so an attachment on one turn of a long session does not skip the rest. Score those sessions with a metric that does not read the input or the output, such as a JSON Field Match metric or a self-hosted metric. A metric that reads `conversation_turns` is never skipped for this: the turns are scored as you logged them. Nine of the eleven predefined Galtea metrics declare `input`, so expect most of them to skip on a session carrying a document. The Run Evaluation dialog greys these metrics out before the run, with the same reason under the name.</td>
    </tr>

    <tr>
      <td>**Missing from product settings**</td>
      <td>Open the product and fill in the missing field (Description, Capabilities, or Inabilities). Capabilities and Inabilities can also be derived from the product's Specifications.</td>
    </tr>

    <tr>
      <td>**Missing from test case**</td>
      <td>Edit the test case and provide the missing value. Some fields (Goal, User Persona, Scenario, Stopping Criteria) are only populated on SCENARIOS-type test cases. If your test cases use a different type, use a metric that does not require these parameters.</td>
    </tr>

    <tr>
      <td>**Missing from endpoint connection's output mapping**</td>
      <td>Edit the endpoint connection and add the missing key to the output mapping. `actual_output` is extracted from the `output` key. See [Templates & Mapping](https://docs.galtea.ai/v5.13/concepts/product/endpoint-connection-configuration.md) for the JSONPath syntax used in mapping values.</td>
    </tr>

    <tr>
      <td>**Missing trace data**</td>
      <td>The session has no conversation turns yet. Run the metric against a session that contains at least one turn.</td>
    </tr>

    <tr>
      <td>**Missing span data**</td>
      <td>The metric needs span-derived data (`traces`, `tools_used`, or `retrieval_context`). Configure your product to send span data — see the [Tracing Agent Operations](https://docs.galtea.ai/v5.13/sdk/tutorials/tracing-agent-operations.md) tutorial for setup, and the [Span concept page](https://docs.galtea.ai/v5.13/concepts/product/version/session/span.md) for what gets captured. `tools_used` (from `TOOL` spans) and `retrieval_context` (from `RETRIEVER` spans) are automatically extracted from your span data; you do not provide them directly. One turn with the data is enough: a `retrieval_context` metric is skipped only when no turn of the session has a `RETRIEVER` span with content.</td>
    </tr>
  </tbody>
</table>

A skipped evaluation consumes no credits. The skips listed above are decided before the evaluation reaches the evaluator. The evaluator can also skip one, for example a metric that does not apply to the session where the [applicability check](https://docs.galtea.ai/v5.13/concepts/metric.md#applicability-check) is on, and that skip consumes no credits either.

Spans can reach Galtea a few seconds after the turn they belong to. So on a run, Galtea waits up to about 9 seconds for late spans before it decides. It does not wait when every turn that misses a span finished more than about 9 seconds ago, because late spans for those turns would already have arrived. When it skips after a wait, the message ends with a line such as `Waited ~9 s for late spans.` A production monitor applies the same wait rule and the same other rules as a run, including the **Cannot be read by the judge** rule and the single-turn limit of deterministic metrics. A monitor decides a session up to about two minutes after the session closes, so it rarely waits.

Once you have provided the missing data, re-run the evaluation against the same session. A **Cannot be read by the judge** section is the exception: no data is missing, so re-running changes nothing until you pick a different metric or log the turn with the text you want scored, without the file.

## Related

> [card] [Evaluation Types](https://docs.galtea.ai/v5.13/concepts/metric/evaluation-types.md)
>
> Understand AI Evaluation, Human Evaluation, and Self-Hosted scoring.

> [card] [Metrics Overview](https://docs.galtea.ai/v5.13/concepts/metric.md)
>
> Browse all available metrics and create custom ones.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.